Commit graph

760 commits

Author SHA1 Message Date
Meta Vens
ea5a07b37a Own serialization options by the document's assembly
The static UntrustedData options are gone. An assembly declares its formatter
resolvers with [assembly: CultCacheFormatterResolver(typeof(R))];
CultDocumentMessagePackSerialization.OptionsFor(assembly) composes them ahead
of CultDocumentResolver and StandardResolver, once per assembly, and hands
assemblies that declare nothing the base Options. Generated codecs and the
reflective SerializeUntyped/DeserializeUntyped path both read
OptionsFor(typeof(X).Assembly). No registration call, no mutable options, no
load order.

CultMessagePackSecurity copies UntrustedData and supplies a collision-resistant
comparer over CultRecordRef<T>.Key.Value, so ref-keyed dictionaries
deserialize.

Tests: RefKeyedDictionaryRoundTrips failed on 825b4f7 ("No hash-resistant
equality comparer available"); DeclaredResolverEncodesValueTypeAsPositionalArray
and UnknownAssemblyGetsBaseOptions did not compile there (new API);
InteropNoteBytesUnchanged is a pin whose payload constant was captured on
825b4f7. BenchItem, the one unkeyed [CultDocument] in any consumer (in-memory
benchmark, no store), gains [Key]s in its existing slot order.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 21:12:53 +02:00
Meta Vens
6fcc2a7704 Stop MessagePack's source generator flowing to consumers
A direct MessagePackAnalyzer reference with PrivateAssets="all" in
Caching.MessagePack and Networking keeps MessagePack 3's generator out of
consumer builds, where it emitted a reference to a nonexistent
GeneratedMessagePackResolver formatter for Dictionary<CultRecordRef<T>, float>
(CS0426). CultLib composes StandardResolver itself and never registers a
generated resolver.

tests/GameCult.Caching.ConsumerBuild is RefKeyedDictionaryCompilesInAConsumer:
its build is the test. On 825b4f7 it failed with CS0426; now it builds and
obj has no MessagePack.SourceGenerator output.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 21:09:16 +02:00
Meta Vens
825b4f7d26 Mirror snapshots carry the cache Sequence they reflect
ObserveAsync, ReactiveAsync and RefreshAsync read a snapshot without a
Sequence, so an older change admitted before the read but delivered after it
overwrote the newer value.

CultCache.GetWithSequence(key) returns the document and the cache's Sequence,
read together under the gate. A cache-backed CultMesh handle passes it to its
mirrors; each mirror subscribes first, then adopts the sequenced read under its
own gate and takes that Sequence as applied, so a straggler at or below it
drops. RefreshAsync on both mirrors uses the same adoption path. Public Mesh
surface unchanged. The read is non-generic because the Mesh route needs the
untyped document for same-schema conversion.

Contract: only Watch/WatchRecord changes carry Sequence; OnUpdate-derived
CultNet streams have no ordering or stale protection; schema-alias handles and
removals are not stale-protected; the sequence constructor argument changed
CultCacheDocumentChange<T>'s binary signature.

Tests: ObserveAfterNewerSnapshotIgnoresStraggler and
RefreshIgnoresStragglerOlderThanSnapshot (observed and reactive cases) fail on
4562340; SequencedReadReflectsEveryChangeAtOrBelowItsSequence.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 21:02:49 +02:00
Meta Vens
456234018f Carry order as data: per-cache change Sequence, no delivery scheduler
bcae483 made the cache a delivery scheduler (tickets, _deliveryGate turn
waits, per-thread delivery queues). It deadlocked inside CultLib: a load's
delivery reaching a CultNetDatabaseSubscriptionServer observer that needs
_lifecycleGate, while a subscribe's demand handler holding that gate writes
the cache and waits for the delivery's turn.

The cache now owns order as data. Every admitted change takes the next value
of a per-cache in-memory sequence under the gate, published as
CultCacheDocumentChange<T>.Sequence. Delivery is the 544c087/b653f68 rule:
each call publishes its own changes on its own thread after releasing the
gate; nothing waits on another call's delivery. Cross-thread delivery order is
not guaranteed.

Deleted: ticket counter, _delivered, _deliveryGate, Monitor.Wait turns,
per-thread delivery queues, per-ticket catch-all, and the save-order contract
text with its observer-must-not-block rule.

Consumer: a cache-backed CultMesh document handle hands its observed and
reactive mirrors each snapshot's Sequence; ApplyCanonicalSnapshot drops one
not newer than the last applied.

Tests: SequenceIncreasesInAdmissionOrder,
ObserverWritingTheSameCacheDoesNotWaitOnOtherDeliveries,
SubscriptionDemandHandlerWritingCacheDuringDeliveryDoesNotDeadlock,
WatchRecordIgnoresStaleSequence. The last three time out on bcae483.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 20:43:11 +02:00
Meta Vens
bcae4839d0 Deliver each cache's changes in save order
The outermost hold takes a ticket under the gate and, after releasing it,
waits on a per-cache delivery gate for its turn, delivers, then advances
the turn. Delivery order across threads equals admission order. A hold
closed while its thread is delivering queues its ticket behind that
delivery on the same thread instead of waiting, so an observer writing
the cache cannot deadlock against a writer: a waiting thread holds no gate
and no turn, and a turn holder never waits.

Hand-over-hand from the gate to a delivery lock was rejected: a writer
blocked on the delivery lock while holding the gate deadlocks with an
observer that writes the same cache.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 20:09:23 +02:00
Meta Vens
b653f68f81 Route every admitting gate entry through a hold
FlushAllBackingStores and Dispose take the gate through Held, so a store
method that loads can never reach Admit under a plain lock; Admit outside
a hold throws a clear InvalidOperationException. Entering a cache's hold
inside another cache's hold throws. PullAllBackingStoresAsync pulls every
store even when an OnUpdate handler throws, then rethrows. The dead Try
around Subject.OnNext is gone: R3 never throws there, so only OnUpdate
exceptions reach the caller. Repull tests wait for the writer to start
before their bounded wait.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 20:01:36 +02:00
Meta Vens
544c087cf1 Publish each hold's own changes to its own caller
The cache-wide unpublished list let any thread drain and deliver another
thread's changes: a throwing observer dropped the rest of the queue and threw
into an unrelated writer, writes returned before their change was published,
and a third-party load could wait indefinitely.

Every admission now runs in a hold (Held). The outermost hold on a thread
collects exactly the changes it admits and publishes them after it leaves the
gate, before it returns, on its caller's thread; nested holds append to it.
Every change is delivered even when an observer throws; the first exception
(AggregateException for several) is then rethrown, and a failing body's own
exception wins over observer exceptions.

Stores take the hold instead of locking a shared Gate object, so a direct
PullAll on an attached store publishes its own load and the empty follow-up
Loaded hand-over is gone. Gate is replaced by protected Held.

Tests: OwnChangeIsPublishedBeforeWriteReturns and
ThrowingObserverDoesNotLoseOtherChanges fail on 4eacd87 and pass here. The
repull tests now record a write that reaches the gate while the pull is paused
instead of inferring readiness from thread state; both fail when the pull does
not hold the gate.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 19:46:18 +02:00
Meta Vens
4eacd876bf Publish observers after the gate, gate every store entry point, stamp under one hold
Observers and OnUpdate never run under the cache gate. Admit queues changes
while the gate is held; the outermost entry point (attach, pull, commit,
upsert, remove) publishes them after releasing it. A store's direct PullAll
hands over an empty load outside its lock so the cache publishes then.

A throwing observer can no longer split store from cache: publication starts
only after the store has adopted its loaded view. The directory store adopts
its durable catalog only after the cache admits the load; the write-only
_durableIndex is deleted.

Single-file Push, Delete, PushAll and CommitBatch take the gate like the
directory store; Dispose flushes under it. Write mints storedAt and admits
under one gate hold, and Land re-stamps staged upserts under the gate that
admits them, so a concurrent pull cannot land a later storedAt in between.

Tests: ObserverDoesNotRunUnderTheGate,
ThrowingOnUpdateHandlerDoesNotResurrectDeletedRecord,
RepullNeverErasesAStagedWriteOnDirectoryStore,
DirectCallsOnAnAttachedStoreShareTheCacheGate; the directory last-writer
result is pinned to {a,c,x}; the single-file re-pull test waits for the write
to reach the store or block instead of sleeping. Pausing tests always release
the pause so a failure cannot deadlock Dispose.

Docs: observer rule, direct store calls share the gate, directory TryCommit
contention and gate wait, per-key last-writer-wins, directory flush merge.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 19:23:44 +02:00
Meta Vens
5aad137e61 Share the cache gate with its stores and make unconditional commit last-writer-wins
An attached store takes its cache's gate as its own lock (CacheBackingStore.Gate),
so a pull's load callback into the cache can no longer take the gate after the
store lock. The directory store's private mutation gate is deleted; the
single-file pull now runs under the gate; PullAllBackingStoresAsync takes the
gate around each store's pull. Fixes the pull/upsert deadlock on directory
stores and the re-pull that erased a staged single-file write.

An unconditional single-file Commit writes the store's whole view plus the batch,
exactly as a flush would, instead of merging onto the file; it persists earlier
staged writes and leaves the store clean. Conditional commits still land onto
the file under the lock. Directory-store commit and flush already share one
generation writer (staged keys onto the current manifest) and are unchanged.

Contracts state both rules, the lock order, that OnUpdate fires for loads only,
that a missing single file keeps staged writes, and that a store failing to
hydrate on open throws and is left byte-identical.

Tests: PullAndUpsertOnDirectoryStoreDoNotDeadlock, RepullNeverErasesAStagedWrite,
UnconditionalCommitIsLastWriterWinsLikeFlush, UnconditionalCommitPersistsStagedWrites,
OpeningACorruptStoreThrowsAndLeavesItUntouched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 18:45:27 +02:00
Meta Vens
2a44209510 Refuse a key in two stores, stop reporting reader contention, prove lock waits and page order
F2: Validate refuses a write or load that would give a key already held by
one home store a second store; the message names the key and both stores and
nothing lands in either store or the cache.
F3: TryCommit enters the cache gate normally; only the store lock attempt is
non-blocking, so Contended means another writer holds the store.
F4: PushAll and Commit are shown to wait on a held <path>.lock; the race test
asserts mismatches occurred and that Task.WaitAll finished in time.
F5: a failed page write leaves the manifest bytes and the previous generation
intact.

Lock order (F1) and commit merge semantics (F6) are unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 18:21:19 +02:00
Meta Vens
03a71ebd66 Pin CultNet publish-after-commit, add write-routed interop, update contract status
- NetworkingTests.CultNetPutInsideCommitPublishesOnlyAfterCommit: a record
  staged in a batch reaches no CultNet observer before the store commits; a
  put made while staging is its own write. Chosen over refusing CultNet writes
  inside a batch, which would need ambient "inside a batch" state.
- Interop peer mode write-routed <catalog.cc> <run.cc>: cultcache.interop-note
  routes to the catalog file, cultcache.interop-run-note to the run file, one
  record each.
- store-composition contract: the C# column reads implemented for routing,
  attach, schema, lookups, globals, batch and conditional commit; overwriting
  a store is open, remove, upsert, flush. persistence-format: C# implements
  batch and conditional commit.
- tests README lists the new suites.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 17:58:00 +02:00
Meta Vens
245eef6fb7 Restore directory store durability coverage with real file locks
Cut 2 deleted the stage probes and the tests that injected faults through them.
DirectoryStoreDurabilityTests takes the faults from held files instead:
FailedManifestReplaceKeepsPreviousGeneration,
SameStoredAtDifferentPayloadWritesADistinctPage, PullAllWaitsForHeldCommitLease,
MutationBlockedBehindFlushStaysDirty, FlushNeverOpensUnchangedPages,
MissingManifestIgnoresOrphanPagesAndFlushDeletesThem,
TamperedPageFailsLoadNamingPage.

The tampered page surfaces as an AggregateException from the parallel page
load whose message names the page; the test asserts the page is named.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 17:58:00 +02:00
Meta Vens
a06d3f71f9 Prove conditional commit on single-file and directory stores
ConditionalCommitTests: StaleExpectFailsWithNothingChanged, ExpectNullIsCreateOnce,
PerEntryCommitSurvivesUnrelatedWrite, ExpectUnchangedFailsOnUnrelatedInsert,
TwoWritersRacingExactlyOneWins, TryCommitReportsContended,
PlainFlushNeverInterleaves, StoredAtIsStrictlyIncreasingPerKey,
DirtyStoreRefusesConditionalCommit, CrossStoreConditionalBatchThrowsBeforeAnyStore,
DirectoryStoreHonorsConditions (the four condition tests on a directory store).

The race is in-process: two store instances on one file, two threads, the
final count exactly 2 + 2N. The lock is a FileShare.None handle, which excludes
a second handle in this process exactly as in another, so the cross-process
TwoProcessesRacingExactlyOneWins and its console project are not added. Each
writer's mismatch is scripted before the free race, so "each writer reports at
least one mismatch" does not depend on scheduling.

PlainFlushNeverInterleaves counts an open that loses to a replace in flight as
no read; every read that gets bytes must deserialize.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 17:57:59 +02:00
Meta Vens
ffdecf7e26 Turn the store-routing tests green and cover routing, batches and loads
Cut 1's fixtures are rewritten as the cut map names: LoadingNeverWritesASecondStore
attaches its second store with a home type, and the global fixture is a real
[CultGlobal] type instead of a reflected backing field.

New: RoutedStoresWriteTheSameBytesAsSingleStores, CatalogRecordNeverLandsInRunStore,
AttachAfterDirtyThrows, SecondUntypedStoreThrows, DuplicateHomeTypeThrows,
WriteWithoutHomeThrows, ForeignRecordIsRefusedOnLoad,
LateRouteOverAdmittedTypeThrows, ReadOnlyStoreRefusesWrites,
ReadOnlyStoreIsNotFlushed, GlobalSingletonIsEnforced,
AmbiguousAssignableLookupThrows, BatchIsAllOrNothingOnStoreFailure,
BatchAcrossTwoHomesThrows, BatchObserversSeeOnlyCommittedRecords,
BatchReadsSeeCommittedStateOnly, SingleUpsertAndBatchShareOneAdmissionPath,
PullAllAdmissionFailureLeavesStoreAndCacheConsistent,
StoreAttachedToSecondCacheThrows.

Two readings where the map could not be followed literally:
- LateRouteOverAdmittedTypeThrows: "the reverse order succeeds" cannot hold
  with the same untyped file holding a Note, because a Note loaded from the
  untyped store when a Note route exists is a foreign record. The reverse case
  puts the Note in the routed store and a RoutingOther in the untyped one.
- BatchIsAllOrNothingOnStoreFailure: the fault is the store file held open
  without delete sharing, so the atomic replace fails, instead of a PushAll
  override; the single-file commit no longer routes through PushAll.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 17:57:58 +02:00
Meta Vens
a3cc394241 Route each document type to one home store; attach hydrates; commit explicit batches
Cut 3 of the CultCache migration (Aetheria docs/cultcache-migration-cut.md, 2.1
and 2.2). One commit, because routing, the batch that replaces the ambient
transaction, and conditional commit share one admission path.

Deleted first:
- globals: InitializeGlobals, MaterializeMissingGlobals, the two-argument
  constructor, the initializeGlobals plumbing in CultCacheMessagePack,
  ContainsDurableRecord (cache and directory store)
- replication: the push-to-every-store loops and the attach-time push of
  existing records
- typeof(T) as the schema owner in AddAsync/UpsertAsync and in
  CultNetDatabase.PutAsync/PutPredictedAsync
- the ambient transaction: ExecuteTransactionAsync x2, CultCacheTransaction,
  CommitTransaction, VisibleStoredDocuments, the AsyncLocal, the SemaphoreSlim
  gate, and every ambient check
- CultCacheOpenOptions.PullOnOpen; OpenAsync is Task.FromResult(Create(...))
- DirectoryMessagePackBackingStore.ResolveLegacyUncataloguedRecordCatalog: the
  catalog precommit (fe55bff) is an ancestor of the v4 page format (eea8248),
  so no v4 manifest can reach it
- CultCache.DeleteAsync<T>; its one caller, CultNetDatabaseSubscriptionClient,
  called it by reflection and now calls Remove(key)

Added:
- AddBackingStore(store, params Type[] homes): attach validates (clean, not
  already attached, one untyped store, no claimed home type, no admitted
  record changes home) and then pulls under the gate
- Home(type): most derived routed home, else the untyped store
- Admit: every add and remove, from a write, a batch or a load, validates
  (home, read-only, loaded-from-home, global singleton), lands durably, then
  applies and publishes; a pull hands its whole delta over in one call, so a
  refused record leaves the store and the cache unchanged
- read-only stores (CacheBackingStore(readOnly), both store constructors,
  CultCacheOpenOptions.ReadOnly); flush skips them and writes only dirty stores
- assignable GetGlobal<T>/GetByName<T>/GetByIndex<T>/Watch<T>; more than one
  candidate throws naming them
- Commit(Action<CultCacheBatch>) returning bool, TryCommit returning
  CultCommitOutcome, CultCacheBatch (Upsert, Remove, Expect, ExpectUnchanged,
  sealed after commit), CultCommitRequest; CacheBackingStore.CommitBatch
  evaluates conditions under the store's exclusive lock
- single-file store: <path>.lock sidecar for CommitBatch and PushAll; a clean
  store commits onto the file as it is, so records it never loaded survive
- storedAt minted in one place and bumped a tick when not later than the record
  it replaces

Consumer adjustments in this repo: Mesh streaming tests drop PullOnOpen; the
Studio bridge stops reflecting on it; BackingStoreTests' global test creates
its global explicitly and its transaction tests are rewritten against Commit
(the two that proved ambient hydration rules are deleted); the pre-Cut 2
refusal test expects the attach to throw.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 17:57:58 +02:00
Meta Vens
14d7c64774 Mark pre-Cut 2 store fixtures binary
The repo-wide text=auto with core.autocrlf=true would rewrite an LF byte in
a content-addressed page on checkout and break its SHA-256 identity.
Committed blobs already match the bytes produced at 0db1fe5.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 17:17:49 +02:00
Meta Vens
997f3997e6 Exercise CultCache.TryGet<T> again in the OpenAsync durability test
AquaSynth calls TryGet<T>; cover both the hit and the miss.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 17:17:14 +02:00
Meta Vens
b26c98f185 Note that CultManagedDocument is parked with SoA; state the page load rule
The parked/cultcache-soa tag message says the managed document was deleted
separately. parked-features.md now supersedes that; the tag is unchanged.
The directory store gains its design statement and a comment that only
manifest-named pages are loaded.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 17:17:14 +02:00
Meta Vens
0ab5564688 Drop the unused GameCult.Logging reference from GameCult.Caching
No Caching source uses it. Networking, the only CultLib user, references
Logging directly; no external consumer (Aquarium, Ymir, Mimir, AquaSynth,
Gjallar, Delvehold) uses the GameCult.Logging namespace.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 17:17:14 +02:00
Meta Vens
b6905c2c1c Pin stores written before Cut 2 with checked-in fixtures
Fixtures were produced by CultLib 0db1fe5 from a temporary worktree: a v4
directory store and a v1 single-file store holding the same three records.
Tests reopen both and assert every value, and assert the directory store
refuses v1, v1.directory, v2 and v3 manifests without touching the bytes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 17:17:14 +02:00
Meta Vens
946e8fcc2d Restore CultCache.TryGet<T>: the audit missed AquaSynth's call
The Cut 2 subtraction audit listed TryGet<T> as unused, but
AquaSynth\src\AquaSynth.Core\CultCachePatchDocument.cs:104 calls
cache.TryGet(key, out CultCachePatchDocument? document). The map's grep
matched only TryGetByName/TryGetByIndex, so the call was missed and AquaSynth.Core,
AquaSynth.Faust, AquaSynth.Dsl.Tests and AquaSynthDaemon stopped compiling
against 85f2089. The method returns with the signature it had at e9b91c1.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 17:00:36 +02:00
Meta Vens
bb10c8241b Retire the last test fixture for deleted surface; update the tests README
SchemaStampedTestEntry was only used by the v1 missing-catalog load test
deleted with the legacy directory formats. The Caching tests README now
describes what the suite covers after the subtraction, including the ambient
transaction tests kept until Cut 3 and the red store-routing contract tests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 16:57:07 +02:00
Meta Vens
60cbfdde6d Strip doc comments from the Caching projects
Every /// line in GameCult.Caching and GameCult.Caching.MessagePack goes; they
restated member names. Three blocks that state a rule stay as // comments:
MaterializeMissingGlobals runs after hydration so persisted globals win,
ExecuteTransactionAsync commits durably before the cache and observers
advance, and CommitBatch implementations restore their staged view on
failure. Both projects add NoWarn CS1591, since GenerateDocumentationFile is
on in src/Directory.Build.props.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 16:54:43 +02:00
Meta Vens
360dd0ae2f Delete the standalone schema catalog serializers
SerializeSchemaCatalog and DeserializeSchemaCatalog had no consumer outside
the snapshot round-trip test. Catalog entries still serialize inside
SerializeSnapshot/DeserializeSnapshot through the same per-entry
reader and writer, and the round-trip test now checks the catalog through the
snapshot.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 16:51:35 +02:00
Meta Vens
900e148d8a Read only v4 directory stores; delete probes and selective hydration
DirectoryMessagePackBackingStore loses HydrationFilter, FlushStageProbe,
ReadStageProbe and their stage constants, ReadPersistedGeneration,
PullSelected, and every legacy path: the v2 directory-indexed and v3
immutable-pages formats, v1 inline records and split pages, the index
upgrade on flush, and the legacy page paths. CacheBackingStore.PullSelected
and CultPersistedRecordMetadata go with them.

A missing manifest is an empty store. An existing manifest whose
FormatVersion is not cultcache.store.v4.directory-content-addressed-pages
throws "Directory store {path} is {format}; only {v4} is readable." A clean
flush writes nothing, since there is no legacy index left to upgrade.

Tests deleted with the surface they injected through: the v1 missing-catalog
load, pages-before-manifest, fault-before-manifest (4 cases), same-timestamp
page, mutation-during-flush and reader-lease tests. The orphan-page test now
checks through a fresh store pull instead of ReadPersistedGeneration.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 16:50:08 +02:00
Meta Vens
59410fc474 Delete unused CultCacheOpenOptions callbacks and directory overrides
ConfigureCache, ConfigureStore, DirectoryStorePath, DirectoryStoreHydrationFilter
and ConfigureDirectoryStore had no consumer outside tests. CreateCore attaches
the directory store at its default record path, or the single-file store,
with no customization hooks. OpenAsync keeps its behavior (pull, then
materialize missing globals); PullOnOpen stays until Cut 3.

The indexed-filter test loses its selective-open half and keeps the v4
manifest assertions: the hot index carries only the SHA-256 page identity and
both records reload.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 16:45:42 +02:00
Meta Vens
85f2089f0e Delete CultCache wrappers and unused members
Removes FlushBackingStore, both PrepareForReloadOrShutdown overloads,
PullBackingStoreRecordsAsync, TryGet<T>, TryGetByName, TryGetByIndex,
Resolve<T>, the cache and store Logger, and LastSuccessfulFlushAtUtc on the
cache and stores. GetStoredDocuments<T> folds into GetAll<T>.

The soft flag goes from FlushAllBackingStores, ExecuteTransactionAsync,
CommitBatch and PushAll. It stays on FlushAsync(bool) because
CultCacheStudioWindow reflects on that signature until Cut 6.
CultMesh.FlushAsync and CultNetLocal.FlushAsync keep their parameter and
stop forwarding it.

The store's EntryAdded/EntryUpdated/EntryDeleted subjects become
protected internal Loaded/Unloaded callbacks set by the cache at attach.
AddStoredDocumentInternal is synchronous, so the GetAwaiter().GetResult()
bridges on the store subscriptions and the global materializer are gone.

Tests: drop the LastSuccessfulFlushAtUtc asserts; OpenAsync and GetAll tests
read through Get/GetAll; the persisted-global test proves the durable global
loads without staging a default instead of going through selective hydration;
the lease-before-observers test observes through CultCache.Watch; the
PullSelected selective-hydration test is deleted.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 16:44:11 +02:00
Meta Vens
1982d73e44 Name the managed document in the parked SoA entry
CultManagedDocument<T> and its two Document<T> openers came in with the SoA
storage (5428870) and are preserved at the same parked/cultcache-soa tag.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 16:41:55 +02:00
Meta Vens
88e1c8e157 Delete CultNetDatabase's transaction wrapper
ExecuteTransactionAsync (both overloads), RequireTransactionsForAuthoritativeWrites,
EnsureAuthoritativeTransaction and the AfterCommit deferral had no consumer
outside CultNetDatabase_RequiredTransaction_DefersRawPutPublicationUntilCommit,
deleted with them. Writes publish immediately, as they already did outside a
transaction. The cache's own transaction machinery stays until Cut 3 replaces
it with the explicit batch commit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 16:35:56 +02:00
Meta Vens
d26f357f3e Delete CultManagedDocument and both Document<T> openers
CultManagedDocument<T>, CultCache.Document<T> and CultNetDatabase.Document<T>
had no consumer outside the tests removed with the SoA table.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 16:23:55 +02:00
Meta Vens
fe7e5fbe37 Delete the parked CultCache SoA table
Removes CultSoaTable, CultSoaColumn, CultCacheSoaStore, CultCacheSoaTypeTable,
CultCacheSoaMember, CultCache.Soa<T>, the _soa field and its four hooks, and
the two tests that exercised them (with their TransformEntry and
MeshManagedPlayer fixtures). The last working form is at
parked/cultcache-soa.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 16:22:54 +02:00
Meta Vens
d904960610 Park the CultCache SoA table at parked/cultcache-soa
The SoA table was built for ECS-style structure-of-arrays columns over
cached documents and has no consumer outside tests. It is tagged before
removal; docs/parked-features.md names the tag, span, and restore path, and
the runtime parity scope says the table is parked rather than wrong.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 16:21:25 +02:00
Meta Vens
0db1fe5448 Apply Soul fixes to the store composition docs and routing tests parked/cultcache-soa
Conditional commit, not plain flush, is what protects against unseen data.
C# batch and conditional commit are specified, not implemented. Home-type
claims use exact equality. TypeScript attach does not read. CultMath has no
encoding. Reflective and generated descriptors differ until Cut 4. Routing
tests now check the inherited Name, the global flag precondition, and that a
refused write publishes no change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 16:13:20 +02:00
Meta Vens
d81112fe44 Add red store-routing tests
Eight tests against today's API, each red for the behavior the store
composition contract removes: attach pushes into a second store, the generic
parameter chooses schema, exact-type name/global/watch lookups, invented
globals destroying a seeded file, and a refused push leaving the document in
memory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 16:01:32 +02:00
Meta Vens
486fe560af Correct stale store-routing, SoA, and generator docs
The README and the Rust README described mirrors as C# behavior; the parity
scope claimed the C# SoA table; the generator README described DatabaseEntry
formatters that no longer exist.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 16:01:32 +02:00
Meta Vens
37f73c4e38 Write the CultCache store composition contract
One home store per document type, attach is hydration, loading never writes,
read-only stores, runtime type decides schema, assignable lookups, globals
never invented, explicit-batch and conditional commit (a plain flush is
last-writer-wins), per-assembly serialization options, and positional-array
value-type encoding. The persistence-format transaction section becomes the
explicit-batch shape; schema compatibility says which schema a record carries.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 16:01:32 +02:00
Meta Vens
e9b91c1f50 Preserve public trust-anchor parent permissions parked/cultnet-discovery-contracts
enroll_service_identity_at and export_service_identity_trust_anchor
both routed through prepare_parent, which chmods the target directory
to 0700 on unix. That is correct for the private enrollment path but
wrong for exporting a public trust anchor: Idunn's provisioning and
control-plane code export the public anchor into a directory another
service already owns and reads from (Idunn/src/provisioning.rs,
Idunn/src/control_plane.rs), and forcibly narrowing that directory's
permissions on every export would lock out the legitimate reader
instead of protecting anything.

Split the helper: ensure_parent only creates the directory, and the
renamed prepare_private_parent (used solely by enrollment) also locks
it down. Ported from GameCult/cultnet-rs commit 69f2498 ("Preserve
public anchor parent permissions"), which fixed the same bug before
that standalone repo was superseded by this package.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-13 14:45:40 +02:00
Meta Vens
e9e9c60b0f REC-102: cut stale generated mirrors and a dead maintenance receipt
Deletes four confirmed-dead tracked files:
- CultLib.txt, src/GameCult.Caching/GameCult.Caching.txt,
  src/GameCult.Unity/Assets/UI/UI.txt: concatenated snapshot copies of live
  source, carrying stale E: drive provenance and obsolete dependency
  versions. Live source under src/ remains the owner.
- tests/GameCult.Networking.Tests/lcov.info: tracked coverage output.
  .vscode/tasks.json:14 still generates it fresh via
  "/p:CoverletOutput=./lcov.info" on every coverage run, so tracking the
  stale committed copy in git is the liability, not the generation step.
- .voidbot/status/void-memory-maintenance.json: a one-time "starting"
  receipt (added whole in 4813570, never touched again) naming a dead
  E:\Projects\CultLib path and an unimplemented repo_face_heartbeat
  launcher. VoidBot's actual heartbeat state lives at
  .voidbot/status/repo-face-heartbeats.{cc,json} (see
  packages/config/src/index.ts REPO_FACE_HEARTBEAT_STATE_PATH) — a
  different file entirely; nothing reads this one.

Kill attempts:
- git grep -n for each of "CultLib\.txt", "GameCult\.Caching\.txt",
  "UI\.txt", "lcov\.info" across origin/main, excluding each file's own
  path: only .voidbot birth/distiller snapshots (auto-generated onboarding
  file-listing catalogs, not functional readers) and the one real lcov.info
  producer in .vscode/tasks.json.
- rg across all of F:/Projects (excluding target/node_modules/.git):
  gamecult-site's docs/research/gamecult-org-repo-ledger.json lists
  "CultLib.txt" as a filename in a static historical repo-file inventory
  (not a functional consumer); Aetheria/docs/cultcache-migration-cut.md
  cites GameCult.Caching.txt:157 as an evidence line number in a
  migration-history note (stale citation after this cut, not a functional
  break — flagged for the record, not fixed here since that doc is out of
  scope).
- voidbot MCP search for void-memory-maintenance.json / repo_face_heartbeat
  found only the unrelated repo-face-heartbeats state path, confirming no
  reader.

Collision check: git diff --stat origin/main against each of the other 4
CultLib worktrees (codex/recover-aetheria-authority,
codex/delvehold-isosurface, claude/media-stream-catalog, and the two
detached-HEAD worktrees, both confirmed ancestors of origin/main) over
these five paths: all empty, no in-flight work touches them.

REC-102

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-13 14:10:10 +02:00
Meta Vens
c2a9a6e526 Windows atomic replace waits out a transient reader
Some checks failed
Publish packages / @gamecult/cultcache-ts to npm (push) Has been skipped
Publish packages / cultcache-py to PyPI (push) Has been cancelled
Publish packages / cultmesh-py to PyPI (push) Has been cancelled
Publish packages / cultnet-py to PyPI (push) Has been cancelled
cultcache-py-v0.2.0
A concurrent reader of the store's destination file (another process
opening it, an indexer, an antivirus scan) makes MoveFileExW fail with a
sharing or access error for as long as it holds the handle. That is
transient by construction; treating the first refusal as the outcome
killed Muninn serve on 2026-09-11 the moment a status CLI overlapped a
write. The replace retries such errors for up to half a second. The test
holds the destination open from another thread and expects the replace to
outlast it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-11 02:44:17 +02:00
Meta Vens
47b9fc4ee3 A reader returns "nothing" only when nothing is waiting
Both RUDP socket readers returned None after consuming a packet that carried
no frame or event for the caller: an acknowledgement, a ping, a fragment
that completed nothing yet. A caller that polls once per send therefore read
one incoming datagram per outgoing one, and fell behind its peer's
acknowledgements as soon as a reliable channel shared the session; every
unacknowledged packet was then resent, and the resends starved the sends.
Measured on the media path 2026-09-10: 7 video frames in 9 s with reliable
audio on, 266 without. Readers now consume no-event packets and keep
reading; None means the socket is empty.

The client no longer acknowledges unreliable frames. They carry no sequence,
so the ACK told the sender nothing and cost it one datagram to read for
every one it sent.

The fragment-set bound test now asserts eviction, which is what the bound
has done since the fuse was cut earlier today; the old assertion was hiding
behind a truncated test run.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-11 01:21:37 +02:00
Meta Vens
c84cb2e862 The consumer dials the producer
A media stream advertisement now carries `media_endpoint`, where the
producer serves it from; the request no longer carries a consumer endpoint.
The producer listens and the consumer connects, the same way as every other
client/server pair on the mesh, so a viewer's host admits nothing inbound
and needs no firewall rule or port forward. Unshipped contract, keys
renumbered in place.

Audio gets its own channel ("audio": reliable, unordered, media expiry) so
one session can carry lossy video under parity and reliable audio together,
which is what lets the producer serve both to one dialled-in consumer.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-10 22:50:20 +02:00
Meta Vens
61b7bf0e24 Say how Odin admits catalog records, and what it refuses
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-10 22:24:04 +02:00
Meta Vens
780e1a48cd Advertise and request media streams without naming a producer
Two state records beside the media wire records, travelling as CultMesh
documents through Odin rather than on the media channel:

- gamecult.media_stream_advertisement.v1: a stream a producer can serve
  on request — its video and audio sources, the codecs it can encode,
  audio sample rate and channels, defaults, and the connection id it
  dials a receiver with. Keyed by stream_id. A consumer's picker is built
  from these.
- gamecult.media_stream_request.v1: a consumer asking a producer to start
  or stop serving a stream to a receiver endpoint, with sources, codecs,
  bitrate and latency budget. Keyed by stream_id:receiver_id so a newer
  request supersedes; the producer answers on the same key by rewriting
  state and detail.

Until now the only way to pick a stream was Muninn's own
muninn.obs_stream_catalog and muninn.capture_stream_command, which live
in odin-core and bind every consumer to one producer. Odin persists any
well-formed document, so nothing there has to change to carry these.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-10 22:07:39 +02:00
Meta Vens
4160ab560b A repair request is not a keyframe request
The builder forced requested_keyframe whenever chunk keys were missing.
A producer answers each new keyframe request with an IDR, so every
repair request would have cost a keyframe — the thing a repair request
exists to avoid. The old C++ receiver sent repairs with the flag off;
this builder had never been driven by a real receiver. A whole missing
frame still implies one, since the repair cache holds chunks.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-10 21:35:04 +02:00
Meta Vens
34c8ea79b9 Build receiver feedback where both ends can see the rules
build_receiver_feedback normalises the record a receiver sends back —
damage lists sorted and unique, chunk keys well-formed, a keyframe
requested whenever something is missing — and until now lived in Muninn,
the one end that never sends it. Ratatoskr sends it; Muninn's sender
tests construct it to exercise the repair cache. A builder both ends
call belongs beside the record it builds.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-10 21:33:06 +02:00
Meta Vens
89687fceb4 Let the browser sample read the provider's advertisement
Eve refuses to build a command intent for a surface that has not
advertised a command boundary and receipt schema, and the intent it
builds now carries the operation under operation.operationId. The
browser sample did neither: the click was refused inside the page and
the checkpoint saw only a 30 s timeout on the receipt.

The C# provider now publishes its gamecult.eve.provider_advertisement.v1
document with the surface's worldInteraction, and the page leases that
and hands it to the lowering as `provider`. The provider owns the
declaration; the page reads it rather than restating it. The command
sink reads intent.operation.operationId.

The checkpoint now forwards the page's console and pageerror output.
"CultMesh browser operation is required." took one run to find with it
and had been a bare timeout without it.

  scripts/verify-eve-getting-started.ps1 -EveRoot ..\Eve (Eve main)
  -> Eve getting-started verification passed: clean .NET and TypeScript
     artifacts, DOM and headless convergence, real Chromium and C#
     clients, Odin discovery, authority-bound routes, receipts,
     persistence, and route replacement.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-10 13:55:46 +02:00
Meta Vens
f94f5581b0 Let the verse catalog schema say what the reference emits
The C# reference publishes authorityRoutes with certificate: null for
loopback routes, and its own verifier treats null as "no certificate" in
local development. The TypeScript JSON schema only allowed the object,
so the browser client rejected every catalog the getting-started
provider served: "certificate: must be object", Odin could not resolve
the Verse, and the real browser/C# checkpoint has failed since f622ee1
(2026-08-18), behind the sample failure that hid it. TypeScript
consumers already guard route.certificate; the schema now accepts null.

The package-closure smoke also still spelled the browser identity with
providerId; it is authorityRuntimeId since c194bd4.

  npm test --workspace packages/cultnet-ts -> 54 pass
  npm run test:pack:ts -> passed

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-10 13:55:46 +02:00
Meta Vens
672235e8f3 Ignore Python package build output
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-10 13:55:46 +02:00
Meta Vens
830a5d718f Make the Eve onboarding checkpoint run against the contract that exists
The portability workflow has been red since 2026-08-22. Three things,
each hidden behind the one before it:

- Eve started refusing to build a command intent for a surface that has
  not advertised a command boundary and receipt schema (Eve 4338a42,
  2026-08-22). The two-runtime sample never advertised one. It now passes
  a provider advertisement with worldInteraction for its surface, which is
  the declaration the lowering was right to demand.
- The sample and the package-closure script still imported "cultcache-ts";
  the package became @gamecult/cultcache-ts on 2026-09-04 (8cb3b72) and
  CI never got far enough to say so. Two READMEs carried the same import.
- The job pinned Eve to codex/eve-browser-host-isolation, a branch that was
  never merged and stopped moving on 2026-08-25. The sample is written
  against Eve main and passes there; the pin now says so.

The interop workflow's Python step ran one test file by path; it now
discovers the three split suites.

  scripts/verify-eve-two-runtime-sample.ps1 -EveRoot F:\Projects\Eve
  (Eve main ccc3ffe) -> browserCount 1, headlessCount 1, restartCount 1

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-10 13:55:46 +02:00
Meta Vens
6d98b03471 Drop the license classifier setuptools now refuses
setuptools >= 77 rejects "License :: OSI Approved :: MIT License" once a
PEP 639 license expression is set, and `pip install -e` builds with the
newest setuptools. That has failed both CI workflows at the install step
since 2026-09-04. `license = "MIT"` is the expression; the classifier
said the same thing and is gone from all three Python manifests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-10 13:55:46 +02:00