Write the CultCache store composition contract

One home store per document type, attach is hydration, loading never writes,
read-only stores, runtime type decides schema, assignable lookups, globals
never invented, explicit-batch and conditional commit (a plain flush is
last-writer-wins), per-assembly serialization options, and positional-array
value-type encoding. The persistence-format transaction section becomes the
explicit-batch shape; schema compatibility says which schema a record carries.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Meta Vens 2026-09-13 16:01:32 +02:00
commit 37f73c4e38
3 changed files with 196 additions and 12 deletions

View file

@ -41,18 +41,24 @@ Those belong to the CultCache persistence layer.
## Transaction Visibility
An authoritative multi-record change uses `CultCache.ExecuteTransactionAsync`.
The executing async flow sees its buffered overlay while staging; other readers
and all observers continue to see the prior committed generation. The backing
store commits the complete batch before the live cache swaps to it. Observer
notifications run only after the transaction context and commit lock have been
released, so an observer-triggered write begins a separate transaction.
A multi-record commit is an explicit batch of records that resolve to one home
store. The store commits the batch as one durable step. Nothing in the batch is
visible, to the committing flow or to observers, until the store has accepted
it; a failed commit changes nothing on disk or in memory.
An exception before durable finality discards the overlay. It cannot become
visible through a later flush. A transaction currently permits at most one
durable backing store because independent stores cannot provide one atomic
commit boundary. Applications that require authoritative writes may configure
`CultNetDatabase` to reject record-at-a-time writes outside this primitive.
A batch may carry conditions: per-record `(schemaId, storedAt)` identity, or
the whole store unchanged since it was last loaded. Conditions are evaluated
under the store's exclusive lock against what is durably on disk. A failed
condition is a lost race, not an error: the commit reports it and writes
nothing.
**Conditional commit is the only safe multi-process write.** A plain flush
writes the whole snapshot and is last-writer-wins, so processes sharing a store
must all use conditional commit.
C# and Rust implement batch and conditional commit. TypeScript and Python
implement neither yet. Routing, batches and conditions are specified in
`cultcache-store-composition.md`.
## Canonical Store Shape

View file

@ -26,6 +26,18 @@ The canonical fixtures currently exercised in C# are:
See `GameCult.Caching.Tests/BackingStoreTests.cs` for the canonical fixture
documents and the expected receipts.
## Which schema a record carries
A record is written under the schema of its runtime type. Writing a derived
document through a base-typed handle or generic parameter (`UpsertAsync<Gear>`
with a `Weapon`) persists the `Weapon` schema, and the record reloads as a
`Weapon`. A runtime type without `[CultDocument]` is refused.
Typed lookups and watches (`Get<T>`, `GetAll<T>`, `GetByName<T>`,
`GetByIndex<T>`, `GetGlobal<T>`, `Watch<T>`) match every record whose runtime
type is assignable to `T`. A single-result lookup with several candidates
throws.
## Soft-migratable drift
CultCache accepts compatible drift only when the local reader can still map the

View file

@ -0,0 +1,166 @@
# CultCache Store Composition
How one cache composes its backing stores: where a record lives, when a store
is read or written, what a global is, how a multi-record commit lands, and who
owns serialization options. `GameCult.Caching` in C# is the reference; the
runtime table at the end says which runtime implements what.
None of this changes bytes on disk. Every store file is a complete
`cultcache.store.v1` snapshot (single file) or a v4 directory manifest, exactly
as `cultcache-persistence-format.md` describes. No file records which cache or
which route wrote it.
## One home store per document type
- A document type has exactly one home store. A cache attaches stores with an
optional list of home types: `AddBackingStore(store, params Type[] homes)`.
- The home of a document is chosen from its **runtime type**
(`document.GetType()`), never from a generic type parameter at the call site.
Among routed stores whose home types are assignable from the runtime type, the
most derived home type wins; otherwise the untyped store; otherwise none.
- C# routes by assignable CLR type. TypeScript, Rust and Python route by the
exact type string they already use.
- A second untyped store is a registration error. A home type already claimed by
another store is a registration error.
- A write whose document has no home, once any store is attached, is an error.
A cache with zero stores is an in-memory cache and needs no home.
- Nothing is mirrored or replicated. A write touches exactly one store.
## Attachment is hydration
- Attaching a store reads it. There is no interval in which a store is attached
but unread, and no push of existing records into a newly attached store.
- A store must be clean (`IsDirty` false) when attached.
- A record's home cannot change after it is admitted. Attaching a store that
would move the home of a record already in the cache is an error. The
consequence is the attach order: routed stores first, the untyped store last.
- A record loaded from a store that is not its home is refused at load, naming
the key and both stores.
- Pulling again after attach is a re-pull. It does not erase mutations staged in
a dirty store.
## Dirtiness: loading never writes
- A store's `IsDirty` derives only from its own writes and deletes. Pulling sets
it false.
- A cache's `IsDirty` is whether any attached store is dirty, or, with no stores,
whether it has been mutated in memory.
- Opening a store and flushing without a mutation leaves every file
byte-identical.
- A mutation reaches the home store before the cache's in-memory view changes.
If the store refuses the write, the cache holds nothing new and publishes no
change.
## Read-only stores
- A store can be opened read-only. Writes, deletes and full pushes on it throw
and name the store.
- A write whose home is a read-only store throws before anything changes.
- Flushing skips read-only stores.
## Lookups and watches honor inheritance
`Get<T>`, `GetAll<T>`, `GetByName<T>`, `GetByIndex<T>`, `GetGlobal<T>` and
`Watch<T>` match every document whose runtime type is assignable to `T`. A
lookup that returns one document and finds several candidates throws, naming
them. Indexes stay keyed by concrete type; lookups enumerate assignable keys.
## Globals
- `[CultGlobal]` means: at most one record of the type per cache, `GetGlobal<T>()`
returns it or `null`, and a write without a handle keys it
`global:{schemaId}`.
- The cache never creates a global. Constructing a cache, attaching a store and
pulling produce no record the store did not contain. The code path that begins
a global's lifecycle creates it with an ordinary upsert.
- A second record of a global type, through a write or a load, is refused
before any store is touched.
- TypeScript and Python key the single record `__global__`; C# keys it
`global:{schemaId}`. The divergence is recorded, not changed, because changing
it changes bytes. Rust has no global concept.
## Atomic commit: an explicit batch
- A multi-record commit is an explicit batch value staged by the caller and
committed with `Commit`. Every record in the batch must resolve to one home
store; a batch spanning two homes throws before any store is touched.
- The store commits the batch as one durable step: one atomic replace for a
single file, pages then manifest for a directory store. A failed commit
restores the store's staging and changes nothing in memory.
- Nothing in a batch is visible, to the staging code or to observers, until the
store has accepted it. Reads during staging see committed state. There is no
ambient transaction and no in-flight overlay.
- After the store accepts, each record is admitted through the same path as a
single write (home, read-only and global checks included) and one change is
published per record.
- A cache with no stores admits a batch in memory.
## Conditional commit (compare-exchange)
A batch may carry conditions, evaluated under the home store's exclusive lock
against what is durably on disk at that moment:
- `Expect(key, current)`: the record at `key` must still be the one this cache
observed, or, with `current` null, must be absent. Only named keys are
constrained; unrelated concurrent writes do not fail the commit.
- `ExpectUnchanged()`: the store's persisted record set must equal what this
cache last loaded from it, compared as the ordered list of
`(key, schemaId, storedAt)`. Any insert, delete or replace fails it.
Record identity for conditions is `(schemaId, storedAt)`. It is sound because
every write to a key mints a `storedAt` strictly later than the record it
replaces; writers bump a minted timestamp by one tick when it is not later.
- A failed condition is a lost race, not an error: `Commit` returns false and
nothing is written, changed in memory, or published. `TryCommit` makes one
non-blocking lock attempt and reports `Contended` instead of waiting.
- A conditional commit on a store with staged single-record writes throws:
flush first.
- Single-file stores lock a sidecar `<path>.lock` opened exclusively. Directory
stores use their existing commit lease.
**A plain flush is last-writer-wins.** It writes the whole snapshot under the
same lock, so two writers never interleave bytes, but it compares nothing and
overwrites whatever another process committed. Conditional commit is the only
safe multi-process write: processes sharing a store must all use it.
A store is written by one runtime at a time. Whether a Rust `fs2` lock and a C#
exclusive open of the same lock file exclude each other is not established, and
another runtime need not obey the `storedAt` rule.
## Serialization options
- Options belong to the assembly that declares a document. That assembly names
its formatter resolvers with `[assembly: CultCacheFormatterResolver(typeof(R))]`;
generated and reflective serializers resolve options from the document's
assembly. There is no mutable static, no registration call, and no load-order
rule.
- An assembly that declares nothing, and the store envelope, use the base
options.
- Deserialization runs under untrusted-data security. `CultRecordRef<T>` is a
permitted dictionary key, compared by its key string.
## Value-type encoding
Vector-like value types (Unity.Mathematics, CultMath, consumer structs) encode
as **fixed-length positional arrays of primitive components**: a two-component
float vector is `[f32, f32]`, a three-component one `[f32, f32, f32]`, an
integer pair `[int, int]`. No ext types, no maps, no names. The persisted member
type name is the CLR full name, the same rule as `System.Int32`. A reader in
another runtime needs only the component count and primitive type.
## Runtime status
| Rule | C# | Rust | TypeScript | Python |
|---|---|---|---|---|
| One home store, no mirrors | contract; implementation replaces replication | routes by type, still pushes to later matching stores | routes by type, still mirrors | routes by type, still writes every matching store |
| Attach hydrates; loading never writes | contract; attach still pushes existing records | hydrates without writing | hydrates without writing | hydrates without writing |
| Runtime type decides schema | contract; generic parameter still decides | n/a (explicit type ids) | n/a | n/a |
| Assignable lookups and watches | `Get<T>`/`GetAll<T>` only | exact type ids | exact type ids | exact type ids |
| Globals never invented, singleton | contract; constructor still invents defaults | no globals | single `__global__`, invents nothing | single `__global__`, invents nothing |
| Explicit batch commit | contract; ambient transaction still present | `put_prepared_batch`, one store, all-or-nothing | not implemented | `put_envelopes`, one type per call, not all-or-nothing |
| Conditional commit | contract | store-level `compare_exchange` family | not implemented | not implemented |
| Per-assembly options, value-type arrays | contract | n/a | n/a | n/a |
A "contract" cell means this document is ahead of the C# code; the C# change
lands with `StoreRoutingTests` turning green.