A rewrite read the header of the file it replaces but not whether the file was
a store: C# ReadStoreHeader skipped one value and ignored bytes after it, and
took a non-array value or an array whose first slot is a string that is not a
store header for a headerless legacy file (so did the C# Mesh raw write, which
uses it); TypeScript's pushAll took any non-array value or non-header first slot
for v1; Rust ignored bytes after the array.
The rule, one per runtime: a file is a store on rewrite only when it is exactly
one complete MessagePack array. Bytes after it, a value that is not an array,
and a first slot that is a string but not a header this runtime reads are
refused, and the file is left untouched. An empty file and a legacy envelope
array (first slot not a string, empty included) carry no header and are still
rewritten.
- C#: ReadStoreHeader refuses non-array values, trailing bytes and a non-header
first string, with a NotSupportedException naming which.
- TypeScript: storeHeader (store-format.ts) decides the rewrite header;
@msgpack/msgpack's decode already refuses trailing bytes.
- Rust: store_header(bytes, whole) walks with a cursor; the rewrite path asks
for the whole file, the read paths keep reading as before.
- Python: push_all already reads the whole file (unpackb refuses trailing
bytes, the legacy decoder refuses anything but a list of maps); pinned by a
test.
Tests in each runtime run the same file matrix (truncated, trailing, scalar,
map, string first; empty, empty array, legacy), plus a C# unconditional commit
over a store with trailing bytes and the C# Mesh raw write.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SingleFileBackingStore.PushAll replaced the file without reading it, so a
truncated store or one in a format this runtime does not read was clobbered,
where Rust, TypeScript and Python refuse. PushAll now reads the header of a
non-empty file under the lock before it writes; a file it cannot read throws
and is left as it was. An empty file and a legacy envelope array carry no
header and are still replaced.
The header is read by the store's codec: SingleFileBackingStore gains a
protected virtual ReadStoreHeader (default: read the whole snapshot), and the
MessagePack store overrides it with
CultDocumentMessagePackSerialization.ReadStoreHeader. An unconditional commit
reads the file it rewrites through the same check.
Doc text: last-writer-wins now says it holds only over a file the writer can
read (CultCache.cs PushAll comment, cultcache-persistence-format.md
"Conditional commit is the only safe multi-process write" and writer rule 4,
cultcache-store-composition.md "A plain flush and an unconditional commit are
last-writer-wins").
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A record's HoldsIds was decided by its admission and trusted afterwards, so a
document changed after it was committed, or loaded and changed in place,
flushed under the header its admission-time content needed (Soul P7, P10).
HoldsIds now has one owner, the record's stored form, read when asked:
- a plain record: CultElementIds.Holds over its document, a scan that reads
and refuses nothing;
- a variant: the same scan over each override value, decoded with the codec
the record was built with (the cache's, or the attached cache's at load);
- a variant built with no codec (a store that loaded it with no cache) cannot
see its values and counts as holding an id, so a store never writes an
unmarked header over ids it could not see.
Cut: the admission path that set HoldsIds on plain records, on variants from
their resolved copies, and on flattened records; the internal setter and its
backing flag; IdPlan.HeldIds and IdPlan.HoldsIds. The Mesh typed write reads
Holds off the document it writes. Admission keeps its own id counting
(IdPlan.Count) for IdsInMemoryOnly.
Tests: an id added to a committed or loaded deck after admission marks a
PushAll v3, and taking it away unmarks it; a variant read without a codec
counts as holding ids while the same record read with one does not.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A CultStoredDocument built by a caller and handed straight to a store never passed an
admission, so HoldsIds stayed false and its ids landed under a v1/v4 header. An admission
still decides it (it counts the ids it fills); a record that never had one reads it off the
document it stores. CacheBackingStore.HeaderFor documents the rule for third-party stores.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CacheBackingStore.HeaderFor(holdsIds, existingHeader, wholeStore, directoryStore) replaces FormatFor,
DirectoryFormatFor and the writers' private marker logic. A writer that rewrites the whole store
decides by content; a writer that touches part of it keeps a header already marked. Flush,
unconditional and conditional commit, the directory store and CultMesh's single-file write all call
it. A typed CultMesh write replaces the file with a document it can see, so it decides by content;
a raw payload is opaque and keeps a marked file marked.
A record's HoldsIds is what its stored form persists: a load keeps a variant's delta as read, so
the ids a load mints in an override exist only in memory and no longer mark the store. A
conditional commit onto the file no longer reads this cache's view of what the file held, so a
stale view cannot over-mark it. IdPlan.HoldsIds is derived from HeldIds and the fills.
Reverts InternalsVisibleTo GameCult.Caching.MessagePack and the public DirectoryFormatV4/V5
constants: the directory store reaches the decision through the protected member it inherits.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A flattened variant's HoldsIds is read off the document re-resolved against the post-batch base, not carried from the
variant or from the document the flatten was planned against. The conditional commit onto a file decides by the same
rule as the unconditional commit (ids in the records the file will hold, ids only this cache minted excepted, a marked
file staying marked). The directory manifest's header choice moves beside FormatFor. Pins the three-batch flatten
probe on the commit and flush paths, the conditional commit, the last id leaving, an id-bearing override that is not
the last, and a failed CultMesh write giving its ids back.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A store is written v3 (v5 for a directory) only when a record in it holds a non-empty
element id. CultElementIds.Plan reports that as IdPlan.HoldsIds; the admission stamps it
on the stored record and every store reads it. The descriptor's CarriesElementIds is
deleted. A file already marked on disk stays marked. CultPersistedStoreSnapshot.FormatFor
is the one header decision.
Plan refuses one element object sitting twice in a list at write, naming the record and
list, instead of writing both ids and refusing the store at load.
CultMesh single-file write mints, refuses and marks as a cache write does; the read applies
the load-time plan, so a write and a load refuse the same things.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Engine tests for the plan (duplicates, format, empty derived source, atomicity, two
cultures, interface unions, key sharing); cache tests for the marker on single-file and
directory stores, override values on write and load, minting that is given back, and the
in-memory-only mark.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
[CultElementId(nameof(Member))] declares an id derived from that member's
invariant text, never random, so content-addressed elements keep encoding to the
same bytes. CultMeshCdnChunkRef derives its id from Offset; PackArtifact sets
it, the Rust pack_content writes it, and the committed content vectors are
re-cut once so both runtimes carry it. Engine tests drive Problems and Assign
directly (InternalsVisibleTo the Caching tests).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
[CultElementId] marks a string member; the registry refuses a document type
whose list element type (union bases included) lacks one, naming the type and
member. A write mints a random 12-hex id for each unset id and refuses a
duplicate within one list. A store written before ids loads with ids minted in
memory from SHA-256(record key, element path), so every reader and every reload
agrees; they persist on the store's first write. CultCache.MintElementIds()
rewrites the records whose ids exist only in memory.
CultLib's own list-element types gain an id (Geometry, Mesh body, discovery,
entity prefab). CultMeshCdnChunkRef and the CultWitness* elements are pinned by
cross-runtime vectors and are left alone; see the C2a report.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>