Commit graph CultLib/tests/GameCult.Caching.Tests/ElementIdTests.cs
Author SHA1 Message Date
Meta Vens
ea1bd6a797 Every runtime rewrites only a file that is exactly one store
A rewrite read the header of the file it replaces but not whether the file was
a store: C# ReadStoreHeader skipped one value and ignored bytes after it, and
took a non-array value or an array whose first slot is a string that is not a
store header for a headerless legacy file (so did the C# Mesh raw write, which
uses it); TypeScript's pushAll took any non-array value or non-header first slot
for v1; Rust ignored bytes after the array.

The rule, one per runtime: a file is a store on rewrite only when it is exactly
one complete MessagePack array. Bytes after it, a value that is not an array,
and a first slot that is a string but not a header this runtime reads are
refused, and the file is left untouched. An empty file and a legacy envelope
array (first slot not a string, empty included) carry no header and are still
rewritten.

- C#: ReadStoreHeader refuses non-array values, trailing bytes and a non-header
  first string, with a NotSupportedException naming which.
- TypeScript: storeHeader (store-format.ts) decides the rewrite header;
  @msgpack/msgpack's decode already refuses trailing bytes.
- Rust: store_header(bytes, whole) walks with a cursor; the rewrite path asks
  for the whole file, the read paths keep reading as before.
- Python: push_all already reads the whole file (unpackb refuses trailing
  bytes, the legacy decoder refuses anything but a list of maps); pinned by a
  test.

Tests in each runtime run the same file matrix (truncated, trailing, scalar,
map, string first; empty, empty array, legacy), plus a C# unconditional commit
over a store with trailing bytes and the C# Mesh raw write.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 00:01:58 +02:00
Meta Vens
4f742d05c3 C#: a flush refuses to overwrite a store file whose header it cannot read
SingleFileBackingStore.PushAll replaced the file without reading it, so a
truncated store or one in a format this runtime does not read was clobbered,
where Rust, TypeScript and Python refuse. PushAll now reads the header of a
non-empty file under the lock before it writes; a file it cannot read throws
and is left as it was. An empty file and a legacy envelope array carry no
header and are still replaced.

The header is read by the store's codec: SingleFileBackingStore gains a
protected virtual ReadStoreHeader (default: read the whole snapshot), and the
MessagePack store overrides it with
CultDocumentMessagePackSerialization.ReadStoreHeader. An unconditional commit
reads the file it rewrites through the same check.

Doc text: last-writer-wins now says it holds only over a file the writer can
read (CultCache.cs PushAll comment, cultcache-persistence-format.md
"Conditional commit is the only safe multi-process write" and writer rule 4,
cultcache-store-composition.md "A plain flush and an unconditional commit are
last-writer-wins").

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 23:51:47 +02:00
Meta Vens
619e089fa6 C#: HoldsIds is read off the stored form each time a store writes
A record's HoldsIds was decided by its admission and trusted afterwards, so a
document changed after it was committed, or loaded and changed in place,
flushed under the header its admission-time content needed (Soul P7, P10).

HoldsIds now has one owner, the record's stored form, read when asked:
- a plain record: CultElementIds.Holds over its document, a scan that reads
  and refuses nothing;
- a variant: the same scan over each override value, decoded with the codec
  the record was built with (the cache's, or the attached cache's at load);
- a variant built with no codec (a store that loaded it with no cache) cannot
  see its values and counts as holding an id, so a store never writes an
  unmarked header over ids it could not see.

Cut: the admission path that set HoldsIds on plain records, on variants from
their resolved copies, and on flattened records; the internal setter and its
backing flag; IdPlan.HeldIds and IdPlan.HoldsIds. The Mesh typed write reads
Holds off the document it writes. Admission keeps its own id counting
(IdPlan.Count) for IdsInMemoryOnly.

Tests: an id added to a committed or loaded deck after admission marks a
PushAll v3, and taking it away unmarks it; a variant read without a codec
counts as holding ids while the same record read with one does not.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 23:46:50 +02:00
Meta Vens
9c62187562 Test: the directory store keeps a marked manifest on a partial flush, so only the single file drops back
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:56:26 +02:00
Meta Vens
c4fbb53a3b C#: HoldsIds means what the stored form persists, wherever the record came from
A CultStoredDocument built by a caller and handed straight to a store never passed an
admission, so HoldsIds stayed false and its ids landed under a v1/v4 header. An admission
still decides it (it counts the ids it fills); a record that never had one reads it off the
document it stores. CacheBackingStore.HeaderFor documents the rule for third-party stores.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:53:13 +02:00
Meta Vens
41d0dd9bfb C# header: one function decides it for every writer; a store's own ids are the only source
CacheBackingStore.HeaderFor(holdsIds, existingHeader, wholeStore, directoryStore) replaces FormatFor,
DirectoryFormatFor and the writers' private marker logic. A writer that rewrites the whole store
decides by content; a writer that touches part of it keeps a header already marked. Flush,
unconditional and conditional commit, the directory store and CultMesh's single-file write all call
it. A typed CultMesh write replaces the file with a document it can see, so it decides by content;
a raw payload is opaque and keeps a marked file marked.

A record's HoldsIds is what its stored form persists: a load keeps a variant's delta as read, so
the ids a load mints in an override exist only in memory and no longer mark the store. A
conditional commit onto the file no longer reads this cache's view of what the file held, so a
stale view cannot over-mark it. IdPlan.HoldsIds is derived from HeldIds and the fills.

Reverts InternalsVisibleTo GameCult.Caching.MessagePack and the public DirectoryFormatV4/V5
constants: the directory store reaches the decision through the protected member it inherits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 21:54:01 +02:00
Meta Vens
6a7d17bdda C2a fix: a flattened record is marked by the document it stores; every writer decides its header by content
A flattened variant's HoldsIds is read off the document re-resolved against the post-batch base, not carried from the
variant or from the document the flatten was planned against. The conditional commit onto a file decides by the same
rule as the unconditional commit (ids in the records the file will hold, ids only this cache minted excepted, a marked
file staying marked). The directory manifest's header choice moves beside FormatFor. Pins the three-batch flatten
probe on the commit and flush paths, the conditional commit, the last id leaving, an id-bearing override that is not
the last, and a failed CultMesh write giving its ids back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 21:20:01 +02:00
Meta Vens
9ed61dd290 C2a test: a flattened variant that holds override ids keeps the flush marked
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 20:09:05 +02:00
Meta Vens
e8886ff8c0 C2a test: a reloaded variant holding override ids keeps the whole-store flush marked
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 20:03:22 +02:00
Meta Vens
dfde61aba3 C2a test: a preset id and a loaded record holding one mark the store
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:59:52 +02:00
Meta Vens
cc6615b675 C2a test: the id-bearing variant overrides its name
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:43:55 +02:00
Meta Vens
490f7b811f C2a: mark the header by content, refuse an element twice in a list, route CultMesh single-file through the id rule
A store is written v3 (v5 for a directory) only when a record in it holds a non-empty
element id. CultElementIds.Plan reports that as IdPlan.HoldsIds; the admission stamps it
on the stored record and every store reads it. The descriptor's CarriesElementIds is
deleted. A file already marked on disk stays marked. CultPersistedStoreSnapshot.FormatFor
is the one header decision.

Plan refuses one element object sitting twice in a list at write, naming the record and
list, instead of writing both ids and refusing the store at load.

CultMesh single-file write mints, refuses and marks as a cache write does; the read applies
the load-time plan, so a write and a load refuse the same things.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:40:36 +02:00
Meta Vens
fed7204d9c Tests: the element id rules, each pinned against its own mutation
Engine tests for the plan (duplicates, format, empty derived source, atomicity, two
cultures, interface unions, key sharing); cache tests for the marker on single-file and
directory stores, override values on write and load, minting that is given back, and the
in-memory-only mark.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 18:27:16 +02:00
Meta Vens
ded5436569 CultCache: derived element ids; the CDN chunk ref carries one
[CultElementId(nameof(Member))] declares an id derived from that member's
invariant text, never random, so content-addressed elements keep encoding to the
same bytes. CultMeshCdnChunkRef derives its id from Offset; PackArtifact sets
it, the Rust pack_content writes it, and the committed content vectors are
re-cut once so both runtimes carry it. Engine tests drive Problems and Assign
directly (InternalsVisibleTo the Caching tests).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 15:03:09 +02:00
x
0d2645d056 CultCache: element ids on every object-list element (C2a)
[CultElementId] marks a string member; the registry refuses a document type
whose list element type (union bases included) lacks one, naming the type and
member. A write mints a random 12-hex id for each unset id and refuses a
duplicate within one list. A store written before ids loads with ids minted in
memory from SHA-256(record key, element path), so every reader and every reload
agrees; they persist on the store's first write. CultCache.MintElementIds()
rewrites the records whose ids exist only in memory.

CultLib's own list-element types gain an id (Geometry, Mesh body, discovery,
entity prefab). CultMeshCdnChunkRef and the CultWitness* elements are pinned by
cross-runtime vectors and are left alone; see the C2a report.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 12:31:12 +02:00