diff --git a/docs/nuget-packaging.md b/docs/nuget-packaging.md index c83929e0..841deedf 100644 --- a/docs/nuget-packaging.md +++ b/docs/nuget-packaging.md @@ -31,7 +31,7 @@ https://github.com/GameCult/CultLib.git?path=/unity/org.gamecult.cultlib#cultlib ## Rolling back across the store format A consumer that upgrades and writes a store holding an element id leaves a -`cultcache.store.v3` single-file store (`v5` directory store). The first CultLib release that refuses such a store by name is `cultlib-unity-v2.0.0` (`caching-unity-v2.0.0`). Earlier pins, the 1.x line (`cultlib-unity-v1.0.61`, `cultlib-v1.0.32`), read the header string and never compare it: rolling back to one of them reads a v3 or v5 store silently, drops the ids, and can rewrite the elements without them. That is the dangerous case. There is +`cultcache.store.v3` single-file store (`v5` directory store). Refusal by name depends on the store kind and the pin. A **directory** store (`v5`) is refused by every Unity pin from `cultlib-unity-v1.0.57` and `caching-unity-v1.2.0` on (the earliest tags of each line whose `DirectoryMessagePackBackingStore` reads only the `v4` stamp: "only ... is readable"). A **single-file** store (`v3`) is refused only from `cultlib-unity-v2.0.0` and `caching-unity-v2.0.0`: `cultlib-unity-v1.0.61` and `caching-unity-v1.4.0` read the header string and never compare it. The NuGet line (`cultlib-v1.0.32`, the last `cultlib-v` tag) refuses neither: its directory store writes `cultcache.store.v1.directory` and never checks the stamp on read. Rolling back a single-file store to any of those pins reads it silently, drops the ids, and can rewrite the elements without them. That is the dangerous case. There is no general carry-back; restore from a copy taken before upgrading, and upgrade readers before writers. Detail: the Store format and rollback section of `src/GameCult.Caching/README.md`. diff --git a/docs/runtime-parity-scope.md b/docs/runtime-parity-scope.md index e001bb1d..2f54e2fe 100644 --- a/docs/runtime-parity-scope.md +++ b/docs/runtime-parity-scope.md @@ -16,7 +16,7 @@ Cross-runtime parity has three layers: Every store reader refuses, by name, a header or record slot it does not understand (vectors: `tests/vectors/document-variants-c0`). Element ids: a store that can hold them declares `cultcache.store.v3`; TypeScript, - Python and Rust read it and keep the header, and readers older than ids refuse it + Python and Rust read it and keep the header, and the C# runtime decides the mark from content, and these runtimes never drop it. A C# reader from `cultlib-unity-v2.0.0` on refuses a v3 single-file store it cannot read (earlier pins do not) (vectors: `tests/vectors/document-variants-c2a`). Document variants: C# resolves them (`cultcache.store.v2`); every other runtime, CultMesh single-file reads, the directory store and CultNet refuse a variant by name diff --git a/packages/cultcache-py/CHANGELOG.md b/packages/cultcache-py/CHANGELOG.md index cf5898c7..d440ddca 100644 --- a/packages/cultcache-py/CHANGELOG.md +++ b/packages/cultcache-py/CHANGELOG.md @@ -11,7 +11,7 @@ - `indexes` are unique, as C# `[CultIndex]`: a write, load or `register_index` that leaves two records of a type holding one non-blank value under one alias is refused (`CultCacheError`) naming both keys and the alias, and writes nothing. Index text is the C# canonical text (`True`/`False`, integers as digits, floats as the exact expansion, blank values never indexed), so an empty or whitespace string is no longer a value. A new `roles` map names the many-row selection aliases (never indexed for lookup, same catalog `index_alias`). Classify every `indexes` key per the README before updating. The verdict is the contract, the text is informational. -- A single-file store written as `cultcache.store.v3` is read and written: a record that holds an element id (an owned list element with an `[CultElementId]` member, carried through every write) marks the store v3, and a v3 store is carried through a rewrite with its ids. A runtime from before this change refuses a v3 store by name. Rolling back to an older release after a store has been written as v3 is one-way: there is no carry-back, so restore from a copy taken before upgrading. +- A single-file store written as `cultcache.store.v3` is read and written: a store the C# runtime marked v3 (it holds a record with an element id: an owned list element with a `[CultElementId]` member) keeps its header here, and is carried through every rewrite with its ids; this runtime never drops the mark. A runtime from before this change refuses a v3 store by name. Rolling back to an older release after a store has been written as v3 is one-way: there is no carry-back, so restore from a copy taken before upgrading. - A write over a file the reader refuses (`StoreUnreadableError`, from `push`, `push_all` and `delete`) refuses and leaves the file as it is. A flush used to replace such a file. A write set cannot merge with a disk it cannot read, so the file is repaired or removed by its owner before the store writes again. - Added `StoreUnreadableError` (a `ValueError`) for a store file this runtime cannot read, and `SchemaConflictError` (a `ValueError`) for a write the catalog cannot describe. A store is exactly three slots; a schema is never recovered from a record's payload. diff --git a/packages/cultcache-rs/CHANGELOG.md b/packages/cultcache-rs/CHANGELOG.md index 61f6e4dc..15011491 100644 --- a/packages/cultcache-rs/CHANGELOG.md +++ b/packages/cultcache-rs/CHANGELOG.md @@ -8,7 +8,7 @@ only in git history. ### Breaking -- A single-file store written as `cultcache.store.v3` is read and written: a record that holds an element id (an owned list element with an `[CultElementId]` member, carried through every write) marks the store v3, and a v3 store is carried through a rewrite with its ids. A runtime from before this change refuses a v3 store by name. Rolling back to an older release after a store has been written as v3 is one-way: there is no carry-back, so restore from a copy taken before upgrading. +- A single-file store written as `cultcache.store.v3` is read and written: a store the C# runtime marked v3 (it holds a record with an element id: an owned list element with a `[CultElementId]` member) keeps its header here, and is carried through every rewrite with its ids; this runtime never drops the mark. A runtime from before this change refuses a v3 store by name. Rolling back to an older release after a store has been written as v3 is one-way: there is no carry-back, so restore from a copy taken before upgrading. - A write over a file the reader refuses (a zero-byte file, a truncated store, a stamp this runtime does not read) refuses with `CultCacheStoreUnreadable` and leaves the file as it is. A whole-store write used to replace such a file. This reverses 0.3.1, where `push_all` replaced a file with no readable stamp (healing it); the stamp rule now has one owner, the reader every write asks under the lock, and it accepts v1 and v3 (0.3.1 refused v3). - `#[cultcache(index)]` means unique, as in C# `RefuseIndexSharing`: within one type and one alias no two records hold one non-blank value. A put, batch, push or load that would make two records hold a value is refused, naming both keys and the alias, and a store that already holds a duplicate does not open. A grouping index is no longer expressible; a consumer that relied on shared values must drop the `index` and look up by another way. - `bool` index and lookup text renders `True` / `False`, as C# does, where it rendered `true` / `false`. diff --git a/packages/cultcache-ts/CHANGELOG.md b/packages/cultcache-ts/CHANGELOG.md index 84d6592c..1d0e81a2 100644 --- a/packages/cultcache-ts/CHANGELOG.md +++ b/packages/cultcache-ts/CHANGELOG.md @@ -28,7 +28,7 @@ All notable changes to `@gamecult/cultcache-ts` are recorded here, following ### Breaking -- A single-file store written as `cultcache.store.v3` is read and written: a record that holds an element id (an owned list element with an `[CultElementId]` member, carried through every write) marks the store v3, and a v3 store is carried through a rewrite with its ids. A runtime from before this change refuses a v3 store by name. +- A single-file store written as `cultcache.store.v3` is read and written: a store the C# runtime marked v3 (it holds a record with an element id: an owned list element with a `[CultElementId]` member) keeps its header here, and is carried through every rewrite with its ids; this runtime never drops the mark. A runtime from before this change refuses a v3 store by name. - A single-file store refuses a zero-byte file as `StoreUnreadableError`; only a path with nothing at it is an empty store. A header this runtime does not read is refused, and the refusal echoes the header only as `cultcache.store.v`. - A write over a file the reader refuses (`StoreUnreadableError`, from `push`, `pushAll` and `delete`) refuses and leaves the file as it is. A flush used to replace such a file. A write set cannot merge with a disk it cannot read, so the file is repaired or removed by its owner before the store writes again. A full `pushAll` over a file with no readable stamp does not replace it: the reader refuses it. The stamp rule has one owner, `refuseUnreadFormat`, asked only by the one reader that every write consults under the lock. The reader judges the stamp from the header before it decodes the body, so a newer store is refused by name whether or not its body is whole. - A dangling symbolic link at the store path is an I/O error with code `ENOENT`, not an empty store. diff --git a/src/GameCult.Caching.MessagePack/README.md b/src/GameCult.Caching.MessagePack/README.md index 9643372d..5e7156b0 100644 --- a/src/GameCult.Caching.MessagePack/README.md +++ b/src/GameCult.Caching.MessagePack/README.md @@ -61,7 +61,7 @@ await cache.PullAllBackingStoresAsync(); ## Store format and rollback -A store whose elements hold an id is marked `cultcache.store.v3` (a directory store `v5`); the single-file and directory stores here write the mark from content. See the Store format and rollback section of the GameCult.Caching README. The first CultLib release that refuses such a store by name is `cultlib-unity-v2.0.0` (`caching-unity-v2.0.0`). Earlier pins, the 1.x line (`cultlib-unity-v1.0.61`, `cultlib-v1.0.32`), read the header string and never compare it: rolling back to one of them reads a v3 or v5 store silently, drops the ids, and can rewrite the elements without them. That is the dangerous case. The only carry-back is narrow and lossy (a whole-store write that leaves no id in the store drops the mark and the ids); otherwise restore from a copy taken before upgrading, and upgrade every reader before any writer. +A store whose elements hold an id is marked `cultcache.store.v3` (a directory store `v5`); the single-file and directory stores here write the mark from content. See the Store format and rollback section of the GameCult.Caching README. Refusal by name depends on the store kind and the pin. A **directory** store (`v5`) is refused by every Unity pin from `cultlib-unity-v1.0.57` and `caching-unity-v1.2.0` on (the earliest tags of each line whose `DirectoryMessagePackBackingStore` reads only the `v4` stamp: "only ... is readable"). A **single-file** store (`v3`) is refused only from `cultlib-unity-v2.0.0` and `caching-unity-v2.0.0`: `cultlib-unity-v1.0.61` and `caching-unity-v1.4.0` read the header string and never compare it. The NuGet line (`cultlib-v1.0.32`, the last `cultlib-v` tag) refuses neither: its directory store writes `cultcache.store.v1.directory` and never checks the stamp on read. Rolling back a single-file store to any of those pins reads it silently, drops the ids, and can rewrite the elements without them. That is the dangerous case. The only carry-back is narrow and lossy (a whole-store write that leaves no id in the store drops the mark and the ids); otherwise restore from a copy taken before upgrading, and upgrade every reader before any writer. ## MessagePack source generator diff --git a/src/GameCult.Caching/README.md b/src/GameCult.Caching/README.md index 63faa68f..6defed61 100644 --- a/src/GameCult.Caching/README.md +++ b/src/GameCult.Caching/README.md @@ -260,8 +260,8 @@ hex), and that is deliberate: Adopting the recipe on data that already exists: the existing member becomes an id source, so it must be non-empty and unique within its list. A record with a -duplicate value refuses to load (`CultElementIdException`, duplicate id) and an -empty one is refused at write, so clean such data first. +duplicate value and an empty one both refuse to load +(`CultElementIdException`), so clean such data first. Take the next free `[Key]` slot, after the highest key on the type (and above the subclasses' keys on a union base). @@ -274,7 +274,7 @@ A single-file store that holds a variant but no id is `v2`, and a store with neither stays `v1`. The mark follows content: a whole-store write that leaves no element holding an id drops it. -The first CultLib release that refuses such a store by name is `cultlib-unity-v2.0.0` (`caching-unity-v2.0.0`). Earlier pins, the 1.x line (`cultlib-unity-v1.0.61`, `cultlib-v1.0.32`), read the header string and never compare it: rolling back to one of them reads a v3 or v5 store silently, drops the ids, and can rewrite the elements without them. That is the dangerous case. +Refusal by name depends on the store kind and the pin. A **directory** store (`v5`) is refused by every Unity pin from `cultlib-unity-v1.0.57` and `caching-unity-v1.2.0` on (the earliest tags of each line whose `DirectoryMessagePackBackingStore` reads only the `v4` stamp: "only ... is readable"). A **single-file** store (`v3`) is refused only from `cultlib-unity-v2.0.0` and `caching-unity-v2.0.0`: `cultlib-unity-v1.0.61` and `caching-unity-v1.4.0` read the header string and never compare it. The NuGet line (`cultlib-v1.0.32`, the last `cultlib-v` tag) refuses neither: its directory store writes `cultcache.store.v1.directory` and never checks the stamp on read. Rolling back a single-file store to any of those pins reads it silently, drops the ids, and can rewrite the elements without them. That is the dangerous case. Rolling a consumer back after it has written a v3 or v5 store is therefore one-way. The only carry-back is narrow and lossy: a whole-store write that leaves diff --git a/unity/org.gamecult.cultlib/CHANGELOG.md b/unity/org.gamecult.cultlib/CHANGELOG.md index 83389b7b..3d03439a 100644 --- a/unity/org.gamecult.cultlib/CHANGELOG.md +++ b/unity/org.gamecult.cultlib/CHANGELOG.md @@ -10,7 +10,7 @@ Releases as 3.0.0 (major). - A single-file store flush refuses a file it cannot read (empty, truncated, an unreadable stamp or body) with `CultStoreUnreadableException` and leaves the file as it is. A flush used to replace such a file. A write set cannot merge with a disk it cannot read, so the file is repaired or removed by its owner before the store writes again. - A single-file store reads v1, v2 and v3 stamps and refuses v4 and later by name; a directory store reads v4 and v5 and refuses v6 and later by name. Each judges the stamp before the body, so an unreadable body never makes a newer store look like damage. -- A store is written in the format its content needs, and CultLib 2.0 refuses the newer ones by name. A single-file store holding a document variant is written as `cultcache.store.v2`, one holding an element id as `cultcache.store.v3`; a directory store holding an element id is written as `cultcache.store.v5` (`v4` otherwise). A store is marked only when a record actually holds an id or a variant. Upgrade readers before writers: a 2.0 reader that opens such a store is refused, not misread. Rolling back to a 2.0 pin after a store has been written as v3 or v5 is one-way: the only carry-back is narrow and lossy (a whole-store write that leaves no element holding an id writes the store unmarked, and the ids are gone), so restore from a copy taken before upgrading. 1.x pins do not refuse such a store at all; rolling back to one reads it silently and can rewrite the elements without ids. +- A store is written in the format its content needs, and CultLib 2.0 refuses the newer single-file ones by name. A single-file store holding a document variant is written as `cultcache.store.v2`, one holding an element id as `cultcache.store.v3`; a directory store holding an element id is written as `cultcache.store.v5` (`v4` otherwise). A store is marked only when a record actually holds an id or a variant. Upgrade readers before writers: a 2.0 reader that opens such a store is refused, not misread. Rolling back to a 2.0 pin after a store has been written as v3 or v5 is one-way: the only carry-back is narrow and lossy (a whole-store write that leaves no element holding an id writes the store unmarked, and the ids are gone), so restore from a copy taken before upgrading. Refusal by name is not uniform in 1.x: a v5 directory store is refused from `cultlib-unity-v1.0.57` (`caching-unity-v1.2.0`), but a v3 single-file store is read silently by every pin before `cultlib-unity-v2.0.0` (including `cultlib-unity-v1.0.61`, `caching-unity-v1.4.0`, `cultlib-v1.0.32`), which drops the ids and can rewrite the elements without them. - Writes are write sets. A flush or commit replaces the records it staged and copies every other record forward as stored, under the store's lock, instead of rewriting the whole store from the cache's memory. A record the cache does not know is kept, and an untouched record keeps a dropped member's slot through every write (never reuse a dropped slot; see the schema-compatibility contract). - A write or removal at a key whose stored record the writer does not own is refused with `CultSchemaConflictException`, and the file is left as it was. A writer owns a record whose schema id it declares (owns or lists compatible) and a record its type reads whole under compatible drift: that record is writable and is restamped under the type's own id on write, as in 2.0.1. A record a type reads only by dropping persisted slots stays read-only until its id is declared compatible, which accepts the loss (this differs from 2.0.1, which shed the slots silently). A cache owns through its registry; a cacheless CultMesh write owns through the catalog entry it writes. - A write replaces the record at a key only when the type it writes there owns that record, decided per key and the same for a cache and for a cacheless `WriteDocument`, `WriteRecord` or unattached-store `PushAll`, in a single-file store and a directory store. The written type owns the stored record when its entry declares the stored id (its own id, or one it lists compatible), or when it is the type that reads the record's published entry and reads it whole (no persisted slot unread): an older id of its own schema name that it reads whole is its own and is restamped, as in 2.0.1. Anything else is refused with `CultSchemaConflictException` and the file left as it was: a record of another registered type, under its current id or an old id it lists, and a record the written type would read only by dropping persisted slots. No registry enters the decision, `CultDocumentRegistry.Shared` included, so a lent registry never changes the answer. This changes a cache: where 2.x let a cache holding two types replace one's record at a key with the other's, it now refuses, with or without a backing store. Retype a key by removing it and adding the new record, in one batch (`batch.Remove(key)` then `batch.Upsert(...)`) or staged apart (`Remove` then `UpsertAsync`); the removal is judged as a removal, so a record no registered type owns is still refused. A removal or check goes to the registry's removal rule, and a cacheless call, which has no registry, removes a record only when an entry it writes owns it (a `Delete` then `PushAll` on an unattached store writes no entry and is refused).