Docs and tests: 1.x pins do not refuse v3/v5, the mark follows content, non-C# runtimes never drop it; pin the recipe and the narrow drop to v1

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Libby 2026-11-11 08:43:06 +00:00
commit 567eac9056
8 changed files with 78 additions and 21 deletions

View file

@ -31,8 +31,7 @@ https://github.com/GameCult/CultLib.git?path=/unity/org.gamecult.cultlib#cultlib
## Rolling back across the store format
A consumer that upgrades and writes a store holding an element id leaves a
`cultcache.store.v3` single-file store (`v5` directory store). A `GameCult.Caching`
from before element ids refuses it by name, so pinning an older package after
writing is one-way. There is no carry-back; restore from a copy taken before
upgrading. Upgrade readers before writers. Detail: the Element ids section of
`cultcache.store.v3` single-file store (`v5` directory store). The first CultLib release that refuses such a store by name is `cultlib-unity-v2.0.0` (`caching-unity-v2.0.0`). Earlier pins, the 1.x line (`cultlib-unity-v1.0.61`, `cultlib-v1.0.32`), read the header string and never compare it: rolling back to one of them reads a v3 or v5 store silently, drops the ids, and can rewrite the elements without them. That is the dangerous case. There is
no general carry-back; restore from a copy taken before upgrading, and upgrade
readers before writers. Detail: the Store format and rollback section of
`src/GameCult.Caching/README.md`.

View file

@ -141,7 +141,7 @@ Large corpora should use a sharded store or a real database. A single snapshot f
## Store format and rollback
A store whose records hold an element id is marked `cultcache.store.v3` (`v5` for a C# directory store). This runtime reads and writes a v3 single-file store and carries ids through a rewrite. A CultLib or cultcache-py from before element ids refuses a v3 or v5 store by name, so rolling back to an older pin after a store has been written that way is one-way. There is no carry-back: a whole-store write drops the mark only when no record holds an id. Restore from a copy taken before upgrading, and upgrade every reader before any writer.
A store whose elements hold an id is marked `cultcache.store.v3` by the C# runtime. This runtime reads and writes a v3 single-file store, keeps the header it read and carries the ids through a rewrite; it never drops the mark, so it has no carry-back. A cultcache-py from before element ids refuses a v3 store by name, so rolling back to it after a store has been written as v3 is one-way: restore from a copy taken before upgrading, and upgrade every reader before any writer. (The C# line is different: CultLib 1.x does not refuse; see the GameCult.Caching README.)
## DatabaseEntry Slot Contract

View file

@ -396,7 +396,7 @@ fn outcome(error: &anyhow::Error) {
## Store format and rollback
A store whose records hold an element id is marked `cultcache.store.v3`. This runtime reads and writes a v3 single-file store and carries ids through a rewrite. A CultLib or cultcache-rs from before element ids refuses it by name, so rolling back to an older pin after a store has been written that way is one-way. There is no carry-back: a whole-store write drops the mark only when no record holds an id. Restore from a copy taken before upgrading, and upgrade every reader before any writer.
A store whose elements hold an id is marked `cultcache.store.v3` by the C# runtime. This runtime reads and writes a v3 single-file store, keeps the header it read and carries the ids through a rewrite; it never drops the mark, so it has no carry-back. A cultcache-rs from before element ids refuses a v3 store by name, so rolling back to it after a store has been written as v3 is one-way: restore from a copy taken before upgrading, and upgrade every reader before any writer. (The C# line is different: CultLib 1.x does not refuse; see the GameCult.Caching README.)
## Near-Term Ergonomic Improvements

View file

@ -65,7 +65,7 @@ Browsers import the same names from `@gamecult/cultcache-ts/document`, which hol
## Store format and rollback
A store whose records hold an element id is marked `cultcache.store.v3` (`v5` for a C# directory store). This runtime reads a v3 store and keeps the mark and the ids through a rewrite; it does not decide when a document carries ids. A CultLib or cultcache-ts from before element ids refuses a v3 or v5 store by name, so rolling back to an older pin after a store has been written that way is one-way. There is no carry-back: a whole-store write drops the mark only when no record holds an id. Restore from a copy taken before upgrading, and upgrade every reader before any writer.
A store whose elements hold an id is marked `cultcache.store.v3` by the C# runtime. This runtime reads a v3 store, keeps the header it read and carries the ids through a rewrite; it never drops the mark, so it has no carry-back. A cultcache-ts from before element ids refuses a v3 store by name, so rolling back to it after a store has been written as v3 is one-way: restore from a copy taken before upgrading, and upgrade every reader before any writer. (The C# line is different: CultLib 1.x does not refuse; see the GameCult.Caching README.)
## `.cc` Inspection

View file

@ -61,7 +61,7 @@ await cache.PullAllBackingStoresAsync();
## Store format and rollback
A store whose records hold an element id is marked `cultcache.store.v3` (a directory store `v5`). The single-file and directory stores here write the mark from content; see the Element ids section of the GameCult.Caching README. A CultLib or GameCult.Caching from before element ids refuses it by name, so rolling back to an older pin after a store has been written that way is one-way. There is no carry-back: a whole-store write drops the mark only when no record holds an id. Restore from a copy taken before upgrading, and upgrade every reader before any writer.
A store whose elements hold an id is marked `cultcache.store.v3` (a directory store `v5`); the single-file and directory stores here write the mark from content. See the Store format and rollback section of the GameCult.Caching README. The first CultLib release that refuses such a store by name is `cultlib-unity-v2.0.0` (`caching-unity-v2.0.0`). Earlier pins, the 1.x line (`cultlib-unity-v1.0.61`, `cultlib-v1.0.32`), read the header string and never compare it: rolling back to one of them reads a v3 or v5 store silently, drops the ids, and can rewrite the elements without them. That is the dangerous case. The only carry-back is narrow and lossy (a whole-store write that leaves no id in the store drops the mark and the ids); otherwise restore from a copy taken before upgrading, and upgrade every reader before any writer.
## MessagePack source generator

View file

@ -233,7 +233,7 @@ non-derived list id is refused; the same member outside a list is unconstrained.
If the element type already has a readable string id (`Name`, `Slug`, a catalog
key), leave it as it is. Add a trailing keyed string member marked
`[CultElementId(nameof(ExistingId))]`; the element id is derived from the
existing member the first time the element is written, and then persisted:
existing member on load or first write, and then persisted:
```csharp
[MessagePackObject]
@ -258,23 +258,29 @@ hex), and that is deliberate:
`ElementId`, so the element and every override that addresses it stay put.
`ElementId` is the address; `Name` goes back to being ordinary data.
Adopting the recipe on data that already exists: the existing member becomes an
id source, so it must be non-empty and unique within its list. A record with a
duplicate value refuses to load (`CultElementIdException`, duplicate id) and an
empty one is refused at write, so clean such data first.
Take the next free `[Key]` slot, after the highest key on the type (and above
the subclasses' keys on a union base).
## Store format and rollback
A store that holds an element id is written as `cultcache.store.v3` (a directory
store as `cultcache.store.v5.directory-content-addressed-pages`, its `v4` form
otherwise). A single-file store that holds a variant but no id is `v2`, and a store with
neither stays `v1` (`v4` for a directory). Any CultLib from before element ids refuses a v3 or v5 store by name.
That refusal is on purpose, since an older reader would skip the id slots and
rewrite the elements without them.
store as `cultcache.store.v5.directory-content-addressed-pages`, `v4` otherwise).
A single-file store that holds a variant but no id is `v2`, and a store with
neither stays `v1`. The mark follows content: a whole-store write that leaves no
element holding an id drops it.
Rolling a consumer back to an older CultLib pin after it has written a v3 or v5
store is therefore one-way. There is no carry-back: a whole-store write drops the
mark only when nothing in the store holds an id, and a type with a
`[CultElementId]` member always writes one. Restore from a copy taken before
upgrading. Upgrade every reader before any writer.
The first CultLib release that refuses such a store by name is `cultlib-unity-v2.0.0` (`caching-unity-v2.0.0`). Earlier pins, the 1.x line (`cultlib-unity-v1.0.61`, `cultlib-v1.0.32`), read the header string and never compare it: rolling back to one of them reads a v3 or v5 store silently, drops the ids, and can rewrite the elements without them. That is the dangerous case.
Rolling a consumer back after it has written a v3 or v5 store is therefore
one-way. The only carry-back is narrow and lossy: a whole-store write that leaves
no id anywhere in the store (every record rewritten without list elements) writes
the store unmarked again, and the ids are gone. Otherwise restore from a copy
taken before upgrading. Upgrade every reader before any writer.
## Persistence discipline

View file

@ -389,6 +389,44 @@ namespace GameCult.Caching.Tests
Assert.That(refusal.Message, Does.Contain("'7'"));
}
// The documented recipe for a type that already has a readable id (src/GameCult.Caching/README.md, Element ids).
[Test]
public void ATrailingMemberDerivedFromAnExistingReadableIdIsPersistedOnceAndTheStoreIsMarkedV3()
{
var registry = CultDocumentRegistry.ForTypes(new[] { typeof(RecipeBook) });
var path = PathOf("recipes.cc");
var key = new CultRecordKey("book");
using (var cache = CultCacheMessagePack.Create(path, new CultCacheOpenOptions { Registry = registry }))
cache.Commit(batch => batch.Upsert(typeof(RecipeBook), new RecipeBook { Recipes = { new IdRecipe { Name = "bread" }, new IdRecipe { Name = "soup" } } }, key));
Assert.That(HeaderOf(path), Is.EqualTo("cultcache.store.v3"));
using (var cache = CultCacheMessagePack.Create(path, new CultCacheOpenOptions { Registry = registry }))
{
var book = cache.Get<RecipeBook>(key)!;
Assert.That(book.Recipes.Select(recipe => recipe.ElementId), Is.EqualTo(new[] { "bread", "soup" }));
book.Recipes[0].Name = "loaf";
cache.Commit(batch => batch.Upsert(typeof(RecipeBook), book, key));
}
using var again = CultCacheMessagePack.Create(path, new CultCacheOpenOptions { Registry = registry });
var reread = again.Get<RecipeBook>(key)!;
Assert.That(reread.Recipes.Select(recipe => recipe.Name), Is.EqualTo(new[] { "loaf", "soup" }));
Assert.That(reread.Recipes.Select(recipe => recipe.ElementId), Is.EqualTo(new[] { "bread", "soup" }), "renaming the source does not move the element");
}
[Test]
public void AWholeStoreWriteThatLeavesNoIdInTheStoreDropsTheMarkBackToV1()
{
var registry = CultDocumentRegistry.ForTypes(new[] { typeof(RecipeBook) });
var path = PathOf("drop.cc");
var key = new CultRecordKey("book");
using var cache = CultCacheMessagePack.Create(path, new CultCacheOpenOptions { Registry = registry });
cache.Commit(batch => batch.Upsert(typeof(RecipeBook), new RecipeBook { Recipes = { new IdRecipe { Name = "bread" } } }, key));
Assert.That(HeaderOf(path), Is.EqualTo("cultcache.store.v3"));
cache.Commit(batch => batch.Upsert(typeof(RecipeBook), new RecipeBook(), key));
Assert.That(HeaderOf(path), Is.EqualTo("cultcache.store.v1"), "the mark follows content: no element holds an id, so the store is unmarked");
}
// ---- C2a fix batch: the marker, refusals at load, override values, atomic minting ----
private const string HexA = "aaaaaaaaaaaa";
@ -1359,6 +1397,20 @@ namespace GameCult.Caching.Tests
[Key(1)] public List<IdLabel> Labels { get; set; } = new();
}
[CultDocument("tests.element_id_recipe_book", "tests.element_id_recipe_book.v1")]
[MessagePackObject]
public sealed class RecipeBook
{
[Key(0)] public List<IdRecipe> Recipes { get; set; } = new();
}
[MessagePackObject]
public sealed class IdRecipe
{
[Key(0)] public string Name { get; set; } = "";
[Key(1)] [CultElementId(nameof(Name))] public string ElementId { get; set; } = "";
}
[MessagePackObject]
public sealed class IdLabel
{

View file

@ -10,7 +10,7 @@ Releases as 3.0.0 (major).
- A single-file store flush refuses a file it cannot read (empty, truncated, an unreadable stamp or body) with `CultStoreUnreadableException` and leaves the file as it is. A flush used to replace such a file. A write set cannot merge with a disk it cannot read, so the file is repaired or removed by its owner before the store writes again.
- A single-file store reads v1, v2 and v3 stamps and refuses v4 and later by name; a directory store reads v4 and v5 and refuses v6 and later by name. Each judges the stamp before the body, so an unreadable body never makes a newer store look like damage.
- A store is written in the format its content needs, and CultLib 2.0 refuses the newer ones by name. A single-file store holding a document variant is written as `cultcache.store.v2`, one holding an element id as `cultcache.store.v3`; a directory store holding an element id is written as `cultcache.store.v5` (`v4` otherwise). A store is marked only when a record actually holds an id or a variant. Upgrade readers before writers: a 2.0 reader that opens such a store is refused, not misread. Rolling back to a 2.0 pin after a store has been written as v3 or v5 is one-way: there is no carry-back (a whole-store write drops the mark only when no record holds an id, and a type with a `[CultElementId]` member always writes one), so restore from a copy taken before upgrading.
- A store is written in the format its content needs, and CultLib 2.0 refuses the newer ones by name. A single-file store holding a document variant is written as `cultcache.store.v2`, one holding an element id as `cultcache.store.v3`; a directory store holding an element id is written as `cultcache.store.v5` (`v4` otherwise). A store is marked only when a record actually holds an id or a variant. Upgrade readers before writers: a 2.0 reader that opens such a store is refused, not misread. Rolling back to a 2.0 pin after a store has been written as v3 or v5 is one-way: the only carry-back is narrow and lossy (a whole-store write that leaves no element holding an id writes the store unmarked, and the ids are gone), so restore from a copy taken before upgrading. 1.x pins do not refuse such a store at all; rolling back to one reads it silently and can rewrite the elements without ids.
- Writes are write sets. A flush or commit replaces the records it staged and copies every other record forward as stored, under the store's lock, instead of rewriting the whole store from the cache's memory. A record the cache does not know is kept, and an untouched record keeps a dropped member's slot through every write (never reuse a dropped slot; see the schema-compatibility contract).
- A write or removal at a key whose stored record the writer does not own is refused with `CultSchemaConflictException`, and the file is left as it was. A writer owns a record whose schema id it declares (owns or lists compatible) and a record its type reads whole under compatible drift: that record is writable and is restamped under the type's own id on write, as in 2.0.1. A record a type reads only by dropping persisted slots stays read-only until its id is declared compatible, which accepts the loss (this differs from 2.0.1, which shed the slots silently). A cache owns through its registry; a cacheless CultMesh write owns through the catalog entry it writes.
- A write replaces the record at a key only when the type it writes there owns that record, decided per key and the same for a cache and for a cacheless `WriteDocument`, `WriteRecord` or unattached-store `PushAll`, in a single-file store and a directory store. The written type owns the stored record when its entry declares the stored id (its own id, or one it lists compatible), or when it is the type that reads the record's published entry and reads it whole (no persisted slot unread): an older id of its own schema name that it reads whole is its own and is restamped, as in 2.0.1. Anything else is refused with `CultSchemaConflictException` and the file left as it was: a record of another registered type, under its current id or an old id it lists, and a record the written type would read only by dropping persisted slots. No registry enters the decision, `CultDocumentRegistry.Shared` included, so a lent registry never changes the answer. This changes a cache: where 2.x let a cache holding two types replace one's record at a key with the other's, it now refuses, with or without a backing store. Retype a key by removing it and adding the new record, in one batch (`batch.Remove(key)` then `batch.Upsert(...)`) or staged apart (`Remove` then `UpsertAsync`); the removal is judged as a removal, so a record no registered type owns is still refused. A removal or check goes to the registry's removal rule, and a cacheless call, which has no registry, removes a record only when an entry it writes owns it (a `Delete` then `PushAll` on an unattached store writes no entry and is refused).