GameCult/CultLib
Watch
9
0
Fork
You've already forked CultLib
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

C# store header: the durable header owns "holds element ids"; StoredHoldsIds and KnownIdFree are deleted

Browse source 
A record's bytes cannot say whether it holds an element id (an id is an ordinary keyed
string), so the per-record fact decoded by whichever type loaded the record was a split
authority: a narrower type unmarked a v3 store over bytes holding ids. HeaderForWrite is now
a pure function of the durable header, the HoldsIds of the records the write encodes, and
whether it copies any record forward: a write that copies keeps a marked header, a write
that encodes every record decides by content. The directory store takes the same rule.

Also pins the three Low test findings deferred from c2a-write-set s3: the reload cause is
observed by type and message, the author's own Expect after a commit, and the directory
store's re-mint branches.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Meta Vens 2026-10-03 07:04:48 +02:00
commit 25d6cb2951
5 changed files with 115 additions and 55 deletions
Download patch file Download diff file Expand all files Collapse all files

7
src/GameCult.Caching.MessagePack/DirectoryMessagePackBackingStore.cs
View file

@ -311,17 +311,14 @@ public sealed class DirectoryMessagePackBackingStore : CacheBackingStore
var holdsIds = written.ToDictionary(key => key, key => Entries[key].HoldsIds, StringComparer.Ordinal);
var header = HeaderForWrite(
currentManifest.FormatVersion,
holdsIds.Values.Select(holds => (bool?)holds),
currentIndex.Values.Where(record => !written.Contains(record.Key)),
holdsIds.Values.Select(holds => (bool?)holds).ToArray(),
copiesAny: currentIndex.Values.Any(record => !written.Contains(record.Key)),
directoryStore: true,
holdsVariants: false);
WriteManifest(targetCatalog, currentIndex.Values
.OrderBy(record => record.Key, StringComparer.Ordinal)
.ToArray(), header);
foreach (var key in written)
Entries[key].StoredHoldsIds = holdsIds[key];
DeleteUnreferencedRecordPages(currentIndex.Values);
_dirtyKeys.Clear();

16
src/GameCult.Caching/Contracts/cultcache-persistence-format.md
View file

@ -182,7 +182,7 @@ A record that resolves to no local type is a foreign record, for example one of
have, or one renamed without declaring its old id. A store never destroys or relabels it: the store carries it
byte for byte under its own id and its own catalog entry, lists it as foreign (`CacheBackingStore.ForeignRecords`
in C#), and the cache never holds it. Writes of other records proceed and copy it forward as the file holds it. A write that would replace or remove it is refused with the typed schema conflict
naming its key and id, and nothing is written. The refused keys are reloaded as the store holds them (a foreign record is not held), and their staged changes are forgotten. A marked header stays marked while a store carries a record the writer cannot show to hold no element id. Declaring its id on a type claims it at the next load.
naming its key and id, and nothing is written. The refused keys are reloaded as the store holds them (a foreign record is not held), and their staged changes are forgotten. A write that carries a record forward keeps a marked header marked. Declaring its id on a type claims it at the next load.
A reader resolves a record's schema id to the entry that owns it, that entry's own `schemaId`, and
only when no entry owns the id to an entry that lists it as a compatible id (the first such entry
@ -311,12 +311,14 @@ The headers are `v1` (no variant, no element ids), `v2` (holds a variant) and `v
an element id, variants or not). A single-file store declares `v3` when any record it
holds carries an element id: a non-empty `[CultElementId]` value in a record's document,
or in a variant's override values. A store that cannot read a variant's override values
(no codec attached) counts the variant as holding one. A whole-store writer decides by
what the store holds now, so the file drops back to `v1` (or `v2` with a variant) when
the last id is taken away; a writer that touches only part of the file cannot see the
rest, so a file already declared `v3` stays `v3` when it merges onto it. A directory
store's manifest says `cultcache.store.v5.directory-content-addressed-pages` under the
same rule, and `v4` otherwise.
(no codec attached) counts the variant as holding one. An element id is an ordinary keyed
string in the payload, so no reader can tell from a record's bytes whether it holds one;
a writer knows it only for the records it encodes. A write that copies any record forward
therefore keeps a `v3` header it found. Only a write that encodes every record the store
will hold decides by content, and drops to `v1` (or `v2` with a variant) when none holds
an id. A directory store's manifest says
`cultcache.store.v5.directory-content-addressed-pages` under the same rule, and `v4`
otherwise.
A single-file store is exactly one MessagePack array of three slots: the header, the schema
catalog and the records. Every runtime refuses a file that is truncated, has bytes after the

42
src/GameCult.Caching/CultCache.cs
View file

@ -526,11 +526,6 @@ namespace GameCult.Caching
// True while this record holds element ids minted when it loaded, that no write has persisted (see MintElementIds).
internal bool IdsInMemoryOnly { get; set; }
// Whether the stored bytes of this record hold an element id: read when a store loaded the record, before any id was minted
// for it, and set when a store wrote it. Null when not known (a variant is not read at load). HoldsIds is what the document
// holds now; this is what the store holds, so a document changed in place and never written does not change it.
internal bool? StoredHoldsIds { get; set; }
// True when some element this record persists holds a non-empty id, and the only thing a store reads to mark its header.
// It is read off the stored form each time it is asked, so a store asks it as it writes: a document changed after it was
// admitted, or loaded and changed in place, writes the header its content needs. A plain record persists its document; a
@ -3462,24 +3457,17 @@ namespace GameCult.Caching
return directoryStore ? DirectoryFormatV4 : holdsVariants ? CultPersistedStoreSnapshot.FormatV2 : CultPersistedStoreSnapshot.FormatV1;
}
// The header a write leaves, for either store kind: HeaderFor decides, given what the write can see. Marked when a record it
// writes holds an id; kept marked while the store carries a record it cannot show holds none (an opaque payload it writes, or
// a copied record that is not known id-free). A copied record is known id-free when this store holds
// that key under the same stored identity as the file and knows its stored bytes hold no id (StoredHoldsIds).
protected string HeaderForWrite(string? durableHeader, IEnumerable<bool?> written, IEnumerable<CultPersistedRecord> copied, bool directoryStore, bool holdsVariants)
{
var holds = written.ToArray();
var holdsIds = holds.Any(value => value == true);
var seen = HeaderFor(holdsIds, durableHeader, wholeStore: true, directoryStore, holdsVariants);
var unseen = HeaderFor(holdsIds, durableHeader, wholeStore: false, directoryStore, holdsVariants);
return seen == unseen || (holds.All(value => value != null) && copied.All(KnownIdFree)) ? seen : unseen;
}
private bool KnownIdFree(CultPersistedRecord record) =>
Entries.TryGetValue(record.Key, out var held) &&
held.StoredSchemaId == record.SchemaId &&
held.StoredAt == record.StoredAt &&
held.StoredHoldsIds == false;
// The header a write leaves, for either store kind: HeaderFor decides. A record's bytes do not say whether it holds an
// element id (an id is an ordinary keyed string), so only the store header carries that fact. A write that copies any
// record forward keeps a marked header; a write that encodes every record the store will hold decides by content. An
// opaque payload (null) never marks an unmarked store, and keeps a marked one marked.
protected static string HeaderForWrite(string? durableHeader, IReadOnlyCollection<bool?> written, bool copiesAny, bool directoryStore, bool holdsVariants) =>
HeaderFor(
holdsIds: written.Any(value => value == true),
durableHeader,
wholeStore: !copiesAny && written.All(value => value != null),
directoryStore,
holdsVariants);
// Once attached, a hold on the cache's gate: the store and its cache share one lock, so a load calling back into
// the cache can never take the gate after the store lock, and a direct call publishes what it loaded when it
@ -3591,8 +3579,7 @@ namespace GameCult.Caching
resolution.Descriptor,
document)
{
StoredSchemaId = record.SchemaId,
StoredHoldsIds = CultElementIds.Holds(document)
StoredSchemaId = record.SchemaId
};
}
@ -3957,8 +3944,8 @@ namespace GameCult.Caching
{
FormatVersion = HeaderForWrite(
durable?.FormatVersion,
upserts.Select(staged => staged.HoldsIds),
copied,
upserts.Select(staged => staged.HoldsIds).ToArray(),
copiesAny: copied.Length > 0,
directoryStore: false,
holdsVariants: records.Any(record => record.Variant != null)),
SchemaCatalog = CultSchemaCatalogEntry.Derive(
@ -4074,7 +4061,6 @@ namespace GameCult.Caching
continue;
entry.StoredSchemaId = write.Record.SchemaId;
entry.StoredAt = write.Record.StoredAt;
entry.StoredHoldsIds = write.HoldsIds;
}
Cache?.Wrote(staged.Select(write => write.Record));