Compare commits

...
Sign in to create a new pull request.
Author SHA1 Message Date
Meta Vens
8c7252eae7 RetypeEvictionTests: a Flatten of a staged variant, then a second Flatten, is refused as staged apart
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 08:07:58 +01:00
Meta Vens
6f31d90a9a A batch stages an operation after a Flatten against the plain record it leaves: a second Flatten and an UpsertVariant are refused, a plain Upsert replaces it; one answer in a batch and staged apart; pinned
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 08:01:15 +01:00
Meta Vens
fec51f59a8 RetypeEvictionTests: Remove, a plain Upsert, then Flatten on a held variant is refused and never drops the plain write
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 08:01:04 +01:00
Meta Vens
b3221c2d56 OutsideStoreJudgeTests: the outside store tracks the versions it took itself (StoredVersion is internal)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 07:25:03 +01:00
Meta Vens
8c0b55c450 OutsideStoreJudgeTests: a store kind in an assembly with no InternalsVisibleTo applies the whole per-key rule through the protected judge
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 07:24:19 +01:00
Meta Vens
d8a2385080 Flatten acts on what the batch stages at the key: a staged variant is flattened as it lands, a removal or plain record there leaves nothing to flatten; pinned
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 07:23:59 +01:00
Meta Vens
a9601cdd48 Validate: an evicted key is absent to the one-key-one-store check, so a one-batch retype across home stores is the batch-spanning refusal with its own text; contract states it; pinned
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 07:23:22 +01:00
Meta Vens
c8563e533f Single-file judge: the removals need not travel as checks (a removal has no upsert); pin the flatten and retype refusal texts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 06:39:26 +01:00
Meta Vens
2239fb9b5f Contract and CHANGELOG: a one-batch retype is an eviction then an add, a rebase follows the plain-write rule, the protected per-key judge
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 06:29:21 +01:00
Meta Vens
1ff5ae79dc Validate: the key-home check needs no eviction exception (an evicted key's home is already in the batch's homes)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 06:26:50 +01:00
Meta Vens
edad333a4d RetypeEvictionTests: RbY lists RbX's real schema id; the refused-flush test refuses k1 under the lock
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 06:20:52 +01:00
Meta Vens
60474bd2b3 Single-file CommitBatch: a deleted key the batch also upserts reaches the judge as its removal half
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 06:19:56 +01:00
Meta Vens
ad51e45183 RetypeEvictionTests: pin the retype as an eviction, the variant and global retypes, the removal bookkeeping and the rebase rule
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 06:19:07 +01:00
Meta Vens
e19d98d0b8 A one-batch retype is the held record's eviction then the add, on every write path; the stores judge through one protected per-key judge
Delete StagedOperation.Replaces and Admit's replaced set. CultCacheBatch records the keys Remove named; Land evicts the held record at each (or checks the key when held by no one) and the add lands after it, so watchers, the globals copy, the variant guard, the key-home check, Claim, the store judges and the change journal see what they see when the two are staged apart. A Flatten after a Remove in the batch is refused as a flatten of a non-variant.

CacheBackingStore gains one protected RefuseUndeclaredOnDisk overload taking the catalog, the held record, the written entry and the removed keys; both shipped stores call it. The old overload and Undeclared stay.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 06:19:06 +01:00
Meta Vens
6f1930b354 ElementIdTests: pin the StoredPublishedBy guard through a push that relabels a loaded record in place
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 05:31:46 +01:00
Meta Vens
009de19879 WriteOwnershipTests: a removal is spent by the flush that judged it
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 05:29:31 +01:00
Meta Vens
20be80cc23 Restore the codec-less variant test's cache; the one-batch retype test keeps the replacement through a later add of the batch
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 05:16:32 +01:00
Meta Vens
3e283c81d4 Keep the StoredPublishedBy staleness guard and pin it: a record minting relabels in place is owned by a newer version that reads its current publisher whole
The probe (a throw where a loaded record is relabelled in place) tripped 32 existing tests, so
the guard is reachable, not dead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 05:14:10 +01:00
Meta Vens
36873e696e VariantTests: the store-less rebase test builds its cache with the codec
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 05:12:19 +01:00
Meta Vens
abf4f0387a WriteOwnershipTests, ForeignRecordTests, VariantTests: pin the one-batch retype, the store-less refusal, the late declarer's removal and the rebase refusal
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 05:11:09 +01:00
Meta Vens
e163e6b006 Retype in one batch or staged apart, and a store-less cache refuses it too
A removal followed by an add at the same key is a retype (variants:ruling:cache-refuses-retype):
the batch's staged add records that it replaces, the key is judged as the removal it names, and
the record lands. The store carries the same fact for removals the cache staged apart (Remove then
UpsertAsync), so the flush no longer reads the add as an upsert over the record it removed and
discards it. A cache with no backing store judges the upsert against the record it holds. The
variant plan's rebase-type check is deleted: write ownership refuses that retype first. The
StoredPublishedBy staleness guard is deleted (a loaded record is never relabelled in place).
Protected RefuseUndeclaredOnDisk doc states the per-key rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 05:11:09 +01:00
Meta Vens
cf9f8dc0e1 WriteOwnershipTests: a batch removal of a staged key is judged as a removal
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-28 04:31:46 +01:00
Meta Vens
d1032b1617 WriteOwnershipTests: judge a stale two-key batch per key; the unattached store cannot carry upserts
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-28 02:45:17 +01:00
Meta Vens
0b147ea519 ForeignRecordTests, VariantTests: retype a key by removal and add, and a late type writes its own record
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-28 02:43:27 +01:00
Meta Vens
244ad2280e WriteOwnershipTests, WrittenOwnsTests: pin the per-key write predicate in both arms and both store kinds
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-28 02:36:17 +01:00
Meta Vens
340c9507e4 Write ownership by the written type: one per-key predicate, the registry leaves upserts
An upsert may replace the record at its key only when the type it writes there owns the record
(CultDocumentRegistry.WrittenOwns): the entry declares the stored id, or reads the stored entry whole
under its own schema name. Cache and cacheless writers, single-file and directory stores ask the same
per-key judge (CacheBackingStore.UndeclaredAt); a cache no longer replaces another registered type's
record. Registry.Owns is the removal/check rule only. Deleted: CatalogEntryLookup's registry half,
CultDocumentRegistry.Lookup, Owns<T>'s mayClaim, the per-batch union and the per-id memo.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-28 02:36:17 +01:00
14 changed files with 2023 additions and 230 deletions

View file

@ -51,6 +51,8 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "GameCult.Caching.ConsumerBu
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "GameCult.Caching.GeneratorConsumer", "tests\GameCult.Caching.GeneratorConsumer\GameCult.Caching.GeneratorConsumer.csproj", "{1A75EA8F-00B3-49AB-8398-B1B7723BE10B}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "GameCult.Caching.OutsideStoreTests", "tests\GameCult.Caching.OutsideStoreTests\GameCult.Caching.OutsideStoreTests.csproj", "{A54C2C29-941A-46A9-9DDE-9711B2F2EADD}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
@ -289,6 +291,18 @@ Global
{1A75EA8F-00B3-49AB-8398-B1B7723BE10B}.Release|x64.Build.0 = Release|Any CPU
{1A75EA8F-00B3-49AB-8398-B1B7723BE10B}.Release|x86.ActiveCfg = Release|Any CPU
{1A75EA8F-00B3-49AB-8398-B1B7723BE10B}.Release|x86.Build.0 = Release|Any CPU
{A54C2C29-941A-46A9-9DDE-9711B2F2EADD}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{A54C2C29-941A-46A9-9DDE-9711B2F2EADD}.Debug|Any CPU.Build.0 = Debug|Any CPU
{A54C2C29-941A-46A9-9DDE-9711B2F2EADD}.Debug|x64.ActiveCfg = Debug|Any CPU
{A54C2C29-941A-46A9-9DDE-9711B2F2EADD}.Debug|x64.Build.0 = Debug|Any CPU
{A54C2C29-941A-46A9-9DDE-9711B2F2EADD}.Debug|x86.ActiveCfg = Debug|Any CPU
{A54C2C29-941A-46A9-9DDE-9711B2F2EADD}.Debug|x86.Build.0 = Debug|Any CPU
{A54C2C29-941A-46A9-9DDE-9711B2F2EADD}.Release|Any CPU.ActiveCfg = Release|Any CPU
{A54C2C29-941A-46A9-9DDE-9711B2F2EADD}.Release|Any CPU.Build.0 = Release|Any CPU
{A54C2C29-941A-46A9-9DDE-9711B2F2EADD}.Release|x64.ActiveCfg = Release|Any CPU
{A54C2C29-941A-46A9-9DDE-9711B2F2EADD}.Release|x64.Build.0 = Release|Any CPU
{A54C2C29-941A-46A9-9DDE-9711B2F2EADD}.Release|x86.ActiveCfg = Release|Any CPU
{A54C2C29-941A-46A9-9DDE-9711B2F2EADD}.Release|x86.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
@ -314,5 +328,6 @@ Global
{FEF9DA88-2177-4FD0-885D-7096ACFA8A9D} = {B48EAE30-152F-01E3-6E31-82B350494D3C}
{D3138F3C-9531-4906-A5D3-74BD857D0685} = {39D316AB-ABB5-49A2-8E83-CAF4C75B48F4}
{1A75EA8F-00B3-49AB-8398-B1B7723BE10B} = {39D316AB-ABB5-49A2-8E83-CAF4C75B48F4}
{A54C2C29-941A-46A9-9DDE-9711B2F2EADD} = {39D316AB-ABB5-49A2-8E83-CAF4C75B48F4}
EndGlobalSection
EndGlobal

View file

@ -185,7 +185,12 @@ public sealed class DirectoryMessagePackBackingStore : CacheBackingStore
manifest.Records.ToDictionary(record => record.Key, record => VersionOf(record.Payload), StringComparer.Ordinal), Entries.Values, ForeignRecords))
return CultCommitOutcome.Mismatch;
RefuseUndeclaredOnDisk(manifest, request.Upserts.Concat(request.Deletes).Select(entry => entry.Key.Value).Concat(request.Checks).Concat(StagedKeys));
RefuseUndeclaredOnDisk(
manifest,
request.Upserts.Concat(request.Deletes).Select(entry => entry.Key.Value).Concat(request.Checks).Concat(StagedKeys),
request.Upserts,
request.Deletes,
request.Checks);
if (request.Upserts.Count == 0 && request.Deletes.Count == 0 && !IsDirty)
{
// Nothing to write: the batch only judged keys it does not hold.
@ -243,7 +248,7 @@ public sealed class DirectoryMessagePackBackingStore : CacheBackingStore
Directory.CreateDirectory(_manifestFile.DirectoryName!);
using var commitLease = AcquireCommitLease(wait: true, create: true);
var (manifest, catalogBytes) = ReadManifest();
RefuseUndeclaredOnDisk(manifest, StagedKeys);
RefuseUndeclaredOnDisk(manifest, StagedKeys, Array.Empty<CultStoredDocument>(), Array.Empty<CultStoredDocument>(), Array.Empty<string>());
if (IsDirty)
WriteGeneration(manifest, catalogBytes);
else
@ -275,7 +280,7 @@ public sealed class DirectoryMessagePackBackingStore : CacheBackingStore
using var commitLease = AcquireCommitLease(wait, create: true);
if (commitLease == null)
return CultCommitOutcome.Contended;
RefuseUndeclaredOnDisk(ReadManifest().Snapshot, keys);
RefuseUndeclaredOnDisk(ReadManifest().Snapshot, keys, Array.Empty<CultStoredDocument>(), Array.Empty<CultStoredDocument>(), Array.Empty<string>());
return CultCommitOutcome.Committed;
});
}
@ -286,14 +291,24 @@ public sealed class DirectoryMessagePackBackingStore : CacheBackingStore
// The manifest's other records stay as they are, foreign and read-only ones included; a write that would replace or remove a
// record the manifest holds that the writer does not own is refused before any page is written, by the one rule every
// store shares (CacheBackingStore.RefuseUndeclaredOnDisk). The refused records reload as a pull loads them.
private void RefuseUndeclaredOnDisk(CultPersistedStoreSnapshot currentManifest, IEnumerable<string> written)
// store shares (CacheBackingStore.UndeclaredAt, CacheBackingStore.RefuseUndeclaredOnDisk). The refused records reload as a
// pull loads them. A key is owned by the entry the write puts there: the batch's upsert, or the staged document. A key it
// removes (the batch's deletes, the staged deletes) or only checks goes to the registry's removal rule.
private void RefuseUndeclaredOnDisk(CultPersistedStoreSnapshot currentManifest, IEnumerable<string> written, IReadOnlyList<CultStoredDocument> upserts, IReadOnlyList<CultStoredDocument> deletes, IEnumerable<string> checks)
{
CultPersistedRecord? HeldAt(string key) =>
currentManifest.Records.FirstOrDefault(record => string.Equals(record.Key, key, StringComparison.Ordinal));
var batch = upserts.ToDictionary(entry => entry.Key.Value, entry => entry.Descriptor, StringComparer.Ordinal);
CultSchemaCatalogEntry? WrittenAt(string key) =>
batch.TryGetValue(key, out var descriptor) ? descriptor.ToCatalogEntry()
: _dirtyKeys.ContainsKey(key) && Entries.TryGetValue(key, out var staged) ? staged.Descriptor.ToCatalogEntry()
: null;
RefuseUndeclaredOnDisk(
written,
key => HeldAt(key) is { } held ? Undeclared(held, currentManifest.SchemaCatalog, Registry.Owns) : null,
currentManifest.SchemaCatalog,
HeldAt,
WrittenAt,
deletes.Select(entry => entry.Key.Value).Concat(checks),
key => HeldAt(key) is { } held && Foreign(held, currentManifest.SchemaCatalog, VersionOf(held.Payload)) == null
? LoadPage(held, currentManifest.SchemaCatalog, out _, out _)
: null,

View file

@ -206,16 +206,16 @@ The rule is checked again where the write lands, under the store's lock: a write
whose durable record is not owned, because another writer rewrote it after this cache
loaded, is refused typed, and the record is reloaded as the store holds it, read-only when a type can
read it. This holds for every write through the store, the store's own `Push`, `Delete` and
`CommitBatch` included. A CultMesh single-file write (`WriteDocument`, `WriteRecord`, or `PushAll` on an unattached store) asks the same rule: it owns the ids its
written entries declare (their own ids and the ids they list compatible) and an id whose publishing entry its claimant reads whole (no persisted
slot unread). The claimant is picked by the one precedence over the written entries and the call's registry, a written entry being the only
description of its schema id (the registry's type for that id is never asked), and the record is owned only when the claimant is a written
entry. A registry adds claimants and declares nothing, so it only adds refusals; with none the written entries are the only claimants. Refused:
a record another registered type claims, under that type's current id or an old one it lists; a record whose stored entry lists another
registered type's id ahead of the written one's, whatever name the stored entry carries; a record two registered types list compatible or two
registered versions of the written schema could read; a record the picked written entry reads only by dropping persisted slots; and the
removal of any stored record by a call that writes no entry. A registry-less `WriteDocument` reads `CultDocumentRegistry.Shared`. It is
refused the same way otherwise.
`CommitBatch` included. Whether a write may replace the record at a key is decided per key by the type it writes there, for a cache
and for a CultMesh single-file write (`WriteDocument`, `WriteRecord`, or `PushAll` on an unattached store) alike, in a single-file store and
a directory store alike: the written type owns the stored record when its entry declares the stored id (its own id, or one it lists
compatible), or when it is the type that reads the stored record's published entry and reads it whole (no persisted slot unread), so an older
id of its own schema name that it reads whole is its own and is restamped. Anything else is refused, another registered type's record
included, and so is a record the written type would read only by dropping persisted slots. No registry enters the decision, `Shared` included:
the answer for a given written entry, stored id and catalog is the same lent a registry or not, and a cache's registered types do not make
each other's records writable. Retyping a key is a removal and an add: in one batch (the removal called before the add) or staged apart, with or without a backing store. The removal half is judged as a removal, so the key is retyped only when the removal rule allows it. A one-batch retype is staged as the held record's eviction and then the add: a watcher of the old type sees Removed and a watcher of the new type sees Added, a global's slot is freed, and every check, validator and store judge sees what it sees when the two are staged apart; this holds for plain records, variants and flattens alike (a Flatten after a Remove in the same batch flattens nothing and is refused). A one-batch retype across two home stores is refused, by the rule that a commit lands in one home store: the removal is in the first store and the add in the second, so the batch spans both. Stage the removal and the add apart, as two commits (not atomic); the key is then in exactly one store. A variant rebased onto a base of another type follows the plain-write rule: where a plain write of the new type would own the variant's record (the new type declares the old id) the rebase commits and the variant becomes that type, and where a plain write would be refused the rebase is refused. A removal or check of a key goes to the
registry's removal rule (a record a registered type declares, or reads whole, may be removed), and a cacheless call, which has no registry,
removes a record only when an entry it writes owns it; a call that writes no entry removes nothing.
A record that resolves to no local type is a foreign record, for example one of a type this build does not
have, or one renamed under a new schema name without declaring its old id. A store never destroys or relabels it: the store carries it

View file

@ -524,6 +524,13 @@ namespace GameCult.Caching
// asked.
internal bool StoredSchemaReadWhole { get; set; }
// The catalog entry that publishes StoredSchemaId in the store this record was loaded from, with the id it was published for,
// so staging can ask WrittenOwns the question the store's judge asks, over the catalog the record came with. Null for a record
// this cache constructed or wrote, which is stored under Descriptor's own id and published by Descriptor's entry. A write that
// relabels a loaded record in place (minting its element ids, then a flush) moves StoredSchemaId and leaves this behind, so a
// reader uses it only while the id still matches StoredSchemaId.
internal (string Id, CultSchemaCatalogEntry Entry)? StoredPublishedBy { get; set; }
// Names the bytes this record was read from or written as: the SHA-256 of its persisted encoding, set by the store that
// read or wrote them. A record no store has read or written has a version of its own that equals nothing else.
internal string StoredVersion { get; set; }
@ -540,6 +547,9 @@ namespace GameCult.Caching
// A plain record staged by Flatten: the one way a plain record may replace a variant at its key.
internal bool Flattens { get; set; }
// A plain record a batch staged after a Flatten at its key: it replaces the plain record that flatten left, so it may land on the variant the cache holds.
internal bool ReplacesFlattened { get; set; }
// True while this record holds element ids minted when it loaded, that no write has persisted (see MintElementIds).
internal bool IdsInMemoryOnly { get; set; }
@ -593,45 +603,28 @@ namespace GameCult.Caching
public string TypeLabel { get; }
}
// The lookup of a cacheless writer: the catalog entries the call writes, and the lent registry's other types. A written entry is
// the only description of its schema id, so the registry's type for that id is never asked: a registry adds claimants but never
// re-describes the written type.
// The lookup a write's ownership asks: only the catalog entries the call writes. No registry enters it, so the answer is the
// written type's own (variants:ruling:write-ownership-by-written-type).
internal sealed class CatalogEntryLookup : IClaimantLookup<CultSchemaCatalogEntry>
{
private readonly IReadOnlyCollection<CultSchemaCatalogEntry> _entries;
private readonly IClaimantLookup<CultDocumentDescriptor>? _others;
public CatalogEntryLookup(IReadOnlyCollection<CultSchemaCatalogEntry> entries, CultDocumentRegistry? others)
public CatalogEntryLookup(IReadOnlyCollection<CultSchemaCatalogEntry> entries)
{
_entries = entries;
_others = others?.Lookup;
}
public CultSchemaCatalogEntry? Owner(string schemaId) =>
_entries.FirstOrDefault(entry => string.Equals(entry.SchemaId, schemaId, StringComparison.Ordinal))
?? Other(_others?.Owner(schemaId));
_entries.FirstOrDefault(entry => string.Equals(entry.SchemaId, schemaId, StringComparison.Ordinal));
public IReadOnlyList<CultSchemaCatalogEntry> Listing(string schemaId) =>
_entries.Where(entry => entry.CompatibleSchemaIds.Contains(schemaId, StringComparer.Ordinal))
.Concat(Others(_others?.Listing(schemaId)))
.ToArray();
_entries.Where(entry => entry.CompatibleSchemaIds.Contains(schemaId, StringComparer.Ordinal)).ToArray();
public IReadOnlyList<CultSchemaCatalogEntry> Named(string schemaName) =>
_entries.Where(entry => string.Equals(entry.SchemaName, schemaName, StringComparison.Ordinal))
.Concat(Others(_others?.Named(schemaName)))
.ToArray();
_entries.Where(entry => string.Equals(entry.SchemaName, schemaName, StringComparison.Ordinal)).ToArray();
public ClaimantFacts Describe(CultSchemaCatalogEntry claimant) =>
new(claimant.SchemaName, claimant.SchemaVersion, claimant.SchemaId);
// A registry type counts only when no written entry has its schema id.
private CultSchemaCatalogEntry? Other(CultDocumentDescriptor? descriptor) =>
descriptor == null || _entries.Any(entry => string.Equals(entry.SchemaId, descriptor.SchemaId, StringComparison.Ordinal))
? null
: descriptor.ToCatalogEntry();
private IEnumerable<CultSchemaCatalogEntry> Others(IEnumerable<CultDocumentDescriptor>? descriptors) =>
(descriptors ?? Array.Empty<CultDocumentDescriptor>()).Select(Other).OfType<CultSchemaCatalogEntry>();
}
public sealed class CultDocumentRegistry
@ -832,29 +825,42 @@ namespace GameCult.Caching
catalog.FirstOrDefault(entry => string.Equals(entry.SchemaId, schemaId, StringComparison.Ordinal))
?? catalog.FirstOrDefault(entry => entry.CompatibleSchemaIds.Contains(schemaId, StringComparer.Ordinal));
// The one answer to whether a writer may write, relabel or remove a record stored under this id
// (variants:ruling:drift-record-write-ownership): a registered type declares the id (owns it or lists it compatible), or the
// type that reads it reads it whole. Read live, so a type registered later that declares the id claims its records at their
// next write. A record read only by dropping slots is not owned: declaring its id compatible is the consent to the loss.
// Header only: the persisted catalog entry that publishes the id is compared with the claimant's entry, never a payload. A
// claimant that cannot be named (ambiguous, id missing from the catalog, shapes incompatible) owns nothing.
// Whether a removal, or a check of a key, may proceed over a record stored under this id, and the load-time marking of a
// record read-only: a registered type declares the id (owns it or lists it compatible), or the type that reads it reads it
// whole (variants:ruling:drift-record-write-ownership). It never decides an upsert: WrittenOwns does
// (variants:ruling:write-ownership-by-written-type). Read live, so a type registered later that declares the id claims its
// records at their next removal. A record read only by dropping slots is not owned: declaring its id compatible is the
// consent to the loss. Header only: the persisted catalog entry that publishes the id is compared with the claimant's
// entry, never a payload. A claimant that cannot be named (ambiguous, id missing from the catalog, shapes incompatible) owns
// nothing.
internal bool Owns(string schemaId, IReadOnlyCollection<CultSchemaCatalogEntry> catalog) =>
Owns(schemaId, catalog, Declared, _indexes, descriptor => descriptor.ToCatalogEntry(), _ => true);
Owns(schemaId, catalog, Declared, _indexes, descriptor => descriptor.ToCatalogEntry());
// The registry's lookup, for a cacheless writer that lends it only its other types.
internal IClaimantLookup<CultDocumentDescriptor> Lookup => _indexes;
// The one answer to whether an upsert of the written type may replace the record stored under this id at its key, for a cache
// and a cacheless writer, a single-file store and a directory store alike. The written entry owns the record when it declares
// the stored id (its own id, or one it lists compatible), or when it is the type that reads the record's published entry
// (the one Claimant precedence over this one entry) and reads it whole, so an older id of its own schema name that it reads
// whole is its own (variants:ruling:own-entry-reads-whole). Anything else, another registered type's record included, is
// refused: retyping a key is a removal and an add (variants:ruling:cache-refuses-retype). No registry enters it.
internal static bool WrittenOwns(
CultSchemaCatalogEntry written,
string storedId,
IReadOnlyCollection<CultSchemaCatalogEntry> catalog) =>
Owns(storedId, catalog, id => DeclaredBy(written, id), new CatalogEntryLookup(new[] { written }), entry => entry);
// The rule itself, for any writer: declared, or the claimant (picked by the one Claimant over a lookup) reads the record's
// published entry whole. The lookup is a registry's indexes, or the entries a registry-less cacheless writer writes. mayClaim
// is a condition on the claimant's answer, not a second lookup: a cache accepts any claimant, a cacheless writer only one of
// the entries the call writes, so it never owns a record another registered type reads.
// WrittenOwns' declared half: the entry's own id or one it lists compatible.
internal static bool DeclaredBy(CultSchemaCatalogEntry written, string storedId) =>
string.Equals(written.SchemaId, storedId, StringComparison.Ordinal) ||
written.CompatibleSchemaIds.Contains(storedId, StringComparer.Ordinal);
// The rule itself: declared, or the claimant (picked by the one Claimant over a lookup) reads the record's published entry
// whole. The lookup is a registry's indexes (the removal rule) or the one entry a write puts at the key (WrittenOwns).
internal static bool Owns<T>(
string schemaId,
IReadOnlyCollection<CultSchemaCatalogEntry> catalog,
Func<string, bool> declared,
IClaimantLookup<T> lookup,
Func<T, CultSchemaCatalogEntry> entryOf,
Func<CultSchemaCatalogEntry, bool> mayClaim)
Func<T, CultSchemaCatalogEntry> entryOf)
where T : class
{
if (declared(schemaId))
@ -864,8 +870,7 @@ namespace GameCult.Caching
var claimant = Claimant(schemaId, catalog, lookup, out var persisted);
if (persisted == null || claimant == null)
return false;
var claimantEntry = entryOf(claimant);
return mayClaim(claimantEntry) && ReadsWhole(persisted, claimantEntry);
return ReadsWhole(persisted, entryOf(claimant));
}
catch (InvalidOperationException)
{
@ -2074,8 +2079,9 @@ namespace GameCult.Caching
// One staged write: a plain record (Stored), a variant (Variant), a flatten, or, with none of those, a removal.
internal sealed class StagedOperation
{
public StagedOperation(CultRecordKey key, CultStoredDocument? stored, CultVariantDelta? variant, bool flatten)
public StagedOperation(CultRecordKey key, CultStoredDocument? stored, CultVariantDelta? variant, bool flatten, bool afterFlatten = false)
{
AfterFlatten = afterFlatten;
Key = key;
Stored = stored;
Variant = variant;
@ -2086,6 +2092,8 @@ namespace GameCult.Caching
public CultStoredDocument? Stored { get; }
public CultVariantDelta? Variant { get; }
public bool Flatten { get; }
// A plain record staged after a Flatten at its key: the key holds a plain record by then, so it replaces that, not a variant.
public bool AfterFlatten { get; }
public bool IsRemoval => Stored == null && Variant == null && !Flatten;
}
@ -2094,6 +2102,9 @@ namespace GameCult.Caching
{
private readonly CultCache _cache;
internal readonly Dictionary<string, StagedOperation> Operations = new(StringComparer.Ordinal);
// Every key Remove named, in whatever order: Land evicts the record the cache holds at each (or checks the key when it holds
// none) before anything lands, so a record staged at a removed key is an add after an eviction, as when staged apart.
internal readonly HashSet<string> Removals = new(StringComparer.Ordinal);
internal readonly List<(CultRecordKey Key, string? Version)> Expected = new();
internal bool ExpectsUnchanged;
internal bool Sealed;
@ -2118,6 +2129,7 @@ namespace GameCult.Caching
public void Remove(CultRecordKey key)
{
ThrowIfSealed();
Removals.Add(key.Value);
Operations[key.Value] = new StagedOperation(key, null, null, false);
}
@ -2126,6 +2138,10 @@ namespace GameCult.Caching
public CultRecordKey UpsertVariant(CultRecordKey key, CultRecordKey baseKey, IEnumerable<CultVariantOverride>? overrides = null)
{
ThrowIfSealed();
// A flatten staged here leaves a plain record at the key, which cannot become a variant in place, as when staged apart.
if (Operations.TryGetValue(key.Value, out var flattened) && flattened.Flatten)
throw new InvalidOperationException(
$"Record {key.Value} is a plain record; it cannot become a variant of {baseKey.Value} in place. Use a new key.");
Operations[key.Value] = new StagedOperation(
key, null, new CultVariantDelta(baseKey.Value, (overrides ?? Array.Empty<CultVariantOverride>()).ToArray()), false);
return key;
@ -2135,6 +2151,17 @@ namespace GameCult.Caching
public void Flatten(CultRecordKey key)
{
ThrowIfSealed();
// A flatten acts on what the key will hold, as staged apart: a variant staged here is flattened as it lands; a removal or a
// plain record staged here, or an earlier flatten, leaves nothing to flatten; with nothing staged it flattens the variant
// the cache holds.
if (Operations.TryGetValue(key.Value, out var staged))
{
if (staged.Variant == null || staged.Flatten)
throw new InvalidOperationException($"Flatten({key.Value}): the record is not a variant.");
Operations[key.Value] = new StagedOperation(key, null, staged.Variant, true);
return;
}
Operations[key.Value] = new StagedOperation(key, null, null, true);
}
@ -2155,7 +2182,8 @@ namespace GameCult.Caching
{
ThrowIfSealed();
var stored = _cache.CreateStoredDocument(document, key);
Operations[stored.Key.Value] = new StagedOperation(stored.Key, stored, null, false);
Operations[stored.Key.Value] = new StagedOperation(
stored.Key, stored, null, false, afterFlatten: Operations.TryGetValue(stored.Key.Value, out var prior) && prior.Flatten);
return stored.Key;
}
@ -2555,6 +2583,7 @@ namespace GameCult.Caching
Admit(Array.Empty<CultStoredDocument>(), new[] { existing }, null, (home, _) =>
{
home?.Delete(existing);
home?.NoteRemoved(existing.Key.Value);
return CultCommitOutcome.Committed;
});
return true;
@ -2715,12 +2744,13 @@ namespace GameCult.Caching
}
// Call under the gate that admits the result, so no load can land a later storedAt in between.
private CultStoredDocument Stamp(CultRecordKey key, CultDocumentDescriptor descriptor, object document) =>
new(key, MintStoredAt(PreviousStoredAt(key), Clock()), descriptor, document);
private CultStoredDocument Stamp(CultRecordKey key, CultDocumentDescriptor descriptor, object document, bool replacesFlattened = false) =>
new(key, MintStoredAt(PreviousStoredAt(key), Clock()), descriptor, document) { ReplacesFlattened = replacesFlattened };
// An unresolved variant: the admitting hold resolves it against its base before anything lands.
private CultStoredDocument StampVariant(CultRecordKey key, CultDocumentDescriptor descriptor, CultVariantDelta delta) =>
new(key, MintStoredAt(PreviousStoredAt(key), Clock()), descriptor, delta, null, _codec);
// A staged variant that a Flatten followed lands as the plain record of its resolution (flattens).
private CultStoredDocument StampVariant(CultRecordKey key, CultDocumentDescriptor descriptor, CultVariantDelta delta, bool flattens = false) =>
new(key, MintStoredAt(PreviousStoredAt(key), Clock()), descriptor, delta, null, _codec) { Flattens = flattens };
// The plain record a Flatten stages: the variant's resolved document, under the variant's own key and schema.
private CultStoredDocument StampFlatten(CultRecordKey key)
@ -2820,20 +2850,16 @@ namespace GameCult.Caching
// Staging stamped outside this gate; a pull since then may hold a later storedAt, so stamp again.
var upserts = batch.Operations.Values
.Where(op => !op.IsRemoval)
.Select(op => op.Stored != null ? Stamp(op.Key, op.Stored.Descriptor, op.Stored.Document)
: op.Variant != null ? StampVariant(op.Key, DescriptorOfBase(op.Key, op.Variant.BaseKey, batch), op.Variant)
.Select(op => op.Stored != null ? Stamp(op.Key, op.Stored.Descriptor, op.Stored.Document, op.AfterFlatten)
: op.Variant != null ? StampVariant(op.Key, DescriptorOfBase(op.Key, op.Variant.BaseKey, batch), op.Variant, op.Flatten)
: StampFlatten(op.Key))
.ToArray();
// A removal of a key the cache does not hold removes nothing: it is a check every store the commit reaches judges under its lock.
var checks = batch.Operations.Values
.Where(op => op.IsRemoval && !_entries.ContainsKey(op.Key.Value))
.Select(op => op.Key.Value)
.ToArray();
var deletes = batch.Operations.Values
.Where(op => op.IsRemoval && _entries.ContainsKey(op.Key.Value))
.Select(op => _entries[op.Key.Value])
.ToArray();
return Admit(upserts, deletes, null, (home, resolved) =>
// A removal evicts the record the cache holds at the key, and a record staged at the key lands after the eviction, as
// when the two are staged apart (variants:ruling:one-batch-retype-is-eviction-then-add). A removal of a key the cache
// does not hold removes nothing: it is a check every store the commit reaches judges under its lock.
var deletes = batch.Removals.Where(_entries.ContainsKey).Select(key => _entries[key]).ToArray();
var checks = batch.Removals.Where(key => !_entries.ContainsKey(key)).ToArray();
return Admit(upserts, deletes, null, removed: batch.Removals, land: (home, resolved) =>
{
// A batch that only checks keys names no home; it commits through the cache's one writable store, if it has one.
if (home == null && checks.Length > 0 && _stores.Where(entry => !entry.Store.IsReadOnly).Take(2).ToArray() is { Length: 1 } only)
@ -2883,11 +2909,12 @@ namespace GameCult.Caching
IReadOnlyList<CultStoredDocument> admitted,
IReadOnlyList<CultStoredDocument> evicted,
CacheBackingStore? source,
Func<CacheBackingStore?, IReadOnlyList<CultStoredDocument>, CultCommitOutcome> land) => Held(() =>
Func<CacheBackingStore?, IReadOnlyList<CultStoredDocument>, CultCommitOutcome> land,
IReadOnlyCollection<string>? removed = null) => Held(() =>
{
if (_held == null)
throw new InvalidOperationException("An admission reached the cache under a plain lock on its gate; every admission runs in a hold.");
var home = Validate(admitted, evicted, source);
var home = Validate(admitted, evicted, source, removed);
// Ids are decided before anything lands, and the caller's documents change only inside this admission: a plan that
// refuses (a duplicate, a malformed id) has changed nothing, and a write that does not commit gives its ids back.
// A write mints random ids; a load of a record written before ids existed mints the ids its reload will mint
@ -3095,30 +3122,34 @@ namespace GameCult.Caching
private CacheBackingStore? Validate(
IReadOnlyList<CultStoredDocument> admitted,
IReadOnlyList<CultStoredDocument> evicted,
CacheBackingStore? source)
CacheBackingStore? source,
IReadOnlyCollection<string>? removed)
{
var removedKeys = removed?.ToHashSet(StringComparer.Ordinal) ?? new HashSet<string>(StringComparer.Ordinal);
var homes = new List<CacheBackingStore>();
var globals = new Dictionary<Type, string>(_globals);
var evictedKeys = evicted.Select(stored => stored.Key.Value).ToHashSet(StringComparer.Ordinal);
foreach (var stored in evicted)
{
var type = stored.Descriptor.DocumentType;
if (globals.TryGetValue(type, out var key) && key == stored.Key.Value)
globals.Remove(type);
if (source == null)
Claim(stored);
Claim(stored, upsert: false);
}
foreach (var stored in admitted)
{
var descriptor = stored.Descriptor;
if (source == null)
Claim(stored);
Claim(stored, upsert: !removedKeys.Contains(stored.Key.Value));
else if (Home(descriptor.DocumentType) is var home && home != source)
throw new InvalidOperationException(
$"{descriptor.SchemaName} record {stored.Key.Value} was loaded from {source} but its home is {home?.ToString() ?? "no store"}.");
// One key, one store: a key never moves between homes, and two stores never both deliver it.
if (_entries.TryGetValue(stored.Key.Value, out var present) &&
// One key, one store: a key never moves between homes, and two stores never both deliver it. An evicted key is absent
// here, as it is staged apart; a retype across two home stores is then the batch spanning both (below): one rule, one text.
if (!evictedKeys.Contains(stored.Key.Value) && _entries.TryGetValue(stored.Key.Value, out var present) &&
Home(present.Descriptor.DocumentType) is var held &&
Home(descriptor.DocumentType) is var incoming &&
held != incoming)
@ -3137,23 +3168,27 @@ namespace GameCult.Caching
var distinct = homes.Distinct().ToArray();
if (distinct.Length > 1)
throw new InvalidOperationException(
$"The batch spans {distinct[0]} and {distinct[1]}; a commit lands in one home store.");
$"The batch spans {distinct[0]} and {distinct[1]}; a commit lands in one home store. " +
"To move a key between stores, stage its removal and its add apart (two commits, not one atomic commit).");
return distinct.FirstOrDefault();
void Claim(CultStoredDocument stored)
void Claim(CultStoredDocument stored, bool upsert)
{
var home = Home(stored.Descriptor.DocumentType);
if (home == null)
{
if (_stores.Count > 0)
throw new InvalidOperationException($"No backing store is home to {stored.Descriptor.SchemaName}.");
// No store, no file to judge, but the same rule: a record is replaced only by its own type's write.
if (upsert && _entries.TryGetValue(stored.Key.Value, out var inMemory) && !CacheBackingStore.UpsertOwned(inMemory, stored.Descriptor))
throw CacheBackingStore.Retyped(inMemory, stored.Key.Value, stored.Descriptor);
return;
}
if (home.IsReadOnly)
throw new InvalidOperationException(
$"{stored.Descriptor.SchemaName} record {stored.Key.Value} routes to read-only backing store {home}.");
home.RefuseUndeclared(stored.Key.Value);
home.RefuseUndeclared(stored.Key.Value, upsert ? stored.Descriptor : null);
homes.Add(home);
}
}
@ -3264,23 +3299,21 @@ namespace GameCult.Caching
// A variant is written only through the variant API: a plain record never lands on a variant's key, and the
// reverse is a rebase or a flatten, never an accident. A load reports what is on disk and is not a write.
// A rebase onto a base of another type would retype the key; write ownership refused that in Claim before this runs.
if (source == null)
{
foreach (var stored in admitted)
{
_entries.TryGetValue(stored.Key.Value, out var present);
if (stored.Variant == null && !stored.Flattens && present?.Variant != null)
if (gone.Contains(stored.Key.Value))
present = null;
if (stored.Variant == null && !stored.Flattens && !stored.ReplacesFlattened && present?.Variant != null)
throw new InvalidOperationException(
$"Record {stored.Key.Value} is a variant of {present.Variant.BaseKey}; a plain write cannot replace it. " +
"Change it through the variant API, or Flatten it.");
if (stored.Variant != null && present != null && present.Variant == null)
throw new InvalidOperationException(
$"Record {stored.Key.Value} is a plain record; it cannot become a variant of {stored.Variant.BaseKey} in place. Use a new key.");
if (stored.Variant != null && present?.Variant != null &&
present.Descriptor.DocumentType != stored.Descriptor.DocumentType)
throw new InvalidOperationException(
$"Variant {stored.Key.Value} is a {present.Descriptor.DocumentType.Name}; " +
$"rebasing it onto {stored.Variant.BaseKey} would make it a {stored.Descriptor.DocumentType.Name}.");
}
}
@ -3399,7 +3432,7 @@ namespace GameCult.Caching
?? throw new InvalidOperationException($"Variant {key} resolved to nothing.");
stack.RemoveAt(stack.Count - 1);
return done[key] = isVariant
return done[key] = isVariant && !variant.Flattens
? variant.Resolved(document, minted ? new CultVariantDelta(delta.BaseKey, overrides) : delta, variant.IdsInMemoryOnly || (minted && source != null), _codec)
: new CultStoredDocument(variant.Key, variant.StoredAt, variant.Descriptor, document)
{
@ -3944,6 +3977,7 @@ namespace GameCult.Caching
{
StoredSchemaId = record.SchemaId,
StoredSchemaReadWhole = readWhole,
StoredPublishedBy = PublishedByIn(record.SchemaId, catalog),
StoredVersion = version
};
}
@ -3968,10 +4002,14 @@ namespace GameCult.Caching
{
StoredSchemaId = record.SchemaId,
StoredSchemaReadWhole = readWhole,
StoredPublishedBy = PublishedByIn(record.SchemaId, catalog),
StoredVersion = version
};
}
private static (string Id, CultSchemaCatalogEntry Entry)? PublishedByIn(string schemaId, IReadOnlyCollection<CultSchemaCatalogEntry> catalog) =>
CultDocumentRegistry.PublishedBy(schemaId, catalog) is { } entry ? (schemaId, entry) : null;
protected void SetLastSchemaMigrationReports(IEnumerable<CultSchemaMigrationReport> reports)
{
_lastSchemaMigrationReports = reports?.ToArray() ?? Array.Empty<CultSchemaMigrationReport>();
@ -3991,19 +4029,46 @@ namespace GameCult.Caching
_foreignRecords = foreign.OrderBy(record => record.Key, StringComparer.Ordinal).ToArray();
}
// A cache never writes, relabels or removes a record it does not own: the staging half of rule 3. Refused when the store
// carries a foreign record at the key, or holds one that is not owned: its stored id is not declared by a registered type and
// the type that reads it drops persisted slots (read-only). A record read whole is owned; ToStoredDocument decided that from
// the same comparison Owns makes. Called for every record a cache write admits or evicts, before anything is staged.
protected internal void RefuseUndeclared(string key)
// A cache never writes, relabels or removes a record it does not own: the staging half of rule 3. A removal or check of a key
// is refused when the store carries a foreign record at the key, or holds one that is not owned: its stored id is not
// declared by a registered type and the type that reads it drops persisted slots (read-only). A record read whole is owned;
// ToStoredDocument decided that from the same comparison Owns makes.
protected internal void RefuseUndeclared(string key) => RefuseUndeclared(key, null);
// The same, for every record a cache write admits or evicts, before anything is staged: an upsert (written is its type) may
// replace only a record that type owns, WrittenOwns' rule (variants:ruling:write-ownership-by-written-type), asked over the
// catalog entry that published the held record's id when it was loaded, the question the store's judge asks of the file.
internal void RefuseUndeclared(string key, CultDocumentDescriptor? written)
{
var foreign = _foreignRecords.FirstOrDefault(record => string.Equals(record.Key, key, StringComparison.Ordinal));
if (foreign != null)
throw Overwrites(null, new[] { (foreign.Key, foreign.SchemaId, foreign.SchemaName) });
if (Entries.TryGetValue(key, out var held) && !Registry.Declared(held.StoredSchemaId) && !held.StoredSchemaReadWhole)
if (Entries.TryGetValue(key, out var held) && !HeldOwned(held, written))
throw Overwrites(null, new[] { (key, held.StoredSchemaId, held.Descriptor.SchemaName) });
}
private bool HeldOwned(CultStoredDocument held, CultDocumentDescriptor? written) =>
written == null
? Registry.Declared(held.StoredSchemaId) || held.StoredSchemaReadWhole
: UpsertOwned(held, written);
// Whether a write of the written type may replace the record a cache holds: WrittenOwns over the entry that published the held
// record's id when it was loaded (a record this cache wrote is published by its own type). The one answer for a store-backed
// cache's staging and a store-less cache's.
internal static bool UpsertOwned(CultStoredDocument held, CultDocumentDescriptor written) =>
CultDocumentRegistry.WrittenOwns(
written.ToCatalogEntry(),
held.StoredSchemaId,
new[] { held.StoredPublishedBy is { } loaded && string.Equals(loaded.Id, held.StoredSchemaId, StringComparison.Ordinal) ? loaded.Entry : held.Descriptor.ToCatalogEntry() });
// The refusal of a store-less cache that would retype a key.
internal static CultSchemaConflictException Retyped(CultStoredDocument held, string key, CultDocumentDescriptor written) => new(
$"Record '{key}' is a {held.Descriptor.SchemaName} (schema id '{held.StoredSchemaId}'); writing a {written.SchemaName} there would retype the key. " +
"Remove the record first, in the same batch or an earlier commit, then add the other type's.",
held.StoredSchemaId,
new[] { held.Descriptor.SchemaName },
key);
/// <summary>
/// Called when the cache removes keys it does not hold, outside a batch that commits through this store. Such a removal
/// removes nothing, but a store that holds a record at one of these keys under an id no registered type declares must
@ -4069,6 +4134,17 @@ namespace GameCult.Caching
internal bool HasStagedChecks => _checks.Count > 0;
// Keys this store's cache removed here since its last flush. A write of the same key after that is a removal and an add
// (variants:ruling:cache-refuses-retype): the key is judged as the removal it names, not as an upsert over the record it held.
// Cleared like the checks, by a refusal of the key or a successful flush or commit.
private readonly HashSet<string> _removed = new(StringComparer.Ordinal);
internal void NoteRemoved(string key) => Held(() => _removed.Add(key));
// Every key a removal named since the last flush (held or not), which this store judges as a removal even when a record is
// staged there again.
internal IReadOnlyCollection<string> StagedRemovals => _removed.Concat(_checks).ToArray();
// A stored record under an id the writer does not own, named by the catalog entry that publishes the id. Header only: no
// payload is decoded. Null for a record under an owned id.
protected static (string Key, string SchemaId, string SchemaName)? Undeclared(
@ -4077,9 +4153,91 @@ namespace GameCult.Caching
? null
: (record.Key, record.SchemaId, CultDocumentRegistry.PublishedBy(record.SchemaId, catalog)?.SchemaName ?? string.Empty);
// The apply half of rule 3, for every store kind, checked where the write lands under the store's lock. A key this write names
// whose durable record is under an id no registered type declares is refused, every such key at once, through Refuse, which
// reloads it as the store holds it: listed foreign when no type reads it, held read-only when one does.
// The one per-key judge of a write over a durable catalog, for every store kind: the stored record at each key the write names
// is owned by the entry the write puts at that key (WrittenOwns), or, when the write only removes or checks the key, by the
// removal rule. heldAt gives the durable record at a key and writtenAt the catalog entry the write puts there (null for a
// removal or a check). Memoised per written schema id and stored id, so a large store pays one comparison per pair, and an
// answer for one written type is never reused for another.
internal static Func<string, (string Key, string SchemaId, string SchemaName)?> UndeclaredAt(
Func<string, CultPersistedRecord?> heldAt,
IReadOnlyCollection<CultSchemaCatalogEntry> catalog,
Func<string, CultSchemaCatalogEntry?> writtenAt,
Func<string, IReadOnlyCollection<CultSchemaCatalogEntry>, bool> removalOwns)
{
var answers = new Dictionary<(string? Written, string Stored), bool>();
return key =>
{
if (heldAt(key) is not { } held)
return null;
var written = writtenAt(key);
return Undeclared(held, catalog, (storedId, entries) =>
{
var asked = (written?.SchemaId, storedId);
if (!answers.TryGetValue(asked, out var answer))
answers[asked] = answer = written == null
? removalOwns(storedId, entries)
: CultDocumentRegistry.WrittenOwns(written, storedId, entries);
return answer;
});
};
}
/// <summary>
/// The per-key ownership judge of a write, for a store kind outside this library: the rule the shipped stores apply under their
/// lock, as one call. Each key in <paramref name="written"/> and in <paramref name="removed"/> is judged against the durable
/// record <paramref name="heldAt"/> gives for it. A key the write puts a record at (<paramref name="writtenAt"/> gives the
/// catalog entry of the record written there) is owned only by that type: a record under its own schema id, an id it lists
/// compatible, or an older id of its own schema name it reads whole; another registered type's record is refused. A key the
/// write removes or only checks, and every key this store's cache removed since the last flush or commit (the removal half of
/// a retype, whose add then lands at it), is owned by any registered type that declares or reads the record whole. A refusal is
/// <see cref="Refuse"/>: it reloads what it refuses, unstages it through <paramref name="unstage"/>, and throws.
/// </summary>
/// <param name="written">Every key the write names.</param>
/// <param name="catalog">The schema catalog of the durable store.</param>
/// <param name="heldAt">The durable record at a key, or null.</param>
/// <param name="writtenAt">The catalog entry of the record the write puts at a key, or null when it puts none.</param>
/// <param name="removed">The keys the write removes or checks.</param>
/// <param name="durableAt">See <see cref="Refuse"/>.</param>
/// <param name="foreignAt">See <see cref="Refuse"/>.</param>
/// <param name="unstage">See <see cref="Refuse"/>.</param>
/// <param name="cacheless">True for a write no cache made: its refusal is the cacheless one.</param>
protected void RefuseUndeclaredOnDisk(
IEnumerable<string> written,
IReadOnlyCollection<CultSchemaCatalogEntry> catalog,
Func<string, CultPersistedRecord?> heldAt,
Func<string, CultSchemaCatalogEntry?> writtenAt,
IEnumerable<string> removed,
Func<string, CultStoredDocument?> durableAt,
Func<string, CultForeignRecord?> foreignAt,
Action<string> unstage,
bool cacheless = false)
{
var keys = written.Distinct(StringComparer.Ordinal).ToArray();
var removedKeys = removed.Concat(StagedRemovals).ToHashSet(StringComparer.Ordinal);
// A cacheless call has no registry that decides a removal: the entries it writes do (a call that writes no entry owns
// nothing, variants:ruling:mesh-writes-bound-by-declaration).
var entries = keys.Select(writtenAt).OfType<CultSchemaCatalogEntry>().ToArray();
Func<string, IReadOnlyCollection<CultSchemaCatalogEntry>, bool> removalOwns = Cache != null
? Registry.Owns
: (id, entriesInCatalog) => entries.Any(entry => CultDocumentRegistry.WrittenOwns(entry, id, entriesInCatalog));
RefuseUndeclaredOnDisk(
keys,
UndeclaredAt(heldAt, catalog, key => removedKeys.Contains(key) ? null : writtenAt(key), removalOwns),
durableAt,
foreignAt,
unstage,
cacheless);
}
// The apply half of rule 3, for every store kind, checked where the write lands under the store's lock. undeclaredAt is the
// store's per-key judge: for each key the write names it says whether the durable record there is one the write may not
// replace, remove or relabel. The rule it must apply is per key and per written type, never one predicate per stored id: a
// key the write puts a record at is owned only by the type written there (its own id, an id it lists, or an older id of its
// own schema name it reads whole), so another registered type's record is refused; a key the write only removes or checks
// is owned by any registered type that declares or reads it whole; and a key the cache removed and the write then adds again
// is judged as the removal it names. Every key at once is refused through Refuse, which reloads it as the store holds it:
// listed foreign when no type reads it, held read-only when one does. The shipped stores call the overload that builds
// this judge; this one takes a judge a store kind outside the library built itself.
protected void RefuseUndeclaredOnDisk(
IEnumerable<string> written,
Func<string, (string Key, string SchemaId, string SchemaName)?> undeclaredAt,
@ -4147,6 +4305,7 @@ namespace GameCult.Caching
{
unstage(key);
_checks.Remove(key);
_removed.Remove(key);
}
foreach (var held in dropped)
Entries.TryRemove(held.Key.Value, out _);
@ -4203,6 +4362,7 @@ namespace GameCult.Caching
{
IsDirty = false;
_checks.Clear();
_removed.Clear();
}
}
@ -4424,7 +4584,9 @@ namespace GameCult.Caching
return CultCommitOutcome.Mismatch;
var (upserts, removals) = StagedWrites(request.Upserts, request.Deletes);
var written = ApplyWriteSet(disk, upserts, removals, request.Checks);
// A key the batch deletes and also upserts is a retype: the upsert wins in the write set, so the deletion travels as a check,
// and the judge reads the key as the removal it names.
var written = ApplyWriteSet(disk, upserts, removals, request.Checks.Concat(request.Deletes.Select(entry => entry.Key.Value)));
foreach (var entry in request.Deletes)
Entries.TryRemove(entry.Key.Value, out _);
foreach (var entry in request.Upserts)
@ -4484,7 +4646,7 @@ namespace GameCult.Caching
/// <param name="record">The record to store.</param>
/// <param name="catalogEntry">The schema catalog entry for the record's schema id.</param>
/// <param name="holdsIds">Whether the record's payload holds element ids, or null when the caller cannot tell.</param>
/// <param name="registry">The registry the payload was encoded with. Given, the write decides the header for copied records that registry can read, and its other types also claim stored records as a cache registering them would; it declares nothing, so it can only refuse more. <paramref name="catalogEntry"/> stays the only description of its own schema id and the registry's type for that id is never asked. Owned are a record whose id <paramref name="catalogEntry"/> declares (its own or one it lists compatible) and a record the entry reads whole when it is the picked claimant; refused are a record another registered type claims, one whose stored entry lists another registered type's id ahead of the entry's, and one that two registered types list or two versions of the schema could read. Omitted, it decides nothing for copied records and the one entry is the only claimant. It serves this call only: the store keeps no registry.</param>
/// <param name="registry">The registry the payload was encoded with. Given, the write decides the header for copied records that registry can read. It never decides ownership: the answer is the same lent or not. <paramref name="catalogEntry"/> alone owns the record at the key when it declares the stored id (its own or one it lists compatible), or when it is the type that reads the stored entry and reads it whole, so an older id of its own schema name it reads whole is its own and is restamped. Everything else is refused with the file left as it was: a record of another registered type included, whatever the registry holds, and a record it would read only by dropping persisted slots. Retyping a key is a removal and an add. It serves this call only: the store keeps no registry.</param>
/// <remarks>The call holds the store's gate while it runs. User code it runs must not enter a cache or another store; that throws <see cref="InvalidOperationException"/>.</remarks>
/// <exception cref="InvalidOperationException">The store is attached to a cache; write through the cache.</exception>
/// <exception cref="CultStoreUnreadableException">The file at the path is not a store this runtime can read.</exception>
@ -4507,7 +4669,7 @@ namespace GameCult.Caching
/// <param name="key">The record key the document is stored under.</param>
/// <param name="document">The document to store.</param>
/// <param name="storedAt">The stored-at stamp, or null for now.</param>
/// <param name="registry">The registry that describes and encodes the document, or null for <see cref="CultDocumentRegistry.Shared"/>. The document type's entry is taken from it and is the only description of that schema id. Its other types claim stored records as a cache registering them would, and it declares nothing, so it can only refuse more: the document's entry owns a record whose id it declares (its own or one it lists compatible), and a record it reads whole when it is the picked claimant; refused are a record another registered type claims, one whose stored entry lists another registered type's id ahead of the entry's, and one that two registered types list or two versions of the schema could read. A wider registry (Shared when none is passed) refuses at least as much. It serves this call only: the store keeps no registry.</param>
/// <param name="registry">The registry that describes and encodes the document, or null for <see cref="CultDocumentRegistry.Shared"/>. The document type's entry is taken from it, and the registry decides the header for copied records it can read. It never decides ownership, so the answer is the same for any registry that holds the type: the document's entry owns a record whose id it declares (its own or one it lists compatible), and a record of an older id of its own schema name that it reads whole; everything else is refused, another registered type's record included, and a record it would read only by dropping persisted slots. Retyping a key is a removal and an add. It serves this call only: the store keeps no registry.</param>
/// <remarks>The call holds the store's gate while it runs. User code it runs (property getters while the document is encoded, constructors while a copied record is read) must not enter a cache or another store; that throws <see cref="InvalidOperationException"/>, carried by the encoder's wrapper when the code ran inside it.</remarks>
/// <exception cref="InvalidOperationException">The store is attached to a cache; write through the cache.</exception>
/// <exception cref="CultStoreUnreadableException">The file at the path is not a store this runtime can read.</exception>
@ -4723,53 +4885,37 @@ namespace GameCult.Caching
using var fileLock = AcquireLock(wait);
if (fileLock == null)
return CultCommitOutcome.Contended;
RefuseUndeclaredOnDisk(ReadSnapshot(), keys, OwnsOnDisk());
RefuseUndeclaredOnDisk(ReadSnapshot(), keys, Array.Empty<CultStagedRecord>(), Array.Empty<string>());
return CultCommitOutcome.Committed;
});
}
// A cache owns through its registry (CultDocumentRegistry.Owns). A cacheless CultMesh write owns what the catalog entries it
// writes declare: their own ids and the ids they list compatible, or an id whose publishing entry its claimant reads whole,
// the claimant being picked by the one precedence over the written entries and the lent registry's other types, and owning
// only when it is a written entry (variants:ruling:mesh-writes-bound-by-declaration). The keys judged are every key staged,
// written, removed or checked.
// Every key staged, written, removed or checked is judged, each by the one per-key judge (UndeclaredAt).
private void RefuseUndeclaredOnDisk(Durable? durable, IReadOnlyCollection<CultStagedRecord> upserts, IReadOnlyCollection<string> removals, IEnumerable<string> checks) =>
RefuseUndeclaredOnDisk(
durable,
_staged.Concat(upserts.Select(staged => staged.Record.Key)).Concat(removals).Concat(checks).Concat(StagedChecks),
OwnsOnDisk(upserts));
upserts,
checks);
// The owns predicate a write over the durable catalog asks. A cacheless writer declares what the entries it writes declare
// (variants:ruling:mesh-writes-bound-by-declaration) and reads a published entry with one lookup: the written entries shadow
// the lent registry's type for their ids, and the registry adds other claimants. The record is owned only when the claimant
// is a written entry, so the call is never more permissive than that declaration, and a registry can only refuse more.
private Func<string, IReadOnlyCollection<CultSchemaCatalogEntry>, bool> OwnsOnDisk(IReadOnlyCollection<CultStagedRecord>? upserts = null) =>
Cache != null || upserts == null
? Registry.Owns
: (id, catalog) =>
{
var writes = upserts.Select(staged => staged.Entry).ToArray();
Func<string, bool> declared = candidate => writes.Any(entry =>
string.Equals(entry.SchemaId, candidate, StringComparison.Ordinal) ||
entry.CompatibleSchemaIds.Contains(candidate, StringComparer.Ordinal));
return CultDocumentRegistry.Owns(
id,
catalog,
declared,
new CatalogEntryLookup(writes, AttachedRegistry),
entry => entry,
claimant => writes.Any(entry => string.Equals(entry.SchemaId, claimant.SchemaId, StringComparison.Ordinal)));
};
// A refused key's durable version is what this store has now read of it.
private void RefuseUndeclaredOnDisk(Durable? durable, IEnumerable<string> written, Func<string, IReadOnlyCollection<CultSchemaCatalogEntry>, bool> owns)
// A key the write also names as a removal or check (the request's, or one the cache removed here since its last flush) is a
// retype's removal half: it is judged by the removal rule, and the record staged there then lands. A refused key's durable
// version is what this store has now read of it.
private void RefuseUndeclaredOnDisk(Durable? durable, IEnumerable<string> written, IReadOnlyCollection<CultStagedRecord> upserts, IEnumerable<string> checks)
{
var undeclared = UndeclaredOnDisk(durable, owns);
if (undeclared.Count == 0)
return;
var keys = written.Distinct(StringComparer.Ordinal).ToArray();
var wanted = keys.ToHashSet(StringComparer.Ordinal);
var held = (durable?.Snapshot.Records ?? Array.Empty<CultPersistedRecord>())
.Where(record => wanted.Contains(record.Key))
.ToDictionary(record => record.Key, StringComparer.Ordinal);
var removed = checks.ToHashSet(StringComparer.Ordinal);
var writtenEntries = upserts.ToDictionary(staged => staged.Record.Key, staged => staged.Entry, StringComparer.Ordinal);
RefuseUndeclaredOnDisk(
written,
key => undeclared.TryGetValue(key, out var carried) ? carried : null,
keys,
durable?.Snapshot.SchemaCatalog ?? Array.Empty<CultSchemaCatalogEntry>(),
key => held.TryGetValue(key, out var record) ? record : null,
key => writtenEntries.TryGetValue(key, out var entry) ? entry : null,
removed,
key => Cache == null ? null : DurableAt(durable, key),
key => ForeignAt(durable, key),
key =>
@ -4781,23 +4927,6 @@ namespace GameCult.Caching
cacheless: Cache == null);
}
// Every durable record under an id the writer does not own, by key, named by the catalog entry that publishes its id. The
// answer is decided once per distinct schema id, so a large store pays one comparison per id, not per record.
private static Dictionary<string, (string Key, string SchemaId, string SchemaName)> UndeclaredOnDisk(Durable? durable, Func<string, IReadOnlyCollection<CultSchemaCatalogEntry>, bool> owns)
{
var undeclared = new Dictionary<string, (string Key, string SchemaId, string SchemaName)>(StringComparer.Ordinal);
var answers = new Dictionary<string, bool>(StringComparer.Ordinal);
bool OwnsOnce(string id, IReadOnlyCollection<CultSchemaCatalogEntry> catalog) =>
answers.TryGetValue(id, out var answer) ? answer : answers[id] = owns(id, catalog);
foreach (var record in durable?.Snapshot.Records ?? Array.Empty<CultPersistedRecord>())
{
if (Undeclared(record, durable!.Snapshot.SchemaCatalog, OwnsOnce) is { } carried)
undeclared[record.Key] = carried;
}
return undeclared;
}
// The durable record at a key as a load lists it when no registered type claims its id; null otherwise.
private CultForeignRecord? ForeignAt(Durable? durable, string key)
{

View file

@ -0,0 +1,22 @@
<Project Sdk="Microsoft.NET.Sdk">
<!-- No InternalsVisibleTo reaches this assembly: a store kind written here sees only the library's public and protected surface. -->
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<IsPackable>false</IsPackable>
<Nullable>enable</Nullable>
<RootNamespace>GameCult.Caching.OutsideStoreTests</RootNamespace>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\..\src\GameCult.Caching\GameCult.Caching.csproj" />
<ProjectReference Include="..\..\src\GameCult.Caching.MessagePack\GameCult.Caching.MessagePack.csproj" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="NUnit" Version="4.*" />
<PackageReference Include="NUnit3TestAdapter" Version="6.2.0" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.4.0" />
</ItemGroup>
</Project>

View file

@ -0,0 +1,368 @@
using System;
using System.Collections.Generic;
using System.Linq;
using GameCult.Caching.MessagePack;
using MessagePack;
using NUnit.Framework;
namespace GameCult.Caching.OutsideStoreTests
{
// variants:ruling:store-kind-per-key-judge: a store kind written outside the library applies the whole per-key ownership rule
// through the one protected judge (CacheBackingStore.RefuseUndeclaredOnDisk), using only public and protected members. This
// assembly has no InternalsVisibleTo, so a judge parameter type, or something it needs, turning internal breaks it here.
public class OutsideStoreJudgeTests
{
[CultDocument("tests.out_a", "tests.out_a.v1")]
[MessagePackObject]
public sealed class OutA
{
[Key(0)] [CultName] public string Name { get; set; } = string.Empty;
[Key(1)] public int Value { get; set; }
}
[CultDocument("tests.out_b", "tests.out_b.v1")]
[MessagePackObject]
public sealed class OutB
{
[Key(0)] [CultName] public string Name { get; set; } = string.Empty;
[Key(1)] public int Value { get; set; }
}
// RbX's schema id, which RbY lists; the test below asserts it, so a changed derivation fails there.
private const string RbXId = "sha256:25087e0d76fd5b9ac950723ccd47f3f3ac1428b53b49ca9bda9aae8b38a0220a";
[CultDocument("tests.rb_x", "tests.rb_x.v1")]
[MessagePackObject]
public sealed class RbX
{
[Key(0)] [CultName] public string Name { get; set; } = string.Empty;
[Key(1)] public int Value { get; set; }
}
[CultDocument("tests.rb_y", "tests.rb_y.v1", CompatibleSchemaIds = new[] { RbXId })]
[MessagePackObject]
public sealed class RbY
{
[Key(0)] [CultName] public string Name { get; set; } = string.Empty;
[Key(1)] public int Value { get; set; }
}
[CultDocument("tests.rb_z", "tests.rb_z.v1")]
[MessagePackObject]
public sealed class RbZ
{
[Key(0)] [CultName] public string Name { get; set; } = string.Empty;
[Key(1)] public int Value { get; set; }
}
// What the "disk" holds: records by key and the catalog that publishes their ids.
private sealed class Disk
{
public readonly Dictionary<string, (CultPersistedRecord Record, string Version)> Records = new();
public readonly List<CultSchemaCatalogEntry> Catalog = new();
private int _version;
public void Put(CultPersistedRecord record, CultSchemaCatalogEntry entry)
{
if (Catalog.All(known => known.SchemaId != entry.SchemaId))
Catalog.Add(entry);
Records[record.Key] = (record, (++_version).ToString());
}
public string? SchemaIdAt(string key) => Records.TryGetValue(key, out var held) ? held.Record.SchemaId : null;
}
// A store kind with no InternalsVisibleTo: an in-memory disk, flushed and committed under the protected judge.
private sealed class OutsideStore : CacheBackingStore
{
private readonly Disk _disk;
private readonly CultCodec _codec;
private readonly HashSet<string> _staged = new(StringComparer.Ordinal);
// StoredVersion is internal to the library, so a store kind tracks the version it last took of each key itself.
private readonly Dictionary<string, string> _versions = new(StringComparer.Ordinal);
public OutsideStore(Disk disk, CultCodec codec)
{
_disk = disk;
_codec = codec;
}
private object Decode(Type type, byte[] payload) => _codec.Deserialize(type, type, payload)!;
private CultStoredDocument? DurableAt(string key)
{
if (!_disk.Records.TryGetValue(key, out var held))
return null;
try
{
return ToStoredDocument(held.Record, held.Version, _disk.Catalog, Decode, out _);
}
catch (Exception)
{
return null;
}
}
public override void PullAll() => Held(() =>
{
var loaded = _disk.Records
.Where(pair => !_versions.TryGetValue(pair.Key, out var known) || !SameStoredIdentity(known, pair.Value.Version))
.Select(pair => DurableAt(pair.Key))
.OfType<CultStoredDocument>()
.ToArray();
var dropped = Entries.Values.Where(known => !_disk.Records.ContainsKey(known.Key.Value)).ToArray();
if (loaded.Length > 0 || dropped.Length > 0)
Loaded?.Invoke(loaded, dropped);
foreach (var stored in dropped)
Entries.TryRemove(stored.Key.Value, out _);
foreach (var stored in loaded)
{
Entries[stored.Key.Value] = stored;
_versions[stored.Key.Value] = _disk.Records[stored.Key.Value].Version;
}
});
public override void Push(CultStoredDocument entry) => Held(() =>
{
Entries[entry.Key.Value] = entry;
_staged.Add(entry.Key.Value);
IsDirty = true;
});
public override void Delete(CultStoredDocument entry) => Held(() =>
{
Entries.TryRemove(entry.Key.Value, out _);
_staged.Add(entry.Key.Value);
IsDirty = true;
});
public override void PushAll() => Held(() => Apply(Array.Empty<CultStoredDocument>(), Array.Empty<CultStoredDocument>(), Array.Empty<string>()));
public override CultCommitOutcome CommitBatch(CultCommitRequest request, bool wait) => Held(() =>
{
Apply(request.Upserts, request.Deletes, request.Checks);
foreach (var entry in request.Deletes)
Entries.TryRemove(entry.Key.Value, out _);
foreach (var entry in request.Upserts)
Entries[entry.Key.Value] = entry;
return CultCommitOutcome.Committed;
});
private void Apply(IReadOnlyList<CultStoredDocument> upserts, IReadOnlyList<CultStoredDocument> deletes, IEnumerable<string> checks)
{
var writes = upserts.ToDictionary(entry => entry.Key.Value, StringComparer.Ordinal);
foreach (var key in _staged)
{
if (Entries.TryGetValue(key, out var pushed))
writes.TryAdd(key, pushed);
}
var removals = deletes.Select(entry => entry.Key.Value)
.Concat(_staged.Where(key => !writes.ContainsKey(key) && !Entries.ContainsKey(key)))
.ToArray();
var judged = removals.Concat(checks).ToArray();
var entries = writes.ToDictionary(pair => pair.Key, pair => pair.Value.Descriptor.ToCatalogEntry(), StringComparer.Ordinal);
// The whole rule, from the protected judge: the same call both shipped stores make.
RefuseUndeclaredOnDisk(
writes.Keys.Concat(judged).Concat(StagedChecks),
_disk.Catalog,
key => _disk.Records.TryGetValue(key, out var held) ? held.Record : null,
key => entries.TryGetValue(key, out var entry) ? entry : null,
judged,
DurableAt,
_ => null,
key =>
{
_staged.Remove(key);
IsDirty = _staged.Count > 0;
});
foreach (var (key, document) in writes)
{
var type = document.Descriptor.DocumentType;
_disk.Put(
new CultPersistedRecord
{
Key = key,
SchemaId = document.Descriptor.SchemaId,
StoredAt = document.StoredAt,
Payload = _codec.Serialize(document.Document, type, type)
},
entries[key]);
_versions[key] = _disk.Records[key].Version;
}
foreach (var key in removals.Where(key => !writes.ContainsKey(key)))
{
_disk.Records.Remove(key);
_versions.Remove(key);
}
_staged.Clear();
MarkFlushSucceeded();
}
}
private static readonly CultDocumentRegistry AB = CultDocumentRegistry.ForTypes(new[] { typeof(OutA), typeof(OutB) });
private static readonly CultDocumentRegistry Rb = CultDocumentRegistry.ForTypes(new[] { typeof(RbX), typeof(RbY), typeof(RbZ) });
private static readonly CultRecordKey K = new("k");
private static readonly CultRecordKey K2 = new("k2");
private static CultCache Open(Disk disk, CultDocumentRegistry registry)
{
var cache = new CultCache(registry, CultCacheMessagePack.CreateCodec(registry));
cache.AddBackingStore(new OutsideStore(disk, cache.Codec!));
cache.PullAllBackingStoresAsync().GetAwaiter().GetResult();
return cache;
}
// Another writer puts a record of this type at the key on the disk, as the cache's own store would have.
private static void PutOnDisk<T>(Disk disk, CultDocumentRegistry registry, string key, T document, string? asSchemaId = null, CultSchemaCatalogEntry? entry = null)
where T : class
{
var codec = CultCacheMessagePack.CreateCodec(registry);
var descriptor = registry.GetRequired<T>();
disk.Put(
new CultPersistedRecord
{
Key = key,
SchemaId = asSchemaId ?? descriptor.SchemaId,
StoredAt = DateTimeOffset.UtcNow.ToString("O"),
Payload = codec.Serialize(document, typeof(T), typeof(T))
},
entry ?? descriptor.ToCatalogEntry());
}
private static Type HeldType(CultCache cache, CultRecordKey key) => cache.GetStored(key)!.Descriptor.DocumentType;
// The written type's own id: a record of another registered type that this cache never loaded.
[Test]
public void AnotherRegisteredTypesUnheldRecordIsRefusedUnderTheLockAndReloaded()
{
var disk = new Disk();
using var cache = Open(disk, AB);
PutOnDisk(disk, AB, "k", new OutB { Name = "k", Value = 2 });
var before = disk.Records["k"];
Assert.That(
() => cache.Commit(batch => batch.Upsert(typeof(OutA), new OutA { Name = "k", Value = 1 }, K)),
Throws.InstanceOf<CultSchemaConflictException>());
Assert.That(disk.Records["k"], Is.EqualTo(before), "the refused write changed nothing on the disk");
Assert.That(HeldType(cache, K), Is.EqualTo(typeof(OutB)), "the refusal reloaded the record the store holds");
}
[Test]
public void AFlushOfAnotherRegisteredTypesUnheldRecordIsRefusedToo()
{
var disk = new Disk();
using var cache = Open(disk, AB);
PutOnDisk(disk, AB, "k", new OutB { Name = "k", Value = 2 });
cache.UpsertAsync(typeof(OutA), new OutA { Name = "k", Value = 1 }, K).GetAwaiter().GetResult();
Assert.That(() => cache.FlushAsync().GetAwaiter().GetResult(), Throws.InstanceOf<CultSchemaConflictException>());
Assert.That(disk.SchemaIdAt("k"), Is.EqualTo(AB.GetRequired<OutB>().SchemaId));
Assert.That(HeldType(cache, K), Is.EqualTo(typeof(OutB)));
}
// Claimant precedence, a listed id: the written type owns a record under an id it lists, and a type that lists nothing does not.
[Test]
public void ARecordUnderAnIdTheWrittenTypeListsIsOwnedAndOneItDoesNotListIsRefused()
{
Assert.That(Rb.GetRequired<RbX>().SchemaId, Is.EqualTo(RbXId));
var disk = new Disk();
using var cache = Open(disk, Rb);
PutOnDisk(disk, Rb, "k", new RbX { Name = "k" });
PutOnDisk(disk, Rb, "k2", new RbX { Name = "k2" });
Assert.That(cache.Commit(batch => batch.Upsert(typeof(RbY), new RbY { Name = "k", Value = 1 }, K)), Is.True);
Assert.That(disk.SchemaIdAt("k"), Is.EqualTo(Rb.GetRequired<RbY>().SchemaId));
Assert.That(
() => cache.Commit(batch => batch.Upsert(typeof(RbZ), new RbZ { Name = "k2" }, K2)),
Throws.InstanceOf<CultSchemaConflictException>());
Assert.That(disk.SchemaIdAt("k2"), Is.EqualTo(RbXId), "the type that lists nothing replaced nothing");
}
// Reads whole: an older id of the written type's own schema name, which the type reads whole, is its own and is restamped.
[Test]
public void AnOlderIdOfTheWrittenSchemaNameThatIsReadWholeIsOwnedAndRestamped()
{
var disk = new Disk();
using var cache = Open(disk, AB);
var current = AB.GetRequired<OutA>().ToCatalogEntry();
var older = new CultSchemaCatalogEntry
{
SchemaId = "sha256:older-out-a",
SchemaName = current.SchemaName,
SchemaVersion = "tests.out_a.v0",
ContentHash = current.ContentHash,
CanonicalSchemaJson = current.CanonicalSchemaJson,
Members = current.Members
};
PutOnDisk(disk, AB, "k", new OutA { Name = "k" }, asSchemaId: older.SchemaId, entry: older);
Assert.That(cache.Commit(batch => batch.Upsert(typeof(OutA), new OutA { Name = "k", Value = 3 }, K)), Is.True);
Assert.That(disk.SchemaIdAt("k"), Is.EqualTo(current.SchemaId));
}
// Staged retype removals: a held key retypes as a removal then an add, in one batch and staged apart.
[TestCase(true)]
[TestCase(false)]
public void ARetypeOfAHeldKeyCommitsInOneBatchAndStagedApart(bool oneBatch)
{
var disk = new Disk();
PutOnDisk(disk, AB, "k", new OutB { Name = "k", Value = 2 });
using var cache = Open(disk, AB);
Assert.That(HeldType(cache, K), Is.EqualTo(typeof(OutB)));
if (oneBatch)
Assert.That(cache.Commit(batch =>
{
batch.Remove(K);
batch.Upsert(typeof(OutA), new OutA { Name = "k", Value = 1 }, K);
}), Is.True);
else
{
Assert.That(cache.Remove(K), Is.True);
Assert.That(cache.Commit(batch => batch.Upsert(typeof(OutA), new OutA { Name = "k", Value = 1 }, K)), Is.True);
}
Assert.That(disk.SchemaIdAt("k"), Is.EqualTo(AB.GetRequired<OutA>().SchemaId));
Assert.That(HeldType(cache, K), Is.EqualTo(typeof(OutA)));
}
[Test]
public void ARetypeOfAKeyTheCacheNeverLoadedIsJudgedByTheRemovalRule()
{
var disk = new Disk();
using var cache = Open(disk, AB);
PutOnDisk(disk, AB, "k", new OutB { Name = "k", Value = 2 });
Assert.That(cache.Commit(batch =>
{
batch.Remove(K);
batch.Upsert(typeof(OutA), new OutA { Name = "k", Value = 1 }, K);
}), Is.True, "a record a registered type reads may be removed, so the key retypes");
Assert.That(disk.SchemaIdAt("k"), Is.EqualTo(AB.GetRequired<OutA>().SchemaId));
}
// A removal is never licensed over a record no registered type owns, retype or not.
[Test]
public void ACommitNeverRemovesOrRetypesARecordNoRegisteredTypeOwns()
{
var disk = new Disk();
using var cache = Open(disk, AB);
var unknown = new CultSchemaCatalogEntry { SchemaId = "tests.out_unknown", SchemaName = "tests.out_unknown" };
disk.Put(new CultPersistedRecord { Key = "k", SchemaId = unknown.SchemaId, StoredAt = "t", Payload = new byte[] { 0x90 } }, unknown);
var before = disk.Records["k"];
Assert.That(() => cache.Commit(batch =>
{
batch.Remove(K);
batch.Upsert(typeof(OutA), new OutA { Name = "k" }, K);
}), Throws.InstanceOf<CultSchemaConflictException>(), "retype");
Assert.That(() => cache.Commit(batch => batch.Remove(K)), Throws.InstanceOf<CultSchemaConflictException>(), "removal");
Assert.That(disk.Records["k"], Is.EqualTo(before));
}
}
}

View file

@ -274,6 +274,39 @@ namespace GameCult.Caching.Tests
Assert.That(DiskRecord(path, "a").SchemaId, Is.Not.EqualTo(otherRuntimeId));
}
// Pushing a loaded record rewrites it in place under this type's id, and the cache keeps that entry. A newer version of the type,
// registered afterwards, reads the entry the record is now published by whole, so it owns the record: staging asks the entry
// that publishes the id the record carries now, not the one that published the id it was loaded under.
[Test]
public void ANewerVersionReadsWholeTheRecordAPushRelabelledInPlace()
{
const string otherRuntimeId = "other.runtime.id_deck";
var path = PathOf("mint-then-newer.cc");
WritePreIdStore(path, null, ("a", "a", 1));
var snapshot = CultDocumentMessagePackSerialization.DeserializeSnapshot(File.ReadAllBytes(path));
var entry = snapshot.SchemaCatalog.Single();
entry.SchemaId = otherRuntimeId;
entry.ContentHash = otherRuntimeId;
entry.CompatibleSchemaIds = new[] { otherRuntimeId };
snapshot.Records.Single().SchemaId = otherRuntimeId;
File.WriteAllBytes(path, CultDocumentMessagePackSerialization.SerializeSnapshot(snapshot));
var registry = CultDocumentRegistry.ForTypes(new[] { typeof(IdDeck) });
var key = new CultRecordKey("a");
using (var cache = CultCacheMessagePack.Create(path, new CultCacheOpenOptions { Registry = registry }))
{
var loaded = cache.GetStored(key)!;
cache.BackingStores[0].Push(loaded);
cache.BackingStores[0].PushAll();
Assert.That(ReferenceEquals(cache.GetStored(key), loaded), Is.True, "the cache keeps the entry the store relabelled");
Assert.That(loaded.StoredSchemaId, Is.Not.EqualTo(otherRuntimeId), "the push relabelled the loaded entry in place");
registry.GetRequired(typeof(IdDeckNext));
Assert.That(cache.Commit(batch => batch.Upsert(typeof(IdDeckNext), new IdDeckNext { Name = "newer" }, key)), Is.True);
}
Assert.That(DiskRecord(path, "a").SchemaId, Is.EqualTo(registry.GetRequired(typeof(IdDeckNext)).SchemaId));
}
// A pre-id record this cache reads only by dropping a persisted slot is read-only, so the rewrite that would persist its
// minted ids is refused typed, naming it, and the store is left byte for byte.
[Test]
@ -1401,6 +1434,16 @@ namespace GameCult.Caching.Tests
[Key(2)] public List<IdMove> Moves { get; set; } = new();
}
// The next version of IdDeck: the same slots under a new schema version.
[CultDocument("tests.element_id_deck", "tests.element_id_deck.v2")]
[MessagePackObject]
public sealed class IdDeckNext
{
[Key(0)] [CultName] public string Name { get; set; } = "";
[Key(1)] public List<IdReel> Reels { get; set; } = new();
[Key(2)] public List<IdMove> Moves { get; set; } = new();
}
// The shapes before ids: same slots, no id member. Not documents, so never registered.
[MessagePackObject]
public sealed class OldMark

View file

@ -67,6 +67,13 @@ namespace GameCult.Caching.Tests
private static CultCache Open(string path, CultDocumentRegistry registry, bool directory) =>
CultCacheMessagePack.Create(path, new CultCacheOpenOptions { Registry = registry, UseDirectoryStore = directory });
// The other writer retypes a key: a removal and an add in separate commits (variants:ruling:cache-refuses-retype).
private static void Retype(CultCache writer, Type type, object document, CultRecordKey key)
{
Assert.That(writer.Commit(batch => batch.Remove(key)), Is.True);
Assert.That(writer.Commit(batch => batch.Upsert(type, document, key)), Is.True);
}
private static object DeckOf(string name) => New(Deck, ("Name", name));
private static object WidgetOf(string label) => New(Widget, ("Label", label));
@ -205,7 +212,7 @@ namespace GameCult.Caching.Tests
var path = Seeded("replaced.cc", directory);
using var cache = Open(path, DeckOnly, directory);
using (var other = Open(path, Full, directory))
other.Commit(batch => batch.Upsert(Widget, WidgetOf("now a widget"), D));
Retype(other, Widget, WidgetOf("now a widget"), D);
var before = Fingerprint(path);
var refusal = Assert.Throws<CultSchemaConflictException>(() => Land(cache, write, D, DeckOf(Canary)))!;
@ -228,7 +235,7 @@ namespace GameCult.Caching.Tests
var path = Seeded("unstuck.cc", directory);
using var cache = Open(path, DeckOnly, directory);
using (var other = Open(path, Full, directory))
other.Commit(batch => batch.Upsert(Widget, WidgetOf("now a widget"), D));
Retype(other, Widget, WidgetOf("now a widget"), D);
var widget = Stored(path, D);
Assert.Throws<CultSchemaConflictException>(() => Land(cache, refused, D, DeckOf(Canary)));
@ -249,7 +256,7 @@ namespace GameCult.Caching.Tests
var path = Seeded("not-served.cc", directory);
using var cache = Open(path, DeckOnly, directory);
using (var other = Open(path, Full, directory))
other.Commit(batch => batch.Upsert(Widget, WidgetOf("now a widget"), D));
Retype(other, Widget, WidgetOf("now a widget"), D);
var refusal = Assert.Throws<CultSchemaConflictException>(() => Land(cache, refused, D, DeckOf(Canary)))!;
Assert.That(refusal.Message, Does.StartWith("Record 'd' (schema id '" + WidgetId + "'"), "one refused key reads as one record");
@ -287,11 +294,10 @@ namespace GameCult.Caching.Tests
});
using var cache = Open(path, DeckOnly, directory);
using (var other = Open(path, Full, directory))
other.Commit(batch =>
{
batch.Upsert(Widget, WidgetOf("w1"), D);
batch.Upsert(Widget, WidgetOf("w2"), d2);
});
{
Retype(other, Widget, WidgetOf("w1"), D);
Retype(other, Widget, WidgetOf("w2"), d2);
}
var refusal = Assert.Throws<CultSchemaConflictException>(() =>
{
@ -332,7 +338,7 @@ namespace GameCult.Caching.Tests
var path = Seeded("untouched.cc", directory);
using var cache = Open(path, DeckOnly, directory);
using (var other = Open(path, Full, directory))
other.Commit(batch => batch.Upsert(Widget, WidgetOf("now a widget"), D));
Retype(other, Widget, WidgetOf("now a widget"), D);
var widget = Stored(path, D);
Land(cache, write, E, DeckOf("e"));
@ -356,7 +362,7 @@ namespace GameCult.Caching.Tests
var path = Seeded("restamp.cc", directory);
using var cache = Open(path, DeckOnly, directory);
using (var other = Open(path, Full, directory))
other.Commit(batch => batch.Upsert(Widget, WidgetOf("now a widget"), D));
Retype(other, Widget, WidgetOf("now a widget"), D);
Land(cache, write, E, DeckOf("e"));
@ -374,7 +380,7 @@ namespace GameCult.Caching.Tests
var path = Seeded("stale.cc", directory);
using var cache = Open(path, DeckOnly, directory);
using (var other = Open(path, Full, directory))
other.Commit(batch => batch.Upsert(Widget, WidgetOf("now a widget"), D));
Retype(other, Widget, WidgetOf("now a widget"), D);
var widget = Stored(path, D);
Assert.Throws<CultSchemaConflictException>(() => Land(cache, Write.Commit, D, DeckOf(Canary)));
@ -392,7 +398,7 @@ namespace GameCult.Caching.Tests
var path = Seeded("staged-commit.cc", directory);
using var cache = Open(path, DeckOnly, directory);
using (var other = Open(path, Full, directory))
other.Commit(batch => batch.Upsert(Widget, WidgetOf("now a widget"), D));
Retype(other, Widget, WidgetOf("now a widget"), D);
var widget = Stored(path, D);
cache.UpsertAsync(Deck, DeckOf(Canary), D).GetAwaiter().GetResult();
@ -413,7 +419,7 @@ namespace GameCult.Caching.Tests
using var cache = Open(path, DeckOnly, directory);
Land(cache, landed, D, DeckOf("d again"));
using (var other = Open(path, Full, directory))
other.Commit(batch => batch.Upsert(Widget, WidgetOf("now a widget"), D));
Retype(other, Widget, WidgetOf("now a widget"), D);
var widget = Stored(path, D);
Land(cache, Write.Commit, E, DeckOf("e"));
@ -431,7 +437,7 @@ namespace GameCult.Caching.Tests
cache.UpsertAsync(Deck, DeckOf("d staged"), D).GetAwaiter().GetResult();
Land(cache, Write.Commit, E, DeckOf("e"));
using (var other = Open(path, Full, directory))
other.Commit(batch => batch.Upsert(Widget, WidgetOf("now a widget"), D));
Retype(other, Widget, WidgetOf("now a widget"), D);
var widget = Stored(path, D);
Land(cache, Write.Commit, new CultRecordKey("e2"), DeckOf("e2"));
@ -448,7 +454,7 @@ namespace GameCult.Caching.Tests
var path = Seeded("staged-removal.cc", directory);
using var cache = Open(path, DeckOnly, directory);
using (var other = Open(path, Full, directory))
other.Commit(batch => batch.Upsert(Widget, WidgetOf("now a widget"), D));
Retype(other, Widget, WidgetOf("now a widget"), D);
var widget = Stored(path, D);
cache.Remove(D);
@ -468,7 +474,7 @@ namespace GameCult.Caching.Tests
var path = Seeded("pending.cc", directory);
using var cache = Open(path, DeckOnly, directory);
using (var other = Open(path, Full, directory))
other.Commit(batch => batch.Upsert(Widget, WidgetOf("now a widget"), D));
Retype(other, Widget, WidgetOf("now a widget"), D);
cache.UpsertAsync(Deck, DeckOf(Canary), D).GetAwaiter().GetResult();
Assert.Throws<CultSchemaConflictException>(() => cache.FlushAsync().GetAwaiter().GetResult());
@ -478,7 +484,7 @@ namespace GameCult.Caching.Tests
cache.UpsertAsync(Deck, DeckOf(Canary), E).GetAwaiter().GetResult();
cache.UpsertAsync(Deck, DeckOf("v"), V).GetAwaiter().GetResult();
using (var other = Open(path, Full, directory))
other.Commit(batch => batch.Upsert(Widget, WidgetOf("now a widget"), E));
Retype(other, Widget, WidgetOf("now a widget"), E);
Assert.Throws<CultSchemaConflictException>(() => cache.FlushAsync().GetAwaiter().GetResult());
Assert.That(cache.IsDirty, Is.True, "v is still staged");
cache.FlushAsync().GetAwaiter().GetResult();
@ -495,7 +501,7 @@ namespace GameCult.Caching.Tests
using var cache = Open(path, DeckOnly, directory);
cache.Commit(batch => batch.Upsert(Deck, DeckOf("e"), E));
using (var other = Open(path, Full, directory))
other.Commit(batch => batch.Upsert(Widget, WidgetOf("now a widget"), D));
Retype(other, Widget, WidgetOf("now a widget"), D);
var before = Fingerprint(path);
// A condition keeps the single-file commit on the file as it is, so only the removal reaches d.
@ -516,7 +522,7 @@ namespace GameCult.Caching.Tests
var path = Seeded("pulled.cc", directory);
using var cache = Open(path, DeckOnly, directory);
using (var other = Open(path, Full, directory))
other.Commit(batch => batch.Upsert(Widget, WidgetOf("now a widget"), D));
Retype(other, Widget, WidgetOf("now a widget"), D);
cache.PullAllBackingStoresAsync().GetAwaiter().GetResult();
@ -562,7 +568,7 @@ namespace GameCult.Caching.Tests
using var cache = Open(path, DeckOnly, directory);
var z = new CultRecordKey("z");
using (var other = Open(path, Full, directory))
other.Commit(batch => batch.Upsert(Widget, WidgetOf("z"), z));
Retype(other, Widget, WidgetOf("z"), z);
var before = Stored(path, z);
Land(cache, Write.Flush, E, DeckOf("e"));
@ -803,8 +809,9 @@ namespace GameCult.Caching.Tests
Assert.That(EmittedDocumentTypes.Read(reopened.Get(D)!, "Name"), Is.EqualTo("claimed"));
}
// Whether a record is read-only is read live from the registry at the write, not fixed at load: a type registered after the
// load that declares the id makes the record writable, and the write stores it under the writing type's id, not the declarer's.
// Whether a record is writable is decided at the write, not fixed at load, and by the writing type alone: a type registered
// after the load that declares the id writes the record and stores it under its own id, and a registry holding it does not make
// another type the record's writer.
[Test]
public void ATypeRegisteredAfterLoadThatDeclaresTheIdMakesItWritable()
{
@ -815,14 +822,35 @@ namespace GameCult.Caching.Tests
{
Assert.Throws<CultSchemaConflictException>(() => cache.Commit(batch => batch.Upsert(Deck, DeckOf("refused"), D)));
registry.GetRequired(LateClaimingDeck);
cache.Commit(batch => batch.Upsert(Deck, DeckOf("claimed"), D));
Assert.Throws<CultSchemaConflictException>(() => cache.Commit(batch => batch.Upsert(Deck, DeckOf("still refused"), D)), "the registry's other type does not make the record the writing type's");
cache.Commit(batch => batch.Upsert(LateClaimingDeck, New(LateClaimingDeck, ("Name", "claimed")), D));
}
Assert.That(Read(path).Records.Single().SchemaId, Is.EqualTo(registry.GetRequired(Deck).SchemaId));
using var reopened = Open(path, DeckOnly, directory: false);
Assert.That(Read(path).Records.Single().SchemaId, Is.EqualTo(registry.GetRequired(LateClaimingDeck).SchemaId));
using var reopened = Open(path, CultDocumentRegistry.ForTypes(new[] { LateClaimingDeck }), directory: false);
Assert.That(EmittedDocumentTypes.Read(reopened.Get(D)!, "Name"), Is.EqualTo("claimed"));
}
// The removal rule asks the registry as it is when the removal is judged: a record read-only at load becomes removable once a
// type that declares its id is registered. The write rule is not the removal rule (the test above): the late type may remove
// the record, and a removal is judged by what the registry declares, not by the type that last wrote the key.
[Test]
public void ATypeRegisteredAfterLoadThatDeclaresTheIdMakesTheRecordRemovable()
{
var path = OtherRuntimeStore("late-remove.cc");
var registry = CultDocumentRegistry.ForTypes(new[] { Deck });
using (var cache = Open(path, registry, directory: false))
{
Assert.Throws<CultSchemaConflictException>(() => cache.Commit(batch => batch.Remove(D)), "no registered type declares the id yet");
registry.GetRequired(LateClaimingDeck);
Assert.That(cache.Commit(batch => batch.Remove(D)), Is.True);
Assert.That(cache.Get(D), Is.Null);
}
Assert.That(Read(path).Records, Is.Empty);
}
// A commit onto the file that involves a variant is judged on the set the file will hold. A foreign record is not decoded
// for that judgement, so it neither fails the commit nor joins the set.
[Test]
@ -1132,7 +1160,7 @@ namespace GameCult.Caching.Tests
a.UpsertAsync(Deck, DeckOf("e"), E).GetAwaiter().GetResult();
Assert.That(a.Remove(W), Is.False);
using (var b = Open(path, Full, directory))
b.Commit(batch => batch.Upsert(Deck, DeckOf("theirs"), W));
Retype(b, Deck, DeckOf("theirs"), W);
var theirs = Stored(path, W);
Assert.DoesNotThrow(() => a.FlushAsync().GetAwaiter().GetResult());
@ -1140,7 +1168,7 @@ namespace GameCult.Caching.Tests
Assert.That(Read(path).Records.Select(record => record.Key), Is.EquivalentTo(new[] { "d", "e", "w" }), "the staged write landed beside the checked key");
Assert.That(Stored(path, W), Is.EqualTo(theirs), "the record at the checked key is still the other writer's bytes");
using (var c = Open(path, Full, directory))
c.Commit(batch => batch.Upsert(Widget, WidgetOf("now a widget"), W));
Retype(c, Widget, WidgetOf("now a widget"), W);
Assert.DoesNotThrow(() => a.FlushAsync().GetAwaiter().GetResult(), "a flush clears what it judged: the next one has nothing to judge");
}
@ -1181,7 +1209,7 @@ namespace GameCult.Caching.Tests
Assert.That(StoreOf(a).ForeignRecords.Select(record => record.Key), Is.EqualTo(new[] { "w" }));
// The writer declares the widget's id, so it may replace the widget with a deck record under an id a reads but does not declare.
using (var other = Open(path, CultDocumentRegistry.ForTypes(new[] { MovingDeck, Widget }), directory))
Assert.That(other.Commit(batch => batch.Upsert(MovingDeck, New(MovingDeck, ("Name", "moved")), W)), Is.True);
Retype(other, MovingDeck, New(MovingDeck, ("Name", "moved")), W);
Assert.That(a.Remove(W), Is.False);
var refusal = Assert.Throws<CultSchemaConflictException>(() => a.FlushAsync().GetAwaiter().GetResult())!;
@ -1293,7 +1321,7 @@ namespace GameCult.Caching.Tests
else
cache.FlushAsync().GetAwaiter().GetResult();
using (var other = Open(path, Full, directory))
other.Commit(batch => batch.Upsert(Widget, WidgetOf("now a widget"), staged));
Retype(other, Widget, WidgetOf("now a widget"), staged);
Assert.DoesNotThrow(() => cache.FlushAsync().GetAwaiter().GetResult());
}
@ -1371,7 +1399,7 @@ namespace GameCult.Caching.Tests
using var cache = Open(path, DeckOnly, directory);
Assert.That(StoreOf(cache).ForeignRecords.Select(record => record.Key), Is.EqualTo(new[] { "w" }));
using (var other = Open(path, Full, directory))
other.Commit(batch => batch.Upsert(Deck, DeckOf("theirs"), W));
Retype(other, Deck, DeckOf("theirs"), W);
var theirs = Stored(path, W);
Assert.That(cache.Commit(batch => batch.Remove(W)), Is.True);
@ -1455,7 +1483,7 @@ namespace GameCult.Caching.Tests
using var _ = cache;
Assert.That(cache.BackingStores[1].ForeignRecords.Select(record => record.Key), Is.EqualTo(new[] { "w" }));
using (var late = Open(other, CultDocumentRegistry.ForTypes(new[] { Gizmo, Widget }), directory))
late.Commit(batch => batch.Upsert(Gizmo, New(Gizmo, ("Tag", "theirs")), W));
Retype(late, Gizmo, New(Gizmo, ("Tag", "theirs")), W);
var theirs = Stored(other, W);
Assert.That(cache.Commit(batch =>

View file

@ -480,17 +480,18 @@ namespace GameCult.Caching.Tests
private static readonly CultDocumentRegistry NarrowShortRegistry = CultDocumentRegistry.ForTypes(new[] { typeof(NoteNarrow), typeof(NoteShort) });
// Which entry reads a persisted entry is one precedence: a cacheless writer lent a registry picks the claimant that registry
// picks, never one by schema name over its own entry. Here the entry is named and shaped like Narrow but lists Short's id.
// A write's ownership is the written entry's alone (variants:ruling:own-entry-reads-whole): the stored entry is named and shaped
// like Narrow but lists Short's id, and a registry that holds Short (which would read it by listing the id) changes nothing, for
// a cache and a cacheless writer alike.
[TestCase(false)]
[TestCase(true)]
public void ACachelessWriterPicksTheClaimantItsRegistryPicks(bool cacheless)
public void AWrittenEntryReadingTheRecordWholeOwnsItWhateverTheRegistryHolds(bool cacheless)
{
var path = SeedRenamed(listsNarrow: false, alsoLists: NarrowShortRegistry.GetRequired<NoteShort>().SchemaId, name: "tests.restamp_note");
var before = Fingerprint(path);
Assert.Throws<CultSchemaConflictException>(() => WriteNarrow(path, cacheless, NarrowShortRegistry));
Assert.That(Fingerprint(path), Is.EqualTo(before));
WriteNarrow(path, cacheless, NarrowShortRegistry);
Assert.That(Snapshot(path).Records.Single(record => record.Key == "note").SchemaId, Is.EqualTo(NarrowRegistry.GetRequired<NoteNarrow>().SchemaId));
}
[TestCase(false)]
@ -668,28 +669,6 @@ namespace GameCult.Caching.Tests
Assert.That(Fingerprint(path), Is.EqualTo(before));
}
// The shadow, where it is decided: a written entry answers for its schema id whatever the registry holds, and the registry
// still adds its other types.
[Test]
public void AWrittenEntryShadowsTheLentRegistrysTypeForItsId()
{
var written = NarrowRegistry.GetRequired<NoteNarrow>().ToCatalogEntry();
var lookup = new CatalogEntryLookup(new[] { written }, NarrowListerRegistry);
var lister = NarrowListerRegistry.GetRequired<NoteLister>();
Assert.That(lookup.Owner(written.SchemaId), Is.SameAs(written));
Assert.That(lookup.Named(written.SchemaName), Is.EqualTo(new[] { written }).Using<CultSchemaCatalogEntry>((x, y) => ReferenceEquals(x, y)));
Assert.That(lookup.Listing(written.SchemaId), Is.EqualTo(new[] { written }).Using<CultSchemaCatalogEntry>((x, y) => ReferenceEquals(x, y)));
Assert.That(lookup.Owner(lister.SchemaId)!.SchemaId, Is.EqualTo(lister.SchemaId));
Assert.That(lookup.Listing(ListerOldId).Select(entry => entry.SchemaId), Is.EqualTo(new[] { lister.SchemaId }));
Assert.That(lookup.Named(lister.SchemaName).Select(entry => entry.SchemaId), Is.EqualTo(new[] { lister.SchemaId }));
// A write of several entries shadows by the entry that has the id, not by all of them having it.
var other = CultDocumentRegistry.ForTypes(new[] { typeof(NoteOther) }).GetRequired<NoteOther>().ToCatalogEntry();
var several = new CatalogEntryLookup(new[] { other, written }, NarrowListerRegistry);
Assert.That(several.Named(written.SchemaName), Is.EqualTo(new[] { written }).Using<CultSchemaCatalogEntry>((x, y) => ReferenceEquals(x, y)));
}
// The registry's other types change nothing about a record the written type claims itself.
[Test]
public void AnotherRegisteredTypeDoesNotTakeAWritersOwnDriftRecord()
@ -701,16 +680,16 @@ namespace GameCult.Caching.Tests
Assert.That(Snapshot(path).Records.Single(record => record.Key == "note").SchemaId, Is.EqualTo(NarrowRegistry.GetRequired<NoteNarrow>().SchemaId));
}
// What the lent registry registers beyond the written type can only make the claim more ambiguous, never own more: a second
// version of the written schema's name refuses a record the write owned by name alone.
// What the lent registry registers beyond the written type changes nothing, in either direction: a second version of the
// written schema's name neither refuses nor widens a record the write owns by name alone.
[Test]
public void ASecondVersionOfTheWrittenNameInTheLentRegistryOnlyRefuses()
public void ASecondVersionOfTheWrittenNameInTheLentRegistryChangesNothing()
{
var path = SeedRenamed(listsNarrow: false, name: "tests.restamp_note");
var before = Fingerprint(path);
Assert.Throws<CultSchemaConflictException>(() => WriteNarrow(path, cacheless: true, CultDocumentRegistry.ForTypes(new[] { typeof(NoteNarrow), typeof(NoteGrown) })));
Assert.That(Fingerprint(path), Is.EqualTo(before));
WriteNarrow(path, cacheless: true, CultDocumentRegistry.ForTypes(new[] { typeof(NoteNarrow), typeof(NoteGrown) }));
Assert.That(Snapshot(path).Records.Single(record => record.Key == "note").SchemaId, Is.EqualTo(NarrowRegistry.GetRequired<NoteNarrow>().SchemaId));
}
// A record stored under an id its catalog entry only lists (no entry owns it) is published by that entry, and the type that
@ -765,8 +744,8 @@ namespace GameCult.Caching.Tests
var writes = new[] { narrow };
Func<string, bool> declares = id => id == narrow.SchemaId;
Assert.That(CultDocumentRegistry.Owns("tests.restamp.unpublished", writes, declares, new CatalogEntryLookup(writes, null), entry => entry, _ => true), Is.False);
Assert.That(CultDocumentRegistry.Owns(narrow.SchemaId, writes, declares, new CatalogEntryLookup(writes, null), entry => entry, _ => true), Is.True);
Assert.That(CultDocumentRegistry.Owns("tests.restamp.unpublished", writes, declares, new CatalogEntryLookup(writes), entry => entry), Is.False);
Assert.That(CultDocumentRegistry.Owns(narrow.SchemaId, writes, declares, new CatalogEntryLookup(writes), entry => entry), Is.True);
}
private static string VectorPath(string name)

View file

@ -0,0 +1,659 @@
#nullable enable
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using GameCult.Caching.MessagePack;
using MessagePack;
using NUnit.Framework;
using R3;
namespace GameCult.Caching.Tests
{
// variants:ruling:one-batch-retype-is-eviction-then-add: a batch that removes a key and adds a record there stages the held
// record's eviction and then the add, so watchers, the batch validators, the store judges and the removal bookkeeping see what they
// see when the two are staged apart (cache.Remove, then UpsertAsync). Where a test runs both shapes it asserts the same answer.
public class RetypeEvictionTests
{
[CultDocument("tests.ev_a", "tests.ev_a.v1")]
[MessagePackObject(AllowPrivate = true)]
internal sealed class EvA
{
[Key(0)] [CultName] public string Name = string.Empty;
[Key(1)] public int Value;
}
[CultDocument("tests.ev_b", "tests.ev_b.v1")]
[MessagePackObject(AllowPrivate = true)]
internal sealed class EvB
{
[Key(0)] [CultName] public string Name = string.Empty;
[Key(1)] public int Value;
}
// A type only the second writer registers.
[CultDocument("tests.ev_c", "tests.ev_c.v1")]
[MessagePackObject(AllowPrivate = true)]
internal sealed class EvC
{
[Key(0)] [CultName] public string Name = string.Empty;
}
[CultDocument("tests.ev_g", "tests.ev_g.v1")]
[MessagePackObject(AllowPrivate = true)]
[CultGlobal]
internal sealed class EvG
{
[Key(0)] public string Value = string.Empty;
}
// RbX's schema id, which RbY lists. The test below asserts it, so a changed derivation fails there.
private const string RbXId = "sha256:25087e0d76fd5b9ac950723ccd47f3f3ac1428b53b49ca9bda9aae8b38a0220a";
// The rebase pair: RbY lists RbX's id compatible, so a plain write of RbY owns an RbX record; RbZ lists nothing.
[CultDocument("tests.rb_x", "tests.rb_x.v1")]
[MessagePackObject(AllowPrivate = true)]
internal sealed class RbX
{
[Key(0)] [CultName] public string Name = string.Empty;
[Key(1)] public int Value;
}
[CultDocument("tests.rb_y", "tests.rb_y.v1", CompatibleSchemaIds = new[] { RbXId })]
[MessagePackObject(AllowPrivate = true)]
internal sealed class RbY
{
[Key(0)] [CultName] public string Name = string.Empty;
[Key(1)] public int Value;
}
[CultDocument("tests.rb_z", "tests.rb_z.v1")]
[MessagePackObject(AllowPrivate = true)]
internal sealed class RbZ
{
[Key(0)] [CultName] public string Name = string.Empty;
[Key(1)] public int Value;
}
private static readonly CultDocumentRegistry AB = CultDocumentRegistry.ForTypes(new[] { typeof(EvA), typeof(EvB), typeof(EvG) });
private static readonly CultDocumentRegistry ABC = CultDocumentRegistry.ForTypes(new[] { typeof(EvA), typeof(EvB), typeof(EvC), typeof(EvG) });
private static readonly CultDocumentRegistry Rb = CultDocumentRegistry.ForTypes(new[] { typeof(RbX), typeof(RbY), typeof(RbZ) });
private static readonly CultRecordKey K1 = new("k1");
private static readonly CultRecordKey K2 = new("k2");
private static readonly CultRecordKey K3 = new("k3");
// The stores a retype is judged in, and no store at all. The directory store holds no variant, so variant tests skip it.
public static readonly object[] Kinds = { "single-file", "directory", "storeless" };
public static readonly object[] VariantKinds = { "single-file", "storeless" };
private string _directory = string.Empty;
private int _paths;
[SetUp]
public void CreateDirectory()
{
_directory = Path.Combine(Path.GetTempPath(), $"cultlib-retypeeviction-{Guid.NewGuid():N}");
Directory.CreateDirectory(_directory);
}
[TearDown]
public void DeleteDirectory()
{
if (Directory.Exists(_directory))
Directory.Delete(_directory, recursive: true);
}
// A fresh store path per call, so one test can run a scenario twice.
private string NewPath(string kind) => Path.Combine(_directory, $"{kind}-{_paths++}", kind == "directory" ? "store" : "store.cc");
private static CultCache Open(string path, string kind, CultDocumentRegistry registry)
{
var cache = new CultCache(registry, CultCacheMessagePack.CreateCodec(registry));
if (kind != "storeless")
{
Directory.CreateDirectory(Path.GetDirectoryName(path)!);
cache.AddBackingStore(kind == "directory" ? new DirectoryMessagePackBackingStore(path) : new SingleFileMessagePackBackingStore(path));
cache.PullAllBackingStoresAsync().GetAwaiter().GetResult();
}
return cache;
}
private static Type TypeAt(string path, string kind, CultDocumentRegistry registry, CultRecordKey key)
{
using var cache = Open(path, kind, registry);
return cache.GetStored(key)!.Descriptor.DocumentType;
}
// Retypes a held key as one batch (Remove, then the add the caller stages) or staged apart (cache.Remove, then the add's own call).
private static void Retype(CultCache cache, bool oneBatch, CultRecordKey key, Action<CultCacheBatch> add, Action apart)
{
if (oneBatch)
Assert.That(cache.Commit(batch =>
{
batch.Remove(key);
add(batch);
}), Is.True);
else
{
Assert.That(cache.Remove(key), Is.True);
apart();
}
}
private static void PutB(CultCache cache, CultRecordKey key, int value = 2) =>
Assert.That(cache.Commit(batch => batch.Upsert(typeof(EvB), new EvB { Name = key.Value, Value = value }, key)), Is.True);
// finding retype-not-an-eviction: Watch<old> sees Removed and Watch<new> sees Added, in a batch and staged apart alike.
[TestCaseSource(nameof(Kinds))]
public void ARetypeIsRemovedForTheOldTypeAndAddedForTheNewWhetherOneBatchOrStagedApart(string kind)
{
foreach (var oneBatch in new[] { true, false })
{
using var cache = Open(NewPath(kind), kind, AB);
PutB(cache, K2);
var removedLog = new List<CultCacheDocumentChangeKind>();
var addedLog = new List<CultCacheDocumentChangeKind>();
using var oldWatch = cache.Watch<EvB>().Subscribe(change => removedLog.Add(change.Kind));
using var newWatch = cache.Watch<EvA>().Subscribe(change => addedLog.Add(change.Kind));
Retype(cache, oneBatch, K2,
batch => batch.Upsert(typeof(EvA), new EvA { Name = "k2", Value = 9 }, K2),
() => cache.UpsertAsync(typeof(EvA), new EvA { Name = "k2", Value = 9 }, K2).GetAwaiter().GetResult());
Assert.That(removedLog, Is.EqualTo(new[] { CultCacheDocumentChangeKind.Removed }), $"Watch<old>, one batch {oneBatch}");
Assert.That(addedLog, Is.EqualTo(new[] { CultCacheDocumentChangeKind.Added }), $"Watch<new>, one batch {oneBatch}");
Assert.That(cache.GetStored(K2)!.Descriptor.DocumentType, Is.EqualTo(typeof(EvA)));
}
}
// finding retype-not-an-eviction, global key, and finding global-eviction-uncovered: evicting a global frees its slot, and the
// batch that retypes a global's key may land the next global of its type elsewhere, as staged apart.
[TestCaseSource(nameof(Kinds))]
public void EvictingAGlobalFreesItsSlotAndAGlobalKeyRetypesInOneBatch(string kind)
{
using var cache = Open(NewPath(kind), kind, AB);
var g1 = new CultRecordKey("g1");
var g2 = new CultRecordKey("g2");
Assert.That(cache.Commit(batch => batch.Upsert(typeof(EvG), new EvG { Value = "one" }, g1)), Is.True);
Assert.Throws<InvalidOperationException>(
() => cache.Commit(batch => batch.Upsert(typeof(EvG), new EvG { Value = "two" }, g2)), "a second global while the first stands is refused");
Assert.That(cache.Commit(batch =>
{
batch.Remove(g1);
batch.Upsert(typeof(EvG), new EvG { Value = "two" }, g2);
}), Is.True, "the eviction freed the slot");
Assert.That(cache.GetGlobal<EvG>()!.Value, Is.EqualTo("two"));
Assert.That(cache.GetStored(g1), Is.Null);
Assert.That(cache.Commit(batch =>
{
batch.Remove(g2);
batch.Upsert(typeof(EvA), new EvA { Name = "g2", Value = 1 }, g2);
batch.Upsert(typeof(EvG), new EvG { Value = "three" }, g1);
}), Is.True, "staged apart the retype of the global's key and the next global commit");
Assert.That(cache.GetStored(g2)!.Descriptor.DocumentType, Is.EqualTo(typeof(EvA)));
Assert.That(cache.GetGlobal<EvG>()!.Value, Is.EqualTo("three"));
}
// finding retype-not-an-eviction (variant key) and variant-ops-drop-the-removal: a variant key retypes to a plain record of
// another type, a plain key becomes a variant of another type's base, and a key removed, added and then added as a variant
// lands as the variant, each answering as it does staged apart.
[TestCaseSource(nameof(VariantKinds))]
public void VariantKeysRetypeInOneBatchAsStagedApart(string kind)
{
foreach (var oneBatch in new[] { true, false })
{
using var cache = Open(NewPath(kind), kind, AB);
var baseA = new CultRecordKey("base-a");
Assert.That(cache.Commit(batch =>
{
batch.Upsert(typeof(EvA), new EvA { Name = "base-a", Value = 1 }, baseA);
batch.UpsertVariant(K3, baseA, new[] { cache.Override<EvA>(nameof(EvA.Name), "v3") });
batch.Upsert(typeof(EvB), new EvB { Name = "k2", Value = 2 }, K2);
batch.Upsert(typeof(EvB), new EvB { Name = "k1", Value = 2 }, K1);
}), Is.True);
Retype(cache, oneBatch, K3,
batch => batch.Upsert(typeof(EvB), new EvB { Name = "k3", Value = 3 }, K3),
() => cache.UpsertAsync(typeof(EvB), new EvB { Name = "k3", Value = 3 }, K3).GetAwaiter().GetResult());
Assert.That(cache.GetStored(K3)!.Variant, Is.Null, $"one batch {oneBatch}");
Assert.That(cache.GetStored(K3)!.Descriptor.DocumentType, Is.EqualTo(typeof(EvB)));
Retype(cache, oneBatch, K2,
batch => batch.UpsertVariant(K2, baseA, new[] { cache.Override<EvA>(nameof(EvA.Name), "v2") }),
() => cache.UpsertVariantAsync(K2, baseA, new[] { cache.Override<EvA>(nameof(EvA.Name), "v2") }).GetAwaiter().GetResult());
Assert.That(cache.GetStored(K2)!.Variant?.BaseKey, Is.EqualTo(baseA.Value), $"one batch {oneBatch}");
Assert.That(cache.GetStored(K2)!.Descriptor.DocumentType, Is.EqualTo(typeof(EvA)));
// Removed and then added as a variant (in one batch, with a plain add of the same type between): the variant lands.
Retype(cache, oneBatch, K1,
batch =>
{
batch.Upsert(typeof(EvA), new EvA { Name = "k1", Value = 2 }, K1);
batch.UpsertVariant(K1, baseA, new[] { cache.Override<EvA>(nameof(EvA.Name), "v1") });
},
() => cache.UpsertVariantAsync(K1, baseA, new[] { cache.Override<EvA>(nameof(EvA.Name), "v1") }).GetAwaiter().GetResult());
Assert.That(cache.GetStored(K1)!.Variant?.BaseKey, Is.EqualTo(baseA.Value), $"one batch {oneBatch}");
}
}
[TestCaseSource(nameof(VariantKinds))]
public void AFlattenAfterARemovalFlattensNothingInOneBatchAndStagedApart(string kind)
{
using var cache = Open(NewPath(kind), kind, AB);
var baseA = new CultRecordKey("base-a");
cache.Commit(batch =>
{
batch.Upsert(typeof(EvA), new EvA { Name = "base-a", Value = 1 }, baseA);
batch.UpsertVariant(K3, baseA, new[] { cache.Override<EvA>(nameof(EvA.Name), "v3") });
});
var flatten = Assert.Throws<InvalidOperationException>(() => cache.Commit(batch =>
{
batch.Remove(K3);
batch.Flatten(K3);
}))!;
Assert.That(flatten.Message, Does.Contain("Flatten(k3)").And.Contain("not a variant"));
Assert.That(cache.GetStored(K3)!.Variant, Is.Not.Null, "the refused batch changed nothing");
Assert.That(cache.Remove(K3), Is.True);
Assert.Throws<InvalidOperationException>(() => cache.FlattenAsync(K3).GetAwaiter().GetResult());
}
// finding flatten-guard-reads-removals: a Flatten acts on what the key will hold, as staged apart. A variant staged at the key
// (after a removal or not) is flattened as it lands; a plain record staged there leaves nothing to flatten.
[TestCaseSource(nameof(VariantKinds))]
public void AFlattenOfAVariantStagedInTheSameBatchFlattensItAsStagedApart(string kind)
{
foreach (var oneBatch in new[] { true, false })
{
using var cache = Open(NewPath(kind), kind, AB);
var baseA = new CultRecordKey("base-a");
Assert.That(cache.Commit(batch =>
{
batch.Upsert(typeof(EvA), new EvA { Name = "base-a", Value = 1 }, baseA);
batch.UpsertVariant(K3, baseA, new[] { cache.Override<EvA>(nameof(EvA.Name), "v3") });
batch.Upsert(typeof(EvB), new EvB { Name = "k1", Value = 2 }, K1);
}), Is.True);
// Remove, UpsertVariant, Flatten: the plain record of the variant's resolution, retyped.
if (oneBatch)
Assert.That(cache.Commit(batch =>
{
batch.Remove(K1);
batch.UpsertVariant(K1, baseA, new[] { cache.Override<EvA>(nameof(EvA.Name), "v1") });
batch.Flatten(K1);
}), Is.True);
else
{
Assert.That(cache.Remove(K1), Is.True);
cache.UpsertVariantAsync(K1, baseA, new[] { cache.Override<EvA>(nameof(EvA.Name), "v1") }).GetAwaiter().GetResult();
cache.FlattenAsync(K1).GetAwaiter().GetResult();
}
var flattened = cache.GetStored(K1)!;
Assert.That(flattened.Variant, Is.Null, $"one batch {oneBatch}");
Assert.That(flattened.Descriptor.DocumentType, Is.EqualTo(typeof(EvA)));
Assert.That(((EvA)flattened.Document).Name, Is.EqualTo("v1"));
// A held variant rebased and flattened in one batch flattens the rebase, not the held record.
if (oneBatch)
Assert.That(cache.Commit(batch =>
{
batch.UpsertVariant(K3, baseA, new[] { cache.Override<EvA>(nameof(EvA.Name), "v3b") });
batch.Flatten(K3);
}), Is.True);
else
{
cache.UpsertVariantAsync(K3, baseA, new[] { cache.Override<EvA>(nameof(EvA.Name), "v3b") }).GetAwaiter().GetResult();
cache.FlattenAsync(K3).GetAwaiter().GetResult();
}
Assert.That(cache.GetStored(K3)!.Variant, Is.Null, $"one batch {oneBatch}");
Assert.That(((EvA)cache.GetStored(K3)!.Document).Name, Is.EqualTo("v3b"));
}
}
[TestCaseSource(nameof(VariantKinds))]
public void AFlattenOfAPlainRecordStagedInTheSameBatchIsRefusedAsStagedApart(string kind)
{
using var cache = Open(NewPath(kind), kind, AB);
PutB(cache, K2);
var refused = Assert.Throws<InvalidOperationException>(() => cache.Commit(batch =>
{
batch.Remove(K2);
batch.Upsert(typeof(EvA), new EvA { Name = "k2" }, K2);
batch.Flatten(K2);
}))!;
Assert.That(refused.Message, Does.Contain("Flatten(k2)").And.Contain("not a variant"));
Assert.That(cache.GetStored(K2)!.Descriptor.DocumentType, Is.EqualTo(typeof(EvB)), "the refused batch changed nothing");
Assert.That(cache.Remove(K2), Is.True);
cache.UpsertAsync(typeof(EvA), new EvA { Name = "k2" }, K2).GetAwaiter().GetResult();
Assert.Throws<InvalidOperationException>(() => cache.FlattenAsync(K2).GetAwaiter().GetResult());
}
// finding flatten-after-staged-plain-unpinned: where the key held a variant, Remove, a plain Upsert, then Flatten is refused
// as staged apart, and the plain write is never dropped for a flatten of the variant the cache held.
[TestCaseSource(nameof(VariantKinds))]
public void AFlattenAfterARemovalAndAPlainWriteOnAHeldVariantIsRefusedAndKeepsTheHeldVariant(string kind)
{
using var cache = Open(NewPath(kind), kind, AB);
var baseA = new CultRecordKey("base-a");
cache.Commit(batch =>
{
batch.Upsert(typeof(EvA), new EvA { Name = "base-a", Value = 1 }, baseA);
batch.UpsertVariant(K3, baseA, new[] { cache.Override<EvA>(nameof(EvA.Name), "v3") });
});
var refused = Assert.Throws<InvalidOperationException>(() => cache.Commit(batch =>
{
batch.Remove(K3);
batch.Upsert(typeof(EvA), new EvA { Name = "plain" }, K3);
batch.Flatten(K3);
}))!;
Assert.That(refused.Message, Does.Contain("Flatten(k3)").And.Contain("not a variant"));
Assert.That(cache.GetStored(K3)!.Variant, Is.Not.Null, "the refused batch changed nothing");
Assert.That(cache.Remove(K3), Is.True);
cache.UpsertAsync(typeof(EvA), new EvA { Name = "plain" }, K3).GetAwaiter().GetResult();
var apart = Assert.Throws<InvalidOperationException>(() => cache.FlattenAsync(K3).GetAwaiter().GetResult())!;
Assert.That(apart.Message, Is.EqualTo(refused.Message));
Assert.That(((EvA)cache.GetStored(K3)!.Document).Name, Is.EqualTo("plain"));
}
// finding op-after-staged-flatten-diverges: an operation after a Flatten at the same key acts on the plain record that flatten
// leaves, so one intent has one answer in a batch and staged apart, in both stores kinds and after a reload from disk.
[TestCaseSource(nameof(VariantKinds))]
public void AnOperationAfterAStagedFlattenAnswersAsStagedApart(string kind)
{
var scenarios = new[] { "F,F", "F,P", "F,V", "V,F,V", "F,P,F", "V,F,P", "V,F,F" };
foreach (var scenario in scenarios)
{
var steps = scenario.Split(',');
var results = new List<string>();
foreach (var oneBatch in new[] { true, false })
{
var path = NewPath(kind);
string outcome;
using (var cache = Open(path, kind, AB))
{
var baseA = new CultRecordKey("base-a");
cache.Commit(batch =>
{
batch.Upsert(typeof(EvA), new EvA { Name = "base-a", Value = 1 }, baseA);
batch.UpsertVariant(K3, baseA, new[] { cache.Override<EvA>(nameof(EvA.Name), "v3") });
});
var variant = new[] { cache.Override<EvA>(nameof(EvA.Name), "vv") };
try
{
if (oneBatch)
cache.Commit(batch =>
{
foreach (var step in steps)
{
if (step == "F") batch.Flatten(K3);
else if (step == "P") batch.Upsert(typeof(EvA), new EvA { Name = "plain" }, K3);
else batch.UpsertVariant(K3, baseA, variant);
}
});
else
foreach (var step in steps)
{
if (step == "F") cache.FlattenAsync(K3).GetAwaiter().GetResult();
else if (step == "P") cache.UpsertAsync(typeof(EvA), new EvA { Name = "plain" }, K3).GetAwaiter().GetResult();
else cache.UpsertVariantAsync(K3, baseA, variant).GetAwaiter().GetResult();
}
outcome = "committed";
}
catch (InvalidOperationException refused)
{
outcome = "refused: " + refused.Message;
}
outcome += " | " + Describe(cache);
cache.FlushAsync().GetAwaiter().GetResult();
}
if (kind != "storeless" && outcome.StartsWith("committed", StringComparison.Ordinal))
{
using var reloaded = Open(path, kind, AB);
outcome += " | reloaded " + Describe(reloaded);
}
results.Add(outcome);
}
// A refused batch changes nothing, while staged apart the steps before the refused one have landed: the answer compared
// is the verdict and its text, and the end state only where both committed.
var batchAnswer = results[0].Split(" | ")[0];
var apartAnswer = results[1].Split(" | ")[0];
Assert.That(batchAnswer, Is.EqualTo(apartAnswer), $"{kind} {scenario}");
if (batchAnswer == "committed")
Assert.That(results[0], Is.EqualTo(results[1]), $"{kind} {scenario}");
}
static string Describe(CultCache cache)
{
var held = cache.GetStored(K3)!;
return $"{held.Descriptor.DocumentType.Name} variant={held.Variant?.BaseKey} name={((EvA)held.Document).Name}";
}
}
// finding cross-store-retype-decided-by-key-home: a one-batch retype between two home stores is refused by the rule that a commit
// lands in one home store, with that rule's text (an evicted key is absent to the one-key-one-store check, as when staged
// apart); staged apart, as two commits, it leaves the key in exactly one store.
[Test]
public void ARetypeAcrossTwoHomeStoresInOneBatchIsRefusedByTheBatchSpanningRuleAndStagedApartLeavesOneStore()
{
string[] KeysIn(string path) =>
new SingleFileMessagePackBackingStore(path).ReadStoreSnapshot()?.Records.Select(record => record.Key).ToArray() ?? Array.Empty<string>();
CultCache OpenTwo(string a, string b)
{
var cache = new CultCache(AB, CultCacheMessagePack.CreateCodec(AB));
cache.AddBackingStore(new SingleFileMessagePackBackingStore(a), typeof(EvA));
cache.AddBackingStore(new SingleFileMessagePackBackingStore(b), typeof(EvB));
cache.PullAllBackingStoresAsync().GetAwaiter().GetResult();
return cache;
}
var pathA = NewPath("single-file");
var pathB = NewPath("single-file");
Directory.CreateDirectory(Path.GetDirectoryName(pathA)!);
Directory.CreateDirectory(Path.GetDirectoryName(pathB)!);
using (var cache = OpenTwo(pathA, pathB))
{
PutB(cache, K1);
Assert.That(KeysIn(pathB), Is.EqualTo(new[] { "k1" }));
var refused = Assert.Throws<InvalidOperationException>(() => cache.Commit(batch =>
{
batch.Remove(K1);
batch.Upsert(typeof(EvA), new EvA { Name = "k1" }, K1);
}))!;
Assert.That(refused.Message, Does.Contain("The batch spans").And.Contain("a commit lands in one home store")
.And.Contain("stage its removal and its add apart").And.Not.Contain("second store"));
Assert.That(KeysIn(pathA), Is.Empty, "the refused batch wrote nothing");
Assert.That(KeysIn(pathB), Is.EqualTo(new[] { "k1" }));
Assert.That(cache.GetStored(K1)!.Descriptor.DocumentType, Is.EqualTo(typeof(EvB)));
Assert.That(cache.Remove(K1), Is.True);
cache.FlushAsync().GetAwaiter().GetResult();
cache.UpsertAsync(typeof(EvA), new EvA { Name = "k1" }, K1).GetAwaiter().GetResult();
cache.FlushAsync().GetAwaiter().GetResult();
}
Assert.That(KeysIn(pathA), Is.EqualTo(new[] { "k1" }), "staged apart the key is in the add's store");
Assert.That(KeysIn(pathB), Is.Empty, "and no longer in the removal's");
}
// finding removal-bookkeeping-unpinned, mutant X2(a retype elsewhere in the batch skips every key's staging claim): a batch
// holding two retypes commits, and a cross-type add at a key it did not remove is still refused beside a retype.
[TestCaseSource(nameof(Kinds))]
public void EachKeyOfABatchIsJudgedByItsOwnRemoval(string kind)
{
using var cache = Open(NewPath(kind), kind, AB);
PutB(cache, K1);
PutB(cache, K2);
PutB(cache, K3);
Assert.That(cache.Commit(batch =>
{
batch.Remove(K1);
batch.Upsert(typeof(EvA), new EvA { Name = "k1" }, K1);
batch.Remove(K2);
batch.Upsert(typeof(EvA), new EvA { Name = "k2" }, K2);
}), Is.True, "two retypes");
var bad = Assert.Throws<CultSchemaConflictException>(() => cache.Commit(batch =>
{
batch.Remove(K1);
batch.Upsert(typeof(EvB), new EvB { Name = "k1" }, K1);
batch.Upsert(typeof(EvA), new EvA { Name = "k3" }, K3);
}))!;
Assert.That(bad.RecordKey, Is.EqualTo(K3.Value), "one key's removal licenses no other key's retype");
Assert.That(cache.GetStored(K3)!.Descriptor.DocumentType, Is.EqualTo(typeof(EvB)));
Assert.That(cache.GetStored(K1)!.Descriptor.DocumentType, Is.EqualTo(typeof(EvA)), "the refused batch changed nothing");
Assert.That(cache.Commit(batch =>
{
batch.Upsert(typeof(EvB), new EvB { Name = "k3", Value = 5 }, K3);
batch.Remove(K3);
}), Is.True, "an add and then a removal removes");
Assert.That(cache.GetStored(K3), Is.Null);
}
// finding removal-bookkeeping-unpinned, mutant X13 (any second add counts as a replacement): two adds of one batch with no
// removal are an upsert, judged against what the key holds.
[TestCaseSource(nameof(Kinds))]
public void TwoAddsWithNoRemovalAreAnUpsert(string kind)
{
using var cache = Open(NewPath(kind), kind, AB);
PutB(cache, K2);
var refused = Assert.Throws<CultSchemaConflictException>(() => cache.Commit(batch =>
{
batch.Upsert(typeof(EvA), new EvA { Name = "k2", Value = 1 }, K2);
batch.Upsert(typeof(EvA), new EvA { Name = "k2", Value = 2 }, K2);
}))!;
if (kind == "storeless")
Assert.That(refused.Message, Does.Contain("would retype the key").And.Contain("Remove the record first, in the same batch or an earlier commit, then add the other type's."));
Assert.That(cache.GetStored(K2)!.Descriptor.DocumentType, Is.EqualTo(typeof(EvB)));
}
// finding removal-bookkeeping-unpinned, mutant X18 (no removal noted for a record this cache wrote itself): a cache that wrote
// the key itself, then removes it and stages another type there, lands that type at the flush.
[TestCaseSource(nameof(Kinds))]
public void ARecordThisCacheWroteItselfRetypesStagedApart(string kind)
{
var path = NewPath(kind);
using (var cache = Open(path, kind, AB))
{
PutB(cache, K2);
cache.FlushAllBackingStores();
Assert.That(cache.Remove(K2), Is.True);
cache.UpsertAsync(typeof(EvA), new EvA { Name = "k2", Value = 9 }, K2).GetAwaiter().GetResult();
cache.FlushAllBackingStores();
}
if (kind != "storeless")
Assert.That(TypeAt(path, kind, AB, K2), Is.EqualTo(typeof(EvA)));
}
// finding removal-bookkeeping-unpinned, mutant M (a refusal keeps the staged removal): a removal the store refused is gone, so
// a later add at a key another writer has since rewritten is judged as an upsert, not as the stale removal.
[TestCaseSource(nameof(Kinds))]
public void ARefusedRemovalIsNotReusedByALaterAdd(string kind)
{
if (kind == "storeless")
Assert.Ignore("a refusal is a store's.");
var path = NewPath(kind);
using (var seed = Open(path, kind, ABC))
PutB(seed, K1);
using var cache = Open(path, kind, AB);
using var other = Open(path, kind, ABC);
Assert.That(cache.Remove(K1), Is.True);
// Another writer retypes the key to a type this cache does not know: the cache's staged removal meets a record it may not remove.
Assert.That(other.Commit(batch =>
{
batch.Remove(K1);
batch.Upsert(typeof(EvC), new EvC { Name = "k1" }, K1);
}), Is.True);
Assert.Throws<CultSchemaConflictException>(() => cache.FlushAllBackingStores());
// The writer removes it again and puts a B there; this cache pulls and then stages an A with no removal in front of it.
Assert.That(other.Commit(batch => batch.Remove(K1)), Is.True);
cache.PullAllBackingStoresAsync().GetAwaiter().GetResult();
Assert.That(other.Commit(batch => batch.Upsert(typeof(EvB), new EvB { Name = "k1", Value = 7 }, K1)), Is.True);
cache.UpsertAsync(typeof(EvA), new EvA { Name = "k1", Value = 8 }, K1).GetAwaiter().GetResult();
Assert.Throws<CultSchemaConflictException>(() => cache.FlushAllBackingStores());
Assert.That(TypeAt(path, kind, AB, K1), Is.EqualTo(typeof(EvB)));
}
// finding removal-bookkeeping-unpinned, mutant X9 (a refusal clears every staged removal): a refusal of one key leaves another
// key's staged retype, and the next flush lands it.
[TestCaseSource(nameof(Kinds))]
public void ARefusedFlushKeepsAnotherKeysStagedRetype(string kind)
{
if (kind == "storeless")
Assert.Ignore("a refusal is a store's.");
var path = NewPath(kind);
using (var seed = Open(path, kind, ABC))
PutB(seed, K2);
using var cache = Open(path, kind, AB);
using var other = Open(path, kind, ABC);
Assert.That(cache.Remove(K2), Is.True);
cache.UpsertAsync(typeof(EvA), new EvA { Name = "k2", Value = 9 }, K2).GetAwaiter().GetResult();
cache.UpsertAsync(typeof(EvA), new EvA { Name = "k1", Value = 1 }, K1).GetAwaiter().GetResult();
// Another writer puts a type this cache does not know at k1 before the flush: k1's staged add is refused under the lock.
Assert.That(other.Commit(batch => batch.Upsert(typeof(EvC), new EvC { Name = "k1" }, K1)), Is.True);
var refusal = Assert.Throws<CultSchemaConflictException>(() => cache.FlushAllBackingStores())!;
Assert.That(refusal.RecordKey, Is.EqualTo(K1.Value));
cache.FlushAllBackingStores();
Assert.That(TypeAt(path, kind, ABC, K2), Is.EqualTo(typeof(EvA)), "the retype of k2 survived the refusal of k1");
}
// finding rebase-check-was-reachable, ruling variant-rebase-follows-plain-write: a variant rebased onto a base of another type
// follows the plain-write rule. Where RbY lists RbX's id (a plain write of RbY would own an RbX record) the rebase commits and
// the variant becomes an RbY; where it would be refused (RbZ) the rebase is refused and the variant is untouched.
[TestCaseSource(nameof(VariantKinds))]
public void ARebaseChangesTypeExactlyWhereAPlainWriteMay(string kind)
{
Assert.That(Rb.GetRequired(typeof(RbX)).SchemaId, Is.EqualTo(RbXId));
using var cache = Open(NewPath(kind), kind, Rb);
var baseX = new CultRecordKey("base-x");
var baseY = new CultRecordKey("base-y");
var baseZ = new CultRecordKey("base-z");
var variant = new CultRecordKey("v");
Assert.That(cache.Commit(batch =>
{
batch.Upsert(typeof(RbX), new RbX { Name = "base-x" }, baseX);
batch.Upsert(typeof(RbY), new RbY { Name = "base-y" }, baseY);
batch.Upsert(typeof(RbZ), new RbZ { Name = "base-z" }, baseZ);
batch.UpsertVariant(variant, baseX, new[] { cache.Override<RbX>(nameof(RbX.Name), "v") });
}), Is.True);
// The plain write of the same types gives the same two answers.
Assert.Throws<CultSchemaConflictException>(() => cache.Commit(batch => batch.Upsert(typeof(RbZ), new RbZ { Name = "base-x" }, baseX)));
Assert.Throws<CultSchemaConflictException>(
() => cache.Commit(batch => batch.UpsertVariant(variant, baseZ, new[] { cache.Override<RbZ>(nameof(RbZ.Name), "v") })));
Assert.That(cache.GetStored(variant)!.Descriptor.DocumentType, Is.EqualTo(typeof(RbX)));
Assert.That(cache.GetStored(variant)!.Variant!.BaseKey, Is.EqualTo(baseX.Value));
Assert.That(cache.Commit(batch => batch.UpsertVariant(variant, baseY, new[] { cache.Override<RbY>(nameof(RbY.Name), "v") })), Is.True);
Assert.That(cache.GetStored(variant)!.Descriptor.DocumentType, Is.EqualTo(typeof(RbY)));
Assert.That(cache.GetStored(variant)!.Variant!.BaseKey, Is.EqualTo(baseY.Value));
}
}
}

View file

@ -250,8 +250,10 @@ namespace GameCult.Caching.Tests
SeedBig(cache);
cache.Commit(batch => batch.Upsert(typeof(VariantOther), new VariantOther { Name = "other" }, new CultRecordKey("other")));
var rebase = Refused(() => cache.Commit(batch => batch.UpsertVariant(BigKey, new CultRecordKey("other"))));
Assert.That(rebase.Message, Does.Contain(BigKey.Value).And.Contain("VariantOther"));
var rebase = Assert.Throws<CultSchemaConflictException>(() => cache.Commit(batch => batch.UpsertVariant(BigKey, new CultRecordKey("other"))))!;
// Held in a store, the key's record is a VariantGear and the write is a VariantOther: the ownership rule refuses it first
// (variants:ruling:cache-refuses-retype), so this refusal no longer names the types.
Assert.That(rebase.Message, Does.Contain(BigKey.Value));
}
// A store whose variant record names another type than its base's: the loader refuses it naming both keys.
@ -280,6 +282,24 @@ namespace GameCult.Caching.Tests
Assert.That(load.Message, Does.Contain("other-variant").And.Contain("gear-base"));
}
// Rebasing a variant onto a base of another type would retype its key: refused with or without a store, the record untouched.
[Test]
public void ARebaseOntoABaseOfAnotherTypeIsRefusedWithoutAStore()
{
using var cache = new CultCache(Registry, CultCacheMessagePack.CreateCodec(Registry));
SeedBase(cache);
SeedBig(cache);
var other = new CultRecordKey("other");
cache.Commit(batch => batch.Upsert(typeof(VariantOther), new VariantOther { Name = "other" }, other));
var rebase = Assert.Throws<CultSchemaConflictException>(() => cache.Commit(batch => batch.UpsertVariant(BigKey, other)))!;
Assert.That(rebase.RecordKey, Is.EqualTo(BigKey.Value));
Assert.That(cache.GetStored(BigKey)!.Variant!.BaseKey, Is.EqualTo(BaseKey.Value));
Assert.That(cache.GetStored(BigKey)!.Descriptor.DocumentType, Is.EqualTo(typeof(VariantGear)));
Assert.Throws<CultSchemaConflictException>(() => cache.UpsertVariantAsync(BigKey, other).GetAwaiter().GetResult());
}
[Test]
public void AVariantOfAGlobalIsRefused()
{

View file

@ -0,0 +1,417 @@
#nullable enable
using System;
using System.IO;
using System.Linq;
using System.Security.Cryptography;
using GameCult.Caching.MessagePack;
using MessagePack;
using NUnit.Framework;
namespace GameCult.Caching.Tests
{
// variants:ruling:write-ownership-by-written-type and variants:ruling:cache-refuses-retype: an upsert replaces the record stored at
// its key only when the type it writes there owns that record. Which type owns the id, and which other types a registry holds,
// change nothing; retyping a key is a removal and an add. The rule is one per-key predicate (CultDocumentRegistry.WrittenOwns)
// for a cache and a cacheless writer, in a single-file store and a directory store.
public class WriteOwnershipTests
{
// Two schemas of one shape that are never the same type.
[CultDocument("tests.wo_a", "tests.wo_a.v1")]
[MessagePackObject(AllowPrivate = true)]
internal sealed class WoA
{
[Key(0)] [CultName] public string Name = string.Empty;
[Key(1)] public int Value;
}
[CultDocument("tests.wo_b", "tests.wo_b.v1")]
[MessagePackObject(AllowPrivate = true)]
internal sealed class WoB
{
[Key(0)] [CultName] public string Name = string.Empty;
[Key(1)] public int Value;
}
private static readonly CultDocumentRegistry ARegistry = CultDocumentRegistry.ForTypes(new[] { typeof(WoA) });
private static readonly CultDocumentRegistry ABRegistry = CultDocumentRegistry.ForTypes(new[] { typeof(WoA), typeof(WoB) });
private static readonly CultRecordKey K1 = new("k1");
private static readonly CultRecordKey K2 = new("k2");
private string _directory = string.Empty;
[SetUp]
public void CreateDirectory()
{
_directory = Path.Combine(Path.GetTempPath(), $"cultlib-writeownership-{Guid.NewGuid():N}");
Directory.CreateDirectory(_directory);
}
[TearDown]
public void DeleteDirectory()
{
if (Directory.Exists(_directory))
Directory.Delete(_directory, recursive: true);
}
public static readonly object[] StoreKinds = { false, true };
private string PathFor(bool directory) => Path.Combine(_directory, directory ? "store" : "store.cc");
private static CultCache Open(string path, bool directory, CultDocumentRegistry registry)
{
var cache = new CultCache(registry);
cache.AddBackingStore(directory ? new DirectoryMessagePackBackingStore(path) : new SingleFileMessagePackBackingStore(path));
cache.PullAllBackingStoresAsync().GetAwaiter().GetResult();
return cache;
}
private static string Fingerprint(string path)
{
using var sha = SHA256.Create();
if (File.Exists(path))
return Convert.ToHexString(sha.ComputeHash(File.ReadAllBytes(path)));
return string.Join("|", Directory.GetFiles(path, "*", SearchOption.AllDirectories)
.OrderBy(file => file, StringComparer.Ordinal)
.Select(file => Path.GetRelativePath(path, file) + ":" + Convert.ToHexString(sha.ComputeHash(File.ReadAllBytes(file)))));
}
private static Type TypeAt(string path, bool directory, CultDocumentRegistry registry, CultRecordKey key)
{
using var cache = Open(path, directory, registry);
return cache.GetStored(key)!.Descriptor.DocumentType;
}
// A store that a cache holding both types wrote: A at k1 and B at k2.
private string SeedAAtK1BAtK2(bool directory)
{
var path = PathFor(directory);
using var cache = Open(path, directory, ABRegistry);
Assert.That(cache.Commit(batch =>
{
batch.Upsert(new WoA { Name = "k1", Value = 1 }, new CultRecordHandle<WoA>(K1));
batch.Upsert(new WoB { Name = "k2", Value = 2 }, new CultRecordHandle<WoB>(K2));
}), Is.True);
return path;
}
// P1: a cache holding A and B may not replace B's record with A's, whoever else the registry declares.
[TestCaseSource(nameof(StoreKinds))]
public void ACacheRefusesToWriteOneRegisteredTypeOverAnothersRecord(bool directory)
{
var path = SeedAAtK1BAtK2(directory);
var before = Fingerprint(path);
using (var cache = Open(path, directory, ABRegistry))
{
Assert.Throws<CultSchemaConflictException>(() => cache.Commit(batch => batch.Upsert(new WoA { Name = "k2", Value = 9 }, new CultRecordHandle<WoA>(K2))));
Assert.That(cache.GetStored(K2)!.Descriptor.DocumentType, Is.EqualTo(typeof(WoB)), "the cache still holds B's record");
Assert.Throws<CultSchemaConflictException>(() => cache.UpsertAsync(typeof(WoA), new WoA { Name = "k2", Value = 9 }, K2));
Assert.That(cache.GetStored(K2)!.Descriptor.DocumentType, Is.EqualTo(typeof(WoB)), "a staged write is refused at the call, not at the flush");
}
Assert.That(Fingerprint(path), Is.EqualTo(before));
Assert.That(TypeAt(path, directory, ABRegistry, K2), Is.EqualTo(typeof(WoB)));
// The pair that differs only in the type written: B over B's own record.
using (var cache = Open(path, directory, ABRegistry))
Assert.That(cache.Commit(batch => batch.Upsert(new WoB { Name = "k2", Value = 3 }, new CultRecordHandle<WoB>(K2))), Is.True);
}
// Retype is a removal and an add, in separate commits.
[TestCaseSource(nameof(StoreKinds))]
public void ARetypeIsARemovalAndAnAddInSeparateCommits(bool directory)
{
var path = SeedAAtK1BAtK2(directory);
using (var cache = Open(path, directory, ABRegistry))
{
Assert.That(cache.Commit(batch => batch.Remove(K2)), Is.True);
Assert.That(cache.Commit(batch => batch.Upsert(new WoA { Name = "k2", Value = 9 }, new CultRecordHandle<WoA>(K2))), Is.True);
}
Assert.That(TypeAt(path, directory, ABRegistry, K2), Is.EqualTo(typeof(WoA)));
Assert.That(TypeAt(path, directory, ABRegistry, K1), Is.EqualTo(typeof(WoA)), "the bystander is untouched");
}
// The store is judged again where the write lands, under its lock: another writer retyped the key after this cache loaded it,
// so this cache's own-type write of the old record is now a write over a record that is not its type's.
[TestCaseSource(nameof(StoreKinds))]
public void AnotherWritersRetypeRefusesThisCachesStaleWriteUnderTheLock(bool directory)
{
var path = SeedAAtK1BAtK2(directory);
using var stale = Open(path, directory, ABRegistry);
Assert.That(stale.GetStored(K1)!.Descriptor.DocumentType, Is.EqualTo(typeof(WoA)));
using (var other = Open(path, directory, ABRegistry))
{
Assert.That(other.Commit(batch => batch.Remove(K1)), Is.True);
Assert.That(other.Commit(batch => batch.Upsert(new WoB { Name = "k1", Value = 5 }, new CultRecordHandle<WoB>(K1))), Is.True);
}
var before = Fingerprint(path);
Assert.Throws<CultSchemaConflictException>(() => stale.Commit(batch => batch.Upsert(new WoA { Name = "k1", Value = 9 }, new CultRecordHandle<WoA>(K1))));
Assert.That(Fingerprint(path), Is.EqualTo(before));
Assert.That(TypeAt(path, directory, ABRegistry, K1), Is.EqualTo(typeof(WoB)));
}
public static readonly object[] Registries = { "none", "A", "AB", "Shared" };
private static CultDocumentRegistry? Lent(object kind) => kind switch
{
"A" => ARegistry,
"AB" => ABRegistry,
"Shared" => CultDocumentRegistry.Shared,
_ => null,
};
// P3: a cacheless write of A over B's record is refused whatever registry is lent, and the same lent or not.
[TestCaseSource(nameof(Registries))]
public void ACachelessWriteOfOneTypeOverAnothersRecordIsRefusedWhateverRegistryIsLent(object lent)
{
var path = SeedAAtK1BAtK2(directory: false);
var before = Fingerprint(path);
var store = new SingleFileMessagePackBackingStore(path);
Assert.Throws<CultSchemaConflictException>(() => store.WriteDocument(K2, new WoA { Name = "k2", Value = 9 }, registry: Lent(lent)));
Assert.That(Fingerprint(path), Is.EqualTo(before));
// WriteRecord names its entry and nothing else.
var entry = ARegistry.GetRequired<WoA>().ToCatalogEntry();
var record = new CultPersistedRecord
{
Key = K2.Value,
SchemaId = entry.SchemaId,
StoredAt = "2026-10-28T00:00:00.0000000+00:00",
Payload = MessagePackSerializer.Serialize(new WoA { Name = "k2", Value = 9 }),
};
Assert.Throws<CultSchemaConflictException>(() => store.WriteRecord(record, entry, null, Lent(lent)));
Assert.That(Fingerprint(path), Is.EqualTo(before));
// The pair that differs only in whose record it is: A over A's own, which commits for every registry.
store.WriteDocument(K1, new WoA { Name = "k1", Value = 9 }, registry: Lent(lent) ?? ARegistry);
Assert.That(TypeAt(path, false, ABRegistry, K1), Is.EqualTo(typeof(WoA)));
}
// The judge is per key, and an answer for one written type is never reused for another. Another writer left A's record at two
// keys; this stale cache holds A at one and B at the other and writes both. The key A owns lands, the key where B meets A's
// record is the one refused, whichever is judged first.
[TestCase("a-own", "b-bad", true, false)]
[TestCase("a-own", "b-bad", false, false)]
[TestCase("b-own", "a-bad", true, true)]
[TestCase("b-own", "a-bad", false, true)]
public void AStaleBatchIsJudgedPerKeyByEachKeysWrittenType(string ownKey, string badKey, bool ownFirst, bool directory)
{
var path = PathFor(directory);
var own = new CultRecordKey(ownKey);
var bad = new CultRecordKey(badKey);
using (var seed = Open(path, directory, ABRegistry))
Assert.That(seed.Commit(batch =>
{
batch.Upsert(new WoA { Name = ownKey, Value = 1 }, new CultRecordHandle<WoA>(own));
batch.Upsert(new WoB { Name = badKey, Value = 1 }, new CultRecordHandle<WoB>(bad));
}), Is.True);
using var stale = Open(path, directory, ABRegistry);
using (var other = Open(path, directory, ABRegistry))
{
Assert.That(other.Commit(batch => batch.Remove(bad)), Is.True);
Assert.That(other.Commit(batch => batch.Upsert(new WoA { Name = badKey, Value = 2 }, new CultRecordHandle<WoA>(bad))), Is.True);
}
var before = Fingerprint(path);
void StageOwn(CultCacheBatch batch) => batch.Upsert(new WoA { Name = ownKey, Value = 3 }, new CultRecordHandle<WoA>(own));
void StageBad(CultCacheBatch batch) => batch.Upsert(new WoB { Name = badKey, Value = 3 }, new CultRecordHandle<WoB>(bad));
var refusal = Assert.Throws<CultSchemaConflictException>(() => stale.Commit(batch =>
{
if (ownFirst) { StageOwn(batch); StageBad(batch); } else { StageBad(batch); StageOwn(batch); }
}))!;
Assert.That(refusal.RecordKey, Is.EqualTo(badKey));
Assert.That(refusal.Message, Does.Not.Contain($"'{ownKey}'"), "the key A owns is not among the refused");
Assert.That(Fingerprint(path), Is.EqualTo(before));
}
// A batch that removes a key is a removal of it, whatever this cache has staged there: the removal rule judges it, so a key
// another writer retyped can still be removed by the cache that staged a write of the old type.
[TestCaseSource(nameof(StoreKinds))]
public void ABatchRemovalOfAStagedKeyIsJudgedAsARemoval(bool directory)
{
var path = SeedAAtK1BAtK2(directory);
using var stale = Open(path, directory, ABRegistry);
stale.UpsertAsync(typeof(WoA), new WoA { Name = "k1", Value = 9 }, K1).GetAwaiter().GetResult();
using (var other = Open(path, directory, ABRegistry))
{
Assert.That(other.Commit(batch => batch.Remove(K1)), Is.True);
Assert.That(other.Commit(batch => batch.Upsert(new WoB { Name = "k1", Value = 5 }, new CultRecordHandle<WoB>(K1))), Is.True);
}
Assert.That(stale.Commit(batch => batch.Remove(K1)), Is.True);
using var reopened = Open(path, directory, ABRegistry);
Assert.That(reopened.GetStored(K1), Is.Null);
Assert.That(reopened.GetStored(K2)!.Descriptor.DocumentType, Is.EqualTo(typeof(WoB)), "the bystander is untouched");
}
// The ruling's retype: one batch holds the removal and the add, in the order they were called. The removal half is judged as a
// removal (a registered type's record is removable), and the record lands at the key.
[TestCaseSource(nameof(StoreKinds))]
public void ARetypeIsOneBatchHoldingTheRemovalAndTheAdd(bool directory)
{
var path = SeedAAtK1BAtK2(directory);
using (var cache = Open(path, directory, ABRegistry))
{
Assert.That(cache.Commit(batch =>
{
batch.Remove(K2);
batch.Upsert(new WoB { Name = "k2", Value = 8 }, new CultRecordHandle<WoB>(K2));
batch.Upsert(new WoA { Name = "k2", Value = 9 }, new CultRecordHandle<WoA>(K2));
}), Is.True, "the key stays a replacement through a later add of the same batch");
Assert.That(cache.GetStored(K2)!.Descriptor.DocumentType, Is.EqualTo(typeof(WoA)));
}
Assert.That(TypeAt(path, directory, ABRegistry, K2), Is.EqualTo(typeof(WoA)));
Assert.That(TypeAt(path, directory, ABRegistry, K1), Is.EqualTo(typeof(WoA)), "the bystander is untouched");
}
// The order is the meaning: a removal called after the add leaves the key removed.
[TestCaseSource(nameof(StoreKinds))]
public void ARemovalCalledAfterTheAddLeavesTheKeyRemoved(bool directory)
{
var path = SeedAAtK1BAtK2(directory);
using (var cache = Open(path, directory, ABRegistry))
{
Assert.That(cache.Commit(batch =>
{
batch.Remove(K2);
batch.Upsert(new WoA { Name = "k2", Value = 9 }, new CultRecordHandle<WoA>(K2));
batch.Remove(K2);
}), Is.True);
Assert.That(cache.GetStored(K2), Is.Null);
}
using var reopened = Open(path, directory, ABRegistry);
Assert.That(reopened.GetStored(K2), Is.Null);
}
// The removal half is a removal and nothing more: a record no registered type owns is not replaced by adding after its removal.
[TestCaseSource(nameof(StoreKinds))]
public void ARetypeOfARecordNoRegisteredTypeOwnsIsStillRefused(bool directory)
{
var path = SeedAAtK1BAtK2(directory);
var before = Fingerprint(path);
using (var cache = Open(path, directory, ARegistry))
{
Assert.Throws<CultSchemaConflictException>(() => cache.Commit(batch =>
{
batch.Remove(K2);
batch.Upsert(new WoA { Name = "k2", Value = 9 }, new CultRecordHandle<WoA>(K2));
}));
}
Assert.That(Fingerprint(path), Is.EqualTo(before));
}
// A removal of a key the cache does not hold is judged under the store's lock, and so is the add after it: another writer put
// B there, this cache never loaded it, and removing then adding A is a retype of whatever is stored.
[TestCaseSource(nameof(StoreKinds))]
public void ARetypeOfAKeyTheCacheDoesNotHoldCommitsInOneBatch(bool directory)
{
var path = PathFor(directory);
using var early = Open(path, directory, ABRegistry);
using (var other = Open(path, directory, ABRegistry))
Assert.That(other.Commit(batch => batch.Upsert(new WoB { Name = "k1", Value = 1 }, new CultRecordHandle<WoB>(K1))), Is.True);
Assert.That(early.Commit(batch =>
{
batch.Remove(K1);
batch.Upsert(new WoA { Name = "k1", Value = 2 }, new CultRecordHandle<WoA>(K1));
}), Is.True);
Assert.That(TypeAt(path, directory, ABRegistry, K1), Is.EqualTo(typeof(WoA)));
}
// A removal and an add called separately on the cache are one retype too: the add, staged after the removal, is not lost to the
// flush that judges it, and the type changes in the file.
[TestCaseSource(nameof(StoreKinds))]
public void AStagedRemovalAndAddOfAnotherTypeIsNotLostAtTheFlush(bool directory)
{
var path = SeedAAtK1BAtK2(directory);
using (var cache = Open(path, directory, ABRegistry))
{
Assert.That(cache.Remove(K2), Is.True);
cache.UpsertAsync(typeof(WoA), new WoA { Name = "k2", Value = 9 }, K2).GetAwaiter().GetResult();
cache.FlushAllBackingStores();
Assert.That(cache.GetStored(K2)!.Descriptor.DocumentType, Is.EqualTo(typeof(WoA)));
}
Assert.That(TypeAt(path, directory, ABRegistry, K2), Is.EqualTo(typeof(WoA)));
}
[TestCaseSource(nameof(StoreKinds))]
public void AStagedRemovalOfAKeyTheCacheDoesNotHoldAndAnAddOfAnotherTypeIsNotLostAtTheFlush(bool directory)
{
var path = PathFor(directory);
using var early = Open(path, directory, ABRegistry);
using (var other = Open(path, directory, ABRegistry))
Assert.That(other.Commit(batch => batch.Upsert(new WoB { Name = "k1", Value = 1 }, new CultRecordHandle<WoB>(K1))), Is.True);
Assert.That(early.Remove(K1), Is.False, "the cache holds nothing there");
early.UpsertAsync(typeof(WoA), new WoA { Name = "k1", Value = 2 }, K1).GetAwaiter().GetResult();
early.FlushAllBackingStores();
Assert.That(TypeAt(path, directory, ABRegistry, K1), Is.EqualTo(typeof(WoA)));
}
// A removal is spent by the flush that judged it: a later add of another type, with no removal in front of it, is judged as an upsert again.
[TestCaseSource(nameof(StoreKinds))]
public void ARemovalIsSpentByTheFlushThatJudgedIt(bool directory)
{
var path = SeedAAtK1BAtK2(directory);
using var cache = Open(path, directory, ABRegistry);
Assert.That(cache.Remove(K2), Is.True);
cache.FlushAllBackingStores();
// Another writer puts B at the key this cache removed; this cache never loaded it, and stages an A there with no removal.
using (var other = Open(path, directory, ABRegistry))
Assert.That(other.Commit(batch => batch.Upsert(new WoB { Name = "k2", Value = 4 }, new CultRecordHandle<WoB>(K2))), Is.True);
cache.UpsertAsync(typeof(WoA), new WoA { Name = "k2", Value = 5 }, K2).GetAwaiter().GetResult();
Assert.Throws<CultSchemaConflictException>(() => cache.FlushAllBackingStores());
Assert.That(TypeAt(path, directory, ABRegistry, K2), Is.EqualTo(typeof(WoB)));
}
// No store, same rule: a cache holding A and B refuses one's write over the other's record, on Commit and on UpsertAsync, and
// a retype is a removal and an add, in one batch or staged apart.
[Test]
public void AStorelessCacheRefusesToWriteOneRegisteredTypeOverAnothersRecord()
{
using var cache = new CultCache(ABRegistry);
Assert.That(cache.Commit(batch => batch.Upsert(new WoB { Name = "k2", Value = 2 }, new CultRecordHandle<WoB>(K2))), Is.True);
var commit = Assert.Throws<CultSchemaConflictException>(() => cache.Commit(batch => batch.Upsert(new WoA { Name = "k2", Value = 9 }, new CultRecordHandle<WoA>(K2))))!;
Assert.That(commit.RecordKey, Is.EqualTo(K2.Value));
Assert.Throws<CultSchemaConflictException>(() => cache.UpsertAsync(typeof(WoA), new WoA { Name = "k2", Value = 9 }, K2));
Assert.That(cache.GetStored(K2)!.Descriptor.DocumentType, Is.EqualTo(typeof(WoB)));
// The pair that differs only in the type written: B over B's own record.
Assert.That(cache.Commit(batch => batch.Upsert(new WoB { Name = "k2", Value = 3 }, new CultRecordHandle<WoB>(K2))), Is.True);
Assert.That(cache.Commit(batch =>
{
batch.Remove(K2);
batch.Upsert(new WoA { Name = "k2", Value = 9 }, new CultRecordHandle<WoA>(K2));
}), Is.True);
Assert.That(cache.GetStored(K2)!.Descriptor.DocumentType, Is.EqualTo(typeof(WoA)));
Assert.That(cache.Remove(K2), Is.True);
cache.UpsertAsync(typeof(WoB), new WoB { Name = "k2", Value = 4 }, K2).GetAwaiter().GetResult();
Assert.That(cache.GetStored(K2)!.Descriptor.DocumentType, Is.EqualTo(typeof(WoB)));
}
}
}

View file

@ -0,0 +1,97 @@
#nullable enable
using System;
using GameCult.Caching.MessagePack;
using MessagePack;
using NUnit.Framework;
namespace GameCult.Caching.Tests
{
// The one per-key predicate of write ownership (variants:ruling:write-ownership-by-written-type, own-entry-reads-whole), over
// catalogs the production types publish. Each refusal is paired with the accepted case that differs in only the one thing the
// rule looks at. Where the predicate is wired in is WriteOwnershipTests.
public class WrittenOwnsTests
{
// One schema name at three versions, nothing declared: N2 reads N1's records whole (it adds a slot) and Wide's only by
// dropping a slot; Renamed has N2's exact shape under another schema name.
[CultDocument("tests.wo_n", "tests.wo_n.v1")]
[MessagePackObject(AllowPrivate = true)]
internal sealed class WoN1
{
[Key(0)] [CultName] public string Name = string.Empty;
[Key(1)] public int Value;
}
[CultDocument("tests.wo_n", "tests.wo_n.v2")]
[MessagePackObject(AllowPrivate = true)]
internal sealed class WoN2
{
[Key(0)] [CultName] public string Name = string.Empty;
[Key(1)] public int Value;
[Key(2)] public string Colour = string.Empty;
}
[CultDocument("tests.wo_n", "tests.wo_n.v3")]
[MessagePackObject(AllowPrivate = true)]
internal sealed class WoNWide
{
[Key(0)] [CultName] public string Name = string.Empty;
[Key(1)] public int Value;
[Key(2)] public string Colour = string.Empty;
[Key(3)] public string Extra = string.Empty;
}
[CultDocument("tests.wo_nx", "tests.wo_nx.v1")]
[MessagePackObject(AllowPrivate = true)]
internal sealed class WoNRenamed
{
[Key(0)] [CultName] public string Name = string.Empty;
[Key(1)] public int Value;
[Key(2)] public string Colour = string.Empty;
}
private static CultSchemaCatalogEntry EntryOf<T>() where T : class =>
CultDocumentRegistry.ForTypes(new[] { typeof(T) }).GetRequired<T>().ToCatalogEntry();
private static CultSchemaCatalogEntry Listing(CultSchemaCatalogEntry entry, params string[] ids) =>
new()
{
SchemaId = entry.SchemaId,
ContentHash = entry.ContentHash,
SchemaName = entry.SchemaName,
SchemaVersion = entry.SchemaVersion,
CanonicalSchemaJson = entry.CanonicalSchemaJson,
Members = entry.Members,
CompatibleSchemaIds = ids,
};
[Test]
public void WrittenOwnsDeclaredIdsAreTheEntrysOwnOrAnyItListsNotOnlyTheFirst()
{
var a = EntryOf<WriteOwnershipTests.WoA>();
var b = EntryOf<WriteOwnershipTests.WoB>();
var catalog = new[] { a };
Assert.That(CultDocumentRegistry.WrittenOwns(a, a.SchemaId, catalog), Is.True, "its own id");
Assert.That(CultDocumentRegistry.WrittenOwns(b, a.SchemaId, catalog), Is.False, "same shape, another schema: not its record");
Assert.That(CultDocumentRegistry.WrittenOwns(Listing(b, b.SchemaId, a.SchemaId), a.SchemaId, catalog), Is.True, "listed last");
Assert.That(CultDocumentRegistry.WrittenOwns(Listing(b, a.SchemaId, b.SchemaId), a.SchemaId, catalog), Is.True, "listed first");
Assert.That(CultDocumentRegistry.WrittenOwns(Listing(b, b.SchemaId, "tests.wo.other"), a.SchemaId, catalog), Is.False, "lists another id");
Assert.That(CultDocumentRegistry.WrittenOwns(a, a.SchemaId, Array.Empty<CultSchemaCatalogEntry>()), Is.True, "declared needs no catalog");
}
[Test]
public void WrittenOwnsAnOlderIdOfItsOwnNameOnlyWhenItReadsTheRecordWhole()
{
var n1 = EntryOf<WoN1>();
var n2 = EntryOf<WoN2>();
var wide = EntryOf<WoNWide>();
var renamed = EntryOf<WoNRenamed>();
Assert.That(CultDocumentRegistry.WrittenOwns(n2, n1.SchemaId, new[] { n1 }), Is.True, "same name, reads whole");
Assert.That(CultDocumentRegistry.WrittenOwns(renamed, n1.SchemaId, new[] { n1 }), Is.False, "the same shape under another name");
Assert.That(CultDocumentRegistry.WrittenOwns(n2, wide.SchemaId, new[] { wide }), Is.False, "same name, drops a persisted slot");
Assert.That(CultDocumentRegistry.WrittenOwns(n2, "tests.wo.unpublished", new[] { n1 }), Is.False, "an id no catalog entry publishes");
Assert.That(CultDocumentRegistry.WrittenOwns(n2, n1.SchemaId, new[] { n1, wide }), Is.True, "another entry in the catalog changes nothing");
}
}
}

View file

@ -13,7 +13,7 @@ Releases as 3.0.0 (major).
- A store is written in the format its content needs, and CultLib 2.0 refuses the newer ones by name. A single-file store holding a document variant is written as `cultcache.store.v2`, one holding an element id as `cultcache.store.v3`; a directory store holding an element id is written as `cultcache.store.v5` (`v4` otherwise). A store is marked only when a record actually holds an id or a variant. Upgrade readers before writers: a 2.0 reader that opens such a store is refused, not misread.
- Writes are write sets. A flush or commit replaces the records it staged and copies every other record forward as stored, under the store's lock, instead of rewriting the whole store from the cache's memory. A record the cache does not know is kept, and an untouched record keeps a dropped member's slot through every write (never reuse a dropped slot; see the schema-compatibility contract).
- A write or removal at a key whose stored record the writer does not own is refused with `CultSchemaConflictException`, and the file is left as it was. A writer owns a record whose schema id it declares (owns or lists compatible) and a record its type reads whole under compatible drift: that record is writable and is restamped under the type's own id on write, as in 2.0.1. A record a type reads only by dropping persisted slots stays read-only until its id is declared compatible, which accepts the loss (this differs from 2.0.1, which shed the slots silently). A cache owns through its registry; a cacheless CultMesh write owns through the catalog entry it writes.
- A cacheless `WriteDocument`, `WriteRecord` or unattached-store `PushAll` owns a stored record in two ways only: the entries it writes declare the record's schema id (an entry's own id, or one it lists compatible), or the type picked to read the record is one of the written entries and reads it whole (no persisted slot unread). The type is picked as a cache registering the call's registry would pick it, except that a written entry is the only description of its schema id: `WriteDocument` takes its entry from the registry (`CultDocumentRegistry.Shared` when none is passed), `WriteRecord` writes the entry it is given, and a registry's own type for that id is never asked. A registry adds claimants and declares nothing, so it can only refuse more; without one the written entries are the only claimants. Refused, with the file left as it was: a record another registered type claims, under that type's current id or an old id it lists; a record whose stored entry lists another registered type's id ahead of the written one's, whatever name the stored entry carries; a record that two registered types list compatible, or that two registered versions of the written schema could read; a record the picked written entry would read only by dropping persisted slots; and the removal of any stored record by a call that writes no entry (a `Delete` then `PushAll`). As in 2.0.1, a record the written entry's own type reads whole under drift is owned and restamped. A registry-less `WriteDocument` reads `Shared` today, so any type loaded there can refuse the write; pass a registry to pin the answer.
- A write replaces the record at a key only when the type it writes there owns that record, decided per key and the same for a cache and for a cacheless `WriteDocument`, `WriteRecord` or unattached-store `PushAll`, in a single-file store and a directory store. The written type owns the stored record when its entry declares the stored id (its own id, or one it lists compatible), or when it is the type that reads the record's published entry and reads it whole (no persisted slot unread): an older id of its own schema name that it reads whole is its own and is restamped, as in 2.0.1. Anything else is refused with `CultSchemaConflictException` and the file left as it was: a record of another registered type, under its current id or an old id it lists, and a record the written type would read only by dropping persisted slots. No registry enters the decision, `CultDocumentRegistry.Shared` included, so a lent registry never changes the answer. This changes a cache: where 2.x let a cache holding two types replace one's record at a key with the other's, it now refuses, with or without a backing store. Retype a key by removing it and adding the new record, in one batch (`batch.Remove(key)` then `batch.Upsert(...)`) or staged apart (`Remove` then `UpsertAsync`); the removal is judged as a removal, so a record no registered type owns is still refused. A removal or check goes to the registry's removal rule, and a cacheless call, which has no registry, removes a record only when an entry it writes owns it (a `Delete` then `PushAll` on an unattached store writes no entry and is refused).
- Registration refuses an owned object-list element type that has no `[CultElementId]` member (a public string with `[Key]`), naming the type and the list. Element ids let a write set merge a list by identity; a list of record references or plain values is a whole value and is not affected.
- The abstract and protected surface for store kinds outside this library changed. On `SingleFileBackingStore`, `SerializeSnapshot` and `DeserializeSnapshot` are replaced by the abstract `SerializeStore`, `SerializeRecord`, `SerializeCatalogEntry` and `ReadStore`, and the abstract `SerializePayload(object)` becomes `SerializePayload(object document, Type documentType)`: a store encodes a document as the type it is given. On `CacheBackingStore`, `Judging` is removed, `ToStoredDocument` takes the record's version (`record, version, catalog, deserialize, out report`), and these protected members are new for a store kind to implement or call: `HeaderFor`, `HeaderForWrite`, `Foreign`, `SetForeignRecords`, `Undeclared`, `RefuseUndeclared`, `RefuseUndeclaredOnDisk`, `RefuseRemovalOfUndeclared`, `Refuse`, `SameStoredIdentity`, `VersionOf`, `StoredAtOver`, `Overwrites`, `StagedChecks`, `StageChecks` and `JudgeChecks`. `MarkFlushSucceeded` is unchanged. `CacheBackingStore.ForeignRecords` is new and public: the records a store holds under a schema id its cache does not declare.
- `CultCommitRequest.Expected` is now `IReadOnlyList<(CultRecordKey Key, string? Version)>` (it was a key with a schema id and a stored-at), `CultCommitRequest.Checks` is new, and `ConditionsHold` takes `(IReadOnlyDictionary<string, string> durable, IEnumerable<CultStoredDocument> observed, IEnumerable<CultForeignRecord> foreign)` in place of `(IReadOnlyCollection<CultPersistedRecord>, IEnumerable<CultStoredDocument>)`. A store kind outside the library lists the record versions it holds and passes its foreign records.
@ -25,6 +25,7 @@ Releases as 3.0.0 (major).
### Added
- `CacheBackingStore.RefuseUndeclaredOnDisk(written, catalog, heldAt, writtenAt, removed, durableAt, foreignAt, unstage, cacheless)` is the one protected per-key ownership judge for a store kind outside the library: it applies the whole rule under the store's lock (the written type owns a record by its own id, an id it lists or an older id of its own name it reads whole; a key the write removes or checks, and a key the cache removed since the last flush, is owned by any registered type that declares or reads the record whole) and refuses through `Refuse`. Both shipped stores call it. Additive: the earlier overload taking a store-built `undeclaredAt` judge is unchanged. A batch that removes a key and adds a record there is staged as the held record's eviction and then the add, so a watcher of the old type sees `Removed`, a watcher of the new type sees `Added`, and a variant rebase onto a base of another type changes type exactly where a plain write may.
- A cacheless `WriteRecord` or `WriteDocument`, and an attach, hold the store's gate while they run, and user code they run (a document's property getters, a constructor while a copied record is read) now throws `InvalidOperationException` if it enters a cache it does not hold or a second store; it used to wait forever against an attach of the same store. A store call that queued on an attach that then failed runs under the gate that owns the store, not beside a cacheless call or a second cache's attach. An attach that fails takes its registry back, so a `PullAll` on the store afterwards is refused as unattached instead of loading records the next cache never sees. A cacheless `WriteRecord` or `WriteDocument` made while the same thread already holds that store from user code another cacheless write of the store runs, or from a journal an attach's pull runs (then "The store is attached to a cache, write through the cache.", as after the attach) throws `InvalidOperationException`; it used to write the file behind the attaching cache and take that cache's registry, so the cache could never flush or pull the store again. A journal that calls its cache's store (`Push`, `Delete`, `PushAll`, `CommitBatch`, `PullAll`, `ReadStoreSnapshot`, `JudgeChecks`) while that cache's `AddBackingStore` runs the journal now throws "A journal entered the cache it journals", as it does after the attach; it used to stage or write records the cache never admitted, so the cache and the file disagreed.
- `SingleFileBackingStore.ReadStoreSnapshot()` reads a store file once, under the store's in-process gate, with the reader a cache's store opens with, and returns the durable snapshot or null when nothing is at the path. `SingleFileBackingStore.WriteRecord(record, catalogEntry, holdsIds, registry)` writes one record without a cache: under the store's lock every other record and catalog entry is copied as stored, the file is replaced atomically, and the header is decided as a flush decides it. It refuses on a store attached to a cache.
- `SingleFileBackingStore.WriteDocument<T>(key, document, storedAt, registry)` writes one typed document without a cache, as a cache write does: it plans element ids, encodes the document as `T` (the type its schema id and ids are taken from, also for a derived instance) and marks the store when it holds an id; a failed write gives the minted ids back. It refuses on a store attached to a cache. The `registry` of `WriteRecord` and `WriteDocument` serves that call only; a store keeps none.