2026-05-09 13:32:30 +01:00
|
|
|
using System;
|
2026-05-31 17:32:42 +01:00
|
|
|
using System.IO;
|
2026-05-09 13:32:30 +01:00
|
|
|
using System.Threading.Tasks;
|
|
|
|
|
|
|
|
|
|
namespace GameCult.Caching.MessagePack
|
|
|
|
|
{
|
|
|
|
|
public sealed class CultCacheOpenOptions
|
|
|
|
|
{
|
|
|
|
|
public CultDocumentRegistry? Registry { get; set; }
|
|
|
|
|
|
Route each document type to one home store; attach hydrates; commit explicit batches
Cut 3 of the CultCache migration (Aetheria docs/cultcache-migration-cut.md, 2.1
and 2.2). One commit, because routing, the batch that replaces the ambient
transaction, and conditional commit share one admission path.
Deleted first:
- globals: InitializeGlobals, MaterializeMissingGlobals, the two-argument
constructor, the initializeGlobals plumbing in CultCacheMessagePack,
ContainsDurableRecord (cache and directory store)
- replication: the push-to-every-store loops and the attach-time push of
existing records
- typeof(T) as the schema owner in AddAsync/UpsertAsync and in
CultNetDatabase.PutAsync/PutPredictedAsync
- the ambient transaction: ExecuteTransactionAsync x2, CultCacheTransaction,
CommitTransaction, VisibleStoredDocuments, the AsyncLocal, the SemaphoreSlim
gate, and every ambient check
- CultCacheOpenOptions.PullOnOpen; OpenAsync is Task.FromResult(Create(...))
- DirectoryMessagePackBackingStore.ResolveLegacyUncataloguedRecordCatalog: the
catalog precommit (fe55bff) is an ancestor of the v4 page format (eea8248),
so no v4 manifest can reach it
- CultCache.DeleteAsync<T>; its one caller, CultNetDatabaseSubscriptionClient,
called it by reflection and now calls Remove(key)
Added:
- AddBackingStore(store, params Type[] homes): attach validates (clean, not
already attached, one untyped store, no claimed home type, no admitted
record changes home) and then pulls under the gate
- Home(type): most derived routed home, else the untyped store
- Admit: every add and remove, from a write, a batch or a load, validates
(home, read-only, loaded-from-home, global singleton), lands durably, then
applies and publishes; a pull hands its whole delta over in one call, so a
refused record leaves the store and the cache unchanged
- read-only stores (CacheBackingStore(readOnly), both store constructors,
CultCacheOpenOptions.ReadOnly); flush skips them and writes only dirty stores
- assignable GetGlobal<T>/GetByName<T>/GetByIndex<T>/Watch<T>; more than one
candidate throws naming them
- Commit(Action<CultCacheBatch>) returning bool, TryCommit returning
CultCommitOutcome, CultCacheBatch (Upsert, Remove, Expect, ExpectUnchanged,
sealed after commit), CultCommitRequest; CacheBackingStore.CommitBatch
evaluates conditions under the store's exclusive lock
- single-file store: <path>.lock sidecar for CommitBatch and PushAll; a clean
store commits onto the file as it is, so records it never loaded survive
- storedAt minted in one place and bumped a tick when not later than the record
it replaces
Consumer adjustments in this repo: Mesh streaming tests drop PullOnOpen; the
Studio bridge stops reflecting on it; BackingStoreTests' global test creates
its global explicitly and its transaction tests are rewritten against Commit
(the two that proved ambient hydration rules are deleted); the pre-Cut 2
refusal test expects the attach to throw.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 17:57:58 +02:00
|
|
|
public bool ReadOnly { get; set; }
|
2026-05-09 13:32:30 +01:00
|
|
|
|
|
|
|
|
public bool FlushOnDispose { get; set; }
|
|
|
|
|
|
|
|
|
|
public bool StoreFlushOnDispose { get; set; }
|
|
|
|
|
|
2026-05-31 17:32:42 +01:00
|
|
|
public bool UseDirectoryStore { get; set; }
|
2026-05-09 13:32:30 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
public static class CultCacheMessagePack
|
|
|
|
|
{
|
C# core: variant records, top-level Set overrides, resolution in the admitting hold
A variant record stores [baseKey, overrides[]] in record slot 4 and an empty
payload; the cache resolves it against its base inside the hold that admits it or
any record on its base chain, and re-resolves dependents in the same Apply, so a
base change reaches its variants with contiguous sequences and no stamp. Plain
Upsert at a variant key is refused (Flatten is the explicit way); cycle, missing
base, different concrete type, evicted base with survivors and a name that
shadows the base are refused naming the keys. Overrides of a dropped slot or an
undecodable value are ignored with a warning and reported as compatible drift.
The store writes cultcache.store.v2 only when it holds a variant and reads v1 and
v2. The codec pair CultInspectorModel took becomes CultCodec, shared with the
cache, which needs it to clone documents and overlay override values. The
directory store, CultMesh single-file reads and CultNet row building refuse a
variant by name until their own cuts.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 03:51:53 +02:00
|
|
|
// The store's encoding as one port. Values serialize without the document guard but with their owning document's
|
|
|
|
|
// options, resolvers included, so dictionary keys compare by the bytes the store writes; a document type
|
|
|
|
|
// deserializes as a registered document, a member type as itself.
|
|
|
|
|
public static CultCodec CreateCodec(CultDocumentRegistry registry) => new CultCodec(
|
2026-10-14 21:52:36 +02:00
|
|
|
(value, type, document) => global::MessagePack.MessagePackSerializer.Serialize(type, value, CultDocumentMessagePackSerialization.OptionsForDocument(document)),
|
C# core: variant records, top-level Set overrides, resolution in the admitting hold
A variant record stores [baseKey, overrides[]] in record slot 4 and an empty
payload; the cache resolves it against its base inside the hold that admits it or
any record on its base chain, and re-resolves dependents in the same Apply, so a
base change reaches its variants with contiguous sequences and no stamp. Plain
Upsert at a variant key is refused (Flatten is the explicit way); cycle, missing
base, different concrete type, evicted base with survivors and a name that
shadows the base are refused naming the keys. Overrides of a dropped slot or an
undecodable value are ignored with a warning and reported as compatible drift.
The store writes cultcache.store.v2 only when it holds a variant and reads v1 and
v2. The codec pair CultInspectorModel took becomes CultCodec, shared with the
cache, which needs it to clone documents and overlay override values. The
directory store, CultMesh single-file reads and CultNet row building refuse a
variant by name until their own cuts.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 03:51:53 +02:00
|
|
|
(type, document, bytes) => type == document
|
|
|
|
|
? CultDocumentMessagePackSerialization.DeserializeUntyped(type, bytes, registry)
|
2026-10-14 21:52:36 +02:00
|
|
|
: global::MessagePack.MessagePackSerializer.Deserialize(type, bytes, CultDocumentMessagePackSerialization.OptionsForDocument(document)),
|
C# core: variant records, top-level Set overrides, resolution in the admitting hold
A variant record stores [baseKey, overrides[]] in record slot 4 and an empty
payload; the cache resolves it against its base inside the hold that admits it or
any record on its base chain, and re-resolves dependents in the same Apply, so a
base change reaches its variants with contiguous sequences and no stamp. Plain
Upsert at a variant key is refused (Flatten is the explicit way); cycle, missing
base, different concrete type, evicted base with survivors and a name that
shadows the base are refused naming the keys. Overrides of a dropped slot or an
undecodable value are ignored with a warning and reported as compatible drift.
The store writes cultcache.store.v2 only when it holds a variant and reads v1 and
v2. The codec pair CultInspectorModel took becomes CultCodec, shared with the
cache, which needs it to clone documents and overlay override values. The
directory store, CultMesh single-file reads and CultNet row building refuse a
variant by name until their own cuts.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 03:51:53 +02:00
|
|
|
CultDocumentMessagePackSerialization.OverlaySlots);
|
|
|
|
|
|
|
|
|
|
// The inspection model over that codec.
|
|
|
|
|
public static CultInspectorModel CreateInspectorModel(CultDocumentRegistry registry) =>
|
|
|
|
|
new CultInspectorModel(registry, CreateCodec(registry));
|
2026-09-14 14:03:00 +02:00
|
|
|
|
Route each document type to one home store; attach hydrates; commit explicit batches
Cut 3 of the CultCache migration (Aetheria docs/cultcache-migration-cut.md, 2.1
and 2.2). One commit, because routing, the batch that replaces the ambient
transaction, and conditional commit share one admission path.
Deleted first:
- globals: InitializeGlobals, MaterializeMissingGlobals, the two-argument
constructor, the initializeGlobals plumbing in CultCacheMessagePack,
ContainsDurableRecord (cache and directory store)
- replication: the push-to-every-store loops and the attach-time push of
existing records
- typeof(T) as the schema owner in AddAsync/UpsertAsync and in
CultNetDatabase.PutAsync/PutPredictedAsync
- the ambient transaction: ExecuteTransactionAsync x2, CultCacheTransaction,
CommitTransaction, VisibleStoredDocuments, the AsyncLocal, the SemaphoreSlim
gate, and every ambient check
- CultCacheOpenOptions.PullOnOpen; OpenAsync is Task.FromResult(Create(...))
- DirectoryMessagePackBackingStore.ResolveLegacyUncataloguedRecordCatalog: the
catalog precommit (fe55bff) is an ancestor of the v4 page format (eea8248),
so no v4 manifest can reach it
- CultCache.DeleteAsync<T>; its one caller, CultNetDatabaseSubscriptionClient,
called it by reflection and now calls Remove(key)
Added:
- AddBackingStore(store, params Type[] homes): attach validates (clean, not
already attached, one untyped store, no claimed home type, no admitted
record changes home) and then pulls under the gate
- Home(type): most derived routed home, else the untyped store
- Admit: every add and remove, from a write, a batch or a load, validates
(home, read-only, loaded-from-home, global singleton), lands durably, then
applies and publishes; a pull hands its whole delta over in one call, so a
refused record leaves the store and the cache unchanged
- read-only stores (CacheBackingStore(readOnly), both store constructors,
CultCacheOpenOptions.ReadOnly); flush skips them and writes only dirty stores
- assignable GetGlobal<T>/GetByName<T>/GetByIndex<T>/Watch<T>; more than one
candidate throws naming them
- Commit(Action<CultCacheBatch>) returning bool, TryCommit returning
CultCommitOutcome, CultCacheBatch (Upsert, Remove, Expect, ExpectUnchanged,
sealed after commit), CultCommitRequest; CacheBackingStore.CommitBatch
evaluates conditions under the store's exclusive lock
- single-file store: <path>.lock sidecar for CommitBatch and PushAll; a clean
store commits onto the file as it is, so records it never loaded survive
- storedAt minted in one place and bumped a tick when not later than the record
it replaces
Consumer adjustments in this repo: Mesh streaming tests drop PullOnOpen; the
Studio bridge stops reflecting on it; BackingStoreTests' global test creates
its global explicitly and its transaction tests are rewritten against Commit
(the two that proved ambient hydration rules are deleted); the pre-Cut 2
refusal test expects the attach to throw.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 17:57:58 +02:00
|
|
|
// Opening hydrates. A consumer that wants a fresh store removes the records it does not keep, upserts, and
|
|
|
|
|
// flushes; the atomic replace does the rest.
|
2026-05-09 13:32:30 +01:00
|
|
|
public static CultCache Create(string filePath, CultCacheOpenOptions? options = null)
|
|
|
|
|
{
|
|
|
|
|
if (string.IsNullOrWhiteSpace(filePath))
|
|
|
|
|
{
|
|
|
|
|
throw new ArgumentException("File path must be non-empty.", nameof(filePath));
|
|
|
|
|
}
|
|
|
|
|
|
Route each document type to one home store; attach hydrates; commit explicit batches
Cut 3 of the CultCache migration (Aetheria docs/cultcache-migration-cut.md, 2.1
and 2.2). One commit, because routing, the batch that replaces the ambient
transaction, and conditional commit share one admission path.
Deleted first:
- globals: InitializeGlobals, MaterializeMissingGlobals, the two-argument
constructor, the initializeGlobals plumbing in CultCacheMessagePack,
ContainsDurableRecord (cache and directory store)
- replication: the push-to-every-store loops and the attach-time push of
existing records
- typeof(T) as the schema owner in AddAsync/UpsertAsync and in
CultNetDatabase.PutAsync/PutPredictedAsync
- the ambient transaction: ExecuteTransactionAsync x2, CultCacheTransaction,
CommitTransaction, VisibleStoredDocuments, the AsyncLocal, the SemaphoreSlim
gate, and every ambient check
- CultCacheOpenOptions.PullOnOpen; OpenAsync is Task.FromResult(Create(...))
- DirectoryMessagePackBackingStore.ResolveLegacyUncataloguedRecordCatalog: the
catalog precommit (fe55bff) is an ancestor of the v4 page format (eea8248),
so no v4 manifest can reach it
- CultCache.DeleteAsync<T>; its one caller, CultNetDatabaseSubscriptionClient,
called it by reflection and now calls Remove(key)
Added:
- AddBackingStore(store, params Type[] homes): attach validates (clean, not
already attached, one untyped store, no claimed home type, no admitted
record changes home) and then pulls under the gate
- Home(type): most derived routed home, else the untyped store
- Admit: every add and remove, from a write, a batch or a load, validates
(home, read-only, loaded-from-home, global singleton), lands durably, then
applies and publishes; a pull hands its whole delta over in one call, so a
refused record leaves the store and the cache unchanged
- read-only stores (CacheBackingStore(readOnly), both store constructors,
CultCacheOpenOptions.ReadOnly); flush skips them and writes only dirty stores
- assignable GetGlobal<T>/GetByName<T>/GetByIndex<T>/Watch<T>; more than one
candidate throws naming them
- Commit(Action<CultCacheBatch>) returning bool, TryCommit returning
CultCommitOutcome, CultCacheBatch (Upsert, Remove, Expect, ExpectUnchanged,
sealed after commit), CultCommitRequest; CacheBackingStore.CommitBatch
evaluates conditions under the store's exclusive lock
- single-file store: <path>.lock sidecar for CommitBatch and PushAll; a clean
store commits onto the file as it is, so records it never loaded survive
- storedAt minted in one place and bumped a tick when not later than the record
it replaces
Consumer adjustments in this repo: Mesh streaming tests drop PullOnOpen; the
Studio bridge stops reflecting on it; BackingStoreTests' global test creates
its global explicitly and its transaction tests are rewritten against Commit
(the two that proved ambient hydration rules are deleted); the pre-Cut 2
refusal test expects the attach to throw.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 17:57:58 +02:00
|
|
|
options ??= new CultCacheOpenOptions();
|
C# core: variant records, top-level Set overrides, resolution in the admitting hold
A variant record stores [baseKey, overrides[]] in record slot 4 and an empty
payload; the cache resolves it against its base inside the hold that admits it or
any record on its base chain, and re-resolves dependents in the same Apply, so a
base change reaches its variants with contiguous sequences and no stamp. Plain
Upsert at a variant key is refused (Flatten is the explicit way); cycle, missing
base, different concrete type, evicted base with survivors and a name that
shadows the base are refused naming the keys. Overrides of a dropped slot or an
undecodable value are ignored with a warning and reported as compatible drift.
The store writes cultcache.store.v2 only when it holds a variant and reads v1 and
v2. The codec pair CultInspectorModel took becomes CultCodec, shared with the
cache, which needs it to clone documents and overlay override values. The
directory store, CultMesh single-file reads and CultNet row building refuse a
variant by name until their own cuts.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 03:51:53 +02:00
|
|
|
var registry = options.Registry ?? CultDocumentRegistry.Shared;
|
|
|
|
|
var cache = new CultCache(registry, CreateCodec(registry))
|
2026-05-09 13:32:30 +01:00
|
|
|
{
|
|
|
|
|
FlushAttachedStoresOnDispose = options.FlushOnDispose
|
|
|
|
|
};
|
|
|
|
|
|
Route each document type to one home store; attach hydrates; commit explicit batches
Cut 3 of the CultCache migration (Aetheria docs/cultcache-migration-cut.md, 2.1
and 2.2). One commit, because routing, the batch that replaces the ambient
transaction, and conditional commit share one admission path.
Deleted first:
- globals: InitializeGlobals, MaterializeMissingGlobals, the two-argument
constructor, the initializeGlobals plumbing in CultCacheMessagePack,
ContainsDurableRecord (cache and directory store)
- replication: the push-to-every-store loops and the attach-time push of
existing records
- typeof(T) as the schema owner in AddAsync/UpsertAsync and in
CultNetDatabase.PutAsync/PutPredictedAsync
- the ambient transaction: ExecuteTransactionAsync x2, CultCacheTransaction,
CommitTransaction, VisibleStoredDocuments, the AsyncLocal, the SemaphoreSlim
gate, and every ambient check
- CultCacheOpenOptions.PullOnOpen; OpenAsync is Task.FromResult(Create(...))
- DirectoryMessagePackBackingStore.ResolveLegacyUncataloguedRecordCatalog: the
catalog precommit (fe55bff) is an ancestor of the v4 page format (eea8248),
so no v4 manifest can reach it
- CultCache.DeleteAsync<T>; its one caller, CultNetDatabaseSubscriptionClient,
called it by reflection and now calls Remove(key)
Added:
- AddBackingStore(store, params Type[] homes): attach validates (clean, not
already attached, one untyped store, no claimed home type, no admitted
record changes home) and then pulls under the gate
- Home(type): most derived routed home, else the untyped store
- Admit: every add and remove, from a write, a batch or a load, validates
(home, read-only, loaded-from-home, global singleton), lands durably, then
applies and publishes; a pull hands its whole delta over in one call, so a
refused record leaves the store and the cache unchanged
- read-only stores (CacheBackingStore(readOnly), both store constructors,
CultCacheOpenOptions.ReadOnly); flush skips them and writes only dirty stores
- assignable GetGlobal<T>/GetByName<T>/GetByIndex<T>/Watch<T>; more than one
candidate throws naming them
- Commit(Action<CultCacheBatch>) returning bool, TryCommit returning
CultCommitOutcome, CultCacheBatch (Upsert, Remove, Expect, ExpectUnchanged,
sealed after commit), CultCommitRequest; CacheBackingStore.CommitBatch
evaluates conditions under the store's exclusive lock
- single-file store: <path>.lock sidecar for CommitBatch and PushAll; a clean
store commits onto the file as it is, so records it never loaded survive
- storedAt minted in one place and bumped a tick when not later than the record
it replaces
Consumer adjustments in this repo: Mesh streaming tests drop PullOnOpen; the
Studio bridge stops reflecting on it; BackingStoreTests' global test creates
its global explicitly and its transaction tests are rewritten against Commit
(the two that proved ambient hydration rules are deleted); the pre-Cut 2
refusal test expects the attach to throw.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 17:57:58 +02:00
|
|
|
CacheBackingStore store = options.UseDirectoryStore || Directory.Exists(DirectoryMessagePackBackingStore.DefaultRecordDirectoryPath(filePath))
|
|
|
|
|
? new DirectoryMessagePackBackingStore(filePath, readOnly: options.ReadOnly)
|
|
|
|
|
: new SingleFileMessagePackBackingStore(filePath, options.ReadOnly);
|
|
|
|
|
store.FlushOnDispose = options.StoreFlushOnDispose;
|
|
|
|
|
cache.AddBackingStore(store);
|
2026-05-09 13:32:30 +01:00
|
|
|
return cache;
|
|
|
|
|
}
|
Route each document type to one home store; attach hydrates; commit explicit batches
Cut 3 of the CultCache migration (Aetheria docs/cultcache-migration-cut.md, 2.1
and 2.2). One commit, because routing, the batch that replaces the ambient
transaction, and conditional commit share one admission path.
Deleted first:
- globals: InitializeGlobals, MaterializeMissingGlobals, the two-argument
constructor, the initializeGlobals plumbing in CultCacheMessagePack,
ContainsDurableRecord (cache and directory store)
- replication: the push-to-every-store loops and the attach-time push of
existing records
- typeof(T) as the schema owner in AddAsync/UpsertAsync and in
CultNetDatabase.PutAsync/PutPredictedAsync
- the ambient transaction: ExecuteTransactionAsync x2, CultCacheTransaction,
CommitTransaction, VisibleStoredDocuments, the AsyncLocal, the SemaphoreSlim
gate, and every ambient check
- CultCacheOpenOptions.PullOnOpen; OpenAsync is Task.FromResult(Create(...))
- DirectoryMessagePackBackingStore.ResolveLegacyUncataloguedRecordCatalog: the
catalog precommit (fe55bff) is an ancestor of the v4 page format (eea8248),
so no v4 manifest can reach it
- CultCache.DeleteAsync<T>; its one caller, CultNetDatabaseSubscriptionClient,
called it by reflection and now calls Remove(key)
Added:
- AddBackingStore(store, params Type[] homes): attach validates (clean, not
already attached, one untyped store, no claimed home type, no admitted
record changes home) and then pulls under the gate
- Home(type): most derived routed home, else the untyped store
- Admit: every add and remove, from a write, a batch or a load, validates
(home, read-only, loaded-from-home, global singleton), lands durably, then
applies and publishes; a pull hands its whole delta over in one call, so a
refused record leaves the store and the cache unchanged
- read-only stores (CacheBackingStore(readOnly), both store constructors,
CultCacheOpenOptions.ReadOnly); flush skips them and writes only dirty stores
- assignable GetGlobal<T>/GetByName<T>/GetByIndex<T>/Watch<T>; more than one
candidate throws naming them
- Commit(Action<CultCacheBatch>) returning bool, TryCommit returning
CultCommitOutcome, CultCacheBatch (Upsert, Remove, Expect, ExpectUnchanged,
sealed after commit), CultCommitRequest; CacheBackingStore.CommitBatch
evaluates conditions under the store's exclusive lock
- single-file store: <path>.lock sidecar for CommitBatch and PushAll; a clean
store commits onto the file as it is, so records it never loaded survive
- storedAt minted in one place and bumped a tick when not later than the record
it replaces
Consumer adjustments in this repo: Mesh streaming tests drop PullOnOpen; the
Studio bridge stops reflecting on it; BackingStoreTests' global test creates
its global explicitly and its transaction tests are rewritten against Commit
(the two that proved ambient hydration rules are deleted); the pre-Cut 2
refusal test expects the attach to throw.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 17:57:58 +02:00
|
|
|
|
|
|
|
|
public static Task<CultCache> OpenAsync(string filePath, CultCacheOpenOptions? options = null)
|
|
|
|
|
{
|
|
|
|
|
return Task.FromResult(Create(filePath, options));
|
|
|
|
|
}
|
2026-05-09 13:32:30 +01:00
|
|
|
}
|
|
|
|
|
}
|