CultLib/src/GameCult.Caching/README.md

336 lines
12 KiB
Markdown

# GameCult.Caching
`GameCult.Caching` is an attribute-driven typed cache with a canonical
single-file MessagePack store. Domain models stay clean. The cache owns record
keys, schema ids, timestamps, and persistence metadata instead of smearing
that sludge across every class.
## Authoring Model
Cacheable models are plain classes annotated with cache intent.
```csharp
using GameCult.Caching;
using MessagePack;
[CultDocument("gamecult.item_data", "gamecult.item_data.v1")]
public sealed class ItemData
{
[Key(0)]
[CultName]
public string Name = string.Empty;
[Key(1)]
[CultIndex]
public string Sku = string.Empty;
[Key(2)]
public string Category = string.Empty;
[Key(3)]
public int Value;
}
```
Available cache attributes:
- `CultDocument(schemaName, schemaVersion)`
- `CultName`
- `CultIndex(alias?)`: a unique lookup. Within one document type, a non-blank value names one record; a commit that would leave two records holding one is refused, and a store holding such a pair refuses to load. Uniqueness is judged by a `CultCache` over the single-file store; a cacheless `WriteDocument`/`WriteRecord` of such a type is refused (also when the catalog declares an index alias the call cannot classify), and concurrent writers on a directory store are not covered (`unique-index-directory-store`). A value is culture-invariant, and a number is its canonical decimal (`1.0m` and `1.00m`, `0.0` and `-0.0` are one value).
- `CultRole(alias?)`: a selection and reference role. Many records may share a value; it is never a lookup.
- `CultGlobal`
- `CultReference(targetType?, many: false)`
`CultDocumentRegistry` builds every document descriptor by reflection over
these attributes and MessagePack's `[Key]`/`[IgnoreMember]`; it is the only
descriptor authority.
## Main Concepts
### Record keys and handles
`CultCache` gives each stored document a cache-owned `CultRecordKey`. You work
with typed handles instead of forcing storage identity into the domain object.
```csharp
var cache = new CultCache();
var handle = await cache.AddAsync(new ItemData
{
Name = "Potion",
Sku = "POT-1",
Category = "Consumable",
Value = 50
});
var potion = cache.Get<ItemData>(handle.Key);
```
### Name and index lookups
Name and index lookups are driven from attributes, not manual registration.
```csharp
var byName = cache.GetByName<ItemData>("Potion");
var byIndex = cache.GetByIndex<ItemData>("Sku", "POT-1");
```
### Explicit references
References stay explicit in the domain. The cache will resolve them on demand,
but it will not quietly build an entire haunted object graph behind your back.
```csharp
[CultDocument("gamecult.quest_data", "gamecult.quest_data.v1")]
public sealed class QuestData
{
[Key(0)] public string Title = string.Empty;
[Key(1)] [CultReference] public CultRecordRef<ItemData> Reward;
}
```
## Persistence Shape
CultCache persistence is store-shaped, not entry-shaped:
1. store header
2. embedded schema catalog
3. records
Each persisted record carries:
- `key`
- `schemaId`
- `storedAt`
- `payload`
The domain payload stays free of storage metadata.
## Backing Store
CultCache now has one sanctioned persistence format:
- `SingleFileMessagePackBackingStore`
It persists the whole store as:
1. store format version
2. embedded schema catalog
3. persisted records
Each persisted record carries:
- `key`
- `schemaId`
- `storedAt`
- `payload`
## Happy Path
```csharp
using GameCult.Caching;
using GameCult.Caching.MessagePack;
var cache = await CultCacheMessagePack.OpenAsync("Data.msgpack");
var handle = await cache.UpsertAsync(new ItemData
{
Name = "Potion",
Sku = "POT-1",
Category = "Consumable",
Value = 50
});
await cache.FlushAsync();
```
If you want the explicit assembly rite, the lower-level `CultCache` +
`SingleFileMessagePackBackingStore` path is still there. The helper is just the
blessed front door.
## Managed Documents And SoA Storage
Gameplay code keeps authoring normal typed documents. CultCache owns the record
identity, reactive presentation, and CPU-local structure-of-arrays table behind
that POCO surface.
```csharp
[CultDocument("game.player_transform", "game.player_transform.v1")]
public sealed class PlayerTransform
{
[CultName] public string Name = string.Empty;
public float PositionX;
public float PositionY;
public int Health;
}
var player = cache.Document<PlayerTransform>(new CultRecordKey("player:alice"));
await player.ReplaceAsync(new PlayerTransform
{
Name = "Alice",
PositionX = 4,
PositionY = 8,
Health = 100
});
using var subscription = player.Watch().Subscribe(current => Render(current));
player.Value!.Health = 90;
await player.CommitAsync();
var table = cache.Soa<PlayerTransform>();
Span<float> x = table.Column<float>(nameof(PlayerTransform.PositionX)).Span;
Span<float> y = table.Column<float>(nameof(PlayerTransform.PositionY)).Span;
Span<int> health = table.Column<int>(nameof(PlayerTransform.Health)).Span;
for (var row = 0; row < table.Count; row++)
{
x[row] += 0.016f;
}
```
`Document<T>()` is the standard presentation. `Soa<T>()` exposes the
cache-owned physical table for hot loops and diagnostics; it is not a second
domain model callers have to maintain.
## Element ids
A document type that holds a list of objects gives every object type in that list
(union arms included) a string member marked `[CultElementId]`, next to its
`[Key]`. The id is what lets a variant override one list element instead of the
whole list. Lists of strings, numbers and other non-object values need none.
The rule covers the shapes the id walk owns: arrays, `IEnumerable<T>` classes and
interfaces, dictionary values, and integer-keyed `[MessagePackObject]` and `[Union]`
types. Any other holder is one whole value: `ImmutableArray<T>`, `ArraySegment<T>`,
`Memory<T>`, `KeyValuePair`, tuples, `Lazy<T>`, a class without `[MessagePackObject]`,
a `[DataContract]` or string-keyed type, and a type or member declaring
`[MessagePackFormatter]`. Its elements carry no element ids: a `[CultElementId]`
member inside it is a plain string, never minted or checked, and a variant can only
replace the holder whole.
```csharp
[MessagePackObject]
public sealed class Line
{
[Key(0)] [CultElementId] public string Id = string.Empty;
[Key(1)] public string Text = string.Empty;
}
```
Leave `Id` empty and the cache mints a random id when the record is written. An
id you set yourself must be exactly 12 lowercase hex characters
(`0123456789ab`); anything else is refused with a `CultElementIdException`.
For readable ids, derive them: `[CultElementId(nameof(Name))]` makes an unset id
the text of that member (a string or number on the same type, unique within its
list), and a derived id may be any text. A readable name set by hand on a
non-derived list id is refused; the same member outside a list is unconstrained.
### A type that already has a readable id
If the element type already has a readable string id (`Name`, `Slug`, a catalog
key), leave it as it is. Add a trailing keyed string member marked
`[CultElementId(nameof(ExistingId))]`; the element id is derived from the
existing member on load or first write, and then persisted:
```csharp
[MessagePackObject]
public sealed class Recipe
{
[Key(0)] public string Name = string.Empty; // existing readable id
[Key(1)] public int Yield;
[Key(2)] [CultElementId(nameof(Name))] public string ElementId = string.Empty; // new, last
}
```
Marking `Name` itself `[CultElementId]` is refused (`Name` is not 12 lowercase
hex), and that is deliberate:
- Minted and hand-written ids share one grammar, 12 lowercase hex, so they can
never collide; a derived id is the one readable form and is checked for
uniqueness within its list.
- An id the consumer can edit is not a stable override address. Variants address
an element by its id, so a field that is renamed in place would silently move
the overrides to a different element, or to none.
- A derived id is persisted once. Renaming `Name` later does not change
`ElementId`, so the element and every override that addresses it stay put.
`ElementId` is the address; `Name` goes back to being ordinary data.
Adopting the recipe on data that already exists: the existing member becomes an
id source, so it must be non-empty and unique within its list. A record with a
duplicate value and an empty one both refuse to load
(`CultElementIdException`), so clean such data first.
Take the next free `[Key]` slot, after the highest key on the type (and above
the subclasses' keys on a union base).
## Store format and rollback
A store that holds an element id is written as `cultcache.store.v3` (a directory
store as `cultcache.store.v5.directory-content-addressed-pages`, `v4` otherwise).
A single-file store that holds a variant but no id is `v2`, and a store with
neither stays `v1`. The mark follows content: a whole-store write that leaves no
element holding an id drops it.
Refusal by name depends on the store kind and the pin. A **directory** store (`v5`) is refused by every Unity pin from `cultlib-unity-v1.0.57` and `caching-unity-v1.2.0` on (the earliest tags of each line whose `DirectoryMessagePackBackingStore` reads only the `v4` stamp: "only ... is readable"). A **single-file** store (`v3`) is refused only from `cultlib-unity-v2.0.0` and `caching-unity-v2.0.0`: `cultlib-unity-v1.0.61` and `caching-unity-v1.4.0` read the header string and never compare it. The NuGet line (`cultlib-v1.0.32`, the last `cultlib-v` tag) refuses neither: its directory store writes `cultcache.store.v1.directory` and never checks the stamp on read. Rolling back a single-file store to any of those pins reads it silently, drops the ids, and can rewrite the elements without them. That is the dangerous case.
Rolling a consumer back after it has written a v3 or v5 store is therefore
one-way. The only carry-back is narrow and lossy: a whole-store write that leaves
no id anywhere in the store (every record rewritten without list elements) writes
the store unmarked again, and the ids are gone. Otherwise restore from a copy
taken before upgrading. Upgrade every reader before any writer.
## Persistence discipline
Persistence stays manual by default. Mutations mark the cache and attached
backing stores dirty; nothing silently flushes every object write just because
that felt convenient in the moment.
Useful surfaces:
- `CultCacheMessagePack.OpenAsync(path)`
- `cache.IsDirty`
- `store.IsDirty`
- `cache.Document<T>(key)`
- `cache.Soa<T>()`
- `cache.UpsertAsync(document)`
- `cache.FlushAllBackingStores()`
- `cache.FlushAsync()`
- `cache.FlushBackingStore(store)`
- `cache.PrepareForReloadOrShutdown()`
- `cache.PrepareForReloadOrShutdownAsync()`
- `cache.FlushAttachedStoresOnDispose`
- `store.FlushOnDispose`
This is built for callers like Aquarium that want durable runtime state while
still choosing *when* to pay the write cost.
A single-file store admits any number of writers, in one process or many, one at
a time. Every write (flush, `PushAll`, `CommitBatch`, cacheless `WriteRecord`)
takes an exclusive advisory lock on the sidecar `<path>.lock`, reads the file as
it is then, applies its own staged keys onto it and replaces the file by a
write-to-temp-then-rename. Two writers never interleave bytes, and a record
another writer put there survives; a key both staged holds the last write
(`Contracts/cultcache-store-composition.md`, "Locking"). What the lock does not
give is cross-key or cross-store atomicity, a lock that binds a program that
ignores it, or protection on a filesystem without working `flock`. A second
writer waits (30 seconds, then `IOException`; `TryCommit` reports `Contended`
instead of waiting); readers take no lock and always see one complete file. The
operator procedures for a leftover temp file and for backing up two stores are
in the same contract document, "Operating a single-file store".
## Schema migration diagnostics
CultCache does not quietly fall back from exact `schemaId` resolution to
"close enough, probably." When a persisted schema drifts, the cache now emits a
typed migration report describing:
- exact vs compatible-drift resolution
- ignored extra slots
- defaulted missing slots
- warning codes/messages
Compatibility rules and canonical fixture receipts live in
`Contracts/cultcache-schema-compatibility.md`.