Document CultNet distributed cache sharding

This commit is contained in:
Meta Vens 2026-05-19 10:27:51 +01:00
commit 8ca975a69b
2 changed files with 112 additions and 0 deletions

View file

@ -412,6 +412,71 @@ should mean the same thing across persistence and wire discovery.
Where CultNet needs additional wire-oriented metadata, extend around the shared
core instead of redefining the core.
## CultNet As A Sharding Layer
CultNet should be able to initialize directly as a CultCache distribution layer.
The goal is a single distributed realtime database surface: clients work with
typed CultCache documents, while CultNet decides which node owns the relevant
shard and how committed changes reach subscribers.
This is closer to the application experience of RethinkDB or Firebase than a
traditional request/response game protocol. CultNet should still keep its own
shape: explicit schemas, MessagePack payloads, local-first operation where
useful, and honest rejection when compatibility or authority is unclear.
The ownership split is:
- CultCache owns document identity, schema compatibility, local indexing,
persistence, and domain-level change diffing.
- CultNet owns peer membership, shard routing, mutation transport,
authorization, remote subscription fanout, and resynchronization after
reconnect.
- R3 owns C# stream composition for observed domain changes.
A distributed CultCache cluster should expose:
- shard descriptors: which schema/key ranges or named partitions a node owns
- shard epochs: monotonic authority versions for ownership changes
- mutation messages: raw document put/delete records with source generation
and shard epoch
- snapshot messages: bounded catch-up from a known generation or empty local
state
- subscription messages: schema/key/index/global queries that stream matching
domain changes
- conflict reports: explicit outcomes when a mutation cannot be applied cleanly
Client-facing APIs should preserve the database illusion:
```csharp
var db = await CultNetDatabase.ConnectAsync("localhost", 3075);
var player = await db.GetAsync<PlayerData>(playerKey);
await db.PutAsync(playerKey, player);
using var subscription = db
.WatchByIndex<PlayerData>("Region", "eu-west")
.Subscribe(change => Render(change.Document));
```
The illusion has limits and those limits should be visible in the contract.
CultNet is not allowed to pretend that arbitrary split-brain writes merged
cleanly. Each shard needs one current write authority, or a declared conflict
strategy that can be explained without ritual smoke. If neither exists, the
write is rejected and surfaced as a domain-visible failure.
The simplest coherent first implementation is primary-shard ownership:
1. Each shard has one authoritative writer at a time.
2. Followers subscribe to that shard's committed mutation stream.
3. Clients may connect to any node; non-owner nodes forward writes to the shard
owner or reject when forwarding is unavailable.
4. Reconnect performs schema discovery, shard epoch comparison, then snapshot
catch-up before live events resume.
More advanced replicated or offline-first conflict policies can come later, but
they must be explicit shard policies rather than accidental behavior leaking out
of transport retries.
## Migration From Current C# Shape
The current C# `DatabaseEntry` base type exposes `ID` directly. That is useful

View file

@ -92,6 +92,53 @@ same payload schema and MessagePack semantics. It carries exact payload bytes
plus `schemaId`/record-key metadata; it does not guess what a blob "probably"
means.
## Distributed CultCache
CultNet should be able to start as a sharding layer over CultCache. In that
mode, applications treat the cluster as one typed realtime database instead of
manually juggling a local cache, a network client, and a synchronization loop.
Target feel:
```csharp
var db = await CultNetDatabase.ConnectAsync("localhost", 3075);
var player = await db.GetAsync<PlayerData>(playerKey);
await db.PutAsync(playerKey, player);
using var subscription = db
.WatchByIndex<PlayerData>("Region", "eu-west")
.Subscribe(change => Render(change.Document));
```
CultCache remains the document model:
- schema identity and compatibility
- record keys and handles
- local indexes, globals, and typed lookups
- persistence and local domain-change diffing
CultNet becomes the distribution model:
- peer membership
- shard ownership
- mutation routing
- snapshot catch-up
- remote subscription fanout
- reconnect resynchronization
The first coherent shard policy is primary ownership. Each shard has one
authoritative writer at a time. Clients may connect to any node; a non-owner
node forwards a write to the owner or rejects it honestly when forwarding is
not available. Followers subscribe to committed shard mutations and apply them
through the same CultCache reconciliation path as local file changes.
This should feel like the local-first parts of Firebase or RethinkDB without
copying their hidden machinery wholesale. The hard line is conflict honesty:
split-brain writes, stale shard epochs, and schema-incompatible documents must
surface as explicit failures instead of being laundered into cheerful-looking
state.
## Authentication Model
The built-in flow is: