Watch

Meta Vens a1f25af2e0 Keep exact live subscriptions active before publication

2026-07-19 11:49:18 +01:00

17 KiB

Raw Blame History

CultMesh

CultMesh is the distributed realtime database and simulation-consensus layer for GameCult projects. The NuGet package and assembly are named GameCult.Mesh; the public surface is branded as CultMesh.

CultMesh sits over:

GameCult.Caching for typed domain documents, local indexes, persistence, and file-compatible cache storage.
GameCult.Networking for CultNet schema-v0 wire messages, native RUDP, transport adapters, and secure client/server sessions.

The intent is direct: a game can treat a mesh of clients and servers as one reactive database for persistent state, prediction-friendly input state, and simulation observations.

First Node

using GameCult.Mesh;

using var node = await CultMesh.StartNodeAsync("world.ccmp");

var player = await node.Database.GetAsync<PlayerData>(playerKey);
await node.Database.PutAsync(playerKey, player);

If you want one inspectable path that starts with a durable local cache file and ends with a typed read/write through the public CultMesh surface, see docs/durable-node-quickstart.md. If you want the lower-level typed-to-raw handoff that CultNet moves across the wire, see docs/typed-document-path.md.

Eve / CultUI Surfaces

CultMesh includes the portable Eve/CultUI surface document contract and a fluent builder for daemon-published UI. The builder is deliberately shaped like the legacy CultUI/Unity properties panel API: compose menus and forms in readable imperative code, while the emitted document carries typed CultMesh operation bindings, state bindings, style tokens, and embedded surface slots.

var menu = EveSurface.Create("aetheria.main_menu.root")
    .Provider("aetheria", "game.menu")
    .TitleSubtitle("AETHERIA", "TERMINUS")
    .ButtonColumn("aetheria.main_menu.root.actions", actions => actions
        .Button("Continue", "aetheria.main_menu.root.continue")
        .Button("New Game", "aetheria.main_menu.root.new_game")
        .Button("Settings", "aetheria.main_menu.root.show_settings")
        .Button("Quit", "aetheria.main_menu.root.quit"))
    .Style("font.title.family", "Montserrat")
    .Style("font.body.family", "Ubuntu")
    .Build();

Forms accept typed CultMesh descriptors at authoring time and flatten them into MessagePack-safe Eve records in the document:

var setName = CultMesh.OperationBinding(
    "settings.player.name.set",
    "Set Player Name",
    "gamecult.settings.name.v1");

var playerName = new CultMeshStateBindingDescriptor(
    "value",
    "player.settings.name",
    "global:aetheria.player_settings.v1",
    "gamecult.aetheria.player_settings.v1");

var settings = EveSurface.Create("aetheria.main_menu.player_settings")
    .Form("aetheria.main_menu.player_settings.form", form => form
        .Text("Name", "Meta", setName, playerName))
    .Build();

Nested UI is expressed with surface.slot components and EveEmbeddedDocumentSlot records. A runtime lowers the parent surface, resolves the child document through CultMesh, and renders the embedded document without the renderer becoming the owner of that UI.

Enable durable authoritative shard logs when a node should serve replica catch-up after restart:

using var node = await CultMesh.StartNodeAsync("world.ccmp", new CultMeshNodeOptions
{
    EnableDurableShardLogs = true
});

If ShardLogPath is omitted, CultMesh stores logs beside the cache under world.cultmesh/shard-logs.

Distributed CDN

CultMesh can distribute signed, content-addressed game assets through the same typed document lane as gameplay state. Asset manifests, chunks, and portable entity prefab packages are CultCache documents that can be replicated by CultNet snapshots, shard logs, central servers, LAN peers, or opted-out-capable internet peer caches.

This is the canonical asset pipeline target for Aetheria. Brokkr-authored Blender prefab snapshots deploy into CultMeshEntityPrefabPackage documents that reference CDN artifact manifests for meshes, textures, materials, and other payloads. Unity and TypeScript clients should unpack the same package graph rather than maintaining separate Unity-prefab and Electron asset formats.

See docs/distributed-cdn.md for document shapes, publish/read helpers, peer asset sharing policy, and the Brokkr prefab flow.

Verses

A Verse is a rule-bearing consensus graph. Aetheria Main can be a cloud-authoritative Verse backed by GameCult regional simulators. A modded community branch can be transport-compatible with Aetheria Main while declaring different rules, required plugins, and authority policy. A peer-hosted game can choose a peer-to-peer authority model.

var rulesHash = CultMeshVerseDescriptor.ComputeRulesHash(
    "aetheria",
    "rules:v1",
    "plugins:vanilla");

var aetheria = new CultMeshVerseDescriptor(
    "aetheria-main",
    "Aetheria",
    CultMeshVerseAuthorityModel.OperatorCluster,
    new CultMeshVerseCompatibility("cultmesh.v0", rulesHash),
    discoveryEndpoints: ["cultmesh://us-east.aetheria.gamecult.net:3075"]);

using var catalog = CultMesh.CreateVerseCatalog();
catalog.Upsert(aetheria);

using var discovery = CultMesh.ServeVerseCatalog(node, catalog);

Peers can request Verse catalogs over schema-v0 with cultmesh.verse_catalog_request.v0 and receive transport-filtered cultmesh.verse_catalog_response.v0 descriptors.

Peer Exchange

Verse discovery tells a node which consensus graph exists. Peer exchange tells it who else is already in that graph.

using var peers = CultMesh.CreatePeerCatalog();
peers.Upsert(new CultMeshPeerCard(
    "gc-us-east-1",
    "aetheria-main",
    ["cultnet://us-east.aetheria.gamecult.net:3075"],
    roles: [CultMeshPeerRoles.Discovery, CultMeshPeerRoles.ShardPrimary],
    shardIds: ["players-us-east"],
    authorityLeaseId: "lease:players-us-east:42"));

using var exchange = CultMesh.ServePeerExchange(node, peers);

Peer cards are contact candidates, not authority by themselves. Public Verses should validate authority leases or signatures before trusting a peer for committed state.

var leases = CultMesh.CreateAuthorityLeaseCatalog();
leases.Upsert(new CultMeshAuthorityLease(
    "lease:players-us-east:42",
    "aetheria-main",
    "gc-us-east-1",
    [CultMeshPeerRoles.ShardPrimary],
    ["players-us-east"],
    "gamecult-operator",
    DateTimeOffset.UtcNow.AddMinutes(-1),
    DateTimeOffset.UtcNow.AddMinutes(10),
    signature: "operator-signature"));

CultMeshPeerCatalog.FindAuthorized(...) and FirstAuthorized(...) compose peer lookup with the lease catalog, so contact gossip stays separate from trust. The branded RUDP helpers use the same boundary:

using var rudpClient = CultMesh.ConnectRudpClientForAuthorizedPeer(
    "aetheria-client",
    connectionId: 0x10203040,
    peers,
    leases,
    "aetheria-main",
    CultMeshPeerRoles.ShardPrimary,
    shardId: "players-us-east");

CreateRudpServer(...), ParseRudpEndpoint(...), ConnectRudpClient(...), and ConnectRudpClientForPeer(...) delegate to the CultNet RUDP socket transport and return the same schema-message-capable transport after handshake. CreateRudpClient(...) and CreateRudpClientForPeer(...) remain available when a caller intentionally owns the handshake and polling loop. CultMesh owns peer/authority ergonomics; CultNet owns packet reliability, resend pressure, and channel semantics.

Clients can fetch more peers from known peers:

var exchangeClient = CultMesh.CreatePeerExchangeClient();
await exchangeClient.DiscoverAsync(
    peers,
    aetheria.DiscoveryEndpoints,
    "aetheria-main",
    roles: [CultMeshPeerRoles.ReadReplica]);

Client-Side Prediction

CultMesh lets a runtime declare which input documents it may author optimistically. The local cache updates immediately, game simulation can proceed against that predicted state, and the authoritative shard log later reconciles the record.

var db = node.Database;
var input = CultMesh.Document<PlayerInput>(db, inputKey, verse);

await input.SubmitPredictionAsync(inputState);

using var sub = input.Watch()
    .Subscribe(latest =>
    {
        ReconcilePrediction(latest);
    });

Read-only CultCache publications can be lifted into the same document handle shape without local store plumbing:

var health = CultMesh.DocumentFromSingleFile<DaemonHealth>(
    "daemon-publication.ccmp",
    new CultRecordKey("daemon:aetheria.health.v1"),
    verse);

using var healthSub = health.Watch().Subscribe(RenderHealth);
var latestHealth = await health.LatestAsync();

DocumentFromStore<T>() accepts any CacheBackingStore; both helpers pull, decode, resolve persisted schema metadata, and poll for updates behind the CultMeshDocumentHandle<T> surface. Use DocumentFromPublication<T>() when the caller should provide a configured source instead of branching between local files, backing stores, and remote CultNet snapshots itself.

When the remote snapshot should hydrate a local node, use the direct sync facade and keep the rest of the code on local typed handles:

var syncedHealth = await CultMesh.SyncDocumentFromPeerSnapshotAsync<DaemonHealthUi>(
    node,
    "cultnet://daemon.local:3075",
    "daemon:aetheria.health.v1",
    new CultMeshSnapshotEndpointOptions { DocumentRegistry = daemonRegistry });

var localHealth = await node
    .Document<DaemonHealthUi>("daemon:aetheria.health.v1", verse)
    .LatestAsync();

Game Session

CultMeshGameSession wires the common gameplay loop: local catalogs, observation hub, observation server, peer exchange, Verse discovery, prediction, and quorum fact commits.

using var session = CultMesh.CreateGameSession(node, new CultMeshGameSessionOptions
{
    ConsensusOptions = new CultNetSimulationConsensusOptions
    {
        MinimumWitnesses = 2,
        QuorumRatio = 0.66d
    }
});

await session.PredictAsync(inputKey, inputState);

var committed = await session.SubmitAndCommitAsync(new CultNetSimulationObservation
{
    WitnessRuntimeId = "watcher-1",
    ShardId = "arena",
    ShardEpoch = 4,
    Frame = 100,
    SubjectId = "bob",
    ClaimKind = "hit",
    ClaimHash = CultNetSimulationObservation.ComputeClaimHash("hit", "alice", "bob", "frame:100")
});

Managed Documents

CultMesh presents distributed state as the same managed POCO document shape as CultCache. Local edits commit through shard authority; remote shard-log or network events update the same watched value.

using GameCult.Caching;
using GameCult.Mesh;

var player = node.Database.Document<PlayerTransform>(new CultRecordKey("player:alice"));

using var subscription = player.Watch().Subscribe(current => Render(current));

await player.ReplaceAsync(new PlayerTransform
{
    Name = "Alice",
    PositionX = 4,
    Health = 100
});

player.Value!.Health = 90;
await player.CommitAsync();

Remote publications use the same document handle shape:

var health = CultMesh.DocumentFromPeerSnapshot<DaemonHealthDocument>(
    "cultnet://daemon.local:3075",
    "daemon:aetheria.health.v1",
    verse);

var latest = await health.LatestAsync();

The physical cache table remains SoA for CPU-local scans:

Span<float> x = node.Cache.Soa<PlayerTransform>()
    .Column<float>(nameof(PlayerTransform.PositionX))
    .Span;

Witness Consensus

CultMesh nodes can publish observations about simulation facts: shard epoch, frame, subject, claim kind, and claim hash. CultNetSimulationConsensus aggregates those observations deterministically into candidate facts with support weight, total observed weight, confidence, and quorum status.

Those candidates are not committed world state by themselves. They are the fast mesh opinion layer that can feed an authoritative shard commit path. Once a candidate has quorum, a shard-authoritative node can commit it as a CultMeshSimulationFact.

var committer = CultMesh.CreateSimulationFactCommitter(node.Database);
await committer.CommitAsync(candidate);

The central server does not need to decide what every witness saw. It only needs to accept the quorum result through the shard log.

Durable Catch-Up

CultMesh can persist authoritative shard logs through ICultNetShardMutationLogStore. The included file store writes one MessagePack log per shard, so a restarted primary can still serve replica catch-up without depending on warm process memory. When retained log history is compacted, stale replicas get an explicit resync requirement instead of a partial log pretending to be enough. The replicator can answer that by fetching a shard-bounded snapshot, replacing the local shard view, and advancing its cursor to the snapshot's represented log sequence.

Streaming Mode

CultMesh streaming mode is the low-latency body path for Mimir, Fensalir, Eve, and other realtime runtimes. CultCache and CultNet own typed state, discovery, stream descriptors, negotiations, and frame handles. The frame bodies stay in runtime-native storage: shared D3D12 textures, shared memory rings, platform GPU handles, DMA buffers, or paged CultCache fallbacks when a zero-copy transport is not available.

For entity SoA and other CPU-visible hot state, subscribe to the exact control records and logical bodies the lowerer consumes. The subscription advertises the transports the runtime can import; the provider's demand tracker selects the cheapest mutually usable representation. A same-machine consumer normally receives one shared-memory bootstrap descriptor, then observes later commits through the mapped ring header. Per-frame CultNet documents are neither needed nor desirable.

using var entities = await CultMesh.SubscribeLiveBodyAsync(
    subscriptions,
    bodyResolver,
    new CultMeshLiveBodySubscription(
        "pilot-entities",
        "eve-unity",
        "eve:entity-soa:aetheria.daemon:pilot"));

using var input = await subscriptions.SubscribeLiveValueAsync<EveInputCapabilityDocument>(
    "pilot-input",
    "eve:input:aetheria.daemon:pilot");

// Open after each descriptor change. CultMesh chooses shared memory, shared-file
// mapping, or network from observed locality and the resolver's real adapters.
if (entities.HasValue && input.HasValue)
{
    using var frame = entities.OpenCurrentReadOnly().Lease;
    Present(frame, input.Current);
}

CultMeshLiveBody watches only the small latest-publication record and declares the logical body demand. It never routes the body through the subscription snapshot. CultNetLiveValue<T> does the corresponding job for exact scalar or small structured state without creating a renderer-owned replica cache. Both subscriptions may begin before the provider's first publication: HasValue remains false while the retained demand stays active, then the first typed value arrives through the normal reactive change path.

Snapshots remain bootstrap and recovery machinery. Reactive document changes carry small control state; mapped or negotiated network bodies carry hot data.

using GameCult.Mesh;

var streams = CultMesh.CreateStreamCatalog();

streams.Declare(new CultMeshStreamDescriptor(
    "mimir:kiyo-pro:rgba",
    "mimir-live",
    "starfire",
    CultMeshStreamKind.Video,
    new CultMeshStreamClock("mimir:clock", "kiyo-pro", sampleRate: 90_000, confidence: 0.92),
    new[]
    {
        CultMeshStreamBodyTransport.SharedD3D12Texture,
        CultMeshStreamBodyTransport.SharedMemory,
        CultMeshStreamBodyTransport.CultCachePage
    },
    video: new CultMeshVideoStreamFormat(1920, 1080, "rgba8", framesPerSecond: 60),
    maxInFlightFrames: 4));

var fensalir = new CultMeshStreamConsumerProfile(
    "fensalir",
    "mimir-live",
    new[] { CultMeshStreamBodyTransport.SharedD3D12Texture },
    acceptedKinds: new[] { CultMeshStreamKind.Video },
    canImportGpuHandles: true);

var negotiation = streams.Negotiate("mimir:kiyo-pro:rgba", fensalir);

For CPU-visible data, CultMeshSharedMemoryFrameRing preallocates fixed slots and hands producers writable leases. Consumers hold read leases so the producer does not overwrite a visible frame. TryPublishCopy exists for unavoidable fallbacks and increments copy telemetry; the normal hot path writes directly into a lease and commits a CultMeshStreamFrameHandle.

streams.Declare(new CultMeshStreamDescriptor(
    "mimir:leap:depth",
    "mimir-live",
    "starfire",
    CultMeshStreamKind.Tensor,
    new CultMeshStreamClock("mimir:clock", "leap"),
    new[] { CultMeshStreamBodyTransport.SharedMemory }));

using var ring = streams.CreateSharedMemoryRing("mimir:leap:depth", slotCount: 4, slotByteLength: 640 * 480 * 2);

if (ring.TryAcquireWriteSlot(out var write))
{
    FillDepthFrame(write.Span);
    var frame = ring.CommitWriteSlot(write, timestampNs: GetVerseTimeNs(), byteLength: 640 * 480 * 2);
    streams.PublishFrame(frame);
}

The invariant is simple: typed mesh state describes and synchronizes streams; hot frame bodies remain in the cheapest storage both endpoints can share.

17 KiB Raw Blame History