CultLib/packages/cultcache-py

cultcache-py

cultcache-py is the Python runtime package for CultCache, CultNet, and CultMesh. Callers work with registered domain documents while the cache owns schema identity, routing, globals, name lookups, indexes, and backing-store envelopes.

It is intentionally small. It is not an ORM, a database, or distributed consensus wearing rented authority.

Current Shape

• document types are registered explicitly
• a registered cache can be used in memory without attaching a backing store
• SingleFileMessagePackBackingStore writes cultcache.store.v1 snapshots: [format, schemaCatalog, records]
• records store raw MessagePack payload bytes under schema-catalog identity
• payloads decode only through the registered document definition for that type
• unknown persisted types fail closed
• global documents are singleton-style per type
• type-specific backing stores beat generic backing stores
• the JSONL store uses only the Python standard library for bootstrap consumers
• define_database_entry_type(...) emits Rust/C#-style slot-indexed MessagePack array payloads for cross-runtime DatabaseEntry contracts
• cultnet_py exposes schema-v0 MessagePack helpers and 4-byte big-endian frame helpers for raw document state, subscriptions, shard catch-up, simulation observations, and witness artifact bundles
• cultmesh_py exposes a cache-backed local node surface for Python tools

Example

from dataclasses import dataclass, asdict
from cultcache_py import CultCache, define_document_type

@dataclass
class Settings:
    theme: str
    retries: int

settings_doc = define_document_type(
    "settings",
    encode=lambda value: asdict(value),
    decode=lambda payload: Settings(**payload),
    global_document=True,
)

cache = (
    CultCache.builder()
    .register_document_type(settings_doc)
    .build()
)

cache.put_global(settings_doc, Settings(theme="ash", retries=3))
settings = cache.get_required_global(settings_doc)

Add JsonLinesBackingStore or SingleFileMessagePackBackingStore when the cache should persist state; local in-memory cache reads and writes do not need a store.

Public Surface

• define_document_type(...)
• define_database_entry_type(...)
• database_entry_field(...)
• define_document_registry(...)
• CultCache.builder()
• register_document_type(...)
• register_registry(...)
• register_name_lookup(...)
• register_index(...)
• add_backing_store(...)
• add_generic_store(...)
• pull_all_backing_stores()
• get(...)
• get_required(...)
• get_all(...)
• get_key_by_name(...)
• get_by_name(...)
• get_key_by_index(...)
• get_by_index(...)
• get_global(...)
• get_required_global(...)
• put(...)
• put_envelopes(...)
• put_global(...)
• update(...)
• update_global(...)
• delete(...)
• delete_global(...)
• snapshot()

Persistence Model

Backing stores persist envelopes, not caller domain objects. The cache decodes payloads through registered document definitions and rejects unknown type discriminators. This keeps Python's dynamic runtime from turning persistence into an open polymorphic sewer with a cheerful docstring.

JsonLinesBackingStore is the dependency-free control-plane store. It rewrites an atomic JSONL snapshot and base64-encodes payload bytes. It is designed for compact state spines, settings, ledgers, and bootstrap surfaces.

SingleFileMessagePackBackingStore follows the shared CultCache v1 store shape used by the TypeScript, Rust, and C# runtimes:

• top-level MessagePack value: [format, schemaCatalog, records]
• format: cultcache.store.v1
• catalog entries: [schemaId, schemaName, schemaVersion, contentHash, canonicalSchemaJson, compatibleSchemaIds, members]
• records: [key, schemaId, storedAt, payload]

Large corpora should use a sharded store or a real database. A single snapshot file is a scalpel, not a forklift.

DatabaseEntry Slot Contract

For cross-runtime cache entries, use define_database_entry_type(...) instead of the generic JSON formatter:

from dataclasses import dataclass
from cultcache_py import define_database_entry_type

@dataclass
class Settings:
    theme: str
    retries: int = 0

settings_doc = define_database_entry_type(
    "settings",
    [
        ("theme", 0),
        ("retries", 1, 0),
    ],
    cls=Settings,
)

The payload is a MessagePack array. Field keys are durable slot indexes:

• key 0 writes array slot 0
• key 4 writes array slot 4
• unused slots are written as nil
• deleted fields should leave their slots reserved
• newly added fields should use new keys
• fields with defaults tolerate missing or nil slots when older payloads are read

That matches the Rust #[derive(DatabaseEntry)] formatter shape and the C# [Key(n)] intent: schema evolution comes from stable field slots, not source member order.

CultNet And CultMesh

cultnet_py provides the Python schema-v0 wire helpers and an interop peer:

from datetime import UTC, datetime

from cultnet_py import (
    CultNetSecret,
    CultNetServerSecurityOptions,
    CultNetRawClient,
    CultNetSimulationObservation,
    CultNetWitnessArtifactBundle,
    apply_raw_snapshot,
    apply_shard_log_response,
    compute_simulation_claim_hash,
    database_subscribe,
    decode_frame,
    encode_frame,
    hello,
    parse_message,
    shard_catalog_request,
    simulation_observation,
    witness_artifact_bundle,
)

payload = hello(runtime_id="python-runtime").to_bytes()
message = parse_message(decode_frame(encode_frame(payload)))

client = CultNetRawClient("127.0.0.1", 3075)
catalog = client.fetch_schema_catalog(kinds=["wire_message"])
snapshot = client.fetch_snapshot_response(schema_ids=["cultnet.interop-note"])
shard_catalog = client.fetch_shard_catalog(schema_ids=["cultnet.interop-note"])
# With a local CultCache and the matching registered document definitions:
# applied = apply_raw_snapshot(cache, [note_doc], snapshot)

subscription = database_subscribe(subscription_id="ui", schema_ids=["cultnet.interop-note"])
shard_request = shard_catalog_request(message_id="shards", schema_ids=["cultnet.interop-note"])
shard_log = client.fetch_shard_log_response(shard_id="interop", shard_epoch=1, after_sequence=0)
# log_changes = apply_shard_log_response(cache, [note_doc], shard_log)
with client.subscribe_database(subscription_id="ui", schema_ids=["cultnet.interop-note"]) as live:
    initial_snapshot = live.read_next_snapshot_response()
claim_hash = compute_simulation_claim_hash("frame:42", "subject:player-1", "hit")
observation = simulation_observation(
    message_id="obs-1",
    witness_runtime_id="python-runtime",
    shard_id="interop",
    shard_epoch=1,
    frame=42,
    subject_id="player-1",
    claim_kind="hit",
    claim_hash=claim_hash,
)
typed_observation = CultNetSimulationObservation.from_wire(observation.to_wire())
witness = witness_artifact_bundle(
    bundle_id="bundle-1",
    witness_kind="interop-proof",
    captured_at="2026-06-13T00:00:00Z",
    subject={"documentType": "cultnet.interop-note", "subjectId": "note:python"},
    contracts=[{"role": "payload", "schemaId": "cultnet.interop-note"}],
    artifacts=[{"role": "log", "uri": "cultcache://bundle-1/log", "mediaType": "text/plain"}],
    provenance={"pipelineId": "interop", "runId": "run-1", "runtimeId": "python-runtime"},
)
typed_witness = CultNetWitnessArtifactBundle.from_wire(witness)
witness_payload = typed_witness.to_payload()

security = CultNetServerSecurityOptions.development()
token = CultNetSecret.create_session_token(
    "318fb4b6-ff5e-4c4f-b911-d81807de53a8",
    datetime(2035, 1, 1, tzinfo=UTC),
    security,
    session_version=1,
)
session = CultNetSecret.try_validate_session_token(token, security)

AES-GCM string/byte encryption helpers are available through the same CultNetSecret surface when the optional crypto extra is installed.

The peer can serve, dial, and probe the same raw-state interop lane used by the TypeScript, Rust, and C# test peers:

python -m cultnet_py.interop_peer serve --runtime-id python-peer --runtime-kind python --display-name "Python Peer" --agent-id python-agent --advertise-host 127.0.0.1 --tcp-port 3075 --discovery-port 4075 --discovery-group 239.77.44.11 --schema-path ..\cultnet-ts\integration\contracts\cultnet.interop-note.schema.json
python -m cultnet_py.interop_peer dial --runtime-id python-client --runtime-kind python --display-name "Python Client" --agent-id python-client --target-host 127.0.0.1 --target-port 3075 --schema-path ..\cultnet-ts\integration\contracts\cultnet.interop-note.schema.json
python -m cultnet_py.interop_peer probe --runtime-id python-prober --discovery-port 4075 --discovery-group 239.77.44.11

cultmesh_py includes a local cache-backed node, schema-v0 helpers for the CultMesh Verse catalog and peer exchange wire messages, local and verified authority lease checks, stream transport negotiation, committed simulation fact documents, and local prediction reconciliation:

from datetime import UTC, datetime, timedelta

from cultcache_py import define_database_entry_type
from cultnet_py import (
    CultNetClientAuthorityScope,
    CultNetFileShardReplicaCursorStore,
    CultNetRawClient,
    CultNetSchemaShardLogFetcher,
    CultNetSchemaShardSnapshotFetcher,
    CultNetShardDescriptor,
    CultNetShardReplicator,
    CultNetShardReplicatorOptions,
)
from cultmesh_py import (
    CultMesh,
    CultMeshAuthorityLease,
    CultMeshGameSessionOptions,
    CultMeshHmacAuthorityLeaseVerifier,
    CultMeshNodeOptions,
    peer_exchange_request,
)

note_doc = define_database_entry_type(
    "mesh.note",
    [("body", 0), ("name", 1), ("kind", 2)],
    name="name",
    indexes={"kind": "kind"},
)
node = CultMesh.start_node(
    "mesh.cc",
    runtime_id="python-runtime",
    options=CultMeshNodeOptions(enable_durable_shard_logs=True),
)
node.database.register_document(note_doc)
unsubscribe = node.database.watch_record(note_doc, "note:1", lambda change: print(change.change_kind))
unsubscribe_named = node.database.watch_by_name(note_doc, "intro", lambda change: print(change.record_key))
unsubscribe_kind = node.database.watch_by_index(note_doc, "kind", "guide", lambda change: print(change.record_key))
node.database.put(note_doc, "note:1", {"body": "hello", "name": "intro", "kind": "guide"})
live_note = node.database.get_required(note_doc, "note:1")
put_message = node.database.put_raw_message(
    note_doc,
    "note:2",
    {"body": "wire me", "name": "wire", "kind": "interop"},
    shard_id="interop",
    shard_epoch=1,
)
snapshot_response = node.database.build_snapshot_response(schema_ids=[note_doc.catalog_entry().schema_id])
snapshot_wire = node.database.create_snapshot_response(schema_ids=[note_doc.catalog_entry().schema_id])
delete_message = node.database.delete_raw_message(note_doc, "note:2", shard_id="interop", shard_epoch=1)
shard_log_response = node.database.build_shard_log_response(shard_id="interop", shard_epoch=1)
shard_log_wire = node.database.create_shard_log_response(shard_id="interop", shard_epoch=1)

peers = CultMesh.create_peer_catalog()
response = peers.create_response(peer_exchange_request("pex-1", verse_id="local"))
lease_verifier = CultMeshHmacAuthorityLeaseVerifier({"odin": b"shared-lease-key"})
lease_catalog = CultMesh.create_authority_lease_catalog(
    signature_verifier=lease_verifier.verify,
    require_verified_signatures=True,
)
lease_valid_from = datetime.now(UTC)
lease = lease_verifier.issue(CultMeshAuthorityLease(
    lease_id="lease:python-runtime",
    verse_id="local",
    peer_id="python-runtime",
    roles=("shard-primary",),
    valid_from=lease_valid_from,
    expires_at=lease_valid_from + timedelta(minutes=5),
    shard_ids=("interop",),
    issuer_runtime_id="odin",
))
lease_catalog.upsert(lease)
session = CultMesh.create_game_session(
    node,
    CultMeshGameSessionOptions(
        client_authority_scopes=(
            CultNetClientAuthorityScope("python-runtime", schema_ids=(note_doc.catalog_entry().schema_id,), key_prefix="input:python"),
        ),
    ),
)
server = CultMesh.serve_node(
    node,
    peer_catalog=peers,
    observation_hub=session.observation_hub,
    port=3075,
    max_snapshot_documents=1000,
    max_snapshot_bytes=4 * 1024 * 1024,
)

The same local server can be launched as a package daemon when another runtime or operator process needs a CultNet endpoint without embedding Python glue:

python -m cultmesh_py.daemon --runtime-id python-runtime --display-name "Python Runtime" --host 127.0.0.1 --port 3075 --cache-file mesh.cc --enable-durable-shard-logs --seed-interop-note --seed-shard-id interop --verse-id local --role shard-primary --ready-file mesh.ready.json

The ready file and stdout line are process readiness hints. The served state and capability truth still live on the framed CultNet/CultMesh endpoint. Use --register-interop-note to expose the package interop-note schema without seeding a note, or --seed-interop-note to register it and publish one local record for wire probes. Seeded records are committed through the raw mutation path, so peers can read them through both snapshot and shard-log catch-up. Add --verse-id and one or more --role values to publish the launched endpoint in its own Verse and peer catalogs for discovery clients. When the daemon is launched with --cache-file, --enable-durable-shard-logs, and --shard-log-file, a later daemon process can rehydrate and serve the persisted snapshot and shard log without reseeding.

CultMeshPeerHealthMonitor probes peer-card endpoints with cultnet.hello.v0 and preserves the runtime id, display name, document types, message versions, and mutation contracts reported by the peer.

client = CultMesh.create_verse_discovery_client("127.0.0.1", 3075)
verses = client.fetch_verses(transport_version="cultmesh.v0")
mesh_peers = client.fetch_peers(verse_id="python-interop", roles=["read-replica"])
client.sync_peer_catalog(peers, verse_id="python-interop", roles=["read-replica"])
client.fanout_peer_catalog(peers, verse_id="python-interop", roles=["read-replica"])

raw_client = CultNetRawClient("127.0.0.1", 3075)
node.database.sync_snapshot(raw_client, schema_ids=[note_doc.catalog_entry().schema_id])
node.database.sync_shard_log(raw_client, shard_id="interop", shard_epoch=1)
replicator = CultNetShardReplicator(
    node.database,
    CultNetShardReplicatorOptions(
        fetcher=CultNetSchemaShardLogFetcher(),
        snapshot_fetcher=CultNetSchemaShardSnapshotFetcher(),
        cursor_store=CultNetFileShardReplicaCursorStore("mesh.replica-cursors.msgpack"),
        poll_interval_seconds=1.0,
        on_error=lambda error: print(error),
    ),
)
replica_shard = CultNetShardDescriptor(
    shard_id="interop",
    owner_runtime_id="primary-runtime",
    epoch=1,
    schema_ids=(note_doc.catalog_entry().schema_id,),
    primary_endpoints=("cultnet://127.0.0.1:3075",),
)
replicator.pull_once(replica_shard)
replicator.start([replica_shard])
replicator.stop()
server.stop()

streams = CultMesh.create_stream_catalog()

claim_hash = "accepted-claim-hash"
facts = CultMesh.create_simulation_fact_committer(node)
committed = facts.commit({
    "shardId": "arena",
    "shardEpoch": 4,
    "frame": 100,
    "subjectId": "bob",
    "claimKind": "hit",
    "claimHash": claim_hash,
    "witnessCount": 2,
    "supportWeight": 2.0,
    "totalWeight": 2.0,
    "confidence": 1.0,
    "hasQuorum": True,
})
session_commits = session.submit_and_commit(typed_observation)
prediction = session.predict(note_doc, "input:python:move", {"body": "predicted input"})
unsubscribe()

Wire Parity

See PARITY.md for the current evidence map, gates, and things this package still does not claim.

The Python interop peer is cultcache_py.interop:

python -m cultcache_py.interop write --file cache.cc --runtime-id python
python -m cultcache_py.interop read --file cache.cc

Current receipts:

• packages/cultcache-ts/test/cult-cache.test.ts includes Python in the shared CultCache v1 parity matrix with TypeScript, Rust, and C#.
• packages/cultnet-ts/test/interop/cultnet-interop.test.ts includes Python in the live TS/Rust/C#/Python schema-v0 peer ring: discovery, hello, schema catalog, wire-message catalog discovery, raw snapshot, raw document put/delete, mutation receipt, fire-command receipt, database subscription changes, shard catalog, shard log catch-up, simulation consensus candidates, committed simulation fact snapshots, Python session prediction reconciliation, and witness artifact bundle round-trips. The same test asks the Python peer for CultMesh Verse catalog and peer exchange responses over the CultNet pipe, then verifies the public CultMeshDiscoveryClient can fetch typed Python Verse and peer descriptors from that live endpoint. It also verifies CultNetRawClient can fetch the live Python peer's schema catalog, raw snapshot, and shard catalog.
• packages/cultcache-py/tests/test_cultcache.py covers Python CultMesh Verse catalog, peer exchange, authority lease, stream negotiation, CultNet helper shapes, raw snapshot/shard-log application, simulation claim hashing, consensus aggregation, game-session fact commits, local prediction reconciliation, committed simulation fact payload slots, and witness artifact bundle payload slots.
• cultcache_py, cultnet_py, and cultmesh_py ship py.typed markers so downstream type checkers can inspect the package surface instead of treating the runtime as an untyped xenos swamp.

Performance Baseline

The package ships a lightweight benchmark for Python-owned hot paths. In a CultLib source checkout, it can also run the C# reference benchmark beside it:

python -m cultcache_py.benchmark --records 1000 --json
python -m cultcache_py.compare_csharp --records 1000 --json
python -m cultcache_py.verify --json

It reports slot-indexed DatabaseEntry encode/decode throughput, framed CultNet MessagePack parse throughput, raw snapshot application into a registered cache, and public CultCache upsert/get throughput. The C# compare command also runs packages/cultcache-py/tools/GameCult.Caching.Benchmark and reports median Python-to-C# ratios for the shared public cache operations. It uses three samples by default; pass --samples when you need a different evidence shape. Both runtimes precompute record keys before measured cache upsert/get loops so the comparison is about public cache access, not string construction noise. The public in-memory cache owner stores records in type-partitioned dictionaries so hot get(...) calls avoid per-call composite-key allocation. Treat the benchmark numbers as a local baseline, not a claim that Python matches the C# reference in every workload.