95493862bd
One distribution named cultcache-py carried three libraries. Rust and TypeScript layer them as cultcache < cultnet < cultmesh; Python now does the same, one package per layer, module names unchanged so every `from cultmesh_py import ...` keeps working. Two imports ran against that layering and are resolved by moving code to where the dependency direction says it lives, not by adding a shim: - cultnet_py.interop_peer imported the Verse/peer catalog, authority lease, stream, and simulation-fact contracts from cultmesh_py. They are wire contracts a bare peer has to speak without owning a node, which is why cultnet-rs and cultnet-ts keep them in the net layer. They now live in cultnet_py.cultmesh_contracts; cultmesh_py re-exports them. - cultcache_py.benchmark imported cultnet_py. It measures framed CultNet parse beside cache paths, so it and compare_csharp move to cultnet_py. verify checks all three packages and moves to cultmesh_py. The single 6,693-line test module is split by what each test touches: a test using any cultmesh_py name lands in cultmesh-py, else cultnet_py in cultnet-py, else cultcache-py. Each suite runs with only its layer and the layers below on the path, so a test that reaches up fails to import rather than quietly binding the packages back together. 11 / 53 / 78 = the same 142. Workflows install all three; the publish job is one matrix over the three packages keyed by tag prefix. The TS test harnesses put all three src directories on PYTHONPATH. cultcache-py bumps to 0.2.0 because it no longer contains cultnet_py or cultmesh_py; the new packages start at 0.1.0. Nothing was ever published, so no PyPI consumer moves. cultcache-py 11 tests, path: cultcache-py/src cultnet-py 53 tests, path: + cultnet-py/src cultmesh-py 78 tests, path: + cultmesh-py/src python -m cultmesh_py.verify -> ok, all five checks cultnet_py.interop_peer imports with cultmesh_py absent Not run here: the cultnet-ts interop ring and cultcache-ts store matrix need dotnet/kotlin peers; their PYTHONPATH change is four lines. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
330 lines
21 KiB
Markdown
330 lines
21 KiB
Markdown
# Python Runtime Parity Map
|
|
|
|
This file is the current audit map for the Python runtime packages
|
|
`cultcache-py`, `cultnet-py`, and `cultmesh-py`. It is evidence, not confetti.
|
|
|
|
## Proven Surfaces
|
|
|
|
- CultCache v1 store parity:
|
|
`packages/cultcache-ts/test/cult-cache.test.ts` includes Python in the shared
|
|
TypeScript/Rust/C#/Python store matrix.
|
|
- CultNet schema-v0 live interop:
|
|
`packages/cultnet-ts/test/interop/cultnet-interop.test.ts` runs a live
|
|
TS/Rust/C#/Python peer ring.
|
|
The Python interop peer filters shard-scoped raw snapshots by its live
|
|
shard-log membership before stamping shard cursor metadata.
|
|
- Python CultNet raw client:
|
|
Fetches schema catalogs, raw snapshots, shard catalogs, shard logs, and live
|
|
database subscription changes. Request-response reads and database
|
|
subscription streams raise typed `CultNetPeerError` values for
|
|
`cultnet.error.v0` peer responses while preserving the raw error document for
|
|
callers that need the wire evidence. `CultNetRawClient` and database
|
|
subscriptions now consume a caller-provided `CultNetSchemaTransport` factory,
|
|
with `create_tcp_framed_schema_transport(...)` as the default, so request and
|
|
subscription semantics are no longer owned by inline TCP socket setup.
|
|
`create_rudp_schema_transport(...)` opens the same raw-client request path
|
|
over a socket-backed RUDP reliable ordered `schema` channel, and the branded
|
|
`CultMesh.create_client(endpoint="rudp://...")` helper selects that transport
|
|
from a RUDP endpoint without making callers assemble a factory closure.
|
|
- Python CultNet raw snapshot:
|
|
Provides typed `CultNetRawSnapshotResponse` and `CultNetRawDocumentRecord`
|
|
helpers plus `CultNetRawClient.fetch_snapshot_response(...)` for inspecting
|
|
peer snapshots before handing them to the existing replication helpers.
|
|
`apply_raw_document_record(...)`, `apply_raw_snapshot(...)`,
|
|
`CultMeshDatabase.apply_snapshot_response(...)`, and sync paths accept typed
|
|
records/responses while preserving the existing replication owner.
|
|
`CultMeshDatabase.build_snapshot_response(...)` projects local envelopes into
|
|
the same typed response shape; the older `create_snapshot_response(...)`
|
|
remains the wire-dict compatibility projection.
|
|
- Python CultNet schema catalog:
|
|
Exposes shared wire-message schema descriptors from the package surface so
|
|
both the interop peer and CultMesh local server describe the CultNet/CultMesh
|
|
messages they speak through the same catalog authority. Python callers can
|
|
also ingest and filter remote schema catalog responses through typed
|
|
`CultNetSchemaCatalog` and `CultNetSchemaDescriptor` helpers, including
|
|
`CultNetRawClient.sync_schema_catalog(...)`. The catalog includes
|
|
`cultnet.error.v0`, and the local server advertises that version in hello
|
|
responses because oversized snapshots and other peer failures use that
|
|
contract on the wire. It also advertises the shared
|
|
`cultnet.transport_profile.v0` contract so Python peers can describe their
|
|
TCP-framed and RUDP transport capabilities honestly.
|
|
- Python CultNet security/session primitives:
|
|
`CultNetClientSecurityOptions`, `CultNetServerSecurityOptions`, and
|
|
`CultNetSecret` mirror the C#/TS connection-key, server signing-secret,
|
|
development-default, environment-validation, base64url, and HMAC-SHA256
|
|
session-token helper surface. Python creates C#-compatible versioned session
|
|
tokens with `userGuidN|expiresUnix|sessionVersion` payloads and still
|
|
validates legacy two-field tokens with session version `0`. AES-GCM string
|
|
and byte helpers use the same tag-prefix payload shape when the optional
|
|
`cryptography` dependency is available; without that dependency they fail
|
|
explicitly instead of pretending plaintext is secure.
|
|
- Python CultNet shard catalog:
|
|
Provides typed `CultNetShardCatalog` and `CultNetShardDescriptor` helpers for
|
|
ingesting, filtering, and emitting shard topology responses, plus
|
|
`CultNetRawClient.sync_shard_catalog(...)` for peer topology discovery over
|
|
the same framed CultNet pipe.
|
|
- Python CultNet shard log:
|
|
Provides typed `CultNetShardLogResponse` and `CultNetShardLogEntry` helpers
|
|
for inspecting mutation-log entries, resync pressure, and sequence cursors,
|
|
including typed put-document and delete identity projections on individual
|
|
entries while cache mutation remains owned by the existing replication
|
|
helpers.
|
|
`apply_shard_log_response(...)`, `CultMeshDatabase`, and
|
|
`CultMeshGameSession` can consume typed shard-log responses directly for
|
|
database sync and prediction reconciliation.
|
|
`CultMeshDatabase.build_shard_log_response(...)` creates the typed response
|
|
from the package-local mutation log and reports resync-required responses for
|
|
unknown shards or stale epoch cursors; `create_shard_log_response(...)`
|
|
remains the raw wire-dict compatibility projection.
|
|
- Python CultNet shard replication:
|
|
Provides schema-v0 shard-log and shard-snapshot fetchers, a schema-v0
|
|
write-forwarder for raw put/delete messages, plus a
|
|
`CultNetShardReplicator` that tracks replica cursors, applies log responses
|
|
through the existing database owner, runs explicit background pull loops for
|
|
non-primary shards with advertised primaries, reports loop errors through a
|
|
caller-owned callback, and recovers from compacted-history resync pressure by
|
|
fetching a shard snapshot. Replica cursor storage can be in-memory or
|
|
restart-safe through a local MessagePack cursor file. The write-forwarder is
|
|
also exercised against a launched Python daemon endpoint: forwarded raw
|
|
put/delete messages mutate daemon state and are recorded in the daemon-served
|
|
shard log.
|
|
- Python CultNet database subscription changes:
|
|
Provides typed `CultNetDatabaseChange` parsing plus
|
|
`CultNetDatabaseSubscription.read_next_change(...)` / `iter_changes(...)` for
|
|
live raw database subscriptions. Put/update changes expose their contained
|
|
raw document as a typed `CultNetRawDocumentRecord` while preserving the legacy
|
|
dict-shaped `document` projection. Initial snapshot frames can be read as
|
|
typed `CultNetRawSnapshotResponse` values through
|
|
`read_next_snapshot_response(...)` / `iter_snapshot_responses(...)`, while
|
|
raw message reads remain available for callers that own their own dispatch
|
|
loop.
|
|
- Python CultNet witness artifacts:
|
|
Provides typed `CultNetWitnessArtifactBundle` ownership for the C#-compatible
|
|
8-slot witness artifact payload while preserving dict helpers for existing
|
|
wire callers.
|
|
- Python CultNet RUDP transport:
|
|
Provides the shared `cultnet.transport.rudp.v0` packet codec, deterministic
|
|
RUDP session state machine, socket-backed single-peer transport connection,
|
|
transport profile advertisement, ping/pong, disconnect reason propagation,
|
|
timeout state, fragmentation/reassembly, reliable ordered `schema` delivery,
|
|
unreliable sequenced `latest`, unreliable unordered `realtime`, resend
|
|
pressure, bounded reliable-send backpressure, and the portable
|
|
`cultnet.reconnect_policy.v0` delay-policy helper plus reconnect controller
|
|
advertised through RUDP transport profiles. `CultNetRudpReconnectLoop`
|
|
consumes that controller for caller-owned RUDP transport factories, matching
|
|
the TypeScript/Kotlin reconnect-loop shape without making the socket factory
|
|
a hidden global. The TypeScript interop harness proves bidirectional Python/TypeScript RUDP socket exchange,
|
|
packet-loss recovery, reordering recovery, fragmentation, ping/pong, and
|
|
disconnect reason propagation through the shared wire language.
|
|
- Python CultMesh discovery client:
|
|
Fetches Verse catalogs and peer exchange responses from live endpoints over
|
|
CultNet frames. It can also parse `cultnet://host:port` endpoints, discover
|
|
from endpoint lists, de-duplicate discovery targets, fan out one-shot
|
|
discovery from endpoints already advertised in local Verse and peer catalogs,
|
|
fan out raw snapshot fetches and simulation observations to peer-card
|
|
endpoints, apply peer snapshots through a caller-owned database, run
|
|
lightweight scheduled snapshot and simulation-observation fanout loops, probe
|
|
peer-card endpoints for hello-level reachability, preserve hello capability
|
|
metadata on health results including document types, message versions, and
|
|
mutation contracts, run immediate scheduled health probes on start, and
|
|
upsert results into local catalogs with the same count-returning ergonomics
|
|
as the C# discovery clients.
|
|
- Python CultMesh facade:
|
|
Exposes C#-matching client entrypoints for Verse discovery, peer exchange,
|
|
raw CultNet client construction, local nodes, Verse catalogs, peer catalogs,
|
|
schema catalogs, built-in schema catalogs, shard catalogs, sessions, streams,
|
|
authority leases, local TCP node serving, simulation fact commits, RUDP
|
|
endpoint parsing, and RUDP client/server construction from endpoint or
|
|
peer-card contact hints. Peer catalogs expose authorized-peer lookup, and the
|
|
branded facade can open a RUDP client from the first peer authorized by the
|
|
authority lease catalog for a Verse role, keeping endpoint contact separate
|
|
from trust. The durable shard-log node option attaches a file-backed
|
|
authoritative log store beside the cache file by default. Schema, shard, and
|
|
RUDP factories delegate to the existing `cultnet_py` owners, so the branded
|
|
entrypoint does not create a second source of discovery or transport truth.
|
|
The `cultmesh-py-daemon` /
|
|
`python -m cultmesh_py.daemon` entrypoint launches that same local TCP server
|
|
body and emits a bounded readiness document for process supervisors while the
|
|
served CultNet endpoint remains the capability owner. Readiness now mirrors
|
|
the served hello capability metadata, shard ids, and configured snapshot
|
|
limits so supervisors can inspect startup state without inventing a second
|
|
protocol truth. Readiness also includes the same Verse descriptors and peer
|
|
cards served by the live discovery endpoints, keeping operator and agent
|
|
inspection tied to the catalogs that own mesh identity. The daemon can
|
|
register and seed the package interop-note
|
|
document so launched processes advertise a real payload schema and serve both
|
|
a real snapshot and a real shard-log entry over the same CultNet lane. When
|
|
launched with a Verse id, it publishes its actual bound endpoint in the Verse
|
|
and peer catalogs so discovery clients can find the running Python peer
|
|
without embedding code-owned catalogs. With a cache file and durable shard-log
|
|
path, a restarted daemon can rehydrate the persisted interop-note snapshot and
|
|
shard-log entry without reseeding. The launched daemon also accepts raw puts
|
|
over a live database subscription stream, emits the matching change
|
|
notification, and records the mutation in the served shard log. If its durable
|
|
shard log has been compacted, stale shard-log requests return the same
|
|
explicit `compacted_history` resync boundary over the daemon wire endpoint
|
|
instead of silently looking empty. Simulation observation serving is opt-in:
|
|
the daemon only advertises `cultnet.simulation_observation.v0` when an
|
|
observation hub is attached, and an opt-in launched daemon answers witness
|
|
observations with schema-v0 consensus candidates. When such a daemon
|
|
self-advertises into a Verse, its peer card includes the `simulation-observer`
|
|
role so discovery clients can find the peer by the capability it actually
|
|
serves.
|
|
The daemon CLI snapshot document/byte limits feed the same public server edge;
|
|
oversized daemon snapshot responses are returned as structured
|
|
`cultnet.error.v0` peer errors rather than partial payloads.
|
|
- Python CultMesh authority leases:
|
|
Lease objects own validity and peer/role/shard coverage checks; the catalog
|
|
owns sorted listing, lookup, upsert, signature verification policy, and
|
|
authorization delegation, matching the C# reference authority boundary.
|
|
Callers can require verified signatures and use the packaged HMAC verifier to
|
|
issue and verify canonical lease payloads keyed by issuer runtime.
|
|
- Python CultMesh local server:
|
|
Serves a `CultMeshNode` plus Verse and peer catalogs over the same framed
|
|
MessagePack request/response lane used by `CultNetRawClient` and
|
|
`CultMeshDiscoveryClient`. Accepted connections now read and write through
|
|
the shared `TcpFramedTransportConnection` port rather than owning raw frame
|
|
I/O in the server loop. It also supports live raw database subscriptions for
|
|
snapshot, put, and delete notifications on a connected client stream.
|
|
When given a `CultNetSimulationObservationHub`, it accepts
|
|
`cultnet.simulation_observation.v0` messages and replies with current
|
|
`cultnet.simulation_consensus_candidate.v0` messages through the same framed
|
|
lane.
|
|
Its schema catalog includes both registered document payload schemas and
|
|
shared wire-message descriptors for the CultNet/CultMesh messages it handles.
|
|
Hello and payload schema descriptors advertise the raw snapshot, put, delete,
|
|
and shard-log mutation surfaces the server actually serves.
|
|
Unsupported request schemas, malformed raw put/delete messages, and snapshot
|
|
limit violations return structured `cultnet.error.v0` documents instead of
|
|
silent stream stalls.
|
|
Its shard catalog advertises the database-owned shard ids, schema ids, and
|
|
latest shard epochs so discovery and shard-log catch-up agree on the same
|
|
authority.
|
|
Public mesh edges can set snapshot document and encoded-byte limits; oversized
|
|
snapshot responses are rejected as `cultnet.error.v0` before the payload is
|
|
sent.
|
|
- Python CultMesh reactive catalogs:
|
|
Verse and peer catalogs expose sorted local views, `get(...)`, response
|
|
application, and `watch(...)` callbacks for local discovery updates. Verse
|
|
descriptors can evaluate transfer compatibility with the same transport/rules
|
|
shape as the C# reference surface.
|
|
- Python CultMesh database facade:
|
|
`node.database` owns the local typed document read/write/snapshot/sync surface
|
|
so Python callers do not need to reach through the node composition wrapper.
|
|
It includes global document helpers that delegate through the same singleton
|
|
key invariant as `CultCache`.
|
|
It also exposes local `watch(...)`, `watch_record(...)`, `watch_global(...)`,
|
|
`watch_by_name(...)`, and `watch_by_index(...)` callbacks for database changes,
|
|
including authoritative shard-log reconciliation applied through the session
|
|
facade.
|
|
Change notifications carry previous values when the database can observe the
|
|
old record before local or replicated application.
|
|
It can project local envelopes into schema-v0 raw snapshot responses and apply
|
|
those responses back through the shared CultNet replication path. Typed
|
|
builders own the local response shape and raw dict methods serialize that
|
|
shape for compatibility with older callers and wire handlers.
|
|
Shard-scoped raw snapshots filter by shard-log membership when the database
|
|
has an authoritative log for the requested shard.
|
|
Raw shard-scoped writes append a package-local mutation log that can be
|
|
projected into schema-v0 shard-log responses for peer catch-up.
|
|
- Python CultMesh committed simulation facts:
|
|
The live interop harness sends Python a simulation observation, receives a
|
|
consensus candidate, then requests the committed `gamecult.mesh.simulation_fact`
|
|
document back over the raw snapshot lane and checks its MessagePack slots.
|
|
Python exposes typed `CultNetSimulationObservation` and
|
|
`CultNetSimulationConsensusCandidate` objects for local consensus and
|
|
game-session commit paths while preserving the schema-v0 dict projections
|
|
used by the wire harness. Scheduled simulation-observation fanout keeps
|
|
queued observations pending when no peer produces a candidate, even when
|
|
endpoint failures are reported through a caller-owned error callback.
|
|
- Python CultMesh prediction reconciliation:
|
|
The live interop harness runs a Python CultMesh session client, writes a local
|
|
prediction, pulls the live Python peer shard log, applies the authoritative
|
|
entry, and verifies the session reports the record as reconciled.
|
|
Python sessions also expose pending prediction inspection, resimulation input
|
|
listing, explicit rollback of one or all pending predictions, and
|
|
authoritative-delete rollback when a shard log rejects speculative local
|
|
input.
|
|
- Python CultMesh session watches:
|
|
`CultMeshGameSession.watch_candidates(...)` and
|
|
`watch_simulation_facts(...)` expose the same gameplay-facing observation and
|
|
committed-fact watch points as the C# session facade. A game session can also
|
|
serve its own Verse catalog, peer catalog, and simulation observation hub over
|
|
the local framed CultMesh server so peer-facing session state uses the same
|
|
wire contracts as lower-level server callers. Session serving options can keep
|
|
individual discovery, peer exchange, or simulation surfaces unserved without
|
|
advertising disabled capabilities.
|
|
- Python package health:
|
|
`cultmesh-py-verify --json` checks the public CultCache/CultNet/CultMesh
|
|
export surface declared by each package `__all__`, `py.typed` markers, a live
|
|
local CultMesh framed-wire smoke over hello/schema/snapshot/shard
|
|
catalog/shard log, typed peer-error export availability, peer-advertised
|
|
mutation contracts, payload schema wire contracts, simulation capability
|
|
truth, and benchmark sanity for Python-owned hot paths.
|
|
- Python/C# public cache baseline:
|
|
`cultnet_py.compare_csharp` runs Python and C# `CultCache` upsert/get
|
|
benchmarks with the same operation names and reports median Python-to-C# ratios
|
|
from three samples by default. It also emits per-metric threshold checks and an
|
|
overall parity status so performance claims have an explicit evidence gate.
|
|
Both benchmark lanes precompute record keys before measuring public cache
|
|
reads and writes, matching the C# reference benchmark shape instead of
|
|
charging one runtime for key construction inside the hot path.
|
|
The Python in-memory record owner is type-partitioned so public `get(...)`
|
|
reads avoid per-call composite-key allocation on the hot path.
|
|
|
|
## Local Python Gates
|
|
|
|
```powershell
|
|
$env:PYTHONPATH="$PWD\packages\cultcache-py\src;$PWD\packages\cultnet-py\src;$PWD\packages\cultmesh-py\src"
|
|
python -m unittest discover -s packages\cultcache-py\tests
|
|
python -m unittest discover -s packages\cultnet-py\tests
|
|
python -m unittest discover -s packages\cultmesh-py\tests
|
|
python -m cultmesh_py.verify --json
|
|
python -m cultnet_py.benchmark --records 1000 --json
|
|
python -m cultnet_py.compare_csharp --records 1000 --samples 3 --json
|
|
python -m pip wheel --no-deps -w $env:TEMP packages\cultcache-py packages\cultnet-py packages\cultmesh-py
|
|
```
|
|
|
|
## Cross-Runtime Gate
|
|
|
|
```powershell
|
|
$env:PYTHON='C:\Users\Meta\.cache\codex-runtimes\codex-primary-runtime\dependencies\python\python.exe'
|
|
npx tsc -p packages\cultnet-ts\tsconfig.test.json --pretty false
|
|
node --test packages\cultnet-ts\dist-test\test\interop\cultnet-interop.test.js
|
|
```
|
|
|
|
## Still Not Claimed
|
|
|
|
- Python does not claim to match the C# runtime for every raw-throughput
|
|
workload. It has a measured local baseline plus a shared public `CultCache`
|
|
upsert/get comparison harness with threshold status, not a universal
|
|
performance victory certificate.
|
|
- Python does not implement the full LiteNetLib client/server auth flow. It now
|
|
owns the shared security/session primitives for helper parity, while the live
|
|
transport harness still speaks the schema-v0 interop lane.
|
|
- Python does not make RUDP the only public pipe yet. The local CultMesh server,
|
|
raw client, and interop peer still keep the TCP-framed schema lane for
|
|
compatibility, while the Python interop peer can now serve and dial schema-v0
|
|
over fragmented RUDP. `docs/cultnet-transport-parity.md` tracks the
|
|
remaining work to put broader server-side bodies behind the shared transport
|
|
port instead of cloning LiteNetLib. The raw client request/subscription layer
|
|
now routes through a schema-transport factory, so the caller-facing semantics
|
|
are transport-port-shaped even when the default factory opens TCP-framed
|
|
connections. Python raw clients can now use a RUDP schema transport directly
|
|
through `create_rudp_schema_transport(...)` or the `CultMesh.create_client`
|
|
RUDP endpoint helper. The reconnect policy/controller primitive is portable
|
|
and advertised by RUDP profiles, and `CultNetRudpReconnectLoop` gives Python
|
|
callers the same RUDP retry-scheduling helper as TypeScript/Kotlin. Automatic
|
|
reconnect use by Python service bodies is not yet claimed.
|
|
- Python does not implement the full C# daemon/server stack. Its current role is
|
|
package runtime, raw interop peer, launchable local CultMesh endpoint, local
|
|
CultMesh session facade, and typed state participant.
|
|
- Python does not claim the full C# shard mutation-log service surface. It can
|
|
create/apply raw snapshots, create/apply package-local raw shard-log
|
|
responses, consume remote shard-log responses, and run pull-once replica
|
|
catch-up with snapshot recovery, a file-backed replica cursor store, and
|
|
schema-v0 write forwarding. It also has an explicit package-level background
|
|
pull loop for caller-provided shard descriptors. Its authoritative shard-log
|
|
store is file-backed, supports compaction watermarks, and can be attached by
|
|
`CultMeshNodeOptions(enable_durable_shard_logs=True)`, but it is still
|
|
package-local rather than the full C# daemon/server stack.
|