Split the Python runtime into cultcache-py, cultnet-py, cultmesh-py
One distribution named cultcache-py carried three libraries. Rust and TypeScript layer them as cultcache < cultnet < cultmesh; Python now does the same, one package per layer, module names unchanged so every `from cultmesh_py import ...` keeps working. Two imports ran against that layering and are resolved by moving code to where the dependency direction says it lives, not by adding a shim: - cultnet_py.interop_peer imported the Verse/peer catalog, authority lease, stream, and simulation-fact contracts from cultmesh_py. They are wire contracts a bare peer has to speak without owning a node, which is why cultnet-rs and cultnet-ts keep them in the net layer. They now live in cultnet_py.cultmesh_contracts; cultmesh_py re-exports them. - cultcache_py.benchmark imported cultnet_py. It measures framed CultNet parse beside cache paths, so it and compare_csharp move to cultnet_py. verify checks all three packages and moves to cultmesh_py. The single 6,693-line test module is split by what each test touches: a test using any cultmesh_py name lands in cultmesh-py, else cultnet_py in cultnet-py, else cultcache-py. Each suite runs with only its layer and the layers below on the path, so a test that reaches up fails to import rather than quietly binding the packages back together. 11 / 53 / 78 = the same 142. Workflows install all three; the publish job is one matrix over the three packages keyed by tag prefix. The TS test harnesses put all three src directories on PYTHONPATH. cultcache-py bumps to 0.2.0 because it no longer contains cultnet_py or cultmesh_py; the new packages start at 0.1.0. Nothing was ever published, so no PyPI consumer moves. cultcache-py 11 tests, path: cultcache-py/src cultnet-py 53 tests, path: + cultnet-py/src cultmesh-py 78 tests, path: + cultmesh-py/src python -m cultmesh_py.verify -> ok, all five checks cultnet_py.interop_peer imports with cultmesh_py absent Not run here: the cultnet-ts interop ring and cultcache-ts store matrix need dotnet/kotlin peers; their PYTHONPATH change is four lines. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
parent
a8aeddacab
commit
95493862bd
55 changed files with 7332 additions and 7008 deletions
4
.github/workflows/cultmesh-portability.yml
vendored
4
.github/workflows/cultmesh-portability.yml
vendored
|
|
@ -77,8 +77,8 @@ jobs:
|
|||
- name: Install TypeScript workspace dependencies
|
||||
run: npm install --no-package-lock
|
||||
|
||||
- name: Install Python CultCache package
|
||||
run: python -m pip install -e packages/cultcache-py
|
||||
- name: Install Python CultLib packages
|
||||
run: python -m pip install -e packages/cultcache-py -e packages/cultnet-py -e packages/cultmesh-py
|
||||
|
||||
- name: Verify explicit reactive scheduling across runtimes
|
||||
shell: pwsh
|
||||
|
|
|
|||
2
.github/workflows/cultnet-interop.yml
vendored
2
.github/workflows/cultnet-interop.yml
vendored
|
|
@ -69,7 +69,7 @@ jobs:
|
|||
run: npm install --no-package-lock
|
||||
|
||||
- name: Install Python package dependencies
|
||||
run: python -m pip install -e packages/cultcache-py
|
||||
run: python -m pip install -e packages/cultcache-py -e packages/cultnet-py -e packages/cultmesh-py
|
||||
|
||||
- name: Build TypeScript workspace dependencies
|
||||
run: npm run build --workspace packages/cultcache-ts
|
||||
|
|
|
|||
56
.github/workflows/publish-packages.yml
vendored
56
.github/workflows/publish-packages.yml
vendored
|
|
@ -8,15 +8,18 @@ name: Publish packages
|
|||
# directly, which is already how Ghostlight, Epiphany, CodexConnector and Odin
|
||||
# consume the core stack.
|
||||
#
|
||||
# Tag a release to publish, for example `cultcache-ts-v0.13.6`. The tag's
|
||||
# version must match the version in the package manifest; the workflow refuses
|
||||
# the publish otherwise rather than shipping a mislabelled artifact.
|
||||
# Tag a release to publish, for example `cultcache-ts-v0.13.6` or
|
||||
# `cultnet-py-v0.1.0`. The tag's version must match the version in the package
|
||||
# manifest; the workflow refuses the publish otherwise rather than shipping a
|
||||
# mislabelled artifact.
|
||||
|
||||
on:
|
||||
push:
|
||||
tags:
|
||||
- "cultcache-ts-v*"
|
||||
- "cultcache-py-v*"
|
||||
- "cultnet-py-v*"
|
||||
- "cultmesh-py-v*"
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
package:
|
||||
|
|
@ -26,6 +29,8 @@ on:
|
|||
options:
|
||||
- cultcache-ts
|
||||
- cultcache-py
|
||||
- cultnet-py
|
||||
- cultmesh-py
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
|
@ -72,9 +77,16 @@ jobs:
|
|||
env:
|
||||
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
|
||||
|
||||
cultcache-py:
|
||||
name: cultcache-py to PyPI
|
||||
if: startsWith(github.ref, 'refs/tags/cultcache-py-v') || inputs.package == 'cultcache-py'
|
||||
# The three Python packages layer cultcache-py < cultnet-py < cultmesh-py, the
|
||||
# same way the Rust crates and npm packages do. One job shape serves all
|
||||
# three; the matrix entry names the package and the tag prefix that releases it.
|
||||
python:
|
||||
name: ${{ matrix.package }} to PyPI
|
||||
if: startsWith(github.ref, 'refs/tags/cultcache-py-v') || startsWith(github.ref, 'refs/tags/cultnet-py-v') || startsWith(github.ref, 'refs/tags/cultmesh-py-v') || endsWith(inputs.package, '-py')
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
package: [cultcache-py, cultnet-py, cultmesh-py]
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 20
|
||||
environment: pypi
|
||||
|
|
@ -84,31 +96,51 @@ jobs:
|
|||
# long-lived API token is stored in the repository.
|
||||
id-token: write
|
||||
steps:
|
||||
- name: Skip packages this run does not name
|
||||
id: selected
|
||||
run: |
|
||||
if [ "${GITHUB_REF_NAME#${{ matrix.package }}-v}" != "$GITHUB_REF_NAME" ] || [ "${{ inputs.package }}" = "${{ matrix.package }}" ]; then
|
||||
echo "run=true" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "run=false" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
- name: Check out CultLib
|
||||
if: steps.selected.outputs.run == 'true'
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Python
|
||||
if: steps.selected.outputs.run == 'true'
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Verify the tag matches the manifest version
|
||||
if: startsWith(github.ref, 'refs/tags/cultcache-py-v')
|
||||
if: steps.selected.outputs.run == 'true' && startsWith(github.ref, 'refs/tags/')
|
||||
run: |
|
||||
manifest="$(python -c "import tomllib,pathlib;print(tomllib.loads(pathlib.Path('packages/cultcache-py/pyproject.toml').read_text())['project']['version'])")"
|
||||
tagged="${GITHUB_REF_NAME#cultcache-py-v}"
|
||||
manifest="$(python -c "import tomllib,pathlib;print(tomllib.loads(pathlib.Path('packages/${{ matrix.package }}/pyproject.toml').read_text())['project']['version'])")"
|
||||
tagged="${GITHUB_REF_NAME#${{ matrix.package }}-v}"
|
||||
if [ "$manifest" != "$tagged" ]; then
|
||||
echo "tag $GITHUB_REF_NAME does not match manifest version $manifest" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Lower layers install from source so a release of one package is tested
|
||||
# against the sibling revision it was cut with, not whatever PyPI has.
|
||||
- name: Install and test
|
||||
if: steps.selected.outputs.run == 'true'
|
||||
run: |
|
||||
python -m pip install -e packages/cultcache-py -e packages/cultnet-py -e packages/cultmesh-py
|
||||
python -m unittest discover -s packages/${{ matrix.package }}/tests
|
||||
|
||||
- name: Build sdist and wheel
|
||||
if: steps.selected.outputs.run == 'true'
|
||||
run: |
|
||||
python -m pip install --upgrade build
|
||||
python -m build packages/cultcache-py
|
||||
python -m build packages/${{ matrix.package }}
|
||||
|
||||
- name: Publish
|
||||
if: startsWith(github.ref, 'refs/tags/cultcache-py-v')
|
||||
if: steps.selected.outputs.run == 'true' && startsWith(github.ref, 'refs/tags/')
|
||||
uses: pypa/gh-action-pypi-publish@release/v1
|
||||
with:
|
||||
packages-dir: packages/cultcache-py/dist
|
||||
packages-dir: packages/${{ matrix.package }}/dist
|
||||
|
|
|
|||
4
.gitignore
vendored
4
.gitignore
vendored
|
|
@ -303,6 +303,10 @@ node_modules/
|
|||
!packages/cultmesh-ts/**
|
||||
!packages/cultcache-py/
|
||||
!packages/cultcache-py/**
|
||||
!packages/cultnet-py/
|
||||
!packages/cultnet-py/**
|
||||
!packages/cultmesh-py/
|
||||
!packages/cultmesh-py/**
|
||||
!packages/cultcache-rs/
|
||||
!packages/cultcache-rs/**
|
||||
!packages/cultnet-rs/
|
||||
|
|
|
|||
|
|
@ -112,7 +112,9 @@ The solution includes:
|
|||
- `packages/cultnet-ts`: TypeScript CultNet schema-v0 contracts, framing, discovery, raw document replication, and interop tests
|
||||
- `packages/cultmesh-ts`: TypeScript CultMesh local node and mesh catalog surface for local runtimes such as VoidBot
|
||||
- `packages/cultmesh-browser`: browser-safe, read-only CultMesh document leases and typed operations over an Odin-resolved WebSocket route
|
||||
- `packages/cultcache-py`: Python CultCache/CultNet/CultMesh package with CultCache v1 wire parity
|
||||
- `packages/cultcache-py`: Python CultCache with CultCache v1 wire parity
|
||||
- `packages/cultnet-py`: Python CultNet contracts, framing, RUDP transport, replication, and interop peer
|
||||
- `packages/cultmesh-py`: Python CultMesh local node, discovery, sessions, local server, and daemon
|
||||
- `packages/cultcache-rs`: Rust CultCache and derive macro
|
||||
- `packages/cultnet-rs`: Rust CultNet contracts, framing, discovery, and interop peer
|
||||
- `packages/cultmesh-kotlin`: Kotlin/JVM CultMesh and CultNet surface for Android-adjacent runtimes
|
||||
|
|
@ -145,6 +147,8 @@ packages/
|
|||
cultmesh-browser/
|
||||
cultmesh-kotlin/
|
||||
cultcache-py/
|
||||
cultnet-py/
|
||||
cultmesh-py/
|
||||
cultcache-rs/
|
||||
cultnet-rs/
|
||||
```
|
||||
|
|
|
|||
|
|
@ -31,7 +31,7 @@ CultNet owns cross-runtime transport semantics:
|
|||
| C# interop peer | Schema-v0 MessagePack over TCP-framed or shared RUDP transport | UDP multicast probe/announce with TCP and RUDP transport profiles | Test harness transport parity |
|
||||
| TypeScript `cultnet-ts` | `CultNetPeer` over any Node `Duplex`, TCP-framed transport, or single-peer RUDP socket transport; interop uses TCP | UDP multicast probe/announce in the interop peer | First UDP socket binding for the shared RUDP reliability owner |
|
||||
| Rust `cultnet-rs` | Interop example serves and dials schema-v0 MessagePack over TCP-framed or shared RUDP transport; single-peer RUDP socket transport exists in the library | UDP multicast probe/announce with TCP and RUDP transport profiles | UDP socket binding for the shared RUDP reliability owner |
|
||||
| Python `cultcache-py` | TCP sockets with 4-byte length-prefixed MessagePack frames for local CultMesh/CultNet server and client; single-peer RUDP socket transport exists in the library | Endpoint lists and CultMesh peer/Verse catalogs | UDP socket binding for the shared RUDP reliability owner |
|
||||
| Python `cultnet-py` | TCP sockets with 4-byte length-prefixed MessagePack frames for local CultMesh/CultNet server and client; single-peer RUDP socket transport exists in the library | Endpoint lists and CultMesh peer/Verse catalogs | UDP socket binding for the shared RUDP reliability owner |
|
||||
| Kotlin `cultmesh-kotlin` | Channel-aware WebSocket transport connection for reliable ordered `schema` frames; single-peer RUDP socket transport exists in the library; interop CLI serves and dials schema-v0 MessagePack over TCP-framed or shared RUDP transport | CultMesh Verse and peer catalogs, plus endpoint lists carried in schema-v0 catalog messages | WebSocket/TCP adapters plus UDP socket binding for the shared RUDP reliability owner, with build-script and TypeScript harness proof |
|
||||
|
||||
The live split is therefore: payload language and the native RUDP packet/session
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
# Python Runtime Parity Map
|
||||
|
||||
This file is the current audit map for `cultcache-py` as a Python runtime for
|
||||
CultCache, CultNet, and CultMesh. It is evidence, not confetti.
|
||||
This file is the current audit map for the Python runtime packages
|
||||
`cultcache-py`, `cultnet-py`, and `cultmesh-py`. It is evidence, not confetti.
|
||||
|
||||
## Proven Surfaces
|
||||
|
||||
|
|
@ -255,14 +255,14 @@ CultCache, CultNet, and CultMesh. It is evidence, not confetti.
|
|||
individual discovery, peer exchange, or simulation surfaces unserved without
|
||||
advertising disabled capabilities.
|
||||
- Python package health:
|
||||
`cultcache-py-verify --json` checks the public CultCache/CultNet/CultMesh
|
||||
`cultmesh-py-verify --json` checks the public CultCache/CultNet/CultMesh
|
||||
export surface declared by each package `__all__`, `py.typed` markers, a live
|
||||
local CultMesh framed-wire smoke over hello/schema/snapshot/shard
|
||||
catalog/shard log, typed peer-error export availability, peer-advertised
|
||||
mutation contracts, payload schema wire contracts, simulation capability
|
||||
truth, and benchmark sanity for Python-owned hot paths.
|
||||
- Python/C# public cache baseline:
|
||||
`cultcache_py.compare_csharp` runs Python and C# `CultCache` upsert/get
|
||||
`cultnet_py.compare_csharp` runs Python and C# `CultCache` upsert/get
|
||||
benchmarks with the same operation names and reports median Python-to-C# ratios
|
||||
from three samples by default. It also emits per-metric threshold checks and an
|
||||
overall parity status so performance claims have an explicit evidence gate.
|
||||
|
|
@ -275,12 +275,14 @@ CultCache, CultNet, and CultMesh. It is evidence, not confetti.
|
|||
## Local Python Gates
|
||||
|
||||
```powershell
|
||||
$env:PYTHONPATH="$PWD\packages\cultcache-py\src"
|
||||
$env:PYTHONPATH="$PWD\packages\cultcache-py\src;$PWD\packages\cultnet-py\src;$PWD\packages\cultmesh-py\src"
|
||||
python -m unittest discover -s packages\cultcache-py\tests
|
||||
python -m cultcache_py.verify --json
|
||||
python -m cultcache_py.benchmark --records 1000 --json
|
||||
python -m cultcache_py.compare_csharp --records 1000 --samples 3 --json
|
||||
python -m pip wheel --no-deps -w $env:TEMP packages\cultcache-py
|
||||
python -m unittest discover -s packages\cultnet-py\tests
|
||||
python -m unittest discover -s packages\cultmesh-py\tests
|
||||
python -m cultmesh_py.verify --json
|
||||
python -m cultnet_py.benchmark --records 1000 --json
|
||||
python -m cultnet_py.compare_csharp --records 1000 --samples 3 --json
|
||||
python -m pip wheel --no-deps -w $env:TEMP packages\cultcache-py packages\cultnet-py packages\cultmesh-py
|
||||
```
|
||||
|
||||
## Cross-Runtime Gate
|
||||
|
|
@ -52,7 +52,7 @@ CultMesh.
|
|||
`reconnectPolicy` profile field from C#, Rust, Python, and Kotlin peers.
|
||||
Runtime-local tests cover the deterministic delay helper and reconnect
|
||||
controller in C#, TypeScript, Rust, Python, and Kotlin.
|
||||
- `packages/cultcache-py/PARITY.md` is the Python-specific audit ledger. Its
|
||||
- `docs/python-runtime-parity.md` is the Python-specific audit ledger. Its
|
||||
"Still Not Claimed" section is part of the contract, not an apology.
|
||||
- `docs/cultnet-transport-parity.md` is the transport ownership map. It tracks
|
||||
the remaining work to lower older LiteNetLib/WebSocket bodies behind the
|
||||
|
|
|
|||
|
|
@ -1,12 +1,16 @@
|
|||
# cultcache-py
|
||||
# cultcache-py
|
||||
|
||||
`cultcache-py` is the Python runtime package for CultCache, CultNet, and
|
||||
CultMesh. Callers work with registered domain documents while the cache owns
|
||||
schema identity, routing, globals, name lookups, indexes, and backing-store
|
||||
envelopes.
|
||||
`cultcache-py` is the Python CultCache runtime. Callers work with registered
|
||||
domain documents while the cache owns schema identity, routing, globals, name
|
||||
lookups, indexes, and backing-store envelopes.
|
||||
|
||||
It is intentionally small. It is not an ORM, a database, or distributed consensus wearing rented authority.
|
||||
|
||||
It is the bottom of the Python CultLib stack and depends only on `msgpack`.
|
||||
`cultnet-py` (transport and wire contracts) and `cultmesh-py` (nodes,
|
||||
discovery, sessions, local server) build on it, mirroring `cultcache-rs` /
|
||||
`cultnet-rs` / `cultmesh-rs` and the npm packages.
|
||||
|
||||
## Current Shape
|
||||
|
||||
- document types are registered explicitly
|
||||
|
|
@ -21,10 +25,6 @@ It is intentionally small. It is not an ORM, a database, or distributed consensu
|
|||
- the JSONL store uses only the Python standard library for bootstrap consumers
|
||||
- `define_database_entry_type(...)` emits Rust/C#-style slot-indexed
|
||||
MessagePack array payloads for cross-runtime `DatabaseEntry` contracts
|
||||
- `cultnet_py` exposes schema-v0 MessagePack helpers and 4-byte big-endian
|
||||
frame helpers for raw document state, subscriptions, shard catch-up,
|
||||
simulation observations, and witness artifact bundles
|
||||
- `cultmesh_py` exposes a cache-backed local node surface for Python tools
|
||||
|
||||
## Example
|
||||
|
||||
|
|
@ -144,382 +144,28 @@ That matches the Rust `#[derive(DatabaseEntry)]` formatter shape and the C#
|
|||
`[Key(n)]` intent: schema evolution comes from stable field slots, not source
|
||||
member order.
|
||||
|
||||
## CultNet And CultMesh
|
||||
|
||||
`cultnet_py` provides the Python schema-v0 wire helpers and an interop peer:
|
||||
|
||||
```python
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from cultnet_py import (
|
||||
CultNetSecret,
|
||||
CultNetServerSecurityOptions,
|
||||
CultNetRawClient,
|
||||
CultNetSimulationObservation,
|
||||
CultNetWitnessArtifactBundle,
|
||||
apply_raw_snapshot,
|
||||
apply_shard_log_response,
|
||||
compute_simulation_claim_hash,
|
||||
database_subscribe,
|
||||
decode_frame,
|
||||
encode_frame,
|
||||
hello,
|
||||
parse_message,
|
||||
shard_catalog_request,
|
||||
simulation_observation,
|
||||
witness_artifact_bundle,
|
||||
)
|
||||
from cultmesh_py import CultMesh
|
||||
|
||||
payload = hello(runtime_id="python-runtime").to_bytes()
|
||||
message = parse_message(decode_frame(encode_frame(payload)))
|
||||
|
||||
client = CultNetRawClient("127.0.0.1", 3075)
|
||||
catalog = client.fetch_schema_catalog(kinds=["wire_message"])
|
||||
snapshot = client.fetch_snapshot_response(schema_ids=["cultnet.interop-note"])
|
||||
shard_catalog = client.fetch_shard_catalog(schema_ids=["cultnet.interop-note"])
|
||||
# With a local CultCache and the matching registered document definitions:
|
||||
# applied = apply_raw_snapshot(cache, [note_doc], snapshot)
|
||||
|
||||
subscription = database_subscribe(subscription_id="ui", schema_ids=["cultnet.interop-note"])
|
||||
shard_request = shard_catalog_request(message_id="shards", schema_ids=["cultnet.interop-note"])
|
||||
shard_log = client.fetch_shard_log_response(shard_id="interop", shard_epoch=1, after_sequence=0)
|
||||
# log_changes = apply_shard_log_response(cache, [note_doc], shard_log)
|
||||
with client.subscribe_database(subscription_id="ui", schema_ids=["cultnet.interop-note"]) as live:
|
||||
initial_snapshot = live.read_next_snapshot_response()
|
||||
|
||||
# The same raw-client request surface can ride RUDP when the peer advertises a
|
||||
# rudp:// endpoint and shares the connection id for that lane.
|
||||
rudp_client = CultMesh.create_client(
|
||||
endpoint="rudp://127.0.0.1:3076",
|
||||
connection_id=0x43554C54,
|
||||
)
|
||||
rudp_catalog = rudp_client.fetch_schema_catalog(kinds=["wire_message"])
|
||||
|
||||
claim_hash = compute_simulation_claim_hash("frame:42", "subject:player-1", "hit")
|
||||
observation = simulation_observation(
|
||||
message_id="obs-1",
|
||||
witness_runtime_id="python-runtime",
|
||||
shard_id="interop",
|
||||
shard_epoch=1,
|
||||
frame=42,
|
||||
subject_id="player-1",
|
||||
claim_kind="hit",
|
||||
claim_hash=claim_hash,
|
||||
)
|
||||
typed_observation = CultNetSimulationObservation.from_wire(observation.to_wire())
|
||||
witness = witness_artifact_bundle(
|
||||
bundle_id="bundle-1",
|
||||
witness_kind="interop-proof",
|
||||
captured_at="2026-06-13T00:00:00Z",
|
||||
subject={"documentType": "cultnet.interop-note", "subjectId": "note:python"},
|
||||
contracts=[{"role": "payload", "schemaId": "cultnet.interop-note"}],
|
||||
artifacts=[{"role": "log", "uri": "cultcache://bundle-1/log", "mediaType": "text/plain"}],
|
||||
provenance={"pipelineId": "interop", "runId": "run-1", "runtimeId": "python-runtime"},
|
||||
)
|
||||
typed_witness = CultNetWitnessArtifactBundle.from_wire(witness)
|
||||
witness_payload = typed_witness.to_payload()
|
||||
|
||||
security = CultNetServerSecurityOptions.development()
|
||||
token = CultNetSecret.create_session_token(
|
||||
"318fb4b6-ff5e-4c4f-b911-d81807de53a8",
|
||||
datetime(2035, 1, 1, tzinfo=UTC),
|
||||
security,
|
||||
session_version=1,
|
||||
)
|
||||
session = CultNetSecret.try_validate_session_token(token, security)
|
||||
```
|
||||
|
||||
AES-GCM string/byte encryption helpers are available through the same
|
||||
`CultNetSecret` surface when the optional `crypto` extra is installed.
|
||||
|
||||
The peer can serve, dial, and probe the same raw-state interop lane used by the
|
||||
TypeScript, Rust, and C# test peers:
|
||||
|
||||
```powershell
|
||||
python -m cultnet_py.interop_peer serve --runtime-id python-peer --runtime-kind python --display-name "Python Peer" --agent-id python-agent --advertise-host 127.0.0.1 --tcp-port 3075 --discovery-port 4075 --discovery-group 239.77.44.11 --schema-path ..\cultnet-ts\integration\contracts\cultnet.interop-note.schema.json
|
||||
python -m cultnet_py.interop_peer dial --runtime-id python-client --runtime-kind python --display-name "Python Client" --agent-id python-client --target-host 127.0.0.1 --target-port 3075 --schema-path ..\cultnet-ts\integration\contracts\cultnet.interop-note.schema.json
|
||||
python -m cultnet_py.interop_peer probe --runtime-id python-prober --discovery-port 4075 --discovery-group 239.77.44.11
|
||||
```
|
||||
|
||||
`cultmesh_py` includes a local cache-backed node, schema-v0 helpers for the
|
||||
CultMesh Verse catalog and peer exchange wire messages, local and verified
|
||||
authority lease checks, stream transport negotiation, committed simulation fact
|
||||
documents, and local prediction reconciliation:
|
||||
|
||||
```python
|
||||
from datetime import UTC, datetime, timedelta
|
||||
|
||||
from cultcache_py import define_database_entry_type
|
||||
from cultnet_py import (
|
||||
CultNetClientAuthorityScope,
|
||||
CultNetFileShardReplicaCursorStore,
|
||||
CultNetRawClient,
|
||||
CultNetSchemaShardLogFetcher,
|
||||
CultNetSchemaShardSnapshotFetcher,
|
||||
CultNetShardDescriptor,
|
||||
CultNetShardReplicator,
|
||||
CultNetShardReplicatorOptions,
|
||||
)
|
||||
from cultmesh_py import (
|
||||
CultMesh,
|
||||
CultMeshAuthorityLease,
|
||||
CultMeshGameSessionOptions,
|
||||
CultMeshHmacAuthorityLeaseVerifier,
|
||||
CultMeshNodeOptions,
|
||||
CultMeshSnapshotFanout,
|
||||
peer_exchange_request,
|
||||
)
|
||||
|
||||
note_doc = define_database_entry_type(
|
||||
"mesh.note",
|
||||
[("body", 0), ("name", 1), ("kind", 2)],
|
||||
name="name",
|
||||
indexes={"kind": "kind"},
|
||||
)
|
||||
node = CultMesh.start_node(
|
||||
"mesh.cc",
|
||||
runtime_id="python-runtime",
|
||||
options=CultMeshNodeOptions(enable_durable_shard_logs=True),
|
||||
)
|
||||
node.database.register_document(note_doc)
|
||||
unsubscribe = node.database.watch_record(note_doc, "note:1", lambda change: print(change.change_kind))
|
||||
unsubscribe_named = node.database.watch_by_name(note_doc, "intro", lambda change: print(change.record_key))
|
||||
unsubscribe_kind = node.database.watch_by_index(note_doc, "kind", "guide", lambda change: print(change.record_key))
|
||||
node.database.put(note_doc, "note:1", {"body": "hello", "name": "intro", "kind": "guide"})
|
||||
live_note = node.database.get_required(note_doc, "note:1")
|
||||
put_message = node.database.put_raw_message(
|
||||
note_doc,
|
||||
"note:2",
|
||||
{"body": "wire me", "name": "wire", "kind": "interop"},
|
||||
shard_id="interop",
|
||||
shard_epoch=1,
|
||||
)
|
||||
snapshot_response = node.database.build_snapshot_response(schema_ids=[note_doc.catalog_entry().schema_id])
|
||||
snapshot_wire = node.database.create_snapshot_response(schema_ids=[note_doc.catalog_entry().schema_id])
|
||||
delete_message = node.database.delete_raw_message(note_doc, "note:2", shard_id="interop", shard_epoch=1)
|
||||
shard_log_response = node.database.build_shard_log_response(shard_id="interop", shard_epoch=1)
|
||||
shard_log_wire = node.database.create_shard_log_response(shard_id="interop", shard_epoch=1)
|
||||
|
||||
station_stock_doc = define_database_entry_type(
|
||||
"aetheria.station_stock",
|
||||
[("missiles", 0), ("coolant", 1)],
|
||||
schema_id="gamecult.aetheria.station_stock.v1",
|
||||
)
|
||||
station_stock_ui_doc = define_database_entry_type(
|
||||
"aetheria.station_stock.ui",
|
||||
[("missiles", 0), ("coolant", 1)],
|
||||
schema_id="gamecult.aetheria.station_stock.v1",
|
||||
)
|
||||
node.database.register_document(station_stock_doc)
|
||||
node.database.put(station_stock_ui_doc, "station:starbridge:stock", {"missiles": 24, "coolant": 80})
|
||||
stock = node.database.get_required(station_stock_ui_doc, "station:starbridge:stock")
|
||||
|
||||
peers = CultMesh.create_peer_catalog()
|
||||
response = peers.create_response(peer_exchange_request("pex-1", verse_id="local"))
|
||||
lease_verifier = CultMeshHmacAuthorityLeaseVerifier({"odin": b"shared-lease-key"})
|
||||
lease_catalog = CultMesh.create_authority_lease_catalog(
|
||||
signature_verifier=lease_verifier.verify,
|
||||
require_verified_signatures=True,
|
||||
)
|
||||
lease_valid_from = datetime.now(UTC)
|
||||
lease = lease_verifier.issue(CultMeshAuthorityLease(
|
||||
lease_id="lease:python-runtime",
|
||||
verse_id="local",
|
||||
peer_id="python-runtime",
|
||||
roles=("shard-primary",),
|
||||
valid_from=lease_valid_from,
|
||||
expires_at=lease_valid_from + timedelta(minutes=5),
|
||||
shard_ids=("interop",),
|
||||
issuer_runtime_id="odin",
|
||||
))
|
||||
lease_catalog.upsert(lease)
|
||||
session = CultMesh.create_game_session(
|
||||
node,
|
||||
CultMeshGameSessionOptions(
|
||||
client_authority_scopes=(
|
||||
CultNetClientAuthorityScope("python-runtime", schema_ids=(note_doc.catalog_entry().schema_id,), key_prefix="input:python"),
|
||||
),
|
||||
),
|
||||
)
|
||||
server = CultMesh.serve_node(
|
||||
node,
|
||||
peer_catalog=peers,
|
||||
observation_hub=session.observation_hub,
|
||||
port=3075,
|
||||
max_snapshot_documents=1000,
|
||||
max_snapshot_bytes=4 * 1024 * 1024,
|
||||
)
|
||||
```
|
||||
|
||||
The same local server can be launched as a package daemon when another runtime
|
||||
or operator process needs a CultNet endpoint without embedding Python glue:
|
||||
|
||||
```powershell
|
||||
python -m cultmesh_py.daemon --runtime-id python-runtime --display-name "Python Runtime" --host 127.0.0.1 --port 3075 --cache-file mesh.cc --enable-durable-shard-logs --seed-interop-note --seed-shard-id interop --verse-id local --role shard-primary --ready-file mesh.ready.json
|
||||
```
|
||||
|
||||
The ready file and stdout line are process readiness hints. The served state and
|
||||
capability truth still live on the framed CultNet/CultMesh endpoint. Use
|
||||
`--register-interop-note` to expose the package interop-note schema without
|
||||
seeding a note, or `--seed-interop-note` to register it and publish one local
|
||||
record for wire probes. Seeded records are committed through the raw mutation
|
||||
path, so peers can read them through both snapshot and shard-log catch-up. Add
|
||||
`--verse-id` and one or more `--role` values to publish the launched endpoint in
|
||||
its own Verse and peer catalogs for discovery clients. When the daemon is
|
||||
launched with `--cache-file`, `--enable-durable-shard-logs`, and
|
||||
`--shard-log-file`, a later daemon process can rehydrate and serve the persisted
|
||||
snapshot and shard log without reseeding.
|
||||
|
||||
`CultMeshPeerHealthMonitor` probes peer-card endpoints with `cultnet.hello.v0`
|
||||
and preserves the runtime id, display name, document types, message versions,
|
||||
and mutation contracts reported by the peer.
|
||||
|
||||
```python
|
||||
|
||||
client = CultMesh.create_verse_discovery_client("127.0.0.1", 3075)
|
||||
verses = client.fetch_verses(transport_version="cultmesh.v0")
|
||||
mesh_peers = client.fetch_peers(verse_id="python-interop", roles=["read-replica"])
|
||||
client.sync_peer_catalog(peers, verse_id="python-interop", roles=["read-replica"])
|
||||
client.fanout_peer_catalog(peers, verse_id="python-interop", roles=["read-replica"])
|
||||
replica_notes = client.sync_documents(
|
||||
node.database,
|
||||
peers,
|
||||
verse_id="python-interop",
|
||||
roles=["read-replica"],
|
||||
documents=[(note_doc, "note:remote")],
|
||||
)
|
||||
snapshot_fanout = CultMeshSnapshotFanout(
|
||||
client,
|
||||
node.database,
|
||||
peers,
|
||||
verse_id="python-interop",
|
||||
roles=["read-replica"],
|
||||
documents=[(note_doc, "note:remote")],
|
||||
on_document=lambda note: print(note),
|
||||
)
|
||||
snapshot_fanout.sync_once()
|
||||
|
||||
raw_client = CultNetRawClient("127.0.0.1", 3075)
|
||||
synced_note = node.database.sync_document(raw_client, note_doc, "note:remote")
|
||||
synced_stock = CultMesh.sync_document(
|
||||
node,
|
||||
raw_client,
|
||||
station_stock_ui_doc,
|
||||
"station:starbridge:stock",
|
||||
)
|
||||
with CultMesh.subscribe_document(
|
||||
node,
|
||||
raw_client,
|
||||
station_stock_ui_doc,
|
||||
"station:starbridge:stock",
|
||||
) as stock_subscription:
|
||||
stock = stock_subscription.sync_initial()
|
||||
next_change = stock_subscription.read_next_change()
|
||||
node.database.sync_shard_log(raw_client, shard_id="interop", shard_epoch=1)
|
||||
replicator = CultNetShardReplicator(
|
||||
node.database,
|
||||
CultNetShardReplicatorOptions(
|
||||
fetcher=CultNetSchemaShardLogFetcher(),
|
||||
snapshot_fetcher=CultNetSchemaShardSnapshotFetcher(),
|
||||
cursor_store=CultNetFileShardReplicaCursorStore("mesh.replica-cursors.msgpack"),
|
||||
poll_interval_seconds=1.0,
|
||||
on_error=lambda error: print(error),
|
||||
),
|
||||
)
|
||||
replica_shard = CultNetShardDescriptor(
|
||||
shard_id="interop",
|
||||
owner_runtime_id="primary-runtime",
|
||||
epoch=1,
|
||||
schema_ids=(note_doc.catalog_entry().schema_id,),
|
||||
primary_endpoints=("cultnet://127.0.0.1:3075",),
|
||||
)
|
||||
replicator.pull_once(replica_shard)
|
||||
replicator.start([replica_shard])
|
||||
replicator.stop()
|
||||
server.stop()
|
||||
|
||||
streams = CultMesh.create_stream_catalog()
|
||||
|
||||
claim_hash = "accepted-claim-hash"
|
||||
facts = CultMesh.create_simulation_fact_committer(node)
|
||||
committed = facts.commit({
|
||||
"shardId": "arena",
|
||||
"shardEpoch": 4,
|
||||
"frame": 100,
|
||||
"subjectId": "bob",
|
||||
"claimKind": "hit",
|
||||
"claimHash": claim_hash,
|
||||
"witnessCount": 2,
|
||||
"supportWeight": 2.0,
|
||||
"totalWeight": 2.0,
|
||||
"confidence": 1.0,
|
||||
"hasQuorum": True,
|
||||
})
|
||||
session_commits = session.submit_and_commit(typed_observation)
|
||||
prediction = session.predict(note_doc, "input:python:move", {"body": "predicted input"})
|
||||
unsubscribe()
|
||||
```
|
||||
|
||||
## Wire Parity
|
||||
|
||||
See [PARITY.md](PARITY.md) for the current evidence map, gates, and things this
|
||||
package still does not claim.
|
||||
See [docs/python-runtime-parity.md](../../docs/python-runtime-parity.md) for
|
||||
the evidence map across the three Python packages.
|
||||
|
||||
The Python interop peer is `cultcache_py.interop`:
|
||||
The store interop peer is `cultcache_py.interop`:
|
||||
|
||||
```powershell
|
||||
python -m cultcache_py.interop write --file cache.cc --runtime-id python
|
||||
python -m cultcache_py.interop read --file cache.cc
|
||||
```
|
||||
|
||||
Current receipts:
|
||||
`packages/cultcache-ts/test/cult-cache.test.ts` includes Python in the shared
|
||||
CultCache v1 parity matrix with TypeScript, Rust, and C#. `cultcache_py` ships
|
||||
a `py.typed` marker so downstream type checkers can inspect the package surface.
|
||||
|
||||
- `packages/cultcache-ts/test/cult-cache.test.ts` includes Python in the shared
|
||||
CultCache v1 parity matrix with TypeScript, Rust, and C#.
|
||||
- `packages/cultnet-ts/test/interop/cultnet-interop.test.ts` includes Python in
|
||||
the live TS/Rust/C#/Python schema-v0 peer ring: discovery, hello, schema
|
||||
catalog, wire-message catalog discovery, raw snapshot, raw document
|
||||
put/delete, mutation receipt, fire-command receipt, database subscription
|
||||
changes, shard catalog, shard log catch-up, simulation consensus candidates,
|
||||
committed simulation fact snapshots, Python session prediction reconciliation,
|
||||
and witness artifact bundle round-trips. The same test asks the Python peer
|
||||
for CultMesh Verse catalog and peer exchange responses over the CultNet pipe,
|
||||
then verifies the public `CultMeshDiscoveryClient` can fetch typed Python
|
||||
Verse and peer descriptors from that live endpoint. It also verifies
|
||||
`CultNetRawClient` can fetch the live Python peer's schema catalog, raw
|
||||
snapshot, and shard catalog.
|
||||
- `packages/cultcache-py/tests/test_cultcache.py` covers Python CultMesh
|
||||
Verse catalog, peer exchange, authority lease, stream negotiation, CultNet
|
||||
helper shapes, raw snapshot/shard-log application, simulation claim hashing,
|
||||
consensus aggregation, game-session fact commits, local prediction
|
||||
reconciliation, committed simulation fact payload slots, the local CultMesh
|
||||
server's TCP and RUDP schema request paths, and witness artifact bundle
|
||||
payload slots.
|
||||
- `cultcache_py`, `cultnet_py`, and `cultmesh_py` ship `py.typed` markers so
|
||||
downstream type checkers can inspect the package surface instead of treating
|
||||
the runtime as an untyped xenos swamp.
|
||||
|
||||
## Performance Baseline
|
||||
|
||||
The package ships a lightweight benchmark for Python-owned hot paths. In a
|
||||
CultLib source checkout, it can also run the C# reference benchmark beside it:
|
||||
## Tests
|
||||
|
||||
```powershell
|
||||
python -m cultcache_py.benchmark --records 1000 --json
|
||||
python -m cultcache_py.compare_csharp --records 1000 --json
|
||||
python -m cultcache_py.verify --json
|
||||
$env:PYTHONPATH="$PWD\packages\cultcache-py\src"
|
||||
python -m unittest discover -s packages\cultcache-py\tests
|
||||
```
|
||||
|
||||
It reports slot-indexed `DatabaseEntry` encode/decode throughput, framed
|
||||
CultNet MessagePack parse throughput, raw snapshot application into a
|
||||
registered cache, and public `CultCache` upsert/get throughput. The C# compare
|
||||
command also runs `packages/cultcache-py/tools/GameCult.Caching.Benchmark` and
|
||||
reports median Python-to-C# ratios for the shared public cache operations. It
|
||||
uses three samples by default; pass `--samples` when you need a different
|
||||
evidence shape. Both runtimes precompute record keys before measured cache
|
||||
upsert/get loops so the comparison is about public cache access, not string
|
||||
construction noise. The public in-memory cache owner stores records in
|
||||
type-partitioned dictionaries so hot `get(...)` calls avoid per-call
|
||||
composite-key allocation. Treat the benchmark numbers as a local baseline, not
|
||||
a claim that Python matches the C# reference in every workload.
|
||||
The suite runs with only this package on the path. If a test here needs
|
||||
`cultnet_py` or `cultmesh_py`, it belongs in that package.
|
||||
|
|
|
|||
|
|
@ -4,8 +4,8 @@ build-backend = "setuptools.build_meta"
|
|||
|
||||
[project]
|
||||
name = "cultcache-py"
|
||||
version = "0.1.0"
|
||||
description = "Python CultCache, CultNet, and CultMesh runtime package for GameCult typed state."
|
||||
version = "0.2.0"
|
||||
description = "Python CultCache runtime for GameCult typed state."
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.10"
|
||||
license = "MIT"
|
||||
|
|
@ -18,23 +18,11 @@ classifiers = [
|
|||
"Programming Language :: Python :: 3.12",
|
||||
]
|
||||
dependencies = ["msgpack>=1.0"]
|
||||
|
||||
[project.optional-dependencies]
|
||||
msgpack = ["msgpack>=1.0"]
|
||||
crypto = ["cryptography>=42"]
|
||||
|
||||
[project.scripts]
|
||||
cultcache-py-interop = "cultcache_py.interop:main"
|
||||
cultnet-py-interop = "cultnet_py.interop_peer:main"
|
||||
cultcache-py-benchmark = "cultcache_py.benchmark:main"
|
||||
cultcache-py-compare-csharp = "cultcache_py.compare_csharp:main"
|
||||
cultcache-py-verify = "cultcache_py.verify:main"
|
||||
cultmesh-py-daemon = "cultmesh_py.daemon:main"
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
where = ["src"]
|
||||
|
||||
[tool.setuptools.package-data]
|
||||
cultcache_py = ["py.typed"]
|
||||
cultnet_py = ["py.typed"]
|
||||
cultmesh_py = ["py.typed"]
|
||||
|
|
|
|||
|
|
@ -1,135 +0,0 @@
|
|||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from datetime import UTC, datetime
|
||||
import hashlib
|
||||
from typing import Any
|
||||
|
||||
from cultcache_py import define_database_entry_type
|
||||
from cultcache_py.documents import DocumentDefinition
|
||||
|
||||
from .node import CultMeshNode
|
||||
|
||||
SIMULATION_FACT_DOCUMENT_TYPE = "gamecult.mesh.simulation_fact"
|
||||
SIMULATION_FACT_SCHEMA_VERSION = "gamecult.mesh.simulation_fact.v1"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CultMeshSimulationFact:
|
||||
fact_id: str
|
||||
shard_id: str
|
||||
shard_epoch: int
|
||||
frame: int
|
||||
subject_id: str
|
||||
claim_kind: str
|
||||
claim_hash: str
|
||||
claim_summary: str | None
|
||||
witness_count: int
|
||||
support_weight: float
|
||||
total_weight: float
|
||||
confidence: float
|
||||
committed_at: str
|
||||
|
||||
@staticmethod
|
||||
def compute_fact_id(candidate: dict[str, Any]) -> str:
|
||||
canonical = "\x1f".join([
|
||||
_candidate_string(candidate, "shardId", "shard_id"),
|
||||
str(_candidate_int(candidate, "shardEpoch", "shard_epoch")),
|
||||
str(_candidate_int(candidate, "frame", "frame")),
|
||||
_candidate_string(candidate, "subjectId", "subject_id"),
|
||||
_candidate_string(candidate, "claimKind", "claim_kind"),
|
||||
])
|
||||
return hashlib.sha256(canonical.encode("utf-8")).hexdigest()
|
||||
|
||||
@staticmethod
|
||||
def create_record_key(candidate: dict[str, Any]) -> str:
|
||||
return f"simulation:{CultMeshSimulationFact.compute_fact_id(candidate)}"
|
||||
|
||||
@staticmethod
|
||||
def from_candidate(candidate: dict[str, Any], *, committed_at: str | None = None) -> "CultMeshSimulationFact":
|
||||
return CultMeshSimulationFact(
|
||||
fact_id=CultMeshSimulationFact.compute_fact_id(candidate),
|
||||
shard_id=_candidate_string(candidate, "shardId", "shard_id"),
|
||||
shard_epoch=_candidate_int(candidate, "shardEpoch", "shard_epoch"),
|
||||
frame=_candidate_int(candidate, "frame", "frame"),
|
||||
subject_id=_candidate_string(candidate, "subjectId", "subject_id"),
|
||||
claim_kind=_candidate_string(candidate, "claimKind", "claim_kind"),
|
||||
claim_hash=_candidate_string(candidate, "claimHash", "claim_hash"),
|
||||
claim_summary=_candidate_optional_string(candidate, "claimSummary", "claim_summary"),
|
||||
witness_count=_candidate_int(candidate, "witnessCount", "witness_count"),
|
||||
support_weight=_candidate_float(candidate, "supportWeight", "support_weight"),
|
||||
total_weight=_candidate_float(candidate, "totalWeight", "total_weight"),
|
||||
confidence=_candidate_float(candidate, "confidence", "confidence"),
|
||||
committed_at=committed_at or datetime.now(UTC).isoformat(),
|
||||
)
|
||||
|
||||
|
||||
simulation_fact_document: DocumentDefinition[CultMeshSimulationFact] = define_database_entry_type(
|
||||
SIMULATION_FACT_DOCUMENT_TYPE,
|
||||
[
|
||||
("fact_id", 0),
|
||||
("shard_id", 1),
|
||||
("shard_epoch", 2),
|
||||
("frame", 3),
|
||||
("subject_id", 4),
|
||||
("claim_kind", 5),
|
||||
("claim_hash", 6),
|
||||
("claim_summary", 7, None),
|
||||
("witness_count", 8),
|
||||
("support_weight", 9),
|
||||
("total_weight", 10),
|
||||
("confidence", 11),
|
||||
("committed_at", 12),
|
||||
],
|
||||
cls=CultMeshSimulationFact,
|
||||
name="fact_id",
|
||||
indexes={"shard": "shard_id", "frame": "frame", "subject": "subject_id", "claim_kind": "claim_kind"},
|
||||
schema_name=SIMULATION_FACT_DOCUMENT_TYPE,
|
||||
schema_version=SIMULATION_FACT_SCHEMA_VERSION,
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CultMeshSimulationFactCommit:
|
||||
key: str
|
||||
fact: CultMeshSimulationFact
|
||||
|
||||
|
||||
@dataclass
|
||||
class CultMeshSimulationFactCommitter:
|
||||
node: CultMeshNode
|
||||
document: DocumentDefinition[CultMeshSimulationFact] = simulation_fact_document
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
if all(document.type != self.document.type for document in self.node.documents):
|
||||
self.node.register_document(self.document)
|
||||
|
||||
def commit(self, candidate: dict[str, Any], *, committed_at: str | None = None) -> CultMeshSimulationFactCommit:
|
||||
if not bool(candidate.get("hasQuorum") if "hasQuorum" in candidate else candidate.get("has_quorum")):
|
||||
raise ValueError("Simulation candidate cannot be committed before quorum")
|
||||
key = CultMeshSimulationFact.create_record_key(candidate)
|
||||
fact = CultMeshSimulationFact.from_candidate(candidate, committed_at=committed_at)
|
||||
self.node.put(self.document, key, fact)
|
||||
return CultMeshSimulationFactCommit(key=key, fact=fact)
|
||||
|
||||
|
||||
def _candidate_string(candidate: dict[str, Any], camel: str, snake: str) -> str:
|
||||
value = candidate.get(camel, candidate.get(snake, ""))
|
||||
if value is None:
|
||||
return ""
|
||||
return str(value)
|
||||
|
||||
|
||||
def _candidate_optional_string(candidate: dict[str, Any], camel: str, snake: str) -> str | None:
|
||||
value = candidate.get(camel, candidate.get(snake))
|
||||
if value is None:
|
||||
return None
|
||||
return str(value)
|
||||
|
||||
|
||||
def _candidate_int(candidate: dict[str, Any], camel: str, snake: str) -> int:
|
||||
return int(candidate.get(camel, candidate.get(snake, 0)))
|
||||
|
||||
|
||||
def _candidate_float(candidate: dict[str, Any], camel: str, snake: str) -> float:
|
||||
return float(candidate.get(camel, candidate.get(snake, 0.0)))
|
||||
File diff suppressed because it is too large
Load diff
|
|
@ -3,7 +3,7 @@ import { exec, execFile } from "node:child_process";
|
|||
import { existsSync } from "node:fs";
|
||||
import { access, mkdtemp, readFile, writeFile } from "node:fs/promises";
|
||||
import { homedir, tmpdir } from "node:os";
|
||||
import { dirname, join, resolve } from "node:path";
|
||||
import { dirname, join, resolve, delimiter } from "node:path";
|
||||
import { test } from "node:test";
|
||||
import { promisify } from "node:util";
|
||||
|
||||
|
|
@ -24,7 +24,9 @@ const pythonCommand = process.env.PYTHON ?? "python";
|
|||
const cultCacheTsRoot = resolve(__dirname, "../..");
|
||||
const cultLibRoot = findAncestor(cultCacheTsRoot, "CultLib.sln") ?? resolve(cultCacheTsRoot, "..", "CultLib");
|
||||
const cultcachePyRoot = resolve(cultLibRoot, "packages", "cultcache-py");
|
||||
const cultcachePySrc = resolve(cultcachePyRoot, "src");
|
||||
const cultcachePySrc = ["cultcache-py", "cultnet-py", "cultmesh-py"]
|
||||
.map((name) => resolve(cultLibRoot, "packages", name, "src"))
|
||||
.join(delimiter);
|
||||
const cultcacheRsRoot = existsSync(resolve(cultLibRoot, "packages", "cultcache-rs"))
|
||||
? resolve(cultLibRoot, "packages", "cultcache-rs")
|
||||
: resolve(cultCacheTsRoot, "..", "cultcache-rs");
|
||||
|
|
|
|||
252
packages/cultmesh-py/README.md
Normal file
252
packages/cultmesh-py/README.md
Normal file
|
|
@ -0,0 +1,252 @@
|
|||
# cultmesh-py
|
||||
|
||||
`cultmesh-py` is the Python CultMesh runtime: a cache-backed local node,
|
||||
reactive and observed documents, the branded `CultMesh` facade, Verse and peer
|
||||
discovery clients, snapshot and observation fan-out, peer health monitoring,
|
||||
game sessions with prediction reconciliation, committed simulation facts, the
|
||||
local TCP/RUDP server, and a package daemon.
|
||||
|
||||
It depends on `cultcache-py` and `cultnet-py`, mirroring `cultmesh-rs` and
|
||||
`cultmesh-ts`. The Verse catalog, peer exchange, authority lease, stream
|
||||
negotiation, and simulation fact contracts are defined in
|
||||
`cultnet_py.cultmesh_contracts` and re-exported here.
|
||||
|
||||
This is the package a Python process embeds to serve a Verse — the Brokkr
|
||||
Blender add-on serves its editor mirror through `CultMesh.serve_node(...)`
|
||||
from inside Blender's own interpreter.
|
||||
|
||||
## Example
|
||||
|
||||
```python
|
||||
from datetime import UTC, datetime, timedelta
|
||||
|
||||
from cultcache_py import define_database_entry_type
|
||||
from cultnet_py import (
|
||||
CultNetClientAuthorityScope,
|
||||
CultNetFileShardReplicaCursorStore,
|
||||
CultNetRawClient,
|
||||
CultNetSchemaShardLogFetcher,
|
||||
CultNetSchemaShardSnapshotFetcher,
|
||||
CultNetShardDescriptor,
|
||||
CultNetShardReplicator,
|
||||
CultNetShardReplicatorOptions,
|
||||
)
|
||||
from cultmesh_py import (
|
||||
CultMesh,
|
||||
CultMeshAuthorityLease,
|
||||
CultMeshGameSessionOptions,
|
||||
CultMeshHmacAuthorityLeaseVerifier,
|
||||
CultMeshNodeOptions,
|
||||
CultMeshSnapshotFanout,
|
||||
peer_exchange_request,
|
||||
)
|
||||
|
||||
note_doc = define_database_entry_type(
|
||||
"mesh.note",
|
||||
[("body", 0), ("name", 1), ("kind", 2)],
|
||||
name="name",
|
||||
indexes={"kind": "kind"},
|
||||
)
|
||||
node = CultMesh.start_node(
|
||||
"mesh.cc",
|
||||
runtime_id="python-runtime",
|
||||
options=CultMeshNodeOptions(enable_durable_shard_logs=True),
|
||||
)
|
||||
node.database.register_document(note_doc)
|
||||
unsubscribe = node.database.watch_record(note_doc, "note:1", lambda change: print(change.change_kind))
|
||||
unsubscribe_named = node.database.watch_by_name(note_doc, "intro", lambda change: print(change.record_key))
|
||||
unsubscribe_kind = node.database.watch_by_index(note_doc, "kind", "guide", lambda change: print(change.record_key))
|
||||
node.database.put(note_doc, "note:1", {"body": "hello", "name": "intro", "kind": "guide"})
|
||||
live_note = node.database.get_required(note_doc, "note:1")
|
||||
put_message = node.database.put_raw_message(
|
||||
note_doc,
|
||||
"note:2",
|
||||
{"body": "wire me", "name": "wire", "kind": "interop"},
|
||||
shard_id="interop",
|
||||
shard_epoch=1,
|
||||
)
|
||||
snapshot_response = node.database.build_snapshot_response(schema_ids=[note_doc.catalog_entry().schema_id])
|
||||
snapshot_wire = node.database.create_snapshot_response(schema_ids=[note_doc.catalog_entry().schema_id])
|
||||
delete_message = node.database.delete_raw_message(note_doc, "note:2", shard_id="interop", shard_epoch=1)
|
||||
shard_log_response = node.database.build_shard_log_response(shard_id="interop", shard_epoch=1)
|
||||
shard_log_wire = node.database.create_shard_log_response(shard_id="interop", shard_epoch=1)
|
||||
|
||||
station_stock_doc = define_database_entry_type(
|
||||
"aetheria.station_stock",
|
||||
[("missiles", 0), ("coolant", 1)],
|
||||
schema_id="gamecult.aetheria.station_stock.v1",
|
||||
)
|
||||
station_stock_ui_doc = define_database_entry_type(
|
||||
"aetheria.station_stock.ui",
|
||||
[("missiles", 0), ("coolant", 1)],
|
||||
schema_id="gamecult.aetheria.station_stock.v1",
|
||||
)
|
||||
node.database.register_document(station_stock_doc)
|
||||
node.database.put(station_stock_ui_doc, "station:starbridge:stock", {"missiles": 24, "coolant": 80})
|
||||
stock = node.database.get_required(station_stock_ui_doc, "station:starbridge:stock")
|
||||
|
||||
peers = CultMesh.create_peer_catalog()
|
||||
response = peers.create_response(peer_exchange_request("pex-1", verse_id="local"))
|
||||
lease_verifier = CultMeshHmacAuthorityLeaseVerifier({"odin": b"shared-lease-key"})
|
||||
lease_catalog = CultMesh.create_authority_lease_catalog(
|
||||
signature_verifier=lease_verifier.verify,
|
||||
require_verified_signatures=True,
|
||||
)
|
||||
lease_valid_from = datetime.now(UTC)
|
||||
lease = lease_verifier.issue(CultMeshAuthorityLease(
|
||||
lease_id="lease:python-runtime",
|
||||
verse_id="local",
|
||||
peer_id="python-runtime",
|
||||
roles=("shard-primary",),
|
||||
valid_from=lease_valid_from,
|
||||
expires_at=lease_valid_from + timedelta(minutes=5),
|
||||
shard_ids=("interop",),
|
||||
issuer_runtime_id="odin",
|
||||
))
|
||||
lease_catalog.upsert(lease)
|
||||
session = CultMesh.create_game_session(
|
||||
node,
|
||||
CultMeshGameSessionOptions(
|
||||
client_authority_scopes=(
|
||||
CultNetClientAuthorityScope("python-runtime", schema_ids=(note_doc.catalog_entry().schema_id,), key_prefix="input:python"),
|
||||
),
|
||||
),
|
||||
)
|
||||
server = CultMesh.serve_node(
|
||||
node,
|
||||
peer_catalog=peers,
|
||||
observation_hub=session.observation_hub,
|
||||
port=3075,
|
||||
max_snapshot_documents=1000,
|
||||
max_snapshot_bytes=4 * 1024 * 1024,
|
||||
)
|
||||
```
|
||||
|
||||
The same local server can be launched as a package daemon when another runtime
|
||||
or operator process needs a CultNet endpoint without embedding Python glue:
|
||||
|
||||
```powershell
|
||||
python -m cultmesh_py.daemon --runtime-id python-runtime --display-name "Python Runtime" --host 127.0.0.1 --port 3075 --cache-file mesh.cc --enable-durable-shard-logs --seed-interop-note --seed-shard-id interop --verse-id local --role shard-primary --ready-file mesh.ready.json
|
||||
```
|
||||
|
||||
The ready file and stdout line are process readiness hints. The served state and
|
||||
capability truth still live on the framed CultNet/CultMesh endpoint. Use
|
||||
`--register-interop-note` to expose the package interop-note schema without
|
||||
seeding a note, or `--seed-interop-note` to register it and publish one local
|
||||
record for wire probes. Seeded records are committed through the raw mutation
|
||||
path, so peers can read them through both snapshot and shard-log catch-up. Add
|
||||
`--verse-id` and one or more `--role` values to publish the launched endpoint in
|
||||
its own Verse and peer catalogs for discovery clients. When the daemon is
|
||||
launched with `--cache-file`, `--enable-durable-shard-logs`, and
|
||||
`--shard-log-file`, a later daemon process can rehydrate and serve the persisted
|
||||
snapshot and shard log without reseeding.
|
||||
|
||||
`CultMeshPeerHealthMonitor` probes peer-card endpoints with `cultnet.hello.v0`
|
||||
and preserves the runtime id, display name, document types, message versions,
|
||||
and mutation contracts reported by the peer.
|
||||
|
||||
```python
|
||||
client = CultMesh.create_verse_discovery_client("127.0.0.1", 3075)
|
||||
verses = client.fetch_verses(transport_version="cultmesh.v0")
|
||||
mesh_peers = client.fetch_peers(verse_id="python-interop", roles=["read-replica"])
|
||||
client.sync_peer_catalog(peers, verse_id="python-interop", roles=["read-replica"])
|
||||
client.fanout_peer_catalog(peers, verse_id="python-interop", roles=["read-replica"])
|
||||
replica_notes = client.sync_documents(
|
||||
node.database,
|
||||
peers,
|
||||
verse_id="python-interop",
|
||||
roles=["read-replica"],
|
||||
documents=[(note_doc, "note:remote")],
|
||||
)
|
||||
snapshot_fanout = CultMeshSnapshotFanout(
|
||||
client,
|
||||
node.database,
|
||||
peers,
|
||||
verse_id="python-interop",
|
||||
roles=["read-replica"],
|
||||
documents=[(note_doc, "note:remote")],
|
||||
on_document=lambda note: print(note),
|
||||
)
|
||||
snapshot_fanout.sync_once()
|
||||
|
||||
raw_client = CultNetRawClient("127.0.0.1", 3075)
|
||||
synced_note = node.database.sync_document(raw_client, note_doc, "note:remote")
|
||||
synced_stock = CultMesh.sync_document(
|
||||
node,
|
||||
raw_client,
|
||||
station_stock_ui_doc,
|
||||
"station:starbridge:stock",
|
||||
)
|
||||
with CultMesh.subscribe_document(
|
||||
node,
|
||||
raw_client,
|
||||
station_stock_ui_doc,
|
||||
"station:starbridge:stock",
|
||||
) as stock_subscription:
|
||||
stock = stock_subscription.sync_initial()
|
||||
next_change = stock_subscription.read_next_change()
|
||||
node.database.sync_shard_log(raw_client, shard_id="interop", shard_epoch=1)
|
||||
replicator = CultNetShardReplicator(
|
||||
node.database,
|
||||
CultNetShardReplicatorOptions(
|
||||
fetcher=CultNetSchemaShardLogFetcher(),
|
||||
snapshot_fetcher=CultNetSchemaShardSnapshotFetcher(),
|
||||
cursor_store=CultNetFileShardReplicaCursorStore("mesh.replica-cursors.msgpack"),
|
||||
poll_interval_seconds=1.0,
|
||||
on_error=lambda error: print(error),
|
||||
),
|
||||
)
|
||||
replica_shard = CultNetShardDescriptor(
|
||||
shard_id="interop",
|
||||
owner_runtime_id="primary-runtime",
|
||||
epoch=1,
|
||||
schema_ids=(note_doc.catalog_entry().schema_id,),
|
||||
primary_endpoints=("cultnet://127.0.0.1:3075",),
|
||||
)
|
||||
replicator.pull_once(replica_shard)
|
||||
replicator.start([replica_shard])
|
||||
replicator.stop()
|
||||
server.stop()
|
||||
|
||||
streams = CultMesh.create_stream_catalog()
|
||||
|
||||
facts = CultMesh.create_simulation_fact_committer(node)
|
||||
committed = facts.commit({
|
||||
"shardId": "arena",
|
||||
"shardEpoch": 4,
|
||||
"frame": 100,
|
||||
"subjectId": "bob",
|
||||
"claimKind": "hit",
|
||||
"claimHash": "accepted-claim-hash",
|
||||
"witnessCount": 2,
|
||||
"supportWeight": 2.0,
|
||||
"totalWeight": 2.0,
|
||||
"confidence": 1.0,
|
||||
"hasQuorum": True,
|
||||
})
|
||||
prediction = session.predict(note_doc, "input:python:move", {"body": "predicted input"})
|
||||
unsubscribe()
|
||||
```
|
||||
|
||||
## Verify
|
||||
|
||||
`cultmesh_py.verify` checks the public exports and `py.typed` markers of all
|
||||
three Python packages, smokes a local node over the wire, checks the served
|
||||
capability truth, and runs the benchmark for sanity:
|
||||
|
||||
```powershell
|
||||
python -m cultmesh_py.verify --json
|
||||
```
|
||||
|
||||
## Tests
|
||||
|
||||
```powershell
|
||||
$env:PYTHONPATH="$PWD\packages\cultcache-py\src;$PWD\packages\cultnet-py\src;$PWD\packages\cultmesh-py\src"
|
||||
python -m unittest discover -s packages\cultmesh-py\tests
|
||||
```
|
||||
|
||||
`packages\cultnet-ts\test\interop\cultnet-interop.test.ts` verifies the public
|
||||
`CultMeshDiscoveryClient` can fetch typed Verse and peer descriptors from the
|
||||
live Python peer. See
|
||||
[docs/python-runtime-parity.md](../../docs/python-runtime-parity.md) for the
|
||||
evidence map across the three Python packages.
|
||||
29
packages/cultmesh-py/pyproject.toml
Normal file
29
packages/cultmesh-py/pyproject.toml
Normal file
|
|
@ -0,0 +1,29 @@
|
|||
[build-system]
|
||||
requires = ["setuptools>=68"]
|
||||
build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "cultmesh-py"
|
||||
version = "0.1.0"
|
||||
description = "Python CultMesh node, discovery, sessions, and local server for GameCult typed state."
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.10"
|
||||
license = "MIT"
|
||||
authors = [{ name = "GameCult" }]
|
||||
classifiers = [
|
||||
"License :: OSI Approved :: MIT License",
|
||||
"Programming Language :: Python :: 3",
|
||||
"Programming Language :: Python :: 3.10",
|
||||
"Programming Language :: Python :: 3.11",
|
||||
"Programming Language :: Python :: 3.12",
|
||||
]
|
||||
dependencies = ["cultcache-py>=0.2.0", "cultnet-py>=0.1.0"]
|
||||
[project.scripts]
|
||||
cultmesh-py-daemon = "cultmesh_py.daemon:main"
|
||||
cultmesh-py-verify = "cultmesh_py.verify:main"
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
where = ["src"]
|
||||
|
||||
[tool.setuptools.package-data]
|
||||
cultmesh_py = ["py.typed"]
|
||||
|
|
@ -39,7 +39,7 @@ from .simulation import (
|
|||
CultMeshSimulationFactCommitter,
|
||||
simulation_fact_document,
|
||||
)
|
||||
from .wire import (
|
||||
from cultnet_py.cultmesh_contracts import (
|
||||
PEER_EXCHANGE_REQUEST,
|
||||
PEER_EXCHANGE_RESPONSE,
|
||||
VERSE_CATALOG_REQUEST,
|
||||
|
|
@ -9,7 +9,7 @@ from urllib.parse import urlparse
|
|||
from cultcache_py.documents import DocumentDefinition
|
||||
from cultnet_py import CultNetDatabaseSubscription, CultNetRawClient, CultNetRawSnapshotResponse, hello
|
||||
|
||||
from .wire import (
|
||||
from cultnet_py.cultmesh_contracts import (
|
||||
PEER_EXCHANGE_RESPONSE,
|
||||
VERSE_CATALOG_RESPONSE,
|
||||
CultMeshPeerCard,
|
||||
|
|
@ -13,7 +13,7 @@ from cultnet_py import CultNetSimulationConsensusOptions, CultNetSimulationObser
|
|||
|
||||
from .facade import CultMesh
|
||||
from .server import CultMeshLocalServer
|
||||
from .wire import (
|
||||
from cultnet_py.cultmesh_contracts import (
|
||||
CultMeshPeerCard,
|
||||
CultMeshPeerCatalog,
|
||||
CultMeshVerseCatalog,
|
||||
|
|
@ -34,7 +34,7 @@ from .node import (
|
|||
from .server import CultMeshLocalServer
|
||||
from .session import CultMeshGameSession, CultMeshGameSessionOptions
|
||||
from .simulation import CultMeshSimulationFactCommitter
|
||||
from .wire import (
|
||||
from cultnet_py.cultmesh_contracts import (
|
||||
AuthorityLeaseVerifier,
|
||||
CultMeshAuthorityLeaseCatalog,
|
||||
CultMeshPeerCard,
|
||||
|
|
@ -24,7 +24,7 @@ from cultnet_py import (
|
|||
)
|
||||
|
||||
from .node import CultMeshNode
|
||||
from .wire import (
|
||||
from cultnet_py.cultmesh_contracts import (
|
||||
PEER_EXCHANGE_REQUEST,
|
||||
VERSE_CATALOG_REQUEST,
|
||||
CultMeshPeerCatalog,
|
||||
|
|
@ -23,7 +23,7 @@ from .simulation import (
|
|||
CultMeshSimulationFactCommitter,
|
||||
simulation_fact_document,
|
||||
)
|
||||
from .wire import CultMeshAuthorityLeaseCatalog, CultMeshPeerCatalog, CultMeshVerseCatalog
|
||||
from cultnet_py.cultmesh_contracts import CultMeshAuthorityLeaseCatalog, CultMeshPeerCatalog, CultMeshVerseCatalog
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from .server import CultMeshLocalServer
|
||||
47
packages/cultmesh-py/src/cultmesh_py/simulation.py
Normal file
47
packages/cultmesh-py/src/cultmesh_py/simulation.py
Normal file
|
|
@ -0,0 +1,47 @@
|
|||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
from cultcache_py.documents import DocumentDefinition
|
||||
from cultnet_py.cultmesh_contracts import (
|
||||
SIMULATION_FACT_DOCUMENT_TYPE,
|
||||
SIMULATION_FACT_SCHEMA_VERSION,
|
||||
CultMeshSimulationFact,
|
||||
simulation_fact_document,
|
||||
)
|
||||
|
||||
from .node import CultMeshNode
|
||||
|
||||
__all__ = [
|
||||
"SIMULATION_FACT_DOCUMENT_TYPE",
|
||||
"SIMULATION_FACT_SCHEMA_VERSION",
|
||||
"CultMeshSimulationFact",
|
||||
"CultMeshSimulationFactCommit",
|
||||
"CultMeshSimulationFactCommitter",
|
||||
"simulation_fact_document",
|
||||
]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CultMeshSimulationFactCommit:
|
||||
key: str
|
||||
fact: CultMeshSimulationFact
|
||||
|
||||
|
||||
@dataclass
|
||||
class CultMeshSimulationFactCommitter:
|
||||
node: CultMeshNode
|
||||
document: DocumentDefinition[CultMeshSimulationFact] = simulation_fact_document
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
if all(document.type != self.document.type for document in self.node.documents):
|
||||
self.node.register_document(self.document)
|
||||
|
||||
def commit(self, candidate: dict[str, Any], *, committed_at: str | None = None) -> CultMeshSimulationFactCommit:
|
||||
if not bool(candidate.get("hasQuorum") if "hasQuorum" in candidate else candidate.get("has_quorum")):
|
||||
raise ValueError("Simulation candidate cannot be committed before quorum")
|
||||
key = CultMeshSimulationFact.create_record_key(candidate)
|
||||
fact = CultMeshSimulationFact.from_candidate(candidate, committed_at=committed_at)
|
||||
self.node.put(self.document, key, fact)
|
||||
return CultMeshSimulationFactCommit(key=key, fact=fact)
|
||||
|
|
@ -6,7 +6,7 @@ import json
|
|||
from importlib import resources
|
||||
from typing import Any
|
||||
|
||||
from .benchmark import run_benchmark
|
||||
from cultnet_py.benchmark import run_benchmark
|
||||
|
||||
|
||||
EXPECTED_EXPORTS: dict[str, tuple[str, ...]] = {
|
||||
|
|
@ -150,7 +150,7 @@ EXPECTED_EXPORTS: dict[str, tuple[str, ...]] = {
|
|||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(prog="cultcache-py-verify")
|
||||
parser = argparse.ArgumentParser(prog="cultmesh-py-verify")
|
||||
parser.add_argument("--records", type=int, default=64)
|
||||
parser.add_argument("--json", action="store_true", dest="emit_json")
|
||||
args = parser.parse_args(argv)
|
||||
|
|
@ -159,7 +159,7 @@ def main(argv: list[str] | None = None) -> int:
|
|||
if args.emit_json:
|
||||
print(json.dumps(result, indent=2, sort_keys=True))
|
||||
else:
|
||||
print(f"cultcache-py verify: {result['status']}")
|
||||
print(f"cultmesh-py verify: {result['status']}")
|
||||
for check in result["checks"]:
|
||||
print(f"- {check['name']}: {check['status']}")
|
||||
return 0 if result["status"] == "ok" else 1
|
||||
4491
packages/cultmesh-py/tests/test_cultmesh.py
Normal file
4491
packages/cultmesh-py/tests/test_cultmesh.py
Normal file
File diff suppressed because it is too large
Load diff
156
packages/cultnet-py/README.md
Normal file
156
packages/cultnet-py/README.md
Normal file
|
|
@ -0,0 +1,156 @@
|
|||
# cultnet-py
|
||||
|
||||
`cultnet-py` is the Python CultNet runtime: schema-v0 MessagePack wire helpers,
|
||||
4-byte big-endian framing, the raw client, database subscriptions, shard
|
||||
catch-up and replication, simulation observations and consensus, witness
|
||||
artifact bundles, the RUDP codec/session/socket transport, and the CultMesh
|
||||
wire contracts that both a node and a bare peer have to agree on.
|
||||
|
||||
It depends on `cultcache-py` and `msgpack`; AES-GCM helpers on `CultNetSecret`
|
||||
light up with the optional `crypto` extra. `cultmesh-py` builds on it. The
|
||||
layering mirrors `cultnet-rs` and `cultnet-ts`.
|
||||
|
||||
## Example
|
||||
|
||||
```python
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from cultnet_py import (
|
||||
CultNetSecret,
|
||||
CultNetServerSecurityOptions,
|
||||
CultNetRawClient,
|
||||
CultNetSimulationObservation,
|
||||
CultNetWitnessArtifactBundle,
|
||||
apply_raw_snapshot,
|
||||
apply_shard_log_response,
|
||||
compute_simulation_claim_hash,
|
||||
database_subscribe,
|
||||
decode_frame,
|
||||
encode_frame,
|
||||
hello,
|
||||
parse_message,
|
||||
shard_catalog_request,
|
||||
simulation_observation,
|
||||
witness_artifact_bundle,
|
||||
)
|
||||
|
||||
payload = hello(runtime_id="python-runtime").to_bytes()
|
||||
message = parse_message(decode_frame(encode_frame(payload)))
|
||||
|
||||
client = CultNetRawClient("127.0.0.1", 3075)
|
||||
catalog = client.fetch_schema_catalog(kinds=["wire_message"])
|
||||
snapshot = client.fetch_snapshot_response(schema_ids=["cultnet.interop-note"])
|
||||
shard_catalog = client.fetch_shard_catalog(schema_ids=["cultnet.interop-note"])
|
||||
# With a local CultCache and the matching registered document definitions:
|
||||
# applied = apply_raw_snapshot(cache, [note_doc], snapshot)
|
||||
|
||||
subscription = database_subscribe(subscription_id="ui", schema_ids=["cultnet.interop-note"])
|
||||
shard_request = shard_catalog_request(message_id="shards", schema_ids=["cultnet.interop-note"])
|
||||
shard_log = client.fetch_shard_log_response(shard_id="interop", shard_epoch=1, after_sequence=0)
|
||||
# log_changes = apply_shard_log_response(cache, [note_doc], shard_log)
|
||||
with client.subscribe_database(subscription_id="ui", schema_ids=["cultnet.interop-note"]) as live:
|
||||
initial_snapshot = live.read_next_snapshot_response()
|
||||
|
||||
claim_hash = compute_simulation_claim_hash("frame:42", "subject:player-1", "hit")
|
||||
observation = simulation_observation(
|
||||
message_id="obs-1",
|
||||
witness_runtime_id="python-runtime",
|
||||
shard_id="interop",
|
||||
shard_epoch=1,
|
||||
frame=42,
|
||||
subject_id="player-1",
|
||||
claim_kind="hit",
|
||||
claim_hash=claim_hash,
|
||||
)
|
||||
typed_observation = CultNetSimulationObservation.from_wire(observation.to_wire())
|
||||
witness = witness_artifact_bundle(
|
||||
bundle_id="bundle-1",
|
||||
witness_kind="interop-proof",
|
||||
captured_at="2026-06-13T00:00:00Z",
|
||||
subject={"documentType": "cultnet.interop-note", "subjectId": "note:python"},
|
||||
contracts=[{"role": "payload", "schemaId": "cultnet.interop-note"}],
|
||||
artifacts=[{"role": "log", "uri": "cultcache://bundle-1/log", "mediaType": "text/plain"}],
|
||||
provenance={"pipelineId": "interop", "runId": "run-1", "runtimeId": "python-runtime"},
|
||||
)
|
||||
typed_witness = CultNetWitnessArtifactBundle.from_wire(witness)
|
||||
witness_payload = typed_witness.to_payload()
|
||||
|
||||
security = CultNetServerSecurityOptions.development()
|
||||
token = CultNetSecret.create_session_token(
|
||||
"318fb4b6-ff5e-4c4f-b911-d81807de53a8",
|
||||
datetime(2035, 1, 1, tzinfo=UTC),
|
||||
security,
|
||||
session_version=1,
|
||||
)
|
||||
session = CultNetSecret.try_validate_session_token(token, security)
|
||||
```
|
||||
|
||||
The raw-client request surface can ride RUDP when the peer advertises a
|
||||
`rudp://` endpoint and shares the connection id for that lane;
|
||||
`create_rudp_schema_transport(...)` builds that transport, and
|
||||
`cultmesh_py.CultMesh.create_client(endpoint="rudp://...")` selects it from an
|
||||
endpoint string.
|
||||
|
||||
## CultMesh wire contracts
|
||||
|
||||
`cultnet_py.cultmesh_contracts` carries the Verse catalog, peer exchange,
|
||||
authority lease, stream negotiation, and committed simulation fact contracts.
|
||||
They live here rather than in `cultmesh-py` for the same reason
|
||||
`CultMeshPeerCard` lives in `cultnet-rs`: the interop peer has to speak them
|
||||
without owning a node. `cultmesh_py` re-exports every one of them, so callers
|
||||
that already import from there keep working.
|
||||
|
||||
## RUDP
|
||||
|
||||
`CultNetRudpSession` is the shared reliable/unreliable/sequenced/ordered
|
||||
session over the RUDP packet codec, with fragmentation above
|
||||
`max_fragment_bytes`, a bounded reliable window, reliable expiry, and a
|
||||
bounded fragment reassembly map that evicts the oldest stranded set and counts
|
||||
it in `fragment_sets_evicted` rather than refusing new payloads. The
|
||||
socket-backed transport, reconnect policy/controller, and reconnect loop follow
|
||||
the Rust and TypeScript entrypoint pattern.
|
||||
|
||||
## Interop peer
|
||||
|
||||
The peer can serve, dial, and probe the same raw-state interop lane used by the
|
||||
TypeScript, Rust, and C# test peers, and answers Verse catalog and peer exchange
|
||||
requests without a CultMesh node:
|
||||
|
||||
```powershell
|
||||
python -m cultnet_py.interop_peer serve --runtime-id python-peer --runtime-kind python --display-name "Python Peer" --agent-id python-agent --advertise-host 127.0.0.1 --tcp-port 3075 --discovery-port 4075 --discovery-group 239.77.44.11 --schema-path ..\cultnet-ts\integration\contracts\cultnet.interop-note.schema.json
|
||||
python -m cultnet_py.interop_peer dial --runtime-id python-client --runtime-kind python --display-name "Python Client" --agent-id python-client --target-host 127.0.0.1 --target-port 3075 --schema-path ..\cultnet-ts\integration\contracts\cultnet.interop-note.schema.json
|
||||
python -m cultnet_py.interop_peer probe --runtime-id python-prober --discovery-port 4075 --discovery-group 239.77.44.11
|
||||
```
|
||||
|
||||
`packages/cultnet-ts/test/interop/cultnet-interop.test.ts` drives it inside the
|
||||
live TS/Rust/C#/Python peer ring.
|
||||
|
||||
## Performance baseline
|
||||
|
||||
The package ships a lightweight benchmark for Python-owned hot paths. In a
|
||||
CultLib source checkout, it can also run the C# reference benchmark beside it:
|
||||
|
||||
```powershell
|
||||
python -m cultnet_py.benchmark --records 1000 --json
|
||||
python -m cultnet_py.compare_csharp --records 1000 --json
|
||||
```
|
||||
|
||||
It reports slot-indexed `DatabaseEntry` encode/decode throughput, framed
|
||||
CultNet MessagePack parse throughput, raw snapshot application into a
|
||||
registered cache, and public `CultCache` upsert/get throughput. The C# compare
|
||||
command also runs `packages/cultnet-py/tools/GameCult.Caching.Benchmark` and
|
||||
reports median Python-to-C# ratios for the shared public cache operations. It
|
||||
uses three samples by default; pass `--samples` when you need a different
|
||||
evidence shape. Treat the numbers as a local baseline, not a claim that Python
|
||||
matches the C# reference in every workload.
|
||||
|
||||
## Tests
|
||||
|
||||
```powershell
|
||||
$env:PYTHONPATH="$PWD\packages\cultcache-py\src;$PWD\packages\cultnet-py\src"
|
||||
python -m unittest discover -s packages\cultnet-py\tests
|
||||
```
|
||||
|
||||
The suite runs without `cultmesh_py` on the path. See
|
||||
[docs/python-runtime-parity.md](../../docs/python-runtime-parity.md) for the
|
||||
evidence map across the three Python packages.
|
||||
33
packages/cultnet-py/pyproject.toml
Normal file
33
packages/cultnet-py/pyproject.toml
Normal file
|
|
@ -0,0 +1,33 @@
|
|||
[build-system]
|
||||
requires = ["setuptools>=68"]
|
||||
build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "cultnet-py"
|
||||
version = "0.1.0"
|
||||
description = "Python CultNet transport, wire contracts, and replication for GameCult typed state."
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.10"
|
||||
license = "MIT"
|
||||
authors = [{ name = "GameCult" }]
|
||||
classifiers = [
|
||||
"License :: OSI Approved :: MIT License",
|
||||
"Programming Language :: Python :: 3",
|
||||
"Programming Language :: Python :: 3.10",
|
||||
"Programming Language :: Python :: 3.11",
|
||||
"Programming Language :: Python :: 3.12",
|
||||
]
|
||||
dependencies = ["cultcache-py>=0.2.0", "msgpack>=1.0"]
|
||||
[project.optional-dependencies]
|
||||
crypto = ["cryptography>=42"]
|
||||
|
||||
[project.scripts]
|
||||
cultnet-py-interop = "cultnet_py.interop_peer:main"
|
||||
cultnet-py-benchmark = "cultnet_py.benchmark:main"
|
||||
cultnet-py-compare-csharp = "cultnet_py.compare_csharp:main"
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
where = ["src"]
|
||||
|
||||
[tool.setuptools.package-data]
|
||||
cultnet_py = ["py.typed"]
|
||||
|
|
@ -7,6 +7,34 @@ from .client import (
|
|||
create_rudp_schema_transport,
|
||||
create_tcp_framed_schema_transport,
|
||||
)
|
||||
from .cultmesh_contracts import (
|
||||
PEER_EXCHANGE_REQUEST,
|
||||
PEER_EXCHANGE_RESPONSE,
|
||||
SIMULATION_FACT_DOCUMENT_TYPE,
|
||||
SIMULATION_FACT_SCHEMA_VERSION,
|
||||
VERSE_CATALOG_REQUEST,
|
||||
VERSE_CATALOG_RESPONSE,
|
||||
AuthorityLeaseVerifier,
|
||||
CultMeshAuthorityLease,
|
||||
CultMeshAuthorityLeaseCatalog,
|
||||
CultMeshHmacAuthorityLeaseVerifier,
|
||||
CultMeshPeerCard,
|
||||
CultMeshPeerCatalog,
|
||||
CultMeshSimulationFact,
|
||||
CultMeshStreamCatalog,
|
||||
CultMeshStreamConsumerProfile,
|
||||
CultMeshStreamDescriptor,
|
||||
CultMeshStreamFrameHandle,
|
||||
CultMeshStreamNegotiation,
|
||||
CultMeshVerseCatalog,
|
||||
CultMeshVerseCompatibility,
|
||||
CultMeshVerseDescriptor,
|
||||
peer_exchange_request,
|
||||
peer_from_wire,
|
||||
simulation_fact_document,
|
||||
verse_catalog_request,
|
||||
verse_from_wire,
|
||||
)
|
||||
from .framing import decode_frame, encode_frame, read_frame, write_frame
|
||||
from .messages import (
|
||||
CultNetMessage,
|
||||
|
|
@ -108,6 +136,32 @@ from .simulation import (
|
|||
)
|
||||
|
||||
__all__ = [
|
||||
"PEER_EXCHANGE_REQUEST",
|
||||
"PEER_EXCHANGE_RESPONSE",
|
||||
"SIMULATION_FACT_DOCUMENT_TYPE",
|
||||
"SIMULATION_FACT_SCHEMA_VERSION",
|
||||
"VERSE_CATALOG_REQUEST",
|
||||
"VERSE_CATALOG_RESPONSE",
|
||||
"AuthorityLeaseVerifier",
|
||||
"CultMeshAuthorityLease",
|
||||
"CultMeshAuthorityLeaseCatalog",
|
||||
"CultMeshHmacAuthorityLeaseVerifier",
|
||||
"CultMeshPeerCard",
|
||||
"CultMeshPeerCatalog",
|
||||
"CultMeshSimulationFact",
|
||||
"CultMeshStreamCatalog",
|
||||
"CultMeshStreamConsumerProfile",
|
||||
"CultMeshStreamDescriptor",
|
||||
"CultMeshStreamFrameHandle",
|
||||
"CultMeshStreamNegotiation",
|
||||
"CultMeshVerseCatalog",
|
||||
"CultMeshVerseCompatibility",
|
||||
"CultMeshVerseDescriptor",
|
||||
"peer_exchange_request",
|
||||
"peer_from_wire",
|
||||
"simulation_fact_document",
|
||||
"verse_catalog_request",
|
||||
"verse_from_wire",
|
||||
"CultNetMessage",
|
||||
"CultNetWitnessArtifactBundle",
|
||||
"CultNetDatabaseSubscription",
|
||||
|
|
@ -19,7 +19,7 @@ class BenchItem:
|
|||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(prog="cultcache-py-benchmark")
|
||||
parser = argparse.ArgumentParser(prog="cultnet-py-benchmark")
|
||||
parser.add_argument("--records", type=int, default=5000)
|
||||
parser.add_argument("--json", action="store_true", dest="as_json")
|
||||
args = parser.parse_args(argv)
|
||||
|
|
@ -16,7 +16,7 @@ DEFAULT_PARITY_THRESHOLD = 0.90
|
|||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(prog="cultcache-py-compare-csharp")
|
||||
parser = argparse.ArgumentParser(prog="cultnet-py-compare-csharp")
|
||||
parser.add_argument("--records", type=int, default=5000)
|
||||
parser.add_argument("--samples", type=int, default=DEFAULT_SAMPLE_COUNT)
|
||||
parser.add_argument("--parity-threshold", type=float, default=DEFAULT_PARITY_THRESHOLD)
|
||||
|
|
@ -1,14 +1,18 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import hashlib
|
||||
import hmac
|
||||
from dataclasses import dataclass, field, replace
|
||||
from datetime import datetime
|
||||
from datetime import UTC, datetime
|
||||
from hashlib import sha256
|
||||
from typing import Any, Callable, Mapping
|
||||
|
||||
import msgpack
|
||||
|
||||
from cultcache_py import define_database_entry_type
|
||||
from cultcache_py.documents import DocumentDefinition
|
||||
|
||||
|
||||
VERSE_CATALOG_REQUEST = "cultmesh.verse_catalog_request.v0"
|
||||
VERSE_CATALOG_RESPONSE = "cultmesh.verse_catalog_response.v0"
|
||||
|
|
@ -613,3 +617,104 @@ def copy_budget_for(transport: str) -> str:
|
|||
if transport == "inline-bytes":
|
||||
return "opaque-runtime"
|
||||
return "opaque-runtime"
|
||||
|
||||
|
||||
SIMULATION_FACT_DOCUMENT_TYPE = "gamecult.mesh.simulation_fact"
|
||||
SIMULATION_FACT_SCHEMA_VERSION = "gamecult.mesh.simulation_fact.v1"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CultMeshSimulationFact:
|
||||
fact_id: str
|
||||
shard_id: str
|
||||
shard_epoch: int
|
||||
frame: int
|
||||
subject_id: str
|
||||
claim_kind: str
|
||||
claim_hash: str
|
||||
claim_summary: str | None
|
||||
witness_count: int
|
||||
support_weight: float
|
||||
total_weight: float
|
||||
confidence: float
|
||||
committed_at: str
|
||||
|
||||
@staticmethod
|
||||
def compute_fact_id(candidate: dict[str, Any]) -> str:
|
||||
canonical = "\x1f".join([
|
||||
_candidate_string(candidate, "shardId", "shard_id"),
|
||||
str(_candidate_int(candidate, "shardEpoch", "shard_epoch")),
|
||||
str(_candidate_int(candidate, "frame", "frame")),
|
||||
_candidate_string(candidate, "subjectId", "subject_id"),
|
||||
_candidate_string(candidate, "claimKind", "claim_kind"),
|
||||
])
|
||||
return hashlib.sha256(canonical.encode("utf-8")).hexdigest()
|
||||
|
||||
@staticmethod
|
||||
def create_record_key(candidate: dict[str, Any]) -> str:
|
||||
return f"simulation:{CultMeshSimulationFact.compute_fact_id(candidate)}"
|
||||
|
||||
@staticmethod
|
||||
def from_candidate(candidate: dict[str, Any], *, committed_at: str | None = None) -> "CultMeshSimulationFact":
|
||||
return CultMeshSimulationFact(
|
||||
fact_id=CultMeshSimulationFact.compute_fact_id(candidate),
|
||||
shard_id=_candidate_string(candidate, "shardId", "shard_id"),
|
||||
shard_epoch=_candidate_int(candidate, "shardEpoch", "shard_epoch"),
|
||||
frame=_candidate_int(candidate, "frame", "frame"),
|
||||
subject_id=_candidate_string(candidate, "subjectId", "subject_id"),
|
||||
claim_kind=_candidate_string(candidate, "claimKind", "claim_kind"),
|
||||
claim_hash=_candidate_string(candidate, "claimHash", "claim_hash"),
|
||||
claim_summary=_candidate_optional_string(candidate, "claimSummary", "claim_summary"),
|
||||
witness_count=_candidate_int(candidate, "witnessCount", "witness_count"),
|
||||
support_weight=_candidate_float(candidate, "supportWeight", "support_weight"),
|
||||
total_weight=_candidate_float(candidate, "totalWeight", "total_weight"),
|
||||
confidence=_candidate_float(candidate, "confidence", "confidence"),
|
||||
committed_at=committed_at or datetime.now(UTC).isoformat(),
|
||||
)
|
||||
|
||||
|
||||
simulation_fact_document: DocumentDefinition[CultMeshSimulationFact] = define_database_entry_type(
|
||||
SIMULATION_FACT_DOCUMENT_TYPE,
|
||||
[
|
||||
("fact_id", 0),
|
||||
("shard_id", 1),
|
||||
("shard_epoch", 2),
|
||||
("frame", 3),
|
||||
("subject_id", 4),
|
||||
("claim_kind", 5),
|
||||
("claim_hash", 6),
|
||||
("claim_summary", 7, None),
|
||||
("witness_count", 8),
|
||||
("support_weight", 9),
|
||||
("total_weight", 10),
|
||||
("confidence", 11),
|
||||
("committed_at", 12),
|
||||
],
|
||||
cls=CultMeshSimulationFact,
|
||||
name="fact_id",
|
||||
indexes={"shard": "shard_id", "frame": "frame", "subject": "subject_id", "claim_kind": "claim_kind"},
|
||||
schema_name=SIMULATION_FACT_DOCUMENT_TYPE,
|
||||
schema_version=SIMULATION_FACT_SCHEMA_VERSION,
|
||||
)
|
||||
|
||||
|
||||
def _candidate_string(candidate: dict[str, Any], camel: str, snake: str) -> str:
|
||||
value = candidate.get(camel, candidate.get(snake, ""))
|
||||
if value is None:
|
||||
return ""
|
||||
return str(value)
|
||||
|
||||
|
||||
def _candidate_optional_string(candidate: dict[str, Any], camel: str, snake: str) -> str | None:
|
||||
value = candidate.get(camel, candidate.get(snake))
|
||||
if value is None:
|
||||
return None
|
||||
return str(value)
|
||||
|
||||
|
||||
def _candidate_int(candidate: dict[str, Any], camel: str, snake: str) -> int:
|
||||
return int(candidate.get(camel, candidate.get(snake, 0)))
|
||||
|
||||
|
||||
def _candidate_float(candidate: dict[str, Any], camel: str, snake: str) -> float:
|
||||
return float(candidate.get(camel, candidate.get(snake, 0.0)))
|
||||
|
|
@ -24,7 +24,7 @@ from cultcache_py import (
|
|||
define_document_type,
|
||||
)
|
||||
from cultcache_py.documents import DocumentDefinition
|
||||
from cultmesh_py import (
|
||||
from .cultmesh_contracts import (
|
||||
PEER_EXCHANGE_REQUEST,
|
||||
SIMULATION_FACT_DOCUMENT_TYPE,
|
||||
VERSE_CATALOG_REQUEST,
|
||||
2046
packages/cultnet-py/tests/test_cultnet.py
Normal file
2046
packages/cultnet-py/tests/test_cultnet.py
Normal file
File diff suppressed because it is too large
Load diff
|
|
@ -24,7 +24,9 @@ const pythonCommand = process.env.PYTHON ?? resolvePythonCommand();
|
|||
const cultNetTsRoot = resolve(__dirname, "../../..");
|
||||
const cultLibRoot = findAncestor(cultNetTsRoot, "CultLib.sln") ?? resolve(cultNetTsRoot, "..", "CultLib");
|
||||
const cultcachePyRoot = resolve(cultLibRoot, "packages", "cultcache-py");
|
||||
const cultcachePySrc = resolve(cultcachePyRoot, "src");
|
||||
const cultcachePySrc = ["cultcache-py", "cultnet-py", "cultmesh-py"]
|
||||
.map((name) => resolve(cultLibRoot, "packages", name, "src"))
|
||||
.join(delimiter);
|
||||
const cultmeshKotlinRoot = resolve(cultLibRoot, "packages", "cultmesh-kotlin");
|
||||
const kotlinHome = process.env.KOTLIN_HOME ?? "C:\\Program Files\\Android\\Android Studio\\plugins\\Kotlin\\kotlinc";
|
||||
const kotlinJavaHome = process.env.KOTLIN_JAVA_HOME ?? "C:\\Program Files\\Android\\Android Studio\\jbr";
|
||||
|
|
|
|||
|
|
@ -69,7 +69,7 @@ if ($LASTEXITCODE -ne 0) {
|
|||
throw "CultMesh TypeScript reactive document performance probe failed."
|
||||
}
|
||||
|
||||
$pythonSource = Join-Path $root "packages\cultcache-py\src"
|
||||
$pythonSource = (@("cultcache-py", "cultnet-py", "cultmesh-py") | ForEach-Object { Join-Path $root "packages\$_\src" }) -join [IO.Path]::PathSeparator
|
||||
$previousPythonPath = $env:PYTHONPATH
|
||||
$env:PYTHONPATH = if ([string]::IsNullOrWhiteSpace($previousPythonPath)) {
|
||||
$pythonSource
|
||||
|
|
@ -78,7 +78,7 @@ else {
|
|||
$pythonSource + [IO.Path]::PathSeparator + $previousPythonPath
|
||||
}
|
||||
try {
|
||||
& $pythonExecutable (Join-Path $root "packages\cultcache-py\tools\reactive_performance_probe.py") @runtimeArguments
|
||||
& $pythonExecutable (Join-Path $root "packages\cultmesh-py\tools\reactive_performance_probe.py") @runtimeArguments
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
throw "CultMesh Python reactive document performance probe failed."
|
||||
}
|
||||
|
|
|
|||
|
|
@ -81,7 +81,7 @@ finally {
|
|||
Pop-Location
|
||||
}
|
||||
|
||||
$pythonPath = Join-Path $root "packages\cultcache-py\src"
|
||||
$pythonPath = (@("cultcache-py", "cultnet-py", "cultmesh-py") | ForEach-Object { Join-Path $root "packages\$_\src" }) -join [IO.Path]::PathSeparator
|
||||
$previousPythonPath = $env:PYTHONPATH
|
||||
$env:PYTHONPATH = if ([string]::IsNullOrWhiteSpace($previousPythonPath)) {
|
||||
$pythonPath
|
||||
|
|
@ -91,8 +91,8 @@ else {
|
|||
}
|
||||
try {
|
||||
& $pythonExecutable -m unittest discover `
|
||||
-s (Join-Path $root "packages\cultcache-py\tests") `
|
||||
-p "test_cultcache.py" `
|
||||
-s (Join-Path $root "packages\cultmesh-py\tests") `
|
||||
-p "test_cultmesh.py" `
|
||||
-k "reactive_scheduling_scales_with_changed_documents_only"
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
throw "CultMesh Python reactive scaling verification failed."
|
||||
|
|
|
|||
Loading…
Reference in a new issue