diff --git a/research/witness-authoritative-networking/prior-art-2026-10-26.md b/research/witness-authoritative-networking/prior-art-2026-10-26.md new file mode 100644 index 00000000..739fdac3 --- /dev/null +++ b/research/witness-authoritative-networking/prior-art-2026-10-26.md @@ -0,0 +1,294 @@ +# Witness-authoritative networking: prior art (2026-10-26) + +## Framing (operator, 2026-10-26) + +The concept began as witness-authoritative networking; the consensus literature is the closest existing body of work, not the origin. The goal: no centralized authoritative server should be needed to resolve local disputes such as whether Alice shot Bob first or Bob shot Alice first. A quorum of nearby clients reaches agreement on the ordering, and whoever is determined to be wrong is corrected exactly as in a server-authoritative model with client-side prediction, except that the correction can happen before the server hears about it. + +The report below was produced by a read-only Eyes pass. It describes the CultMesh implementation as read from source on 2026-10-26 and compares it with prior art; it does not recommend a design. Items marked as from memory were not re-fetched. + +--- + +# CultMesh "witness consensus": source reading and prior art + +Eyes pass, 2026-10-26. Read-only. Eureka mind (eureka-state) not used. No design recommendation is made here. + +Evidence labels: [SRC] read from the CultLib source this pass. [WEB] confirmed by a web search result this pass (title/venue/authors seen in a result). [MEM] from model memory, NOT re-fetched this pass; verify before citing. + +--------------------------------------------------------------------------- + +## STEP 1. What CultMesh witness consensus is today (from source) + +All paths under `F:\Projects\CultLib\`. + +### 1.1 The machine, as implemented [SRC] + +Three pieces plus a facade: + +1. **Observation** `CultNetSimulationObservation`, `src\GameCult.Networking\CultNetSimulationConsensus.cs:15-69`. One witness report: `witnessRuntimeId`, `shardId`, `shardEpoch`, `frame`, `subjectId`, `claimKind` ("hit, input, collision, or ownership", line 39), `claimHash` (SHA-256 of caller-chosen parts joined by U+001F, lines 61-68), optional `claimSummary`, `weight` (default 1, "Normal clients should use 1", lines 50-52), `observedAt`. There is no signature field, no nonce, no reference to the witness's own state hash or prior observations. +2. **Aggregator** `CultNetSimulationConsensus.BuildCandidates`, same file lines 177-293. + - Drops invalid observations (empty ids/hash or weight <= 0; lines 257-266). + - De-duplicates per (witness, shard, epoch, frame, subject, kind), keeping the highest weight (lines 196-203, 273-282). So a witness that reports two different claim hashes for the same subject/frame keeps only one (heaviest, then ordinal-smallest witness id), not both. + - Groups by (shard, epoch, frame, subject, kind) (lines 205-206, 284-292). Per group: total weight = sum of all weights; best claim = the claimHash with greatest summed weight, ties broken by smallest hash ordinal (lines 219-230). + - `HasQuorum = distinctWitnesses >= MinimumWitnesses && bestWeight >= MinimumWeight && bestWeight/totalObservedWeight >= QuorumRatio` (lines 238-241). Defaults: MinimumWitnesses 1, MinimumWeight 1.0, QuorumRatio 0.5 (lines 79-87). + - The denominator is weight actually OBSERVED, not weight of an enrolled witness set. +3. **Hub** `CultNetSimulationObservationHub`, lines 298-398. In-memory `List<>` of every observation; each `Submit` appends, re-runs `BuildCandidates` over the whole list (line 347) and emits candidates for that subject on an R3 `Subject`. No eviction, no persistence, not thread-safe (plain List). +4. **Wire bridge** `src\GameCult.Networking\CultNetSimulationObservationServer.cs`: receives `cultnet.simulation_observation.v0` (`CultNetSchemaMessages.cs:133, 1255-1268`), submits to the hub, replies to the sending peer with candidate messages. In the lines read (61-70) the handler submits without comparing `WitnessRuntimeId` to the authenticated peer identity [SRC, partial read; verify the rest of the handler]. +5. **Commit** `src\GameCult.Mesh\CultMeshSimulationFacts.cs:12-188`. `CultMeshSimulationFact` is a CultCache document (`gamecult.mesh.simulation_fact.v1`). `FactId` = SHA-256 over shard|epoch|frame|subject|kind, deliberately NOT over claimHash (lines 114-126). `CultMeshSimulationFactCommitter.CommitAsync` throws unless `HasQuorum` (lines 179-182) then `PutAsync` into `CultNetDatabase`, which enforces the shard's primary and epoch (`CultNetDatabase.cs:1034, 1054` call `EnsurePrimary(shard, schemaId, key, message.ShardEpoch)`; stale-epoch errors at 829-834, 1086-1091). +6. **Session facade** `src\GameCult.Mesh\CultMeshGameSession.cs:60-224`. `SubmitAndCommitAsync` submits, then `CommitQuorumCandidatesAsync` commits each quorum candidate once per session (`_committedFactIds` HashSet, then a DB `GetAsync` existence check, lines 190-221). The session constructs an `AuthorityResolver` (line 77) but nothing in the witness path calls it (grep of the file: only lines 35, 77, 128). + +### 1.2 What the design says it is for [SRC] + +- `src\GameCult.Mesh\README.md:3-4`: "distributed realtime database and simulation-consensus layer". README "Witness Consensus" section lines 360-378: observations are "the fast mesh opinion layer that can feed an authoritative shard commit path"; candidates "are not committed world state by themselves"; "The central server does not need to decide what every witness saw. It only needs to accept the quorum result through the shard log." Example "who shot first" (`contracts\cultnet\cultnet-distributed-database.md` ~line 175). +- `src\GameCult.Mesh\docs\research.md:21-41`: live model is primary-shard authority plus replica catch-up; steps 4-6: witnesses publish immutable observations, nodes aggregate to candidates, "Authoritative world state is still committed through shard logs". Next research cut includes "add observation fanout across peers", "simulation rollback/resimulation helpers", then "revisit membership, failover, and consensus protocol depth". +- `src\GameCult.Mesh\docs\public-api.md:528-531, 574-583`: "Each shard has one primary writer for now. Non-primary writes are rejected or explicitly forwarded. Stale epochs fail loudly." Candidates are "opinions". +- Verse authority models enum, `src\GameCult.Mesh\CultMeshVerse.cs:14-32`: OperatorCluster, FederatedCluster, PeerToPeer ("Peers participate directly in authority decisions"), SubscribedOverlay. Only descriptors; no peer-to-peer authority code found. +- `src\GameCult.Mesh\docs\cross-runtime-primitives-roadmap.md:80`: "Authority scope: Runtime claim, lease, witness, or delegated simulation ownership expressed as typed policy state. First consumers: Trusted co-op, future quorum/witness modes". The words "trusted co-op" mark the current trust model. +- `research\cultnet-distributed-database\summary.md` (193 lines): borrowed Raft (primary per shard first, election deferred), Dynamo, SWIM (later), CRDTs (only with declared merge law), Hashgraph. Hashgraph was explicitly REJECTED for now: "CultNet does not currently need asynchronous Byzantine total ordering"; keep only gossip-about-gossip as a provenance idea (summary lines on Hashgraph; also `docs\research.md:15-16`). Stored PDFs: hashgraph-swirlds-tr-2016-01.pdf, raft-extended.pdf, dynamo-sosp2007.pdf, swim.pdf, crdt-arxiv-1805.06358.html, plus changefeed/Firebase pages. + +### 1.3 Who simulates, who witnesses, what is agreed + +- **Who simulates:** the code does not say. Nothing in CultLib implements a simulation, input exchange, rollback or resimulation. The only simulation-shaped objects are the observation fields. `docs\research.md` lists "simulation rollback/resimulation helpers" as NOT yet built; `contracts\cultnet\cultnet-distributed-database.md` lists cross-runtime rollback/resimulation helpers as "standardize ... beyond Python's local pending-prediction rollback surface" (Python only). +- **Who witnesses:** any runtime that submits a `CultNetSimulationObservation`. No witness registry, selection rule, enrollment, stake or lease check exists in the witness path. +- **What is agreed:** a per-(shard, epoch, frame, subject, claimKind) claim HASH; not inputs, not state hashes of a whole tick, not an ordered log. The summary text is carried but not part of the vote. Agreement is a weighted plurality over whatever observations arrived, evaluated incrementally. +- **Ordering / time:** `frame` is a number the witness asserts; no clock, no tick schedule, no ordering across subjects, no finality rule. Commit fires the first time a candidate has quorum (session-local `_committedFactIds` plus DB existence check). Because FactId excludes claimHash, a second, different quorum-winning claim for the same subject/frame/kind later would be silently skipped (line 205-215 `GetAsync != null -> continue`). +- **Disagreement:** the minority claim is only reflected in `Confidence` (= support/total) and `TotalWeight`. No dispute, challenge, slashing, re-simulation or escalation path. + +### 1.4 Fault model (as built) [SRC, inferred from code] + +- Not Byzantine-tolerant. Reasons visible in code: (a) observations are unsigned and the witness id is a self-asserted string; (b) `weight` is supplied by the observer (line 52); (c) quorum denominator is observed weight, so n observers out of an unknown population can reach "quorum"; with defaults, one witness alone commits (MinimumWitnesses=1, ratio 0.5); (d) no Sybil bound; (e) the commit authority is the single shard primary, so a crashed or malicious primary is outside the model (no election, no failover: contract doc "Not implemented yet: leader election or automatic shard failover"). +- The effective model is "trusted co-op / operator-cluster": the shard primary is trusted; witnesses are an advisory vote the primary chooses to commit. Cheating resistance is whatever the primary's trust gives. +- Authority leases (separate subsystem) are the only signature-backed element: `CultMeshAuthorityLease` (`CultMeshAuthority.cs:8-60`) binds verse, peer, roles, shard ids, issuer, validity window, optional epoch, signature; decisions belong to `CultMeshAuthorityResolver` with signature verifier and revocation source (`docs\reliability-migration-plan.md:120-175`: "Forbidden writers: peer catalogs, transport success, caches, and UI state must never confer authority"). `CultMeshAuthorityProof.cs` verifies P-256 (IEEE P1363, 64-byte) Odin route certificates and provider session proofs (lines 111-143, 305-316). Leases cover roles like `ShardPrimary` and `ReadReplica`; no "Witness" role was found in the witness path [SRC grep, verify]. + +### 1.5 Implemented vs planned + +Implemented [SRC]: observation/candidate types, deterministic aggregation, schema-v0 messages, in-memory hub, server bridge that replies with candidates, fact document and committer through primary shard with epoch check, session facade, durable shard logs and snapshot fallback, primary-shard authority, client prediction scopes (`Predicted`/`Reconciled`), authority lease/resolver, Python parity for consensus types, committer and session. +Planned / not implemented [SRC docs]: observation fanout across peers ("standardize cross-runtime scheduled peer-to-peer fanout for observation gossip and candidate propagation", `cultnet-distributed-database.md` Next Coherent Slice), rollback/resimulation helpers (C#), SWIM membership, leader election/failover, witness selection, signatures on observations, "future quorum/witness modes" (roadmap line 80). + +### 1.6 Tests [SRC] + +`tests\GameCult.Networking.Tests\NetworkingTests.cs`: `CultNetSimulationConsensus_Builds_QuorumCandidate_FromWitnesses` (6647), `..._DeduplicatesWitness_AndBreaksTiesDeterministically` (6702), `CultNetSimulationObservationHub_Emits_ReactiveCandidates` (6760), `CultNetSimulationObservationServer_Creates_CandidateMessages` (6806), `CultMeshSimulationFactCommitter_Commits_QuorumCandidate_ToShardLog` (6856), `CultMeshGameSession_SubmitsWitnesses_AndCommitsQuorumFact` (6902), `CultMeshSimulationFactCommitter_Rejects_CandidateWithoutQuorum` (6970); message round-trips at 3908-3955. Python: `packages\cultmesh-py\tests\test_cultmesh.py` ~1866-2019 (committer, game session commits quorum once). TS interop test mentions simulation messages (`packages\cultnet-ts\test\interop\cultnet-interop.test.ts`, not read in detail). No test found for: Byzantine/lying witnesses, forged witness ids, weight inflation, conflicting claims for one FactId, network partition, latency, or a real simulation. All tests are in-process, happy-path or arithmetic. + +### 1.7 Name collision + +`CultWitnessArtifactBundle` (`src\GameCult.Networking\CultWitnessArtifactBundle.cs`, schema `cultnet.witness_artifact_bundle`) is a build/pipeline evidence manifest (subject pin, contract pins, artifact URIs, timing "witnesses"), unrelated to simulation witnesses. "Witness" in the postmortem docs (cold-release witness) means the same pipeline-evidence sense. Do not conflate. + +--------------------------------------------------------------------------- + +## STEP 2. Prior art + +### A. Crash-fault consensus + +| Item | Core idea | Fault model | Agreed | Latency / throughput | Group size | Implementations | Relation to CultMesh | +|---|---|---|---|---|---|---|---| +| Paxos: Lamport, "The Part-Time Parliament", ACM TOCS 1998; "Paxos Made Simple", 2001 [MEM] | Quorum-based single-value agreement with proposal numbers | Crash, message loss, asynchronous; needs majority (n >= 2f+1) | One value per instance | 2 round trips for a fresh proposer, 1 with stable leader | 3-7 | Chubby, Spanner (Multi-Paxos) [MEM] | Different: agrees on an ordered log, no observed-weight plurality; majority is of a fixed membership | +| Multi-Paxos [MEM] | Stable leader skips phase 1 per slot | same | Ordered log | ~1 RTT to a majority, high throughput | 3-7 | Spanner, many | Similar shape to CultMesh's primary-shard log; CultMesh has no replication quorum on the log yet | +| Raft: Ongaro & Ousterhout, "In Search of an Understandable Consensus Algorithm", USENIX ATC 2014 [MEM; PDF is in `research\...\raft-extended.pdf`] | Leader election by term, log replication, majority commit | Crash (non-Byzantine), partitions | Replicated log | 1 RTT commit, ms-level in a LAN, 10^4-10^5 ops/s | 3-7 | etcd, Consul, TiKV, CockroachDB [MEM] | Similar: CultMesh docs explicitly call primary-per-shard "Raft-shaped" (`docs\research.md:11`); epoch ~ term (`stale_epoch` errors). Different: no election, no majority ack of writes, witnesses are not voters on the log | +| Viewstamped Replication: Oki & Liskov PODC 1988; Liskov & Cowling, "VR Revisited", MIT TR 2012 [MEM] | Primary + view changes, state-machine replication | Crash | Ordered ops | like Raft | 3-5 | few production | Same family as Raft; view number ~ shard epoch | +| EPaxos: Moraru, Andersen, Kaminsky, SOSP 2013 [MEM] | Leaderless, commit non-conflicting commands in 1 RTT fast path | Crash | Partial order by dependency | Fast path 1 WAN RTT; conflicts slow | 3-5 | research/few prod | Different; relevant only to "avoid a single primary" | +| ZooKeeper/ZAB: Junqueira, Reed, Serafini, DSN 2011 [MEM] | Primary-order atomic broadcast | Crash | Ordered broadcast | ms | 3-7 | ZooKeeper | Same family; coordination service for leases | +| Leases: Gray & Cheriton, SOSP 1989 [MEM] | Time-bounded grants | Crash plus bounded clock drift | Exclusive holder | low | any | Chubby, etcd leases | Same idea as `CultMeshAuthorityLease` (validFrom/expiresAt, `CultMeshAuthority.cs:14-60`); CultMesh adds signed issuer and epoch | + +Group sizes in A are small (3-7) because every write waits on a majority. Throughput characteristics of etcd/ZooKeeper are written for control-plane use, not 20-60 Hz game state; no frame-rate use of Raft quorum was surveyed this pass. + +### B. Convergence without consensus + +| Item | Core idea | Fault model | Notes | +|---|---|---|---| +| CRDTs: Shapiro, Preguica, Baquero, Zawirski, "Conflict-free Replicated Data Types", SSS 2011 (INRIA RR-7687 [MEM]); survey arXiv 1805.06358 (local copy in research dir) | Merge function is commutative, associative, idempotent so replicas converge with no coordination | Crash/partition, NOT Byzantine; every replica trusted | Cannot enforce cross-object invariants (e.g. "hp cannot go below 0 and only one player gets the last item", exclusive ownership) without coordination. Related: Bailis et al., "Coordination Avoidance in Database Systems" (invariant confluence), VLDB 2015 [MEM] | +| Operational Transformation: Ellis & Gibbs, SIGMOD 1989 [MEM] | Transform concurrent ops against each other | Trusted peers, usually central server in practice | Text editing; correctness proofs notoriously hard | +| Automerge, Yjs [MEM] | Op-based JSON/text CRDTs; Yjs derives from YATA (Nicolaescu et al. 2016) | Trusted peers | Document sync, not game rules; Automerge has no authority/permission model | +| Local-first software: Kleppmann, Wiggins, van Hardenberg, McGranaghan, Onward! 2019 [MEM] | Seven ideals; CRDT sync, user-owned data | Trusted collaborators | Explicitly collaboration documents | + +Relation to CultMesh: the research summary already says CRDTs only for documents with declared merge laws and "generic automatic merge ... is a language cop with a nicer hat" (`research\...\summary.md`). CultMesh's witness layer is NOT a CRDT: it is a vote over claim hashes, not a merge. Game rules (hit validity, ownership transfer) are invariant-bearing, which is the standard limit of CRDTs. + +### C. Byzantine fault tolerance and witness-style constructs + +| Item | Core idea | Fault model | What is agreed | Latency / throughput | Group size | Implementations | Relation to CultMesh | +|---|---|---|---|---|---|---|---| +| PBFT: Castro & Liskov, OSDI 1999 [MEM] | 3-phase (pre-prepare, prepare, commit) with signed/MAC'd messages and view change | Byzantine, n >= 3f+1, partial synchrony | Total order of requests | O(n^2) messages; ms in LAN; throughput falls with n | 4-~20 | BFT-SMaRt, Hyperledger Fabric (older) [MEM] | Different: quorum is of a FIXED membership (2f+1 of 3f+1) with authenticated messages; CultMesh's quorum is of observed weight and unauthenticated | +| Tendermint: Buchman, Kwon, Milosevic, arXiv 1807.04938 (2018) [MEM] | PBFT-like rounds with proposer rotation, locked blocks | Byzantine, partial sync | Block per height | seconds per block | tens-~150 validators | Cosmos | Different; shows round time ~ seconds | +| HotStuff: Yin, Malkhi, Reiter, Gueta, Abraham, PODC 2019 [MEM] | Linear view change via chained quorum certificates | Byzantine, partial sync | Ordered blocks | O(n) per view | tens-100s | Libra/Diem, Aptos lineage | Different; its "quorum certificate" (n-f signatures on a value) is the formal object CultMesh's `HasQuorum` resembles without signatures | +| Hashgraph: Baird, "The Swirlds Hashgraph Consensus Algorithm", SWIRLDS-TR-2016-01 (local PDF in CultLib research dir) | Gossip-about-gossip builds a DAG; virtual voting; "witnesses" are the first events of each round per member, "famous witnesses" decide order | Asynchronous BFT, n >= 3f+1 | Total order + consensus timestamp | seconds | ~tens-100s; Hedera runs 30-ish councils [MEM] | Hedera | NAME ONLY: a Hashgraph witness is a DAG vertex, not an attester of a simulation fact. CultLib already evaluated and rejected adopting it | +| Narwhal and Tusk: Danezis, Kokoris-Kogias, Sonnino, Spiegelman, EuroSys 2022 [WEB confirmed: EuroSys '22 pp 34-50; ~160k tx/s at ~3 s latency, 600k tx/s with workers] | Separate reliable dissemination (DAG mempool) from ordering | Async BFT | Ordered txs | seconds | 10-50 | Sui/Mysten lineage [MEM] | Different latency class | +| Bullshark: Spiegelman, Giridharan, Sonnino, Kokoris-Kogias, ACM CCS 2022, arXiv 2201.05677 [WEB confirmed; DOI 10.1145/3548606.3559361; abstract: partially-synchronous variant ~125k tx/s at ~2 s latency, 50 parties] | DAG BFT with fast path, no extra messages beyond the DAG | Async liveness, partial sync fast path | Ordered vertices | ~2 s | ~50 | Sui | Different latency class | +| DAG-Rider: Keidar, Kokoris-Kogias, Naor, Spiegelman, "All You Need is DAG", PODC 2021 [MEM, not confirmed this pass] | First DAG-based async BFT with optimal amortized communication | Async BFT | Ordered | seconds | tens | foundation of Narwhal/Tusk | Different | +| Certificate Transparency / transparency-log witnesses: C2SP `tlog-cosignature` and `tlog-witness`; Sigsum; litewitness [WEB: spec names and purpose confirmed via search; spec pages not opened] | Independent witnesses verify a log's new checkpoint is consistent with the previous one (Merkle consistency proof) and cosign it; clients require a quorum of cosignatures to defeat split-view attacks | Byzantine LOG, but witnesses only check append-only consistency; they do not execute the application | A signed checkpoint (tree head) | Seconds to minutes; sync problems reported when logs update faster than witnesses | handful to dozens of witnesses | Sigsum, Go checksum DB witnesses, Let's Encrypt/Boulder adoption in progress [WEB, partial] | CLOSEST use of the word "witness": independent parties attest a statement (checkpoint hash) with a signature and a client-side quorum policy. Similar to a candidate with signed support; different in that CT witnesses verify a cryptographic proof, not an independent re-execution | +| Ethereum sync committee / light-client attestations (Altair, 2021) [MEM] | 512-validator committee signs headers; light clients accept with 2/3 signatures | Honest-majority of a randomly sampled committee | Block header | ~12 s slots | 512 | Ethereum | Relates to witness SELECTION by random sampling | +| Ouroboros: Kiayias, Russell, David, Oliynykov, CRYPTO 2017 [MEM] | Stake-weighted leader/committee selection by verifiable randomness | Honest-majority stake, Byzantine | Chain | slot-based | thousands | Cardano | Relates to weight = stake and Sybil resistance via a scarce resource | +| Sybil attack: Douceur, IPTPS 2002 [MEM] | Without a trusted identity authority, cheap identities defeat voting | n/a | n/a | n/a | n/a | n/a | Defines the open problem for CultMesh's self-assigned `weight` and `witnessRuntimeId` | + +Group-size/latency pattern in C: all classical BFT agrees on an ordered log, authenticated, with quorum 2f+1 of n >= 3f+1 over a fixed or sampled membership. Latencies from ms (small PBFT in LAN) to seconds (WAN, large n). I found no BFT protocol in this group specialised to a 20-60 Hz tick budget. + +### D. Game networking + +| Item | Core idea | Fault model | What is agreed/authoritative | Latency | Scale | Notes / relation | +|---|---|---|---|---|---|---| +| Deterministic lockstep (RTS): Bettner & Terrano, "1500 Archers on a 28.8", GDC 2001 [MEM]; Fiedler, "Deterministic Lockstep", gafferongames.com [MEM] | All peers exchange only inputs and run a bit-identical deterministic simulation | Honest peers; desync = bug (state hash compare detects it) | Inputs per turn | Input delay of a turn (~100-200 ms), stalls on slowest peer | 2-8 (RTS), hundreds of units | Related: agreement on INPUTS, detect divergence by state hash. A cheater who modifies local state is detected only by hash comparison; map-hack cannot be prevented because every peer has full state. CultMesh agrees on claim hashes about facts, not inputs | +| Rollback netcode / GGPO: Tony Cannon, 2009 [MEM] | Predict remote inputs, run ahead, rewind and resimulate on late inputs | Honest peers | Inputs; resimulate | Hides latency up to ~8 frames | 2-4 (fighting games) | Requires fast deterministic save/load and resimulate; CultMesh docs list "rollback/resimulation helpers" as planned, not built | +| Photon Quantum (Exit Games) [MEM; docs not fetched] | Deterministic ECS with fixed-point math, predict/rollback, a server-side input-relay and optional authoritative verification | Server-trusted; can run a replay/verification on the server | Inputs and checkpoints | Predict/rollback hides latency | Up to ~hundreds in some modes [unverified] | A commercial instance of "peers simulate, server verifies" | +| Client prediction with server authority (Quake/Source): Gambetta, "Fast-Paced Multiplayer" series, gabrielgambetta.com [MEM]; Valve Source networking / lag compensation doc [MEM]; Bernier, "Latency Compensating Methods", GDC 2001 [MEM] | Server simulates authoritatively; client predicts own input, reconciles to server snapshots; server rewinds for hit validation (lag compensation) | Cheating clients; server trusted | Server state | Tick 20-128 Hz, 50-100 ms perceived budget met via prediction | tens to ~100 per server | Direct match to CultMesh's current LIVE model (primary shard + `PutPredictedAsync`/`Reconciled`); the witness layer is an addition to it | +| Fiedler networked physics: "Snapshot Interpolation", "State Synchronization", "Deterministic Lockstep" [MEM] | Compare interpolation, delta compression, priority accumulators | n/a | n/a | n/a | n/a | Engineering reference for bandwidth/ticks | +| Interest management / AOI: Colyseus (Bharambe et al., NSDI 2006) [MEM]; Donnybrook (Bharambe et al., SIGCOMM 2008) [MEM]; Boulanger, Kienzle, Verbrugge survey (NetGames 2006) [MEM] | Send each client only entities near it; distribute zones/objects over servers or peers | Mostly trusted servers/peers | Authority per object/zone | Same as client-server | 100s-10,000s | Relates to CultMesh `shardId` per zone; shards are the authority unit | +| SpatialOS (Improbable) [partly WEB] | Cloud runtime that shards a world across worker servers by entity authority and AOI | Trusted operator | Per-entity authority delegated to workers | n/a | thousands of entities/clients per world claimed [MEM] | WEB: Jan 2019 Unity license dispute threatened SpatialOS-on-Unity games; Improbable sold Inflexion (Tencent, Feb 2022) and Midwinter (Behaviour, 2022) and sold its gaming unit in Dec 2023; pivoted to "Morpheus". The WEB results did NOT confirm an official SpatialOS end-of-life date or GDK deprecation (UNVERIFIED) | + +Pattern in D: realtime games meet 50-100 ms either by a trusted authoritative server with prediction, or by lockstep/rollback with an input delay among trusted peers. None of the mainstream techniques use a voting quorum among untrusted witnesses on the tick path. + +### E. Decentralised / P2P anti-cheat and verification + +| Item | Core idea | Fault model | Known limits | +|---|---|---|---| +| Lockstep with commitments: Baughman & Levine, "Cheat-Proof Playout for Centralized and Distributed Online Games", IEEE INFOCOM 2001, pp 104-113; revised as Baughman, Liberatore, Levine, "Cheat-Proof Playout for Centralized and Peer-to-Peer Gaming", IEEE/ACM ToN 2007 [WEB confirmed] | Each round, players first commit (hash) their move, then reveal; prevents look-ahead cheating. "Asynchronous synchronization" variant removes the slowest-player penalty | Cheating peers (protocol-level cheats: fixed delay, timestamp, suppressed update, inconsistency) | Round time bounded by the slowest player; needs all players; no defence against collusion or against a compromised client reading state (information exposure) [MEM for the limits] | +| NEO: GauthierDickey, Zappala, Lo, Marr, "Low latency and cheat-proof event ordering for peer-to-peer games", NOSSDAV 2004, pp 134-139 [WEB confirmed]; also NetGames 2004 paper (distributed games) | Rounds whose duration bounds latency to a MAJORITY of players, not the slowest; cheat-proof event ordering | Protocol-level cheats | WEB: a 2008 paper reports Corman et al. showed NEO cannot prevent all cheats as claimed and proposed Secure Event Agreement (SEA); full details UNVERIFIED (not opened) | +| Watchmen: Yahyavi, Huguenin, Gascon-Samson, Kienzle, Kemme, "Watchmen: Scalable Cheat-Resistant Support for Distributed Multi-player Online Games", IEEE ICDCS 2013, pp 134-144 [WEB confirmed] | Randomized dynamic proxy scheme for dissemination AND verification of actions; players verify each other's actions via proxies | Cheating players, including collusion (per abstract) | WEB abstract: prototype on Quake III, scales to hundreds of players, "significantly reducing" cheating "even under collusion" (not eliminating). Full evaluation numbers not read | +| RACS: Webb, Soh, Lau, "RACS: A Referee Anti-Cheat Scheme for P2P Gaming", NOSSDAV 2007 [MEM, search hit "Referee-based architectures for MMOGs" Curtin repo suggests Webb/Soh thesis; not confirmed] | A referee node (trusted/semitrusted) validates state updates; peers do the rest | Cheating players, trusted referee | Referee is single point of trust | +| Kabus, Terpstra, Cilia, Buchmann, "Addressing Cheating in Distributed MMOGs", NetGames 2005 [MEM] | Peers replicate objects and a subset acts as witnesses/validators checking each other's updates | Cheating peers | Reliant on honest majority in the object's replica set | +| Mutual checking / peer verification, Mönch et al., "Protecting online games against cheating", NetGames 2006 [MEM]; Corman et al. SEA, 2006 [MEM] | See above | | | +| Trusted execution (Intel SGX etc.) for cheat prevention [MEM] | Run authoritative logic in an enclave | Hardware-rooted trust | Repeated side-channel breaks of SGX; dependence on vendor attestation | +| Replay/verification servers (server re-simulates client input stream) [MEM] | Server accepts client-reported results only if re-simulation matches | Cheating clients | Costs server CPU equal to the simulation | +| Server-side authoritative simulation | The common industrial answer | Server trusted | Operator pays CPU/bandwidth | + +Known limitations across E (recurring in the literature) [MEM unless noted]: collusion (witnesses/peers that are friends), Sybil identities with no costly identity, latency of verification vs 50-100 ms budgets, information exposure (every peer holding full state can map-hack), network-level DoS on proxies/referees, and the need for deterministic simulation across heterogeneous clients (floating point, JS engines). WEB-confirmed for Watchmen: collusion is explicitly in scope and results are "significantly reduced", not eliminated. + +### F. Other close matches to "peers simulate, witnesses attest" + +| Item | Core idea | Relation | +|---|---|---| +| Optimistic rollups, fraud proofs: Arbitrum (Kalodner et al., "Arbitrum: Scalable, private smart contracts", USENIX Security 2018 [MEM]); Optimism [MEM] | Sequencer posts results; anyone re-executes; a challenge game (bisection over execution steps) proves one step on-chain; economic bond slashed | Closest structural match to "one party simulates, others independently re-simulate and dispute": needs deterministic VM, a challenge window (days on mainnet), and bonds. Latency class is far outside 50-100 ms; CultMesh has no challenge path today | +| Refereed delegation: Canetti, Riva, Rothblum, "Practical Delegation of Computation using Multiple Servers", CCS 2011 [MEM]; TrueBit (Teutsch & Reitwiessner, 2017/2019) [MEM] | Multiple servers compute; disagreement resolved by referee via bisection | Same family; assumes at least one honest server | +| Verifiable computation / zk (SNARKs, STARKs; zkVMs) [MEM] | Prover returns a proof the computation was done correctly | Removes trust in witnesses; proving cost far above real-time tick budget for a general sim (no real-time game case surveyed this pass) | +| N-version / replicated execution (BFT state-machine replication, Schneider 1990 "Implementing Fault-Tolerant Services Using the State Machine Approach", ACM Computing Surveys [MEM]) | Replicas execute the same deterministic ops and vote on outputs | The textbook ancestor of "peers independently simulate and vote on results"; the standard requires deterministic execution and agreed input order, which CultMesh does not yet provide | +| Output/response voting in BFT clients (PBFT client waits for f+1 matching replies) [MEM] | Client accepts a result when f+1 replicas agree | Structurally the same as accepting a candidate once enough matching claim hashes arrive; differs in that f is fixed and replicas are authenticated | +| Transparency-log witness cosigning (see C) | Quorum of independent signed attestations over a hash | Closest to CultMesh's "claim hash + quorum" at the data-structure level | + +--------------------------------------------------------------------------- + +## STEP 3. Synthesis + +### (1) Comparison across A-F + +"Realtime .io fit" = tens to hundreds of players, 50-100 ms budgets. Ratings are descriptive judgments from the facts above, with confidence noted. + +| Family | Fault model | What is agreed | Latency class | Group size | Cheating resistance | Realtime .io fit (descriptive) | +|---|---|---|---|---|---|---| +| A. Crash consensus (Paxos/Raft/VR/ZAB) | Crash only | Ordered log | ms (LAN) to 100s of ms (WAN) per commit | 3-7 | None (trusts members) | Fine as a coordination/lease/failover backend for shards, not per-tick; high confidence | +| B. CRDTs/OT | Crash/partition, trusted replicas | Converged state, no order | Zero coordination latency | Any | None | Fits presence/cosmetic state; cannot enforce game invariants; high confidence | +| C. BFT (PBFT/HotStuff/Tendermint/DAG) | Byzantine, n>=3f+1 | Ordered log of signed requests | ~100 ms-seconds, grows with n | 4-100ish | Strong against <1/3 faulty identities; none against Sybil without identity scarcity | Latency/size mismatch for tick-rate; plausible for low-frequency events; medium confidence | +| C'. Transparency-log witnesses | Byzantine log, honest-enough witnesses | Signed checkpoint hash | seconds-minutes | handful-dozens | Detects equivocation (split view) | Suited to auditing/after-the-fact, not tick path; medium confidence | +| D1. Authoritative server + prediction | Server trusted | Server state | one tick, 50-100 ms perceived hidden by prediction | tens-100s per server | Strong vs clients if server honest | The incumbent for the stated target | +| D2. Lockstep/rollback | Honest peers | Inputs | input delay 100-200 ms (lockstep) or hidden by rollback | 2-8 typical | Detects desync; no information hiding | Small-group only; high confidence | +| E. P2P anti-cheat (NEO, Watchmen, RACS, Kabus) | Cheating peers, limited collusion | Event order / verified actions via proxies/referees | Designed for realtime (Watchmen: Quake III prototype) | up to hundreds (Watchmen abstract) | Reduced, not eliminated; collusion and Sybil are known gaps | Closest in goal; research prototypes, not shipped infrastructure (to my knowledge, unverified) | +| F. Fraud proofs / VC | Byzantine with >=1 honest verifier or crypto proof | Execution result | Challenge window (minutes-days) or proving time | Open | Strong given determinism + bonds/proofs | Not on the tick path; plausible for deferred settlement | +| CultMesh today (witness plurality + primary shard commit) | Trusted primary; witnesses unauthenticated | Claim hash per (shard, epoch, frame, subject, kind) | In-process aggregation O(total observations) per submit | Unspecified | None against a lying witness or primary (see 1.4) | Latency is not the problem (no network round in aggregator); the trust model is | + +### (2) Design questions CultMesh's witness consensus must answer, tied to prior art + +| # | Question | Where prior art answered it | State in CultMesh source | +|---|---|---|---| +| 1 | What exactly is agreed: inputs, per-tick state hash, or sparse facts? | Lockstep/state-machine replication agree on inputs (Schneider 1990; Bettner & Terrano); rollups agree on state roots; CT on tree heads; CultMesh agrees on sparse fact hashes | Sparse facts (claimHash). No input log or state hash | +| 2 | Determinism requirement: can independent peers compute the same claim bits? | Lockstep (fixed-point, ordered inputs), Photon Quantum (fixed-point ECS), rollups (deterministic VM) | Not addressed; `ComputeClaimHash` hashes caller-chosen strings. CultMath exists in CultLib as parity-oriented math (`packages\cultmath`) but no link to witnesses found | +| 3 | Who may witness; selection and enrollment | BFT: fixed membership; Ethereum: random committee sampling; Ouroboros: stake-weighted VRF; Watchmen: randomized dynamic proxies; Sigsum: operator-chosen witness policy | None. Any runtime may submit; `AuthorityLease` exists for roles but isn't applied to witnesses | +| 4 | Authenticated identity of an observation | PBFT signed messages; CT cosignatures; HotStuff QCs | Unsigned, witness id self-asserted; bridge does not bind to peer identity in lines read | +| 5 | Sybil resistance / weight | Douceur 2002; stake (Ouroboros); invite-only membership (BFT) | `weight` is observer-supplied (line 52) | +| 6 | Quorum arithmetic: threshold over a fixed membership vs over observed votes | PBFT/HotStuff 2f+1 of 3f+1; Raft majority of configured members | Ratio of observed weight; MinimumWitnesses is the only count floor; default 1 | +| 7 | Collusion | Watchmen (random proxies per round), NEO/SEA, "honest majority of a randomly sampled set" (Ethereum) | Not addressed | +| 8 | Time/ordering: what is frame N, who orders facts, when is a fact final | Lockstep rounds; Raft log index; BFT sequence numbers; rollup challenge window finality | `frame` asserted by witness; commit at first quorum; no finality/retraction | +| 9 | Disagreement handling | Fraud proofs / bisection (Arbitrum, TrueBit); view change (PBFT); desync detection by state hash (lockstep); slashing | Minority recorded only as lower `Confidence`; later conflicting quorum silently skipped (FactId excludes claimHash) | +| 10 | Equivocation (same witness, two claims) | PBFT/Tendermint treat as Byzantine evidence; CT witnesses detect split views | Aggregator keeps one of them by weight then id, no evidence retained | +| 11 | Commit authority and its failure | Raft/Paxos leader election; BFT view change; Verse `PeerToPeer` is only a descriptor | Single shard primary, no election/failover (documented "Not implemented") | +| 12 | Cost per tick | Narwhal/Bullshark: seconds; Raft: 1 RTT; lockstep: input exchange each turn | Aggregator is O(N) rescans of the full observation list per submit; list is unbounded (lines 300, 347) | +| 13 | Bandwidth: observation fanout topology | Gossip (SWIM, Hashgraph), interest management (Colyseus/Donnybrook), proxies (Watchmen) | Fanout listed as "not implemented"; peer exchange exists for contact discovery only | +| 14 | Information exposure (peers hold full state) | Known limit of lockstep/P2P (Baughman & Levine; Kabus; Watchmen); server authority hides state | Not addressed | +| 15 | Replay/audit: can a third party verify a committed fact later | CT consistency proofs; rollup state roots; replay verification servers | Facts store counts and a hash only; no witness signatures or evidence archived | + +### (3) Novelty vs known constructions + +Judgement is descriptive; all of it rests on what was read, and I did not search the academic literature exhaustively (see unverified list). + +Known constructions under another name: +- Primary-per-shard with epochs and log catch-up = Raft/VR primary/term structure (the docs say so themselves). +- Authority lease with validity window, signed issuer, epoch = classical leases (Gray & Cheriton) with signed capabilities. +- "Quorum candidate" with claim hash and support count = result voting in replicated state machines (f+1 matching replies) or a CT-style quorum of cosignatures, minus signatures and fixed membership. +- Predicted/Reconciled input documents = client-side prediction with server reconciliation (Quake/Source; Gambetta). +- Shard = zone-based authority partition (Colyseus/SpatialOS-style). +- Fact keyed by (shard, epoch, frame, subject, kind) = idempotent event id. +- Peers checking each other's claimed events = the E family (Kabus, RACS, Watchmen), whose designs add selection, authentication and verification rules that CultMesh lacks. + +Possibly distinctive in this codebase (not shown to be novel in the literature): +- Combining a typed-document database (CultCache) with a witness-fact document type so witness facts travel through the same shard log, schema catalog and replication path as all other state (`CultMeshSimulationFact` is an ordinary `[CultDocument]`). I found no direct counterpart in the sources I looked at; this is an integration choice rather than a protocol. +- Treating witness opinion as an explicitly non-authoritative "fast mesh opinion layer" that feeds, rather than replaces, a primary commit. The literature has analogues (referee schemes with a trusted referee, RACS), so this is a framing difference more than a new construction. + +Nothing in the implemented witness path is a new consensus algorithm; it is a deterministic weighted-plurality aggregator (the tie-break and dedup rules are the only protocol content). + +### (4) Ten most relevant primary sources + +1. Yahyavi, Huguenin, Gascon-Samson, Kienzle, Kemme. Watchmen. IEEE ICDCS 2013, pp 134-144. https://hal.archives-ouvertes.fr/hal-00797352 [WEB-confirmed bibliographic data] +2. Baughman & Levine. Cheat-Proof Playout for Centralized and Distributed Online Games. IEEE INFOCOM 2001. https://www.comp.nus.edu.sg/~bleong/hydra/related/baughman01cheatproof.pdf and ToN 2007 version https://people.cs.umass.edu/~liberato/home/publication/cheat-proof-playout-for-centralized-and [WEB] +3. GauthierDickey, Zappala, Lo, Marr. NEO, NOSSDAV 2004. https://www.cs.du.edu/~chrisg/publications/chrisg-nossdav04.pdf [WEB]; read with the later critique/SEA (Corman et al.) cited at https://staff.csie.ncu.edu.tw/%7Ejrjiang/network2008/EASES.pdf [WEB result; content not opened] +4. Castro & Liskov. Practical Byzantine Fault Tolerance. OSDI 1999 [MEM; link not fetched: https://pmg.csail.mit.edu/papers/osdi99.pdf] +5. Yin et al. HotStuff. PODC 2019, arXiv 1803.05069 [MEM] +6. Ongaro & Ousterhout. Raft. USENIX ATC 2014; local copy `F:\Projects\CultLib\research\cultnet-distributed-database\raft-extended.pdf` +7. Baird. Swirlds Hashgraph TR-2016-01; local copy `F:\Projects\CultLib\research\cultnet-distributed-database\hashgraph-swirlds-tr-2016-01.pdf` +8. Schneider. Implementing Fault-Tolerant Services Using the State Machine Approach. ACM Computing Surveys 1990 [MEM] +9. C2SP tlog-cosignature and tlog-witness specs, https://github.com/C2SP/C2SP (tlog-cosignature at https://github.com/C2SP/C2SP/blob/tlog-cosignature/v1.0.0/tlog-cosignature.md) [WEB, specs not opened]; Sigsum https://www.sigsum.org +10. Kalodner et al. Arbitrum, USENIX Security 2018 [MEM] (fraud proof / bisection), paired with Spiegelman et al. Bullshark, CCS 2022, https://arxiv.org/abs/2201.05677 [WEB] as the current DAG-BFT state of the art. Honourable mentions: Shapiro et al. CRDT survey arXiv 1805.06358 (local copy); Gambetta, Fast-Paced Multiplayer https://www.gabrielgambetta.com/client-server-game-architecture.html [MEM]; Fiedler, gafferongames.com [MEM]; Douceur, The Sybil Attack, IPTPS 2002 [MEM]; Narwhal and Tusk arXiv 2105.11827 [WEB]. + +--------------------------------------------------------------------------- + +## Unverified / gaps + +- Everything tagged [MEM] was not re-fetched this pass: Paxos/Raft/VR/EPaxos/ZAB publication details, CRDT/OT/Automerge/Yjs/local-first details, PBFT, Tendermint, HotStuff, DAG-Rider, Ouroboros, Ethereum sync committees, GGPO, Photon Quantum, Gambetta/Fiedler/Valve pages, Colyseus/Donnybrook, RACS, Kabus et al., Mönch et al., Arbitrum/TrueBit/Canetti, Douceur, Schneider. +- WEB-confirmed only at citation/abstract level, not full text: Watchmen, Baughman & Levine, NEO, Narwhal/Tusk, Bullshark, C2SP witness specs, Improbable corporate events. NEO "cannot prevent all cheats" claim comes from a snippet of a 2008 paper, not read. +- SpatialOS end-of-life/GDK deprecation: NOT confirmed by the searches (they returned the 2019 Unity dispute and 2022-2023 divestments only). +- Photon Quantum scale figures ("hundreds") and its verification mode are from memory; no docs fetched. +- In CultLib: I read the observation server only to line ~70 (no peer-identity check seen), did not open the TS interop test body, `cultnet-py` simulation module, or `CultNetSchemaRegistry`. No "Witness" lease role was found by grep but the Authority resolver internals (`CultMeshAuthority*.cs`) were read only partially. Did not search Aetheria/StreamPixels design docs for witness-consensus intent beyond a filename hit list (`F:\Projects\Aetheria\docs\research\ew-prior-art.md` uses "compliance witness" in an unrelated game-fiction sense). `.claude\worktrees\agent-a1917e987955dada5` under CultLib duplicates some files; ignored. +- Latency/throughput numbers for A, C families are typical-literature figures from memory except where marked WEB. +- No empirical measurement of CultMesh aggregator cost was made. + +--- + +## Design notes from discussion (operator and Self, 2026-10-26) + +These are working directions from a design conversation, not rulings. Points marked "operator" are the operator's stated intent; the rest are analysis offered in that conversation. + +### Shape of the system + +- **Goal (operator):** a quorum of clients resolves local ordering disputes (who shot first) without a central authoritative server. The loser of a dispute is corrected exactly as under server authority with client-side prediction, but before the server has heard of the dispute. +- **Shape this implies:** an optimistic fast path among peers, with the server reduced to a slow path: audit and arbitration in the rare case where the quorum fails, splits, or is later shown to have lied. The server can overturn a peer decision after the fact and penalise provably dishonest witnesses. This is the structure of optimistic rollups (fraud proofs), applied at realtime latency; the closest game-networking prior art is Watchmen and NEO. +- **Cost (operator):** every client carries the lag-compensation machinery that is normally only on the server. + +### Prerequisites before any quorum can agree + +1. **Shared timeline.** Each party sees the other through latency, so "first" is undefined without a common reference: tick numbers plus signed, tick-stamped inputs. Without it, witnesses vote on their own latency. +2. **Commit before reveal.** Without it, a client can wait for the opponent's input and then claim an earlier action. Baughman and Levine (2001) solved this with commit-reveal lockstep. +3. **Escalation when the quorum fails or splits,** handled by the server slow path above. + +### Witness selection: network proximity over game proximity + +The operator questioned whether game-space proximity (area of interest) adds much over geographic or network proximity. Analysis in favour of network proximity: + +- **Neutrality.** Network-near clients are statistically disinterested in the dispute and rarely on either party's team. Game-near clients are disproportionately the parties' teammates, which makes collusion the main threat rather than an edge case. +- **Latency.** Low RTT among witnesses is what the quorum round actually needs. Game-near players can be 200 ms apart on the network. +- **Calculable risk.** Random selection from a network-near pool makes the probability of a captured quorum a binomial function of the colluding fraction. Game-near selection makes it depend on team composition. + +### Dispute-scoped verification keeps witnessing cheap + +A witness does not simulate the world, only the dispute. A dispute packet carries the last committed state of the entities involved and both parties' signed inputs with tick numbers. The witness rewinds the relevant hitboxes over a few ticks and signs its verdict: lag compensation in miniature, small in both CPU and bandwidth. Watchmen's randomly assigned proxies are the nearest precedent. + +### Problems this introduces + +- **Information leakage.** A witness learns positions it would not otherwise see; a witness colluding with a third party could act as a wallhack relay. Possible mitigations: pseudonymous entity identities in dispute packets, and adjudicating only past ticks. +- **History provenance.** If one party supplies the history, it can lie. Both parties' inputs must be signed by their authors and checked against previously committed state, which makes commit-before-reveal mandatory. +- **Cross-runtime determinism.** A witness may run a different runtime from the parties (for example a browser TypeScript client adjudicating between native C# clients). Every hit test must then produce bit-identical results across runtimes, which requires deterministic arithmetic (fixed-point or strictly specified floating point). CultLib's cross-runtime parity, and CultMath's HLSL-parity math, become security properties here rather than conveniences. This is also the strongest argument that witness-authoritative networking depends on a substrate like CultLib. + +### Gap to the current implementation (from the report above) + +The implemented path is a weighted plurality over self-asserted, unsigned claim hashes, committed by a trusted shard primary, with no fixed witness set, no failover, no determinism contract tied to witnesses, and happy-path tests only. A proof of concept (for example a browser .io game, which first needs CultMesh browser clients that can write) would need, at minimum: + +- signed observations +- an authenticated witness set whose weights are not self-declared +- explicit handling of disagreement and of a later contradicting quorum +- primary failover +- the shared timeline and commit-reveal described above +- a determinism contract across runtimes + +Its demonstration case is a cheating client being caught and corrected by witnesses.