GameCult/CultLib
Watch
9
0
Fork
You've already forked CultLib
 0
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions

Merge CultCache R1 (f789c8d1) into C2a: one reader per runtime, R1's failure taxonomy

Browse source 
R1's rules win where both sides decided the same question: a zero-byte file is
not a store (refused as unreadable in every runtime), only nothing at the path is
empty, a dangling link is an I/O error, refusals never echo stored values and echo
a header only as cultcache.store.v<digits>, writes report Rejected, NotReplaced
and ReplacedNotDurable. C2a's rules win on what it added: the v3 element-id
header, variants, whole-file proof, typed unreadable errors in C#, TypeScript and
Python. Rust keeps R1's CultCacheStoreUnreadable{path, kind}; C2a's
StoreUnreadableError is deleted. The readability manifest's zero-byte row is
refuses x4.

Not yet built or tested: verification follows in the next commits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Meta Vens 2026-10-02 19:41:35 +02:00
commit bad511f5ac
102 changed files with 16581 additions and 1837 deletions
Download patch file Download diff file Expand all files Collapse all files

10
contracts/cultnet/cultnet-distributed-database.md
View file

@ -104,6 +104,16 @@ Not implemented yet:
- cross-runtime rollback/resimulation helpers for simulation frames
- declared CRDT merge policies
## Error Codes
`cultnet.error.v0` carries an optional machine-readable `code`. The set of codes is a closed enum
(`cultnet.error.schema.json`): Rust and TypeScript refuse a message whose code they do not know, they do
not guess. Adding a code is therefore a coordinated change across every runtime, made together: the schema (both
copies), the C# `CultNetErrorMessage` factory, the Rust `CultNetErrorCode`, and a captured-bytes case in
`packages/cultnet-rs/tests/error_message.rs` for it. TypeScript reads the schema. The current codes are
`selection_invalid`, `cursor_stale`, `cursor_invalid`, `reference_outside_target`, `variant_unsupported` and
`unowned_schema` (a write, or a remote put or delete, for a schema and key no shard owns).
## Live Invariants
- CultCache owns document schema, identity, local indexes, and local diffing.

8
contracts/cultnet/cultnet.error.schema.json
View file

@ -26,9 +26,9 @@
}
},
"code": {
"description": "Machine-readable refusal code (R-N). Additive: a v0 peer that does not know this field ignores it.",
"description": "Machine-readable refusal code (R-N). Additive: a v0 peer that does not know this field ignores it. The set is a closed enum: Rust and TypeScript refuse a message whose code is not listed, so adding a code is one coordinated change across every runtime (this schema, the C# factory, the Rust CultNetErrorCode, and a captured-bytes case for each).",
"type": ["string", "null"],
"enum": [null, "selection_invalid", "cursor_stale", "cursor_invalid", "reference_outside_target", "variant_unsupported"]
"enum": [null, "selection_invalid", "cursor_stale", "cursor_invalid", "reference_outside_target", "variant_unsupported", "unowned_schema"]
},
"details": {
"description": "Code-specific structured detail (R-N). Every field is populated only for the code that defines it; the others are null, not omitted.",
@ -36,8 +36,8 @@
"additionalProperties": false,
"required": ["field", "value", "asOf", "current"],
"properties": {
"field": { "description": "selection_invalid: the selection field that failed. variant_unsupported: \"subscriptionId\", when a subscription was ended.", "type": ["string", "null"] },
"value": { "description": "selection_invalid: the offending value, when there is one string worth naming. variant_unsupported: the id of the subscription that ended.", "type": ["string", "null"] },
"field": { "description": "selection_invalid: the selection field that failed. variant_unsupported: \"subscriptionId\", when a subscription was ended. unowned_schema: \"schemaId\".", "type": ["string", "null"] },
"value": { "description": "selection_invalid: the offending value, when there is one string worth naming. variant_unsupported: the id of the subscription that ended. unowned_schema: the schema id no shard owns.", "type": ["string", "null"] },
"asOf": { "description": "cursor_stale: the cursor's own minted asOf.", "type": ["integer", "null"], "minimum": 0 },
"current": { "description": "cursor_stale: the answering server's current asOf.", "type": ["integer", "null"], "minimum": 0 }
}

12
docs/cultcache-publication-cut.md
View file

@ -826,3 +826,15 @@ Recorded, not forked:
- **Crash between the store write and the log append** leaves the log lagging the store, and nothing detects
it; a restarted primary reuses that sequence (A2's divergence reached by a plain kill). The spec called this
lag detectable; for a crash it is not.
- **Cut 4 final Soul (4d), recorded 2026-09-30 (Self), not fixed in Cut 4:**
- **F2, read cost.** `CultNetDatabase.Serves` (`:1980-1989`) runs per row in `GetAll`/`GetByName`/`GetByIndex` for every
layout except a single owns-everything shard: a reflection `GetTrackedKey`, a gate lock and a shard re-scan per row.
`database.GetAll` measured 24.8x `cache.GetAll` on 20,000 rows under a key-prefixed shard. Fix: filter by key inside
the cache enumeration (the `rowFilter` path already has keys) and hoist the owns-everything check.
- **F4, dispose drain edges.** An observer that removed its stash entry before the drain can publish after `Dispose`
returns (dropped); a throwing `_cacheJournal.Dispose()` skips the drain. Neither double-publishes or deadlocks.
- **F5, refusal naming.** A write whose schema is owned but whose key is outside the shard's prefix also answers
`unowned_schema` with details `{field:"schemaId"}`; the peer cannot tell a key refusal from a schema refusal.
- **Pre-existing, separate task.** Python emits error code `unsupported_schema_version`
(`cultnet-py/.../interop_peer.py:415`, `cultmesh-py/.../server.py:161`), which is not in the error-code enum; TS and
Rust refuse the whole message.

644
docs/cultnet-error-contract-cut.md Normal file
View file

@ -0,0 +1,644 @@
# CultNet error contract: cut map
Status: cut map, not started. Imagination pass 1, 2026-09-30, against CultLib `main` at `8fc74c70`.
This document owns the means. There is no separate target doc: section 1 holds the ends until the
operator rules on section 3. After that, Self moves the rulings and invariants into
`contracts/cultnet/cultnet-distributed-database.md` ("Error Codes") and the schema's own description.
Open: operator questions Q1-Q8 (section 3). No cut is dispatched until Q1, Q2, Q3 and Q8 are ruled. Q4-Q7
change only single emit sites and can be ruled during Cut 1.
Depends on: `hands/cultmesh-put-serve-bound` merging first. It introduces `document_unservable` in Python
(`fd394d54`), a codeless unservable refusal in cultmesh-ts (`0eb7434a`) and `CultMeshRudpRejectionReason` in
cultmesh-rs (`c9cdad12`, `85288a5a`). The cuts below convert those sites. Their anchors are by commit, not by
line, because the branch is not on main.
Map branch: `main`.
---
## 0. What the Body does today (probed)
Every claim in this table was established by running code. The C#, Rust and TypeScript probes ran on
Yggdrasil through `ygg-verify.sh` at `8fc74c70`. The Python bytes came from `msgpack.packb` on the exact
dicts `cultmesh_py/server.py:573-589` and `cultmesh-ts/src/index.ts:5921-5926` build. The six probe inputs:
| Input | Bytes from |
|---|---|
| `cs_cursor_stale` | Rust `tests/error_message.rs` captured C# hex (`cursor_stale`, details `{asOf:1,current:2}`) |
| `cs_routing` | C# `new CultNetErrorMessage { Error, RoutingHint = { Reason="not_primary", Shard=... } }` serialized with `CultNetSchemaMessageSerialization` |
| `cs_codeless` | C# `new CultNetErrorMessage { Error = "boom" }` |
| `cs_unknown_code` | C# with `Code = "document_unservable"`, empty `CultNetErrorDetails` |
| `py_snapshot_byte_limit` | the Python `_error_response` shape: `messageId`, `code: "snapshot_byte_limit_exceeded"`, `details {responseBytes, maxSnapshotBytes}` |
| `ts_two_key` | the cultmesh-ts shape `{schemaVersion, error}` |
Decoded by:
| Input | C# `CultNetSchemaMessageSerialization.Deserialize` | Rust `decode_cultnet_message_from_slice` | TS `parseCultNetMessage` |
|---|---|---|---|
| `cs_cursor_stale` | ok | ok, typed | ok |
| `cs_routing` | ok | ok, **routingHint dropped** (`code: None, details: None`) | ok (untyped passthrough) |
| `cs_codeless` | ok | ok | ok |
| `cs_unknown_code` | ok (free string) | **Err: "not a recognized cultnet.error.v0 code"** | **throws: /code must be one of the allowed values** |
| `py_snapshot_byte_limit` | ok, but **details silently become an all-null object** (`responseBytes`, `maxSnapshotBytes` lost), `messageId` dropped | **Err: not a recognized code** | **throws** (missing routingHint, extra messageId, code enum, details keys) |
| `ts_two_key` | ok | ok | **throws: missing routingHint, code, details** — TypeScript cannot parse the errors TypeScript sends |
Captured C# bytes for the two shapes Rust's vector file does not hold (for Hands' reuse):
- `cs_codeless`: `85AD736368656D6156657273696F6EB063756C746E65742E6572726F722E7630A56572726F72A4626F6F6DAB726F7574696E6748696E74C0A4636F6465C0A764657461696C73C0`
- `cs_routing`: see the probe; the shard descriptor writes ten keys, nil included.
Other findings, from reading source at `8fc74c70`:
1. **The interop CI lane has been red for 40 consecutive `main` runs.** It fails at "Verify CultNet schema
contracts stayed consolidated" (`.github/workflows/cultnet-interop.yml:67-90`). That step forbids every
`*.schema.json` outside `contracts/cultnet/`, and R-AB deliberately vendors 22 copies into
`packages/cultnet-rs/contracts/cultnet/`. Those copies are held byte-equal by
`packages/cultnet-rs/tests/schema_discovery_contracts.rs:74-99`. Nothing after that step (every
cross-runtime interop test) has run on main in that time. Run `36714768445` is the latest.
2. **The schema contradicts itself.** `contracts/cultnet/cultnet.error.schema.json:29` says the code is
"Additive: a v0 peer that does not know this field ignores it" and, in the same sentence, "a closed
enum: Rust and TypeScript refuse". The closed-enum text came from Soul pass 4d (`3f942a57`). It
described what the code did; no operator ruled it. `docs/` holds no ruling on unknown codes.
3. **Emitters that break the schema:**
- cultmesh-ts sends `{schemaVersion, error}` only, at `packages/cultmesh-ts/src/index.ts:5875-5878`,
`5885-5888`, `5913-5916` and `5922-5925`, and on the branch at `0eb7434a`.
- cultmesh-py `_error_response` (`packages/cultmesh-py/src/cultmesh_py/server.py:573-589`) adds a
top-level `messageId`, omits `routingHint`, omits `code` and `details` rather than sending nil, and
sends eight out-of-enum codes: `unsupported_schema_version` (`:161`), `malformed_document_put`
(`:358`), `unregistered_document_put` (`:365`), `malformed_document_delete` (`:386`),
`unregistered_document_delete` (`:393`), `snapshot_document_limit_exceeded` (`:558`),
`snapshot_byte_limit_exceeded` (`:567`) and `simulation_observations_disabled` (`:596`). The branch
adds a ninth, `document_unservable` (`fd394d54`).
- `packages/cultnet-py/src/cultnet_py/interop_peer.py:409-417` also sends `unsupported_schema_version`.
R-2 (`docs/cultnet-selection-cut.md:2046-2053`) required that code, but it never entered the schema
enum.
4. **A second schema authority.** `packages/cultnet-py/src/cultnet_py/schema_catalog.py:17`, `:272` and
`:357-362` publish a hand-built `cultnet.error.v0` descriptor under the canonical `$id`. It has a
different shape: `messageId`, `code` as any string, `details` open, and only `schemaVersion` and `error`
required. The same file does this for every wire message (about 350 lines). This is recorded as
follow-up F-1, not cut here.
5. **Silent refusals.** cultmesh-rs's RUDP document server returns every rejection to the local caller only
(`packages/cultmesh-rs/src/rudp_document_server.rs:470-590`, `CultMeshRudpApplicationRejection`). The
peer gets nothing and times out. On the branch this includes the new `DocumentUnservable`.
6. **Receivers drop the code.**
- Every C# receiver turns an error into `new IOException(error.Error)` or
`new InvalidOperationException(error.Error)`, so a caller cannot branch on the code. The sites:
`src/GameCult.Mesh/CultMeshBodySessions.cs:310`, `CultMeshContentSessions.cs:178/256-260`,
`CultMeshPeerExchange.cs:430/466`, `CultMeshSnapshots.cs:91/107/222-226/743` and
`CultMeshVerseDiscovery.cs:273`, plus `src/GameCult.Networking/CultNetShardReplication.cs:440/574`.
- Rust reads `Error { error, .. }` at `packages/cultmesh-rs/src/lib.rs:823`.
- TS types `CultNetErrorMessage` as `{schemaVersion, error}` only (`packages/cultnet-ts/src/contracts.ts:167-170`).
- Kotlin has no error type. A request that gets an error back fails a
`require(response.schemaVersion == ...)` (`packages/cultmesh-kotlin/src/main/kotlin/org/gamecult/cultmesh/CultMesh.kt:1278`,
`:1334`), and the peer's text is lost.
7. **No correlation.** Only Python echoes `messageId`.
- `cultmesh-browser` rejects an error only when exactly one operation is pending
(`packages/cultmesh-browser/src/index.ts:724-731`); otherwise the error is dropped and the operation
times out.
- C# `CultMeshContentSessions.OnError` (`:256-260`) fails every pending request on any error.
8. **The reference is inconsistent with itself.** `src/GameCult.Networking/CultNetDatabaseServer.cs:560`
refuses a variant change with a codeless error. The subscription server refuses the same condition with
`ForVariantUnsupported` (`CultNetDatabaseSubscriptionServer.cs:211/333/366`).
9. **routingHint is never consumed.** No receiver in any runtime reads it. C# builds it at
`CultNetDatabaseServer.cs:638-650`, and Rust's decode discards it.
10. **Consumer audit** (read-only, all of `F:\Projects` except CultLib; no git in Aetheria):
- Outside CultLib, nothing sets `code`, `details` or `routingHint`, and nothing branches on a code.
- Emitters:
- Huginn: `crates/huginn-daemon/src/serve.rs:288/307/312`, Rust struct literals
`Error { error, code: None, details: None }`, pinned at cultnet-rs `8fc74c70`.
- AetheriaEve: `Aetheria.State.Daemon/Program.cs:1737/1778`, C# object initializers.
- Decoders read `.error` only: Gjallar `Program.cs:574`, Mimir
`MimirOdinMoveProofEvidenceRingProvider.cs:80`, Hermodr `hermodr-daemon.cjs:1380` (hand-rolled),
AetheriaEve's smoke test (text match against its own emit text), and Huginn's tests (text match).
- Stale vendored TS:
- `StreamPixels/vendor/CultLib` carries the current closed schema, so any change here breaks its
TS error decode until it is re-vendored.
- `Heimdall/vendor/CultLib` carries a pre-R-X schema with `additionalProperties:false` and no
`routingHint`, so it already refuses every error C# sends today.
- Odin, Idunn, Muninn, Ghostlight and voidbot: zero hits.
## 1. Ends
- **One schema.** `contracts/cultnet/cultnet.error.schema.json` is the only authority on the shape and on the
code registry. C# embeds it (`GameCult.Networking.csproj:19`), TS mirrors it at build time, and Rust
vendors a copy held byte-equal by test. Nothing else publishes a `cultnet.error.v0` shape (F-1 covers
Python).
- **Every runtime emits only schema-valid errors.** This is proven per emit path against the schema, and
byte-equal to the C# reference for every registry code.
- **Every runtime decodes every error it can receive into a typed value.** A known code becomes its typed
variant with its typed details. An unknown code, if Q1 says so, becomes a typed "other" that keeps the
code string and the raw details. No well-formed error is ever refused, and no field is silently dropped
(routingHint, messageId).
- **Code and details have one owner.** The details a message carries are decided by its code. The types in
every runtime make "cursor_stale with a field" unrepresentable. No free-standing code string plus a
grab-bag details object.
- **A refusal is an answer, not a silence** (R-2 precedent, and Q4). A server that refuses a peer's request,
and can still send, sends a coded error that correlates to the request.
- **The public surface is complete in every runtime**, per the consumer-expectation doctrine:
- a typed error value;
- a constructor per registry code;
- a typed exception or error that request APIs raise when the peer refuses, carrying the typed value.
## 2. Identity, lifecycle and authority
| Kind | What names it | Lifecycle | Who decides |
|---|---|---|---|
| Error message | `schemaVersion: cultnet.error.v0`; correlated to its request by `messageId` (Q3) | Sent once, in reply to one request or subscription. Never persisted. | The server that refused. |
| Error code | A snake_case string in the schema's registry (`$defs/knownCode`) | Added by one coordinated change: schema, C# constructor, a C#-written vector, and a typed variant in every runtime. Never renamed and never reused. A retired code stays in the registry marked retired, so an old reader still decodes it. | The operator rules meaning; Self keeps the registry. |
| Details for a code | The code | Frozen with the code. Adding a key to an existing code's details is a new code (or a v1), never an in-place edit (see Q2). | Same as the code. |
| Unknown code, at a reader | The raw string | Preserved as typed `Other`. The reader acts on the `error` text and the absence of a known code. | The reader (Q1). |
| Vector file | `contracts/cultnet/interop/error-vectors.cs-written.json` | Regenerated only by the C# writer test under `CULTNET_WRITE_VECTORS=1` and committed. Every other runtime judges it read-only. | C#, the reference. |
| routingHint | Shard authority refusal | Sent with a null code (unchanged). Typed in every decoder. | CultNetDatabaseServer's authority path. |
## 3. Operator questions
**Rulings (operator, 2026-09-30):** "Q1: B / Q2: B / Q3: A / Q8: A".
- **Q1:** closed for emitters, open for readers. An unknown code decodes to a typed `Other`.
- **Q2:** per-code details.
- **Q3:** a nullable top-level `messageId`.
- **Q8:** v0 is corrected in place, with the semver bumps the policy requires.
- Q4-Q7 are still open.
- Cut 1 waits for `hands/cultmesh-put-serve-bound` to merge.
### Q1. What does a reader do with an error code it does not know?
Today Rust and TypeScript refuse the whole message. The probes above show the result: the peer's error text
is lost and the caller sees "not a recognized code". In cultmesh-rs's snapshot client the `?` at `lib.rs:812-815`
even turns a peer refusal into a local decode failure. So under a closed enum every new code breaks every
reader that has not upgraded. By `docs/semver-policy.md:86-91` that makes each new code a breaking wire
change, and nine codes are already in flight.
- **A. Closed enum (today).** Readers refuse unknown codes. Every code addition is a coordinated,
semver-major wire change across all runtimes and consumers.
- **B. Closed for emitters, open for readers.** The schema keeps the registry, and every runtime's emit
constructors can produce only registry codes (tests prove each emit path valid against the schema).
Readers decode an unrecognized code (matching `^[a-z][a-z0-9_]*$`) into a typed `Other { code, details }`.
A new code is an additive, minor change: old readers still surface the text and know "an error I cannot
classify". This is how gRPC status codes and protobuf open enums behave.
- **C. Fully open.** No registry; any string, emit and decode.
**Recommended: B.** A refusal must never fail to arrive because it is more specific than the reader
expected. B keeps the registry as the single meaning authority. C loses the meaning authority, and A makes
the error channel the most brittle message in CultNet.
Depends on this: schema `code` shape, every decoder, the doc text at `cultnet-distributed-database.md:107-115`.
### Q2. How are details shaped on the wire?
Today C# writes all four detail keys (`field`, `value`, `asOf`, `current`), nil included, because
`CultNetErrorDetails` (`CultNetSchemaMessages.cs:592-603`) is a MessagePack map-mode object. The codes in
flight need new keys: `responseBytes`, `maxSnapshotBytes`, `documentCount`, `maxSnapshotDocuments` and
`schemaVersion`.
- **A. One shared details record; grow it.** Every detail key of every code sits on one record, written as
nil when unused. Each new key changes the bytes of every existing error that has details, so all of the
C# vectors must be recaptured on every addition. It also breaks every strict reader: TS Ajv, and
StreamPixels' vendored schema, have `additionalProperties:false`. And the type cannot tie a key to its
code.
- **B. Per-code details.** `details` carries exactly the keys its code defines. A defined-but-optional key
is present as nil; another code's keys are absent. Keys are named for their meaning, which retires the
`field: "schemaId"` / `field: "subscriptionId"` indirection that exists only because the record is
shared. Adding a code never changes an existing code's bytes. In every runtime, code and details are
fused into one tagged type:
- Rust: an enum with data.
- C#: sealed subclasses, with a custom formatter for `CultNetErrorMessage`. There is precedent in
`src/GameCult.Mesh/CultMeshRouteRecordFormatter.cs`.
- TS: a discriminated union.
- Kotlin: a sealed class.
- Python: constructors per code over a frozen dataclass.
The churn is one time. The five existing C# cases with details change bytes, and all eight Rust hex
cases move into the shared vector file anyway (Cut 1).
- **C. Opaque string map.** Untyped; no runtime can offer a typed accessor.
**Recommended: B.** A is the churn the brief warned of, and it recurs on every future code. B pays it once,
at the moment every vector is being recaptured regardless.
Depends on this: the schema's `details`, the C# model and formatter, `encode_error_details`
(`packages/cultnet-rs/src/contracts.rs:1339-1366`), and every decoder.
### Q3. Does an error carry the `messageId` of the request it refuses?
Python already sends `messageId` (`server.py:583`); the schema forbids it. Without it:
- `cultmesh-browser` drops an error whenever two operations are pending (`index.ts:724-731`);
- C# content sessions fail every pending request on any error (`CultMeshContentSessions.cs:256-260`).
Adding a top-level key changes the bytes of every C# error. That happens once, in the same recapture as Q2.
- **A.** Add a nullable top-level `messageId`. A reply to a request echoes the request's id, and an error
with no request (a subscription push, an unsolicited refusal) sends nil. Receivers correlate by it and
fall back to their current behavior when it is nil.
- **B.** No correlation. Python deletes the field.
**Recommended: A.** Correlation is a capability any multiplexing client needs, and one runtime already
depends on it.
### Q4. Do servers send `document_unservable` (and every other refusal they can) to the peer?
Self ruled that "unservable" must be distinguishable as a typed code in every runtime. On the branch:
Python sends it coded, TypeScript sends it codeless, and Rust tells only its local caller, so the Rust
peer times out. The same Rust silence covers every other document-server rejection: a sink refusal, a
failed snapshot source, too many snapshot documents, and a snapshot response that is too large.
- **A.** Every refusal of a peer's request that the server can still send is answered with a coded error
carrying the request's `messageId`, and is also reported locally (Rust's `ApplicationRejected` stays as
the local report). The refusals that cannot be answered are the ones where the answer itself cannot be
queued: Rust `PayloadBudgetFull` and `ResponseQueueFailed`. Those stay local-only, and the doc says so.
- **B.** Local-only in Rust; peer replies only where a runtime already sends them.
**Recommended: A.** R-2 already ruled that a silence is worse than a missing feature
(`docs/cultnet-selection-cut.md:2049-2053`), and the operator's ruling that CultMesh "refuses" puts only
means something to the party that sent the put.
### Q5. Does Python's `unregistered_document_put` / `unregistered_document_delete` become `unowned_schema`?
Python sends these when the node has no registered document type for the record's schemaId
(`cultmesh_py/node.py:653-666`). C# `unowned_schema` means "no shard owns this schema and key"
(`CultNetSchemaMessages.cs:537-544`). To a peer both mean "this server will not take that schema here; route
elsewhere". The put and delete variants duplicate what the peer already knows (it sent the message).
- **A.** Both collapse into `unowned_schema { schemaId }`. The registry text broadens to "this server
stores no document for that schema (and key)".
- **B.** Join as a separate `schema_unregistered { schemaId }`.
- **C.** Join both codes as they are.
**Recommended: A.** It is one meaning at the peer, and there is one action to take.
### Q6. Does Python's `malformed_document_put` / `malformed_document_delete` become one `message_invalid { field }`?
These are sent when the request lacks a required field (`server.py:355-359`, `383-387`). C# has the same
condition, codeless, at `CultNetDatabaseServer.cs:373/415` and `CultNetDatabaseSubscriptionServer.cs:148`
("requires a subscriptionId or messageId").
- **A.** One `message_invalid { field }`, naming the missing or ill-typed field. It is used by Python's four
sites and C#'s three.
- **B.** Join both per-message codes and leave C# codeless.
**Recommended: A.** The per-message suffix repeats the request's own schemaVersion, and C# gains a code for
free.
### Q7. Does `simulation_observations_disabled` become `unsupported_schema_version`?
Python sends it when the observation hub is off (`server.py:593-598`), with
`details { schemaVersion: "cultnet.simulation_observation.v0" }`, which is exactly the
`unsupported_schema_version` shape. cultmesh-ts's "no cache for snapshot requests" and "no operation
handler" refusals (`index.ts:5875`, `5913`) mean the same thing: this server does not serve that message.
- **A.** Collapse. `unsupported_schema_version` means "this server does not serve messages of this
schemaVersion: unknown, a newer version, or not enabled here". The three TS sites and Huginn's "answers
only X, Y, Z" adopt it.
- **B.** Keep a separate `feature_disabled`-style code.
**Recommended: A.** A peer cannot act differently on "not enabled" and "not known". Python's
`verify.py:335` check follows the rename.
### Q8. Is this a correction of `cultnet.error.v0` in place, or a new `cultnet.error.v1`?
Under Q2-B and Q3-A, strict v0 readers (TS Ajv and vendored copies) refuse the new bytes. Tolerant readers
(C#, Rust, Python) keep working, because the probes show they ignore unknown keys and treat a missing key
as null.
- **A. In place.** The v0 contract has never been decodable across all five runtimes (section 0). No
consumer outside CultLib reads code or details (finding 10). Every publishing package takes the semver
bump its policy requires:
- `org.gamecult.cultlib` major, because the public `CultNetErrorDetails` properties change;
- `cultnet-py` and `cultmesh-py` major, because emitted codes change;
- Rust pins move deliberately.
StreamPixels and Heimdall re-vendor.
- **B. v1.** Add `cultnet.error.v1`, keep v0 emit for peers that did not advertise v1 in hello, and run
two error encoders in every runtime until v0 retires.
**Recommended: A.** B builds a version-negotiation path for a message nobody outside CultLib parses beyond
`.error`. The one thing every old reader does need, the `error` text, survives A in every tolerant runtime.
## 4. The registry (as recommended; rewrite on each ruling)
| Code | Meaning | Details (per Q2-B; `?` = present-as-nil when absent) | Emitted by after this map |
|---|---|---|---|
| `selection_invalid` | A selection field failed the door. | `{ field, value? }` | C#, Rust (selection) |
| `cursor_stale` | The cursor predates the server's retained history. | `{ asOf, current }` | C#, Rust |
| `cursor_invalid` | The cursor could not be parsed. | none (details nil) | C#, Rust |
| `reference_outside_target` | A hop reference left the declared target. | none | C#, Rust |
| `variant_unsupported` | The selection selects a document variant CultNet does not carry. | `{ subscriptionId? }` | C# (incl. `DatabaseServer.cs:560`) |
| `unowned_schema` | This server stores no document for that schema (and key). | `{ schemaId }` | C#, Python (Q5) |
| `unsupported_schema_version` | This server does not serve messages of this schemaVersion. | `{ schemaVersion }` | Python, TS, Huginn (Q7) |
| `message_invalid` | The request lacks a required field or has an ill-typed one. | `{ field }` | C#, Python (Q6) |
| `snapshot_document_limit_exceeded` | The snapshot answer would exceed the document bound. | `{ documentCount, maxSnapshotDocuments }` | Python, Rust (Q4) |
| `snapshot_byte_limit_exceeded` | The snapshot answer would exceed the byte bound. | `{ responseBytes, maxSnapshotBytes }` | Python, Rust (Q4) |
| `document_unservable` | A put was refused because no snapshot could ever return the stored document under the byte bound. | `{ responseBytes, maxSnapshotBytes }` | Python, TS, Rust (Q4) |
A null code is legal and means "unclassified". Examples: an exception message from a failed snapshot
source, an authority refusal carrying `routingHint`, and "not authorized"
(`CultNetDatabaseSubscriptionServer.cs:165`). This map does not invent codes for those.
`snapshot_byte_limit_exceeded` and `document_unservable` have identical details. That is deliberate, and it
is the fixture trap named in the verification rules.
---
## 5. Cuts
All cuts are made on one campaign branch, `hands/error-contract`, from `main` after
`hands/cultmesh-put-serve-bound` merges. The branch merges to `main` once, after Cut 5's Soul pass. The
reason: Cut 1 changes C# bytes, and Rust's static hex cases would stay green against the stale bytes
(finding: `tests/error_message.rs` pins constants, not the reference). A partial merge would ship a
wire split while every suite passed. Each cut gets its own worktree, created by Self. Heavy verification
runs through `ygg-verify.sh`, never on Starfire.
### Cut 0. Unblock the interop lane
- **Repo/branch:** CultLib, `hands/interop-guard` from `main`, merged alone ahead of the campaign. It
does not depend on the rulings.
- **Deletes first:** nothing.
- **Per-file changes:** `.github/workflows/cultnet-interop.yml:72`. The `$stray` filter also accepts
`^packages/cultnet-rs/contracts/cultnet/`, with a comment naming R-AB and
`packages/cultnet-rs/tests/schema_discovery_contracts.rs:74-99` as the equality guard. The namespace
check is unchanged.
- **Authority map:** no ownership change. The canonical owner stays `contracts/cultnet/`, and the Rust
mirror is derived from it, with drift judged by the Rust test.
- **Verification:**
- Push the branch and run the workflow. Report every step that goes red after the guard. None of those
steps has run on main for 40 runs, so expect more red, and treat each failure as its own finding;
do not fix it in this cut.
- Negative: edit one byte of a vendored copy locally and check that the Rust equality test fails.
Reach it on Yggdrasil with `cd packages/cultnet-rs && cargo test --test schema_discovery_contracts`.
- **Ledger:** +3 lines.
### Cut 1. The contract and the C# reference
- **Repo/branch:** CultLib `hands/error-contract`, worktree named by Self.
- **First:** capture the current C# bytes for all eight factory cases, plus `cs_codeless` and `cs_routing`,
into the Hands report. This is the before-image a Soul pass compares against.
- **Deletes first:**
- `CultNetErrorDetails` (`src/GameCult.Networking/CultNetSchemaMessages.cs:588-603`, 16 lines) as a
shared record.
- The `field`/`value`/`asOf`/`current` property block and the `[MessagePackObject]`/`[Key]` attributes
on `CultNetErrorMessage` (`:497-521`). Its serialization moves to one formatter.
- `SoulP4dProbeTests.Wire_UnownedSchemaBytes` (`tests/GameCult.Networking.Tests/SoulP4dProbeTests.cs:191-199`).
It is an `[Explicit]` hex printer, superseded by the vector writer.
- The closed-enum paragraph at `contracts/cultnet/cultnet-distributed-database.md:107-115` and
the `code`/`details` descriptions in the schema (`:28-42`).
- **Adds:**
- **Schema** (`contracts/cultnet/cultnet.error.schema.json`, rewritten; the Rust mirror is re-copied
byte-equal):
- Top level: `schemaVersion`, `error` and `routingHint` as today, plus `messageId: string|null` (Q3)
and `code`. All five are required, present-as-nil (R-X is unchanged at the top level).
- `$defs/knownCode` is the registry enum from section 4.
- `code` is `null`, or a string matching `^[a-z][a-z0-9_]*$` (Q1-B).
- `details` is decided by a top-level `oneOf`, one branch per known code, each fixing `code` by
`const` and `details` by exact `properties`/`required`/`additionalProperties:false`. Two more
branches: a null code with null details, and an unknown code (`not: {$ref knownCode}`) with details
`object|null`, open.
- Use `oneOf`/`not`/`const` only. `tests/GameCult.Networking.Tests/MiniJsonSchemaValidator.cs:30-35`
implements those and **not** `if`/`then`/`allOf`, and it refuses unimplemented keywords. TS
compiles with Ajv2020 (`packages/cultnet-ts/src/contracts.ts:1,674`), which handles both.
- **C# model** (`GameCult.Networking`):
- `CultNetErrorMessage { SchemaVersion, Error, MessageId?, RoutingHint?, Refusal? }`, where
`Code => Refusal?.Code` is derived and not settable.
- `abstract CultNetRefusal { string Code }`, with one sealed subclass per registry code carrying its
typed details (`CultNetCursorStaleRefusal(ulong AsOf, ulong Current)` and so on), plus
`CultNetUnknownRefusal(string Code, IReadOnlyDictionary<string, object?>? Details)` for Q1-B.
- `CultNetErrorCodes`: string constants, one per registry code.
- `CultNetErrorMessageFormatter : IMessagePackFormatter<CultNetErrorMessage>`. It writes the five
top-level keys in order (schemaVersion, error, messageId, routingHint, code, details), and details
as exactly the refusal's keys. It reads by key name, tolerates unknown keys at both levels, and
maps an unknown code to `CultNetUnknownRefusal`. Register it through `[MessagePackFormatter]` on
the type, the way `CultMeshRouteRecordFormatter` is wired.
- Factories: keep `ForSelectionInvalid`, `ForCursor`, `ForReferenceOutsideTarget`,
`ForVariantUnsupported` and `ForUnownedSchema` as thin constructors over the refusal types, and add
one per new code. Each takes an optional `messageId`.
- `CultNetPeerErrorException : IOException`, carrying the `CultNetErrorMessage`.
- **Vector writer:** `ErrorVectorTests.WriteErrorVectors`, gated on `CULTNET_WRITE_VECTORS=1` like
`SelectionParityVectorTests.cs:460-552`. It writes `contracts/cultnet/interop/error-vectors.cs-written.json`:
one entry per registry code (both optional-key states where a key is optional), `cs_codeless`,
`cs_routing`, one unknown code with open details, and one error with `messageId` set. Each entry
holds `{ name, messagePackBase64, expected: { error, messageId, code, details, routingHint } }`,
where `expected` is a JSON projection every runtime compares its typed decode against.
- **Per-file changes** (`8fc74c70`):
- `CultNetDatabaseServer.cs:560`: `ForVariantUnsupported(refusal.Message, subscriptionId)`.
- `CultNetDatabaseServer.cs:373`, `:415` and `CultNetDatabaseSubscriptionServer.cs:148` become
`message_invalid { field: "subscriptionId" }` (Q6-A).
- Every `SendCultNet(CultNetErrorMessage...)` that answers a request passes the request's `MessageId`.
- Each receiver listed in finding 6 throws `new CultNetPeerErrorException(error)` instead of
wrapping `error.Error`. `CultMeshContentSessions.OnError` (`:256-260`) fails only the pending entry
whose id matches `error.MessageId`, and falls back to all entries only when the id is nil.
- `CultNetSchemaContractTests.cs:165-214`: the per-factory tests are replaced by one test that validates
every vector's decoded JSON against the schema with `MiniJsonSchemaValidator`, and one test that
serializes each registry factory and asserts byte equality with its committed vector.
- Rewrite `contracts/cultnet/cultnet-distributed-database.md` "Error Codes" to the registry, and state
the Q1 and Q2 rules.
- **Authority map:**
- Owner: the schema's registry owns code meaning and detail shape. `CultNetErrorMessageFormatter` owns
C# bytes. The refusal subclass owns the code-to-details binding.
- Inputs: the refusal instance, messageId, routingHint.
- Outputs: bytes, and `cs-written.json` for the other runtimes.
- Derived state: `CultNetErrorMessage.Code` becomes derived; the refusal decides it.
- Forbidden writers: any `Code = "..."` assignment (it cannot compile); `[Key]` attribute serialization
of the message; hand-typed hex anywhere.
- Shared paths: every server send uses the factories; every receiver uses `CultNetPeerErrorException`.
- Deletion line: `CultNetErrorDetails` and the settable `Code`.
- **Verification** (`dotnet` image):
- Builds: `tests/GameCult.Networking.Tests`, `tests/GameCult.Mesh.Tests`.
- `ErrorVectorTests` with `CULTNET_WRITE_VECTORS=1`, then without, and check that the committed file
reproduces byte-for-byte.
- Tests and the rules they pin:
- vector round trip: bytes, decode, bytes are equal for every entry;
- a registry factory is byte-equal to its vector;
- every vector validates against the schema;
- an unknown code decodes to `CultNetUnknownRefusal` with its raw details intact;
- a Python-shaped error (`py_snapshot_byte_limit` above, with the Q2/Q3 shape) decodes typed with
`responseBytes` and `maxSnapshotBytes` intact. Today's C# silently empties those.
- content-session correlation: two pending requests, an error for the second, and the first still
completes.
- Mutation: Stryker.NET on the formatter and factories. The survivors that must die:
- swapping two known codes whose details are identical (`snapshot_byte_limit_exceeded` and
`document_unservable`);
- padding a nil key into details;
- dropping `messageId` on encode;
- mapping an unknown code to null.
- Negative greps (checked at HEAD, where they hit the sites being cut):
- `git grep -nE 'new (IOException|InvalidOperationException)\(error\.Error\)' -- src` must return 0
(9 hits at HEAD).
- `git grep -nE 'Field = "(schemaId|subscriptionId)"' -- src` must return 0 (2 at HEAD).
- Operator: none.
- **Ledger:** about -60 C# (the details record, the settable code, the probe printer, and the per-factory
schema tests), +250 (the formatter, the refusal types, the vector writer). The schema grows by about 90
JSON lines. The net is positive, bought by Q1/Q2 typing in the reference.
### Cut 2. Rust: cultnet-rs and cultmesh-rs
- **Deletes first:**
- `CultNetErrorCode` and `CultNetErrorDetails` as separate types (`packages/cultnet-rs/src/contracts.rs:252-310`,
59 lines);
- `require_error_details` (`:1121-1146`);
- `encode_error_details` (`:1339-1366`);
- the eight hex `Case`s and the header comment in `packages/cultnet-rs/tests/error_message.rs:1-160`.
That file is rewritten to read the vector file.
- Keep `require_legacy_optional_u64` (`:1149`). Check with `git grep` for other callers first; at HEAD,
only the error details use it.
- **Adds:**
- `CultNetMessage::Error { error: String, message_id: Option<String>, routing_hint: Option<CultNetShardRoutingHint>, refusal: Option<CultNetRefusal> }`.
- `enum CultNetRefusal`, with one variant per registry code and its typed fields, plus
`Other { code: String, details: Option<rmpv::Value> }`.
- `CultNetRefusal::code(&self) -> &str`.
- A constructor `CultNetMessage::error(text)` and builder-style `with_refusal` / `with_message_id`, so
consumers stop writing struct literals. Huginn has three.
- Decode reads every top-level key by name and routes unknown codes to `Other` (Q1-B).
- Encode writes the Cut 1 key order.
- `CultNetShardRoutingHint` is typed, reusing the crate's existing shard descriptor type.
- **Per-file changes:**
- `packages/cultnet-rs/src/contracts.rs:361-368` (variant), `:1097-1115` (decode), `:1300-1336` (encode).
- `packages/cultnet-rs/src/selection.rs:1210-1255`: the four refusals build `CultNetRefusal` variants.
- `packages/cultnet-rs/src/content.rs:316`: uses `CultNetMessage::error`.
- `packages/cultmesh-rs/src/lib.rs:812-827`: a peer error becomes a typed error carrying the refusal,
instead of `anyhow!` text. Name it `CultMeshPeerError`, next to the crate's existing error types.
- `packages/cultmesh-rs/src/rudp_document_server.rs` (Q4-A): each `CultMeshRudpRejectionReason` that
can be answered sends the matching coded error, correlated by the request's message id, before
returning `ApplicationRejected`. `PayloadBudgetFull` and `ResponseQueueFailed` stay local.
- **Authority map:**
- Owner: `CultNetRefusal` owns code and details. The schema owns the registry.
- Forbidden writers: string codes (`from_wire_str` survives only inside decode); struct literals of
details.
- Shared paths: selection refusals, document-server refusals and content errors all go through the one
encoder.
- **Verification** (`rust` image, `cd packages/cultnet-rs` then `cd ../cultmesh-rs`):
- `cargo test --workspace`.
- `tests/error_message.rs` pins: every vector decodes to `expected`; encoding the decoded value
reproduces the vector's bytes; the unknown-code vector decodes to `Other`; the routing vector keeps
`routing_hint`.
- cultmesh-rs: a put over the bound makes the peer receive `document_unservable` with the request's
message id, and details equal to the computed size and the bound. Run it at two bounds so that a
constant or an offset in `maxSnapshotBytes` dies.
- A peer that sends a Python-shaped `snapshot_byte_limit_exceeded` produces a `CultMeshPeerError`, not
a decode failure.
- Mutation: `cargo mutants --in-diff` on the cut's diff. Triage every survivor by name.
- Negative: `git grep -n 'CultNetErrorDetails' -- packages` must return 0.
- **Consumer note:** Huginn's `serve.rs:288/307/312` and its tests stop compiling at the next pin bump.
The migration is mechanical (`CultNetMessage::error(...)`), and `:312` adopts `unsupported_schema_version`
(Q7). Self routes it as a Huginn change; it is not part of this branch.
### Cut 3. TypeScript: cultnet-ts, cultmesh-ts and cultmesh-browser
- **Deletes first:**
- the two-field `CultNetErrorMessage` interface (`packages/cultnet-ts/src/contracts.ts:167-170`);
- the four literal error constructions in `packages/cultmesh-ts/src/index.ts:5875-5878`, `5885-5888`,
`5913-5916` and `5922-5925`, and the branch's codeless unservable site (`0eb7434a`);
- `rejectCorrelated`'s single-pending heuristic (`packages/cultmesh-browser/src/index.ts:724-731`).
- **Adds:**
- `CultNetErrorMessage` with every field.
- `type CultNetRefusal` as a discriminated union on `code`, with an `{ code: string; details: Record<string, unknown> | null; known: false }` arm.
- `createCultNetError(text, { messageId, refusal })` as the only constructor.
- `class CultNetPeerError extends Error { message: CultNetErrorMessage }`.
- `parseCultNetMessage` keeps Ajv validation against the canonical schema; it now accepts every schema
branch, including unknown codes.
- **Per-file changes:**
- cultmesh-ts: the three "does not serve this" sites (`5875`, `5913`, `5922`) send
`unsupported_schema_version` (Q7-A); the snapshot catch (`5885`) sends a null code; the unservable site
sends `document_unservable` with details.
- cultmesh-ts `6527-6529` throws `CultNetPeerError`.
- cultmesh-browser `163`, `584` and `690`: correlate by `messageId`, falling back to today's behavior only
for a nil id.
- `packages/cultnet-ts/src/idunn-odin-presence-publisher.ts:118-120` throws `CultNetPeerError`.
- The legacy `gamecult.networking.v0` path (`contracts.ts:840-844`, `:899-900`) is unchanged. That is a
separate legacy contract.
- **Verification** (`kotlin` image, which carries Node 24; `npm install`, then build cultcache-ts before
cultnet-ts):
- A new `test/error-vectors.test.ts` pins that every vector parses and deep-equals `expected`, and that
`encode(createCultNetError(...))` is byte-equal to each registry vector.
- The cultmesh-ts document-server tests assert that each refusal parses with TypeScript's own
`parseCultNetMessage`. That is the probe's `ts_two_key` failure, pinned.
- Browser: two pending operations, an error for the second, and the first still resolves.
- Mutation: StrykerJS on the diff.
- Negative: `git grep -nE 'schemaVersion: "cultnet\.error\.v0",' -- packages/cultmesh-ts/src packages/cultmesh-browser/src`
must return 0. Exclude `cultnet-ts/src/schema-discovery.ts:302` (a descriptor) and
`contracts.ts:842` (legacy decode); both are legitimate hits for the unscoped pattern.
### Cut 4. Python: cultnet-py and cultmesh-py
- **Deletes first:**
- `_error_response` (`packages/cultmesh-py/src/cultmesh_py/server.py:573-589`);
- `unsupported_schema_version_error` (`packages/cultnet-py/src/cultnet_py/interop_peer.py:409-417`);
- every string literal code in `server.py` (lines in finding 3).
- **Adds:**
- `cultnet_py/errors.py`, holding:
- `CultNetErrorCode(StrEnum)`, the registry;
- a frozen `CultNetError` dataclass (`error`, `message_id`, `routing_hint`, `code: CultNetErrorCode | str | None`,
`details: Mapping | None`);
- one constructor function per code, taking exactly that code's keys;
- `to_wire()`, which writes the five top-level keys in the Cut 1 order, nil included;
- `parse_cultnet_error(dict)`.
- `CultNetPeerError` (`client.py:46-50`) keeps `.response` and gains `.error: CultNetError`.
- **Per-file changes:**
- Sites `server.py:158-163`, `355-394`, `555-569` and `593-598` and the branch's `document_unservable`
site use the constructors, with the Q5-A/Q6-A/Q7-A codes.
- `interop_peer.py` uses the constructor.
- `verify.py:335` reads `error.error.code`.
- Tests `packages/cultmesh-py/tests/test_cultmesh.py:2088`, `3791`, `4323` and `4363-4371`, and
`packages/cultnet-py/tests/test_cultnet.py:2342`, follow the renames.
- **Verification** (the image the campaign already uses for Python; state it in the report):
- A vector test pins that every vector parses to `expected` and that each constructor's `msgpack.packb`
is byte-equal to its vector.
- Mutation: mutmut on `errors.py` and the changed server sites.
- Negative: `git grep -nE '"schemaVersion": "cultnet\.error\.v0"' -- packages/cultnet-py/src packages/cultmesh-py/src`
returns only `errors.py`.
- **Not here:** F-1 (`schema_catalog.py` publishes its own shapes).
### Cut 5. Kotlin: cultmesh-kotlin
- **Adds:**
- A `CultNetErrorMessage` data class and a `sealed class CultNetRefusal` (known codes plus `Other`),
with a `CultNetMessage.asError(): CultNetErrorMessage?` view.
- `class CultNetPeerException(val error: CultNetErrorMessage) : IOException`.
- At `CultMesh.kt:1278` and `:1334`, and at every `receiveSchemaMessage` consumer that expects a
response, an error response throws `CultNetPeerException` before the `require`.
- **Verification** (`kotlin` image, `pwsh -File build.ps1`):
- A vector test pins that every vector decodes to `expected`.
- Kotlin emits no errors today, so encode parity applies only if a Kotlin server path is added. Say
so in the report; do not invent one.
- There is no mutation tool. Behavioural tests cover it: an error reply to a schema-catalog request
raises `CultNetPeerException` carrying the code.
### Cut 6. Merge, release, consumers
- Soul runs over the whole branch before the merge:
- rerun every vector suite;
- decode each runtime's live emissions with every other runtime. The interop lane from Cut 0 is the
harness; add an error round to `tests/GameCult.Networking.InteropPeer` and
`packages/cultnet-ts/test/interop` if Soul finds a runtime pair unreached.
- Release per `docs/semver-policy.md`, with a Soul pass before tags:
- `org.gamecult.cultlib` major, and the tracked Unity DLLs rebuilt, since they embed the schema;
- `cultnet-py` and `cultmesh-py` major.
- Consumers, routed by Self as separate repo changes:
- Huginn: pin bump plus Cut 2's note.
- StreamPixels and Heimdall: re-vendor. Heimdall is already broken by R-X.
- AetheriaEve, Gjallar and Mimir compile against the live CultLib path and read only `.Error`, so they
need no change. That is inferred from the audit; the next build of each confirms it.
- Hermodr: none.
## 6. Subtraction ledger
| Cut | Removed (estimate) | Added (estimate) | Formats / targets |
|---|---|---|---|
| 0 | 0 | 3 | none |
| 1 | ~60 C#, schema text | ~250 C#, ~90 schema | + `error-vectors.cs-written.json`; - `[Explicit]` hex printer |
| 2 | ~150 (types, two helpers, 160-line hex test) | ~200 | hex constants gone; vectors read from file |
| 3 | ~40 | ~120 | none |
| 4 | ~40 | ~120 (`errors.py`) | 8 ad-hoc codes → registry |
| 5 | 0 | ~80 | none |
The net is positive, by roughly 500 lines, across five runtimes. It buys typed decode where there was
none (Kotlin, TS and Python), a correlation capability, and one vector file in place of hand-copied hex.
The pressure point: if Cut 1's formatter passes 150 lines, report it rather than trimming typing.
## 7. Follow-ups outside this map
- **F-1.** `packages/cultnet-py/src/cultnet_py/schema_catalog.py` publishes about 350 lines of hand-built
schemas under canonical `$id`s. Python should package and serve `contracts/cultnet/*.schema.json`, the
way Rust vendors them. It is a separate cut, because it covers every message, not errors. It can wait
because the catalog is advisory: no runtime validates a peer's message against a published descriptor.
- **F-2.** Codes for the remaining codeless refusals (authorization, failed snapshot sources, routing).
Wait for a consumer that branches on them.
- **F-3.** Whatever the Cut 0 lane reveals once its later steps run again.

50
docs/cultnet-rudp-ack-cut.md
View file

@ -622,6 +622,12 @@ The ack cuts build on these behaviours and do not alter them:
drops what it owed. How a new generation is recognised (Cut 1c) needs one (Q-A6). The two are separable
because 1b does not forget peer state: the peer may not know the session ended, and forgetting what was
received would let its retransmits be delivered twice.
**Correction (Cut 1/1b Soul, 2026-09-30): they are separable as specs, not as merges.** 1b drops owed
writes, which leaves holes the sender will never fill; the receiver keeps `R` across the ending, so if the
next handshake does not reset it, every later ordered frame is held and acked, and flush succeeds while
nothing is delivered. Only 1c's reset on a non-repeated handshake removes that. **1b merges only together
with 1c.** Soul also found `R` seeded by the first reliable packet rather than the handshake; ruled:
`R` is seeded only by the handshake, and reliable data before the Accept is neither acked nor delivered.
- **What must hold after both.** A write is Acknowledged or Invalidated within the generation it was
issued in. A restarted client is admitted. A new server session's frames are delivered after a
reconnect. A retransmitted Connect is still a repeat.
@ -1543,3 +1549,47 @@ Each question stands alone.
`packetsDropped` until K1. Record the stats shape in the parity doc once both land.
- **The P4 probe at the old pins** predates its fix; rerun `09318d13`'s P4 against a pinned sender if
old-sender evidence is needed.
**Follow-up (media FEC Cut 3 Hands, 2026-09-30):** `channel_send_options` in `rudp.rs` has no test for the
`"schema"` and `"latest"` channel arms. Deleting either falls to the default and every test stays green. Pin their send
flags on the wire the way the media arm is now pinned (raw peer socket), in whichever cut next touches channel
profiles.
**Rulings after the Cut 1/1b/1c Soul pass (Self, 2026-09-30):**
- **Stale data after a reconnect.** A reliable sequence at or before the generation's seeded `R` (serial arithmetic
within the receive window) is a duplicate: acknowledged, not delivered. This is what §6/§8 already promised.
- **Stale Connect (RFC 5961's challenge ACK).** A Connect whose sequence precedes the current generation's Connect
within the receive window is stale: it is answered with an Ack naming the current generation and does not reset.
A client whose Connect gets no Accept before its connect attempt times out starts a fresh attempt with a newly
drawn initial sequence, so a restarted client whose random sequence lands in that window still connects.
- **Server mode.** A Connect from a different endpoint is never a repeat; the session predicate stays sequence-only.
(Hands' deviation 5 was rejected: pinned clients restarting on a new port were locked out.)
- **The random default belongs to the session.** A session draws its own initial sequence when none is set, so one
options object reused across sessions or reconnects never repeats a sequence.
- **Connect attempts (Self, 2026-09-30, after the Cut 1c fix-batch Soul pass).** While a Connect awaits its Accept, only an
Accept the client honours retires it; an Ack never does (an Ack naming the pending Connect had wedged the client for
good). A fresh attempt after the connect-attempt timeout uses exactly the abandoned sequence + (receive window − 1),
not a random draw: that keeps a late copy of the abandoned Connect inside the new one's stale window while escaping the
server's stale window (unless the server sits exactly there, in which case the next attempt escapes). Only a client's
first Connect draws randomly.
- **Recorded divergence (not this campaign's to fix yet).** Only Rust has a receiver receive-ahead window (1,024). TS,
C# and Python acknowledge a far-ahead reliable frame by name and hold it behind an ordered gap indefinitely, so a
stranded ordered channel is silent loss there and "still owed" in Rust. Map a receive window for the three runtimes.
- **Residuals after the final Cut 1c Soul pass (recorded 2026-09-30, low, not fixed):**
- A late copy of a Connect from two or more attempts back (reordered > ~3 s) still restarts the server; the +window−1
rule keeps only the immediately previous attempt stale. Rust then times out and reconnects; TS/C#/Python hit the
receive-window divergence above (silent loss behind the gap).
- A client pinning `initial_sequence` that restarts while its old data is still in flight after the restarted Connect
can be matched to the old session's resent Accept (byte-identical; the client cannot tell). Pinning forfeits restart
detection; the random default closes it. Document on the option.
- Repeated failed attempts walk the sequence up 4095 per 3 s and clamp at `u32::MAX−1` after ~18-27 days on one session;
make exhaustion a hard error the caller sees (the clamp removal mutant survives).
- **Cuts 3 and D Soul pass (Self, 2026-09-30).** Cut 3: the 4 MiB bound counts every byte sent above the lowest
unacknowledged sequence, acked or not (the spec's wording; the first implementation counted only pending packets).
Cut D: a send failure is either permanent (the datagram can never be sent as built, e.g. `EMSGSIZE`) or transient;
a permanent failure is returned from a caller-directed send and, inside a poll, ends only that peer's session with a
typed reason; transient failures stay counted losses. Windows `ConnectionResetError` on receive is idle, never fatal,
in every server. Cut D's negative grep `send_packet(...)?;` is obsolete: the `?` now propagates encode errors only.
- **Recorded (liveness, not fixed):** a lowest reliable packet lost forever stalls the sender 1,023 sequences ahead
while the peer stays alive; session timeout doesn't end it. TCP ends such a connection after a retransmission
limit (R2). Decide with Cut F (abandonment) whether a reliable packet outstanding past a bound ends the session.

28
docs/document-variants-cut.md
View file

@ -369,6 +369,34 @@ stripped (probe). Other findings:
refuses v3 also contains C2a, so no released reader strips ids. C0 and C1 are never released on their own.
Raw and cross-runtime writers that cannot see ids stay sticky-only. Fix batch 4 is in Hands.
**C2a F2 ruled (operator, 2026-09-30, accepting Self's recommendation B): records nobody claims are carried
untouched.** A store may hold records under a schema id that no local type owns or lists as compatible, for
example after a rename that declared nothing.
- Both store kinds carry those records byte for byte under their own id. The catalog lists them as foreign
records.
- Writes to other records proceed.
- A write that would overwrite or relabel an unclaimed record is refused with a typed schema conflict.
- The single-file store's whole-view write stops relabelling such records onto local types. The directory
store stops refusing every commit while it holds one.
- Why: batches 8-9 showed that the danger is rewriting a record under a new identity (F1, F6), not an
undeclared old id. Batch 10's owner-beats-lister rule makes declaring an old id cheap anyway.
- Self's first lean, "refuse loudly" (A), is superseded.
**C2a batches 9-10 (Soul pass on `fdb8a60a`, 2026-09-30): hold.**
- **F6, A-B-A lost update on the single-file store.** Ruled by Self: any write that changes a record's bytes or
schema id mints a later `storedAt`.
- **F8, batch 9 over-refused.** An owner beats a lister, so v1 and v2 coexist.
- **F9.** Compatible ids compare as a set.
- **F10.** Typed refusal errors in TS and Python.
- Batch 10 is in Hands.
- **F7 ruled (operator, 2026-09-30): retire `AsSchemaAlias`.** "Schema aliasing sounds like a really bad idea, I
support retiring it."
- C# refuses a second type for one schema, as TS, Python and Rust already do. There is one type per schema in
every runtime.
- The API, its registry path (`IsExactWireAlias`) and the tests that exercised it (7 Mesh, 2 Caching) are
deleted or rewritten.
- No consumer called it.
**Operator rulings, 2026-09-30:**
- **Duplicate unique-index values do not load** ("Duplicate indices should not load"). This covers
plain records too, so R6-for-indexes becomes one general rule:

57
packages/cultcache-py/src/cultcache_py/stores.py
View file

@ -5,7 +5,7 @@ import json
import os
import threading
from pathlib import Path
from typing import Any
from typing import Any, Callable, TypeVar
from .backing_store import (
CultCacheEnvelope,
@ -21,6 +21,7 @@ STORE_FORMAT_ELEMENT_IDS = "cultcache.store.v3"
_READABLE_STORE_FORMATS = (STORE_FORMAT_VERSION, STORE_FORMAT_ELEMENT_IDS)
_STORE_FORMAT_PREFIX = "cultcache.store."
_PERSISTED_RECORD_SLOTS = 4
_Contents = TypeVar("_Contents")
class StoreUnreadableError(ValueError):
@ -56,10 +57,11 @@ class JsonLinesBackingStore:
def pull_all(self) -> list[CultCacheEnvelope]:
with self._lock:
if not self.path.exists():
text = _read_store(self.path, lambda path: path.read_text(encoding="utf-8"))
if text is None:
return []
envelopes: list[CultCacheEnvelope] = []
for line_number, line in enumerate(self.path.read_text(encoding="utf-8").splitlines(), start=1):
for line_number, line in enumerate(text.splitlines(), start=1):
if not line.strip():
continue
raw = json.loads(line)
@ -121,12 +123,11 @@ class SingleFileMessagePackBackingStore:
def pull_all(self) -> list[CultCacheEnvelope]:
msgpack = self._msgpack()
with self._lock:
# The disk decides the header: a file that is gone or empty is not marked.
# An empty file is not a store (no writer leaves one), so it is decoded and refused.
# The disk decides the header: a file that is gone is not marked.
self._format = STORE_FORMAT_VERSION
if not self.path.exists():
return []
data = self.path.read_bytes()
if not data:
data = _read_store(self.path, Path.read_bytes)
if data is None:
return []
try:
decoded = msgpack.unpackb(data, raw=False)
@ -261,13 +262,44 @@ def _encode_snapshot(
return [format_version, catalog, records]
def _read_store(path: Path, read: Callable[[Path], _Contents]) -> _Contents | None:
"""Reads a store file, or returns None when nothing is at its path.
Only nothing is absent. A dangling symbolic link is something: reading it as
empty would let the writer replace the link and move the store off its volume.
Any other failure to reach the file stays an OSError.
"""
try:
return read(path)
except FileNotFoundError:
if path.is_symlink():
raise
return None
def _describe_identity(value: Any) -> str:
"""A record key or schema id as a refusal may show it: a string is an identity and is
named; anything else is described by its type, since the value is the store's."""
if isinstance(value, str):
return repr(value)
return f"<{type(value).__name__}>"
def _describe_header(header: str) -> str:
"""Echoes a header only in the shape cultcache.store.v<digits>; the bytes are the store's."""
version = header[len(_STORE_FORMAT_PREFIX) + 1:] if header.startswith(_STORE_FORMAT_PREFIX + "v") else ""
if version.isascii() and version.isdigit():
return repr(header)
return f"an unrecognised {_STORE_FORMAT_PREFIX}* header of {len(header.encode('utf-8'))} bytes"
def _decode_snapshot(decoded: Any) -> tuple[str, list[CultCacheEnvelope]] | None:
if not isinstance(decoded, list) or not decoded:
return None
if isinstance(decoded[0], str) and decoded[0].startswith(_STORE_FORMAT_PREFIX) and decoded[0] not in _READABLE_STORE_FORMATS:
raise ValueError(
f"CultCache store format {decoded[0]!r} is not readable; this runtime reads {STORE_FORMAT_VERSION!r} and {STORE_FORMAT_ELEMENT_IDS!r} only. "
"The store needs a runtime that resolves document variants."
f"CultCache store format {_describe_header(decoded[0])} is not one this runtime reads; "
f"it reads {STORE_FORMAT_VERSION!r} and {STORE_FORMAT_ELEMENT_IDS!r} only"
)
if decoded[0] not in _READABLE_STORE_FORMATS:
return None
@ -291,8 +323,9 @@ def _decode_snapshot(decoded: Any) -> tuple[str, list[CultCacheEnvelope]] | None
raise ValueError("CultCache persisted records must be MessagePack arrays")
if len(raw_record) > _PERSISTED_RECORD_SLOTS:
raise ValueError(
f"CultCache record {raw_record[0]!r} (schema {raw_record[1]!r}) has {len(raw_record)} slots; "
f"this runtime reads {_PERSISTED_RECORD_SLOTS}. The store needs a runtime that resolves document variants."
f"CultCache record {_describe_identity(raw_record[0])} (schema {_describe_identity(raw_record[1])}) "
f"has {len(raw_record)} slots, more than "
f"the {_PERSISTED_RECORD_SLOTS} of a {STORE_FORMAT_VERSION} record, so this is not a valid store"
)
key, schema_id, stored_at, payload = raw_record
if not isinstance(key, str) or not key:

124
packages/cultcache-py/tests/test_cultcache.py
View file

@ -1,5 +1,6 @@
from __future__ import annotations
import os
import tempfile
import threading
import unittest
@ -1321,6 +1322,129 @@ class CultCacheTests(unittest.TestCase):
self.assertEqual(envelopes[0].payload, b"\x92\xa5alpha\x01")
self.assertEqual(envelopes[1].payload, b"\x92\xa4beta\x02")
def test_single_file_refusals_say_what_was_found_without_blaming_variants(self) -> None:
with self.assertRaises(ValueError) as unsupported:
self._pull_vector("unknown-header.msgpack")
self.assertIn("not one this runtime reads", str(unsupported.exception))
self.assertNotIn("variant", str(unsupported.exception))
with self.assertRaises(ValueError) as invalid:
self._pull_vector("extra-slot-full-payload.msgpack")
self.assertIn("not a valid store", str(invalid.exception))
self.assertNotIn("variant", str(invalid.exception))
def test_single_file_only_a_missing_store_reads_as_empty(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
self.assertEqual(SingleFileMessagePackBackingStore(Path(tmp) / "store.cc").pull_all(), [])
# An existing empty file is not a store: no CultCache writer leaves one.
empty = Path(tmp) / "empty.cc"
empty.write_bytes(b"")
with self.assertRaises(ValueError):
SingleFileMessagePackBackingStore(empty).pull_all()
self.assertEqual(empty.read_bytes(), b"")
# An empty array is the legacy envelope array with no envelopes: no header, no format claimed.
legacy_empty = Path(tmp) / "legacy-empty.cc"
legacy_empty.write_bytes(b"\x90")
self.assertEqual(SingleFileMessagePackBackingStore(legacy_empty).pull_all(), [])
def test_single_file_refusals_never_echo_a_value_from_the_store(self) -> None:
import msgpack # type: ignore
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.cc"
cases = [
(["cultcache.store.v12", [], []], True),
(["cultcache.store.SECRET-HEADER", [], []], False),
(["cultcache.store.v12SECRET", [], []], False),
(["cultcache.store.v", [], []], False),
(["cultcache.store.v1", ["SECRET-CATALOG-TEXT"], []], None),
]
for value, echoed in cases:
path.write_bytes(msgpack.packb(value, use_bin_type=True))
with self.assertRaises(ValueError) as caught:
SingleFileMessagePackBackingStore(path).pull_all()
message = str(caught.exception)
if echoed:
self.assertIn(repr(value[0]), message)
else:
self.assertNotIn("SECRET", message)
if echoed is False:
self.assertNotIn(repr(value[0]), message)
self.assertIn(f"of {len(value[0])} bytes", message)
def test_single_file_record_refusal_names_only_a_string_key(self) -> None:
import msgpack # type: ignore
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.cc"
for key, schema in ((["SECRET-IN-LIST"], "s"), (b"SECRET-BYTES", "s"), ("k", {"SECRET": 1})):
path.write_bytes(msgpack.packb(
["cultcache.store.v1", [], [[key, schema, "t", b"\x90", "extra"]]], use_bin_type=True))
with self.assertRaises(ValueError) as caught:
SingleFileMessagePackBackingStore(path).pull_all()
self.assertNotIn("SECRET", str(caught.exception))
self.assertIn("not a valid store", str(caught.exception))
# What a write does through a symbolic link today. R3 decides whether a write resolves
# the link or refuses it; this pins the current behaviour so that change is deliberate.
@unittest.skipIf(os.name != "posix", "symbolic links need POSIX here")
def test_r3_decides_resolve_or_refuse_a_push_through_a_live_link_replaces_the_link(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
volume = Path(tmp) / "volume"
volume.mkdir()
target = volume / "store.cc"
envelope = CultCacheEnvelope(key="a", type="t", payload=b"\x90", stored_at="2026-09-30T00:00:00Z")
SingleFileMessagePackBackingStore(target).push(envelope)
before = target.read_bytes()
link = Path(tmp) / "store.cc"
link.symlink_to(target)
SingleFileMessagePackBackingStore(link).push(replace(envelope, key="b"))
self.assertFalse(link.is_symlink())
self.assertTrue(link.is_file())
self.assertEqual(target.read_bytes(), before)
@unittest.skipIf(os.name != "posix", "symbolic links need POSIX here")
def test_a_dangling_symlink_at_a_store_path_is_an_os_error_and_nothing_is_written(self) -> None:
for store_type in (SingleFileMessagePackBackingStore, JsonLinesBackingStore):
with tempfile.TemporaryDirectory() as tmp:
volume = Path(tmp) / "volume"
volume.mkdir()
path = Path(tmp) / "store.cc"
path.symlink_to(volume / "store.cc")
store = store_type(path)
with self.assertRaises(FileNotFoundError):
store.pull_all()
with self.assertRaises(FileNotFoundError):
store.push(CultCacheEnvelope(key="k", type="t", payload=b"\x90", stored_at="2026-09-30T00:00:00Z"))
self.assertTrue(path.is_symlink())
self.assertEqual(list(volume.iterdir()), [])
@unittest.skipIf(os.name != "posix", "symlink loops and file-as-parent errors are POSIX errno cases")
def test_json_lines_store_that_cannot_be_reached_is_an_os_error(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
self.assertEqual(JsonLinesBackingStore(Path(tmp) / "missing.jsonl").pull_all(), [])
loop = Path(tmp) / "loop.jsonl"
loop.symlink_to(loop)
parent = Path(tmp) / "body"
parent.write_bytes(b"not a directory")
for path in (loop, parent / "store.jsonl"):
with self.assertRaises(OSError) as caught:
JsonLinesBackingStore(path).pull_all()
self.assertNotIsInstance(caught.exception, FileNotFoundError)
@unittest.skipIf(os.name != "posix", "symlink loops and file-as-parent errors are POSIX errno cases")
def test_single_file_store_that_cannot_be_reached_is_an_os_error(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
loop = Path(tmp) / "loop.cc"
loop.symlink_to(loop)
parent = Path(tmp) / "body"
parent.write_bytes(b"not a directory")
for path in (loop, parent / "store.cc"):
with self.assertRaises(OSError) as caught:
SingleFileMessagePackBackingStore(path).pull_all()
self.assertNotIsInstance(caught.exception, FileNotFoundError)
# The element-id marker: tests/vectors/document-variants-c2a/v3-base.msgpack is v1-base with its header replaced.
_C2A_VECTORS = Path(__file__).resolve().parents[3] / "tests" / "vectors" / "document-variants-c2a"

55
packages/cultcache-rs/README.md
View file

@ -293,6 +293,61 @@ writes. That protects ordinary multi-process access to the same file. It is
still not a multi-master replication protocol, and it is not a substitute for a
coordinator when higher-level write ordering matters.
A write that fails part way, for example on a full disk, removes its staging
file and leaves the store as it was. A read that fails tells you why through
the error chain:
```rust
use cultcache_rs::{CacheBackingStore, CultCacheStoreUnreadable, CultCacheStoreUnreadableKind, SingleFileMessagePackBackingStore};
let store = SingleFileMessagePackBackingStore::new("cache.cc");
if let Err(error) = store.pull_all() {
match error.downcast_ref::<CultCacheStoreUnreadable>().map(|unreadable| unreadable.kind) {
// The bytes were read and are not a store: truncated, garbled, or invalid.
Some(CultCacheStoreUnreadableKind::Undecodable) => {}
// The header names a format this runtime does not read, such as a newer one.
Some(CultCacheStoreUnreadableKind::UnsupportedFormat) => {}
// The bytes could not be read at all: a std::io::Error is in the chain.
_ => {}
}
}
```
An I/O failure never carries `CultCacheStoreUnreadable`, and the chain under
`CultCacheStoreUnreadable` never holds a `std::io::Error`.
Only a store file that does not exist (`std::io::ErrorKind::NotFound`) reads as
an empty store. Any other failure to reach it, such as a symlink loop, a file
where a parent directory should be, or a missing permission, is an I/O error.
A store file that exists and holds zero bytes is `Undecodable`: no CultCache
writer leaves one. A symbolic link whose target is gone is not a missing store:
reading it is an I/O error, so a read-then-write (`push`, `delete`, the exchanges)
fails before writing. Writes do not yet follow links: a write through a live link,
and a `push_all` through a dangling one, replace the link with a regular file and
leave its target alone. A refusal names no
value read from the file; it names a record only by its key and schema id, and
a header only when it has the shape `cultcache.store.v<digits>`.
A write that fails says what it left on disk:
```rust
use cultcache_rs::{CultCacheStoreWriteFailed, CultCacheStoreWriteFailedKind};
fn outcome(error: &anyhow::Error) {
match error.downcast_ref::<CultCacheStoreWriteFailed>().map(|failed| failed.kind) {
// The entries cannot be stored; retrying them fails the same way.
Some(CultCacheStoreWriteFailedKind::Rejected) => {}
// Staging, fsync or rename failed: the store file holds what it held before,
// and a retry may succeed once the fault (a full disk, say) clears.
Some(CultCacheStoreWriteFailedKind::NotReplaced) => {}
// The rename landed and the directory sync failed: the new snapshot is in
// place but may not survive a crash.
Some(CultCacheStoreWriteFailedKind::ReplacedNotDurable) => {}
_ => {}
}
}
```
## Near-Term Ergonomic Improvements
1. **Derive macro**

1101
packages/cultcache-rs/src/lib.rs
View file

File diff suppressed because it is too large Load diff

59
packages/cultcache-ts/src/single-file-messagepack-backing-store.ts
View file

@ -1,4 +1,4 @@
import { mkdir, readFile, rename, rm, writeFile } from "node:fs/promises";
import { lstat, mkdir, readFile, rename, rm, writeFile } from "node:fs/promises";
import { dirname, resolve } from "node:path";
import { decode, encode } from "@msgpack/msgpack";
@ -44,7 +44,7 @@ export class SingleFileMessagePackBackingStore implements CacheBackingStore {
}
async pullAll(): Promise<CultCacheEnvelope[]> {
// The disk decides the header: a file that is gone, empty or legacy is not marked.
// The disk decides the header: a file that is gone or legacy is not marked.
this.#format = STORE_FORMAT_VERSION;
const disk = await this.#readDisk();
this.#format = disk.format;
@ -77,16 +77,8 @@ export class SingleFileMessagePackBackingStore implements CacheBackingStore {
async pushAll(entries: CultCacheEnvelope[], options: PushAllOptions = {}): Promise<void> {
await this.#enqueue(async () => {
if (options.soft) {
try {
await readFile(this.filePath);
return;
} catch (error) {
const code = (error as NodeJS.ErrnoException).code;
if (code !== "ENOENT") {
throw error;
}
}
if (options.soft && (await readStore(this.filePath)) !== undefined) {
return;
}
// A flush of the whole store writes the header the file on disk carries, read now: a file marked for element ids
@ -101,18 +93,10 @@ export class SingleFileMessagePackBackingStore implements CacheBackingStore {
// The one reader of the store file, asked by open, push, delete and flush alike. A file that is gone or zero bytes is an
// empty unmarked store; anything else must decode completely or the read throws StoreUnreadableError.
async #readDisk(): Promise<DiskStore> {
let data: Uint8Array;
try {
data = await readFile(this.filePath);
} catch (error) {
if ((error as NodeJS.ErrnoException).code === "ENOENT") {
return { format: STORE_FORMAT_VERSION, envelopes: [], repairedLegacyPayload: false };
}
throw error;
}
if (data.length === 0) {
// An empty file is not a store, since no CultCache writer leaves one, so it is decoded and refused.
// Only a read that finds nothing at the path is an empty store.
const data = await readStore(this.filePath);
if (data === undefined) {
return { format: STORE_FORMAT_VERSION, envelopes: [], repairedLegacyPayload: false };
}
@ -156,6 +140,33 @@ export class SingleFileMessagePackBackingStore implements CacheBackingStore {
}
}
/**
* Reads a store file, or returns undefined when nothing is at its path. A dangling symbolic
* link is something: reading it as empty would let the writer rename a file over the link
* and move the store off its volume. Any other failure to reach the file is thrown.
*/
async function readStore(path: string): Promise<Uint8Array | undefined> {
try {
return await readFile(path);
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== "ENOENT") {
throw error;
}
}
try {
await lstat(path);
} catch (error) {
if ((error as NodeJS.ErrnoException).code === "ENOENT") {
return undefined;
}
throw error;
}
throw Object.assign(new Error(`CultCache store ${path} is a symbolic link whose target does not exist.`), { code: "ENOENT" });
}
async function renameWithRetry(source: string, destination: string): Promise<void> {
let delayMs = 10;
let lastError: unknown;

36
packages/cultcache-ts/src/store-format.ts
View file

@ -24,8 +24,7 @@ export function isStoreSnapshot(decoded: unknown): decoded is [StoreFormat, ...u
if (decoded[0] !== STORE_FORMAT_VERSION && decoded[0] !== STORE_FORMAT_ELEMENT_IDS) {
throw new Error(
`CultCache store format "${decoded[0]}" is not readable; this runtime reads "${STORE_FORMAT_VERSION}" and "${STORE_FORMAT_ELEMENT_IDS}" only. ` +
"The store needs a runtime that resolves document variants.",
`CultCache store format ${describeHeader(decoded[0])} is not one this runtime reads; it reads "${STORE_FORMAT_VERSION}" and "${STORE_FORMAT_ELEMENT_IDS}" only.`,
);
}
@ -67,12 +66,41 @@ export class StoreUnreadableError extends Error {
}
}
/**
* A store header as a refusal may show it: echoed in the shape `cultcache.store.v<digits>`,
* and otherwise described only by its length, since the bytes are the store's.
*/
function describeHeader(header: string): string {
return /^cultcache\.store\.v[0-9]+$/u.test(header)
? `"${header}"`
: `an unrecognised ${STORE_FORMAT_PREFIX}* header of ${new TextEncoder().encode(header).length} bytes`;
}
/**
* A record key or schema id as a refusal may show it: a string is an identity and is named;
* anything else is described by its type, since the value is the store's.
*/
function describeIdentity(value: unknown): string {
if (typeof value === "string") {
return `"${value}"`;
}
const kind = Array.isArray(value)
? "array"
: value instanceof Uint8Array
? "bytes"
: value === null
? "nil"
: typeof value;
return `<${kind}>`;
}
/** Refuses a persisted record with more slots than v1 defines, naming its key and schema id. */
export function requireV1RecordSlots(record: unknown[]): void {
if (record.length > PERSISTED_RECORD_SLOTS) {
throw new Error(
`CultCache record "${String(record[0])}" (schema "${String(record[1])}") has ${record.length} slots; ` +
`this runtime reads ${PERSISTED_RECORD_SLOTS}. The store needs a runtime that resolves document variants.`,
`CultCache record ${describeIdentity(record[0])} (schema ${describeIdentity(record[1])}) has ${record.length} slots, more than ` +
`the ${PERSISTED_RECORD_SLOTS} of a ${STORE_FORMAT_VERSION} record, so this is not a valid store.`,
);
}
}

125
packages/cultcache-ts/test/cult-cache.test.ts
View file

@ -2,7 +2,7 @@ import assert from "node:assert/strict";
import { exec, execFile } from "node:child_process";
import { EventEmitter } from "node:events";
import { existsSync } from "node:fs";
import { access, mkdtemp, readFile, readdir, rm, writeFile } from "node:fs/promises";
import { access, lstat, mkdir, mkdtemp, readdir, readFile, rm, symlink, writeFile } from "node:fs/promises";
import { homedir, tmpdir } from "node:os";
import { dirname, join, resolve, delimiter } from "node:path";
import { test } from "node:test";
@ -1988,6 +1988,129 @@ test("SingleFileMessagePackBackingStore refuses the C#-written variant store by
);
});
test("SingleFileMessagePackBackingStore refusals say what was found without blaming variants", async () => {
await assert.rejects(
() => pullVector("unknown-header.msgpack"),
(error: Error) => error.message.includes("not one this runtime reads") && !error.message.includes("variant"),
);
await assert.rejects(
() => pullVector("extra-slot-full-payload.msgpack"),
(error: Error) => error.message.includes("not a valid store") && !error.message.includes("variant"),
);
});
test("SingleFileMessagePackBackingStore reads only a missing store as empty", async () => {
const dir = await mkdtemp(join(tmpdir(), "cultcache-absent-"));
assert.deepEqual(await new SingleFileMessagePackBackingStore(join(dir, "store.cc")).pullAll(), []);
// An existing empty file is not a store: no CultCache writer leaves one.
const empty = join(dir, "empty.cc");
await writeFile(empty, new Uint8Array());
await assert.rejects(() => new SingleFileMessagePackBackingStore(empty).pullAll());
assert.equal((await readFile(empty)).length, 0);
// An empty array is the legacy envelope array with no envelopes: no header, no format claimed.
const legacyEmpty = join(dir, "legacy-empty.cc");
await writeFile(legacyEmpty, Uint8Array.of(0x90));
assert.deepEqual(await new SingleFileMessagePackBackingStore(legacyEmpty).pullAll(), []);
});
test("SingleFileMessagePackBackingStore reports a store it cannot reach as an I/O error", { skip: process.platform === "win32" }, async () => {
const dir = await mkdtemp(join(tmpdir(), "cultcache-unreachable-"));
const loop = join(dir, "loop.cc");
await symlink(loop, loop);
const parent = join(dir, "body");
await writeFile(parent, "not a directory");
for (const [path, code] of [[loop, "ELOOP"], [join(parent, "store.cc"), "ENOTDIR"]] as const) {
await assert.rejects(
() => new SingleFileMessagePackBackingStore(path).pullAll(),
(error: NodeJS.ErrnoException) => error.code === code,
);
}
});
test("SingleFileMessagePackBackingStore refusals never echo a value from the store", async () => {
const dir = await mkdtemp(join(tmpdir(), "cultcache-canary-"));
const file = join(dir, "store.cc");
const cases: Array<[unknown[], boolean | undefined]> = [
[["cultcache.store.v12", [], []], true],
[["cultcache.store.SECRET-HEADER", [], []], false],
[["cultcache.store.v12SECRET", [], []], false],
[["cultcache.store.v", [], []], false],
[["cultcache.store.v1", ["SECRET-CATALOG-TEXT"], []], undefined],
];
for (const [value, echoed] of cases) {
await writeFile(file, encode(value));
const header = value[0] as string;
await assert.rejects(
() => new SingleFileMessagePackBackingStore(file).pullAll(),
(error: Error) => {
if (echoed) {
return error.message.includes(`"${header}"`);
}
return !error.message.includes("SECRET")
&& (echoed === undefined || (!error.message.includes(`"${header}"`) && error.message.includes(`of ${header.length} bytes`)));
},
);
}
});
test("SingleFileMessagePackBackingStore refuses a dangling symlink at its path and writes nothing", { skip: process.platform === "win32" }, async () => {
const dir = await mkdtemp(join(tmpdir(), "cultcache-dangling-"));
const volume = join(dir, "volume");
await mkdir(volume);
const file = join(dir, "store.cc");
await symlink(join(volume, "store.cc"), file);
const store = new SingleFileMessagePackBackingStore(file);
const envelope = { key: "k", type: "t", payload: Uint8Array.of(0x90), storedAt: "2026-09-30T00:00:00Z" };
await assert.rejects(() => store.pullAll(), (error: NodeJS.ErrnoException) => error.code === "ENOENT");
await assert.rejects(() => store.push(envelope), (error: NodeJS.ErrnoException) => error.code === "ENOENT");
await assert.rejects(() => store.pushAll([envelope], { soft: true }), (error: NodeJS.ErrnoException) => error.code === "ENOENT");
assert.ok((await lstat(file)).isSymbolicLink());
assert.deepEqual(await readdir(volume), []);
});
test("SingleFileMessagePackBackingStore record refusal names only a string key", async () => {
const dir = await mkdtemp(join(tmpdir(), "cultcache-record-key-"));
const file = join(dir, "store.cc");
const secret = new TextEncoder().encode("SECRET-BYTES");
for (const [key, schema] of [[["SECRET-IN-ARRAY"], "s"], [secret, "s"], ["k", { SECRET: 1 }]] as const) {
await writeFile(file, encode(["cultcache.store.v1", [], [[key, schema, "t", Uint8Array.of(0x90), "extra"]]]));
await assert.rejects(
() => new SingleFileMessagePackBackingStore(file).pullAll(),
(error: Error) => !error.message.includes("SECRET") && !error.message.includes("83,69,67") && error.message.includes("not a valid store"),
);
}
});
// What a write does through a symbolic link today. R3 decides whether a write resolves the link
// or refuses it; these pin the current behaviour so that change is deliberate.
test("R3 decides resolve-or-refuse: a push through a live link replaces the link", { skip: process.platform === "win32" }, async () => {
const dir = await mkdtemp(join(tmpdir(), "cultcache-live-link-"));
const volume = join(dir, "volume");
await mkdir(volume);
const target = join(volume, "store.cc");
const envelope = { key: "a", type: "t", payload: Uint8Array.of(0x90), storedAt: "2026-09-30T00:00:00Z" };
await new SingleFileMessagePackBackingStore(target).push(envelope);
const before = await readFile(target);
const link = join(dir, "store.cc");
await symlink(target, link);
await new SingleFileMessagePackBackingStore(link).push({ ...envelope, key: "b" });
assert.ok((await lstat(link)).isFile());
assert.deepEqual(await readFile(target), before);
});
test("R3 decides resolve-or-refuse: a pushAll through a dangling link replaces the link", { skip: process.platform === "win32" }, async () => {
const dir = await mkdtemp(join(tmpdir(), "cultcache-dangling-push-all-"));
const volume = join(dir, "volume");
await mkdir(volume);
const link = join(dir, "store.cc");
await symlink(join(volume, "store.cc"), link);
await new SingleFileMessagePackBackingStore(link).pushAll([{ key: "a", type: "t", payload: Uint8Array.of(0x90), storedAt: "2026-09-30T00:00:00Z" }]);
assert.ok((await lstat(link)).isFile());
assert.deepEqual(await readdir(volume), []);
});
test("CultCache inspector refuses the same vectors by name", async () => {
const inspect = async (name: string) => inspectCultCacheBytes(name, await readFile(join(variantVectors, name)));
await assert.rejects(() => inspect("unknown-header.msgpack"), /cultcache\.store\.v9/u);

61
packages/cultmesh-kotlin/src/main/kotlin/org/gamecult/cultmesh/CultMesh.kt
View file

@ -11,6 +11,7 @@ import java.io.InputStream
import java.io.OutputStream
import java.net.DatagramPacket
import java.net.DatagramSocket
import java.net.Inet6Address
import java.net.InetAddress
import java.net.InetSocketAddress
import java.net.ServerSocket
@ -604,7 +605,7 @@ object CultMesh {
connectionId: Long,
remoteHost: String,
remotePort: Int,
bindHost: String = "127.0.0.1",
bindHost: String? = null,
bindPort: Int = 0,
tuning: CultNetRudpSocketTuning = CultNetRudpSocketTuning(),
): CultNetRudpSocketTransportConnection = cultNetRudpClient(
@ -623,7 +624,7 @@ object CultMesh {
runtimeId: String,
connectionId: Long,
endpoint: CultNetRudpEndpoint,
bindHost: String = "127.0.0.1",
bindHost: String? = null,
bindPort: Int = 0,
tuning: CultNetRudpSocketTuning = CultNetRudpSocketTuning(),
): CultNetRudpSocketTransportConnection = createRudpClient(
@ -640,7 +641,7 @@ object CultMesh {
runtimeId: String,
connectionId: Long,
endpoint: String,
bindHost: String = "127.0.0.1",
bindHost: String? = null,
bindPort: Int = 0,
tuning: CultNetRudpSocketTuning = CultNetRudpSocketTuning(),
): CultNetRudpSocketTransportConnection = createRudpClient(
@ -656,7 +657,7 @@ object CultMesh {
runtimeId: String,
connectionId: Long,
peer: CultMeshPeerCard,
bindHost: String = "127.0.0.1",
bindHost: String? = null,
bindPort: Int = 0,
tuning: CultNetRudpSocketTuning = CultNetRudpSocketTuning(),
): CultNetRudpSocketTransportConnection {
@ -674,7 +675,7 @@ object CultMesh {
role: String,
shardId: String? = null,
at: Instant = Instant.now(),
bindHost: String = "127.0.0.1",
bindHost: String? = null,
bindPort: Int = 0,
tuning: CultNetRudpSocketTuning = CultNetRudpSocketTuning(),
): CultNetRudpSocketTransportConnection {
@ -687,7 +688,7 @@ object CultMesh {
runtimeId: String,
connectionId: Long,
endpoint: CultNetRudpEndpoint,
bindHost: String = "127.0.0.1",
bindHost: String? = null,
bindPort: Int = 0,
tuning: CultNetRudpSocketTuning = CultNetRudpSocketTuning(),
connectPayload: ByteArray = ByteArray(0),
@ -706,7 +707,7 @@ object CultMesh {
runtimeId: String,
connectionId: Long,
endpoint: String,
bindHost: String = "127.0.0.1",
bindHost: String? = null,
bindPort: Int = 0,
tuning: CultNetRudpSocketTuning = CultNetRudpSocketTuning(),
connectPayload: ByteArray = ByteArray(0),
@ -728,7 +729,7 @@ object CultMesh {
runtimeId: String,
connectionId: Long,
peer: CultMeshPeerCard,
bindHost: String = "127.0.0.1",
bindHost: String? = null,
bindPort: Int = 0,
tuning: CultNetRudpSocketTuning = CultNetRudpSocketTuning(),
connectPayload: ByteArray = ByteArray(0),
@ -759,7 +760,7 @@ object CultMesh {
role: String,
shardId: String? = null,
at: Instant = Instant.now(),
bindHost: String = "127.0.0.1",
bindHost: String? = null,
bindPort: Int = 0,
tuning: CultNetRudpSocketTuning = CultNetRudpSocketTuning(),
connectPayload: ByteArray = ByteArray(0),
@ -3051,22 +3052,36 @@ fun cultNetRudpServer(
)
}
/**
* The address an RUDP client binds when its caller names none: loopback for a
* loopback endpoint, otherwise the unspecified address of the endpoint's family.
* A socket bound to loopback cannot send off the host on Windows.
*/
fun cultNetRudpClientBindAddress(remote: InetAddress): InetAddress = when {
remote is Inet6Address && remote.isLoopbackAddress -> InetAddress.getByName("::1")
remote is Inet6Address -> InetAddress.getByName("::")
remote.isLoopbackAddress -> InetAddress.getByName("127.0.0.1")
else -> InetAddress.getByName("0.0.0.0")
}
fun cultNetRudpClient(
runtimeId: String,
connectionId: Long,
remoteHost: String,
remotePort: Int,
bindHost: String = "127.0.0.1",
bindHost: String? = null,
bindPort: Int = 0,
tuning: CultNetRudpSocketTuning = CultNetRudpSocketTuning(),
): CultNetRudpSocketTransportConnection {
val socket = DatagramSocket(bindPort, InetAddress.getByName(bindHost)).also { it.soTimeout = 20 }
val remote = InetAddress.getByName(remoteHost)
val bindAddress = bindHost?.let(InetAddress::getByName) ?: cultNetRudpClientBindAddress(remote)
val socket = DatagramSocket(bindPort, bindAddress).also { it.soTimeout = 20 }
return CultNetRudpSocketTransportConnection(
socket = socket,
mode = CultNetRudpSocketMode.Client,
runtimeId = runtimeId,
connectionId = connectionId,
remoteAddress = InetSocketAddress(InetAddress.getByName(remoteHost), remotePort),
remoteAddress = InetSocketAddress(remote, remotePort),
initialSequence = tuning.initialSequence,
resendDelayMs = tuning.resendDelayMs,
maxFragmentBytes = tuning.maxFragmentBytes,
@ -3391,6 +3406,7 @@ fun main(args: Array<String>) {
rudpSessionFragmentsAndReassemblesReliableOrderedPayloads()
rudpSessionAdvancesLargeFragmentSetsThroughBoundedReliableWindow()
rudpSocketTransportErgonomicFactoriesCarrySchemaFrames()
rudpClientBindsLoopbackOnlyForALoopbackEndpoint()
rudpSocketTransportHandshakesAndCarriesReliableOrderedSchemaFrames()
rudpSocketTransportCarriesFragmentedReliableOrderedSchemaFrames()
return
@ -5437,6 +5453,27 @@ private fun rudpSessionAdvancesLargeFragmentSetsThroughBoundedReliableWindow() {
check(oldAck.ackMask == 0L)
}
private fun rudpClientBoundHost(endpoint: String, bindHost: String? = null): InetAddress =
CultMesh.createRudpClient("kotlin-rudp-bind-client", 0x10203060L, endpoint, bindHost).use { client ->
InetAddress.getByName(client.profile.transports.single().host)
}
// A client bound to loopback cannot send to another host on Windows, so a
// client for a remote endpoint binds the unspecified address. A JVM wildcard
// socket is dual-stack and reports one wildcard for either family, so the
// family is not observable here.
private fun rudpClientBindsLoopbackOnlyForALoopbackEndpoint() {
check(rudpClientBoundHost("rudp://10.77.0.1:17872").isAnyLocalAddress)
check(rudpClientBoundHost("rudp://[2001:db8::1]:17872").isAnyLocalAddress)
check(rudpClientBoundHost("rudp://127.0.0.1:17872") == InetAddress.getByName("127.0.0.1"))
check(rudpClientBoundHost("rudp://[::1]:17872") == InetAddress.getByName("::1"))
check(rudpClientBoundHost("rudp://10.77.0.1:17872", "127.0.0.1") == InetAddress.getByName("127.0.0.1"))
check(rudpClientBoundHost("rudp://127.0.0.1:17872", "0.0.0.0").isAnyLocalAddress)
CultMesh.createRudpServer("kotlin-rudp-bind-server", 0x10203061L).use { server ->
check(InetAddress.getByName(server.profile.transports.single().host) == InetAddress.getByName("127.0.0.1"))
}
}
private fun rudpSocketTransportErgonomicFactoriesCarrySchemaFrames() {
val connectionId = 0x10203042L
CultMesh.createRudpServer(

21
packages/cultmesh-py/src/cultmesh_py/facade.py
View file

@ -18,6 +18,7 @@ from cultnet_py import (
CultNetShardCatalog,
CultNetSimulationObservationHub,
create_rudp_schema_transport,
rudp_client_bind_host,
wire_message_schema_catalog,
)
@ -157,7 +158,7 @@ class CultMesh:
bind_host: str = "127.0.0.1",
bind_port: int = 0,
socket: socket_module.socket | None = None,
initial_sequence: int = 1,
initial_sequence: int | None = None,
resend_delay_ms: int = 250,
transport_id: str = "rudp",
max_payload_bytes: int | None = None,
@ -188,10 +189,10 @@ class CultMesh:
connection_id: int,
endpoint: str | CultMeshRudpEndpoint,
*,
bind_host: str = "127.0.0.1",
bind_host: str | None = None,
bind_port: int = 0,
socket: socket_module.socket | None = None,
initial_sequence: int = 1,
initial_sequence: int | None = None,
resend_delay_ms: int = 250,
transport_id: str = "rudp",
max_payload_bytes: int | None = None,
@ -204,7 +205,10 @@ class CultMesh:
if isinstance(endpoint, str)
else endpoint
)
transport_socket = socket or _bind_rudp_socket(bind_host, bind_port)
transport_socket = socket or _bind_rudp_socket(
bind_host if bind_host is not None else rudp_client_bind_host(parsed_endpoint.host),
bind_port,
)
return CultNetRudpSocketTransportConnection(
CultNetRudpSocketTransportOptions(
runtime_id=runtime_id,
@ -429,7 +433,7 @@ class CultMesh:
endpoint: str | CultMeshRudpEndpoint | None = None,
connection_id: int = 0x43554C54,
runtime_id: str = "cultmesh-python-rudp-client",
bind_host: str = "127.0.0.1",
bind_host: str | None = None,
bind_port: int = 0,
) -> CultNetRawClient:
if endpoint is None and host.lower().startswith("rudp://"):
@ -465,7 +469,7 @@ class CultMesh:
endpoint: str | CultMeshRudpEndpoint | None = None,
connection_id: int = 0x43554C54,
runtime_id: str = "cultmesh-python-rudp-client",
bind_host: str = "127.0.0.1",
bind_host: str | None = None,
bind_port: int = 0,
) -> CultNetRawClient:
return CultMesh.create_client(
@ -481,7 +485,10 @@ class CultMesh:
def _bind_rudp_socket(bind_host: str, bind_port: int) -> socket_module.socket:
transport_socket = socket_module.socket(socket_module.AF_INET, socket_module.SOCK_DGRAM)
transport_socket = socket_module.socket(
socket_module.AF_INET6 if ":" in bind_host else socket_module.AF_INET,
socket_module.SOCK_DGRAM,
)
transport_socket.bind((bind_host, bind_port))
transport_socket.settimeout(0.02)
return transport_socket

18
packages/cultmesh-py/src/cultmesh_py/server.py
View file

@ -224,15 +224,19 @@ class CultMeshLocalServer:
now_ms = _now_ms()
peer = peers.get(remote_addr)
if packet.packet_type == CultNetRudpPacketType.CONNECT:
peer = _RudpPeerConnection(
CultNetRudpSession(
CultNetRudpSessionOptions(
connection_id=self.rudp_connection_id,
resend_delay_ms=self.rudp_resend_delay_ms,
# The session decides whether this Connect repeats the one it
# accepted. Any other Connect is a new client session, with new
# subscriptions.
if peer is None or not peer.session.connect_repeats(packet):
peer = _RudpPeerConnection(
CultNetRudpSession(
CultNetRudpSessionOptions(
connection_id=self.rudp_connection_id,
resend_delay_ms=self.rudp_resend_delay_ms,
)
)
)
)
peers[remote_addr] = peer
peers[remote_addr] = peer
self._send_rudp_packet(
rudp_socket,
remote_addr,

31
packages/cultmesh-py/tests/test_cultmesh.py
View file

@ -213,6 +213,37 @@ class CultMeshTests(unittest.TestCase):
self.assertEqual(received_messages[0]["schemaVersion"], "cultnet.schema_catalog_request.v0")
self.assertEqual(received_messages[0]["messageId"], "rudp-transport-factory")
def _rudp_client_bound_host(self, endpoint: str, bind_host: str | None = None) -> str:
client = CultMesh.create_rudp_client("python-cultmesh-bind-client", 0x10203060, endpoint, bind_host=bind_host)
try:
return client.socket.getsockname()[0]
finally:
client.socket.close()
# A client bound to loopback cannot send to another host on Windows, so a
# client for a remote endpoint binds the unspecified address of its family.
def test_cultmesh_rudp_client_for_a_remote_endpoint_binds_the_unspecified_address(self) -> None:
self.assertEqual(self._rudp_client_bound_host("rudp://10.77.0.1:17872"), "0.0.0.0")
self.assertEqual(self._rudp_client_bound_host("rudp://[2001:db8::1]:17872"), "::")
def test_cultmesh_rudp_client_for_a_loopback_endpoint_binds_loopback(self) -> None:
self.assertEqual(self._rudp_client_bound_host("rudp://127.0.0.1:17872"), "127.0.0.1")
self.assertEqual(self._rudp_client_bound_host("rudp://[::1]:17872"), "::1")
def test_cultmesh_rudp_client_binds_an_explicit_bind_host_whatever_the_endpoint(self) -> None:
self.assertEqual(self._rudp_client_bound_host("rudp://10.77.0.1:17872", "127.0.0.1"), "127.0.0.1")
self.assertEqual(self._rudp_client_bound_host("rudp://127.0.0.1:17872", "0.0.0.0"), "0.0.0.0")
def test_cultmesh_rudp_server_binds_loopback_unless_told_otherwise(self) -> None:
loopback = CultMesh.create_rudp_server("python-cultmesh-bind-server", 0x10203061)
anywhere = CultMesh.create_rudp_server("python-cultmesh-bind-server", 0x10203062, bind_host="0.0.0.0")
try:
self.assertEqual(loopback.socket.getsockname()[0], "127.0.0.1")
self.assertEqual(anywhere.socket.getsockname()[0], "0.0.0.0")
finally:
loopback.socket.close()
anywhere.socket.close()
def test_cultmesh_facade_creates_rudp_client_from_peer_endpoint(self) -> None:
connection_id = 0x10203042
server = CultMesh.create_rudp_server(

159
packages/cultmesh-py/tests/test_rudp_server_generation.py Normal file
View file

@ -0,0 +1,159 @@
"""The CultMesh local server's RUDP loop asks each peer's session whether a Connect
repeats. A retransmitted Connect keeps the peer; a restarted client on the same
address is a new one, with none of the old one's subscriptions."""
from __future__ import annotations
import socket
import time
import unittest
from dataclasses import dataclass
import msgpack
from cultcache_py import define_database_entry_type
from cultmesh_py import create_node
from cultmesh_py.server import CultMeshLocalServer
from cultnet_py import (
CultNetRudpSendOptions,
CultNetRudpSession,
CultNetRudpSessionOptions,
document_put_raw,
)
from cultnet_py.transport import CultNetRudpPacketType, decode_rudp_packet, encode_rudp_packet
CONNECTION_ID = 0x10203091
SCHEMA_ID = "mesh.rudp_generation_note.v1"
RELIABLE = CultNetRudpSendOptions(reliable=True, ordered=True)
@dataclass
class Note:
body: str
NOTE = define_database_entry_type(
"mesh.rudp_generation_note",
[("body", 0)],
cls=Note,
schema_id=SCHEMA_ID,
schema_name="mesh.rudp_generation_note",
schema_version=SCHEMA_ID,
)
def wire(message: dict) -> bytes:
return msgpack.packb(message, use_bin_type=True)
class RudpServerGenerationTests(unittest.TestCase):
def setUp(self) -> None:
node = create_node(runtime_id="rudp-generation")
node.database.register_document(NOTE)
self.server = CultMeshLocalServer(node=node, rudp_connection_id=CONNECTION_ID)
self.server.start()
self.sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
self.sock.bind(("127.0.0.1", 0))
self.sock.settimeout(0.02)
self.target = ("127.0.0.1", self.server.port)
# Every schema data packet the server sent, by sequence, whether or not the receiving
# session delivered it: a new server-side session numbers its packets afresh.
self.sent: dict[int, str] = {}
def tearDown(self) -> None:
self.sock.close()
self.server.stop()
def to_server(self, packet) -> None:
self.sock.sendto(encode_rudp_packet(packet), self.target)
def pump(self, session: CultNetRudpSession, until, seconds: float = 2.0) -> bool:
deadline = time.monotonic() + seconds
while time.monotonic() < deadline:
if until():
return True
try:
data, _ = self.sock.recvfrom(65535)
except TimeoutError:
continue
packet = decode_rudp_packet(data)
session.receive(packet, int(time.monotonic() * 1000))
if packet.reliable or packet.packet_type == CultNetRudpPacketType.ACCEPT:
self.to_server(session.create_ack_for_received(packet.sequence))
if packet.packet_type == CultNetRudpPacketType.DATA and packet.channel_id == "schema":
self.sent[packet.sequence] = msgpack.unpackb(packet.payload, raw=False).get("schemaVersion")
return until()
def count(self, schema_version: str) -> int:
return sum(1 for value in self.sent.values() if value == schema_version)
def new_session(self, initial_sequence: int) -> CultNetRudpSession:
return CultNetRudpSession(CultNetRudpSessionOptions(connection_id=CONNECTION_ID, initial_sequence=initial_sequence))
def test_a_retransmitted_connect_keeps_the_peer_and_a_restarted_client_is_a_new_one(self) -> None:
first = self.new_session(50_000)
connect = first.create_connect(0, b"same")
self.to_server(connect)
self.assertTrue(self.pump(first, lambda: first.connected), "the first Accept")
hello = first.send(
"schema", wire({"schemaVersion": "cultnet.hello.v0", "runtimeId": "probe", "runtimeKind": "test"}), RELIABLE
)
self.to_server(hello)
self.assertTrue(self.pump(first, lambda: self.count("cultnet.hello.v0") == 1), "the first answer")
self.to_server(
first.send(
"schema",
wire({
"schemaVersion": "cultnet.database_subscribe.v0",
"messageId": "subscribe",
"subscriptionId": "old-incarnation",
"schemaIds": [SCHEMA_ID],
"includeSnapshot": False,
}),
RELIABLE,
)
)
self.pump(first, lambda: False, 0.2)
first_put = document_put_raw(
message_id="first-put",
key="note:a",
schema_id=SCHEMA_ID,
stored_at="2026-09-30T00:00:00Z",
payload=NOTE.encode_payload(Note("first")),
)
self.to_server(first.send("schema", first_put.to_bytes(), RELIABLE))
self.assertTrue(
self.pump(first, lambda: self.count("cultnet.database_change_raw.v0") == 1),
"the subscription did not notify its own connection",
)
# A retransmitted Connect repeats: the data already delivered is not delivered, or answered, again.
self.to_server(connect)
self.to_server(hello)
self.pump(first, lambda: False, 0.4)
self.assertEqual(self.count("cultnet.hello.v0"), 1, "the retransmitted Connect reset the peer")
# The same client restarted on the same address, connection id and payload: a new peer.
second = self.new_session(7)
self.to_server(second.create_connect(0, b"same"))
self.assertTrue(self.pump(second, lambda: second.connected), "the restarted client's Accept")
self.to_server(
second.send(
"schema", wire({"schemaVersion": "cultnet.hello.v0", "runtimeId": "probe", "runtimeKind": "test"}), RELIABLE
)
)
self.assertTrue(self.pump(second, lambda: self.count("cultnet.hello.v0") == 2), "the restarted client's answer")
# It inherited none of the old incarnation's subscriptions.
put = document_put_raw(
message_id="second-put",
key="note:a",
schema_id=SCHEMA_ID,
stored_at="2026-09-30T00:00:01Z",
payload=NOTE.encode_payload(Note("second")),
)
self.to_server(second.send("schema", put.to_bytes(), RELIABLE))
self.pump(second, lambda: False, 0.4)
self.assertEqual(self.count("cultnet.database_change_raw.v0"), 1, "the new peer inherited a subscription")
if __name__ == "__main__":
unittest.main()

4
packages/cultmesh-rs/src/lib.rs
View file

@ -961,7 +961,7 @@ fn connect_rudp_client(
mode: cultnet_rs::CultNetRudpSocketMode::Client,
remote_addr: Some(target),
connection_id: fresh_rudp_connection_epoch(),
initial_sequence: 1,
initial_sequence: None,
resend_delay_ms,
transport_id: Some(transport_id.to_string()),
max_payload_bytes: None,
@ -1406,7 +1406,7 @@ mod tests {
let mut session =
cultnet_rs::CultNetRudpSession::new(cultnet_rs::CultNetRudpSessionOptions {
connection_id: connect.connection_id,
initial_sequence: 1,
initial_sequence: Some(1),
resend_delay_ms: 5,
max_pending_reliable_packets: Some(64),
});

16
packages/cultmesh-rs/src/rudp_document_server.rs
View file

@ -410,12 +410,20 @@ where
packet: &CultNetRudpPacket,
now: u64,
) -> Result<bool> {
// The session decides whether a Connect repeats the one it accepted. Any
// other Connect from this key is a new client session: the old one ends
// with its budget, and the new one starts fresh.
if self
.sessions
.get(&key)
.is_some_and(|entry| !entry.session.connect_repeats(packet))
{
self.sessions.remove(&key);
}
let reply = match self.sessions.get_mut(&key) {
// The same key is a retransmitted Connect for the existing epoch.
// A genuinely fresh client incarnation must choose a fresh id.
Some(entry) => {
entry.last_activity_monotonic_millis = now;
match entry.session.answer_repeated_connect(packet, now) {
match entry.session.accept_connect(packet, now, Vec::new()) {
Ok(reply) => reply,
Err(_) => {
self.end_refused_session(key)?;
@ -429,7 +437,7 @@ where
}
let mut session = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: key.connection_id,
initial_sequence: 1,
initial_sequence: None,
resend_delay_ms: duration_millis(self.options.resend_delay),
max_pending_reliable_packets: Some(
self.options.max_pending_reliable_packets_per_session,

107
packages/cultmesh-rs/tests/rudp_document_server.rs
View file

@ -160,7 +160,15 @@ fn server(
}
fn client(target: SocketAddr, id: u32) -> Result<CultNetRudpSocketTransportConnection> {
let socket = UdpSocket::bind("127.0.0.1:0")?;
client_on(UdpSocket::bind("127.0.0.1:0")?, target, id, 1)
}
fn client_on(
socket: UdpSocket,
target: SocketAddr,
id: u32,
initial_sequence: u32,
) -> Result<CultNetRudpSocketTransportConnection> {
socket.set_nonblocking(true)?;
CultNetRudpSocketTransportConnection::new(CultNetRudpSocketTransportOptions {
media_delivery: None,
@ -169,7 +177,7 @@ fn client(target: SocketAddr, id: u32) -> Result<CultNetRudpSocketTransportConne
mode: CultNetRudpSocketMode::Client,
remote_addr: Some(target),
connection_id: id,
initial_sequence: 1,
initial_sequence: Some(initial_sequence),
resend_delay_ms: 10,
transport_id: None,
max_payload_bytes: None,
@ -291,7 +299,7 @@ fn same_connection_id_is_peer_scoped_and_raw_bytes_are_untouched() -> Result<()>
}
#[test]
fn duplicate_connect_preserves_epoch_and_fresh_epoch_is_separate() -> Result<()> {
fn a_second_connect_from_a_client_replaces_its_session_and_a_fresh_epoch_is_separate() -> Result<()> {
let clock = Clock::new(45_000);
let sink = Sink::default();
let mut server = server(Default::default(), clock, sink.clone(), Source::default())?;
@ -325,6 +333,97 @@ fn duplicate_connect_preserves_epoch_and_fresh_epoch_is_separate() -> Result<()>
Ok(())
}
/// A client that restarts on the same address and connection id starts a new
/// session: its Connect carries a sequence the accepted one did not, so the
/// server does not mistake it for a retransmit, and what it sends is delivered.
#[test]
fn a_restarted_client_on_the_same_address_and_connection_id_is_admitted() -> Result<()> {
let sink = Sink::default();
let mut server = server(Default::default(), Clock::new(46_000), sink.clone(), Source::default())?;
let target = server.local_addr()?;
let shared = UdpSocket::bind("127.0.0.1:0")?;
let twin = shared.try_clone()?;
let mut first = client_on(shared, target, 93, 50_000)?;
connect(&mut server, &mut [&mut first])?;
for index in 0..3 {
send(
&mut first,
&CultNetMessage::DocumentPutRaw {
message_id: format!("before-{index}"),
document: document("before", vec![index]),
},
)?;
}
for _ in 0..40 {
server.poll_once()?;
first.receive_once()?;
}
assert_eq!(sink.0.lock().unwrap().receipts.len(), 3);
let mut restarted = client_on(twin, target, 93, 7)?;
connect(&mut server, &mut [&mut restarted])?;
assert_eq!(server.session_count(), 1);
send(
&mut restarted,
&CultNetMessage::DocumentPutRaw {
message_id: "after-restart".into(),
document: document("after", vec![9]),
},
)?;
for _ in 0..40 {
server.poll_once()?;
restarted.receive_once()?;
}
let receipts = sink.0.lock().unwrap();
assert!(
receipts.receipts.iter().any(|r| r.message_id == "after-restart"),
"the restarted client's frame was not delivered"
);
Ok(())
}
/// A restarted client is a new session, and starts with a new payload budget:
/// what the old one spent is not charged to it.
#[test]
fn a_restarted_client_starts_with_a_fresh_payload_budget() -> Result<()> {
let put = |id: &str| CultNetMessage::DocumentPutRaw {
message_id: id.into(),
document: document("budget", vec![7; 64]),
};
let encoded = encode_cultnet_message_to_vec(&put("put-a"), CultNetWireContract::CultNetSchemaV0)?;
let options = CultMeshRudpDocumentServerOptions {
max_admitted_payload_bytes: encoded.len() * 2,
max_admitted_payload_bytes_per_session: encoded.len(),
max_snapshot_response_bytes: encoded.len(),
..Default::default()
};
let sink = Sink::default();
let mut server = server(options, Clock::new(47_000), sink.clone(), Source::default())?;
let target = server.local_addr()?;
let shared = UdpSocket::bind("127.0.0.1:0")?;
let twin = shared.try_clone()?;
let mut first = client_on(shared, target, 94, 50_000)?;
connect(&mut server, &mut [&mut first])?;
send(&mut first, &put("put-a"))?;
for _ in 0..20 {
server.poll_once()?;
}
assert_eq!(sink.0.lock().unwrap().receipts.len(), 1);
let mut restarted = client_on(twin, target, 94, 7)?;
connect(&mut server, &mut [&mut restarted])?;
send(&mut restarted, &put("put-b"))?;
for _ in 0..20 {
server.poll_once()?;
}
assert_eq!(
sink.0.lock().unwrap().receipts.len(),
2,
"the restarted client was charged what the old session spent"
);
Ok(())
}
#[test]
fn application_rejection_is_nonfatal_peer_scoped_and_unacknowledged() -> Result<()> {
let clock = Clock::new(48_000);
@ -576,7 +675,7 @@ fn a_connect_storm_and_stray_frames_are_dropped_not_fatal() -> Result<()> {
let session = |id| {
CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: id,
initial_sequence: 1,
initial_sequence: Some(1),
resend_delay_ms: 10,
max_pending_reliable_packets: None,
})

30
packages/cultmesh-ts/src/index.ts
View file

@ -16,6 +16,7 @@ import {
CultNetRudpSession,
CultNetPeer,
CultNetRudpSocketTransportConnection,
rudpClientBindHost,
CultNetSchemaCatalog,
CultNetShardCatalog,
cultNetBuiltinSchemaRegistry,
@ -4191,6 +4192,11 @@ export interface CultMeshRudpEndpoint {
}
export interface CultMeshRudpSocketOptions {
/**
* Local address to bind. Unset, a server binds loopback, and a client binds
* loopback for a loopback endpoint and the unspecified address of the
* endpoint's family otherwise, so it can reach a remote host.
*/
bindHost?: string;
bindPort?: number;
socket?: Socket;
@ -5654,7 +5660,7 @@ export class CultMesh {
if (packet.packetType === "connect") {
const connectPayload = Uint8Array.from(packet.payload ?? []);
if (record && byteArraysEqual(record.connectPayload, connectPayload)) {
if (record?.session.connectRepeats(packet)) {
socket.send(
encodeRudpPacket(record.session.acceptConnect(packet, nowMs)),
remote.port,
@ -5707,7 +5713,7 @@ export class CultMesh {
}
if (packet.reliable) {
socket.send(
encodeRudpPacket(record.session.createAckFor(packet.sequence)),
encodeRudpPacket(record.session.createAckForReceived(packet.sequence)),
record.remote.port,
record.remote.address,
);
@ -5996,7 +6002,7 @@ export class CultMesh {
options: CultMeshRudpSocketOptions = {},
): Promise<CultNetRudpSocketTransportConnection> {
requireNonEmpty(runtimeId, "runtimeId");
const socket = options.socket ?? (await bindRudpSocket(options));
const socket = options.socket ?? (await bindRudpSocket(options.bindHost ?? "127.0.0.1", options));
return new CultNetRudpSocketTransportConnection({
runtimeId,
socket,
@ -6022,7 +6028,10 @@ export class CultMesh {
requireNonEmpty(runtimeId, "runtimeId");
const parsedEndpoint =
typeof endpoint === "string" ? CultMesh.parseRudpEndpoint(endpoint) : endpoint;
const socket = options.socket ?? (await bindRudpSocket(options));
const socket = options.socket ?? (await bindRudpSocket(
options.bindHost ?? (await rudpClientBindHost(parsedEndpoint.host)),
options,
));
return new CultNetRudpSocketTransportConnection({
runtimeId,
socket,
@ -6419,9 +6428,8 @@ function toUint8Array(value: unknown): Uint8Array {
throw new Error("CultNet raw document payload was not binary.");
}
async function bindRudpSocket(options: CultMeshRudpSocketOptions): Promise<Socket> {
const socket = createSocket("udp4");
const host = options.bindHost ?? "127.0.0.1";
async function bindRudpSocket(host: string, options: CultMeshRudpSocketOptions): Promise<Socket> {
const socket = createSocket(host.includes(":") ? "udp6" : "udp4");
const port = options.bindPort ?? 0;
await new Promise<void>((resolve, reject) => {
socket.once("error", reject);
@ -6504,14 +6512,6 @@ function nonBlankOr(value?: string, fallback = ""): string {
return value && value.trim() ? value : fallback;
}
function byteArraysEqual(left: Uint8Array, right: Uint8Array): boolean {
if (left.byteLength !== right.byteLength) return false;
for (let index = 0; index < left.byteLength; index += 1) {
if (left[index] !== right[index]) return false;
}
return true;
}
function parseCultMeshLocalityKind(
value: string | undefined,
fallback: CultMeshLocalityKind,

38
packages/cultmesh-ts/test/cultmesh.test.ts
View file

@ -2635,6 +2635,44 @@ test("CultMesh TS branded facade exposes schema and shard catalogs", () => {
);
});
async function rudpClientBoundHost(endpoint: string, bindHost?: string): Promise<string | undefined> {
const client = await CultMesh.createRudpClient("cultmesh-ts-bind-client", 0x10203060, endpoint, { bindHost });
try {
return client.profile.transports[0]?.host;
} finally {
client.close();
}
}
// A client bound to loopback cannot send to another host on Windows, so a
// client for a remote endpoint binds the unspecified address of its family.
test("CultMesh TS RUDP client for a remote endpoint binds the unspecified address", async () => {
assert.equal(await rudpClientBoundHost("rudp://10.77.0.1:17872"), "0.0.0.0");
assert.equal(await rudpClientBoundHost("rudp://[2001:db8::1]:17872"), "::");
});
test("CultMesh TS RUDP client for a loopback endpoint binds loopback", async () => {
assert.equal(await rudpClientBoundHost("rudp://127.0.0.1:17872"), "127.0.0.1");
assert.equal(await rudpClientBoundHost("rudp://[::1]:17872"), "::1");
});
test("CultMesh TS RUDP client binds an explicit bindHost whatever the endpoint", async () => {
assert.equal(await rudpClientBoundHost("rudp://10.77.0.1:17872", "127.0.0.1"), "127.0.0.1");
assert.equal(await rudpClientBoundHost("rudp://127.0.0.1:17872", "0.0.0.0"), "0.0.0.0");
});
test("CultMesh TS RUDP server binds loopback unless told otherwise", async () => {
const loopback = await CultMesh.createRudpServer("cultmesh-ts-bind-server", 0x10203061);
const any = await CultMesh.createRudpServer("cultmesh-ts-bind-server", 0x10203062, { bindHost: "0.0.0.0" });
try {
assert.equal(loopback.profile.transports[0]?.host, "127.0.0.1");
assert.equal(any.profile.transports[0]?.host, "0.0.0.0");
} finally {
loopback.close();
any.close();
}
});
test("CultMesh TS branded facade creates RUDP clients from peer endpoints", async () => {
const connectionId = 0x10203044;
const server = await CultMesh.createRudpServer(

75
packages/cultmesh-ts/test/provider-rudp-transport.test.ts
View file

@ -8,6 +8,7 @@ import {
decodeRudpPacket,
encodeRudpPacket,
type CultNetOperationRequestMessage,
type CultNetRudpPacket,
} from "cultnet-ts";
import {
@ -320,6 +321,80 @@ test("a replaced RUDP generation drops operations queued behind an active handle
}
});
test("a client restarted on the same address with the same Connect payload replaces its RUDP session, and a retransmitted Connect does not", async () => {
const connectionId = 0x43554c54;
let closedSessions = 0;
const server = CultMesh.createRudpDocumentServer("odin-restart-test", connectionId, {
bindPort: 0,
documents: new CultNetDocumentRegistry(),
onSessionClosed: () => { closedSessions += 1; },
onError: () => undefined,
});
await server.start();
const socket = createSocket("udp4");
const received: CultNetRudpPacket[] = [];
socket.on("message", wire => received.push(decodeRudpPacket(wire)));
const toServer = (packet: CultNetRudpPacket) =>
socket.send(encodeRudpPacket(packet), server.bind.port, "127.0.0.1");
const evidence = Buffer.from("same connect payload");
try {
const first = new CultNetRudpSession({ connectionId, initialSequence: 50_000 });
const connect = first.createConnect(Date.now(), evidence);
toServer(connect);
await waitFor(() => received.some(packet => packet.packetType === "accept"), "the first Accept");
// A retransmitted Connect repeats: the session stays.
toServer(connect);
await new Promise(resolve => setTimeout(resolve, 100));
assert.equal(closedSessions, 0, "a retransmitted Connect closed the session");
// The same client restarted: another sequence, the same payload.
received.length = 0;
const second = new CultNetRudpSession({ connectionId, initialSequence: 7 });
toServer(second.createConnect(Date.now(), evidence));
await waitFor(() => closedSessions === 1, "the old RUDP session to close");
await waitFor(() => received.some(packet => packet.packetType === "accept"), "the restarted client's Accept");
} finally {
socket.close();
server.close();
}
});
test("the RUDP document server does not acknowledge by name a reliable packet its session did not receive", async () => {
const connectionId = 0x43554c55;
const server = CultMesh.createRudpDocumentServer("odin-ack-test", connectionId, {
bindPort: 0,
documents: new CultNetDocumentRegistry(),
onError: () => undefined,
});
await server.start();
const socket = createSocket("udp4");
const received: CultNetRudpPacket[] = [];
socket.on("message", wire => received.push(decodeRudpPacket(wire)));
const toServer = (packet: CultNetRudpPacket) =>
socket.send(encodeRudpPacket(packet), server.bind.port, "127.0.0.1");
try {
const client = new CultNetRudpSession({ connectionId, initialSequence: 50_000 });
const connect = client.createConnect(Date.now());
toServer(connect);
await waitFor(() => received.some(packet => packet.packetType === "accept"), "the Accept");
received.length = 0;
// A reliable packet the session ignores (a client does not send an Accept).
// The session never received it, so the Ack must not name it.
const ignored: CultNetRudpPacket = { ...client.createAck(), packetType: "accept", reliable: true, sequence: connect.sequence + 5 };
toServer(ignored);
await waitFor(() => received.some(packet => packet.packetType === "ack"), "the acknowledgement");
const ack = received.find(packet => packet.packetType === "ack")!;
const names = ack.ack === ignored.sequence
|| Array.from({ length: 32 }, (_, bit) => bit).some(bit => (ack.ackMask & (1 << bit)) !== 0 && ack.ack - bit - 1 === ignored.sequence);
assert.equal(names, false, "the server acknowledged by name a packet its session refused");
} finally {
socket.close();
server.close();
}
});
function deferred<T>(): { promise: Promise<T>; resolve(value: T): void } {
let resolve!: (value: T) => void;
return { promise: new Promise<T>(done => resolve = done), resolve };

2
packages/cultnet-py/src/cultnet_py/__init__.py
View file

@ -6,6 +6,7 @@ from .client import (
CultNetSchemaTransportFactory,
create_rudp_schema_transport,
create_tcp_framed_schema_transport,
rudp_client_bind_host,
)
from .cultmesh_contracts import (
PEER_EXCHANGE_REQUEST,
@ -228,6 +229,7 @@ __all__ = [
"compute_reconnect_delay_ms",
"create_reconnect_policy",
"create_rudp_schema_transport",
"rudp_client_bind_host",
"create_tcp_framed_schema_transport",
"create_rudp_transport_profile",
"create_tcp_framed_transport_profile",

27
packages/cultnet-py/src/cultnet_py/client.py
View file

@ -1,5 +1,6 @@
from __future__ import annotations
import ipaddress
import socket
import time
from collections.abc import Callable
@ -384,6 +385,22 @@ def create_tcp_framed_schema_transport(
)
def rudp_client_bind_host(endpoint_host: str) -> str:
"""Return the address an RUDP client binds when its caller names none.
Loopback for a loopback endpoint, otherwise the unspecified address of the
endpoint's family. A socket bound to loopback cannot send off the host on
Windows.
"""
family, _, _, _, address = socket.getaddrinfo(
endpoint_host.removeprefix("[").removesuffix("]"), None, type=socket.SOCK_DGRAM
)[0]
loopback = ipaddress.ip_address(address[0]).is_loopback
if family == socket.AF_INET6:
return "::1" if loopback else "::"
return "127.0.0.1" if loopback else "0.0.0.0"
def create_rudp_schema_transport(
*,
host: str,
@ -391,16 +408,20 @@ def create_rudp_schema_transport(
connection_id: int,
timeout_seconds: float = 4.0,
runtime_id: str = "cultnet-python-rudp-client",
bind_host: str = "127.0.0.1",
bind_host: str | None = None,
bind_port: int = 0,
initial_sequence: int = 1,
initial_sequence: int | None = None,
resend_delay_ms: int = 25,
transport_id: str = "rudp",
max_payload_bytes: int | None = None,
max_fragment_bytes: int | None = None,
max_pending_reliable_packets: int | None = None,
) -> CultNetRudpSocketTransportConnection:
transport_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
if bind_host is None:
bind_host = rudp_client_bind_host(host)
transport_socket = socket.socket(
socket.AF_INET6 if ":" in bind_host else socket.AF_INET, socket.SOCK_DGRAM
)
transport_socket.bind((bind_host, bind_port))
transport_socket.settimeout(min(timeout_seconds, 0.02))
transport = CultNetRudpSocketTransportConnection(

425
packages/cultnet-py/src/cultnet_py/transport.py
View file

@ -1,5 +1,6 @@
from __future__ import annotations
import secrets
import socket
import threading
import time
@ -292,10 +293,45 @@ class CultNetRudpReceiveResult:
disconnect_reason: bytes = b""
# How long a client keeps retransmitting a Connect nobody answered before it
# abandons that attempt for a fresh one. A Connect the server answers with an
# Ack, never an Accept, is one the server judged stale (see `accept_connect`);
# retransmitting the same sequence would never change its mind.
_RUDP_CONNECT_ATTEMPT_MS = 3_000
def _draw_sequence() -> int:
"""A sequence drawn from a secure source in [1, 2^31). The Connect's
sequence is what tells a peer whether a Connect repeats one it already
accepted or starts a new session, so two sessions must not share one by
default."""
return 1 + secrets.randbelow(2**31 - 1)
def _at_or_before(sequence: int, mark: int, window: int) -> bool:
"""Whether `sequence` is at or before `mark` in serial order, within the
receive window. The compare is modular, so it holds across the wrap of the
32-bit space; a sequence further back than the window is the duplicate
test's below-window clause, and one ahead of the mark is never before it."""
return ((mark - sequence) & 0xFFFFFFFF) < window
def _packet_acknowledges(packet: "CultNetRudpPacket", sequence: int) -> bool:
if packet.ack == sequence:
return True
return any(
packet.ack_mask & (1 << bit) and packet.ack > bit and packet.ack - bit - 1 == sequence
for bit in range(32)
)
@dataclass(frozen=True)
class CultNetRudpSessionOptions:
connection_id: int
initial_sequence: int = 1
# The first sequence this session issues. Unset, the session draws its own
# at random: every session is a new identity to its peer, and an options
# object reused for several must not give them one sequence.
initial_sequence: int | None = None
resend_delay_ms: int = 250
max_pending_reliable_packets: int | None = None
@ -320,7 +356,7 @@ class CultNetRudpSocketTransportOptions:
mode: CultNetRudpSocketMode
connection_id: int
remote_addr: tuple[str, int] | None = None
initial_sequence: int = 1
initial_sequence: int | None = None
resend_delay_ms: int = 250
transport_id: str = "rudp"
max_payload_bytes: int | None = None
@ -409,20 +445,49 @@ class CultNetRudpSession:
if options.max_pending_reliable_packets is not None and options.max_pending_reliable_packets <= 0:
raise ValueError("RUDP max_pending_reliable_packets must be greater than zero")
self.max_pending_reliable_packets = options.max_pending_reliable_packets
self._next_sequence = _uint32(options.initial_sequence, "initial_sequence")
self._next_sequence = _uint32(
_draw_sequence() if options.initial_sequence is None else options.initial_sequence,
"initial_sequence",
)
if self._next_sequence == 0xFFFFFFFF:
raise ValueError("RUDP initial_sequence must leave room for a reliable packet")
self._next_sequenced_by_channel: dict[str, int] = {}
self._next_fragment_id = 1
self._connected = False
# Advances every time a generation ends. Everything issued in a
# generation (writes, flushes) belongs to it and dies with it.
self._generation = 0
# True from the end of a generation until the next Connect or Accept
# begins one. A flush started in that interval has no live generation
# to wait on.
self._ended = False
# The sequence of the Connect that started the current generation: sent
# by this side, or accepted from the peer. A Connect repeats exactly when
# the session is connected and the Connect carries this sequence.
self._connect_sequence: int | None = None
# A Connect this side sent is unanswered. Only then is an Accept honoured.
self._awaiting_accept = False
# When the unanswered Connect first went out; `due_resends` abandons it
# for a fresh attempt once _RUDP_CONNECT_ATTEMPT_MS has passed.
self._connect_started_at_ms = 0
# The payload of the Connect this side sent, kept so a fresh attempt can
# always be built whatever became of the pending packet.
self._connect_payload = b""
self._last_received_at_ms: int | None = None
self._highest_received_sequence: int | None = None
self._received_sequences: set[int] = set()
self._latest_sequenced_by_channel: dict[str, int] = {}
self._pending_reliable: dict[int, _PendingReliablePacket] = {}
self._queued_reliable: deque[CultNetRudpPacket] = deque()
self._ordered_next_sequence_by_channel: dict[str, int] = {}
self._ordered_buffers: dict[str, dict[int, tuple[CultNetRudpDeliveredFrame, int]]] = {}
# Every reliable sequence up to and including this one has been received
# since the peer state was last reset. Only the handshake seeds it: the
# peer's Connect on the accepting side, the peer's Accept on the
# connecting side. Reliable data that arrives before that is refused.
self._received_through: int | None = None
# Ordered frames received but not yet deliverable, keyed by first
# sequence. A frame is held for exactly one reason: a reliable sequence
# below it has not arrived.
self._ordered_held: dict[int, CultNetRudpDeliveredFrame] = {}
self._fragment_buffers: dict[tuple[str, int], dict[str, Any]] = {}
# Matches cultnet-rs max_pending_fragment_sets. A set that loses one
# fragment is never completed; without a bound it is kept for the life
@ -455,23 +520,49 @@ class CultNetRudpSession:
def outstanding_reliable_packet_count(self) -> int:
return len(self._pending_reliable) + len(self._queued_reliable)
def reset_peer_state(self) -> None:
"""Forgets everything learned from the peer; sequence numbers already issued stay issued."""
@property
def generation(self) -> int:
return self._generation
@property
def ended(self) -> bool:
return self._ended
def _end_session(self) -> None:
"""The one way a session generation ends. The session stops being
connected and what it still owed the peer dies with it: a write not yet
acknowledged is dropped, so no later session retransmits it or credits
an ack to it. What was learned from the peer is not touched: the peer
may not know the session ended, and forgetting what it sent would let its
retransmits be delivered twice."""
self._connected = False
self._ended = True
self._awaiting_accept = False
self._generation += 1
self._pending_reliable.clear()
self._queued_reliable.clear()
def reset_peer_state(self) -> None:
"""Ends the current generation and forgets everything learned from the peer; sequence numbers already issued stay issued."""
self._end_session()
self._last_received_at_ms = None
self._highest_received_sequence = None
self._received_sequences.clear()
self._next_sequenced_by_channel.clear()
self._latest_sequenced_by_channel.clear()
self._pending_reliable.clear()
self._queued_reliable.clear()
self._ordered_next_sequence_by_channel.clear()
self._ordered_buffers.clear()
self._received_through = None
self._ordered_held.clear()
self._fragment_buffers.clear()
self._fragment_sets_evicted = 0
def create_connect(self, now_ms: int = 0, payload: bytes = b"") -> CultNetRudpPacket:
# A session that has had a peer starts a new generation: nothing it
# learned from that peer describes the one this Connect reaches, and
# what it still owed that peer no longer takes room in the queue.
if self._connect_sequence is not None:
self.reset_peer_state()
self._ensure_reliable_capacity(1)
self._ended = False
packet = self._create_packet(
CultNetRudpPacketType.CONNECT,
"control",
@ -479,22 +570,75 @@ class CultNetRudpSession:
reliable=True,
ordered=True,
)
self._connect_sequence = packet.sequence
self._awaiting_accept = True
self._connect_started_at_ms = now_ms
self._connect_payload = packet.payload
self._track_reliable(packet, now_ms)
return packet
def connect_repeats(self, packet: CultNetRudpPacket) -> bool:
"""Whether `packet` is a Connect this session's current generation
already owns: a retransmit of the Connect that started it (the session
is connected and the sequence is that Connect's), or a stale copy of an
earlier attempt by the same client (a sequence before it, within the
receive window). Servers that keep one session per peer ask this to tell
a Connect that starts a new session from one that does not; anything
else that reaches `accept_connect` starts a new generation."""
return (
packet.packet_type == CultNetRudpPacketType.CONNECT
and self._connected
and self._connect_sequence is not None
and (
self._connect_sequence == packet.sequence
or self._connect_is_stale(packet.sequence, self._connect_sequence)
)
)
def _connect_is_stale(self, sequence: int, current: int) -> bool:
"""A Connect that precedes the current generation's within the receive
window is the client's earlier attempt, delayed in the network: a client
that retried never sends a lower sequence again. Restarting on it would
strand the client, which honours only the Accept for its newest Connect.
The rule is TCP's answer to a delayed SYN (RFC 5961's challenge ACK):
keep the connection and answer with an Ack. A restarted client whose
random initial sequence lands in this window is answered the same way
and abandons the attempt for a fresh draw (_RUDP_CONNECT_ATTEMPT_MS)."""
return sequence != current and _at_or_before(sequence, current, self.RECEIVED_SEQUENCE_WINDOW)
def accept_connect(
self,
packet: CultNetRudpPacket,
now_ms: int = 0,
payload: bytes = b"",
) -> CultNetRudpPacket:
"""Answers a Connect. A repeat of the accepted one queues nothing, so a
Connect storm cannot grow the reliable queue: the reply is the Accept
still awaiting acknowledgement, or an Ack once it was acknowledged. A
stale copy of an earlier attempt gets the same reply and changes nothing
else: it is not evidence the peer is alive. Any other Connect ends the
current generation, forgets the peer and accepts a new one."""
self._require_connection(packet)
if packet.packet_type != CultNetRudpPacketType.CONNECT:
raise ValueError(f"Expected RUDP connect packet, got {packet.packet_type.value}")
if self.connect_repeats(packet):
if self._connect_sequence == packet.sequence:
self._apply_acknowledgements(packet)
self._remember_received(packet.sequence)
self._last_received_at_ms = now_ms
for pending in self._pending_reliable.values():
if pending.packet.packet_type == CultNetRudpPacketType.ACCEPT:
pending.last_sent_at_ms = now_ms
return pending.packet
return self.create_ack()
self.reset_peer_state()
self._ensure_reliable_capacity(1)
self._remember_received(packet.sequence)
self._seed_received(packet.sequence)
self._last_received_at_ms = now_ms
self._connect_sequence = packet.sequence
self._connected = True
self._ended = False
response = self._create_packet(
CultNetRudpPacketType.ACCEPT,
"control",
@ -505,25 +649,6 @@ class CultNetRudpSession:
self._track_reliable(response, now_ms)
return response
def answer_repeated_connect(self, packet: CultNetRudpPacket, now_ms: int = 0) -> CultNetRudpPacket:
"""Answers a Connect from a peer this session already accepted.
The packet's sequence is remembered and nothing is queued, so a Connect
storm cannot grow the reliable queue. The reply is the Accept still awaiting
acknowledgement, or an Ack once it was acknowledged.
"""
self._require_connection(packet)
if packet.packet_type != CultNetRudpPacketType.CONNECT:
raise ValueError(f"Expected RUDP connect packet, got {packet.packet_type.value}")
self._apply_acknowledgements(packet)
self._remember_received(packet.sequence)
self._last_received_at_ms = now_ms
for pending in self._pending_reliable.values():
if pending.packet.packet_type == CultNetRudpPacketType.ACCEPT:
pending.last_sent_at_ms = now_ms
return pending.packet
return self.create_ack()
def send(
self,
channel_id: str,
@ -589,17 +714,28 @@ class CultNetRudpSession:
def receive(self, packet: CultNetRudpPacket, now_ms: int = 0) -> CultNetRudpReceiveResult:
self._require_connection(packet)
self._apply_acknowledgements(packet)
# An Accept counts only while this side's Connect is unanswered and the
# Accept names it. A late or duplicate one, or one from an earlier
# generation, must not seed the watermark or revive an ended session.
honours_accept = (
packet.packet_type == CultNetRudpPacketType.ACCEPT
and self._awaiting_accept
and self._connect_sequence is not None
and _packet_acknowledges(packet, self._connect_sequence)
)
if packet.packet_type == CultNetRudpPacketType.ACCEPT and not honours_accept:
return CultNetRudpReceiveResult()
# While this side's Connect awaits its Accept, only the Accept it
# honours retires it: an Ack that names the Connect (a server's reply to
# a repeat or a stale copy) says the server did not start a session.
if honours_accept or not self._awaiting_accept:
self._apply_acknowledgements(packet)
ready_to_send = self._promote_queued_reliable(now_ms)
self._last_received_at_ms = now_ms
expected_sequence_if_uninitialized = (
packet.sequence
if self._highest_received_sequence is None
else self._highest_received_sequence + 1
)
if packet.packet_type == CultNetRudpPacketType.ACCEPT:
self._remember_received(packet.sequence)
if honours_accept:
self._awaiting_accept = False
self._seed_received(packet.sequence)
self._connected = True
return CultNetRudpReceiveResult(ready_to_send=ready_to_send)
@ -617,7 +753,7 @@ class CultNetRudpSession:
)
if packet.packet_type == CultNetRudpPacketType.DISCONNECT:
self._connected = False
self._end_session()
return CultNetRudpReceiveResult(
ready_to_send=ready_to_send,
disconnected=True,
@ -627,37 +763,38 @@ class CultNetRudpSession:
if packet.packet_type != CultNetRudpPacketType.DATA:
return CultNetRudpReceiveResult(ready_to_send=ready_to_send)
duplicate = packet.reliable and (
packet.sequence in self._received_sequences or (
self._highest_received_sequence is not None
and packet.sequence < self._highest_received_sequence
and self._highest_received_sequence - packet.sequence >= self.RECEIVED_SEQUENCE_WINDOW
)
)
# Reliable data before the handshake has seeded the watermark has no place
# in the order: refuse it unremembered, so it is not acknowledged and the
# sender retransmits it once the handshake is done.
if packet.reliable and self._received_through is None:
return CultNetRudpReceiveResult(ready_to_send=ready_to_send)
duplicate = packet.reliable and self._was_received(packet.sequence)
if packet.reliable:
self._remember_received(packet.sequence)
if duplicate:
return CultNetRudpReceiveResult(ready_to_send=ready_to_send)
delivered: list[CultNetRudpDeliveredFrame] = []
reassembled = self._reassemble(packet)
if reassembled is None:
return CultNetRudpReceiveResult(ready_to_send=ready_to_send)
frame, ordered, next_sequence = reassembled
if not ordered and packet.sequenced:
newest_sequence = next_sequence - 1
latest_sequence = self._latest_sequenced_by_channel.get(frame.channel_id)
if latest_sequence is not None and newest_sequence <= latest_sequence:
return CultNetRudpReceiveResult(ready_to_send=ready_to_send)
self._latest_sequenced_by_channel[frame.channel_id] = newest_sequence
return CultNetRudpReceiveResult(delivered=(frame,), ready_to_send=ready_to_send)
if not ordered:
return CultNetRudpReceiveResult(delivered=(frame,), ready_to_send=ready_to_send)
return CultNetRudpReceiveResult(
delivered=tuple(
self._deliver_ordered(frame, next_sequence, expected_sequence_if_uninitialized)
),
ready_to_send=ready_to_send,
)
if reassembled is not None:
frame, ordered, next_sequence = reassembled
if ordered:
self._ordered_held[frame.sequence] = frame
elif not packet.sequenced:
delivered.append(frame)
else:
newest_sequence = next_sequence - 1
latest_sequence = self._latest_sequenced_by_channel.get(frame.channel_id)
if latest_sequence is None or newest_sequence > latest_sequence:
self._latest_sequenced_by_channel[frame.channel_id] = newest_sequence
delivered.append(frame)
# Any reliable packet may have advanced the watermark, a fragment or an
# unordered frame as much as an ordered one, so the drain runs after all
# of them.
delivered.extend(self._drain_ordered())
return CultNetRudpReceiveResult(delivered=tuple(delivered), ready_to_send=ready_to_send)
def create_ack(self) -> CultNetRudpPacket:
ack, ack_mask = self._ack_state()
@ -683,7 +820,11 @@ class CultNetRudpSession:
)
def create_ack_for_received(self, sequence: int) -> CultNetRudpPacket:
# What `receive` refused is not acknowledged by name (the ack carries only
# what was received), so its sender retransmits it.
received_sequence = _uint32(sequence, "received sequence")
if not self._was_received(received_sequence):
return self.create_ack()
ack, _ = self._ack_state()
if ack >= received_sequence and ack - received_sequence <= 32:
return self.create_ack()
@ -693,7 +834,7 @@ class CultNetRudpSession:
return self._create_packet(CultNetRudpPacketType.PING, "control", payload)
def create_disconnect(self, reason: bytes = b"") -> CultNetRudpPacket:
self._connected = False
self._end_session()
return self._create_packet(CultNetRudpPacketType.DISCONNECT, "control", reason)
def check_timeout(self, now_ms: int, timeout_ms: int) -> bool:
@ -701,10 +842,13 @@ class CultNetRudpSession:
return False
if now_ms - self._last_received_at_ms <= timeout_ms:
return False
self._connected = False
self._end_session()
return True
def due_resends(self, now_ms: int) -> tuple[CultNetRudpPacket, ...]:
fresh = self._abandon_unanswered_connect(now_ms)
if fresh is not None:
return (fresh,)
due: list[CultNetRudpPacket] = []
for pending in self._pending_reliable.values():
if now_ms - pending.last_sent_at_ms >= self.resend_delay_ms:
@ -712,6 +856,24 @@ class CultNetRudpSession:
due.append(pending.packet)
return tuple(sorted(due, key=lambda packet: packet.sequence))
def _abandon_unanswered_connect(self, now_ms: int) -> CultNetRudpPacket | None:
"""A Connect unanswered for _RUDP_CONNECT_ATTEMPT_MS is replaced, not
retransmitted further, by a Connect with the same payload and the
abandoned sequence plus the receive window less one. Nothing was ever
sent in an unanswered generation, so no sequence issued so far is owed.
The jump keeps a late copy of the abandoned Connect inside the new one's
stale window, so a server that took the new Connect answers the copy
with an Ack instead of restarting; and it leaves the stale window of
whatever generation the server holds, unless that generation sits
exactly at the jump, and then the next attempt leaves it. The first
Connect of a session is the only one drawn at random."""
if not self._awaiting_accept or now_ms - self._connect_started_at_ms < _RUDP_CONNECT_ATTEMPT_MS:
return None
if self._connect_sequence is None:
return None
self._next_sequence = min(self._connect_sequence + self.RECEIVED_SEQUENCE_WINDOW - 1, 0xFFFFFFFE)
return self.create_connect(now_ms, self._connect_payload)
def _create_packet(
self,
packet_type: CultNetRudpPacketType,
@ -806,8 +968,39 @@ class CultNetRudpSession:
if packet.ack_mask & (1 << bit):
self._pending_reliable.pop(packet.ack - bit - 1, None)
def _seed_received(self, sequence: int) -> None:
"""The handshake's one act on the watermark: the peer's Connect or Accept
is the first sequence of the session, whatever else it has sent."""
self._received_through = sequence
self._remember_received(sequence)
def _was_received(self, sequence: int) -> bool:
"""The one duplicate test: true for a sequence in the window that was
received, for one below the window, and for one at or before the
watermark. The watermark starts at the handshake's seed, so a frame the
peer sent in an earlier generation (every sequence it issued is below the
Connect that began this one) is a duplicate of something already
delivered and is acknowledged, never delivered again."""
return (
sequence in self._received_sequences
or (
self._highest_received_sequence is not None
and sequence < self._highest_received_sequence
and self._highest_received_sequence - sequence >= self.RECEIVED_SEQUENCE_WINDOW
)
or (
self._received_through is not None
and _at_or_before(sequence, self._received_through, self.RECEIVED_SEQUENCE_WINDOW)
)
)
def _remember_received(self, sequence: int) -> None:
self._received_sequences.add(sequence)
if self._received_through is not None:
through = self._received_through
while through + 1 in self._received_sequences:
through += 1
self._received_through = through
if self._highest_received_sequence is None or sequence > self._highest_received_sequence:
self._highest_received_sequence = sequence
if len(self._received_sequences) > self.RECEIVED_SEQUENCE_WINDOW:
@ -865,58 +1058,15 @@ class CultNetRudpSession:
max(sequences) + 1,
)
def _deliver_ordered(
self,
frame: CultNetRudpDeliveredFrame,
next_after_frame: int,
expected_sequence_if_uninitialized: int,
) -> list[CultNetRudpDeliveredFrame]:
next_sequence = self._ordered_next_sequence_by_channel.get(frame.channel_id)
if next_sequence is None:
next_sequence = min(expected_sequence_if_uninitialized, frame.sequence)
self._ordered_next_sequence_by_channel[frame.channel_id] = next_sequence
while (
frame.sequence > next_sequence
and next_sequence in self._received_sequences
and next_sequence not in self._ordered_buffers.get(frame.channel_id, {})
):
next_sequence += 1
self._ordered_next_sequence_by_channel[frame.channel_id] = next_sequence
if frame.sequence < next_sequence:
def _drain_ordered(self) -> list[CultNetRudpDeliveredFrame]:
"""Delivers, in sequence order, every held ordered frame whose first
sequence is at most one past the watermark. Ordered delivery has one
owner: the watermark. A frame this call does not deliver waits for the
sequence below it, and nothing else releases it."""
if self._received_through is None:
return []
if frame.sequence > next_sequence:
self._ordered_buffers.setdefault(frame.channel_id, {})[frame.sequence] = (frame, next_after_frame)
return []
self._ordered_next_sequence_by_channel[frame.channel_id] = next_after_frame
return [frame, *self._drain_ordered(frame.channel_id)]
def _drain_ordered(self, channel_id: str) -> list[CultNetRudpDeliveredFrame]:
delivered: list[CultNetRudpDeliveredFrame] = []
buffer = self._ordered_buffers.get(channel_id)
if buffer is None:
return delivered
while True:
next_sequence = self._ordered_next_sequence_by_channel[channel_id]
pending = buffer.pop(next_sequence, None)
if pending is None:
break
frame, next_after_frame = pending
delivered.append(frame)
self._ordered_next_sequence_by_channel[channel_id] = next_after_frame
self._skip_received_non_channel_sequences(channel_id)
return delivered
def _skip_received_non_channel_sequences(self, channel_id: str) -> None:
next_sequence = self._ordered_next_sequence_by_channel.get(channel_id)
while (
next_sequence is not None
and next_sequence in self._received_sequences
and next_sequence not in self._ordered_buffers.get(channel_id, {})
):
next_sequence += 1
self._ordered_next_sequence_by_channel[channel_id] = next_sequence
releasable = sorted(sequence for sequence in self._ordered_held if sequence <= self._received_through + 1)
return [self._ordered_held.pop(sequence) for sequence in releasable]
def _allocate_fragment_id(self) -> int:
fragment_id = self._next_fragment_id
@ -1033,6 +1183,10 @@ class CultNetRudpSocketTransportConnection:
if packet.connection_id != self.session.connection_id:
self._packets_dropped += 1
return None
# A Connect from another endpoint is a new client, whatever sequence it
# carries: the endpoint moves and the Connect starts a new generation.
# Only a Connect from the peer's own endpoint can be its repeat.
connect_from_new_endpoint = False
if self.remote_addr is None:
# Only a Connect claims the endpoint of a server-mode transport.
if self.mode == CultNetRudpSocketMode.SERVER and packet.packet_type != CultNetRudpPacketType.CONNECT:
@ -1040,18 +1194,21 @@ class CultNetRudpSocketTransportConnection:
return None
self.remote_addr = remote_addr
elif remote_addr != self.remote_addr:
self._packets_dropped += 1
return None
if self.mode == CultNetRudpSocketMode.SERVER and packet.packet_type == CultNetRudpPacketType.CONNECT:
self.remote_addr = remote_addr
connect_from_new_endpoint = True
else:
self._packets_dropped += 1
return None
if self.mode == CultNetRudpSocketMode.SERVER and packet.packet_type == CultNetRudpPacketType.CONNECT:
# A Connect from the peer this transport already accepted repeats:
# answer it with the Accept already owed and queue nothing.
# The session answers a repeat of the Connect it accepted with the
# Accept already owed, and anything else with a new generation.
try:
if self.session.connected:
reply = self.session.answer_repeated_connect(packet, _now_ms())
else:
reply = self.session.accept_connect(packet, _now_ms())
self.disconnect_reason = None
if connect_from_new_endpoint:
self.session.reset_peer_state()
reply = self.session.accept_connect(packet, _now_ms())
self.disconnect_reason = None
except ValueError:
self._packets_dropped += 1
return None
@ -1105,14 +1262,19 @@ class CultNetRudpSocketTransportConnection:
preserved = deque(self._delivered_frames)
self._delivered_frames.clear()
self.socket.settimeout(poll_timeout)
# A flush belongs to the generation it started in: whatever ends that
# generation, and whatever begins after it, the writes being waited on
# are gone.
generation = self.session.generation
try:
while True:
# An ended session forgot its unacknowledged writes; reporting
# them flushed would be a lie.
if self.disconnect_reason is not None:
if self.session.ended or self.session.generation != generation:
reason = self.disconnect_reason
raise ConnectionError(
"RUDP session ended before its reliable writes were acknowledged: "
+ self.disconnect_reason.decode("utf-8", errors="replace")
+ ("no reason given" if reason is None else reason.decode("utf-8", errors="replace"))
)
if self.session.outstanding_reliable_packet_count == 0:
return
@ -1137,7 +1299,10 @@ class CultNetRudpSocketTransportConnection:
self._send_packet(self.session.create_ping(payload))
def check_timeout(self, timeout_ms: int, now_ms: int | None = None) -> bool:
return self.session.check_timeout(_now_ms() if now_ms is None else now_ms, timeout_ms)
timed_out = self.session.check_timeout(_now_ms() if now_ms is None else now_ms, timeout_ms)
if timed_out:
self.disconnect_reason = b"session timed out"
return timed_out
def poll_resends(self) -> None:
for packet in self.session.due_resends(_now_ms()):

286
packages/cultnet-py/tests/test_cultnet.py
View file

@ -30,6 +30,8 @@ from cultnet_py.interop_peer import (
)
from cultnet_py import (
compute_simulation_claim_hash,
create_rudp_schema_transport,
rudp_client_bind_host,
CultNetDatabaseChange,
CultNetRawClient,
CultNetRawDocumentRecord,
@ -94,6 +96,17 @@ from cultnet_py import (
)
def acknowledges(packet, sequence: int) -> bool:
"""Whether the packet's ack field and mask cover the sequence."""
if packet.ack == sequence:
return True
return (
packet.ack > sequence
and packet.ack - sequence <= 32
and (packet.ack_mask & (1 << (packet.ack - sequence - 1))) != 0
)
def bind_udp_socket() -> socket.socket:
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind(("127.0.0.1", 0))
@ -616,12 +629,7 @@ class CultNetTests(unittest.TestCase):
receiver = CultNetRudpSession(
CultNetRudpSessionOptions(connection_id=7, initial_sequence=200, resend_delay_ms=100)
)
sender.receive(
CultNetRudpPacket(CultNetRudpPacketType.ACCEPT, 7, 1, 0, 0, "control")
)
receiver.receive(
CultNetRudpPacket(CultNetRudpPacketType.ACCEPT, 7, 2, 0, 0, "control")
)
sender.receive(receiver.accept_connect(sender.create_connect(0), 0), 0)
first = sender.send(
"schema",
@ -638,27 +646,28 @@ class CultNetTests(unittest.TestCase):
b"third",
CultNetRudpSendOptions(reliable=True, ordered=True, now_ms=0),
)
self.assertEqual(sender.pending_reliable_sequences, (10, 11, 12))
self.assertEqual(sender.pending_reliable_sequences, (11, 12, 13))
receiver.receive(first)
receiver.receive(third)
ack_with_gap = receiver.create_ack()
self.assertEqual(ack_with_gap.ack, 12)
self.assertEqual(ack_with_gap.ack_mask, 0b10 | (1 << 9))
self.assertEqual(ack_with_gap.ack, 13)
self.assertEqual(ack_with_gap.ack_mask, 0b110)
sender.receive(ack_with_gap)
self.assertEqual(sender.pending_reliable_sequences, (11,))
self.assertEqual(sender.pending_reliable_sequences, (12,))
receiver.receive(second)
full_ack = receiver.create_ack()
self.assertEqual(full_ack.ack, 12)
self.assertEqual(full_ack.ack_mask, 0b11 | (1 << 9))
self.assertEqual(full_ack.ack, 13)
self.assertEqual(full_ack.ack_mask, 0b111)
sender.receive(full_ack)
self.assertEqual(sender.pending_reliable_sequences, ())
def test_cultnet_rudp_session_schedules_reliable_resends_until_acked(self) -> None:
session = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=99, initial_sequence=1, resend_delay_ms=100))
session.receive(
CultNetRudpPacket(CultNetRudpPacketType.ACCEPT, 99, 50, 0, 0, "control")
CultNetRudpSession(CultNetRudpSessionOptions(connection_id=99, initial_sequence=900)).accept_connect(session.create_connect(0), 0),
0,
)
sent = session.send(
"schema",
@ -700,7 +709,7 @@ class CultNetTests(unittest.TestCase):
server = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=103, initial_sequence=100))
connect = client.create_connect(0, b"join")
accept = server.accept_connect(connect, 0)
reply = server.answer_repeated_connect(replace(connect, ack=accept.sequence), 1)
reply = server.accept_connect(replace(connect, ack=accept.sequence), 1)
self.assertEqual(reply.packet_type, CultNetRudpPacketType.ACK)
self.assertEqual(server.outstanding_reliable_packet_count, 0)
@ -708,17 +717,23 @@ class CultNetTests(unittest.TestCase):
session = CultNetRudpSession(
CultNetRudpSessionOptions(connection_id=102, initial_sequence=1, max_pending_reliable_packets=2)
)
session.receive(CultNetRudpPacket(CultNetRudpPacketType.ACCEPT, 102, 50, 0, 0, "control"))
session.receive(
CultNetRudpSession(CultNetRudpSessionOptions(connection_id=102, initial_sequence=900)).accept_connect(session.create_connect(0), 0),
0,
)
session.send("schema", b"first", CultNetRudpSendOptions(reliable=True, ordered=True))
session.send("schema", b"second", CultNetRudpSendOptions(reliable=True, ordered=True))
with self.assertRaisesRegex(ValueError, "reliable send queue is full"):
session.send("schema", b"third", CultNetRudpSendOptions(reliable=True, ordered=True))
self.assertEqual(session.pending_reliable_sequences, (1, 2))
self.assertEqual(session.pending_reliable_sequences, (2, 3))
fragmented = CultNetRudpSession(
CultNetRudpSessionOptions(connection_id=103, initial_sequence=1, max_pending_reliable_packets=3)
)
fragmented.receive(CultNetRudpPacket(CultNetRudpPacketType.ACCEPT, 103, 50, 0, 0, "control"))
fragmented.receive(
CultNetRudpSession(CultNetRudpSessionOptions(connection_id=103, initial_sequence=900)).accept_connect(fragmented.create_connect(0), 0),
0,
)
with self.assertRaisesRegex(ValueError, "reliable send queue is full"):
fragmented.send_many(
"schema",
@ -731,12 +746,7 @@ class CultNetTests(unittest.TestCase):
def test_cultnet_rudp_session_suppresses_duplicates_and_delivers_reliable_ordered_payloads(self) -> None:
sender = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=123, initial_sequence=1))
receiver = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=123, initial_sequence=100))
sender.receive(
CultNetRudpPacket(CultNetRudpPacketType.ACCEPT, 123, 90, 0, 0, "control")
)
receiver.receive(
CultNetRudpPacket(CultNetRudpPacketType.ACCEPT, 123, 91, 0, 0, "control")
)
sender.receive(receiver.accept_connect(sender.create_connect(0), 0))
first = sender.send("schema", b"first", CultNetRudpSendOptions(reliable=True, ordered=True))
second = sender.send("schema", b"second", CultNetRudpSendOptions(reliable=True, ordered=True))
@ -753,8 +763,7 @@ class CultNetTests(unittest.TestCase):
def test_cultnet_rudp_lossy_packets_cannot_create_reliable_ordered_gaps(self) -> None:
sender = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=198, initial_sequence=1))
receiver = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=198, initial_sequence=100))
sender.receive(CultNetRudpPacket(CultNetRudpPacketType.ACCEPT, 198, 90, 0, 0, "control"))
receiver.receive(CultNetRudpPacket(CultNetRudpPacketType.ACCEPT, 198, 91, 0, 0, "control"))
sender.receive(receiver.accept_connect(sender.create_connect(0), 0), 0)
realtime = sender.send("realtime", b"discarded realtime", CultNetRudpSendOptions())
latest = sender.send("latest", b"discarded latest state", CultNetRudpSendOptions(sequenced=True))
@ -770,8 +779,7 @@ class CultNetTests(unittest.TestCase):
def test_cultnet_rudp_unreliable_sequenced_delivery_is_scoped_to_its_channel(self) -> None:
sender = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=197, initial_sequence=50))
receiver = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=197, initial_sequence=100))
sender.receive(CultNetRudpPacket(CultNetRudpPacketType.ACCEPT, 197, 90, 0, 0, "control"))
receiver.receive(CultNetRudpPacket(CultNetRudpPacketType.ACCEPT, 197, 91, 0, 0, "control"))
sender.receive(receiver.accept_connect(sender.create_connect(0), 0), 0)
older = sender.send("latest", b"older", CultNetRudpSendOptions(sequenced=True))
newer = sender.send("latest", b"newer", CultNetRudpSendOptions(sequenced=True))
@ -786,7 +794,10 @@ class CultNetTests(unittest.TestCase):
def test_cultnet_rudp_rejects_unreliable_ordered_delivery(self) -> None:
session = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=196, initial_sequence=1))
session.receive(CultNetRudpPacket(CultNetRudpPacketType.ACCEPT, 196, 50, 0, 0, "control"))
session.receive(
CultNetRudpSession(CultNetRudpSessionOptions(connection_id=196, initial_sequence=900)).accept_connect(session.create_connect(0), 0),
0,
)
with self.assertRaises(ValueError) as raised:
session.send(
@ -799,12 +810,7 @@ class CultNetTests(unittest.TestCase):
def test_cultnet_rudp_session_skips_control_packets_while_ordering_schema_payloads(self) -> None:
sender = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=124, initial_sequence=1))
receiver = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=124, initial_sequence=100))
sender.receive(
CultNetRudpPacket(CultNetRudpPacketType.ACCEPT, 124, 90, 0, 0, "control")
)
receiver.receive(
CultNetRudpPacket(CultNetRudpPacketType.ACCEPT, 124, 91, 0, 0, "control")
)
sender.receive(receiver.accept_connect(sender.create_connect(0), 0), 0)
first = sender.send("schema", b"first", CultNetRudpSendOptions(reliable=True, ordered=True))
control = sender.create_ack()
@ -817,12 +823,7 @@ class CultNetTests(unittest.TestCase):
def test_cultnet_rudp_session_fragments_and_reassembles_reliable_ordered_payloads(self) -> None:
sender = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=456, initial_sequence=1))
receiver = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=456, initial_sequence=100))
sender.receive(
CultNetRudpPacket(CultNetRudpPacketType.ACCEPT, 456, 90, 0, 0, "control")
)
receiver.receive(
CultNetRudpPacket(CultNetRudpPacketType.ACCEPT, 456, 91, 0, 0, "control")
)
sender.receive(receiver.accept_connect(sender.create_connect(0), 0), 0)
packets = sender.send_many(
"schema",
@ -847,8 +848,7 @@ class CultNetTests(unittest.TestCase):
connection_id = 457
sender = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=connection_id, initial_sequence=1))
receiver = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=connection_id, initial_sequence=100))
sender.receive(CultNetRudpPacket(CultNetRudpPacketType.ACCEPT, connection_id, 0, 0, 0, "control"))
receiver.receive(CultNetRudpPacket(CultNetRudpPacketType.ACCEPT, connection_id, 0, 0, 0, "control"))
sender.receive(receiver.accept_connect(sender.create_connect(0), 0), 0)
fragment_count = CultNetRudpSession.RELIABLE_SEND_WINDOW_PACKETS + 17
payload = bytes(index % 251 for index in range(fragment_count * 8))
@ -1068,14 +1068,13 @@ class CultNetTests(unittest.TestCase):
# The peer is told, and the goodbye does not acknowledge the refused frame.
(goodbye,) = [p for p in drain_peer() if p.packet_type == CultNetRudpPacketType.DISCONNECT]
self.assertTrue(peer.receive(goodbye, 1).disconnected)
self.assertIn(poison.sequence, peer.pending_reliable_sequences)
self.assertFalse(acknowledges(goodbye, poison.sequence), "the goodbye acknowledged the refused frame")
# A retransmit is dropped, not acknowledged.
to_server(poison)
self.assertIsNone(server.receive_once())
self.assertEqual(server.stats.packets_dropped, 2)
self.assertEqual(drain_peer(), [])
self.assertIn(poison.sequence, peer.pending_reliable_sequences)
# A Connect claims the endpoint again.
again = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=connection_id))
@ -1145,6 +1144,135 @@ class CultNetTests(unittest.TestCase):
peer_socket.close()
server.close()
def test_cultnet_rudp_server_mode_admits_a_restarted_client_and_keeps_its_session_on_a_retransmit(self) -> None:
server, _, peer_socket, _, to_server, drain_peer = self._server_mode_pair(0x10203058)
connection_id = 0x10203058
reliable = CultNetRudpSendOptions(reliable=True, ordered=True)
try:
first = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=connection_id, initial_sequence=50_000))
connect = first.create_connect(0, b"same")
to_server(connect)
server.receive_once()
first.receive([p for p in drain_peer() if p.packet_type == CultNetRudpPacketType.ACCEPT][0], 0)
sent = [first.send("schema", f"c{index}".encode(), reliable) for index in range(3)]
delivered = []
for packet in sent:
to_server(packet)
delivered.append(server.receive_once().payload)
self.assertEqual(delivered, [b"c0", b"c1", b"c2"])
# A retransmitted Connect is a repeat: what was delivered stays delivered once.
to_server(connect)
server.receive_once()
to_server(sent[0])
self.assertIsNone(server.receive_once(), "the retransmitted Connect reset the session")
# The same client restarted on the same address, connection id and payload.
drain_peer()
second = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=connection_id, initial_sequence=7))
to_server(second.create_connect(0, b"same"))
server.receive_once()
second.receive([p for p in drain_peer() if p.packet_type == CultNetRudpPacketType.ACCEPT][0], 0)
to_server(second.send("schema", b"after restart", reliable))
self.assertEqual(server.receive_once().payload, b"after restart")
finally:
peer_socket.close()
server.close()
def test_cultnet_rudp_server_mode_admits_a_pinned_client_that_restarts_on_a_new_port(self) -> None:
server, server_socket, peer_socket, _, to_server, drain_peer = self._server_mode_pair(0x1020305B)
connection_id = 0x1020305B
reliable = CultNetRudpSendOptions(reliable=True, ordered=True)
other_socket = bind_udp_socket()
try:
first = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=connection_id, initial_sequence=1))
to_server(first.create_connect(0, b"join"))
server.receive_once()
first.receive([p for p in drain_peer() if p.packet_type == CultNetRudpPacketType.ACCEPT][0], 0)
to_server(first.send("schema", b"hello", reliable))
self.assertEqual(server.receive_once().payload, b"hello")
# The process restarted on a new port with the same pinned sequence.
second = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=connection_id, initial_sequence=1))
other_socket.sendto(encode_rudp_packet(second.create_connect(0, b"join")), server_socket.getsockname())
server.receive_once()
wire, _ = other_socket.recvfrom(65535)
second.receive(decode_rudp_packet(wire), 0)
self.assertTrue(second.connected, "the restarted client on a new port was not admitted")
other_socket.sendto(encode_rudp_packet(second.send("schema", b"after", reliable)), server_socket.getsockname())
self.assertEqual(server.receive_once().payload, b"after")
finally:
other_socket.close()
peer_socket.close()
server.close()
def test_cultnet_rudp_server_mode_admits_a_pinned_client_that_restarts_on_the_same_address(self) -> None:
server, _, peer_socket, _, to_server, drain_peer = self._server_mode_pair(0x1020305C)
connection_id = 0x1020305C
try:
first = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=connection_id, initial_sequence=1))
to_server(first.create_connect(0, b"join"))
server.receive_once()
accept = [p for p in drain_peer() if p.packet_type == CultNetRudpPacketType.ACCEPT][0]
first.receive(accept, 0)
to_server(first.create_ack_for_received(accept.sequence))
server.receive_once()
drain_peer()
# The process restarted on the same address with the same pinned sequence.
restarted = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=connection_id, initial_sequence=1))
to_server(restarted.create_connect(0, b"join"))
server.receive_once()
for packet in drain_peer():
restarted.receive(packet, 1)
self.assertFalse(restarted.connected)
to_server(restarted.due_resends(3_000)[0])
server.receive_once()
for packet in drain_peer():
restarted.receive(packet, 3_001)
self.assertTrue(restarted.connected, "the restarted pinned client was never admitted")
finally:
peer_socket.close()
server.close()
def test_cultnet_rudp_server_mode_admits_a_connect_from_a_new_endpoint(self) -> None:
server, server_socket, peer_socket, peer, to_server, drain_peer = self._server_mode_pair(0x10203059)
other_socket = bind_udp_socket()
try:
to_server(peer.create_connect(0, b"a"))
server.receive_once()
drain_peer()
other = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=0x10203059, initial_sequence=9))
other_socket.sendto(encode_rudp_packet(other.create_connect(0, b"b")), server_socket.getsockname())
server.receive_once()
wire, _ = other_socket.recvfrom(65535)
other.receive(decode_rudp_packet(wire), 0)
self.assertTrue(other.connected, "the new endpoint was not admitted")
other_socket.sendto(
encode_rudp_packet(other.send("schema", b"from b", CultNetRudpSendOptions(reliable=True, ordered=True))),
server_socket.getsockname(),
)
self.assertEqual(server.receive_once().payload, b"from b")
finally:
other_socket.close()
peer_socket.close()
server.close()
def test_cultnet_rudp_flush_after_a_timeout_names_the_timeout(self) -> None:
server, _, peer_socket, peer, to_server, drain_peer = self._server_mode_pair(0x1020305A)
try:
to_server(peer.create_connect(0, b"join"))
server.receive_once()
server.send("schema", b"unanswered")
self.assertTrue(server.check_timeout(0, now_ms=10**14))
with self.assertRaisesRegex(ConnectionError, "session timed out"):
server.flush_reliable(0.1)
finally:
peer_socket.close()
server.close()
def test_cultnet_rudp_session_rejects_limits_that_cannot_admit_a_connect(self) -> None:
with self.assertRaises(ValueError):
CultNetRudpSession(CultNetRudpSessionOptions(connection_id=1, initial_sequence=0xFFFFFFFF))
@ -2320,12 +2448,78 @@ class CultNetTests(unittest.TestCase):
self.assertEqual(CultNetSimulationConsensusCandidate.from_wire(candidates[0]), typed_candidates[0])
def _non_loopback_ipv4_address() -> str | None:
probe = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
try:
probe.connect(("192.0.2.1", 9))
address = probe.getsockname()[0]
except OSError:
return None
finally:
probe.close()
return None if address.startswith("127.") or address == "0.0.0.0" else address
class CultNetRudpClientBindTests(unittest.TestCase):
def test_an_rudp_client_binds_loopback_only_for_a_loopback_endpoint(self) -> None:
self.assertEqual(rudp_client_bind_host("10.77.0.1"), "0.0.0.0")
self.assertEqual(rudp_client_bind_host("2001:db8::1"), "::")
self.assertEqual(rudp_client_bind_host("[2001:db8::1]"), "::")
self.assertEqual(rudp_client_bind_host("127.0.0.1"), "127.0.0.1")
self.assertEqual(rudp_client_bind_host("127.0.0.2"), "127.0.0.1")
self.assertEqual(rudp_client_bind_host("::1"), "::1")
self.assertEqual(rudp_client_bind_host("[::1]"), "::1")
# A socket bound to loopback cannot send off the host on Windows. On Linux
# it reaches a local non-loopback address, but from 127.0.0.1, so the
# source address shows which one the client bound.
def test_a_schema_transport_for_a_non_loopback_endpoint_does_not_send_from_loopback(self) -> None:
address = _non_loopback_ipv4_address()
if address is None:
self.skipTest("no non-loopback IPv4 address on this host")
listener = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
listener.bind(("0.0.0.0", 0))
listener.settimeout(2.0)
try:
with self.assertRaises(TimeoutError):
create_rudp_schema_transport(
host=address,
port=listener.getsockname()[1],
connection_id=0x43554C54,
timeout_seconds=0.2,
)
_, source = listener.recvfrom(65535)
self.assertEqual(source[0], address)
finally:
listener.close()
def test_a_schema_transport_binds_an_explicit_bind_host(self) -> None:
address = _non_loopback_ipv4_address()
if address is None:
self.skipTest("no non-loopback IPv4 address on this host")
listener = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
listener.bind(("0.0.0.0", 0))
listener.settimeout(2.0)
try:
with self.assertRaises(TimeoutError):
create_rudp_schema_transport(
host=address,
port=listener.getsockname()[1],
connection_id=0x43554C54,
timeout_seconds=0.2,
bind_host="127.0.0.1",
)
_, source = listener.recvfrom(65535)
self.assertEqual(source[0], "127.0.0.1")
finally:
listener.close()
class CultNetRudpFragmentBoundTests(unittest.TestCase):
def _pair(self) -> tuple[CultNetRudpSession, CultNetRudpSession]:
sender = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=311, initial_sequence=1))
receiver = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=311, initial_sequence=100))
sender.receive(CultNetRudpPacket(CultNetRudpPacketType.ACCEPT, 311, 90, 0, 0, "control"))
receiver.receive(CultNetRudpPacket(CultNetRudpPacketType.ACCEPT, 311, 91, 0, 0, "control"))
sender.receive(receiver.accept_connect(sender.create_connect(0), 0), 0)
return sender, receiver
def test_stranded_fragment_sets_are_bounded_and_evicted_oldest_first(self) -> None:

427
packages/cultnet-py/tests/test_rudp_connect_generation.py Normal file
View file

@ -0,0 +1,427 @@
"""A Connect's sequence says whether it repeats the one a session accepted, and
the handshake alone seeds the watermark that orders delivery."""
from __future__ import annotations
import unittest
from dataclasses import replace
from cultnet_py.transport import (
CultNetRudpPacketType,
CultNetRudpSendOptions,
CultNetRudpSession,
CultNetRudpSessionOptions,
CultNetRudpSocketMode,
CultNetRudpSocketTransportOptions,
)
CONNECTION_ID = 0x10203090
def session(initial_sequence: int) -> CultNetRudpSession:
return CultNetRudpSession(CultNetRudpSessionOptions(connection_id=CONNECTION_ID, initial_sequence=initial_sequence))
def send(from_session: CultNetRudpSession, text: str, *, ordered: bool = True, channel: str = "schema"):
return from_session.send(channel, text.encode(), CultNetRudpSendOptions(reliable=True, ordered=ordered))
def names(result) -> list[str]:
return [frame.payload.decode() for frame in result.delivered]
def handshake(client: CultNetRudpSession, server: CultNetRudpSession):
connect = client.create_connect(0)
accept = server.accept_connect(connect, 0)
client.receive(accept, 0)
assert client.connected and server.connected
return connect, accept
class CultNetRudpConnectGenerationTests(unittest.TestCase):
def test_a_reconnect_on_the_same_session_is_not_stranded_by_a_write_the_ended_session_owed(self) -> None:
client, server = session(1), session(500)
handshake(client, server)
send(server, "lost")
server.create_disconnect(b"gone")
client.receive(server.accept_connect(client.create_connect(1), 1), 1)
self.assertEqual(names(client.receive(send(server, "after"), 2)), ["after"])
def test_a_server_that_accepts_a_new_client_forgets_the_old_clients_sequences(self) -> None:
old_client, server = session(50), session(500)
handshake(old_client, server)
for name in ("c1", "c2", "c3"):
self.assertEqual(names(server.receive(send(old_client, name), 1)), [name])
server.create_disconnect(b"restart")
new_client = session(2)
new_client.receive(server.accept_connect(new_client.create_connect(2), 2), 2)
self.assertEqual(names(server.receive(send(new_client, "fresh"), 3)), ["fresh"])
def test_a_retransmitted_connect_after_data_has_flowed_does_not_reset_the_session(self) -> None:
client, server = session(1), session(500)
connect, _ = handshake(client, server)
frame = send(client, "once")
self.assertEqual(names(server.receive(frame, 1)), ["once"])
self.assertTrue(server.connect_repeats(connect))
self.assertEqual(server.accept_connect(connect, 2).packet_type, CultNetRudpPacketType.ACK)
self.assertEqual(server.outstanding_reliable_packet_count, 0, "a repeat queued something")
self.assertEqual(
names(server.receive(frame, 3)), [], "a retransmitted Connect made the server forget what it had delivered"
)
def test_a_connect_with_another_sequence_is_not_a_repeat_whatever_it_carries(self) -> None:
server = session(500)
connect = session(1).create_connect(0, b"same payload")
server.accept_connect(connect, 0)
self.assertTrue(server.connect_repeats(connect))
self.assertFalse(server.connect_repeats(session(9).create_connect(0, b"same payload")))
self.assertFalse(server.connect_repeats(replace(connect, packet_type=CultNetRudpPacketType.PING)))
server.create_disconnect()
self.assertFalse(server.connect_repeats(connect), "an ended session repeats nothing")
def test_an_accept_that_does_not_name_the_pending_connect_is_ignored(self) -> None:
client, server = session(10), session(500)
connect = client.create_connect(0)
accept = server.accept_connect(connect, 0)
client.receive(replace(accept, ack=connect.sequence + 50, ack_mask=0), 1)
self.assertFalse(client.connected, "a stale Accept connected the session")
self.assertEqual(client.pending_reliable_sequences, (connect.sequence,))
client.receive(replace(accept, ack=connect.sequence + 3, ack_mask=1 << 2), 2)
self.assertTrue(client.connected, "the mask names the Connect")
def test_an_accept_after_a_local_disconnect_does_not_reconnect_the_session(self) -> None:
client, server = session(1), session(500)
_, accept = handshake(client, server)
client.create_disconnect(b"bye")
self.assertFalse(client.connected)
client.receive(accept, 5)
self.assertFalse(client.connected, "a late Accept revived an ended session")
def test_a_reconnect_forgets_what_the_old_server_sent(self) -> None:
client, old_server = session(1), session(500)
handshake(client, old_server)
for name in ("d1", "d2", "d3"):
self.assertEqual(names(client.receive(send(old_server, name), 1)), [name])
new_server = session(501)
client.receive(new_server.accept_connect(client.create_connect(2), 2), 2)
self.assertEqual(names(client.receive(send(new_server, "fresh"), 3)), ["fresh"])
def test_an_accept_for_an_abandoned_connect_does_not_connect_the_session(self) -> None:
client, server = session(1), session(500)
accept = server.accept_connect(client.create_connect(0), 0)
client.create_disconnect(b"never mind")
client.receive(accept, 1)
self.assertFalse(client.connected, "an Accept revived an abandoned Connect")
def test_a_full_queue_owed_to_a_vanished_peer_does_not_refuse_a_new_generation(self) -> None:
def bounded(initial_sequence: int) -> CultNetRudpSession:
return CultNetRudpSession(
CultNetRudpSessionOptions(
connection_id=CONNECTION_ID, initial_sequence=initial_sequence, max_pending_reliable_packets=1
)
)
server = bounded(500)
server.accept_connect(session(1).create_connect(0), 0)
self.assertEqual(server.outstanding_reliable_packet_count, 1, "the Accept is owed and never acknowledged")
following = session(9).create_connect(1)
server.accept_connect(following, 1)
self.assertTrue(server.connect_repeats(following))
client = bounded(1)
client.create_connect(0)
client.create_connect(1)
self.assertEqual(client.outstanding_reliable_packet_count, 1, "only the new Connect is owed")
def test_a_duplicate_accept_does_not_reseed_the_watermark(self) -> None:
client, server = session(1), session(500)
_, accept = handshake(client, server)
s1, s2 = send(server, "s1"), send(server, "s2")
client.receive(replace(accept, sequence=s2.sequence), 1)
self.assertEqual(names(client.receive(s2, 2)), [], "s1 is still missing")
self.assertEqual(names(client.receive(s1, 3)), ["s1", "s2"])
def test_reliable_data_before_the_accept_is_neither_delivered_nor_acknowledged(self) -> None:
client, server = session(1), session(500)
accept = server.accept_connect(client.create_connect(0), 0)
a, b = send(server, "A"), send(server, "B")
for early in (b, a):
self.assertEqual(names(client.receive(early, 1)), [])
ack = client.create_ack_for_received(early.sequence)
self.assertEqual((ack.ack, ack.ack_mask), (0, 0), "an unhandled packet was acknowledged")
client.receive(accept, 2)
self.assertTrue(client.connected)
self.assertEqual(server.outstanding_reliable_packet_count, 3, "nothing was acknowledged")
self.assertEqual(names(client.receive(a, 3)), ["A"])
self.assertEqual(names(client.receive(b, 3)), ["B"])
def test_ordered_frames_after_the_accept_are_delivered_in_order_whatever_arrives_first(self) -> None:
client, server = session(1), session(500)
handshake(client, server)
a, b = send(server, "A"), send(server, "B")
self.assertEqual(names(client.receive(b, 1)), [])
self.assertEqual(names(client.receive(a, 2)), ["A", "B"])
def test_an_unordered_reliable_frame_is_delivered_on_receipt_never_held_behind_a_gap(self) -> None:
client, server = session(1), session(500)
handshake(client, server)
send(server, "lost")
unordered = send(server, "now", ordered=False, channel="media")
ordered = send(server, "later")
self.assertEqual(names(client.receive(unordered, 1)), ["now"])
self.assertEqual(names(client.receive(ordered, 2)), [], "the ordered frame waits for the gap")
def test_a_peer_disconnect_does_not_forget_what_the_peer_sent(self) -> None:
client, server = session(1), session(500)
handshake(client, server)
s1, s2 = send(client, "s1"), send(client, "s2")
self.assertEqual(names(server.receive(s2, 1)), [], "s1 is missing, so s2 is held")
server.receive(client.create_disconnect(b"bye"), 2)
self.assertEqual(names(server.receive(s1, 3)), ["s1", "s2"], "the Disconnect forgot the held frame")
self.assertEqual(names(server.receive(s2, 4)), [], "the Disconnect forgot what was received")
def test_sessions_from_one_options_object_each_draw_their_own_initial_sequence(self) -> None:
self.assertIsNone(CultNetRudpSessionOptions(connection_id=1).initial_sequence)
self.assertIsNone(
CultNetRudpSocketTransportOptions(
runtime_id="r", socket=None, mode=CultNetRudpSocketMode.CLIENT, connection_id=1
).initial_sequence
)
options = CultNetRudpSessionOptions(connection_id=1)
firsts = {CultNetRudpSession(options).create_connect(0).sequence for _ in range(64)}
self.assertGreater(len(firsts), 32, "a default session does not draw its own sequence")
self.assertTrue(all(1 <= value < 2**31 for value in firsts), "a draw left [1, 2^31)")
# Ack Cut 1d: a frame from an earlier generation is a duplicate, a delayed
# Connect from an earlier attempt is stale, a Connect from a new endpoint is
# a new client.
@staticmethod
def names_sequence(ack, sequence: int) -> bool:
return ack.ack == sequence or any(
ack.ack_mask & (1 << bit) and ack.ack > bit and ack.ack - bit - 1 == sequence for bit in range(32)
)
def test_a_frame_from_the_clients_earlier_generation_is_acknowledged_not_delivered_again(self) -> None:
client, server = session(10), session(500)
handshake(client, server)
old = send(client, "old")
self.assertEqual(names(server.receive(old, 1)), ["old"])
connect = client.create_connect(2)
client.receive(server.accept_connect(connect, 2), 2)
self.assertLess(old.sequence, connect.sequence)
self.assertEqual(names(server.receive(old, 3)), [], "an earlier generation's frame was delivered again")
self.assertTrue(self.names_sequence(server.create_ack_for_received(old.sequence), old.sequence))
self.assertEqual(names(server.receive(send(client, "fresh"), 4)), ["fresh"])
def test_a_frame_from_the_servers_earlier_generation_is_acknowledged_not_delivered_again(self) -> None:
client, server = session(10), session(500)
handshake(client, server)
old = send(server, "old")
self.assertEqual(names(client.receive(old, 1)), ["old"])
client.receive(server.accept_connect(client.create_connect(2), 2), 2)
self.assertEqual(names(client.receive(old, 3)), [], "an earlier generation's frame was delivered again")
self.assertTrue(self.names_sequence(client.create_ack_for_received(old.sequence), old.sequence))
self.assertEqual(names(client.receive(send(server, "fresh"), 4)), ["fresh"])
def test_a_duplicate_below_the_receive_window_is_acknowledged_by_name(self) -> None:
client, server = session(10), session(500)
handshake(client, server)
first = send(client, "first")
self.assertEqual(names(server.receive(first, 1)), ["first"])
for _ in range(4_200):
server.receive(send(client, "x"), 1)
client.receive(server.create_ack(), 1)
self.assertEqual(names(server.receive(first, 2)), [])
self.assertEqual(
server.create_ack_for_received(first.sequence).ack,
first.sequence,
"a duplicate below the window was not acknowledged by name",
)
def test_a_delayed_connect_from_an_earlier_attempt_does_not_strand_the_client(self) -> None:
client, server = session(10), session(500)
earlier = client.create_connect(0)
retried = client.create_connect(300)
accept = server.accept_connect(retried, 301)
client.receive(accept, 302)
server.receive(client.create_ack_for_received(accept.sequence), 302)
self.assertTrue(client.connected)
a = send(client, "a")
self.assertEqual(names(server.receive(a, 303)), ["a"])
client.receive(server.create_ack_for_received(a.sequence), 303)
self.assertTrue(server.connect_repeats(earlier), "the delayed Connect starts nothing")
reply = server.accept_connect(earlier, 304)
self.assertEqual(reply.packet_type, CultNetRudpPacketType.ACK)
client.receive(reply, 305)
self.assertTrue(client.connected)
b = send(client, "b")
self.assertEqual(names(server.receive(b, 306)), ["b"], "the delayed Connect reset the server")
client.receive(server.create_ack_for_received(b.sequence), 307)
self.assertNotIn(b.sequence, client.pending_reliable_sequences)
def test_a_repeated_connect_refreshes_liveness_and_a_stale_one_does_not(self) -> None:
client, server = session(10), session(500)
earlier = client.create_connect(0)
current = client.create_connect(1)
server.accept_connect(current, 0)
server.accept_connect(current, 900)
self.assertFalse(server.check_timeout(1_000, 500), "a repeated Connect did not refresh liveness")
server.accept_connect(earlier, 1_400)
self.assertTrue(server.check_timeout(1_600, 500), "a stale Connect refreshed liveness")
def test_stale_connects_are_recognised_by_serial_arithmetic(self) -> None:
server = session(500)
server.accept_connect(session(3).create_connect(0), 0)
def connect(sequence: int):
return session(sequence).create_connect(0)
self.assertTrue(server.connect_repeats(connect(2**32 - 2)), "just before the wrap")
self.assertFalse(server.connect_repeats(connect(4)), "just after")
self.assertFalse(server.connect_repeats(connect(3 + 4_096)), "far after")
self.assertTrue(server.connect_repeats(connect(2**32 - 4_092)), "the window's edge")
self.assertFalse(server.connect_repeats(connect(2**32 - 4_093)), "past the window")
def test_a_restarted_client_whose_first_sequence_is_stale_connects_after_redrawing(self) -> None:
server, old = session(500), session(1_000)
old_accept = server.accept_connect(old.create_connect(0), 0)
old.receive(old_accept, 0)
restarted = session(900)
first = restarted.create_connect(0, b"join")
# The old client has not acknowledged its Accept yet, so that is the
# reply: it names the old Connect, not this one.
early = server.accept_connect(first, 1)
self.assertEqual(early.packet_type, CultNetRudpPacketType.ACCEPT)
restarted.receive(early, 1)
self.assertFalse(restarted.connected, "an Accept for another Connect connected the client")
server.receive(old.create_ack_for_received(old_accept.sequence), 1)
reply = server.accept_connect(first, 2)
self.assertEqual(reply.packet_type, CultNetRudpPacketType.ACK)
restarted.receive(reply, 2)
self.assertFalse(restarted.connected)
retransmitted = restarted.due_resends(1_000)
self.assertEqual(len(retransmitted), 1)
self.assertEqual(retransmitted[0].sequence, first.sequence, "the attempt is still young")
fresh = restarted.due_resends(3_500)
self.assertEqual(len(fresh), 1)
self.assertEqual(fresh[0].packet_type, CultNetRudpPacketType.CONNECT)
self.assertNotEqual(fresh[0].sequence, first.sequence, "the same Connect was retransmitted for ever")
self.assertEqual(fresh[0].payload, b"join", "the fresh attempt lost the Connect payload")
accept = server.accept_connect(fresh[0], 3_500)
self.assertEqual(accept.packet_type, CultNetRudpPacketType.ACCEPT)
restarted.receive(accept, 3_500)
self.assertTrue(restarted.connected)
self.assertEqual(names(server.receive(send(restarted, "hello"), 3_501)), ["hello"])
def test_an_answered_connect_is_never_replaced_by_the_attempt_timeout(self) -> None:
client, server = session(10), session(500)
connect, _ = handshake(client, server)
resent = client.due_resends(60_000)
self.assertTrue(
all(p.packet_type != CultNetRudpPacketType.CONNECT or p.sequence == connect.sequence for p in resent),
"a connected client started a new attempt",
)
self.assertTrue(client.connected)
def test_an_accepted_peer_that_goes_silent_times_out(self) -> None:
server = session(500)
server.accept_connect(session(10).create_connect(0), 0)
self.assertTrue(server.check_timeout(100_000, 1_000))
# Ack Cut 1d, batch 2: the connect-attempt timeout.
def test_an_ack_naming_the_pending_connect_does_not_retire_it(self) -> None:
server, old = session(500), session(1)
old_accept = server.accept_connect(old.create_connect(0), 0)
old.receive(old_accept, 0)
server.receive(old.create_ack_for_received(old_accept.sequence), 0)
restarted = session(1)
first = restarted.create_connect(0, b"join")
reply = server.accept_connect(first, 1)
self.assertEqual(reply.packet_type, CultNetRudpPacketType.ACK)
self.assertTrue(self.names_sequence(reply, first.sequence), "the Ack must name the pending Connect for this test to bite")
restarted.receive(reply, 1)
self.assertFalse(restarted.connected)
self.assertEqual(restarted.pending_reliable_sequences, (first.sequence,), "an Ack retired the Connect")
fresh = restarted.due_resends(3_000)
self.assertEqual(len(fresh), 1, "the client waits for ever with nothing to resend")
self.assertEqual(fresh[0].payload, b"join")
accept = server.accept_connect(fresh[0], 3_000)
self.assertEqual(accept.packet_type, CultNetRudpPacketType.ACCEPT)
restarted.receive(accept, 3_000)
self.assertTrue(restarted.connected)
def test_a_late_copy_of_an_abandoned_connect_is_stale_once_its_replacement_is_accepted(self) -> None:
server, client = session(500), session(10)
abandoned = client.create_connect(0)
fresh = client.due_resends(3_000)
self.assertEqual(len(fresh), 1)
self.assertEqual(fresh[0].sequence, abandoned.sequence + 4_095)
accept = server.accept_connect(fresh[0], 3_001)
client.receive(accept, 3_002)
server.receive(client.create_ack_for_received(accept.sequence), 3_002)
self.assertTrue(client.connected)
a = send(client, "a")
self.assertEqual(names(server.receive(a, 3_003)), ["a"])
client.receive(server.create_ack_for_received(a.sequence), 3_003)
self.assertTrue(server.connect_repeats(abandoned), "the late copy would restart the server")
reply = server.accept_connect(abandoned, 3_004)
self.assertEqual(reply.packet_type, CultNetRudpPacketType.ACK)
client.receive(reply, 3_005)
self.assertTrue(client.connected)
self.assertEqual(names(server.receive(send(client, "b"), 3_006)), ["b"])
def test_a_server_sitting_at_the_jump_is_left_by_the_next_attempt(self) -> None:
server, old = session(500), session(5_095)
old_accept = server.accept_connect(old.create_connect(0), 0)
old.receive(old_accept, 0)
server.receive(old.create_ack_for_received(old_accept.sequence), 0)
restarted = session(1_000)
first = restarted.create_connect(0, b"join")
restarted.receive(server.accept_connect(first, 1), 1)
self.assertFalse(restarted.connected)
second = restarted.due_resends(3_000)[0]
self.assertEqual(second.sequence, 5_095, "the jump lands on the server's generation")
reply = server.accept_connect(second, 3_001)
self.assertEqual(reply.packet_type, CultNetRudpPacketType.ACK)
restarted.receive(reply, 3_001)
self.assertFalse(restarted.connected)
third = restarted.due_resends(6_000)[0]
self.assertEqual(third.sequence, 5_095 + 4_095)
accept = server.accept_connect(third, 6_001)
self.assertEqual(accept.packet_type, CultNetRudpPacketType.ACCEPT)
restarted.receive(accept, 6_001)
self.assertTrue(restarted.connected)
if __name__ == "__main__":
unittest.main()

115
packages/cultnet-py/tests/test_rudp_ordered_delivery.py Normal file
View file

@ -0,0 +1,115 @@
"""Ordered delivery has one owner: the contiguous received-through watermark.
A frame is held for exactly one reason, a reliable sequence below it has not
arrived, and it is delivered in the call that fills the last such gap,
whichever channel the gap belonged to.
"""
from __future__ import annotations
import unittest
from cultnet_py.transport import (
CultNetRudpPacket,
CultNetRudpSendOptions,
CultNetRudpSession,
CultNetRudpSessionOptions,
)
def handshake() -> tuple[CultNetRudpSession, CultNetRudpSession]:
client = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=410, initial_sequence=1))
server = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=410, initial_sequence=500))
accept = server.accept_connect(client.create_connect(0), 0)
client.receive(accept, 0)
return client, server
def send(session: CultNetRudpSession, channel: str, payload: str, ordered: bool) -> CultNetRudpPacket:
return session.send(channel, payload.encode(), CultNetRudpSendOptions(reliable=True, ordered=ordered))
def names(result) -> list[str]:
return [frame.payload.decode() for frame in result.delivered]
class CultNetRudpOrderedDeliveryTests(unittest.TestCase):
def test_ordered_frame_waits_for_a_gap_filled_by_another_channel(self) -> None:
sender, receiver = handshake()
o1 = send(sender, "schema", "o1", True)
u = send(sender, "rel", "u", False)
o2 = send(sender, "schema", "o2", True)
o3 = send(sender, "schema", "o3", True)
o4 = send(sender, "schema", "o4", True)
self.assertEqual(names(receiver.receive(o1, 1)), ["o1"])
self.assertEqual(names(receiver.receive(o2, 2)), [], "u is missing")
self.assertEqual(names(receiver.receive(o3, 3)), [], "u is missing")
self.assertEqual(names(receiver.receive(o4, 4)), [], "u is missing")
self.assertEqual(names(receiver.receive(u, 5)), ["u", "o2", "o3", "o4"])
def test_two_ordered_channels_release_each_other_in_sequence_order(self) -> None:
sender, receiver = handshake()
a1 = send(sender, "schema", "A1", True)
b1 = send(sender, "other", "B1", True)
a2 = send(sender, "schema", "A2", True)
a3 = send(sender, "schema", "A3", True)
delivered = names(receiver.receive(a1, 1)) + names(receiver.receive(a2, 2)) + names(receiver.receive(a3, 3))
self.assertEqual(delivered, ["A1"], "B1 is missing, so A2 and A3 wait")
self.assertEqual(names(receiver.receive(b1, 4)), ["B1", "A2", "A3"])
def test_channel_first_used_after_other_traffic_loses_nothing(self) -> None:
sender, receiver = handshake()
c1 = send(sender, "late", "C1", True)
c2 = send(sender, "late", "C2", True)
x = send(sender, "schema", "X", True)
self.assertEqual(names(receiver.receive(x, 1)), [], "C1 is missing")
self.assertEqual(names(receiver.receive(c2, 2)), [], "C1 is missing")
self.assertEqual(names(receiver.receive(c1, 3)), ["C1", "C2", "X"])
def test_accept_seeds_the_watermark_of_the_connecting_side(self) -> None:
client, server = handshake()
s1 = send(server, "schema", "s1", True)
s2 = send(server, "schema", "s2", True)
self.assertEqual(names(client.receive(s2, 1)), [])
self.assertEqual(names(client.receive(s1, 2)), ["s1", "s2"])
def test_connect_seeds_the_watermark_of_the_accepting_side(self) -> None:
client, server = handshake()
first = send(client, "schema", "first", True)
second = send(client, "schema", "second", True)
self.assertEqual(names(server.receive(second, 1)), [])
self.assertEqual(names(server.receive(first, 2)), ["first", "second"])
def test_duplicate_of_a_held_frame_is_not_delivered_twice(self) -> None:
sender, receiver = handshake()
s1 = send(sender, "schema", "s1", True)
s2 = send(sender, "schema", "s2", True)
self.assertEqual(names(receiver.receive(s2, 1)), [])
self.assertEqual(names(receiver.receive(s2, 2)), [])
self.assertEqual(names(receiver.receive(s1, 3)), ["s1", "s2"])
self.assertEqual(names(receiver.receive(s2, 4)), [])
def test_reset_peer_state_forgets_held_frames_and_reseeds_the_watermark_from_the_next_handshake(self) -> None:
sender, receiver = handshake()
send(sender, "schema", "s1", True)
s2 = send(sender, "schema", "s2", True)
self.assertEqual(names(receiver.receive(s2, 1)), [])
receiver.reset_peer_state()
following = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=410, initial_sequence=100))
following.receive(receiver.accept_connect(following.create_connect(0), 2), 2)
d1 = send(following, "schema", "d1", True)
d2 = send(following, "schema", "d2", True)
# The held s2 died with the reset, and the watermark follows the new peer.
self.assertEqual(names(receiver.receive(d2, 3)), [])
self.assertEqual(names(receiver.receive(d1, 4)), ["d1", "d2"])
if __name__ == "__main__":
unittest.main()

223
packages/cultnet-py/tests/test_rudp_session_ending.py Normal file
View file

@ -0,0 +1,223 @@
"""A write belongs to the session generation it was issued in.
Every way a generation ends drops the writes it still owed, and a flush
belongs to the generation it started in.
"""
from __future__ import annotations
import socket
import threading
import time
import unittest
from cultnet_py.transport import (
CultNetRudpPacketType,
CultNetRudpSendOptions,
CultNetRudpSession,
CultNetRudpSessionOptions,
CultNetRudpSocketMode,
CultNetRudpSocketTransportConnection,
CultNetRudpSocketTransportOptions,
decode_rudp_packet,
encode_rudp_packet,
)
CONNECTION_ID = 0x10203070
def bind_udp_socket() -> socket.socket:
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind(("127.0.0.1", 0))
sock.settimeout(0.02)
return sock
def client_with_a_lost_write() -> tuple[CultNetRudpSession, CultNetRudpSession]:
client = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=CONNECTION_ID, initial_sequence=1))
server = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=CONNECTION_ID, initial_sequence=500))
client.receive(server.accept_connect(client.create_connect(0), 0), 0)
client.send("schema", b"owed to the old session", CultNetRudpSendOptions(reliable=True, ordered=True))
# A fragmented write larger than the send window leaves part of it queued.
client.send_many(
"schema", bytes(40 * 8), CultNetRudpSendOptions(reliable=True, ordered=True), max_fragment_bytes=8
)
assert client.outstanding_reliable_packet_count == 41
assert client.queued_reliable_packet_count > 0
return client, server
def drain(sock: socket.socket) -> list:
packets = []
while True:
try:
wire, _ = sock.recvfrom(65535)
except TimeoutError:
return packets
packets.append(decode_rudp_packet(wire))
def connected_client() -> tuple[CultNetRudpSocketTransportConnection, socket.socket, tuple]:
peer_socket = bind_udp_socket()
client = CultNetRudpSocketTransportConnection(
CultNetRudpSocketTransportOptions(
runtime_id="python-rudp-client",
socket=bind_udp_socket(),
mode=CultNetRudpSocketMode.CLIENT,
remote_addr=peer_socket.getsockname(),
connection_id=CONNECTION_ID,
)
)
client.connect(b"hello")
wire, client_addr = peer_socket.recvfrom(65535)
peer = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=CONNECTION_ID, initial_sequence=500))
accept = peer.accept_connect(decode_rudp_packet(wire), 0)
peer_socket.sendto(encode_rudp_packet(accept), client_addr)
client.receive_once()
assert client.connected
return client, peer_socket, client_addr
class CultNetRudpSessionEndingTests(unittest.TestCase):
def test_every_way_a_session_ends_drops_the_writes_it_owed(self) -> None:
endings = {
"peer Disconnect": lambda client, server: client.receive(server.create_disconnect(b"bye"), 1),
"local disconnect": lambda client, server: client.create_disconnect(b"bye"),
"timeout": lambda client, server: self.assertTrue(client.check_timeout(1_000, 10)),
"reset": lambda client, server: client.reset_peer_state(),
}
for name, end in endings.items():
with self.subTest(ending=name):
client, server = client_with_a_lost_write()
end(client, server)
self.assertEqual(client.outstanding_reliable_packet_count, 0, "the write survived the end")
next_server = CultNetRudpSession(
CultNetRudpSessionOptions(connection_id=CONNECTION_ID, initial_sequence=900)
)
next_server.accept_connect(client.create_connect(2_000), 2_000)
for resend in client.due_resends(60_000):
self.assertEqual(
next_server.receive(resend, 60_000).delivered,
(),
"the old session's write was delivered in the next session",
)
def test_a_flush_fails_after_a_local_disconnect_or_a_timeout(self) -> None:
client, peer_socket, _ = connected_client()
try:
client.send("schema", b"owed")
client.disconnect(b"bye")
with self.assertRaises(ConnectionError):
client.flush_reliable(0.2)
finally:
client.close()
peer_socket.close()
client, peer_socket, _ = connected_client()
try:
client.send("schema", b"owed")
time.sleep(0.005)
self.assertTrue(client.check_timeout(1))
with self.assertRaises(ConnectionError):
client.flush_reliable(0.2)
finally:
client.close()
peer_socket.close()
def test_a_flush_that_started_before_the_end_fails_even_after_a_later_connect(self) -> None:
client, peer_socket, _ = connected_client()
outcome: list[BaseException | None] = []
def flush() -> None:
try:
client.flush_reliable(5.0)
outcome.append(None)
except BaseException as error: # noqa: BLE001 - the outcome is the assertion
outcome.append(error)
try:
client.send("schema", b"owed")
worker = threading.Thread(target=flush)
worker.start()
time.sleep(0.05)
client.disconnect(b"bye")
client.connect(b"hello again")
worker.join(10)
self.assertEqual(len(outcome), 1)
self.assertIsInstance(outcome[0], ConnectionError)
finally:
client.close()
peer_socket.close()
def test_a_flush_started_after_a_reconnect_waits_on_the_new_session(self) -> None:
client, peer_socket, client_addr = connected_client()
try:
client.send("schema", b"owed to the old session")
client.disconnect(b"bye")
client.connect(b"hello again")
connect = [p for p in drain(peer_socket) if p.packet_type == CultNetRudpPacketType.CONNECT][-1]
accept = CultNetRudpSession(
CultNetRudpSessionOptions(connection_id=CONNECTION_ID, initial_sequence=700)
).accept_connect(connect, 0)
peer_socket.sendto(encode_rudp_packet(accept), client_addr)
client.receive_once()
client.flush_reliable(0.5)
finally:
client.close()
peer_socket.close()
def test_a_flush_started_after_the_server_accepts_a_new_connect_waits_on_the_new_session(self) -> None:
server_socket = bind_udp_socket()
peer_socket = bind_udp_socket()
server = CultNetRudpSocketTransportConnection(
CultNetRudpSocketTransportOptions(
runtime_id="python-rudp-server",
socket=server_socket,
mode=CultNetRudpSocketMode.SERVER,
connection_id=CONNECTION_ID,
)
)
to_server = lambda packet: peer_socket.sendto(encode_rudp_packet(packet), server_socket.getsockname())
try:
first = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=CONNECTION_ID))
to_server(first.create_connect(0))
server.receive_once()
to_server(first.create_disconnect())
server.receive_once()
following = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=CONNECTION_ID))
to_server(following.create_connect(0))
server.receive_once()
accept = [p for p in drain(peer_socket) if p.packet_type == CultNetRudpPacketType.ACCEPT][-1]
following.receive(accept, 0)
to_server(following.create_ack())
server.receive_once()
server.flush_reliable(0.5)
finally:
server.close()
peer_socket.close()
def test_an_ending_does_not_forget_what_was_received_from_the_peer(self) -> None:
client = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=CONNECTION_ID, initial_sequence=1))
server = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=CONNECTION_ID, initial_sequence=500))
client.receive(server.accept_connect(client.create_connect(0), 0), 0)
ordered = CultNetRudpSendOptions(reliable=True, ordered=True)
s1 = client.send("schema", b"s1", ordered)
s2 = client.send("schema", b"s2", ordered)
self.assertEqual(server.receive(s2, 1).delivered, (), "s1 is missing, so s2 is held")
self.assertTrue(server.check_timeout(1_000, 10))
self.assertEqual(
[frame.payload for frame in server.receive(s1, 1_001).delivered],
[b"s1", b"s2"],
"the ending forgot the held frame",
)
self.assertEqual(server.receive(s2, 1_002).delivered, (), "the ending forgot what was received")
if __name__ == "__main__":
unittest.main()

6
packages/cultnet-rs/Cargo.toml
View file

@ -19,6 +19,7 @@ cultcache-rs = { path = "../cultcache-rs" }
ed25519-dalek = "2.2"
hmac = "0.12"
rand = "0.9"
reed-solomon-erasure = "6"
rmp-serde = "1.3"
rmpv = { version = "1.3", features = ["with-serde"] }
serde = { version = "1", features = ["derive"] }
@ -35,3 +36,8 @@ windows-sys = { version = "0.61", features = ["Win32_Foundation", "Win32_Securit
[dev-dependencies]
pretty_assertions = "1.4"
tempfile = "3"
[[example]]
name = "cultnet_impair"
path = "examples/cultnet_impair/main.rs"
test = true

8
packages/cultnet-rs/contracts/cultnet/cultnet.error.schema.json
View file

@ -26,9 +26,9 @@
}
},
"code": {
"description": "Machine-readable refusal code (R-N). Additive: a v0 peer that does not know this field ignores it.",
"description": "Machine-readable refusal code (R-N). Additive: a v0 peer that does not know this field ignores it. The set is a closed enum: Rust and TypeScript refuse a message whose code is not listed, so adding a code is one coordinated change across every runtime (this schema, the C# factory, the Rust CultNetErrorCode, and a captured-bytes case for each).",
"type": ["string", "null"],
"enum": [null, "selection_invalid", "cursor_stale", "cursor_invalid", "reference_outside_target", "variant_unsupported"]
"enum": [null, "selection_invalid", "cursor_stale", "cursor_invalid", "reference_outside_target", "variant_unsupported", "unowned_schema"]
},
"details": {
"description": "Code-specific structured detail (R-N). Every field is populated only for the code that defines it; the others are null, not omitted.",
@ -36,8 +36,8 @@
"additionalProperties": false,
"required": ["field", "value", "asOf", "current"],
"properties": {
"field": { "description": "selection_invalid: the selection field that failed. variant_unsupported: \"subscriptionId\", when a subscription was ended.", "type": ["string", "null"] },
"value": { "description": "selection_invalid: the offending value, when there is one string worth naming. variant_unsupported: the id of the subscription that ended.", "type": ["string", "null"] },
"field": { "description": "selection_invalid: the selection field that failed. variant_unsupported: \"subscriptionId\", when a subscription was ended. unowned_schema: \"schemaId\".", "type": ["string", "null"] },
"value": { "description": "selection_invalid: the offending value, when there is one string worth naming. variant_unsupported: the id of the subscription that ended. unowned_schema: the schema id no shard owns.", "type": ["string", "null"] },
"asOf": { "description": "cursor_stale: the cursor's own minted asOf.", "type": ["integer", "null"], "minimum": 0 },
"current": { "description": "cursor_stale: the answering server's current asOf.", "type": ["integer", "null"], "minimum": 0 }
}

336
packages/cultnet-rs/examples/cultnet_impair/main.rs Normal file
View file

@ -0,0 +1,336 @@
//! `cultnet-impair`: a deterministic bidirectional UDP proxy for pressure-testing
//! real CultNet RUDP sessions. It owns transport impairment only; what a stream
//! does with the loss is the stream's business.
//!
//! The proxy listens at the endpoint a client dials and forwards to `--upstream`
//! through one upstream socket per client, so the reverse ACK and feedback
//! traffic crosses the same schedule. The schedule is `policy.rs`.
//!
//! ```text
//! cargo run --release --example cultnet_impair -- \
//! --listen 0.0.0.0:17890 --upstream 127.0.0.1:17990 \
//! --profile tests/impairment/loss-1pct.toml --seed 424242 --metrics loss-1pct.csv
//! ```
//!
//! Profiles are a small TOML subset of unsigned integers; see
//! `tests/impairment/`. The receiver binds the upstream endpoint and the client
//! dials the proxy's; never one port for both. For a two-host test, run the proxy
//! on the receiver host so reverse traffic is impaired too.
mod policy;
use std::collections::HashMap;
use std::env;
use std::fs::{self, File};
use std::io::{self, Write};
use std::net::{SocketAddr, UdpSocket};
use std::path::{Path, PathBuf};
use std::thread;
use std::time::{Duration, Instant};
use policy::{Direction, Profile, Scheduler, Stats};
const MAX_DATAGRAM: usize = 65_535;
const MAX_CLIENT_FLOWS: usize = 64;
const CLIENT_FLOW_IDLE_TIMEOUT_MS: u64 = 30_000;
const MAX_DATAGRAMS_PER_FLOW_PER_TURN: usize = 16;
#[derive(Debug)]
struct UpstreamFlow {
socket: UdpSocket,
last_activity_ms: u64,
}
fn open_upstream_flow(upstream: SocketAddr, now_ms: u64) -> io::Result<UpstreamFlow> {
let socket = UdpSocket::bind("0.0.0.0:0")?;
socket.connect(upstream)?;
socket.set_nonblocking(true)?;
Ok(UpstreamFlow {
socket,
last_activity_ms: now_ms,
})
}
fn ensure_upstream_flow(
flows: &mut HashMap<SocketAddr, UpstreamFlow>,
client: SocketAddr,
upstream: SocketAddr,
now_ms: u64,
stats: &mut Stats,
) -> io::Result<()> {
if flows.contains_key(&client) {
return Ok(());
}
if flows.len() >= MAX_CLIENT_FLOWS
&& let Some(oldest) = flows
.iter()
.min_by_key(|(_, flow)| flow.last_activity_ms)
.map(|(address, _)| *address)
{
flows.remove(&oldest);
stats.flow_evicted += 1;
}
flows.insert(client, open_upstream_flow(upstream, now_ms)?);
stats.max_flows = stats.max_flows.max(flows.len());
Ok(())
}
fn reap_idle_flows(
flows: &mut HashMap<SocketAddr, UpstreamFlow>,
now_ms: u64,
stats: &mut Stats,
) {
let before = flows.len();
flows.retain(|_, flow| {
now_ms.saturating_sub(flow.last_activity_ms) < CLIENT_FLOW_IDLE_TIMEOUT_MS
});
stats.flow_evicted += (before - flows.len()) as u64;
}
#[derive(Debug)]
struct Options {
listen: SocketAddr,
upstream: SocketAddr,
profile: PathBuf,
seed: u64,
metrics: Option<PathBuf>,
}
fn parse_options() -> Result<Options, String> {
let mut args = env::args().skip(1);
let mut listen = None;
let mut upstream = None;
let mut profile = None;
let mut seed = 1;
let mut metrics = None;
while let Some(arg) = args.next() {
let mut value = || args.next().ok_or_else(|| format!("{arg} requires a value"));
match arg.as_str() {
"--listen" => {
listen = Some(
value()?
.parse()
.map_err(|_| "--listen must be a socket address")?,
)
}
"--upstream" => {
upstream = Some(
value()?
.parse()
.map_err(|_| "--upstream must be a socket address")?,
)
}
"--profile" => profile = Some(PathBuf::from(value()?)),
"--seed" => {
seed = value()?
.parse()
.map_err(|_| "--seed must be an unsigned integer")?
}
"--metrics" => metrics = Some(PathBuf::from(value()?)),
"--help" | "-h" => return Err(usage().into()),
_ => return Err(format!("unknown argument {arg}\n{}", usage())),
}
}
Ok(Options {
listen: listen.ok_or("--listen is required")?,
upstream: upstream.ok_or("--upstream is required")?,
profile: profile.ok_or("--profile is required")?,
seed,
metrics,
})
}
fn usage() -> &'static str {
"usage: cultnet-impair --listen HOST:PORT --upstream HOST:PORT --profile PATH [--seed N] [--metrics PATH]"
}
fn elapsed_ms(start: Instant) -> u64 {
start.elapsed().as_millis().try_into().unwrap_or(u64::MAX)
}
fn write_metrics(path: &Path, stats: &Stats, active_flows: usize) -> io::Result<()> {
if let Some(parent) = path.parent() {
fs::create_dir_all(parent)?;
}
let mut file = File::create(path)?;
writeln!(
file,
"received,forwarded,dropped,duplicated,reordered,stalled,queue_overflow,max_queue,flow_evicted,max_flows,active_flows"
)?;
writeln!(
file,
"{},{},{},{},{},{},{},{},{},{},{}",
stats.received,
stats.forwarded,
stats.dropped,
stats.duplicated,
stats.reordered,
stats.stalled,
stats.queue_overflow,
stats.max_queue,
stats.flow_evicted,
stats.max_flows,
active_flows
)
}
fn run(options: Options) -> Result<(), String> {
let profile = Profile::load(&options.profile)?;
let client_socket = UdpSocket::bind(options.listen)
.map_err(|error| format!("binding {}: {error}", options.listen))?;
client_socket
.set_nonblocking(true)
.map_err(|error| error.to_string())?;
println!(
"cultnet-impair listen={} upstream={} seed={} profile={}",
options.listen,
options.upstream,
options.seed,
options.profile.display()
);
let start = Instant::now();
let mut scheduler = Scheduler::new(profile, options.seed);
let mut flows = HashMap::<SocketAddr, UpstreamFlow>::new();
let mut last_flow_reap_at = 0_u64;
let mut buffer = vec![0; MAX_DATAGRAM];
loop {
let now = elapsed_ms(start);
let mut did_work = false;
for _ in 0..MAX_DATAGRAMS_PER_FLOW_PER_TURN {
match client_socket.recv_from(&mut buffer) {
Ok((size, address)) => {
did_work = true;
ensure_upstream_flow(
&mut flows,
address,
options.upstream,
now,
&mut scheduler.stats,
)
.map_err(|error| format!("opening upstream flow for {address}: {error}"))?;
if let Some(flow) = flows.get_mut(&address) {
flow.last_activity_ms = now;
}
scheduler.admit(
now,
Direction::ClientToUpstream,
&buffer[..size],
address,
);
}
Err(error) if error.kind() == io::ErrorKind::WouldBlock => break,
Err(error) => return Err(error.to_string()),
}
}
for (address, flow) in &mut flows {
for _ in 0..MAX_DATAGRAMS_PER_FLOW_PER_TURN {
match flow.socket.recv(&mut buffer) {
Ok(size) => {
did_work = true;
flow.last_activity_ms = now;
scheduler.admit(
now,
Direction::UpstreamToClient,
&buffer[..size],
*address,
);
}
Err(error) if error.kind() == io::ErrorKind::WouldBlock => break,
Err(error) => return Err(error.to_string()),
}
}
}
while let Some(item) = scheduler.pop_due(now) {
did_work = true;
let result = match item.direction {
Direction::ClientToUpstream => match flows.get_mut(&item.client) {
Some(flow) => {
flow.last_activity_ms = now;
flow.socket.send(&item.bytes)
}
None => {
scheduler.stats.dropped += 1;
continue;
}
},
Direction::UpstreamToClient => client_socket.send_to(&item.bytes, item.client),
};
if result.is_ok() {
scheduler.stats.forwarded += 1;
}
}
if let Some(path) = options.metrics.as_deref() {
if now % 1000 == 0 {
let _ = write_metrics(path, &scheduler.stats, flows.len());
}
}
if now.saturating_sub(last_flow_reap_at) >= 1_000 {
reap_idle_flows(&mut flows, now, &mut scheduler.stats);
last_flow_reap_at = now;
}
if did_work {
thread::yield_now();
} else {
thread::sleep(Duration::from_millis(1));
}
}
}
fn main() {
match parse_options().and_then(run) {
Ok(()) => {}
Err(error) => {
eprintln!("{error}");
std::process::exit(2);
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn independent_clients_keep_independent_upstream_udp_identities() {
let mut flows = HashMap::new();
let mut stats = Stats::default();
let upstream = "127.0.0.1:54321".parse().unwrap();
let first = "127.0.0.1:12001".parse().unwrap();
let second = "127.0.0.1:12002".parse().unwrap();
ensure_upstream_flow(&mut flows, first, upstream, 1, &mut stats).unwrap();
ensure_upstream_flow(&mut flows, second, upstream, 2, &mut stats).unwrap();
assert_eq!(flows.len(), 2);
assert_ne!(
flows[&first].socket.local_addr().unwrap(),
flows[&second].socket.local_addr().unwrap()
);
assert_eq!(stats.max_flows, 2);
assert_eq!(stats.flow_evicted, 0);
}
#[test]
fn idle_client_flows_are_reaped_without_touching_active_flows() {
let mut flows = HashMap::new();
let mut stats = Stats::default();
let upstream = "127.0.0.1:54321".parse().unwrap();
let idle = "127.0.0.1:12001".parse().unwrap();
let active = "127.0.0.1:12002".parse().unwrap();
ensure_upstream_flow(&mut flows, idle, upstream, 0, &mut stats).unwrap();
ensure_upstream_flow(
&mut flows,
active,
upstream,
CLIENT_FLOW_IDLE_TIMEOUT_MS - 1,
&mut stats,
)
.unwrap();
reap_idle_flows(&mut flows, CLIENT_FLOW_IDLE_TIMEOUT_MS, &mut stats);
assert!(!flows.contains_key(&idle));
assert!(flows.contains_key(&active));
assert_eq!(stats.flow_evicted, 1);
}
}

298
packages/cultnet-rs/examples/cultnet_impair/policy.rs Normal file
View file

@ -0,0 +1,298 @@
//! The impairment policy: what happens to each datagram, and nothing about
//! sockets. `Profile` says what to do, `Scheduler` decides per datagram and
//! holds the delayed ones, and every decision is a pure function of the
//! profile, the seed and the datagram's position, so a schedule replays
//! exactly. The proxy in `main.rs` and the loss-matrix test in
//! `tests/media_fec_loss.rs` both take their schedule from here.
use std::collections::VecDeque;
use std::fs;
use std::net::SocketAddr;
use std::path::Path;
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct Profile {
pub loss_basis_points: u32,
pub burst_every: u64,
pub burst_length: u64,
pub duplicate_every: u64,
pub reorder_every: u64,
pub reorder_delay_ms: u64,
pub delay_ms: u64,
pub jitter_ms: u64,
pub stall_at_ms: u64,
pub stall_for_ms: u64,
pub max_scheduled: usize,
}
impl Default for Profile {
fn default() -> Self {
Self {
loss_basis_points: 0,
burst_every: 0,
burst_length: 0,
duplicate_every: 0,
reorder_every: 0,
reorder_delay_ms: 0,
delay_ms: 0,
jitter_ms: 0,
stall_at_ms: 0,
stall_for_ms: 0,
max_scheduled: 4096,
}
}
}
impl Profile {
pub fn load(path: &Path) -> Result<Self, String> {
let text = fs::read_to_string(path)
.map_err(|error| format!("reading {}: {error}", path.display()))?;
let mut profile = Self::default();
for (index, raw) in text.lines().enumerate() {
let line = raw.split('#').next().unwrap_or("").trim();
if line.is_empty() || line.starts_with('[') {
continue;
}
let (key, value) = line
.split_once('=')
.ok_or_else(|| format!("{}:{}: expected key = value", path.display(), index + 1))?;
let key = key.trim();
let value = value.trim().trim_matches('"');
let number = value.parse::<u64>().map_err(|_| {
format!(
"{}:{}: {key} must be an unsigned integer",
path.display(),
index + 1
)
})?;
match key {
"loss_basis_points" => {
profile.loss_basis_points = number
.try_into()
.map_err(|_| "loss_basis_points is too large")?
}
"burst_every" => profile.burst_every = number,
"burst_length" => profile.burst_length = number,
"duplicate_every" => profile.duplicate_every = number,
"reorder_every" => profile.reorder_every = number,
"reorder_delay_ms" => profile.reorder_delay_ms = number,
"delay_ms" => profile.delay_ms = number,
"jitter_ms" => profile.jitter_ms = number,
"stall_at_ms" => profile.stall_at_ms = number,
"stall_for_ms" => profile.stall_for_ms = number,
"max_scheduled" => {
profile.max_scheduled = number
.try_into()
.map_err(|_| "max_scheduled is too large")?
}
_ => {
return Err(format!(
"{}:{}: unknown profile key {key}",
path.display(),
index + 1
));
}
}
}
if profile.loss_basis_points > 10_000 {
return Err("loss_basis_points must be <= 10000".into());
}
if profile.max_scheduled == 0 {
return Err("max_scheduled must be greater than zero".into());
}
Ok(profile)
}
}
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Direction {
ClientToUpstream,
UpstreamToClient,
}
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct Decision {
pub drop: bool,
pub copies: u8,
pub delay_ms: u64,
pub reordered: bool,
pub stalled: bool,
}
#[derive(Clone, Debug)]
pub struct DeterministicRng(u64);
impl DeterministicRng {
pub fn new(seed: u64) -> Self {
Self(if seed == 0 { 0x9e3779b97f4a7c15 } else { seed })
}
pub fn next(&mut self) -> u64 {
let mut x = self.0;
x ^= x << 13;
x ^= x >> 7;
x ^= x << 17;
self.0 = x;
x
}
}
#[derive(Debug)]
pub struct Scheduler {
pub profile: Profile,
pub rng: DeterministicRng,
pub seen: u64,
pub burst_remaining: u64,
pub queue: VecDeque<Scheduled>,
pub stats: Stats,
}
#[derive(Debug)]
pub struct Scheduled {
pub due_ms: u64,
pub direction: Direction,
pub bytes: Vec<u8>,
pub client: SocketAddr,
}
#[derive(Debug, Default)]
pub struct Stats {
pub received: u64,
pub forwarded: u64,
pub dropped: u64,
pub duplicated: u64,
pub reordered: u64,
pub stalled: u64,
pub queue_overflow: u64,
pub max_queue: usize,
pub flow_evicted: u64,
pub max_flows: usize,
}
impl Scheduler {
pub fn new(profile: Profile, seed: u64) -> Self {
Self {
profile,
rng: DeterministicRng::new(seed),
seen: 0,
burst_remaining: 0,
queue: VecDeque::new(),
stats: Stats::default(),
}
}
pub fn decide(&mut self, now_ms: u64) -> Decision {
self.seen += 1;
let stalled = self.profile.stall_for_ms > 0
&& now_ms >= self.profile.stall_at_ms
&& now_ms
< self
.profile
.stall_at_ms
.saturating_add(self.profile.stall_for_ms);
if stalled {
return Decision {
drop: true,
copies: 0,
delay_ms: 0,
reordered: false,
stalled: true,
};
}
if self.burst_remaining == 0
&& self.profile.burst_every > 0
&& self.seen % self.profile.burst_every == 0
{
self.burst_remaining = self.profile.burst_length;
}
if self.burst_remaining > 0 {
self.burst_remaining -= 1;
return Decision {
drop: true,
copies: 0,
delay_ms: 0,
reordered: false,
stalled: false,
};
}
if self.profile.loss_basis_points > 0
&& self.rng.next() % 10_000 < u64::from(self.profile.loss_basis_points)
{
return Decision {
drop: true,
copies: 0,
delay_ms: 0,
reordered: false,
stalled: false,
};
}
let duplicate =
self.profile.duplicate_every > 0 && self.seen % self.profile.duplicate_every == 0;
let reordered =
self.profile.reorder_every > 0 && self.seen % self.profile.reorder_every == 0;
let jitter = if self.profile.jitter_ms == 0 {
0
} else {
self.rng.next() % (self.profile.jitter_ms + 1)
};
Decision {
drop: false,
copies: if duplicate { 2 } else { 1 },
delay_ms: self.profile.delay_ms
+ jitter
+ if reordered {
self.profile.reorder_delay_ms
} else {
0
},
reordered,
stalled: false,
}
}
pub fn admit(&mut self, now_ms: u64, direction: Direction, bytes: &[u8], client: SocketAddr) {
self.stats.received += 1;
let decision = self.decide(now_ms);
if decision.drop {
self.stats.dropped += 1;
if decision.stalled {
self.stats.stalled += 1;
}
return;
}
if decision.copies == 2 {
self.stats.duplicated += 1;
}
if decision.reordered {
self.stats.reordered += 1;
}
for copy in 0..decision.copies {
if self.queue.len() >= self.profile.max_scheduled {
self.stats.queue_overflow += 1;
self.stats.dropped += 1;
continue;
}
let delay = decision.delay_ms + u64::from(copy);
let item = Scheduled {
due_ms: now_ms.saturating_add(delay),
direction,
bytes: bytes.to_vec(),
client,
};
let position = self
.queue
.iter()
.position(|queued| queued.due_ms > item.due_ms)
.unwrap_or(self.queue.len());
self.queue.insert(position, item);
self.stats.max_queue = self.stats.max_queue.max(self.queue.len());
}
}
pub fn pop_due(&mut self, now_ms: u64) -> Option<Scheduled> {
if self.queue.front().is_some_and(|item| item.due_ms <= now_ms) {
self.queue.pop_front()
} else {
None
}
}
}

6
packages/cultnet-rs/examples/cultnet_interop_peer.rs
View file

@ -507,7 +507,7 @@ fn rudp_serve_once(options: &BTreeMap<String, String>) -> Result<()> {
let local_port = socket.local_addr()?.port();
let mut transport_options =
CultNetRudpSocketTransportOptions::server("rust-rudp-interop", socket, 0x22446688);
transport_options.initial_sequence = 100;
transport_options.initial_sequence = Some(100);
transport_options.resend_delay_ms = 25;
transport_options.max_fragment_bytes = max_fragment_bytes;
let mut transport = CultNetRudpSocketTransportConnection::new(transport_options)?;
@ -608,7 +608,7 @@ fn rudp_serve_message_once(options: &BTreeMap<String, String>) -> Result<()> {
let local_port = socket.local_addr()?.port();
let mut transport_options =
CultNetRudpSocketTransportOptions::server("rust-rudp-message-interop", socket, 0x22446689);
transport_options.initial_sequence = 100;
transport_options.initial_sequence = Some(100);
transport_options.resend_delay_ms = 25;
let mut transport = CultNetRudpSocketTransportConnection::new(transport_options)?;
@ -967,7 +967,7 @@ fn run_rudp_server(
let peer = peers.entry(remote_addr).or_insert_with(|| RudpPeerSession {
session: CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: RUDP_INTEROP_CONNECTION_ID,
initial_sequence: 100,
initial_sequence: Some(100),
resend_delay_ms: RUDP_INTEROP_RESEND_DELAY_MS,
max_pending_reliable_packets: None,
}),

7
packages/cultnet-rs/src/contracts.rs
View file

@ -250,8 +250,8 @@ impl CultNetShardDescriptor {
}
}
/// Machine-readable `cultnet.error.v0` refusal code (R-N). Mirrors the five strings
/// `contracts/cultnet/cultnet.error.schema.json` enumerates for `code`; a peer that receives a fifth
/// Machine-readable `cultnet.error.v0` refusal code (R-N). Mirrors the strings
/// `contracts/cultnet/cultnet.error.schema.json` enumerates for `code`; a peer that receives a
/// string it does not know refuses to decode rather than guess a meaning for it.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
@ -261,6 +261,7 @@ pub enum CultNetErrorCode {
CursorInvalid,
ReferenceOutsideTarget,
VariantUnsupported,
UnownedSchema,
}
impl CultNetErrorCode {
@ -271,6 +272,7 @@ impl CultNetErrorCode {
Self::CursorInvalid => "cursor_invalid",
Self::ReferenceOutsideTarget => "reference_outside_target",
Self::VariantUnsupported => "variant_unsupported",
Self::UnownedSchema => "unowned_schema",
}
}
@ -281,6 +283,7 @@ impl CultNetErrorCode {
"cursor_invalid" => Ok(Self::CursorInvalid),
"reference_outside_target" => Ok(Self::ReferenceOutsideTarget),
"variant_unsupported" => Ok(Self::VariantUnsupported),
"unowned_schema" => Ok(Self::UnownedSchema),
other => Err(anyhow!("ErrorMessage.Code {other:?} is not a recognized cultnet.error.v0 code")),
}
}

41
packages/cultnet-rs/src/cultmesh.rs
View file

@ -3,6 +3,9 @@ use anyhow::anyhow;
use chrono::DateTime;
use chrono::Utc;
use std::collections::BTreeMap;
use std::net::IpAddr;
use std::net::Ipv4Addr;
use std::net::Ipv6Addr;
use std::net::SocketAddr;
use std::net::ToSocketAddrs;
use std::net::UdpSocket;
@ -240,10 +243,13 @@ impl CultNetRudpEndpoint {
#[derive(Clone, Debug)]
pub struct CultMeshRudpSocketOptions {
pub bind_host: String,
/// Local address to bind. Unset, a server binds loopback, and a client binds
/// loopback for a loopback endpoint and the unspecified address of the
/// endpoint's family otherwise, so it can reach a remote host.
pub bind_host: Option<String>,
pub bind_port: u16,
pub read_timeout: Option<Duration>,
pub initial_sequence: u32,
pub initial_sequence: Option<u32>,
pub resend_delay_ms: u64,
pub max_payload_bytes: Option<u32>,
pub max_fragment_bytes: Option<u32>,
@ -253,10 +259,10 @@ pub struct CultMeshRudpSocketOptions {
impl Default for CultMeshRudpSocketOptions {
fn default() -> Self {
Self {
bind_host: "127.0.0.1".to_string(),
bind_host: None,
bind_port: 0,
read_timeout: Some(Duration::from_millis(20)),
initial_sequence: 1,
initial_sequence: None,
resend_delay_ms: 250,
max_payload_bytes: None,
max_fragment_bytes: None,
@ -316,7 +322,8 @@ impl CultMesh {
connection_id: u32,
options: CultMeshRudpSocketOptions,
) -> Result<CultNetRudpSocketTransportConnection> {
let socket = bind_rudp_socket(&options)?;
let bind_host = options.bind_host.as_deref().unwrap_or("127.0.0.1");
let socket = bind_rudp_socket(bind_host, &options)?;
CultNetRudpSocketTransportConnection::new(CultNetRudpSocketTransportOptions {
media_reliable_expire_after_ms: None,
media_delivery: None,
@ -341,14 +348,19 @@ impl CultMesh {
endpoint: &CultNetRudpEndpoint,
options: CultMeshRudpSocketOptions,
) -> Result<CultNetRudpSocketTransportConnection> {
let socket = bind_rudp_socket(&options)?;
let remote_addr = endpoint.socket_addr()?;
let bind_host = match options.bind_host.as_deref() {
Some(host) => host.to_string(),
None => client_bind_ip(remote_addr).to_string(),
};
let socket = bind_rudp_socket(&bind_host, &options)?;
CultNetRudpSocketTransportConnection::new(CultNetRudpSocketTransportOptions {
media_reliable_expire_after_ms: None,
media_delivery: None,
runtime_id: runtime_id.into(),
socket,
mode: CultNetRudpSocketMode::Client,
remote_addr: Some(endpoint.socket_addr()?),
remote_addr: Some(remote_addr),
connection_id,
initial_sequence: options.initial_sequence,
resend_delay_ms: options.resend_delay_ms,
@ -503,10 +515,21 @@ pub fn cultnet_rudp_endpoint(endpoint: &str) -> Result<CultNetRudpEndpoint> {
Ok(CultNetRudpEndpoint { host, port })
}
fn bind_rudp_socket(options: &CultMeshRudpSocketOptions) -> Result<UdpSocket> {
/// A socket bound to loopback cannot send off the host on Windows, so a client
/// keeps loopback only when its peer is on loopback.
fn client_bind_ip(remote_addr: SocketAddr) -> IpAddr {
match (remote_addr.ip().is_loopback(), remote_addr) {
(true, SocketAddr::V4(_)) => IpAddr::V4(Ipv4Addr::LOCALHOST),
(true, SocketAddr::V6(_)) => IpAddr::V6(Ipv6Addr::LOCALHOST),
(false, SocketAddr::V4(_)) => IpAddr::V4(Ipv4Addr::UNSPECIFIED),
(false, SocketAddr::V6(_)) => IpAddr::V6(Ipv6Addr::UNSPECIFIED),
}
}
fn bind_rudp_socket(bind_host: &str, options: &CultMeshRudpSocketOptions) -> Result<UdpSocket> {
let socket = UdpSocket::bind(format!(
"{}:{}",
socket_host(&options.bind_host),
socket_host(bind_host),
options.bind_port
))?;
socket.set_read_timeout(options.read_timeout)?;

2
packages/cultnet-rs/src/lib.rs
View file

@ -2,6 +2,7 @@ mod content;
mod contracts;
mod idunn_deployment_brake;
mod idunn_lifecycle_brake;
mod media_fec;
mod media_stream_contracts;
mod media_stream_wire;
mod provider_health_contracts;
@ -24,6 +25,7 @@ pub use content::*;
pub use contracts::*;
pub use idunn_deployment_brake::*;
pub use idunn_lifecycle_brake::*;
pub use media_fec::*;
pub use media_stream_contracts::*;
pub use media_stream_wire::*;
pub use provider_health_contracts::*;

749
packages/cultnet-rs/src/media_fec.rs Normal file
View file

@ -0,0 +1,749 @@
//! Forward error correction for CultMesh Media Stream records.
//!
//! Pure functions over the records in [`crate::media_stream_contracts`]: no
//! sockets, no clock, no randomness. A producer calls [`protect_video_frame`]
//! and [`protect_audio_block`]; a consumer calls [`recover_video_block`] and
//! [`recover_audio_block`]. Both ends link this one implementation, so a
//! producer and a consumer cannot disagree about the math without one of them
//! failing to build against the same version of this crate.
//!
//! # The code
//!
//! Every block is a systematic Reed-Solomon code over GF(2^8): `k` data shards
//! are sent unchanged and `m` parity shards are computed from them. Any `k` of
//! the `k + m` shards recover the block (the code is MDS), so any `m` erasures
//! are repaired and any more are reported as [`MediaFecError::BeyondRepair`],
//! never as wrong bytes. The construction is named on every parity record by
//! [`MEDIA_FEC_SCHEME_RS_GF256_V1`] and backed by the `reed-solomon-erasure`
//! crate. The library is not the contract: the construction below is, and
//! `tests/media_fec.rs` pins the parity bytes of known inputs, so a library that
//! changes the matrix fails there instead of on the wire, and a decoder in
//! another runtime has vectors to meet.
//!
//! # The `rs-gf256-v1` construction
//!
//! Enough to implement it in another runtime without this crate.
//!
//! * The field is GF(2^8) over the polynomial `x^8 + x^4 + x^3 + x^2 + 1`
//! (`0x11D`). Addition is XOR; multiplication is carry-less multiplication
//! reduced by `0x11D`.
//! * For a block of `k` data shards and `m` parity shards, let `n = k + m`
//! (`n <= 256`) and build the `n x k` Vandermonde matrix `V` whose entry in
//! row `r`, column `c` is `r^c`, where the row number `r` (0 through
//! `n - 1`) is itself read as a field element and `0^0 = 1`. The evaluation
//! points are the integers 0 through `n - 1`, not powers of a generator.
//! * The encoding matrix is `E = V * inverse(top)`, where `top` is the first
//! `k` rows of `V`. Its first `k` rows are the identity, so data shards are
//! sent unchanged; its last `m` rows generate parity.
//! * Every shard of a block has the same length, `shard_payload_bytes`. A
//! shorter final video chunk is zero-padded to that length before encoding.
//! Byte `j` of parity shard `p` is the XOR over data shards `i` of
//! `E[k + p][i] * data[i][j]`.
//! * Any `k` surviving shards recover the block: invert the `k x k` submatrix
//! of `E` for the surviving rows and multiply.
//!
//! The known-answer vectors in `tests/media_fec.rs` were checked against an
//! independent implementation of exactly this text.
//!
//! # What a block is
//!
//! * Video: a frame's chunks split into `ceil(n / 16)` contiguous, near-equal
//! blocks, each protected by `max(2, ceil(k / 4))` parity shards. A block
//! never spans two frames, so one frame's deadline governs its recovery.
//! [`protect_video_frame`] returns the frame's records in send order: blocks
//! are interleaved, so a burst of consecutive losses lands as one loss per
//! block rather than many in one.
//! * Audio: `4` consecutive packets of one payload length, protected by `2`
//! parity shards.
//!
//! # One datagram
//!
//! Every record the protect functions return, data or parity, fits one
//! datagram once wrapped for the wire, so parity never reintroduces the
//! fragment amplification it exists to remove. The protect functions enforce
//! that against [`MediaFecPolicy::max_wire_bytes`]: they encode each record
//! with the caller's provenance and refuse the block if any is larger.
//! The default budget is [`MEDIA_FEC_MAX_WIRE_BYTES`], an IPv4 path with a
//! 1500-byte MTU. Pass a smaller budget when the path carries less: IPv6 over
//! 1500 leaves a UDP payload of 1452 bytes, and a WireGuard or Tailscale
//! tunnel about 1252, so subtract the 41 bytes of RUDP header and channel name
//! from the path's UDP payload limit.
//!
//! The budget is measured with the provenance given, so a producer whose
//! `stored_at` or identifiers grow later must pass the widest it will use.
//!
//! # Audio playout
//!
//! A recovered audio packet has the block's latest `deadline_ticks`, so a
//! packet recovered early in the block reports a deadline later than its own.
//! Audio consumers must gate playout on `pts_ticks`, which recovery derives
//! exactly, and use `deadline_ticks` only to give up on the block as a whole.
//!
//! # What this module does not own
//!
//! When to give up on a block is the receiver's deadline, and the `(k, m)`
//! choice is the producer's policy ([`MediaFecPolicy`]); this module only makes
//! and consumes shards.
use std::collections::BTreeMap;
use std::fmt;
use reed_solomon_erasure::galois_8::ReedSolomon;
use crate::media_stream_contracts::{
GameCultMediaAudioPacketRecord, GameCultMediaAudioParityShardRecord,
GameCultMediaVideoAccessUnitRecord, GameCultMediaVideoParityShardRecord,
};
use crate::media_stream_wire::{
GAMECULT_MEDIA_CHANNEL, GAMECULT_MEDIA_MAX_WIRE_BYTES, GameCultMediaWireRecord, MediaWireProvenance,
encode_media_wire_record, validate_audio_parity_record, validate_audio_record,
validate_video_parity_record, validate_video_record,
};
use crate::rudp::RUDP_FIXED_HEADER_BYTES;
/// Names the construction on every parity record: systematic Reed-Solomon over
/// GF(2^8), as built by `reed-solomon-erasure`'s `galois_8` field. A different
/// matrix is a different scheme id.
pub const MEDIA_FEC_SCHEME_RS_GF256_V1: &str = "rs-gf256-v1";
/// The largest UDP payload an IPv4 datagram carries on a 1500-byte MTU.
pub const MEDIA_IPV4_UDP_PAYLOAD_BYTES: usize = 1_472;
/// The default datagram budget: the most a wrapped media record may occupy so
/// that the RUDP header and the channel name still leave it in one IPv4
/// datagram on a 1500-byte MTU (1,431 bytes).
pub const MEDIA_FEC_MAX_WIRE_BYTES: usize =
MEDIA_IPV4_UDP_PAYLOAD_BYTES - RUDP_FIXED_HEADER_BYTES - GAMECULT_MEDIA_CHANNEL.len();
/// A producer's choice of `(k, m)` and datagram budget.
/// [`MediaFecPolicy::STANDARD`] is the ruled default; each record carries the
/// `(k, m)` it was made with, so a decoder needs no policy.
///
/// Build one by overriding fields of the default, `MediaFecPolicy {
/// max_wire_bytes: 1_211, ..MediaFecPolicy::STANDARD }`, not by naming every
/// field: a field added later then cannot break, or silently mis-set, the
/// caller. `max_wire_bytes` may not exceed
/// [`GAMECULT_MEDIA_MAX_WIRE_BYTES`], the ceiling a decoder holds every record
/// to.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct MediaFecPolicy {
/// The most data chunks in one video block.
pub video_max_block_data_shards: u16,
/// The fewest parity shards a video block carries.
pub video_min_parity_shards: u16,
/// A video block of `k` data shards carries at least `ceil(k / this)` parity
/// shards.
pub video_parity_divisor: u16,
/// Packets per audio block.
pub audio_data_shards: u16,
/// Parity shards per audio block.
pub audio_parity_shards: u16,
/// The most a wrapped record may occupy. The protect functions refuse a
/// block with any record over it. See the module docs for smaller paths.
pub max_wire_bytes: usize,
}
impl MediaFecPolicy {
/// Video `k <= 16` with `m = max(2, ceil(k / 4))`; audio 4+2; the IPv4
/// datagram budget.
pub const STANDARD: Self = Self {
video_max_block_data_shards: 16,
video_min_parity_shards: 2,
video_parity_divisor: 4,
audio_data_shards: 4,
audio_parity_shards: 2,
max_wire_bytes: MEDIA_FEC_MAX_WIRE_BYTES,
};
/// `m` for a video block of `data_shards` chunks.
pub fn video_parity_shards(&self, data_shards: u16) -> u16 {
data_shards
.div_ceil(self.video_parity_divisor.max(1))
.max(self.video_min_parity_shards)
}
/// How a frame of `chunk_count` chunks splits: `ceil(n / max)` blocks whose
/// sizes differ by at most one, larger blocks first.
pub fn video_block_sizes(&self, chunk_count: u16) -> Vec<u16> {
let max = self.video_max_block_data_shards.max(1);
let blocks = chunk_count.div_ceil(max);
if blocks == 0 {
return Vec::new();
}
let base = chunk_count / blocks;
let extra = chunk_count % blocks;
(0..blocks).map(|index| base + u16::from(index < extra)).collect()
}
fn validate(&self) -> Result<(), MediaFecError> {
let widest = self.video_max_block_data_shards;
if widest == 0
|| self.video_min_parity_shards == 0
|| self.video_parity_divisor == 0
|| self.audio_data_shards == 0
|| self.audio_parity_shards == 0
{
return Err(MediaFecError::invalid("media FEC policy fields must be non-zero"));
}
if usize::from(widest) + usize::from(self.video_parity_shards(widest)) > 256
|| usize::from(self.audio_data_shards) + usize::from(self.audio_parity_shards) > 256
{
return Err(MediaFecError::invalid("media FEC policy exceeds 256 shards per block"));
}
if self.max_wire_bytes > GAMECULT_MEDIA_MAX_WIRE_BYTES {
return Err(MediaFecError::invalid(format!(
"media FEC policy max_wire_bytes {} exceeds the {GAMECULT_MEDIA_MAX_WIRE_BYTES}-byte media record ceiling",
self.max_wire_bytes
)));
}
Ok(())
}
}
impl Default for MediaFecPolicy {
fn default() -> Self {
Self::STANDARD
}
}
/// Why a block could not be protected or recovered.
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum MediaFecError {
/// Fewer than `k` of the block's shards are present. Nothing is returned:
/// a block that cannot be repaired is reported, never guessed at.
BeyondRepair { shards_present: usize, shards_needed: usize },
/// The inputs are not one well-formed block.
Invalid(String),
}
impl MediaFecError {
fn invalid(message: impl Into<String>) -> Self {
Self::Invalid(message.into())
}
}
impl fmt::Display for MediaFecError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::BeyondRepair { shards_present, shards_needed } => write!(
f,
"media FEC block is beyond repair: {shards_present} shards present, {shards_needed} needed"
),
Self::Invalid(message) => write!(f, "invalid media FEC block: {message}"),
}
}
}
impl std::error::Error for MediaFecError {}
// ---------------------------------------------------------------------------
// The code: equal-length shards in, equal-length shards out.
// ---------------------------------------------------------------------------
fn parity_shards(data: &[Vec<u8>], parity_count: usize) -> Result<Vec<Vec<u8>>, MediaFecError> {
let code = ReedSolomon::new(data.len(), parity_count)
.map_err(|error| MediaFecError::invalid(format!("{error:?}")))?;
let mut parity = vec![vec![0_u8; data[0].len()]; parity_count];
code.encode_sep(data, &mut parity)
.map_err(|error| MediaFecError::invalid(format!("{error:?}")))?;
Ok(parity)
}
/// `shards` holds the block's `k` data slots then `m` parity slots, `None` for
/// each missing. Returns the `k` data shards.
fn recover_data_shards(
data_shards: usize,
mut shards: Vec<Option<Vec<u8>>>,
) -> Result<Vec<Vec<u8>>, MediaFecError> {
let parity_shards = shards.len() - data_shards;
let present = shards.iter().flatten().count();
if present < data_shards {
return Err(MediaFecError::BeyondRepair {
shards_present: present,
shards_needed: data_shards,
});
}
let code = ReedSolomon::new(data_shards, parity_shards)
.map_err(|error| MediaFecError::invalid(format!("{error:?}")))?;
code.reconstruct_data(&mut shards)
.map_err(|error| MediaFecError::invalid(format!("{error:?}")))?;
Ok(shards
.into_iter()
.take(data_shards)
.map(|shard| shard.expect("reconstruct_data fills every data shard"))
.collect())
}
fn padded(payload: &[u8], shard_bytes: usize) -> Vec<u8> {
let mut shard = vec![0_u8; shard_bytes];
shard[..payload.len()].copy_from_slice(payload);
shard
}
fn from_anyhow(error: anyhow::Error) -> MediaFecError {
MediaFecError::Invalid(error.to_string())
}
/// Refuses a record that would not fit one datagram once wrapped.
fn fits_datagram(
record: &GameCultMediaWireRecord,
provenance: MediaWireProvenance<'_>,
policy: &MediaFecPolicy,
) -> Result<(), MediaFecError> {
let wire = encode_media_wire_record(record, provenance).map_err(from_anyhow)?;
if wire.len() > policy.max_wire_bytes {
return Err(MediaFecError::invalid(format!(
"a {} record wraps to {} bytes, over the {}-byte datagram budget: cut chunks smaller or raise max_wire_bytes",
record.schema_id(),
wire.len(),
policy.max_wire_bytes
)));
}
Ok(())
}
/// Where slot `slot` of an audio block lands: its packet id and presentation
/// time, or `None` if either overflows. The one place block arithmetic lives;
/// the protect path, the wire validator and recovery all ask it.
pub(crate) fn audio_slot(
base_packet_id: u64,
base_pts_ticks: i64,
packet_duration_ticks: u32,
slot: u64,
) -> Option<(u64, i64)> {
let packet_id = base_packet_id.checked_add(slot)?;
let pts_ticks = i64::from(packet_duration_ticks)
.checked_mul(i64::try_from(slot).ok()?)?
.checked_add(base_pts_ticks)?;
Some((packet_id, pts_ticks))
}
// ---------------------------------------------------------------------------
// Video
// ---------------------------------------------------------------------------
/// Protects one video frame. `chunks` are all of the frame's access-unit
/// records, in any order; the result is every data and parity record in send
/// order.
///
/// Within a block, data and parity alternate; across blocks, the sequence
/// rotates one record from each block at a time. Where every block is the same
/// size, no two consecutive `block_count` records share a block. Blocks that
/// differ in size run out at the end, and the rotation then cycles over the
/// blocks remaining.
///
/// Errors if `chunks` are not exactly one whole frame (mixed frames, a missing
/// or repeated chunk index, disagreeing framing), if a chunk is not shard
/// shaped (every chunk of a block but its last must be the same length, the
/// last no longer), or if any resulting record, wrapped with `provenance`,
/// exceeds `policy.max_wire_bytes`.
pub fn protect_video_frame(
chunks: &[GameCultMediaVideoAccessUnitRecord],
policy: &MediaFecPolicy,
provenance: MediaWireProvenance<'_>,
) -> Result<Vec<GameCultMediaWireRecord>, MediaFecError> {
policy.validate()?;
let first = chunks
.first()
.ok_or_else(|| MediaFecError::invalid("a video frame needs at least one chunk"))?;
let chunk_count = usize::from(first.chunk_count);
if chunks.len() != chunk_count {
return Err(MediaFecError::invalid(format!(
"frame {} declares {} chunks but {} were given",
first.frame_id,
chunk_count,
chunks.len()
)));
}
let mut by_index: BTreeMap<u16, &GameCultMediaVideoAccessUnitRecord> = BTreeMap::new();
for chunk in chunks {
validate_video_record(chunk).map_err(from_anyhow)?;
if !same_frame(first, chunk) {
return Err(MediaFecError::invalid(
"a block never spans frames: chunks disagree on stream, session, frame or framing",
));
}
if by_index.insert(chunk.chunk_index, chunk).is_some() {
return Err(MediaFecError::invalid(format!(
"chunk {} appears twice",
chunk.chunk_index
)));
}
}
let block_sizes = policy.video_block_sizes(first.chunk_count);
let block_count = block_sizes.len() as u16;
let mut lanes: Vec<Vec<GameCultMediaWireRecord>> = Vec::with_capacity(block_sizes.len());
let mut start = 0_u16;
for (block_index, &size) in block_sizes.iter().enumerate() {
let members: Vec<&GameCultMediaVideoAccessUnitRecord> = (start..start + size)
.map(|index| by_index[&index])
.collect();
let shard_bytes = members[0].payload.len();
let (last, body) = members.split_last().expect("block has a member");
if body.iter().any(|chunk| chunk.payload.len() != shard_bytes)
|| last.payload.len() > shard_bytes
{
return Err(MediaFecError::invalid(
"every chunk of a block but its last must share one length, and the last no longer",
));
}
let data: Vec<Vec<u8>> = members
.iter()
.map(|chunk| padded(&chunk.payload, shard_bytes))
.collect();
let parity_count = policy.video_parity_shards(size);
let parity = parity_shards(&data, usize::from(parity_count))?;
let parity_records: Vec<GameCultMediaWireRecord> = parity
.into_iter()
.enumerate()
.map(|(parity_index, payload)| {
GameCultMediaWireRecord::VideoParity(GameCultMediaVideoParityShardRecord {
stream_id: first.stream_id.clone(),
session_id: first.session_id.clone(),
frame_id: first.frame_id,
codec: first.codec.clone(),
pts_ticks: first.pts_ticks,
duration_ticks: first.duration_ticks,
timebase_num: first.timebase_num,
timebase_den: first.timebase_den,
keyframe: first.keyframe,
dependency_frame_id: first.dependency_frame_id,
deadline_ticks: first.deadline_ticks,
chunk_count: first.chunk_count,
fec_scheme: MEDIA_FEC_SCHEME_RS_GF256_V1.to_string(),
block_index: block_index as u16,
block_count,
block_data_start: start,
block_data_count: size,
parity_index: parity_index as u16,
parity_count,
shard_payload_bytes: shard_bytes as u32,
last_chunk_payload_bytes: last.payload.len() as u32,
payload,
})
})
.collect();
let data_records = members
.iter()
.map(|chunk| GameCultMediaWireRecord::Video((*chunk).clone()));
lanes.push(interleave(data_records.collect(), parity_records));
start += size;
}
let records = rotate(lanes);
for record in &records {
fits_datagram(record, provenance, policy)?;
}
Ok(records)
}
fn same_frame(a: &GameCultMediaVideoAccessUnitRecord, b: &GameCultMediaVideoAccessUnitRecord) -> bool {
a.stream_id == b.stream_id
&& a.session_id == b.session_id
&& a.frame_id == b.frame_id
&& a.codec == b.codec
&& a.pts_ticks == b.pts_ticks
&& a.duration_ticks == b.duration_ticks
&& a.timebase_num == b.timebase_num
&& a.timebase_den == b.timebase_den
&& a.keyframe == b.keyframe
&& a.dependency_frame_id == b.dependency_frame_id
&& a.deadline_ticks == b.deadline_ticks
&& a.chunk_count == b.chunk_count
}
/// Data and parity alternate: `D0 P0 D1 P1 ...`, the longer list finishing the
/// lane.
fn interleave<T>(data: Vec<T>, parity: Vec<T>) -> Vec<T> {
let mut data = data.into_iter();
let mut parity = parity.into_iter();
let mut lane = Vec::new();
loop {
let next_data = data.next();
let next_parity = parity.next();
if next_data.is_none() && next_parity.is_none() {
return lane;
}
lane.extend(next_data);
lane.extend(next_parity);
}
}
/// One record from each lane in turn, until every lane is empty.
fn rotate<T>(lanes: Vec<Vec<T>>) -> Vec<T> {
let mut lanes: Vec<_> = lanes.into_iter().map(Vec::into_iter).collect();
let mut out = Vec::new();
loop {
let before = out.len();
for lane in &mut lanes {
out.extend(lane.next());
}
if out.len() == before {
return out;
}
}
}
/// Rebuilds the data chunks a block lost.
///
/// `parity` are the block's parity shards that arrived (at least one, for its
/// geometry) and `data` its data chunks that arrived; each may be in any order
/// and may repeat. Returns the missing chunks, ordered by `chunk_index`: empty
/// if none were missing. Fails with [`MediaFecError::BeyondRepair`] if fewer
/// than `block_data_count` shards of the block are present, and with
/// [`MediaFecError::Invalid`] if the records are not from one block.
pub fn recover_video_block(
parity: &[GameCultMediaVideoParityShardRecord],
data: &[GameCultMediaVideoAccessUnitRecord],
) -> Result<Vec<GameCultMediaVideoAccessUnitRecord>, MediaFecError> {
let geometry = parity
.first()
.ok_or_else(|| MediaFecError::invalid("recovery needs at least one parity shard"))?;
let k = usize::from(geometry.block_data_count);
let m = usize::from(geometry.parity_count);
let start = usize::from(geometry.block_data_start);
let shard_bytes = geometry.shard_payload_bytes as usize;
let mut shards: Vec<Option<Vec<u8>>> = vec![None; k + m];
for shard in parity {
validate_video_parity_record(shard).map_err(from_anyhow)?;
if !same_block(geometry, shard) {
return Err(MediaFecError::invalid("parity shards are not from one block"));
}
shards[k + usize::from(shard.parity_index)].get_or_insert_with(|| shard.payload.clone());
}
for chunk in data {
validate_video_record(chunk).map_err(from_anyhow)?;
let index = usize::from(chunk.chunk_index);
let framing_agrees = chunk.stream_id == geometry.stream_id
&& chunk.session_id == geometry.session_id
&& chunk.frame_id == geometry.frame_id
&& chunk.chunk_count == geometry.chunk_count;
if !framing_agrees || index < start || index >= start + k {
return Err(MediaFecError::invalid("data chunk does not belong to the parity's block"));
}
let expected = if index == start + k - 1 {
geometry.last_chunk_payload_bytes as usize
} else {
shard_bytes
};
if chunk.payload.len() != expected {
return Err(MediaFecError::invalid(format!(
"chunk {index} is {} bytes but the block declares {expected}",
chunk.payload.len()
)));
}
shards[index - start].get_or_insert_with(|| padded(&chunk.payload, shard_bytes));
}
let missing: Vec<usize> = (0..k).filter(|&slot| shards[slot].is_none()).collect();
let recovered = recover_data_shards(k, shards)?;
Ok(missing
.into_iter()
.map(|slot| {
let mut payload = recovered[slot].clone();
if slot == k - 1 {
payload.truncate(geometry.last_chunk_payload_bytes as usize);
}
GameCultMediaVideoAccessUnitRecord {
stream_id: geometry.stream_id.clone(),
session_id: geometry.session_id.clone(),
frame_id: geometry.frame_id,
codec: geometry.codec.clone(),
pts_ticks: geometry.pts_ticks,
duration_ticks: geometry.duration_ticks,
timebase_num: geometry.timebase_num,
timebase_den: geometry.timebase_den,
keyframe: geometry.keyframe,
dependency_frame_id: geometry.dependency_frame_id,
deadline_ticks: geometry.deadline_ticks,
chunk_index: (start + slot) as u16,
chunk_count: geometry.chunk_count,
payload,
}
})
.collect())
}
fn same_block(a: &GameCultMediaVideoParityShardRecord, b: &GameCultMediaVideoParityShardRecord) -> bool {
a.stream_id == b.stream_id
&& a.session_id == b.session_id
&& a.frame_id == b.frame_id
&& a.codec == b.codec
&& a.pts_ticks == b.pts_ticks
&& a.duration_ticks == b.duration_ticks
&& a.timebase_num == b.timebase_num
&& a.timebase_den == b.timebase_den
&& a.keyframe == b.keyframe
&& a.dependency_frame_id == b.dependency_frame_id
&& a.deadline_ticks == b.deadline_ticks
&& a.chunk_count == b.chunk_count
&& a.fec_scheme == b.fec_scheme
&& a.block_index == b.block_index
&& a.block_count == b.block_count
&& a.block_data_start == b.block_data_start
&& a.block_data_count == b.block_data_count
&& a.parity_count == b.parity_count
&& a.shard_payload_bytes == b.shard_payload_bytes
&& a.last_chunk_payload_bytes == b.last_chunk_payload_bytes
}
// ---------------------------------------------------------------------------
// Audio
// ---------------------------------------------------------------------------
/// Protects one block of audio packets: exactly `policy.audio_data_shards`
/// packets with contiguous ids and presentation times, one payload length, and
/// one codec and timebase. Returns the parity records in `parity_index` order.
/// Errors if a parity record, wrapped with `provenance`, exceeds
/// `policy.max_wire_bytes`; the packets are smaller than their parity.
pub fn protect_audio_block(
packets: &[GameCultMediaAudioPacketRecord],
policy: &MediaFecPolicy,
provenance: MediaWireProvenance<'_>,
) -> Result<Vec<GameCultMediaAudioParityShardRecord>, MediaFecError> {
policy.validate()?;
if packets.len() != usize::from(policy.audio_data_shards) {
return Err(MediaFecError::invalid(format!(
"an audio block is {} packets, got {}",
policy.audio_data_shards,
packets.len()
)));
}
let first = &packets[0];
let shard_bytes = first.payload.len();
for (index, packet) in packets.iter().enumerate() {
validate_audio_record(packet).map_err(from_anyhow)?;
let expected =
audio_slot(first.packet_id, first.pts_ticks, first.duration_ticks, index as u64);
if packet.stream_id != first.stream_id
|| packet.session_id != first.session_id
|| packet.codec != first.codec
|| packet.timebase_num != first.timebase_num
|| packet.timebase_den != first.timebase_den
|| packet.duration_ticks != first.duration_ticks
|| Some((packet.packet_id, packet.pts_ticks)) != expected
|| packet.payload.len() != shard_bytes
{
return Err(MediaFecError::invalid(
"audio block packets must be contiguous and share codec, timebase, duration and payload length",
));
}
}
let data: Vec<Vec<u8>> = packets.iter().map(|packet| packet.payload.clone()).collect();
let deadline_ticks = packets.iter().map(|packet| packet.deadline_ticks).max().unwrap_or(0);
let parity: Vec<GameCultMediaAudioParityShardRecord> =
parity_shards(&data, usize::from(policy.audio_parity_shards))?
.into_iter()
.enumerate()
.map(|(parity_index, payload)| GameCultMediaAudioParityShardRecord {
stream_id: first.stream_id.clone(),
session_id: first.session_id.clone(),
codec: first.codec.clone(),
fec_scheme: MEDIA_FEC_SCHEME_RS_GF256_V1.to_string(),
base_packet_id: first.packet_id,
base_pts_ticks: first.pts_ticks,
packet_duration_ticks: first.duration_ticks,
timebase_num: first.timebase_num,
timebase_den: first.timebase_den,
deadline_ticks,
data_shard_count: policy.audio_data_shards,
parity_index: parity_index as u16,
parity_shard_count: policy.audio_parity_shards,
shard_payload_bytes: shard_bytes as u32,
payload,
})
.collect();
// A parity record repeats every field of a packet and adds the scheme and
// the geometry, so it is the larger of the block's records: if it fits, the
// packets do.
for shard in &parity {
fits_datagram(&GameCultMediaWireRecord::AudioParity(shard.clone()), provenance, policy)?;
}
Ok(parity)
}
/// Rebuilds the packets an audio block lost, ordered by `packet_id`. A
/// recovered packet takes its id and presentation time from the block's
/// arithmetic and its deadline from the block's, the latest in the block:
/// gate playout on `pts_ticks`, not `deadline_ticks`. Errors as [`recover_video_block`] does.
pub fn recover_audio_block(
parity: &[GameCultMediaAudioParityShardRecord],
data: &[GameCultMediaAudioPacketRecord],
) -> Result<Vec<GameCultMediaAudioPacketRecord>, MediaFecError> {
let geometry = parity
.first()
.ok_or_else(|| MediaFecError::invalid("recovery needs at least one parity shard"))?;
let k = usize::from(geometry.data_shard_count);
let m = usize::from(geometry.parity_shard_count);
let shard_bytes = geometry.shard_payload_bytes as usize;
let mut shards: Vec<Option<Vec<u8>>> = vec![None; k + m];
for shard in parity {
validate_audio_parity_record(shard).map_err(from_anyhow)?;
if !same_audio_block(geometry, shard) {
return Err(MediaFecError::invalid("parity shards are not from one block"));
}
shards[k + usize::from(shard.parity_index)].get_or_insert_with(|| shard.payload.clone());
}
for packet in data {
validate_audio_record(packet).map_err(from_anyhow)?;
let slot = packet
.packet_id
.checked_sub(geometry.base_packet_id)
.filter(|slot| *slot < k as u64)
.ok_or_else(|| MediaFecError::invalid("audio packet does not belong to the parity's block"))?
as usize;
if packet.stream_id != geometry.stream_id
|| packet.session_id != geometry.session_id
|| packet.payload.len() != shard_bytes
{
return Err(MediaFecError::invalid("audio packet disagrees with the parity's block"));
}
shards[slot].get_or_insert_with(|| packet.payload.clone());
}
let missing: Vec<usize> = (0..k).filter(|&slot| shards[slot].is_none()).collect();
let recovered = recover_data_shards(k, shards)?;
missing
.into_iter()
.map(|slot| {
let (packet_id, pts_ticks) = audio_slot(
geometry.base_packet_id,
geometry.base_pts_ticks,
geometry.packet_duration_ticks,
slot as u64,
)
.ok_or_else(|| MediaFecError::invalid("audio block runs past the packet id or pts range"))?;
Ok(GameCultMediaAudioPacketRecord {
stream_id: geometry.stream_id.clone(),
session_id: geometry.session_id.clone(),
packet_id,
codec: geometry.codec.clone(),
pts_ticks,
duration_ticks: geometry.packet_duration_ticks,
timebase_num: geometry.timebase_num,
timebase_den: geometry.timebase_den,
deadline_ticks: geometry.deadline_ticks,
payload: recovered[slot].clone(),
})
})
.collect()
}
fn same_audio_block(a: &GameCultMediaAudioParityShardRecord, b: &GameCultMediaAudioParityShardRecord) -> bool {
a.stream_id == b.stream_id
&& a.session_id == b.session_id
&& a.codec == b.codec
&& a.fec_scheme == b.fec_scheme
&& a.base_packet_id == b.base_packet_id
&& a.base_pts_ticks == b.base_pts_ticks
&& a.packet_duration_ticks == b.packet_duration_ticks
&& a.timebase_num == b.timebase_num
&& a.timebase_den == b.timebase_den
&& a.deadline_ticks == b.deadline_ticks
&& a.data_shard_count == b.data_shard_count
&& a.parity_shard_count == b.parity_shard_count
&& a.shard_payload_bytes == b.shard_payload_bytes
}

104
packages/cultnet-rs/src/media_stream_contracts.rs
View file

@ -36,15 +36,15 @@
//! Nothing here specifies a delivery guarantee. Whether a stream rides a
//! reliable or lossy CultNet channel, and whether it carries parity, is a
//! transport decision made per stream — not a property of the payload shape.
//! Video parity shards exist for producers that choose forward error correction;
//! audio currently has no parity record, and a producer emitting audio on a
//! lossy channel needs one.
//! Video and audio parity shards exist for producers that choose forward error
//! correction; [`crate::media_fec`] owns the code that makes and consumes them.
use cultcache_rs::DatabaseEntry;
pub const GAMECULT_MEDIA_VIDEO_ACCESS_UNIT_SCHEMA: &str = "gamecult.media_video_access_unit.v1";
pub const GAMECULT_MEDIA_VIDEO_PARITY_SHARD_SCHEMA: &str = "gamecult.media_video_parity_shard.v2";
pub const GAMECULT_MEDIA_VIDEO_PARITY_SHARD_SCHEMA: &str = "gamecult.media_video_parity_shard.v3";
pub const GAMECULT_MEDIA_AUDIO_PACKET_SCHEMA: &str = "gamecult.media_audio_packet.v1";
pub const GAMECULT_MEDIA_AUDIO_PARITY_SHARD_SCHEMA: &str = "gamecult.media_audio_parity_shard.v1";
pub const GAMECULT_MEDIA_RECEIVER_FEEDBACK_SCHEMA: &str = "gamecult.media_receiver_feedback.v1";
pub const GAMECULT_MEDIA_STREAM_ADVERTISEMENT_SCHEMA: &str =
"gamecult.media_stream_advertisement.v1";
@ -464,15 +464,25 @@ pub struct GameCultMediaVideoAccessUnitRecord {
pub payload: Vec<u8>,
}
/// Forward-error-correction parity for one video access unit. Carries the
/// framing fields of the access unit it protects so a receiver can recover
/// without having seen the original, plus the shard geometry needed to run
/// recovery: `parity_index` within `parity_count`, and the chunk sizes the
/// parity was computed over.
/// Forward-error-correction parity for one block of a video access unit.
///
/// A frame's `chunk_count` chunks are split into `block_count` contiguous
/// blocks, and each block carries `parity_count` parity shards computed over its
/// `block_data_count` data chunks (`block_data_start..block_data_start +
/// block_data_count`). Any `block_data_count` of a block's
/// `block_data_count + parity_count` shards recover it. The math is
/// [`crate::media_fec`]'s and is named by `fec_scheme`; a decoder that does not
/// know the scheme must drop the shard rather than guess.
///
/// It carries the framing fields of the access unit it protects, so a receiver
/// can rebuild a lost chunk without having seen any chunk of the frame.
/// `shard_payload_bytes` is the padded shard length; `last_chunk_payload_bytes`
/// is the true length of the block's final data chunk, the only one that may be
/// shorter.
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
#[cultcache(
type = "gamecult.media_video_parity_shard",
schema = "gamecult.media_video_parity_shard.v2"
schema = "gamecult.media_video_parity_shard.v3"
)]
pub struct GameCultMediaVideoParityShardRecord {
#[cultcache(key = 0)]
@ -500,14 +510,26 @@ pub struct GameCultMediaVideoParityShardRecord {
#[cultcache(key = 11)]
pub chunk_count: u16,
#[cultcache(key = 12)]
pub parity_index: u16,
pub fec_scheme: String,
#[cultcache(key = 13)]
pub parity_count: u16,
pub block_index: u16,
#[cultcache(key = 14)]
pub chunk_payload_bytes: u32,
pub block_count: u16,
#[cultcache(key = 15)]
pub block_data_start: u16,
/// `k`: how many data chunks this block protects.
#[cultcache(key = 16)]
pub block_data_count: u16,
#[cultcache(key = 17)]
pub parity_index: u16,
/// `m`: how many parity shards this block carries.
#[cultcache(key = 18)]
pub parity_count: u16,
#[cultcache(key = 19)]
pub shard_payload_bytes: u32,
#[cultcache(key = 20)]
pub last_chunk_payload_bytes: u32,
#[cultcache(key = 16, bytes)]
#[cultcache(key = 21, bytes)]
pub payload: Vec<u8>,
}
@ -541,6 +563,60 @@ pub struct GameCultMediaAudioPacketRecord {
pub payload: Vec<u8>,
}
/// Forward-error-correction parity for a block of consecutive audio packets.
///
/// The block is `data_shard_count` packets with contiguous ids starting at
/// `base_packet_id`, all with one payload length (`shard_payload_bytes`) and
/// contiguous presentation times spaced `packet_duration_ticks` apart. A packet
/// recovered from parity takes its id and time from that arithmetic, and its
/// `deadline_ticks` from the block's, which is the latest in the block: the
/// per-packet deadline is not recoverable, so a recovered packet may be held
/// past its own but never past the block's.
///
/// Codecs whose frames vary in length pad each one to a shared shard length
/// before parity is computed; that framing belongs to the codec, not to this
/// record.
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
#[cultcache(
type = "gamecult.media_audio_parity_shard",
schema = "gamecult.media_audio_parity_shard.v1"
)]
pub struct GameCultMediaAudioParityShardRecord {
#[cultcache(key = 0)]
pub stream_id: String,
#[cultcache(key = 1)]
pub session_id: String,
#[cultcache(key = 2)]
pub codec: String,
#[cultcache(key = 3)]
pub fec_scheme: String,
#[cultcache(key = 4)]
pub base_packet_id: u64,
#[cultcache(key = 5)]
pub base_pts_ticks: i64,
#[cultcache(key = 6)]
pub packet_duration_ticks: u32,
#[cultcache(key = 7)]
pub timebase_num: u32,
#[cultcache(key = 8)]
pub timebase_den: u32,
/// The latest `deadline_ticks` among the block's data packets.
#[cultcache(key = 9)]
pub deadline_ticks: i64,
/// `k`.
#[cultcache(key = 10)]
pub data_shard_count: u16,
#[cultcache(key = 11)]
pub parity_index: u16,
/// `m`.
#[cultcache(key = 12)]
pub parity_shard_count: u16,
#[cultcache(key = 13)]
pub shard_payload_bytes: u32,
#[cultcache(key = 14, bytes)]
pub payload: Vec<u8>,
}
/// What a receiver observed, published back to the producer so it can adapt:
/// what decoded, what is missing, what arrived too late, and how much queue and
/// jitter the receiver is carrying.

161
packages/cultnet-rs/src/media_stream_wire.rs
View file

@ -28,20 +28,32 @@ use crate::contracts::{
CultNetMessage, CultNetRawDocumentRecord, CultNetRawPayloadEncoding, CultNetWireContract,
decode_cultnet_message_from_slice, encode_cultnet_message_to_vec,
};
use crate::media_fec::{MEDIA_FEC_SCHEME_RS_GF256_V1, audio_slot};
use crate::media_stream_contracts::{
GAMECULT_MEDIA_AUDIO_PACKET_SCHEMA, GAMECULT_MEDIA_RECEIVER_FEEDBACK_SCHEMA,
GAMECULT_MEDIA_VIDEO_ACCESS_UNIT_SCHEMA, GAMECULT_MEDIA_VIDEO_PARITY_SHARD_SCHEMA,
GameCultMediaAudioPacketRecord, GameCultMediaReceiverFeedbackRecord,
GAMECULT_MEDIA_AUDIO_PACKET_SCHEMA, GAMECULT_MEDIA_AUDIO_PARITY_SHARD_SCHEMA,
GAMECULT_MEDIA_RECEIVER_FEEDBACK_SCHEMA, GAMECULT_MEDIA_VIDEO_ACCESS_UNIT_SCHEMA,
GAMECULT_MEDIA_VIDEO_PARITY_SHARD_SCHEMA, GameCultMediaAudioPacketRecord,
GameCultMediaAudioParityShardRecord, GameCultMediaReceiverFeedbackRecord,
GameCultMediaVideoAccessUnitRecord, GameCultMediaVideoParityShardRecord,
};
/// The CultNet channel video and receiver feedback ride.
/// The CultNet channel every media record rides: video, audio, their parity,
/// and receiver feedback. Whether it is reliable is the transport profile's
/// choice per session; stale media is dropped, not retransmitted.
pub const GAMECULT_MEDIA_CHANNEL: &str = "media";
/// The CultNet channel audio rides. Audio has its own channel so one session
/// can carry it reliably with a short expiry while video stays lossy under
/// parity; the RUDP transport profile owns each channel's delivery.
pub const GAMECULT_MEDIA_AUDIO_CHANNEL: &str = "audio";
/// The largest wrapped media record the decoder accepts, and the most the RUDP
/// `media` channel reassembles: 64 KiB.
///
/// It is the bound a receiver holds a peer to. A parity shard's length is
/// whatever its producer's [`MediaFecPolicy::max_wire_bytes`](crate::MediaFecPolicy::max_wire_bytes) allowed, and
/// recovery work grows with the square of the block size times that length, so
/// without a ceiling one hostile record could name any length RUDP would
/// reassemble (16 MiB). 64 KiB is one maximum UDP datagram: it covers every
/// path MTU in use, jumbo frames (9,000 bytes) with a wide margin, and a
/// record is still one message rather than a stream. A policy may not ask for
/// more; see [`MediaFecPolicy`](crate::MediaFecPolicy).
pub const GAMECULT_MEDIA_MAX_WIRE_BYTES: usize = 64 * 1024;
/// One media record on its way somewhere.
#[derive(Clone, Debug, PartialEq, Eq)]
@ -49,6 +61,7 @@ pub enum GameCultMediaWireRecord {
Video(GameCultMediaVideoAccessUnitRecord),
VideoParity(GameCultMediaVideoParityShardRecord),
Audio(GameCultMediaAudioPacketRecord),
AudioParity(GameCultMediaAudioParityShardRecord),
Feedback(GameCultMediaReceiverFeedbackRecord),
}
@ -84,6 +97,7 @@ impl GameCultMediaWireRecord {
Self::Video(_) => GAMECULT_MEDIA_VIDEO_ACCESS_UNIT_SCHEMA,
Self::VideoParity(_) => GAMECULT_MEDIA_VIDEO_PARITY_SHARD_SCHEMA,
Self::Audio(_) => GAMECULT_MEDIA_AUDIO_PACKET_SCHEMA,
Self::AudioParity(_) => GAMECULT_MEDIA_AUDIO_PARITY_SHARD_SCHEMA,
Self::Feedback(_) => GAMECULT_MEDIA_RECEIVER_FEEDBACK_SCHEMA,
}
}
@ -96,13 +110,21 @@ impl GameCultMediaWireRecord {
record.stream_id, record.session_id, record.frame_id, record.chunk_index
),
Self::VideoParity(record) => format!(
"{}:{}:video-parity:{}:{}",
record.stream_id, record.session_id, record.frame_id, record.parity_index
"{}:{}:video-parity:{}:{}:{}",
record.stream_id,
record.session_id,
record.frame_id,
record.block_index,
record.parity_index
),
Self::Audio(record) => format!(
"{}:{}:audio:{}",
record.stream_id, record.session_id, record.packet_id
),
Self::AudioParity(record) => format!(
"{}:{}:audio-parity:{}:{}",
record.stream_id, record.session_id, record.base_packet_id, record.parity_index
),
Self::Feedback(record) => format!(
"{}:{}:feedback:{}",
record.stream_id, record.session_id, record.receiver_id
@ -124,6 +146,7 @@ impl GameCultMediaWireRecord {
Self::Video(record) => rmp_serde::to_vec(record),
Self::VideoParity(record) => rmp_serde::to_vec(record),
Self::Audio(record) => rmp_serde::to_vec(record),
Self::AudioParity(record) => rmp_serde::to_vec(record),
Self::Feedback(record) => rmp_serde::to_vec(record),
}
.map_err(Into::into)
@ -159,11 +182,24 @@ pub fn encode_media_wire_record(
},
};
encode_cultnet_message_to_vec(&message, CultNetWireContract::CultNetSchemaV0).map_err(Into::into)
let wire = encode_cultnet_message_to_vec(&message, CultNetWireContract::CultNetSchemaV0)?;
if wire.len() > GAMECULT_MEDIA_MAX_WIRE_BYTES {
return Err(anyhow!(
"media record is {} bytes, over the {GAMECULT_MEDIA_MAX_WIRE_BYTES}-byte ceiling",
wire.len()
));
}
Ok(wire)
}
/// Unwraps a media record from the wire.
pub fn decode_media_wire_record(payload: &[u8]) -> Result<GameCultMediaWireRecord> {
if payload.len() > GAMECULT_MEDIA_MAX_WIRE_BYTES {
return Err(anyhow!(
"media record is {} bytes, over the {GAMECULT_MEDIA_MAX_WIRE_BYTES}-byte ceiling",
payload.len()
));
}
let message = decode_cultnet_message_from_slice(payload, CultNetWireContract::CultNetSchemaV0)?;
let CultNetMessage::DocumentPutRaw { document, .. } = message else {
return Err(anyhow!("expected cultnet.document_put_raw.v0"));
@ -191,6 +227,12 @@ pub fn decode_media_wire_record(payload: &[u8]) -> Result<GameCultMediaWireRecor
validate_audio_record(&record)?;
GameCultMediaWireRecord::Audio(record)
}
GAMECULT_MEDIA_AUDIO_PARITY_SHARD_SCHEMA => {
let record: GameCultMediaAudioParityShardRecord =
rmp_serde::from_slice(&document.payload)?;
validate_audio_parity_record(&record)?;
GameCultMediaWireRecord::AudioParity(record)
}
GAMECULT_MEDIA_RECEIVER_FEEDBACK_SCHEMA => {
let record: GameCultMediaReceiverFeedbackRecord =
rmp_serde::from_slice(&document.payload)?;
@ -349,39 +391,114 @@ pub fn validate_video_parity_record(record: &GameCultMediaVideoParityShardRecord
"video parity media record deadline_ticks must not precede pts_ticks"
));
}
if record.fec_scheme != MEDIA_FEC_SCHEME_RS_GF256_V1 {
return Err(anyhow!(
"video parity media record fec_scheme {:?} is not supported",
record.fec_scheme
));
}
if record.chunk_count == 0 {
return Err(anyhow!(
"video parity media record chunk_count must be non-zero"
));
}
if record.block_count == 0 || record.block_index >= record.block_count {
return Err(anyhow!(
"video parity media record block_index {} is outside block_count {}",
record.block_index,
record.block_count
));
}
if record.block_data_count == 0
|| u32::from(record.block_data_start) + u32::from(record.block_data_count)
> u32::from(record.chunk_count)
{
return Err(anyhow!(
"video parity media record block does not lie inside the frame's chunks"
));
}
if record.parity_count == 0 || record.parity_index >= record.parity_count {
return Err(anyhow!(
"video parity media record has invalid parity stripe metadata"
"video parity media record parity_index {} is outside parity_count {}",
record.parity_index,
record.parity_count
));
}
if record.parity_count > record.chunk_count {
if usize::from(record.block_data_count) + usize::from(record.parity_count) > 256 {
return Err(anyhow!(
"video parity media record parity_count exceeds chunk_count"
"video parity media record block exceeds 256 shards"
));
}
if record.chunk_payload_bytes == 0 || record.last_chunk_payload_bytes == 0 {
if record.shard_payload_bytes == 0 {
return Err(anyhow!(
"video parity media record chunk lengths must be non-zero"
"video parity media record shard_payload_bytes must be non-zero"
));
}
if record.last_chunk_payload_bytes > record.chunk_payload_bytes {
if record.last_chunk_payload_bytes == 0
|| record.last_chunk_payload_bytes > record.shard_payload_bytes
{
return Err(anyhow!(
"video parity media record last chunk length exceeds regular chunk length"
"video parity media record last chunk length must be in 1..=shard_payload_bytes"
));
}
if record.payload.is_empty() {
if record.payload.len() != record.shard_payload_bytes as usize {
return Err(anyhow!(
"video parity media record payload must be non-empty"
"video parity media record payload does not match declared shard length"
));
}
if record.payload.len() > record.chunk_payload_bytes as usize {
Ok(())
}
pub fn validate_audio_parity_record(record: &GameCultMediaAudioParityShardRecord) -> Result<()> {
if record.stream_id.is_empty() || record.session_id.is_empty() || record.codec.is_empty() {
return Err(anyhow!(
"video parity media record payload exceeds declared chunk length"
"audio parity media record stream_id, session_id and codec must be non-empty"
));
}
if record.timebase_num == 0 || record.timebase_den == 0 || record.packet_duration_ticks == 0 {
return Err(anyhow!(
"audio parity media record timing metadata must be non-zero"
));
}
if record.fec_scheme != MEDIA_FEC_SCHEME_RS_GF256_V1 {
return Err(anyhow!(
"audio parity media record fec_scheme {:?} is not supported",
record.fec_scheme
));
}
if record.data_shard_count == 0
|| record.parity_shard_count == 0
|| record.parity_index >= record.parity_shard_count
|| usize::from(record.data_shard_count) + usize::from(record.parity_shard_count) > 256
{
return Err(anyhow!("audio parity media record stripe metadata is invalid"));
}
let Some((_, last_pts_ticks)) = audio_slot(
record.base_packet_id,
record.base_pts_ticks,
record.packet_duration_ticks,
u64::from(record.data_shard_count) - 1,
) else {
return Err(anyhow!(
"audio parity media record block runs past the packet id or pts range"
));
};
// Recovery gives every recovered packet the block's deadline, so it must
// not precede the last packet's pts or a recovered packet would fail
// `validate_audio_record`.
if record.deadline_ticks < last_pts_ticks {
return Err(anyhow!(
"audio parity media record deadline_ticks must not precede the block's last pts_ticks"
));
}
if record.shard_payload_bytes == 0 {
return Err(anyhow!(
"audio parity media record shard_payload_bytes must be non-zero"
));
}
if record.payload.len() != record.shard_payload_bytes as usize {
return Err(anyhow!(
"audio parity media record payload does not match declared shard length"
));
}
Ok(())

675
packages/cultnet-rs/src/rudp.rs
View file

File diff suppressed because it is too large Load diff

650
packages/cultnet-rs/tests/cultnet.rs
View file

@ -432,12 +432,6 @@ fn rudp_transport_profile_advertises_state_and_realtime_channels() {
CultNetTransportOrdering::Unordered,
Some(64)
),
(
"audio",
CultNetTransportDelivery::Reliable,
CultNetTransportOrdering::Unordered,
Some(64)
),
]
);
}
@ -558,13 +552,13 @@ fn rudp_reconnect_loop_consumes_shared_controller() -> Result<()> {
fn rudp_session_handshake_acks_reliable_connect_and_accept_packets() -> Result<()> {
let mut client = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 0x0a0b0c0d,
initial_sequence: 1,
initial_sequence: Some(1),
resend_delay_ms: 50,
..CultNetRudpSessionOptions::default()
});
let mut server = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 0x0a0b0c0d,
initial_sequence: 100,
initial_sequence: Some(100),
resend_delay_ms: 50,
..CultNetRudpSessionOptions::default()
});
@ -595,52 +589,19 @@ fn rudp_session_handshake_acks_reliable_connect_and_accept_packets() -> Result<(
fn rudp_session_computes_ack_masks_and_clears_pending_reliable_packets() -> Result<()> {
let mut sender = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 7,
initial_sequence: 10,
initial_sequence: Some(10),
resend_delay_ms: 100,
..CultNetRudpSessionOptions::default()
});
let mut receiver = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 7,
initial_sequence: 200,
initial_sequence: Some(200),
resend_delay_ms: 100,
..CultNetRudpSessionOptions::default()
});
sender.receive(
&CultNetRudpPacket {
packet_type: CultNetRudpPacketType::Accept,
connection_id: 7,
sequence: 1,
ack: 0,
ack_mask: 0,
channel_id: "control".to_string(),
reliable: false,
ordered: false,
sequenced: false,
fragment_id: 0,
fragment_index: 0,
fragment_count: 0,
payload: Vec::new(),
},
0,
)?;
receiver.receive(
&CultNetRudpPacket {
packet_type: CultNetRudpPacketType::Accept,
connection_id: 7,
sequence: 2,
ack: 0,
ack_mask: 0,
channel_id: "control".to_string(),
reliable: false,
ordered: false,
sequenced: false,
fragment_id: 0,
fragment_index: 0,
fragment_count: 0,
payload: Vec::new(),
},
0,
)?;
let connect = sender.create_connect(0, Vec::new())?;
let accept = receiver.accept_connect(&connect, 0, Vec::new())?;
sender.receive(&accept, 0)?;
let options = CultNetRudpSendOptions {
reliable: true,
@ -650,20 +611,20 @@ fn rudp_session_computes_ack_masks_and_clears_pending_reliable_packets() -> Resu
let first = sender.send("schema", b"first".to_vec(), options.clone())?;
let second = sender.send("schema", b"second".to_vec(), options.clone())?;
let third = sender.send("schema", b"third".to_vec(), options)?;
assert_eq!(sender.pending_reliable_sequences(), vec![10, 11, 12]);
assert_eq!(sender.pending_reliable_sequences(), vec![11, 12, 13]);
receiver.receive(&first, 0)?;
receiver.receive(&third, 0)?;
let ack_with_gap = receiver.create_ack();
assert_eq!(ack_with_gap.ack, 12);
assert_eq!(ack_with_gap.ack_mask, 0b10 | (1 << 9));
assert_eq!(ack_with_gap.ack, 13);
assert_eq!(ack_with_gap.ack_mask, 0b110);
sender.receive(&ack_with_gap, 0)?;
assert_eq!(sender.pending_reliable_sequences(), vec![11]);
assert_eq!(sender.pending_reliable_sequences(), vec![12]);
receiver.receive(&second, 0)?;
let full_ack = receiver.create_ack();
assert_eq!(full_ack.ack, 12);
assert_eq!(full_ack.ack_mask, 0b11 | (1 << 9));
assert_eq!(full_ack.ack, 13);
assert_eq!(full_ack.ack_mask, 0b111);
sender.receive(&full_ack, 0)?;
assert!(sender.pending_reliable_sequences().is_empty());
Ok(())
@ -673,28 +634,11 @@ fn rudp_session_computes_ack_masks_and_clears_pending_reliable_packets() -> Resu
fn rudp_session_schedules_reliable_resends_until_acked() -> Result<()> {
let mut session = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 99,
initial_sequence: 1,
initial_sequence: Some(1),
resend_delay_ms: 100,
..CultNetRudpSessionOptions::default()
});
session.receive(
&CultNetRudpPacket {
packet_type: CultNetRudpPacketType::Accept,
connection_id: 99,
sequence: 50,
ack: 0,
ack_mask: 0,
channel_id: "control".to_string(),
reliable: false,
ordered: false,
sequenced: false,
fragment_id: 0,
fragment_index: 0,
fragment_count: 0,
payload: Vec::new(),
},
0,
)?;
session.assume_connected(0);
let sent = session.send(
"schema",
b"payload".to_vec(),
@ -743,12 +687,12 @@ fn rudp_session_schedules_reliable_resends_until_acked() -> Result<()> {
fn rudp_session_pings_and_detects_receive_timeout() -> Result<()> {
let mut client = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 101,
initial_sequence: 1,
initial_sequence: Some(1),
..CultNetRudpSessionOptions::default()
});
let mut server = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 101,
initial_sequence: 100,
initial_sequence: Some(100),
..CultNetRudpSessionOptions::default()
});
let connect = client.create_connect(0, b"join".to_vec())?;
@ -774,12 +718,12 @@ fn rudp_session_pings_and_detects_receive_timeout() -> Result<()> {
fn rudp_lossy_packets_cannot_create_reliable_ordered_gaps() -> Result<()> {
let mut client = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 198,
initial_sequence: 1,
initial_sequence: Some(1),
..CultNetRudpSessionOptions::default()
});
let mut server = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 198,
initial_sequence: 100,
initial_sequence: Some(100),
..CultNetRudpSessionOptions::default()
});
let connect = client.create_connect(0, Vec::new())?;
@ -830,12 +774,12 @@ fn rudp_lossy_packets_cannot_create_reliable_ordered_gaps() -> Result<()> {
fn rudp_unreliable_sequenced_delivery_is_scoped_to_its_channel() -> Result<()> {
let mut sender = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 197,
initial_sequence: 50,
initial_sequence: Some(50),
..CultNetRudpSessionOptions::default()
});
let mut receiver = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 197,
initial_sequence: 100,
initial_sequence: Some(100),
..CultNetRudpSessionOptions::default()
});
let connect = sender.create_connect(0, Vec::new())?;
@ -861,24 +805,7 @@ fn rudp_rejects_unreliable_ordered_delivery() -> Result<()> {
connection_id: 196,
..CultNetRudpSessionOptions::default()
});
session.receive(
&CultNetRudpPacket {
packet_type: CultNetRudpPacketType::Accept,
connection_id: 196,
sequence: 50,
ack: 0,
ack_mask: 0,
channel_id: "control".to_string(),
reliable: false,
ordered: false,
sequenced: false,
fragment_id: 0,
fragment_index: 0,
fragment_count: 0,
payload: Vec::new(),
},
0,
)?;
session.assume_connected(0);
let error = session
.send(
"schema",
@ -901,28 +828,11 @@ fn rudp_rejects_unreliable_ordered_delivery() -> Result<()> {
fn rudp_session_bounds_pending_reliable_packets_before_enqueue() -> Result<()> {
let mut session = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 102,
initial_sequence: 1,
initial_sequence: Some(1),
max_pending_reliable_packets: Some(2),
..CultNetRudpSessionOptions::default()
});
session.receive(
&CultNetRudpPacket {
packet_type: CultNetRudpPacketType::Accept,
connection_id: 102,
sequence: 50,
ack: 0,
ack_mask: 0,
channel_id: "control".to_string(),
reliable: false,
ordered: false,
sequenced: false,
fragment_id: 0,
fragment_index: 0,
fragment_count: 0,
payload: Vec::new(),
},
0,
)?;
session.assume_connected(0);
session.send(
"schema",
b"first".to_vec(),
@ -957,28 +867,11 @@ fn rudp_session_bounds_pending_reliable_packets_before_enqueue() -> Result<()> {
let mut fragmented = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 103,
initial_sequence: 1,
initial_sequence: Some(1),
max_pending_reliable_packets: Some(3),
..CultNetRudpSessionOptions::default()
});
fragmented.receive(
&CultNetRudpPacket {
packet_type: CultNetRudpPacketType::Accept,
connection_id: 103,
sequence: 50,
ack: 0,
ack_mask: 0,
channel_id: "control".to_string(),
reliable: false,
ordered: false,
sequenced: false,
fragment_id: 0,
fragment_index: 0,
fragment_count: 0,
payload: Vec::new(),
},
0,
)?;
fragmented.assume_connected(0);
let error = fragmented
.send_many(
"schema",
@ -1001,50 +894,17 @@ fn rudp_session_suppresses_duplicates_and_delivers_reliable_ordered_payloads_in_
-> Result<()> {
let mut sender = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 123,
initial_sequence: 1,
initial_sequence: Some(1),
..CultNetRudpSessionOptions::default()
});
let mut receiver = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 123,
initial_sequence: 100,
initial_sequence: Some(100),
..CultNetRudpSessionOptions::default()
});
sender.receive(
&CultNetRudpPacket {
packet_type: CultNetRudpPacketType::Accept,
connection_id: 123,
sequence: 90,
ack: 0,
ack_mask: 0,
channel_id: "control".to_string(),
reliable: false,
ordered: false,
sequenced: false,
fragment_id: 0,
fragment_index: 0,
fragment_count: 0,
payload: Vec::new(),
},
0,
)?;
receiver.receive(
&CultNetRudpPacket {
packet_type: CultNetRudpPacketType::Accept,
connection_id: 123,
sequence: 91,
ack: 0,
ack_mask: 0,
channel_id: "control".to_string(),
reliable: false,
ordered: false,
sequenced: false,
fragment_id: 0,
fragment_index: 0,
fragment_count: 0,
payload: Vec::new(),
},
0,
)?;
let connect = sender.create_connect(0, Vec::new())?;
let accept = receiver.accept_connect(&connect, 0, Vec::new())?;
sender.receive(&accept, 0)?;
let options = CultNetRudpSendOptions {
reliable: true,
@ -1082,50 +942,17 @@ fn rudp_session_suppresses_duplicates_and_delivers_reliable_ordered_payloads_in_
fn rudp_session_skips_control_packets_while_ordering_schema_payloads() -> Result<()> {
let mut sender = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 124,
initial_sequence: 1,
initial_sequence: Some(1),
..CultNetRudpSessionOptions::default()
});
let mut receiver = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 124,
initial_sequence: 100,
initial_sequence: Some(100),
..CultNetRudpSessionOptions::default()
});
sender.receive(
&CultNetRudpPacket {
packet_type: CultNetRudpPacketType::Accept,
connection_id: 124,
sequence: 90,
ack: 0,
ack_mask: 0,
channel_id: "control".to_string(),
reliable: false,
ordered: false,
sequenced: false,
fragment_id: 0,
fragment_index: 0,
fragment_count: 0,
payload: Vec::new(),
},
0,
)?;
receiver.receive(
&CultNetRudpPacket {
packet_type: CultNetRudpPacketType::Accept,
connection_id: 124,
sequence: 91,
ack: 0,
ack_mask: 0,
channel_id: "control".to_string(),
reliable: false,
ordered: false,
sequenced: false,
fragment_id: 0,
fragment_index: 0,
fragment_count: 0,
payload: Vec::new(),
},
0,
)?;
let connect = sender.create_connect(0, Vec::new())?;
let accept = receiver.accept_connect(&connect, 0, Vec::new())?;
sender.receive(&accept, 0)?;
let options = CultNetRudpSendOptions {
reliable: true,
@ -1164,50 +991,17 @@ fn rudp_session_skips_control_packets_while_ordering_schema_payloads() -> Result
fn rudp_session_fragments_and_reassembles_reliable_ordered_payloads() -> Result<()> {
let mut sender = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 456,
initial_sequence: 1,
initial_sequence: Some(1),
..CultNetRudpSessionOptions::default()
});
let mut receiver = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 456,
initial_sequence: 100,
initial_sequence: Some(100),
..CultNetRudpSessionOptions::default()
});
sender.receive(
&CultNetRudpPacket {
packet_type: CultNetRudpPacketType::Accept,
connection_id: 456,
sequence: 90,
ack: 0,
ack_mask: 0,
channel_id: "control".to_string(),
reliable: false,
ordered: false,
sequenced: false,
fragment_id: 0,
fragment_index: 0,
fragment_count: 0,
payload: Vec::new(),
},
0,
)?;
receiver.receive(
&CultNetRudpPacket {
packet_type: CultNetRudpPacketType::Accept,
connection_id: 456,
sequence: 91,
ack: 0,
ack_mask: 0,
channel_id: "control".to_string(),
reliable: false,
ordered: false,
sequenced: false,
fragment_id: 0,
fragment_index: 0,
fragment_count: 0,
payload: Vec::new(),
},
0,
)?;
let connect = sender.create_connect(0, Vec::new())?;
let accept = receiver.accept_connect(&connect, 0, Vec::new())?;
sender.receive(&accept, 0)?;
let packets = sender.send_many(
"schema",
@ -1265,7 +1059,7 @@ fn rudp_socket_transport_handshakes_and_carries_reliable_ordered_schema_frames()
mode: CultNetRudpSocketMode::Server,
remote_addr: None,
connection_id,
initial_sequence: 100,
initial_sequence: Some(100),
resend_delay_ms: 25,
transport_id: None,
max_payload_bytes: None,
@ -1282,7 +1076,7 @@ fn rudp_socket_transport_handshakes_and_carries_reliable_ordered_schema_frames()
mode: CultNetRudpSocketMode::Client,
remote_addr: Some(server_addr),
connection_id,
initial_sequence: 1,
initial_sequence: Some(1),
resend_delay_ms: 25,
transport_id: None,
max_payload_bytes: None,
@ -1359,7 +1153,7 @@ fn rudp_socket_transport_carries_fragmented_reliable_ordered_schema_frames() ->
mode: CultNetRudpSocketMode::Server,
remote_addr: None,
connection_id,
initial_sequence: 100,
initial_sequence: Some(100),
resend_delay_ms: 25,
transport_id: None,
max_payload_bytes: None,
@ -1376,7 +1170,7 @@ fn rudp_socket_transport_carries_fragmented_reliable_ordered_schema_frames() ->
mode: CultNetRudpSocketMode::Client,
remote_addr: Some(server_addr),
connection_id,
initial_sequence: 1,
initial_sequence: Some(1),
resend_delay_ms: 25,
transport_id: None,
max_payload_bytes: None,
@ -1404,36 +1198,19 @@ fn rudp_session_advances_large_fragment_sets_through_a_bounded_reliable_window()
let connection_id = 0x1020_3042;
let mut sender = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id,
initial_sequence: 1,
initial_sequence: Some(1),
resend_delay_ms: 25,
max_pending_reliable_packets: None,
});
let mut receiver = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id,
initial_sequence: 100,
initial_sequence: Some(100),
resend_delay_ms: 25,
max_pending_reliable_packets: None,
});
for session in [&mut sender, &mut receiver] {
session.receive(
&CultNetRudpPacket {
packet_type: CultNetRudpPacketType::Accept,
connection_id,
sequence: 0,
ack: 0,
ack_mask: 0,
channel_id: "control".into(),
reliable: false,
ordered: false,
sequenced: false,
fragment_id: 0,
fragment_index: 0,
fragment_count: 0,
payload: Vec::new(),
},
0,
)?;
}
let connect = sender.create_connect(0, Vec::new())?;
let accept = receiver.accept_connect(&connect, 0, Vec::new())?;
sender.receive(&accept, 0)?;
let fragment_count = CULTNET_RUDP_RELIABLE_SEND_WINDOW_PACKETS + 17;
let payload = (0..fragment_count * 8)
@ -1491,7 +1268,7 @@ fn rudp_socket_flush_waits_for_large_fragment_delivery() -> Result<()> {
mode: CultNetRudpSocketMode::Server,
remote_addr: None,
connection_id,
initial_sequence: 100,
initial_sequence: Some(100),
resend_delay_ms: 25,
transport_id: None,
max_payload_bytes: None,
@ -1508,7 +1285,7 @@ fn rudp_socket_flush_waits_for_large_fragment_delivery() -> Result<()> {
mode: CultNetRudpSocketMode::Client,
remote_addr: Some(server_addr),
connection_id,
initial_sequence: 1,
initial_sequence: Some(1),
resend_delay_ms: 25,
transport_id: None,
max_payload_bytes: None,
@ -1557,6 +1334,71 @@ fn rudp_socket_flush_waits_for_large_fragment_delivery() -> Result<()> {
Ok(())
}
fn rudp_client_bound_host(endpoint: &str, bind_host: Option<&str>) -> Result<String> {
let client = CultMesh::create_rudp_client_for_endpoint(
"rust-cultmesh-bind-client",
0x2030_4060,
endpoint,
CultMeshRudpSocketOptions {
bind_host: bind_host.map(str::to_string),
..CultMeshRudpSocketOptions::default()
},
)?;
Ok(client.profile.transports[0]
.host
.clone()
.expect("RUDP client profile advertises its bound host"))
}
/// A client bound to loopback cannot send to another host on Windows, so a
/// client for a remote endpoint binds the unspecified address of its family.
#[test]
fn cultmesh_rudp_client_for_a_remote_endpoint_binds_the_unspecified_address() -> Result<()> {
assert_eq!(rudp_client_bound_host("rudp://10.77.0.1:17872", None)?, "0.0.0.0");
assert_eq!(rudp_client_bound_host("rudp://[2001:db8::1]:17872", None)?, "::");
Ok(())
}
#[test]
fn cultmesh_rudp_client_for_a_loopback_endpoint_binds_loopback() -> Result<()> {
assert_eq!(rudp_client_bound_host("rudp://127.0.0.1:17872", None)?, "127.0.0.1");
assert_eq!(rudp_client_bound_host("rudp://[::1]:17872", None)?, "::1");
Ok(())
}
#[test]
fn cultmesh_rudp_client_binds_an_explicit_bind_host_whatever_the_endpoint() -> Result<()> {
assert_eq!(
rudp_client_bound_host("rudp://10.77.0.1:17872", Some("127.0.0.1"))?,
"127.0.0.1"
);
assert_eq!(
rudp_client_bound_host("rudp://127.0.0.1:17872", Some("0.0.0.0"))?,
"0.0.0.0"
);
Ok(())
}
#[test]
fn cultmesh_rudp_server_binds_loopback_unless_told_otherwise() -> Result<()> {
let server = CultMesh::create_rudp_server(
"rust-cultmesh-bind-server",
0x2030_4061,
CultMeshRudpSocketOptions::default(),
)?;
assert_eq!(server.profile.transports[0].host.as_deref(), Some("127.0.0.1"));
let server = CultMesh::create_rudp_server(
"rust-cultmesh-bind-server",
0x2030_4062,
CultMeshRudpSocketOptions {
bind_host: Some("0.0.0.0".to_string()),
..CultMeshRudpSocketOptions::default()
},
)?;
assert_eq!(server.profile.transports[0].host.as_deref(), Some("0.0.0.0"));
Ok(())
}
#[test]
fn cultmesh_facade_creates_rudp_client_from_peer_endpoint() -> Result<()> {
let connection_id = 0x2030_4050;
@ -2669,12 +2511,12 @@ fn schema_discovery_round_trips_over_legacy_gamecult_contract_when_inline_schema
fn rudp_sequenced_channel_tracking_is_bounded() -> Result<()> {
let mut sender = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 401,
initial_sequence: 1,
initial_sequence: Some(1),
..CultNetRudpSessionOptions::default()
});
let mut receiver = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 401,
initial_sequence: 100,
initial_sequence: Some(100),
..CultNetRudpSessionOptions::default()
});
let connect = sender.create_connect(0, Vec::new())?;
@ -2709,18 +2551,30 @@ fn rudp_sequenced_channel_tracking_is_bounded() -> Result<()> {
fn rudp_ordered_channel_tracking_is_bounded() -> Result<()> {
let mut sender = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 402,
initial_sequence: 1,
initial_sequence: Some(1),
..CultNetRudpSessionOptions::default()
});
let mut receiver = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 402,
initial_sequence: 100,
initial_sequence: Some(100),
..CultNetRudpSessionOptions::default()
});
let connect = sender.create_connect(0, Vec::new())?;
let accept = receiver.accept_connect(&connect, 1, Vec::new())?;
sender.receive(&accept, 2)?;
// Channels are tracked only while they hold a frame, so the bound is on
// channels held behind one missing sequence.
let gap = sender.send(
"gap",
b"lost".to_vec(),
CultNetRudpSendOptions {
reliable: true,
ordered: false,
..CultNetRudpSendOptions::default()
},
)?;
sender.receive(&receiver.create_ack_for(gap.sequence), 3)?;
let mut error = None;
for channel in 0..256u32 {
let packet = sender.send(
@ -2733,7 +2587,8 @@ fn rudp_ordered_channel_tracking_is_bounded() -> Result<()> {
},
)?;
match receiver.receive(&packet, 3) {
Ok(_) => {
Ok(result) => {
assert!(result.delivered.is_empty(), "the gap holds every frame");
sender.receive(&receiver.create_ack_for(packet.sequence), 3)?;
}
Err(refused) => {
@ -2754,12 +2609,12 @@ fn rudp_ordered_channel_tracking_is_bounded() -> Result<()> {
fn rudp_ordered_hold_buffer_is_bounded() -> Result<()> {
let mut sender = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 403,
initial_sequence: 1,
initial_sequence: Some(1),
..CultNetRudpSessionOptions::default()
});
let mut receiver = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 403,
initial_sequence: 100,
initial_sequence: Some(100),
..CultNetRudpSessionOptions::default()
});
let connect = sender.create_connect(0, Vec::new())?;
@ -2797,16 +2652,196 @@ fn rudp_ordered_hold_buffer_is_bounded() -> Result<()> {
Ok(())
}
fn rudp_ordered_pair() -> Result<(CultNetRudpSession, CultNetRudpSession)> {
let mut client = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 410,
initial_sequence: Some(1),
..CultNetRudpSessionOptions::default()
});
let mut server = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 410,
initial_sequence: Some(500),
..CultNetRudpSessionOptions::default()
});
let connect = client.create_connect(0, Vec::new())?;
let accept = server.accept_connect(&connect, 0, Vec::new())?;
client.receive(&accept, 0)?;
Ok((client, server))
}
fn rudp_send_options(ordered: bool) -> CultNetRudpSendOptions {
CultNetRudpSendOptions {
reliable: true,
ordered,
..CultNetRudpSendOptions::default()
}
}
fn rudp_delivered_names(delivered: &[cultnet_rs::CultNetRudpDeliveredFrame]) -> Vec<String> {
delivered
.iter()
.map(|frame| String::from_utf8(frame.payload.clone()).unwrap())
.collect()
}
#[test]
fn rudp_ordered_frame_waits_for_a_gap_filled_by_another_channel() -> Result<()> {
let (mut sender, mut receiver) = rudp_ordered_pair()?;
let o1 = sender.send("schema", b"o1".to_vec(), rudp_send_options(true))?;
let u = sender.send("media", b"u".to_vec(), rudp_send_options(false))?;
let o2 = sender.send("schema", b"o2".to_vec(), rudp_send_options(true))?;
let o3 = sender.send("schema", b"o3".to_vec(), rudp_send_options(true))?;
let o4 = sender.send("schema", b"o4".to_vec(), rudp_send_options(true))?;
assert_eq!(rudp_delivered_names(&receiver.receive(&o1, 1)?.delivered), ["o1"]);
// `u` is lost, so nothing behind it may be delivered.
assert!(receiver.receive(&o2, 2)?.delivered.is_empty());
assert!(receiver.receive(&o3, 3)?.delivered.is_empty());
assert!(receiver.receive(&o4, 4)?.delivered.is_empty());
// The other channel's packet fills the gap and releases every ordered
// frame behind it in the same call.
assert_eq!(
rudp_delivered_names(&receiver.receive(&u, 5)?.delivered),
["u", "o2", "o3", "o4"]
);
Ok(())
}
#[test]
fn rudp_two_ordered_channels_release_each_other_in_sequence_order() -> Result<()> {
let (mut sender, mut receiver) = rudp_ordered_pair()?;
let a1 = sender.send("schema", b"A1".to_vec(), rudp_send_options(true))?;
let b1 = sender.send("other", b"B1".to_vec(), rudp_send_options(true))?;
let a2 = sender.send("schema", b"A2".to_vec(), rudp_send_options(true))?;
let a3 = sender.send("schema", b"A3".to_vec(), rudp_send_options(true))?;
let mut delivered = rudp_delivered_names(&receiver.receive(&a1, 1)?.delivered);
delivered.extend(rudp_delivered_names(&receiver.receive(&a2, 2)?.delivered));
delivered.extend(rudp_delivered_names(&receiver.receive(&a3, 3)?.delivered));
assert_eq!(delivered, ["A1"], "B1 is missing, so A2 and A3 wait");
assert_eq!(
rudp_delivered_names(&receiver.receive(&b1, 4)?.delivered),
["B1", "A2", "A3"]
);
Ok(())
}
#[test]
fn rudp_channel_first_used_after_other_traffic_loses_nothing() -> Result<()> {
let (mut sender, mut receiver) = rudp_ordered_pair()?;
let c1 = sender.send("late", b"C1".to_vec(), rudp_send_options(true))?;
let c2 = sender.send("late", b"C2".to_vec(), rudp_send_options(true))?;
let x = sender.send("schema", b"X".to_vec(), rudp_send_options(true))?;
// X, then C2, arrive while C1 is still in flight.
let mut delivered = rudp_delivered_names(&receiver.receive(&x, 1)?.delivered);
delivered.extend(rudp_delivered_names(&receiver.receive(&c2, 2)?.delivered));
assert!(delivered.is_empty(), "C1 is missing, so nothing is delivered");
assert_eq!(
rudp_delivered_names(&receiver.receive(&c1, 3)?.delivered),
["C1", "C2", "X"]
);
Ok(())
}
#[test]
fn rudp_accept_seeds_the_watermark_of_the_connecting_side() -> Result<()> {
let (mut client, mut server) = rudp_ordered_pair()?;
let s1 = server.send("schema", b"s1".to_vec(), rudp_send_options(true))?;
let s2 = server.send("schema", b"s2".to_vec(), rudp_send_options(true))?;
assert!(client.receive(&s2, 1)?.delivered.is_empty());
assert_eq!(
rudp_delivered_names(&client.receive(&s1, 2)?.delivered),
["s1", "s2"]
);
Ok(())
}
#[test]
fn rudp_reset_forgets_held_frames_and_reseeds_the_watermark_from_the_next_handshake() -> Result<()> {
let (mut sender, mut receiver) = rudp_ordered_pair()?;
sender.send("schema", b"s1".to_vec(), rudp_send_options(true))?;
let s2 = sender.send("schema", b"s2".to_vec(), rudp_send_options(true))?;
assert!(receiver.receive(&s2, 1)?.delivered.is_empty());
receiver.reset_peer_state();
let mut next = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 410,
initial_sequence: Some(100),
..CultNetRudpSessionOptions::default()
});
let accept = receiver.accept_connect(&next.create_connect(0, Vec::new())?, 2, Vec::new())?;
next.receive(&accept, 2)?;
let d1 = next.send("schema", b"d1".to_vec(), rudp_send_options(true))?;
let d2 = next.send("schema", b"d2".to_vec(), rudp_send_options(true))?;
// The held s2 died with the reset, and the watermark follows the new peer.
assert!(receiver.receive(&d2, 3)?.delivered.is_empty());
assert_eq!(rudp_delivered_names(&receiver.receive(&d1, 4)?.delivered), ["d1", "d2"]);
Ok(())
}
#[test]
fn rudp_large_ordered_frame_without_a_gap_is_never_held_or_refused() -> Result<()> {
let (mut sender, mut receiver) = rudp_ordered_pair()?;
let packets = sender.send_many(
"schema",
vec![7u8; 5 * 1024 * 1024],
rudp_send_options(true),
Some(60_000),
)?;
let mut queue = packets;
let mut delivered = Vec::new();
while !queue.is_empty() {
let mut next = Vec::new();
for packet in &queue {
delivered.extend(receiver.receive(packet, 1)?.delivered);
let ack = receiver.create_ack_for_received(packet.sequence);
next.extend(sender.receive(&ack, 1)?.ready_to_send);
}
queue = next;
}
assert_eq!(delivered.len(), 1);
assert_eq!(delivered[0].payload.len(), 5 * 1024 * 1024);
Ok(())
}
#[test]
fn rudp_ordered_hold_buffer_is_bounded_in_bytes() -> Result<()> {
let (mut sender, mut receiver) = rudp_ordered_pair()?;
let withheld = sender.send("schema", b"first".to_vec(), rudp_send_options(true))?;
sender.receive(&receiver.create_ack_for(withheld.sequence), 1)?;
// Four 1 MiB frames fit the 4 MiB hold; the fifth does not.
let mut refused_at = None;
for step in 0..8u32 {
let packet = sender.send("schema", vec![7u8; 1024 * 1024], rudp_send_options(true))?;
match receiver.receive(&packet, 2) {
Ok(result) => {
assert!(result.delivered.is_empty(), "the gap holds every frame");
sender.receive(&receiver.create_ack_for(packet.sequence), 2)?;
}
Err(refusal) => {
assert!(refusal.to_string().contains("ordered hold buffer is full"), "{refusal}");
refused_at = Some(step);
break;
}
}
}
assert_eq!(refused_at, Some(4));
Ok(())
}
#[test]
fn rudp_far_ahead_sequence_cannot_starve_the_session() -> Result<()> {
let mut sender = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 404,
initial_sequence: 1,
initial_sequence: Some(1),
..CultNetRudpSessionOptions::default()
});
let mut receiver = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 404,
initial_sequence: 100,
initial_sequence: Some(100),
..CultNetRudpSessionOptions::default()
});
let connect = sender.create_connect(0, Vec::new())?;
@ -2855,12 +2890,12 @@ fn rudp_reliable_sequence_exhaustion_is_reported_not_fatal() -> Result<()> {
connection_id: 405,
// One sequence left: the handshake takes it, so the next send is the
// first allocation with nowhere to go.
initial_sequence: u32::MAX - 1,
initial_sequence: Some(u32::MAX - 1),
..CultNetRudpSessionOptions::default()
});
let mut receiver = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 405,
initial_sequence: 100,
initial_sequence: Some(100),
..CultNetRudpSessionOptions::default()
});
let connect = sender.create_connect(0, Vec::new())?;
@ -2924,7 +2959,7 @@ fn tcp_framed_receive_refuses_a_length_beyond_the_advertised_maximum() -> Result
fn rudp_reliable_expiry_drops_late_packets_and_leaves_durable_sends_alone() -> Result<()> {
let mut sender = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 7,
initial_sequence: 10,
initial_sequence: Some(10),
resend_delay_ms: 100,
..CultNetRudpSessionOptions::default()
});
@ -2988,7 +3023,7 @@ fn rudp_reliable_expiry_drops_late_packets_and_leaves_durable_sends_alone() -> R
fn rudp_reliable_expiry_also_reclaims_the_send_backlog() -> Result<()> {
let mut sender = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 9,
initial_sequence: 10,
initial_sequence: Some(10),
resend_delay_ms: 100,
..CultNetRudpSessionOptions::default()
});
@ -3037,30 +3072,13 @@ fn rudp_reliable_expiry_also_reclaims_the_send_backlog() -> Result<()> {
Ok(())
}
fn accept_rudp_session(session: &mut CultNetRudpSession, connection_id: u32) -> Result<()> {
session.receive(
&CultNetRudpPacket {
packet_type: CultNetRudpPacketType::Accept,
connection_id,
sequence: 0,
ack: 0,
ack_mask: 0,
channel_id: "control".into(),
reliable: false,
ordered: false,
sequenced: false,
fragment_id: 0,
fragment_index: 0,
fragment_count: 0,
payload: Vec::new(),
},
0,
)?;
fn accept_rudp_session(session: &mut CultNetRudpSession, _connection_id: u32) -> Result<()> {
session.assume_connected(0);
Ok(())
}
#[test]
fn rudp_profile_expires_only_the_media_channels() -> Result<()> {
fn rudp_profile_expires_only_the_media_channel() -> Result<()> {
let socket = std::net::UdpSocket::bind("127.0.0.1:0")?;
let peer = socket.local_addr()?;
let options = CultNetRudpSocketTransportOptions::client("expiry-profile", socket, peer, 11);
@ -3074,14 +3092,12 @@ fn rudp_profile_expires_only_the_media_channels() -> Result<()> {
.map(|channel| (channel.channel_id.as_str(), channel.reliable_expire_after_ms))
.collect::<Vec<_>>();
for media_channel in ["media", "audio"] {
assert!(
channels.contains(&(media_channel, Some(75))),
"the {media_channel} channel advertises its expiry so a peer can see it: {channels:?}"
);
}
assert!(
channels.contains(&("media", Some(75))),
"the media channel advertises its expiry so a peer can see it: {channels:?}"
);
for (channel_id, expiry) in &channels {
if !matches!(*channel_id, "media" | "audio") {
if *channel_id != "media" {
assert_eq!(
*expiry, None,
"state channel {channel_id} must never expire a reliable send"
@ -3094,12 +3110,12 @@ fn rudp_profile_expires_only_the_media_channels() -> Result<()> {
fn connected_rudp_pair() -> Result<(CultNetRudpSession, CultNetRudpSession)> {
let mut client = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 311,
initial_sequence: 1,
initial_sequence: Some(1),
..CultNetRudpSessionOptions::default()
});
let mut server = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 311,
initial_sequence: 100,
initial_sequence: Some(100),
..CultNetRudpSessionOptions::default()
});
let connect = client.create_connect(0, Vec::new())?;

40
packages/cultnet-rs/tests/error_message.rs
View file

@ -118,6 +118,46 @@ fn cases() -> Vec<Case> {
details: None,
},
},
Case {
name: "variant_unsupported_subscription_ended",
// C#: ForVariantUnsupported("variant unsupported here", "sub-1")
csharp_hex: "85AD736368656D6156657273696F6EB063756C746E65742E6572726F722E7630A56572726F72B876617269616E7420756E737570706F727465642068657265AB726F7574696E6748696E74C0A4636F6465B376617269616E745F756E737570706F72746564A764657461696C7384A56669656C64AE737562736372697074696F6E4964A576616C7565A57375622D31A461734F66C0A763757272656E74C0",
message: || CultNetMessage::Error {
error: "variant unsupported here".to_string(),
code: Some(CultNetErrorCode::VariantUnsupported),
details: Some(CultNetErrorDetails {
field: Some("subscriptionId".to_string()),
value: Some("sub-1".to_string()),
as_of: None,
current: None,
}),
},
},
Case {
name: "variant_unsupported_no_subscription",
// C#: ForVariantUnsupported("variant unsupported here") - no subscription ended, so Details stays null.
csharp_hex: "85AD736368656D6156657273696F6EB063756C746E65742E6572726F722E7630A56572726F72B876617269616E7420756E737570706F727465642068657265AB726F7574696E6748696E74C0A4636F6465B376617269616E745F756E737570706F72746564A764657461696C73C0",
message: || CultNetMessage::Error {
error: "variant unsupported here".to_string(),
code: Some(CultNetErrorCode::VariantUnsupported),
details: None,
},
},
Case {
name: "unowned_schema",
// C#: ForUnownedSchema(new CultNetUnownedSchemaException("some.schema.v1", new CultRecordKey("some:key")))
csharp_hex: "85AD736368656D6156657273696F6EB063756C746E65742E6572726F722E7630A56572726F72D95C756E6F776E65645F736368656D613A204E6F207368617264206F776E7320736368656D612027736F6D652E736368656D612E763127206B65792027736F6D653A6B6579273B207468652077726974652077617320726566757365642EAB726F7574696E6748696E74C0A4636F6465AE756E6F776E65645F736368656D61A764657461696C7384A56669656C64A8736368656D614964A576616C7565AE736F6D652E736368656D612E7631A461734F66C0A763757272656E74C0",
message: || CultNetMessage::Error {
error: "unowned_schema: No shard owns schema 'some.schema.v1' key 'some:key'; the write was refused.".to_string(),
code: Some(CultNetErrorCode::UnownedSchema),
details: Some(CultNetErrorDetails {
field: Some("schemaId".to_string()),
value: Some("some.schema.v1".to_string()),
as_of: None,
current: None,
}),
},
},
]
}

3
packages/cultnet-rs/tests/impairment/burst-4.toml Normal file
View file

@ -0,0 +1,3 @@
burst_every = 200
burst_length = 4
max_scheduled = 4096

3
packages/cultnet-rs/tests/impairment/burst-8.toml Normal file
View file

@ -0,0 +1,3 @@
burst_every = 200
burst_length = 8
max_scheduled = 4096

2
packages/cultnet-rs/tests/impairment/clean.toml Normal file
View file

@ -0,0 +1,2 @@
# No impairment. The proxy itself remains in the measured path.
max_scheduled = 4096

3
packages/cultnet-rs/tests/impairment/delay-jitter.toml Normal file
View file

@ -0,0 +1,3 @@
delay_ms = 5
jitter_ms = 10
max_scheduled = 4096

2
packages/cultnet-rs/tests/impairment/duplicate.toml Normal file
View file

@ -0,0 +1,2 @@
duplicate_every = 50
max_scheduled = 4096

2
packages/cultnet-rs/tests/impairment/loss-1pct.toml Normal file
View file

@ -0,0 +1,2 @@
loss_basis_points = 100
max_scheduled = 4096

2
packages/cultnet-rs/tests/impairment/loss-3pct.toml Normal file
View file

@ -0,0 +1,2 @@
loss_basis_points = 300
max_scheduled = 4096

2
packages/cultnet-rs/tests/impairment/loss-5pct.toml Normal file
View file

@ -0,0 +1,2 @@
loss_basis_points = 500
max_scheduled = 4096

37
packages/cultnet-rs/tests/impairment/loss-matrix.txt Normal file
View file

@ -0,0 +1,37 @@
profile chunks frames whole/no-fec whole/fec recovered beyond
clean 8 200 200 200 0 0
clean 16 200 200 200 0 0
clean 17 200 200 200 0 0
clean 24 200 200 200 0 0
clean 32 200 200 200 0 0
clean 48 200 200 200 0 0
loss-1pct 8 200 183 200 17 0
loss-1pct 16 200 170 200 30 0
loss-1pct 17 200 170 200 31 0
loss-1pct 24 200 154 200 48 0
loss-1pct 32 200 144 200 59 0
loss-1pct 48 200 137 200 75 0
loss-3pct 8 200 158 199 41 1
loss-3pct 16 200 126 200 74 0
loss-3pct 17 200 119 199 90 0
loss-3pct 24 200 101 200 116 0
loss-3pct 32 200 78 200 147 0
loss-3pct 48 200 57 200 208 0
loss-5pct 8 200 130 197 67 3
loss-5pct 16 200 93 199 106 1
loss-5pct 17 200 92 198 132 1
loss-5pct 24 200 59 198 180 2
loss-5pct 32 200 41 200 214 0
loss-5pct 48 200 21 199 318 1
burst-4 8 200 181 191 10 9
burst-4 16 200 161 200 39 0
burst-4 17 200 175 200 46 0
burst-4 24 200 161 200 68 0
burst-4 32 200 121 200 118 0
burst-4 48 200 121 200 177 0
burst-8 8 200 181 191 10 0
burst-8 16 200 161 181 20 19
burst-8 17 200 171 183 20 30
burst-8 24 200 161 171 19 49
burst-8 32 200 121 200 118 0
burst-8 48 200 121 200 197 0

3
packages/cultnet-rs/tests/impairment/reorder.toml Normal file
View file

@ -0,0 +1,3 @@
reorder_every = 20
reorder_delay_ms = 12
max_scheduled = 4096

3
packages/cultnet-rs/tests/impairment/stall-250ms.toml Normal file
View file

@ -0,0 +1,3 @@
stall_at_ms = 10000
stall_for_ms = 250
max_scheduled = 4096

219
packages/cultnet-rs/tests/impairment_policy.rs Normal file
View file

@ -0,0 +1,219 @@
//! The impairment policy is deterministic, or a loss experiment cannot be
//! replayed. These pin each decision the proxy example makes, against the same
//! `policy.rs` the example compiles.
#[allow(dead_code)]
#[path = "../examples/cultnet_impair/policy.rs"]
mod policy;
use std::net::SocketAddr;
use policy::{Direction, Profile, Scheduler};
fn client() -> SocketAddr {
"127.0.0.1:1234".parse().unwrap()
}
#[test]
fn iid_decisions_are_seed_reproducible() {
let profile = Profile {
loss_basis_points: 2500,
..Default::default()
};
let mut a = Scheduler::new(profile.clone(), 42);
let mut b = Scheduler::new(profile, 42);
let left: Vec<_> = (0..100).map(|_| a.decide(0)).collect();
let right: Vec<_> = (0..100).map(|_| b.decide(0)).collect();
assert_eq!(left, right);
assert!(left.iter().any(|d| d.drop));
assert!(left.iter().any(|d| !d.drop));
}
#[test]
fn burst_drop_is_exact() {
let profile = Profile {
burst_every: 5,
burst_length: 3,
..Default::default()
};
let mut scheduler = Scheduler::new(profile, 1);
let drops: Vec<_> = (1..=9).filter(|_| scheduler.decide(0).drop).collect();
assert_eq!(drops, vec![5, 6, 7], "a burst starts on the Nth datagram and runs burst_length");
}
#[test]
fn duplicate_delay_and_reorder_are_deterministic() {
let profile = Profile {
duplicate_every: 2,
reorder_every: 3,
reorder_delay_ms: 20,
delay_ms: 5,
..Default::default()
};
let mut scheduler = Scheduler::new(profile, 1);
assert_eq!(scheduler.decide(0).copies, 1);
assert_eq!(scheduler.decide(0).copies, 2);
let third = scheduler.decide(0);
assert!(third.reordered);
assert_eq!(third.delay_ms, 25);
}
#[test]
fn stall_drops_only_inside_window() {
let profile = Profile {
stall_at_ms: 100,
stall_for_ms: 50,
..Default::default()
};
let mut scheduler = Scheduler::new(profile, 1);
assert!(!scheduler.decide(99).drop);
let stalled = scheduler.decide(100);
assert!(stalled.drop && stalled.stalled);
assert!(scheduler.decide(149).drop);
assert!(!scheduler.decide(150).drop);
}
#[test]
fn reorder_delay_places_packet_behind_later_packet() {
let profile = Profile {
reorder_every: 2,
reorder_delay_ms: 20,
..Default::default()
};
let mut scheduler = Scheduler::new(profile, 1);
scheduler.admit(0, Direction::ClientToUpstream, b"first", client());
scheduler.admit(0, Direction::ClientToUpstream, b"held", client());
scheduler.admit(1, Direction::ClientToUpstream, b"later", client());
assert_eq!(scheduler.pop_due(1).unwrap().bytes, b"first");
assert_eq!(scheduler.pop_due(1).unwrap().bytes, b"later");
assert!(scheduler.pop_due(19).is_none());
assert_eq!(scheduler.pop_due(20).unwrap().bytes, b"held");
}
#[test]
fn scheduled_queue_is_bounded_and_counts_overflow() {
let profile = Profile {
delay_ms: 100,
max_scheduled: 2,
..Default::default()
};
let mut scheduler = Scheduler::new(profile, 1);
for value in 0..5 {
scheduler.admit(0, Direction::ClientToUpstream, &[value], client());
}
assert_eq!(scheduler.queue.len(), 2);
assert_eq!(scheduler.stats.max_queue, 2);
assert_eq!(scheduler.stats.queue_overflow, 3);
assert_eq!(scheduler.stats.dropped, 3);
}
fn profile_path(name: &str) -> std::path::PathBuf {
std::path::Path::new(env!("CARGO_MANIFEST_DIR"))
.join("tests/impairment")
.join(format!("{name}.toml"))
}
#[test]
fn every_committed_profile_loads_with_its_named_meaning() {
let load = |name: &str| Profile::load(&profile_path(name)).unwrap_or_else(|e| panic!("{name}: {e}"));
assert_eq!(load("clean"), Profile::default());
assert_eq!(load("loss-1pct").loss_basis_points, 100);
assert_eq!(load("loss-3pct").loss_basis_points, 300);
assert_eq!(load("loss-5pct").loss_basis_points, 500);
let burst4 = load("burst-4");
assert_eq!((burst4.burst_every, burst4.burst_length), (200, 4));
let burst8 = load("burst-8");
assert_eq!((burst8.burst_every, burst8.burst_length), (200, 8));
let jitter = load("delay-jitter");
assert_eq!((jitter.delay_ms, jitter.jitter_ms), (5, 10));
assert_eq!(load("duplicate").duplicate_every, 50);
let reorder = load("reorder");
assert_eq!((reorder.reorder_every, reorder.reorder_delay_ms), (20, 12));
let stall = load("stall-250ms");
assert_eq!((stall.stall_at_ms, stall.stall_for_ms), (10_000, 250));
}
#[test]
fn a_malformed_profile_is_refused_with_its_line() {
let dir = tempfile::tempdir().unwrap();
let write = |name: &str, text: &str| {
let path = dir.path().join(name);
std::fs::write(&path, text).unwrap();
Profile::load(&path).unwrap_err()
};
assert!(write("unknown.toml", "jitter = 3
").contains("unknown profile key jitter"));
assert!(write("word.toml", "delay_ms = soon
").contains("unsigned integer"));
assert!(write("shape.toml", "delay_ms
").contains("key = value"));
assert!(write("loss.toml", "loss_basis_points = 10001
").contains("<= 10000"));
assert!(write("queue.toml", "max_scheduled = 0
").contains("greater than zero"));
assert!(write("line.toml", "
delay_ms = x
").contains(":3:"));
}
#[test]
fn admission_counts_what_it_did_to_each_datagram() {
let mut scheduler = Scheduler::new(
Profile { duplicate_every: 2, reorder_every: 3, reorder_delay_ms: 10, ..Default::default() },
1,
);
for value in 0..6 {
scheduler.admit(0, Direction::ClientToUpstream, &[value], client());
}
// Datagrams 2, 4 and 6 are duplicated, 3 and 6 are reordered (6 is both).
assert_eq!(scheduler.stats.received, 6);
assert_eq!(scheduler.stats.duplicated, 3);
assert_eq!(scheduler.stats.reordered, 2);
assert_eq!(scheduler.queue.len(), 9);
let mut stalled = Scheduler::new(Profile { stall_at_ms: 0, stall_for_ms: 10, ..Default::default() }, 1);
stalled.admit(5, Direction::UpstreamToClient, b"x", client());
assert_eq!((stalled.stats.stalled, stalled.stats.dropped, stalled.queue.len()), (1, 1, 0));
}
#[test]
fn jitter_is_seeded_uniform_within_its_bound() {
let profile = Profile { delay_ms: 7, jitter_ms: 5, ..Default::default() };
let mut scheduler = Scheduler::new(profile, 9);
let delays: Vec<u64> = (0..300).map(|_| scheduler.decide(0).delay_ms).collect();
assert!(delays.iter().all(|delay| (7..=12).contains(delay)), "{delays:?}");
for delay in 7..=12 {
assert!(delays.contains(&delay), "delay {delay} never drawn");
}
}
#[test]
fn the_loss_ceiling_is_inclusive() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("all.toml");
std::fs::write(&path, "loss_basis_points = 10000\n").unwrap();
let profile = Profile::load(&path).unwrap();
let mut scheduler = Scheduler::new(profile, 1);
assert!((0..50).all(|_| scheduler.decide(0).drop), "10000 basis points drops everything");
}
/// The seeded stream is part of the contract: the same seed must drop the same
/// datagrams on every run, on every host. 100,000 draws of xorshift64 from seed
/// 42, taken modulo 10,000 and compared against 100, give 1,008.
#[test]
fn seeded_loss_drops_the_exact_datagrams_the_stream_dictates() {
let mut scheduler = Scheduler::new(Profile { loss_basis_points: 100, ..Default::default() }, 42);
let dropped = (0..100_000).filter(|_| scheduler.decide(0).drop).count();
assert_eq!(dropped, 1_008);
}
#[test]
fn datagrams_due_together_leave_in_arrival_order() {
let mut scheduler = Scheduler::new(Profile { delay_ms: 5, ..Default::default() }, 1);
for value in 0..4_u8 {
scheduler.admit(0, Direction::ClientToUpstream, &[value], client());
}
let order: Vec<u8> = std::iter::from_fn(|| scheduler.pop_due(5)).map(|item| item.bytes[0]).collect();
assert_eq!(order, vec![0, 1, 2, 3]);
}

118
packages/cultnet-rs/tests/media_channel_delivery.rs
View file

@ -90,3 +90,121 @@ fn the_other_channels_keep_their_fixed_delivery() {
assert_eq!(delivery_of("schema"), CultNetTransportDelivery::Reliable);
assert_eq!(delivery_of("realtime"), CultNetTransportDelivery::Unreliable);
}
// ---------------------------------------------------------------------------
// What the media channel puts on the wire
// ---------------------------------------------------------------------------
use std::thread::sleep;
use std::time::Duration;
use cultnet_rs::{
CultNetRudpPacket, CultNetRudpSession, CultNetRudpSessionOptions, decode_rudp_packet,
encode_rudp_packet,
};
/// A connected client transport and the raw socket standing in for its peer.
struct Link {
client: CultNetRudpSocketTransportConnection,
peer: UdpSocket,
}
impl Link {
fn open(delivery: Option<CultNetTransportDelivery>, expire_after_ms: Option<u64>) -> Self {
let peer = UdpSocket::bind("127.0.0.1:0").expect("binds");
peer.set_read_timeout(Some(Duration::from_millis(500))).unwrap();
let socket = UdpSocket::bind("127.0.0.1:0").expect("binds");
socket.set_read_timeout(Some(Duration::from_millis(300))).unwrap();
let mut options = CultNetRudpSocketTransportOptions::client(
"test-media",
socket,
peer.local_addr().unwrap(),
7,
);
options.media_delivery = delivery;
options.media_reliable_expire_after_ms = expire_after_ms;
options.resend_delay_ms = 10;
let mut client = CultNetRudpSocketTransportConnection::new(options).expect("opens");
client.connect(Vec::new()).expect("sends connect");
let mut wire = vec![0_u8; 2_048];
let (received, client_addr) = peer.recv_from(&mut wire).expect("the connect arrives");
let connect = decode_rudp_packet(&wire[..received]).expect("decodes");
let mut server = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 7,
..CultNetRudpSessionOptions::default()
});
let accept = server.accept_connect(&connect, 0, Vec::new()).expect("accepts");
peer.send_to(&encode_rudp_packet(&accept).unwrap(), client_addr).unwrap();
client.receive_once().expect("takes the accept");
assert!(client.connected());
let mut link = Self { client, peer };
link.datagrams(Duration::from_millis(100));
link
}
/// Everything the peer receives within `window`.
fn datagrams(&mut self, window: Duration) -> Vec<CultNetRudpPacket> {
self.peer.set_read_timeout(Some(window)).unwrap();
let mut packets = Vec::new();
let mut wire = vec![0_u8; 2_048];
while let Ok((received, _)) = self.peer.recv_from(&mut wire) {
packets.push(decode_rudp_packet(&wire[..received]).expect("decodes"));
}
packets
}
fn send_media(&mut self, window: Duration) -> Vec<CultNetRudpPacket> {
self.client.send("media", vec![1, 2, 3]).expect("sends");
self.datagrams(window)
}
}
fn media_only(packets: Vec<CultNetRudpPacket>) -> Vec<CultNetRudpPacket> {
packets.into_iter().filter(|packet| packet.channel_id == "media").collect()
}
/// The channel the profile describes as reliable is sent reliable, unordered
/// and unsequenced; asking for an unreliable lane sends it unreliable.
#[test]
fn the_media_channel_is_sent_with_the_delivery_the_profile_advertises() {
let mut reliable = Link::open(None, None);
let sent = media_only(reliable.send_media(Duration::from_millis(100)));
assert_eq!(sent.len(), 1);
assert!(sent[0].reliable, "the default media lane is reliable");
assert!(!sent[0].ordered && !sent[0].sequenced, "media is neither ordered nor sequenced");
let mut unreliable = Link::open(Some(CultNetTransportDelivery::Unreliable), None);
let sent = media_only(unreliable.send_media(Duration::from_millis(100)));
assert_eq!(sent.len(), 1);
assert!(!sent[0].reliable, "an unreliable media lane is not retransmitted");
assert!(!sent[0].ordered && !sent[0].sequenced);
}
/// A reliable media packet with an expiry is dropped once it has passed, and
/// one without is retransmitted until acknowledged. The session reclaims
/// expired packets on the next send, so the probe sends a second packet.
fn first_packet_is_resent_after_a_later_send(expire_after_ms: Option<u64>) -> bool {
let mut link = Link::open(None, expire_after_ms);
let first = media_only(link.send_media(Duration::from_millis(50)));
assert_eq!(first.len(), 1);
sleep(Duration::from_millis(150));
assert_eq!(media_only(link.send_media(Duration::from_millis(50))).len(), 1);
sleep(Duration::from_millis(15));
link.client.poll_resends().unwrap();
media_only(link.datagrams(Duration::from_millis(100)))
.iter()
.any(|packet| packet.sequence == first[0].sequence)
}
#[test]
fn the_media_channels_expiry_is_applied_to_its_packets() {
assert!(
!first_packet_is_resent_after_a_later_send(Some(20)),
"an expired media packet is dropped, not retransmitted"
);
assert!(
first_packet_is_resent_after_a_later_send(None),
"without an expiry the unacknowledged packet is retransmitted"
);
}

1143
packages/cultnet-rs/tests/media_fec.rs Normal file
View file

File diff suppressed because it is too large Load diff

208
packages/cultnet-rs/tests/media_fec_loss.rs Normal file
View file

@ -0,0 +1,208 @@
//! Video frames protected by the media FEC codec, sent through the impairment
//! schedules `cultnet_impair` applies to real sockets, with no sockets: each
//! record of a frame in send order takes the next datagram decision, and what
//! survives is handed to `recover_video_block`.
//!
//! Two kinds of claim. The guarantee is asserted: a block recovers exactly when
//! at most `m` of its records were lost, and a burst short enough that the send
//! order spreads it under `m` per block always leaves every frame whole. The
//! recovered-frame ratios are evidence, not assertions; `--nocapture` prints the
//! table and `tests/impairment/loss-matrix.txt` is that output (200 frames per cell, seed 20260930).
#[allow(dead_code)]
#[path = "../examples/cultnet_impair/policy.rs"]
mod policy;
use std::fmt::Write as _;
use std::path::Path;
use cultnet_rs::{
GameCultMediaVideoAccessUnitRecord, GameCultMediaWireRecord, MediaFecError, MediaFecPolicy,
MediaWireProvenance, protect_video_frame, recover_video_block,
};
use policy::{Profile, Scheduler};
const POLICY: MediaFecPolicy = MediaFecPolicy::STANDARD;
const SEED: u64 = 20_260_930;
const FRAMES: u64 = 200;
fn provenance() -> MediaWireProvenance<'static> {
MediaWireProvenance {
stored_at: "unix:1700000000000",
runtime_id: "raven-muninn-primary",
role: "muninn.media",
producer: "muninn",
}
}
fn frame(frame_id: u64, chunk_count: u16) -> Vec<GameCultMediaVideoAccessUnitRecord> {
(0..chunk_count)
.map(|index| GameCultMediaVideoAccessUnitRecord {
stream_id: "muninn.raven.av.rudp".to_string(),
session_id: "session-1".to_string(),
frame_id,
codec: "h264".to_string(),
pts_ticks: frame_id as i64 * 3_000,
duration_ticks: 3_000,
timebase_num: 1,
timebase_den: 90_000,
keyframe: false,
dependency_frame_id: frame_id.checked_sub(1),
deadline_ticks: frame_id as i64 * 3_000 + 22_500,
chunk_index: index,
chunk_count,
payload: (0..if index + 1 == chunk_count { 60 } else { 100 })
.map(|byte| (frame_id as usize * 31 + usize::from(index) * 7 + byte) as u8)
.collect(),
})
.collect()
}
#[derive(Default)]
struct Tally {
frames: u64,
whole_without_parity: u64,
whole_with_parity: u64,
blocks_recovered: u64,
blocks_beyond_repair: u64,
}
/// Runs one shape through one profile's schedule and asserts the per-block
/// guarantee on every block of every frame.
fn run(profile: &Profile, chunk_count: u16) -> Tally {
let mut scheduler = Scheduler::new(profile.clone(), SEED);
let sizes = POLICY.video_block_sizes(chunk_count);
let mut tally = Tally::default();
for frame_id in 0..FRAMES {
let original = frame(frame_id, chunk_count);
let records = protect_video_frame(&original, &POLICY, provenance()).unwrap();
let survivors: Vec<GameCultMediaWireRecord> = records
.into_iter()
.filter(|_| !scheduler.decide(0).drop)
.collect();
let mut whole = true;
let mut whole_without_parity = true;
let mut start = 0_u16;
for (block, &size) in sizes.iter().enumerate() {
let m = usize::from(POLICY.video_parity_shards(size));
let mut data = Vec::new();
let mut parity = Vec::new();
for record in &survivors {
match record {
GameCultMediaWireRecord::Video(chunk)
if chunk.chunk_index >= start && chunk.chunk_index < start + size =>
{
data.push(chunk.clone())
}
GameCultMediaWireRecord::VideoParity(shard) if usize::from(shard.block_index) == block => {
parity.push(shard.clone())
}
_ => {}
}
}
let lost = usize::from(size) + m - data.len() - parity.len();
let missing_data = usize::from(size) - data.len();
whole_without_parity &= missing_data == 0;
let block_whole = if missing_data == 0 {
true
} else if parity.is_empty() {
assert!(lost > m, "no parity survived, so more than m shards were lost");
false
} else {
match recover_video_block(&parity, &data) {
Ok(recovered) => {
assert!(lost <= m, "block {block} of frame {frame_id} recovered {lost} losses with m = {m}");
assert_eq!(recovered.len(), missing_data);
for chunk in &recovered {
assert_eq!(
chunk, &original[usize::from(chunk.chunk_index)],
"a recovered chunk must be the chunk that was sent"
);
}
tally.blocks_recovered += 1;
true
}
Err(MediaFecError::BeyondRepair { .. }) => {
assert!(lost > m, "block {block} of frame {frame_id} refused a recoverable {lost} losses with m = {m}");
tally.blocks_beyond_repair += 1;
false
}
Err(other) => panic!("{other}"),
}
};
whole &= block_whole;
start += size;
}
tally.frames += 1;
tally.whole_with_parity += u64::from(whole);
tally.whole_without_parity += u64::from(whole_without_parity);
assert!(whole || !whole_without_parity, "parity never makes a whole frame worse");
}
tally
}
fn profile(name: &str) -> Profile {
Profile::load(&Path::new(env!("CARGO_MANIFEST_DIR")).join(format!("tests/impairment/{name}.toml")))
.unwrap()
}
/// Every shape here splits into equal blocks, so a burst of `L` consecutive
/// datagrams costs each of `B` blocks at most `ceil(L / B)` of its records.
const EQUAL_BLOCK_SHAPES: [u16; 5] = [8, 16, 24, 32, 48];
#[test]
fn the_loss_matrix() {
let cells = ["clean", "loss-1pct", "loss-3pct", "loss-5pct", "burst-4", "burst-8"];
let shapes = [8_u16, 16, 17, 24, 32, 48];
let mut table = String::new();
writeln!(
table,
"{:<10} {:>6} {:>7} {:>13} {:>12} {:>10} {:>9}",
"profile", "chunks", "frames", "whole/no-fec", "whole/fec", "recovered", "beyond"
)
.unwrap();
for cell in cells {
let loaded = profile(cell);
for &chunks in &shapes {
let tally = run(&loaded, chunks);
writeln!(
table,
"{:<10} {:>6} {:>7} {:>13} {:>12} {:>10} {:>9}",
cell,
chunks,
tally.frames,
tally.whole_without_parity,
tally.whole_with_parity,
tally.blocks_recovered,
tally.blocks_beyond_repair
)
.unwrap();
if cell == "clean" {
assert_eq!(tally.whole_with_parity, tally.frames, "a clean link loses nothing");
}
let burst = match cell {
"burst-4" => 4_usize,
"burst-8" => 8,
_ => continue,
};
if EQUAL_BLOCK_SHAPES.contains(&chunks) {
let sizes = POLICY.video_block_sizes(chunks);
let per_block = burst.div_ceil(sizes.len());
let m = usize::from(POLICY.video_parity_shards(sizes[0]));
if per_block <= m {
assert_eq!(
tally.whole_with_parity, tally.frames,
"{cell} over {chunks} chunks costs each of {} blocks at most {per_block} of m = {m}: no frame may be lost",
sizes.len()
);
}
}
}
}
println!("{table}");
}

73
packages/cultnet-rs/tests/media_stream_contracts.rs
View file

@ -5,9 +5,10 @@
use cultcache_rs::DatabaseEntry;
use cultnet_rs::{
GAMECULT_MEDIA_AUDIO_PACKET_SCHEMA, GAMECULT_MEDIA_RECEIVER_FEEDBACK_SCHEMA,
GAMECULT_MEDIA_VIDEO_ACCESS_UNIT_SCHEMA, GAMECULT_MEDIA_VIDEO_PARITY_SHARD_SCHEMA,
GameCultMediaAudioPacketRecord, GameCultMediaReceiverFeedbackRecord,
GAMECULT_MEDIA_AUDIO_PACKET_SCHEMA, GAMECULT_MEDIA_AUDIO_PARITY_SHARD_SCHEMA,
GAMECULT_MEDIA_RECEIVER_FEEDBACK_SCHEMA, GAMECULT_MEDIA_VIDEO_ACCESS_UNIT_SCHEMA,
GAMECULT_MEDIA_VIDEO_PARITY_SHARD_SCHEMA, GameCultMediaAudioPacketRecord,
GameCultMediaAudioParityShardRecord, GameCultMediaReceiverFeedbackRecord,
GameCultMediaVideoAccessUnitRecord, GameCultMediaVideoParityShardRecord,
};
use rmpv::Value;
@ -129,9 +130,14 @@ fn parity_shards_carry_the_geometry_recovery_needs() {
dependency_frame_id: Some(4241),
deadline_ticks: 108_000,
chunk_count: 5,
fec_scheme: "rs-gf256-v1".to_string(),
block_index: 0,
block_count: 1,
block_data_start: 0,
block_data_count: 5,
parity_index: 1,
parity_count: 2,
chunk_payload_bytes: 1024,
shard_payload_bytes: 1024,
last_chunk_payload_bytes: 512,
payload: vec![9, 9, 9],
};
@ -141,13 +147,21 @@ fn parity_shards_carry_the_geometry_recovery_needs() {
panic!("parity shards must encode as a positional array");
};
assert_eq!(slots.len(), 17, "field count is wire contract");
assert_eq!(slots[12].as_u64(), Some(1), "slot 12 is parity_index");
assert_eq!(slots[13].as_u64(), Some(2), "slot 13 is parity_count");
assert_eq!(slots.len(), 22, "field count is wire contract");
assert_eq!(slots[12].as_str(), Some("rs-gf256-v1"), "slot 12 is fec_scheme");
assert_eq!(slots[13].as_u64(), Some(0), "slot 13 is block_index");
assert_eq!(slots[16].as_u64(), Some(5), "slot 16 is block_data_count (k)");
assert_eq!(slots[17].as_u64(), Some(1), "slot 17 is parity_index");
assert_eq!(slots[18].as_u64(), Some(2), "slot 18 is parity_count (m)");
assert_eq!(
slots[15].as_u64(),
slots[20].as_u64(),
Some(512),
"slot 15 is last_chunk_payload_bytes, which recovery needs to size the tail"
"slot 20 is last_chunk_payload_bytes, which recovery needs to size the tail"
);
assert_eq!(
slots[21].as_slice(),
Some([9, 9, 9].as_slice()),
"the parity payload is bin, not an array of integers"
);
let decoded: GameCultMediaVideoParityShardRecord =
@ -155,6 +169,42 @@ fn parity_shards_carry_the_geometry_recovery_needs() {
assert_eq!(decoded, shard);
}
#[test]
fn audio_parity_shards_carry_the_geometry_recovery_needs() {
let shard = GameCultMediaAudioParityShardRecord {
stream_id: "raven-primary-av".to_string(),
session_id: "session-1".to_string(),
codec: "opus".to_string(),
fec_scheme: "rs-gf256-v1".to_string(),
base_packet_id: 77,
base_pts_ticks: 960,
packet_duration_ticks: 960,
timebase_num: 1,
timebase_den: 48_000,
deadline_ticks: 4_800,
data_shard_count: 4,
parity_index: 1,
parity_shard_count: 2,
shard_payload_bytes: 3,
payload: vec![7, 8, 9],
};
let encoded = rmp_serde::to_vec(&shard).expect("encodes");
let Value::Array(slots) = decode_as_value(&encoded) else {
panic!("audio parity shards must encode as a positional array");
};
assert_eq!(slots.len(), 15, "field count is wire contract");
assert_eq!(slots[4].as_u64(), Some(77), "slot 4 is base_packet_id");
assert_eq!(slots[10].as_u64(), Some(4), "slot 10 is data_shard_count (k)");
assert_eq!(slots[12].as_u64(), Some(2), "slot 12 is parity_shard_count (m)");
assert_eq!(slots[14].as_slice(), Some([7, 8, 9].as_slice()), "the payload is bin");
let decoded: GameCultMediaAudioParityShardRecord =
rmp_serde::from_slice(&encoded).expect("round-trips");
assert_eq!(decoded, shard);
}
/// `missing_video_chunk_keys` was appended after producers were already writing
/// feedback. A record written without it must still decode, or the append was a
/// breaking change wearing a `default` attribute.
@ -205,6 +255,11 @@ fn published_schema_constants_match_the_derived_identity() {
GameCultMediaAudioPacketRecord::SCHEMA_NAME,
GAMECULT_MEDIA_AUDIO_PACKET_SCHEMA
);
assert_eq!(
GameCultMediaAudioParityShardRecord::SCHEMA_NAME,
GAMECULT_MEDIA_AUDIO_PARITY_SHARD_SCHEMA
);
assert_eq!(GAMECULT_MEDIA_VIDEO_PARITY_SHARD_SCHEMA, "gamecult.media_video_parity_shard.v3");
assert_eq!(
GameCultMediaReceiverFeedbackRecord::SCHEMA_NAME,
GAMECULT_MEDIA_RECEIVER_FEEDBACK_SCHEMA

463
packages/cultnet-rs/tests/media_stream_wire.rs
View file

@ -2,7 +2,10 @@
//! nothing on the wire.
use cultnet_rs::{
GAMECULT_MEDIA_CHANNEL, GameCultMediaAudioPacketRecord, GameCultMediaReceiverFeedbackRecord,
CultNetRudpDeliveredFrame, CultNetRudpSendOptions, CultNetRudpSession, CultNetRudpSessionOptions,
GAMECULT_MEDIA_MAX_WIRE_BYTES, RudpTransportProfileOptions, create_rudp_transport_profile,
GAMECULT_MEDIA_CHANNEL, GameCultMediaAudioPacketRecord, GameCultMediaAudioParityShardRecord,
GameCultMediaReceiverFeedbackRecord,
GameCultMediaVideoAccessUnitRecord, GameCultMediaVideoParityShardRecord,
GameCultMediaWireRecord, MediaWireProvenance, decode_media_wire_record,
encode_media_wire_record,
@ -116,12 +119,34 @@ fn every_variant_round_trips() {
dependency_frame_id: Some(0),
deadline_ticks: 1,
chunk_count: 4,
fec_scheme: "rs-gf256-v1".to_string(),
block_index: 0,
block_count: 1,
block_data_start: 0,
block_data_count: 4,
parity_index: 1,
parity_count: 2,
chunk_payload_bytes: 848,
last_chunk_payload_bytes: 400,
shard_payload_bytes: 3,
last_chunk_payload_bytes: 2,
payload: vec![1, 2, 3],
}),
GameCultMediaWireRecord::AudioParity(GameCultMediaAudioParityShardRecord {
stream_id: "s".to_string(),
session_id: "x".to_string(),
codec: "opus".to_string(),
fec_scheme: "rs-gf256-v1".to_string(),
base_packet_id: 8,
base_pts_ticks: 0,
packet_duration_ticks: 960,
timebase_num: 1,
timebase_den: 48_000,
deadline_ticks: 4_000,
data_shard_count: 4,
parity_index: 0,
parity_shard_count: 2,
shard_payload_bytes: 3,
payload: vec![4, 5, 6],
}),
GameCultMediaWireRecord::Audio(GameCultMediaAudioPacketRecord {
stream_id: "s".to_string(),
session_id: "x".to_string(),
@ -215,3 +240,435 @@ fn provenance_must_be_complete() {
fn the_media_channel_is_named_once() {
assert_eq!(GAMECULT_MEDIA_CHANNEL, "media");
}
fn video_parity() -> GameCultMediaVideoParityShardRecord {
GameCultMediaVideoParityShardRecord {
stream_id: "s".to_string(),
session_id: "x".to_string(),
frame_id: 7,
codec: "h264".to_string(),
pts_ticks: 0,
duration_ticks: 1,
timebase_num: 1,
timebase_den: 90_000,
keyframe: false,
dependency_frame_id: None,
deadline_ticks: 1,
chunk_count: 20,
fec_scheme: "rs-gf256-v1".to_string(),
block_index: 1,
block_count: 2,
block_data_start: 10,
block_data_count: 10,
parity_index: 1,
parity_count: 3,
shard_payload_bytes: 4,
last_chunk_payload_bytes: 2,
payload: vec![1, 2, 3, 4],
}
}
fn audio_parity() -> GameCultMediaAudioParityShardRecord {
GameCultMediaAudioParityShardRecord {
stream_id: "s".to_string(),
session_id: "x".to_string(),
codec: "opus".to_string(),
fec_scheme: "rs-gf256-v1".to_string(),
base_packet_id: 8,
base_pts_ticks: 0,
packet_duration_ticks: 960,
timebase_num: 1,
timebase_den: 48_000,
deadline_ticks: 4_000,
data_shard_count: 4,
parity_index: 1,
parity_shard_count: 2,
shard_payload_bytes: 3,
payload: vec![4, 5, 6],
}
}
fn decodes(record: GameCultMediaWireRecord) -> Result<GameCultMediaWireRecord, String> {
let wire = encode_media_wire_record(&record, provenance()).map_err(|error| error.to_string())?;
decode_media_wire_record(&wire).map_err(|error| error.to_string())
}
/// A parity key names the block as well as the shard, or two blocks' parity
/// index 0 would share an address.
#[test]
fn parity_record_keys_name_the_block_and_the_shard() {
let mut other_block = video_parity();
other_block.block_index = 0;
let mut other_shard = video_parity();
other_shard.parity_index = 0;
let keys = [
GameCultMediaWireRecord::VideoParity(video_parity()).record_key(),
GameCultMediaWireRecord::VideoParity(other_block).record_key(),
GameCultMediaWireRecord::VideoParity(other_shard).record_key(),
];
assert_eq!(keys[0], "s:x:video-parity:7:1:1");
assert_ne!(keys[0], keys[1]);
assert_ne!(keys[0], keys[2]);
let audio = GameCultMediaWireRecord::AudioParity(audio_parity()).record_key();
assert_eq!(audio, "s:x:audio-parity:8:1");
let mut other = audio_parity();
other.parity_index = 0;
assert_ne!(audio, GameCultMediaWireRecord::AudioParity(other).record_key());
}
#[test]
fn a_well_formed_parity_record_is_admitted() {
assert!(decodes(GameCultMediaWireRecord::VideoParity(video_parity())).is_ok());
assert!(decodes(GameCultMediaWireRecord::AudioParity(audio_parity())).is_ok());
}
/// The decoder is where a consumer's idea of a well-formed shard is enforced.
/// Each corruption below would otherwise reach the recovery arithmetic.
#[test]
fn a_malformed_video_parity_record_is_refused_with_its_reason() {
let cases: Vec<(&str, Box<dyn Fn(&mut GameCultMediaVideoParityShardRecord)>)> = vec![
("fec_scheme", Box::new(|r| r.fec_scheme = "xor-v0".to_string())),
("block_index", Box::new(|r| r.block_index = 2)),
("block_index", Box::new(|r| r.block_count = 0)),
("inside the frame", Box::new(|r| r.block_data_start = 11)),
("inside the frame", Box::new(|r| r.block_data_count = 0)),
("parity_index", Box::new(|r| r.parity_index = 3)),
("parity_index", Box::new(|r| r.parity_count = 0)),
("256 shards", Box::new(|r| {
r.block_data_count = 254;
r.chunk_count = 300;
r.parity_count = 3;
})),
("shard_payload_bytes", Box::new(|r| {
r.shard_payload_bytes = 0;
r.payload.clear();
})),
("last chunk length", Box::new(|r| r.last_chunk_payload_bytes = 0)),
("last chunk length", Box::new(|r| r.last_chunk_payload_bytes = 5)),
("declared shard length", Box::new(|r| r.payload.push(0))),
("chunk_count", Box::new(|r| r.chunk_count = 0)),
("256 shards", Box::new(|r| {
r.block_data_count = 256;
r.chunk_count = 300;
r.parity_count = 1;
r.parity_index = 0;
})),
("deadline_ticks", Box::new(|r| r.deadline_ticks = -1)),
];
for (expected, corrupt) in cases {
let mut record = video_parity();
corrupt(&mut record);
let error = decodes(GameCultMediaWireRecord::VideoParity(record)).unwrap_err();
assert!(error.contains(expected), "expected {expected:?} in {error:?}");
}
}
#[test]
fn a_malformed_audio_parity_record_is_refused_with_its_reason() {
let cases: Vec<(&str, Box<dyn Fn(&mut GameCultMediaAudioParityShardRecord)>)> = vec![
("fec_scheme", Box::new(|r| r.fec_scheme = String::new())),
("stripe", Box::new(|r| r.parity_index = 2)),
("stripe", Box::new(|r| r.data_shard_count = 0)),
("stripe", Box::new(|r| r.data_shard_count = 255)),
("timing", Box::new(|r| r.packet_duration_ticks = 0)),
("last pts_ticks", Box::new(|r| r.deadline_ticks = -1)),
("packet id or pts range", Box::new(|r| r.base_packet_id = u64::MAX - 2)),
("packet id or pts range", Box::new(|r| {
r.base_pts_ticks = i64::MAX - 960 * 3 + 1;
r.deadline_ticks = i64::MAX;
})),
("declared shard length", Box::new(|r| { r.payload.pop(); })),
("codec", Box::new(|r| r.codec = String::new())),
("stream_id", Box::new(|r| r.stream_id = String::new())),
("stream_id", Box::new(|r| r.session_id = String::new())),
("timing", Box::new(|r| r.timebase_num = 0)),
("timing", Box::new(|r| r.timebase_den = 0)),
];
for (expected, corrupt) in cases {
let mut record = audio_parity();
corrupt(&mut record);
let error = decodes(GameCultMediaWireRecord::AudioParity(record)).unwrap_err();
assert!(error.contains(expected), "expected {expected:?} in {error:?}");
}
}
/// The 256-shard ceiling and the deadline floor are inclusive on the side that
/// keeps a valid record valid.
#[test]
fn records_on_the_edge_of_each_bound_are_admitted() {
let mut video = video_parity();
video.chunk_count = 300;
video.block_data_count = 254;
video.parity_count = 2;
video.parity_index = 1;
assert!(decodes(GameCultMediaWireRecord::VideoParity(video)).is_ok(), "254 + 2 = 256 shards");
let mut audio = audio_parity();
audio.data_shard_count = 254;
audio.deadline_ticks = i64::MAX;
audio.parity_shard_count = 2;
assert!(decodes(GameCultMediaWireRecord::AudioParity(audio)).is_ok(), "254 + 2 = 256 shards");
let mut audio = audio_parity();
audio.deadline_ticks = audio.base_pts_ticks + 3 * i64::from(audio.packet_duration_ticks);
assert!(decodes(GameCultMediaWireRecord::AudioParity(audio)).is_ok(), "a deadline at the last pts");
let mut audio = audio_parity();
audio.base_packet_id = u64::MAX - 3;
assert!(decodes(GameCultMediaWireRecord::AudioParity(audio)).is_ok(), "the last packet id is u64::MAX");
let mut audio = audio_parity();
audio.base_pts_ticks = i64::MAX - 960 * 3;
audio.deadline_ticks = i64::MAX;
assert!(decodes(GameCultMediaWireRecord::AudioParity(audio)).is_ok(), "the last pts is i64::MAX");
}
// ---------------------------------------------------------------------------
// The ceiling on one media record
// ---------------------------------------------------------------------------
fn sized_audio_parity(payload_bytes: usize, deadline_ticks: i64) -> GameCultMediaAudioParityShardRecord {
GameCultMediaAudioParityShardRecord {
stream_id: "s".to_string(),
session_id: "x".to_string(),
codec: "opus".to_string(),
fec_scheme: "rs-gf256-v1".to_string(),
base_packet_id: 8,
base_pts_ticks: 0,
packet_duration_ticks: 100,
timebase_num: 1,
timebase_den: 48_000,
deadline_ticks,
data_shard_count: 4,
parity_index: 0,
parity_shard_count: 2,
shard_payload_bytes: payload_bytes as u32,
payload: vec![7; payload_bytes],
}
}
fn sized_video_parity(payload_bytes: usize) -> GameCultMediaVideoParityShardRecord {
GameCultMediaVideoParityShardRecord {
stream_id: "s".to_string(),
session_id: "x".to_string(),
frame_id: 1,
codec: "h264".to_string(),
pts_ticks: 0,
duration_ticks: 1,
timebase_num: 1,
timebase_den: 90_000,
keyframe: false,
dependency_frame_id: Some(0),
deadline_ticks: 1,
chunk_count: 4,
fec_scheme: "rs-gf256-v1".to_string(),
block_index: 0,
block_count: 1,
block_data_start: 0,
block_data_count: 4,
parity_index: 1,
parity_count: 2,
shard_payload_bytes: payload_bytes as u32,
last_chunk_payload_bytes: 2,
payload: vec![7; payload_bytes],
}
}
fn wire_len(record: &GameCultMediaWireRecord) -> usize {
encode_media_wire_record(record, provenance()).unwrap().len()
}
/// The payload length whose wrapped record is exactly `wire_bytes` long, found
/// from the record's own linear growth in its payload.
fn payload_for_wire_len(
wire_bytes: usize,
wrap: &dyn Fn(usize) -> GameCultMediaWireRecord,
) -> usize {
let probe = 1_000;
let payload = probe + wire_bytes - wire_len(&wrap(probe));
assert_eq!(wire_len(&wrap(payload)), wire_bytes, "the record grows one byte per payload byte here");
payload
}
/// RUDP would reassemble a 16 MiB record, and recovery work scales with the
/// shard length, so the decoder holds every record to one ceiling.
#[test]
fn a_media_record_one_byte_over_the_ceiling_is_refused_at_decode() {
let ceiling = GAMECULT_MEDIA_MAX_WIRE_BYTES;
let wraps: [(&str, Box<dyn Fn(usize) -> GameCultMediaWireRecord>); 2] = [
(
"audio parity",
Box::new(|payload| GameCultMediaWireRecord::AudioParity(sized_audio_parity(payload, 300))),
),
(
"video parity",
Box::new(|payload| GameCultMediaWireRecord::VideoParity(sized_video_parity(payload))),
),
];
for (name, wrap) in &wraps {
let at = payload_for_wire_len(ceiling, wrap.as_ref());
let wire = encode_media_wire_record(&wrap(at), provenance()).unwrap();
assert_eq!(wire.len(), ceiling);
decode_media_wire_record(&wire)
.unwrap_or_else(|error| panic!("{name} at the ceiling is refused: {error}"));
// The encoder refuses to make an over-ceiling record, so one byte of
// padding stands in for a peer that sent one.
let mut over = wire.clone();
over.push(0);
let error = decode_media_wire_record(&over).unwrap_err();
assert!(error.to_string().contains("ceiling"), "{name}: {error}");
}
}
#[test]
fn a_jumbo_frame_sized_record_is_accepted() {
let wire = encode_media_wire_record(
&GameCultMediaWireRecord::AudioParity(sized_audio_parity(8_800, 300)),
provenance(),
)
.unwrap();
assert!(wire.len() > 8_800 && wire.len() < GAMECULT_MEDIA_MAX_WIRE_BYTES);
decode_media_wire_record(&wire).expect("a jumbo-frame record decodes");
}
/// Recovery hands every recovered packet the block's deadline, so a parity
/// whose deadline precedes the last packet's pts would recover a packet that
/// fails `validate_audio_record`.
#[test]
fn audio_parity_must_not_have_a_deadline_before_its_last_packets_pts() {
// Four packets, 100 ticks each from 0: the last plays at 300.
let decodes = |deadline_ticks: i64| {
let wire = encode_media_wire_record(
&GameCultMediaWireRecord::AudioParity(sized_audio_parity(8, deadline_ticks)),
provenance(),
)
.unwrap();
decode_media_wire_record(&wire)
};
assert!(decodes(300).is_ok());
assert!(decodes(301).is_ok());
let error = decodes(299).unwrap_err();
assert!(error.to_string().contains("last pts_ticks"), "{error}");
let error = decodes(0).unwrap_err();
assert!(error.to_string().contains("last pts_ticks"), "{error}");
}
// ---------------------------------------------------------------------------
// The media channel's own payload cap
// ---------------------------------------------------------------------------
fn connected_session() -> CultNetRudpSession {
let mut session = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 9,
..CultNetRudpSessionOptions::default()
});
session.assume_connected(0);
session
}
const FRAGMENT_BYTES: usize = 16 * 1024;
fn media_payload_limit(session_limit: Option<u32>) -> Option<u32> {
let profile = create_rudp_transport_profile(
"raven",
RudpTransportProfileOptions {
max_payload_bytes: session_limit,
..RudpTransportProfileOptions::default()
},
);
profile.transports[0]
.channels
.iter()
.find(|channel| channel.channel_id == GAMECULT_MEDIA_CHANNEL)
.expect("the profile advertises the media channel")
.max_payload_bytes
}
#[test]
fn the_media_channel_advertises_the_ceiling_or_a_tighter_session_cap() {
let ceiling = GAMECULT_MEDIA_MAX_WIRE_BYTES as u32;
assert_eq!(media_payload_limit(None), Some(ceiling));
assert_eq!(media_payload_limit(Some(16 * 1024 * 1024)), Some(ceiling));
assert_eq!(media_payload_limit(Some(4_096)), Some(4_096));
}
#[test]
fn a_session_refuses_to_send_more_than_the_ceiling_on_the_media_channel_only() {
let ceiling = GAMECULT_MEDIA_MAX_WIRE_BYTES;
let send = |channel: &str, payload: usize| {
connected_session().send_many(
channel,
vec![0; payload],
CultNetRudpSendOptions::default(),
Some(FRAGMENT_BYTES),
)
};
assert!(send(GAMECULT_MEDIA_CHANNEL, ceiling).is_ok());
let error = send(GAMECULT_MEDIA_CHANNEL, ceiling + 1).unwrap_err();
assert!(error.to_string().contains("max_payload_bytes"), "{error}");
assert!(send("schema", ceiling + 1).is_ok(), "other channels keep the session cap");
}
/// Reassembly is where memory is spent, so the receiver enforces the ceiling
/// itself, not only the sender.
#[test]
fn a_session_refuses_to_reassemble_more_than_the_ceiling_on_the_media_channel() {
let ceiling = GAMECULT_MEDIA_MAX_WIRE_BYTES;
let packets = connected_session()
.send_many(
"schema",
vec![0; ceiling + 1],
CultNetRudpSendOptions::default(),
Some(FRAGMENT_BYTES),
)
.unwrap();
assert!(packets.len() > 1);
let deliver = |channel: &str| -> anyhow::Result<Vec<CultNetRudpDeliveredFrame>> {
let mut receiver = connected_session();
let mut delivered = Vec::new();
for packet in &packets {
let mut packet = packet.clone();
packet.channel_id = channel.to_string();
delivered.extend(receiver.receive(&packet, 0)?.delivered);
}
Ok(delivered)
};
let frames = deliver("schema").expect("the same fragments reassemble on another channel");
assert_eq!(frames.len(), 1);
assert_eq!(frames[0].payload.len(), ceiling + 1);
let error = deliver(GAMECULT_MEDIA_CHANNEL).unwrap_err();
assert!(error.to_string().contains("max_payload_bytes"), "{error}");
}
/// A producer learns at encode time that a record is over the ceiling, rather
/// than the receiver dropping it.
#[test]
fn the_encoder_refuses_a_record_over_the_ceiling() {
let wrap = |payload: usize| GameCultMediaWireRecord::AudioParity(sized_audio_parity(payload, 300));
let at = payload_for_wire_len(GAMECULT_MEDIA_MAX_WIRE_BYTES, &wrap);
assert!(encode_media_wire_record(&wrap(at), provenance()).is_ok());
let error = encode_media_wire_record(&wrap(at + 1), provenance()).unwrap_err();
assert!(error.to_string().contains("ceiling"), "{error}");
}
/// A record that arrives unfragmented is held to the same ceiling.
#[test]
fn a_session_refuses_an_unfragmented_media_packet_over_the_ceiling() {
let packets = connected_session()
.send_many(
"schema",
vec![0; GAMECULT_MEDIA_MAX_WIRE_BYTES + 1],
CultNetRudpSendOptions::default(),
None,
)
.unwrap();
assert_eq!(packets.len(), 1, "sent whole");
let mut packet = packets[0].clone();
assert!(connected_session().receive(&packet, 0).is_ok());
packet.channel_id = GAMECULT_MEDIA_CHANNEL.to_string();
let error = connected_session().receive(&packet, 0).unwrap_err();
assert!(error.to_string().contains("max_payload_bytes"), "{error}");
}

1001
packages/cultnet-rs/tests/rudp_connect_generation.rs Normal file
View file

File diff suppressed because it is too large Load diff

22
packages/cultnet-rs/tests/rudp_server_hub.rs
View file

@ -168,7 +168,7 @@ fn hub_keeps_independent_peer_sessions_and_exposes_connect_evidence() -> Result<
}
#[test]
fn hub_fences_replaced_generations_and_does_not_replace_connect_retransmits() -> Result<()> {
fn hub_fences_replaced_generations() -> Result<()> {
let server_socket = socket()?;
let server_addr = server_socket.local_addr()?;
let mut hub = CultNetRudpServerHub::new(CultNetRudpServerHubOptions::new(
@ -189,12 +189,6 @@ fn hub_fences_replaced_generations_and_does_not_replace_connect_retransmits() ->
})?;
let original = connect(&mut hub, &mut provider, &original_evidence)?;
provider.connect(original_evidence.clone())?;
for _ in 0..3 {
assert!(hub.receive_event_once()?.is_none());
}
assert_eq!(hub.session(original.remote_addr), Some(&original));
provider.connect(replacement_evidence.clone())?;
let mut disconnected = None;
for _ in 0..20 {
@ -253,7 +247,7 @@ fn identical_connect_retransmit_reuses_pending_accept_after_loss() -> Result<()>
let client_socket = socket()?;
let mut client_session = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: CONNECTION_ID,
initial_sequence: 1,
initial_sequence: Some(1),
resend_delay_ms: 250,
max_pending_reliable_packets: None,
});
@ -292,13 +286,13 @@ fn identical_connect_retransmit_reuses_pending_accept_after_loss() -> Result<()>
fn exact_reliable_acks_clear_more_than_one_ack_window_of_fragments() -> Result<()> {
let mut sender = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: CONNECTION_ID,
initial_sequence: 1,
initial_sequence: Some(1),
resend_delay_ms: 250,
max_pending_reliable_packets: None,
});
let mut receiver = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: CONNECTION_ID,
initial_sequence: 1,
initial_sequence: Some(1),
resend_delay_ms: 250,
max_pending_reliable_packets: None,
});
@ -352,7 +346,7 @@ fn exact_reliable_acks_clear_more_than_one_ack_window_of_fragments() -> Result<(
fn session_and_hub_reject_configured_memory_bounds() -> Result<()> {
let mut receiver = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: CONNECTION_ID,
initial_sequence: 1,
initial_sequence: Some(1),
resend_delay_ms: 250,
max_pending_reliable_packets: None,
});
@ -360,7 +354,7 @@ fn session_and_hub_reject_configured_memory_bounds() -> Result<()> {
receiver.set_max_pending_fragment_sets(1)?;
let mut sender = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: CONNECTION_ID,
initial_sequence: 1,
initial_sequence: Some(1),
resend_delay_ms: 250,
max_pending_reliable_packets: None,
});
@ -428,13 +422,13 @@ fn session_and_hub_reject_configured_memory_bounds() -> Result<()> {
fn replay_history_is_bounded_and_idle_hub_sessions_expire() -> Result<()> {
let mut sender = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: CONNECTION_ID,
initial_sequence: 1,
initial_sequence: Some(1),
resend_delay_ms: 250,
max_pending_reliable_packets: None,
});
let mut receiver = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: CONNECTION_ID,
initial_sequence: 1,
initial_sequence: Some(1),
resend_delay_ms: 250,
max_pending_reliable_packets: None,
});

232
packages/cultnet-rs/tests/rudp_stray_packets.rs
View file

@ -30,9 +30,13 @@ fn socket() -> Result<UdpSocket> {
}
fn raw_session(connection_id: u32) -> CultNetRudpSession {
raw_session_from(connection_id, 1)
}
fn raw_session_from(connection_id: u32, initial_sequence: u32) -> CultNetRudpSession {
CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id,
initial_sequence: 1,
initial_sequence: Some(initial_sequence),
resend_delay_ms: 250,
max_pending_reliable_packets: None,
})
@ -442,7 +446,7 @@ fn server_mode_ends_the_session_when_the_session_refuses_a_packet() -> Result<()
#[test]
fn constructors_reject_limits_that_make_a_connect_unadmittable() -> Result<()> {
let mut hub = CultNetRudpServerHubOptions::new("hub", socket()?, CONNECTION_ID);
hub.initial_sequence = u32::MAX;
hub.initial_sequence = Some(u32::MAX);
assert!(CultNetRudpServerHub::new(hub).is_err());
let mut hub = CultNetRudpServerHubOptions::new("hub", socket()?, CONNECTION_ID);
hub.max_pending_reliable_packets = Some(0);
@ -455,13 +459,13 @@ fn constructors_reject_limits_that_make_a_connect_unadmittable() -> Result<()> {
] {
let addr = "127.0.0.1:9".parse()?;
let mut options = build(socket()?, addr);
options.initial_sequence = u32::MAX;
options.initial_sequence = Some(u32::MAX);
assert!(CultNetRudpSocketTransportConnection::new(options).is_err());
let mut options = build(socket()?, addr);
options.max_pending_reliable_packets = Some(0);
assert!(CultNetRudpSocketTransportConnection::new(options).is_err());
let mut options = build(socket()?, addr);
options.initial_sequence = u32::MAX - 1;
options.initial_sequence = Some(u32::MAX - 1);
options.max_pending_reliable_packets = Some(1);
assert!(CultNetRudpSocketTransportConnection::new(options).is_ok());
}
@ -725,7 +729,7 @@ fn a_flush_fails_when_a_new_endpoint_replaces_the_peer_mid_wait() -> Result<()>
send_to(&peer_a, server_addr, &raw_session(CONNECTION_ID).create_connect(0, b"a".to_vec())?)?;
let _ = server.receive_once()?;
server.send_reliable("schema", b"for A".to_vec())?;
send_to(&peer_b, server_addr, &raw_session(CONNECTION_ID).create_connect(0, b"b".to_vec())?)?;
send_to(&peer_b, server_addr, &raw_session_from(CONNECTION_ID, 2).create_connect(0, b"b".to_vec())?)?;
let error = server.flush_reliable(Duration::from_millis(300)).expect_err("A's write was forgotten");
assert!(
@ -745,8 +749,224 @@ fn a_repeated_connect_acknowledges_what_it_carries() -> Result<()> {
let accept = server.accept_connect(&connect, 0, Vec::new())?;
let mut repeat = connect.clone();
repeat.ack = accept.sequence;
let reply = server.answer_repeated_connect(&repeat, 1)?;
let reply = server.accept_connect(&repeat, 1, Vec::new())?;
assert_eq!(reply.packet_type, CultNetRudpPacketType::Ack);
assert_eq!(server.outstanding_reliable_packet_count(), 0);
Ok(())
}
/// A connected client session with one ordered write the peer never received.
fn client_with_a_lost_write() -> Result<(CultNetRudpSession, CultNetRudpSession)> {
let mut client = raw_session(CONNECTION_ID);
let mut server = raw_session(CONNECTION_ID);
let accept = server.accept_connect(&client.create_connect(0, Vec::new())?, 0, Vec::new())?;
client.receive(&accept, 0)?;
client.send(
"schema",
b"owed to the old session".to_vec(),
CultNetRudpSendOptions {
reliable: true,
ordered: true,
..Default::default()
},
)?;
// A fragmented write larger than the send window leaves part of it queued.
client.send_many(
"schema",
vec![7u8; 40 * 8],
CultNetRudpSendOptions {
reliable: true,
ordered: true,
..Default::default()
},
Some(8),
)?;
assert_eq!(client.outstanding_reliable_packet_count(), 41);
assert!(client.queued_reliable_packet_count() > 0);
Ok((client, server))
}
/// A write belongs to the session it was issued in. Whichever way that session
/// ends, the write it still owed dies with it: it is not retransmitted into a
/// later session, where the new peer would deliver it.
#[test]
fn every_way_a_session_ends_drops_the_writes_it_owed() -> Result<()> {
type End = fn(&mut CultNetRudpSession, &mut CultNetRudpSession) -> Result<()>;
let endings: [(&str, End); 4] = [
("peer Disconnect", |client, server| {
client.receive(&server.create_disconnect(b"bye".to_vec()), 1)?;
Ok(())
}),
("local disconnect", |client, _| {
client.create_disconnect(b"bye".to_vec());
Ok(())
}),
("timeout", |client, _| {
assert!(client.check_timeout(1_000, 10));
Ok(())
}),
("refusal", |client, _| {
client.end_refused_session();
Ok(())
}),
];
for (name, end) in endings {
let (mut client, mut server) = client_with_a_lost_write()?;
end(&mut client, &mut server)?;
assert_eq!(client.outstanding_reliable_packet_count(), 0, "{name}: the write survived the end");
let mut next_server = raw_session(CONNECTION_ID);
let connect = client.create_connect(2_000, Vec::new())?;
next_server.accept_connect(&connect, 2_000, Vec::new())?;
for resend in client.due_resends(60_000) {
let result = next_server.receive(&resend, 60_000)?;
assert!(
result.delivered.is_empty(),
"{name}: the old session's write was delivered in the next session"
);
}
}
Ok(())
}
fn connected_client(server_socket: &UdpSocket, server_addr: SocketAddr) -> Result<(CultNetRudpSocketTransportConnection, CultNetRudpSession, SocketAddr)> {
let mut client = CultNetRudpSocketTransportConnection::new(
CultNetRudpSocketTransportOptions::client("client", socket()?, server_addr, CONNECTION_ID),
)?;
client.connect(b"hello".to_vec())?;
let mut buffer = vec![0_u8; 65_535];
let (received, client_addr) = server_socket.recv_from(&mut buffer)?;
let mut peer = raw_session(CONNECTION_ID);
let accept = peer.accept_connect(&decode_rudp_packet(&buffer[..received])?, 0, Vec::new())?;
send_to(server_socket, client_addr, &accept)?;
let _ = client.receive_once()?;
Ok((client, peer, client_addr))
}
/// A local `disconnect()` ends the session like any other end: the receipt of a
/// write the peer never acknowledged is Invalidated, stays so after a
/// reconnect, and the write is not carried into the new session.
#[test]
fn a_receipt_invalidated_by_a_local_disconnect_stays_invalidated_after_reconnecting() -> Result<()> {
let server = socket()?;
let server_addr = server.local_addr()?;
let (mut client, _peer, _) = connected_client(&server, server_addr)?;
let receipt = client.send_reliable("schema", b"written before the disconnect".to_vec())?;
assert_eq!(client.reliable_send_status(&receipt), CultNetRudpReliableSendStatus::Pending);
client.disconnect(b"bye".to_vec())?;
assert_eq!(client.reliable_send_status(&receipt), CultNetRudpReliableSendStatus::Invalidated);
assert_eq!(client.outstanding_reliable_packet_count(), 0, "the write survived the disconnect");
client.connect(b"hello again".to_vec())?;
assert_eq!(
client.reliable_send_status(&receipt),
CultNetRudpReliableSendStatus::Invalidated,
"reconnecting brought the write back"
);
assert_eq!(client.outstanding_reliable_packet_count(), 1, "only the new Connect is owed");
Ok(())
}
/// A flush that starts after the session ended has no live session to wait on.
#[test]
fn a_flush_fails_after_a_local_disconnect_or_a_timeout() -> Result<()> {
let server = socket()?;
let server_addr = server.local_addr()?;
let (mut client, _peer, _) = connected_client(&server, server_addr)?;
client.send_reliable("schema", b"owed".to_vec())?;
client.disconnect(b"bye".to_vec())?;
let error = client.flush_reliable(Duration::from_millis(200)).expect_err("the write was dropped");
assert!(error.to_string().contains("ended before its reliable writes were acknowledged"), "{error}");
let server = socket()?;
let server_addr = server.local_addr()?;
let (mut client, _peer, _) = connected_client(&server, server_addr)?;
client.send_reliable("schema", b"owed".to_vec())?;
std::thread::sleep(Duration::from_millis(5));
assert!(client.check_timeout(1));
let error = client.flush_reliable(Duration::from_millis(200)).expect_err("the write was dropped");
assert!(error.to_string().contains("ended before its reliable writes were acknowledged"), "{error}");
Ok(())
}
/// A flush that starts after a reconnect waits on the new session: the end of
/// the old one does not fail it.
#[test]
fn a_flush_started_after_a_reconnect_waits_on_the_new_session() -> Result<()> {
let server = socket()?;
let server_addr = server.local_addr()?;
let (mut client, _peer, client_addr) = connected_client(&server, server_addr)?;
client.send_reliable("schema", b"owed to the old session".to_vec())?;
client.disconnect(b"bye".to_vec())?;
client.connect(b"hello again".to_vec())?;
let connect = drain(&server)?
.into_iter()
.rfind(|packet| packet.packet_type == CultNetRudpPacketType::Connect)
.expect("the reconnect reached the peer");
let accept = raw_session(CONNECTION_ID).accept_connect(&connect, 0, Vec::new())?;
send_to(&server, client_addr, &accept)?;
let _ = client.receive_once()?;
client.flush_reliable(Duration::from_millis(500))?;
Ok(())
}
/// A server-mode transport that accepts a new Connect after its peer left has a
/// live session again, and a flush started in it waits on that session.
#[test]
fn a_flush_started_after_the_server_accepts_a_new_connect_waits_on_the_new_session() -> Result<()> {
let server_socket = socket()?;
let server_addr = server_socket.local_addr()?;
let mut server = CultNetRudpSocketTransportConnection::new(
CultNetRudpSocketTransportOptions::server("server", server_socket, CONNECTION_ID),
)?;
let peer = socket()?;
let mut first = raw_session(CONNECTION_ID);
send_to(&peer, server_addr, &first.create_connect(0, b"peer".to_vec())?)?;
let _ = server.receive_once()?;
send_to(&peer, server_addr, &first.create_disconnect(b"bye".to_vec()))?;
let _ = server.receive_once()?;
let mut next = raw_session(CONNECTION_ID);
send_to(&peer, server_addr, &next.create_connect(0, b"peer".to_vec())?)?;
let _ = server.receive_once()?;
let accept = drain(&peer)?
.into_iter()
.rfind(|packet| packet.packet_type == CultNetRudpPacketType::Accept)
.expect("the new Connect was accepted");
next.receive(&accept, 0)?;
send_to(&peer, server_addr, &next.create_ack_for_received(accept.sequence))?;
let _ = server.receive_once()?;
server.flush_reliable(Duration::from_millis(500))?;
Ok(())
}
/// The session ended, the peer did not notice: what it sent before the end is
/// still remembered, held frames included, so its retransmits are duplicates.
#[test]
fn an_ending_does_not_forget_what_was_received_from_the_peer() -> Result<()> {
let mut client = raw_session(CONNECTION_ID);
let mut server = raw_session(CONNECTION_ID);
let accept = server.accept_connect(&client.create_connect(0, Vec::new())?, 0, Vec::new())?;
client.receive(&accept, 0)?;
let ordered = CultNetRudpSendOptions {
reliable: true,
ordered: true,
..Default::default()
};
let s1 = client.send("schema", b"s1".to_vec(), ordered.clone())?;
let s2 = client.send("schema", b"s2".to_vec(), ordered)?;
assert!(server.receive(&s2, 1)?.delivered.is_empty(), "s1 is missing, so s2 is held");
assert!(server.check_timeout(1_000, 10));
let released = server.receive(&s1, 1_001)?.delivered;
assert_eq!(
released.iter().map(|frame| frame.payload.clone()).collect::<Vec<_>>(),
[b"s1".to_vec(), b"s2".to_vec()],
"the ending forgot the held frame"
);
assert!(server.receive(&s2, 1_002)?.delivered.is_empty(), "the ending forgot what was received");
Ok(())
}

1
packages/cultnet-ts/src/index.ts
View file

@ -106,6 +106,7 @@ export {
createRudpTransportProfile,
decodeRudpPacket,
encodeRudpPacket,
rudpClientBindHost,
CULTNET_RUDP_RELIABLE_SEND_WINDOW_PACKETS,
CultNetRudpReconnectLoop,
CultNetRudpSession,

6
packages/cultnet-ts/src/operation-service.ts
View file

@ -12,6 +12,7 @@ import {
CultNetRudpSocketTransportConnection,
decodeRudpPacket,
encodeRudpPacket,
rudpClientBindHost,
type CultNetRudpPacket,
} from "./rudp";
import { CultNetPeer } from "./peer";
@ -112,8 +113,9 @@ export async function invokeCultNetOperation(
options: CultNetOperationClientOptions,
): Promise<CultNetOperationResponseMessage> {
const target = parseRudpEndpoint(endpoint);
const socket = createSocket("udp4");
await bindSocket(socket, 0, "127.0.0.1");
const bindHost = await rudpClientBindHost(target.host);
const socket = createSocket(bindHost.includes(":") ? "udp6" : "udp4");
await bindSocket(socket, 0, bindHost);
const transport = new CultNetRudpSocketTransportConnection({
mode: "client",
runtimeId: options.runtimeId,

457
packages/cultnet-ts/src/rudp.ts
View file

@ -1,7 +1,22 @@
import { randomInt } from "node:crypto";
import { EventEmitter } from "node:events";
import { type RemoteInfo, type Socket } from "node:dgram";
import { lookup } from "node:dns/promises";
import type { CultNetTransportProfile } from "./contracts";
/**
* The address an RUDP client binds when its caller names none: loopback for a
* loopback endpoint, otherwise the unspecified address of the endpoint's family.
* A socket bound to loopback cannot send off the host on Windows.
*/
export async function rudpClientBindHost(endpointHost: string): Promise<string> {
const { address, family } = await lookup(endpointHost.replace(/^\[(.*)\]$/, "$1"));
if (family === 6) {
return address === "::1" ? "::1" : "::";
}
return address.startsWith("127.") ? "127.0.0.1" : "0.0.0.0";
}
import {
CultNetReconnectController,
createCultNetReconnectPolicy,
@ -18,6 +33,34 @@ const RUDP_FIXED_HEADER_BYTES = 36;
const MAX_CHANNEL_ID_BYTES = 255;
export const CULTNET_RUDP_RELIABLE_SEND_WINDOW_PACKETS = 32;
const RUDP_RECEIVED_SEQUENCE_WINDOW = 4_096;
/// How long a client keeps retransmitting a Connect nobody answered before it
/// abandons that attempt for a fresh one. A Connect the server answers with an
/// Ack, never an Accept, is one the server judged stale (see `acceptConnect`);
/// retransmitting the same sequence would never change its mind.
const RUDP_CONNECT_ATTEMPT_MS = 3_000;
/// A sequence drawn from a secure source in [1, 2^31). The Connect's sequence
/// is what tells a peer whether a Connect repeats one it already accepted or
/// starts a new session, so two sessions must not share one by default.
function drawSequence(): number {
return randomInt(1, 2 ** 31);
}
/// Whether `sequence` is at or before `mark` in serial order, within the
/// receive window. The compare is modular, so it holds across the wrap of the
/// 32-bit space; a sequence further back than the window is the duplicate
/// test's below-window clause, and one ahead of the mark is never before it.
function atOrBefore(sequence: number, mark: number): boolean {
return ((mark - sequence) >>> 0) < RUDP_RECEIVED_SEQUENCE_WINDOW;
}
function packetAcknowledges(packet: CultNetRudpPacket, sequence: number): boolean {
if (packet.ack === sequence) return true;
for (let bit = 0; bit < 32; bit += 1) {
if ((packet.ackMask & (1 << bit)) !== 0 && packet.ack - bit - 1 === sequence) return true;
}
return false;
}
export type CultNetRudpPacketType =
| "connect"
@ -72,11 +115,6 @@ type PendingReliablePacket = {
lastSentAtMs: number;
};
type PendingOrderedFrame = {
frame: CultNetRudpDeliveredFrame;
nextSequence: number;
};
type FragmentBuffer = {
channelId: string;
reliable: boolean;
@ -150,6 +188,24 @@ export class CultNetRudpSession {
readonly #nextSequencedByChannel = new Map<string, number>();
#nextFragmentId = 1;
#connected = false;
/// Advances every time a generation ends. Everything issued in a generation
/// (writes, flushes) belongs to it and dies with it.
#generation = 0;
/// True from the end of a generation until the next Connect or Accept begins
/// one. A flush started in that interval has no live generation to wait on.
#ended = false;
/// The sequence of the Connect that started the current generation: sent by
/// this side, or accepted from the peer. A Connect repeats exactly when the
/// session is connected and the Connect carries this sequence.
#connectSequence: number | undefined;
/// A Connect this side sent is unanswered. Only then is an Accept honoured.
#awaitingAccept = false;
/// The payload of the Connect this side sent, kept so a fresh attempt can
/// always be built whatever became of the pending packet.
#connectPayload = new Uint8Array();
/// When the unanswered Connect first went out; `dueResends` abandons it for a
/// fresh attempt once `RUDP_CONNECT_ATTEMPT_MS` has passed.
#connectStartedAtMs = 0;
readonly #maxPendingReliablePackets: number | undefined;
#lastReceivedAtMs: number | undefined;
#highestReceivedSequence: number | undefined;
@ -157,8 +213,15 @@ export class CultNetRudpSession {
readonly #latestSequencedByChannel = new Map<string, number>();
readonly #pendingReliable = new Map<number, PendingReliablePacket>();
readonly #queuedReliable: CultNetRudpPacket[] = [];
readonly #orderedNextSequenceByChannel = new Map<string, number>();
readonly #orderedBuffers = new Map<string, Map<number, PendingOrderedFrame>>();
/// Every reliable sequence up to and including this one has been received
/// since the peer state was last reset. Only the handshake seeds it: the
/// peer's Connect on the accepting side, the peer's Accept on the connecting
/// side. Reliable data that arrives before that is refused.
#receivedThrough: number | undefined;
/// Ordered frames received but not yet deliverable, keyed by first sequence.
/// A frame is held for exactly one reason: a reliable sequence below it has
/// not arrived.
readonly #orderedHeld = new Map<number, CultNetRudpDeliveredFrame>();
readonly #fragmentBuffers = new Map<string, FragmentBuffer>();
/// Matches cultnet-rs's `max_pending_fragment_sets`. A fragment set is only
/// removed on successful reassembly, so a set that loses one fragment is
@ -169,7 +232,7 @@ export class CultNetRudpSession {
constructor(options: CultNetRudpSessionOptions) {
this.connectionId = toUint32(options.connectionId, "connectionId");
this.#nextSequence = toUint32(options.initialSequence ?? 1, "initialSequence");
this.#nextSequence = toUint32(options.initialSequence ?? drawSequence(), "initialSequence");
if (this.#nextSequence === 0xffff_ffff) {
throw new Error("RUDP initialSequence must leave room for a reliable packet.");
}
@ -211,23 +274,53 @@ export class CultNetRudpSession {
return this.#lastReceivedAtMs;
}
resetPeerState(): void {
get generation(): number {
return this.#generation;
}
get ended(): boolean {
return this.#ended;
}
/// The one way a session generation ends. The session stops being connected
/// and what it still owed the peer dies with it: a write not yet acknowledged
/// is dropped, so no later session retransmits it or credits an ack to it.
/// What was learned from the peer is not touched: the peer may not know the
/// session ended, and forgetting what it sent would let its retransmits be
/// delivered twice.
#endSession(): void {
this.#connected = false;
this.#ended = true;
this.#awaitingAccept = false;
this.#generation += 1;
this.#pendingReliable.clear();
this.#queuedReliable.splice(0);
}
/// Ends the current generation and forgets everything learned from the peer;
/// sequence numbers already issued stay issued.
resetPeerState(): void {
this.#endSession();
this.#lastReceivedAtMs = undefined;
this.#highestReceivedSequence = undefined;
this.#receivedSequences.clear();
this.#nextSequencedByChannel.clear();
this.#latestSequencedByChannel.clear();
this.#pendingReliable.clear();
this.#queuedReliable.splice(0);
this.#orderedNextSequenceByChannel.clear();
this.#orderedBuffers.clear();
this.#receivedThrough = undefined;
this.#orderedHeld.clear();
this.#fragmentBuffers.clear();
this.#fragmentSetsEvicted = 0;
}
createConnect(nowMs = 0, payload = new Uint8Array()): CultNetRudpPacket {
// A session that has had a peer starts a new generation: nothing it learned
// from that peer describes the one this Connect reaches, and what it still
// owed that peer no longer takes room in the queue.
if (this.#connectSequence !== undefined) {
this.resetPeerState();
}
this.#ensureReliableCapacity(1);
this.#ended = false;
const packet = this.#createPacket({
packetType: "connect",
channelId: "control",
@ -235,24 +328,68 @@ export class CultNetRudpSession {
ordered: true,
payload,
});
this.#connectSequence = packet.sequence;
this.#awaitingAccept = true;
this.#connectStartedAtMs = nowMs;
this.#connectPayload = new Uint8Array(payload);
this.#trackReliable(packet, nowMs);
return packet;
}
/// Whether `packet` is a Connect this session's current generation already
/// owns: a retransmit of the Connect that started it (the session is
/// connected and the sequence is that Connect's), or a stale copy of an
/// earlier attempt by the same client (a sequence before it, within the
/// receive window). Servers that keep one session per peer ask this to tell a
/// Connect that starts a new session from one that does not; anything else
/// that reaches `acceptConnect` starts a new generation.
connectRepeats(packet: CultNetRudpPacket): boolean {
return packet.packetType === "connect"
&& this.#connected
&& this.#connectSequence !== undefined
&& (this.#connectSequence === packet.sequence
|| this.#connectIsStale(packet.sequence, this.#connectSequence));
}
/// A Connect that precedes the current generation's within the receive window
/// is the client's earlier attempt, delayed in the network: a client that
/// retried never sends a lower sequence again. Restarting on it would strand
/// the client, which honours only the Accept for its newest Connect. The rule
/// is TCP's answer to a delayed SYN (RFC 5961's challenge ACK): keep the
/// connection and answer with an Ack. A restarted client whose random initial
/// sequence lands in this window is answered the same way and abandons the
/// attempt for a fresh draw (`RUDP_CONNECT_ATTEMPT_MS`).
#connectIsStale(sequence: number, current: number): boolean {
return sequence !== current && atOrBefore(sequence, current);
}
/// Answers a Connect. A repeat of the accepted one queues nothing, so a
/// Connect storm cannot grow the reliable queue: the reply is the Accept
/// still awaiting acknowledgement, or an Ack once it was acknowledged. A
/// stale copy of an earlier attempt gets the same reply and changes nothing
/// else: it is not evidence the peer is alive. Any other Connect ends the
/// current generation, forgets the peer and accepts a new one.
acceptConnect(packet: CultNetRudpPacket, nowMs = 0, payload = new Uint8Array()): CultNetRudpPacket {
this.#requireConnection(packet);
if (packet.packetType !== "connect") {
throw new Error(`Expected RUDP connect packet, got ${packet.packetType}.`);
}
// A Connect from a peer this session already accepted repeats.
if (this.#connected) {
return this.answerRepeatedConnect(packet, nowMs);
if (this.connectRepeats(packet)) {
if (this.#connectSequence === packet.sequence) {
this.#applyAcknowledgements(packet);
this.#rememberReceived(packet.sequence);
this.#lastReceivedAtMs = nowMs;
}
return this.#pendingAcceptForResend(nowMs) ?? this.createAck();
}
this.resetPeerState();
this.#ensureReliableCapacity(1);
this.#rememberReceived(packet.sequence);
this.#seedReceived(packet.sequence);
this.#lastReceivedAtMs = nowMs;
this.#connectSequence = packet.sequence;
this.#connected = true;
this.#ended = false;
const response = this.#createPacket({
packetType: "accept",
channelId: "control",
@ -264,24 +401,12 @@ export class CultNetRudpSession {
return response;
}
/**
* Answers a Connect from a peer this session has already accepted. The packet's
* sequence is remembered and nothing is queued, so a Connect storm cannot grow
* the reliable queue. The reply is the Accept still awaiting acknowledgement,
* or an Ack once it was acknowledged.
*/
answerRepeatedConnect(packet: CultNetRudpPacket, nowMs = 0): CultNetRudpPacket {
this.#requireConnection(packet);
if (packet.packetType !== "connect") {
throw new Error(`Expected RUDP connect packet, got ${packet.packetType}.`);
}
this.#applyAcknowledgements(packet);
this.#rememberReceived(packet.sequence);
this.#lastReceivedAtMs = nowMs;
/// The Accept still awaiting acknowledgement, resent.
#pendingAcceptForResend(nowMs: number): CultNetRudpPacket | undefined {
const pendingAccept = [...this.#pendingReliable.values()]
.find(pending => pending.packet.packetType === "accept");
if (!pendingAccept) {
return this.createAck();
return undefined;
}
pendingAccept.lastSentAtMs = nowMs;
return {
@ -369,15 +494,28 @@ export class CultNetRudpSession {
receive(packet: CultNetRudpPacket, nowMs = 0): CultNetRudpReceiveResult {
this.#requireConnection(packet);
this.#applyAcknowledgements(packet);
// An Accept counts only while this side's Connect is unanswered and the
// Accept names it. A late or duplicate one, or one from an earlier
// generation, must not seed the watermark or revive an ended session.
const honoursAccept = packet.packetType === "accept"
&& this.#awaitingAccept
&& this.#connectSequence !== undefined
&& packetAcknowledges(packet, this.#connectSequence);
if (packet.packetType === "accept" && !honoursAccept) {
return { delivered: [], readyToSend: [] };
}
// While this side's Connect awaits its Accept, only the Accept it honours
// retires it: an Ack that names the Connect (a server's reply to a repeat or
// a stale copy) says the server did not start a session.
if (honoursAccept || !this.#awaitingAccept) {
this.#applyAcknowledgements(packet);
}
const readyToSend = this.#promoteQueuedReliable(nowMs);
this.#lastReceivedAtMs = nowMs;
const expectedSequenceIfUninitialized = this.#highestReceivedSequence === undefined
? packet.sequence
: this.#highestReceivedSequence + 1;
if (packet.packetType === "accept") {
this.#rememberReceived(packet.sequence);
if (honoursAccept) {
this.#awaitingAccept = false;
this.#seedReceived(packet.sequence);
this.#connected = true;
return { delivered: [], readyToSend };
}
@ -404,7 +542,7 @@ export class CultNetRudpSession {
}
if (packet.packetType === "disconnect") {
this.#connected = false;
this.#endSession();
return {
delivered: [],
readyToSend,
@ -417,39 +555,41 @@ export class CultNetRudpSession {
return { delivered: [], readyToSend };
}
const isDuplicate = (packet.reliable ?? false)
&& (this.#receivedSequences.has(packet.sequence)
|| (this.#highestReceivedSequence !== undefined
&& packet.sequence < this.#highestReceivedSequence
&& this.#highestReceivedSequence - packet.sequence >= RUDP_RECEIVED_SEQUENCE_WINDOW));
// Reliable data before the handshake has seeded the watermark has no place
// in the order: refuse it unremembered, so it is not acknowledged and the
// sender retransmits it once the handshake is done.
if ((packet.reliable ?? false) && this.#receivedThrough === undefined) {
return { delivered: [], readyToSend };
}
const isDuplicate = (packet.reliable ?? false) && this.#wasReceived(packet.sequence);
if (packet.reliable ?? false) this.#rememberReceived(packet.sequence);
if (isDuplicate) {
return { delivered: [], readyToSend };
}
const delivered: CultNetRudpDeliveredFrame[] = [];
const reassembled = this.#reassemble(packet);
if (!reassembled) {
return { delivered: [], readyToSend };
}
if (!reassembled.ordered && (packet.sequenced ?? false)) {
const newestSequence = reassembled.nextSequence - 1;
const latestSequence = this.#latestSequencedByChannel.get(reassembled.frame.channelId);
if (latestSequence !== undefined && newestSequence <= latestSequence) {
return { delivered: [], readyToSend };
if (reassembled) {
if (reassembled.ordered) {
this.#orderedHeld.set(reassembled.frame.sequence, reassembled.frame);
} else if (!(packet.sequenced ?? false)) {
delivered.push(reassembled.frame);
} else {
const newestSequence = reassembled.nextSequence - 1;
const latestSequence = this.#latestSequencedByChannel.get(reassembled.frame.channelId);
if (latestSequence === undefined || newestSequence > latestSequence) {
this.#latestSequencedByChannel.set(reassembled.frame.channelId, newestSequence);
delivered.push(reassembled.frame);
}
}
this.#latestSequencedByChannel.set(reassembled.frame.channelId, newestSequence);
return { delivered: [reassembled.frame], readyToSend };
}
if (!reassembled.ordered) {
return { delivered: [reassembled.frame], readyToSend };
}
return {
delivered: this.#deliverOrdered(reassembled.frame, reassembled.nextSequence, expectedSequenceIfUninitialized),
readyToSend,
};
// Any reliable packet may have advanced the watermark, a fragment or an
// unordered frame as much as an ordered one, so the drain runs after all of
// them.
delivered.push(...this.#drainOrdered());
return { delivered, readyToSend };
}
createAck(): CultNetRudpPacket {
@ -490,7 +630,12 @@ export class CultNetRudpSession {
}
createAckForReceived(sequence: number): CultNetRudpPacket {
// What `receive` refused is not acknowledged by name (the ack carries only
// what was received), so its sender retransmits it.
const receivedSequence = toUint32(sequence, "received sequence");
if (!this.#wasReceived(receivedSequence)) {
return this.createAck();
}
const { ack } = this.#ackState();
return ack >= receivedSequence && ack - receivedSequence <= 32
? this.createAck()
@ -506,7 +651,7 @@ export class CultNetRudpSession {
}
createDisconnect(reason = new Uint8Array()): CultNetRudpPacket {
this.#connected = false;
this.#endSession();
return this.#createPacket({
packetType: "disconnect",
channelId: "control",
@ -521,11 +666,15 @@ export class CultNetRudpSession {
if (nowMs - this.#lastReceivedAtMs <= timeoutMs) {
return false;
}
this.#connected = false;
this.#endSession();
return true;
}
dueResends(nowMs: number): CultNetRudpPacket[] {
const fresh = this.#abandonUnansweredConnect(nowMs);
if (fresh) {
return [fresh];
}
const due: CultNetRudpPacket[] = [];
for (const pending of this.#pendingReliable.values()) {
if (nowMs - pending.lastSentAtMs >= this.resendDelayMs) {
@ -536,6 +685,27 @@ export class CultNetRudpSession {
return due.sort((left, right) => left.sequence - right.sequence);
}
/// A Connect unanswered for `RUDP_CONNECT_ATTEMPT_MS` is replaced, not
/// retransmitted further, by a Connect with the same payload and the
/// abandoned sequence plus the receive window less one. Nothing was ever sent
/// in an unanswered generation, so no sequence issued so far is owed. The jump
/// keeps a late copy of the abandoned Connect inside the new one's stale
/// window, so a server that took the new Connect answers the copy with an Ack
/// instead of restarting; and it leaves the stale window of whatever
/// generation the server holds, unless that generation sits exactly at the
/// jump, and then the next attempt leaves it. The first Connect of a session
/// is the only one drawn at random.
#abandonUnansweredConnect(nowMs: number): CultNetRudpPacket | undefined {
if (!this.#awaitingAccept || nowMs - this.#connectStartedAtMs < RUDP_CONNECT_ATTEMPT_MS) {
return undefined;
}
if (this.#connectSequence === undefined) {
return undefined;
}
this.#nextSequence = Math.min(this.#connectSequence + RUDP_RECEIVED_SEQUENCE_WINDOW - 1, 0xffff_fffe);
return this.createConnect(nowMs, this.#connectPayload);
}
#createPacket(packet: {
packetType: CultNetRudpPacketType;
channelId: string;
@ -616,8 +786,34 @@ export class CultNetRudpSession {
}
}
/// The handshake's one act on the watermark: the peer's Connect or Accept is
/// the first sequence of the session, whatever else it has sent.
#seedReceived(sequence: number): void {
this.#receivedThrough = sequence;
this.#rememberReceived(sequence);
}
/// The one duplicate test: true for a sequence in the window that was
/// received, for one below the window, and for one at or before the
/// watermark. The watermark starts at the handshake's seed, so a frame the
/// peer sent in an earlier generation (every sequence it issued is below the
/// Connect that began this one) is a duplicate of something already delivered
/// and is acknowledged, never delivered again.
#wasReceived(sequence: number): boolean {
return this.#receivedSequences.has(sequence)
|| (this.#highestReceivedSequence !== undefined
&& sequence < this.#highestReceivedSequence
&& this.#highestReceivedSequence - sequence >= RUDP_RECEIVED_SEQUENCE_WINDOW)
|| (this.#receivedThrough !== undefined && atOrBefore(sequence, this.#receivedThrough));
}
#rememberReceived(sequence: number): void {
this.#receivedSequences.add(sequence);
if (this.#receivedThrough !== undefined) {
let through = this.#receivedThrough;
while (this.#receivedSequences.has(through + 1)) through += 1;
this.#receivedThrough = through;
}
if (this.#highestReceivedSequence === undefined || sequence > this.#highestReceivedSequence) {
this.#highestReceivedSequence = sequence;
}
@ -734,68 +930,20 @@ export class CultNetRudpSession {
};
}
#deliverOrdered(
frame: CultNetRudpDeliveredFrame,
nextSequence: number,
expectedSequenceIfUninitialized: number,
): CultNetRudpDeliveredFrame[] {
let next = this.#orderedNextSequenceByChannel.get(frame.channelId);
if (next === undefined) {
next = Math.min(expectedSequenceIfUninitialized, frame.sequence);
this.#orderedNextSequenceByChannel.set(frame.channelId, next);
}
while (frame.sequence > next && this.#receivedSequences.has(next) && !this.#orderedBuffers.get(frame.channelId)?.has(next)) {
next += 1;
this.#orderedNextSequenceByChannel.set(frame.channelId, next);
}
if (frame.sequence < next) {
return [];
}
if (frame.sequence > next) {
let buffer = this.#orderedBuffers.get(frame.channelId);
if (!buffer) {
buffer = new Map();
this.#orderedBuffers.set(frame.channelId, buffer);
}
buffer.set(frame.sequence, { frame, nextSequence });
return [];
}
this.#orderedNextSequenceByChannel.set(frame.channelId, nextSequence);
return [
frame,
...this.#drainOrdered(frame.channelId),
];
}
#drainOrdered(channelId: string): CultNetRudpDeliveredFrame[] {
const delivered: CultNetRudpDeliveredFrame[] = [];
const buffer = this.#orderedBuffers.get(channelId);
if (!buffer) {
return delivered;
}
let next = this.#orderedNextSequenceByChannel.get(channelId);
while (next !== undefined && buffer.has(next)) {
const pending = buffer.get(next)!;
buffer.delete(next);
delivered.push(pending.frame);
next = pending.nextSequence;
this.#orderedNextSequenceByChannel.set(channelId, next);
this.#skipReceivedNonChannelSequences(channelId);
}
return delivered;
}
#skipReceivedNonChannelSequences(channelId: string): void {
let next = this.#orderedNextSequenceByChannel.get(channelId);
while (next !== undefined && this.#receivedSequences.has(next) && !this.#orderedBuffers.get(channelId)?.has(next)) {
next += 1;
this.#orderedNextSequenceByChannel.set(channelId, next);
}
/// Delivers, in sequence order, every held ordered frame whose first sequence
/// is at most one past the watermark. Ordered delivery has one owner: the
/// watermark. A frame this call does not deliver waits for the sequence below
/// it, and nothing else releases it.
#drainOrdered(): CultNetRudpDeliveredFrame[] {
if (this.#receivedThrough === undefined) return [];
const releasable = [...this.#orderedHeld.keys()]
.filter((sequence) => sequence <= this.#receivedThrough! + 1)
.sort((left, right) => left - right);
return releasable.map((sequence) => {
const frame = this.#orderedHeld.get(sequence)!;
this.#orderedHeld.delete(sequence);
return frame;
});
}
#allocateFragmentId(): number {
@ -824,12 +972,8 @@ export class CultNetRudpSocketTransportConnection extends EventEmitter implement
#remoteHost: string | undefined;
#remotePort: number | undefined;
#closed = false;
// A flush belongs to the session it started in. Each session start advances
// the generation; each session end records the generation it ended and why.
// A reconnect from a disconnect listener starts a new generation but cannot
// un-end the old one.
#generation = 0;
#endedGeneration = -1;
// Why the last generation ended. A report only: the session owns whether it
// ended.
#endedReason: Uint8Array | undefined;
readonly #stats: CultNetTransportStats = {
bytesReceived: 0,
@ -892,7 +1036,6 @@ export class CultNetRudpSocketTransportConnection extends EventEmitter implement
if (this.#mode !== "client") {
throw new Error("Only a client RUDP socket transport can initiate connect.");
}
this.#generation += 1;
this.#sendPacket(this.#session.createConnect(Date.now(), payload));
}
@ -910,11 +1053,14 @@ export class CultNetRudpSocketTransportConnection extends EventEmitter implement
async flush(timeoutMs = 30_000): Promise<void> {
const deadline = Date.now() + Math.max(0, timeoutMs);
const generation = this.#generation;
// A flush belongs to the generation it started in: whatever ends that
// generation, and whatever begins after it (a reconnect from a disconnect
// listener included), the writes being waited on are gone.
const generation = this.#session.generation;
for (;;) {
// An ended session forgot its unacknowledged writes; reporting them
// flushed would be a lie, even if a new session has since begun.
if (this.#endedGeneration >= generation) {
// flushed would be a lie.
if (this.#session.ended || this.#session.generation !== generation) {
throw new Error(
`RUDP session ended before its reliable writes were acknowledged: ${Buffer.from(this.#endedReason ?? new Uint8Array()).toString("utf8")}`,
);
@ -934,11 +1080,6 @@ export class CultNetRudpSocketTransportConnection extends EventEmitter implement
}
}
#endGeneration(reason: Uint8Array): void {
this.#endedGeneration = this.#generation;
this.#endedReason = reason;
}
ping(payload = new Uint8Array()): void {
this.#sendPacket(this.#session.createPing(payload));
}
@ -946,7 +1087,7 @@ export class CultNetRudpSocketTransportConnection extends EventEmitter implement
checkTimeout(timeoutMs: number, nowMs = Date.now()): boolean {
const timedOut = this.#session.checkTimeout(nowMs, timeoutMs);
if (timedOut) {
this.#endGeneration(Buffer.from("session timed out", "utf8"));
this.#endedReason = Buffer.from("session timed out", "utf8");
this.emit("timeout");
this.emit("close");
}
@ -973,7 +1114,7 @@ export class CultNetRudpSocketTransportConnection extends EventEmitter implement
// The goodbye is built after the reset, or its ack field would acknowledge
// the very frame the session refused.
this.#session.resetPeerState();
this.#endGeneration(reason);
this.#endedReason = reason;
try {
this.#sendPacket(this.#session.createDisconnect(reason));
} catch {
@ -1008,7 +1149,10 @@ export class CultNetRudpSocketTransportConnection extends EventEmitter implement
return;
}
let movedEndpoint = false;
// A Connect from another endpoint is a new client, whatever sequence it
// carries: the endpoint moves and the Connect starts a new generation. Only
// a Connect from the peer's own endpoint can be its repeat.
let connectFromNewEndpoint = false;
if (!this.#remoteHost || this.#remotePort === undefined) {
if (this.#mode === "server" && packet.packetType !== "connect") {
this.#stats.packetsDropped += 1;
@ -1018,9 +1162,9 @@ export class CultNetRudpSocketTransportConnection extends EventEmitter implement
this.#remotePort = remote.port;
} else if (remote.address !== this.#remoteHost || remote.port !== this.#remotePort) {
if (this.#mode === "server" && packet.packetType === "connect") {
movedEndpoint = true;
this.#remoteHost = remote.address;
this.#remotePort = remote.port;
connectFromNewEndpoint = true;
} else {
this.#stats.packetsDropped += 1;
return;
@ -1028,22 +1172,19 @@ export class CultNetRudpSocketTransportConnection extends EventEmitter implement
}
if (this.#mode === "server" && packet.packetType === "connect") {
// Only a Connect from the endpoint already accepted repeats, and the
// session answers it with the Accept owed. A Connect from another
// endpoint replaces the peer: the old session's writes die with it.
const repeated = this.#session.connected && !movedEndpoint;
if (!repeated) {
// A timed-out session already ended its generation; any other
// predecessor ends here, connected or not.
if (this.#endedGeneration < this.#generation) {
this.#endGeneration(Buffer.from("replaced by a new Connect", "utf8"));
}
// The session decides whether this Connect repeats the one it accepted; a
// repeat is answered with the Accept owed, anything else replaces the
// peer and the old session's writes die with it. A timed-out session
// already ended its generation; any other predecessor ends here.
if ((connectFromNewEndpoint || !this.#session.connectRepeats(packet)) && !this.#session.ended) {
this.#endedReason = Buffer.from("replaced by a new Connect", "utf8");
}
if (connectFromNewEndpoint) {
this.#session.resetPeerState();
}
let accept: CultNetRudpPacket;
try {
accept = this.#session.acceptConnect(packet, Date.now());
if (!repeated) this.#generation += 1;
} catch {
this.#stats.packetsDropped += 1;
return;
@ -1081,7 +1222,7 @@ export class CultNetRudpSocketTransportConnection extends EventEmitter implement
} satisfies CultNetTransportFrame);
}
if (result.disconnected) {
this.#endGeneration(result.disconnectReason ?? new Uint8Array());
this.#endedReason = result.disconnectReason ?? new Uint8Array();
this.emit("disconnect", { reason: result.disconnectReason ?? new Uint8Array() });
this.emit("close");
return;

289
packages/cultnet-ts/test/cultnet.test.ts
View file

@ -5,10 +5,10 @@ import { Duplex } from "node:stream";
import dgram, { type Socket } from "node:dgram";
import { rmSync, mkdtempSync } from "node:fs";
import { join } from "node:path";
import { tmpdir } from "node:os";
import { networkInterfaces, tmpdir } from "node:os";
import { z } from "zod";
import { encode } from "@msgpack/msgpack";
import { decode as decodeMsgpack, encode } from "@msgpack/msgpack";
import {
CultCache,
SchemaConflictError,
@ -49,6 +49,7 @@ import {
ghostlightAgentStateGeneratedContract,
parseCultNetMessage,
invokeCultNetOperation,
rudpClientBindHost,
startCultNetOperationServer,
validateGhostlightAgentStateGenerated,
validateGhostlightAgentState,
@ -168,6 +169,14 @@ function createDuplexPair(): { a: Duplex; b: Duplex } {
return { a, b };
}
/// Whether the packet's ack field and mask cover the sequence.
function acknowledges(packet: CultNetRudpPacket, sequence: number): boolean {
if (packet.ack === sequence) return true;
return packet.ack > sequence
&& packet.ack - sequence <= 32
&& (packet.ackMask & (1 << (packet.ack - sequence - 1))) !== 0;
}
async function bindUdpSocket(): Promise<Socket> {
const socket = dgram.createSocket("udp4");
await new Promise<void>((resolve) => socket.bind(0, "127.0.0.1", resolve));
@ -502,33 +511,32 @@ test("rudp session handshake acks reliable connect and accept packets", () => {
test("rudp session computes ack masks and clears pending reliable packets", () => {
const sender = new CultNetRudpSession({ connectionId: 7, initialSequence: 10, resendDelayMs: 100 });
const receiver = new CultNetRudpSession({ connectionId: 7, initialSequence: 200, resendDelayMs: 100 });
sender.receive({ packetType: "accept", connectionId: 7, sequence: 1, ack: 0, ackMask: 0, channelId: "control" });
receiver.receive({ packetType: "accept", connectionId: 7, sequence: 2, ack: 0, ackMask: 0, channelId: "control" });
sender.receive(receiver.acceptConnect(sender.createConnect(0), 0), 0);
const first = sender.send("schema", Buffer.from("first"), { reliable: true, ordered: true, nowMs: 0 });
const second = sender.send("schema", Buffer.from("second"), { reliable: true, ordered: true, nowMs: 0 });
const third = sender.send("schema", Buffer.from("third"), { reliable: true, ordered: true, nowMs: 0 });
assert.deepEqual(sender.pendingReliableSequences, [10, 11, 12]);
assert.deepEqual(sender.pendingReliableSequences, [11, 12, 13]);
receiver.receive(first);
receiver.receive(third);
const ackWithGap = receiver.createAck();
assert.equal(ackWithGap.ack, 12);
assert.equal(ackWithGap.ackMask, 0b10 | (1 << 9));
assert.equal(ackWithGap.ack, 13);
assert.equal(ackWithGap.ackMask, 0b110);
sender.receive(ackWithGap);
assert.deepEqual(sender.pendingReliableSequences, [11]);
assert.deepEqual(sender.pendingReliableSequences, [12]);
receiver.receive(second);
const fullAck = receiver.createAck();
assert.equal(fullAck.ack, 12);
assert.equal(fullAck.ackMask, 0b11 | (1 << 9));
assert.equal(fullAck.ack, 13);
assert.equal(fullAck.ackMask, 0b111);
sender.receive(fullAck);
assert.deepEqual(sender.pendingReliableSequences, []);
});
test("rudp session schedules reliable resends until acked", () => {
const session = new CultNetRudpSession({ connectionId: 99, initialSequence: 1, resendDelayMs: 100 });
session.receive({ packetType: "accept", connectionId: 99, sequence: 50, ack: 0, ackMask: 0, channelId: "control" });
session.receive(new CultNetRudpSession({ connectionId: 99, initialSequence: 900 }).acceptConnect(session.createConnect(0), 0), 0);
const sent = session.send("schema", Buffer.from("payload"), { reliable: true, ordered: true, nowMs: 10 });
assert.deepEqual(session.dueResends(90), []);
@ -599,7 +607,7 @@ test("rudp unreliable sequenced delivery is scoped to its channel", () => {
test("rudp rejects unreliable ordered delivery", () => {
const session = new CultNetRudpSession({ connectionId: 196 });
session.receive({ packetType: "accept", connectionId: 196, sequence: 50, ack: 0, ackMask: 0, channelId: "control" });
session.receive(new CultNetRudpSession({ connectionId: 196, initialSequence: 900 }).acceptConnect(session.createConnect(0), 0), 0);
assert.throws(
() => session.send("schema", Buffer.from("cannot order what will not retransmit"), { ordered: true }),
/ordered delivery requires reliability/,
@ -608,7 +616,7 @@ test("rudp rejects unreliable ordered delivery", () => {
test("rudp session bounds pending reliable packets before enqueue", () => {
const session = new CultNetRudpSession({ connectionId: 102, initialSequence: 1, maxPendingReliablePackets: 2 });
session.receive({ packetType: "accept", connectionId: 102, sequence: 50, ack: 0, ackMask: 0, channelId: "control" });
session.receive(new CultNetRudpSession({ connectionId: 102, initialSequence: 900 }).acceptConnect(session.createConnect(0), 0), 0);
session.send("schema", Buffer.from("first"), { reliable: true, ordered: true });
session.send("schema", Buffer.from("second"), { reliable: true, ordered: true });
@ -616,10 +624,10 @@ test("rudp session bounds pending reliable packets before enqueue", () => {
() => session.send("schema", Buffer.from("third"), { reliable: true, ordered: true }),
/reliable send queue is full/,
);
assert.deepEqual(session.pendingReliableSequences, [1, 2]);
assert.deepEqual(session.pendingReliableSequences, [2, 3]);
const fragmented = new CultNetRudpSession({ connectionId: 103, initialSequence: 1, maxPendingReliablePackets: 3 });
fragmented.receive({ packetType: "accept", connectionId: 103, sequence: 50, ack: 0, ackMask: 0, channelId: "control" });
fragmented.receive(new CultNetRudpSession({ connectionId: 103, initialSequence: 900 }).acceptConnect(fragmented.createConnect(0), 0), 0);
assert.throws(
() => fragmented.sendMany("schema", Buffer.from("fragment-me"), { reliable: true, ordered: true, maxFragmentBytes: 3 }),
/reliable send queue is full/,
@ -630,8 +638,7 @@ test("rudp session bounds pending reliable packets before enqueue", () => {
test("rudp session suppresses duplicates and delivers reliable ordered payloads in sequence", () => {
const sender = new CultNetRudpSession({ connectionId: 123, initialSequence: 1 });
const receiver = new CultNetRudpSession({ connectionId: 123, initialSequence: 100 });
sender.receive({ packetType: "accept", connectionId: 123, sequence: 90, ack: 0, ackMask: 0, channelId: "control" });
receiver.receive({ packetType: "accept", connectionId: 123, sequence: 91, ack: 0, ackMask: 0, channelId: "control" });
sender.receive(receiver.acceptConnect(sender.createConnect(0), 0));
const first = sender.send("schema", Buffer.from("first"), { reliable: true, ordered: true });
const second = sender.send("schema", Buffer.from("second"), { reliable: true, ordered: true });
@ -649,8 +656,7 @@ test("rudp session suppresses duplicates and delivers reliable ordered payloads
test("rudp session skips received control packets while ordering schema payloads", () => {
const sender = new CultNetRudpSession({ connectionId: 124, initialSequence: 1 });
const receiver = new CultNetRudpSession({ connectionId: 124, initialSequence: 100 });
sender.receive({ packetType: "accept", connectionId: 124, sequence: 90, ack: 0, ackMask: 0, channelId: "control" });
receiver.receive({ packetType: "accept", connectionId: 124, sequence: 91, ack: 0, ackMask: 0, channelId: "control" });
sender.receive(receiver.acceptConnect(sender.createConnect(0), 0), 0);
const first = sender.send("schema", Buffer.from("first"), { reliable: true, ordered: true });
const control = sender.createAck();
@ -667,8 +673,7 @@ test("rudp session skips received control packets while ordering schema payloads
test("rudp session fragments and reassembles reliable ordered payloads", () => {
const sender = new CultNetRudpSession({ connectionId: 456, initialSequence: 1 });
const receiver = new CultNetRudpSession({ connectionId: 456, initialSequence: 100 });
sender.receive({ packetType: "accept", connectionId: 456, sequence: 90, ack: 0, ackMask: 0, channelId: "control" });
receiver.receive({ packetType: "accept", connectionId: 456, sequence: 91, ack: 0, ackMask: 0, channelId: "control" });
sender.receive(receiver.acceptConnect(sender.createConnect(0), 0), 0);
const packets = sender.sendMany("schema", Buffer.from("fragment-me-please"), {
reliable: true,
@ -1427,8 +1432,7 @@ test("rudp session advances large fragment sets through a bounded reliable windo
const connectionId = 457;
const sender = new CultNetRudpSession({ connectionId, initialSequence: 1, resendDelayMs: 25 });
const receiver = new CultNetRudpSession({ connectionId, initialSequence: 100, resendDelayMs: 25 });
sender.receive({ packetType: "accept", connectionId, sequence: 0, ack: 0, ackMask: 0, channelId: "control" });
receiver.receive({ packetType: "accept", connectionId, sequence: 0, ack: 0, ackMask: 0, channelId: "control" });
sender.receive(receiver.acceptConnect(sender.createConnect(0), 0), 0);
const fragmentCount = CULTNET_RUDP_RELIABLE_SEND_WINDOW_PACKETS + 17;
const payload = Uint8Array.from(
@ -1578,14 +1582,13 @@ test("RUDP operation service correlates typed payload envelopes", async () => {
test("rudp sequence-neutral acknowledgements interoperate with ordered receivers", () => {
const sender = new CultNetRudpSession({ connectionId: 125, initialSequence: 1 });
const receiver = new CultNetRudpSession({ connectionId: 125, initialSequence: 100 });
sender.receive({ packetType: "accept", connectionId: 125, sequence: 90, ack: 0, ackMask: 0, channelId: "control" });
receiver.receive({ packetType: "accept", connectionId: 125, sequence: 91, ack: 0, ackMask: 0, channelId: "control" });
sender.receive(receiver.acceptConnect(sender.createConnect(0), 0), 0);
const ack = sender.createAck();
const request = sender.send("schema", Buffer.from("snapshot"), { reliable: true, ordered: true });
assert.equal(ack.sequence, 0);
assert.equal(request.sequence, 1);
assert.equal(request.sequence, 2);
assert.deepEqual(receiver.receive(ack).delivered, []);
assert.deepEqual(receiver.receive(request).delivered.map(frame => Buffer.from(frame.payload).toString("utf8")), ["snapshot"]);
});
@ -1926,7 +1929,7 @@ test("rudp receiver evicts stranded fragment sets instead of refusing new ones",
// Heimdall fragmented envelopes (max_fragment_bytes = 2048), so on the live
// auth path this map grew for every lost fragment and was never swept.
const receiver = new CultNetRudpSession({ connectionId: 9, initialSequence: 1, resendDelayMs: 50 });
receiver.receive({ packetType: "accept", connectionId: 9, sequence: 1, ack: 0, ackMask: 0, channelId: "control" });
receiver.acceptConnect(new CultNetRudpSession({ connectionId: 9, initialSequence: 1 }).createConnect(0), 0);
// 200 fragment sets that each lose their second fragment.
for (let id = 1; id <= 200; id += 1) {
@ -2070,7 +2073,7 @@ test("rudp socket transport ends the session that refuses a reliable frame and n
// neither in the goodbye nor after it retransmits.
const goodbye = received.find((p) => p.packetType === "disconnect")!;
assert.equal(peer.receive(goodbye, 1).disconnected, true);
assert.ok(peer.pendingReliableSequences.includes(poison.sequence), "the refused frame was acknowledged");
assert.equal(acknowledges(goodbye, poison.sequence), false, "the goodbye acknowledged the refused frame");
const before = received.length;
toServer(poison);
await waitFor(() => server.stats.packetsDropped === 2, "the retransmit dropped");
@ -2316,6 +2319,47 @@ test("operation service frees the slots of abandoned sessions once they idle out
}
});
test("an RUDP client binds loopback only for a loopback endpoint", async () => {
assert.equal(await rudpClientBindHost("10.77.0.1"), "0.0.0.0");
assert.equal(await rudpClientBindHost("2001:db8::1"), "::");
assert.equal(await rudpClientBindHost("[2001:db8::1]"), "::");
assert.equal(await rudpClientBindHost("127.0.0.1"), "127.0.0.1");
assert.equal(await rudpClientBindHost("127.0.0.2"), "127.0.0.1");
assert.equal(await rudpClientBindHost("::1"), "::1");
assert.equal(await rudpClientBindHost("[::1]"), "::1");
});
// A socket bound to loopback cannot send off the host on Windows. On Linux it
// reaches a local non-loopback address, but from 127.0.0.1, so the source
// address shows which one the client bound.
test("an operation client for a non-loopback endpoint does not send from loopback", async (t) => {
const address = Object.values(networkInterfaces())
.flat()
.find((entry) => entry && entry.family === "IPv4" && !entry.internal)?.address;
if (!address) {
t.skip("no non-loopback IPv4 interface on this host");
return;
}
const listener = dgram.createSocket("udp4");
await new Promise<void>((resolve) => listener.bind(0, "0.0.0.0", resolve));
const source = new Promise<string>((resolve) => listener.once("message", (_wire, remote) => resolve(remote.address)));
const request: CultNetOperationRequestMessage = {
schemaVersion: "cultnet.operation_request.v0",
messageId: "bind-host",
serviceId: "sai.vn",
operation: "describe",
payloadSchema: "gamecult.eve.plugin_abi.request.v1",
payloadEncoding: "messagepack-base64",
payload: "gaZzY2hlbWE=",
};
try {
await assert.rejects(invokeCultNetOperation(`rudp://${address}:${udpPort(listener)}`, request, { runtimeId: "bind-test", timeoutMs: 300 }));
assert.equal(await source, address);
} finally {
listener.close();
}
});
test("server-mode transport replaces its peer when a Connect arrives from another endpoint", async () => {
const serverSocket = await bindUdpSocket();
const socketA = await bindUdpSocket();
@ -2447,7 +2491,7 @@ test("a repeated Connect acknowledges what it carries", () => {
const server = new CultNetRudpSession({ connectionId });
const connect = client.createConnect(0);
const accept = server.acceptConnect(connect, 0);
const reply = server.answerRepeatedConnect({ ...connect, ack: accept.sequence }, 1);
const reply = server.acceptConnect({ ...connect, ack: accept.sequence }, 1);
assert.equal(reply.packetType, "ack");
assert.equal(server.outstandingReliablePacketCount, 0);
});
@ -2584,3 +2628,190 @@ test("operation service sweeps a session that ran a handler once it idles out",
await server.close();
}
});
test("a flush started after a reconnect waits on the new session", async () => {
const peerSocket = await bindUdpSocket();
const clientSocket = await bindUdpSocket();
const connectionId = 0x1020305b;
const client = new CultNetRudpSocketTransportConnection({
runtimeId: "rudp-client",
socket: clientSocket,
mode: "client",
remoteHost: "127.0.0.1",
remotePort: udpPort(peerSocket),
connectionId,
resendDelayMs: 1000,
resendPollMs: 1000,
});
const received: CultNetRudpPacket[] = [];
let clientPort = 0;
peerSocket.on("message", (wire, remote) => {
clientPort = remote.port;
received.push(decodeRudpPacket(wire));
});
try {
client.connect();
await waitFor(() => received.some((p) => p.packetType === "connect"), "the Connect");
const first = new CultNetRudpSession({ connectionId });
peerSocket.send(encodeRudpPacket(first.acceptConnect(received[0]!, 0)), clientPort, "127.0.0.1");
await waitFor(() => client.connected, "the client connected");
client.send("schema", Buffer.from("owed to the old session"));
client.checkTimeout(50, Date.now() + 10_000);
assert.equal(client.connected, false);
client.connect();
await waitFor(() => received.filter((p) => p.packetType === "connect").length === 2, "the reconnect");
const second = new CultNetRudpSession({ connectionId });
const reconnect = received.filter((p) => p.packetType === "connect")[1]!;
peerSocket.send(encodeRudpPacket(second.acceptConnect(reconnect, 0)), clientPort, "127.0.0.1");
await waitFor(() => client.connected, "the client reconnected");
await client.flush(500);
} finally {
peerSocket.close();
client.close();
}
});
test("a flush started after the server accepts a new Connect waits on the new session", async () => {
const serverSocket = await bindUdpSocket();
const peerSocket = await bindUdpSocket();
const connectionId = 0x1020305c;
const server = new CultNetRudpSocketTransportConnection({
runtimeId: "rudp-server",
socket: serverSocket,
mode: "server",
connectionId,
resendDelayMs: 1000,
resendPollMs: 1000,
});
const received: CultNetRudpPacket[] = [];
peerSocket.on("message", (wire) => received.push(decodeRudpPacket(wire)));
const toServer = (packet: CultNetRudpPacket) =>
peerSocket.send(encodeRudpPacket(packet), udpPort(serverSocket), "127.0.0.1");
try {
const first = new CultNetRudpSession({ connectionId });
toServer(first.createConnect(0));
await waitFor(() => received.some((p) => p.packetType === "accept"), "the first Accept");
toServer(first.createDisconnect(Buffer.from("bye")));
await waitFor(() => !server.connected, "the first session ended");
const next = new CultNetRudpSession({ connectionId });
toServer(next.createConnect(0));
await waitFor(() => received.filter((p) => p.packetType === "accept").length === 2, "the second Accept");
const accept = received.filter((p) => p.packetType === "accept")[1]!;
next.receive(accept, 0);
toServer(next.createAckForReceived(accept.sequence));
await waitFor(() => server.outstandingReliablePacketCount === 0, "the second Accept acknowledged");
await server.flush(500);
} finally {
peerSocket.close();
server.close();
}
});
test("server-mode transport admits a restarted client and keeps its session when a Connect is retransmitted", async () => {
const serverSocket = await bindUdpSocket();
const peerSocket = await bindUdpSocket();
const connectionId = 0x10203059;
const server = new CultNetRudpSocketTransportConnection({
runtimeId: "rudp-server",
socket: serverSocket,
mode: "server",
connectionId,
resendPollMs: 1000,
});
let received: CultNetRudpPacket[] = [];
peerSocket.on("message", (wire) => received.push(decodeRudpPacket(wire)));
const frames: string[] = [];
server.on("frame", (frame: { payload: Uint8Array }) => frames.push(Buffer.from(frame.payload).toString("utf8")));
const toServer = (packet: CultNetRudpPacket) =>
peerSocket.send(encodeRudpPacket(packet), udpPort(serverSocket), "127.0.0.1");
const reliable = { reliable: true, ordered: true };
try {
const first = new CultNetRudpSession({ connectionId, initialSequence: 50_000 });
const connect = first.createConnect(0, Buffer.from("same"));
toServer(connect);
await waitFor(() => received.some((p) => p.packetType === "accept"), "the first Accept");
first.receive(received.find((p) => p.packetType === "accept")!, 0);
const sent = [0, 1, 2].map((index) => first.send("schema", Buffer.from(`c${index}`), reliable));
sent.forEach(toServer);
await waitFor(() => frames.length === 3, "the first client's frames");
// A retransmitted Connect is a repeat: what was delivered stays delivered once.
toServer(connect);
toServer(sent[0]!);
await new Promise((resolve) => setTimeout(resolve, 100));
assert.deepEqual(frames, ["c0", "c1", "c2"]);
// The same client restarted on the same address, connection id and payload.
received = [];
const second = new CultNetRudpSession({ connectionId, initialSequence: 7 });
toServer(second.createConnect(0, Buffer.from("same")));
await waitFor(() => received.some((p) => p.packetType === "accept"), "the restarted client's Accept");
second.receive(received.find((p) => p.packetType === "accept")!, 0);
toServer(second.send("schema", Buffer.from("after restart"), reliable));
await waitFor(() => frames.includes("after restart"), "the restarted client's frame");
} finally {
peerSocket.close();
server.close();
}
});
test("RUDP operation service admits a restarted client on the same address", async () => {
const server = await startCultNetOperationServer({
runtimeId: "sai-sidecar",
handler: request => ({
schemaVersion: "cultnet.operation_response.v0",
messageId: request.messageId,
serviceId: request.serviceId,
operation: request.operation,
status: "ok",
payloadSchema: "gamecult.eve.plugin_abi.response.v1",
payloadEncoding: "messagepack-base64",
payload: request.payload,
diagnostics: [],
sourceRuntimeId: "sai-sidecar",
}),
});
const connectionId = 0x43554c54;
const port = Number(new URL(server.endpoint).port);
const socket = await bindUdpSocket();
let session = new CultNetRudpSession({ connectionId, initialSequence: 50_000 });
const answered: string[] = [];
socket.on("message", (wire) => {
const packet = decodeRudpPacket(wire);
const result = session.receive(packet, Date.now());
if (packet.packetType === "accept" || packet.reliable) {
socket.send(encodeRudpPacket(session.createAckForReceived(packet.sequence)), port, "127.0.0.1");
}
for (const frame of result.delivered) {
answered.push((decodeMsgpack(frame.payload) as { messageId: string }).messageId);
}
});
const toServer = (packet: CultNetRudpPacket) => socket.send(encodeRudpPacket(packet), port, "127.0.0.1");
const request = (messageId: string) => Buffer.from(encode(encodeCultNetMessageForWire({
schemaVersion: "cultnet.operation_request.v0",
messageId,
serviceId: "sai.vn",
operation: "describe",
payloadSchema: "gamecult.eve.plugin_abi.request.v1",
payloadEncoding: "messagepack-base64",
payload: "gaZzY2hlbWE=",
} satisfies CultNetOperationRequestMessage)));
try {
toServer(session.createConnect(Date.now()));
await waitFor(() => session.connected, "the first Accept");
toServer(session.send("schema", request("before-restart"), { reliable: true, ordered: true }));
await waitFor(() => answered.includes("before-restart"), "the first answer");
session = new CultNetRudpSession({ connectionId, initialSequence: 7 });
toServer(session.createConnect(Date.now()));
await waitFor(() => session.connected, "the restarted client's Accept");
toServer(session.send("schema", request("after-restart"), { reliable: true, ordered: true }));
await waitFor(() => answered.includes("after-restart"), "the restarted client's answer");
} finally {
socket.close();
await server.close();
}
});

535
packages/cultnet-ts/test/rudp-connect-generation.test.ts Normal file
View file

@ -0,0 +1,535 @@
// A Connect's sequence says whether it repeats the one a session accepted, and
// the handshake alone seeds the watermark that orders delivery.
import assert from "node:assert/strict";
import test from "node:test";
import { createSocket, type Socket } from "node:dgram";
import { once } from "node:events";
import {
CultNetRudpSession,
CultNetRudpSocketTransportConnection,
decodeRudpPacket,
encodeRudpPacket,
type CultNetRudpPacket,
} from "../src";
const connectionId = 0x10203080;
const enc = (text: string) => new TextEncoder().encode(text);
const dec = (bytes: Uint8Array) => new TextDecoder().decode(bytes);
const session = (initialSequence: number) => new CultNetRudpSession({ connectionId, initialSequence });
const send = (from: CultNetRudpSession, text: string) => from.send("schema", enc(text), { reliable: true, ordered: true });
const names = (result: { delivered: { payload: Uint8Array }[] }) => result.delivered.map((frame) => dec(frame.payload));
function handshake(client: CultNetRudpSession, server: CultNetRudpSession): { connect: CultNetRudpPacket; accept: CultNetRudpPacket } {
const connect = client.createConnect(0);
const accept = server.acceptConnect(connect, 0);
client.receive(accept, 0);
assert.ok(client.connected && server.connected);
return { connect, accept };
}
test("a reconnect on the same session is not stranded by a write the ended session owed", () => {
const client = session(1);
const server = session(500);
handshake(client, server);
send(server, "lost");
server.createDisconnect(enc("gone"));
client.receive(server.acceptConnect(client.createConnect(1), 1), 1);
assert.deepEqual(names(client.receive(send(server, "after"), 2)), ["after"]);
});
test("a server that accepts a new client forgets the old client's sequences", () => {
const oldClient = session(50);
const server = session(500);
handshake(oldClient, server);
for (const name of ["c1", "c2", "c3"]) {
assert.deepEqual(names(server.receive(send(oldClient, name), 1)), [name]);
}
server.createDisconnect(enc("restart"));
const newClient = session(2);
newClient.receive(server.acceptConnect(newClient.createConnect(2), 2), 2);
assert.deepEqual(names(server.receive(send(newClient, "fresh"), 3)), ["fresh"]);
});
test("a retransmitted Connect after data has flowed does not reset the session", () => {
const client = session(1);
const server = session(500);
const { connect } = handshake(client, server);
const frame = send(client, "once");
assert.deepEqual(names(server.receive(frame, 1)), ["once"]);
assert.equal(server.connectRepeats(connect), true);
assert.equal(server.acceptConnect(connect, 2).packetType, "ack", "the Accept was already acknowledged");
assert.equal(server.outstandingReliablePacketCount, 0, "a repeat queued something");
assert.deepEqual(server.receive(frame, 3).delivered, [], "a retransmitted Connect made the server forget what it had delivered");
});
test("a Connect with another sequence is not a repeat whatever it carries", () => {
const server = session(500);
const connect = session(1).createConnect(0, enc("same payload"));
server.acceptConnect(connect, 0);
assert.equal(server.connectRepeats(connect), true);
assert.equal(server.connectRepeats(session(9).createConnect(0, enc("same payload"))), false);
assert.equal(server.connectRepeats({ ...connect, packetType: "ping" }), false);
server.createDisconnect();
assert.equal(server.connectRepeats(connect), false, "an ended session repeats nothing");
});
test("an Accept that does not name the pending Connect is ignored", () => {
const client = session(10);
const server = session(500);
const connect = client.createConnect(0);
const accept = server.acceptConnect(connect, 0);
client.receive({ ...accept, ack: connect.sequence + 50, ackMask: 0 }, 1);
assert.equal(client.connected, false, "a stale Accept connected the session");
assert.deepEqual(client.pendingReliableSequences, [connect.sequence]);
client.receive({ ...accept, ack: connect.sequence + 3, ackMask: 1 << 2 }, 2);
assert.equal(client.connected, true, "the mask names the Connect");
});
test("an Accept after a local disconnect does not reconnect the session", () => {
const client = session(1);
const server = session(500);
const { accept } = handshake(client, server);
client.createDisconnect(enc("bye"));
assert.equal(client.connected, false);
client.receive(accept, 5);
assert.equal(client.connected, false, "a late Accept revived an ended session");
});
test("a reconnect forgets what the old server sent", () => {
const client = session(1);
const oldServer = session(500);
handshake(client, oldServer);
for (const name of ["d1", "d2", "d3"]) {
assert.deepEqual(names(client.receive(send(oldServer, name), 1)), [name]);
}
const newServer = session(501);
client.receive(newServer.acceptConnect(client.createConnect(2), 2), 2);
assert.deepEqual(names(client.receive(send(newServer, "fresh"), 3)), ["fresh"]);
});
test("an Accept for an abandoned Connect does not connect the session", () => {
const client = session(1);
const server = session(500);
const accept = server.acceptConnect(client.createConnect(0), 0);
client.createDisconnect(enc("never mind"));
client.receive(accept, 1);
assert.equal(client.connected, false, "an Accept revived an abandoned Connect");
});
test("a full queue owed to a vanished peer does not refuse a new generation", () => {
const bounded = (initialSequence: number) => new CultNetRudpSession({ connectionId, initialSequence, maxPendingReliablePackets: 1 });
const server = bounded(500);
server.acceptConnect(session(1).createConnect(0), 0);
assert.equal(server.outstandingReliablePacketCount, 1, "the Accept is owed and never acknowledged");
const next = session(9).createConnect(1);
server.acceptConnect(next, 1);
assert.equal(server.connectRepeats(next), true);
const client = bounded(1);
client.createConnect(0);
client.createConnect(1);
assert.equal(client.outstandingReliablePacketCount, 1, "only the new Connect is owed");
});
test("a duplicate Accept does not reseed the watermark", () => {
const client = session(1);
const server = session(500);
const { accept } = handshake(client, server);
const s1 = send(server, "s1");
const s2 = send(server, "s2");
client.receive({ ...accept, sequence: s2.sequence }, 1);
assert.deepEqual(client.receive(s2, 2).delivered, [], "s1 is still missing");
assert.deepEqual(names(client.receive(s1, 3)), ["s1", "s2"]);
});
test("reliable data before the Accept is neither delivered nor acknowledged", () => {
const client = session(1);
const server = session(500);
const accept = server.acceptConnect(client.createConnect(0), 0);
const a = send(server, "A");
const b = send(server, "B");
for (const early of [b, a]) {
assert.deepEqual(client.receive(early, 1).delivered, []);
const ack = client.createAckForReceived(early.sequence);
assert.deepEqual([ack.ack, ack.ackMask], [0, 0], "an unhandled packet was acknowledged");
}
client.receive(accept, 2);
assert.ok(client.connected);
assert.equal(server.outstandingReliablePacketCount, 3, "nothing was acknowledged");
assert.deepEqual(names(client.receive(a, 3)), ["A"]);
assert.deepEqual(names(client.receive(b, 3)), ["B"]);
});
test("ordered frames after the Accept are delivered in order whatever arrives first", () => {
const client = session(1);
const server = session(500);
handshake(client, server);
const a = send(server, "A");
const b = send(server, "B");
assert.deepEqual(client.receive(b, 1).delivered, []);
assert.deepEqual(names(client.receive(a, 2)), ["A", "B"]);
});
test("an unordered reliable frame is delivered on receipt, never held behind a gap", () => {
const client = session(1);
const server = session(500);
handshake(client, server);
send(server, "lost");
const unordered = server.send("media", enc("now"), { reliable: true, ordered: false });
const ordered = send(server, "later");
assert.deepEqual(names(client.receive(unordered, 1)), ["now"]);
assert.deepEqual(client.receive(ordered, 2).delivered, [], "the ordered frame waits for the gap");
});
test("a peer Disconnect does not forget what the peer sent", () => {
const client = session(1);
const server = session(500);
handshake(client, server);
const s1 = send(client, "s1");
const s2 = send(client, "s2");
assert.deepEqual(server.receive(s2, 1).delivered, [], "s1 is missing, so s2 is held");
server.receive(client.createDisconnect(enc("bye")), 2);
assert.deepEqual(names(server.receive(s1, 3)), ["s1", "s2"], "the Disconnect forgot the held frame");
assert.deepEqual(server.receive(s2, 4).delivered, [], "the Disconnect forgot what was received");
});
test("sessions from one options object each draw their own initial sequence from one to two to the thirty first", () => {
const options = { connectionId };
const firsts = new Set(Array.from({ length: 64 }, () => new CultNetRudpSession(options).createConnect(0).sequence));
assert.ok(firsts.size > 32, `a default session does not draw its own sequence: ${[...firsts]}`);
assert.ok([...firsts].every((value) => value >= 1 && value < 2 ** 31), `left [1, 2^31): ${[...firsts]}`);
});
// Ack Cut 1d: a frame from an earlier generation is a duplicate, a delayed
// Connect from an earlier attempt is stale, a Connect from a new endpoint is a
// new client.
const namesSequence = (ack: CultNetRudpPacket, sequence: number) =>
ack.ack === sequence
|| Array.from({ length: 32 }, (_, bit) => bit).some(
(bit) => (ack.ackMask & (1 << bit)) !== 0 && ack.ack - bit - 1 === sequence,
);
test("a frame from the client's earlier generation is acknowledged, not delivered again", () => {
const client = session(10);
const server = session(500);
handshake(client, server);
const old = send(client, "old");
assert.deepEqual(names(server.receive(old, 1)), ["old"]);
const connect = client.createConnect(2);
client.receive(server.acceptConnect(connect, 2), 2);
assert.ok(old.sequence < connect.sequence);
assert.deepEqual(server.receive(old, 3).delivered, [], "an earlier generation's frame was delivered again");
assert.ok(namesSequence(server.createAckForReceived(old.sequence), old.sequence), "the duplicate was not acknowledged");
assert.deepEqual(names(server.receive(send(client, "fresh"), 4)), ["fresh"]);
});
test("a frame from the server's earlier generation is acknowledged, not delivered again", () => {
const client = session(10);
const server = session(500);
handshake(client, server);
const old = send(server, "old");
assert.deepEqual(names(client.receive(old, 1)), ["old"]);
client.receive(server.acceptConnect(client.createConnect(2), 2), 2);
assert.deepEqual(client.receive(old, 3).delivered, [], "an earlier generation's frame was delivered again");
assert.ok(namesSequence(client.createAckForReceived(old.sequence), old.sequence));
assert.deepEqual(names(client.receive(send(server, "fresh"), 4)), ["fresh"]);
});
test("a duplicate below the receive window is acknowledged by name", () => {
const client = session(10);
const server = session(500);
handshake(client, server);
const first = send(client, "first");
assert.deepEqual(names(server.receive(first, 1)), ["first"]);
for (let index = 0; index < 4_200; index += 1) {
server.receive(send(client, "x"), 1);
client.receive(server.createAck(), 1);
}
assert.deepEqual(server.receive(first, 2).delivered, []);
assert.equal(server.createAckForReceived(first.sequence).ack, first.sequence, "a duplicate below the window was not acknowledged by name");
});
test("a delayed Connect from an earlier attempt does not strand the client", () => {
const client = session(10);
const server = session(500);
const earlier = client.createConnect(0);
const retried = client.createConnect(300);
const accept = server.acceptConnect(retried, 301);
client.receive(accept, 302);
server.receive(client.createAckForReceived(accept.sequence), 302);
assert.ok(client.connected);
const a = send(client, "a");
assert.deepEqual(names(server.receive(a, 303)), ["a"]);
client.receive(server.createAckForReceived(a.sequence), 303);
assert.equal(server.connectRepeats(earlier), true, "the delayed Connect starts nothing");
const reply = server.acceptConnect(earlier, 304);
assert.equal(reply.packetType, "ack");
client.receive(reply, 305);
assert.ok(client.connected);
const b = send(client, "b");
assert.deepEqual(names(server.receive(b, 306)), ["b"], "the delayed Connect reset the server");
client.receive(server.createAckForReceived(b.sequence), 307);
assert.ok(!client.pendingReliableSequences.includes(b.sequence));
});
test("a repeated Connect refreshes liveness and a stale one does not", () => {
const client = session(10);
const server = session(500);
const earlier = client.createConnect(0);
const current = client.createConnect(1);
server.acceptConnect(current, 0);
server.acceptConnect(current, 900);
assert.equal(server.checkTimeout(1_000, 500), false, "a repeated Connect did not refresh liveness");
server.acceptConnect(earlier, 1_400);
assert.equal(server.checkTimeout(1_600, 500), true, "a stale Connect refreshed liveness");
});
test("stale Connects are recognised by serial arithmetic", () => {
const server = session(500);
server.acceptConnect(session(3).createConnect(0), 0);
const connect = (sequence: number) => session(sequence).createConnect(0);
assert.equal(server.connectRepeats(connect(2 ** 32 - 2)), true, "just before the wrap");
assert.equal(server.connectRepeats(connect(4)), false, "just after");
assert.equal(server.connectRepeats(connect(3 + 4_096)), false, "far after");
assert.equal(server.connectRepeats(connect(2 ** 32 - 4_092)), true, "the window's edge");
assert.equal(server.connectRepeats(connect(2 ** 32 - 4_093)), false, "past the window");
});
test("a restarted client whose first sequence is stale connects after redrawing", () => {
const server = session(500);
const old = session(1_000);
const oldAccept = server.acceptConnect(old.createConnect(0), 0);
old.receive(oldAccept, 0);
const restarted = session(900);
const first = restarted.createConnect(0, enc("join"));
// The old client has not acknowledged its Accept yet, so that is the reply:
// it names the old Connect, not this one.
const early = server.acceptConnect(first, 1);
assert.equal(early.packetType, "accept");
restarted.receive(early, 1);
assert.equal(restarted.connected, false, "an Accept for another Connect connected the client");
server.receive(old.createAckForReceived(oldAccept.sequence), 1);
const reply = server.acceptConnect(first, 2);
assert.equal(reply.packetType, "ack");
restarted.receive(reply, 2);
assert.equal(restarted.connected, false);
const retransmitted = restarted.dueResends(1_000);
assert.equal(retransmitted.length, 1);
assert.equal(retransmitted[0]!.sequence, first.sequence, "the attempt is still young");
const fresh = restarted.dueResends(3_500);
assert.equal(fresh.length, 1);
assert.equal(fresh[0]!.packetType, "connect");
assert.notEqual(fresh[0]!.sequence, first.sequence, "the same Connect was retransmitted for ever");
assert.equal(dec(fresh[0]!.payload ?? new Uint8Array()), "join", "the fresh attempt lost the Connect payload");
const accept = server.acceptConnect(fresh[0]!, 3_500);
assert.equal(accept.packetType, "accept");
restarted.receive(accept, 3_500);
assert.ok(restarted.connected);
assert.deepEqual(names(server.receive(send(restarted, "hello"), 3_501)), ["hello"]);
});
test("an answered Connect is never replaced by the attempt timeout", () => {
const client = session(10);
const server = session(500);
const { connect } = handshake(client, server);
const resent = client.dueResends(60_000);
assert.ok(resent.every((packet) => packet.packetType !== "connect" || packet.sequence === connect.sequence), "a connected client started a new attempt");
assert.ok(client.connected);
});
async function bind(): Promise<Socket> {
const socket = createSocket("udp4");
socket.bind(0, "127.0.0.1");
await once(socket, "listening");
return socket;
}
const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
test("server mode admits a pinned client that restarts on a new port", async () => {
const serverSocket = await bind();
const a = await bind();
const b = await bind();
const server = new CultNetRudpSocketTransportConnection({
runtimeId: "srv", socket: serverSocket, mode: "server", connectionId, resendPollMs: 20, resendDelayMs: 20,
});
const frames: string[] = [];
server.on("frame", (frame: { payload: Uint8Array }) => frames.push(dec(frame.payload)));
const port = (serverSocket.address() as { port: number }).port;
const atA: CultNetRudpPacket[] = [];
const atB: CultNetRudpPacket[] = [];
a.on("message", (wire) => atA.push(decodeRudpPacket(wire)));
b.on("message", (wire) => atB.push(decodeRudpPacket(wire)));
try {
const first = session(1);
a.send(encodeRudpPacket(first.createConnect(0, enc("join"))), port, "127.0.0.1");
await sleep(100);
const accept = atA.find((packet) => packet.packetType === "accept");
assert.ok(accept);
first.receive(accept, 0);
a.send(encodeRudpPacket(first.createAckForReceived(accept.sequence)), port, "127.0.0.1");
a.send(encodeRudpPacket(send(first, "hello")), port, "127.0.0.1");
await sleep(100);
assert.deepEqual(frames, ["hello"]);
const second = session(1); // the process restarted on a new port, pinned
b.send(encodeRudpPacket(second.createConnect(0, enc("join"))), port, "127.0.0.1");
await sleep(100);
const acceptB = atB.find((packet) => packet.packetType === "accept");
assert.ok(acceptB, `the restarted client on a new port was not admitted: ${JSON.stringify(atB.map((packet) => packet.packetType))}`);
second.receive(acceptB, 1);
assert.ok(second.connected);
b.send(encodeRudpPacket(send(second, "after")), port, "127.0.0.1");
await sleep(100);
assert.deepEqual(frames, ["hello", "after"]);
} finally {
server.close();
a.close();
b.close();
}
});
test("an accepted peer that goes silent times out", () => {
const server = session(500);
server.acceptConnect(session(10).createConnect(0), 0);
assert.equal(server.checkTimeout(100_000, 1_000), true);
});
// Ack Cut 1d, batch 2: the connect-attempt timeout.
test("an Ack naming the pending Connect does not retire it", () => {
const server = session(500);
const old = session(1);
const oldAccept = server.acceptConnect(old.createConnect(0), 0);
old.receive(oldAccept, 0);
server.receive(old.createAckForReceived(oldAccept.sequence), 0);
const restarted = session(1);
const first = restarted.createConnect(0, enc("join"));
const reply = server.acceptConnect(first, 1);
assert.equal(reply.packetType, "ack");
assert.ok(namesSequence(reply, first.sequence), "the Ack must name the pending Connect for this test to bite");
restarted.receive(reply, 1);
assert.equal(restarted.connected, false);
assert.deepEqual(restarted.pendingReliableSequences, [first.sequence], "an Ack retired the Connect");
const fresh = restarted.dueResends(3_000);
assert.equal(fresh.length, 1, "the client waits for ever with nothing to resend");
assert.equal(dec(fresh[0]!.payload ?? new Uint8Array()), "join");
const accept = server.acceptConnect(fresh[0]!, 3_000);
assert.equal(accept.packetType, "accept");
restarted.receive(accept, 3_000);
assert.ok(restarted.connected);
});
test("a late copy of an abandoned Connect is stale once its replacement is accepted", () => {
const server = session(500);
const client = session(10);
const abandoned = client.createConnect(0);
const fresh = client.dueResends(3_000);
assert.equal(fresh.length, 1);
assert.equal(fresh[0]!.sequence, abandoned.sequence + 4_095);
const accept = server.acceptConnect(fresh[0]!, 3_001);
client.receive(accept, 3_002);
server.receive(client.createAckForReceived(accept.sequence), 3_002);
assert.ok(client.connected);
const a = send(client, "a");
assert.deepEqual(names(server.receive(a, 3_003)), ["a"]);
client.receive(server.createAckForReceived(a.sequence), 3_003);
assert.equal(server.connectRepeats(abandoned), true, "the late copy would restart the server");
const reply = server.acceptConnect(abandoned, 3_004);
assert.equal(reply.packetType, "ack");
client.receive(reply, 3_005);
assert.ok(client.connected);
assert.deepEqual(names(server.receive(send(client, "b"), 3_006)), ["b"]);
});
test("a server sitting at the jump is left by the next attempt", () => {
const server = session(500);
const old = session(5_095);
const oldAccept = server.acceptConnect(old.createConnect(0), 0);
old.receive(oldAccept, 0);
server.receive(old.createAckForReceived(oldAccept.sequence), 0);
const restarted = session(1_000);
const first = restarted.createConnect(0, enc("join"));
restarted.receive(server.acceptConnect(first, 1), 1);
assert.equal(restarted.connected, false);
const second = restarted.dueResends(3_000)[0]!;
assert.equal(second.sequence, 5_095, "the jump lands on the server's generation");
const reply = server.acceptConnect(second, 3_001);
assert.equal(reply.packetType, "ack");
restarted.receive(reply, 3_001);
assert.equal(restarted.connected, false);
const third = restarted.dueResends(6_000)[0]!;
assert.equal(third.sequence, 5_095 + 4_095);
const accept = server.acceptConnect(third, 6_001);
assert.equal(accept.packetType, "accept");
restarted.receive(accept, 6_001);
assert.ok(restarted.connected);
});
test("server mode admits a pinned client that restarts on the same address", async () => {
const serverSocket = await bind();
const peer = await bind();
const server = new CultNetRudpSocketTransportConnection({
runtimeId: "srv", socket: serverSocket, mode: "server", connectionId, resendPollMs: 20, resendDelayMs: 20,
});
const port = (serverSocket.address() as { port: number }).port;
const received: CultNetRudpPacket[] = [];
peer.on("message", (wire) => received.push(decodeRudpPacket(wire)));
const take = async (type: string) => {
for (let index = 0; index < 50; index += 1) {
const found = received.findIndex((packet) => packet.packetType === type);
if (found >= 0) return received.splice(found, 1)[0]!;
await sleep(10);
}
throw new Error(`no ${type} arrived`);
};
try {
const first = session(1);
peer.send(encodeRudpPacket(first.createConnect(0, enc("join"))), port, "127.0.0.1");
const accept = await take("accept");
first.receive(accept, 0);
peer.send(encodeRudpPacket(first.createAckForReceived(accept.sequence)), port, "127.0.0.1");
await sleep(50);
received.length = 0;
const restarted = session(1);
peer.send(encodeRudpPacket(restarted.createConnect(0, enc("join"))), port, "127.0.0.1");
restarted.receive(await take("ack"), 1);
assert.equal(restarted.connected, false);
peer.send(encodeRudpPacket(restarted.dueResends(3_000)[0]!), port, "127.0.0.1");
restarted.receive(await take("accept"), 3_001);
assert.ok(restarted.connected, "the restarted pinned client was never admitted");
} finally {
server.close();
peer.close();
}
});

110
packages/cultnet-ts/test/rudp-ordered-delivery.test.ts Normal file
View file

@ -0,0 +1,110 @@
// Ordered delivery has one owner: the contiguous received-through watermark. A
// frame is held for exactly one reason, a reliable sequence below it has not
// arrived, and it is delivered in the call that fills the last such gap,
// whichever channel the gap belonged to.
import assert from "node:assert/strict";
import test from "node:test";
import { CultNetRudpSession } from "../src";
const enc = (text: string) => new TextEncoder().encode(text);
const dec = (bytes: Uint8Array) => new TextDecoder().decode(bytes);
function handshake(): { client: CultNetRudpSession; server: CultNetRudpSession } {
const client = new CultNetRudpSession({ connectionId: 410, initialSequence: 1 });
const server = new CultNetRudpSession({ connectionId: 410, initialSequence: 500 });
const accept = server.acceptConnect(client.createConnect(0), 0);
client.receive(accept, 0);
return { client, server };
}
const names = (result: { delivered: { payload: Uint8Array }[] }) => result.delivered.map((frame) => dec(frame.payload));
const send = (session: CultNetRudpSession, channel: string, payload: string, ordered: boolean) =>
session.send(channel, enc(payload), { reliable: true, ordered });
test("an ordered frame waits for a gap filled by another channel", () => {
const { client: sender, server: receiver } = handshake();
const o1 = send(sender, "schema", "o1", true);
const u = send(sender, "rel", "u", false);
const o2 = send(sender, "schema", "o2", true);
const o3 = send(sender, "schema", "o3", true);
const o4 = send(sender, "schema", "o4", true);
assert.deepEqual(names(receiver.receive(o1, 1)), ["o1"]);
assert.deepEqual(names(receiver.receive(o2, 2)), [], "u is missing");
assert.deepEqual(names(receiver.receive(o3, 3)), [], "u is missing");
assert.deepEqual(names(receiver.receive(o4, 4)), [], "u is missing");
assert.deepEqual(names(receiver.receive(u, 5)), ["u", "o2", "o3", "o4"]);
});
test("two ordered channels release each other in sequence order", () => {
const { client: sender, server: receiver } = handshake();
const a1 = send(sender, "schema", "A1", true);
const b1 = send(sender, "other", "B1", true);
const a2 = send(sender, "schema", "A2", true);
const a3 = send(sender, "schema", "A3", true);
const delivered = [
...names(receiver.receive(a1, 1)),
...names(receiver.receive(a2, 2)),
...names(receiver.receive(a3, 3)),
];
assert.deepEqual(delivered, ["A1"], "B1 is missing, so A2 and A3 wait");
assert.deepEqual(names(receiver.receive(b1, 4)), ["B1", "A2", "A3"]);
});
test("a channel first used after other traffic loses nothing", () => {
const { client: sender, server: receiver } = handshake();
const c1 = send(sender, "late", "C1", true);
const c2 = send(sender, "late", "C2", true);
const x = send(sender, "schema", "X", true);
assert.deepEqual(names(receiver.receive(x, 1)), [], "C1 is missing");
assert.deepEqual(names(receiver.receive(c2, 2)), [], "C1 is missing");
assert.deepEqual(names(receiver.receive(c1, 3)), ["C1", "C2", "X"]);
});
test("Accept seeds the watermark of the connecting side", () => {
const { client, server } = handshake();
const s1 = send(server, "schema", "s1", true);
const s2 = send(server, "schema", "s2", true);
assert.deepEqual(names(client.receive(s2, 1)), []);
assert.deepEqual(names(client.receive(s1, 2)), ["s1", "s2"]);
});
test("Connect seeds the watermark of the accepting side", () => {
const { client, server } = handshake();
const first = send(client, "schema", "first", true);
const second = send(client, "schema", "second", true);
assert.deepEqual(names(server.receive(second, 1)), []);
assert.deepEqual(names(server.receive(first, 2)), ["first", "second"]);
});
test("a duplicate of a held frame is not delivered twice", () => {
const { client: sender, server: receiver } = handshake();
const s1 = send(sender, "schema", "s1", true);
const s2 = send(sender, "schema", "s2", true);
assert.deepEqual(names(receiver.receive(s2, 1)), []);
assert.deepEqual(names(receiver.receive(s2, 2)), []);
assert.deepEqual(names(receiver.receive(s1, 3)), ["s1", "s2"]);
assert.deepEqual(names(receiver.receive(s2, 4)), []);
});
test("resetting peer state forgets held frames and reseeds the watermark from the next handshake", () => {
const { client: sender, server: receiver } = handshake();
send(sender, "schema", "s1", true);
const s2 = send(sender, "schema", "s2", true);
assert.deepEqual(names(receiver.receive(s2, 1)), []);
receiver.resetPeerState();
const next = new CultNetRudpSession({ connectionId: 410, initialSequence: 100 });
next.receive(receiver.acceptConnect(next.createConnect(0), 2), 2);
const d1 = send(next, "schema", "d1", true);
const d2 = send(next, "schema", "d2", true);
// The held s2 died with the reset, and the watermark follows the new peer.
assert.deepEqual(names(receiver.receive(d2, 3)), []);
assert.deepEqual(names(receiver.receive(d1, 4)), ["d1", "d2"]);
});

72
packages/cultnet-ts/test/rudp-session-ending.test.ts Normal file
View file

@ -0,0 +1,72 @@
// A write belongs to the session generation it was issued in. Every way a
// generation ends drops the writes it still owed, so none is retransmitted
// into a later session where the new peer would deliver it.
import assert from "node:assert/strict";
import test from "node:test";
import { CultNetRudpSession } from "../src";
const connectionId = 0x10203070;
const enc = (text: string) => new TextEncoder().encode(text);
function clientWithALostWrite(): { client: CultNetRudpSession; server: CultNetRudpSession } {
const client = new CultNetRudpSession({ connectionId, initialSequence: 1 });
const server = new CultNetRudpSession({ connectionId, initialSequence: 500 });
client.receive(server.acceptConnect(client.createConnect(0), 0), 0);
client.send("schema", enc("owed to the old session"), { reliable: true, ordered: true });
// A fragmented write larger than the send window leaves part of it queued.
client.sendMany("schema", new Uint8Array(40 * 8), { reliable: true, ordered: true, maxFragmentBytes: 8 });
assert.equal(client.outstandingReliablePacketCount, 41);
assert.ok(client.queuedReliablePacketCount > 0);
return { client, server };
}
const endings: Record<string, (client: CultNetRudpSession, server: CultNetRudpSession) => void> = {
"peer Disconnect": (client, server) => {
client.receive(server.createDisconnect(enc("bye")), 1);
},
"local disconnect": (client) => {
client.createDisconnect(enc("bye"));
},
timeout: (client) => {
assert.equal(client.checkTimeout(1_000, 10), true);
},
reset: (client) => {
client.resetPeerState();
},
};
for (const [name, end] of Object.entries(endings)) {
test(`a ${name} drops the writes the session owed`, () => {
const { client, server } = clientWithALostWrite();
end(client, server);
assert.equal(client.outstandingReliablePacketCount, 0, "the write survived the end");
const nextServer = new CultNetRudpSession({ connectionId, initialSequence: 900 });
nextServer.acceptConnect(client.createConnect(2_000), 2_000);
for (const resend of client.dueResends(60_000)) {
assert.deepEqual(
nextServer.receive(resend, 60_000).delivered,
[],
"the old session's write was delivered in the next session",
);
}
});
}
test("an ending does not forget what was received from the peer", () => {
const client = new CultNetRudpSession({ connectionId, initialSequence: 1 });
const server = new CultNetRudpSession({ connectionId, initialSequence: 500 });
client.receive(server.acceptConnect(client.createConnect(0), 0), 0);
const s1 = client.send("schema", enc("s1"), { reliable: true, ordered: true });
const s2 = client.send("schema", enc("s2"), { reliable: true, ordered: true });
assert.deepEqual(server.receive(s2, 1).delivered, [], "s1 is missing, so s2 is held");
assert.equal(server.checkTimeout(1_000, 10), true);
assert.deepEqual(
server.receive(s1, 1_001).delivered.map((frame) => new TextDecoder().decode(frame.payload)),
["s1", "s2"],
"the ending forgot the held frame",
);
assert.deepEqual(server.receive(s2, 1_002).delivered, [], "the ending forgot what was received");
});

13
src/GameCult.Caching.MessagePack/CultDocumentMessagePackSerialization.cs
View file

@ -284,7 +284,7 @@ public static class CultDocumentMessagePackSerialization
!string.Equals(version, CultPersistedStoreSnapshot.FormatV3, StringComparison.Ordinal))
{
throw new CultStoreUnreadableException(
$"Store format {version} is not readable; this runtime reads {CultPersistedStoreSnapshot.FormatV1}, {CultPersistedStoreSnapshot.FormatV2} and {CultPersistedStoreSnapshot.FormatV3}.");
$"Store format {DescribeHeader(version)} is not readable; this runtime reads {CultPersistedStoreSnapshot.FormatV1}, {CultPersistedStoreSnapshot.FormatV2} and {CultPersistedStoreSnapshot.FormatV3}.");
}
// Only a v2 or v3 store may hold a variant: a v1 header over a variant record is a store no reader could trust.
@ -299,6 +299,17 @@ public static class CultDocumentMessagePackSerialization
}
}
// A store header as a refusal may show it: echoed in the shape cultcache.store.v<digits>, and otherwise described
// only by its length, since the bytes are the store's.
private static string DescribeHeader(string header)
{
const string prefix = "cultcache.store.v";
var version = header.StartsWith(prefix, StringComparison.Ordinal) ? header.Substring(prefix.Length) : "";
return version.Length > 0 && version.All(character => character >= '0' && character <= '9')
? header
: $"an unrecognised cultcache.store.* header of {System.Text.Encoding.UTF8.GetByteCount(header)} bytes";
}
private static void WritePersistedRecord(ref MessagePackWriter writer, CultPersistedRecord record)
{
writer.WriteArrayHeader(record.Variant == null ? PersistedRecordFieldCount : VariantRecordFieldCount);

4
src/GameCult.Caching/Contracts/cultcache-persistence-format.md
View file

@ -298,8 +298,8 @@ array (a second store appended included), is not an array, has fewer or more tha
(a header alone included), whose first slot is not a header it reads, or whose catalog or
records it cannot decode (a record with more slots than its header allows, a variant record
it does not read). A runtime never skips a slot it does not understand. It refuses on open
and on rewrite alike, because the rewrite asks the same full reader as the open. A zero-byte
file and an empty array are an empty store. A flush or a commit replaces a file only when the
and on rewrite alike, because the rewrite asks the same full reader as the open. A missing
file and an empty array are an empty store; a zero-byte file is not a store and is refused as unreadable. A flush or a commit replaces a file only when the
runtime's own reader opens it, so one verdict covers open, flush and commit, and a file the
runtime cannot read is left as it is. Each runtime refuses with one typed error:
`CultStoreUnreadableException` (C#), `StoreUnreadableError` (Rust, TypeScript, Python), with

22
src/GameCult.Caching/Contracts/cultcache-store-composition.md
View file

@ -47,10 +47,20 @@ which route wrote it.
- Changes published through `Watch`/`WatchRecord` carry a `Sequence`: a
per-cache, in-memory number assigned when the cache admits the change, under
its gate. It increases in admission order and is not persisted.
`OnUpdate(previous, current)` carries none. Streams CultNet derives from
`OnUpdate` (`CultNetDatabase.WatchAllChanges`, the subscription server,
database-backed Mesh handles) have no ordering guarantee and no stale
protection.
`OnUpdate(previous, current)` carries none. CultNet derives its streams
(`CultNetDatabase.WatchAllChanges`, the subscription server, database-backed
Mesh handles) from `Watch` plus one journal (`AddJournal`), so a load, a
commit and a variant dependent's re-resolution all reach subscribers and the
mutation log. The journal runs under the gate in `Sequence` order, so the
shard log is written in cache order; the `Watch` observer publishes what the
journal decided. A database write door only hands the journal the wire message
or replicated entry for the change it admitted. A replica publishes but never
mints log sequences. A primary logs every change it admits, loads and pulls
included; a store attached after the database exists has its hydration logged
too. A primary that cannot log a change compacts through that change's
sequence and throws `CultNetShardLogException` to the writer after publication:
the commit stands, and a replica behind it resynchronizes from a snapshot. The
streams carry no `Sequence` and have no stale protection.
- `GetWithSequence(key)` returns the document and the cache's current
`Sequence`, read together under the gate: every change with a `Sequence` at
or below it is reflected in the document.
@ -125,6 +135,10 @@ which route wrote it.
needs the cache or its gate: another thread's write, a lock a writer holds.
Its own leaf locks, taken only inside the journal, are the only locks that may
nest under the gate.
- A shard mutation log store (`ICultNetShardMutationLogStore`) runs inside the
CultNet journal, under the cache's gate: it must not call the cache or wait
for a thread that does. A store that writes its own cache throws through the
journal's reentry guard, and the admission stands.
- A thread holds one cache's gate at a time: entering a cache's hold inside
another cache's hold throws.
- A store doing I/O on behalf of its cache (pull, flush, commit) holds the gate,

17
src/GameCult.Caching/CultCache.cs
View file

@ -2335,7 +2335,15 @@ namespace GameCult.Caching
Remove(handle.Key);
}
// false: a condition failed; nothing was written, changed in memory, or published.
/// <summary>
/// Commits the staged batch. Returns false when a condition failed: nothing was written, changed in memory, or published.
/// </summary>
/// <remarks>
/// A commit that succeeds and then fails in a journal or observer still stands and is published, and the writer
/// receives the failure. One failure arrives as itself, several as an <see cref="AggregateException"/>. A writer that
/// wants a particular failure (a CultNetShardLogException, when a database's log refused the commit) catches that type,
/// or unwraps <see cref="AggregateException.InnerExceptions"/> and looks for it.
/// </remarks>
public bool Commit(Action<CultCacheBatch> stage)
{
return Land(stage, wait: true) == CultCommitOutcome.Committed;
@ -2673,7 +2681,8 @@ namespace GameCult.Caching
// run under the gate, and a write an observer makes is a new outermost hold. Cross-thread delivery order is not
// guaranteed; each change carries the Sequence it was admitted with. Journals run before that, under the gate,
// in Sequence order (AddJournal); no hold may be entered on this cache while one runs. Every change reaches every observer even
// if an observer or an OnUpdate handler throws; then the first exception (an AggregateException for several) is rethrown.
// if an observer or an OnUpdate handler throws; then the exception (an AggregateException for several, whatever their
// origin: a journal, an observer, a second database) is rethrown. A writer looking for one type unwraps InnerExceptions.
// If the body threw, its exception wins.
internal T Held<T>(Func<T> body)
{
@ -3984,7 +3993,7 @@ namespace GameCult.Caching
}
// Open, flush and commit read the file through this one call, so they agree on which files are stores. A file that is
// gone or zero bytes is an empty store: nothing to read, nothing to lose.
// gone is an empty store: nothing to read, nothing to lose. A zero-byte file is not a store; it is refused.
private CultPersistedStoreSnapshot? ReadSnapshot()
{
var bytes = ReadDisk();
@ -4007,7 +4016,7 @@ namespace GameCult.Caching
return null;
try
{
return ReadAllBytesShared(FileInfo.FullName) is { Length: > 0 } bytes ? bytes : null;
return ReadAllBytesShared(FileInfo.FullName);
}
catch (FileNotFoundException)
{

41
src/GameCult.Mesh/CultMesh.cs
View file

@ -537,14 +537,18 @@ namespace GameCult.Mesh
/// </summary>
public sealed class CultMeshRudpSocketOptions
{
/// <summary>Gets or sets the local bind host.</summary>
public string BindHost { get; set; } = "0.0.0.0";
/// <summary>
/// Gets or sets the local bind host. Unset, a server binds every IPv4 interface, and a
/// client binds loopback for a loopback endpoint and the unspecified address of the
/// endpoint's family otherwise, so it can reach a remote host.
/// </summary>
public string? BindHost { get; set; }
/// <summary>Gets or sets the local bind port.</summary>
public int BindPort { get; set; }
/// <summary>Gets or sets a caller-owned bound socket.</summary>
public Socket? Socket { get; set; }
/// <summary>Gets or sets the first local packet sequence.</summary>
public uint InitialSequence { get; set; } = 1;
/// <summary>Gets or sets the first local packet sequence. Unset, each session draws its own at random.</summary>
public uint? InitialSequence { get; set; }
/// <summary>Gets or sets the reliable resend delay in milliseconds.</summary>
public long ResendDelayMs { get; set; } = 250;
/// <summary>Gets or sets the advertised transport id.</summary>
@ -2931,7 +2935,7 @@ namespace GameCult.Mesh
return new CultNetRudpSocketTransportConnection(new CultNetRudpSocketTransportOptions
{
RuntimeId = runtimeId,
Socket = options.Socket ?? BindRudpSocket(options.BindHost, options.BindPort),
Socket = options.Socket ?? BindRudpSocket(ResolveRudpAddress(options.BindHost ?? "0.0.0.0"), options.BindPort),
Mode = CultNetRudpSocketMode.Server,
ConnectionId = connectionId,
InitialSequence = options.InitialSequence,
@ -2955,12 +2959,16 @@ namespace GameCult.Mesh
{
if (endpoint == null) throw new ArgumentNullException(nameof(endpoint));
options ??= new CultMeshRudpSocketOptions();
var remote = ResolveRudpAddress(endpoint.Host);
var bindAddress = options.BindHost == null
? RudpClientBindAddress(remote)
: ResolveRudpAddress(options.BindHost);
return new CultNetRudpSocketTransportConnection(new CultNetRudpSocketTransportOptions
{
RuntimeId = runtimeId,
Socket = options.Socket ?? BindRudpSocket(options.BindHost, options.BindPort),
Socket = options.Socket ?? BindRudpSocket(bindAddress, options.BindPort),
Mode = CultNetRudpSocketMode.Client,
RemoteEndPoint = new IPEndPoint(ResolveRudpAddress(endpoint.Host), endpoint.Port),
RemoteEndPoint = new IPEndPoint(remote, endpoint.Port),
ConnectionId = connectionId,
InitialSequence = options.InitialSequence,
ResendDelayMs = options.ResendDelayMs,
@ -3232,9 +3240,21 @@ namespace GameCult.Mesh
return CultNetLocal.ConnectClient(host, port, security, configureClient);
}
private static Socket BindRudpSocket(string host, int port)
// A socket bound to loopback cannot send off the host on Windows, so a client keeps
// loopback only when its peer is on loopback.
private static IPAddress RudpClientBindAddress(IPAddress remote)
{
var ipv6 = remote.AddressFamily == AddressFamily.InterNetworkV6;
if (IPAddress.IsLoopback(remote))
{
return ipv6 ? IPAddress.IPv6Loopback : IPAddress.Loopback;
}
return ipv6 ? IPAddress.IPv6Any : IPAddress.Any;
}
private static Socket BindRudpSocket(IPAddress address, int port)
{
var address = ResolveRudpAddress(host);
var socket = new Socket(address.AddressFamily, SocketType.Dgram, ProtocolType.Udp);
socket.Bind(new IPEndPoint(address, port));
socket.ReceiveTimeout = 20;
@ -3377,7 +3397,8 @@ namespace GameCult.Mesh
if (document != null)
return document;
var untyped = database.Cache.Get(key);
// Through the database, which serves only what its shards own; the cache behind it may hold more.
var untyped = await database.GetAsync<object>(key).ConfigureAwait(false);
if (untyped != null && IsSameCultDocumentSchema<TDocument>(untyped.GetType()))
return ConvertUntypedDocument<TDocument>(untyped);

935
src/GameCult.Networking/CultNetDatabase.cs
View file

File diff suppressed because it is too large Load diff

33
src/GameCult.Networking/CultNetDatabaseServer.cs
View file

@ -112,10 +112,13 @@ namespace GameCult.Networking
request);
}
// A database serves only what its shards own.
return _database.Documents.CreateRawSnapshotResponse(
_database.Cache,
string.IsNullOrWhiteSpace(request.MessageId) ? Guid.NewGuid().ToString("N") : request.MessageId,
request);
request,
options: null,
rowFilter: _database.Owns);
}
/// <summary>
@ -135,6 +138,9 @@ namespace GameCult.Networking
// R-Q: asOf is one shard-log watermark, so a selection whose matched rows span more than
// one shard's log is refused rather than answered against a watermark that is not
// exact for all of them.
// A row whose schema no shard owns is not part of what this database serves: it is left out, not an error.
options: null,
rowFilter: _database.Owns,
shardIdOf: (schemaId, key) => _database.ResolveShard(schemaId, key).ShardId,
cursorKey: _database.CursorKey,
// S-9: once the matched rows resolve to one shard, asOf is that shard's own watermark,
@ -312,6 +318,16 @@ namespace GameCult.Networking
peer.SendCultNet(CreateRoutingError(ex));
}
}
catch (CultNetUnownedSchemaException ex)
{
_server.Logger.LogWarning($"CultNet raw put refused: {ex.Message}");
peer.SendCultNet(CultNetErrorMessage.ForUnownedSchema(ex));
}
catch (Exception ex) when (CommittedWithLogFailure(ex))
{
// The write committed and was published; only its shard log failed, so the peer is not told it was refused.
_server.Logger.LogWarning($"CultNet raw put committed but its shard log failed: {ex.Message}");
}
catch (Exception ex)
{
_server.Logger.LogError($"CultNet raw put failed: {ex.Message}");
@ -332,6 +348,15 @@ namespace GameCult.Networking
peer.SendCultNet(CreateRoutingError(ex));
}
}
catch (CultNetUnownedSchemaException ex)
{
_server.Logger.LogWarning($"CultNet raw delete refused: {ex.Message}");
peer.SendCultNet(CultNetErrorMessage.ForUnownedSchema(ex));
}
catch (Exception ex) when (CommittedWithLogFailure(ex))
{
_server.Logger.LogWarning($"CultNet raw delete committed but its shard log failed: {ex.Message}");
}
catch (Exception ex)
{
peer.SendCultNet(new CultNetErrorMessage { Error = ex.Message });
@ -604,6 +629,12 @@ namespace GameCult.Networking
: messageId;
}
// A CultNetShardLogException means the write committed and was published (the writer hears of a failed log afterwards);
// the cache wraps it in an AggregateException when something else failed in the same hold.
private static bool CommittedWithLogFailure(Exception exception) =>
exception is CultNetShardLogException ||
exception is AggregateException aggregate && aggregate.Flatten().InnerExceptions.Any(inner => inner is CultNetShardLogException);
private static CultNetErrorMessage CreateRoutingError(CultNetShardAuthorityException exception)
{
return new CultNetErrorMessage

3
src/GameCult.Networking/CultNetDatabaseSubscriptionServer.cs
View file

@ -421,7 +421,8 @@ namespace GameCult.Networking
documentSelection,
ordinalOf: (schemaId, key) => _database.LastWriteSequence(schemaId, key) ?? 0,
asOf: _database.CurrentAsOf(),
options: null);
options: null,
rowFilter: _database.Owns);
// The snapshot is one row per record key by construction, so the duplicate-key throw this
// used to guard can no longer fire (docs/cultnet-selection-cut.md, S2-2); deleted rather
// than tested around.

31
src/GameCult.Networking/CultNetDocumentRegistry.cs
View file

@ -301,7 +301,16 @@ namespace GameCult.Networking
CultCache cache,
string messageId,
CultNetSnapshotRequestMessage? filter = null,
CultNetDocumentMessageOptions? options = null)
CultNetDocumentMessageOptions? options = null) =>
CreateRawSnapshotResponse(cache, messageId, filter, options, rowFilter: null);
// rowFilter leaves out the cached rows it rejects before evaluation (a database serves only what its shards own).
internal CultNetSnapshotResponseRawMessage CreateRawSnapshotResponse(
CultCache cache,
string messageId,
CultNetSnapshotRequestMessage? filter,
CultNetDocumentMessageOptions? options,
Func<CultDocumentDescriptor, CultRecordKey, bool>? rowFilter)
{
if (cache == null) throw new ArgumentNullException(nameof(cache));
var lowSchemas = CultNetV0SelectionLowering.Lower(filter?.SchemaIds);
@ -344,7 +353,7 @@ namespace GameCult.Networking
ordinalOf = static (_, _) => 0;
}
var page = SelectAll(cache, selection, ordinalOf, asOf: 0, options);
var page = SelectAll(cache, selection, ordinalOf, asOf: 0, options, rowFilter);
return new CultNetSnapshotResponseRawMessage
{
@ -369,13 +378,27 @@ namespace GameCult.Networking
CultNetDocumentMessageOptions? options = null,
Func<string, CultRecordKey, string>? shardIdOf = null,
CultNetSelectionCursorKey? cursorKey = null,
Func<string, ulong>? asOfForShard = null)
Func<string, ulong>? asOfForShard = null) =>
CreateSelectionResponse(cache, messageId, selection, ordinalOf, asOf, options, shardIdOf, cursorKey, asOfForShard, rowFilter: null);
// rowFilter leaves out the cached rows it rejects before evaluation (a database serves only what its shards own).
internal CultNetSnapshotResponseRawV1Message CreateSelectionResponse(
CultCache cache,
string messageId,
CultNetSelection selection,
Func<string, CultRecordKey, long> ordinalOf,
ulong asOf,
CultNetDocumentMessageOptions? options,
Func<string, CultRecordKey, string>? shardIdOf,
CultNetSelectionCursorKey? cursorKey,
Func<string, ulong>? asOfForShard,
Func<CultDocumentDescriptor, CultRecordKey, bool>? rowFilter)
{
if (cache == null) throw new ArgumentNullException(nameof(cache));
if (selection == null) throw new ArgumentNullException(nameof(selection));
if (ordinalOf == null) throw new ArgumentNullException(nameof(ordinalOf));
var page = SelectPage(cache, selection, ordinalOf, asOf, options, shardIdOf: shardIdOf, cursorKey: cursorKey, asOfForShard: asOfForShard);
var page = SelectPage(cache, selection, ordinalOf, asOf, options, rowFilter, shardIdOf, cursorKey, asOfForShard);
return new CultNetSnapshotResponseRawV1Message
{
MessageId = RequireNonEmpty(messageId, nameof(messageId)),

5
src/GameCult.Networking/CultNetRudpSchemaServer.cs
View file

@ -43,9 +43,10 @@ namespace GameCult.Networking
/// </summary>
public uint ConnectionId { get; set; } = 0x43554c54;
/// <summary>
/// Gets or sets the first local packet sequence for each accepted peer.
/// Gets or sets the first local packet sequence for each accepted peer. Unset, each peer's
/// session draws its own at random.
/// </summary>
public uint InitialSequence { get; set; } = 100;
public uint? InitialSequence { get; set; }
/// <summary>
/// Gets or sets the resend delay in milliseconds.
/// </summary>

13
src/GameCult.Networking/CultNetSchemaMessages.cs
View file

@ -511,7 +511,7 @@ namespace GameCult.Networking
[Key("routingHint")] public CultNetShardRoutingHint? RoutingHint { get; set; }
/// <summary>
/// Gets or sets the machine-readable refusal code (R-N), e.g. <c>selection_invalid</c>,
/// <c>cursor_stale</c>, <c>cursor_invalid</c>, <c>reference_outside_target</c>. Additive to
/// <c>cursor_stale</c>, <c>cursor_invalid</c>, <c>reference_outside_target</c>, <c>unowned_schema</c>. Additive to
/// <c>cultnet.error.v0</c>; a v0 peer that does not know this field ignores it.
/// </summary>
[Key("code")] public string? Code { get; set; }
@ -532,6 +532,17 @@ namespace GameCult.Networking
Details = new CultNetErrorDetails { Field = ex.Field, Value = ex.Value }
};
/// <summary>
/// Builds the wire error for a write no shard owns: code <c>unowned_schema</c>, details <c>{ field: "schemaId", value }</c>.
/// </summary>
public static CultNetErrorMessage ForUnownedSchema(CultNetUnownedSchemaException ex) =>
new CultNetErrorMessage
{
Error = $"unowned_schema: {ex.Message}",
Code = "unowned_schema",
Details = new CultNetErrorDetails { Field = "schemaId", Value = ex.SchemaId }
};
/// <summary>
/// Builds the wire error for a cursor refusal (R-N): code <c>cursor_stale</c> with details
/// <c>{ asOf, current }</c>, or code <c>cursor_invalid</c> with no details.

5
src/GameCult.Networking/CultNetShardMutationLogStorage.cs
View file

@ -10,6 +10,11 @@ namespace GameCult.Networking
/// <summary>
/// Stores accepted shard-log entries in their replica catch-up wire shape.
/// </summary>
/// <remarks>
/// A database calls its store from a cache journal, under that cache's gate, so a slow store slows every writer of
/// the cache and blocks its readers too, because reads take the same gate. A store must not call the cache, or wait for a thread that does. Appends must throw rather than
/// swallow a failure: the database burns the sequence and compacts past it, and tells the writer.
/// </remarks>
public interface ICultNetShardMutationLogStore
{
/// <summary>

512
src/GameCult.Networking/CultNetTransport.cs
View file

@ -768,9 +768,11 @@ namespace GameCult.Networking
/// </summary>
public uint ConnectionId { get; set; }
/// <summary>
/// Gets or sets the first local packet sequence.
/// Gets or sets the first local packet sequence. Unset, each session draws its own at random:
/// every session is a new identity to its peer, and an options object reused for several
/// must not give them one sequence.
/// </summary>
public uint InitialSequence { get; set; } = 1;
public uint? InitialSequence { get; set; }
/// <summary>
/// Gets or sets the resend delay in milliseconds.
/// </summary>
@ -845,9 +847,11 @@ namespace GameCult.Networking
/// </summary>
public uint ConnectionId { get; set; }
/// <summary>
/// Gets or sets the first local packet sequence.
/// Gets or sets the first local packet sequence. Unset, each session draws its own at random:
/// every session is a new identity to its peer, and an options object reused for several
/// must not give them one sequence.
/// </summary>
public uint InitialSequence { get; set; } = 1;
public uint? InitialSequence { get; set; }
/// <summary>
/// Gets or sets the resend delay in milliseconds.
/// </summary>
@ -882,18 +886,17 @@ namespace GameCult.Networking
/// <summary>Maximum reliable packets admitted to the wire before acknowledgements advance the window.</summary>
public const int ReliableSendWindowPackets = 32;
private const int ReceivedSequenceWindow = 4096;
// How long a client keeps retransmitting a Connect nobody answered before it abandons that
// attempt for a fresh one. A Connect the server answers with an Ack, never an Accept, is one
// the server judged stale (see AcceptConnect); retransmitting the same sequence would never
// change its mind.
private const long ConnectAttemptMs = 3_000;
private sealed class PendingReliablePacket
{
public CultNetRudpPacket Packet { get; set; } = new CultNetRudpPacket();
public long LastSentAtMs { get; set; }
}
private sealed class PendingOrderedFrame
{
public CultNetRudpDeliveredFrame Frame { get; set; } = new CultNetRudpDeliveredFrame();
public uint NextSequence { get; set; }
}
private sealed class FragmentBuffer
{
public string ChannelId { get; set; } = string.Empty;
@ -908,6 +911,24 @@ namespace GameCult.Networking
private ushort _nextFragmentId = 1;
private readonly int? _maxPendingReliablePackets;
private volatile bool _connected;
// Advances every time a generation ends. Everything issued in a generation (writes, flushes)
// belongs to it and dies with it.
private long _generation;
// True from the end of a generation until the next Connect or Accept begins one. A flush started
// in that interval has no live generation to wait on.
private bool _ended;
// The sequence of the Connect that started the current generation: sent by this side, or
// accepted from the peer. A Connect repeats exactly when the session is connected and the
// Connect carries this sequence.
private uint? _connectSequence;
// A Connect this side sent is unanswered. Only then is an Accept honoured.
private bool _awaitingAccept;
// When the unanswered Connect first went out; DueResends abandons it for a fresh attempt once
// ConnectAttemptMs has passed.
private long _connectStartedAtMs;
// The payload of the Connect this side sent, kept so a fresh attempt can always be built
// whatever became of the pending packet.
private byte[] _connectPayload = Array.Empty<byte>();
private long? _lastReceivedAtMs;
private uint? _highestReceivedSequence;
private readonly HashSet<uint> _receivedSequences = new HashSet<uint>();
@ -915,9 +936,14 @@ namespace GameCult.Networking
private readonly object _pendingReliableGate = new object();
private readonly Dictionary<uint, PendingReliablePacket> _pendingReliable = new Dictionary<uint, PendingReliablePacket>();
private readonly Queue<CultNetRudpPacket> _queuedReliable = new Queue<CultNetRudpPacket>();
private readonly Dictionary<string, uint> _orderedNextSequenceByChannel = new Dictionary<string, uint>(StringComparer.Ordinal);
private readonly Dictionary<string, SortedDictionary<uint, PendingOrderedFrame>> _orderedBuffers =
new Dictionary<string, SortedDictionary<uint, PendingOrderedFrame>>(StringComparer.Ordinal);
// Every reliable sequence up to and including this one has been received since the peer state
// was last reset. Only the handshake seeds it: the peer's Connect on the accepting side, the
// peer's Accept on the connecting side. Reliable data that arrives before that is refused.
private uint? _receivedThrough;
// Ordered frames received but not yet deliverable, keyed by first sequence. A frame is held for
// exactly one reason: a reliable sequence below it has not arrived.
private readonly SortedDictionary<uint, CultNetRudpDeliveredFrame> _orderedHeld =
new SortedDictionary<uint, CultNetRudpDeliveredFrame>();
private readonly Dictionary<string, FragmentBuffer> _fragmentBuffers = new Dictionary<string, FragmentBuffer>(StringComparer.Ordinal);
/// <summary>
@ -929,18 +955,39 @@ namespace GameCult.Networking
ValidateLimits(options.InitialSequence, options.MaxPendingReliablePackets);
ConnectionId = options.ConnectionId;
_nextSequence = options.InitialSequence;
_nextSequence = options.InitialSequence ?? DrawSequence();
ResendDelayMs = options.ResendDelayMs;
_maxPendingReliablePackets = options.MaxPendingReliablePackets;
}
/// <summary>
/// A sequence drawn from a secure source in [1, 2^31). The Connect's sequence is what tells a
/// peer whether a Connect repeats one it already accepted or starts a new session, so two
/// sessions must not share one by default.
/// </summary>
private static uint DrawSequence()
{
const uint ceiling = 1u << 31;
var bytes = new byte[4];
using (var random = System.Security.Cryptography.RandomNumberGenerator.Create())
random.GetBytes(bytes);
return 1u + BitConverter.ToUInt32(bytes, 0) % (ceiling - 1u);
}
// Whether `sequence` is at or before `mark` in serial order, within the receive window. The
// compare is modular, so it holds across the wrap of the 32-bit space; a sequence further back
// than the window is the duplicate test's below-window clause, and one ahead of the mark is
// never before it.
private static bool AtOrBefore(uint sequence, uint mark) =>
unchecked(mark - sequence) < ReceivedSequenceWindow;
internal static readonly byte[] RefusedPacketReason = Encoding.UTF8.GetBytes("session refused a packet");
/// <summary>
/// A session cannot admit a Connect when its sequence space starts exhausted or its reliable
/// queue holds nothing, so neither is a usable configuration.
/// </summary>
internal static void ValidateLimits(uint initialSequence, int? maxPendingReliablePackets)
internal static void ValidateLimits(uint? initialSequence, int? maxPendingReliablePackets)
{
if (initialSequence == uint.MaxValue)
{
@ -965,6 +1012,8 @@ namespace GameCult.Networking
/// Gets whether the session has completed the connect/accept handshake.
/// </summary>
public bool Connected => _connected;
internal long Generation => Interlocked.Read(ref _generation);
internal bool Ended => _ended;
/// <summary>
/// Gets the logical time of the last received packet.
/// </summary>
@ -1000,14 +1049,57 @@ namespace GameCult.Networking
/// </summary>
public CultNetRudpPacket CreateConnect(long nowMs = 0, byte[]? payload = null)
{
// A session that has had a peer starts a new generation: nothing it learned from that
// peer describes the one this Connect reaches, and what it still owed that peer no
// longer takes room in the queue.
if (_connectSequence.HasValue)
ResetPeerState();
EnsureReliableCapacity(1);
_ended = false;
var packet = CreatePacket(CultNetRudpPacketType.Connect, "control", payload ?? Array.Empty<byte>(), reliable: true, ordered: true, sequenced: false);
_connectSequence = packet.Sequence;
_awaitingAccept = true;
_connectStartedAtMs = nowMs;
_connectPayload = packet.Payload ?? Array.Empty<byte>();
TrackReliable(packet, nowMs);
return packet;
}
/// <summary>
/// Accepts a connect packet and returns a reliable ordered accept packet.
/// Whether <paramref name="packet"/> is a Connect this session's current generation already owns:
/// a retransmit of the Connect that started it (the session is connected and the sequence is
/// that Connect's), or a stale copy of an earlier attempt by the same client (a sequence before
/// it, within the receive window). Servers that keep one session per peer ask this to tell a
/// Connect that starts a new session from one that does not; anything else that reaches
/// <see cref="AcceptConnect"/> starts a new generation.
/// </summary>
public bool ConnectRepeats(CultNetRudpPacket packet)
{
if (packet == null) throw new ArgumentNullException(nameof(packet));
return packet.PacketType == CultNetRudpPacketType.Connect
&& _connected
&& _connectSequence.HasValue
&& (_connectSequence.Value == packet.Sequence
|| ConnectIsStale(packet.Sequence, _connectSequence.Value));
}
// A Connect that precedes the current generation's within the receive window is the client's
// earlier attempt, delayed in the network: a client that retried never sends a lower sequence
// again. Restarting on it would strand the client, which honours only the Accept for its newest
// Connect. The rule is TCP's answer to a delayed SYN (RFC 5961's challenge ACK): keep the
// connection and answer with an Ack. A restarted client whose random initial sequence lands in
// this window is answered the same way and abandons the attempt for a fresh draw
// (ConnectAttemptMs).
private static bool ConnectIsStale(uint sequence, uint current) =>
sequence != current && AtOrBefore(sequence, current);
/// <summary>
/// Answers a connect packet. A repeat of the accepted connect queues nothing, so a connect storm
/// cannot grow the reliable queue: the reply is the accept still awaiting acknowledgement, or an
/// ack once it was acknowledged. A stale copy of an earlier attempt gets the same reply and
/// changes nothing else: it is not evidence the peer is alive. Any other connect ends the
/// current generation, forgets the peer and accepts a new one with a reliable ordered accept
/// packet.
/// </summary>
public CultNetRudpPacket AcceptConnect(CultNetRudpPacket packet, long nowMs = 0, byte[]? payload = null)
{
@ -1017,30 +1109,31 @@ namespace GameCult.Networking
throw new InvalidOperationException($"Expected RUDP connect packet, got {packet.PacketType}.");
}
if (ConnectRepeats(packet))
{
if (_connectSequence == packet.Sequence)
{
ApplyAcknowledgements(packet);
RememberReceived(packet.Sequence);
_lastReceivedAtMs = nowMs;
}
return PendingAcceptForResend(nowMs) ?? CreateAck();
}
ResetPeerState();
EnsureReliableCapacity(1);
RememberReceived(packet.Sequence);
SeedReceived(packet.Sequence);
_lastReceivedAtMs = nowMs;
_connectSequence = packet.Sequence;
_connected = true;
_ended = false;
var response = CreatePacket(CultNetRudpPacketType.Accept, "control", payload ?? Array.Empty<byte>(), reliable: true, ordered: true, sequenced: false);
TrackReliable(response, nowMs);
return response;
}
/// <summary>
/// Answers a connect from a peer this session has already accepted. The packet's sequence is
/// remembered and nothing is queued, so a connect storm cannot grow the reliable queue. The reply
/// is the accept still awaiting acknowledgement, or an ack once it was acknowledged.
/// </summary>
public CultNetRudpPacket AnswerRepeatedConnect(CultNetRudpPacket packet, long nowMs = 0)
// The Accept still awaiting acknowledgement, resent.
private CultNetRudpPacket? PendingAcceptForResend(long nowMs)
{
RequireConnection(packet);
if (packet.PacketType != CultNetRudpPacketType.Connect)
{
throw new InvalidOperationException($"Expected RUDP connect packet, got {packet.PacketType}.");
}
ApplyAcknowledgements(packet);
RememberReceived(packet.Sequence);
_lastReceivedAtMs = nowMs;
lock (_pendingReliableGate)
{
foreach (var pending in _pendingReliable.Values)
@ -1052,7 +1145,7 @@ namespace GameCult.Networking
}
}
return CreateAck();
return null;
}
/// <summary>
@ -1145,16 +1238,29 @@ namespace GameCult.Networking
public CultNetRudpReceiveResult Receive(CultNetRudpPacket packet, long nowMs = 0)
{
RequireConnection(packet);
ApplyAcknowledgements(packet);
// An Accept counts only while this side's Connect is unanswered and the Accept names it.
// A late or duplicate one, or one from an earlier generation, must not seed the watermark
// or revive an ended session.
var honoursAccept = packet.PacketType == CultNetRudpPacketType.Accept
&& _awaitingAccept
&& _connectSequence.HasValue
&& PacketAcknowledges(packet, _connectSequence.Value);
if (packet.PacketType == CultNetRudpPacketType.Accept && !honoursAccept)
{
return new CultNetRudpReceiveResult();
}
// While this side's Connect awaits its Accept, only the Accept it honours retires it: an Ack
// that names the Connect (a server's reply to a repeat or a stale copy) says the server did
// not start a session.
if (honoursAccept || !_awaitingAccept)
ApplyAcknowledgements(packet);
var readyToSend = PromoteQueuedReliable(nowMs);
_lastReceivedAtMs = nowMs;
var expectedSequenceIfUninitialized = _highestReceivedSequence.HasValue
? _highestReceivedSequence.Value + 1
: packet.Sequence;
if (packet.PacketType == CultNetRudpPacketType.Accept)
if (honoursAccept)
{
RememberReceived(packet.Sequence);
_awaitingAccept = false;
SeedReceived(packet.Sequence);
_connected = true;
return new CultNetRudpReceiveResult { ReadyToSend = readyToSend };
}
@ -1189,7 +1295,7 @@ namespace GameCult.Networking
if (packet.PacketType == CultNetRudpPacketType.Disconnect)
{
_connected = false;
EndSession();
return new CultNetRudpReceiveResult
{
ReadyToSend = readyToSend,
@ -1203,11 +1309,15 @@ namespace GameCult.Networking
return new CultNetRudpReceiveResult { ReadyToSend = readyToSend };
}
var duplicate = packet.Reliable
&& (_receivedSequences.Contains(packet.Sequence)
|| (_highestReceivedSequence.HasValue
&& packet.Sequence < _highestReceivedSequence.Value
&& _highestReceivedSequence.Value - packet.Sequence >= ReceivedSequenceWindow));
// Reliable data before the handshake has seeded the watermark has no place in the order:
// refuse it unremembered, so it is not acknowledged and the sender retransmits it once
// the handshake is done.
if (packet.Reliable && !_receivedThrough.HasValue)
{
return new CultNetRudpReceiveResult { ReadyToSend = readyToSend };
}
var duplicate = packet.Reliable && WasReceived(packet.Sequence);
if (packet.Reliable)
{
RememberReceived(packet.Sequence);
@ -1217,30 +1327,34 @@ namespace GameCult.Networking
return new CultNetRudpReceiveResult { ReadyToSend = readyToSend };
}
var delivered = new List<CultNetRudpDeliveredFrame>();
var reassembled = Reassemble(packet);
if (reassembled == null)
if (reassembled != null)
{
return new CultNetRudpReceiveResult { ReadyToSend = readyToSend };
}
if (!reassembled.Ordered && packet.Sequenced)
{
var newestSequence = reassembled.NextSequence - 1;
if (_latestSequencedByChannel.TryGetValue(reassembled.Frame.ChannelId, out var latestSequence)
&& newestSequence <= latestSequence)
if (reassembled.Ordered)
{
return new CultNetRudpReceiveResult { ReadyToSend = readyToSend };
_orderedHeld[reassembled.Frame.Sequence] = reassembled.Frame;
}
else if (!packet.Sequenced)
{
delivered.Add(reassembled.Frame);
}
else
{
var newestSequence = reassembled.NextSequence - 1;
if (!_latestSequencedByChannel.TryGetValue(reassembled.Frame.ChannelId, out var latestSequence)
|| newestSequence > latestSequence)
{
_latestSequencedByChannel[reassembled.Frame.ChannelId] = newestSequence;
delivered.Add(reassembled.Frame);
}
}
_latestSequencedByChannel[reassembled.Frame.ChannelId] = newestSequence;
}
return new CultNetRudpReceiveResult
{
ReadyToSend = readyToSend,
Delivered = reassembled.Ordered
? DeliverOrdered(reassembled.Frame, reassembled.NextSequence, expectedSequenceIfUninitialized)
: new[] { reassembled.Frame }
};
// Any reliable packet may have advanced the watermark, a fragment or an unordered frame as
// much as an ordered one, so the drain runs after all of them.
delivered.AddRange(DrainOrdered());
return new CultNetRudpReceiveResult { ReadyToSend = readyToSend, Delivered = delivered };
}
/// <summary>
@ -1272,6 +1386,10 @@ namespace GameCult.Networking
/// </summary>
public CultNetRudpPacket CreateAckForReceived(uint receivedSequence)
{
// What Receive refused is not acknowledged by name (the ack carries only what was
// received), so its sender retransmits it.
if (!WasReceived(receivedSequence))
return CreateAck();
var (ack, _) = AckState();
return ack >= receivedSequence && ack - receivedSequence <= 32
? CreateAck()
@ -1287,23 +1405,39 @@ namespace GameCult.Networking
}
/// <summary>
/// Forgets everything learned from the peer; sequence numbers already issued stay issued.
/// The one way a session generation ends. The session stops being connected and what it still
/// owed the peer dies with it: a write not yet acknowledged is dropped, so no later session
/// retransmits it or credits an ack to it. What was learned from the peer is not touched: the
/// peer may not know the session ended, and forgetting what it sent would let its retransmits
/// be delivered twice.
/// </summary>
public void ResetPeerState()
private void EndSession()
{
_connected = false;
_lastReceivedAtMs = null;
_highestReceivedSequence = null;
_receivedSequences.Clear();
_nextSequencedByChannel.Clear();
_latestSequencedByChannel.Clear();
_ended = true;
_awaitingAccept = false;
Interlocked.Increment(ref _generation);
lock (_pendingReliableGate)
{
_pendingReliable.Clear();
_queuedReliable.Clear();
}
_orderedNextSequenceByChannel.Clear();
_orderedBuffers.Clear();
}
/// <summary>
/// Ends the current generation and forgets everything learned from the peer; sequence numbers
/// already issued stay issued.
/// </summary>
public void ResetPeerState()
{
EndSession();
_lastReceivedAtMs = null;
_highestReceivedSequence = null;
_receivedSequences.Clear();
_nextSequencedByChannel.Clear();
_latestSequencedByChannel.Clear();
_receivedThrough = null;
_orderedHeld.Clear();
_fragmentBuffers.Clear();
}
@ -1323,7 +1457,7 @@ namespace GameCult.Networking
/// </summary>
public CultNetRudpPacket CreateDisconnect(byte[]? reason = null)
{
_connected = false;
EndSession();
return CreatePacket(CultNetRudpPacketType.Disconnect, "control", reason ?? Array.Empty<byte>(), reliable: false, ordered: false, sequenced: false);
}
@ -1340,7 +1474,7 @@ namespace GameCult.Networking
{
return false;
}
_connected = false;
EndSession();
return true;
}
@ -1349,6 +1483,9 @@ namespace GameCult.Networking
/// </summary>
public IReadOnlyList<CultNetRudpPacket> DueResends(long nowMs)
{
var fresh = AbandonUnansweredConnect(nowMs);
if (fresh != null)
return new[] { fresh };
var due = new List<CultNetRudpPacket>();
lock (_pendingReliableGate)
{
@ -1515,9 +1652,69 @@ namespace GameCult.Networking
}
}
private static bool PacketAcknowledges(CultNetRudpPacket packet, uint sequence)
{
if (packet.Ack == sequence)
return true;
for (var bit = 0; bit < 32; bit++)
{
if ((packet.AckMask & (1u << bit)) != 0 && packet.Ack > bit && packet.Ack - (uint)bit - 1 == sequence)
return true;
}
return false;
}
// A Connect unanswered for ConnectAttemptMs is replaced, not retransmitted further, by a Connect
// with the same payload and the abandoned sequence plus the receive window less one. Nothing
// was ever sent in an unanswered generation, so no sequence issued so far is owed. The jump
// keeps a late copy of the abandoned Connect inside the new one's stale window, so a server
// that took the new Connect answers the copy with an Ack instead of restarting; and it leaves
// the stale window of whatever generation the server holds, unless that generation sits
// exactly at the jump, and then the next attempt leaves it. The first Connect of a session is
// the only one drawn at random.
private CultNetRudpPacket? AbandonUnansweredConnect(long nowMs)
{
if (!_awaitingAccept || nowMs - _connectStartedAtMs < ConnectAttemptMs || !_connectSequence.HasValue)
return null;
var jumped = (ulong)_connectSequence.Value + ReceivedSequenceWindow - 1;
_nextSequence = (uint)Math.Min(jumped, uint.MaxValue - 1);
return CreateConnect(nowMs, _connectPayload);
}
// The handshake's one act on the watermark: the peer's Connect or Accept is the first sequence
// of the session, whatever else it has sent.
private void SeedReceived(uint sequence)
{
_receivedThrough = sequence;
RememberReceived(sequence);
}
// The one duplicate test: true for a sequence in the window that was received, for one below
// the window, and for one at or before the watermark. The watermark starts at the handshake's
// seed, so a frame the peer sent in an earlier generation (every sequence it issued is below
// the Connect that began this one) is a duplicate of something already delivered and is
// acknowledged, never delivered again.
private bool WasReceived(uint sequence)
{
return _receivedSequences.Contains(sequence)
|| (_highestReceivedSequence.HasValue
&& sequence < _highestReceivedSequence.Value
&& _highestReceivedSequence.Value - sequence >= ReceivedSequenceWindow)
|| (_receivedThrough.HasValue && AtOrBefore(sequence, _receivedThrough.Value));
}
private void RememberReceived(uint sequence)
{
_receivedSequences.Add(sequence);
if (_receivedThrough.HasValue)
{
var through = _receivedThrough.Value;
while (through != uint.MaxValue && _receivedSequences.Contains(through + 1))
{
through++;
}
_receivedThrough = through;
}
if (!_highestReceivedSequence.HasValue || sequence > _highestReceivedSequence.Value)
{
_highestReceivedSequence = sequence;
@ -1625,80 +1822,35 @@ namespace GameCult.Networking
};
}
private IReadOnlyList<CultNetRudpDeliveredFrame> DeliverOrdered(
CultNetRudpDeliveredFrame frame,
uint nextSequenceAfterFrame,
uint expectedSequenceIfUninitialized)
{
if (!_orderedNextSequenceByChannel.TryGetValue(frame.ChannelId, out var next))
{
next = Math.Min(expectedSequenceIfUninitialized, frame.Sequence);
_orderedNextSequenceByChannel[frame.ChannelId] = next;
}
while (frame.Sequence > next
&& _receivedSequences.Contains(next)
&& (!_orderedBuffers.TryGetValue(frame.ChannelId, out var pendingBuffer) || !pendingBuffer.ContainsKey(next)))
{
next++;
_orderedNextSequenceByChannel[frame.ChannelId] = next;
}
if (frame.Sequence < next)
{
return Array.Empty<CultNetRudpDeliveredFrame>();
}
if (frame.Sequence > next)
{
if (!_orderedBuffers.TryGetValue(frame.ChannelId, out var buffer))
{
buffer = new SortedDictionary<uint, PendingOrderedFrame>();
_orderedBuffers[frame.ChannelId] = buffer;
}
buffer[frame.Sequence] = new PendingOrderedFrame { Frame = frame, NextSequence = nextSequenceAfterFrame };
return Array.Empty<CultNetRudpDeliveredFrame>();
}
_orderedNextSequenceByChannel[frame.ChannelId] = nextSequenceAfterFrame;
return new[] { frame }.Concat(DrainOrdered(frame.ChannelId)).ToArray();
}
private IReadOnlyList<CultNetRudpDeliveredFrame> DrainOrdered(string channelId)
/// <summary>
/// Delivers, in sequence order, every held ordered frame whose first sequence is at most one past
/// the watermark. Ordered delivery has one owner: the watermark. A frame this call does not
/// deliver waits for the sequence below it, and nothing else releases it.
/// </summary>
private IReadOnlyList<CultNetRudpDeliveredFrame> DrainOrdered()
{
var delivered = new List<CultNetRudpDeliveredFrame>();
if (!_orderedBuffers.TryGetValue(channelId, out var buffer))
if (!_receivedThrough.HasValue)
{
return delivered;
}
while (_orderedNextSequenceByChannel.TryGetValue(channelId, out var next) && buffer.TryGetValue(next, out var pending))
var through = _receivedThrough.Value;
foreach (var pair in _orderedHeld)
{
buffer.Remove(next);
delivered.Add(pending.Frame);
_orderedNextSequenceByChannel[channelId] = pending.NextSequence;
SkipReceivedNonChannelSequences(channelId);
if (through != uint.MaxValue && pair.Key > through + 1)
{
break;
}
delivered.Add(pair.Value);
}
foreach (var frame in delivered)
{
_orderedHeld.Remove(frame.Sequence);
}
return delivered;
}
private void SkipReceivedNonChannelSequences(string channelId)
{
if (!_orderedNextSequenceByChannel.TryGetValue(channelId, out var next))
{
return;
}
while (_receivedSequences.Contains(next)
&& (!_orderedBuffers.TryGetValue(channelId, out var buffer) || !buffer.ContainsKey(next)))
{
next++;
_orderedNextSequenceByChannel[channelId] = next;
}
}
private ushort AllocateFragmentId()
{
var fragmentId = _nextFragmentId;
@ -1917,18 +2069,23 @@ namespace GameCult.Networking
var preserved = new List<CultNetTransportFrame>();
while (_deliveredFrames.Count > 0)
preserved.Add(_deliveredFrames.Dequeue());
// A flush belongs to the generation it started in: whatever ends that generation, and
// whatever begins after it, the writes being waited on are gone.
long generation;
lock (_sessionGate)
generation = _session.Generation;
try
{
while (true)
{
// An ended session forgot its unacknowledged writes; reporting them
// flushed would be a lie.
var ended = DisconnectReason;
if (ended != null)
throw new InvalidOperationException(
"RUDP session ended before its reliable writes were acknowledged: " + Encoding.UTF8.GetString(ended));
lock (_sessionGate)
{
// An ended session forgot its unacknowledged writes; reporting them flushed
// would be a lie.
if (_session.Ended || _session.Generation != generation)
throw new InvalidOperationException(
"RUDP session ended before its reliable writes were acknowledged: " +
(DisconnectReason == null ? "no reason given" : Encoding.UTF8.GetString(DisconnectReason)));
if (_session.OutstandingReliablePacketCount == 0)
return;
}
@ -2171,7 +2328,6 @@ namespace GameCult.Networking
error.SocketErrorCode == SocketError.ConnectionAborted ||
error.SocketErrorCode == SocketError.Shutdown)
{
DisconnectReason = Array.Empty<byte>();
return false;
}
@ -2198,6 +2354,10 @@ namespace GameCult.Networking
_stats.PacketsDropped++;
return true;
}
// A Connect from another endpoint is a new client, whatever sequence it carries: the endpoint
// moves and the Connect starts a new generation. Only a Connect from the peer's own
// endpoint can be its repeat.
var connectFromNewEndpoint = false;
if (_remoteEndPoint == null)
{
// Only a Connect claims the endpoint of a server-mode transport.
@ -2211,8 +2371,16 @@ namespace GameCult.Networking
}
else if (!_remoteEndPoint.Equals(remote))
{
_stats.PacketsDropped++;
return true;
if (_mode == CultNetRudpSocketMode.Server && packet.PacketType == CultNetRudpPacketType.Connect)
{
_remoteEndPoint = remote;
connectFromNewEndpoint = true;
}
else
{
_stats.PacketsDropped++;
return true;
}
}
TracePacket("rx", packet, received, remote);
@ -2223,17 +2391,12 @@ namespace GameCult.Networking
CultNetRudpPacket accept;
try
{
// A connect from the peer this transport already accepted repeats: answer it
// with the accept already owed and queue nothing.
if (_session.Connected)
{
accept = _session.AnswerRepeatedConnect(packet, NowMs());
}
else
{
accept = _session.AcceptConnect(packet, NowMs());
DisconnectReason = null;
}
// The session answers a repeat of the connect it accepted with the accept
// already owed, and anything else with a new generation.
if (connectFromNewEndpoint)
_session.ResetPeerState();
accept = _session.AcceptConnect(packet, NowMs());
DisconnectReason = null;
}
catch (InvalidOperationException)
{
@ -2399,9 +2562,10 @@ namespace GameCult.Networking
/// </summary>
public uint ConnectionId { get; set; }
/// <summary>
/// Gets or sets the first local packet sequence for each accepted peer session.
/// Gets or sets the first local packet sequence for each accepted peer session. Unset, each
/// session draws its own at random.
/// </summary>
public uint InitialSequence { get; set; } = 100;
public uint? InitialSequence { get; set; }
/// <summary>
/// Gets or sets the resend delay in milliseconds.
/// </summary>
@ -2493,7 +2657,7 @@ namespace GameCult.Networking
private const int WireBurstPackets = 32;
private readonly Socket _socket;
private readonly uint _connectionId;
private readonly uint _initialSequence;
private readonly uint? _initialSequence;
private readonly long _resendDelayMs;
private readonly int? _maxFragmentBytes;
private readonly int? _maxPendingReliablePackets;
@ -2668,25 +2832,29 @@ namespace GameCult.Networking
{
var connectPayload = packet.Payload ?? Array.Empty<byte>();
_peers.TryGetValue(peerKey, out var admitted);
// The same payload from an admitted peer is a repeated Connect: answer it with the
// Accept already owed and queue nothing. A different payload is a new generation.
if (admitted != null && admitted.ConnectPayload.AsSpan().SequenceEqual(connectPayload))
// The admitted peer's session decides whether this Connect repeats the one it accepted:
// answer a repeat with the Accept already owed and queue nothing. Any other Connect is
// a new generation.
if (admitted != null)
{
lock (admitted.SessionGate)
{
CultNetRudpPacket reply;
try
if (admitted.Session.ConnectRepeats(packet))
{
reply = admitted.Session.AnswerRepeatedConnect(packet, NowMs());
}
catch (InvalidOperationException)
{
_stats.PacketsDropped++;
CultNetRudpPacket reply;
try
{
reply = admitted.Session.AcceptConnect(packet, NowMs(), _acceptPayload);
}
catch (InvalidOperationException)
{
_stats.PacketsDropped++;
return true;
}
SendPacket(admitted.RemoteEndPoint, reply);
return true;
}
SendPacket(admitted.RemoteEndPoint, reply);
}
return true;
}
var peer = new CultNetRudpSocketServerPeer(

55
tests/GameCult.Caching.Tests/PreCut2StoreFormatTests.cs
View file

@ -178,6 +178,61 @@ namespace GameCult.Caching.Tests
Throws.TypeOf<CultStoreUnreadableException>().With.Message.Contains("format version"));
}
// Every runtime reads a store file the same way: a missing file is an empty store, an existing empty file is not
// a store (no CultCache writer leaves one), and an empty array carries no header, so it claims no format and is an
// empty store too.
[Test]
public async Task SingleFileRefusesAnEmptyFileAndOpensAMissingFileOrAnEmptyArrayAsEmpty()
{
var root = Path.Combine(Path.GetTempPath(), $"cultlib-empty-{Guid.NewGuid():N}");
Directory.CreateDirectory(root);
try
{
var empty = Path.Combine(root, "empty.msgpack");
File.WriteAllBytes(empty, Array.Empty<byte>());
using (var cache = new CultCache())
{
Assert.Catch(() => cache.AddBackingStore(new SingleFileMessagePackBackingStore(empty)));
Assert.That(cache.BackingStores, Is.Empty);
}
Assert.That(File.ReadAllBytes(empty), Is.Empty);
var emptyArray = Path.Combine(root, "empty-array.msgpack");
File.WriteAllBytes(emptyArray, new byte[] { 0x90 });
foreach (var path in new[] { Path.Combine(root, "missing.msgpack"), emptyArray })
{
using var cache = new CultCache();
cache.AddBackingStore(new SingleFileMessagePackBackingStore(path));
await cache.PullAllBackingStoresAsync();
Assert.That(cache.GetAll<PreCut2FixtureItem>(), Is.Empty, path);
}
}
finally
{
if (Directory.Exists(root)) Directory.Delete(root, recursive: true);
}
}
// A refusal echoes a header only in the shape cultcache.store.v<digits>; any other is described by its length.
[TestCase("cultcache.store.v12", true)]
[TestCase("cultcache.store.SECRET-HEADER", false)]
[TestCase("cultcache.store.v12SECRET", false)]
[TestCase("cultcache.store.v", false)]
public void SingleFileRefusalEchoesAHeaderOnlyInTheKnownShape(string header, bool echoed)
{
var snapshot = new CultPersistedStoreSnapshot { FormatVersion = header };
var error = Assert.Throws<CultStoreUnreadableException>(() => CultDocumentMessagePackSerialization.RequireSingleFileFormat(snapshot))!;
if (echoed)
{
Assert.That(error.Message, Does.Contain("format " + header + " is not"));
}
else
{
Assert.That(error.Message, Does.Not.Contain("SECRET").And.Not.Contain("format " + header + " is not"));
Assert.That(error.Message, Does.Contain($"of {header.Length} bytes"));
}
}
[Test]
public void V1StoreWrittenAtTheBaseCommitStillDecodesByteForByte()
{

1
tests/GameCult.Caching.Tests/StoreReadabilityTests.cs
View file

@ -426,7 +426,6 @@ namespace GameCult.Caching.Tests
// A commit that lands onto the file as it is keeps the header the file carries: a store already v3 stays v3.
[TestCase("../v3-base.msgpack", "cultcache.store.v3")]
[TestCase("compatible-id-only-v3.bin", "cultcache.store.v3")]
[TestCase("zero-byte.bin", "cultcache.store.v1")]
[TestCase("empty-array.bin", "cultcache.store.v1")]
public void AConditionalCommitOntoAReadableFileKeepsItsHeader(string vector, string header)
{

26
tests/GameCult.Mesh.Tests/CultMeshStreamingTests.cs
View file

@ -902,6 +902,32 @@ public sealed class CultMeshStreamingTests
await act.Should().ThrowAsync<NotSupportedException>();
}
// A database serves only what its shards own, including when a handle reads a same-schema instance of another CLR type
// out of the cache behind it.
[TestCase("tests.mesh_note.v1", true)]
[TestCase("tests.unrelated.v1", false)]
public async Task DocumentHandle_ReadsASameSchemaInstanceOnlyWhenAShardOwnsIt(string ownedSchemaId, bool served)
{
var cache = new CultCache();
var key = new CultRecordKey("note:one");
await cache.UpsertAsync(new MeshNoteDocument { Schema = "tests.mesh_note.v1", Text = "cached" }, new CultRecordHandle<MeshNoteDocument>(key));
var database = new CultNetDatabase(cache, new CultNetDatabaseOptions
{
RuntimeId = "reader",
Shards = new[] { new CultNetShardDescriptor("notes", "reader", epoch: 1, isPrimary: true, schemaIds: new[] { ownedSchemaId }) }
});
var handle = CultMesh.Document<MeshNoteAliasDocument>(database, key, CultMesh.Verse("starbridge", "reader"));
if (served)
{
(await handle.LatestAsync()).Text.Should().Be("cached");
}
else
{
await FluentActions.Awaiting(() => handle.LatestAsync()).Should().ThrowAsync<KeyNotFoundException>();
}
}
[Test]
public async Task DocumentHandle_SubmitsPredictionsThroughCultNetDatabase()
{

1
tests/GameCult.Networking.InteropPeer/Program.cs
View file

@ -873,6 +873,7 @@ static void RudpServeOnce(Dictionary<string, string> options)
}
if (disconnectReason != null)
{
transport.FlushReliable(TimeSpan.FromSeconds(2));
transport.Disconnect(disconnectReason);
}
PollRudpAfterSend(transport, TimeSpan.FromMilliseconds(250));

1199
tests/GameCult.Networking.Tests/CultNetJournalTests.cs Normal file
View file

File diff suppressed because it is too large Load diff

351
tests/GameCult.Networking.Tests/CultNetPublicationOrderTests.cs Normal file
View file

@ -0,0 +1,351 @@
#nullable enable
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using GameCult.Caching;
using NUnit.Framework;
using R3;
using static GameCult.Networking.Tests.NetworkingTests;
namespace GameCult.Networking.Tests
{
public sealed class CultNetPublicationOrderTests
{
private static readonly TimeSpan Wait = TimeSpan.FromSeconds(5);
private static NetworkSchemaNote Note(string text) => new() { Schema = "tests.networking_note.v1", Text = text };
private static string? TextOf(object? document) => (document as NetworkSchemaNote)?.Text;
// A1: two writers to one key. A is admitted first, B second; the cache ends at B. A's publication is delayed
// (the window between leaving the cache gate and reaching the handler), B runs to completion meanwhile.
[Test]
public async Task S1_TheLogOrdersOneKeysWritesInAdmissionOrder()
{
var cache = new CultCache();
var entered = new ManualResetEventSlim();
var release = new ManualResetEventSlim();
using var widen = cache.Watch<NetworkSchemaNote>().Subscribe(change =>
{
if (change.Document?.Text == "A")
{
entered.Set();
release.Wait(Wait);
}
});
var database = new CultNetDatabase(cache);
var seen = new List<string?>();
using var subscriber = database.Watch<NetworkSchemaNote>().Subscribe(change => { lock (seen) seen.Add(change.Document?.Text); });
var key = new CultRecordKey("soul:order");
var a = Task.Run(() => cache.UpsertAsync(Note("A"), new CultRecordHandle<NetworkSchemaNote>(key)).GetAwaiter().GetResult());
Assert.That(entered.Wait(Wait), Is.True);
await database.PutAsync(key, Note("B"));
release.Set();
await a;
var shard = database.Shards[0].ShardId;
var last = database.GetMutationLog(shard).Where(entry => entry.Key.Value == key.Value).OrderBy(entry => entry.Sequence).Last();
// A replica built from the log.
var replicaCache = new CultCache();
var replica = new CultNetDatabase(replicaCache, new CultNetDatabaseOptions
{
Shards = [new CultNetShardDescriptor(shard, "other", epoch: database.Shards[0].Epoch, isPrimary: false)]
});
await replica.ApplyShardLogResponseAsync(new CultNetShardLogResponseMessage
{
ShardId = shard,
ShardEpoch = database.Shards[0].Epoch,
Entries = database.GetMutationLogMessages(shard).ToArray()
});
Assert.Multiple(() =>
{
Assert.That(cache.Get<NetworkSchemaNote>(key)!.Text, Is.EqualTo("B"), "the cache admitted A then B");
Assert.That(TextOf(last.Document), Is.EqualTo("B"), "the log's last write for the key must be the cache's last write");
Assert.That(replicaCache.Get<NetworkSchemaNote>(key)?.Text, Is.EqualTo("B"), "a replica built from the log must match the primary");
// Delivery is not ordered across threads (order is data: a wire peer drops the stale change by Sequence, Cut 6),
// so a subscriber sees both changes, once each, in either order.
Assert.That(seen, Is.EquivalentTo(new[] { "A", "B" }), "each change is published exactly once");
});
}
// Same property, no instrumentation: two writers hammer one key, then the log's last entry is compared with the cache.
[Test]
public void S1b_UninstrumentedOneKeyRaceLeavesLogAndCacheAgreeing()
{
var divergent = 0;
const int rounds = 300;
for (var round = 0; round < rounds; round++)
{
var cache = new CultCache();
var database = new CultNetDatabase(cache);
var key = new CultRecordKey("soul:hammer");
using var start = new Barrier(2);
Parallel.For(0, 2, writer =>
{
start.SignalAndWait();
for (var i = 0; i < 40; i++)
cache.UpsertAsync(Note($"{writer}:{i}"), new CultRecordHandle<NetworkSchemaNote>(key)).GetAwaiter().GetResult();
});
var last = database.GetMutationLog(database.Shards[0].ShardId).OrderBy(entry => entry.Sequence).Last();
if (!ReferenceEquals(last.Document, cache.Get<NetworkSchemaNote>(key)))
divergent++;
}
TestContext.Progress.WriteLine($"SOUL S1b divergent rounds: {divergent}/{rounds}");
Assert.That(divergent, Is.EqualTo(0), $"{divergent}/{rounds} rounds ended with the log's last write differing from the cache");
}
// A2: a door's context is keyed by instance. A plain removal of the instance the door just admitted, landing in
// the window before the door's own change is handled, consumes the door's context.
[Test]
public async Task S2_ARemovalNeverWearsAReconcilingPutsContext()
{
var cache = new CultCache();
var entered = new ManualResetEventSlim();
var release = new ManualResetEventSlim();
using var widen = cache.Watch<NetworkSchemaNote>().Subscribe(change =>
{
if (change.Document?.Text == "authoritative")
{
entered.Set();
release.Wait(Wait);
}
});
var database = new CultNetDatabase(cache, new CultNetDatabaseOptions
{
RuntimeId = "local",
ClientAuthorityScopes = [new CultNetClientAuthorityScope("local")]
});
var key = new CultRecordKey("soul:reconcile");
await database.PutPredictedAsync(key, Note("predicted"));
var published = new ConcurrentQueue<(CultNetDatabaseChangeKind Kind, string? Text)>();
using var subscriber = database.Watch<NetworkSchemaNote>().Subscribe(change => published.Enqueue((change.Kind, change.Document?.Text)));
var logged = database.GetMutationLog(database.Shards[0].ShardId).Count;
var message = database.Documents.CreateRawDocumentPutMessage("m", new CultRecordHandle<NetworkSchemaNote>(key), Note("authoritative"));
message.ShardId = database.Shards[0].ShardId;
message.ShardEpoch = database.Shards[0].Epoch;
var a = Task.Run(() => database.ApplyPutAsync(message));
Assert.That(entered.Wait(Wait), Is.True);
cache.Remove(key);
release.Set();
await a;
var log = database.GetMutationLog(database.Shards[0].ShardId, logged).ToArray();
Assert.Multiple(() =>
{
Assert.That(published.Count(change => change.Kind == CultNetDatabaseChangeKind.Removed), Is.EqualTo(1),
"the removal is published as a removal: " + string.Join(", ", published));
Assert.That(published.Where(change => change.Kind == CultNetDatabaseChangeKind.Reconciled).Select(change => change.Text),
Is.EqualTo(new[] { "authoritative" }), "the reconcile carries the authoritative document");
Assert.That(log.Where(entry => entry.Kind != CultNetDatabaseChangeKind.Removed).All(entry => entry.Document != null), Is.True,
"no put entry without a document: " + string.Join(", ", log.Select(entry => entry.Kind + ":" + TextOf(entry.Document))));
});
}
// A4: chained replication. B is a replica of P, C a replica of B. A replica-applied delete of a key B does not
// hold (B removed it locally, a supported write) must still keep B's log contiguous for C.
[Test]
public async Task S3_AChainedReplicaSeesNoGapWhenItsSourceAppliesADeleteOfAnAbsentKey()
{
const string shardId = "chain";
var schemaId = new CultCache().Registry.GetRequired<NetworkSchemaNote>().SchemaId;
CultNetDatabase Make(CultCache cache, bool primary) => new(cache, new CultNetDatabaseOptions
{
Shards = [new CultNetShardDescriptor(shardId, "p", epoch: 2, isPrimary: primary, schemaIds: [schemaId])]
});
var p = Make(new CultCache(), true);
var bCache = new CultCache();
var b = Make(bCache, false);
var c = Make(new CultCache(), false);
async Task Pull(CultNetDatabase from, CultNetDatabase to)
{
await to.ApplyShardLogResponseAsync(new CultNetShardLogResponseMessage
{
ShardId = shardId,
ShardEpoch = 2,
Entries = from.GetMutationLogMessages(shardId, to.GetAppliedShardSequence(shardId)).ToArray()
});
}
var k = new CultRecordKey("soul:chain:k");
await p.PutAsync(k, Note("x"));
await Pull(p, b);
await Pull(b, c);
bCache.Remove(k);
await p.DeleteAsync<NetworkSchemaNote>(k);
await p.PutAsync(new CultRecordKey("soul:chain:k2"), Note("y"));
await Pull(p, b);
Assert.Multiple(() =>
{
Assert.That(b.GetMutationLog(shardId).Select(entry => entry.Sequence), Is.EqualTo(new[] { 1L, 2L, 3L }),
"B's log holds every primary entry it applied");
Assert.DoesNotThrowAsync(() => Pull(b, c), "C pulls B's log without a gap");
});
}
// A5: a subscriber that answers a Reconciled change by predicting the same key again keeps its prediction.
[Test]
public async Task S4_APredictionMadeInReactionToAReconcileSurvives()
{
var cache = new CultCache();
var database = new CultNetDatabase(cache, new CultNetDatabaseOptions
{
RuntimeId = "local",
ClientAuthorityScopes = [new CultNetClientAuthorityScope("local")]
});
var key = new CultRecordKey("soul:repredict");
var kinds = new ConcurrentQueue<CultNetDatabaseChangeKind>();
var repredicted = false;
using var subscriber = database.Watch<NetworkSchemaNote>().Subscribe(change =>
{
kinds.Enqueue(change.Kind);
if (change.Kind == CultNetDatabaseChangeKind.Reconciled && !repredicted)
{
repredicted = true;
database.PutPredictedAsync(key, Note("predicted again")).GetAwaiter().GetResult();
}
});
CultNetDocumentPutRawMessage Put(string text)
{
var message = database.Documents.CreateRawDocumentPutMessage(text, new CultRecordHandle<NetworkSchemaNote>(key), Note(text));
message.ShardId = database.Shards[0].ShardId;
message.ShardEpoch = database.Shards[0].Epoch;
return message;
}
await database.PutPredictedAsync(key, Note("predicted"));
await database.ApplyPutAsync(Put("auth1"));
await database.ApplyPutAsync(Put("auth2"));
Assert.That(kinds, Is.EqualTo(new[]
{
CultNetDatabaseChangeKind.Predicted,
CultNetDatabaseChangeKind.Reconciled,
CultNetDatabaseChangeKind.Predicted,
CultNetDatabaseChangeKind.Reconciled
}));
}
// Target of AWriteAfterADatabaseDeleteIsStillPublishedAndLogged: a door context must not outlive its door. A
// refused prediction leaves nothing behind that a later write of the same instance would wear.
[Test]
public async Task S6_ARefusedDoorLeavesNoContextForALaterWriteOfTheSameInstance()
{
var registry = CultDocumentRegistry.Shared;
var cache = new CultCache(registry, GameCult.Caching.MessagePack.CultCacheMessagePack.CreateCodec(registry));
var documents = new CultNetDocumentRegistry(registry)
.Register(CultNetDocumentBinding.ForDocument<CultNetDocumentRegistryVariantTests.VariantWireFixture>(registry));
var database = new CultNetDatabase(cache, new CultNetDatabaseOptions
{
DocumentRegistry = documents,
RuntimeId = "local",
ClientAuthorityScopes = [new CultNetClientAuthorityScope("local")]
});
var baseKey = new CultRecordKey("soul-refuse-base");
var variantKey = new CultRecordKey("soul-refuse-variant");
await cache.UpsertAsync(new CultNetDocumentRegistryVariantTests.VariantWireFixture { Name = "base", Power = 1 },
new CultRecordHandle<CultNetDocumentRegistryVariantTests.VariantWireFixture>(baseKey));
await cache.UpsertVariantAsync(variantKey, baseKey, new[]
{
cache.Override<CultNetDocumentRegistryVariantTests.VariantWireFixture>(nameof(CultNetDocumentRegistryVariantTests.VariantWireFixture.Name), "v")
});
var instance = new CultNetDocumentRegistryVariantTests.VariantWireFixture { Name = "reused", Power = 3 };
Assert.That(async () => await database.PutPredictedAsync(variantKey, instance), Throws.InvalidOperationException);
var kinds = new ConcurrentQueue<CultNetDatabaseChangeKind>();
using var subscriber = database.Watch<CultNetDocumentRegistryVariantTests.VariantWireFixture>().Subscribe(change => kinds.Enqueue(change.Kind));
var logged = database.GetMutationLog(database.Shards[0].ShardId).Count;
await cache.UpsertAsync(instance, new CultRecordHandle<CultNetDocumentRegistryVariantTests.VariantWireFixture>(new CultRecordKey("soul-refuse-other")));
Assert.Multiple(() =>
{
Assert.That(kinds, Is.EqualTo(new[] { CultNetDatabaseChangeKind.Added }));
Assert.That(database.GetMutationLog(database.Shards[0].ShardId, logged), Has.Count.EqualTo(1));
});
}
// Target of ADisposedDatabaseNoLongerObservesTheCache: after Dispose the database must stop logging, which a
// durable log store can observe even though the disposed Subject hides publication.
[Test]
public async Task S7_ADisposedDatabaseAppendsNothingToItsDurableLog()
{
var root = System.IO.Path.Combine(System.IO.Path.GetTempPath(), "soul-log-" + Guid.NewGuid().ToString("N"));
var store = new CultNetFileShardMutationLogStore(root);
var cache = new CultCache();
var database = new CultNetDatabase(cache, new CultNetDatabaseOptions { MutationLogStore = store });
var shard = database.Shards[0].ShardId;
await cache.UpsertAsync(Note("before"), new CultRecordHandle<NetworkSchemaNote>(new CultRecordKey("soul:dispose:before")));
Assert.That(store.Read(shard), Has.Count.EqualTo(1));
database.Dispose();
try { await cache.UpsertAsync(Note("after"), new CultRecordHandle<NetworkSchemaNote>(new CultRecordKey("soul:dispose:after"))); }
catch { /* a throwing handler would also be a failure below */ }
Assert.That(store.Read(shard), Has.Count.EqualTo(1));
System.IO.Directory.Delete(root, true);
}
// The log entry exists before its change is published: a subscriber that reads the log from its handler finds it.
[Test]
public async Task TheLogEntryIsWrittenBeforeTheChangeIsPublished()
{
var cache = new CultCache();
var database = new CultNetDatabase(cache);
var key = new CultRecordKey("soul:log-first");
var loggedWhenPublished = new ConcurrentQueue<bool>();
using var subscriber = database.Watch<NetworkSchemaNote>().Subscribe(change =>
loggedWhenPublished.Enqueue(database.GetMutationLog(database.Shards[0].ShardId).Any(entry => entry.Key.Value == key.Value)));
await database.PutAsync(key, Note("one"));
cache.Remove(key);
Assert.That(loggedWhenPublished, Is.EqualTo(new[] { true, true }));
}
// An observer that mutates the admitted instance in place and upserts it again writes a second change of the same
// instance. It is published as its own update and never wears the context of the door's change, even when it
// reaches the database first.
[Test]
public async Task AnInPlaceReUpsertOfTheDoorsInstanceDoesNotWearTheDoorsContext()
{
var cache = new CultCache();
using var normalizer = cache.Watch<NetworkSchemaNote>().Subscribe(change =>
{
if (change.Document?.Text == "authoritative")
{
change.Document.Text = "normalized";
cache.UpsertAsync(change.Document, new CultRecordHandle<NetworkSchemaNote>(change.Key)).GetAwaiter().GetResult();
}
});
var database = new CultNetDatabase(cache, new CultNetDatabaseOptions
{
RuntimeId = "local",
ClientAuthorityScopes = [new CultNetClientAuthorityScope("local")]
});
var key = new CultRecordKey("soul:in-place");
var seen = new ConcurrentQueue<(CultNetDatabaseChangeKind Kind, string? Previous)>();
using var subscriber = database.Watch<NetworkSchemaNote>().Subscribe(change => seen.Enqueue((change.Kind, change.PreviousDocument?.Text)));
var message = database.Documents.CreateRawDocumentPutMessage("m", new CultRecordHandle<NetworkSchemaNote>(key), Note("authoritative"));
message.ShardId = database.Shards[0].ShardId;
message.ShardEpoch = database.Shards[0].Epoch;
await database.PutPredictedAsync(key, Note("predicted"));
await database.ApplyPutAsync(message);
// The observer's nested write is delivered depth-first, inside the delivery of the change that caused it, so
// the database may publish the two in either order; what is pinned is that each carries its own context.
Assert.That(seen, Is.EquivalentTo(new (CultNetDatabaseChangeKind, string?)[]
{
(CultNetDatabaseChangeKind.Predicted, null),
(CultNetDatabaseChangeKind.Reconciled, "predicted"),
(CultNetDatabaseChangeKind.Updated, "normalized")
}));
}
}
}

429
tests/GameCult.Networking.Tests/CultNetPublishesCommitsTests.cs Normal file
View file

@ -0,0 +1,429 @@
#nullable enable
using System;
using System.Collections.Concurrent;
using System.IO;
using System.Linq;
using System.Net;
using System.Net.Sockets;
using System.Threading;
using System.Threading.Tasks;
using GameCult.Caching;
using GameCult.Caching.MessagePack;
using NUnit.Framework;
using R3;
using static GameCult.Networking.Tests.NetworkingTests;
using static GameCult.Networking.Tests.CultNetDocumentRegistryVariantTests;
namespace GameCult.Networking.Tests
{
// The cache is the one owner of "a change was committed": whatever door wrote it, a subscriber over the wire and the
// mutation log see it once. A database write door publishes its own change (it carries the shard-log entry); every
// other change in the hold publishes from the cache.
public sealed class CultNetPublishesCommitsTests
{
private static CultNetDatabaseChangeRawMessage[] Of(Wire wire, string key) =>
wire.Changes.Where(change => KeyOf(change) == key).ToArray();
private static string? KeyOf(CultNetDatabaseChangeRawMessage change) => change.Document?.RecordKey ?? change.RecordKey;
private sealed class Wire : IDisposable
{
private readonly CancellationTokenSource _cancellation = new();
private readonly Thread _thread;
private readonly RudpCultNetSchemaServer _server;
private readonly CultNetDatabaseSubscriptionServer _subscriptions;
private readonly ICultNetSchemaClient _client;
public ConcurrentQueue<CultNetDatabaseChangeRawMessage> Changes { get; } = new();
public int Snapshots;
public ConcurrentQueue<CultNetErrorMessage> Errors { get; } = new();
private readonly CultNetDatabase _database;
public Wire(CultNetDatabase database)
{
_database = database;
var socket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
socket.Bind(new IPEndPoint(IPAddress.Loopback, 0));
socket.ReceiveTimeout = 20;
_server = new RudpCultNetSchemaServer(new RudpCultNetSchemaServerOptions { RuntimeId = "publish-server", Socket = socket });
_subscriptions = new CultNetDatabaseSubscriptionServer(_server, database);
_thread = new Thread(() =>
{
while (!_cancellation.IsCancellationRequested)
{
_ = _server.PollOnceAsync().GetAwaiter().GetResult();
Thread.Sleep(1);
}
}) { IsBackground = true };
_thread.Start();
_client = CultNetSchemaClients.CreateRudp("publish-client");
_client.OnCultNet<CultNetDatabaseChangeRawMessage>(Changes.Enqueue);
_client.OnCultNet<CultNetErrorMessage>(Errors.Enqueue);
_client.OnCultNet<CultNetSnapshotResponseRawMessage>(_ => Interlocked.Increment(ref Snapshots));
_client.Connect("127.0.0.1", _server.LocalEndPoint.Port);
}
public async Task SubscribeAsync(bool includeSnapshot, params Type[] documentTypes)
{
await Until(() => _client.Connected);
_client.SendCultNet(new CultNetDatabaseSubscribeMessage
{
MessageId = "subscribe",
SubscriptionId = "all",
SchemaIds = documentTypes.Select(type => _database.Cache.Registry.GetRequired(type).SchemaId).ToArray(),
IncludeSnapshot = includeSnapshot
});
await Until(() => Snapshots > 0 || !Errors.IsEmpty);
Assert.That(Errors.Select(error => error.Error), Is.Empty, "the subscription was refused");
}
public void Dispose()
{
_cancellation.Cancel();
_thread.Join();
_client.Dispose();
_subscriptions.Dispose();
_server.Dispose();
}
}
private static async Task Until(Func<bool> condition)
{
var deadline = DateTimeOffset.UtcNow + TimeSpan.FromSeconds(5);
while (!condition())
{
if (DateTimeOffset.UtcNow > deadline)
Assert.Fail("Condition was not satisfied before timeout.");
await Task.Delay(5);
}
}
// One reliable ordered channel: a sentinel write that has arrived proves every earlier write has had its chance
// to arrive twice.
private static async Task SettleAsync(CultNetDatabase database, Wire wire)
{
var sentinel = "tests:publish:sentinel:" + Guid.NewGuid().ToString("N");
await database.PutAsync(new CultRecordKey(sentinel), new NetworkSchemaNote { Schema = "tests.networking_note.v1" });
await Until(() => wire.Changes.Any(change => KeyOf(change) == sentinel));
}
private static NetworkSchemaNote Note(string text) => new() { Schema = "tests.networking_note.v1", Text = text };
[Test]
public async Task ACacheCommitAndUpsertReachASubscriberOnce()
{
var cache = new CultCache();
var database = new CultNetDatabase(cache);
using var wire = new Wire(database);
await wire.SubscribeAsync(false, typeof(NetworkSchemaNote));
var key = new CultRecordKey("tests:publish:commit");
cache.Commit(batch => batch.Upsert(Note("committed"), new CultRecordHandle<NetworkSchemaNote>(key)));
await Until(() => Of(wire, key.Value).Length >= 1);
cache.Commit(batch => batch.Remove(key));
await Until(() => Of(wire, key.Value).Length >= 2);
await cache.UpsertAsync(Note("upserted"), new CultRecordHandle<NetworkSchemaNote>(key));
await SettleAsync(database, wire);
Assert.That(Of(wire, key.Value).Select(change => change.ChangeKind), Is.EqualTo(new[] { "added", "removed", "added" }));
Assert.That(database.GetMutationLog(database.Shards[0].ShardId).Count(entry => entry.Key.Value == key.Value), Is.EqualTo(3));
}
[Test]
public async Task ADatabasePutAndDeleteStillPublishExactlyOnce()
{
var database = new CultNetDatabase(new CultCache());
using var wire = new Wire(database);
await wire.SubscribeAsync(false, typeof(NetworkSchemaNote));
var key = new CultRecordKey("tests:publish:put");
await database.PutAsync(key, Note("put"));
await database.DeleteAsync<NetworkSchemaNote>(key);
await SettleAsync(database, wire);
Assert.That(Of(wire, key.Value).Select(change => change.ChangeKind), Is.EqualTo(new[] { "added", "removed" }));
Assert.That(database.GetMutationLog(database.Shards[0].ShardId).Count(entry => entry.Key.Value == key.Value), Is.EqualTo(2));
}
[Test]
public async Task AReplicatedWriteIsPublishedAndLoggedOnceNotEchoed()
{
var cache = new CultCache();
var database = new CultNetDatabase(cache);
using var wire = new Wire(database);
await wire.SubscribeAsync(false, typeof(NetworkSchemaNote));
var key = new CultRecordKey("tests:publish:replicated");
var message = database.Documents.CreateRawDocumentPutMessage(
"put-replicated", new CultRecordHandle<NetworkSchemaNote>(key), Note("remote"));
message.ShardId = database.Shards[0].ShardId;
message.ShardEpoch = database.Shards[0].Epoch;
await database.ApplyPutAsync(message);
await SettleAsync(database, wire);
Assert.That(Of(wire, key.Value), Has.Length.EqualTo(1));
Assert.That(database.GetMutationLog(database.Shards[0].ShardId).Count(entry => entry.Key.Value == key.Value), Is.EqualTo(1));
}
[Test]
public async Task ALoadRemovalCarriesItsKey()
{
var path = Path.Combine(Path.GetTempPath(), $"cultnet-load-{Guid.NewGuid():N}.cc");
try
{
var key = new CultRecordKey("tests:publish:loaded");
var writer = new CultCache();
writer.AddBackingStore(new SingleFileMessagePackBackingStore(path));
await writer.UpsertAsync(Note("there"), new CultRecordHandle<NetworkSchemaNote>(key));
writer.FlushAllBackingStores();
var cache = new CultCache();
cache.AddBackingStore(new SingleFileMessagePackBackingStore(path));
var database = new CultNetDatabase(cache);
Assert.That(cache.Get<NetworkSchemaNote>(key), Is.Not.Null);
using var wire = new Wire(database);
await wire.SubscribeAsync(true, typeof(NetworkSchemaNote));
writer.Remove(key);
writer.FlushAllBackingStores();
await cache.PullAllBackingStoresAsync();
await SettleAsync(database, wire);
Assert.That(Of(wire, key.Value).Select(change => change.ChangeKind), Is.EqualTo(new[] { "removed" }));
}
finally
{
if (File.Exists(path)) File.Delete(path);
if (File.Exists(path + ".lock")) File.Delete(path + ".lock");
}
}
[Test]
public async Task ABaseEditPublishesItsVariantDependentAndTheSubscriberIsRefusedItByKey()
{
var registry = CultDocumentRegistry.Shared;
var cache = new CultCache(registry, CultCacheMessagePack.CreateCodec(registry));
var documents = new CultNetDocumentRegistry(registry)
.Register(CultNetDocumentBinding.ForDocument<VariantWireFixture>(registry));
var database = new CultNetDatabase(cache, new CultNetDatabaseOptions { DocumentRegistry = documents });
var baseKey = new CultRecordKey("publish-variant-base");
var variantKey = new CultRecordKey("publish-variant-big");
await cache.UpsertAsync(new VariantWireFixture { Name = "base", Power = 1 }, new CultRecordHandle<VariantWireFixture>(baseKey));
await cache.UpsertVariantAsync(variantKey, baseKey, new[]
{
cache.Override<VariantWireFixture>(nameof(VariantWireFixture.Name), "big"),
cache.Override<VariantWireFixture>(nameof(VariantWireFixture.Power), 9)
});
using var wire = new Wire(database);
await wire.SubscribeAsync(false, typeof(VariantWireFixture));
var errorsBefore = wire.Errors.Count;
await database.PutAsync(baseKey, new VariantWireFixture { Name = "base edited", Power = 2 });
await Until(() => wire.Errors.Count > errorsBefore);
await Until(() => Of(wire, baseKey.Value).Length >= 1);
Assert.That(wire.Errors.Skip(errorsBefore).Single().Error, Does.Contain(variantKey.Value),
"the re-resolved dependent is published, then refused (Q6) by key");
Assert.That(Of(wire, baseKey.Value), Has.Length.EqualTo(1));
Assert.That(Of(wire, variantKey.Value), Is.Empty, "a variant is never carried as a record");
}
// An observer that rewrites the very key a door just wrote is a second write: it publishes and logs on its own.
[Test]
public async Task AnObserverRewriteOfTheDoorsOwnKeyIsPublishedAndLoggedToo()
{
var cache = new CultCache();
var database = new CultNetDatabase(cache);
using var wire = new Wire(database);
await wire.SubscribeAsync(false, typeof(NetworkSchemaNote));
var key = new CultRecordKey("tests:publish:normalized");
using var normalizer = cache.Watch<NetworkSchemaNote>().Subscribe(change =>
{
if (change.Document?.Text == "raw")
cache.UpsertAsync(Note("normalized"), new CultRecordHandle<NetworkSchemaNote>(change.Key)).GetAwaiter().GetResult();
});
await database.PutAsync(key, Note("raw"));
await Until(() => Of(wire, key.Value).Length >= 2);
await SettleAsync(database, wire);
Assert.That(cache.Get<NetworkSchemaNote>(key)!.Text, Is.EqualTo("normalized"));
Assert.That(Of(wire, key.Value).Select(change => change.ChangeKind), Is.EqualTo(new[] { "added", "updated" }));
Assert.That(database.GetMutationLog(database.Shards[0].ShardId).Count(entry => entry.Key.Value == key.Value), Is.EqualTo(2));
}
private static (CultCache Cache, CultNetDocumentRegistry Documents) VariantCache()
{
var registry = CultDocumentRegistry.Shared;
var cache = new CultCache(registry, CultCacheMessagePack.CreateCodec(registry));
var documents = new CultNetDocumentRegistry(registry)
.Register(CultNetDocumentBinding.ForDocument<VariantWireFixture>(registry));
return (cache, documents);
}
private static System.Collections.Generic.List<string> Record(CultNetDatabase database, Func<object, string> describe)
{
var seen = new System.Collections.Generic.List<string>();
database.WatchAllChanges().Subscribe(change => seen.Add(describe(change)));
return seen;
}
private static string KeyOfChange(object change) => ((dynamic)change).Key.Value;
[Test]
public async Task ABaseWriteAndItsReResolvedVariantAreLoggedInCacheOrder()
{
var (cache, documents) = VariantCache();
var database = new CultNetDatabase(cache, new CultNetDatabaseOptions { DocumentRegistry = documents });
var baseKey = new CultRecordKey("publish-variant-base");
var variantKey = new CultRecordKey("publish-variant-big");
await cache.UpsertAsync(new VariantWireFixture { Name = "base", Power = 1 }, new CultRecordHandle<VariantWireFixture>(baseKey));
await cache.UpsertVariantAsync(variantKey, baseKey, new[]
{
cache.Override<VariantWireFixture>(nameof(VariantWireFixture.Name), "big")
});
var published = Record(database, KeyOfChange);
var logged = database.GetMutationLog(database.Shards[0].ShardId).Count;
await database.PutAsync(baseKey, new VariantWireFixture { Name = "base edited", Power = 2 });
Assert.That(published, Is.EqualTo(new[] { baseKey.Value, variantKey.Value }));
Assert.That(database.GetMutationLog(database.Shards[0].ShardId, logged).Select(entry => entry.Key.Value),
Is.EqualTo(new[] { baseKey.Value, variantKey.Value }));
}
private static CultNetDatabase Replica(CultCache cache, CultNetDocumentRegistry? documents, string schemaId) =>
new(cache, new CultNetDatabaseOptions
{
DocumentRegistry = documents,
Shards =
[
new CultNetShardDescriptor("replica-shard", "runtime-a", epoch: 3, isPrimary: false, schemaIds: [schemaId])
]
});
[Test]
public async Task AReplicaLogHoldsOnlyThePrimarysSequencesWhateverItResolvesLocally()
{
var (sourceCache, sourceDocuments) = VariantCache();
var (replicaCache, replicaDocuments) = VariantCache();
var schemaId = sourceCache.Registry.GetRequired<VariantWireFixture>().SchemaId;
var source = new CultNetDatabase(sourceCache, new CultNetDatabaseOptions
{
DocumentRegistry = sourceDocuments,
Shards = [new CultNetShardDescriptor("replica-shard", "runtime-a", epoch: 3, isPrimary: true, schemaIds: [schemaId])]
});
var replica = Replica(replicaCache, replicaDocuments, schemaId);
var baseKey = new CultRecordKey("publish-variant-base");
var variantKey = new CultRecordKey("publish-variant-big");
await replicaCache.UpsertAsync(new VariantWireFixture { Name = "base", Power = 1 }, new CultRecordHandle<VariantWireFixture>(baseKey));
await replicaCache.UpsertVariantAsync(variantKey, baseKey, new[]
{
replicaCache.Override<VariantWireFixture>(nameof(VariantWireFixture.Name), "big")
});
await source.PutAsync(baseKey, new VariantWireFixture { Name = "base edited", Power = 2 });
await replica.ApplyShardLogResponseAsync(new CultNetShardLogResponseMessage
{
ShardId = "replica-shard",
ShardEpoch = 3,
Entries = source.GetMutationLogMessages("replica-shard").ToArray()
});
Assert.That(replicaCache.Get<VariantWireFixture>(variantKey)!.Power, Is.EqualTo(2), "the dependent re-resolved");
Assert.That(replica.GetMutationLog("replica-shard").Select(entry => entry.Sequence), Is.EqualTo(new[] { 1L }),
"the replica log holds the primary's entry and mints none of its own");
}
[Test]
public async Task ALocalCacheWriteOnAReplicaIsPublishedOnceAndNeverLogged()
{
var cache = new CultCache();
var schemaId = cache.Registry.GetRequired<NetworkSchemaNote>().SchemaId;
var replica = Replica(cache, null, schemaId);
var published = Record(replica, KeyOfChange);
await cache.UpsertAsync(Note("local"), new CultRecordHandle<NetworkSchemaNote>(new CultRecordKey("tests:publish:local")));
Assert.That(published, Is.EqualTo(new[] { "tests:publish:local" }));
Assert.That(replica.GetMutationLog("replica-shard"), Is.Empty);
}
[Test]
public void ConcurrentWritersLoseNoLogEntryAndReuseNoSequence()
{
var cache = new CultCache();
var database = new CultNetDatabase(cache);
const int writers = 4, perWriter = 5000;
Parallel.For(0, writers, writer =>
{
for (var i = 0; i < perWriter; i++)
cache.UpsertAsync(Note("n"), new CultRecordHandle<NetworkSchemaNote>(new CultRecordKey($"tests:publish:c:{writer}:{i}")))
.GetAwaiter().GetResult();
});
var sequences = database.GetMutationLog(database.Shards[0].ShardId).Select(entry => entry.Sequence).ToArray();
Assert.That(sequences, Has.Length.EqualTo(writers * perWriter));
Assert.That(sequences.Distinct().Count(), Is.EqualTo(writers * perWriter));
}
[Test]
public async Task AWriteAfterADatabaseDeleteIsStillPublishedAndLogged()
{
var cache = new CultCache();
var database = new CultNetDatabase(cache);
var key = new CultRecordKey("tests:publish:after-delete");
await database.PutAsync(key, Note("one"));
await database.DeleteAsync<NetworkSchemaNote>(key);
var published = Record(database, KeyOfChange);
await cache.UpsertAsync(Note("two"), new CultRecordHandle<NetworkSchemaNote>(key));
Assert.That(published, Is.EqualTo(new[] { key.Value }));
Assert.That(database.GetMutationLog(database.Shards[0].ShardId).Select(entry => entry.Kind),
Is.EqualTo(new[]
{
CultNetDatabaseChangeKind.Added, CultNetDatabaseChangeKind.Removed, CultNetDatabaseChangeKind.Added
}));
}
[Test]
public async Task AShardSnapshotResyncPublishesItsRemovalsAndPutsOnceAndLogsNothing()
{
var cache = new CultCache();
var schemaId = cache.Registry.GetRequired<NetworkSchemaNote>().SchemaId;
var replica = Replica(cache, null, schemaId);
var stale = new CultRecordKey("tests:publish:stale");
var fresh = new CultRecordKey("tests:publish:fresh");
await cache.UpsertAsync(Note("stale"), new CultRecordHandle<NetworkSchemaNote>(stale));
var published = Record(replica, change => ((dynamic)change).Kind.ToString() + ":" + ((dynamic)change).Key.Value);
var put = replica.Documents.CreateRawDocumentPutMessage(
"snapshot-fresh", new CultRecordHandle<NetworkSchemaNote>(fresh), Note("fresh"));
await replica.ApplyShardSnapshotResponseAsync(replica.Shards[0], new CultNetSnapshotResponseRawMessage
{
ShardId = "replica-shard",
ShardEpoch = 3,
Documents = [put.Document!],
ShardLogSequence = 5
});
Assert.That(published, Is.EqualTo(new[] { "Removed:" + stale.Value, "Added:" + fresh.Value }));
Assert.That(replica.GetMutationLog("replica-shard"), Is.Empty);
}
[Test]
public async Task ADisposedDatabaseNoLongerObservesTheCache()
{
var cache = new CultCache();
var database = new CultNetDatabase(cache);
var published = Record(database, KeyOfChange);
database.Dispose();
await cache.UpsertAsync(Note("after"), new CultRecordHandle<NetworkSchemaNote>(new CultRecordKey("tests:publish:disposed")));
Assert.That(published, Is.Empty);
}
}
}

840
tests/GameCult.Networking.Tests/CultNetRudpConnectGenerationTests.cs Normal file
View file

@ -0,0 +1,840 @@
#nullable enable
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Net.Sockets;
using System.Text;
using GameCult.Mesh;
using NUnit.Framework;
namespace GameCult.Networking.Tests
{
/// <summary>
/// A Connect's sequence says whether it repeats the one a session accepted, and the handshake
/// alone seeds the watermark that orders delivery.
/// </summary>
public sealed class CultNetRudpConnectGenerationTests
{
private const uint ConnectionId = 0x10203090;
private static readonly CultNetRudpSendOptions Ordered = new CultNetRudpSendOptions { Reliable = true, Ordered = true };
private static CultNetRudpSession Session(uint initialSequence) =>
new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = ConnectionId, InitialSequence = initialSequence });
private static CultNetRudpPacket Send(CultNetRudpSession session, string text, bool ordered = true, string channel = "schema") =>
session.Send(channel, Encoding.UTF8.GetBytes(text), new CultNetRudpSendOptions { Reliable = true, Ordered = ordered });
private static string[] Names(CultNetRudpReceiveResult result) =>
result.Delivered.Select(frame => Encoding.UTF8.GetString(frame.Payload)).ToArray();
private static (CultNetRudpPacket Connect, CultNetRudpPacket Accept) Handshake(CultNetRudpSession client, CultNetRudpSession server)
{
var connect = client.CreateConnect(0);
var accept = server.AcceptConnect(connect, 0);
client.Receive(accept, 0);
Assert.That(client.Connected && server.Connected, Is.True);
return (connect, accept);
}
private static Socket Bind()
{
var socket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
socket.Bind(new IPEndPoint(IPAddress.Loopback, 0));
socket.ReceiveTimeout = 20;
return socket;
}
private static List<CultNetRudpPacket> Drain(Socket socket)
{
var packets = new List<CultNetRudpPacket>();
var buffer = new byte[65535];
EndPoint from = new IPEndPoint(IPAddress.Any, 0);
while (socket.Poll(20_000, SelectMode.SelectRead))
{
var received = socket.ReceiveFrom(buffer, ref from);
packets.Add(CultNetRudpPacketCodec.Decode(buffer.AsSpan(0, received).ToArray()));
}
return packets;
}
[Test]
public void AReconnectOnTheSameSessionIsNotStrandedByAWriteTheEndedSessionOwed()
{
var client = Session(1);
var server = Session(500);
Handshake(client, server);
Send(server, "lost");
server.CreateDisconnect(Encoding.UTF8.GetBytes("gone"));
client.Receive(server.AcceptConnect(client.CreateConnect(1), 1), 1);
Assert.That(Names(client.Receive(Send(server, "after"), 2)), Is.EqualTo(new[] { "after" }));
}
[Test]
public void AServerThatAcceptsANewClientForgetsTheOldClientsSequences()
{
var oldClient = Session(50);
var server = Session(500);
Handshake(oldClient, server);
foreach (var name in new[] { "c1", "c2", "c3" })
Assert.That(Names(server.Receive(Send(oldClient, name), 1)), Is.EqualTo(new[] { name }));
server.CreateDisconnect(Encoding.UTF8.GetBytes("restart"));
var newClient = Session(2);
newClient.Receive(server.AcceptConnect(newClient.CreateConnect(2), 2), 2);
Assert.That(Names(server.Receive(Send(newClient, "fresh"), 3)), Is.EqualTo(new[] { "fresh" }));
}
[Test]
public void ARetransmittedConnectAfterDataHasFlowedDoesNotResetTheSession()
{
var client = Session(1);
var server = Session(500);
var (connect, _) = Handshake(client, server);
var frame = Send(client, "once");
Assert.That(Names(server.Receive(frame, 1)), Is.EqualTo(new[] { "once" }));
Assert.That(server.ConnectRepeats(connect), Is.True);
Assert.That(server.AcceptConnect(connect, 2).PacketType, Is.EqualTo(CultNetRudpPacketType.Ack), "the Accept was already acknowledged");
Assert.That(server.OutstandingReliablePacketCount, Is.EqualTo(0), "a repeat queued something");
Assert.That(server.Receive(frame, 3).Delivered, Is.Empty, "a retransmitted Connect made the server forget what it had delivered");
}
[Test]
public void AConnectWithAnotherSequenceIsNotARepeatWhateverItCarries()
{
var server = Session(500);
var connect = Session(1).CreateConnect(0, Encoding.UTF8.GetBytes("same payload"));
server.AcceptConnect(connect, 0);
Assert.That(server.ConnectRepeats(connect), Is.True);
Assert.That(server.ConnectRepeats(Session(9).CreateConnect(0, Encoding.UTF8.GetBytes("same payload"))), Is.False);
var notAConnect = CultNetRudpPacketCodec.Decode(CultNetRudpPacketCodec.Encode(connect));
notAConnect.PacketType = CultNetRudpPacketType.Ping;
Assert.That(server.ConnectRepeats(notAConnect), Is.False);
server.CreateDisconnect();
Assert.That(server.ConnectRepeats(connect), Is.False, "an ended session repeats nothing");
}
[Test]
public void AnAcceptThatDoesNotNameThePendingConnectIsIgnored()
{
var client = Session(10);
var server = Session(500);
var connect = client.CreateConnect(0);
var accept = server.AcceptConnect(connect, 0);
var stale = CultNetRudpPacketCodec.Decode(CultNetRudpPacketCodec.Encode(accept));
stale.Ack = connect.Sequence + 50;
stale.AckMask = 0;
client.Receive(stale, 1);
Assert.That(client.Connected, Is.False, "a stale Accept connected the session");
Assert.That(client.PendingReliableSequences, Is.EqualTo(new[] { connect.Sequence }));
var named = CultNetRudpPacketCodec.Decode(CultNetRudpPacketCodec.Encode(accept));
named.Ack = connect.Sequence + 3;
named.AckMask = 1u << 2;
client.Receive(named, 2);
Assert.That(client.Connected, Is.True, "the mask names the Connect");
}
[Test]
public void AnAcceptAfterALocalDisconnectDoesNotReconnectTheSession()
{
var client = Session(1);
var server = Session(500);
var (_, accept) = Handshake(client, server);
client.CreateDisconnect(Encoding.UTF8.GetBytes("bye"));
Assert.That(client.Connected, Is.False);
client.Receive(accept, 5);
Assert.That(client.Connected, Is.False, "a late Accept revived an ended session");
}
[Test]
public void AReconnectForgetsWhatTheOldServerSent()
{
var client = Session(1);
var oldServer = Session(500);
Handshake(client, oldServer);
foreach (var name in new[] { "d1", "d2", "d3" })
Assert.That(Names(client.Receive(Send(oldServer, name), 1)), Is.EqualTo(new[] { name }));
var newServer = Session(501);
client.Receive(newServer.AcceptConnect(client.CreateConnect(2), 2), 2);
Assert.That(Names(client.Receive(Send(newServer, "fresh"), 3)), Is.EqualTo(new[] { "fresh" }));
}
[Test]
public void AnAcceptForAnAbandonedConnectDoesNotConnectTheSession()
{
var client = Session(1);
var server = Session(500);
var accept = server.AcceptConnect(client.CreateConnect(0), 0);
client.CreateDisconnect(Encoding.UTF8.GetBytes("never mind"));
client.Receive(accept, 1);
Assert.That(client.Connected, Is.False, "an Accept revived an abandoned Connect");
}
[Test]
public void AFullQueueOwedToAVanishedPeerDoesNotRefuseANewGeneration()
{
static CultNetRudpSession Bounded(uint initialSequence) => new CultNetRudpSession(new CultNetRudpSessionOptions
{
ConnectionId = ConnectionId,
InitialSequence = initialSequence,
MaxPendingReliablePackets = 1
});
var server = Bounded(500);
server.AcceptConnect(Session(1).CreateConnect(0), 0);
Assert.That(server.OutstandingReliablePacketCount, Is.EqualTo(1), "the Accept is owed and never acknowledged");
var next = Session(9).CreateConnect(1);
server.AcceptConnect(next, 1);
Assert.That(server.ConnectRepeats(next), Is.True);
var client = Bounded(1);
client.CreateConnect(0);
client.CreateConnect(1);
Assert.That(client.OutstandingReliablePacketCount, Is.EqualTo(1), "only the new Connect is owed");
}
[Test]
public void ADuplicateAcceptDoesNotReseedTheWatermark()
{
var client = Session(1);
var server = Session(500);
var (_, accept) = Handshake(client, server);
var s1 = Send(server, "s1");
var s2 = Send(server, "s2");
var forged = CultNetRudpPacketCodec.Decode(CultNetRudpPacketCodec.Encode(accept));
forged.Sequence = s2.Sequence;
client.Receive(forged, 1);
Assert.That(client.Receive(s2, 2).Delivered, Is.Empty, "s1 is still missing");
Assert.That(Names(client.Receive(s1, 3)), Is.EqualTo(new[] { "s1", "s2" }));
}
[Test]
public void ReliableDataBeforeTheAcceptIsNeitherDeliveredNorAcknowledged()
{
var client = Session(1);
var server = Session(500);
var accept = server.AcceptConnect(client.CreateConnect(0), 0);
var a = Send(server, "A");
var b = Send(server, "B");
foreach (var early in new[] { b, a })
{
Assert.That(client.Receive(early, 1).Delivered, Is.Empty);
var ack = client.CreateAckForReceived(early.Sequence);
Assert.That((ack.Ack, ack.AckMask), Is.EqualTo((0u, 0u)), "an unhandled packet was acknowledged");
}
client.Receive(accept, 2);
Assert.That(client.Connected, Is.True);
Assert.That(server.OutstandingReliablePacketCount, Is.EqualTo(3), "nothing was acknowledged");
Assert.That(Names(client.Receive(a, 3)), Is.EqualTo(new[] { "A" }));
Assert.That(Names(client.Receive(b, 3)), Is.EqualTo(new[] { "B" }));
}
[Test]
public void OrderedFramesAfterTheAcceptAreDeliveredInOrderWhateverArrivesFirst()
{
var client = Session(1);
var server = Session(500);
Handshake(client, server);
var a = Send(server, "A");
var b = Send(server, "B");
Assert.That(client.Receive(b, 1).Delivered, Is.Empty);
Assert.That(Names(client.Receive(a, 2)), Is.EqualTo(new[] { "A", "B" }));
}
[Test]
public void AnUnorderedReliableFrameIsDeliveredOnReceiptNeverHeldBehindAGap()
{
var client = Session(1);
var server = Session(500);
Handshake(client, server);
Send(server, "lost");
var unordered = Send(server, "now", ordered: false, channel: "media");
var ordered = Send(server, "later");
Assert.That(Names(client.Receive(unordered, 1)), Is.EqualTo(new[] { "now" }));
Assert.That(client.Receive(ordered, 2).Delivered, Is.Empty, "the ordered frame waits for the gap");
}
[Test]
public void APeerDisconnectDoesNotForgetWhatThePeerSent()
{
var client = Session(1);
var server = Session(500);
Handshake(client, server);
var s1 = Send(client, "s1");
var s2 = Send(client, "s2");
Assert.That(server.Receive(s2, 1).Delivered, Is.Empty, "s1 is missing, so s2 is held");
server.Receive(client.CreateDisconnect(Encoding.UTF8.GetBytes("bye")), 2);
Assert.That(Names(server.Receive(s1, 3)), Is.EqualTo(new[] { "s1", "s2" }), "the Disconnect forgot the held frame");
Assert.That(server.Receive(s2, 4).Delivered, Is.Empty, "the Disconnect forgot what was received");
}
/// <summary>
/// The options say nothing about the initial sequence; each session draws its own, so one options
/// object serves many sessions without giving them one sequence.
/// </summary>
[Test]
public void SessionsFromOneOptionsObjectEachDrawTheirOwnInitialSequence()
{
Assert.That(new CultNetRudpSessionOptions().InitialSequence, Is.Null);
Assert.That(new CultNetRudpSocketTransportOptions().InitialSequence, Is.Null);
Assert.That(new CultNetRudpSocketTransportServerOptions().InitialSequence, Is.Null);
Assert.That(new RudpCultNetSchemaServerOptions().InitialSequence, Is.Null);
Assert.That(new CultMeshRudpSocketOptions().InitialSequence, Is.Null);
var options = new CultNetRudpSessionOptions { ConnectionId = ConnectionId };
var firsts = Enumerable.Range(0, 64)
.Select(_ => new CultNetRudpSession(options).CreateConnect(0).Sequence)
.ToArray();
Assert.That(firsts.Distinct().Count(), Is.GreaterThan(32), "a default session does not draw its own sequence");
Assert.That(firsts.All(value => value >= 1 && value < (1u << 31)), Is.True, "a draw left [1, 2^31)");
}
/// <summary>
/// A transport built from the same options each time (a reconnect loop's factory) opens each
/// connection with its own Connect sequence.
/// </summary>
[Test]
public void AReusedOptionsObjectYieldsADifferentConnectSequenceForEachTransport()
{
using var listener = Bind();
using var clientSocket = Bind();
var options = new CultNetRudpSocketTransportOptions
{
RuntimeId = "csharp-rudp-client",
Socket = clientSocket,
Mode = CultNetRudpSocketMode.Client,
RemoteEndPoint = listener.LocalEndPoint,
ConnectionId = ConnectionId
};
var sequences = new HashSet<uint>();
for (var index = 0; index < 8; index++)
{
var client = new CultNetRudpSocketTransportConnection(options);
client.Connect("hello");
sequences.Add(Drain(listener).Last(p => p.PacketType == CultNetRudpPacketType.Connect).Sequence);
}
Assert.That(sequences, Has.Count.EqualTo(8), "two connections opened with one Connect sequence");
}
/// <summary>
/// A listener builds a session for every client it admits; each draws its own sequence, so no two
/// clients are accepted with the same one.
/// </summary>
[Test]
public void TheListenerAcceptsEachClientWithItsOwnSequence()
{
using var serverSocket = Bind();
using var server = new CultNetRudpSocketTransportServer(new CultNetRudpSocketTransportServerOptions
{
RuntimeId = "csharp-rudp-listener",
Socket = serverSocket,
ConnectionId = ConnectionId
});
var accepts = new HashSet<uint>();
for (var index = 0; index < 8; index++)
{
using var peerSocket = Bind();
peerSocket.SendTo(CultNetRudpPacketCodec.Encode(Session(1).CreateConnect(0)), serverSocket.LocalEndPoint!);
server.ReceiveOnce();
accepts.Add(Drain(peerSocket).First(p => p.PacketType == CultNetRudpPacketType.Accept).Sequence);
}
Assert.That(accepts, Has.Count.EqualTo(8), "two clients were accepted with one sequence");
}
// Ack Cut 1d: a frame from an earlier generation is a duplicate, a delayed Connect from an
// earlier attempt is stale, a Connect from a new endpoint is a new client.
private static bool NamesSequence(CultNetRudpPacket ack, uint sequence) =>
ack.Ack == sequence
|| Enumerable.Range(0, 32).Any(bit => (ack.AckMask & (1u << bit)) != 0 && ack.Ack > bit && ack.Ack - (uint)bit - 1 == sequence);
/// <summary>
/// A frame the client's old generation already delivered, retransmitted after the client
/// reconnected, is acknowledged and not delivered into the new generation.
/// </summary>
[Test]
public void AFrameFromTheClientsEarlierGenerationIsAcknowledgedNotDeliveredAgain()
{
var client = Session(10);
var server = Session(500);
Handshake(client, server);
var old = Send(client, "old");
Assert.That(Names(server.Receive(old, 1)), Is.EqualTo(new[] { "old" }));
var connect = client.CreateConnect(2);
client.Receive(server.AcceptConnect(connect, 2), 2);
Assert.That(old.Sequence, Is.LessThan(connect.Sequence));
Assert.That(server.Receive(old, 3).Delivered, Is.Empty, "an earlier generation's frame was delivered again");
Assert.That(NamesSequence(server.CreateAckForReceived(old.Sequence), old.Sequence), Is.True, "the duplicate was not acknowledged");
Assert.That(Names(server.Receive(Send(client, "fresh"), 4)), Is.EqualTo(new[] { "fresh" }));
}
[Test]
public void AFrameFromTheServersEarlierGenerationIsAcknowledgedNotDeliveredAgain()
{
var client = Session(10);
var server = Session(500);
Handshake(client, server);
var old = Send(server, "old");
Assert.That(Names(client.Receive(old, 1)), Is.EqualTo(new[] { "old" }));
client.Receive(server.AcceptConnect(client.CreateConnect(2), 2), 2);
Assert.That(client.Receive(old, 3).Delivered, Is.Empty, "an earlier generation's frame was delivered again");
Assert.That(NamesSequence(client.CreateAckForReceived(old.Sequence), old.Sequence), Is.True);
Assert.That(Names(client.Receive(Send(server, "fresh"), 4)), Is.EqualTo(new[] { "fresh" }));
}
[Test]
public void ADuplicateBelowTheReceiveWindowIsAcknowledgedByName()
{
var client = Session(10);
var server = Session(500);
Handshake(client, server);
var first = Send(client, "first");
Assert.That(Names(server.Receive(first, 1)), Is.EqualTo(new[] { "first" }));
for (var index = 0; index < 4_200; index++)
{
server.Receive(Send(client, "x"), 1);
client.Receive(server.CreateAck(), 1);
}
Assert.That(server.Receive(first, 2).Delivered, Is.Empty);
Assert.That(server.CreateAckForReceived(first.Sequence).Ack, Is.EqualTo(first.Sequence), "a duplicate below the window was not acknowledged by name");
}
/// <summary>
/// A Connect delayed from an earlier attempt of the client that is now connected does not
/// restart the session: the server answers with an Ack and the client's ordered frames keep
/// flowing.
/// </summary>
[Test]
public void ADelayedConnectFromAnEarlierAttemptDoesNotStrandTheClient()
{
var client = Session(10);
var server = Session(500);
var earlier = client.CreateConnect(0);
var retried = client.CreateConnect(300);
var accept = server.AcceptConnect(retried, 301);
client.Receive(accept, 302);
server.Receive(client.CreateAckForReceived(accept.Sequence), 302);
Assert.That(client.Connected, Is.True);
var a = Send(client, "a");
Assert.That(Names(server.Receive(a, 303)), Is.EqualTo(new[] { "a" }));
client.Receive(server.CreateAckForReceived(a.Sequence), 303);
Assert.That(server.ConnectRepeats(earlier), Is.True, "the delayed Connect starts nothing");
var reply = server.AcceptConnect(earlier, 304);
Assert.That(reply.PacketType, Is.EqualTo(CultNetRudpPacketType.Ack));
client.Receive(reply, 305);
Assert.That(client.Connected, Is.True);
var b = Send(client, "b");
Assert.That(Names(server.Receive(b, 306)), Is.EqualTo(new[] { "b" }), "the delayed Connect reset the server");
client.Receive(server.CreateAckForReceived(b.Sequence), 307);
Assert.That(client.PendingReliableSequences, Does.Not.Contain(b.Sequence));
}
/// <summary>
/// A stale Connect is the peer's echo from the past, not evidence it is alive; a retransmit of
/// the accepted Connect is.
/// </summary>
[Test]
public void ARepeatedConnectRefreshesLivenessAndAStaleOneDoesNot()
{
var client = Session(10);
var server = Session(500);
var earlier = client.CreateConnect(0);
var current = client.CreateConnect(1);
server.AcceptConnect(current, 0);
server.AcceptConnect(current, 900);
Assert.That(server.CheckTimeout(1_000, 500), Is.False, "a repeated Connect did not refresh liveness");
server.AcceptConnect(earlier, 1_400);
Assert.That(server.CheckTimeout(1_600, 500), Is.True, "a stale Connect refreshed liveness");
}
/// <summary>
/// An accepted peer that never speaks again times out: accepting a Connect is the first thing
/// heard from it.
/// </summary>
[Test]
public void AnAcceptedPeerThatGoesSilentTimesOut()
{
var server = Session(500);
server.AcceptConnect(Session(10).CreateConnect(0), 0);
Assert.That(server.CheckTimeout(100_000, 1_000), Is.True);
}
[Test]
public void StaleConnectsAreRecognisedBySerialArithmetic()
{
var server = Session(500);
server.AcceptConnect(Session(3).CreateConnect(0), 0);
CultNetRudpPacket Connect(uint sequence) => Session(sequence).CreateConnect(0);
Assert.That(server.ConnectRepeats(Connect(uint.MaxValue - 1)), Is.True, "just before the wrap");
Assert.That(server.ConnectRepeats(Connect(4)), Is.False, "just after");
Assert.That(server.ConnectRepeats(Connect(3 + 4_096)), Is.False, "far after");
Assert.That(server.ConnectRepeats(Connect(unchecked(3u - 4_095))), Is.True, "the window's edge");
Assert.That(server.ConnectRepeats(Connect(unchecked(3u - 4_096))), Is.False, "past the window");
}
/// <summary>
/// A restarted client whose random first sequence lands just before the server's current
/// Connect is answered with an Ack, never an Accept. It does not retransmit that Connect for
/// ever: once the attempt times out it starts a fresh one with a newly drawn sequence, and is
/// admitted.
/// </summary>
[Test]
public void ARestartedClientWhoseFirstSequenceIsStaleConnectsAfterRedrawing()
{
var server = Session(500);
var old = Session(1_000);
var oldAccept = server.AcceptConnect(old.CreateConnect(0), 0);
old.Receive(oldAccept, 0);
var restarted = Session(900);
var first = restarted.CreateConnect(0, Encoding.UTF8.GetBytes("join"));
// The old client has not acknowledged its Accept yet, so that is the reply: it names the
// old Connect, not this one.
var early = server.AcceptConnect(first, 1);
Assert.That(early.PacketType, Is.EqualTo(CultNetRudpPacketType.Accept));
restarted.Receive(early, 1);
Assert.That(restarted.Connected, Is.False, "an Accept for another Connect connected the client");
server.Receive(old.CreateAckForReceived(oldAccept.Sequence), 1);
var reply = server.AcceptConnect(first, 2);
Assert.That(reply.PacketType, Is.EqualTo(CultNetRudpPacketType.Ack));
restarted.Receive(reply, 2);
Assert.That(restarted.Connected, Is.False);
var retransmitted = restarted.DueResends(1_000);
Assert.That(retransmitted, Has.Count.EqualTo(1));
Assert.That(retransmitted[0].Sequence, Is.EqualTo(first.Sequence), "the attempt is still young");
var fresh = restarted.DueResends(3_500);
Assert.That(fresh, Has.Count.EqualTo(1));
Assert.That(fresh[0].PacketType, Is.EqualTo(CultNetRudpPacketType.Connect));
Assert.That(fresh[0].Sequence, Is.Not.EqualTo(first.Sequence), "the same Connect was retransmitted for ever");
Assert.That(Encoding.UTF8.GetString(fresh[0].Payload), Is.EqualTo("join"), "the fresh attempt lost the Connect payload");
var accept = server.AcceptConnect(fresh[0], 3_500);
Assert.That(accept.PacketType, Is.EqualTo(CultNetRudpPacketType.Accept));
restarted.Receive(accept, 3_500);
Assert.That(restarted.Connected, Is.True);
Assert.That(Names(server.Receive(Send(restarted, "hello"), 3_501)), Is.EqualTo(new[] { "hello" }));
}
[Test]
public void AnAnsweredConnectIsNeverReplacedByTheAttemptTimeout()
{
var client = Session(10);
var server = Session(500);
var (connect, _) = Handshake(client, server);
var resent = client.DueResends(60_000);
Assert.That(resent.All(p => p.PacketType != CultNetRudpPacketType.Connect || p.Sequence == connect.Sequence), Is.True, "a connected client started a new attempt");
Assert.That(client.Connected, Is.True);
}
/// <summary>
/// A pinned client that restarts on a new port against a server-mode transport is a new client:
/// the Connect starts a new generation and the client is admitted, though its sequence is the
/// one the old client's Connect carried.
/// </summary>
[Test]
public void ServerModeAdmitsAPinnedClientThatRestartsOnANewPort()
{
using var serverSocket = Bind();
using var firstSocket = Bind();
using var secondSocket = Bind();
using var server = new CultNetRudpSocketTransportConnection(new CultNetRudpSocketTransportOptions
{
RuntimeId = "csharp-rudp-server",
Socket = serverSocket,
Mode = CultNetRudpSocketMode.Server,
ConnectionId = ConnectionId
});
var serverEndPoint = serverSocket.LocalEndPoint!;
void ToServer(Socket from, CultNetRudpPacket packet) => from.SendTo(CultNetRudpPacketCodec.Encode(packet), serverEndPoint);
var first = Session(1);
ToServer(firstSocket, first.CreateConnect(0, Encoding.UTF8.GetBytes("join")));
server.ReceiveOnce();
first.Receive(Drain(firstSocket).First(p => p.PacketType == CultNetRudpPacketType.Accept), 0);
ToServer(firstSocket, Send(first, "hello"));
Assert.That(Encoding.UTF8.GetString(server.ReceiveOnce()!.Payload), Is.EqualTo("hello"));
var second = Session(1);
ToServer(secondSocket, second.CreateConnect(0, Encoding.UTF8.GetBytes("join")));
server.ReceiveOnce();
var accepts = Drain(secondSocket).Where(p => p.PacketType == CultNetRudpPacketType.Accept).ToArray();
Assert.That(accepts, Is.Not.Empty, "the restarted client on a new port was not admitted");
second.Receive(accepts[0], 0);
ToServer(secondSocket, Send(second, "after"));
Assert.That(Encoding.UTF8.GetString(server.ReceiveOnce()!.Payload), Is.EqualTo("after"));
}
// Ack Cut 1d, batch 2: the connect-attempt timeout.
/// <summary>
/// A server that keeps a session answers a client's repeated or stale Connect with an Ack that
/// may name the client's pending Connect. That Ack says no session started: only an Accept the
/// client honours retires its Connect, so the attempt still times out and is replaced.
/// </summary>
[Test]
public void AnAckNamingThePendingConnectDoesNotRetireIt()
{
var server = Session(500);
var old = Session(1);
var oldAccept = server.AcceptConnect(old.CreateConnect(0), 0);
old.Receive(oldAccept, 0);
server.Receive(old.CreateAckForReceived(oldAccept.Sequence), 0);
var restarted = Session(1);
var first = restarted.CreateConnect(0, Encoding.UTF8.GetBytes("join"));
var reply = server.AcceptConnect(first, 1);
Assert.That(reply.PacketType, Is.EqualTo(CultNetRudpPacketType.Ack));
Assert.That(NamesSequence(reply, first.Sequence), Is.True, "the Ack must name the pending Connect for this test to bite");
restarted.Receive(reply, 1);
Assert.That(restarted.Connected, Is.False);
Assert.That(restarted.PendingReliableSequences, Is.EqualTo(new[] { first.Sequence }), "an Ack retired the Connect");
var fresh = restarted.DueResends(3_000);
Assert.That(fresh, Has.Count.EqualTo(1), "the client waits for ever with nothing to resend");
Assert.That(Encoding.UTF8.GetString(fresh[0].Payload), Is.EqualTo("join"));
var accept = server.AcceptConnect(fresh[0], 3_000);
Assert.That(accept.PacketType, Is.EqualTo(CultNetRudpPacketType.Accept));
restarted.Receive(accept, 3_000);
Assert.That(restarted.Connected, Is.True);
}
[Test]
public void ALateCopyOfAnAbandonedConnectIsStaleOnceItsReplacementIsAccepted()
{
var server = Session(500);
var client = Session(10);
var abandoned = client.CreateConnect(0);
var fresh = client.DueResends(3_000);
Assert.That(fresh, Has.Count.EqualTo(1));
Assert.That(fresh[0].Sequence, Is.EqualTo(abandoned.Sequence + 4_095));
var accept = server.AcceptConnect(fresh[0], 3_001);
client.Receive(accept, 3_002);
server.Receive(client.CreateAckForReceived(accept.Sequence), 3_002);
Assert.That(client.Connected, Is.True);
var a = Send(client, "a");
Assert.That(Names(server.Receive(a, 3_003)), Is.EqualTo(new[] { "a" }));
client.Receive(server.CreateAckForReceived(a.Sequence), 3_003);
Assert.That(server.ConnectRepeats(abandoned), Is.True, "the late copy would restart the server");
var reply = server.AcceptConnect(abandoned, 3_004);
Assert.That(reply.PacketType, Is.EqualTo(CultNetRudpPacketType.Ack));
client.Receive(reply, 3_005);
Assert.That(client.Connected, Is.True);
Assert.That(Names(server.Receive(Send(client, "b"), 3_006)), Is.EqualTo(new[] { "b" }));
}
[Test]
public void AServerSittingAtTheJumpIsLeftByTheNextAttempt()
{
var server = Session(500);
var old = Session(5_095);
var oldAccept = server.AcceptConnect(old.CreateConnect(0), 0);
old.Receive(oldAccept, 0);
server.Receive(old.CreateAckForReceived(oldAccept.Sequence), 0);
var restarted = Session(1_000);
var first = restarted.CreateConnect(0, Encoding.UTF8.GetBytes("join"));
restarted.Receive(server.AcceptConnect(first, 1), 1);
Assert.That(restarted.Connected, Is.False);
var second = restarted.DueResends(3_000)[0];
Assert.That(second.Sequence, Is.EqualTo(5_095u), "the jump lands on the server's generation");
var reply = server.AcceptConnect(second, 3_001);
Assert.That(reply.PacketType, Is.EqualTo(CultNetRudpPacketType.Ack));
restarted.Receive(reply, 3_001);
Assert.That(restarted.Connected, Is.False);
var third = restarted.DueResends(6_000)[0];
Assert.That(third.Sequence, Is.EqualTo(5_095u + 4_095u));
var accept = server.AcceptConnect(third, 6_001);
Assert.That(accept.PacketType, Is.EqualTo(CultNetRudpPacketType.Accept));
restarted.Receive(accept, 6_001);
Assert.That(restarted.Connected, Is.True);
}
/// <summary>
/// A pinned client that restarts on the same address is a repeat by sequence, so the listener
/// answers with an Ack; the client's attempt times out, and the fresh Connect (another
/// sequence) starts a new generation the listener admits.
/// </summary>
[Test]
public void TheListenerAdmitsAPinnedClientThatRestartsOnTheSameAddress()
{
using var serverSocket = Bind();
using var peerSocket = Bind();
using var server = new CultNetRudpSocketTransportServer(new CultNetRudpSocketTransportServerOptions
{
RuntimeId = "csharp-rudp-listener",
Socket = serverSocket,
ConnectionId = ConnectionId
});
var serverEndPoint = serverSocket.LocalEndPoint!;
void ToServer(CultNetRudpPacket packet) => peerSocket.SendTo(CultNetRudpPacketCodec.Encode(packet), serverEndPoint);
var first = Session(1);
ToServer(first.CreateConnect(0, Encoding.UTF8.GetBytes("join")));
server.ReceiveOnce();
var accept = Drain(peerSocket).First(p => p.PacketType == CultNetRudpPacketType.Accept);
first.Receive(accept, 0);
ToServer(first.CreateAckForReceived(accept.Sequence));
server.ReceiveOnce();
Drain(peerSocket);
var restarted = Session(1);
ToServer(restarted.CreateConnect(0, Encoding.UTF8.GetBytes("join")));
server.ReceiveOnce();
foreach (var packet in Drain(peerSocket))
restarted.Receive(packet, 1);
Assert.That(restarted.Connected, Is.False);
ToServer(restarted.DueResends(3_000)[0]);
server.ReceiveOnce();
foreach (var packet in Drain(peerSocket))
restarted.Receive(packet, 3_001);
Assert.That(restarted.Connected, Is.True, "the restarted pinned client was never admitted");
}
// Transports.
[Test]
public void ServerModeAdmitsARestartedClientKeepsItsSessionOnARetransmitAndAdmitsANewEndpoint()
{
using var serverSocket = Bind();
using var peerSocket = Bind();
using var otherSocket = Bind();
using var server = new CultNetRudpSocketTransportConnection(new CultNetRudpSocketTransportOptions
{
RuntimeId = "csharp-rudp-server",
Socket = serverSocket,
Mode = CultNetRudpSocketMode.Server,
ConnectionId = ConnectionId
});
var serverEndPoint = serverSocket.LocalEndPoint!;
void ToServer(Socket from, CultNetRudpPacket packet) => from.SendTo(CultNetRudpPacketCodec.Encode(packet), serverEndPoint);
var first = Session(50_000);
var connect = first.CreateConnect(0, Encoding.UTF8.GetBytes("same"));
ToServer(peerSocket, connect);
server.ReceiveOnce();
first.Receive(Drain(peerSocket).First(p => p.PacketType == CultNetRudpPacketType.Accept), 0);
var sent = new[] { "c0", "c1", "c2" }.Select(name => Send(first, name)).ToArray();
foreach (var packet in sent)
{
ToServer(peerSocket, packet);
Assert.That(Encoding.UTF8.GetString(server.ReceiveOnce()!.Payload), Is.EqualTo(Encoding.UTF8.GetString(packet.Payload)));
}
// A retransmitted Connect is a repeat: what was delivered stays delivered once.
ToServer(peerSocket, connect);
server.ReceiveOnce();
ToServer(peerSocket, sent[0]);
Assert.That(server.ReceiveOnce(), Is.Null, "the retransmitted Connect reset the session");
// The same client restarted on the same address, connection id and payload.
Drain(peerSocket);
var second = Session(7);
ToServer(peerSocket, second.CreateConnect(0, Encoding.UTF8.GetBytes("same")));
server.ReceiveOnce();
second.Receive(Drain(peerSocket).First(p => p.PacketType == CultNetRudpPacketType.Accept), 0);
ToServer(peerSocket, Send(second, "after restart"));
Assert.That(Encoding.UTF8.GetString(server.ReceiveOnce()!.Payload), Is.EqualTo("after restart"));
// A Connect from a new endpoint claims the transport.
var third = Session(9);
ToServer(otherSocket, third.CreateConnect(0, Encoding.UTF8.GetBytes("elsewhere")));
server.ReceiveOnce();
third.Receive(Drain(otherSocket).First(p => p.PacketType == CultNetRudpPacketType.Accept), 0);
ToServer(otherSocket, Send(third, "from elsewhere"));
Assert.That(Encoding.UTF8.GetString(server.ReceiveOnce()!.Payload), Is.EqualTo("from elsewhere"));
}
[Test]
public void TheListenerAdmitsARestartedClientWithTheSameAddressIdAndPayload()
{
using var serverSocket = Bind();
using var peerSocket = Bind();
using var server = new CultNetRudpSocketTransportServer(new CultNetRudpSocketTransportServerOptions
{
RuntimeId = "csharp-rudp-listener",
Socket = serverSocket,
ConnectionId = ConnectionId
});
var ended = new List<CultNetRudpSocketServerPeer>();
server.PeerDisconnected += ended.Add;
var serverEndPoint = serverSocket.LocalEndPoint!;
void ToServer(CultNetRudpPacket packet) => peerSocket.SendTo(CultNetRudpPacketCodec.Encode(packet), serverEndPoint);
var first = Session(50_000);
ToServer(first.CreateConnect(0, Encoding.UTF8.GetBytes("same")));
server.ReceiveOnce();
first.Receive(Drain(peerSocket).First(p => p.PacketType == CultNetRudpPacketType.Accept), 0);
for (var index = 0; index < 3; index++)
{
ToServer(Send(first, $"c{index}"));
Assert.That(server.ReceiveOnce(), Is.Not.Null);
}
Assert.That(ended, Is.Empty);
var second = Session(7);
ToServer(second.CreateConnect(0, Encoding.UTF8.GetBytes("same")));
server.ReceiveOnce();
Assert.That(ended, Has.Count.EqualTo(1), "the restarted client was taken for a retransmit");
second.Receive(Drain(peerSocket).First(p => p.PacketType == CultNetRudpPacketType.Accept), 0);
ToServer(Send(second, "after restart"));
var frame = server.ReceiveOnce();
Assert.That(frame, Is.Not.Null);
Assert.That(Encoding.UTF8.GetString(frame!.Frame.Payload), Is.EqualTo("after restart"));
}
/// <summary>
/// Windows reports the ICMP port-unreachable for a datagram sent to a closed port as a connection
/// reset on the next receive. Nothing ended, so nothing may be reported. Other platforms do not
/// raise it, so the test is inconclusive there.
/// </summary>
[Test]
public void AnIcmpConnectionResetDoesNotEndTheSession()
{
Assume.That(OperatingSystem.IsWindows(), "only Windows raises an ICMP connection reset on UDP");
using var serverSocket = Bind();
using var peerSocket = Bind();
using var server = new CultNetRudpSocketTransportConnection(new CultNetRudpSocketTransportOptions
{
RuntimeId = "csharp-rudp-server",
Socket = serverSocket,
Mode = CultNetRudpSocketMode.Server,
ConnectionId = ConnectionId
});
peerSocket.SendTo(CultNetRudpPacketCodec.Encode(Session(1).CreateConnect(0)), serverSocket.LocalEndPoint!);
server.ReceiveOnce();
Assert.That(server.Connected, Is.True);
var closed = Bind();
var closedEndPoint = closed.LocalEndPoint!;
closed.Dispose();
serverSocket.SendTo(new byte[] { 1 }, closedEndPoint);
System.Threading.Thread.Sleep(50);
server.ReceiveOnce();
Assert.That(server.DisconnectReason, Is.Null, "an ICMP reset claimed an ending that did not happen");
Assert.That(server.Connected, Is.True);
}
}
}

134
tests/GameCult.Networking.Tests/CultNetRudpOrderedDeliveryTests.cs Normal file
View file

@ -0,0 +1,134 @@
#nullable enable
using System.Collections.Generic;
using System.Linq;
using System.Text;
using NUnit.Framework;
namespace GameCult.Networking.Tests
{
/// <summary>
/// Ordered delivery has one owner: the contiguous received-through watermark. A frame is held for
/// exactly one reason, a reliable sequence below it has not arrived, and it is delivered in the call
/// that fills the last such gap, whichever channel the gap belonged to.
/// </summary>
public sealed class CultNetRudpOrderedDeliveryTests
{
private static (CultNetRudpSession Client, CultNetRudpSession Server) Handshake()
{
var client = new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = 410, InitialSequence = 1 });
var server = new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = 410, InitialSequence = 500 });
var accept = server.AcceptConnect(client.CreateConnect(0), 0);
client.Receive(accept, 0);
return (client, server);
}
private static CultNetRudpSendOptions Reliable(bool ordered) => new CultNetRudpSendOptions { Reliable = true, Ordered = ordered };
private static CultNetRudpPacket Send(CultNetRudpSession session, string channel, string payload, bool ordered)
=> session.Send(channel, Encoding.UTF8.GetBytes(payload), Reliable(ordered));
private static string[] Names(CultNetRudpReceiveResult result)
=> result.Delivered.Select(frame => Encoding.UTF8.GetString(frame.Payload)).ToArray();
[Test]
public void OrderedFrameWaitsForAGapFilledByAnotherChannel()
{
var (sender, receiver) = Handshake();
var o1 = Send(sender, "schema", "o1", true);
var u = Send(sender, "rel", "u", false);
var o2 = Send(sender, "schema", "o2", true);
var o3 = Send(sender, "schema", "o3", true);
var o4 = Send(sender, "schema", "o4", true);
Assert.That(Names(receiver.Receive(o1, 1)), Is.EqualTo(new[] { "o1" }));
Assert.That(Names(receiver.Receive(o2, 2)), Is.Empty, "u is missing");
Assert.That(Names(receiver.Receive(o3, 3)), Is.Empty, "u is missing");
Assert.That(Names(receiver.Receive(o4, 4)), Is.Empty, "u is missing");
Assert.That(Names(receiver.Receive(u, 5)), Is.EqualTo(new[] { "u", "o2", "o3", "o4" }));
}
[Test]
public void TwoOrderedChannelsReleaseEachOtherInSequenceOrder()
{
var (sender, receiver) = Handshake();
var a1 = Send(sender, "schema", "A1", true);
var b1 = Send(sender, "other", "B1", true);
var a2 = Send(sender, "schema", "A2", true);
var a3 = Send(sender, "schema", "A3", true);
var delivered = new List<string>();
delivered.AddRange(Names(receiver.Receive(a1, 1)));
delivered.AddRange(Names(receiver.Receive(a2, 2)));
delivered.AddRange(Names(receiver.Receive(a3, 3)));
Assert.That(delivered, Is.EqualTo(new[] { "A1" }), "B1 is missing, so A2 and A3 wait");
Assert.That(Names(receiver.Receive(b1, 4)), Is.EqualTo(new[] { "B1", "A2", "A3" }));
}
[Test]
public void ChannelFirstUsedAfterOtherTrafficLosesNothing()
{
var (sender, receiver) = Handshake();
var c1 = Send(sender, "late", "C1", true);
var c2 = Send(sender, "late", "C2", true);
var x = Send(sender, "schema", "X", true);
Assert.That(Names(receiver.Receive(x, 1)), Is.Empty, "C1 is missing");
Assert.That(Names(receiver.Receive(c2, 2)), Is.Empty, "C1 is missing");
Assert.That(Names(receiver.Receive(c1, 3)), Is.EqualTo(new[] { "C1", "C2", "X" }));
}
[Test]
public void AcceptSeedsTheWatermarkOfTheConnectingSide()
{
var (client, server) = Handshake();
var s1 = Send(server, "schema", "s1", true);
var s2 = Send(server, "schema", "s2", true);
Assert.That(Names(client.Receive(s2, 1)), Is.Empty);
Assert.That(Names(client.Receive(s1, 2)), Is.EqualTo(new[] { "s1", "s2" }));
}
[Test]
public void ConnectSeedsTheWatermarkOfTheAcceptingSide()
{
var (client, server) = Handshake();
var first = Send(client, "schema", "first", true);
var second = Send(client, "schema", "second", true);
Assert.That(Names(server.Receive(second, 1)), Is.Empty);
Assert.That(Names(server.Receive(first, 2)), Is.EqualTo(new[] { "first", "second" }));
}
[Test]
public void DuplicateOfAHeldFrameIsNotDeliveredTwice()
{
var (sender, receiver) = Handshake();
var s1 = Send(sender, "schema", "s1", true);
var s2 = Send(sender, "schema", "s2", true);
Assert.That(Names(receiver.Receive(s2, 1)), Is.Empty);
Assert.That(Names(receiver.Receive(s2, 2)), Is.Empty);
Assert.That(Names(receiver.Receive(s1, 3)), Is.EqualTo(new[] { "s1", "s2" }));
Assert.That(Names(receiver.Receive(s2, 4)), Is.Empty);
}
[Test]
public void ResetPeerStateForgetsHeldFramesAndReseedsTheWatermarkFromTheNextHandshake()
{
var (sender, receiver) = Handshake();
Send(sender, "schema", "s1", true);
var s2 = Send(sender, "schema", "s2", true);
Assert.That(Names(receiver.Receive(s2, 1)), Is.Empty);
receiver.ResetPeerState();
var next = new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = 410, InitialSequence = 100 });
next.Receive(receiver.AcceptConnect(next.CreateConnect(0), 2), 2);
var d1 = Send(next, "schema", "d1", true);
var d2 = Send(next, "schema", "d2", true);
// The held s2 died with the reset, and the watermark follows the new peer.
Assert.That(Names(receiver.Receive(d2, 3)), Is.Empty);
Assert.That(Names(receiver.Receive(d1, 4)), Is.EqualTo(new[] { "d1", "d2" }));
}
}
}

215
tests/GameCult.Networking.Tests/CultNetRudpSessionEndingTests.cs Normal file
View file

@ -0,0 +1,215 @@
#nullable enable
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Net.Sockets;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using NUnit.Framework;
namespace GameCult.Networking.Tests
{
/// <summary>
/// A write belongs to the session generation it was issued in. Every way a generation ends drops
/// the writes it still owed, and a flush belongs to the generation it started in.
/// </summary>
public sealed class CultNetRudpSessionEndingTests
{
private const uint ConnectionId = 0x10203070;
private static Socket Bind()
{
var socket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
socket.Bind(new IPEndPoint(IPAddress.Loopback, 0));
socket.ReceiveTimeout = 20;
return socket;
}
private static (CultNetRudpSession Client, CultNetRudpSession Server) ClientWithALostWrite()
{
var client = new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = ConnectionId, InitialSequence = 1 });
var server = new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = ConnectionId, InitialSequence = 500 });
client.Receive(server.AcceptConnect(client.CreateConnect(0), 0), 0);
client.Send("schema", Encoding.UTF8.GetBytes("owed to the old session"), new CultNetRudpSendOptions { Reliable = true, Ordered = true });
// A fragmented write larger than the send window leaves part of it queued.
client.SendMany("schema", new byte[40 * 8], new CultNetRudpSendOptions { Reliable = true, Ordered = true }, maxFragmentBytes: 8);
Assert.That(client.OutstandingReliablePacketCount, Is.EqualTo(41));
Assert.That(client.QueuedReliablePacketCount, Is.GreaterThan(0));
return (client, server);
}
[TestCase("peer Disconnect")]
[TestCase("local disconnect")]
[TestCase("timeout")]
[TestCase("reset")]
public void EveryWayASessionEndsDropsTheWritesItOwed(string ending)
{
var (client, server) = ClientWithALostWrite();
switch (ending)
{
case "peer Disconnect": client.Receive(server.CreateDisconnect(Encoding.UTF8.GetBytes("bye")), 1); break;
case "local disconnect": client.CreateDisconnect(Encoding.UTF8.GetBytes("bye")); break;
case "timeout": Assert.That(client.CheckTimeout(1_000, 10), Is.True); break;
case "reset": client.ResetPeerState(); break;
default: throw new ArgumentException(ending);
}
Assert.That(client.OutstandingReliablePacketCount, Is.EqualTo(0), "the write survived the end");
var nextServer = new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = ConnectionId, InitialSequence = 900 });
nextServer.AcceptConnect(client.CreateConnect(2_000), 2_000);
foreach (var resend in client.DueResends(60_000))
{
Assert.That(
nextServer.Receive(resend, 60_000).Delivered,
Is.Empty,
"the old session's write was delivered in the next session");
}
}
private static (CultNetRudpSocketTransportConnection Client, Socket Peer, EndPoint ClientEndPoint) ConnectedClient()
{
var peerSocket = Bind();
var client = new CultNetRudpSocketTransportConnection(new CultNetRudpSocketTransportOptions
{
RuntimeId = "csharp-rudp-client",
Socket = Bind(),
Mode = CultNetRudpSocketMode.Client,
RemoteEndPoint = peerSocket.LocalEndPoint!,
ConnectionId = ConnectionId
});
client.Connect("hello");
var buffer = new byte[65535];
EndPoint from = new IPEndPoint(IPAddress.Any, 0);
var received = peerSocket.ReceiveFrom(buffer, ref from);
var peer = new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = ConnectionId, InitialSequence = 500 });
var accept = peer.AcceptConnect(CultNetRudpPacketCodec.Decode(buffer.AsSpan(0, received).ToArray()), 0);
peerSocket.SendTo(CultNetRudpPacketCodec.Encode(accept), from);
client.ReceiveOnce();
Assert.That(client.Connected, Is.True);
return (client, peerSocket, from);
}
[Test]
public void AFlushFailsAfterALocalDisconnectOrATimeout()
{
var (client, peerSocket, _) = ConnectedClient();
using (client)
using (peerSocket)
{
client.Send("schema", Encoding.UTF8.GetBytes("owed"));
client.Disconnect();
Assert.Throws<InvalidOperationException>(() => client.FlushReliable(TimeSpan.FromMilliseconds(200)));
}
var (timedOut, secondPeer, _) = ConnectedClient();
using (timedOut)
using (secondPeer)
{
timedOut.Send("schema", Encoding.UTF8.GetBytes("owed"));
Thread.Sleep(5);
Assert.That(timedOut.CheckTimeout(1), Is.True);
Assert.Throws<InvalidOperationException>(() => timedOut.FlushReliable(TimeSpan.FromMilliseconds(200)));
}
}
[Test]
public void AFlushThatStartedBeforeTheEndFailsEvenAfterALaterConnect()
{
var (client, peerSocket, _) = ConnectedClient();
using (client)
using (peerSocket)
{
client.Send("schema", Encoding.UTF8.GetBytes("owed"));
var flush = Task.Run(() => client.FlushReliable(TimeSpan.FromSeconds(5)));
Thread.Sleep(50);
client.Disconnect();
client.Connect("hello again");
var error = Assert.Throws<AggregateException>(() => flush.Wait(TimeSpan.FromSeconds(10)))!;
Assert.That(error.InnerException, Is.InstanceOf<InvalidOperationException>());
}
}
private static List<CultNetRudpPacket> Drain(Socket socket)
{
var packets = new List<CultNetRudpPacket>();
var buffer = new byte[65535];
EndPoint from = new IPEndPoint(IPAddress.Any, 0);
while (socket.Poll(20_000, SelectMode.SelectRead))
{
var received = socket.ReceiveFrom(buffer, ref from);
packets.Add(CultNetRudpPacketCodec.Decode(buffer.AsSpan(0, received).ToArray()));
}
return packets;
}
[Test]
public void AFlushStartedAfterAReconnectWaitsOnTheNewSession()
{
var (client, peerSocket, clientEndPoint) = ConnectedClient();
using (client)
using (peerSocket)
{
client.Send("schema", Encoding.UTF8.GetBytes("owed to the old session"));
client.Disconnect();
client.Connect("hello again");
var connect = Drain(peerSocket).Last(packet => packet.PacketType == CultNetRudpPacketType.Connect);
var accept = new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = ConnectionId, InitialSequence = 700 }).AcceptConnect(connect, 0);
peerSocket.SendTo(CultNetRudpPacketCodec.Encode(accept), clientEndPoint);
client.ReceiveOnce();
Assert.DoesNotThrow(() => client.FlushReliable(TimeSpan.FromMilliseconds(500)));
}
}
[Test]
public void AFlushStartedAfterTheServerAcceptsANewConnectWaitsOnTheNewSession()
{
using var serverSocket = Bind();
using var peerSocket = Bind();
using var server = new CultNetRudpSocketTransportConnection(new CultNetRudpSocketTransportOptions
{
RuntimeId = "csharp-rudp-server",
Socket = serverSocket,
Mode = CultNetRudpSocketMode.Server,
ConnectionId = ConnectionId
});
var first = new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = ConnectionId });
peerSocket.SendTo(CultNetRudpPacketCodec.Encode(first.CreateConnect(0)), serverSocket.LocalEndPoint!);
server.ReceiveOnce();
peerSocket.SendTo(CultNetRudpPacketCodec.Encode(first.CreateDisconnect()), serverSocket.LocalEndPoint!);
server.ReceiveOnce();
var next = new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = ConnectionId });
peerSocket.SendTo(CultNetRudpPacketCodec.Encode(next.CreateConnect(0)), serverSocket.LocalEndPoint!);
server.ReceiveOnce();
var accept = Drain(peerSocket).Last(packet => packet.PacketType == CultNetRudpPacketType.Accept);
next.Receive(accept, 0);
peerSocket.SendTo(CultNetRudpPacketCodec.Encode(next.CreateAck()), serverSocket.LocalEndPoint!);
server.ReceiveOnce();
Assert.DoesNotThrow(() => server.FlushReliable(TimeSpan.FromMilliseconds(500)));
}
[Test]
public void AnEndingDoesNotForgetWhatWasReceivedFromThePeer()
{
var client = new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = ConnectionId, InitialSequence = 1 });
var server = new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = ConnectionId, InitialSequence = 500 });
client.Receive(server.AcceptConnect(client.CreateConnect(0), 0), 0);
var ordered = new CultNetRudpSendOptions { Reliable = true, Ordered = true };
var s1 = client.Send("schema", Encoding.UTF8.GetBytes("s1"), ordered);
var s2 = client.Send("schema", Encoding.UTF8.GetBytes("s2"), ordered);
Assert.That(server.Receive(s2, 1).Delivered, Is.Empty, "s1 is missing, so s2 is held");
Assert.That(server.CheckTimeout(1_000, 10), Is.True);
Assert.That(
server.Receive(s1, 1_001).Delivered.Select(frame => Encoding.UTF8.GetString(frame.Payload)).ToArray(),
Is.EqualTo(new[] { "s1", "s2" }),
"the ending forgot the held frame");
Assert.That(server.Receive(s2, 1_002).Delivered, Is.Empty, "the ending forgot what was received");
}
}
}

11
tests/GameCult.Networking.Tests/CultNetSchemaContractTests.cs
View file

@ -195,6 +195,17 @@ namespace GameCult.Networking.Tests
AssertErrorMatchesSchema(CultNetErrorMessage.ForReferenceOutsideTarget(ex));
}
[Test]
public void ErrorMessage_UnownedSchemaRealBytes_MatchesItsSchema()
{
var message = CultNetErrorMessage.ForUnownedSchema(new CultNetUnownedSchemaException("some.schema.v1", new CultRecordKey("some:key")));
Assert.That(message.Code, Is.EqualTo("unowned_schema"));
Assert.That(message.Details!.Field, Is.EqualTo("schemaId"));
Assert.That(message.Details.Value, Is.EqualTo("some.schema.v1"));
AssertErrorMatchesSchema(message);
}
private static void AssertErrorMatchesSchema(CultNetErrorMessage message)
{
var validator = new MiniJsonSchemaValidator(SchemaDir());

137
tests/GameCult.Networking.Tests/NetworkingTests.cs
View file

@ -730,26 +730,25 @@ namespace GameCult.Networking.Tests
InitialSequence = 200,
ResendDelayMs = 100
});
sender.Receive(new CultNetRudpPacket { PacketType = CultNetRudpPacketType.Accept, ConnectionId = 7, Sequence = 1, ChannelId = "control" });
receiver.Receive(new CultNetRudpPacket { PacketType = CultNetRudpPacketType.Accept, ConnectionId = 7, Sequence = 2, ChannelId = "control" });
sender.Receive(receiver.AcceptConnect(sender.CreateConnect(0), 0), 0);
var first = sender.Send("schema", Encoding.UTF8.GetBytes("first"), new CultNetRudpSendOptions { Reliable = true, Ordered = true });
var second = sender.Send("schema", Encoding.UTF8.GetBytes("second"), new CultNetRudpSendOptions { Reliable = true, Ordered = true });
var third = sender.Send("schema", Encoding.UTF8.GetBytes("third"), new CultNetRudpSendOptions { Reliable = true, Ordered = true });
Assert.That(sender.PendingReliableSequences, Is.EqualTo(new uint[] { 10, 11, 12 }));
Assert.That(sender.PendingReliableSequences, Is.EqualTo(new uint[] { 11, 12, 13 }));
receiver.Receive(first);
receiver.Receive(third);
var ackWithGap = receiver.CreateAck();
Assert.That(ackWithGap.Ack, Is.EqualTo(12));
Assert.That(ackWithGap.AckMask, Is.EqualTo(0b10u | (1u << 9)));
Assert.That(ackWithGap.Ack, Is.EqualTo(13));
Assert.That(ackWithGap.AckMask, Is.EqualTo(0b110u));
sender.Receive(ackWithGap);
Assert.That(sender.PendingReliableSequences, Is.EqualTo(new uint[] { 11 }));
Assert.That(sender.PendingReliableSequences, Is.EqualTo(new uint[] { 12 }));
receiver.Receive(second);
var fullAck = receiver.CreateAck();
Assert.That(fullAck.Ack, Is.EqualTo(12));
Assert.That(fullAck.AckMask, Is.EqualTo(0b11u | (1u << 9)));
Assert.That(fullAck.Ack, Is.EqualTo(13));
Assert.That(fullAck.AckMask, Is.EqualTo(0b111u));
sender.Receive(fullAck);
Assert.That(sender.PendingReliableSequences, Is.Empty);
}
@ -763,7 +762,7 @@ namespace GameCult.Networking.Tests
InitialSequence = 1,
ResendDelayMs = 100
});
session.Receive(new CultNetRudpPacket { PacketType = CultNetRudpPacketType.Accept, ConnectionId = 99, Sequence = 50, ChannelId = "control" });
session.Receive(new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = 99, InitialSequence = 900 }).AcceptConnect(session.CreateConnect(0), 0), 0);
var sent = session.Send("schema", Encoding.UTF8.GetBytes("payload"), new CultNetRudpSendOptions { Reliable = true, Ordered = true, NowMs = 10 });
Assert.That(session.DueResends(90), Is.Empty);
@ -788,17 +787,14 @@ namespace GameCult.Networking.Tests
ConnectionId = 991,
InitialSequence = 500
});
sender.Receive(new CultNetRudpPacket
{ PacketType = CultNetRudpPacketType.Accept, ConnectionId = 991, Sequence = 400, ChannelId = "control" });
receiver.Receive(new CultNetRudpPacket
{ PacketType = CultNetRudpPacketType.Accept, ConnectionId = 991, Sequence = 0, ChannelId = "control" });
sender.Receive(receiver.AcceptConnect(sender.CreateConnect(0), 0), 0);
var oldPacket = sender.Send(
"snapshot",
new byte[] { 1 },
new CultNetRudpSendOptions { Reliable = true, Ordered = true, NowMs = 0 });
receiver.Receive(oldPacket);
foreach (var sequence in Enumerable.Range(2, 40).Select(value => (uint)value))
foreach (var sequence in Enumerable.Range(3, 40).Select(value => (uint)value))
{
receiver.Receive(new CultNetRudpPacket
{
@ -813,16 +809,16 @@ namespace GameCult.Networking.Tests
}
sender.Receive(receiver.CreateAck());
Assert.That(sender.PendingReliableSequences.First(), Is.EqualTo(1u));
Assert.That(sender.PendingReliableSequences.First(), Is.EqualTo(2u));
var oldRetransmit = sender.DueResends(10).Single(packet => packet.Sequence == 1);
var oldRetransmit = sender.DueResends(10).Single(packet => packet.Sequence == 2);
receiver.Receive(oldRetransmit);
var exactAck = receiver.CreateAckForReceived(oldRetransmit.Sequence);
Assert.That(exactAck.Ack, Is.EqualTo(oldRetransmit.Sequence));
Assert.That(exactAck.AckMask, Is.Zero);
sender.Receive(exactAck);
Assert.That(sender.PendingReliableSequences, Does.Not.Contain(1u));
Assert.That(sender.PendingReliableSequences, Does.Not.Contain(2u));
}
[Test]
@ -834,13 +830,7 @@ namespace GameCult.Networking.Tests
InitialSequence = 1,
ResendDelayMs = 1
});
session.Receive(new CultNetRudpPacket
{
PacketType = CultNetRudpPacketType.Accept,
ConnectionId = 100,
Sequence = 5000,
ChannelId = "control"
});
session.Receive(new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = 100, InitialSequence = 900 }).AcceptConnect(session.CreateConnect(0), 0), 0);
var packets = Enumerable.Range(0, CultNetRudpSession.ReliableSendWindowPackets)
.Select(index => session.Send("schema", Encoding.UTF8.GetBytes(index.ToString()),
new CultNetRudpSendOptions { Reliable = true, Ordered = true, NowMs = 0 }))
@ -909,7 +899,7 @@ namespace GameCult.Networking.Tests
var accept = server.AcceptConnect(connect, 0);
var repeat = CultNetRudpPacketCodec.Decode(CultNetRudpPacketCodec.Encode(connect));
repeat.Ack = accept.Sequence;
var reply = server.AnswerRepeatedConnect(repeat, 1);
var reply = server.AcceptConnect(repeat, 1);
Assert.That(reply.PacketType, Is.EqualTo(CultNetRudpPacketType.Ack));
Assert.That(server.OutstandingReliablePacketCount, Is.EqualTo(0));
}
@ -968,13 +958,7 @@ namespace GameCult.Networking.Tests
public void RudpSession_RejectsUnreliableOrderedDelivery()
{
var session = new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = 196 });
session.Receive(new CultNetRudpPacket
{
PacketType = CultNetRudpPacketType.Accept,
ConnectionId = 196,
Sequence = 50,
ChannelId = "control"
});
session.Receive(new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = 196, InitialSequence = 900 }).AcceptConnect(session.CreateConnect(0), 0), 0);
var error = Assert.Throws<InvalidOperationException>(() =>
session.Send(
"schema",
@ -992,14 +976,14 @@ namespace GameCult.Networking.Tests
InitialSequence = 1,
MaxPendingReliablePackets = 2
});
session.Receive(new CultNetRudpPacket { PacketType = CultNetRudpPacketType.Accept, ConnectionId = 102, Sequence = 50, ChannelId = "control" });
session.Receive(new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = 102, InitialSequence = 900 }).AcceptConnect(session.CreateConnect(0), 0), 0);
session.Send("schema", Encoding.UTF8.GetBytes("first"), new CultNetRudpSendOptions { Reliable = true, Ordered = true });
session.Send("schema", Encoding.UTF8.GetBytes("second"), new CultNetRudpSendOptions { Reliable = true, Ordered = true });
var error = Assert.Throws<InvalidOperationException>(() =>
session.Send("schema", Encoding.UTF8.GetBytes("third"), new CultNetRudpSendOptions { Reliable = true, Ordered = true }));
Assert.That(error!.Message, Does.Contain("reliable send queue is full"));
Assert.That(session.PendingReliableSequences, Is.EqualTo(new uint[] { 1, 2 }));
Assert.That(session.PendingReliableSequences, Is.EqualTo(new uint[] { 2, 3 }));
var fragmented = new CultNetRudpSession(new CultNetRudpSessionOptions
{
@ -1007,7 +991,7 @@ namespace GameCult.Networking.Tests
InitialSequence = 1,
MaxPendingReliablePackets = 3
});
fragmented.Receive(new CultNetRudpPacket { PacketType = CultNetRudpPacketType.Accept, ConnectionId = 103, Sequence = 50, ChannelId = "control" });
fragmented.Receive(new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = 103, InitialSequence = 900 }).AcceptConnect(fragmented.CreateConnect(0), 0), 0);
error = Assert.Throws<InvalidOperationException>(() =>
fragmented.SendMany(
@ -1032,8 +1016,8 @@ namespace GameCult.Networking.Tests
ConnectionId = 123,
InitialSequence = 100
});
sender.Receive(new CultNetRudpPacket { PacketType = CultNetRudpPacketType.Accept, ConnectionId = 123, Sequence = 90, ChannelId = "control" });
receiver.Receive(new CultNetRudpPacket { PacketType = CultNetRudpPacketType.Accept, ConnectionId = 123, Sequence = 91, ChannelId = "control" });
var accept = receiver.AcceptConnect(sender.CreateConnect(0), 0);
sender.Receive(accept);
var first = sender.Send("schema", Encoding.UTF8.GetBytes("first"), new CultNetRudpSendOptions { Reliable = true, Ordered = true });
var second = sender.Send("schema", Encoding.UTF8.GetBytes("second"), new CultNetRudpSendOptions { Reliable = true, Ordered = true });
@ -1062,8 +1046,7 @@ namespace GameCult.Networking.Tests
ConnectionId = 124,
InitialSequence = 100
});
sender.Receive(new CultNetRudpPacket { PacketType = CultNetRudpPacketType.Accept, ConnectionId = 124, Sequence = 90, ChannelId = "control" });
receiver.Receive(new CultNetRudpPacket { PacketType = CultNetRudpPacketType.Accept, ConnectionId = 124, Sequence = 91, ChannelId = "control" });
sender.Receive(receiver.AcceptConnect(sender.CreateConnect(0), 0), 0);
var first = sender.Send("schema", Encoding.UTF8.GetBytes("first"), new CultNetRudpSendOptions { Reliable = true, Ordered = true });
var control = sender.CreateAck();
@ -1091,8 +1074,7 @@ namespace GameCult.Networking.Tests
ConnectionId = 456,
InitialSequence = 100
});
sender.Receive(new CultNetRudpPacket { PacketType = CultNetRudpPacketType.Accept, ConnectionId = 456, Sequence = 90, ChannelId = "control" });
receiver.Receive(new CultNetRudpPacket { PacketType = CultNetRudpPacketType.Accept, ConnectionId = 456, Sequence = 91, ChannelId = "control" });
sender.Receive(receiver.AcceptConnect(sender.CreateConnect(0), 0), 0);
var packets = sender.SendMany(
"schema",
@ -1130,8 +1112,7 @@ namespace GameCult.Networking.Tests
InitialSequence = 100,
ResendDelayMs = 25
});
sender.Receive(new CultNetRudpPacket { PacketType = CultNetRudpPacketType.Accept, ConnectionId = connectionId, Sequence = 0, ChannelId = "control" });
receiver.Receive(new CultNetRudpPacket { PacketType = CultNetRudpPacketType.Accept, ConnectionId = connectionId, Sequence = 0, ChannelId = "control" });
sender.Receive(receiver.AcceptConnect(sender.CreateConnect(0), 0), 0);
var fragmentCount = CultNetRudpSession.ReliableSendWindowPackets + 17;
var payload = Enumerable.Range(0, fragmentCount * 8).Select(index => (byte)(index % 251)).ToArray();
@ -1269,6 +1250,16 @@ namespace GameCult.Networking.Tests
Assert.That(Encoding.UTF8.GetString(ReceiveRudpFrame(client).Payload), Is.EqualTo("server still here"));
}
/// <summary>Whether the packet's ack field and mask cover the sequence.</summary>
private static bool Acknowledges(CultNetRudpPacket packet, uint sequence)
{
if (packet.Ack == sequence)
return true;
return packet.Ack > sequence
&& packet.Ack - sequence <= 32
&& (packet.AckMask & (1u << (int)(packet.Ack - sequence - 1))) != 0;
}
private static List<CultNetRudpPacket> DrainPackets(Socket socket)
{
var packets = new List<CultNetRudpPacket>();
@ -1397,7 +1388,7 @@ namespace GameCult.Networking.Tests
var goodbye = DrainPackets(peerSocket).Single();
Assert.That(goodbye.PacketType, Is.EqualTo(CultNetRudpPacketType.Disconnect));
Assert.That(peer.Receive(goodbye, 1).Disconnected, Is.True);
Assert.That(peer.PendingReliableSequences, Does.Contain(poison.Sequence));
Assert.That(Acknowledges(goodbye, poison.Sequence), Is.False, "the goodbye acknowledged the refused frame");
// A retransmit is dropped, not acknowledged.
peerSocket.SendTo(CultNetRudpPacketCodec.Encode(poison), serverEndPoint);
@ -1444,7 +1435,7 @@ namespace GameCult.Networking.Tests
var goodbye = DrainPackets(peerSocket).Single();
Assert.That(goodbye.PacketType, Is.EqualTo(CultNetRudpPacketType.Disconnect));
Assert.That(peer.Receive(goodbye, 1).Disconnected, Is.True);
Assert.That(peer.PendingReliableSequences, Does.Contain(poison.Sequence));
Assert.That(Acknowledges(goodbye, poison.Sequence), Is.False, "the goodbye acknowledged the refused frame");
peerSocket.SendTo(CultNetRudpPacketCodec.Encode(poison), serverEndPoint);
Assert.That(server.ReceiveOnce(), Is.Null);
@ -1898,7 +1889,7 @@ namespace GameCult.Networking.Tests
Assert.That(cache.Get(batched), Is.Null);
}), Is.True);
Assert.That(published, Is.EqualTo(new[] { put }));
Assert.That(published, Is.EqualTo(new[] { put, batched }), "the batch publishes when the store commits it");
Assert.That(cache.Get<NetworkSchemaNote>(batched)?.Text, Is.EqualTo("batched"));
var reopened = new CultCache();
reopened.AddBackingStore(new SingleFileMessagePackBackingStore(path));
@ -2627,7 +2618,8 @@ namespace GameCult.Networking.Tests
var cache = new CultCache();
var delivering = new ManualResetEventSlim();
cache.OnUpdate += (_, _) => delivering.Set();
// Subscribed before the database, so it runs before the database's own observer of the same change.
using var loadDelivering = cache.Watch<object>().Subscribe(_ => delivering.Set());
var database = new CultNetDatabase(cache);
var server = new RudpCultNetSchemaServer(new RudpCultNetSchemaServerOptions
{
@ -4211,13 +4203,13 @@ namespace GameCult.Networking.Tests
Schema = "tests.networking_note.v1",
Text = "two"
});
// Outside the shard's key prefix: must reach the non-shard snapshot but never the
// Outside the shard's key prefix (written to the cache directly: the database refuses a write no shard owns): must reach the non-shard snapshot but never the
// shard-scoped one, so an unfiltered request still proves shard membership is enforced.
await database.PutAsync(new CultRecordKey("other-note:outside-shard"), new NetworkSchemaNote
await cache.UpsertAsync(new NetworkSchemaNote
{
Schema = "tests.networking_note.v1",
Text = "outside"
});
}, new CultRecordHandle<NetworkSchemaNote>(new CultRecordKey("other-note:outside-shard")));
var nonShard = registry.CreateRawSnapshotResponse(cache, "snapshot-nonshard-unfiltered", filter: null);
var shardScoped = database.CreateShardSnapshotResponse(shard, "snapshot-shard-unfiltered", filter: null);
@ -5270,10 +5262,49 @@ namespace GameCult.Networking.Tests
#pragma warning restore CS0618
}
[Test]
public void CultMeshRudpClient_DefaultBindAllowsRemoteRoutes()
private static string? RudpClientBoundHost(string endpoint, string? bindHost = null)
{
Assert.That(new CultMeshRudpSocketOptions().BindHost, Is.EqualTo("0.0.0.0"));
using var client = CultMesh.CreateRudpClient(
"csharp-cultmesh-bind-client",
0x10203060u,
endpoint,
new CultMeshRudpSocketOptions { BindHost = bindHost });
return client.Profile.Transports[0].Host;
}
// A client bound to loopback cannot send to another host on Windows, so a client for a
// remote endpoint binds the unspecified address of its family.
[Test]
public void CultMeshRudpClient_ForARemoteEndpoint_BindsTheUnspecifiedAddress()
{
Assert.That(RudpClientBoundHost("rudp://10.77.0.1:17872"), Is.EqualTo("0.0.0.0"));
Assert.That(RudpClientBoundHost("rudp://[2001:db8::1]:17872"), Is.EqualTo("::"));
}
[Test]
public void CultMeshRudpClient_ForALoopbackEndpoint_BindsLoopback()
{
Assert.That(RudpClientBoundHost("rudp://127.0.0.1:17872"), Is.EqualTo("127.0.0.1"));
Assert.That(RudpClientBoundHost("rudp://[::1]:17872"), Is.EqualTo("::1"));
}
[Test]
public void CultMeshRudpClient_BindsAnExplicitBindHostWhateverTheEndpoint()
{
Assert.That(RudpClientBoundHost("rudp://10.77.0.1:17872", "127.0.0.1"), Is.EqualTo("127.0.0.1"));
Assert.That(RudpClientBoundHost("rudp://127.0.0.1:17872", "0.0.0.0"), Is.EqualTo("0.0.0.0"));
}
[Test]
public void CultMeshRudpServer_BindsEveryIPv4InterfaceUnlessToldOtherwise()
{
using var anywhere = CultMesh.CreateRudpServer("csharp-cultmesh-bind-server", 0x10203061u);
using var loopback = CultMesh.CreateRudpServer(
"csharp-cultmesh-bind-server",
0x10203062u,
new CultMeshRudpSocketOptions { BindHost = "127.0.0.1" });
Assert.That(anywhere.Profile.Transports[0].Host, Is.EqualTo("0.0.0.0"));
Assert.That(loopback.Profile.Transports[0].Host, Is.EqualTo("127.0.0.1"));
}
[Test]

304
tests/GameCult.Networking.Tests/SoulP4ProbeTests.cs Normal file
View file

@ -0,0 +1,304 @@
#nullable enable
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Linq;
using System.Reflection;
using System.Threading;
using System.Threading.Tasks;
using GameCult.Caching;
using NUnit.Framework;
using R3;
using static GameCult.Networking.Tests.NetworkingTests;
namespace GameCult.Networking.Tests
{
// Soul pass on Cut 4 (scratch; not for merge). Category Finding asserts the invariant Soul expects and is expected to
// fail on 13072fc5; Pin asserts a promise that should hold; Measure reports numbers.
public sealed class SoulP4ProbeTests
{
private const string ShardId = "soul-shard";
private static readonly CultRecordKey One = new("soul:p4:one");
private static readonly CultRecordKey Two = new("soul:p4:two");
private static readonly CultRecordKey Three = new("soul:p4:three");
private static NetworkSchemaNote Note(string text) => new() { Schema = "tests.networking_note.v1", Text = text };
private static string SchemaId(CultCache cache) => cache.Registry.GetRequired<NetworkSchemaNote>().SchemaId;
private static CultNetDatabase Db(CultCache cache, bool primary, ICultNetShardMutationLogStore? store) =>
new(cache, new CultNetDatabaseOptions
{
Shards = [new CultNetShardDescriptor(ShardId, "primary", epoch: 1, isPrimary: primary, schemaIds: [SchemaId(cache)])],
MutationLogStore = store
});
private sealed class Store : ICultNetShardMutationLogStore
{
private readonly object _gate = new();
private readonly SortedDictionary<long, CultNetShardLogEntryMessage> _entries = new();
private long _compacted;
public Func<long, bool> RefuseAppend { get; set; } = _ => false;
public bool RefuseCompact { get; set; }
public int AppendDelayMs { get; set; }
public ManualResetEventSlim InAppend { get; } = new();
public IReadOnlyList<CultNetShardLogEntryMessage> Read(string shardId, long afterSequence = 0, int? limit = null)
{
lock (_gate)
{
var e = _entries.Values.Where(x => x.Sequence > afterSequence);
return (limit.HasValue ? e.Take(limit.Value) : e).ToArray();
}
}
public void Append(string shardId, CultNetShardLogEntryMessage entry)
{
InAppend.Set();
if (AppendDelayMs > 0) Thread.Sleep(AppendDelayMs);
lock (_gate)
{
if (RefuseAppend(entry.Sequence)) throw new IOException($"refused {entry.Sequence}");
_entries[entry.Sequence] = entry;
}
}
public long GetCompactedThrough(string shardId) { lock (_gate) return _compacted; }
public void CompactThrough(string shardId, long sequence)
{
lock (_gate)
{
if (RefuseCompact) throw new IOException("refused compact");
_compacted = Math.Max(_compacted, sequence);
foreach (var k in _entries.Keys.Where(k => k <= sequence).ToArray()) _entries.Remove(k);
}
}
}
private sealed class Peer : ICultNetShardLogFetcher, ICultNetShardSnapshotFetcher, IDisposable
{
private readonly Server _server;
private readonly CultNetDatabaseServer _dbServer;
private readonly CultNetDatabase _db;
public Peer(CultCache cache, CultNetDatabase db)
{
_db = db;
_server = new Server(cache, ServerSecurityOptions.Development());
_dbServer = new CultNetDatabaseServer(_server, db);
}
public CultNetShardLogResponseMessage Log(long after) => _dbServer.CreateShardLogResponse(new CultNetShardLogRequestMessage
{ MessageId = "pull", ShardId = ShardId, ShardEpoch = 1, AfterSequence = after });
public Task<CultNetShardLogResponseMessage> FetchAsync(CultNetShardDescriptor shard, long afterSequence, int? limit = null) => Task.FromResult(Log(afterSequence));
public Task<CultNetSnapshotResponseRawMessage> FetchAsync(CultNetShardDescriptor shard) => Task.FromResult(_db.CreateShardSnapshotResponse(_db.Shards[0], "snap"));
public void Dispose() { _dbServer.Dispose(); _server.Dispose(); }
}
private static int StashCount(CultNetDatabase db)
{
var field = typeof(CultNetDatabase).GetField("_stash", BindingFlags.NonPublic | BindingFlags.Instance)!;
var stash = (System.Collections.ICollection)field.GetValue(db)!;
lock (typeof(CultNetDatabase).GetField("_logGate", BindingFlags.NonPublic | BindingFlags.Instance)!.GetValue(db)!)
return stash.Count;
}
// F1: the recorded residual is "tail reuse". A hole in the middle of the log after a double refusal and a restart
// strands a replica behind it for good: the restarted primary has no floor, the store serves S+1 after S-1.
[Test, Category("SoulFinding")]
public async Task F1_MidLogHoleAfterDoubleRefusalAndRestartStrandsAReplica()
{
var store = new Store { RefuseAppend = s => s == 2, RefuseCompact = true };
var cache = new CultCache();
var primary = Db(cache, true, store);
var replicaCache = new CultCache();
var replica = Db(replicaCache, false, null);
await primary.PutAsync(One, Note("one"));
using (var p0 = new Peer(cache, primary))
using (var r0 = new CultNetShardReplicator(replica, new CultNetShardReplicatorOptions { Fetcher = p0, SnapshotFetcher = p0 }))
await r0.PullOnceAsync(ShardId);
try { await primary.PutAsync(Two, Note("two")); } catch (CultNetShardLogException) { }
store.RefuseCompact = false;
await primary.PutAsync(Three, Note("three"));
primary.Dispose();
var restarted = Db(cache, true, store);
using var peer = new Peer(cache, restarted);
using var replicator = new CultNetShardReplicator(replica, new CultNetShardReplicatorOptions { Fetcher = peer, SnapshotFetcher = peer });
var errors = new List<string>();
for (var attempt = 0; attempt < 3; attempt++)
{
try { await replicator.PullOnceAsync(ShardId); }
catch (Exception e) { errors.Add(e.GetType().Name + ": " + e.Message); }
}
TestContext.Out.WriteLine($"store={string.Join(",", store.Read(ShardId).Select(e => e.Sequence))} floor={restarted.GetCompactedMutationLogSequence(ShardId)} errors=[{string.Join(" | ", errors)}]");
Assert.That(new[] { One, Two, Three }.Select(k => replicaCache.Get<NetworkSchemaNote>(k)?.Text), Is.EqualTo(new[] { "one", "two", "three" }));
}
// F2: Q-P5 A says the writer gets a typed CultNetShardLogException. A commit of two changes on a refusing store
// hands the writer an AggregateException; a writer's catch (CultNetShardLogException) misses it.
[Test, Category("SoulFinding")]
public void F2_ABatchOnARefusingStoreReachesTheWriterAsTheTypedException()
{
var store = new Store { RefuseAppend = _ => true };
var cache = new CultCache();
_ = Db(cache, true, store);
Exception? caught = null;
try
{
cache.Commit(batch =>
{
batch.Upsert(Note("one"), new CultRecordHandle<NetworkSchemaNote>(One));
batch.Upsert(Note("two"), new CultRecordHandle<NetworkSchemaNote>(Two));
});
}
catch (Exception e) { caught = e; }
TestContext.Out.WriteLine($"writer caught {caught?.GetType().FullName}");
Assert.That(caught, Is.InstanceOf<CultNetShardLogException>());
}
// F3: main publishes PutPredictedAsync as Predicted and records the prediction whatever instance it is given. The
// shape rule refuses a door whose instance is already the cached one, so a mutate-then-predict is published as
// Updated, is logged on a primary, and its reconciliation is published as Updated.
[Test, Category("SoulFinding")]
public async Task F3_APredictionOfTheCachedInstanceIsPublishedAsAPrediction()
{
var cache = new CultCache();
var db = new CultNetDatabase(cache, new CultNetDatabaseOptions
{
RuntimeId = "local",
ClientAuthorityScopes = [new CultNetClientAuthorityScope("local")]
});
var shard = db.Shards[0];
var kinds = new List<CultNetDatabaseChangeKind>();
db.Watch<NetworkSchemaNote>().Subscribe(c => kinds.Add(c.Kind));
await db.PutPredictedAsync(One, Note("p1"));
var held = cache.Get<NetworkSchemaNote>(One)!;
held.Text = "p2";
await db.PutPredictedAsync(One, held);
var auth = db.Documents.CreateRawDocumentPutMessage("a", new CultRecordHandle<NetworkSchemaNote>(One), Note("auth"));
auth.ShardId = shard.ShardId;
auth.ShardEpoch = shard.Epoch;
await db.ApplyPutAsync(auth);
TestContext.Out.WriteLine($"kinds=[{string.Join(",", kinds)}] log=[{string.Join(",", db.GetMutationLog(shard.ShardId).Select(e => e.Kind))}]");
Assert.That(kinds, Is.EqualTo(new[] { CultNetDatabaseChangeKind.Predicted, CultNetDatabaseChangeKind.Predicted, CultNetDatabaseChangeKind.Reconciled }));
Assert.That(db.GetMutationLog(shard.ShardId).Select(e => e.Kind), Is.EqualTo(new[] { CultNetDatabaseChangeKind.Updated }));
}
// F4: the journal holds the cache gate across the log append, and cache reads take the gate, so a reader of an
// unrelated key waits for the whole append.
[Test, Category("SoulFinding"), Ignore("Accepted follow-up F4: a log append runs under the cache gate, so readers wait; documented on ICultNetShardMutationLogStore.")]
public async Task F4_AReaderOfAnotherKeyDoesNotWaitForALogAppend()
{
var store = new Store { AppendDelayMs = 400 };
var cache = new CultCache();
await cache.UpsertAsync(Note("two"), new CultRecordHandle<NetworkSchemaNote>(Two));
var db = Db(cache, true, store);
var writer = Task.Run(() => cache.UpsertAsync(Note("one"), new CultRecordHandle<NetworkSchemaNote>(One)).GetAwaiter().GetResult());
Assert.That(store.InAppend.Wait(5000), Is.True);
var clock = Stopwatch.StartNew();
_ = cache.Get<NetworkSchemaNote>(Two);
var waited = clock.Elapsed.TotalMilliseconds;
await writer;
TestContext.Out.WriteLine($"reader of an unrelated key waited {waited:0.0} ms for a 400 ms append");
Assert.That(waited, Is.LessThan(100));
}
// P1: the constructor registers the observer before the journal, so a change stashed by the journal always finds
// its publisher, even while another thread writes the cache during construction.
[Test, Category("SoulPin")]
public void P1_ConstructionUnderConcurrentWritesOrphansNoStash()
{
var orphans = 0;
for (var round = 0; round < 200; round++)
{
var cache = new CultCache();
using var stop = new CancellationTokenSource();
var writer = Task.Run(() =>
{
var i = 0;
while (!stop.IsCancellationRequested)
cache.UpsertAsync(Note("w" + i++), new CultRecordHandle<NetworkSchemaNote>(new CultRecordKey("soul:p4:race:" + (i % 4)))).GetAwaiter().GetResult();
});
Thread.SpinWait(200 * (round % 7));
var db = Db(cache, true, null);
Thread.SpinWait(2000);
stop.Cancel();
writer.GetAwaiter().GetResult();
orphans += StashCount(db);
db.Dispose();
}
Assert.That(orphans, Is.EqualTo(0), "no stash entry outlives its admission");
}
// P2: every stash entry is removed by its publication.
[Test, Category("SoulPin")]
public async Task P2_TheStashDrainsAfterEveryWrite()
{
var cache = new CultCache();
var db = Db(cache, true, new Store());
await db.PutAsync(One, Note("one"));
await cache.UpsertAsync(Note("two"), new CultRecordHandle<NetworkSchemaNote>(Two));
cache.Commit(batch => batch.Upsert(Note("three"), new CultRecordHandle<NetworkSchemaNote>(Three)));
await db.DeleteAsync<NetworkSchemaNote>(One);
Assert.That(StashCount(db), Is.EqualTo(0));
}
// P3: a replica never applies an entry past a hole. The primary burns 2 with compaction refused, then logs 3; a
// replica at 1 is answered compacted_history and resyncs, never applying 3 on top of 1.
[Test, Category("SoulPin")]
public async Task P3_AReplicaAtTheHoleResyncsRatherThanSkippingIt()
{
var store = new Store { RefuseAppend = s => s == 2, RefuseCompact = true };
var cache = new CultCache();
var primary = Db(cache, true, store);
using var peer = new Peer(cache, primary);
var replicaCache = new CultCache();
var replica = Db(replicaCache, false, null);
using var replicator = new CultNetShardReplicator(replica, new CultNetShardReplicatorOptions { Fetcher = peer, SnapshotFetcher = peer });
await primary.PutAsync(One, Note("one"));
await replicator.PullOnceAsync(ShardId);
try { await primary.PutAsync(Two, Note("two")); } catch (CultNetShardLogException) { }
await primary.PutAsync(Three, Note("three"));
Assert.That(peer.Log(1).Reason, Is.EqualTo("compacted_history"));
var applied = await replicator.PullOnceAsync(ShardId);
Assert.That(applied, Is.EqualTo(3));
Assert.That(new[] { One, Two, Three }.Select(k => replicaCache.Get<NetworkSchemaNote>(k)?.Text), Is.EqualTo(new[] { "one", "two", "three" }));
}
[Test, Category("SoulMeasure"), Explicit("Measurement, no assertion; writes a log file.")]
public async Task M1_BareWriteLatencyWithTheFileLogStore()
{
var root = Path.Combine(Path.GetTempPath(), "soulp4-log-" + Guid.NewGuid().ToString("N"));
try
{
var cache = new CultCache();
_ = Db(cache, true, new CultNetFileShardMutationLogStore(root));
var times = new List<double>();
for (var i = 0; i < 1000; i++)
{
var clock = Stopwatch.StartNew();
await cache.UpsertAsync(Note("n" + i), new CultRecordHandle<NetworkSchemaNote>(new CultRecordKey("soul:p4:lat:" + i)));
times.Add(clock.Elapsed.TotalMilliseconds);
}
TestContext.Out.WriteLine($"file store bare write ms: first100 avg {times.Take(100).Average():0.000}, 400-500 avg {times.Skip(400).Take(100).Average():0.000}, last100 avg {times.Skip(900).Average():0.000}, max {times.Max():0.0}");
var bare = new CultCache();
var t2 = new List<double>();
for (var i = 0; i < 1000; i++)
{
var clock = Stopwatch.StartNew();
await bare.UpsertAsync(Note("n" + i), new CultRecordHandle<NetworkSchemaNote>(new CultRecordKey("soul:p4:lat:" + i)));
t2.Add(clock.Elapsed.TotalMilliseconds);
}
TestContext.Out.WriteLine($"no database bare write ms: last100 avg {t2.Skip(900).Average():0.000}");
}
finally
{
if (Directory.Exists(root)) Directory.Delete(root, true);
}
}
}
}

142
tests/GameCult.Networking.Tests/SoulP4cProbeTests.cs Normal file
View file

@ -0,0 +1,142 @@
#nullable enable
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using GameCult.Caching;
using NUnit.Framework;
using R3;
using static GameCult.Networking.Tests.NetworkingTests;
namespace GameCult.Networking.Tests
{
// Soul pass 4c probes. They guard: a committed put whose shard log failed is not answered as a refusal (Q1); reads and watch agree
// on what a database serves (Q2); a replica log apply leaves out a row its shard does not own (Q3); the unowned refusal is not a
// shard-authority exception (Q4); two databases over one cache aggregate their log failures (Q5).
public sealed class SoulP4cProbeTests
{
private const string ShardId = "soulc-shard";
private static readonly CultRecordKey One = new("soulc:one");
private static readonly CultRecordKey Unowned = new("soulc:unowned");
private static NetworkSchemaNote Note(string text) => new() { Schema = "tests.networking_note.v1", Text = text };
private static string SchemaId(CultCache cache) => cache.Registry.GetRequired<NetworkSchemaNote>().SchemaId;
private sealed class RefusingStore : ICultNetShardMutationLogStore
{
public IReadOnlyList<CultNetShardLogEntryMessage> Read(string shardId, long afterSequence = 0, int? limit = null) => Array.Empty<CultNetShardLogEntryMessage>();
public void Append(string shardId, CultNetShardLogEntryMessage entry) => throw new IOException("refused " + entry.Sequence);
public long GetCompactedThrough(string shardId) => 0;
public void CompactThrough(string shardId, long sequence) { }
}
private sealed class Logger : GameCult.Logging.ILogger
{
public List<string> Errors { get; } = new();
public void LogInfo(string message) { }
public void LogWarning(string message) { }
public void LogError(string message) => Errors.Add(message);
public void LogDebug(string message) { }
}
private static CultNetDatabase Db(CultCache cache, bool primary, ICultNetShardMutationLogStore? store) =>
new(cache, new CultNetDatabaseOptions
{
Shards = [new CultNetShardDescriptor(ShardId, "primary", epoch: 1, isPrimary: primary, schemaIds: [SchemaId(cache)])],
MutationLogStore = store
});
// Q1: the server's own remote-put path is a writer. A put that committed and could not be logged is answered as a
// failure: the peer is told its write failed, while the cache holds it and it was published.
[Test, Category("SoulFinding")]
public async Task Q1_ServerAnswersACommittedButUnloggedRemotePutAsAFailure()
{
var cache = new CultCache();
var database = Db(cache, primary: true, new RefusingStore());
using var server = new Server(cache, ServerSecurityOptions.Development());
var logger = new Logger();
server.Logger = logger;
using var databaseServer = new CultNetDatabaseServer(server, database);
var handler = (Func<CultNetDocumentPutRawMessage, CultNetServerPeer, Task>)typeof(CultNetDatabaseServer)
.GetField("_putHandler", System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Instance)!
.GetValue(databaseServer)!;
var peer = (CultNetServerPeer)System.Runtime.CompilerServices.RuntimeHelpers.GetUninitializedObject(typeof(CultNetServerPeer));
var message = database.Documents.CreateRawDocumentPutMessage("wire", new CultRecordHandle<NetworkSchemaNote>(One), Note("wire"));
message.ShardId = ShardId;
message.ShardEpoch = 1;
try { await handler(message, peer); } catch (Exception) { }
TestContext.WriteLine("errors: " + string.Join(" | ", logger.Errors));
Assert.That(cache.Get<NetworkSchemaNote>(One)?.Text, Is.EqualTo("wire"), "the put committed");
Assert.That(logger.Errors, Has.None.Contains("raw put failed"), "a committed put must not be reported to the peer as a failed one");
}
// Q2: reads serve a cached row no shard owns, the database watch never publishes its changes: a view built from
// GetAll + Watch (CultMesh Collection, the subscription server's snapshot + WatchAllChanges) goes stale.
[Test, Category("SoulFinding")]
public async Task Q2_ReadsServeAnUnownedRowWhoseChangesTheWatchNeverPublishes()
{
var cache = new CultCache();
var database = Db(cache, primary: true, store: null);
var seen = new List<string>();
database.Watch<MeshQuickstartNote>().Subscribe(change => seen.Add(change.Document?.Body ?? "<removed>"));
await cache.UpsertAsync(new MeshQuickstartNote { NoteId = "u", Body = "v1" }, new CultRecordHandle<MeshQuickstartNote>(Unowned));
await cache.UpsertAsync(new MeshQuickstartNote { NoteId = "u", Body = "v2" }, new CultRecordHandle<MeshQuickstartNote>(Unowned));
var read = database.GetAll<MeshQuickstartNote>().Select(note => note.Body).ToArray();
TestContext.WriteLine($"GetAll={string.Join(",", read)} watch={string.Join(",", seen)}");
Assert.That(read.Length == 0 || seen.Count > 0, Is.True, "a row the database reads out must be one whose changes it publishes");
}
// Q3: a replica's log apply door writes a row its shard does not own. The snapshot apply path filters by
// ShardMatchesRawDocument; the log path does not.
[Test, Category("SoulFinding")]
public async Task Q3_AReplicaLogApplyWritesARowItsShardDoesNotOwn()
{
var cache = new CultCache();
var replica = Db(cache, primary: false, store: null);
var put = replica.Documents.CreateRawDocumentPutMessage("wire", new CultRecordHandle<MeshQuickstartNote>(Unowned), new MeshQuickstartNote { NoteId = "r", Body = "replicated" });
put.ShardId = ShardId;
put.ShardEpoch = 1;
Exception? thrown = null;
try
{
await replica.ApplyShardLogResponseAsync(new CultNetShardLogResponseMessage
{
MessageId = "log",
ShardId = ShardId,
ShardEpoch = 1,
Entries = [new CultNetShardLogEntryMessage { Sequence = 1, CommittedAt = DateTimeOffset.UtcNow.ToString("O"), ChangeKind = "added", Put = put }]
});
}
catch (Exception exception) { thrown = exception; }
TestContext.WriteLine($"thrown={thrown?.GetType().Name}: {thrown?.Message}; cached={cache.Get<MeshQuickstartNote>(Unowned)?.Body}; log={replica.GetMutationLog(ShardId).Count}");
Assert.That(cache.Get<MeshQuickstartNote>(Unowned), Is.Null, "no shard owns it here, so no path writes it");
}
// Q4 (pin): the unowned refusal is not a shard-authority exception, so the server's routing path does not see it.
[Test, Category("SoulPin")]
public void Q4_TheUnownedRefusalIsNotAShardAuthorityException()
{
var cache = new CultCache();
var database = Db(cache, primary: true, store: null);
var put = database.Documents.CreateRawDocumentPutMessage("wire", new CultRecordHandle<MeshQuickstartNote>(Unowned), new MeshQuickstartNote { NoteId = "w", Body = "w" });
var refusal = Assert.ThrowsAsync<CultNetUnownedSchemaException>(async () => await database.ApplyPutAsync(put))!;
Assert.That(refusal, Is.Not.InstanceOf<CultNetShardAuthorityException>());
}
// Q5 (pin): two databases over one cache, both refusing: a writer's catch (CultNetShardLogException) misses.
[Test, Category("SoulPin")]
public void Q5_TwoDatabasesOnOneCacheAggregateTheirLogFailures()
{
var cache = new CultCache();
_ = Db(cache, primary: true, new RefusingStore());
_ = Db(cache, primary: true, new RefusingStore());
var failure = Assert.Throws<AggregateException>(() => cache.Commit(batch =>
batch.Upsert(Note("x"), new CultRecordHandle<NetworkSchemaNote>(One))))!;
Assert.That(failure.InnerExceptions.OfType<CultNetShardLogException>().Count(), Is.EqualTo(2));
}
}
}

201
tests/GameCult.Networking.Tests/SoulP4dProbeTests.cs Normal file
View file

@ -0,0 +1,201 @@
#nullable enable
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using GameCult.Caching;
using NUnit.Framework;
using static GameCult.Networking.Tests.NetworkingTests;
namespace GameCult.Networking.Tests
{
// Soul pass 4d probes. They guard the replica log relay (a skipped entry is relayed as the primary sent it, in memory and on
// disk), key-prefixed shard reads, and measure two costs; the measurements are explicit.
public sealed class SoulP4dProbeTests
{
private const string ShardId = "soupd-shard";
private static readonly string Now = DateTimeOffset.UtcNow.ToString("O");
private static string NoteSchema(CultCache cache) => cache.Registry.GetRequired<NetworkSchemaNote>().SchemaId;
private static string MeshSchema(CultCache cache) => cache.Registry.GetRequired<MeshQuickstartNote>().SchemaId;
private static CultNetDatabase Db(CultCache cache, bool primary, string[] schemas, string? keyPrefix = null, ICultNetShardMutationLogStore? store = null) =>
new(cache, new CultNetDatabaseOptions
{
Shards = [new CultNetShardDescriptor(ShardId, "primary", epoch: 1, isPrimary: primary, schemaIds: schemas, keyPrefix: keyPrefix)],
MutationLogStore = store
});
private static CultNetShardLogResponseMessage UnownedPutLog(CultNetDatabase anyDb, string key)
{
var put = anyDb.Documents.CreateRawDocumentPutMessage("wire", new CultRecordHandle<MeshQuickstartNote>(new CultRecordKey(key)), new MeshQuickstartNote { NoteId = "p", Body = "primary-put" });
put.ShardId = ShardId;
put.ShardEpoch = 1;
return new CultNetShardLogResponseMessage
{
MessageId = "log",
ShardId = ShardId,
ShardEpoch = 1,
Entries = [new CultNetShardLogEntryMessage { Sequence = 1, CommittedAt = Now, ChangeKind = "added", Put = put }]
};
}
// Q3a: a replica (no store) that skips an unowned put relays it to a chained replica as a delete.
[Test, Category("SoulFinding")]
public async Task Q3a_MemoryLogRelaysASkippedPutAsARemoval()
{
var cache = new CultCache();
var replica = Db(cache, primary: false, [NoteSchema(cache)]);
await replica.ApplyShardLogResponseAsync(UnownedPutLog(replica, "relay:k"));
var relayed = replica.GetMutationLogMessages(ShardId);
TestContext.Out.WriteLine($"memory relay: kind={relayed[0].ChangeKind} put={relayed[0].Put != null} delete={relayed[0].Delete != null}");
Assert.That(relayed[0].ChangeKind, Is.EqualTo("added"), "the relayed entry keeps the primary's kind");
}
// Q3b: the same replica with a durable store relays the primary's put (disk and memory disagree).
[Test, Category("SoulFinding")]
public async Task Q3b_StoreBackedLogRelaysTheSkippedPutAsAPut()
{
var dir = Path.Combine(Path.GetTempPath(), "soulp4d-" + Guid.NewGuid().ToString("N"));
var cache = new CultCache();
var replica = Db(cache, primary: false, [NoteSchema(cache)], store: new CultNetFileShardMutationLogStore(dir));
await replica.ApplyShardLogResponseAsync(UnownedPutLog(replica, "relay:k"));
var relayed = replica.GetMutationLogMessages(ShardId);
var memory = replica.GetMutationLog(ShardId);
TestContext.Out.WriteLine($"store relay: kind={relayed[0].ChangeKind} put={relayed[0].Put != null}; memory entry kind={memory[0].Kind} doc={(memory[0].Document == null ? "null" : "set")}");
Assert.That(relayed[0].ChangeKind, Is.EqualTo("added"));
}
// Q3c: a chained replica that does own the row (config drift) and holds it deletes it on the relayed entry.
[Test, Category("SoulFinding")]
public async Task Q3c_AChainedReplicaThatOwnsTheRowDeletesWhatThePrimaryPut()
{
var cacheA = new CultCache();
var a = Db(cacheA, primary: false, [NoteSchema(cacheA)]);
await a.ApplyShardLogResponseAsync(UnownedPutLog(a, "relay:k"));
var cacheC = new CultCache();
var c = Db(cacheC, primary: false, [NoteSchema(cacheC), MeshSchema(cacheC)]);
await cacheC.UpsertAsync(new MeshQuickstartNote { NoteId = "p", Body = "earlier" }, new CultRecordHandle<MeshQuickstartNote>(new CultRecordKey("relay:k")));
await c.ApplyShardLogResponseAsync(new CultNetShardLogResponseMessage
{
MessageId = "chain",
ShardId = ShardId,
ShardEpoch = 1,
Entries = a.GetMutationLogMessages(ShardId).ToArray()
});
var row = cacheC.Get<MeshQuickstartNote>(new CultRecordKey("relay:k"));
TestContext.Out.WriteLine($"chained row after relay: {row?.Body ?? "<deleted>"}");
Assert.That(row, Is.Not.Null, "the primary put this row; a relay must not delete it");
}
// A skipped entry is relayed exactly as the primary sent it, whether the replica keeps its log in memory or on disk.
[Test]
public async Task SkippedEntriesAreRelayedIdenticallyFromMemoryAndFromDisk()
{
var dir = Path.Combine(Path.GetTempPath(), "soulp4d-parity-" + Guid.NewGuid().ToString("N"));
try
{
var memoryCache = new CultCache();
var inMemory = Db(memoryCache, primary: false, [NoteSchema(memoryCache)]);
var diskCache = new CultCache();
var onDisk = Db(diskCache, primary: false, [NoteSchema(diskCache)], store: new CultNetFileShardMutationLogStore(dir));
var log = UnownedPutLog(inMemory, "relay:k");
var removal = new CultNetShardLogEntryMessage
{
Sequence = 2,
CommittedAt = Now,
ChangeKind = "removed",
Delete = new CultNetDocumentDeleteMessage
{
MessageId = "d",
SchemaId = memoryCache.Registry.GetRequired<MeshQuickstartNote>().SchemaId,
RecordKey = "relay:k",
ShardId = ShardId,
ShardEpoch = 1
}
};
log.Entries = [log.Entries[0], removal];
await inMemory.ApplyShardLogResponseAsync(log);
await onDisk.ApplyShardLogResponseAsync(log);
var memory = inMemory.GetMutationLogMessages(ShardId);
var disk = onDisk.GetMutationLogMessages(ShardId);
Assert.That(memory.Select(e => e.ChangeKind), Is.EqualTo(new[] { "added", "removed" }));
Assert.That(memory.Select(e => e.ChangeKind), Is.EqualTo(disk.Select(e => e.ChangeKind)));
Assert.That(memory.Select(e => e.Put != null), Is.EqualTo(new[] { true, false }));
Assert.That(memory.Select(e => e.Put != null), Is.EqualTo(disk.Select(e => e.Put != null)));
Assert.That(memory.Select(e => e.Delete != null), Is.EqualTo(new[] { false, true }));
Assert.That(memory.Select(e => e.Delete != null), Is.EqualTo(disk.Select(e => e.Delete != null)));
Assert.That(memory[0].Put!.Document!.RecordKey, Is.EqualTo(disk[0].Put!.Document!.RecordKey));
Assert.That(memory[0].Put!.Document!.Payload, Is.EqualTo(disk[0].Put!.Document!.Payload), "the document bytes are the primary's");
}
finally
{
if (Directory.Exists(dir)) Directory.Delete(dir, true);
}
}
// Q2: key-prefixed shard: the schema is owned, a key outside the prefix is not.
[Test, Category("SoulPin")]
public async Task Q2_KeyPrefixedReadsServeOnlyTheOwnedKeys()
{
var cache = new CultCache();
var database = Db(cache, primary: true, [NoteSchema(cache)], keyPrefix: "a:");
await cache.UpsertAsync(new NetworkSchemaNote { Text = "a" }, new CultRecordHandle<NetworkSchemaNote>(new CultRecordKey("a:1")));
await cache.UpsertAsync(new NetworkSchemaNote { Text = "b" }, new CultRecordHandle<NetworkSchemaNote>(new CultRecordKey("b:1")));
Assert.That(database.GetAll<NetworkSchemaNote>().Select(n => n.Text), Is.EqualTo(new[] { "a" }));
Assert.That(await database.GetAsync<NetworkSchemaNote>(new CultRecordKey("b:1")), Is.Null);
Assert.That(await database.GetAsync<NetworkSchemaNote>(new CultRecordKey("a:1")), Is.Not.Null);
using var server = new Server(cache, ServerSecurityOptions.Development());
using var databaseServer = new CultNetDatabaseServer(server, database);
var snapshot = databaseServer.CreateSnapshotResponse(new CultNetSnapshotRequestMessage { MessageId = "all" });
Assert.That(snapshot.Documents.Select(d => d.RecordKey), Is.EqualTo(new[] { "a:1" }));
}
[Test, Category("SoulPin")]
public async Task Q2_PrimaryAllServesEveryCachedRow()
{
var cache = new CultCache();
var database = new CultNetDatabase(cache, new CultNetDatabaseOptions { Shards = [CultNetShardDescriptor.PrimaryAll()] });
await cache.UpsertAsync(new MeshQuickstartNote { NoteId = "x", Body = "x" }, new CultRecordHandle<MeshQuickstartNote>(new CultRecordKey("any:1")));
Assert.That(database.GetAll<object>().Count(), Is.EqualTo(1));
Assert.That(database.GetByName<MeshQuickstartNote>("x"), Is.Not.Null);
}
// Q2 cost: GetAll through the database versus the cache, scoped shard (reflection per row).
[Test, Category("SoulMeasure"), Explicit("Measurement, no assertion.")]
public void Q2_GetAllCost()
{
var cache = new CultCache();
var database = Db(cache, primary: false, [NoteSchema(cache)], keyPrefix: "n:");
cache.Commit(batch =>
{
for (var i = 0; i < 20000; i++)
batch.Upsert(new NetworkSchemaNote { Text = "t" + i }, new CultRecordHandle<NetworkSchemaNote>(new CultRecordKey("n:" + i)));
});
_ = cache.GetAll<NetworkSchemaNote>().Count();
_ = database.GetAll<NetworkSchemaNote>().Count();
var sw = Stopwatch.StartNew();
for (var r = 0; r < 5; r++) _ = cache.GetAll<NetworkSchemaNote>().Count();
var cacheMs = sw.Elapsed.TotalMilliseconds / 5;
sw.Restart();
for (var r = 0; r < 5; r++) _ = database.GetAll<NetworkSchemaNote>().Count();
var dbMs = sw.Elapsed.TotalMilliseconds / 5;
TestContext.Out.WriteLine($"GetAll 20000 rows: cache={cacheMs:F2}ms database={dbMs:F2}ms ratio={dbMs / Math.Max(cacheMs, 0.001):F1}x");
}
// Wire: the bytes a peer receives for the unowned refusal.
[Test, Category("SoulMeasure"), Explicit("Prints the wire bytes the Rust captured-bytes case pins.")]
public void Wire_UnownedSchemaBytes()
{
var message = CultNetErrorMessage.ForUnownedSchema(new CultNetUnownedSchemaException("some.schema.v1", new CultRecordKey("some:key")));
var bytes = MessagePack.MessagePackSerializer.Serialize(message, CultNetSchemaMessageSerialization.Options);
TestContext.Out.WriteLine("HEX=" + Convert.ToHexString(bytes));
TestContext.Out.WriteLine("JSON=" + MessagePack.MessagePackSerializer.ConvertToJson(bytes));
}
}
}

Some files were not shown because too many files have changed in this diff Show more