GameCult/CultLib
Watch
9
0
Fork
You've already forked CultLib
 0
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions

Merge origin/main: CultCache publication Cuts 1-3 and the RUDP hardening

Browse source 
One conflict, at the end of Admit in CultCache.cs. The branch had added the
IdsInMemoryOnly marking of a load's minted ids before Apply; main had turned
the thread's hold into a Hold carrying Changes and Failures and run the
journals right after Apply. Both are kept, in that order: the marking touches
only the records and runs no user code, then Apply mints the Sequences, the
hold records the changes, and the journals run under the gate.

Id planning and Project do not collide. Ids are planned and applied first,
Resolve (variants, then Project, then the name and index rules) reads the
filled documents, land runs, and only then Apply mints. A Project getter that
throws is caught by the same handler that undoes the id plans, so a refused
admission changes no document and mints nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Meta Vens 2026-09-29 23:33:11 +02:00
commit cf16cb7c1f
44 changed files with 7388 additions and 493 deletions
Download patch file Download diff file Expand all files Collapse all files

664
docs/cultcache-publication-cut.md Normal file
View file

@ -0,0 +1,664 @@
# CultCache publication: the cache owns "published once", and order stays data
Status, 2026-09-30: Cuts 1-2 landed on `main` at `2a1b8a8`, from `hands/cultcache-pub-c1c2` at `5fb3fca`.
Soul verdict: merge. Suites green: Caching 323, Networking 295, Mesh 262. Cut 3 is in Hands on
`hands/cultcache-pub-c3`, with Soul's Cuts 1-2 follow-ups folded in:
- F4: pin `Held`'s publish-on-throw guard. This is needed before Cut 4 relies on sequence density; Stryker could
not mutate `Held` (CS0165).
- F3: pin observer removal on unsubscribe.
- F1: the "getters run once" claim is false for conditional commits on variant stores.
Recorded:
- F2, the Q-P2 hazard. A rethrown observer exception makes a committed write look cancelled or refused. The
only live typed catch around a cache write is AetheriaEve `Aetheria.State.Daemon/Program.cs:3722`. It needs a
fail-fast handler, and no production code installs one.
- F5: subscribe and unsubscribe cost O(n).
- F6: writes after `Dispose` are dropped silently.
Map written by Imagination pass 1, 2026-09-29. The ends are the
operator ruling of 2026-09-30: every committed cache change reaches subscribers
exactly once. This document owns the means.
- **Body.** CultLib `main` `65311c9ad770828adafe40c2c14b2d3da517628e`. Every
`file:line` is against it unless it names the branch.
- **Superseded attempt.** `origin/hands/cultnet-publish-commits`, tip
`28248eb`, base `f9f107a`. Soul pass 3 said do not merge it, and that stands.
- Main has not touched `CultCache.cs`, `CultNetDatabase.cs`, the composition
doc or the networking tests since `f9f107a`: `git diff --stat` is empty.
- `git merge-tree 65311c9 28248eb` is clean.
- **Scope.** The C# cache's publication and CultNetDatabase's use of it. There
is no wire-format change and no other runtime changes (§6).
## Rulings (operator, 2026-09-30)
- **Q-P1 A: order as data.** Each change is published exactly once, on its writer's thread, before the write
returns, and carries its `Sequence`. The shard log is written in cache order by the gate-held journal. Wire
peers drop changes that are stale by `Sequence`. The turnstile (`bcae483`/`4562340`) and the drainer are
both rejected.
- **Q-P2 A:** a throwing observer does not stop the others. Every observer runs, the exception is rethrown to
the writer afterwards, and the observer stays subscribed. The commit stands.
- **Q-P3 A:** a public `CultCache.AddJournal(...)`, with reentry guarded.
- **Q-P4 A:** Cut 6, which drops stale changes per key on the wire, lands in this campaign.
## The finding that shapes this map
The direction Imagination was handed was: "ordered delivery belongs in the
cache's own publication, on the writer's thread, so a writer delivers its own
changes before returning". That design **has been built before, and it was
deleted**.
- `bcae483` (2026-09-13 20:09) made the cache a delivery scheduler: tickets, a
delivery gate, turn waits and per-thread queues for reentrant writes.
- `4562340`, 34 minutes later, deleted it: "It deadlocked inside CultLib: a
load's delivery reaching a CultNetDatabaseSubscriptionServer observer that
needs _lifecycleGate, while a subscribe's demand handler holding that gate
writes the cache and waits for the delivery's turn."
- It replaced it with **order as data**: a per-cache `Sequence`, and no
delivery scheduler.
- The regression tests it left, `ObserverWritingTheSameCacheDoesNotWaitOnOtherDeliveries`
(`ConditionalCommitTests.cs:763`) and
`SubscriptionDemandHandlerWritingCacheDuringDeliveryDoesNotDeadlock`
(`NetworkingTests.cs:2244`), are live on main.
This pass rebuilt that scheduler as a prototype. It is the per-cache turnstile
with same-thread trampolining at `79f4639`, and it reproduces the deletion:
- Caching tests: 278 of 279 pass, and `ObserverWritingTheSameCacheDoesNotWaitOnOtherDeliveries`
fails.
- The networking suite hangs, and Yggdrasil's idle watchdog killed it at 900 s.
The blame-hang run that names the hung test is recorded below.
The reason is structural, not an implementation slip. Once a writer waits for
another thread's delivery, any lock that an observer takes and a writer holds
becomes a deadlock:
- total cross-thread order;
- a writer delivering before it returns;
- no writer waiting on another thread.
Pick two. The branch's drainer chose order plus no-wait, and lost
before-return (F3 and F4). The turnstile chose order plus before-return, and
lost no-wait (the `bcae483` deadlock).
**So this map keeps order as data (the `4562340` ruling) and moves the one
order-sensitive consumer, the CultNet mutation log, to the place order is
decided: under the cache gate.**
- Every writer keeps delivering its own changes, on its own thread, before it
returns, exactly as main does. F3 is satisfied by not regressing it.
- The log becomes a **journal** written in the admitting hold, in Sequence
order.
- The reorder buffer and drainer die, and no scheduler replaces them. This
departs from Soul's stated direction on purpose, and **Q-P1** puts it to the
operator.
## Probe evidence
All of this ran on Yggdrasil via `ygg-verify.sh`, from scratch commits in a
throwaway worktree that no branch holds.
- Probe file: `tests/GameCult.Caching.Tests/PublicationProbeTests.cs` in scratch commit `79f4639`. The worktree has been removed. The commit is unreferenced in the local object store, where gc can collect it; it also survives on the Yggdrasil mirror as `refs/verify/79f4639f8f5b95db7ce16f244a081c1ec101a468`.
- `9c5abf5` is the probes on main. `79f4639` is the same probes plus the
turnstile prototype in `CultCache.cs`.
| # | Probe | main `65311c9` | Turnstile prototype `79f4639` |
|---|---|---|---|
| Q1 | A batch `{ok, boom}` where boom's `[CultName]` getter throws | `TargetInvocationException`. `_sequence` goes from 0 to 1, because ok was minted. **Both ok and boom are in memory**, and `GetByName("ok")` finds ok. **Nothing is published**, and the next write gets Sequence 2. F1 is worse than stated: memory is corrupt and unpublished. In a store-backed cache, `land` (`:2325`) already ran, so the store holds it too. | No stall, but memory is still corrupt. |
| Q2 | Two `Watch` observers, the first throws | With the default R3 handler, the second observer sees 2 of 2 and the writer sees nothing. **With a rethrowing handler, the second observer sees 0 of 2 and every writer gets the exception.** The comment at `CultCache.cs:2374`, "OnNext does not throw", is false under fail-fast. | With its own observer array and a per-observer catch, the second observer sees 2 of 2 and the writer gets the exception afterwards. |
| Q3 | Writer A's observer blocks on `a`, then B writes `b` | The recorder sees `b(2)` before `a(1)`. There is no cross-thread order, by design since `4562340`. | Sees `a` then `b`, because B waited for A. |
| Q4 | An observer on `x` writes `y` | `y(2)` before `x(1)`: nested, depth-first, and the nested write returns after `y` is published. | `x` then `y`, and the nested write returns before `y` is published. |
| Q5 | `OnUpdate` on Upsert and on Commit | 0 fires, so loads only. `Watch` sees 2. | Unchanged. |
| Q6 | 8 threads × 1000 writes, plus reentrant writes | 67 ms, 2645 of 8800 cross-thread inversions, and 0 writes returned undelivered. | 516 ms, 0 inversions, 0 undelivered. |
| Q7 | Soul's P2 shape: a handler waits 1 s for another thread's write | Observed in time. | Not observed in time. |
| Suites | Caching, Networking and Mesh | (green on main) | Caching fails `ObserverWritingTheSameCacheDoesNotWaitOnOtherDeliveries`, and Networking **hangs** until the 900 s idle kill. Rerun with `--blame-hang-timeout 90s`: 286 passed, then the test host hung and was dumped. The blame sequence file was not extracted, so **the hung test is not named**. It is consistent with, but not proven to be, `SubscriptionDemandHandlerWritingCacheDuringDeliveryDoesNotDeadlock`, whose own comment says "disposal would hang too". |
## Authority map
- **Owner.** `CultCache`. The hold that mints a Sequence range also:
1. journals the range under the gate, in Sequence order (new);
2. publishes the range to every observer, isolated, on the minting writer's
thread, after it releases the gate and before it returns (main's model,
now isolated).
Nothing waits on another call's delivery.
- **Inputs.**
- The changes returned by `Apply` (`CultCache.cs:2707`), the single mutation
site.
- The pre-mint projection of names and indexes (Cut 1).
- The journal list, the observer array and `OnUpdate`.
- **Outputs.**
- Journal calls: one per admission, under the gate, in Sequence order.
- `Watch` / `WatchRecord` notifications that carry `Sequence`.
- `OnUpdate` for loads.
- Observer and journal exceptions, rethrown to the writer.
- **Derived state.**
- `Sequence` is order as data.
- CultNet's primary log sequence is minted in the journal, so log order is
cache order.
- A replica's log sequence is copied from the primary entry.
- The database change kind (`Predicted`, `Reconciled`, `Added`, `Updated`,
`Removed`) is decided in the journal.
- `CultNetDatabaseChange.Sequence` (new) is copied from the cache change.
- Wire projections drop stale by it (Cut 6).
- **Forbidden writers.**
- Nothing re-sequences or schedules cache changes. The branch's `_parked`,
`_ready`, `_nextSequence`, `_draining`, `PendingChange` and the drain body
of `OnCacheChange` die, and so do the turnstile, `bcae483` and the
prototype.
- No R3 `Subject` fans out cache changes (Q2).
- `Apply` and `Index` run no user code.
- Doors never read or write prediction state: the branch's `HasPrediction`
and `_predictionGate` die.
- The database's Watch observer never logs; only the journal logs.
- A journal never enters the cache.
- **Shared paths.** Every path goes `Held` → `Admit` → `Resolve` → **Project**
→ `land` → `Apply` → **journal** → leave gate → **publish**:
- loads: `AddBackingStore` `:1909`, `Loaded` `:1940`, `PullAll*` `:1959` and
`:3052`;
- `Commit` / `TryCommit` (`Land`, `:2272`);
- `UpsertAsync`, `UpsertVariantAsync`, `FlattenAsync` and `Remove`
(`:2004-2056`, `Write` `:2252`);
- a variant dependent's re-resolution (`VariantPlan.Dependents`, in the same
`Apply`);
- direct calls on an attached store (`:2924`);
- `Dispose` (`:2142`);
- the database doors, replica apply and snapshot resync, which only write
the cache and hand context to the journal.
The single exception is the replica's absent-key delete, which has no cache
change. It logs directly (§4).
- **Deletion line.**
- In the cache: `Subject<Change> _changes` (`:1809`), the `_changes.OnNext`
call and its false comment (`:2374-2375`), `_changes.Dispose()`
(`:2154`), and the accessor calls in `Index` (`:2798-2804`).
- In the database, on the salvaged branch: the reorder buffer and drainer,
`_predictionGate` and `HasPrediction`, the logging in `Deliver`, and the
`Subject<object> _changes` fan-out.
## The design, question by question
### 1. Validation before minting: a refused change mints nothing
- **Where.** Add **Project** to `Admit` (`:2313`), between `Resolve` (`:2324`)
and `land` (`:2325`).
- **What.** Over `plan.Admitted ∪ plan.Dependents`, it evaluates once per
record: `NameAccessor(document)` and every `IndexAccessors[alias](document)`.
It keeps `(Name, (Alias, Value)[])` per key on the plan.
- **A throwing getter refuses the whole admission.** It raises an
`InvalidOperationException` that names the key, the member and the unwrapped
message; today the caller sees a bare `TargetInvocationException`. The
refusal comes before `land`, so the admission writes no store, touches no
memory and mints nothing.
- **Readers take the projection, not the document.**
- `Index` (`:2793-2806`).
- `RefuseVariantIndexSharing` (`:2659-2660`).
- The variant-name rule in `ResolveOne` (`:2616`). Its ancestor read at
`:2621` reads held documents, pre-land, and may stay.
`Resolve` currently runs the refusal rules before any projection exists.
Hands makes Project the first pass over the resolved plan and moves both
rules after it, so rules read the projection and only the projection reads
documents.
- **Apply becomes user-code-free.** It then does only dictionary and
`ConditionalWeakTable` work on precomputed data.
- **Loads refuse too.** A store whose record's getter throws fails to open,
loudly. That is the rule "hydration failure on open is loud" (composition doc
`:88-92`). Today it half-indexes silently.
### 2. Observer isolation
- **Replace the `Subject<Change>` with the cache's own observer array**,
copy-on-write under a small lock.
- `Watch<T>` (`:1887`) becomes `Observable.Create<Change>(register)` followed
by the existing `Where` and `Select`.
- `Dispose` calls `OnCompleted` on each observer, which is what
`Subject.Dispose` did.
- **Publish loops change → observer, and wraps each `OnNext` in its own
try/catch.**
- R3's `Observer<T>` already catches handler exceptions (`OnErrorResume`, then
the unhandled handler). So an exception reaches the cache only when fail-fast
was asked for: a rethrowing `ObservableSystem` handler, or a custom observer.
- Escaped exceptions join the `OnUpdate` failures. They are rethrown to the
writer after every observer has received every change of the hold. That is
the existing `OnUpdate` policy (composition doc `:77-82`).
- The observer stays subscribed (**Q-P2**). The commit stands either way.
- **Prototype.** At `79f4639` (Q2) this took about 15 lines. Hands keeps that
part and drops the turnstile around it.
### 3. Delivery thread, order and reentrancy
- **Unchanged from main, on purpose.** Each outermost hold publishes exactly its
own changes, on its thread, after it leaves the gate and before it returns
(composition doc `:59-65`, `Held` `:2339`).
- A write from inside a handler is its own outermost hold. It delivers
depth-first before returning, as Q4 shows on main.
- There is no cross-thread order. Every change carries `Sequence`, and
latest-state consumers drop stale changes by it (doc `:68-73`, and the Mesh
mirrors at `CultMesh.cs:1534-1540`).
- **Why this is not a regression of the ruling.** The ruling asks for
exactly-once, not for a total delivery order.
- Exactly-once in the cache comes from the single mint site plus isolation
plus Cut 1 (§1-2).
- Order is needed by exactly one consumer, the shard log. It gets that order
from the journal (§4), not from delivery.
- **The journal is the new mechanism.**
- Signature: `IDisposable CultCache.AddJournal(Action<IReadOnlyList<CultCacheDocumentChange<object>>>)`.
Its name and shape are **Q-P3**.
- It is called in `Admit` right after `Apply` (`:2328-2329`), under the gate,
once per admission, with that admission's changes in Sequence order.
Journals run in registration order.
- A journal that enters the cache, meaning any hold on this cache while a
journal runs, throws `InvalidOperationException`. It is guarded by a flag
checked in `Held`'s `Monitor.IsEntered` branch (`:2341`). This is the
store rule ("a store call must not wait for a write to its own cache"),
enforced.
- Journal exceptions follow the observer policy: the admission stands, it is
still published, and the exception is rethrown to the writer after
publication.
- Adding or removing a journal takes the gate, so registration is atomic with
respect to admissions.
- **Lock order.** Cache gate, then journal-internal leaf locks (the database's
`_logGate`). Observers still run outside the gate, so the `bcae483` cycle
(an observer needing `_lifecycleGate` while a gate holder writes the cache)
cannot form. No journal takes `_lifecycleGate`, and no writer waits on a
delivery.
- **Soul's P2 passes.** A handler waiting on another thread's write sees it,
because nothing orders deliveries (Q7 on main).
### 4. What CultNetDatabase keeps
The database is rebuilt on the journal. It has no queue.
- **Journal (under the cache gate, per admitted change, in Sequence order):**
1. `TakeDoor(change)`, with the shape rule salvaged from the branch
(`:1219-1234`). A put door matches a change whose new document is its
instance and whose previous document is not. A removal door matches a
change whose previous document is its instance and which has no new one.
- The consumption now happens inside the door's own hold, before any
observer runs. So an observer's later write to the door's key or
instance can never wear the context (the `f3343c8` defect), and
unconditional `Dispose`-removal stays correct.
2. Predictions. A `Predicts` door adds the prediction. On an `Authoritative`
door (`ApplyPutAsync`, replica put), if a prediction exists for the key,
the kind is `Reconciled` and the prediction is removed.
- The gate serializes this, so `_predictionGate` and `HasPrediction` are
deleted (F5).
- Soul's P6 cannot happen: the prediction is recorded before
`PutPredictedAsync` returns.
3. Primacy.
- On a shard primary, where the door is not `NoLog` (prediction, snapshot):
build the log entry, including the wire message from a `Put`/`Delete`
door or `ToLogEntryMessage`, and **only then** mint the per-shard
sequence and append it to memory and to the durable store. A
serialization failure then mints nothing, so the log can never hold a
gap.
- On a replica, record the primary's entry from the `Replica` door,
under its own sequence.
- For a local write on a replica, log nothing.
4. Stash `(cache Sequence → CultNetDatabaseChange)` under `_logGate`, for the
publisher.
- **Watch observer (after the gate, on the writer's thread, before it
returns):** it removes the stash entry for `change.Sequence` and publishes it.
- A change with no stash was admitted before the database existed and is
ignored.
- Register the observer first and the journal second, in the constructor, so
a stash can never be orphaned.
- **Log-before-publish** is structural: the journal runs before the gate opens.
- **The replica's absent-key delete (F6)** logs directly under `_logGate`. It
is in order because every earlier entry of the same pull loop was journaled
inside its own hold, before that `UpsertAsync`/`Remove` returned. This holds
even when a pull runs inside a handler, since nested holds journal
synchronously.
- **The in-memory log is keyed and ordered by sequence.** It becomes a
`SortedList<long, entry>` per shard, to match the durable store
(`CultNetShardMutationLogStorage.cs:55-64, 75-89`: that store replaces by
sequence and orders by sequence).
- **Kept from the branch:**
- the door table keyed by instance identity, and the shape rule;
- "only a primary mints; replicas publish";
- log-before-publish;
- the snapshot resync "logs nothing" rule. Its removal door (branch `:733`)
must set `Removal = true`, or the shape rule never matches it.
- **Database fan-out isolation.** The database's `Subject<object> _changes`
(main `:487`) has the Q2 defect one layer up: the subscription server's
per-peer handlers sit on it. Replace it with the same isolated
observer-array pattern as §2 (about 20 lines). Its exceptions reach the
writer through the cache's own per-observer catch.
- **`CultNetDatabaseChange<T>` gains `long Sequence`**, the cache change's
Sequence, as an optional constructor argument, exactly as
`CultCacheDocumentChange` did (composition doc `:57-58`). Without it,
in-process latest-state consumers cannot drop stale changes.
- **Gone:** F3 (no deferred delivery), F4 (no drainer), F7 (no queue to
strand) and F8 (no guard).
### 5. Backpressure and memory bounds
- **No buffer exists.** A writer delivers its own changes, and they are held
only in its `mine` list for the length of its own call.
- **The journal stash** holds at most the changes of holds between their
journal and their publication. That is one hold per writing thread, and each
entry is removed by the same thread that added it.
- **A slow observer slows only the writer whose change it is handling.** That
is main's behaviour. It does not slow other writers, and it does not grow
memory. This replaces the branch's unbounded drainer backlog (F4, Soul P5).
- **The journal runs under the gate, so a slow journal slows every writer.**
- The database journal does in-memory work plus one durable
`mutationLogStore.Append` per logged change. `Append` rewrites the shard
file (`CultNetShardMutationLogStorage.cs:83-89`).
- That is I/O under the gate, which the locking doctrine already allows for
stores ("A store doing I/O on behalf of its cache ... holds the gate",
composition doc `:98-101`). It is still a new cost, so Cut 4's
verification measures commit latency with a file log store.
- The whole-file rewrite is a pre-existing monolith. It is recorded as a
follow-up, not fixed here.
### 6. Cross-runtime impact
The Eyes survey was run at `65311c9`.
- **`cultcache-rs`, `cultcache-ts` and `cultcache-py` have no publication
surface:** no watch, no listener and no Sequence.
- Rust: `lib.rs:1970-1975, 2213-2420`.
- TypeScript: `cult-cache.ts:164-181`, "Observer delivery is not scheduled
here".
- Python: `cache.py:30-45`, which refuses reentrant mutation.
- **`cultnet-rs`, `cultnet-ts` and `cultnet-py` have no CultNetDatabase
equivalent and no cache watcher.** The only server-side shard log is the
Python interop test peer (`interop_peer.py:565, 583`), which mints
`len(log)+1` under its own lock, inline with the write. That is already the
journal's shape.
- **Wire parity is unaffected.**
- Cache `Sequence` never goes on the wire (composition doc `:47-49`).
- `CultNetShardLogEntryMessage` does not change.
- `CultNetDatabaseChangeRawMessage` (`CultNetSchemaMessages.cs:1016`) carries
no sequence, and this map adds none: stale protection for wire
subscriptions is server-side (Cut 6).
- No vector in `tests/vectors/` encodes publication order.
- **No other runtime must change.**
- **Follow-up.** `cultmesh-py`'s `CultMeshDatabase` watch (`node.py:421-505,
1059-1065`) has F2 (a throwing callback skips the rest and reaches the
writer), and it has no Sequence. When Python grows cache publication, it
takes this contract.
## Cuts
Every cut is on CultLib, branch `hands/cultcache-publication` from `main`
`65311c9`, in a worktree Self creates. Cuts 1-3 touch only the cache and land
first. Cuts 4-6 depend on Cut 3. Heavy verification runs on Yggdrasil.
**The negative checks that stay green in every cut** are the `4562340` tests:
- `ObserverWritingTheSameCacheDoesNotWaitOnOtherDeliveries`;
- `SubscriptionDemandHandlerWritingCacheDuringDeliveryDoesNotDeadlock`;
- `SequenceIncreasesInAdmissionOrder`;
- `WatchRecordIgnoresStaleSequence`.
They are the proof that no scheduler has come back.
### Cut 1. Project before mint (cache, behaviour)
- **First.** Commit Q1 as a failing test named for the rule.
- **Deletes first.**
- Accessor calls in `Index` (`:2798-2804`).
- Direct accessor reads in `RefuseVariantIndexSharing` (`:2659-2660`) and
`ResolveOne` (`:2616`).
- **Adds.**
- `Project` in `Admit` (`:2324-2325`).
- A per-key projection on `VariantPlan` (`:2456`).
- The early returns in `Resolve` (`:2477-2478`, `:2530-2534`) still pass
through `Project`, because it is called from `Admit`.
- **Authority.** Project owns "this admission's name and index values", and
the rules and `Index` read them. `Apply` runs no user code.
- **Verification.**
- Tests:
- **A refused getter mints nothing and changes nothing.** `GetWithSequence`
is unchanged, both keys are absent, the name lookup misses, nothing is
published, and the next write is previous+1.
- **Store-backed batch:** the store file is byte-identical, and a reopen
holds neither key.
- **A load whose getter throws** refuses the open and leaves the file
byte-identical.
- **Once per landed record:** a counting getter runs exactly once per
landed record, including a variant dependent re-resolved by a base edit.
- **The refusal names** the key and the member.
- Soul P4.
- Negative grep: `rg -n "NameAccessor\?\.Invoke|IndexAccessors" src/GameCult.Caching/CultCache.cs`
has no hit in `Apply`, `Index` or `Unindex`. `:315` is store-side and
legitimate.
- Stryker.NET on the diff.
- **Ledger.** About −15 and +45.
### Cut 2. Observer isolation (cache, behaviour)
- **Deletes first.**
- `Subject<Change> _changes` (`:1809`).
- `_changes.OnNext` and the false comment (`:2374-2375`).
- `_changes.Dispose()` (`:2154`).
- **Adds.**
- The observer array and `Observable.Create` in `Watch<T>` (`:1887-1897`).
- The per-observer catch in `Publish` (`:2369`), with escapes joining
`failures`.
- `OnCompleted` on `Dispose`.
- **Docs.** Composition doc `:77-82`: escaped observer exceptions follow the
`OnUpdate` policy.
- **Verification.**
- Tests:
- **Q2 with a rethrowing handler:** the second observer receives every
change. The writer gets the escaped exception after delivery, and the
thrower is still subscribed (per Q-P2).
- **Subscription order:** subscribing and unsubscribing during a delivery
leaves every other observer's stream complete.
- **Dispose** completes observers.
- Soul P3.
- The four `4562340` tests stay green.
- Stryker on the diff, especially removing the catch and breaking out of
the loop on the first failure.
- **Ledger.** About −5 and +30.
### Cut 3. The journal (cache, behaviour)
- **Adds.**
- `AddJournal` (Q-P3), with registration under the gate.
- The invocation in `Admit` after `Apply` (`:2328`).
- The no-reentry guard in `Held` (`:2341`).
- Exceptions routed like observer failures.
- **Docs.** A composition-doc paragraph in "Store composition" and in
"Locking": journals run under the gate, in Sequence order, and must not
enter the cache or wait for anything that needs it.
- **Verification.**
- Tests:
- **Order:** 8 threads × 1000 writes. The journal sees contiguous,
strictly increasing Sequences, and each call's changes arrive together.
- **Timing:** the journal runs before any observer of the same change, and
after `Apply` (`Get` inside the journal sees the new document).
- **Reentry:** a journal that writes the cache throws, and the admission
stands.
- **Refusal:** a refused admission journals nothing.
- **Dependents:** a variant dependent is journaled in the same call as its
base edit.
- **Failure:** a throwing journal still lets the change publish, and the
writer gets the exception.
- The `4562340` tests stay green.
- Stryker on the diff.
- **Ledger.** About +50.
### Cut 4. Salvage the branch and rebuild CultNetDatabase on the journal (networking)
- **4a (history).** `git merge origin/hands/cultnet-publish-commits` into the
cut branch. It is clean. The merge carries the tests; the queue dies in 4b.
Self may choose a squash-salvage instead, with the same content.
- **4b (subtraction, then behaviour).**
- **Deletes first**, at branch `28248eb` `CultNetDatabase.cs`:
- `_parked`, `_ready`, `_nextSequence` and `_draining` (`:502-505`);
- the handshake at `:530`;
- `OnCacheChange`'s queue body (`:1083-1135`);
- `PendingChange` (`:1265-1275`);
- `_predictionGate` and `HasPrediction` (`:506-507`, `:1209-1213`) and
their call sites (`:916`, `:1574`);
- the logging in `Deliver` (`:1168-1201`).
- **Moves.** Logging, predictions and door consumption move into the journal
(§4). `Deliver` becomes the publisher that reads the stash.
- **Changes.**
- Door flags: `Predicts` and `Authoritative`.
- The snapshot removal door sets `Removal`.
- The in-memory log becomes a `SortedList`.
- **Keeps.** Door identity and the shape rule, primacy, `_logGate` for
readers, and the absent-key delete's direct log.
- **Verification.**
- Every salvaged test passes:
- `CultNetPublishesCommitsTests` (13);
- `CultNetPublicationOrderTests`: S1, S1b, S2, S3, S4, S6, S7,
log-before-publish, and in-place re-upsert.
- Soul's P1-P7 from `SoulPub3Tests.cs` are committed, all passing. P2 passes
because nothing orders delivery.
- New tests:
- a reentrant `database.PutAsync` from inside a database handler is logged
with its context and published once;
- a reentrant `ApplyShardLogResponseAsync` from inside a handler keeps the
replica log in primary order;
- a primary serialization failure mints no log sequence.
- Commit latency with `CultNetFileShardMutationLogStore`, main against Cut 4.
It is reported, not gated.
- Negative grep: `rg -n "_parked|_ready|_draining|_nextSequence|_predictionGate|HasPrediction" src/GameCult.Networking`
returns nothing.
- The `4562340` tests stay green.
- Stryker on the diff.
- **Ledger.** Against the branch, about −120 and +60.
### Cut 5. Database fan-out isolation and Sequence (networking, behaviour)
- **Deletes first.** The database's `Subject<object> _changes` (main `:487`).
- **Adds.**
- The isolated observer array.
- `CultNetDatabaseChange<T>.Sequence` (main `:319-371`), set by the journal.
- **Verification.**
- With a rethrowing handler and one throwing database subscriber, the other
subscriber and a live subscription-server peer receive every change.
- The disposed-database test.
- `Sequence` equals the cache change's.
- **Ledger.** About −10 and +30.
### Cut 6. Stale protection on wire projections (networking, behaviour)
This is a separate cut that closes a pre-existing hole. Composition doc
`:50-53` says the CultNet streams "have no ordering guarantee and no stale
protection", and the branch's total order was reaching for this.
- **The rule.** A projection sends a record only if the change's `Sequence` is
above the Sequence it last applied for that key.
- The subscription server applies it under `_lifecycleGate`, keyed in
`SubscriptionProjectionState` next to `DeliveredBySourceRecordKey`
(`CultNetDatabaseSubscriptionServer.cs:446`).
- `CultNetDatabaseServer.CreateSubscription` (`:501-512`) applies it under a
per-subscription lock.
- `Reconcile` (`:347`) stamps each key it reads with a per-key atomic
`GetWithSequence`.
- **Verification.** Two threads write one key with a blocking observer that
forces the inverted delivery (Q3 shape). The peer's last state for the key
equals the cache's, on both servers.
- **Operator.** Q-P4 asks whether this cut belongs here or is recorded.
- **Ledger.** About +40.
### Salvage from `hands/cultnet-publish-commits`
- **Keep:**
- both new test files;
- the `NetworkingTests.cs` edits (`:1528`, `:2257`);
- the door-shape rule;
- "replicas never mint";
- log-before-publish;
- "the snapshot logs nothing";
- the rewritten composition-doc paragraph, rewritten again by Cut 3 to name
the journal as the log's order source.
- **Commit** Soul pass 3's P1-P7: P3 in Cut 2, P4 in Cut 1, and the rest in
Cut 4.
- **Drop:** the reorder buffer, the drainer, `_predictionGate`, and logging
from the observer.
## Subtraction ledger (estimate)
| Cut | Removed | Added | Other |
|---|---|---|---|
| 1 | ~15 | ~45 | none |
| 2 | ~5 | ~30 | the cache no longer uses R3 `Subject` |
| 3 | 0 | ~50 | one public method (Q-P3) |
| 4 | ~120 vs branch | ~60 | the merge brings +1109/−243, of which about 780 are tests |
| 5 | ~10 | ~30 | the database `Subject` goes, and one public property |
| 6 | 0 | ~40 | none |
- **Source, net against main:** about +200.
- **Source against the branch:** about −60, and no scheduler anywhere.
- The budget is pressure, not a metric.
## Operator questions
- **Q-P1. Order.**
- **A. Order as data** (the `4562340` ruling, kept).
- Every change is published exactly once, on its writer's thread, before
the write returns, and carries `Sequence`.
- There is no cross-thread delivery order.
- The shard log is in cache order because the journal writes it under the
gate.
- Wire projections drop stale changes by Sequence (Cut 6).
- **B. Totally ordered delivery in the cache** (a turnstile, which is Soul's
stated direction). It was built as `bcae483`, deleted for deadlock as
`4562340`, and reproduced by this pass: the networking suite hangs.
Making it safe needs a rule that nobody holding a lock an observer takes
may write the cache, and the subscription server itself breaks that rule.
- **C. A dedicated delivery thread or drainer** (the branch). It was refuted
by Soul pass 3: F3 and F4.
- **Recommended: A.** Only A delivers exactly-once, before-return and
no-deadlock together. The one consumer that needs order gets it where order
is decided.
- **Q-P2. An observer whose exception escapes** (a fail-fast R3 handler, or a
custom observer).
- **A.** Rethrow it to the writer after every observer has run, and keep the
observer subscribed.
- **B.** Report it and unsubscribe.
- **C.** Route it to R3's unhandled handler only.
- **Recommended: A.**
- It matches the `OnUpdate` policy and R3's `OnErrorResume`.
- An escape only happens when the owner asked for fail-fast.
- B turns one bad change into permanent deafness.
- C sends the exception back to the handler that just rethrew it.
- The commit stands under every option.
- **Q-P3. The journal's surface.**
- **A.** A public `IDisposable CultCache.AddJournal(Action<IReadOnlyList<CultCacheDocumentChange<object>>>)`,
guarded so that entering the cache from it throws.
- **B.** An internal method, with `InternalsVisibleTo("GameCult.Networking")`.
The comment at `CultCache.cs:214` shows Networking was deliberately kept
without it.
- **Recommended: A.**
- The contract is small, and the dangerous misuse, reentry, is enforced
rather than documented.
- An ordered admission journal is a legitimate library capability: audit,
replication and derived indexes.
- B couples two assemblies to hide one method.
- **Q-P4. Cut 6 now or later?**
- **A.** Land it in this campaign.
- **B.** Record it.
- **Recommended: A.** With every commit now published, two writers of one key
on two threads can leave a peer holding the older state. That was already
true for loads; commits make it common.
Not forks, recorded so they are not re-asked:
- A throwing getter refuses loads too.
- Reentrant writes keep main's depth-first delivery. Order is data.
- Journal I/O under the gate follows the store precedent.
## Follow-ups outside this campaign
- **`cultmesh-py` watch.** It has no isolation and no Sequence
(`node.py:1059-1065`).
- **`RecordMutationLogEntry` memory-before-disk.** A throwing durable `Append`
leaves memory ahead of disk. This predates the campaign.
- **Durable log whole-file rewrite.** `CultNetShardMutationLogStorage.cs:83-89`
rewrites the whole file per `Append`. It is a monolith, and under the gate
its cost is now every writer's.
- **Mesh database-backed handles.** `CultMesh.cs:2362` and `:3396` can drop
stale changes once `CultNetDatabaseChange.Sequence` exists.
- **Skill scar.** The Eureka SKILL.md already says "a ticket system that
serialized delivery was built, deadlocked, and was deleted". The Soul pass 3
direction re-proposed it anyway. The Soul brief template should tell Soul to
check `git log -S` for a proposed mechanism before recommending it.

774
docs/cultnet-rudp-ack-cut.md Normal file
View file

@ -0,0 +1,774 @@
# CultNet RUDP: Acknowledged means delivered, and the receiver owns ordered delivery with one watermark
Status: cut map, Imagination pass 1, 2026-09-30, committed by Self.
**Rulings (operator, 2026-09-30).** These supersede the two-state proposal in the text below. Cuts 4 and 5
need re-mapping to them before Hands.
- **Q-A1 (a):** Delivered is derived at the sender from a correct receiver's cumulative watermark. The wire
does not change.
- **Q-A2:** "I think this is telling us we don't actually need both received and delivered."
- There is one public meaning: Acknowledged means delivered.
- Receipts go Pending, then Acknowledged (delivered), or Invalidated. No public Received state exists.
- Retransmission still stops on each packet's own ack, so resending stays bounded.
- Flush waits for Acknowledged.
- **Open:** Q-A3 (expiring reliable traffic next to ordered traffic), Q-A4 (porting receipts), Q-A5 (Kotlin).
- **Added to Cut 1's scope** (from stray-packet Soul pass 3, pre-existing): a restarted client on the same
endpoint and payload is locked out by repeated-Connect answers. A reconnect on the same transport treats a new
server session's frames as duplicates. Session generation belongs to the sequence space this cut owns.
- **Also in Cut 1's scope** (stray-packet Soul pass 4): the ending paths disagree about what a session owed.
- In Rust, a peer's Disconnect invalidates receipts.
- A local `disconnect()` or a timeout carries owed writes into the next session. There, a receipt from the old
session can become Acknowledged (probe R1).
- TS, Python and C# also carry owed writes across a peer Disconnect.
- One rule: a write belongs to the session it was issued in.
- **Body.** `origin/hands/rudp-stray-packets` at `d85877a7` ("the session no
longer keeps an initial sequence nothing reads"). Every `file:line` is against
it unless it names another revision.
- `main` (`ae8576d8`) has not touched any RUDP file since the branch forked at
`60663f95`. `git diff --stat` over the four transport files and
`packages/cultmesh-kotlin` is empty.
- Hands is still working on that branch. The line numbers are good until it
moves again; the anchors (function names) are the durable reference.
- **Base of every cut.** `main` after `hands/rudp-stray-packets` merges. This
campaign does not start before that merge, and it does not rebase that branch.
- **Scope.** The RUDP session and socket transports in all five runtimes: C#
`src/GameCult.Networking/CultNetTransport.cs`, Rust
`packages/cultnet-rs/src/rudp.rs` (plus `packages/cultmesh-rs`), TypeScript
`packages/cultnet-ts/src/rudp.ts`, Python
`packages/cultnet-py/src/cultnet_py/transport.py`, and Kotlin
`packages/cultmesh-kotlin/.../CultMesh.kt`. Kotlin was not in the brief. It is
a fifth RUDP implementation with the same ordered-delivery code
(`CultMesh.kt:2939-2978`) and the same codec. **Q-A5** decides whether it is in.
- **Wire.** This map changes no byte on the wire (§1).
## The ends (operator, 2026-09-30)
- A frame the peer has buffered but not delivered must not look delivered.
Today an ordered frame that arrives behind a gap is held and acknowledged; if
the session ends it is lost, and the sender saw "Acknowledged".
- Resending stays bounded. The sender stops retransmitting when the peer has
**received** the frame, exactly as today. Delivery-only acknowledgement was
rejected: "I don't like the possibility of unbounded resending on an
unresponsive peer application. TCP sounds like it has the right idea."
- Two meanings: **Received** (the frame is in the peer's session) and
**Delivered** (handed to the peer's application). Unordered and unreliable
channels deliver on receipt. Receipts go Pending, then Received, then
Delivered, plus the existing Invalidated.
- A session that ends with frames held leaves their receipts at Received,
visibly not Delivered.
## The finding that shapes this map
Self's proposal puts a second acknowledgement on the wire: a per-channel
"delivered through N" watermark beside the received ack. Probing the receivers
changed the picture in two ways.
**1. The ordered receivers are broken in every runtime, and the break is
bigger than the session-end case.** A held frame is not only lost at session
end. In a live session it can be stranded forever, or dropped, while the sender
sees it acknowledged. The per-channel `next` pointer
(`CultNetTransport.cs:918`, `rudp.rs:194`, `rudp.ts:160`, `transport.py:424`,
`CultMesh.kt:2643`) has been patched twice (`436654de` added the
first-use initialisation, `4335ab10` added the skip over other channels'
sequences) and is still wrong:
- **P1: a gap filled by another channel strands the ordered channel.** Ordered
`o1`, reliable `u` on another channel (lost), `o2`, `o3`. When `u` arrives,
only `u`'s channel is drained. `o2` and `o3` are never delivered, and the
sender's pending set is empty. The same holds with two ordered channels
(P1b).
- **P6: a channel first used after other traffic loses frames.** `C1`, `C2` on
a new channel, then `X` on another. `X` arrives, then `C2`, then `C1`. The
channel initialises its `next` from `min(highest+1, frame)`, so `C2` is
delivered first and `C1` is **dropped as old** (`frame.sequence < next`). It
was acknowledged.
- Both reproduce at the branch tip, at Odin's pin `a8aeddac` and at Muninn's
pin `c2a9a6e5`, in every runtime probed.
**2. Once the receiver is correct, "Delivered" is not new information.** A
conforming receiver holds an ordered frame for exactly one reason: some
reliable sequence below it has not arrived. When the last missing sequence
arrives, the same `Receive` call drains the held frames, before the
acknowledgement is built. So:
> An ordered frame is Delivered exactly when every reliable sequence up to its
> last one has been Received.
The sender already knows that. It is the sender's own **acknowledged-through**
watermark: one below the lowest sequence that is still pending or queued.
TCP's cumulative ACK means the same thing, "every byte below N is in the
receive buffer", and it sits beside SACK, which means Received. Here the
sender can compute the cumulative value itself, because every sequence is
acknowledged individually and removed from its pending set.
A delivered watermark on the wire would be a second writer of a value the
first one already determines. Old receivers would never send it, so the sender
would need the derived rule as a fallback anyway. **This map derives Delivered
at the sender and changes no wire byte.** The explicit watermark was probed
and is available as an additive option, but it is not recommended. **Q-A1**
puts it to the operator, because it departs from Self's proposal.
The derivation is sound only if nothing the receiver acknowledged is later
dropped while the session lives. The probes found four places where it is:
| # | Where an acknowledged frame is lost while the session lives | Runtimes | Fixed by |
|---|---|---|---|
| P1, P1b | A cross-channel gap strands the ordered channel | all five | Cut 1 |
| P6 | Late channel initialisation drops the earlier frame | all five | Cut 1 |
| P5 | A reliable fragment set is evicted after its fragments were acknowledged | Rust, TS, Python (C# and Kotlin have no eviction) | Cut 2 |
| P8 | A retransmit more than 4,096 sequences behind the receiver's highest counts as a duplicate: acknowledged and dropped | all five | Cut 3 (the sender never lets its span reach it) |
And one place where the sender loses track without the receiver's help:
| # | Sender-side lie | Runtime | Fixed by |
|---|---|---|---|
| P2 | An expired reliable packet leaves a permanent hole. Ordered frames after it are held forever, and `flush_reliable` succeeds (outstanding 0) | Rust (the only runtime with reliable expiry) | Cut 4 reports it honestly; **Q-A3** decides the real fix |
| P9 | A receipt for an expiring send reads `Acknowledged` once the packet expires unacknowledged | Rust | Cut 4 |
| P10 | `send_reliable` with a full window queues the frame, then returns an error ("receipts require a non-empty reliable packet set"), so a caller that retries sends it twice | Rust | Cut 4 |
## Probe evidence
Everything ran on Yggdrasil through `ygg-verify.sh`, from local-only probe
branches in the throwaway worktrees `F:\Projects\CultLib-imag-ack` and
`-imag-ack-old`. Each probe commit also survives on the Yggdrasil mirror as
`refs/verify/<sha>`.
- `de2bdfe8`, `09318d13`, `03eac6ff`: Rust session and socket probes on the
branch at `ca18dfac` (`packages/cultnet-rs/tests/probe_ack.rs`).
- `b70a69ca`: the C#, TS and Python probes (`ProbeAckTests.cs`,
`probe-ack.test.ts`, `probe_ack.py`) on `ca18dfac`.
- `2320a4e5`: the C# probes rebased on `d85877a7`.
- At `ca18dfac`, `GameCult.Networking` did not compile (`CultNetTransport.cs(2707)`: `ReplacedPeerReason` not in scope). `afd52e97` on the branch fixes it.
- `7fe5d726`: P5 and P8 in C#, TS and Python.
- `f0e7136c`: the Rust probes at Odin's pin `a8aeddac`.
- `bf8fc826`: all four runtimes' probes at Muninn's pin `c2a9a6e5`.
- `db7d2886`: **the prototype**, the Rust receiver rebuilt on one watermark
plus sender-derived receipt states (about +130/−125 in `rudp.rs`).
| # | Probe | Branch tip (all runtimes probed) | Pins `a8aeddac` / `c2a9a6e5` | Prototype `db7d2886` |
|---|---|---|---|---|
| P1 | `o1`, reliable `u` lost, `o2`, `o3`; then `u`, then `o3` | Delivered `[o1, u]`. `o2`, `o3` never. Sender pending `[]`. Rust, C#, TS, Python. | Same in Rust (both pins), C#, TS, Python (`c2a9a6e5`) | `[o1, u, o2, o3]` |
| P1b | `A1`, `B1` (second ordered channel) lost, `A2`, `A3` | `[A1, B1]`, all four | Same | `[A1, B1, A2, A3]` |
| P6 | `C1`, `C2` on a new channel, then `X`; arrival `X`, `C2`, `C1` | `[C2]`. `C1` dropped. All four. | Same | `[C1, C2, X]` |
| P2 | Rust: reliable media with expiry 1 ms, lost; then `s1`, `s2` on `schema` | Nothing delivered. Sender outstanding 0, expired 1, no resends. `flush_reliable` would return Ok. | Same | Unchanged (held behind the hole). Cut 4 makes the sender say so. |
| P3 | `s1` lost, `s2` arrives and is acknowledged | `s2` not delivered, but acknowledged: sender pending `[s1]` only. A reset then loses `s2`. | Same | Receipt for `s2`: `Received`; after `s1` arrives, 2 frames delivered and `Delivered` |
| P4 | Rust: `s0` delivered, `s1` lost, then a 5 MiB ordered frame in 60 kB fragments, pumped with acks | `RUDP ordered hold buffer is full`: the session is refused. After the gap it would have delivered. With no gap, delivered. | (probe predates the fix; not evidence) | The prototype held every ordered frame, so a 5 MiB frame was refused **even with no gap**: hold only what is not yet deliverable (Cut 1) |
| P5 | 3-fragment reliable frame, 2 fragments arrive and are acked; 70 lossy unreliable fragment sets; last fragment arrives | Rust, TS, Python: reliable frame never delivered, sender pending `[]` | Same (Rust) | Unchanged (Cut 2) |
| P7 | An Ack with flag bit `0x10`, a nonzero sequence field, or a 5-byte payload | Decoded and applied (ack cleared the pending sequence) in Rust, C#, TS, Python | Same at both pins | n/a |
| P8 | Reliable `g` lost while 4,200+ later sequences are sent and acked; then `g` arrives | `g` not delivered, and acknowledged: the sender stops. Rust, C#, TS, Python. | n/a | Unchanged (Cut 3) |
| P9 | Rust socket: `send_reliable("media")` with expiry 1 ms, never read; a ping triggers the purge | `Pending`, then `Acknowledged` | n/a | `Pending` (the abandoned sequence is never acknowledged) |
| P10 | Rust socket: 40 `schema` sends, then `send_reliable` | Error "receipts require a non-empty reliable packet set", after the frame was queued | n/a | Not addressed by the prototype (Cut 4) |
**Suites on the prototype `db7d2886`** (baseline `03eac6ff`, the probe commit
without the prototype, was all green):
- `cultnet-rs`: 110 + 4 + 24 pass; `tests/cultnet.rs` 51 pass and 2 fail.
Both failures are test shape, not mechanism:
- `rudp_session_suppresses_duplicates_and_delivers_reliable_ordered_payloads_in_sequence`
(`tests/cultnet.rs:1000`) hands the receiver an Accept with sequence 91 from a
sender whose data starts at 1. That handshake is impossible. Under a
watermark seeded by the Accept, every data sequence is "already below" it.
The test must use a consistent handshake.
- `rudp_ordered_channel_tracking_is_bounded` (`:2655`) sends 256 ordered
channels with no gap. The prototype keeps state only for channels with held
frames, so nothing is refused. The bound becomes "channels with held
frames", and the test must hold frames behind a gap.
- `cultmesh-rs` did not compile: `lib.rs:892` matches the status enum
exhaustively and does not cover `Received`. Cut 4 rewrites that call site.
- The probe suite: P1, P1b, P6 and P3b are fixed as the table shows.
**Wire compatibility (P7).** Every decoder ignores flag bits 4-7 and byte 35.
Every session ignores an Ack's sequence field and payload
(`CultNetTransport.cs:1177`, `rudp.rs:572-586`, `rudp.ts:397`,
`transport.py:612`, `CultMesh.kt:2738`). Every decoder rejects an unknown
packet type (`CultNetTransport.cs` `Enum.IsDefined`, `rudp.rs`
`packet_type_from_code`, `rudp.ts:1290`, `transport.py:1212`,
`CultMesh.kt` `fromCode`) and an unknown version byte. So:
- An additive watermark fits in an Ack: flag bit 4 set, the watermark in the
Ack's unused sequence field, zero extra bytes. Old peers apply the ack and
ignore the rest, proven at both pins.
- A new packet type or a version bump is not additive: old peers refuse the
datagram, and since the stray-packet branch they count it as dropped.
- No capability negotiation is needed under this map, because nothing on the
wire changes.
## Authority map
- **Owner of ordered delivery (receiver): the session's contiguous
received-through watermark `R`.**
- Every reliable sequence up to `R` has been received.
- A held ordered frame whose first sequence is at most `R + 1` is delivered,
in sequence order, in the same call that advanced `R`.
- `R` is seeded by the handshake. `AcceptConnect` seeds it with the Connect's
sequence, and receiving the Accept seeds it with the Accept's sequence. A
session connected without a handshake (Rust `assume_connected`,
`rudp.rs:241`, test-only today) seeds it from its first reliable packet.
- It advances in `RememberReceived` and resets in `ResetPeerState`.
- **Owner of Received (sender): the pending and queued sets**, as today. A
sequence leaves them when an ack names it. Retransmission stops there.
- **Owner of Delivered (sender): the acknowledged-through watermark `A`**, one
below the lowest sequence that is pending, queued or abandoned. An ordered
send is Delivered when `A` reaches its last sequence. A reliable unordered
send is Delivered when it is Received.
- **Inputs.** Received sequences (receiver); acks and expiry (sender). Nothing
else.
- **Outputs.** Delivered frames, in order (receiver). Receipt status and flush
completion (sender).
- **Derived, not owners.**
- The per-channel `next` map is **deleted**, not demoted:
`_orderedNextSequenceByChannel` (`CultNetTransport.cs:918`),
`ordered_next_sequence_by_channel` (`rudp.rs:194`),
`#orderedNextSequenceByChannel` (`rudp.ts:160`),
`_ordered_next_sequence_by_channel` (`transport.py:424`),
`orderedNextSequenceByChannel` (`CultMesh.kt:2643`).
- `expectedSequenceIfUninitialized` and its equivalents are deleted
(`CultNetTransport.cs:1151`, `rudp.rs:534`, `rudp.ts:375`,
`transport.py:595`, `CultMesh.kt:2728`).
- `PendingOrderedFrame.NextSequence` is deleted.
- `OutstandingReliablePacketCount` stays, and means "not yet Received".
- **Forbidden writers.** No path delivers an ordered frame except the drain
after `R` advances. No receive path drops a frame whose sequence it
acknowledged while the session lives. No sender path reports Delivered
except through `A`.
- **Shared paths.** Accept, reliable data, fragments, duplicates and
retransmits all advance `R` through `RememberReceived` and then call the
drain. Direct sends, fragmented sends, queued promotion and resends all feed
the same pending and queued sets that `A` reads.
- **Deletion line.** `DeliverOrdered`, `DrainOrdered` and
`SkipReceivedNonChannelSequences` and their equivalents go before the drain
is added: `CultNetTransport.cs:1628-1700`, `rudp.rs:1126-1234`,
`rudp.ts:737-800`, `transport.py:868-920`, `CultMesh.kt:2939-2978`.
## The design, question by question
### 1. Where Delivered comes from
- Derived at the sender from `A` (§ The finding). No wire change.
- **Why it is sound after Cuts 1-3.** A conforming receiver:
- drops nothing it acknowledged while the session lives (P1, P6: Cut 1; P5:
Cut 2; P8: made unreachable by Cut 3);
- holds an ordered frame only while a sequence below it is missing (Cut 1);
- drains held frames in the call that fills the last gap, before it
acknowledges (Cut 1).
- So every sequence up to `s` acknowledged means `s` was delivered.
- **Mixed versions.** A new sender with an old receiver can be wrong only when
that receiver strands a frame (P1, P6). That needs reliable traffic on more
than one channel of one session. The C#, TS, Python and Kotlin socket
transports send reliably only on `schema` (`CultNetTransport.cs:2372-2376`,
`rudp.ts:1233-1241`), so they cannot trigger it. Rust sends reliable
`media`/`audio` too (`rudp.rs` `channel_send_options`). The Eyes survey found
no live session that mixes those with `schema`: Muninn's media session
carries no `schema` traffic (`Muninn crates/muninn-daemon/src/main.rs:3099-3158`),
and neither does Sleipnir's HID stream (`Odin crates/sleipnir-daemon/src/main.rs:2217-2272`).
The old receiver loses such frames today in any case. Bumping the pin is the
fix.
- **Rejected: a wire watermark (Q-A1 B).** It is additive (P7), but it is a
second authority for a value the first one determines. Old receivers never
send it, so the derived rule is needed as a fallback anyway.
### 2. What "delivered to the application" means
- **Delivered means the frame left the session's hold and was handed to the
transport's application boundary.** That boundary is:
- C#: enqueued in the socket transport's `_deliveredFrames` (read by
`TryReceiveOnce`), or returned in `CultNetRudpReceiveResult.Delivered` from
a bare session;
- Rust: pushed to `delivered_frames` (`rudp.rs:2137`) or the hub's
`pending_events`, or handed to the document server's sink
(`rudp_document_server.rs:343-360`), which acknowledges only after the sink
accepted;
- TS: the `"frame"` event was emitted (`rudp.ts:1055-1061`);
- Python and Kotlin: the delivered queue.
- **It is not "consumed".** The operator rejected application-driven
acknowledgement, and TCP's cumulative ACK has the same boundary: in the
receive buffer, readable, not read.
- A listener or sink that throws or rejects has still been delivered to. The
handler-rejection paths from the stray-packet branch end the session, which
is the application's answer, not the transport's.
### 3. The public API
- **Rust receipts** (the only runtime that has them):
- `CultNetRudpReliableSendStatus` becomes `Pending`, `Received`, `Delivered`,
`Invalidated` (`rudp.rs:144-148`). `Acknowledged` is removed, not aliased.
Every caller must choose, and every caller today means Delivered.
- The receipt records whether its send was ordered (`rudp.rs:132-135`).
- After the session ends without a reset: `Pending` becomes `Invalidated`
(as today, `rudp.rs:1919-1928`). `Received` stays `Received`, which the
ruling asks for. `Delivered` stays.
- After a reset (`reset_peer_state`: a new Connect generation, or
`end_refused_session`), the session keeps one frozen snapshot of the ended
epoch: its scope, `A`, and its still-unacknowledged sequences, at most the
reliable window plus the queue. A receipt from that epoch resolves against
the snapshot, so `Received` stays `Received` and `Delivered` stays
`Delivered`. Older epochs are `Invalidated`.
- Today every receipt, delivered or not, reads `Invalidated` after a
reset, because the scope rotates (`rudp.rs:338-339`).
- `send_reliable` receipts cover queued packets (P10). The receipt lists every
sequence the send assigned, not only the ones admitted to the wire.
- `send_reliable` refuses a channel with reliable expiry (P9). The stale
comment at `rudp.rs:1903-1904` says this lineage has no expiry, which is
false since `bac5be88`. Restore the guard it describes.
- New: `CultNetRudpSocketTransportConnection::wait_delivered(&receipt, timeout) -> Result<()>`.
It pumps receive and resends like `flush_reliable`. It fails at once when
the session ended, naming the frozen status. Today four places hand-roll
that loop: `cultmesh-rs/src/lib.rs:806-838` and `:870-900`,
`CodexConnector src/idunn_health.rs:727-745` and
`Ghostlight crates/ghostlight-dungeon/src/idunn_health.rs:1126-1145`.
- **Flush: one wait, and it means Delivered** (**Q-A2**).
- When every sent sequence is Received, `A` is the highest sequence sent, so
every send is Delivered. The two waits coincide, except behind an abandoned
sequence.
- Rust `flush_reliable` (`rudp.rs:1985`) therefore fails at once when an
ordered send follows the session's first abandoned sequence. That case
never becomes Delivered, and today it reports success (P2).
- C# `FlushReliable` (`CultNetTransport.cs:1914`), TS `flush`
(`rudp.ts:906`), Python `flush_reliable` (`transport.py:1101`) and Kotlin
`flushReliable` (`CultMesh.kt:3186`) keep their code. Their contract text
changes to "every reliable frame sent has been delivered to the peer's
application". These runtimes have no expiry.
- A second "flush until received" variant would differ only in the
abandoned-sequence case, where it would report success on frames that
will never be delivered. It has no consumer. Not added.
- **Defaults.** Plain flush waits for Delivered. No parameter.
- **Compatibility.** No C#, TS, Python or Kotlin signature changes. Rust:
removing `Acknowledged` breaks the external callers at their next bump. Each
is a one-line change to `Delivered`, or a switch to `wait_delivered`. The
pinned callers are CodexConnector (`e171eca`) and Ghostlight (`85f7024`).
### 4. Consumers, each assigned
Surveyed across `F:\Projects` (the HEAD of each repo is recorded in the Eyes
report). Every consumer that waits wants **Delivered**. None wants Received
alone.
| Consumer | What it waits on today | Assigned | Change |
|---|---|---|---|
| `cultmesh-rs/src/lib.rs:781-838` `request_raw_snapshot_from_rudp_catalog` (Idunn `src/drivers.rs:5527` uses it) | `Acknowledged`, then the response | Delivered (the response proves it anyway) | Cut 4: `wait_delivered` |
| `cultmesh-rs/src/lib.rs:870-900` `publish_cultnet_message_to_rudp_catalog` (Odin `odin-daemon/src/main.rs:616-658`) | `Acknowledged`, then disconnect. The doc at `:869` claims "application admission". | Delivered. That is true at the document server, which acknowledges after its sink accepted (`rudp_document_server.rs:343-372`). | Cut 4: `wait_delivered`; the doc says Delivered |
| `cultmesh-rs/src/lib.rs:918-938` `publish_cultnet_messages_to_rudp_catalog` (Ghostlight `mesh.rs:323`) | `flush_reliable`, then drop. **Several ordered frames, then close.** | Delivered | None; flush now means it |
| `cultmesh-ts/src/index.ts:6070-6098` `publishRudpDocumentOnce` (Heimdall, StreamPixels, Stonks, weksa, Bifrost, VoidBot, Hermodr) | `peer.flush` (`cultnet-ts/src/peer.ts:109-110`), then close | Delivered | None |
| `cultnet-ts/src/idunn-odin-presence-publisher.ts:100-127` | a hand-rolled session loop that waits for `outstandingReliablePacketCount === 0`, then closes. It never resends, and queued packets are never sent. | Delivered | Cut 5: use the socket transport and `flush` |
| `cultnet-ts/src/signed-daemon-health.ts:113-160` (Hermodr, Odin `idunn-rudp.cjs`) | a hand-rolled loop: sends only the packets `sendMany` admitted, once; waits up to 500 ms for **any** ack packet, swallows the timeout, closes | Delivered | Cut 5: use the socket transport and `flush`. Today a lost datagram is reported as published, and a payload over 32 packets is truncated. |
| `cultnet-ts/src/operation-service.ts` | the operation response | none (the response is the proof) | none |
| C# `FlushReliable` callers: Gjallar `Program.cs:3055-3080` (two ordered frames, flush, dispose) | flush | Delivered | None |
| Heimdall `src/idunn-rudp-health.ts:75-86` (vendored CultLib `b6b1d9c`) | first ack packet | Delivered | Heimdall's own cut at its bump (follow-up) |
| CodexConnector `idunn_health.rs:727-745`, Ghostlight `idunn_health.rs:1126-1145` | `Acknowledged` | Delivered | At their next pin bump |
| Muninn `main.rs:9446-9458`, Sleipnir `main.rs:880-904` (both copies) | nothing (send and drop) | none today | Follow-up: fire-and-forget health can be silently lost |
| Kotlin `flushReliable` | no callers | Delivered | Contract text only |
### 5. The hold buffer and the 4 MiB cap
- **It belongs here**, because the held frame is exactly the state this map
names.
- **Mechanism (P4).** Rust caps held ordered frames at 1,024 frames and 4 MiB
(`rudp.rs:47-48`, checked at `:1169-1175`). A refusal ends the session. The
default payload limit is 16 MiB (`rudp.rs:33`). With one lost packet under a
large ordered transfer, a legal 5 MiB frame is refused and kills the session.
- The sender's window counts pending packets, not span or bytes
(`rudp.rs:898-930`). While one packet stays lost, acknowledged packets keep
freeing slots, so the held data behind the gap grows without limit.
- **Fix: a sender-side flow window, TCP's receive window without a wire
field** (Cut 3). A reliable packet is admitted to the wire only while:
- its sequence is at most 1,023 above the lowest unacknowledged sequence;
and
- the payload bytes of sequences above the lowest unacknowledged one stay
within 4 MiB, with at least one packet always admissible.
- Otherwise it waits in the existing queue.
- **What this buys:**
- a conforming sender never makes a Rust receiver hold more than 4 MiB, so
the cap stops refusing legal traffic;
- a conforming sender never gets 1,024 sequences ahead of the receiver's
highest, so the Rust far-ahead refusal (`rudp.rs:46`) never drops its
packets;
- a conforming sender never leaves a sequence 4,096 behind the receiver's
highest, so P8 cannot happen. The receive-history window stays 4,096 in
every runtime (`CultNetTransport.cs:884`, `rudp.rs:35`, `rudp.ts:20`,
`transport.py:404`, `CultMesh.kt:2625`).
- **What is kept.** The cap and the far-ahead refusal stay. They protect
against hostile peers.
- **Rejected:**
- raising the cap to the payload limit or above: that is 16-64 MiB per
hostile session instead of 4;
- lowering the ordered payload limit to 4 MiB: that is an API regression for
large snapshots.
- **Cost.** Normal flow never has a span beyond the 32-packet window, so the
new limits bind only while a packet is lost.
- **Old senders** (the pinned and vendored ones) can still trip the Rust cap
until they bump.
- **Not in scope.** C#, TS, Python and Kotlin receivers have no hold cap at
all. They hold without limit for a hostile peer. Follow-up.
### 6. Reliable expiry (P2)
- Only Rust has it (`rudp.rs:39`, `:886-896`). An expired packet's sequence
never reaches the receiver. `R` stops below it, and so did the old `next`
pointer. Ordered frames after it are held for the rest of the session.
- **Today's users.** Only Muninn's reliable `audio`
(`Muninn main.rs:3230-3241`), and its session carries no ordered traffic.
`hands/media-fec-cut3` deletes the `audio` profile entry and the `"audio"`
send arm. The `media` channel keeps a caller-chosen reliable-with-expiry
mode.
- **This map makes it honest (Cut 4):**
- the sender records its first abandoned sequence;
- `A` never passes it;
- ordered receipts after it stay `Received`;
- `flush_reliable` fails at once.
- **The real fix is a fork (Q-A3).** The literature's answer is PR-SCTP's
FORWARD-TSN (RFC 3758): the sender tells the receiver to move its cumulative
point past abandoned sequences. That is a wire addition, and every runtime's
receiver would have to learn it.
### 7. Interaction with the stray-packet branch
The ack cuts build on these branch behaviours and do not alter them:
- An ended session fails its flush (`CultNetTransport.cs:1924-1931`,
`rudp.rs:1987-2011`, `rudp.ts:906-916`, `transport.py:1101-1124`).
- A refusing session ends, and its goodbye is built after the reset
(`EndRefused`, `end_refused_session` `rudp.rs:358`). The refused sequence is
never acknowledged, so a refusal never becomes a false Received.
- Rust keeps issued sequences on reset. Cut 4's frozen epoch snapshot relies
on that: an old receipt's sequences can never collide with a new epoch's.
- One Accept per session in C#, TS and Python, and `AnswerRepeatedConnect`. A
repeated Connect's sequence is at most `R` and counts as a duplicate.
- The TS operation service expiry and handler rejection are above the
transport, and do not change.
- One thing the branch left: the Rust socket transport in server mode still
calls `reset_peer_state()` on every Connect (`rudp.rs:2099`), even a repeated
one. Under Cut 4 that rotates the epoch. That makes it visible, but it is the
branch's parity item, not this map's. It is recorded as a follow-up.
## Cuts
Every cut is on CultLib, branch `hands/rudp-delivered` from `main` after
`hands/rudp-stray-packets` merges, in a worktree Self creates. Heavy
verification runs on Yggdrasil. Each cut starts by committing its probes as
failing tests named for the rule, in every runtime it touches.
**Negative checks that stay green in every cut:**
- the cross-runtime reorder, drop and fragment proofs in the TS interop harness
(`docs/cultnet-transport-parity.md:227-248`);
- the stray-packet tests (`rudp_stray_packets.rs`, and their C#, TS and Python
equivalents);
- the codec fixture vectors: no wire byte changes.
### Cut 1. One watermark owns ordered delivery (receiver, all five runtimes)
- **First.** Failing tests P1, P1b and P6 in each runtime. Rust's are in
`probe_ack.rs` at `db7d2886`, and the others are in the probe commits above.
- **Deletes first:**
- the per-channel `next` map and its resets:
- `CultNetTransport.cs:918`, `:1292-1310`;
- `rudp.rs:194`, `:221`, `:348`;
- `rudp.ts:160`, `:214-225`;
- `transport.py:424`, `:458-469`;
- `CultMesh.kt:2643`;
- `expectedSequenceIfUninitialized`:
`CultNetTransport.cs:1151-1153`, `rudp.rs:534-537`, `rudp.ts:375-377`,
`transport.py:595-599`, `CultMesh.kt:2728`;
- `DeliverOrdered`, `DrainOrdered` and `SkipReceivedNonChannelSequences`
(the deletion line above);
- `PendingOrderedFrame.NextSequence`.
- **Adds:**
- `R`, seeded in `AcceptConnect` (`CultNetTransport.cs:1012`, `rudp.rs:377`,
`rudp.ts:242`, `transport.py:485`, `CultMesh.kt:2667`) and on Accept
receipt (`CultNetTransport.cs:1155`, `rudp.rs:539`, `rudp.ts:379`,
`transport.py:601`, `CultMesh.kt:2730`);
- `R` advances in `RememberReceived` (`CultNetTransport.cs:1518`,
`rudp.rs:961`, `rudp.ts:619`, `transport.py:809`, `CultMesh.kt:2891`);
- a reassembled ordered frame is delivered at once when its first sequence
is at most `R + 1` and its channel holds nothing below it. Otherwise it is
held. The caps apply only to what is held: the prototype held everything
and so refused a 5 MiB frame with no gap;
- after every reliable receipt (data, fragment, duplicate, Accept), drain
every channel's held frames whose first sequence is at most `R + 1`, in
sequence order;
- Rust only: the 64-channel cap (`rudp.rs:42`) counts channels with held
frames.
- **Tests changed:** `tests/cultnet.rs:1000` (consistent handshake) and
`:2655` (hold frames behind a gap). The equivalents in the other runtimes are
whatever the change breaks. Each must be read before it is edited, because a
test that encodes `min(highest+1, frame)` encodes P6.
- **Verification:**
- P1, P1b and P6 pass in every runtime;
- the full suites: `cargo test` in `cultnet-rs` and `cultmesh-rs`, `dotnet
test tests/GameCult.Networking.Tests`, `npm test` in `cultnet-ts`, the
Python suite, and Kotlin per Q-A5;
- the TS interop harness across runtimes;
- negative grep: `rg -n "orderedNextSequence|ordered_next_sequence|expectedSequenceIfUninitialized|expected_sequence_if_uninitialized|SkipReceivedNonChannel|skip_received_non_channel" src packages`
returns nothing;
- mutation testing on the diff: cargo-mutants, Stryker.NET, StrykerJS, and
mutmut for Python.
- **Ledger.** Per runtime about −70 and +40. Across five, about −350 and +200.
### Cut 2. A reliable fragment set is never evicted (receiver: Rust, TS, Python)
- **First.** Failing test P5 in each of the three.
- **Change:**
- eviction picks the stalest set that holds no reliable fragment
(`rudp.rs:1015-1035`, `rudp.ts:679-692`, `transport.py:836-842`);
- if every pending set holds reliable fragments, the packet is refused and
the session ends, as every other refusal does.
- A conforming sender cannot reach that bound: its window keeps at most 32
reliable packets in flight, and the bound is 64 sets.
- **Verification:** P5 delivers the reliable frame. `d3bc6e66`'s test (lossy
fragment sets keep being evicted) stays green. `fragment_sets_evicted` still
counts evictions.
- **Ledger.** About −3 and +12 per runtime.
### Cut 3. The sender's flow window: 1,023 sequences and 4 MiB above the lowest unacknowledged (sender, all five runtimes)
- **First.** Failing tests P8 (all runtimes) and P4 (Rust receiver, with a
sender from each runtime).
- **Change.** Admission (`AdmitReliablePackets` / `PromoteQueuedReliable` at
`CultNetTransport.cs:1439-1490`, `rudp.rs:898-930`, `rudp.ts:586-600`,
`transport.py:776-796`, `CultMesh.kt:2856-2872`) admits a packet only while:
- the window has a slot (as today);
- its sequence is at most the lowest unacknowledged sequence plus 1,023;
- the admitted payload bytes above the lowest unacknowledged sequence stay
within 4 MiB.
- The first packet is always admissible. Expired packets (Rust) do not count
as unacknowledged here: they are gone from the wire.
- **Contract.** `docs/cultnet-transport-parity.md` (after `:263`) states the
receive window: a receiver must hold 1,023 sequences and 4 MiB beyond its
contiguous watermark, and a conforming sender never exceeds that.
- **Verification:**
- P8: `g` is delivered;
- P4: the 5 MiB frame is delivered after the gap fills, and the session is
never refused;
- the existing large-snapshot tests (`c398bad1`) and the fragment interop
stay green;
- the Rust benchmark or media probe shows no throughput change on a lossless
link.
- **Ledger.** About +25 per runtime, and nothing removed.
### Cut 4. Rust receipts: Received and Delivered, honest expiry, honest queueing (Rust, cultmesh-rs)
- **First.** Failing tests: P3b (Received, then Delivered), P9, P10, P2
(`flush_reliable` must fail), and the frozen-epoch case (a receipt at
`Received` stays `Received` across `end_refused_session`).
- **Deletes first:**
- `Acknowledged` (`rudp.rs:146`);
- the stale comment and the half guard at `rudp.rs:1903-1908`;
- the hand-rolled status loops at `cultmesh-rs/src/lib.rs:806-838` and
`:870-900`.
- **Adds:**
- `Received` and `Delivered`;
- `ordered` on the receipt;
- `acked_through` (`A`), reading pending, queued and first-abandoned;
- first-abandoned, recorded in `purge_expired_reliable` (`rudp.rs:886`);
- the receipt covers queued sequences: `send_many` (`:450`) reports every
sequence it assigned;
- `send_reliable` refuses a channel with expiry;
- the one-epoch frozen snapshot in `reset_peer_state` (`:338`);
- `wait_delivered`;
- `flush_reliable` fails fast after an abandoned sequence precedes an ordered
send.
- **Changes:**
- `cultmesh-rs/src/lib.rs:832` and `:892` use `wait_delivered`;
- the doc at `:869` says Delivered;
- `tests/rudp_stray_packets.rs:309` and `cultmesh-rs/tests/rudp_document_server.rs:408,415` change from `Acknowledged` to `Delivered`.
- **Verification:**
- the new tests pass;
- `rg -n "Acknowledged" packages/cultnet-rs packages/cultmesh-rs` has no hit;
- both suites are green;
- cargo-mutants on the diff.
- **Ledger.** About −60 and +110 (of which `wait_delivered` is about 35, and
it replaces two loops of about 30 each).
### Cut 5. CultLib-internal consumers and the contract text (TS, docs)
- **Changes:**
- `signed-daemon-health.ts:113-160` and
`idunn-odin-presence-publisher.ts:100-127` send through
`CultNetRudpSocketTransportConnection` and `flush`. Their hand-rolled
session loops are deleted.
- Flush contract text in all five runtimes: "every reliable frame sent has
been delivered to the peer's application".
- `docs/cultnet-transport-parity.md:236-240` (the control-packet guard
paragraph) is replaced by the watermark rule, and `:261-272` gains the
Received/Delivered meanings and the receive window.
- **Verification:**
- `runtime-presence-health.test.ts` and `signed-daemon-health.test.ts` stay
green;
- a new test drops the first datagram and still publishes;
- a new test publishes a payload over 32 packets whole.
- **Ledger.** About −100 and +40.
## Subtraction ledger (estimate)
| Cut | Removed | Added | Other |
|---|---|---|---|
| 1 | ~350 | ~200 | five per-channel state machines become one watermark |
| 2 | ~10 | ~35 | none |
| 3 | 0 | ~125 | a documented receive-window contract |
| 4 | ~60 | ~110 | one public method (`wait_delivered`), two enum variants for one |
| 5 | ~100 | ~40 | two hand-rolled RUDP loops gone |
- **Source, net:** about −10. Tests are extra: about 5 × 6 rule tests.
- **Wire:** zero bytes.
- **Downstream:** each external `Acknowledged` caller needs a one-line change
at its next bump.
- **Prototype check:** the Rust receiver and status prototype is +129/−124 in
`rudp.rs` and includes throwaway public helpers.
## Operator questions
Each question stands alone.
- **Q-A1. Where does Delivered come from?**
- **A.** Derive it at the sender. An ordered send is Delivered when every
reliable sequence up to its last one is Received. A reliable unordered send
is Delivered when Received. Nothing changes on the wire.
- **B.** Add a delivered watermark to the wire. The receiver sets Ack flag
bit `0x10` and puts "delivered through N" in the Ack's unused sequence
field. That is additive: old C#, Rust, TS and Python peers at the branch tip
and at both pins ignore it (P7). Senders use it when it is present, and
fall back to A for old peers.
- **Recommended: A.**
- Once Cuts 1-3 make the receiver correct, B's watermark always equals A's
derivation.
- B is a second authority for one value, plus a capability sniff, plus five
encoder and decoder changes.
- B would only tell the sender more than A if a receiver strands frames,
and that receiver is old, so it does not send B.
- **Depends on it.** Cut 4's shape. B adds a Cut 3b in all five runtimes (+~40
each) and makes the parity codec fixtures change.
- **Q-A2. One flush, or a "received" flush beside the "delivered" one?**
- **A.** One flush. It means Delivered.
- **B.** Add `flushReceived` beside it.
- **Recommended: A.** After this map, all-Received means all-Delivered, except
behind an abandoned sequence. There a received-flush would report success
on frames that will never be delivered, and plain flush fails fast. No
consumer wants Received alone (§4). Receipts keep both states.
- **Depends on it.** B adds a method in five runtimes and a test each.
- **Q-A3. Reliable expiry next to ordered traffic (P2).** An expired reliable
packet leaves a permanent hole, and every ordered frame after it on the same
session is held until the session ends.
- **A.** Report it honestly (Cut 4: receipts stay Received and flush fails
fast). Document "do not send ordered traffic on a session that uses
reliable expiry". No live consumer does.
- **B.** PR-SCTP's FORWARD-TSN (RFC 3758). On expiry the sender sends an
abandonment notice, which rides an Ack-typed packet with flag `0x10` so
that old peers ignore it (P7). New receivers treat the sequence as received
with no frame, and acknowledge it. The sender resends the notice until it
is acknowledged. That is a new cut in all five receivers. Old receivers
never acknowledge it, so the resend needs its own bound.
- **C.** Refuse ordered sends on a session after its first expiry.
- **D.** Delete reliable expiry. Muninn's `audio` is its only user and Muninn
Q4 retires it. This subtracts, but CultLib would lose partial reliability,
which a reasonable consumer can expect.
- **Recommended: A now.** Record B as the complete answer for when a consumer
mixes the two.
- **Depends on it.** Whether Cut 4 carries `first_abandoned` alone (A) or a
new wire notice (B).
- **Q-A4. Receipts in C#, TypeScript, Python and Kotlin?**
- Only Rust has per-send receipts. The other four offer flush and an
outstanding count.
- **A.** Port receipts with the same four states and a `WaitDelivered`, in
this campaign.
- **B.** Record it as a parity gap. Flush covers every surveyed consumer.
- **Recommended: B.** No consumer outside Rust waits per send. A reasonable
C# consumer might want it, but it is additive later and does not change the
wire.
- **Depends on it.** A adds a cut of about +80 per runtime.
- **Q-A5. Is Kotlin in the campaign?**
- `cultmesh-kotlin` carries a fifth RUDP runtime with P1 and P6 (the same code,
`CultMesh.kt:2939-2978`). Its flush has no callers. The stray-packet branch
did not touch it. Its only build path is `build.ps1`, which needs Android
Studio's `kotlinc` and JBR on Windows. Yggdrasil has no Kotlin image in the
verify list.
- **A.** Include it in Cuts 1, 3 and 5, and verify on Starfire as a
Windows-only job, or add a Kotlin image to Yggdrasil first.
- **B.** Record it as parity debt, with the stray-packet gaps. Mark the parity
doc so that nobody reads Kotlin as conforming.
- **Recommended: A, with a Yggdrasil Kotlin image.** Wire parity is the
library's promise. A runtime that strands acknowledged frames should not
stay listed as conforming.
- **Depends on it.** Cut 1 and Cut 3 scope and their verification host.
Not forks, recorded so they are not re-asked:
- `Acknowledged` is removed, not aliased. The ruled states replace it.
- Delivered means handed to the transport's application boundary, not consumed
(the ruling).
- The 4 MiB cap stays. The sender's window keeps legal traffic under it (§5).
- The receiver's 4,096 receive history stays. The sender's window keeps legal
traffic inside it.
- A receipt keeps Received across a session end or reset (the ruling). A reset
keeps one frozen epoch.
## Sequencing and blockers
1. **`hands/rudp-stray-packets` merges to `main`.** It is in Hands now. Every
cut here is based on that merge.
2. **The Odin CultLib pin bump** follows that merge. It is **not** blocked on
this campaign: Odin crashed on stray packets, and it should not wait. Odin
uses `publish_cultnet_message_to_rudp_catalog`, not the enum, so Cut 4 does
not break its build when it bumps again.
3. **Cut 1** is blocked on step 1 only.
4. **Cut 2 and Cut 3** are blocked on step 1 only. They are independent of Cut 1
and of each other, and can run in parallel worktrees. Cut 3 and Cut 1 touch
different functions in the same files. Merge Cut 1 first so that Cut 3's P4
test exercises the new hold path.
5. **Cut 4** is blocked on Cut 1, because Delivered is sound only against a
correct receiver. It is also blocked on Q-A1 and Q-A3.
6. **Cut 5** is blocked on Cut 3 in TypeScript, because the socket transport it
routes through must carry the window.
7. **Parallel campaigns:**
- **`hands/media-fec-cut3`** (Muninn's Cut 3 in CultLib) touches `rudp.rs`
only in the profile's `audio` entry (`~2215`) and the
`"media" | "audio"` arm of `channel_send_options` (`~2380`). Neither
overlaps Cuts 1-4 (the session internals and `send_reliable`
`:1891-1928`). It can merge in either order, and the second to land
rebases trivially. Once it lands, reliable expiry is only for `media` when
the caller chooses Reliable.
- **Muninn's own cuts** use lossy media plus parity. Its reliable `audio`
session has no ordered traffic, so P2 does not bite it. Its next CultLib
pin (the `pins/` commit at Cut 3's CultLib tag, Muninn Q2) picks up
whatever has landed. Nothing here blocks Muninn, and nothing in Muninn
blocks this.
- **CultCache publication Cut 4** (CultNetDatabase on the journal,
`docs/cultcache-publication-cut.md` §4) touches `CultNetDatabase.cs`
only, and needs in-order, exactly-once delivery on `schema`. The C#
socket transports carry one reliable channel (`CultNetTransport.cs:2372-2376`),
so P1 and P6 cannot occur on their sessions. There is no blocking
relation in either direction.
8. **Heimdall and StreamPixels vendored bumps** (`b6b1d9c`, `542ddd3`) come
after Cut 5, so that they pick up honest flushes and the TS window at once.
Heimdall's `idunn-rudp-health.ts` first-ack wait moves to `flush` at that
bump.
9. **CodexConnector (`e171eca`) and Ghostlight (`85f7024`)** change
`Acknowledged` to `Delivered` (or to `wait_delivered`) at their next bump
after Cut 4.
## Follow-ups outside this campaign
- **Hold caps in C#, TS, Python and Kotlin.** Their ordered hold buffers and
per-channel maps are unbounded against a hostile peer. Rust has caps.
- **The Rust server-mode socket transport resets on every Connect**
(`rudp.rs:2099`), even a repeated one. This is a stray-packet parity item.
- **The document server's per-session payload budget never shrinks**
(`rudp_document_server.rs:321`, `:584`). After 4 MiB in one session it drops
data without acknowledging it, and the sender retransmits until it gives up.
- **Fire-and-forget health.** Muninn `main.rs:9446`, Sleipnir `main.rs:880-904`
and Heimdall's first-ack wait can lose a health record silently.
- **FORWARD-TSN** if Q-A3 is A and a consumer later mixes expiry with ordered
traffic.
- **The P4 probe at the pins** predates its fix, so it proves nothing there. If
anyone needs old-sender-to-new-receiver evidence for the cap, rerun
`09318d13`'s P4 against a pinned sender.

55
docs/document-variants-cut.md
View file

@ -314,6 +314,61 @@ not closeable, and a fix batch is in Hands:
**Open for the operator:** plain records with duplicate unique-index values are silently
overwritten today. That predates variants. Fixing it could refuse existing stores.
**C2a status, 2026-09-30.** Hands pushed `hands/variants-c2a` at `640ad9a`.
- Element ids are 12 hex characters. Writes mint them at random. Load mints them deterministically.
Derived ids use `[CultElementId(nameof(Member))]`.
- The CDN chunk ref's id is its offset. The witness types carry ids, and the ids cross C#, TS, Python and
Rust.
Soul held it. Fixes are in Hands:
- ids are never minted inside variant overrides;
- interface unions are never minted;
- a null derived source leaves an empty id;
- load minting is culture-dependent;
- the minted-on-load list goes stale;
- a refused batch leaves minted ids behind;
- the id format is unchecked;
- late union-slot reuse passes registration.
**C2a rulings (operator, 2026-09-30):**
- **Old readers are tripwired.** A store that holds element ids carries a marker that every pre-C2a reader
refuses under C0's unknown-header rule. An old C# reader can therefore never silently strip nested ids.
MessagePack's generated formatters skip unknown slots inside elements.
- **Duplicate element ids refuse at load,** naming the record, the list and the id. This matches the
duplicate-index ruling.
- **C2b precondition (Soul F7).** Whether a pre-id element keeps its load-time id depends on the write path.
C2b must refuse an override that targets a record whose ids exist only in memory, or mint on open.
**C2a Soul pass 2 (`e85b201b`), 2026-09-30: hold.** The tripwire only works against unreleased readers.
Every published release (`cultlib-unity-v1.0.60`, `caching-unity-v1.4.0`, `cultcache-ts-v0.14.0`,
`cultcache-py-v0.3.0`) predates C0. The 1.0.60 C# reader loads a v3 store, and its next write is v1 with the ids
stripped (probe). Other findings:
- F2: the same element object twice in one list commits, then the store refuses to load.
- F3: CultMesh single-file writes bypass the marker.
- F4: TS `pushAll` writes the header without reading it first.
**C2a rulings, 2026-09-30 (operator):**
- **Rollout order closes the released-reader hole.** Ship a CultLib release that refuses v3, move every C#
reader of a store onto it, then let a C2a writer touch that store. There is no format change for this.
This supersedes the claim above that old readers are tripwired: that claim holds only for post-C0 readers.
- **Mark by content.** A store is written as v3 (v5 for directory stores) only when a record actually holds an
element id. An existing v3 header on disk stays.
- Fix batch 3 in Hands: F2, marking by content in every runtime, F3 routed through the cache's header
decision and load checks, F4.
**C2a Soul pass 3 (`9ed61dd2`), 2026-09-30: fix first.**
- A flattened record's `HoldsIds` goes stale when its base changes in the same batch.
- A conditional commit onto disk ignores content.
- Rust `push_all` clobbers headers it cannot read.
- TS keeps a stale header after the file goes away.
- Self's claim that "rollout order makes the raw-writer hole benign" was wrong. The reader that strips ids
from a v1 store is a post-C0, pre-C2a build. That is exactly the reader the v3 marker targets, and rollout
order does not keep it away.
**Ruling, 2026-09-30 (operator): C0 is released only together with C2a.** The first CultLib release that
refuses v3 also contains C2a, so no released reader strips ids. C0 and C1 are never released on their own.
Raw and cross-runtime writers that cannot see ids stay sticky-only. Fix batch 4 is in Hands.
**Operator rulings, 2026-09-30:**
- **Duplicate unique-index values do not load** ("Duplicate indices should not load"). This covers
plain records too, so R6-for-indexes becomes one general rule:

104
packages/cultmesh-rs/src/rudp_document_server.rs
View file

@ -207,6 +207,7 @@ pub struct CultMeshRudpDocumentServer<S, Q, C> {
snapshot_source: Q,
clock: C,
options: CultMeshRudpDocumentServerOptions,
packets_dropped: u64,
}
impl<S, Q, C> CultMeshRudpDocumentServer<S, Q, C>
@ -234,9 +235,18 @@ where
snapshot_source,
clock,
options,
packets_dropped: 0,
})
}
/// Datagrams read and discarded because they belong to no admissible
/// session: malformed frames, unknown sessions, a Connect the session cap
/// refuses, a packet that fails its session. Never an
/// error: a moved flow or a scanner must not end the daemon loop.
pub fn packets_dropped(&self) -> u64 {
self.packets_dropped
}
pub fn local_addr(&self) -> Result<SocketAddr> {
Ok(self.socket.local_addr()?)
}
@ -264,7 +274,10 @@ where
let packet = match decode_rudp_packet(&wire) {
Ok(packet) => packet,
Err(_) => return Ok(CultMeshRudpPollOutcome::Handled),
Err(_) => {
self.packets_dropped += 1;
return Ok(CultMeshRudpPollOutcome::Handled);
}
};
let now_unix = self.clock.now_unix_millis();
let now_monotonic = self.clock.now_monotonic_millis();
@ -274,11 +287,14 @@ where
};
if packet.packet_type == CultNetRudpPacketType::Connect {
self.accept_connection(key, &packet, now_monotonic)?;
if !self.accept_connection(key, &packet, now_monotonic)? {
self.packets_dropped += 1;
}
return Ok(CultMeshRudpPollOutcome::Handled);
}
if !self.sessions.contains_key(&key) {
self.packets_dropped += 1;
return Ok(CultMeshRudpPollOutcome::Handled);
}
if packet.packet_type == CultNetRudpPacketType::Data
@ -309,7 +325,8 @@ where
let result = match result {
Ok(result) => result,
Err(_) => {
self.sessions.remove(&key);
self.end_refused_session(key)?;
self.packets_dropped += 1;
return Ok(CultMeshRudpPollOutcome::Handled);
}
};
@ -385,45 +402,54 @@ where
})
}
/// `Ok(false)` when the Connect was refused because the session cap is full;
/// only a socket failure is an `Err`.
fn accept_connection(
&mut self,
key: CultMeshRudpSessionKey,
packet: &CultNetRudpPacket,
now: u64,
) -> Result<()> {
if !self.sessions.contains_key(&key) {
if self.sessions.len() >= self.options.max_sessions {
return Ok(());
}
self.sessions.insert(
key,
SessionEntry {
session: CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: key.connection_id,
initial_sequence: 1,
resend_delay_ms: duration_millis(self.options.resend_delay),
max_pending_reliable_packets: Some(
self.options.max_pending_reliable_packets_per_session,
),
}),
created_at_monotonic_millis: now,
last_activity_monotonic_millis: now,
admitted_payload_bytes: 0,
},
);
}
let accept = {
let entry = self
.sessions
.get_mut(&key)
.expect("accepted session must be present");
) -> Result<bool> {
let reply = match self.sessions.get_mut(&key) {
// The same key is a retransmitted Connect for the existing epoch.
// A genuinely fresh client incarnation must choose a fresh id.
entry.last_activity_monotonic_millis = now;
entry.session.accept_connect(packet, now, Vec::new())?
Some(entry) => {
entry.last_activity_monotonic_millis = now;
match entry.session.answer_repeated_connect(packet, now) {
Ok(reply) => reply,
Err(_) => {
self.end_refused_session(key)?;
return Ok(false);
}
}
}
None => {
if self.sessions.len() >= self.options.max_sessions {
return Ok(false);
}
let mut session = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: key.connection_id,
initial_sequence: 1,
resend_delay_ms: duration_millis(self.options.resend_delay),
max_pending_reliable_packets: Some(
self.options.max_pending_reliable_packets_per_session,
),
});
let accept = session.accept_connect(packet, now, Vec::new())?;
self.sessions.insert(
key,
SessionEntry {
session,
created_at_monotonic_millis: now,
last_activity_monotonic_millis: now,
admitted_payload_bytes: 0,
},
);
accept
}
};
self.send_packet(key.remote_addr, &accept)
self.send_packet(key.remote_addr, &reply)?;
Ok(true)
}
fn deliver_application_message(
@ -566,6 +592,16 @@ where
Ok(None)
}
/// The session refused a packet: it ends, and the client is told so it does
/// not wait on a session the server no longer holds.
fn end_refused_session(&mut self, key: CultMeshRudpSessionKey) -> Result<()> {
let Some(mut entry) = self.sessions.remove(&key) else {
return Ok(());
};
let goodbye = entry.session.end_refused_session();
self.send_packet(key.remote_addr, &goodbye)
}
fn payload_budget_allows(&self, key: CultMeshRudpSessionKey, bytes: usize) -> bool {
let Some(session_bytes) = self
.sessions

100
packages/cultmesh-rs/tests/rudp_document_server.rs
View file

@ -553,3 +553,103 @@ fn session_cap_and_expiry_use_monotonic_time() -> Result<()> {
assert_eq!(server.session_count(), 0);
Ok(())
}
#[test]
fn a_connect_storm_and_stray_frames_are_dropped_not_fatal() -> Result<()> {
use cultnet_rs::{
CultNetRudpSendOptions, CultNetRudpSession, CultNetRudpSessionOptions,
encode_rudp_packet,
};
let clock = Clock::new(60_000);
let sink = Sink::default();
let options = CultMeshRudpDocumentServerOptions {
max_pending_reliable_packets_per_session: 4,
..Default::default()
};
let mut server = server(options, clock.clone(), sink.clone(), Source::default())?;
let target = server.local_addr()?;
let mut real = client(target, 101)?;
connect(&mut server, &mut [&mut real])?;
assert_eq!(server.packets_dropped(), 0);
let raw = UdpSocket::bind("127.0.0.1:0")?;
let session = |id| {
CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: id,
initial_sequence: 1,
resend_delay_ms: 10,
max_pending_reliable_packets: None,
})
};
let send_raw = |server: &mut Server, packet: &cultnet_rs::CultNetRudpPacket| -> Result<()> {
raw.send_to(&encode_rudp_packet(packet)?, target)?;
server.poll_once()?;
Ok(())
};
// A malformed frame.
raw.send_to(b"not a rudp packet", target)?;
server.poll_once()?;
assert_eq!(server.packets_dropped(), 1);
// Data for a session no Connect admitted.
let mut unadmitted = session(300);
unadmitted.assume_connected(0);
let reliable = CultNetRudpSendOptions {
reliable: true,
..Default::default()
};
send_raw(&mut server, &unadmitted.send("schema", vec![1], reliable.clone())?)?;
assert_eq!(server.packets_dropped(), 2);
// A moved flow re-sends Connect from a socket that never hears the Accept.
// Each repeat re-sends the Accept still awaiting acknowledgement and queues
// nothing, so the storm neither errors nor grows the session's queue.
let mut stormer = session(102);
let connect_packet = stormer.create_connect(0, Vec::new())?;
for _ in 0..200 {
send_raw(&mut server, &connect_packet)?;
}
assert_eq!(server.packets_dropped(), 2);
assert_eq!(server.session_count(), 2);
clock.set(60_000 + 1_000);
assert_eq!(
server.maintain()?.packets_resent,
1,
"one Accept awaits acknowledgement, however many Connects repeated"
);
// A packet its own admitted session refuses ends that session, not the loop.
stormer.assume_connected(0);
let mut poison = stormer.send("schema", vec![1], reliable)?;
poison.fragment_count = 2;
poison.fragment_id = 0;
send_raw(&mut server, &poison)?;
assert_eq!(server.packets_dropped(), 3);
assert_eq!(server.session_count(), 1);
// ...and the refused client is told, after the Accepts its storm drew.
raw.set_read_timeout(Some(Duration::from_millis(200)))?;
let mut buffer = vec![0_u8; 65_535];
let mut told = false;
while let Ok(received) = raw.recv(&mut buffer) {
told |= cultnet_rs::decode_rudp_packet(&buffer[..received])?.packet_type
== cultnet_rs::CultNetRudpPacketType::Disconnect;
}
assert!(told, "the refused client must be sent a goodbye");
send(
&mut real,
&CultNetMessage::DocumentPutRaw {
message_id: "after-storm".into(),
document: document("real", vec![9]),
},
)?;
for _ in 0..20 {
server.poll_once()?;
if sink.0.lock().unwrap().receipts.len() == 1 {
break;
}
}
assert_eq!(sink.0.lock().unwrap().receipts.len(), 1);
Ok(())
}

111
packages/cultnet-py/src/cultnet_py/transport.py
View file

@ -23,6 +23,10 @@ class CultNetTransportStats:
bytes_sent: int = 0
frames_received: int = 0
frames_sent: int = 0
# Datagrams read and discarded because they belong to no session on this
# transport: malformed frames, another session's connection id, a sender
# that is not the peer, or a packet the session refuses.
packets_dropped: int = 0
@dataclass(frozen=True)
@ -406,6 +410,8 @@ class CultNetRudpSession:
raise ValueError("RUDP max_pending_reliable_packets must be greater than zero")
self.max_pending_reliable_packets = options.max_pending_reliable_packets
self._next_sequence = _uint32(options.initial_sequence, "initial_sequence")
if self._next_sequence == 0xFFFFFFFF:
raise ValueError("RUDP initial_sequence must leave room for a reliable packet")
self._next_sequenced_by_channel: dict[str, int] = {}
self._next_fragment_id = 1
self._connected = False
@ -449,6 +455,21 @@ class CultNetRudpSession:
def outstanding_reliable_packet_count(self) -> int:
return len(self._pending_reliable) + len(self._queued_reliable)
def reset_peer_state(self) -> None:
"""Forgets everything learned from the peer; sequence numbers already issued stay issued."""
self._connected = False
self._last_received_at_ms = None
self._highest_received_sequence = None
self._received_sequences.clear()
self._next_sequenced_by_channel.clear()
self._latest_sequenced_by_channel.clear()
self._pending_reliable.clear()
self._queued_reliable.clear()
self._ordered_next_sequence_by_channel.clear()
self._ordered_buffers.clear()
self._fragment_buffers.clear()
self._fragment_sets_evicted = 0
def create_connect(self, now_ms: int = 0, payload: bytes = b"") -> CultNetRudpPacket:
self._ensure_reliable_capacity(1)
packet = self._create_packet(
@ -484,6 +505,25 @@ class CultNetRudpSession:
self._track_reliable(response, now_ms)
return response
def answer_repeated_connect(self, packet: CultNetRudpPacket, now_ms: int = 0) -> CultNetRudpPacket:
"""Answers a Connect from a peer this session already accepted.
The packet's sequence is remembered and nothing is queued, so a Connect
storm cannot grow the reliable queue. The reply is the Accept still awaiting
acknowledgement, or an Ack once it was acknowledged.
"""
self._require_connection(packet)
if packet.packet_type != CultNetRudpPacketType.CONNECT:
raise ValueError(f"Expected RUDP connect packet, got {packet.packet_type.value}")
self._apply_acknowledgements(packet)
self._remember_received(packet.sequence)
self._last_received_at_ms = now_ms
for pending in self._pending_reliable.values():
if pending.packet.packet_type == CultNetRudpPacketType.ACCEPT:
pending.last_sent_at_ms = now_ms
return pending.packet
return self.create_ack()
def send(
self,
channel_id: str,
@ -921,6 +961,7 @@ class CultNetRudpSocketTransportConnection:
self._bytes_sent = 0
self._frames_received = 0
self._frames_sent = 0
self._packets_dropped = 0
self._delivered_frames: deque[CultNetTransportFrame] = deque()
self._closed = False
self.disconnect_reason: bytes | None = None
@ -943,6 +984,7 @@ class CultNetRudpSocketTransportConnection:
bytes_sent=self._bytes_sent,
frames_received=self._frames_received,
frames_sent=self._frames_sent,
packets_dropped=self._packets_dropped,
)
@property
@ -952,6 +994,7 @@ class CultNetRudpSocketTransportConnection:
def connect(self, payload: bytes = b"") -> None:
if self.mode != CultNetRudpSocketMode.CLIENT:
raise ValueError("Only a client RUDP socket transport can initiate connect")
self.disconnect_reason = None
self._send_packet(self.session.create_connect(_now_ms(), payload))
def send(self, channel_id: str, payload: bytes) -> None:
@ -977,17 +1020,53 @@ class CultNetRudpSocketTransportConnection:
return None
self._bytes_received += len(wire)
# What a datagram carries is the sender's business, not a fault of this
# process: a malformed frame, another session's connection id, a sender
# that is not the peer and a packet the session refuses are dropped and
# counted. Only the socket ends a loop. The id is checked before the peer
# endpoint is claimed, so a stray cannot become the peer.
try:
packet = decode_rudp_packet(wire)
except ValueError:
self._packets_dropped += 1
return None
if packet.connection_id != self.session.connection_id:
self._packets_dropped += 1
return None
if self.remote_addr is None:
# Only a Connect claims the endpoint of a server-mode transport.
if self.mode == CultNetRudpSocketMode.SERVER and packet.packet_type != CultNetRudpPacketType.CONNECT:
self._packets_dropped += 1
return None
self.remote_addr = remote_addr
elif remote_addr != self.remote_addr:
self._packets_dropped += 1
return None
packet = decode_rudp_packet(wire)
if self.mode == CultNetRudpSocketMode.SERVER and packet.packet_type == CultNetRudpPacketType.CONNECT:
self._send_packet(self.session.accept_connect(packet, _now_ms()))
# A Connect from the peer this transport already accepted repeats:
# answer it with the Accept already owed and queue nothing.
try:
if self.session.connected:
reply = self.session.answer_repeated_connect(packet, _now_ms())
else:
reply = self.session.accept_connect(packet, _now_ms())
self.disconnect_reason = None
except ValueError:
self._packets_dropped += 1
return None
self._send_packet(reply)
return None
result = self.session.receive(packet, _now_ms())
try:
result = self.session.receive(packet, _now_ms())
except ValueError:
# receive() has already recorded the packet's reliable sequence, so the
# session cannot be kept: a retransmit would be acknowledged and the
# frame silently lost. End it and tell the peer.
self._packets_dropped += 1
self._end_refused_session()
return None
if result.reply is not None:
self._send_packet(result.reply)
for ready in result.ready_to_send:
@ -1027,7 +1106,16 @@ class CultNetRudpSocketTransportConnection:
self._delivered_frames.clear()
self.socket.settimeout(poll_timeout)
try:
while self.session.outstanding_reliable_packet_count > 0:
while True:
# An ended session forgot its unacknowledged writes; reporting
# them flushed would be a lie.
if self.disconnect_reason is not None:
raise ConnectionError(
"RUDP session ended before its reliable writes were acknowledged: "
+ self.disconnect_reason.decode("utf-8", errors="replace")
)
if self.session.outstanding_reliable_packet_count == 0:
return
if time.monotonic() >= deadline:
raise TimeoutError(
"RUDP reliable flush timed out with "
@ -1061,6 +1149,21 @@ class CultNetRudpSocketTransportConnection:
self._closed = True
self.socket.close()
def _end_refused_session(self) -> None:
reason = b"session refused a packet"
# The goodbye is built after the reset, or its ack field would acknowledge
# the very frame the session refused.
self.session.reset_peer_state()
goodbye = self.session.create_disconnect(reason)
self.disconnect_reason = reason
try:
self._send_packet(goodbye)
except OSError:
pass # Best effort: the session ends whether or not the peer hears it.
if self.mode == CultNetRudpSocketMode.SERVER:
# Only a Connect can claim the endpoint again.
self.remote_addr = None
def _send_packet(self, packet: CultNetRudpPacket) -> None:
if self.remote_addr is None:
raise ValueError("RUDP socket transport does not have a remote endpoint")

238
packages/cultnet-py/tests/test_cultnet.py
View file

@ -8,7 +8,7 @@ import unittest
import hashlib
import hmac
import io
from dataclasses import dataclass
from dataclasses import dataclass, replace
from datetime import UTC, datetime
from pathlib import Path
from typing import Callable
@ -695,6 +695,15 @@ class CultNetTests(unittest.TestCase):
self.assertTrue(client.check_timeout(91, 50))
self.assertFalse(client.connected)
def test_a_repeated_connect_acknowledges_what_it_carries(self) -> None:
client = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=103, initial_sequence=1))
server = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=103, initial_sequence=100))
connect = client.create_connect(0, b"join")
accept = server.accept_connect(connect, 0)
reply = server.answer_repeated_connect(replace(connect, ack=accept.sequence), 1)
self.assertEqual(reply.packet_type, CultNetRudpPacketType.ACK)
self.assertEqual(server.outstanding_reliable_packet_count, 0)
def test_cultnet_rudp_session_bounds_pending_reliable_packets_before_enqueue(self) -> None:
session = CultNetRudpSession(
CultNetRudpSessionOptions(connection_id=102, initial_sequence=1, max_pending_reliable_packets=2)
@ -918,6 +927,233 @@ class CultNetTests(unittest.TestCase):
client.close()
server.close()
def test_cultnet_rudp_socket_transport_drops_strays_and_keeps_serving_its_peer(self) -> None:
server_socket = bind_udp_socket()
client_socket = bind_udp_socket()
stray_socket = bind_udp_socket()
connection_id = 0x10203050
server = CultNetRudpSocketTransportConnection(
CultNetRudpSocketTransportOptions(
runtime_id="python-rudp-server",
socket=server_socket,
mode=CultNetRudpSocketMode.SERVER,
connection_id=connection_id,
)
)
client = CultNetRudpSocketTransportConnection(
CultNetRudpSocketTransportOptions(
runtime_id="python-rudp-client",
socket=client_socket,
mode=CultNetRudpSocketMode.CLIENT,
remote_addr=server_socket.getsockname(),
connection_id=connection_id,
)
)
foreign = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=0x0BADF10E))
strays = [
b"not a rudp packet",
b"",
encode_rudp_packet(foreign.create_connect(0, b"foreign")),
]
try:
# A stray that arrives before the peer must not become the peer.
for stray in strays:
stray_socket.sendto(stray, server_socket.getsockname())
for _ in strays:
self.assertIsNone(server.receive_once())
self.assertEqual(server.stats.packets_dropped, len(strays))
client.connect(b"join")
pump_rudp_handshake(client, server)
# A moved flow answers from the peer's own address with someone else's session.
for stray in strays:
server_socket.sendto(stray, client_socket.getsockname())
client_socket.sendto(stray, server_socket.getsockname())
for _ in strays:
self.assertIsNone(client.receive_once())
self.assertIsNone(server.receive_once())
self.assertEqual(client.stats.packets_dropped, len(strays))
self.assertEqual(server.stats.packets_dropped, 2 * len(strays))
client.send("schema", b"still here")
self.assertEqual(receive_rudp_frame(server).payload, b"still here")
server.send("schema", b"server still here")
self.assertEqual(receive_rudp_frame(client).payload, b"server still here")
finally:
stray_socket.close()
client.close()
server.close()
def test_cultnet_rudp_server_mode_is_claimed_only_by_a_connect(self) -> None:
server_socket = bind_udp_socket()
orphan_socket = bind_udp_socket()
client_socket = bind_udp_socket()
connection_id = 0x10203052
server = CultNetRudpSocketTransportConnection(
CultNetRudpSocketTransportOptions(
runtime_id="python-rudp-server",
socket=server_socket,
mode=CultNetRudpSocketMode.SERVER,
connection_id=connection_id,
)
)
client = CultNetRudpSocketTransportConnection(
CultNetRudpSocketTransportOptions(
runtime_id="python-rudp-client",
socket=client_socket,
mode=CultNetRudpSocketMode.CLIENT,
remote_addr=server_socket.getsockname(),
connection_id=connection_id,
)
)
orphan = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=connection_id))
try:
# A right-id packet that is not a Connect must claim nothing.
orphan_socket.sendto(encode_rudp_packet(orphan.create_ack()), server_socket.getsockname())
self.assertIsNone(server.receive_once())
self.assertEqual(server.stats.packets_dropped, 1)
self.assertIsNone(server.remote_addr)
client.connect(b"join")
pump_rudp_handshake(client, server)
self.assertTrue(client.connected)
self.assertEqual(server.remote_addr, client_socket.getsockname())
finally:
orphan_socket.close()
client.close()
server.close()
def test_cultnet_rudp_socket_transport_ends_the_session_that_refuses_a_reliable_frame(self) -> None:
server_socket = bind_udp_socket()
peer_socket = bind_udp_socket()
connection_id = 0x10203053
server = CultNetRudpSocketTransportConnection(
CultNetRudpSocketTransportOptions(
runtime_id="python-rudp-server",
socket=server_socket,
mode=CultNetRudpSocketMode.SERVER,
connection_id=connection_id,
)
)
peer = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=connection_id))
def to_server(packet) -> None:
peer_socket.sendto(encode_rudp_packet(packet), server_socket.getsockname())
def drain_peer() -> list:
packets = []
while True:
try:
wire, _ = peer_socket.recvfrom(65535)
except TimeoutError:
return packets
packets.append(decode_rudp_packet(wire))
try:
to_server(peer.create_connect(0, b"join"))
server.receive_once()
accept = [p for p in drain_peer() if p.packet_type == CultNetRudpPacketType.ACCEPT][0]
peer.receive(accept, 0)
(frame,) = peer.send_many("schema", b"poison", CultNetRudpSendOptions(reliable=True, ordered=True))
poison = replace(frame, fragment_count=2, fragment_id=0)
to_server(poison)
self.assertIsNone(server.receive_once())
self.assertEqual(server.stats.packets_dropped, 1)
self.assertFalse(server.connected)
self.assertEqual(server.disconnect_reason, b"session refused a packet")
# The peer is told, and the goodbye does not acknowledge the refused frame.
(goodbye,) = [p for p in drain_peer() if p.packet_type == CultNetRudpPacketType.DISCONNECT]
self.assertTrue(peer.receive(goodbye, 1).disconnected)
self.assertIn(poison.sequence, peer.pending_reliable_sequences)
# A retransmit is dropped, not acknowledged.
to_server(poison)
self.assertIsNone(server.receive_once())
self.assertEqual(server.stats.packets_dropped, 2)
self.assertEqual(drain_peer(), [])
self.assertIn(poison.sequence, peer.pending_reliable_sequences)
# A Connect claims the endpoint again.
again = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=connection_id))
to_server(again.create_connect(0, b"again"))
server.receive_once()
self.assertEqual(
[p.packet_type for p in drain_peer()], [CultNetRudpPacketType.ACCEPT]
)
finally:
peer_socket.close()
server.close()
def _server_mode_pair(self, connection_id: int):
server_socket = bind_udp_socket()
peer_socket = bind_udp_socket()
server = CultNetRudpSocketTransportConnection(
CultNetRudpSocketTransportOptions(
runtime_id="python-rudp-server",
socket=server_socket,
mode=CultNetRudpSocketMode.SERVER,
connection_id=connection_id,
)
)
peer = CultNetRudpSession(CultNetRudpSessionOptions(connection_id=connection_id))
def to_server(packet) -> None:
peer_socket.sendto(encode_rudp_packet(packet), server_socket.getsockname())
def drain_peer() -> list:
packets = []
while True:
try:
wire, _ = peer_socket.recvfrom(65535)
except TimeoutError:
return packets
packets.append(decode_rudp_packet(wire))
return server, server_socket, peer_socket, peer, to_server, drain_peer
def test_cultnet_rudp_flush_fails_for_a_write_the_ended_session_forgot(self) -> None:
server, _, peer_socket, peer, to_server, drain_peer = self._server_mode_pair(0x10203054)
try:
to_server(peer.create_connect(0, b"join"))
server.receive_once()
peer.receive([p for p in drain_peer() if p.packet_type == CultNetRudpPacketType.ACCEPT][0], 0)
server.send("schema", b"never acknowledged")
(frame,) = peer.send_many("schema", b"poison", CultNetRudpSendOptions(reliable=True, ordered=True))
to_server(replace(frame, fragment_count=2, fragment_id=0))
self.assertIsNone(server.receive_once())
with self.assertRaises(ConnectionError):
server.flush_reliable(0.2)
finally:
peer_socket.close()
server.close()
def test_cultnet_rudp_server_mode_owes_one_reliable_accept_however_many_connects_repeat(self) -> None:
server, _, peer_socket, peer, to_server, drain_peer = self._server_mode_pair(0x10203055)
try:
connect = peer.create_connect(0, b"join")
for _ in range(20):
to_server(connect)
server.receive_once()
accepts = {p.sequence for p in drain_peer() if p.packet_type == CultNetRudpPacketType.ACCEPT}
self.assertEqual(len(accepts), 1)
self.assertLessEqual(server.outstanding_reliable_packet_count, 1)
finally:
peer_socket.close()
server.close()
def test_cultnet_rudp_session_rejects_limits_that_cannot_admit_a_connect(self) -> None:
with self.assertRaises(ValueError):
CultNetRudpSession(CultNetRudpSessionOptions(connection_id=1, initial_sequence=0xFFFFFFFF))
with self.assertRaises(ValueError):
CultNetRudpSession(CultNetRudpSessionOptions(connection_id=1, max_pending_reliable_packets=0))
CultNetRudpSession(
CultNetRudpSessionOptions(connection_id=1, initial_sequence=0xFFFFFFFE, max_pending_reliable_packets=1)
)
def test_cultnet_rudp_socket_transport_carries_fragmented_reliable_ordered_schema_frames(self) -> None:
server_socket = bind_udp_socket()
client_socket = bind_udp_socket()

193
packages/cultnet-rs/src/rudp.rs
View file

@ -126,8 +126,10 @@ pub struct CultNetRudpSendOptions {
}
/// Identifies every transport packet belonging to one non-expiring reliable
/// send. Receipts are local to the session that issued them and cannot survive
/// a peer reset.
/// send. Receipts are local to the session that issued them: a peer reset or
/// a Disconnect received from the peer invalidates them for good. A local
/// `disconnect()` or a timeout does not; the writes it leaves owed carry into
/// the next session.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct CultNetRudpReliableSendReceipt {
session_scope: Uuid,
@ -180,7 +182,6 @@ pub struct CultNetRudpSession {
max_pending_reliable_packets: Option<usize>,
max_payload_bytes: Option<usize>,
max_pending_fragment_sets: usize,
initial_sequence: u32,
next_sequence: u32,
next_sequenced_by_channel: BTreeMap<String, u32>,
next_fragment_id: u16,
@ -208,7 +209,6 @@ impl CultNetRudpSession {
max_pending_reliable_packets: options.max_pending_reliable_packets,
max_payload_bytes: None,
max_pending_fragment_sets: 64,
initial_sequence: options.initial_sequence,
next_sequence: options.initial_sequence,
next_sequenced_by_channel: BTreeMap::new(),
next_fragment_id: 1,
@ -280,6 +280,10 @@ impl CultNetRudpSession {
Some(pending.packet.clone())
}
fn session_scope(&self) -> Uuid {
self.session_scope
}
fn reliable_send_status(
&self,
receipt: &CultNetRudpReliableSendReceipt,
@ -334,11 +338,12 @@ impl CultNetRudpSession {
Ok(())
}
/// Forgets everything learned from the peer. Sequence numbers and fragment
/// ids already issued stay issued: a peer that still remembers them must
/// never see one reused.
pub fn reset_peer_state(&mut self) {
self.session_scope = Uuid::new_v4();
self.next_sequence = self.initial_sequence;
self.next_sequenced_by_channel.clear();
self.next_fragment_id = 1;
self.connected = false;
self.last_received_at_ms = None;
self.highest_received_sequence = None;
@ -351,6 +356,16 @@ impl CultNetRudpSession {
self.fragment_buffers.clear();
}
/// Ends a session that refused a packet. `receive` has already recorded the
/// refused packet's reliable sequence, so the session cannot be kept: a
/// retransmit would be acknowledged and the frame silently lost. Returns the
/// goodbye for the peer, built after the reset or its ack field would
/// acknowledge the very frame that was refused.
pub fn end_refused_session(&mut self) -> CultNetRudpPacket {
self.reset_peer_state();
self.create_disconnect(RUDP_REFUSED_PACKET_REASON.to_vec())
}
pub fn create_connect(&mut self, now_ms: u64, payload: Vec<u8>) -> Result<CultNetRudpPacket> {
self.ensure_reliable_capacity(1)?;
let packet = self.create_packet(
@ -395,6 +410,30 @@ impl CultNetRudpSession {
Ok(response)
}
/// Answers a Connect from a peer this session has already accepted. The
/// packet's sequence is remembered and nothing is queued, so a Connect storm
/// cannot grow the reliable queue. The reply is the Accept still awaiting
/// acknowledgement, or an Ack once it was acknowledged.
pub fn answer_repeated_connect(
&mut self,
packet: &CultNetRudpPacket,
now_ms: u64,
) -> Result<CultNetRudpPacket> {
self.require_connection(packet)?;
if packet.packet_type != CultNetRudpPacketType::Connect {
return Err(anyhow!(
"Expected RUDP connect packet, got {:?}",
packet.packet_type
));
}
self.apply_acknowledgements(packet);
self.remember_received(packet.sequence);
self.last_received_at_ms = Some(now_ms);
Ok(self
.pending_accept_for_resend(now_ms)
.unwrap_or_else(|| self.create_ack()))
}
pub fn send(
&mut self,
channel_id: &str,
@ -555,7 +594,13 @@ impl CultNetRudpSession {
}
if packet.packet_type == CultNetRudpPacketType::Disconnect {
// The session is over. What it still owed the peer dies with it,
// and receipts it issued can never become Acknowledged in a later
// session.
self.connected = false;
self.session_scope = Uuid::new_v4();
self.pending_reliable.clear();
self.queued_reliable.clear();
return Ok(CultNetRudpReceiveResult {
delivered: Vec::new(),
ready_to_send,
@ -1399,6 +1444,10 @@ pub struct CultNetRudpServerHub {
impl CultNetRudpServerHub {
pub fn new(options: CultNetRudpServerHubOptions) -> Result<Self> {
validate_rudp_session_limits(
options.initial_sequence,
options.max_pending_reliable_packets,
)?;
let local_addr = options.socket.local_addr()?;
if options.max_peers == 0 {
return Err(anyhow!(
@ -1572,8 +1621,15 @@ impl CultNetRudpServerHub {
};
wire.truncate(received);
self.stats.bytes_received += received as u64;
let packet = decode_rudp_packet(&wire)?;
// What a datagram carries is the sender's business, not a fault of this
// process: a malformed frame, another session's id and a sender no
// Connect admitted are dropped and counted. Only the socket ends a loop.
let Ok(packet) = decode_rudp_packet(&wire) else {
self.stats.packets_dropped += 1;
return Ok(true);
};
if packet.connection_id != self.connection_id {
self.stats.packets_dropped += 1;
return Ok(true);
}
@ -1581,16 +1637,16 @@ impl CultNetRudpServerHub {
if let Some(peer) = self.peers.get_mut(&remote_addr)
&& peer.context.connect_payload == packet.payload
{
let _ = peer.session.receive(&packet, now_ms())?;
let reply = peer
.session
.pending_accept_for_resend(now_ms())
.unwrap_or_else(|| peer.session.create_ack());
let Ok(reply) = peer.session.answer_repeated_connect(&packet, now_ms()) else {
self.end_refusing_peer(remote_addr)?;
return Ok(true);
};
self.send_packet(remote_addr, &reply)?;
return Ok(true);
}
if !self.peers.contains_key(&remote_addr) && self.peers.len() >= self.max_peers {
return Err(anyhow!("RUDP server hub peer limit reached"));
self.stats.packets_dropped += 1;
return Ok(true);
}
let mut session = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: self.connection_id,
@ -1610,7 +1666,10 @@ impl CultNetRudpServerHub {
session_generation: generation,
connect_payload: packet.payload.clone(),
};
let accept = session.accept_connect(&packet, now_ms(), Vec::new())?;
let Ok(accept) = session.accept_connect(&packet, now_ms(), Vec::new()) else {
self.stats.packets_dropped += 1;
return Ok(true);
};
self.send_packet(remote_addr, &accept)?;
let replaced = self.peers.insert(
remote_addr,
@ -1632,9 +1691,13 @@ impl CultNetRudpServerHub {
}
let Some(peer) = self.peers.get_mut(&remote_addr) else {
self.stats.packets_dropped += 1;
return Ok(true);
};
let Ok(result) = peer.session.receive(&packet, now_ms()) else {
self.end_refusing_peer(remote_addr)?;
return Ok(true);
};
let result = peer.session.receive(&packet, now_ms())?;
let context = peer.context.clone();
let ack = if packet.reliable {
Some(peer.session.create_ack_for_received(packet.sequence))
@ -1678,6 +1741,24 @@ impl CultNetRudpServerHub {
Ok(true)
}
/// An admitted peer sent a packet its session refuses. `receive` has
/// already recorded the packet's reliable sequence, so the session cannot be
/// kept: a retransmit would be acknowledged and the frame silently lost.
/// The session ends, the peer is told, and the caller sees `Disconnected`.
fn end_refusing_peer(&mut self, remote_addr: SocketAddr) -> Result<()> {
self.stats.packets_dropped += 1;
let Some(mut peer) = self.peers.remove(&remote_addr) else {
return Ok(());
};
let goodbye = peer.session.end_refused_session();
self.pending_events
.push_back(CultNetRudpServerEvent::Disconnected {
session: peer.context,
reason: goodbye.payload.clone(),
});
self.send_packet(remote_addr, &goodbye)
}
pub fn poll_resends(&mut self) -> Result<()> {
let now = now_ms();
let mut packets = Vec::new();
@ -1725,6 +1806,10 @@ impl CultNetRudpServerHub {
impl CultNetRudpSocketTransportConnection {
pub fn new(options: CultNetRudpSocketTransportOptions) -> Result<Self> {
validate_rudp_session_limits(
options.initial_sequence,
options.max_pending_reliable_packets,
)?;
let local_addr = options.socket.local_addr()?;
let max_payload_bytes = options
.max_payload_bytes
@ -1793,6 +1878,7 @@ impl CultNetRudpSocketTransportConnection {
));
}
let packet = self.session.create_connect(now_ms(), payload)?;
self.disconnect_reason = None;
self.send_packet(&packet)
}
@ -1911,9 +1997,23 @@ impl CultNetRudpSocketTransportConnection {
self.socket.set_read_timeout(Some(poll_timeout))?;
let deadline = Instant::now() + timeout;
// A flush belongs to the session it started in: a Connect that
// replaces the peer mid-wait forgets the writes being waited on.
let scope = self.session.session_scope();
let mut preserved_frames = VecDeque::new();
let result = (|| {
while self.outstanding_reliable_packet_count() > 0 {
loop {
// An ended session forgot its unacknowledged writes; reporting
// them flushed would be a lie.
if self.session.session_scope() != scope || self.disconnect_reason.is_some() {
return Err(anyhow!(
"RUDP session ended before its reliable writes were acknowledged: {}",
String::from_utf8_lossy(self.disconnect_reason.as_deref().unwrap_or(b"replaced by a new Connect"))
));
}
if self.outstanding_reliable_packet_count() == 0 {
return Ok(());
}
if Instant::now() >= deadline {
return Err(anyhow!(
"RUDP reliable flush timed out with {} packets outstanding",
@ -1925,7 +2025,6 @@ impl CultNetRudpSocketTransportConnection {
}
self.poll_resends()?;
}
Ok(())
})();
preserved_frames.append(&mut self.delivered_frames);
@ -1970,7 +2069,20 @@ impl CultNetRudpSocketTransportConnection {
wire.truncate(received);
self.stats.bytes_received += received as u64;
let packet = decode_rudp_packet(&wire)?;
// What a datagram carries is the sender's business, not a fault of this
// process: a malformed frame, another session's id, a sender that is not
// this transport's peer and a packet the session refuses are dropped and
// counted. Only the socket ends a loop. The id is checked before the
// peer endpoint or session state is touched, so a stray Connect can
// neither move the endpoint nor reset the session.
let Ok(packet) = decode_rudp_packet(&wire) else {
self.stats.packets_dropped += 1;
return Ok(true);
};
if packet.connection_id != self.session.connection_id() {
self.stats.packets_dropped += 1;
return Ok(true);
}
if let Some(expected) = self.remote_addr {
if expected != remote_addr {
if self.mode == CultNetRudpSocketMode::Server
@ -1978,6 +2090,7 @@ impl CultNetRudpSocketTransportConnection {
{
self.remote_addr = Some(remote_addr);
} else {
self.stats.packets_dropped += 1;
return Ok(true);
}
}
@ -1985,6 +2098,7 @@ impl CultNetRudpSocketTransportConnection {
if self.mode == CultNetRudpSocketMode::Server
&& packet.packet_type != CultNetRudpPacketType::Connect
{
self.stats.packets_dropped += 1;
return Ok(true);
}
self.remote_addr = Some(remote_addr);
@ -1993,12 +2107,28 @@ impl CultNetRudpSocketTransportConnection {
&& packet.packet_type == CultNetRudpPacketType::Connect
{
self.session.reset_peer_state();
let accept = self.session.accept_connect(&packet, now_ms(), Vec::new())?;
self.disconnect_reason = None;
let Ok(accept) = self.session.accept_connect(&packet, now_ms(), Vec::new()) else {
self.stats.packets_dropped += 1;
return Ok(true);
};
self.send_packet(&accept)?;
return Ok(true);
}
let result = self.session.receive(&packet, now_ms())?;
let Ok(result) = self.session.receive(&packet, now_ms()) else {
// End the session and tell the peer. A server-mode transport also
// forgets its peer: only a Connect can claim it again.
self.stats.packets_dropped += 1;
let goodbye = self.session.end_refused_session();
self.disconnect_reason = Some(goodbye.payload.clone());
let sent = self.send_packet(&goodbye);
if self.mode == CultNetRudpSocketMode::Server {
self.remote_addr = None;
}
sent?;
return Ok(false);
};
if let Some(reply) = result.reply {
self.send_packet(&reply)?;
}
@ -2142,6 +2272,29 @@ where
}
}
/// Sent to a peer whose session refused one of its packets, and reported to
/// the local caller as the disconnect reason.
const RUDP_REFUSED_PACKET_REASON: &[u8] = b"session refused a packet";
/// A session cannot admit a Connect when its sequence space starts exhausted or
/// its reliable queue holds nothing, so neither is a usable configuration.
fn validate_rudp_session_limits(
initial_sequence: u32,
max_pending_reliable_packets: Option<u32>,
) -> Result<()> {
if initial_sequence == u32::MAX {
return Err(anyhow!(
"RUDP initial_sequence must leave room for a reliable packet"
));
}
if max_pending_reliable_packets == Some(0) {
return Err(anyhow!(
"RUDP max_pending_reliable_packets must be greater than zero"
));
}
Ok(())
}
#[derive(Clone, Debug, Default)]
pub struct RudpTransportProfileOptions {
pub transport_id: Option<String>,

5
packages/cultnet-rs/src/transport.rs
View file

@ -26,6 +26,11 @@ pub struct CultNetTransportStats {
/// payload that lost a fragment on the wire. Zero for non-fragmenting
/// transports.
pub fragment_sets_evicted: u64,
/// Datagrams read and discarded because they belong to no session on this
/// transport: malformed frames, another session's connection id, an
/// unadmitted sender, or a packet the session refused. A steady rate means
/// a flow is being moved or scanned, not that this transport is failing.
pub packets_dropped: u64,
}
#[derive(Clone, Debug, PartialEq, Eq)]

17
packages/cultnet-rs/tests/rudp_server_hub.rs
View file

@ -415,16 +415,13 @@ fn session_and_hub_reject_configured_memory_bounds() -> Result<()> {
let mut second = client(server_addr)?;
connect(&mut hub, &mut first, b"first")?;
second.connect(b"second".to_vec())?;
for _ in 0..20 {
match hub.receive_event_once() {
Err(error) => {
assert!(error.to_string().contains("peer limit"));
return Ok(());
}
Ok(_) => continue,
}
}
Err(anyhow!("hub did not enforce peer limit"))
// The peer limit refuses the second Connect without ending the hub loop:
// a full table is not a fault of the process that holds it.
std::thread::sleep(Duration::from_millis(20));
while hub.receive_event_once()?.is_some() {}
assert_eq!(hub.sessions().len(), 1);
assert_eq!(hub.stats().packets_dropped, 1);
Ok(())
}
#[test]

752
packages/cultnet-rs/tests/rudp_stray_packets.rs Normal file
View file

@ -0,0 +1,752 @@
//! A packet that does not belong to a session is dropped; only socket failures
//! end a transport loop. Each test puts one class of stray in front of a real
//! session and requires the real session to keep being served.
use std::net::{SocketAddr, UdpSocket};
use std::time::Duration;
use anyhow::Result;
use cultnet_rs::*;
const CONNECTION_ID: u32 = 0x4355_4c54;
const FOREIGN_ID: u32 = 0x0BAD_F10E;
#[derive(Clone, Copy)]
enum Stray {
/// Bytes that are not a RUDP frame.
Malformed,
/// A well-formed session that carries someone else's connection id: what a
/// moved UDP flow delivers.
ForeignConnectionId,
/// The right connection id from a sender no Connect ever admitted, or one
/// that already closed.
UnadmittedSender,
}
fn socket() -> Result<UdpSocket> {
let socket = UdpSocket::bind("127.0.0.1:0")?;
socket.set_read_timeout(Some(Duration::from_millis(20)))?;
Ok(socket)
}
fn raw_session(connection_id: u32) -> CultNetRudpSession {
CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id,
initial_sequence: 1,
resend_delay_ms: 250,
max_pending_reliable_packets: None,
})
}
fn reliable() -> CultNetRudpSendOptions {
CultNetRudpSendOptions {
reliable: true,
..Default::default()
}
}
fn send_to(socket: &UdpSocket, to: SocketAddr, packet: &CultNetRudpPacket) -> Result<()> {
socket.send_to(&encode_rudp_packet(packet)?, to)?;
Ok(())
}
/// How many datagrams `send_stray` puts on the wire for `kind`.
fn datagrams(kind: Stray) -> u64 {
match kind {
Stray::Malformed => 2,
Stray::ForeignConnectionId => 4,
Stray::UnadmittedSender => 3,
}
}
/// A packet the sender's own session accepted for sending and its receiver's
/// session refuses: a fragment with no fragment id.
fn poisoned(session: &mut CultNetRudpSession) -> Result<CultNetRudpPacket> {
let mut packet = session.send("schema", b"poison".to_vec(), reliable())?;
packet.fragment_count = 2;
packet.fragment_id = 0;
Ok(packet)
}
fn send_stray(kind: Stray, from: &UdpSocket, to: SocketAddr) -> Result<()> {
match kind {
Stray::Malformed => {
from.send_to(b"not a rudp packet", to)?;
from.send_to(&[], to)?;
}
Stray::ForeignConnectionId => {
let mut foreign = raw_session(FOREIGN_ID);
send_to(from, to, &foreign.create_connect(0, b"foreign".to_vec())?)?;
foreign.assume_connected(0);
send_to(from, to, &foreign.send("schema", b"foreign".to_vec(), reliable())?)?;
send_to(from, to, &foreign.create_ack())?;
send_to(from, to, &foreign.create_disconnect(b"foreign".to_vec()))?;
}
Stray::UnadmittedSender => {
let mut orphan = raw_session(CONNECTION_ID);
orphan.assume_connected(0);
send_to(from, to, &orphan.send("schema", b"orphan".to_vec(), reliable())?)?;
send_to(from, to, &orphan.create_ack())?;
send_to(from, to, &orphan.create_disconnect(b"orphan".to_vec()))?;
}
}
Ok(())
}
fn hub_keeps_serving_its_session(kind: Stray) -> Result<()> {
let mut options = CultNetRudpServerHubOptions::new("hub", socket()?, CONNECTION_ID);
options.max_peers = 1;
let mut hub = CultNetRudpServerHub::new(options)?;
let hub_addr = hub.local_addr()?;
let mut real = CultNetRudpSocketTransportConnection::new(
CultNetRudpSocketTransportOptions::client("real", socket()?, hub_addr, CONNECTION_ID),
)?;
real.connect(b"real".to_vec())?;
let mut admitted = false;
for _ in 0..20 {
if let Some(CultNetRudpServerEvent::Connected { .. }) = hub.receive_event_once()? {
admitted = true;
break;
}
}
assert!(admitted);
let _ = real.receive_once()?;
let stray = socket()?;
send_stray(kind, &stray, hub_addr)?;
std::thread::sleep(Duration::from_millis(20));
while hub.receive_event_once()?.is_some() {}
assert_eq!(hub.stats().packets_dropped, datagrams(kind), "every stray is counted once");
real.send("schema", b"still here".to_vec())?;
std::thread::sleep(Duration::from_millis(20));
match hub.receive_event_once()? {
Some(CultNetRudpServerEvent::Frame { frame, .. }) => {
assert_eq!(frame.payload, b"still here");
}
other => panic!("the real session must still be served, got {other:?}"),
}
Ok(())
}
#[test]
fn hub_drops_a_connect_that_arrives_with_the_peer_table_full() -> Result<()> {
let mut options = CultNetRudpServerHubOptions::new("hub", socket()?, CONNECTION_ID);
options.max_peers = 1;
let mut hub = CultNetRudpServerHub::new(options)?;
let hub_addr = hub.local_addr()?;
let mut real = CultNetRudpSocketTransportConnection::new(
CultNetRudpSocketTransportOptions::client("real", socket()?, hub_addr, CONNECTION_ID),
)?;
real.connect(b"real".to_vec())?;
while !matches!(
hub.receive_event_once()?,
Some(CultNetRudpServerEvent::Connected { .. })
) {}
let _ = real.receive_once()?;
send_to(
&socket()?,
hub_addr,
&raw_session(CONNECTION_ID).create_connect(0, b"late".to_vec())?,
)?;
std::thread::sleep(Duration::from_millis(20));
while hub.receive_event_once()?.is_some() {}
assert_eq!(hub.sessions().len(), 1);
assert_eq!(hub.stats().packets_dropped, 1);
real.send("schema", b"still here".to_vec())?;
std::thread::sleep(Duration::from_millis(20));
assert!(matches!(
hub.receive_event_once()?,
Some(CultNetRudpServerEvent::Frame { .. })
));
Ok(())
}
/// A moved flow answers a client from its server address with someone else's
/// session.
fn client_keeps_its_session(kind: Stray) -> Result<()> {
let server = socket()?;
let server_addr = server.local_addr()?;
let mut client = CultNetRudpSocketTransportConnection::new(
CultNetRudpSocketTransportOptions::client("client", socket()?, server_addr, CONNECTION_ID),
)?;
client.connect(b"hello".to_vec())?;
let mut buffer = vec![0_u8; 65_535];
let (received, client_addr) = server.recv_from(&mut buffer)?;
let connect = decode_rudp_packet(&buffer[..received])?;
let mut server_session = raw_session(CONNECTION_ID);
let accept = server_session.accept_connect(&connect, 0, Vec::new())?;
send_to(&server, client_addr, &accept)?;
let _ = client.receive_once()?;
assert!(client.connected());
send_stray(kind, &server, client_addr)?;
send_to(
&server,
client_addr,
&server_session.send("schema", b"real".to_vec(), reliable())?,
)?;
let frame = client
.receive_once()?
.expect("the real frame behind the strays");
assert_eq!(frame.payload, b"real");
assert_eq!(client.stats().packets_dropped, datagrams(kind), "every stray is counted once");
Ok(())
}
/// A server-mode transport owns one peer. A stray, including a foreign Connect
/// from another address, neither resets that peer's session nor moves the
/// endpoint the transport answers.
fn server_mode_keeps_its_peer(kind: Stray) -> Result<()> {
let server_socket = socket()?;
let server_addr = server_socket.local_addr()?;
let mut server = CultNetRudpSocketTransportConnection::new(
CultNetRudpSocketTransportOptions::server("server", server_socket, CONNECTION_ID),
)?;
let peer = socket()?;
let mut peer_session = raw_session(CONNECTION_ID);
send_to(&peer, server_addr, &peer_session.create_connect(0, b"peer".to_vec())?)?;
let _ = server.receive_once()?;
let mut buffer = vec![0_u8; 65_535];
let (received, _) = peer.recv_from(&mut buffer)?;
peer_session.receive(&decode_rudp_packet(&buffer[..received])?, 0)?;
send_stray(kind, &socket()?, server_addr)?;
send_stray(kind, &peer, server_addr)?;
send_to(
&peer,
server_addr,
&peer_session.send("schema", b"real".to_vec(), reliable())?,
)?;
let frame = server
.receive_once()?
.expect("the real frame behind the strays");
assert_eq!(frame.payload, b"real");
// Once from an unrelated address, once from the peer's own.
assert_eq!(server.stats().packets_dropped, 2 * datagrams(kind), "every stray is counted once");
Ok(())
}
macro_rules! per_stray {
($body:ident, $malformed:ident, $foreign:ident, $unadmitted:ident) => {
#[test]
fn $malformed() -> Result<()> {
$body(Stray::Malformed)
}
#[test]
fn $foreign() -> Result<()> {
$body(Stray::ForeignConnectionId)
}
#[test]
fn $unadmitted() -> Result<()> {
$body(Stray::UnadmittedSender)
}
};
($body:ident, $malformed:ident, $foreign:ident) => {
#[test]
fn $malformed() -> Result<()> {
$body(Stray::Malformed)
}
#[test]
fn $foreign() -> Result<()> {
$body(Stray::ForeignConnectionId)
}
};
}
per_stray!(
hub_keeps_serving_its_session,
hub_survives_malformed_frames,
hub_survives_foreign_connection_ids,
hub_survives_unadmitted_senders
);
per_stray!(
client_keeps_its_session,
client_survives_malformed_frames,
client_survives_foreign_connection_ids
);
per_stray!(
server_mode_keeps_its_peer,
server_mode_survives_malformed_frames,
server_mode_survives_foreign_connection_ids
);
/// A session that refuses a packet has already recorded its reliable sequence,
/// so keeping it would acknowledge the sender's retransmit and lose the frame.
/// The session ends and the sender is told.
fn refused_frame_options() -> (u32, Vec<u8>) {
(64, vec![7_u8; 100])
}
fn client_with_fast_resend(server_addr: SocketAddr) -> Result<CultNetRudpSocketTransportConnection> {
let mut options =
CultNetRudpSocketTransportOptions::client("client", socket()?, server_addr, CONNECTION_ID);
options.resend_delay_ms = 1;
CultNetRudpSocketTransportConnection::new(options)
}
/// The client's view after the server refused its frame: told the session is
/// over, and never shown the frame as acknowledged, even after it retransmits.
fn assert_client_sees_refusal(
client: &mut CultNetRudpSocketTransportConnection,
receipt: &CultNetRudpReliableSendReceipt,
mut serve: impl FnMut() -> Result<()>,
) -> Result<()> {
serve()?;
let _ = client.receive_once()?;
assert!(
client.disconnect_reason().is_some(),
"the client must be told the session ended"
);
std::thread::sleep(Duration::from_millis(10));
client.poll_resends()?;
serve()?;
let _ = client.receive_once()?;
assert_ne!(
client.reliable_send_status(receipt),
CultNetRudpReliableSendStatus::Acknowledged,
"a refused frame must never be acknowledged"
);
Ok(())
}
#[test]
fn hub_ends_the_session_of_a_peer_whose_reliable_frame_it_refuses() -> Result<()> {
let (limit, payload) = refused_frame_options();
let mut options = CultNetRudpServerHubOptions::new("hub", socket()?, CONNECTION_ID);
options.max_payload_bytes = Some(limit);
let mut hub = CultNetRudpServerHub::new(options)?;
let hub_addr = hub.local_addr()?;
let mut client = client_with_fast_resend(hub_addr)?;
client.connect(b"peer".to_vec())?;
assert!(matches!(
hub.receive_event_once()?,
Some(CultNetRudpServerEvent::Connected { .. })
));
let _ = client.receive_once()?;
assert!(client.connected());
let receipt = client.send_reliable("schema", payload)?;
let mut ended = false;
assert_client_sees_refusal(&mut client, &receipt, || {
while let Some(event) = hub.receive_event_once()? {
ended |= matches!(event, CultNetRudpServerEvent::Disconnected { .. });
}
Ok(())
})?;
assert!(ended, "the hub must report the ended session");
assert_eq!(hub.sessions().len(), 0);
assert!(hub.stats().packets_dropped >= 1);
Ok(())
}
#[test]
fn server_mode_ends_the_session_of_a_peer_whose_reliable_frame_it_refuses() -> Result<()> {
let (limit, payload) = refused_frame_options();
let server_socket = socket()?;
let server_addr = server_socket.local_addr()?;
let mut options = CultNetRudpSocketTransportOptions::server("server", server_socket, CONNECTION_ID);
options.max_payload_bytes = Some(limit);
let mut server = CultNetRudpSocketTransportConnection::new(options)?;
let mut client = client_with_fast_resend(server_addr)?;
client.connect(b"peer".to_vec())?;
let _ = server.receive_once()?;
let _ = client.receive_once()?;
assert!(client.connected());
let receipt = client.send_reliable("schema", payload)?;
assert_client_sees_refusal(&mut client, &receipt, || {
let _ = server.receive_once()?;
Ok(())
})?;
assert!(server.disconnect_reason().is_some());
assert!(server.stats().packets_dropped >= 1);
Ok(())
}
#[test]
fn client_ends_its_session_when_it_refuses_a_packet() -> Result<()> {
let server = socket()?;
let server_addr = server.local_addr()?;
let mut client = CultNetRudpSocketTransportConnection::new(
CultNetRudpSocketTransportOptions::client("client", socket()?, server_addr, CONNECTION_ID),
)?;
client.connect(b"hello".to_vec())?;
let mut buffer = vec![0_u8; 65_535];
let (received, client_addr) = server.recv_from(&mut buffer)?;
let mut server_session = raw_session(CONNECTION_ID);
let accept =
server_session.accept_connect(&decode_rudp_packet(&buffer[..received])?, 0, Vec::new())?;
send_to(&server, client_addr, &accept)?;
let _ = client.receive_once()?;
send_to(&server, client_addr, &poisoned(&mut server_session)?)?;
assert!(client.receive_once()?.is_none());
assert_eq!(client.stats().packets_dropped, 1);
assert!(client.disconnect_reason().is_some());
// The server is told, not left holding a live session.
let mut told = false;
while let Ok((received, _)) = server.recv_from(&mut buffer) {
told |= decode_rudp_packet(&buffer[..received])?.packet_type
== CultNetRudpPacketType::Disconnect;
}
assert!(told, "the server must be told the session ended");
Ok(())
}
#[test]
fn server_mode_ends_the_session_when_the_session_refuses_a_packet() -> Result<()> {
let server_socket = socket()?;
let server_addr = server_socket.local_addr()?;
let mut server = CultNetRudpSocketTransportConnection::new(
CultNetRudpSocketTransportOptions::server("server", server_socket, CONNECTION_ID),
)?;
let stray = socket()?;
// Before any peer: a right-id packet that is not a Connect claims nothing.
send_stray(Stray::UnadmittedSender, &stray, server_addr)?;
assert!(server.receive_once()?.is_none());
assert_eq!(server.stats().packets_dropped, 3);
let peer = socket()?;
let mut peer_session = raw_session(CONNECTION_ID);
send_to(&peer, server_addr, &peer_session.create_connect(0, b"peer".to_vec())?)?;
let _ = server.receive_once()?;
let mut buffer = vec![0_u8; 65_535];
let (received, _) = peer.recv_from(&mut buffer)?;
peer_session.receive(&decode_rudp_packet(&buffer[..received])?, 0)?;
assert_eq!(server.stats().packets_dropped, 3);
// After the peer is known: right-id packets from any other address.
send_stray(Stray::UnadmittedSender, &stray, server_addr)?;
assert!(server.receive_once()?.is_none());
assert_eq!(server.stats().packets_dropped, 6);
// A packet from the peer that the session itself refuses ends the session
// and the peer is told.
send_to(&peer, server_addr, &poisoned(&mut peer_session)?)?;
assert!(server.receive_once()?.is_none());
assert_eq!(server.stats().packets_dropped, 7);
assert!(server.disconnect_reason().is_some());
let (received, _) = peer.recv_from(&mut buffer)?;
assert_eq!(
decode_rudp_packet(&buffer[..received])?.packet_type,
CultNetRudpPacketType::Disconnect
);
Ok(())
}
#[test]
fn constructors_reject_limits_that_make_a_connect_unadmittable() -> Result<()> {
let mut hub = CultNetRudpServerHubOptions::new("hub", socket()?, CONNECTION_ID);
hub.initial_sequence = u32::MAX;
assert!(CultNetRudpServerHub::new(hub).is_err());
let mut hub = CultNetRudpServerHubOptions::new("hub", socket()?, CONNECTION_ID);
hub.max_pending_reliable_packets = Some(0);
assert!(CultNetRudpServerHub::new(hub).is_err());
for build in [
(|s, a| CultNetRudpSocketTransportOptions::client("c", s, a, CONNECTION_ID))
as fn(UdpSocket, SocketAddr) -> CultNetRudpSocketTransportOptions,
|s, _| CultNetRudpSocketTransportOptions::server("s", s, CONNECTION_ID),
] {
let addr = "127.0.0.1:9".parse()?;
let mut options = build(socket()?, addr);
options.initial_sequence = u32::MAX;
assert!(CultNetRudpSocketTransportConnection::new(options).is_err());
let mut options = build(socket()?, addr);
options.max_pending_reliable_packets = Some(0);
assert!(CultNetRudpSocketTransportConnection::new(options).is_err());
let mut options = build(socket()?, addr);
options.initial_sequence = u32::MAX - 1;
options.max_pending_reliable_packets = Some(1);
assert!(CultNetRudpSocketTransportConnection::new(options).is_ok());
}
Ok(())
}
/// The distinct reliable Accept sequences a peer has been sent so far.
fn accept_sequences(peer: &UdpSocket) -> Result<std::collections::BTreeSet<u32>> {
let mut buffer = vec![0_u8; 65_535];
let mut sequences = std::collections::BTreeSet::new();
while let Ok((received, _)) = peer.recv_from(&mut buffer) {
let packet = decode_rudp_packet(&buffer[..received])?;
if packet.packet_type == CultNetRudpPacketType::Accept && packet.reliable {
sequences.insert(packet.sequence);
}
}
Ok(sequences)
}
/// A Connect that repeats (a moved flow, a retransmit, a storm) is answered
/// with the Accept already owed, never a fresh reliable Accept per Connect.
#[test]
fn hub_owes_one_reliable_accept_however_many_connects_repeat() -> Result<()> {
let mut hub =
CultNetRudpServerHub::new(CultNetRudpServerHubOptions::new("hub", socket()?, CONNECTION_ID))?;
let hub_addr = hub.local_addr()?;
let peer = socket()?;
let mut peer_session = raw_session(CONNECTION_ID);
let connect = peer_session.create_connect(0, b"peer".to_vec())?;
let mut connected = 0;
for _ in 0..20 {
send_to(&peer, hub_addr, &connect)?;
while let Some(event) = hub.receive_event_once()? {
assert!(
matches!(event, CultNetRudpServerEvent::Connected { .. }),
"a repeated Connect must not replace the admitted session"
);
connected += 1;
}
}
assert_eq!(connected, 1);
assert_eq!(accept_sequences(&peer)?.len(), 1);
Ok(())
}
#[test]
fn server_mode_owes_one_reliable_accept_however_many_connects_repeat() -> Result<()> {
let server_socket = socket()?;
let server_addr = server_socket.local_addr()?;
let mut server = CultNetRudpSocketTransportConnection::new(
CultNetRudpSocketTransportOptions::server("server", server_socket, CONNECTION_ID),
)?;
let peer = socket()?;
let mut peer_session = raw_session(CONNECTION_ID);
let connect = peer_session.create_connect(0, b"peer".to_vec())?;
for _ in 0..20 {
send_to(&peer, server_addr, &connect)?;
let _ = server.receive_once()?;
}
assert!(server.outstanding_reliable_packet_count() <= 1);
Ok(())
}
/// A late duplicate Connect resets what the server learned of the peer, never
/// what it already issued: a reused sequence is dropped by the peer as a
/// duplicate.
#[test]
fn server_mode_never_reuses_a_sequence_after_a_late_duplicate_connect() -> Result<()> {
let server_socket = socket()?;
let server_addr = server_socket.local_addr()?;
let mut server = CultNetRudpSocketTransportConnection::new(
CultNetRudpSocketTransportOptions::server("server", server_socket, CONNECTION_ID),
)?;
let peer = socket()?;
let mut peer_session = raw_session(CONNECTION_ID);
let connect = peer_session.create_connect(0, b"peer".to_vec())?;
send_to(&peer, server_addr, &connect)?;
let _ = server.receive_once()?;
server.send_reliable("schema", b"before".to_vec())?;
send_to(&peer, server_addr, &connect)?;
let _ = server.receive_once()?;
server.send_reliable("schema", b"after".to_vec())?;
let mut buffer = vec![0_u8; 65_535];
let mut delivered = Vec::new();
while let Ok((received, _)) = peer.recv_from(&mut buffer) {
let result = peer_session.receive(&decode_rudp_packet(&buffer[..received])?, 0)?;
delivered.extend(result.delivered.into_iter().map(|frame| frame.payload));
}
assert!(delivered.contains(&b"before".to_vec()));
assert!(
delivered.contains(&b"after".to_vec()),
"the frame sent after the duplicate Connect was dropped as a reused sequence"
);
Ok(())
}
/// A session that ended forgot its unacknowledged writes; a flush that then
/// reports success would claim writes the peer never acknowledged.
#[test]
fn a_write_unacknowledged_when_the_session_is_refused_fails_its_flush() -> Result<()> {
let server = socket()?;
let server_addr = server.local_addr()?;
let mut client = CultNetRudpSocketTransportConnection::new(
CultNetRudpSocketTransportOptions::client("client", socket()?, server_addr, CONNECTION_ID),
)?;
client.connect(b"hello".to_vec())?;
let mut buffer = vec![0_u8; 65_535];
let (received, client_addr) = server.recv_from(&mut buffer)?;
let mut server_session = raw_session(CONNECTION_ID);
let accept =
server_session.accept_connect(&decode_rudp_packet(&buffer[..received])?, 0, Vec::new())?;
send_to(&server, client_addr, &accept)?;
let _ = client.receive_once()?;
let receipt = client.send_reliable("schema", b"never acknowledged".to_vec())?;
send_to(&server, client_addr, &poisoned(&mut server_session)?)?;
let _ = client.receive_once()?;
assert!(client.disconnect_reason().is_some());
let flushed = client.flush_reliable(Duration::from_millis(200));
assert!(flushed.is_err(), "the write was never acknowledged");
assert_eq!(
client.reliable_send_status(&receipt),
CultNetRudpReliableSendStatus::Invalidated
);
Ok(())
}
/// A peer's Disconnect ends the session as surely as a refusal: writes it never
/// acknowledged are lost, and the transport says so.
#[test]
fn a_write_unacknowledged_when_the_peer_disconnects_fails_its_flush() -> Result<()> {
let server = socket()?;
let server_addr = server.local_addr()?;
let mut client = CultNetRudpSocketTransportConnection::new(
CultNetRudpSocketTransportOptions::client("client", socket()?, server_addr, CONNECTION_ID),
)?;
client.connect(b"hello".to_vec())?;
let mut buffer = vec![0_u8; 65_535];
let (received, client_addr) = server.recv_from(&mut buffer)?;
let mut server_session = raw_session(CONNECTION_ID);
let accept =
server_session.accept_connect(&decode_rudp_packet(&buffer[..received])?, 0, Vec::new())?;
send_to(&server, client_addr, &accept)?;
let _ = client.receive_once()?;
let receipt = client.send_reliable("schema", b"never acknowledged".to_vec())?;
send_to(&server, client_addr, &server_session.create_disconnect(b"bye".to_vec()))?;
let _ = client.receive_once()?;
assert!(client.flush_reliable(Duration::from_millis(200)).is_err());
assert_eq!(
client.reliable_send_status(&receipt),
CultNetRudpReliableSendStatus::Invalidated
);
Ok(())
}
/// Drains what a raw peer socket has been sent.
fn drain(socket: &UdpSocket) -> Result<Vec<CultNetRudpPacket>> {
let mut buffer = vec![0_u8; 65_535];
let mut packets = Vec::new();
while let Ok((received, _)) = socket.recv_from(&mut buffer) {
packets.push(decode_rudp_packet(&buffer[..received])?);
}
Ok(packets)
}
/// A receipt belongs to the session it was issued in. A peer's Disconnect ends
/// that session for good: reconnecting must not bring the write back to Pending,
/// and an ack the old peer sent late must not make it Acknowledged.
#[test]
fn a_receipt_invalidated_by_a_peer_disconnect_never_becomes_acknowledged() -> Result<()> {
let server = socket()?;
let server_addr = server.local_addr()?;
let mut client = CultNetRudpSocketTransportConnection::new(
CultNetRudpSocketTransportOptions::client("client", socket()?, server_addr, CONNECTION_ID),
)?;
client.connect(b"hello".to_vec())?;
let mut buffer = vec![0_u8; 65_535];
let (received, client_addr) = server.recv_from(&mut buffer)?;
let mut old_peer = raw_session(CONNECTION_ID);
let accept = old_peer.accept_connect(&decode_rudp_packet(&buffer[..received])?, 0, Vec::new())?;
send_to(&server, client_addr, &accept)?;
let _ = client.receive_once()?;
let receipt = client.send_reliable("schema", b"written before the disconnect".to_vec())?;
let data = drain(&server)?
.into_iter()
.find(|packet| packet.packet_type == CultNetRudpPacketType::Data)
.expect("the write reached the peer");
old_peer.receive(&data, 0)?;
let late_ack = old_peer.create_ack_for_received(data.sequence);
send_to(&server, client_addr, &old_peer.create_disconnect(b"bye".to_vec()))?;
let _ = client.receive_once()?;
assert_eq!(client.reliable_send_status(&receipt), CultNetRudpReliableSendStatus::Invalidated);
client.connect(b"hello again".to_vec())?;
assert_eq!(
client.reliable_send_status(&receipt),
CultNetRudpReliableSendStatus::Invalidated,
"reconnecting brought the write back to Pending"
);
let connect = drain(&server)?
.into_iter()
.find(|packet| packet.packet_type == CultNetRudpPacketType::Connect)
.expect("the reconnect reached the peer");
let mut new_peer = raw_session(CONNECTION_ID);
let new_accept = new_peer.accept_connect(&connect, 0, Vec::new())?;
send_to(&server, client_addr, &new_accept)?;
send_to(&server, client_addr, &late_ack)?;
while client.receive_once()?.is_some() {}
assert_eq!(
client.reliable_send_status(&receipt),
CultNetRudpReliableSendStatus::Invalidated,
"the old session's write was acknowledged in the new session"
);
// The old write is not retransmitted into the new session either: the
// Accept acknowledged the reconnect's Connect, and nothing else is owed.
assert_eq!(client.outstanding_reliable_packet_count(), 0, "the old write survived into the new session");
Ok(())
}
/// A server-mode transport that accepts a new Connect has a live session again;
/// the reason the previous one ended is history, not state.
#[test]
fn server_mode_forgets_the_end_reason_once_a_new_connect_is_accepted() -> Result<()> {
let server_socket = socket()?;
let server_addr = server_socket.local_addr()?;
let mut server = CultNetRudpSocketTransportConnection::new(
CultNetRudpSocketTransportOptions::server("server", server_socket, CONNECTION_ID),
)?;
let peer = socket()?;
let mut peer_session = raw_session(CONNECTION_ID);
send_to(&peer, server_addr, &peer_session.create_connect(0, b"peer".to_vec())?)?;
let _ = server.receive_once()?;
send_to(&peer, server_addr, &peer_session.create_disconnect(b"bye".to_vec()))?;
let _ = server.receive_once()?;
assert_eq!(server.disconnect_reason(), Some(&b"bye"[..]));
let mut next_peer = raw_session(CONNECTION_ID);
send_to(&peer, server_addr, &next_peer.create_connect(0, b"peer".to_vec())?)?;
let _ = server.receive_once()?;
assert!(server.connected());
assert_eq!(server.disconnect_reason(), None, "a stale reason survives the new session");
Ok(())
}
/// A flush waits on one session. A Connect from another endpoint replaces the
/// peer and forgets the writes being waited on; the flush must say so.
#[test]
fn a_flush_fails_when_a_new_endpoint_replaces_the_peer_mid_wait() -> Result<()> {
let server_socket = socket()?;
let server_addr = server_socket.local_addr()?;
let mut server = CultNetRudpSocketTransportConnection::new(
CultNetRudpSocketTransportOptions::server("server", server_socket, CONNECTION_ID),
)?;
let peer_a = socket()?;
let peer_b = socket()?;
send_to(&peer_a, server_addr, &raw_session(CONNECTION_ID).create_connect(0, b"a".to_vec())?)?;
let _ = server.receive_once()?;
server.send_reliable("schema", b"for A".to_vec())?;
send_to(&peer_b, server_addr, &raw_session(CONNECTION_ID).create_connect(0, b"b".to_vec())?)?;
let error = server.flush_reliable(Duration::from_millis(300)).expect_err("A's write was forgotten");
assert!(
error.to_string().contains("ended before its reliable writes were acknowledged"),
"{error}"
);
Ok(())
}
/// A repeated Connect still carries an acknowledgement field, and the session
/// honours it: the Accept it acknowledges is no longer owed.
#[test]
fn a_repeated_connect_acknowledges_what_it_carries() -> Result<()> {
let mut client = raw_session(CONNECTION_ID);
let mut server = raw_session(CONNECTION_ID);
let connect = client.create_connect(0, Vec::new())?;
let accept = server.accept_connect(&connect, 0, Vec::new())?;
let mut repeat = connect.clone();
repeat.ack = accept.sequence;
let reply = server.answer_repeated_connect(&repeat, 1)?;
assert_eq!(reply.packet_type, CultNetRudpPacketType::Ack);
assert_eq!(server.outstanding_reliable_packet_count(), 0);
Ok(())
}

112
packages/cultnet-ts/src/operation-service.ts
View file

@ -17,6 +17,10 @@ import {
import { CultNetPeer } from "./peer";
const DEFAULT_CONNECTION_ID = 0x43554c54;
/** Matches the document server's default session cap. A Connect past it is dropped. */
const MAX_OPERATION_SESSIONS = 64;
/** Matches the document server's default `session_idle_timeout` (30 s). */
const DEFAULT_SESSION_IDLE_TIMEOUT_MS = 30_000;
export interface CultNetOperationServerOptions {
runtimeId: string;
@ -24,12 +28,16 @@ export interface CultNetOperationServerOptions {
port?: number;
connectionId?: number;
maxFragmentBytes?: number;
/** A session that has sent nothing for this long is dropped, freeing its slot. Default 30 000. */
sessionIdleTimeoutMs?: number;
handler: (request: CultNetOperationRequestMessage) =>
CultNetOperationResponseMessage | Promise<CultNetOperationResponseMessage>;
}
export interface CultNetOperationServer {
readonly endpoint: string;
/** Datagrams read and discarded: malformed, unadmitted, refused by their session, or failed in handling. */
readonly packetsDropped: number;
close(): Promise<void>;
}
@ -43,6 +51,10 @@ export interface CultNetOperationClientOptions {
interface RemoteSession {
session: CultNetRudpSession;
remote: RemoteInfo;
/** Handlers running for this session; a session serving a request is not idle. */
handling: number;
/** When the last handler finished: the response still needs its acknowledgement. */
lastHandledAtMs: number;
}
export async function startCultNetOperationServer(
@ -57,11 +69,26 @@ export async function startCultNetOperationServer(
const wire = encodeRudpPacket(packet);
socket.send(wire, remote.port, remote.address);
};
let packetsDropped = 0;
socket.on("message", (wire, remote) => {
void handleServerDatagram(socket, sessions, connectionId, options, wire, remote, sendPacket);
// What a datagram carries is the sender's business: a rejection here must
// drop and count the packet, never become an unhandled rejection that ends
// the process.
handleServerDatagram(sessions, connectionId, options, wire, remote, sendPacket).then(
admitted => { if (!admitted) packetsDropped += 1; },
() => { packetsDropped += 1; },
);
});
await bindSocket(socket, options.port ?? 0, options.host ?? "127.0.0.1");
const idleTimeoutMs = options.sessionIdleTimeoutMs ?? DEFAULT_SESSION_IDLE_TIMEOUT_MS;
const resendTimer = setInterval(() => {
const now = Date.now();
for (const [key, peer] of sessions) {
// An abandoned client must not hold a slot forever, or the cap locks
// every new client out.
if (peer.handling > 0 || now - peer.lastHandledAtMs < idleTimeoutMs) continue;
if (peer.session.checkTimeout(now, idleTimeoutMs)) sessions.delete(key);
}
for (const peer of sessions.values()) {
for (const packet of peer.session.dueResends(Date.now())) sendPacket(peer.remote, packet);
}
@ -71,6 +98,7 @@ export async function startCultNetOperationServer(
const endpoint = `rudp://${address.address}:${address.port}`;
return {
endpoint,
get packetsDropped() { return packetsDropped; },
close: async () => {
clearInterval(resendTimer);
await closeSocket(socket);
@ -118,54 +146,96 @@ export async function invokeCultNetOperation(
}
}
/** Resolves false when the datagram was dropped without an error. */
async function handleServerDatagram(
socket: Socket,
sessions: Map<string, RemoteSession>,
connectionId: number,
options: CultNetOperationServerOptions,
wire: Buffer,
remote: RemoteInfo,
sendPacket: (remote: RemoteInfo, packet: CultNetRudpPacket) => void,
): Promise<void> {
): Promise<boolean> {
let packet: CultNetRudpPacket;
try {
packet = decodeRudpPacket(wire);
} catch {
return;
return false;
}
if (packet.connectionId !== connectionId) return;
if (packet.connectionId !== connectionId) return false;
const key = `${remote.address}:${remote.port}`;
let peer = sessions.get(key);
if (packet.packetType === "connect") {
peer = { session: new CultNetRudpSession({ connectionId, resendDelayMs: 25 }), remote };
sessions.set(key, peer);
if (!peer) {
if (sessions.size >= MAX_OPERATION_SESSIONS) return false;
peer = { session: new CultNetRudpSession({ connectionId, resendDelayMs: 25 }), remote, handling: 0, lastHandledAtMs: 0 };
sessions.set(key, peer);
}
// A Connect from an admitted peer repeats: the session answers with the
// Accept already owed and queues nothing.
sendPacket(remote, peer.session.acceptConnect(packet, Date.now(), encode("cultnet-operation-service")));
return;
return true;
}
if (!peer) return false;
let result: ReturnType<CultNetRudpSession["receive"]>;
try {
result = peer.session.receive(packet, Date.now());
} catch {
endSession(sessions, key, peer, sendPacket);
return false;
}
if (!peer) return;
const result = peer.session.receive(packet, Date.now());
if (result.reply) sendPacket(remote, result.reply);
for (const ready of result.readyToSend ?? []) sendPacket(remote, ready);
if (result.disconnected) {
sessions.delete(key);
return;
return true;
}
for (const frame of result.delivered) {
if (frame.channelId !== "schema") continue;
const message = parseCultNetMessage(decode(frame.payload));
if (message.schemaVersion !== "cultnet.operation_request.v0") continue;
const response = await options.handler(message);
const payload = encode(encodeCultNetMessageForWire(response, "cultnet.schema.v0"));
for (const responsePacket of peer.session.sendMany("schema", payload, {
reliable: true,
ordered: true,
nowMs: Date.now(),
maxFragmentBytes: options.maxFragmentBytes ?? 2048,
})) sendPacket(remote, responsePacket);
try {
const message = parseCultNetMessage(decode(frame.payload));
if (message.schemaVersion !== "cultnet.operation_request.v0") continue;
peer.handling += 1;
let response: CultNetOperationResponseMessage;
try {
response = await options.handler(message);
} finally {
peer.handling -= 1;
peer.lastHandledAtMs = Date.now();
}
const payload = encode(encodeCultNetMessageForWire(response, "cultnet.schema.v0"));
for (const responsePacket of peer.session.sendMany("schema", payload, {
reliable: true,
ordered: true,
nowMs: Date.now(),
maxFragmentBytes: options.maxFragmentBytes ?? 2048,
})) sendPacket(remote, responsePacket);
} catch {
// The session recorded this request's sequence, so keeping it would
// acknowledge the retransmit of a request that was never handled.
endSession(sessions, key, peer, sendPacket);
return false;
}
}
if (packet.packetType === "data" || result.delivered.length > 0) {
sendPacket(remote, peer.session.createAckForReceived(packet.sequence));
}
return true;
}
/**
* Ends a session that took a packet it could not serve, and tells the peer. The
* reset comes first, or the goodbye's ack field would acknowledge the very
* packet that was refused. A session already replaced under the same key is left.
*/
function endSession(
sessions: Map<string, RemoteSession>,
key: string,
peer: RemoteSession,
sendPacket: (remote: RemoteInfo, packet: CultNetRudpPacket) => void,
): void {
if (sessions.get(key) === peer) sessions.delete(key);
peer.session.resetPeerState();
sendPacket(peer.remote, peer.session.createDisconnect(Buffer.from("session refused a packet", "utf8")));
}
function parseRudpEndpoint(endpoint: string): { host: string; port: number } {

149
packages/cultnet-ts/src/rudp.ts
View file

@ -170,6 +170,9 @@ export class CultNetRudpSession {
constructor(options: CultNetRudpSessionOptions) {
this.connectionId = toUint32(options.connectionId, "connectionId");
this.#nextSequence = toUint32(options.initialSequence ?? 1, "initialSequence");
if (this.#nextSequence === 0xffff_ffff) {
throw new Error("RUDP initialSequence must leave room for a reliable packet.");
}
this.resendDelayMs = options.resendDelayMs ?? 250;
if (options.maxPendingReliablePackets !== undefined && options.maxPendingReliablePackets <= 0) {
throw new Error("RUDP maxPendingReliablePackets must be greater than zero.");
@ -242,16 +245,9 @@ export class CultNetRudpSession {
throw new Error(`Expected RUDP connect packet, got ${packet.packetType}.`);
}
const pendingAccept = [...this.#pendingReliable.values()]
.find(pending => pending.packet.packetType === "accept");
if (pendingAccept) {
this.#rememberReceived(packet.sequence);
this.#lastReceivedAtMs = nowMs;
pendingAccept.lastSentAtMs = nowMs;
return {
...pendingAccept.packet,
payload: new Uint8Array(pendingAccept.packet.payload ?? new Uint8Array()),
};
// A Connect from a peer this session already accepted repeats.
if (this.#connected) {
return this.answerRepeatedConnect(packet, nowMs);
}
this.#ensureReliableCapacity(1);
this.#rememberReceived(packet.sequence);
@ -268,6 +264,32 @@ export class CultNetRudpSession {
return response;
}
/**
* Answers a Connect from a peer this session has already accepted. The packet's
* sequence is remembered and nothing is queued, so a Connect storm cannot grow
* the reliable queue. The reply is the Accept still awaiting acknowledgement,
* or an Ack once it was acknowledged.
*/
answerRepeatedConnect(packet: CultNetRudpPacket, nowMs = 0): CultNetRudpPacket {
this.#requireConnection(packet);
if (packet.packetType !== "connect") {
throw new Error(`Expected RUDP connect packet, got ${packet.packetType}.`);
}
this.#applyAcknowledgements(packet);
this.#rememberReceived(packet.sequence);
this.#lastReceivedAtMs = nowMs;
const pendingAccept = [...this.#pendingReliable.values()]
.find(pending => pending.packet.packetType === "accept");
if (!pendingAccept) {
return this.createAck();
}
pendingAccept.lastSentAtMs = nowMs;
return {
...pendingAccept.packet,
payload: new Uint8Array(pendingAccept.packet.payload ?? new Uint8Array()),
};
}
send(
channelId: string,
payload: Uint8Array,
@ -802,11 +824,19 @@ export class CultNetRudpSocketTransportConnection extends EventEmitter implement
#remoteHost: string | undefined;
#remotePort: number | undefined;
#closed = false;
// A flush belongs to the session it started in. Each session start advances
// the generation; each session end records the generation it ended and why.
// A reconnect from a disconnect listener starts a new generation but cannot
// un-end the old one.
#generation = 0;
#endedGeneration = -1;
#endedReason: Uint8Array | undefined;
readonly #stats: CultNetTransportStats = {
bytesReceived: 0,
bytesSent: 0,
framesReceived: 0,
framesSent: 0,
packetsDropped: 0,
};
constructor(options: CultNetRudpSocketTransportOptions) {
@ -862,6 +892,7 @@ export class CultNetRudpSocketTransportConnection extends EventEmitter implement
if (this.#mode !== "client") {
throw new Error("Only a client RUDP socket transport can initiate connect.");
}
this.#generation += 1;
this.#sendPacket(this.#session.createConnect(Date.now(), payload));
}
@ -879,7 +910,18 @@ export class CultNetRudpSocketTransportConnection extends EventEmitter implement
async flush(timeoutMs = 30_000): Promise<void> {
const deadline = Date.now() + Math.max(0, timeoutMs);
while (this.#session.outstandingReliablePacketCount > 0) {
const generation = this.#generation;
for (;;) {
// An ended session forgot its unacknowledged writes; reporting them
// flushed would be a lie, even if a new session has since begun.
if (this.#endedGeneration >= generation) {
throw new Error(
`RUDP session ended before its reliable writes were acknowledged: ${Buffer.from(this.#endedReason ?? new Uint8Array()).toString("utf8")}`,
);
}
if (this.#session.outstandingReliablePacketCount === 0) {
return;
}
if (this.#closed) {
throw new Error("RUDP transport closed before reliable packets were acknowledged.");
}
@ -892,6 +934,11 @@ export class CultNetRudpSocketTransportConnection extends EventEmitter implement
}
}
#endGeneration(reason: Uint8Array): void {
this.#endedGeneration = this.#generation;
this.#endedReason = reason;
}
ping(payload = new Uint8Array()): void {
this.#sendPacket(this.#session.createPing(payload));
}
@ -899,6 +946,7 @@ export class CultNetRudpSocketTransportConnection extends EventEmitter implement
checkTimeout(timeoutMs: number, nowMs = Date.now()): boolean {
const timedOut = this.#session.checkTimeout(nowMs, timeoutMs);
if (timedOut) {
this.#endGeneration(Buffer.from("session timed out", "utf8"));
this.emit("timeout");
this.emit("close");
}
@ -920,36 +968,102 @@ export class CultNetRudpSocketTransportConnection extends EventEmitter implement
}
}
#endRefusedSession(): void {
const reason = Buffer.from("session refused a packet", "utf8");
// The goodbye is built after the reset, or its ack field would acknowledge
// the very frame the session refused.
this.#session.resetPeerState();
this.#endGeneration(reason);
try {
this.#sendPacket(this.#session.createDisconnect(reason));
} catch {
// Best-effort: the session ends whether or not the peer hears it.
}
if (this.#mode === "server") {
// Only a Connect can claim the endpoint again.
this.#remoteHost = undefined;
this.#remotePort = undefined;
}
this.emit("disconnect", { reason });
this.emit("close");
}
#receiveDatagram(wire: Buffer, remote: RemoteInfo): void {
this.#stats.bytesReceived += wire.length;
// What a datagram carries is the sender's business, not a fault of this
// process: a malformed frame, another session's connection id and a packet
// the session refuses are dropped and counted, never emitted as "error"
// (an "error" with no listener throws and ends the process). The id is
// checked before the peer endpoint or session state is touched, so a stray
// Connect can neither move the endpoint nor reset the session.
let packet: CultNetRudpPacket;
try {
packet = decodeRudpPacket(wire);
} catch (error) {
this.emit("error", error instanceof Error ? error : new Error(String(error)));
} catch {
this.#stats.packetsDropped += 1;
return;
}
if (packet.connectionId !== this.#session.connectionId) {
this.#stats.packetsDropped += 1;
return;
}
let movedEndpoint = false;
if (!this.#remoteHost || this.#remotePort === undefined) {
if (this.#mode === "server" && packet.packetType !== "connect") {
this.#stats.packetsDropped += 1;
return;
}
this.#remoteHost = remote.address;
this.#remotePort = remote.port;
} else if (remote.address !== this.#remoteHost || remote.port !== this.#remotePort) {
if (this.#mode === "server" && packet.packetType === "connect") {
movedEndpoint = true;
this.#remoteHost = remote.address;
this.#remotePort = remote.port;
} else {
this.#stats.packetsDropped += 1;
return;
}
}
try {
if (this.#mode === "server" && packet.packetType === "connect") {
if (this.#mode === "server" && packet.packetType === "connect") {
// Only a Connect from the endpoint already accepted repeats, and the
// session answers it with the Accept owed. A Connect from another
// endpoint replaces the peer: the old session's writes die with it.
const repeated = this.#session.connected && !movedEndpoint;
if (!repeated) {
// A timed-out session already ended its generation; any other
// predecessor ends here, connected or not.
if (this.#endedGeneration < this.#generation) {
this.#endGeneration(Buffer.from("replaced by a new Connect", "utf8"));
}
this.#session.resetPeerState();
this.#sendPacket(this.#session.acceptConnect(packet, Date.now()));
}
let accept: CultNetRudpPacket;
try {
accept = this.#session.acceptConnect(packet, Date.now());
if (!repeated) this.#generation += 1;
} catch {
this.#stats.packetsDropped += 1;
return;
}
this.#sendPacket(accept);
return;
}
const result = this.#session.receive(packet, Date.now());
let result: ReturnType<CultNetRudpSession["receive"]>;
try {
result = this.#session.receive(packet, Date.now());
} catch {
// receive() has already recorded the packet's reliable sequence, so the
// session cannot be kept: a retransmit would be acknowledged and the
// frame silently lost. End it and tell the peer.
this.#stats.packetsDropped += 1;
this.#endRefusedSession();
return;
}
try {
if (result.reply) {
this.#sendPacket(result.reply);
}
@ -967,6 +1081,7 @@ export class CultNetRudpSocketTransportConnection extends EventEmitter implement
} satisfies CultNetTransportFrame);
}
if (result.disconnected) {
this.#endGeneration(result.disconnectReason ?? new Uint8Array());
this.emit("disconnect", { reason: result.disconnectReason ?? new Uint8Array() });
this.emit("close");
return;

3
packages/cultnet-ts/src/transport.ts
View file

@ -14,6 +14,8 @@ export interface CultNetTransportStats {
bytesSent: number;
framesReceived: number;
framesSent: number;
/** Datagrams read and discarded because they belong to no session on this transport. */
packetsDropped: number;
}
export interface CultNetReconnectPolicy {
@ -192,6 +194,7 @@ export class TcpFramedTransportConnection extends EventEmitter implements CultNe
bytesSent: 0,
framesReceived: 0,
framesSent: 0,
packetsDropped: 0,
};
constructor(stream: Duplex, profile: CultNetTransportProfile) {

623
packages/cultnet-ts/test/cultnet.test.ts
View file

@ -8,6 +8,7 @@ import { join } from "node:path";
import { tmpdir } from "node:os";
import { z } from "zod";
import { encode } from "@msgpack/msgpack";
import {
CultCache,
SingleFileMessagePackBackingStore,
@ -38,6 +39,9 @@ import {
createRudpTransportProfile,
defineCultNetDocumentBinding,
decodeRudpPacket,
type CultNetOperationRequestMessage,
type CultNetOperationResponseMessage,
type CultNetRudpPacket,
encodeCultNetMessageForWire,
encodeRudpPacket,
ghostlightAgentStateGeneratedContract,
@ -1867,3 +1871,622 @@ test("rudp receiver evicts stranded fragment sets instead of refusing new ones",
assert.equal(second.delivered.length, 1);
assert.equal(Buffer.from(second.delivered[0]!.payload).toString(), "hello");
});
test("rudp socket transport drops strays without an error event and keeps serving its peer", async () => {
const serverSocket = await bindUdpSocket();
const clientSocket = await bindUdpSocket();
const strayHost = await bindUdpSocket();
const connectionId = 0x10203050;
const server = new CultNetRudpSocketTransportConnection({
runtimeId: "rudp-server",
socket: serverSocket,
mode: "server",
connectionId,
resendPollMs: 5,
});
const client = new CultNetRudpSocketTransportConnection({
runtimeId: "rudp-client",
socket: clientSocket,
mode: "client",
remoteHost: "127.0.0.1",
remotePort: udpPort(serverSocket),
connectionId,
resendPollMs: 5,
});
try {
// No "error" listener on either side: an unhandled "error" event throws
// and would end the process, which is what a stray must never do.
const frames: string[] = [];
server.on("frame", (frame) => frames.push(Buffer.from(frame.payload).toString("utf8")));
client.connect();
await waitFor(() => client.connected && server.connected, "RUDP socket handshake");
const foreign = new CultNetRudpSession({ connectionId: 0x0badf10e });
const strays = [
Buffer.from("not a rudp packet"),
Buffer.alloc(0),
Buffer.from(encodeRudpPacket(foreign.createConnect(0, Buffer.from("foreign")))),
];
for (const stray of strays) {
// From an unrelated address, and from the peer's own address.
strayHost.send(stray, udpPort(serverSocket), "127.0.0.1");
clientSocket.send(stray, udpPort(serverSocket), "127.0.0.1");
}
await waitFor(() => server.stats.packetsDropped >= strays.length * 2, "strays counted");
client.send("schema", Buffer.from("still here", "utf8"));
await waitFor(() => frames.length === 1, "the real frame");
assert.deepEqual(frames, ["still here"]);
// The client side too: a moved flow answers from the server address.
for (const stray of strays) {
serverSocket.send(stray, udpPort(clientSocket), "127.0.0.1");
}
await waitFor(() => client.stats.packetsDropped >= strays.length, "client strays counted");
const clientFrames: string[] = [];
client.on("frame", (frame) => clientFrames.push(Buffer.from(frame.payload).toString("utf8")));
server.send("schema", Buffer.from("server still here", "utf8"));
await waitFor(() => clientFrames.length === 1, "the real server frame");
} finally {
strayHost.close();
client.close();
server.close();
}
});
/** A reliable frame no session accepts: a fragment set with no fragment id. */
function poisonedFrame(session: CultNetRudpSession): CultNetRudpPacket {
const [packet] = session.sendMany("schema", Buffer.from("poison"), { reliable: true, ordered: true, nowMs: 0 });
return { ...packet!, fragmentCount: 2, fragmentId: 0, fragmentIndex: 0 };
}
test("rudp socket transport ends the session that refuses a reliable frame and never acknowledges it", async () => {
const serverSocket = await bindUdpSocket();
const peerSocket = await bindUdpSocket();
const connectionId = 0x10203051;
const server = new CultNetRudpSocketTransportConnection({
runtimeId: "rudp-server",
socket: serverSocket,
mode: "server",
connectionId,
resendPollMs: 1000,
});
const received: CultNetRudpPacket[] = [];
peerSocket.on("message", (wire) => received.push(decodeRudpPacket(wire)));
const peer = new CultNetRudpSession({ connectionId });
const toServer = (packet: CultNetRudpPacket) =>
peerSocket.send(encodeRudpPacket(packet), udpPort(serverSocket), "127.0.0.1");
const disconnects: string[] = [];
server.on("disconnect", ({ reason }) => disconnects.push(Buffer.from(reason).toString("utf8")));
try {
toServer(peer.createConnect(0));
await waitFor(() => received.some((p) => p.packetType === "accept"), "the Accept");
peer.receive(received.find((p) => p.packetType === "accept")!, 0);
const poison = poisonedFrame(peer);
toServer(poison);
await waitFor(() => disconnects.length === 1, "the server ending the session");
await waitFor(() => received.some((p) => p.packetType === "disconnect"), "the goodbye");
assert.equal(server.stats.packetsDropped, 1);
assert.equal(server.connected, false);
// The peer sees the session end and never sees the frame acknowledged,
// neither in the goodbye nor after it retransmits.
const goodbye = received.find((p) => p.packetType === "disconnect")!;
assert.equal(peer.receive(goodbye, 1).disconnected, true);
assert.ok(peer.pendingReliableSequences.includes(poison.sequence), "the refused frame was acknowledged");
const before = received.length;
toServer(poison);
await waitFor(() => server.stats.packetsDropped === 2, "the retransmit dropped");
await new Promise((resolve) => setTimeout(resolve, 30));
assert.equal(received.length, before, "the server must not answer a retransmit for an ended session");
// Only a Connect claims the endpoint again.
const again = new CultNetRudpSession({ connectionId });
toServer(again.createConnect(0));
await waitFor(() => received.length > before, "the new Accept");
assert.ok(received.slice(before).some((p) => p.packetType === "accept"));
} finally {
peerSocket.close();
server.close();
}
});
test("rudp sessions reject sequence spaces and queues that cannot admit a Connect", () => {
assert.throws(() => new CultNetRudpSession({ connectionId: 1, initialSequence: 0xffff_ffff }), /initialSequence/);
assert.throws(() => new CultNetRudpSession({ connectionId: 1, maxPendingReliablePackets: 0 }), /greater than zero/);
assert.doesNotThrow(() => new CultNetRudpSession({ connectionId: 1, initialSequence: 0xffff_fffe, maxPendingReliablePackets: 1 }));
});
test("operation service drops and counts a packet its session refuses and keeps serving", async () => {
const handler = (request: CultNetOperationRequestMessage): CultNetOperationResponseMessage => ({
schemaVersion: "cultnet.operation_response.v0",
messageId: request.messageId,
serviceId: request.serviceId,
operation: request.operation,
status: "ok",
payloadSchema: "gamecult.eve.plugin_abi.response.v1",
payloadEncoding: "messagepack-base64",
payload: request.payload,
diagnostics: [],
sourceRuntimeId: "sai-sidecar",
});
const server = await startCultNetOperationServer({ runtimeId: "sai-sidecar", handler });
const raw = await bindUdpSocket();
const port = Number(new URL(server.endpoint).port);
const connectionId = 0x43554c54;
const received: CultNetRudpPacket[] = [];
raw.on("message", (wire) => received.push(decodeRudpPacket(wire)));
const peer = new CultNetRudpSession({ connectionId });
const toServer = (packet: CultNetRudpPacket) => raw.send(encodeRudpPacket(packet), port, "127.0.0.1");
try {
toServer(peer.createConnect(0));
await waitFor(() => received.some((p) => p.packetType === "accept"), "the Accept");
peer.receive(received[0]!, 0);
toServer(poisonedFrame(peer));
await waitFor(() => server.packetsDropped === 1, "the poisoned frame dropped");
await waitFor(() => received.some((p) => p.packetType === "disconnect"), "the session ended");
// A frame the session accepts but the service cannot handle rejects inside
// the handler; that too is a dropped packet, not a dead process.
const garbage = new CultNetRudpSession({ connectionId });
received.length = 0;
toServer(garbage.createConnect(0));
await waitFor(() => received.some((p) => p.packetType === "accept"), "the second Accept");
garbage.receive(received[0]!, 0);
const [notMessagePack] = garbage.sendMany("schema", Buffer.from([0xc1]), { reliable: true, ordered: true, nowMs: 0 });
toServer(notMessagePack!);
await waitFor(() => server.packetsDropped === 2, "the unhandled frame dropped");
// The session that could not serve it ends and the client is told, so its
// retransmit is not acknowledged for a request that was never handled.
await waitFor(() => received.some((p) => p.packetType === "disconnect"), "the goodbye");
const beforeRetransmit = received.length;
toServer(notMessagePack!);
await waitFor(() => server.packetsDropped === 3, "the retransmit dropped");
await new Promise((resolve) => setTimeout(resolve, 30));
assert.equal(received.length, beforeRetransmit, "the retransmit of an unhandled request was answered");
const response = await invokeCultNetOperation(server.endpoint, {
schemaVersion: "cultnet.operation_request.v0",
messageId: "after-poison",
serviceId: "sai.vn",
operation: "describe",
payloadSchema: "gamecult.eve.plugin_abi.request.v1",
payloadEncoding: "messagepack-base64",
payload: "gaZzY2hlbWE=",
}, { runtimeId: "eve-test" });
assert.equal(response.messageId, "after-poison");
} finally {
raw.close();
await server.close();
}
});
test("operation service drops a Connect once its session table is full", async () => {
const server = await startCultNetOperationServer({
runtimeId: "sai-sidecar",
handler: () => { throw new Error("no requests expected"); },
});
const port = Number(new URL(server.endpoint).port);
const sockets: Socket[] = [];
try {
for (let index = 0; index < 66; index += 1) {
const socket = await bindUdpSocket();
sockets.push(socket);
socket.send(encodeRudpPacket(new CultNetRudpSession({ connectionId: 0x43554c54 }).createConnect(0)), port, "127.0.0.1");
}
await waitFor(() => server.packetsDropped === 2, "the Connects past the cap dropped");
await new Promise((resolve) => setTimeout(resolve, 30));
assert.equal(server.packetsDropped, 2);
} finally {
for (const socket of sockets) socket.close();
await server.close();
}
});
test("rudp flush fails for a write the ended session forgot", async () => {
const serverSocket = await bindUdpSocket();
const peerSocket = await bindUdpSocket();
const connectionId = 0x10203056;
const server = new CultNetRudpSocketTransportConnection({
runtimeId: "rudp-server",
socket: serverSocket,
mode: "server",
connectionId,
resendPollMs: 1000,
});
const received: CultNetRudpPacket[] = [];
peerSocket.on("message", (wire) => received.push(decodeRudpPacket(wire)));
const peer = new CultNetRudpSession({ connectionId });
const toServer = (packet: CultNetRudpPacket) =>
peerSocket.send(encodeRudpPacket(packet), udpPort(serverSocket), "127.0.0.1");
try {
toServer(peer.createConnect(0));
await waitFor(() => received.some((p) => p.packetType === "accept"), "the Accept");
peer.receive(received.find((p) => p.packetType === "accept")!, 0);
server.send("schema", Buffer.from("never acknowledged"));
toServer(poisonedFrame(peer));
await waitFor(() => server.stats.packetsDropped === 1, "the refusal");
await assert.rejects(server.flush(200), /ended before its reliable writes were acknowledged/);
} finally {
peerSocket.close();
server.close();
}
});
test("a Connect that repeats after its Accept was acknowledged is answered with an Ack and queues nothing", () => {
const connectionId = 0x10203057;
const client = new CultNetRudpSession({ connectionId });
const server = new CultNetRudpSession({ connectionId });
const connect = client.createConnect(0);
const accept = server.acceptConnect(connect, 10);
client.receive(accept, 11);
server.receive(client.createAckForReceived(accept.sequence), 12);
assert.equal(server.outstandingReliablePacketCount, 0);
for (let attempt = 0; attempt < 20; attempt += 1) {
assert.equal(server.acceptConnect(connect, 20 + attempt).packetType, "ack");
}
assert.equal(server.outstandingReliablePacketCount, 0);
});
test("server-mode transport and operation service owe one reliable Accept however many Connects repeat", async () => {
const connectionId = 0x43554c54;
const serverSocket = await bindUdpSocket();
const transport = new CultNetRudpSocketTransportConnection({
runtimeId: "rudp-server",
socket: serverSocket,
mode: "server",
connectionId,
resendPollMs: 1000,
});
const service = await startCultNetOperationServer({
runtimeId: "sai-sidecar",
handler: () => { throw new Error("no requests expected"); },
});
const raw = await bindUdpSocket();
let received: CultNetRudpPacket[] = [];
raw.on("message", (wire) => received.push(decodeRudpPacket(wire)));
const settle = () => new Promise((resolve) => setTimeout(resolve, 100));
try {
for (const port of [udpPort(serverSocket), Number(new URL(service.endpoint).port)]) {
const peer = new CultNetRudpSession({ connectionId });
const connect = peer.createConnect(0);
received = [];
for (let attempt = 0; attempt < 20; attempt += 1) raw.send(encodeRudpPacket(connect), port, "127.0.0.1");
await settle();
const accepts = received.filter((p) => p.packetType === "accept");
assert.equal(new Set(accepts.map((p) => p.sequence)).size, 1, `reliable Accepts sent by port ${port}`);
// Once the Accept is acknowledged, a repeated Connect is answered with an Ack, never a new Accept.
peer.receive(accepts[0]!, 0);
raw.send(encodeRudpPacket(peer.createAckForReceived(accepts[0]!.sequence)), port, "127.0.0.1");
await settle();
received = [];
for (let attempt = 0; attempt < 20; attempt += 1) raw.send(encodeRudpPacket(connect), port, "127.0.0.1");
await settle();
assert.equal(received.filter((p) => p.packetType === "accept").length, 0, `Accepts sent by port ${port} after the ack`);
assert.ok(received.some((p) => p.packetType === "ack"), `an Ack from port ${port}`);
}
assert.equal(transport.outstandingReliablePacketCount, 0);
} finally {
raw.close();
transport.close();
await service.close();
}
});
test("operation service frees the slots of abandoned sessions once they idle out", async () => {
const server = await startCultNetOperationServer({
runtimeId: "sai-sidecar",
sessionIdleTimeoutMs: 300,
handler: (request): CultNetOperationResponseMessage => ({
schemaVersion: "cultnet.operation_response.v0",
messageId: request.messageId,
serviceId: request.serviceId,
operation: request.operation,
status: "ok",
payloadSchema: "gamecult.eve.plugin_abi.response.v1",
payloadEncoding: "messagepack-base64",
payload: request.payload,
diagnostics: [],
sourceRuntimeId: "sai-sidecar",
}),
});
const port = Number(new URL(server.endpoint).port);
const sockets: Socket[] = [];
const request: CultNetOperationRequestMessage = {
schemaVersion: "cultnet.operation_request.v0",
messageId: "after-abandonment",
serviceId: "sai.vn",
operation: "describe",
payloadSchema: "gamecult.eve.plugin_abi.request.v1",
payloadEncoding: "messagepack-base64",
payload: "gaZzY2hlbWE=",
};
try {
for (let index = 0; index < 64; index += 1) {
const socket = await bindUdpSocket();
sockets.push(socket);
socket.send(encodeRudpPacket(new CultNetRudpSession({ connectionId: 0x43554c54 }).createConnect(0)), port, "127.0.0.1");
}
await new Promise((resolve) => setTimeout(resolve, 60));
await assert.rejects(invokeCultNetOperation(server.endpoint, request, { runtimeId: "eve-test", timeoutMs: 100 }));
await new Promise((resolve) => setTimeout(resolve, 400));
const response = await invokeCultNetOperation(server.endpoint, request, { runtimeId: "eve-test" });
assert.equal(response.messageId, "after-abandonment");
} finally {
for (const socket of sockets) socket.close();
await server.close();
}
});
test("server-mode transport replaces its peer when a Connect arrives from another endpoint", async () => {
const serverSocket = await bindUdpSocket();
const socketA = await bindUdpSocket();
const socketB = await bindUdpSocket();
const connectionId = 0x10203058;
const server = new CultNetRudpSocketTransportConnection({
runtimeId: "rudp-server",
socket: serverSocket,
mode: "server",
connectionId,
resendPollMs: 1000,
});
const receivedByA: CultNetRudpPacket[] = [];
socketA.on("message", (wire) => receivedByA.push(decodeRudpPacket(wire)));
const receivedByB: CultNetRudpPacket[] = [];
socketB.on("message", (wire) => receivedByB.push(decodeRudpPacket(wire)));
const frames: string[] = [];
server.on("frame", (frame: { payload: Uint8Array }) => frames.push(Buffer.from(frame.payload).toString("utf8")));
const peerA = new CultNetRudpSession({ connectionId });
const peerB = new CultNetRudpSession({ connectionId });
const send = (socket: Socket, packet: CultNetRudpPacket) =>
socket.send(encodeRudpPacket(packet), udpPort(serverSocket), "127.0.0.1");
try {
send(socketA, peerA.createConnect(0));
await waitFor(() => receivedByA.some((p) => p.packetType === "accept"), "A's Accept");
const acceptA = receivedByA.find((p) => p.packetType === "accept")!;
peerA.receive(acceptA, 0);
send(socketA, peerA.createAckForReceived(acceptA.sequence));
await waitFor(() => server.outstandingReliablePacketCount === 0, "A's Accept acknowledged");
send(socketB, peerB.createConnect(0));
await waitFor(() => receivedByB.some((p) => p.packetType === "accept"), "B's Accept");
peerB.receive(receivedByB.find((p) => p.packetType === "accept")!, 0);
// B owns the session; A's traffic is a stranger's and is dropped.
const [fromA] = peerA.sendMany("schema", Buffer.from("from-A"), { reliable: true, ordered: true, nowMs: 0 });
send(socketA, fromA!);
const [fromB] = peerB.sendMany("schema", Buffer.from("from-B"), { reliable: true, ordered: true, nowMs: 0 });
send(socketB, fromB!);
await waitFor(() => frames.length > 0, "B's frame");
await new Promise((resolve) => setTimeout(resolve, 30));
assert.deepEqual(frames, ["from-B"]);
assert.equal(server.stats.packetsDropped, 1);
} finally {
socketA.close();
socketB.close();
assert.doesNotThrow(() => server.close());
}
});
test("operation service does not idle out a session while its handler is running", async () => {
const respond = (request: CultNetOperationRequestMessage): CultNetOperationResponseMessage => ({
schemaVersion: "cultnet.operation_response.v0",
messageId: request.messageId,
serviceId: request.serviceId,
operation: request.operation,
status: "ok",
payloadSchema: "gamecult.eve.plugin_abi.response.v1",
payloadEncoding: "messagepack-base64",
payload: request.payload,
diagnostics: [],
sourceRuntimeId: "sai-sidecar",
});
const server = await startCultNetOperationServer({
runtimeId: "sai-sidecar",
sessionIdleTimeoutMs: 200,
handler: async (request) => {
await new Promise((resolve) => setTimeout(resolve, 700));
return respond(request);
},
});
try {
const response = await invokeCultNetOperation(server.endpoint, {
schemaVersion: "cultnet.operation_request.v0",
messageId: "slow",
serviceId: "sai.vn",
operation: "describe",
payloadSchema: "gamecult.eve.plugin_abi.request.v1",
payloadEncoding: "messagepack-base64",
payload: "gaZzY2hlbWE=",
}, { runtimeId: "eve-test", timeoutMs: 3000 });
assert.equal(response.messageId, "slow");
} finally {
await server.close();
}
});
test("a flush is bound to the session it started in, even when a disconnect listener reconnects", async () => {
const peerSocket = await bindUdpSocket();
const clientSocket = await bindUdpSocket();
const connectionId = 0x10203059;
const client = new CultNetRudpSocketTransportConnection({
runtimeId: "rudp-client",
socket: clientSocket,
mode: "client",
remoteHost: "127.0.0.1",
remotePort: udpPort(peerSocket),
connectionId,
resendDelayMs: 1000,
resendPollMs: 1000,
});
const received: CultNetRudpPacket[] = [];
let clientPort = 0;
peerSocket.on("message", (wire, remote) => {
clientPort = remote.port;
received.push(decodeRudpPacket(wire));
});
const peer = new CultNetRudpSession({ connectionId });
try {
client.connect();
await waitFor(() => received.some((p) => p.packetType === "connect"), "the Connect");
const accept = peer.acceptConnect(received.find((p) => p.packetType === "connect")!, 0);
peerSocket.send(encodeRudpPacket(accept), clientPort, "127.0.0.1");
await waitFor(() => client.connected, "the client connected");
client.send("schema", Buffer.from("never acknowledged"));
client.on("disconnect", () => client.connect());
const flushed = client.flush(500);
const outcome = assert.rejects(flushed, /ended before its reliable writes were acknowledged/);
peerSocket.send(encodeRudpPacket(peer.createDisconnect(Buffer.from("bye"))), clientPort, "127.0.0.1");
await outcome;
} finally {
peerSocket.close();
client.close();
}
});
test("a repeated Connect acknowledges what it carries", () => {
const connectionId = 0x1020305a;
const client = new CultNetRudpSession({ connectionId });
const server = new CultNetRudpSession({ connectionId });
const connect = client.createConnect(0);
const accept = server.acceptConnect(connect, 0);
const reply = server.answerRepeatedConnect({ ...connect, ack: accept.sequence }, 1);
assert.equal(reply.packetType, "ack");
assert.equal(server.outstandingReliablePacketCount, 0);
});
async function timedOutServerWithUnackedWrite(connectionId: number) {
const serverSocket = await bindUdpSocket();
const socketA = await bindUdpSocket();
const socketB = await bindUdpSocket();
const server = new CultNetRudpSocketTransportConnection({
runtimeId: "rudp-server",
socket: serverSocket,
mode: "server",
connectionId,
resendDelayMs: 1000,
resendPollMs: 1000,
});
const receivedByA: CultNetRudpPacket[] = [];
socketA.on("message", (wire) => receivedByA.push(decodeRudpPacket(wire)));
const peerA = new CultNetRudpSession({ connectionId });
const send = (socket: Socket, packet: CultNetRudpPacket) =>
socket.send(encodeRudpPacket(packet), udpPort(serverSocket), "127.0.0.1");
send(socketA, peerA.createConnect(0));
await waitFor(() => receivedByA.some((p) => p.packetType === "accept"), "A's Accept");
const acceptA = receivedByA.find((p) => p.packetType === "accept")!;
peerA.receive(acceptA, 0);
send(socketA, peerA.createAckForReceived(acceptA.sequence));
await waitFor(() => server.outstandingReliablePacketCount === 0, "A's Accept acknowledged");
server.send("schema", Buffer.from("never acknowledged"));
assert.ok(server.outstandingReliablePacketCount > 0);
const flushed = server.flush(2000);
const outcome = assert.rejects(flushed, /ended before its reliable writes were acknowledged/);
// The flush must have started before the timeout fires.
await new Promise((resolve) => setTimeout(resolve, 20));
assert.equal(server.checkTimeout(50, Date.now() + 10_000), true);
return {
server, socketA, socketB, outcome, send,
close: () => {
socketA.close();
socketB.close();
server.close();
},
};
}
test("a flush waiting on an unacknowledged write fails as soon as the session times out", async () => {
const t = await timedOutServerWithUnackedWrite(0x1020305a);
try {
await t.outcome;
} finally {
t.close();
}
});
test("a flush fails when a Connect from another endpoint follows a timeout", async () => {
const t = await timedOutServerWithUnackedWrite(0x1020305b);
try {
t.send(t.socketB, new CultNetRudpSession({ connectionId: 0x1020305b }).createConnect(0));
await t.outcome;
} finally {
t.close();
}
});
test("a flush fails when a Connect from the same endpoint follows a timeout", async () => {
const t = await timedOutServerWithUnackedWrite(0x1020305c);
try {
t.send(t.socketA, new CultNetRudpSession({ connectionId: 0x1020305c }).createConnect(0));
await t.outcome;
} finally {
t.close();
}
});
test("operation service sweeps a session that ran a handler once it idles out", async () => {
const server = await startCultNetOperationServer({
runtimeId: "sai-sidecar",
sessionIdleTimeoutMs: 300,
handler: (request): CultNetOperationResponseMessage => ({
schemaVersion: "cultnet.operation_response.v0",
messageId: request.messageId,
serviceId: request.serviceId,
operation: request.operation,
status: "ok",
payloadSchema: "gamecult.eve.plugin_abi.response.v1",
payloadEncoding: "messagepack-base64",
payload: request.payload,
diagnostics: [],
sourceRuntimeId: "sai-sidecar",
}),
});
const port = Number(new URL(server.endpoint).port);
const sockets: Socket[] = [];
const request: CultNetOperationRequestMessage = {
schemaVersion: "cultnet.operation_request.v0",
messageId: "after-handling",
serviceId: "sai.vn",
operation: "describe",
payloadSchema: "gamecult.eve.plugin_abi.request.v1",
payloadEncoding: "messagepack-base64",
payload: "gaZzY2hlbWE=",
};
const connectionId = 0x43554c54;
try {
// Sixty-four clients each run a handler and then go silent without a
// Disconnect. Only the idle sweep can free their slots.
const clients: { socket: Socket; peer: CultNetRudpSession; handled: () => boolean }[] = [];
for (let index = 0; index < 64; index += 1) {
const socket = await bindUdpSocket();
sockets.push(socket);
const peer = new CultNetRudpSession({ connectionId });
let handled = false;
let accepted = false;
socket.on("message", (wire) => {
const packet = decodeRudpPacket(wire);
if (packet.packetType === "accept" && !accepted) {
accepted = true;
peer.receive(packet, 0);
for (const out of peer.sendMany("schema", Buffer.from(encode(encodeCultNetMessageForWire(request, "cultnet.schema.v0"))), { reliable: true, ordered: true, nowMs: 0 })) {
socket.send(encodeRudpPacket(out), port, "127.0.0.1");
}
} else if (packet.channelId === "schema") {
handled = true;
}
});
socket.send(encodeRudpPacket(peer.createConnect(0)), port, "127.0.0.1");
clients.push({ socket, peer, handled: () => handled });
}
await waitFor(() => clients.every((c) => c.handled()), "every handler ran");
await new Promise((resolve) => setTimeout(resolve, 700));
const response = await invokeCultNetOperation(server.endpoint, request, { runtimeId: "eve-test", timeoutMs: 1000 });
assert.equal(response.messageId, "after-handling");
} finally {
for (const socket of sockets) socket.close();
await server.close();
}
});

43
scripts/build-unity-package.ps1
View file

@ -53,6 +53,40 @@ if ($LASTEXITCODE -ne 0) {
throw "CultLib Unity package failed the semver policy check (see docs/semver-policy.md)."
}
# Release gate: cultlib's Unity package resolves CultMath through the org.gamecult.cultmath
# dependency, so a cultlib release must not run ahead of the CultMath package it names.
# (1) The declared dependency version must not exceed the CultMath package's own version.
# (2) The CultMath.dll that package tracks must define every type CultMathResolver formats.
# The names come from the resolver source (its Shape<T> entries); scripts\check-cultmath-types.cs
# looks them up in the assembly's type-definition table, which Windows PowerShell cannot load
# the assembly to do. It checks type names, not signatures.
$cultMathPackageRoot = Join-Path $repoRoot "packages\cultmath\unity\org.gamecult.cultmath"
$cultMathDeclared = (Get-Content -LiteralPath (Join-Path $templateRoot "package.json") -Raw | ConvertFrom-Json).dependencies.'org.gamecult.cultmath'
$cultMathAvailable = (Get-Content -LiteralPath (Join-Path $cultMathPackageRoot "package.json") -Raw | ConvertFrom-Json).version
if ([string]::IsNullOrWhiteSpace($cultMathDeclared)) {
throw "package.json must declare org.gamecult.cultmath, which supplies CultMath.dll."
}
foreach ($declared in @(@("org.gamecult.cultlib's org.gamecult.cultmath dependency", $cultMathDeclared), @("the CultMath package version", $cultMathAvailable))) {
if ($declared[1] -notmatch '^\d+\.\d+\.\d+$') {
throw "$($declared[0]) is '$($declared[1])'. The release gate compares exact release versions (major.minor.patch); prerelease and range versions are not supported."
}
}
if ([version]$cultMathDeclared -gt [version]$cultMathAvailable) {
throw "Release order: org.gamecult.cultlib depends on org.gamecult.cultmath $cultMathDeclared, but the CultMath package is at $cultMathAvailable. Release CultMath first."
}
$resolverSource = Get-Content -LiteralPath (Join-Path $repoRoot "src\GameCult.Caching.MessagePack\CultMathResolver.cs") -Raw
$resolverTypes = @([regex]::Matches($resolverSource, '\bShape<(?!T>)(\w+)>\(') | ForEach-Object { $_.Groups[1].Value } | Sort-Object -Unique)
if ($resolverTypes.Count -eq 0) {
throw "Found no CultMath types in CultMathResolver.cs; the release gate cannot check the CultMath package."
}
$missingTypes = & dotnet run (Join-Path $PSScriptRoot "check-cultmath-types.cs") -- (Join-Path $cultMathPackageRoot "Runtime\Plugins\CultMath.dll") @resolverTypes
if ($LASTEXITCODE -eq 1) {
throw "The CultMath.dll in org.gamecult.cultmath $cultMathAvailable lacks types CultMathResolver formats: $missingTypes. Release CultMath first."
}
if ($LASTEXITCODE -ne 0) {
throw "scripts\check-cultmath-types.cs failed with exit code $LASTEXITCODE."
}
# The tracked DLLs and pdbs are committed beside their source, so none may name a commit or a worktree:
# Source Link writes the commit SHA into the pdb, the informational version carries it too, and each DLL
# carries its pdb's content id. ContinuousIntegrationBuild maps the local source path to /_/.
@ -128,6 +162,15 @@ $expectedAssemblies = @(
"System.Text.Json.dll",
"System.Threading.Channels.dll"
)
# Assemblies the package references but never ships: org.gamecult.cultmath owns CultMath.dll, and the
# package.json dependency delivers it. A second copy in Runtime\Plugins would be a duplicate assembly.
$externalAssemblies = @("CultMath.dll")
$asmdef = Get-Content -LiteralPath (Join-Path $templateRoot "Runtime\GameCult.CultLib.asmdef") -Raw | ConvertFrom-Json
$referencedAssemblies = @($asmdef.precompiledReferences | Sort-Object)
$declaredAssemblies = @($expectedAssemblies + $externalAssemblies | Sort-Object)
if (($referencedAssemblies -join "|") -ne ($declaredAssemblies -join "|")) {
throw "GameCult.CultLib.asmdef precompiledReferences differ from the shipped plus external assembly lists."
}
$publishedByName = @{}
foreach ($assembly in Get-ChildItem -LiteralPath $publishRoot -Filter "*.dll") {
$publishedByName[$assembly.Name] = $assembly

30
scripts/check-cultmath-types.cs Normal file
View file

@ -0,0 +1,30 @@
// Release gate helper for build-unity-package.ps1: reads an assembly's type-definition table and
// reports which of the named CultMath types it does not define as a public top-level type.
// Usage: dotnet run scripts/check-cultmath-types.cs -- <assembly.dll> <TypeName>...
// Exit 0 when every type is defined, 1 when any is missing, 2 on bad usage.
using System.Reflection;
using System.Reflection.Metadata;
using System.Reflection.PortableExecutable;
if (args.Length < 2)
{
Console.Error.WriteLine("usage: check-cultmath-types <assembly.dll> <TypeName>...");
return 2;
}
using var stream = File.OpenRead(args[0]);
using var pe = new PEReader(stream);
var metadata = pe.GetMetadataReader();
var defined = new HashSet<string>();
foreach (var handle in metadata.TypeDefinitions)
{
var type = metadata.GetTypeDefinition(handle);
if (!type.GetDeclaringType().IsNil) continue;
if ((type.Attributes & TypeAttributes.VisibilityMask) != TypeAttributes.Public) continue;
if (metadata.GetString(type.Namespace) == "CultMath") defined.Add(metadata.GetString(type.Name));
}
var missing = args.Skip(1).Where(name => !defined.Contains(name)).ToArray();
if (missing.Length == 0) return 0;
Console.WriteLine(string.Join(", ", missing));
return 1;

2
src/GameCult.Caching.MessagePack/CultDocumentMessagePackSerialization.cs
View file

@ -58,7 +58,7 @@ public static class CultDocumentMessagePackSerialization
{
return MessagePackSerializerOptions.Standard
.WithResolver(CompositeResolver.Create(
consumerResolvers.Append(CultDocumentResolver.Instance).Append(StandardResolver.Instance).ToArray()))
consumerResolvers.Append(CultDocumentResolver.Instance).Append(CultMathResolver.Instance).Append(StandardResolver.Instance).ToArray()))
.WithSecurity(CultMessagePackSecurity.Instance);
}

516
src/GameCult.Caching.MessagePack/CultMathJson.cs Normal file
View file

@ -0,0 +1,516 @@
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Reflection;
using System.Text;
using System.Text.Json;
using System.Text.Json.Serialization;
using CultMath;
using Random = CultMath.Random;
namespace GameCult.Caching.MessagePack;
/// <summary>
/// The canonical System.Text.Json shape of every public CultMath value type, the JSON twin of
/// <see cref="CultMathResolver"/>. Vectors, quaternions and colors are objects keyed by their
/// field names (<c>{"x":1,"y":2}</c>), matrices are arrays of row objects, <c>rect</c> is
/// <c>{"min":..,"max":..}</c>, and <c>CultCellular</c> is <c>{"nearest":..,"edge":..,"id":..}</c>.
/// Property names are fixed: <c>PropertyNamingPolicy</c> does not rename them, and
/// <c>PropertyNameCaseInsensitive</c> makes reading case-insensitive.
/// <para>
/// Reading skips unknown properties (<c>UnmappedMemberHandling.Disallow</c> refuses them) and
/// leaves missing ones at zero (a missing <c>Color32.a</c> is 255). A repeated property replaces
/// the whole earlier value, never merging component-wise, and is refused when
/// <c>AllowDuplicateProperties</c> is false. A wrong token, a number outside its component type,
/// or a matrix that is not an array of exactly its row count throws
/// <see cref="JsonException"/>.
/// </para>
/// <para>
/// Numbers follow <see cref="JsonSerializerOptions.NumberHandling"/> as System.Text.Json's own
/// float does. NaN and the infinities are the strings <c>"NaN"</c>, <c>"Infinity"</c>,
/// <c>"-Infinity"</c>: written when <c>AllowNamedFloatingPointLiterals</c> or
/// <c>WriteAsString</c> is set (otherwise <see cref="JsonException"/>), read when
/// <c>AllowNamedFloatingPointLiterals</c> or <c>AllowReadingFromString</c> is set.
/// <c>AllowReadingFromString</c> reads quoted numbers without surrounding whitespace, and
/// <c>WriteAsString</c> writes quoted numbers. Two intended divergences: a bare number beyond
/// the component's range (<c>1e40</c> as a float) is refused where System.Text.Json reads
/// infinity, and refusals on write are <see cref="JsonException"/> where it throws
/// <see cref="ArgumentException"/>.
/// </para>
/// <para>
/// Every type is also a dictionary key: the flattened components joined by commas in
/// declaration order, invariant culture, shortest round-trip floats (<c>"1,2"</c>,
/// <c>"true,false"</c>, matrices row by row, <c>rect</c> as <c>"minX,minY,maxX,maxY"</c>).
/// Only that exact spelling reads back, so two JSON keys never collapse into one dictionary key.
/// </para>
/// Register with <see cref="AddCultMathConverters"/>; JSON options are the consumer's own.
/// </summary>
public static class CultMathJson
{
/// <summary>The CultMath types that have a converter, for coverage checks.</summary>
public static IReadOnlyCollection<Type> ConvertedTypes => Converters.Keys;
/// <summary>Adds the CultMath converters to <paramref name="options"/> and returns it.</summary>
public static JsonSerializerOptions AddCultMathConverters(this JsonSerializerOptions options)
{
if (options == null) throw new ArgumentNullException(nameof(options));
foreach (var converter in Converters.Values) options.Converters.Add(converter);
return options;
}
private static bool LiteralsWrite(JsonSerializerOptions o) =>
(o.NumberHandling & (JsonNumberHandling.AllowNamedFloatingPointLiterals | JsonNumberHandling.WriteAsString)) != 0;
private static bool LiteralsRead(JsonSerializerOptions o) =>
(o.NumberHandling & (JsonNumberHandling.AllowNamedFloatingPointLiterals | JsonNumberHandling.AllowReadingFromString)) != 0;
private static bool FromString(JsonSerializerOptions o) => (o.NumberHandling & JsonNumberHandling.AllowReadingFromString) != 0;
private static bool AsString(JsonSerializerOptions o) => (o.NumberHandling & JsonNumberHandling.WriteAsString) != 0;
// AllowDuplicateProperties arrived after the System.Text.Json this library compiles against.
private static readonly PropertyInfo? AllowDuplicateProperties = typeof(JsonSerializerOptions).GetProperty("AllowDuplicateProperties");
private static bool DuplicatesAllowed(JsonSerializerOptions o) => AllowDuplicateProperties == null || (bool)AllowDuplicateProperties.GetValue(o)!;
private const NumberStyles FloatStyle = NumberStyles.AllowLeadingSign | NumberStyles.AllowDecimalPoint | NumberStyles.AllowExponent;
// One component kind. Every CultMath type is a flat list of one kind, so a Scalar plus a Node tree describes it.
private abstract class Scalar<S> where S : unmanaged
{
public abstract void Write(Utf8JsonWriter w, S value, JsonSerializerOptions o);
public abstract S Read(ref Utf8JsonReader r, JsonSerializerOptions o);
/// <summary>The canonical key spelling, whatever the options allow.</summary>
public abstract string Spelling(S value);
/// <summary>The key spelling, refusing what the options would refuse to write.</summary>
public virtual string Text(S value, JsonSerializerOptions o) => Spelling(value);
public abstract S Parse(string text, JsonSerializerOptions o);
/// <summary>Reads a key component; keys are always text, so AllowReadingFromString does not apply.</summary>
public virtual S ParseKey(string text, JsonSerializerOptions o) => Parse(text, o);
}
private abstract class FloatingScalar<S> : Scalar<S> where S : unmanaged
{
protected static bool IsLiteral(string s) => s == "NaN" || s == "Infinity" || s == "-Infinity";
protected abstract bool IsFinite(S value);
protected abstract string Literal(S value);
protected abstract S FromLiteral(string literal);
protected abstract bool TryNumber(string text, out S value);
protected abstract bool TryToken(ref Utf8JsonReader r, out S value);
protected abstract void WriteNumber(Utf8JsonWriter w, S value);
protected abstract string Number(S value);
public override void Write(Utf8JsonWriter w, S v, JsonSerializerOptions o)
{
if (!IsFinite(v))
{
if (!LiteralsWrite(o)) throw new JsonException($"A non-finite {typeof(S).Name} needs JsonNumberHandling.AllowNamedFloatingPointLiterals or WriteAsString.");
w.WriteStringValue(Literal(v));
}
else if (AsString(o)) w.WriteStringValue(Number(v));
else WriteNumber(w, v);
}
public override S Read(ref Utf8JsonReader r, JsonSerializerOptions o)
{
if (r.TokenType == JsonTokenType.Number)
{
if (!TryToken(ref r, out var v) || !IsFinite(v)) throw new JsonException($"The number is outside the {typeof(S).Name} range.");
return v;
}
if (r.TokenType == JsonTokenType.String) return Parse(r.GetString()!, o);
throw new JsonException($"Expected a number, found {r.TokenType}.");
}
public override string Spelling(S v) => Number(v);
public override string Text(S v, JsonSerializerOptions o)
{
if (!IsFinite(v) && !LiteralsWrite(o)) throw new JsonException($"A non-finite {typeof(S).Name} needs JsonNumberHandling.AllowNamedFloatingPointLiterals or WriteAsString.");
return Spelling(v);
}
public override S Parse(string text, JsonSerializerOptions o)
{
if (!IsLiteral(text) && !FromString(o)) throw new JsonException($"Expected a number, found the string '{text}'.");
return ParseKey(text, o);
}
public override S ParseKey(string text, JsonSerializerOptions o)
{
if (IsLiteral(text))
{
if (!LiteralsRead(o)) throw new JsonException($"'{text}' needs JsonNumberHandling.AllowNamedFloatingPointLiterals or AllowReadingFromString.");
return FromLiteral(text);
}
if (TryNumber(text, out var v) && IsFinite(v)) return v;
throw new JsonException($"A {typeof(S).Name} cannot be read from '{text}'.");
}
}
private sealed class FloatScalar : FloatingScalar<float>
{
public static readonly FloatScalar Instance = new();
protected override bool IsFinite(float v) => float.IsFinite(v);
protected override string Literal(float v) => float.IsNaN(v) ? "NaN" : v > 0f ? "Infinity" : "-Infinity";
protected override float FromLiteral(string s) => s == "NaN" ? float.NaN : s == "Infinity" ? float.PositiveInfinity : float.NegativeInfinity;
protected override bool TryNumber(string s, out float v) => float.TryParse(s, FloatStyle, CultureInfo.InvariantCulture, out v);
protected override bool TryToken(ref Utf8JsonReader r, out float v) => r.TryGetSingle(out v);
protected override void WriteNumber(Utf8JsonWriter w, float v) => w.WriteNumberValue(v);
protected override string Number(float v) => v.ToString("R", CultureInfo.InvariantCulture);
}
private sealed class DoubleScalar : FloatingScalar<double>
{
public static readonly DoubleScalar Instance = new();
protected override bool IsFinite(double v) => double.IsFinite(v);
protected override string Literal(double v) => double.IsNaN(v) ? "NaN" : v > 0d ? "Infinity" : "-Infinity";
protected override double FromLiteral(string s) => s == "NaN" ? double.NaN : s == "Infinity" ? double.PositiveInfinity : double.NegativeInfinity;
protected override bool TryNumber(string s, out double v) => double.TryParse(s, FloatStyle, CultureInfo.InvariantCulture, out v);
protected override bool TryToken(ref Utf8JsonReader r, out double v) => r.TryGetDouble(out v);
protected override void WriteNumber(Utf8JsonWriter w, double v) => w.WriteNumberValue(v);
protected override string Number(double v) => v.ToString("R", CultureInfo.InvariantCulture);
}
private abstract class IntegerScalar<S> : Scalar<S> where S : unmanaged
{
protected abstract bool TryToken(ref Utf8JsonReader r, out S value);
protected abstract bool TryText(string text, out S value);
protected abstract void WriteNumber(Utf8JsonWriter w, S value);
public override void Write(Utf8JsonWriter w, S v, JsonSerializerOptions o)
{
if (AsString(o)) w.WriteStringValue(Spelling(v)); else WriteNumber(w, v);
}
public override S Read(ref Utf8JsonReader r, JsonSerializerOptions o)
{
if (r.TokenType == JsonTokenType.Number)
{
if (!TryToken(ref r, out var v)) throw new JsonException($"The number is not a {typeof(S).Name}.");
return v;
}
if (r.TokenType == JsonTokenType.String && FromString(o)) return Parse(r.GetString()!, o);
throw new JsonException($"Expected an integer, found {r.TokenType}.");
}
public override S Parse(string text, JsonSerializerOptions o) =>
TryText(text, out var v) ? v : throw new JsonException($"A {typeof(S).Name} cannot be read from '{text}'.");
}
private sealed class IntScalar : IntegerScalar<int>
{
public static readonly IntScalar Instance = new();
protected override bool TryToken(ref Utf8JsonReader r, out int v) => r.TryGetInt32(out v);
protected override bool TryText(string s, out int v) => int.TryParse(s, NumberStyles.AllowLeadingSign, CultureInfo.InvariantCulture, out v);
protected override void WriteNumber(Utf8JsonWriter w, int v) => w.WriteNumberValue(v);
public override string Spelling(int v) => v.ToString(CultureInfo.InvariantCulture);
}
private sealed class UIntScalar : IntegerScalar<uint>
{
public static readonly UIntScalar Instance = new();
protected override bool TryToken(ref Utf8JsonReader r, out uint v) => r.TryGetUInt32(out v);
protected override bool TryText(string s, out uint v) => uint.TryParse(s, NumberStyles.None, CultureInfo.InvariantCulture, out v);
protected override void WriteNumber(Utf8JsonWriter w, uint v) => w.WriteNumberValue(v);
public override string Spelling(uint v) => v.ToString(CultureInfo.InvariantCulture);
}
private sealed class ByteScalar : IntegerScalar<byte>
{
public static readonly ByteScalar Instance = new();
protected override bool TryToken(ref Utf8JsonReader r, out byte v) => r.TryGetByte(out v);
protected override bool TryText(string s, out byte v) => byte.TryParse(s, NumberStyles.None, CultureInfo.InvariantCulture, out v);
protected override void WriteNumber(Utf8JsonWriter w, byte v) => w.WriteNumberValue(v);
public override string Spelling(byte v) => v.ToString(CultureInfo.InvariantCulture);
}
private sealed class BoolScalar : Scalar<bool>
{
public static readonly BoolScalar Instance = new();
public override void Write(Utf8JsonWriter w, bool v, JsonSerializerOptions o) => w.WriteBooleanValue(v);
public override bool Read(ref Utf8JsonReader r, JsonSerializerOptions o) =>
r.TokenType == JsonTokenType.True ? true
: r.TokenType == JsonTokenType.False ? false
: throw new JsonException($"Expected true or false, found {r.TokenType}.");
public override string Spelling(bool v) => v ? "true" : "false";
public override bool Parse(string text, JsonSerializerOptions o) =>
text == "true" ? true : text == "false" ? false : throw new JsonException($"A bool cannot be read from '{text}'.");
}
// The JSON structure over a flat component list: leaves consume one component each, in order.
// Read fills `flat` (pre-loaded with `defaults`), so a missing property keeps its default.
private abstract class Node<S> where S : unmanaged
{
public int Count { get; protected set; }
public abstract void Write(Utf8JsonWriter w, ReadOnlySpan<S> flat, Scalar<S> scalar, JsonSerializerOptions o);
public abstract void Read(ref Utf8JsonReader r, scoped Span<S> flat, scoped ReadOnlySpan<S> defaults, Scalar<S> scalar, JsonSerializerOptions o);
}
private sealed class Leaf<S> : Node<S> where S : unmanaged
{
public Leaf() { Count = 1; }
public override void Write(Utf8JsonWriter w, ReadOnlySpan<S> flat, Scalar<S> scalar, JsonSerializerOptions o) => scalar.Write(w, flat[0], o);
public override void Read(ref Utf8JsonReader r, scoped Span<S> flat, scoped ReadOnlySpan<S> defaults, Scalar<S> scalar, JsonSerializerOptions o) => flat[0] = scalar.Read(ref r, o);
}
private sealed class Fields<S> : Node<S> where S : unmanaged
{
private readonly string[] names;
private readonly byte[][] utf8Names;
private readonly JsonEncodedText[] encoded;
private readonly Node<S>[] children;
private readonly int[] offsets;
public Fields(params (string Name, Node<S> Child)[] fields)
{
names = new string[fields.Length];
utf8Names = new byte[fields.Length][];
encoded = new JsonEncodedText[fields.Length];
children = new Node<S>[fields.Length];
offsets = new int[fields.Length];
for (var i = 0; i < fields.Length; i++)
{
names[i] = fields[i].Name;
utf8Names[i] = Encoding.UTF8.GetBytes(fields[i].Name);
encoded[i] = JsonEncodedText.Encode(fields[i].Name);
children[i] = fields[i].Child;
offsets[i] = Count;
Count += fields[i].Child.Count;
}
}
public override void Write(Utf8JsonWriter w, ReadOnlySpan<S> flat, Scalar<S> scalar, JsonSerializerOptions o)
{
w.WriteStartObject();
for (var i = 0; i < children.Length; i++)
{
w.WritePropertyName(encoded[i]);
children[i].Write(w, flat.Slice(offsets[i], children[i].Count), scalar, o);
}
w.WriteEndObject();
}
public override void Read(ref Utf8JsonReader r, scoped Span<S> flat, scoped ReadOnlySpan<S> defaults, Scalar<S> scalar, JsonSerializerOptions o)
{
if (r.TokenType != JsonTokenType.StartObject) throw new JsonException($"Expected a JSON object, found {r.TokenType}.");
var ignoreCase = o.PropertyNameCaseInsensitive;
var seen = 0;
while (r.Read() && r.TokenType == JsonTokenType.PropertyName)
{
var index = IndexOf(ref r, ignoreCase);
r.Read();
if (index < 0)
{
if (o.UnmappedMemberHandling == JsonUnmappedMemberHandling.Disallow)
throw new JsonException("The JSON object has a property that is not a component.");
r.Skip();
continue;
}
var slice = flat.Slice(offsets[index], children[index].Count);
var bit = 1 << index;
if ((seen & bit) != 0)
{
if (!DuplicatesAllowed(o)) throw new JsonException($"The property '{names[index]}' appears more than once.");
defaults.Slice(offsets[index], slice.Length).CopyTo(slice);
}
seen |= bit;
children[index].Read(ref r, slice, defaults.Slice(offsets[index], slice.Length), scalar, o);
}
}
private int IndexOf(ref Utf8JsonReader r, bool ignoreCase)
{
if (!ignoreCase)
{
for (var i = 0; i < utf8Names.Length; i++)
if (r.ValueTextEquals(utf8Names[i])) return i;
return -1;
}
var name = r.GetString()!;
for (var i = 0; i < names.Length; i++)
if (string.Equals(names[i], name, StringComparison.OrdinalIgnoreCase)) return i;
return -1;
}
}
private sealed class Rows<S> : Node<S> where S : unmanaged
{
private readonly Node<S> row;
private readonly int rows;
public Rows(int rows, Node<S> row)
{
this.rows = rows;
this.row = row;
Count = rows * row.Count;
}
public override void Write(Utf8JsonWriter w, ReadOnlySpan<S> flat, Scalar<S> scalar, JsonSerializerOptions o)
{
w.WriteStartArray();
for (var i = 0; i < rows; i++) row.Write(w, flat.Slice(i * row.Count, row.Count), scalar, o);
w.WriteEndArray();
}
public override void Read(ref Utf8JsonReader r, scoped Span<S> flat, scoped ReadOnlySpan<S> defaults, Scalar<S> scalar, JsonSerializerOptions o)
{
if (r.TokenType != JsonTokenType.StartArray) throw new JsonException($"Expected a JSON array of {rows} rows, found {r.TokenType}.");
for (var i = 0; i < rows; i++)
{
if (!r.Read() || r.TokenType == JsonTokenType.EndArray) throw new JsonException($"Expected {rows} rows.");
row.Read(ref r, flat.Slice(i * row.Count, row.Count), defaults.Slice(i * row.Count, row.Count), scalar, o);
}
if (!r.Read() || r.TokenType != JsonTokenType.EndArray) throw new JsonException($"Expected {rows} rows.");
}
}
private delegate T Make<T, S>(ReadOnlySpan<S> components) where S : unmanaged;
private delegate void Flatten<T, S>(T value, Span<S> components) where S : unmanaged;
private sealed class Shape<T, S> : JsonConverter<T> where S : unmanaged
{
private readonly Scalar<S> scalar;
private readonly Node<S> root;
private readonly S[] defaults;
private readonly Make<T, S> make;
private readonly Flatten<T, S> flatten;
public Shape(Scalar<S> scalar, Node<S> root, Make<T, S> make, Flatten<T, S> flatten, S[]? initial)
{
this.scalar = scalar;
this.root = root;
this.make = make;
this.flatten = flatten;
defaults = initial ?? new S[root.Count];
}
public override T Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
{
Span<S> flat = stackalloc S[root.Count];
defaults.CopyTo(flat);
root.Read(ref reader, flat, defaults, scalar, options);
return make(flat);
}
public override void Write(Utf8JsonWriter writer, T value, JsonSerializerOptions options)
{
Span<S> flat = stackalloc S[root.Count];
flatten(value, flat);
root.Write(writer, flat, scalar, options);
}
public override T ReadAsPropertyName(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
{
var text = reader.GetString() ?? throw new JsonException("A dictionary key cannot be null.");
var parts = text.Split(',');
if (parts.Length != root.Count) throw new JsonException($"A {typeof(T).Name} key has {root.Count} comma-separated components.");
Span<S> flat = stackalloc S[root.Count];
for (var i = 0; i < parts.Length; i++) flat[i] = scalar.ParseKey(parts[i], options);
var value = make(flat);
if (Spell(flat) != text) throw new JsonException($"'{text}' is not the canonical spelling of a {typeof(T).Name} key.");
return value;
}
public override void WriteAsPropertyName(Utf8JsonWriter writer, T value, JsonSerializerOptions options)
{
Span<S> flat = stackalloc S[root.Count];
flatten(value, flat);
for (var i = 0; i < flat.Length; i++) scalar.Text(flat[i], options);
writer.WritePropertyName(Spell(flat));
}
private string Spell(ReadOnlySpan<S> flat)
{
var parts = new string[flat.Length];
for (var i = 0; i < parts.Length; i++) parts[i] = scalar.Spelling(flat[i]);
return string.Join(",", parts);
}
}
private static Node<S> Val<S>() where S : unmanaged => new Leaf<S>();
private static Node<S> Xy<S>() where S : unmanaged => new Fields<S>(("x", Val<S>()), ("y", Val<S>()));
private static Node<S> Xyz<S>() where S : unmanaged => new Fields<S>(("x", Val<S>()), ("y", Val<S>()), ("z", Val<S>()));
private static Node<S> Xyzw<S>() where S : unmanaged => new Fields<S>(("x", Val<S>()), ("y", Val<S>()), ("z", Val<S>()), ("w", Val<S>()));
private static KeyValuePair<Type, JsonConverter> Add<T, S>(
Scalar<S> scalar, Node<S> root, Make<T, S> make, Flatten<T, S> flatten, S[]? initial = null) where S : unmanaged =>
new(typeof(T), new Shape<T, S>(scalar, root, make, flatten, initial));
private static readonly Dictionary<Type, JsonConverter> Converters = Build();
private static Dictionary<Type, JsonConverter> Build()
{
var f = FloatScalar.Instance;
var shapes = new[]
{
Add<float2, float>(f, Xy<float>(), a => new float2(a[0], a[1]), (v, a) => { a[0] = v.x; a[1] = v.y; }),
Add<float3, float>(f, Xyz<float>(), a => new float3(a[0], a[1], a[2]), (v, a) => { a[0] = v.x; a[1] = v.y; a[2] = v.z; }),
Add<float4, float>(f, Xyzw<float>(), a => new float4(a[0], a[1], a[2], a[3]), (v, a) => { a[0] = v.x; a[1] = v.y; a[2] = v.z; a[3] = v.w; }),
Add<quaternion, float>(f, Xyzw<float>(), a => new quaternion(a[0], a[1], a[2], a[3]), (v, a) => { a[0] = v.x; a[1] = v.y; a[2] = v.z; a[3] = v.w; }),
Add<double2, double>(DoubleScalar.Instance, Xy<double>(), a => new double2(a[0], a[1]), (v, a) => { a[0] = v.x; a[1] = v.y; }),
Add<double3, double>(DoubleScalar.Instance, Xyz<double>(), a => new double3(a[0], a[1], a[2]), (v, a) => { a[0] = v.x; a[1] = v.y; a[2] = v.z; }),
Add<int2, int>(IntScalar.Instance, Xy<int>(), a => new int2(a[0], a[1]), (v, a) => { a[0] = v.x; a[1] = v.y; }),
Add<int3, int>(IntScalar.Instance, Xyz<int>(), a => new int3(a[0], a[1], a[2]), (v, a) => { a[0] = v.x; a[1] = v.y; a[2] = v.z; }),
Add<int4, int>(IntScalar.Instance, Xyzw<int>(), a => new int4(a[0], a[1], a[2], a[3]), (v, a) => { a[0] = v.x; a[1] = v.y; a[2] = v.z; a[3] = v.w; }),
Add<bool2, bool>(BoolScalar.Instance, Xy<bool>(), a => new bool2(a[0], a[1]), (v, a) => { a[0] = v.x; a[1] = v.y; }),
Add<bool3, bool>(BoolScalar.Instance, Xyz<bool>(), a => new bool3(a[0], a[1], a[2]), (v, a) => { a[0] = v.x; a[1] = v.y; a[2] = v.z; }),
Add<bool4, bool>(BoolScalar.Instance, Xyzw<bool>(), a => new bool4(a[0], a[1], a[2], a[3]), (v, a) => { a[0] = v.x; a[1] = v.y; a[2] = v.z; a[3] = v.w; }),
Add<Color32, byte>(ByteScalar.Instance,
new Fields<byte>(("r", Val<byte>()), ("g", Val<byte>()), ("b", Val<byte>()), ("a", Val<byte>())),
a => new Color32(a[0], a[1], a[2], a[3]), (v, a) => { a[0] = v.r; a[1] = v.g; a[2] = v.b; a[3] = v.a; },
initial: new byte[] { 0, 0, 0, 255 }),
Add<Random, uint>(UIntScalar.Instance, new Fields<uint>(("state", Val<uint>())),
a => new Random { state = a[0] }, (v, a) => a[0] = v.state),
// Compositions: matrices are arrays of row objects, rect is {min, max}, CultCellular is {nearest, edge, id}.
Add<float2x2, float>(f, new Rows<float>(2, Xy<float>()),
a => new float2x2(a[0], a[1], a[2], a[3]),
(v, a) => { a[0] = v[0].x; a[1] = v[0].y; a[2] = v[1].x; a[3] = v[1].y; }),
Add<float3x3, float>(f, new Rows<float>(3, Xyz<float>()),
a => new float3x3(a[0], a[1], a[2], a[3], a[4], a[5], a[6], a[7], a[8]),
(v, a) =>
{
for (var row = 0; row < 3; row++)
{
a[row * 3] = v[row].x;
a[(row * 3) + 1] = v[row].y;
a[(row * 3) + 2] = v[row].z;
}
}),
Add<rect, float>(f, new Fields<float>(("min", Xy<float>()), ("max", Xy<float>())),
a => new rect(a[0], a[1], a[2], a[3]),
(v, a) => { a[0] = v.min.x; a[1] = v.min.y; a[2] = v.max.x; a[3] = v.max.y; }),
Add<CultCellular, float>(f, new Fields<float>(("nearest", Xyzw<float>()), ("edge", Xyzw<float>()), ("id", Val<float>())),
a => new CultCellular(new float4(a[0], a[1], a[2], a[3]), new float4(a[4], a[5], a[6], a[7]), a[8]),
(v, a) =>
{
a[0] = v.nearest.x; a[1] = v.nearest.y; a[2] = v.nearest.z; a[3] = v.nearest.w;
a[4] = v.edge.x; a[5] = v.edge.y; a[6] = v.edge.z; a[7] = v.edge.w;
a[8] = v.id;
}),
};
var map = new Dictionary<Type, JsonConverter>();
foreach (var shape in shapes) map.Add(shape.Key, shape.Value);
return map;
}
}

230
src/GameCult.Caching.MessagePack/CultMathResolver.cs Normal file
View file

@ -0,0 +1,230 @@
using System;
using System.Collections.Generic;
using CultMath;
using MessagePack;
using MessagePack.Formatters;
using Random = CultMath.Random;
namespace GameCult.Caching.MessagePack;
/// <summary>
/// The canonical MessagePack encoding of every public CultMath value type. Each type is an
/// array of its components in declaration order: vectors and quaternions are flat scalar
/// arrays, matrices are arrays of row vectors, and <c>rect</c> is <c>[min, max]</c>. Decoding
/// skips trailing elements and leaves missing ones at zero (a missing Color32 alpha is 255); nil is refused. The resolver is
/// part of the default <see cref="CultDocumentMessagePackSerialization"/> chain, so a consumer
/// registers nothing.
/// </summary>
public sealed class CultMathResolver : IFormatterResolver
{
public static readonly CultMathResolver Instance = new();
private CultMathResolver() { }
/// <summary>The CultMath types this resolver formats, for coverage checks.</summary>
public static IReadOnlyCollection<Type> FormattedTypes => Formatters.Keys;
public IMessagePackFormatter<T>? GetFormatter<T>() => Cache<T>.Formatter;
private static class Cache<T>
{
public static readonly IMessagePackFormatter<T>? Formatter =
Formatters.TryGetValue(typeof(T), out var formatter) ? (IMessagePackFormatter<T>)formatter : null;
}
private delegate void WriteFn<T>(ref MessagePackWriter writer, T value);
private delegate T ReadFn<T>(ref MessagePackReader reader);
private sealed class ArrayShape<T> : IMessagePackFormatter<T>
{
private readonly WriteFn<T> write;
private readonly ReadFn<T> read;
public ArrayShape(WriteFn<T> write, ReadFn<T> read)
{
this.write = write;
this.read = read;
}
public void Serialize(ref MessagePackWriter writer, T value, MessagePackSerializerOptions options) => write(ref writer, value);
public T Deserialize(ref MessagePackReader reader, MessagePackSerializerOptions options)
{
return read(ref reader);
}
}
private static KeyValuePair<Type, object> Shape<T>(WriteFn<T> write, ReadFn<T> read) =>
new(typeof(T), new ArrayShape<T>(write, read));
// Each reader takes the first dst.Length elements and skips the rest.
private static void Singles(ref MessagePackReader r, scoped Span<float> dst)
{
for (int i = 0, n = r.ReadArrayHeader(); i < n; i++)
if (i < dst.Length) dst[i] = r.ReadSingle(); else r.Skip();
}
private static void Doubles(ref MessagePackReader r, scoped Span<double> dst)
{
for (int i = 0, n = r.ReadArrayHeader(); i < n; i++)
if (i < dst.Length) dst[i] = r.ReadDouble(); else r.Skip();
}
private static void Ints(ref MessagePackReader r, scoped Span<int> dst)
{
for (int i = 0, n = r.ReadArrayHeader(); i < n; i++)
if (i < dst.Length) dst[i] = r.ReadInt32(); else r.Skip();
}
private static void Bools(ref MessagePackReader r, scoped Span<bool> dst)
{
for (int i = 0, n = r.ReadArrayHeader(); i < n; i++)
if (i < dst.Length) dst[i] = r.ReadBoolean(); else r.Skip();
}
private static void Bytes(ref MessagePackReader r, scoped Span<byte> dst)
{
for (int i = 0, n = r.ReadArrayHeader(); i < n; i++)
if (i < dst.Length) dst[i] = r.ReadByte(); else r.Skip();
}
private static void Put(ref MessagePackWriter w, scoped ReadOnlySpan<float> v)
{
w.WriteArrayHeader(v.Length);
foreach (var e in v) w.Write(e);
}
private static void Put(ref MessagePackWriter w, scoped ReadOnlySpan<double> v)
{
w.WriteArrayHeader(v.Length);
foreach (var e in v) w.Write(e);
}
private static void Put(ref MessagePackWriter w, scoped ReadOnlySpan<int> v)
{
w.WriteArrayHeader(v.Length);
foreach (var e in v) w.Write(e);
}
private static void Put(ref MessagePackWriter w, scoped ReadOnlySpan<bool> v)
{
w.WriteArrayHeader(v.Length);
foreach (var e in v) w.Write(e);
}
private static void Put(ref MessagePackWriter w, scoped ReadOnlySpan<byte> v)
{
w.WriteArrayHeader(v.Length);
foreach (var e in v) w.Write(e);
}
private static void PutFloat2(ref MessagePackWriter w, float2 v) => Put(ref w, stackalloc float[] { v.x, v.y });
private static void PutFloat3(ref MessagePackWriter w, float3 v) => Put(ref w, stackalloc float[] { v.x, v.y, v.z });
private static void PutFloat4(ref MessagePackWriter w, float4 v) => Put(ref w, stackalloc float[] { v.x, v.y, v.z, v.w });
private static float2 GetFloat2(ref MessagePackReader r)
{
Span<float> s = stackalloc float[2];
Singles(ref r, s);
return new float2(s[0], s[1]);
}
private static float3 GetFloat3(ref MessagePackReader r)
{
Span<float> s = stackalloc float[3];
Singles(ref r, s);
return new float3(s[0], s[1], s[2]);
}
private static float4 GetFloat4(ref MessagePackReader r)
{
Span<float> s = stackalloc float[4];
Singles(ref r, s);
return new float4(s[0], s[1], s[2], s[3]);
}
private static readonly Dictionary<Type, object> Formatters = Build();
private static Dictionary<Type, object> Build()
{
var shapes = new[]
{
Shape<float2>(PutFloat2, GetFloat2),
Shape<float3>(PutFloat3, GetFloat3),
Shape<float4>(PutFloat4, GetFloat4),
Shape<double2>((ref MessagePackWriter w, double2 v) => Put(ref w, stackalloc double[] { v.x, v.y }),
(ref MessagePackReader r) => { Span<double> s = stackalloc double[2]; Doubles(ref r, s); return new double2(s[0], s[1]); }),
Shape<double3>((ref MessagePackWriter w, double3 v) => Put(ref w, stackalloc double[] { v.x, v.y, v.z }),
(ref MessagePackReader r) => { Span<double> s = stackalloc double[3]; Doubles(ref r, s); return new double3(s[0], s[1], s[2]); }),
Shape<int2>((ref MessagePackWriter w, int2 v) => Put(ref w, stackalloc int[] { v.x, v.y }),
(ref MessagePackReader r) => { Span<int> s = stackalloc int[2]; Ints(ref r, s); return new int2(s[0], s[1]); }),
Shape<int3>((ref MessagePackWriter w, int3 v) => Put(ref w, stackalloc int[] { v.x, v.y, v.z }),
(ref MessagePackReader r) => { Span<int> s = stackalloc int[3]; Ints(ref r, s); return new int3(s[0], s[1], s[2]); }),
Shape<int4>((ref MessagePackWriter w, int4 v) => Put(ref w, stackalloc int[] { v.x, v.y, v.z, v.w }),
(ref MessagePackReader r) => { Span<int> s = stackalloc int[4]; Ints(ref r, s); return new int4(s[0], s[1], s[2], s[3]); }),
Shape<bool2>((ref MessagePackWriter w, bool2 v) => Put(ref w, stackalloc bool[] { v.x, v.y }),
(ref MessagePackReader r) => { Span<bool> s = stackalloc bool[2]; Bools(ref r, s); return new bool2(s[0], s[1]); }),
Shape<bool3>((ref MessagePackWriter w, bool3 v) => Put(ref w, stackalloc bool[] { v.x, v.y, v.z }),
(ref MessagePackReader r) => { Span<bool> s = stackalloc bool[3]; Bools(ref r, s); return new bool3(s[0], s[1], s[2]); }),
Shape<bool4>((ref MessagePackWriter w, bool4 v) => Put(ref w, stackalloc bool[] { v.x, v.y, v.z, v.w }),
(ref MessagePackReader r) => { Span<bool> s = stackalloc bool[4]; Bools(ref r, s); return new bool4(s[0], s[1], s[2], s[3]); }),
Shape<quaternion>((ref MessagePackWriter w, quaternion v) => Put(ref w, stackalloc float[] { v.x, v.y, v.z, v.w }),
(ref MessagePackReader r) => { Span<float> s = stackalloc float[4]; Singles(ref r, s); return new quaternion(s[0], s[1], s[2], s[3]); }),
Shape<Color32>((ref MessagePackWriter w, Color32 v) => Put(ref w, stackalloc byte[] { v.r, v.g, v.b, v.a }),
(ref MessagePackReader r) => { Span<byte> s = stackalloc byte[4]; s[3] = 255; Bytes(ref r, s); return new Color32(s[0], s[1], s[2], s[3]); }),
Shape<Random>((ref MessagePackWriter w, Random v) => { w.WriteArrayHeader(1); w.Write(v.state); },
(ref MessagePackReader r) =>
{
var value = default(Random);
for (int i = 0, n = r.ReadArrayHeader(); i < n; i++)
if (i == 0) value.state = r.ReadUInt32(); else r.Skip();
return value;
}),
// Compositions: matrices are rows, rect is [min, max], CultCellular is [nearest, edge, id].
Shape<float2x2>((ref MessagePackWriter w, float2x2 v) => { w.WriteArrayHeader(2); PutFloat2(ref w, v[0]); PutFloat2(ref w, v[1]); },
(ref MessagePackReader r) =>
{
var rows = new float2[2];
for (int i = 0, n = r.ReadArrayHeader(); i < n; i++)
if (i < 2) rows[i] = GetFloat2(ref r); else r.Skip();
return new float2x2(rows[0], rows[1]);
}),
Shape<float3x3>((ref MessagePackWriter w, float3x3 v) => { w.WriteArrayHeader(3); PutFloat3(ref w, v[0]); PutFloat3(ref w, v[1]); PutFloat3(ref w, v[2]); },
(ref MessagePackReader r) =>
{
var rows = new float3[3];
for (int i = 0, n = r.ReadArrayHeader(); i < n; i++)
if (i < 3) rows[i] = GetFloat3(ref r); else r.Skip();
return new float3x3(rows[0], rows[1], rows[2]);
}),
Shape<rect>((ref MessagePackWriter w, rect v) => { w.WriteArrayHeader(2); PutFloat2(ref w, v.min); PutFloat2(ref w, v.max); },
(ref MessagePackReader r) =>
{
var corners = new float2[2];
for (int i = 0, n = r.ReadArrayHeader(); i < n; i++)
if (i < 2) corners[i] = GetFloat2(ref r); else r.Skip();
return new rect(corners[0], corners[1]);
}),
Shape<CultCellular>((ref MessagePackWriter w, CultCellular v) => { w.WriteArrayHeader(3); PutFloat4(ref w, v.nearest); PutFloat4(ref w, v.edge); w.Write(v.id); },
(ref MessagePackReader r) =>
{
var value = default(CultCellular);
for (int i = 0, n = r.ReadArrayHeader(); i < n; i++)
{
switch (i)
{
case 0: value.nearest = GetFloat4(ref r); break;
case 1: value.edge = GetFloat4(ref r); break;
case 2: value.id = r.ReadSingle(); break;
default: r.Skip(); break;
}
}
return value;
}),
};
var map = new Dictionary<Type, object>();
foreach (var shape in shapes) map.Add(shape.Key, shape.Value);
return map;
}
}

2
src/GameCult.Caching.MessagePack/GameCult.Caching.MessagePack.csproj
View file

@ -6,7 +6,9 @@
<ItemGroup>
<ProjectReference Include="..\GameCult.Caching\GameCult.Caching.csproj" />
<ProjectReference Include="..\..\packages\cultmath\src\CultMath\CultMath.csproj" />
<PackageReference Include="MessagePack" Version="3.1.7" />
<PackageReference Include="System.Text.Json" Version="9.0.10" />
<PackageReference Include="MessagePackAnalyzer" Version="3.1.7" PrivateAssets="all" />
<InternalsVisibleTo Include="GameCult.Caching.Tests" />
</ItemGroup>

40
src/GameCult.Caching.MessagePack/README.md
View file

@ -9,6 +9,7 @@ format for the attribute-first `GameCult.Caching` stack.
- `CultDocumentMessagePackSerialization`
- `CultDocumentResolver`
- `CultRecordRefFormatter<T>`
- `CultMathResolver`: the canonical component-array encoding of every public CultMath value type, part of the default resolver chain
Cult document payloads serialize through `MessagePackSerializer` with the
options of the document's assembly (`CultDocumentMessagePackSerialization.OptionsFor`),
@ -56,3 +57,42 @@ await cache.PullAllBackingStoresAsync();
persisted through hand-written MessagePack array layouts in this package.
- The raw CultNet document lane should treat these payload bytes as already
blessed, not decode and re-encode them for sport.
## CultMath Value Shapes
Every public CultMath value type has one MessagePack shape (`CultMathResolver`,
in the default chain) and one JSON shape (`CultMathJson`, added to a consumer's
options with `new JsonSerializerOptions().AddCultMathConverters()`; CultLib owns
no JSON options of its own). MessagePack decoding skips extra elements, leaves
missing ones at zero (`Color32` alpha at 255) and refuses nil. JSON reading
skips unknown properties (`UnmappedMemberHandling.Disallow` refuses them), leaves
missing ones at zero, honours `PropertyNameCaseInsensitive`, lets a repeated
property replace the whole earlier value (refused when `AllowDuplicateProperties`
is false), and throws `JsonException` for a wrong token or an out-of-range number.
Numbers follow `JsonSerializerOptions.NumberHandling` as System.Text.Json's own
float does: `"NaN"`, `"Infinity"` and `"-Infinity"` are written under
`AllowNamedFloatingPointLiterals` or `WriteAsString` and read under
`AllowNamedFloatingPointLiterals` or `AllowReadingFromString`; quoted numbers read
under `AllowReadingFromString` with no surrounding whitespace.
Every type is also a dictionary key: its flattened components joined by commas,
invariant culture (`"1,2"`, `"true,false"`, matrices row by row, `rect` as
`"minX,minY,maxX,maxY"`), and only that exact spelling reads back. Two intended
divergences from System.Text.Json: a bare number beyond a component's range
(`1e40` as a float) is refused where System.Text.Json reads infinity, and a
refusal on write is a `JsonException` where System.Text.Json throws
`ArgumentException`. A consumer resolver or converter
registered ahead of these wins.
| Type | MessagePack | JSON |
| --- | --- | --- |
| `float2`, `double2`, `int2`, `bool2` | `[x, y]` | `{"x":..,"y":..}` |
| `float3`, `double3`, `int3`, `bool3` | `[x, y, z]` | `{"x":..,"y":..,"z":..}` |
| `float4`, `int4`, `bool4`, `quaternion` | `[x, y, z, w]` | `{"x":..,"y":..,"z":..,"w":..}` |
| `Color32` | `[r, g, b, a]` as uint8 | `{"r":..,"g":..,"b":..,"a":..}` |
| `Random` | `[state]` | `{"state":..}` |
| `float2x2`, `float3x3` | array of row vectors (row-major) | array of row objects |
| `rect` | `[min, max]` of `float2` | `{"min":{..},"max":{..}}` |
| `CultCellular` | `[nearest, edge, id]` | `{"nearest":{..},"edge":{..},"id":..}` |
Floats are MessagePack float32 (`0xCA`), doubles float64 (`0xCB`), integers the
shortest MessagePack integer.

43
src/GameCult.Caching/Contracts/cultcache-store-composition.md
View file

@ -74,17 +74,45 @@ which route wrote it.
- Not stale-protected: schema-alias handles (`AsSchemaAlias`) get no
`Sequence`; removals do not advance a mirror's applied `Sequence`, and the
mirrors do not surface removals.
- `AddJournal` registers an ordered admission journal, for audit, replication or
a derived index. The cache calls each journal under its gate, right after an
admission is applied and before that admission is published, once per
admission that changed something, with that admission's changes (a variant
re-resolved by a base edit included) in `Sequence` order. Journals of one cache
run in registration order, and reading the cache from one sees the new state.
Because the gate serializes admissions, the calls of one cache see contiguous,
strictly increasing `Sequence` values: a journal is the one place where cache
order is a fact, and delivery order is not. A refused admission, or one a store
declines, is not journaled. Disposing the registration removes the journal.
- A journal that throws does not undo the admission: the remaining journals
still run, the change is published, and the exception is rethrown to the
writer after publication, with the same aggregation as an observer's. The
journal stays registered. Any hold on the cache while a journal runs throws
`InvalidOperationException`: a journal must not write to the cache or
register another journal.
- An `OnUpdate` handler exception is rethrown to that caller after all of that
call's changes are delivered (an `AggregateException` if several threw). If
the call itself failed, its own exception is rethrown and handler exceptions
are dropped. A `Watch` subscriber exception is not rethrown; it follows R3's
unhandled-exception handling. A throwing handler cannot undo the store's
adoption of a load; the store and cache already agree when publication starts.
are dropped. Every `Watch` subscriber receives every change: R3 routes a
subscriber's exception to its unhandled-exception handler, and if that handler
rethrows (fail-fast), the exception is treated as an `OnUpdate` handler
exception. The subscriber stays subscribed and the commit stands. A throwing
handler cannot undo the store's adoption of a load; the store and cache
already agree when publication starts.
- An admission runs a record's `[CultName]` and `[CultIndex]` getters once per
landing record, before the store is written and before any `Sequence` is
minted. A conditional commit on a clean file store is judged a second time
against the file it merges onto whenever the cache holds variants in any store,
the batch lands a variant, or the file holds one. That second read runs the
getters again for the records the batch lands, for any variant it re-resolves,
and for the records other writers put in the file (the arriving set); that
judgement is also before the store is written. A getter that throws refuses the whole change
set, loads included: nothing is written, minted or left in memory.
- Pulling all stores pulls every attached store even if a handler throws during
one store's load, then rethrows.
- Hydration failure on open is loud: a corrupt store file, or a record whose
schema the registry cannot resolve, makes the open throw and leaves the file
byte-identical. Consumers never delete and rewrite a store they failed to
schema the registry cannot resolve or whose name or index getter throws, makes
the open throw and leaves the file byte-identical. Consumers never delete and rewrite a store they failed to
open.
## Locking
@ -93,6 +121,11 @@ which route wrote it.
cache takes that cache's gate as its own lock, so a store's load callback into
the cache cannot take the two out of order. A store must not call `Loaded`
while holding its own lock outside the cache's hold.
- A journal runs under the cache's gate, so a slow journal slows every writer of
that cache. It must not enter the cache, and must not wait for anything that
needs the cache or its gate: another thread's write, a lock a writer holds.
Its own leaf locks, taken only inside the journal, are the only locks that may
nest under the gate.
- A thread holds one cache's gate at a time: entering a cache's hold inside
another cache's hold throws.
- A store doing I/O on behalf of its cache (pull, flush, commit) holds the gate,

343
src/GameCult.Caching/CultCache.cs
View file

@ -1825,11 +1825,18 @@ namespace GameCult.Caching
// Keys holding a variant: derived from the entries by Apply, never decided anywhere else.
private readonly HashSet<string> _variantKeys = new(StringComparer.Ordinal);
private readonly ConditionalWeakTable<object, KeyBox> _handles = new();
private readonly Subject<Change> _changes = new();
// The cache's own observers, copy-on-write under _observerGate: publication reads one snapshot per change and
// catches per observer, so no observer decides whether another receives a change.
private Observer<Change>[] _observers = Array.Empty<Observer<Change>>();
private bool _observersCompleted;
private readonly object _observerGate = new();
private readonly object _gate = new();
private long _sequence;
// The changes admitted by this thread's outermost hold, published by that hold when it exits.
[ThreadStatic] private static List<(Change Change, bool Loaded)>? _held;
// What this thread's outermost hold admitted, published by that hold when it exits.
[ThreadStatic] private static Hold? _held;
// Journals run under the gate, in registration order; both fields are read and written only under it.
private readonly List<Action<IReadOnlyList<CultCacheDocumentChange<object>>>> _journals = new();
private bool _journaling;
private bool _dirtyInMemory;
// The codec is the store's encoding (CultCacheMessagePack.Create supplies it). A cache without one holds no variants:
@ -1945,7 +1952,7 @@ namespace GameCult.Caching
public Observable<CultCacheDocumentChange<T>> Watch<T>() where T : class
{
return _changes
return Observable.Create<Change>(Register)
.Where(change => typeof(T).IsAssignableFrom(change.Stored.Descriptor.DocumentType))
.Select(change => new CultCacheDocumentChange<T>(
change.Kind,
@ -1955,11 +1962,70 @@ namespace GameCult.Caching
change.Sequence));
}
// Disposing removes the journal. A dispose the cache refuses (from inside a journal, or inside another cache's hold)
// throws and leaves the registration live and disposable; it counts as disposed only once the removal succeeded.
private sealed class JournalRegistration : IDisposable
{
private readonly CultCache _cache;
private readonly Action<IReadOnlyList<CultCacheDocumentChange<object>>> _journal;
private bool _removed;
public JournalRegistration(CultCache cache, Action<IReadOnlyList<CultCacheDocumentChange<object>>> journal)
{
_cache = cache;
_journal = journal;
}
public void Dispose()
{
if (_removed)
return;
_cache.Held(() => _cache._journals.Remove(_journal));
_removed = true;
}
}
private IDisposable Register(Observer<Change> observer)
{
lock (_observerGate)
{
if (!_observersCompleted)
{
_observers = _observers.Append(observer).ToArray();
return Disposable.Create(() =>
{
lock (_observerGate)
_observers = _observers.Where(held => !ReferenceEquals(held, observer)).ToArray();
});
}
}
observer.OnCompleted();
return Disposable.Empty;
}
public Observable<CultCacheDocumentChange<T>> WatchRecord<T>(CultRecordKey key) where T : class
{
return Watch<T>().Where(change => change.Key.Equals(key));
}
// An ordered admission journal: called under the gate, right after an admission is applied, once per admission
// that changed something, with that admission's changes in Sequence order. Journals of one cache run in
// registration order. A journal sees the new state (reads work) and is the one place cache order is a fact: it
// must not write to the cache or wait for anything that needs it, and any hold on this cache while a journal runs
// throws. A journal's exception does not undo the admission: it is published, then the exception is rethrown to
// the writer, as an observer's is.
public IDisposable AddJournal(Action<IReadOnlyList<CultCacheDocumentChange<object>>> journal)
{
if (journal == null) throw new ArgumentNullException(nameof(journal));
Held(() =>
{
_journals.Add(journal);
return true;
});
return new JournalRegistration(this, journal);
}
// Attaching reads the store; there is no interval in which it is attached but unread.
public void AddBackingStore(CacheBackingStore store, params Type[] homes)
{
@ -2210,7 +2276,31 @@ namespace GameCult.Caching
foreach (var store in BackingStores)
store.Dispose();
_changes.Dispose();
Observer<Change>[] completing;
lock (_observerGate)
{
completing = _observers;
_observers = Array.Empty<Observer<Change>>();
_observersCompleted = true;
}
var failures = new List<Exception>();
foreach (var observer in completing)
{
try
{
observer.OnCompleted();
}
catch (Exception exception)
{
failures.Add(exception);
}
}
if (failures.Count == 1)
ExceptionDispatchInfo.Capture(failures[0]).Throw();
if (failures.Count > 1)
throw new AggregateException(failures);
}
internal static void RequireInstanceOf(Type documentType, object document)
@ -2425,40 +2515,51 @@ namespace GameCult.Caching
stored.IdsInMemoryOnly = idPlan.Count > 0;
}
foreach (var change in Apply(plan, evicted, source))
_held.Add((change, source != null));
var applied = Apply(plan, evicted, source);
foreach (var change in applied)
_held.Changes.Add((change, source != null));
RunJournals(applied, _held.Failures);
return CultCommitOutcome.Committed;
});
// Every admission runs in a hold. A nested hold on the same cache adds to the outermost one, which publishes
// exactly its own changes after it leaves the gate, before it returns, on its caller's thread: observers never
// run under the gate, and a write an observer makes is a new outermost hold. Cross-thread delivery order is not
// guaranteed; each change carries the Sequence it was admitted with. Every change is delivered even if an
// OnUpdate handler throws; then the first handler exception (an AggregateException for several) is rethrown.
// guaranteed; each change carries the Sequence it was admitted with. Journals run before that, under the gate,
// in Sequence order (AddJournal); no hold may be entered on this cache while one runs. Every change reaches every observer even
// if an observer or an OnUpdate handler throws; then the first exception (an AggregateException for several) is rethrown.
// If the body threw, its exception wins.
internal T Held<T>(Func<T> body)
{
if (Monitor.IsEntered(_gate))
{
if (_journaling)
throw new InvalidOperationException("A journal entered the cache it journals; a journal must not write to it.");
return body();
}
if (_held != null)
throw new InvalidOperationException(
"A cache hold was entered inside another cache's hold; a thread holds one cache's gate at a time.");
var mine = _held = new List<(Change Change, bool Loaded)>();
T result;
var mine = _held = new Hold();
var result = default(T)!;
try
{
lock (_gate)
result = body();
}
catch
catch (Exception exception)
{
_held = null;
Publish(mine);
throw;
if (mine.Failures.Count == 0)
throw;
throw new AggregateException(mine.Failures.Prepend(exception));
}
_held = null;
var failures = Publish(mine);
var failures = mine.Failures;
failures.AddRange(Publish(mine));
if (failures.Count == 1)
ExceptionDispatchInfo.Capture(failures[0]).Throw();
if (failures.Count > 1)
@ -2466,13 +2567,25 @@ namespace GameCult.Caching
return result;
}
private List<Exception> Publish(List<(Change Change, bool Loaded)> changes)
private List<Exception> Publish(Hold hold)
{
var failures = new List<Exception>();
foreach (var (change, loaded) in changes)
foreach (var (change, loaded) in hold.Changes)
{
// R3 routes a throwing subscriber to its unhandled-exception handler; OnNext does not throw.
_changes.OnNext(change);
// R3 routes a throwing subscriber to its unhandled-exception handler, and a fail-fast handler rethrows: every
// observer still runs, the exception joins the failures, and the observer stays subscribed.
foreach (var observer in Volatile.Read(ref _observers))
{
try
{
observer.OnNext(change);
}
catch (Exception exception)
{
failures.Add(exception);
}
}
if (!loaded)
continue;
try
@ -2488,6 +2601,36 @@ namespace GameCult.Caching
return failures;
}
// Runs under the gate. A journal that throws joins the hold's failures and the remaining journals still run.
private void RunJournals(List<Change> applied, List<Exception> failures)
{
if (_journals.Count == 0 || applied.Count == 0)
return;
var changes = applied
.Select(change => new CultCacheDocumentChange<object>(
change.Kind, change.Stored.Key, change.Document, change.Previous, change.Sequence))
.ToArray();
_journaling = true;
try
{
foreach (var journal in _journals)
{
try
{
journal(changes);
}
catch (Exception exception)
{
failures.Add(exception);
}
}
}
finally
{
_journaling = false;
}
}
private CacheBackingStore? Validate(
IReadOnlyList<CultStoredDocument> admitted,
IReadOnlyList<CultStoredDocument> evicted,
@ -2565,11 +2708,84 @@ namespace GameCult.Caching
// Variants this admission did not write whose base chain it changed: re-resolved, same delta, same storedAt.
public List<CultStoredDocument> Dependents { get; } = new();
// Name and index values of every record that lands (Admitted and Dependents), by key. Filled once by Project.
public Dictionary<string, Projection> Projections { get; } = new(StringComparer.Ordinal);
}
// What a landing record's [CultName] and [CultIndex] getters said. Indexes holds only non-blank values.
private sealed class Projection
{
public Projection(string? name, (string Alias, string Value)[] indexes)
{
Name = name;
Indexes = indexes;
}
public string? Name { get; }
public (string Alias, string Value)[] Indexes { get; }
}
// The only place an admission runs a name or index getter on a landing record. It runs before land and before any
// sequence is minted, so a getter that throws refuses the whole admission with nothing written and nothing minted.
private static void Project(VariantPlan plan)
{
foreach (var stored in plan.Admitted.Concat(plan.Dependents))
{
var descriptor = stored.Descriptor;
var name = descriptor.NameAccessor == null ? null : NameRead(stored);
var indexes = new List<(string Alias, string Value)>();
foreach (var pair in descriptor.IndexAccessors)
{
var member = descriptor.RichMembers.First(candidate => candidate.IndexAlias == pair.Key).Member.Name;
var value = Read(stored, $"index '{pair.Key}' member {member}", pair.Value);
if (!string.IsNullOrWhiteSpace(value))
indexes.Add((pair.Key, value!));
}
plan.Projections[stored.Key.Value] = new Projection(name, indexes.ToArray());
}
}
// A blank name is no name.
private static string? NameRead(CultStoredDocument stored) =>
Read(stored, $"[CultName] member {stored.Descriptor.NameMember}", stored.Descriptor.NameAccessor!) is { Length: > 0 } name ? name : null;
private static string? Read(CultStoredDocument stored, string what, Func<object, string?> getter)
{
try
{
return getter(stored.Document);
}
catch (Exception exception)
{
throw new InvalidOperationException(
$"Record {stored.Key.Value} ({stored.Descriptor.SchemaName}) is refused: its {what} threw: {exception.GetBaseException().Message}", exception);
}
}
// Every admission is judged here: variants resolved, then every landing record's name and index values read once,
// then the rules that compare them. It refuses naming the keys, and mutates nothing.
private VariantPlan Resolve(
IReadOnlyList<CultStoredDocument> admitted,
IReadOnlyList<CultStoredDocument> evicted,
CacheBackingStore? source)
{
var plan = ResolveVariants(admitted, evicted, source);
Project(plan);
// The variant rules are judged only when a variant is involved.
if (_variantKeys.Count != 0 || admitted.Any(stored => stored.Variant != null))
{
RefuseVariantNameSharing(plan);
RefuseVariantIndexSharing(plan, evicted);
}
return plan;
}
// Resolves every admitted variant and every variant whose base chain this admission changes, base first, against
// the post-admission set. It refuses a broken variant naming the keys, and mutates nothing.
private VariantPlan Resolve(
private VariantPlan ResolveVariants(
IReadOnlyList<CultStoredDocument> admitted,
IReadOnlyList<CultStoredDocument> evicted,
CacheBackingStore? source)
@ -2631,11 +2847,7 @@ namespace GameCult.Caching
}
if (toResolve.Count == 0)
{
var unchanged = new VariantPlan(admitted);
RefuseVariantIndexSharing(unchanged, incoming, gone);
return unchanged;
}
return new VariantPlan(admitted);
var pending = new HashSet<string>(toResolve, StringComparer.Ordinal);
var done = new Dictionary<string, CultStoredDocument>(StringComparer.Ordinal);
@ -2646,7 +2858,6 @@ namespace GameCult.Caching
var plan = new VariantPlan(admitted.Select(stored => done.TryGetValue(stored.Key.Value, out var resolved) ? resolved : stored).ToList());
foreach (var key in toResolve.Where(key => !incoming.ContainsKey(key)))
plan.Dependents.Add(done[key]);
RefuseVariantIndexSharing(plan, incoming, gone);
return plan;
CultStoredDocument ResolveOne(string key)
@ -2729,21 +2940,6 @@ namespace GameCult.Caching
var document = _codec.Deserialize(type, type, kept.Count == 0 ? payload : _codec.Overlay(payload, kept))
?? throw new InvalidOperationException($"Variant {key} resolved to nothing.");
if (isVariant)
{
// R6: the name index maps one name to one key, so a variant that kept its base's name would shadow it.
if (descriptor.NameAccessor?.Invoke(document) is { Length: > 0 } variantName)
{
for (var ancestor = baseStored; ancestor != null;
ancestor = ancestor.Variant == null ? null : done.TryGetValue(ancestor.Variant.BaseKey, out var above) ? above : Lookup(ancestor.Variant.BaseKey))
{
if (descriptor.NameAccessor(ancestor.Document) == variantName)
throw new InvalidOperationException(
$"Variant {key} and its base {ancestor.Key.Value} are both named '{variantName}'; a variant must override its name.");
}
}
}
stack.RemoveAt(stack.Count - 1);
return done[key] = isVariant
? variant.Resolved(document, minted ? new CultVariantDelta(delta.BaseKey, overrides) : delta, variant.IdsInMemoryOnly || (minted && source != null), holdsIds)
@ -2757,20 +2953,47 @@ namespace GameCult.Caching
}
}
// R6: the name index maps one name to one key, so a variant that kept its base's name would shadow it. A variant's
// name is its projection; an ancestor that lands is read from its projection, one that stays is read as held.
private void RefuseVariantNameSharing(VariantPlan plan)
{
var landing = plan.Admitted.Concat(plan.Dependents).ToDictionary(stored => stored.Key.Value, StringComparer.Ordinal);
// Resolution already refused a base that is missing or removed, so an ancestor is landing or held.
CultStoredDocument? Ancestor(string key) =>
landing.TryGetValue(key, out var landed) ? landed
: _entries.TryGetValue(key, out var held) ? held : null;
string? NameOf(CultStoredDocument stored) =>
plan.Projections.TryGetValue(stored.Key.Value, out var projection) ? projection.Name : NameRead(stored);
foreach (var variant in landing.Values.Where(stored => stored.Variant != null))
{
var variantName = plan.Projections[variant.Key.Value].Name;
if (variantName == null)
continue;
for (var ancestor = Ancestor(variant.Variant!.BaseKey); ancestor != null;
ancestor = ancestor.Variant == null ? null : Ancestor(ancestor.Variant.BaseKey))
{
if (NameOf(ancestor) == variantName)
throw new InvalidOperationException(
$"Variant {variant.Key.Value} and its base {ancestor.Key.Value} are both named '{variantName}'; a variant must override its name.");
}
}
}
// The one index rule, judged on the commit's result whatever the write order: a variant may not share an indexed
// value with another record. It reads the truthful held index (every holder of a value), minus the records this
// commit replaces or removes, plus the records it lands. Plain-vs-plain sharing is not this rule's business.
// A unique index maps a value to one key at lookup; a variant sharing it would be found in place of its
// holder or lose the value, and the store would refuse to reopen.
private void RefuseVariantIndexSharing(
VariantPlan plan,
IReadOnlyDictionary<string, CultStoredDocument> incoming,
HashSet<string> gone)
private void RefuseVariantIndexSharing(VariantPlan plan, IReadOnlyList<CultStoredDocument> evicted)
{
var landing = plan.Admitted.Concat(plan.Dependents).ToList();
if (landing.All(stored => stored.Descriptor.IndexAccessors.Count == 0))
return;
var replaced = new HashSet<string>(incoming.Keys.Concat(gone).Concat(plan.Dependents.Select(stored => stored.Key.Value)), StringComparer.Ordinal);
var replaced = new HashSet<string>(
plan.Admitted.Concat(evicted).Concat(plan.Dependents).Select(stored => stored.Key.Value), StringComparer.Ordinal);
var landingByKey = landing.ToDictionary(stored => stored.Key.Value, StringComparer.Ordinal);
bool IsVariant(string key) =>
@ -2781,12 +3004,9 @@ namespace GameCult.Caching
var landed = new Dictionary<((Type Type, string Alias) Index, string Value), List<string>>();
foreach (var stored in landing)
{
foreach (var pair in stored.Descriptor.IndexAccessors)
foreach (var (alias, value) in plan.Projections[stored.Key.Value].Indexes)
{
var value = pair.Value(stored.Document);
if (string.IsNullOrWhiteSpace(value))
continue;
var index = (stored.Descriptor.DocumentType, pair.Key);
var index = (stored.Descriptor.DocumentType, alias);
landedValues.Add((stored, index, value));
if (!landed.TryGetValue((index, value), out var landedKeys))
landed[(index, value)] = landedKeys = new List<string>();
@ -2858,7 +3078,7 @@ namespace GameCult.Caching
_entries[stored.Key.Value] = stored;
_handles.AddOrUpdate(stored.Document, new KeyBox(stored.Key));
Index(stored);
Index(stored, plan.Projections[stored.Key.Value]);
if (stored.Variant == null)
_variantKeys.Remove(stored.Key.Value);
else
@ -2915,19 +3135,15 @@ namespace GameCult.Caching
}
}
private void Index(CultStoredDocument stored)
private void Index(CultStoredDocument stored, Projection projection)
{
var type = stored.Descriptor.DocumentType;
if (stored.Descriptor.IsGlobal)
_globals[type] = stored.Key.Value;
if (stored.Descriptor.NameAccessor?.Invoke(stored.Document) is { Length: > 0 } name)
MapOf(_names, type)[name] = stored.Key.Value;
foreach (var pair in stored.Descriptor.IndexAccessors)
{
var value = pair.Value(stored.Document);
if (!string.IsNullOrWhiteSpace(value))
AddHolder(_indexes, (type, pair.Key), value, stored.Key.Value);
}
if (projection.Name != null)
MapOf(_names, type)[projection.Name] = stored.Key.Value;
foreach (var (alias, value) in projection.Indexes)
AddHolder(_indexes, (type, alias), value, stored.Key.Value);
}
// Documents are mutable, so a name or index value may have changed since it was indexed: drop by key.
@ -2985,6 +3201,13 @@ namespace GameCult.Caching
public CultRecordKey Key { get; }
}
// What one outermost hold admitted, and what its journals threw.
private sealed class Hold
{
public List<(Change Change, bool Loaded)> Changes { get; } = new();
public List<Exception> Failures { get; } = new();
}
private sealed class Change
{
public Change(CultCacheDocumentChangeKind kind, CultStoredDocument stored, object? document, object? previous, long sequence)

5
src/GameCult.Geometry/CultGeometryIsoSurface.cs
View file

@ -39,7 +39,7 @@ namespace GameCult.Geometry
public static CultGeometryTriangleMesh Extract(
float[,,] samples,
float isoValue = 0f,
CultVec3 origin = default,
float3 origin = default,
float cellSize = 1f)
{
if (samples == null) throw new ArgumentNullException(nameof(samples));
@ -56,7 +56,6 @@ namespace GameCult.Geometry
var positions = new List<float>();
var normals = new List<float>();
var indices = new List<uint>();
var originValue = new float3(origin.X, origin.Y, origin.Z);
var cornerPositions = new float3[8];
var cornerValues = new float[8];
var tetraPositions = new float3[4];
@ -66,7 +65,7 @@ namespace GameCult.Geometry
for (var y = 0; y < samples.GetLength(1) - 1; y++)
for (var z = 0; z < samples.GetLength(2) - 1; z++)
{
var cubeOrigin = originValue + new float3(x, y, z) * cellSize;
var cubeOrigin = origin + new float3(x, y, z) * cellSize;
for (var corner = 0; corner < CubeCorners.Length; corner++)
{
var offset = CubeCorners[corner];

231
src/GameCult.Geometry/CultGeometryPrimitives.cs
View file

@ -1,197 +1,10 @@
using System;
using System.Text.Json.Serialization;
using CultMath;
using MessagePack;
namespace GameCult.Geometry
{
/// <summary>
/// Deterministic two-dimensional float vector for cross-runtime geometry and query contracts.
/// </summary>
[MessagePackObject]
public readonly struct CultVec2 : IEquatable<CultVec2>
{
/// <summary>Creates a two-dimensional vector.</summary>
public CultVec2(float x, float y)
{
X = x;
Y = y;
}
/// <summary>Gets the x component.</summary>
[Key(0)]
public float X { get; }
/// <summary>Gets the y component.</summary>
[Key(1)]
public float Y { get; }
/// <summary>Gets the zero vector.</summary>
[IgnoreMember]
[JsonIgnore]
public static CultVec2 Zero => new(0f, 0f);
/// <summary>Gets the squared vector length.</summary>
[IgnoreMember]
[JsonIgnore]
public float LengthSquared => (X * X) + (Y * Y);
/// <summary>Returns whether this vector is equal to another vector.</summary>
public bool Equals(CultVec2 other) => X.Equals(other.X) && Y.Equals(other.Y);
/// <inheritdoc />
public override bool Equals(object? obj) => obj is CultVec2 other && Equals(other);
/// <inheritdoc />
public override int GetHashCode() => HashCode.Combine(X, Y);
/// <inheritdoc />
public override string ToString() => $"({X}, {Y})";
public static CultVec2 operator +(CultVec2 left, CultVec2 right) => new(left.X + right.X, left.Y + right.Y);
public static CultVec2 operator -(CultVec2 left, CultVec2 right) => new(left.X - right.X, left.Y - right.Y);
public static CultVec2 operator *(CultVec2 value, float scalar) => new(value.X * scalar, value.Y * scalar);
public static CultVec2 operator /(CultVec2 value, float scalar) => new(value.X / scalar, value.Y / scalar);
public static bool operator ==(CultVec2 left, CultVec2 right) => left.Equals(right);
public static bool operator !=(CultVec2 left, CultVec2 right) => !left.Equals(right);
}
/// <summary>
/// Deterministic three-dimensional float vector for cross-runtime geometry and query contracts.
/// </summary>
[MessagePackObject]
public readonly struct CultVec3 : IEquatable<CultVec3>
{
/// <summary>Creates a three-dimensional vector.</summary>
public CultVec3(float x, float y, float z)
{
X = x;
Y = y;
Z = z;
}
/// <summary>Gets the x component.</summary>
[Key(0)]
public float X { get; }
/// <summary>Gets the y component.</summary>
[Key(1)]
public float Y { get; }
/// <summary>Gets the z component.</summary>
[Key(2)]
public float Z { get; }
/// <summary>Gets the zero vector.</summary>
[IgnoreMember]
[JsonIgnore]
public static CultVec3 Zero => new(0f, 0f, 0f);
/// <summary>Gets the XY plane projection.</summary>
[IgnoreMember]
[JsonIgnore]
public CultVec2 Xy => new(X, Y);
/// <summary>Gets the XZ plane projection for Unity compatibility adapters.</summary>
[IgnoreMember]
[JsonIgnore]
public CultVec2 Xz => new(X, Z);
/// <summary>Gets the squared vector length.</summary>
[IgnoreMember]
[JsonIgnore]
public float LengthSquared => (X * X) + (Y * Y) + (Z * Z);
/// <summary>Returns whether this vector is equal to another vector.</summary>
public bool Equals(CultVec3 other) => X.Equals(other.X) && Y.Equals(other.Y) && Z.Equals(other.Z);
/// <inheritdoc />
public override bool Equals(object? obj) => obj is CultVec3 other && Equals(other);
/// <inheritdoc />
public override int GetHashCode() => HashCode.Combine(X, Y, Z);
/// <inheritdoc />
public override string ToString() => $"({X}, {Y}, {Z})";
public static CultVec3 operator +(CultVec3 left, CultVec3 right) => new(left.X + right.X, left.Y + right.Y, left.Z + right.Z);
public static CultVec3 operator -(CultVec3 left, CultVec3 right) => new(left.X - right.X, left.Y - right.Y, left.Z - right.Z);
public static CultVec3 operator *(CultVec3 value, float scalar) => new(value.X * scalar, value.Y * scalar, value.Z * scalar);
public static CultVec3 operator /(CultVec3 value, float scalar) => new(value.X / scalar, value.Y / scalar, value.Z / scalar);
public static bool operator ==(CultVec3 left, CultVec3 right) => left.Equals(right);
public static bool operator !=(CultVec3 left, CultVec3 right) => !left.Equals(right);
}
/// <summary>
/// Axis-aligned two-dimensional rectangle represented by canonical min and max corners.
/// </summary>
[MessagePackObject]
public readonly struct CultRect : IEquatable<CultRect>
{
/// <summary>Creates a rectangle and canonicalizes min/max corners.</summary>
public CultRect(CultVec2 min, CultVec2 max)
{
Min = new CultVec2(Math.Min(min.X, max.X), Math.Min(min.Y, max.Y));
Max = new CultVec2(Math.Max(min.X, max.X), Math.Max(min.Y, max.Y));
}
/// <summary>Creates a rectangle and canonicalizes min/max corners.</summary>
public CultRect(float minX, float minY, float maxX, float maxY)
: this(new CultVec2(minX, minY), new CultVec2(maxX, maxY))
{
}
/// <summary>Gets the canonical minimum corner.</summary>
[Key(0)]
public CultVec2 Min { get; }
/// <summary>Gets the canonical maximum corner.</summary>
[Key(1)]
public CultVec2 Max { get; }
/// <summary>Gets the rectangle size.</summary>
[IgnoreMember]
[JsonIgnore]
public CultVec2 Size => Max - Min;
/// <summary>Gets the rectangle center.</summary>
[IgnoreMember]
[JsonIgnore]
public CultVec2 Center => (Min + Max) * 0.5f;
/// <summary>Returns whether the point lies inside or on the rectangle edge.</summary>
public bool Contains(CultVec2 point)
{
return point.X >= Min.X &&
point.X <= Max.X &&
point.Y >= Min.Y &&
point.Y <= Max.Y;
}
/// <summary>Returns whether this rectangle intersects another rectangle.</summary>
public bool Intersects(CultRect other)
{
return Min.X <= other.Max.X &&
Max.X >= other.Min.X &&
Min.Y <= other.Max.Y &&
Max.Y >= other.Min.Y;
}
/// <summary>Returns whether this rectangle is equal to another rectangle.</summary>
public bool Equals(CultRect other) => Min == other.Min && Max == other.Max;
/// <inheritdoc />
public override bool Equals(object? obj) => obj is CultRect other && Equals(other);
/// <inheritdoc />
public override int GetHashCode() => HashCode.Combine(Min, Max);
/// <inheritdoc />
public override string ToString() => $"[{Min}..{Max}]";
public static bool operator ==(CultRect left, CultRect right) => left.Equals(right);
public static bool operator !=(CultRect left, CultRect right) => !left.Equals(right);
}
/// <summary>
/// Two-dimensional circle for spatial queries.
/// </summary>
@ -199,7 +12,8 @@ namespace GameCult.Geometry
public readonly struct CultCircle : IEquatable<CultCircle>
{
/// <summary>Creates a circle.</summary>
public CultCircle(CultVec2 center, float radius)
[JsonConstructor]
public CultCircle(float2 center, float radius)
{
if (radius < 0f)
{
@ -212,7 +26,7 @@ namespace GameCult.Geometry
/// <summary>Gets the circle center.</summary>
[Key(0)]
public CultVec2 Center { get; }
public float2 Center { get; }
/// <summary>Gets the circle radius.</summary>
[Key(1)]
@ -221,29 +35,29 @@ namespace GameCult.Geometry
/// <summary>Gets the circle bounds.</summary>
[IgnoreMember]
[JsonIgnore]
public CultRect Bounds => new(
Center.X - Radius,
Center.Y - Radius,
Center.X + Radius,
Center.Y + Radius);
public rect Bounds => new(
Center.x - Radius,
Center.y - Radius,
Center.x + Radius,
Center.y + Radius);
/// <summary>Returns whether the point lies inside or on the circle edge.</summary>
public bool Contains(CultVec2 point)
public bool Contains(float2 point)
{
var delta = point - Center;
return delta.LengthSquared <= Radius * Radius;
return math.lengthsq(delta) <= Radius * Radius;
}
/// <summary>Returns whether this circle intersects a rectangle.</summary>
public bool Intersects(CultRect rect)
public bool Intersects(rect rect)
{
var clampedX = Math.Max(rect.Min.X, Math.Min(Center.X, rect.Max.X));
var clampedY = Math.Max(rect.Min.Y, Math.Min(Center.Y, rect.Max.Y));
return Contains(new CultVec2(clampedX, clampedY));
var clampedX = Math.Max(rect.min.x, Math.Min(Center.x, rect.max.x));
var clampedY = Math.Max(rect.min.y, Math.Min(Center.y, rect.max.y));
return Contains(new float2(clampedX, clampedY));
}
/// <summary>Returns whether this circle is equal to another circle.</summary>
public bool Equals(CultCircle other) => Center == other.Center && Radius.Equals(other.Radius);
public bool Equals(CultCircle other) => Center.Equals(other.Center) && Radius.Equals(other.Radius);
/// <inheritdoc />
public override bool Equals(object? obj) => obj is CultCircle other && Equals(other);
@ -262,7 +76,8 @@ namespace GameCult.Geometry
public readonly struct CultSphere : IEquatable<CultSphere>
{
/// <summary>Creates a sphere.</summary>
public CultSphere(CultVec3 center, float radius)
[JsonConstructor]
public CultSphere(float3 center, float radius)
{
if (radius < 0f)
{
@ -275,7 +90,7 @@ namespace GameCult.Geometry
/// <summary>Gets the sphere center.</summary>
[Key(0)]
public CultVec3 Center { get; }
public float3 Center { get; }
/// <summary>Gets the sphere radius.</summary>
[Key(1)]
@ -284,17 +99,17 @@ namespace GameCult.Geometry
/// <summary>Gets the XY circle projection.</summary>
[IgnoreMember]
[JsonIgnore]
public CultCircle XyCircle => new(Center.Xy, Radius);
public CultCircle XyCircle => new(Center.xy, Radius);
/// <summary>Returns whether the point lies inside or on the sphere surface.</summary>
public bool Contains(CultVec3 point)
public bool Contains(float3 point)
{
var delta = point - Center;
return delta.LengthSquared <= Radius * Radius;
return math.lengthsq(delta) <= Radius * Radius;
}
/// <summary>Returns whether this sphere is equal to another sphere.</summary>
public bool Equals(CultSphere other) => Center == other.Center && Radius.Equals(other.Radius);
public bool Equals(CultSphere other) => Center.Equals(other.Center) && Radius.Equals(other.Radius);
/// <inheritdoc />
public override bool Equals(object? obj) => obj is CultSphere other && Equals(other);

7
src/GameCult.Geometry/CultGeometrySurfaceNets.cs
View file

@ -44,7 +44,7 @@ namespace GameCult.Geometry
public static CultGeometryQuadMesh Extract(
float[,,] samples,
float isoValue = 0f,
CultVec3 origin = default,
float3 origin = default,
float cellSize = 1f)
{
if (samples == null) throw new ArgumentNullException(nameof(samples));
@ -66,7 +66,7 @@ namespace GameCult.Geometry
throw new ArgumentException("The isovalue must be finite.", nameof(isoValue));
}
if (!float.IsFinite(origin.X) || !float.IsFinite(origin.Y) || !float.IsFinite(origin.Z))
if (!float.IsFinite(origin.x) || !float.IsFinite(origin.y) || !float.IsFinite(origin.z))
{
throw new ArgumentException("The origin must be finite.", nameof(origin));
}
@ -80,7 +80,6 @@ namespace GameCult.Geometry
var sizeY = samples.GetLength(1);
var sizeZ = samples.GetLength(2);
var cellCounts = new[] { sizeX - 1, sizeY - 1, sizeZ - 1 };
var originValue = new float3(origin.X, origin.Y, origin.Z);
var vertexIndex = new Dictionary<(int, int, int), uint>();
var positions = new List<float>();
@ -90,7 +89,7 @@ namespace GameCult.Geometry
for (var cz = 0; cz < cellCounts[2]; cz++)
{
var cellCoord = new[] { cx, cy, cz };
var vertex = CellVertex(samples, cellCoord, isoValue, originValue, cellSize);
var vertex = CellVertex(samples, cellCoord, isoValue, origin, cellSize);
if (vertex == null) continue;
vertexIndex[(cx, cy, cz)] = (uint)(positions.Count / 3);

11
src/GameCult.Geometry/README.md
View file

@ -31,12 +31,9 @@ The package is deliberately substrate-shaped:
## Value Primitives
The small value layer is for cross-runtime query contracts, physics probes,
native slices, and UI surfaces:
native slices, and UI surfaces. Vectors and rectangles are CultMath's `float2`,
`float3` and `rect`, and `GameCult.Caching.MessagePack` serializes them.
- `CultVec2` and `CultVec3`: whole-vector values for positions, velocities,
accelerations, and query inputs. SoA layouts may store them efficiently, but
the semantic API does not force callers to split x/y/z into unrelated fields.
- `CultRect`: canonical XY viewport/query rectangle stored as min/max corners.
- `CultCircle`: 2D influence/query brush with rect intersection helpers.
- `CultSphere`: 3D query primitive with whole-vector center and XY projection.
@ -85,8 +82,8 @@ cache miss; that is split-brain geometry.
## Documents
- `CultVec2`, `CultVec3`, `CultRect`, `CultCircle`, `CultSphere`: shared value
primitives for query and view contracts.
- `CultCircle`, `CultSphere`: shared value primitives for query and view
contracts.
- `CultGeometryDomainDocument`: one hierarchical domain tree, suitable for
Ragnarok/Fensalir-style feature DSL output.
- `CultGeometryBuildRequest`: one LOD/frustum/budget request for workers.

8
src/GameCult.Mesh/docs/cross-runtime-primitives-roadmap.md
View file

@ -116,12 +116,12 @@ var aetheria = verse.Use(AetheriaRuntime.Schema);
var viewport = await aetheria.Zone(zoneId)
.Objects()
.VisibleTo(controlledUnits)
.Within(CultRect.FromBounds(-400, -250, 400, 250))
.Within(new rect(-400f, -250f, 400f, 250f))
.QueryAsync();
await aetheria.Entity(ravenId)
.Pilot()
.MoveAsync(CultVec2.Right, claim: "pilot-control");
.MoveAsync(new float2(1f, 0f), claim: "pilot-control");
```
Current seed:
@ -134,7 +134,7 @@ var verse = await CultMesh.ConnectVerseAsync(
new[] { new CultMeshAuthorityClaim("pilot-control", shardId: "zone:raven") });
var aetheria = verse.Use(context => new AetheriaGeneratedFacade(context));
await aetheria.Entity(ravenId).Pilot.MoveAsync(CultVec2.Right);
await aetheria.Entity(ravenId).Pilot.MoveAsync(new float2(1f, 0f));
```
```ts
@ -177,7 +177,7 @@ let viewport = aetheria
.zone(zone_id)
.objects()
.visible_to(&controlled_units)
.within(CultRect::from_bounds(-400.0, -250.0, 400.0, 250.0))
.within(rect::new(-400.0, -250.0, 400.0, 250.0))
.query()
.await?;
```

4
src/GameCult.Mesh/docs/public-api.md
View file

@ -407,8 +407,8 @@ var view = new CultMeshNativeSliceViewDescriptor(
rowCount,
new[]
{
CultMeshNativeSliceColumn.For<CultVec2>("position"),
CultMeshNativeSliceColumn.For<CultVec2>("velocity")
CultMeshNativeSliceColumn.For<float2>("position"),
CultMeshNativeSliceColumn.For<float2>("velocity")
},
new CultMeshRouteHint(CultMeshLocalityKind.SharedMemory));
```

281
src/GameCult.Networking/CultNetTransport.cs
View file

@ -33,6 +33,12 @@ namespace GameCult.Networking
/// Gets the number of bytes received, including transport framing.
/// </summary>
public long BytesReceived { get; internal set; }
/// <summary>
/// Gets the number of datagrams read and discarded because they belong to no session on
/// this transport: malformed frames, another session's connection id, a sender that is not
/// the peer, or a packet the session refuses.
/// </summary>
public long PacketsDropped { get; internal set; }
internal CultNetTransportStats Snapshot()
{
@ -41,7 +47,8 @@ namespace GameCult.Networking
FramesSent = FramesSent,
FramesReceived = FramesReceived,
BytesSent = BytesSent,
BytesReceived = BytesReceived
BytesReceived = BytesReceived,
PacketsDropped = PacketsDropped
};
}
}
@ -657,12 +664,15 @@ namespace GameCult.Networking
throw new InvalidOperationException("CultNet RUDP packet header length does not match the channel id length.");
}
var payloadLength = checked((int)BinaryPrimitives.ReadUInt32BigEndian(wire.AsSpan(30, 4)));
if (wire.Length != headerBytes + payloadLength)
// Compared as a long: a declared length past int.MaxValue is a malformed packet, not an overflow.
var declaredPayloadLength = BinaryPrimitives.ReadUInt32BigEndian(wire.AsSpan(30, 4));
if (declaredPayloadLength != (long)wire.Length - headerBytes)
{
throw new InvalidOperationException("CultNet RUDP packet payload length does not match the packet size.");
}
var payloadLength = (int)declaredPayloadLength;
var flags = wire[6];
var payload = new byte[payloadLength];
Array.Copy(wire, headerBytes, payload, 0, payload.Length);
@ -916,10 +926,7 @@ namespace GameCult.Networking
public CultNetRudpSession(CultNetRudpSessionOptions options)
{
if (options == null) throw new ArgumentNullException(nameof(options));
if (options.MaxPendingReliablePackets.HasValue && options.MaxPendingReliablePackets.Value <= 0)
{
throw new ArgumentOutOfRangeException(nameof(options), "RUDP MaxPendingReliablePackets must be greater than zero.");
}
ValidateLimits(options.InitialSequence, options.MaxPendingReliablePackets);
ConnectionId = options.ConnectionId;
_nextSequence = options.InitialSequence;
@ -927,6 +934,25 @@ namespace GameCult.Networking
_maxPendingReliablePackets = options.MaxPendingReliablePackets;
}
internal static readonly byte[] RefusedPacketReason = Encoding.UTF8.GetBytes("session refused a packet");
/// <summary>
/// A session cannot admit a Connect when its sequence space starts exhausted or its reliable
/// queue holds nothing, so neither is a usable configuration.
/// </summary>
internal static void ValidateLimits(uint initialSequence, int? maxPendingReliablePackets)
{
if (initialSequence == uint.MaxValue)
{
throw new ArgumentOutOfRangeException(nameof(initialSequence), "RUDP InitialSequence must leave room for a reliable packet.");
}
if (maxPendingReliablePackets.HasValue && maxPendingReliablePackets.Value <= 0)
{
throw new ArgumentOutOfRangeException(nameof(maxPendingReliablePackets), "RUDP MaxPendingReliablePackets must be greater than zero.");
}
}
/// <summary>
/// Gets the connection/session binding id.
/// </summary>
@ -999,6 +1025,36 @@ namespace GameCult.Networking
return response;
}
/// <summary>
/// Answers a connect from a peer this session has already accepted. The packet's sequence is
/// remembered and nothing is queued, so a connect storm cannot grow the reliable queue. The reply
/// is the accept still awaiting acknowledgement, or an ack once it was acknowledged.
/// </summary>
public CultNetRudpPacket AnswerRepeatedConnect(CultNetRudpPacket packet, long nowMs = 0)
{
RequireConnection(packet);
if (packet.PacketType != CultNetRudpPacketType.Connect)
{
throw new InvalidOperationException($"Expected RUDP connect packet, got {packet.PacketType}.");
}
ApplyAcknowledgements(packet);
RememberReceived(packet.Sequence);
_lastReceivedAtMs = nowMs;
lock (_pendingReliableGate)
{
foreach (var pending in _pendingReliable.Values)
{
if (pending.Packet.PacketType != CultNetRudpPacketType.Accept)
continue;
pending.LastSentAtMs = nowMs;
return ClonePacket(pending.Packet);
}
}
return CreateAck();
}
/// <summary>
/// Creates a data packet.
/// </summary>
@ -1230,6 +1286,38 @@ namespace GameCult.Networking
return CreatePacket(CultNetRudpPacketType.Ping, "control", payload ?? Array.Empty<byte>(), reliable: false, ordered: false, sequenced: false);
}
/// <summary>
/// Forgets everything learned from the peer; sequence numbers already issued stay issued.
/// </summary>
public void ResetPeerState()
{
_connected = false;
_lastReceivedAtMs = null;
_highestReceivedSequence = null;
_receivedSequences.Clear();
_nextSequencedByChannel.Clear();
_latestSequencedByChannel.Clear();
lock (_pendingReliableGate)
{
_pendingReliable.Clear();
_queuedReliable.Clear();
}
_orderedNextSequenceByChannel.Clear();
_orderedBuffers.Clear();
_fragmentBuffers.Clear();
}
/// <summary>
/// Ends a session that refused a packet. Receive has already recorded the packet's reliable
/// sequence, so the peer's sequences are forgotten before the goodbye is built: its ack field
/// would otherwise acknowledge the very frame the session refused.
/// </summary>
internal CultNetRudpPacket EndRefused()
{
ResetPeerState();
return CreateDisconnect(RefusedPacketReason);
}
/// <summary>
/// Creates a packet carrying a transport-level disconnect reason.
/// </summary>
@ -1752,6 +1840,7 @@ namespace GameCult.Networking
throw new InvalidOperationException("Only a client RUDP socket transport can initiate connect.");
}
DisconnectReason = null;
lock (_sessionGate)
SendPacket(_session.CreateConnect(NowMs(), payload ?? Array.Empty<byte>()));
}
@ -1832,6 +1921,12 @@ namespace GameCult.Networking
{
while (true)
{
// An ended session forgot its unacknowledged writes; reporting them
// flushed would be a lie.
var ended = DisconnectReason;
if (ended != null)
throw new InvalidOperationException(
"RUDP session ended before its reliable writes were acknowledged: " + Encoding.UTF8.GetString(ended));
lock (_sessionGate)
{
if (_session.OutstandingReliablePacketCount == 0)
@ -2081,23 +2176,72 @@ namespace GameCult.Networking
}
_stats.BytesReceived += received;
// What a datagram carries is the sender's business, not a fault of this process: a
// malformed frame, another session's connection id, a sender that is not the peer and
// a packet the session refuses are dropped and counted. Only the socket ends a loop.
// The id is checked before the peer endpoint is claimed, so a stray cannot become it.
var wire = new byte[received];
Array.Copy(buffer, wire, received);
CultNetRudpPacket packet;
try
{
packet = CultNetRudpPacketCodec.Decode(wire);
}
catch (Exception error) when (error is InvalidOperationException || error is ArgumentException)
{
_stats.PacketsDropped++;
return true;
}
if (packet.ConnectionId != _session.ConnectionId)
{
_stats.PacketsDropped++;
return true;
}
if (_remoteEndPoint == null)
{
// Only a Connect claims the endpoint of a server-mode transport.
if (_mode == CultNetRudpSocketMode.Server && packet.PacketType != CultNetRudpPacketType.Connect)
{
_stats.PacketsDropped++;
return true;
}
_remoteEndPoint = remote;
}
else if (!_remoteEndPoint.Equals(remote))
{
_stats.PacketsDropped++;
return true;
}
var wire = new byte[received];
Array.Copy(buffer, wire, received);
var packet = CultNetRudpPacketCodec.Decode(wire);
TracePacket("rx", packet, received, remote);
if (_mode == CultNetRudpSocketMode.Server && packet.PacketType == CultNetRudpPacketType.Connect)
{
lock (_sessionGate)
SendPacket(_session.AcceptConnect(packet, NowMs()));
{
CultNetRudpPacket accept;
try
{
// A connect from the peer this transport already accepted repeats: answer it
// with the accept already owed and queue nothing.
if (_session.Connected)
{
accept = _session.AnswerRepeatedConnect(packet, NowMs());
}
else
{
accept = _session.AcceptConnect(packet, NowMs());
DisconnectReason = null;
}
}
catch (InvalidOperationException)
{
_stats.PacketsDropped++;
return true;
}
SendPacket(accept);
}
return true;
}
@ -2105,7 +2249,32 @@ namespace GameCult.Networking
CultNetRudpPacket? acknowledgement = null;
lock (_sessionGate)
{
result = _session.Receive(packet, NowMs());
try
{
result = _session.Receive(packet, NowMs());
}
catch (InvalidOperationException)
{
// Receive has already recorded the packet's reliable sequence, so the session
// cannot be kept: a retransmit would be acknowledged and the frame silently lost.
// End it and tell the peer; a server-mode transport also forgets its endpoint, so
// only a Connect can claim it again.
_stats.PacketsDropped++;
var goodbye = _session.EndRefused();
DisconnectReason = CultNetRudpSession.RefusedPacketReason;
try
{
SendPacket(goodbye);
}
catch (SocketException)
{
// Best effort: the session ends whether or not the peer hears it.
}
if (_mode == CultNetRudpSocketMode.Server)
_remoteEndPoint = null;
return true;
}
if (result.Reply != null)
SendPacket(result.Reply);
SendPackets(result.ReadyToSend);
@ -2268,12 +2437,14 @@ namespace GameCult.Networking
/// </summary>
public sealed class CultNetRudpSocketServerPeer
{
internal CultNetRudpSocketServerPeer(EndPoint remoteEndPoint, CultNetRudpSession session)
internal CultNetRudpSocketServerPeer(EndPoint remoteEndPoint, byte[] connectPayload, CultNetRudpSession session)
{
RemoteEndPoint = remoteEndPoint;
ConnectPayload = connectPayload;
Session = session;
}
internal byte[] ConnectPayload { get; }
internal CultNetRudpSession Session { get; }
internal object SessionGate { get; } = new object();
@ -2316,6 +2487,7 @@ namespace GameCult.Networking
public sealed class CultNetRudpSocketTransportServer : IDisposable
{
private const int MinimumReceiveBufferBytes = 4 * 1024 * 1024;
private static readonly byte[] ReplacedPeerReason = Encoding.UTF8.GetBytes("replaced by a new Connect generation");
// Keep server response pacing aligned with the 32-packet ACK mask. Large
// content responses otherwise pay one Windows scheduler quantum per 1 KiB.
private const int WireBurstPackets = 32;
@ -2341,6 +2513,7 @@ namespace GameCult.Networking
_socket = options.Socket ?? throw new ArgumentNullException(nameof(options.Socket));
if (_socket.ReceiveBufferSize < MinimumReceiveBufferBytes)
_socket.ReceiveBufferSize = MinimumReceiveBufferBytes;
CultNetRudpSession.ValidateLimits(options.InitialSequence, options.MaxPendingReliablePackets);
_connectionId = options.ConnectionId;
_initialSequence = options.InitialSequence;
_resendDelayMs = options.ResendDelayMs;
@ -2478,21 +2651,47 @@ namespace GameCult.Networking
{
packet = CultNetRudpPacketCodec.Decode(wire);
}
catch (InvalidOperationException)
catch (Exception error) when (error is InvalidOperationException || error is ArgumentException)
{
_stats.PacketsDropped++;
return true;
}
if (packet.ConnectionId != _connectionId)
{
_stats.PacketsDropped++;
return true;
}
var peerKey = RemoteKey(remote);
if (packet.PacketType == CultNetRudpPacketType.Connect)
{
var connectPayload = packet.Payload ?? Array.Empty<byte>();
_peers.TryGetValue(peerKey, out var admitted);
// The same payload from an admitted peer is a repeated Connect: answer it with the
// Accept already owed and queue nothing. A different payload is a new generation.
if (admitted != null && admitted.ConnectPayload.AsSpan().SequenceEqual(connectPayload))
{
lock (admitted.SessionGate)
{
CultNetRudpPacket reply;
try
{
reply = admitted.Session.AnswerRepeatedConnect(packet, NowMs());
}
catch (InvalidOperationException)
{
_stats.PacketsDropped++;
return true;
}
SendPacket(admitted.RemoteEndPoint, reply);
}
return true;
}
var peer = new CultNetRudpSocketServerPeer(
CloneEndPoint(remote),
connectPayload,
new CultNetRudpSession(new CultNetRudpSessionOptions
{
ConnectionId = _connectionId,
@ -2503,25 +2702,65 @@ namespace GameCult.Networking
_peers[peerKey] = peer;
lock (peer.SessionGate)
SendPacket(peer.RemoteEndPoint, peer.Session.AcceptConnect(packet, NowMs(), _acceptPayload));
if (admitted != null)
{
admitted.DisconnectReason = ReplacedPeerReason;
PeerDisconnected?.Invoke(admitted);
}
return true;
}
if (!_peers.TryGetValue(peerKey, out var existingPeer))
{
_stats.PacketsDropped++;
return true;
}
CultNetRudpReceiveResult result;
CultNetRudpReceiveResult? outcome;
CultNetRudpPacket? acknowledgement = null;
lock (existingPeer.SessionGate)
{
result = existingPeer.Session.Receive(packet, NowMs());
if (result.Reply != null)
SendPacket(existingPeer.RemoteEndPoint, result.Reply);
SendPackets(existingPeer, result.ReadyToSend);
if (packet.PacketType == CultNetRudpPacketType.Data)
acknowledgement = existingPeer.Session.CreateAckForReceived(packet.Sequence);
try
{
outcome = existingPeer.Session.Receive(packet, NowMs());
}
catch (InvalidOperationException)
{
// Receive has already recorded the packet's reliable sequence, so the session
// cannot be kept: a retransmit would be acknowledged and the frame silently lost.
_stats.PacketsDropped++;
outcome = null;
}
if (outcome == null)
{
try
{
SendPacket(existingPeer.RemoteEndPoint, existingPeer.Session.EndRefused());
}
catch (SocketException)
{
// Best effort: the session ends whether or not the peer hears it.
}
}
else
{
if (outcome.Reply != null)
SendPacket(existingPeer.RemoteEndPoint, outcome.Reply);
SendPackets(existingPeer, outcome.ReadyToSend);
if (packet.PacketType == CultNetRudpPacketType.Data)
acknowledgement = existingPeer.Session.CreateAckForReceived(packet.Sequence);
}
}
if (outcome == null)
{
existingPeer.DisconnectReason = CultNetRudpSession.RefusedPacketReason;
_peers.Remove(peerKey);
PeerDisconnected?.Invoke(existingPeer);
return true;
}
var result = outcome;
if (result.Disconnected)
{
existingPeer.DisconnectReason = result.DisconnectReason;

402
tests/GameCult.Caching.Tests/CultCacheJournalTests.cs Normal file
View file

@ -0,0 +1,402 @@
#nullable enable
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using GameCult.Caching;
using GameCult.Caching.MessagePack;
using MessagePack;
using NUnit.Framework;
using R3;
namespace GameCult.Caching.Tests
{
// Cut 3 of docs/cultcache-publication-cut.md: a journal is called under the cache gate, right after an admission is
// applied, once per admission, with that admission's changes in Sequence order. It is the one place cache order is a fact.
[NonParallelizable]
public class CultCacheJournalTests
{
private static readonly CultDocumentRegistry Registry = CultDocumentRegistry.ForTypes(new[] { typeof(JournalNote) });
private string _directory = string.Empty;
[SetUp]
public void Prepare()
{
_directory = Path.Combine(Path.GetTempPath(), $"cultlib-journal-{Guid.NewGuid():N}");
Directory.CreateDirectory(_directory);
JournalNote.Armed = false;
}
[TearDown]
public void Cleanup()
{
JournalNote.Armed = false;
if (Directory.Exists(_directory))
Directory.Delete(_directory, recursive: true);
}
[CultDocument("tests.journal_note", "tests.journal_note.v1")]
[MessagePackObject(AllowPrivate = true)]
internal sealed class JournalNote
{
public static volatile bool Armed;
[IgnoreMember] private string _name = string.Empty;
[Key(0)]
[CultName]
public string Name
{
get
{
if (Armed && _name == "boom")
throw new InvalidOperationException("name getter boom");
return _name;
}
set => _name = value;
}
[Key(1)]
public int Power;
}
private static CultRecordKey KeyOf(string text) => new($"journal:{text}");
private static CultCache Memory() => new(Registry, CultCacheMessagePack.CreateCodec(Registry));
private static void Put(CultCache cache, string name, int power = 1) =>
cache.Commit(batch => batch.Upsert(typeof(JournalNote), new JournalNote { Name = name, Power = power }, KeyOf(name)));
[Test]
public void EveryAdmissionIsJournaledInSequenceOrderAndItsChangesArriveTogether()
{
using var cache = Memory();
var calls = new List<long[]>();
using var journal = cache.AddJournal(changes => calls.Add(changes.Select(change => change.Sequence).ToArray()));
const int threads = 8;
const int writes = 1000;
Parallel.For(0, threads, new ParallelOptions { MaxDegreeOfParallelism = threads }, thread =>
{
for (var i = 0; i < writes; i++)
cache.Commit(batch =>
{
batch.Upsert(typeof(JournalNote), new JournalNote { Name = $"{thread}-{i}-a" }, KeyOf($"{thread}-{i}-a"));
batch.Upsert(typeof(JournalNote), new JournalNote { Name = $"{thread}-{i}-b" }, KeyOf($"{thread}-{i}-b"));
});
});
Assert.That(calls, Has.Count.EqualTo(threads * writes), "one call per admission");
var expected = 1L;
foreach (var call in calls)
{
Assert.That(call, Is.EqualTo(new[] { expected, expected + 1 }), "contiguous, and one call's changes together");
expected += 2;
}
}
[Test]
public void AJournalRunsAfterTheAdmissionIsAppliedAndBeforeAnyObserver()
{
using var cache = Memory();
var order = new List<string>();
using var observer = cache.Watch<JournalNote>().Subscribe(change => order.Add($"observer:{change.Sequence}"));
var seenInside = new List<string?>();
using var journal = cache.AddJournal(changes =>
{
order.Add($"journal:{changes[0].Sequence}");
seenInside.Add(cache.Get<JournalNote>(KeyOf("one"))?.Name);
Assert.That(changes[0].Kind, Is.EqualTo(order.Count == 1 ? CultCacheDocumentChangeKind.Added : CultCacheDocumentChangeKind.Updated));
Assert.That(((JournalNote?)changes[0].Document)?.Power, Is.EqualTo(order.Count == 1 ? 1 : 2));
Assert.That(((JournalNote?)changes[0].PreviousDocument)?.Power, Is.EqualTo(order.Count == 1 ? null : 1));
});
Put(cache, "one", 1);
Put(cache, "one", 2);
Assert.That(order, Is.EqualTo(new[] { "journal:1", "observer:1", "journal:2", "observer:2" }));
Assert.That(seenInside, Is.EqualTo(new[] { "one", "one" }), "the journal reads the new state");
}
[Test]
public void AnAdmissionThatChangesNothingIsNotJournaled()
{
using var cache = Memory();
var calls = 0;
using var journal = cache.AddJournal(_ => calls++);
// Removing a record the cache does not hold is a commit that admits nothing.
cache.Commit(batch => batch.Remove(KeyOf("absent")));
Assert.That(calls, Is.Zero);
}
[Test]
public void JournalsRunInRegistrationOrderAndADisposedJournalStopsRunning()
{
using var cache = Memory();
var order = new List<string>();
using var first = cache.AddJournal(_ => order.Add("first"));
var second = cache.AddJournal(_ => order.Add("second"));
using var third = cache.AddJournal(_ => order.Add("third"));
Put(cache, "one");
second.Dispose();
order.Add("|");
Put(cache, "two");
Assert.That(order, Is.EqualTo(new[] { "first", "second", "third", "|", "first", "third" }));
}
[Test]
public void AJournalThatWritesTheCacheThrowsAndTheAdmissionStands()
{
using var cache = Memory();
var seen = new List<long>();
using var observer = cache.Watch<JournalNote>().Subscribe(change => seen.Add(change.Sequence));
using var journal = cache.AddJournal(_ => Put(cache, "nested"));
var thrown = Assert.Throws<InvalidOperationException>(() => Put(cache, "outer"))!;
Assert.Multiple(() =>
{
Assert.That(thrown.Message, Does.Contain("journal"));
Assert.That(cache.Get<JournalNote>(KeyOf("outer")), Is.Not.Null, "the admission stands");
Assert.That(cache.Get<JournalNote>(KeyOf("nested")), Is.Null, "the journal's own write did not land");
Assert.That(seen, Is.EqualTo(new[] { 1L }), "and it was published");
});
}
[Test]
public void AJournalThatRegistersAnotherJournalThrows()
{
using var cache = Memory();
using var journal = cache.AddJournal(_ => cache.AddJournal(_ => { }));
var thrown = Assert.Throws<InvalidOperationException>(() => Put(cache, "outer"))!;
Assert.That(thrown.Message, Does.Contain("journal"));
Assert.That(cache.Get<JournalNote>(KeyOf("outer")), Is.Not.Null);
}
[Test]
public void TheCacheIsWritableAgainOnceAJournalHasReturned()
{
using var cache = Memory();
using var journal = cache.AddJournal(_ => { });
Put(cache, "one");
cache.Commit(batch => batch.Upsert(typeof(JournalNote), new JournalNote { Name = "two" }, KeyOf("two")));
using var late = cache.AddJournal(_ => { });
Assert.That(cache.Get<JournalNote>(KeyOf("two")), Is.Not.Null);
}
[Test]
public void ARefusedAdmissionIsNotJournaled()
{
using var cache = Memory();
var calls = 0;
using var journal = cache.AddJournal(_ => calls++);
JournalNote.Armed = true;
Assert.Throws<InvalidOperationException>(() => Put(cache, "boom"));
JournalNote.Armed = false;
Assert.That(calls, Is.Zero);
}
[Test]
public void AnAdmissionTheStoreDeclinesIsNotJournaled()
{
using var cache = Memory();
Put(cache, "seed");
var calls = 0;
using var journal = cache.AddJournal(_ => calls++);
var outcome = cache.TryCommit(batch =>
{
batch.Expect(KeyOf("seed"), null);
batch.Upsert(typeof(JournalNote), new JournalNote { Name = "two" }, KeyOf("two"));
});
Assert.That(outcome, Is.EqualTo(CultCommitOutcome.Mismatch));
Assert.That(calls, Is.Zero);
Assert.That(cache.Get<JournalNote>(KeyOf("two")), Is.Null);
}
[Test]
public void AVariantDependentIsJournaledInTheSameCallAsItsBaseEdit()
{
using var cache = Memory();
Put(cache, "base", 10);
cache.Commit(batch => batch.UpsertVariant(KeyOf("big"), KeyOf("base"), new[]
{
cache.Override<JournalNote>(nameof(JournalNote.Name), "big"),
}));
var calls = new List<string[]>();
using var journal = cache.AddJournal(changes => calls.Add(changes.Select(change => change.Key.Value).ToArray()));
Put(cache, "base", 20);
Assert.That(calls, Has.Count.EqualTo(1));
Assert.That(calls[0], Is.EquivalentTo(new[] { KeyOf("base").Value, KeyOf("big").Value }));
}
[Test]
public void AThrowingJournalDoesNotStopTheOthersOrTheAdmissionAndTheWriterHearsOfItAfterPublication()
{
using var cache = Memory();
var order = new List<string>();
using var observer = cache.Watch<JournalNote>().Subscribe(_ => order.Add("observer"));
using var a = cache.AddJournal(_ =>
{
order.Add("a");
throw new InvalidOperationException("a");
});
using var b = cache.AddJournal(_ => order.Add("b"));
using var c = cache.AddJournal(_ =>
{
order.Add("c");
throw new InvalidOperationException("c");
});
var thrown = Assert.Throws<AggregateException>(() => Put(cache, "one"))!;
Assert.Multiple(() =>
{
Assert.That(thrown.InnerExceptions.Select(exception => exception.Message), Is.EqualTo(new[] { "a", "c" }));
Assert.That(order, Is.EqualTo(new[] { "a", "b", "c", "observer" }), "every journal ran, then the change published");
Assert.That(cache.Get<JournalNote>(KeyOf("one")), Is.Not.Null, "the admission stands");
});
using var quiet = cache.Watch<JournalNote>().Subscribe(_ => { });
Assert.Throws<AggregateException>(() => Put(cache, "two"), "a throwing journal stays registered");
}
[Test]
public void ADisposeTheCacheRefusesLeavesTheJournalRegisteredAndALaterDisposeRemovesIt()
{
using var cache = Memory();
var calls = 0;
IDisposable? registration = null;
registration = cache.AddJournal(_ =>
{
calls++;
registration!.Dispose();
});
var thrown = Assert.Throws<InvalidOperationException>(() => Put(cache, "one"))!;
Assert.That(thrown.Message, Does.Contain("journal"), "disposing from inside a journal is refused");
registration.Dispose();
Put(cache, "two");
Assert.That(calls, Is.EqualTo(1), "the outside dispose removed it, though the first attempt threw");
registration.Dispose();
}
[Test]
public void ALoadIsJournaledWithItsChanges()
{
var path = Path.Combine(_directory, "notes.cc");
using (var writer = CultCacheMessagePack.Create(path, new CultCacheOpenOptions { Registry = Registry }))
{
Put(writer, "a");
Put(writer, "b");
}
using var cache = Memory();
var journaled = new List<IReadOnlyList<CultCacheDocumentChange<object>>>();
using var journal = cache.AddJournal(changes => journaled.Add(changes));
cache.AddBackingStore(new SingleFileMessagePackBackingStore(path));
Assert.Multiple(() =>
{
Assert.That(journaled, Has.Count.EqualTo(1), "one admission, one call");
Assert.That(journaled[0].Select(change => change.Key).OrderBy(key => key.Value),
Is.EqualTo(new[] { KeyOf("a"), KeyOf("b") }));
});
}
// A store that admits a load and then fails: the hold's body throws after the journal has already failed.
private sealed class LoadThenFailStore : CacheBackingStore
{
public bool Armed;
public override void PullAll() => Held(() =>
{
if (!Armed)
return;
var record = new CultStoredDocument(
KeyOf("loaded"), "2026-01-01T00:00:00.0000000+00:00", Registry.GetRequired(typeof(JournalNote)), new JournalNote { Name = "loaded" });
Loaded!(new[] { record }, Array.Empty<CultStoredDocument>());
throw new InvalidOperationException("store failed after the load");
});
public override void Push(CultStoredDocument entry) { }
public override void Delete(CultStoredDocument entry) { }
public override void PushAll() { }
public override CultCommitOutcome CommitBatch(CultCommitRequest request, bool wait) => CultCommitOutcome.Committed;
}
[Test]
public void AHoldWhoseBodyThrowsSurfacesTheBodyFirstThenTheJournalFailures()
{
using var cache = Memory();
var store = new LoadThenFailStore();
cache.AddBackingStore(store);
using var a = cache.AddJournal(_ => throw new InvalidOperationException("journal a"));
using var b = cache.AddJournal(_ => throw new InvalidOperationException("journal b"));
store.Armed = true;
var thrown = Assert.Throws<AggregateException>(() => store.PullAll())!;
Assert.Multiple(() =>
{
Assert.That(thrown.InnerExceptions.Select(exception => exception.Message),
Is.EqualTo(new[] { "store failed after the load", "journal a", "journal b" }));
Assert.That(cache.Get<JournalNote>(KeyOf("loaded")), Is.Not.Null, "the load landed");
});
}
[Test]
public void AHoldWhoseBodyThrowsAloneRethrowsItAsItself()
{
using var cache = Memory();
var store = new LoadThenFailStore();
cache.AddBackingStore(store);
using var journal = cache.AddJournal(_ => { });
store.Armed = true;
var thrown = Assert.Throws<InvalidOperationException>(() => store.PullAll())!;
Assert.That(thrown.Message, Is.EqualTo("store failed after the load"));
}
[Test]
public void AJournalThatThrowsDuringALoadLeavesTheStoreAttachedAndTheLoadAdopted()
{
var path = Path.Combine(_directory, "notes.cc");
using (var writer = CultCacheMessagePack.Create(path, new CultCacheOpenOptions { Registry = Registry }))
Put(writer, "stored");
var cache = Memory();
var seen = new List<long>();
using var observer = cache.Watch<JournalNote>().Subscribe(change => seen.Add(change.Sequence));
using var journal = cache.AddJournal(_ => throw new InvalidOperationException("journal boom"));
var thrown = Assert.Throws<InvalidOperationException>(() => cache.AddBackingStore(new SingleFileMessagePackBackingStore(path)))!;
Assert.Multiple(() =>
{
Assert.That(thrown.Message, Is.EqualTo("journal boom"));
Assert.That(cache.BackingStores, Has.Count.EqualTo(1), "the store stays attached: it and the cache agree");
Assert.That(cache.Get<JournalNote>(KeyOf("stored")), Is.Not.Null);
Assert.That(seen, Is.EqualTo(new[] { 1L }));
});
}
}
}

256
tests/GameCult.Caching.Tests/CultCacheObserverTests.cs Normal file
View file

@ -0,0 +1,256 @@
#nullable enable
using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.CompilerServices;
using GameCult.Caching;
using GameCult.Caching.MessagePack;
using MessagePack;
using NUnit.Framework;
using R3;
namespace GameCult.Caching.Tests
{
// Cut 2 of docs/cultcache-publication-cut.md: the cache keeps its own observers and catches per observer, so no
// observer decides whether another receives a change. R3 routes a subscriber's exception to the unhandled handler;
// these tests install a fail-fast (rethrowing) handler, the only way an exception reaches the cache.
[NonParallelizable]
public class CultCacheObserverTests
{
private static readonly CultDocumentRegistry Registry = CultDocumentRegistry.ForTypes(new[] { typeof(ObservedPing) });
private Action<Exception> _previousHandler = _ => { };
[SetUp]
public void FailFast()
{
_previousHandler = ObservableSystem.GetUnhandledExceptionHandler();
ObservableSystem.RegisterUnhandledExceptionHandler(exception => throw exception);
}
[TearDown]
public void Restore() => ObservableSystem.RegisterUnhandledExceptionHandler(_previousHandler);
[CultDocument("tests.observed_ping", "tests.observed_ping.v1")]
[MessagePackObject(AllowPrivate = true)]
internal sealed class ObservedPing
{
[Key(0)]
public string Text = string.Empty;
}
private static CultRecordKey KeyOf(string text) => new($"observed:{text}");
private static void Send(CultCache cache, params string[] texts) =>
cache.Commit(batch =>
{
foreach (var text in texts)
batch.Upsert(typeof(ObservedPing), new ObservedPing { Text = text }, KeyOf(text));
});
[Test]
public void AnEscapedObserverExceptionReachesTheWriterAfterEveryObserverSawEveryChange()
{
using var cache = new CultCache(Registry);
var thrower = new List<string>();
var later = new List<string>();
using var first = cache.Watch<ObservedPing>().Subscribe(change =>
{
thrower.Add(change.Document!.Text);
if (change.Document.Text == "boom")
throw new InvalidOperationException("observer boom");
});
using var second = cache.Watch<ObservedPing>().Subscribe(change => later.Add(change.Document!.Text));
var escaped = Assert.Throws<InvalidOperationException>(() => Send(cache, "boom", "after"))!;
Assert.Multiple(() =>
{
Assert.That(escaped.Message, Is.EqualTo("observer boom"));
Assert.That(later, Is.EqualTo(new[] { "boom", "after" }), "the observer after the thrower saw every change of the hold");
Assert.That(thrower, Is.EqualTo(new[] { "boom", "after" }), "the thrower kept receiving after its own exception");
Assert.That(cache.Get<ObservedPing>(KeyOf("boom")), Is.Not.Null, "the commit stands");
Assert.That(cache.Get<ObservedPing>(KeyOf("after")), Is.Not.Null);
});
Send(cache, "later");
Assert.That(thrower, Is.EqualTo(new[] { "boom", "after", "later" }), "the thrower stayed subscribed");
Assert.That(later, Is.EqualTo(new[] { "boom", "after", "later" }));
}
[Test]
public void SeveralEscapedObserverExceptionsArriveTogether()
{
using var cache = new CultCache(Registry);
using var one = cache.Watch<ObservedPing>().Subscribe(_ => throw new InvalidOperationException("one"));
using var two = cache.Watch<ObservedPing>().Subscribe(_ => throw new InvalidOperationException("two"));
var escaped = Assert.Throws<AggregateException>(() => Send(cache, "boom"))!;
Assert.That(escaped.InnerExceptions.Select(exception => exception.Message), Is.EqualTo(new[] { "one", "two" }));
}
[Test]
public void ObserversMayJoinAndLeaveDuringADeliveryWithoutBreakingTheOthers()
{
using var cache = new CultCache(Registry);
var a = new List<long>();
var b = new List<long>();
var c = new List<long>();
var d = new List<long>();
IDisposable? leaving = null;
IDisposable? joined = null;
using var first = cache.Watch<ObservedPing>().Subscribe(change =>
{
a.Add(change.Sequence);
if (a.Count != 1)
return;
leaving!.Dispose();
joined = cache.Watch<ObservedPing>().Subscribe(next => c.Add(next.Sequence));
});
leaving = cache.Watch<ObservedPing>().Subscribe(change => b.Add(change.Sequence));
using var last = cache.Watch<ObservedPing>().Subscribe(change => d.Add(change.Sequence));
Send(cache, "one", "two");
joined?.Dispose();
Assert.Multiple(() =>
{
Assert.That(a, Is.EqualTo(new[] { 1L, 2L }));
Assert.That(d, Is.EqualTo(new[] { 1L, 2L }), "an observer that did not move still saw everything");
Assert.That(b, Does.Not.Contain(2L), "a disposed observer stops receiving");
Assert.That(c, Is.EqualTo(new[] { 2L }), "an observer that joined mid-hold receives from the next change");
});
}
[Test]
public void AnUnsubscribedObserverIsReleasedByTheCache()
{
using var cache = new CultCache(Registry);
var kept = new List<IDisposable>();
var live = Subscribe(cache, kept, dispose: false);
var gone = Subscribe(cache, kept, dispose: true);
Send(cache, "one");
CollectGarbage();
Assert.Multiple(() =>
{
Assert.That(live.IsAlive, Is.True, "a live subscription is held by the cache");
Assert.That(gone.IsAlive, Is.False, "an unsubscribed observer is no longer held");
});
GC.KeepAlive(kept);
GC.KeepAlive(cache);
}
// A disposed R3 subscription ignores OnNext, so a stale entry in the cache is invisible to a receiver; it shows as
// a leak: the cache keeps the subscriber alive. The test frame must not hold the subscription itself.
[MethodImpl(MethodImplOptions.NoInlining)]
private static WeakReference Subscribe(CultCache cache, List<IDisposable> kept, bool dispose)
{
var target = new object();
var subscription = cache.Watch<ObservedPing>().Subscribe(_ => GC.KeepAlive(target));
if (dispose)
subscription.Dispose();
else
kept.Add(subscription);
return new WeakReference(target);
}
private static void CollectGarbage()
{
GC.Collect();
GC.WaitForPendingFinalizers();
GC.Collect();
}
// A store that admits a load and then fails: the hold's body throws after the changes were minted.
private sealed class LoadThenFailStore : CacheBackingStore
{
public bool Armed;
public override void PullAll() => Held(() =>
{
if (!Armed)
return;
var record = new CultStoredDocument(
KeyOf("loaded"), "2026-01-01T00:00:00.0000000+00:00", Registry.GetRequired(typeof(ObservedPing)), new ObservedPing { Text = "loaded" });
Loaded!(new[] { record }, Array.Empty<CultStoredDocument>());
throw new InvalidOperationException("store failed after the load");
});
public override void Push(CultStoredDocument entry) { }
public override void Delete(CultStoredDocument entry) { }
public override void PushAll() { }
public override CultCommitOutcome CommitBatch(CultCommitRequest request, bool wait) => CultCommitOutcome.Committed;
}
[Test]
public void AHoldWhoseBodyThrowsStillPublishesTheChangesItMinted()
{
using var cache = new CultCache(Registry);
var store = new LoadThenFailStore();
cache.AddBackingStore(store);
var seen = new List<long>();
using var subscription = cache.Watch<ObservedPing>().Subscribe(change => seen.Add(change.Sequence));
store.Armed = true;
var thrown = Assert.Throws<InvalidOperationException>(() => store.PullAll())!;
Assert.Multiple(() =>
{
Assert.That(thrown.Message, Is.EqualTo("store failed after the load"), "the body's own exception wins");
Assert.That(cache.Get<ObservedPing>(KeyOf("loaded")), Is.Not.Null, "the load landed");
Assert.That(seen, Is.EqualTo(new[] { 1L }), "and reached its observers");
});
}
[Test]
public void DisposeCompletesObserversAndACompletedCacheCompletesNewOnes()
{
var cache = new CultCache(Registry);
var completed = 0;
using var before = cache.Watch<ObservedPing>().Subscribe(_ => { }, _ => completed++);
cache.Dispose();
Assert.That(completed, Is.EqualTo(1));
using var after = cache.Watch<ObservedPing>().Subscribe(_ => { }, _ => completed++);
Assert.That(completed, Is.EqualTo(2), "subscribing to a disposed cache completes at once");
}
// Completing an observer is publication too: every observer is completed, and a fail-fast handler's exception
// reaches the disposer afterwards (an AggregateException for several).
[Test]
public void ACompletionThatEscapesReachesTheDisposerAfterEveryObserverCompleted()
{
var cache = new CultCache(Registry);
var completed = new List<string>();
using var one = cache.Watch<ObservedPing>().Subscribe(_ => { }, _ =>
{
completed.Add("one");
throw new InvalidOperationException("one");
});
using var two = cache.Watch<ObservedPing>().Subscribe(_ => { }, _ =>
{
completed.Add("two");
throw new InvalidOperationException("two");
});
var escaped = Assert.Throws<AggregateException>(() => cache.Dispose())!;
Assert.That(completed, Is.EqualTo(new[] { "one", "two" }));
Assert.That(escaped.InnerExceptions.Select(exception => exception.Message), Is.EqualTo(new[] { "one", "two" }));
}
[Test]
public void ASingleEscapedCompletionIsRethrownAsItself()
{
var cache = new CultCache(Registry);
using var only = cache.Watch<ObservedPing>().Subscribe(_ => { }, _ => throw new InvalidOperationException("only"));
var escaped = Assert.Throws<InvalidOperationException>(() => cache.Dispose())!;
Assert.That(escaped.Message, Is.EqualTo("only"));
}
}
}

258
tests/GameCult.Caching.Tests/CultCacheProjectionTests.cs Normal file
View file

@ -0,0 +1,258 @@
#nullable enable
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Threading;
using GameCult.Caching;
using GameCult.Caching.MessagePack;
using MessagePack;
using NUnit.Framework;
using R3;
namespace GameCult.Caching.Tests
{
// Cut 1 of docs/cultcache-publication-cut.md: an admission reads every landing record's name and index values once,
// before land and before any sequence is minted. A getter that throws refuses the whole change set.
[NonParallelizable]
public class CultCacheProjectionTests
{
private static readonly CultDocumentRegistry Registry = CultDocumentRegistry.ForTypes(new[] { typeof(ProjectedRecord) });
private string _directory = string.Empty;
[SetUp]
public void Prepare()
{
_directory = Path.Combine(Path.GetTempPath(), $"cultlib-projection-{Guid.NewGuid():N}");
Directory.CreateDirectory(_directory);
ProjectedRecord.Armed = false;
ProjectedRecord.Reads = 0;
}
[TearDown]
public void Cleanup()
{
ProjectedRecord.Armed = false;
if (Directory.Exists(_directory))
Directory.Delete(_directory, recursive: true);
}
[CultDocument("tests.projected_record", "tests.projected_record.v1")]
[MessagePackObject(AllowPrivate = true)]
internal sealed class ProjectedRecord
{
public static long Reads;
public static volatile bool Armed;
[IgnoreMember] private string _name = string.Empty;
[IgnoreMember] private string _code = string.Empty;
[Key(0)]
[CultName]
public string Name
{
get
{
Interlocked.Increment(ref Reads);
if (Armed && _name == "boom")
throw new InvalidOperationException("name getter boom");
return _name;
}
set => _name = value;
}
[Key(1)]
[CultIndex("code")]
public string Code
{
get
{
Interlocked.Increment(ref Reads);
if (Armed && _code == "boom")
throw new InvalidOperationException("index getter boom");
return _code;
}
set => _code = value;
}
}
private static readonly CultRecordKey OkKey = new("projected:ok");
private static readonly CultRecordKey BoomKey = new("projected:boom");
private static ProjectedRecord Ok() => new() { Name = "ok", Code = "ok-code" };
private static ProjectedRecord Boom(bool name) => name ? new ProjectedRecord { Name = "boom", Code = "boom-code" } : new ProjectedRecord { Name = "boom-name", Code = "boom" };
private string PathOf(string name) => Path.Combine(_directory, name);
private static CultCache Open(string path) => CultCacheMessagePack.Create(path, new CultCacheOpenOptions { Registry = Registry });
private static void Put(CultCache cache, ProjectedRecord document, CultRecordKey key) =>
cache.Commit(batch => batch.Upsert(typeof(ProjectedRecord), document, key));
private static long SequenceOf(CultCache cache) => cache.GetWithSequence(OkKey).Sequence;
private static void StageBoth(CultCacheBatch batch, ProjectedRecord boom)
{
batch.Upsert(typeof(ProjectedRecord), Ok(), OkKey);
batch.Upsert(typeof(ProjectedRecord), boom, BoomKey);
}
[TestCase(true, "[CultName] member Name")]
[TestCase(false, "index 'code' member Code")]
public void ARefusedGetterMintsNothingAndChangesNothing(bool nameThrows, string member)
{
using var cache = new CultCache(Registry);
var seen = new List<long>();
using var watch = cache.Watch<ProjectedRecord>().Subscribe(change => seen.Add(change.Sequence));
Put(cache, new ProjectedRecord { Name = "seed", Code = "seed-code" }, new CultRecordKey("projected:seed"));
var before = SequenceOf(cache);
seen.Clear();
ProjectedRecord.Armed = true;
var refusal = Assert.Throws<InvalidOperationException>(() => cache.Commit(batch => StageBoth(batch, Boom(nameThrows))))!;
ProjectedRecord.Armed = false;
Assert.Multiple(() =>
{
Assert.That(refusal.Message, Does.Contain(BoomKey.Value).And.Contain(member).And.Contain("getter boom"));
Assert.That(SequenceOf(cache), Is.EqualTo(before), "a refused admission mints nothing");
Assert.That(cache.Get<ProjectedRecord>(OkKey), Is.Null, "the good record did not land");
Assert.That(cache.Get<ProjectedRecord>(BoomKey), Is.Null);
Assert.That(cache.GetByName<ProjectedRecord>("ok"), Is.Null, "the name index holds nothing of the refused set");
Assert.That(cache.GetByIndex<ProjectedRecord>("code", "ok-code"), Is.Null);
Assert.That(seen, Is.Empty, "nothing was published");
});
Put(cache, Ok(), OkKey);
Assert.That(SequenceOf(cache), Is.EqualTo(before + 1), "the next write is previous + 1");
Assert.That(seen, Is.EqualTo(new[] { before + 1 }));
}
[Test]
public void ARefusedSingleWriteMintsNothing()
{
using var cache = new CultCache(Registry);
ProjectedRecord.Armed = true;
var refusal = Assert.Throws<InvalidOperationException>(() => Put(cache, Boom(true), BoomKey))!;
ProjectedRecord.Armed = false;
Assert.That(refusal.Message, Does.Contain(BoomKey.Value));
Assert.That(SequenceOf(cache), Is.EqualTo(0));
Assert.That(cache.Get<ProjectedRecord>(BoomKey), Is.Null);
}
[Test]
public void ARefusedBatchLeavesAStoreByteIdenticalAndReopensWithNeitherKey()
{
var path = PathOf("projected.cc");
using (var seed = Open(path))
Put(seed, new ProjectedRecord { Name = "seed", Code = "seed-code" }, new CultRecordKey("projected:seed"));
var before = File.ReadAllBytes(path);
using (var cache = Open(path))
{
ProjectedRecord.Armed = true;
Assert.Throws<InvalidOperationException>(() => cache.Commit(batch => StageBoth(batch, Boom(true))));
ProjectedRecord.Armed = false;
Assert.That(File.ReadAllBytes(path), Is.EqualTo(before));
}
using var reopened = Open(path);
Assert.Multiple(() =>
{
Assert.That(reopened.Get<ProjectedRecord>(OkKey), Is.Null);
Assert.That(reopened.Get<ProjectedRecord>(BoomKey), Is.Null);
Assert.That(File.ReadAllBytes(path), Is.EqualTo(before));
});
}
[Test]
public void ALoadWhoseGetterThrowsRefusesTheOpenAndLeavesTheFileByteIdentical()
{
var path = PathOf("poisoned.cc");
using (var seed = Open(path))
Put(seed, Boom(true), BoomKey);
var before = File.ReadAllBytes(path);
ProjectedRecord.Armed = true;
var refusal = Assert.Throws<InvalidOperationException>(() => Open(path))!;
ProjectedRecord.Armed = false;
Assert.That(refusal.Message, Does.Contain(BoomKey.Value).And.Contain("[CultName] member Name"));
Assert.That(File.ReadAllBytes(path), Is.EqualTo(before));
}
// A counting getter: each landing record is read once for its name and once for its index, and never again by the
// variant rules or the indexer. A variant re-resolved by a base edit lands too, and is read once.
[Test]
public void EachLandedRecordIsReadOncePerGetterIncludingAVariantDependent()
{
var codec = CultCacheMessagePack.CreateCodec(Registry);
using var cache = new CultCache(Registry, codec);
var baseKey = new CultRecordKey("projected:base");
Put(cache, new ProjectedRecord { Name = "base", Code = "base-code" }, baseKey);
cache.Commit(batch => batch.UpsertVariant(new CultRecordKey("projected:variant"), baseKey, new[]
{
cache.Override<ProjectedRecord>(nameof(ProjectedRecord.Name), "variant"),
cache.Override<ProjectedRecord>(nameof(ProjectedRecord.Code), "variant-code")
}));
// Resolving a variant serializes its base, and the serializer reads the same getters: count those reads apart.
ProjectedRecord.Reads = 0;
codec.Serialize(new ProjectedRecord(), typeof(ProjectedRecord), typeof(ProjectedRecord));
var serializerReads = Interlocked.Read(ref ProjectedRecord.Reads);
ProjectedRecord.Reads = 0;
Put(cache, new ProjectedRecord { Name = "base", Code = "base-code-2" }, baseKey);
// Two landing records (the base, and the variant re-resolved against it), two getters each, plus the one serialization.
Assert.That(Interlocked.Read(ref ProjectedRecord.Reads), Is.EqualTo(4 + serializerReads));
Assert.That(cache.GetByIndex<ProjectedRecord>("code", "base-code-2"), Is.Not.Null);
Assert.That(cache.GetByName<ProjectedRecord>("variant"), Is.Not.Null);
}
// The index rule counts the holders a commit leaves behind: a record removed by the same commit no longer holds its value.
[Test]
public void AVariantMayTakeAnIndexedValueFromARecordTheSameCommitRemoves()
{
using var cache = new CultCache(Registry, CultCacheMessagePack.CreateCodec(Registry));
var baseKey = new CultRecordKey("projected:base");
var holderKey = new CultRecordKey("projected:holder");
Put(cache, new ProjectedRecord { Name = "base", Code = "base-code" }, baseKey);
Put(cache, new ProjectedRecord { Name = "holder", Code = "shared" }, holderKey);
var landed = cache.Commit(batch =>
{
batch.Remove(holderKey);
batch.UpsertVariant(new CultRecordKey("projected:variant"), baseKey, new[]
{
cache.Override<ProjectedRecord>(nameof(ProjectedRecord.Name), "variant"),
cache.Override<ProjectedRecord>(nameof(ProjectedRecord.Code), "shared")
});
});
Assert.That(landed, Is.True);
Assert.That(cache.GetByIndex<ProjectedRecord>("code", "shared"), Is.Not.Null);
Assert.That(cache.Get<ProjectedRecord>(holderKey), Is.Null);
}
// A blank name is no name: a variant and its base that are both blank do not shadow each other.
[Test]
public void ABlankNameIsNoNameSoBlankVariantAndBaseDoNotCollide()
{
using var cache = new CultCache(Registry, CultCacheMessagePack.CreateCodec(Registry));
var baseKey = new CultRecordKey("projected:base");
Put(cache, new ProjectedRecord { Name = "", Code = "base-code" }, baseKey);
cache.Commit(batch => batch.UpsertVariant(new CultRecordKey("projected:variant"), baseKey, new[]
{
cache.Override<ProjectedRecord>(nameof(ProjectedRecord.Name), ""),
cache.Override<ProjectedRecord>(nameof(ProjectedRecord.Code), "variant-code")
}));
Assert.That(cache.Get<ProjectedRecord>(new CultRecordKey("projected:variant")), Is.Not.Null);
}
}
}

415
tests/GameCult.Caching.Tests/CultMathJsonTests.cs Normal file
View file

@ -0,0 +1,415 @@
using System;
using System.Collections;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
using System.Text.Json;
using System.Text.Json.Serialization;
using CultMath;
using FluentAssertions;
using GameCult.Caching.MessagePack;
using NUnit.Framework;
namespace GameCult.Caching.Tests
{
public sealed class CultMathJsonTests
{
private static readonly JsonSerializerOptions Options = new JsonSerializerOptions().AddCultMathConverters();
private static string Json<T>(T value) => JsonSerializer.Serialize(value, Options);
private static T Parse<T>(string json) => JsonSerializer.Deserialize<T>(json, Options)!;
[Test]
public void EveryShape_IsPinned()
{
Json(new float2(1f, 2f)).Should().Be("""{"x":1,"y":2}""");
Json(new float3(1f, 2f, 3f)).Should().Be("""{"x":1,"y":2,"z":3}""");
Json(new float4(1f, 2f, 3f, 4f)).Should().Be("""{"x":1,"y":2,"z":3,"w":4}""");
Json(new double2(1d, 2.5d)).Should().Be("""{"x":1,"y":2.5}""");
Json(new double3(1d, 2d, 3d)).Should().Be("""{"x":1,"y":2,"z":3}""");
Json(new int2(3, -4)).Should().Be("""{"x":3,"y":-4}""");
Json(new int3(1, 2, 3)).Should().Be("""{"x":1,"y":2,"z":3}""");
Json(new int4(1, 2, 3, 4)).Should().Be("""{"x":1,"y":2,"z":3,"w":4}""");
Json(new bool2(true, false)).Should().Be("""{"x":true,"y":false}""");
Json(new bool3(true, false, true)).Should().Be("""{"x":true,"y":false,"z":true}""");
Json(new bool4(true, false, false, true)).Should().Be("""{"x":true,"y":false,"z":false,"w":true}""");
Json(new quaternion(0f, 0f, 0f, 1f)).Should().Be("""{"x":0,"y":0,"z":0,"w":1}""");
Json(new Color32(255, 128, 0, 7)).Should().Be("""{"r":255,"g":128,"b":0,"a":7}""");
Json(new CultMath.Random(12345u)).Should().Be("""{"state":12345}""");
Json(new float2x2(1f, 2f, 3f, 4f)).Should().Be("""[{"x":1,"y":2},{"x":3,"y":4}]""");
Json(new float3x3(1f, 2f, 3f, 4f, 5f, 6f, 7f, 8f, 9f)).Should().Be(
"""[{"x":1,"y":2,"z":3},{"x":4,"y":5,"z":6},{"x":7,"y":8,"z":9}]""");
Json(new rect(5f, 7f, -1f, -2f)).Should().Be("""{"min":{"x":-1,"y":-2},"max":{"x":5,"y":7}}""");
Json(new CultCellular(new float4(1f, 2f, 3f, 4f), new float4(5f, 6f, 7f, 8f), 0.25f)).Should().Be(
"""{"nearest":{"x":1,"y":2,"z":3,"w":4},"edge":{"x":5,"y":6,"z":7,"w":8},"id":0.25}""");
}
[Test]
public void EveryShape_RoundTrips()
{
RoundTrip(new float2(1.5f, -2f));
RoundTrip(new float3(1f, 2f, 3f));
RoundTrip(new float4(1f, 2f, 3f, 4f));
RoundTrip(new double2(1e300, -2.5));
RoundTrip(new double3(1, 2, 3));
RoundTrip(new int2(-7, 900000));
RoundTrip(new int3(1, -2, 3));
RoundTrip(new int4(1, 2, 3, int.MinValue));
RoundTrip(new bool2(true, false));
RoundTrip(new bool3(false, true, true));
RoundTrip(new bool4(true, false, false, true));
RoundTrip(new quaternion(0.1f, 0.2f, 0.3f, 0.9f));
RoundTrip(new Color32(255, 128, 0, 7));
RoundTrip(new CultMath.Random(12345u));
RoundTrip(new float2x2(1f, 2f, 3f, 4f));
RoundTrip(new float3x3(1f, 2f, 3f, 4f, 5f, 6f, 7f, 8f, 9f));
RoundTrip(new rect(-1f, -2f, 5f, 7f));
RoundTrip(new CultCellular(new float4(1, 2, 3, 4), new float4(5, 6, 7, 8), 0.25f));
}
[Test]
public void Reading_SkipsUnknownPropertiesAndZeroesMissingOnes()
{
Parse<float3>("""{"z":3,"extra":[1,2],"x":1}""").Should().Be(new float3(1f, 0f, 3f));
Parse<Color32>("""{"r":1,"g":2,"b":3}""").Should().Be(new Color32(1, 2, 3, 255));
Parse<float2?>("null").Should().BeNull();
}
[Test]
public void Reading_RefusesShapesThatAreNotTheType()
{
Action array = () => Parse<float2>("[1,2]");
array.Should().Throw<JsonException>();
Action shortMatrix = () => Parse<float2x2>("""[{"x":1,"y":2}]""");
shortMatrix.Should().Throw<JsonException>();
Action nil = () => Parse<rect>("null");
nil.Should().Throw<JsonException>();
}
[Test]
public void EveryPublicCultMathValueType_HasAConverter()
{
var valueTypes = typeof(float2).Assembly.GetExportedTypes()
.Where(t => t.IsValueType && !t.IsEnum)
.ToHashSet();
valueTypes.Except(CultMathJson.ConvertedTypes).Should().BeEmpty();
CultMathJson.ConvertedTypes.Except(valueTypes).Should().BeEmpty();
foreach (var type in valueTypes)
{
var json = JsonSerializer.Serialize(Activator.CreateInstance(type), type, Options);
JsonSerializer.Deserialize(json, type, Options).Should().Be(Activator.CreateInstance(type));
}
}
private static JsonSerializerOptions With(JsonNumberHandling handling = JsonNumberHandling.Strict, bool ignoreCase = false) =>
new JsonSerializerOptions { NumberHandling = handling, PropertyNameCaseInsensitive = ignoreCase }.AddCultMathConverters();
private const JsonNumberHandling Named = JsonNumberHandling.AllowNamedFloatingPointLiterals;
[Test]
public void NonFiniteFloats_NeedNamedLiterals_AsInSystemTextJson()
{
Action nan = () => JsonSerializer.Serialize(new float2(float.NaN, 1f), Options);
nan.Should().Throw<JsonException>();
Action inf = () => JsonSerializer.Serialize(new double3(1d, double.PositiveInfinity, 0d), Options);
inf.Should().Throw<JsonException>();
Action named = () => JsonSerializer.Deserialize<float2>("""{"x":"NaN","y":1}""", Options);
named.Should().Throw<JsonException>();
var options = With(Named);
JsonSerializer.Serialize(new float2(float.NaN, 1f), options).Should().Be("""{"x":"NaN","y":1}""");
JsonSerializer.Serialize(new float3(float.PositiveInfinity, float.NegativeInfinity, 0f), options)
.Should().Be("""{"x":"Infinity","y":"-Infinity","z":0}""");
var back = JsonSerializer.Deserialize<float3>("""{"x":"Infinity","y":"-Infinity","z":"NaN"}""", options);
back.x.Should().Be(float.PositiveInfinity);
back.y.Should().Be(float.NegativeInfinity);
float.IsNaN(back.z).Should().BeTrue();
JsonSerializer.Deserialize<double2>("""{"x":"NaN","y":"-Infinity"}""", options).y.Should().Be(double.NegativeInfinity);
}
[Test]
public void Numbers_OutsideTheComponentType_AreRefused()
{
foreach (var json in new[] { """{"x":1e40}""", """{"x":-1e40}""" })
{
Action tooBig = () => Parse<float2>(json);
tooBig.Should().Throw<JsonException>(json);
}
Action doubleTooBig = () => Parse<double2>("""{"x":1e400}""");
doubleTooBig.Should().Throw<JsonException>();
Action alpha = () => Parse<Color32>("""{"r":300}""");
alpha.Should().Throw<JsonException>();
Action negativeByte = () => Parse<Color32>("""{"g":-1}""");
negativeByte.Should().Throw<JsonException>();
Action fraction = () => Parse<int2>("""{"x":1.5}""");
fraction.Should().Throw<JsonException>();
Action wrongToken = () => Parse<float2>("""{"x":"a"}""");
wrongToken.Should().Throw<JsonException>();
Action boolean = () => Parse<bool2>("""{"x":1}""");
boolean.Should().Throw<JsonException>();
Action negativeState = () => Parse<CultMath.Random>("""{"state":-1}""");
negativeState.Should().Throw<JsonException>();
}
[Test]
public void NumberHandling_QuotesAndUnquotesNumbers()
{
JsonSerializer.Serialize(new float2(1f, 2.5f), With(JsonNumberHandling.WriteAsString)).Should().Be("""{"x":"1","y":"2.5"}""");
JsonSerializer.Serialize(new int2(1, 2), With(JsonNumberHandling.WriteAsString)).Should().Be("""{"x":"1","y":"2"}""");
JsonSerializer.Deserialize<float2>("""{"x":"1.5","y":2}""", With(JsonNumberHandling.AllowReadingFromString))
.Should().Be(new float2(1.5f, 2f));
JsonSerializer.Deserialize<int2>("""{"x":"7","y":2}""", With(JsonNumberHandling.AllowReadingFromString))
.Should().Be(new int2(7, 2));
Action strict = () => Parse<float2>("""{"x":"1.5"}""");
strict.Should().Throw<JsonException>();
Action tooBig = () => JsonSerializer.Deserialize<float2>("""{"x":"1e40"}""", With(JsonNumberHandling.AllowReadingFromString));
tooBig.Should().Throw<JsonException>();
}
[Test]
public void PropertyNames_FollowTheCaseSensitivityOption()
{
JsonSerializer.Deserialize<float2>("""{"X":1,"Y":2}""", With(ignoreCase: true)).Should().Be(new float2(1f, 2f));
JsonSerializer.Deserialize<rect>("""{"MIN":{"X":1,"y":2},"Max":{"x":3,"Y":4}}""", With(ignoreCase: true))
.Should().Be(new rect(1f, 2f, 3f, 4f));
Parse<float2>("""{"X":1,"Y":2}""").Should().Be(float2.zero);
}
[Test]
public void EveryShape_WorksAsADictionaryKey()
{
Json(new Dictionary<int2, string> { [new int2(1, -2)] = "a" }).Should().Be("""{"1,-2":"a"}""");
Parse<Dictionary<int2, string>>("""{"1,-2":"a"}""")[new int2(1, -2)].Should().Be("a");
KeyOf(new float2(1.5f, -2f)).Should().Be("1.5,-2");
KeyOf(new float3(0.1f, 2f, 3f)).Should().Be("0.1,2,3");
KeyOf(new bool2(true, false)).Should().Be("true,false");
KeyOf(new Color32(255, 128, 0, 7)).Should().Be("255,128,0,7");
KeyOf(new CultMath.Random(12345u)).Should().Be("12345");
KeyOf(new float2x2(1f, 2f, 3f, 4f)).Should().Be("1,2,3,4");
KeyOf(new rect(5f, 7f, -1f, -2f)).Should().Be("-1,-2,5,7");
KeyOf(new float2(float.NaN, float.NegativeInfinity), Named).Should().Be("NaN,-Infinity");
var nan = JsonSerializer.Deserialize<Dictionary<float2, int>>("""{"NaN,Infinity":1}""", With(Named))!;
nan.Keys.Single().y.Should().Be(float.PositiveInfinity);
Action refuseNonFinite = () => KeyOf(new float2(float.NaN, 0f));
refuseNonFinite.Should().Throw<JsonException>();
Action wrongArity = () => Parse<Dictionary<float3, int>>("""{"1,2":1}""");
wrongArity.Should().Throw<JsonException>();
Action overflow = () => Parse<Dictionary<float2, int>>("""{"1e40,0":1}""");
overflow.Should().Throw<JsonException>();
}
[Test]
public void EveryPublicCultMathValueType_RoundTripsAsADictionaryKey()
{
foreach (var type in typeof(float2).Assembly.GetExportedTypes().Where(t => t.IsValueType && !t.IsEnum))
{
var dictionary = (IDictionary)Activator.CreateInstance(typeof(Dictionary<,>).MakeGenericType(type, typeof(int)))!;
dictionary.Add(Activator.CreateInstance(type)!, 7);
var json = JsonSerializer.Serialize(dictionary, dictionary.GetType(), Options);
var back = (IDictionary)JsonSerializer.Deserialize(json, dictionary.GetType(), Options)!;
back.Count.Should().Be(1, type.Name);
back[Activator.CreateInstance(type)!].Should().Be(7, type.Name);
}
}
private static string KeyOf<T>(T value, JsonNumberHandling handling = JsonNumberHandling.Strict) where T : notnull
{
var json = JsonSerializer.Serialize(new Dictionary<T, int> { [value] = 1 }, With(handling));
return JsonDocument.Parse(json).RootElement.EnumerateObject().Single().Name;
}
private const JsonNumberHandling ReadStrings = JsonNumberHandling.AllowReadingFromString;
private const JsonNumberHandling WriteStrings = JsonNumberHandling.WriteAsString;
// Every row was measured against System.Text.Json's own float.
[Test]
public void NamedLiterals_FollowTheSameFlagsAsSystemTextJsonFloat()
{
foreach (var handling in new[] { Named, ReadStrings, ReadStrings | Named })
{
JsonSerializer.Deserialize<float2>("""{"x":"NaN","y":"-Infinity"}""", With(handling))!.y.Should().Be(float.NegativeInfinity, handling.ToString());
}
foreach (var handling in new[] { JsonNumberHandling.Strict, WriteStrings })
{
Action read = () => JsonSerializer.Deserialize<float2>("""{"x":"NaN"}""", With(handling));
read.Should().Throw<JsonException>(handling.ToString());
}
foreach (var handling in new[] { Named, WriteStrings, WriteStrings | Named })
{
JsonSerializer.Serialize(new float2(float.NaN, float.PositiveInfinity), With(handling))
.Should().Be("""{"x":"NaN","y":"Infinity"}""", handling.ToString());
}
foreach (var handling in new[] { JsonNumberHandling.Strict, ReadStrings })
{
Action write = () => JsonSerializer.Serialize(new float2(float.NaN, 0f), With(handling));
write.Should().Throw<JsonException>(handling.ToString());
}
JsonSerializer.Serialize(new float2(1f, float.NaN), With(WriteStrings)).Should().Be("""{"x":"1","y":"NaN"}""");
JsonSerializer.Deserialize<double3>("""{"z":"Infinity"}""", With(ReadStrings))!.z.Should().Be(double.PositiveInfinity);
JsonSerializer.Serialize(new double2(double.NegativeInfinity, 2d), With(WriteStrings)).Should().Be("""{"x":"-Infinity","y":"2"}""");
}
[Test]
public void QuotedNumbers_RefuseWhitespace()
{
var options = With(ReadStrings);
foreach (var json in new[] { """{"x":" 1.5"}""", """{"x":"1.5 "}""", """{"x":"\t1"}""" })
{
Action floatRead = () => JsonSerializer.Deserialize<float2>(json, options);
floatRead.Should().Throw<JsonException>(json);
}
Action doubleRead = () => JsonSerializer.Deserialize<double2>("""{"y":" 2"}""", options);
doubleRead.Should().Throw<JsonException>();
Action intRead = () => JsonSerializer.Deserialize<int2>("""{"x":" 1"}""", options);
intRead.Should().Throw<JsonException>();
Action padded = () => JsonSerializer.Deserialize<float2>("""{"x":" NaN"}""", With(ReadStrings | Named));
padded.Should().Throw<JsonException>();
}
[Test]
public void UnknownProperties_AreSkippedWhateverTheyContain()
{
var expected = new float3(1f, 2f, 3f);
foreach (var json in new[]
{
"""{"u":{"a":[1,{"b":2}]},"x":1,"y":2,"z":3}""",
"""{"x":1,"u":[[1],[2,[3]]],"y":2,"v":{"a":{}},"z":3}""",
"""{"x":1,"y":2,"z":3,"u":{"a":{"b":[]}}}""",
"""{"x":1,"y":2,"z":3,"u":[{"a":[{}]}]}""",
})
{
Parse<float3>(json).Should().Be(expected, json);
}
Parse<rect>("""{"min":{"q":[{}],"x":1,"y":2},"skip":{"a":[1]},"max":{"x":3,"y":4,"q":{"a":[]}}}""")
.Should().Be(new rect(1f, 2f, 3f, 4f));
Parse<float2x2>("""[{"x":1,"u":[1],"y":2},{"y":4,"u":{"a":1},"x":3}]""").Should().Be(new float2x2(1f, 2f, 3f, 4f));
}
[Test]
public void UnmappedMemberHandling_Disallow_RefusesUnknownProperties()
{
var options = new JsonSerializerOptions { UnmappedMemberHandling = JsonUnmappedMemberHandling.Disallow }.AddCultMathConverters();
Action flat = () => JsonSerializer.Deserialize<float2>("""{"x":1,"z":3}""", options);
flat.Should().Throw<JsonException>();
Action nested = () => JsonSerializer.Deserialize<rect>("""{"min":{"x":1,"q":1},"max":{}}""", options);
nested.Should().Throw<JsonException>();
JsonSerializer.Deserialize<float2>("""{"x":1,"y":2}""", options).Should().Be(new float2(1f, 2f));
}
[Test]
public void DuplicateProperties_ReplaceTheWholeValue()
{
Parse<float2>("""{"x":1,"x":2,"y":5}""").Should().Be(new float2(2f, 5f));
Parse<rect>("""{"min":{"x":1,"y":2},"min":{"x":3},"max":{"x":4,"y":5}}""").Should().Be(new rect(3f, 0f, 4f, 5f));
Parse<Color32>("""{"a":9,"a":1}""").Should().Be(new Color32(0, 0, 0, 1));
Parse<Color32>("""{"r":3,"r":4}""").Should().Be(new Color32(4, 0, 0, 255));
Parse<CultCellular>("""{"nearest":{"x":1,"y":2},"nearest":{"z":3},"id":1}""")
.Should().Be(new CultCellular(new float4(0f, 0f, 3f, 0f), float4.zero, 1f));
}
[Test]
public void AllowDuplicatePropertiesFalse_RefusesRepeats()
{
var property = typeof(JsonSerializerOptions).GetProperty("AllowDuplicateProperties");
Assert.That(property, Is.Not.Null, "the test runtime's System.Text.Json has AllowDuplicateProperties");
var options = new JsonSerializerOptions().AddCultMathConverters();
property!.SetValue(options, false);
Action flat = () => JsonSerializer.Deserialize<float2>("""{"x":1,"x":2}""", options);
flat.Should().Throw<JsonException>();
Action nested = () => JsonSerializer.Deserialize<rect>("""{"min":{"x":1,"x":2}}""", options);
nested.Should().Throw<JsonException>();
JsonSerializer.Deserialize<float2>("""{"x":1,"y":2}""", options).Should().Be(new float2(1f, 2f));
}
[Test]
public void DictionaryKeys_ReadOnlyTheCanonicalSpelling()
{
foreach (var key in new[] { "1.0,2", "+1,2", " 1,2", "1, 2", "01,2", "1,2 ", "1e0,2", "1,2,", ",1" })
{
Action read = () => Parse<Dictionary<float2, int>>($$"""{"{{key}}":1}""");
read.Should().Throw<JsonException>(key);
}
foreach (var key in new[] { "01,2", "+1,2", "1,-0" })
{
Action read = () => Parse<Dictionary<int2, int>>($$"""{"{{key}}":1}""");
read.Should().Throw<JsonException>(key);
}
Action bools = () => Parse<Dictionary<bool2, int>>("""{"True,false":1}""");
bools.Should().Throw<JsonException>();
Parse<Dictionary<float2, int>>("""{"1,2":1}""").Should().ContainKey(new float2(1f, 2f));
}
[Test]
public void NegativeZero_IsADistinctKey()
{
var negative = new float2(-0f, 0f);
KeyOf(negative).Should().Be("-0,0");
KeyOf(float2.zero).Should().Be("0,0");
KeyOf(new double2(0d, -0d)).Should().Be("0,-0");
var parsed = Parse<Dictionary<float2, int>>("""{"-0,0":1}""").Keys.Single();
float.IsNegative(parsed.x).Should().BeTrue();
var positive = Parse<Dictionary<float2, int>>("""{"0,0":1}""").Keys.Single();
float.IsNegative(positive.x).Should().BeFalse();
}
[Test]
public void Culture_DoesNotChangeKeysOrValues()
{
var german = (CultureInfo)CultureInfo.InvariantCulture.Clone();
german.NumberFormat.NumberDecimalSeparator = ",";
german.NumberFormat.NumberGroupSeparator = ".";
german.NumberFormat.NegativeSign = "−";
german.NumberFormat.PositiveSign = "➕";
var swedish = (CultureInfo)CultureInfo.InvariantCulture.Clone();
swedish.NumberFormat.NegativeSign = "−";
swedish.NumberFormat.NumberDecimalSeparator = ",";
var original = CultureInfo.CurrentCulture;
try
{
foreach (var culture in new[] { german, swedish })
{
CultureInfo.CurrentCulture = culture;
var options = With(ReadStrings | WriteStrings | Named);
KeyOf(new float2(-1.5f, 2.25f)).Should().Be("-1.5,2.25");
KeyOf(new double3(-1e-7, 1234.5, -0d)).Should().Be("-1E-07,1234.5,-0");
KeyOf(new int2(-3, 4)).Should().Be("-3,4");
KeyOf(new float2(float.NaN, float.NegativeInfinity), Named).Should().Be("NaN,-Infinity");
Parse<Dictionary<float2, int>>("""{"-1.5,2.25":1}""").Keys.Single().Should().Be(new float2(-1.5f, 2.25f));
Parse<Dictionary<int2, int>>("""{"-3,4":1}""").Keys.Single().Should().Be(new int2(-3, 4));
JsonSerializer.Serialize(new float2(-1.5f, 2.25f), options).Should().Be("""{"x":"-1.5","y":"2.25"}""");
JsonSerializer.Deserialize<float2>("""{"x":"-1.5","y":"2.25"}""", options).Should().Be(new float2(-1.5f, 2.25f));
JsonSerializer.Serialize(new int2(-3, 4), options).Should().Be("""{"x":"-3","y":"4"}""");
JsonSerializer.Deserialize<int2>("""{"x":"-3","y":"4"}""", options).Should().Be(new int2(-3, 4));
Json(new float2(-1.5f, 2.25f)).Should().Be("""{"x":-1.5,"y":2.25}""");
}
}
finally
{
CultureInfo.CurrentCulture = original;
}
}
private static void RoundTrip<T>(T value) => Parse<T>(Json(value)).Should().Be(value);
}
}

222
tests/GameCult.Caching.Tests/CultMathSerializationTests.cs Normal file
View file

@ -0,0 +1,222 @@
using System;
using System.Collections.Generic;
using System.Linq;
using FluentAssertions;
using GameCult.Caching.MessagePack;
using CultMath;
using MessagePack;
using MessagePack.Formatters;
using NUnit.Framework;
[assembly: CultCacheFormatterResolver(typeof(GameCult.Caching.Tests.CultMathSerializationTests.Float2Override))]
namespace GameCult.Caching.Tests
{
public sealed class CultMathSerializationTests
{
private static readonly MessagePackSerializerOptions Options = CultDocumentMessagePackSerialization.Options;
private static string Hex<T>(T value) => Convert.ToHexString(MessagePackSerializer.Serialize(value, Options));
private static T Decode<T>(string hex) => MessagePackSerializer.Deserialize<T>(Convert.FromHexString(hex), Options);
[MessagePackObject]
public sealed class Carrier
{
[Key(0)] public float2 Position;
[Key(1)] public float3? Velocity;
[Key(2)] public float2[] Path = Array.Empty<float2>();
[Key(3)] public List<int2> Cells = new();
}
// Bytes the pre-collapse CultVec2/CultVec3/CultRect encodings and Aetheria's Float2/Int2/Bool2/Float3/Float4
// formatters produce: an array of components in declaration order, float32 as 0xCA.
[Test]
public void Float2_MatchesTheLegacyVectorBytes()
{
const string hex = "92CA3F800000CA40000000";
Hex(new float2(1f, 2f)).Should().Be(hex);
Decode<float2>(hex).Should().Be(new float2(1f, 2f));
}
[Test]
public void Float3_MatchesTheLegacyVectorBytes()
{
const string hex = "93CA3F800000CA40000000CA40400000";
Hex(new float3(1f, 2f, 3f)).Should().Be(hex);
Decode<float3>(hex).Should().Be(new float3(1f, 2f, 3f));
}
[Test]
public void Rect_MatchesTheLegacyCultRectBytes()
{
const string hex = "9292CABF800000CAC000000092CA40A00000CA40E00000";
Hex(new rect(5f, 7f, -1f, -2f)).Should().Be(hex);
Decode<rect>(hex).Should().Be(new rect(-1f, -2f, 5f, 7f));
}
[Test]
public void ConsumerFormatterShapes_MatchByteForByte()
{
Hex(new float4(1f, 2f, 3f, 4f)).Should().Be("94CA3F800000CA40000000CA40400000CA40800000");
Hex(new int2(3, -4)).Should().Be("9203FC");
Hex(new bool2(true, false)).Should().Be("92C3C2");
}
[Test]
public void Decode_SkipsExtraElements_AndLeavesMissingOnesZero()
{
Decode<float2>("93CA3F800000CA40000000CA40400000").Should().Be(new float2(1f, 2f));
Decode<float3>("92CA3F800000CA40000000").Should().Be(new float3(1f, 2f, 0f));
Decode<rect>("9392CABF800000CAC000000092CA40A00000CA40E0000001").Should().Be(new rect(-1f, -2f, 5f, 7f));
}
[Test]
public void Decode_RefusesNilAndNonArrays()
{
Action nil = () => Decode<float2>("C0");
nil.Should().Throw<MessagePackSerializationException>();
Action notArray = () => Decode<float3>("01");
notArray.Should().Throw<MessagePackSerializationException>();
}
[Test]
public void EveryFormattedType_RoundTrips()
{
RoundTrip(new float2(1.5f, -2f));
RoundTrip(new float3(1f, 2f, 3f));
RoundTrip(new float4(1f, 2f, 3f, 4f));
RoundTrip(new double2(1e300, -2.5));
RoundTrip(new double3(1, 2, 3));
RoundTrip(new int2(-7, 900000));
RoundTrip(new int3(1, -2, 3));
RoundTrip(new int4(1, 2, 3, int.MinValue));
RoundTrip(new bool2(true, false));
RoundTrip(new bool3(false, true, true));
RoundTrip(new bool4(true, false, false, true));
RoundTrip(new quaternion(0.1f, 0.2f, 0.3f, 0.9f));
RoundTrip(new Color32(255, 128, 0, 7));
RoundTrip(new CultMath.Random(12345u));
RoundTrip(new float2x2(1f, 2f, 3f, 4f));
RoundTrip(new float3x3(1f, 2f, 3f, 4f, 5f, 6f, 7f, 8f, 9f));
RoundTrip(new rect(-1f, -2f, 5f, 7f));
RoundTrip(new CultCellular(new float4(1, 2, 3, 4), new float4(5, 6, 7, 8), 0.25f));
}
[Test]
public void Matrices_AreArraysOfRows()
{
Hex(new float2x2(1f, 2f, 3f, 4f)).Should().Be("9292CA3F800000CA4000000092CA40400000CA40800000");
}
[Test]
public void EveryShape_IsPinnedByteForByte()
{
Hex(new double2(1d, 2d)).Should().Be("92CB3FF0000000000000CB4000000000000000");
Hex(new double3(1d, 2d, 3d)).Should().Be("93CB3FF0000000000000CB4000000000000000CB4008000000000000");
Hex(new int3(1, -2, 3)).Should().Be("9301FE03");
Hex(new int4(1, 2, 3, -1)).Should().Be("94010203FF");
Hex(new bool3(true, false, true)).Should().Be("93C3C2C3");
Hex(new bool4(false, true, true, false)).Should().Be("94C2C3C3C2");
Hex(new quaternion(1f, 2f, 3f, 4f)).Should().Be("94CA3F800000CA40000000CA40400000CA40800000");
Hex(new Color32(255, 128, 0, 7)).Should().Be("94CCFFCC800007");
Hex(new CultMath.Random(12345u)).Should().Be("91CD3039");
Hex(new float3x3(1f, 2f, 3f, 4f, 5f, 6f, 7f, 8f, 9f)).Should().Be(
"93" +
"93CA3F800000CA40000000CA40400000" +
"93CA40800000CA40A00000CA40C00000" +
"93CA40E00000CA41000000CA41100000");
Hex(new CultCellular(new float4(1f, 2f, 3f, 4f), new float4(5f, 6f, 7f, 8f), 0.25f)).Should().Be(
"93" +
"94CA3F800000CA40000000CA40400000CA40800000" +
"94CA40A00000CA40C00000CA40E00000CA41000000" +
"CA3E800000");
}
[Test]
public void Color32_MissingAlphaDecodesAsOpaque()
{
Decode<Color32>("93CCFFCC8000").Should().Be(new Color32(255, 128, 0, 255));
Decode<Color32>("94CCFFCC800007").Should().Be(new Color32(255, 128, 0, 7));
}
[Test]
public void Random_StateZero_IsTheGeneratorDefaultSeed()
{
// CultMath's generator maps state 0 to its default seed, so a zero or missing state is not degenerate.
var reference = new CultMath.Random(0u);
var expected = new[] { reference.NextUInt(), reference.NextUInt(), reference.NextUInt() };
foreach (var hex in new[] { "9100", "90" })
{
var decoded = Decode<CultMath.Random>(hex);
new[] { decoded.NextUInt(), decoded.NextUInt(), decoded.NextUInt() }.Should().Equal(expected);
}
}
[Test]
public void ConsumerResolvers_WinOverCultMathResolver()
{
var overridden = CultDocumentMessagePackSerialization.OptionsFor(typeof(Float2Override).Assembly);
MessagePackSerializer.Serialize(new float2(1f, 2f), overridden).Should().Equal(0xC3);
Hex(new float2(1f, 2f)).Should().Be("92CA3F800000CA40000000");
}
public sealed class Float2Override : IFormatterResolver
{
public static readonly Float2Override Instance = new();
public IMessagePackFormatter<T>? GetFormatter<T>() =>
typeof(T) == typeof(float2) ? (IMessagePackFormatter<T>)(object)Formatter.Instance : null;
private sealed class Formatter : IMessagePackFormatter<float2>
{
public static readonly Formatter Instance = new();
public void Serialize(ref MessagePackWriter writer, float2 value, MessagePackSerializerOptions options) => writer.Write(true);
public float2 Deserialize(ref MessagePackReader reader, MessagePackSerializerOptions options) => throw new NotSupportedException();
}
}
[Test]
public void EveryPublicCultMathValueType_HasAFormatter()
{
var valueTypes = typeof(float2).Assembly.GetExportedTypes()
.Where(t => t.IsValueType && !t.IsEnum)
.ToHashSet();
valueTypes.Except(CultMathResolver.FormattedTypes).Should().BeEmpty(
"every public CultMath value type serializes through CultMathResolver");
CultMathResolver.FormattedTypes.Except(valueTypes).Should().BeEmpty();
}
[Test]
public void TheDefaultChain_ComposesMathIntoNullablesCollectionsAndDocuments()
{
var value = new Carrier
{
Position = new float2(1f, 2f),
Velocity = new float3(3f, 4f, 5f),
Path = new[] { new float2(0f, 1f), new float2(2f, 3f) },
Cells = new List<int2> { new int2(1, 2) },
};
var decoded = MessagePackSerializer.Deserialize<Carrier>(MessagePackSerializer.Serialize(value, Options), Options);
decoded.Position.Should().Be(value.Position);
decoded.Velocity.Should().Be(value.Velocity);
decoded.Path.Should().Equal(value.Path);
decoded.Cells.Should().Equal(value.Cells);
MessagePackSerializer.Serialize(new Carrier(), Options).Should().NotBeEmpty();
var absent = MessagePackSerializer.Deserialize<Carrier>(MessagePackSerializer.Serialize(new Carrier(), Options), Options);
absent.Velocity.Should().BeNull();
}
private static void RoundTrip<T>(T value)
{
var bytes = MessagePackSerializer.Serialize(value, Options);
MessagePackSerializer.Deserialize<T>(bytes, Options).Should().Be(value);
}
}
}

79
tests/GameCult.Geometry.Tests/GeometryDocumentTests.cs
View file

@ -1,4 +1,5 @@
using FluentAssertions;
using CultMath;
using GameCult.Caching;
using GameCult.Caching.MessagePack;
using GameCult.Mesh;
@ -19,54 +20,66 @@ namespace GameCult.Geometry.Tests
[Test]
public void GeometryPrimitives_CanonicalizeAndIntersect()
{
var viewport = new CultRect(8f, 4f, -8f, -4f);
var viewport = new rect(8f, 4f, -8f, -4f);
viewport.Min.Should().Be(new CultVec2(-8f, -4f));
viewport.Max.Should().Be(new CultVec2(8f, 4f));
viewport.Center.Should().Be(CultVec2.Zero);
viewport.Contains(new CultVec2(2f, 3f)).Should().BeTrue();
viewport.Intersects(new CultRect(7f, 3f, 10f, 5f)).Should().BeTrue();
viewport.Intersects(new CultRect(9f, 5f, 10f, 6f)).Should().BeFalse();
viewport.min.Should().Be(new float2(-8f, -4f));
viewport.max.Should().Be(new float2(8f, 4f));
viewport.center.Should().Be(float2.zero);
viewport.Contains(new float2(2f, 3f)).Should().BeTrue();
viewport.Intersects(new rect(7f, 3f, 10f, 5f)).Should().BeTrue();
viewport.Intersects(new rect(9f, 5f, 10f, 6f)).Should().BeFalse();
var gravityBrush = new CultCircle(new CultVec2(10f, 0f), 3f);
var gravityBrush = new CultCircle(new float2(10f, 0f), 3f);
gravityBrush.Intersects(viewport).Should().BeTrue();
gravityBrush.Contains(new CultVec2(12f, 0f)).Should().BeTrue();
gravityBrush.Contains(new CultVec2(14f, 0f)).Should().BeFalse();
gravityBrush.Contains(new float2(12f, 0f)).Should().BeTrue();
gravityBrush.Contains(new float2(14f, 0f)).Should().BeFalse();
}
[Test]
public void GeometryPrimitives_PreserveWholeVectorsForSoaAndPhysicsQueries()
{
var sphere = new CultSphere(new CultVec3(1f, 2f, 3f), 5f);
var sphere = new CultSphere(new float3(1f, 2f, 3f), 5f);
sphere.Contains(new CultVec3(1f, 6f, 3f)).Should().BeTrue();
sphere.Contains(new CultVec3(1f, 8f, 3f)).Should().BeFalse();
sphere.Center.Xy.Should().Be(new CultVec2(1f, 2f));
sphere.Center.Xz.Should().Be(new CultVec2(1f, 3f));
sphere.XyCircle.Bounds.Should().Be(new CultRect(-4f, -3f, 6f, 7f));
}
[Test]
public void GeometryPrimitives_RoundTripThroughMessagePack()
{
var rect = new CultRect(new CultVec2(5f, -2f), new CultVec2(-1f, 7f));
var payload = MessagePackSerializer.Serialize(rect);
var decoded = MessagePackSerializer.Deserialize<CultRect>(payload);
decoded.Should().Be(rect);
decoded.Min.Should().Be(new CultVec2(-1f, -2f));
decoded.Max.Should().Be(new CultVec2(5f, 7f));
sphere.Contains(new float3(1f, 6f, 3f)).Should().BeTrue();
sphere.Contains(new float3(1f, 8f, 3f)).Should().BeFalse();
sphere.Center.xy.Should().Be(new float2(1f, 2f));
sphere.Center.xz.Should().Be(new float2(1f, 3f));
sphere.XyCircle.Bounds.Should().Be(new rect(-4f, -3f, 6f, 7f));
}
[Test]
public void GeometryPrimitives_JsonOmitsComputedConvenienceProperties()
{
var json = JsonSerializer.Serialize(new CultVec3(1f, 2f, 3f));
var options = new JsonSerializerOptions().AddCultMathConverters();
var sphere = new CultSphere(new float3(1f, 2f, 3f), 5f);
var circle = new CultCircle(new float2(1f, 2f), 3f);
json.Should().Be("""{"X":1,"Y":2,"Z":3}""");
json.Should().NotContain("LengthSquared");
json.Should().NotContain("Xy");
json.Should().NotContain("Xz");
var sphereJson = JsonSerializer.Serialize(sphere, options);
var circleJson = JsonSerializer.Serialize(circle, options);
sphereJson.Should().Be("""{"Center":{"x":1,"y":2,"z":3},"Radius":5}""");
circleJson.Should().Be("""{"Center":{"x":1,"y":2},"Radius":3}""");
JsonSerializer.Deserialize<CultSphere>(sphereJson, options).Should().Be(sphere);
JsonSerializer.Deserialize<CultCircle>(circleJson, options).Should().Be(circle);
}
// The bytes CultCircle and CultSphere produced when they held CultVec2/CultVec3 (probed at 65311c9).
[Test]
public void GeometryPrimitives_KeepTheirPreCollapseBytes()
{
var options = CultDocumentMessagePackSerialization.Options;
var circle = new CultCircle(new float2(1f, 2f), 3f);
var sphere = new CultSphere(new float3(1f, 2f, 3f), 5f);
Convert.ToHexString(MessagePackSerializer.Serialize(circle, options))
.Should().Be("9292CA3F800000CA40000000CA40400000");
Convert.ToHexString(MessagePackSerializer.Serialize(sphere, options))
.Should().Be("9293CA3F800000CA40000000CA40400000CA40A00000");
MessagePackSerializer.Deserialize<CultCircle>(MessagePackSerializer.Serialize(circle, options), options)
.Should().Be(circle);
MessagePackSerializer.Deserialize<CultSphere>(MessagePackSerializer.Serialize(sphere, options), options)
.Should().Be(sphere);
}
[Test]

5
tests/GameCult.Geometry.Tests/IsoSurfaceTests.cs
View file

@ -1,4 +1,5 @@
using System;
using CultMath;
using FluentAssertions;
using NUnit.Framework;
@ -77,8 +78,8 @@ namespace GameCult.Geometry.Tests
samples[0, 1, 1] = 1f;
samples[1, 1, 1] = 1f;
var first = CultGeometryIsoSurface.Extract(samples, origin: new CultVec3(10f, 20f, 30f), cellSize: 2f);
var second = CultGeometryIsoSurface.Extract(samples, origin: new CultVec3(10f, 20f, 30f), cellSize: 2f);
var first = CultGeometryIsoSurface.Extract(samples, origin: new float3(10f, 20f, 30f), cellSize: 2f);
var second = CultGeometryIsoSurface.Extract(samples, origin: new float3(10f, 20f, 30f), cellSize: 2f);
second.Positions.Should().Equal(first.Positions);
second.Normals.Should().Equal(first.Normals);

68
tests/GameCult.Geometry.Tests/SurfaceNetsTests.cs
View file

@ -1,5 +1,6 @@
using System;
using System.Collections.Generic;
using CultMath;
using FluentAssertions;
using NUnit.Framework;
@ -73,28 +74,28 @@ namespace GameCult.Geometry.Tests
for (var quad = 0; quad < mesh.QuadCount; quad++)
{
var edge = mesh.QuadEdges[quad];
var insideCoord = new CultVec3(1f, 1f, 1f);
var edgeStart = new CultVec3(edge.X, edge.Y, edge.Z);
var insideCoord = new float3(1f, 1f, 1f);
var edgeStart = new float3(edge.X, edge.Y, edge.Z);
var edgeEnd = OtherEndpoint(edge);
// The single inside sample is (1,1,1); the edge's other endpoint is outside.
var outsideCoord = edgeStart.Equals(insideCoord) ? edgeEnd : edgeStart;
var outward = Subtract(outsideCoord, insideCoord);
var outward = outsideCoord - insideCoord;
var baseIndex = quad * 4;
var v0 = Position(mesh.Positions, mesh.Quads[baseIndex]);
var v1 = Position(mesh.Positions, mesh.Quads[baseIndex + 1]);
var v2 = Position(mesh.Positions, mesh.Quads[baseIndex + 2]);
var v3 = Position(mesh.Positions, mesh.Quads[baseIndex + 3]);
var cross = Cross(Subtract(v2, v0), Subtract(v3, v1));
var cross = math.cross(v2 - v0, v3 - v1);
// Restricted to non-degenerate (non-zero-area) quads: the spec's geometric
// winding rule is only meaningful there. Degenerate/collapsed quads are covered
// separately by AssertEdgeDirectionsBalance, since a zero cross product carries
// no orientation information to check here.
if (cross.X == 0f && cross.Y == 0f && cross.Z == 0f) continue;
if (cross.Equals(float3.zero)) continue;
Dot(cross, outward).Should().BePositive($"quad for edge {edge} should wind outward");
math.dot(cross, outward).Should().BePositive($"quad for edge {edge} should wind outward");
}
}
@ -209,7 +210,7 @@ namespace GameCult.Geometry.Tests
samples[1, 1, 1] = -1f;
var mesh = CultGeometrySurfaceNets.Extract(samples, origin: new CultVec3(10f, 20f, 30f), cellSize: 2f);
var mesh = CultGeometrySurfaceNets.Extract(samples, origin: new float3(10f, 20f, 30f), cellSize: 2f);
mesh.VertexCount.Should().Be(1);
mesh.Positions[0].Should().BeApproximately(10f + (2.9f / 3f * 2f), 1e-5f);
@ -387,10 +388,10 @@ namespace GameCult.Geometry.Tests
// documented c0..c3 order is not swapped.
var expected = new[]
{
new CultVec3(1.5f, 0.5f, 0.5f), // c0 = (1, 0, 0)
new CultVec3(1.5f, 1.5f, 0.5f), // c1 = (1, 1, 0)
new CultVec3(1.5f, 1.5f, 1.5f), // c2 = (1, 1, 1)
new CultVec3(1.5f, 0.5f, 1.5f), // c3 = (1, 0, 1)
new float3(1.5f, 0.5f, 0.5f), // c0 = (1, 0, 0)
new float3(1.5f, 1.5f, 0.5f), // c1 = (1, 1, 0)
new float3(1.5f, 1.5f, 1.5f), // c2 = (1, 1, 1)
new float3(1.5f, 0.5f, 1.5f), // c3 = (1, 0, 1)
};
AssertQuadCorners(mesh, edge, expected);
@ -413,10 +414,10 @@ namespace GameCult.Geometry.Tests
// v3 trade places relative to the raw c0..c3 listing.
var expected = new[]
{
new CultVec3(0.5f, 1.5f, 0.5f), // c0 = (0, 1, 0)
new CultVec3(0.5f, 1.5f, 1.5f), // c3 = (0, 1, 1), swapped into v1
new CultVec3(1.5f, 1.5f, 1.5f), // c2 = (1, 1, 1)
new CultVec3(1.5f, 1.5f, 0.5f), // c1 = (1, 1, 0), swapped into v3
new float3(0.5f, 1.5f, 0.5f), // c0 = (0, 1, 0)
new float3(0.5f, 1.5f, 1.5f), // c3 = (0, 1, 1), swapped into v1
new float3(1.5f, 1.5f, 1.5f), // c2 = (1, 1, 1)
new float3(1.5f, 1.5f, 0.5f), // c1 = (1, 1, 0), swapped into v3
};
AssertQuadCorners(mesh, edge, expected);
@ -438,10 +439,10 @@ namespace GameCult.Geometry.Tests
// mirrored, so the documented c0..c3 order is not swapped.
var expected = new[]
{
new CultVec3(0.5f, 0.5f, 1.5f), // c0 = (0, 0, 1)
new CultVec3(1.5f, 0.5f, 1.5f), // c1 = (1, 0, 1)
new CultVec3(1.5f, 1.5f, 1.5f), // c2 = (1, 1, 1)
new CultVec3(0.5f, 1.5f, 1.5f), // c3 = (0, 1, 1)
new float3(0.5f, 0.5f, 1.5f), // c0 = (0, 0, 1)
new float3(1.5f, 0.5f, 1.5f), // c1 = (1, 0, 1)
new float3(1.5f, 1.5f, 1.5f), // c2 = (1, 1, 1)
new float3(0.5f, 1.5f, 1.5f), // c3 = (0, 1, 1)
};
AssertQuadCorners(mesh, edge, expected);
@ -476,7 +477,7 @@ namespace GameCult.Geometry.Tests
// balance regardless.
var samples = Sphere(64);
var mesh = CultGeometrySurfaceNets.Extract(samples, origin: new CultVec3(1e4f, 1e4f, 1e4f), cellSize: 0.001f);
var mesh = CultGeometrySurfaceNets.Extract(samples, origin: new float3(1e4f, 1e4f, 1e4f), cellSize: 0.001f);
mesh.QuadCount.Should().BeGreaterThan(0);
AssertEdgeDirectionsBalance(mesh);
@ -651,7 +652,7 @@ namespace GameCult.Geometry.Tests
{
var samples = new float[2, 2, 2];
Action act = () => CultGeometrySurfaceNets.Extract(samples, origin: new CultVec3(x, y, z));
Action act = () => CultGeometrySurfaceNets.Extract(samples, origin: new float3(x, y, z));
act.Should().Throw<ArgumentException>();
}
@ -725,28 +726,17 @@ namespace GameCult.Geometry.Tests
return usage;
}
private static CultVec3 OtherEndpoint(CultGeometryGridEdge edge) => edge.Axis switch
private static float3 OtherEndpoint(CultGeometryGridEdge edge) => edge.Axis switch
{
0 => new CultVec3(edge.X + 1, edge.Y, edge.Z),
1 => new CultVec3(edge.X, edge.Y + 1, edge.Z),
_ => new CultVec3(edge.X, edge.Y, edge.Z + 1),
0 => new float3(edge.X + 1, edge.Y, edge.Z),
1 => new float3(edge.X, edge.Y + 1, edge.Z),
_ => new float3(edge.X, edge.Y, edge.Z + 1),
};
private static CultVec3 Subtract(CultVec3 left, CultVec3 right) =>
new(left.X - right.X, left.Y - right.Y, left.Z - right.Z);
private static CultVec3 Cross(CultVec3 left, CultVec3 right) => new(
(left.Y * right.Z) - (left.Z * right.Y),
(left.Z * right.X) - (left.X * right.Z),
(left.X * right.Y) - (left.Y * right.X));
private static float Dot(CultVec3 left, CultVec3 right) =>
(left.X * right.X) + (left.Y * right.Y) + (left.Z * right.Z);
private static CultVec3 Position(float[] positions, uint index)
private static float3 Position(float[] positions, uint index)
{
var offset = (int)index * 3;
return new CultVec3(positions[offset], positions[offset + 1], positions[offset + 2]);
return new float3(positions[offset], positions[offset + 1], positions[offset + 2]);
}
// Hand-derived (double precision, independent of Interpolate) expected position for a
@ -760,7 +750,7 @@ namespace GameCult.Geometry.Tests
return (float)(amount / 3d);
}
private static void AssertQuadCorners(CultGeometryQuadMesh mesh, CultGeometryGridEdge edge, CultVec3[] expected)
private static void AssertQuadCorners(CultGeometryQuadMesh mesh, CultGeometryGridEdge edge, float3[] expected)
{
var quadIndex = Array.IndexOf(mesh.QuadEdges, edge);
quadIndex.Should().BeGreaterThanOrEqualTo(0, $"a quad for edge {edge} should exist");

100
tests/GameCult.Networking.Tests/CultCachePublicationIsolationTests.cs Normal file
View file

@ -0,0 +1,100 @@
#nullable enable
using System;
using System.Collections.Concurrent;
using System.Linq;
using System.Threading.Tasks;
using GameCult.Caching;
using NUnit.Framework;
using R3;
using static GameCult.Networking.Tests.NetworkingTests;
namespace GameCult.Networking.Tests
{
[CultDocument("tests.publication_throwing_name", "tests.publication_throwing_name.v1")]
[MessagePack.MessagePackObject]
public sealed class PublicationThrowingName
{
[MessagePack.Key(0)]
public string Text { get; set; } = string.Empty;
[MessagePack.Key(1)]
[CultName]
public string Name
{
get => Text == "boom" ? throw new InvalidOperationException("name getter boom") : Text;
set { }
}
}
// Cache publication cuts 1 and 2 seen through CultNetDatabase: a refused or failing change never stalls the
// publication and log of the changes after it. (Soul pass 3 probes P4 and P3.)
[NonParallelizable]
public sealed class CultCachePublicationIsolationTests
{
private static NetworkSchemaNote Note(string text) => new() { Schema = "tests.networking_note.v1", Text = text };
private static string SchemaId => CultDocumentRegistry.Shared.GetRequired<NetworkSchemaNote>().SchemaId;
// An admission that a [CultName] getter refuses publishes nothing, leaves nothing in memory, and stalls nothing after it.
[Test]
public async Task ARefusedAdmissionLeavesNothingAndLaterChangesStillPublishAndLog()
{
var cache = new CultCache();
var database = new CultNetDatabase(cache);
var seen = new ConcurrentQueue<string?>();
using var watch = database.Watch<NetworkSchemaNote>().Subscribe(change => seen.Enqueue(change.Document?.Text));
var okKey = new CultRecordKey("pub:ok");
var refusal = Assert.Throws<InvalidOperationException>(() => cache.Commit(batch =>
{
batch.Upsert(new PublicationThrowingName { Text = "ok" }, new CultRecordHandle<PublicationThrowingName>(okKey));
batch.Upsert(new PublicationThrowingName { Text = "boom" }, new CultRecordHandle<PublicationThrowingName>(new CultRecordKey("pub:boomname")));
}))!;
var after = new CultRecordKey("pub:after");
await database.PutAsync(after, Note("after"));
Assert.Multiple(() =>
{
Assert.That(refusal.Message, Does.Contain("pub:boomname").And.Contain("name getter boom"));
Assert.That(cache.Get<PublicationThrowingName>(okKey), Is.Null, "the good record of a refused batch did not land");
Assert.That(seen, Is.EqualTo(new[] { "after" }));
Assert.That(database.LastWriteSequence(SchemaId, after), Is.Not.Null, "a later commit was never logged");
});
}
// One observer's escaped exception (a fail-fast R3 handler) reaches the writer and does not stall later changes.
// A database write is published by the database after its cache write returns, so the failed write itself is not
// published here; that ordering belongs to the CultNetDatabase cut.
[Test]
public async Task AnEscapedObserverExceptionReachesTheWriterAndDoesNotStallLaterChanges()
{
var previous = ObservableSystem.GetUnhandledExceptionHandler();
ObservableSystem.RegisterUnhandledExceptionHandler(exception => throw exception);
try
{
var cache = new CultCache();
using var thrower = cache.Watch<NetworkSchemaNote>().Subscribe(change =>
{
if (change.Document?.Text == "boom") throw new InvalidOperationException("observer boom");
});
var database = new CultNetDatabase(cache);
var seen = new ConcurrentQueue<string?>();
using var watch = database.Watch<NetworkSchemaNote>().Subscribe(change => seen.Enqueue(change.Document?.Text));
var boom = new CultRecordKey("pub:boom");
var escaped = Assert.ThrowsAsync<InvalidOperationException>(() => database.PutAsync(boom, Note("boom")));
ObservableSystem.RegisterUnhandledExceptionHandler(previous);
var after = new CultRecordKey("pub:after");
await database.PutAsync(after, Note("after"));
Assert.Multiple(() =>
{
Assert.That(escaped!.Message, Is.EqualTo("observer boom"));
Assert.That(seen, Does.Contain("after"), "a later commit was never published");
Assert.That(database.LastWriteSequence(SchemaId, after), Is.Not.Null, "a later commit was never logged");
});
}
finally
{
ObservableSystem.RegisterUnhandledExceptionHandler(previous);
}
}
}
}

373
tests/GameCult.Networking.Tests/NetworkingTests.cs
View file

@ -900,6 +900,20 @@ namespace GameCult.Networking.Tests
Assert.That(client.Connected, Is.False);
}
[Test]
public void RudpSession_RepeatedConnectAcknowledgesWhatItCarries()
{
var client = new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = 103, InitialSequence = 1 });
var server = new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = 103, InitialSequence = 100 });
var connect = client.CreateConnect(0, Encoding.UTF8.GetBytes("join"));
var accept = server.AcceptConnect(connect, 0);
var repeat = CultNetRudpPacketCodec.Decode(CultNetRudpPacketCodec.Encode(connect));
repeat.Ack = accept.Sequence;
var reply = server.AnswerRepeatedConnect(repeat, 1);
Assert.That(reply.PacketType, Is.EqualTo(CultNetRudpPacketType.Ack));
Assert.That(server.OutstandingReliablePacketCount, Is.EqualTo(0));
}
[Test]
public void RudpSession_LossyPacketsCannotCreateReliableOrderedGaps()
{
@ -1193,6 +1207,365 @@ namespace GameCult.Networking.Tests
}
[Test]
public void RudpSocketTransport_DropsStraysAndKeepsServingItsPeer()
{
using var serverSocket = BindUdpSocket();
using var clientSocket = BindUdpSocket();
using var strayHost = BindUdpSocket();
var serverEndPoint = serverSocket.LocalEndPoint!;
var clientEndPoint = clientSocket.LocalEndPoint!;
const uint connectionId = 0x10203050;
using var server = new CultNetRudpSocketTransportConnection(new CultNetRudpSocketTransportOptions
{
RuntimeId = "csharp-rudp-server",
Socket = serverSocket,
Mode = CultNetRudpSocketMode.Server,
ConnectionId = connectionId
});
using var client = new CultNetRudpSocketTransportConnection(new CultNetRudpSocketTransportOptions
{
RuntimeId = "csharp-rudp-client",
Socket = clientSocket,
Mode = CultNetRudpSocketMode.Client,
RemoteEndPoint = serverEndPoint,
ConnectionId = connectionId
});
var foreign = new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = 0x0BADF10E });
var strays = new[]
{
Encoding.UTF8.GetBytes("not a rudp packet"),
new byte[] { 1 },
CultNetRudpPacketCodec.Encode(foreign.CreateConnect(0, Encoding.UTF8.GetBytes("foreign")))
};
// A stray that arrives before the peer must not become the peer.
foreach (var stray in strays)
strayHost.SendTo(stray, serverEndPoint);
for (var index = 0; index < strays.Length; index++)
Assert.That(server.ReceiveOnce(), Is.Null);
Assert.That(server.Stats.PacketsDropped, Is.EqualTo(strays.Length));
client.Connect(Encoding.UTF8.GetBytes("join"));
PumpRudpHandshake(client, server);
// A moved flow answers from the peer's own address with someone else's session.
foreach (var stray in strays)
{
serverSocket.SendTo(stray, clientEndPoint);
clientSocket.SendTo(stray, serverEndPoint);
}
for (var index = 0; index < strays.Length; index++)
{
Assert.That(client.ReceiveOnce(), Is.Null);
Assert.That(server.ReceiveOnce(), Is.Null);
}
Assert.That(client.Stats.PacketsDropped, Is.EqualTo(strays.Length));
Assert.That(server.Stats.PacketsDropped, Is.EqualTo(2 * strays.Length));
client.Send("schema", Encoding.UTF8.GetBytes("still here"));
Assert.That(Encoding.UTF8.GetString(ReceiveRudpFrame(server).Payload), Is.EqualTo("still here"));
server.Send("schema", Encoding.UTF8.GetBytes("server still here"));
Assert.That(Encoding.UTF8.GetString(ReceiveRudpFrame(client).Payload), Is.EqualTo("server still here"));
}
private static List<CultNetRudpPacket> DrainPackets(Socket socket)
{
var packets = new List<CultNetRudpPacket>();
var buffer = new byte[65535];
EndPoint from = new IPEndPoint(IPAddress.Any, 0);
while (socket.Poll(20_000, SelectMode.SelectRead))
{
var received = socket.ReceiveFrom(buffer, ref from);
packets.Add(CultNetRudpPacketCodec.Decode(buffer.AsSpan(0, received).ToArray()));
}
return packets;
}
/// <summary>A reliable frame no session accepts: a fragment set with no fragment id.</summary>
private static CultNetRudpPacket PoisonedFrame(CultNetRudpSession session)
{
var packet = session.Send("schema", Encoding.UTF8.GetBytes("poison"), new CultNetRudpSendOptions { Reliable = true, Ordered = true });
packet.FragmentCount = 2;
packet.FragmentId = 0;
return packet;
}
[Test]
public void RudpPacketCodec_TreatsADeclaredLengthPastInt32AsMalformed()
{
var wire = CultNetRudpPacketCodec.Encode(new CultNetRudpPacket
{
PacketType = CultNetRudpPacketType.Ack,
ConnectionId = 0x10203054,
ChannelId = string.Empty
});
Assert.That(wire, Has.Length.EqualTo(36));
wire[30] = 0x80;
wire[31] = 0;
wire[32] = 0;
wire[33] = 0;
Assert.Throws<InvalidOperationException>(() => CultNetRudpPacketCodec.Decode(wire));
using var connectionSocket = BindUdpSocket();
using var listenerSocket = BindUdpSocket();
using var senderSocket = BindUdpSocket();
using var connection = new CultNetRudpSocketTransportConnection(new CultNetRudpSocketTransportOptions
{
RuntimeId = "csharp-rudp-connection",
Socket = connectionSocket,
Mode = CultNetRudpSocketMode.Server,
ConnectionId = 0x10203054
});
using var listener = new CultNetRudpSocketTransportServer(new CultNetRudpSocketTransportServerOptions
{
RuntimeId = "csharp-rudp-listener",
Socket = listenerSocket,
ConnectionId = 0x10203054
});
senderSocket.SendTo(wire, connectionSocket.LocalEndPoint!);
senderSocket.SendTo(wire, listenerSocket.LocalEndPoint!);
Assert.That(connection.ReceiveOnce(), Is.Null);
Assert.That(listener.ReceiveOnce(), Is.Null);
Assert.That(connection.Stats.PacketsDropped, Is.EqualTo(1));
Assert.That(listener.Stats.PacketsDropped, Is.EqualTo(1));
}
[Test]
public void RudpServerMode_IsClaimedOnlyByAConnect()
{
using var serverSocket = BindUdpSocket();
using var orphanSocket = BindUdpSocket();
using var clientSocket = BindUdpSocket();
const uint connectionId = 0x10203055;
using var server = new CultNetRudpSocketTransportConnection(new CultNetRudpSocketTransportOptions
{
RuntimeId = "csharp-rudp-server",
Socket = serverSocket,
Mode = CultNetRudpSocketMode.Server,
ConnectionId = connectionId
});
using var client = new CultNetRudpSocketTransportConnection(new CultNetRudpSocketTransportOptions
{
RuntimeId = "csharp-rudp-client",
Socket = clientSocket,
Mode = CultNetRudpSocketMode.Client,
RemoteEndPoint = serverSocket.LocalEndPoint!,
ConnectionId = connectionId
});
var orphan = new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = connectionId });
// A right-id packet that is not a Connect must claim nothing.
orphanSocket.SendTo(CultNetRudpPacketCodec.Encode(orphan.CreateAck()), serverSocket.LocalEndPoint!);
Assert.That(server.ReceiveOnce(), Is.Null);
Assert.That(server.Stats.PacketsDropped, Is.EqualTo(1));
client.Connect(Encoding.UTF8.GetBytes("join"));
PumpRudpHandshake(client, server);
Assert.That(client.Connected, Is.True);
Assert.That(server.Connected, Is.True);
}
[Test]
public void RudpSocketTransport_EndsTheSessionThatRefusesAReliableFrame()
{
using var serverSocket = BindUdpSocket();
using var peerSocket = BindUdpSocket();
var serverEndPoint = serverSocket.LocalEndPoint!;
const uint connectionId = 0x10203056;
using var server = new CultNetRudpSocketTransportConnection(new CultNetRudpSocketTransportOptions
{
RuntimeId = "csharp-rudp-server",
Socket = serverSocket,
Mode = CultNetRudpSocketMode.Server,
ConnectionId = connectionId
});
var peer = new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = connectionId });
peerSocket.SendTo(CultNetRudpPacketCodec.Encode(peer.CreateConnect(0)), serverEndPoint);
server.ReceiveOnce();
peer.Receive(DrainPackets(peerSocket).Single(p => p.PacketType == CultNetRudpPacketType.Accept), 0);
var poison = PoisonedFrame(peer);
peerSocket.SendTo(CultNetRudpPacketCodec.Encode(poison), serverEndPoint);
Assert.That(server.ReceiveOnce(), Is.Null);
Assert.That(server.Stats.PacketsDropped, Is.EqualTo(1));
Assert.That(server.Connected, Is.False);
Assert.That(server.DisconnectReason, Is.Not.Null);
// The peer is told, and the goodbye does not acknowledge the refused frame.
var goodbye = DrainPackets(peerSocket).Single();
Assert.That(goodbye.PacketType, Is.EqualTo(CultNetRudpPacketType.Disconnect));
Assert.That(peer.Receive(goodbye, 1).Disconnected, Is.True);
Assert.That(peer.PendingReliableSequences, Does.Contain(poison.Sequence));
// A retransmit is dropped, not acknowledged.
peerSocket.SendTo(CultNetRudpPacketCodec.Encode(poison), serverEndPoint);
Assert.That(server.ReceiveOnce(), Is.Null);
Assert.That(server.Stats.PacketsDropped, Is.EqualTo(2));
Assert.That(DrainPackets(peerSocket), Is.Empty);
// A Connect claims the endpoint again.
var again = new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = connectionId });
peerSocket.SendTo(CultNetRudpPacketCodec.Encode(again.CreateConnect(0)), serverEndPoint);
server.ReceiveOnce();
Assert.That(DrainPackets(peerSocket).Select(p => p.PacketType), Is.EqualTo(new[] { CultNetRudpPacketType.Accept }));
}
[Test]
public void RudpSocketTransportServer_EndsThePeerWhoseReliableFrameItRefuses()
{
using var serverSocket = BindUdpSocket();
using var peerSocket = BindUdpSocket();
var serverEndPoint = serverSocket.LocalEndPoint!;
const uint connectionId = 0x10203057;
using var server = new CultNetRudpSocketTransportServer(new CultNetRudpSocketTransportServerOptions
{
RuntimeId = "csharp-rudp-listener",
Socket = serverSocket,
ConnectionId = connectionId
});
var ended = new List<CultNetRudpSocketServerPeer>();
server.PeerDisconnected += ended.Add;
var peer = new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = connectionId });
peerSocket.SendTo(CultNetRudpPacketCodec.Encode(peer.CreateConnect(0)), serverEndPoint);
server.ReceiveOnce();
peer.Receive(DrainPackets(peerSocket).Single(p => p.PacketType == CultNetRudpPacketType.Accept), 0);
Assert.That(server.Peers, Has.Count.EqualTo(1));
var poison = PoisonedFrame(peer);
peerSocket.SendTo(CultNetRudpPacketCodec.Encode(poison), serverEndPoint);
Assert.That(server.ReceiveOnce(), Is.Null);
Assert.That(server.Stats.PacketsDropped, Is.EqualTo(1));
Assert.That(server.Peers, Is.Empty);
Assert.That(ended, Has.Count.EqualTo(1));
var goodbye = DrainPackets(peerSocket).Single();
Assert.That(goodbye.PacketType, Is.EqualTo(CultNetRudpPacketType.Disconnect));
Assert.That(peer.Receive(goodbye, 1).Disconnected, Is.True);
Assert.That(peer.PendingReliableSequences, Does.Contain(poison.Sequence));
peerSocket.SendTo(CultNetRudpPacketCodec.Encode(poison), serverEndPoint);
Assert.That(server.ReceiveOnce(), Is.Null);
Assert.That(server.Stats.PacketsDropped, Is.EqualTo(2));
Assert.That(DrainPackets(peerSocket), Is.Empty);
}
[Test]
public void RudpFlushReliable_FailsForAWriteTheEndedSessionForgot()
{
using var serverSocket = BindUdpSocket();
using var peerSocket = BindUdpSocket();
var serverEndPoint = serverSocket.LocalEndPoint!;
const uint connectionId = 0x10203058;
using var server = new CultNetRudpSocketTransportConnection(new CultNetRudpSocketTransportOptions
{
RuntimeId = "csharp-rudp-server",
Socket = serverSocket,
Mode = CultNetRudpSocketMode.Server,
ConnectionId = connectionId
});
var peer = new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = connectionId });
peerSocket.SendTo(CultNetRudpPacketCodec.Encode(peer.CreateConnect(0)), serverEndPoint);
server.ReceiveOnce();
peer.Receive(DrainPackets(peerSocket).Single(p => p.PacketType == CultNetRudpPacketType.Accept), 0);
server.Send("schema", Encoding.UTF8.GetBytes("never acknowledged"));
peerSocket.SendTo(CultNetRudpPacketCodec.Encode(PoisonedFrame(peer)), serverEndPoint);
Assert.That(server.ReceiveOnce(), Is.Null);
Assert.That(server.DisconnectReason, Is.Not.Null);
Assert.Throws<InvalidOperationException>(() => server.FlushReliable(TimeSpan.FromMilliseconds(200)));
}
[Test]
public void RudpServerMode_OwesOneReliableAcceptHoweverManyConnectsRepeat()
{
using var serverSocket = BindUdpSocket();
using var peerSocket = BindUdpSocket();
const uint connectionId = 0x10203059;
using var server = new CultNetRudpSocketTransportConnection(new CultNetRudpSocketTransportOptions
{
RuntimeId = "csharp-rudp-server",
Socket = serverSocket,
Mode = CultNetRudpSocketMode.Server,
ConnectionId = connectionId
});
var connect = new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = connectionId }).CreateConnect(0);
for (var attempt = 0; attempt < 20; attempt++)
{
peerSocket.SendTo(CultNetRudpPacketCodec.Encode(connect), serverSocket.LocalEndPoint!);
server.ReceiveOnce();
}
var accepts = DrainPackets(peerSocket).Where(p => p.PacketType == CultNetRudpPacketType.Accept && p.Reliable);
Assert.That(accepts.Select(p => p.Sequence).Distinct().Count(), Is.EqualTo(1));
}
[Test]
public void RudpListener_OwesOneReliableAcceptPerGenerationAndReportsAReplacedPeer()
{
using var serverSocket = BindUdpSocket();
using var peerSocket = BindUdpSocket();
const uint connectionId = 0x1020305a;
using var server = new CultNetRudpSocketTransportServer(new CultNetRudpSocketTransportServerOptions
{
RuntimeId = "csharp-rudp-listener",
Socket = serverSocket,
ConnectionId = connectionId
});
var ended = new List<CultNetRudpSocketServerPeer>();
server.PeerDisconnected += ended.Add;
var connect = new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = connectionId }).CreateConnect(0, Encoding.UTF8.GetBytes("join"));
for (var attempt = 0; attempt < 20; attempt++)
{
peerSocket.SendTo(CultNetRudpPacketCodec.Encode(connect), serverSocket.LocalEndPoint!);
server.ReceiveOnce();
}
var accepts = DrainPackets(peerSocket).Where(p => p.PacketType == CultNetRudpPacketType.Accept && p.Reliable);
Assert.That(accepts.Select(p => p.Sequence).Distinct().Count(), Is.EqualTo(1));
Assert.That(server.Peers, Has.Count.EqualTo(1));
Assert.That(ended, Is.Empty);
// A different payload from the same endpoint is a new generation, and the old one is reported.
var other = new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = connectionId }).CreateConnect(0, Encoding.UTF8.GetBytes("rejoin"));
peerSocket.SendTo(CultNetRudpPacketCodec.Encode(other), serverSocket.LocalEndPoint!);
server.ReceiveOnce();
Assert.That(server.Peers, Has.Count.EqualTo(1));
Assert.That(ended, Has.Count.EqualTo(1));
}
[Test]
public void RudpConfiguration_RejectsLimitsThatCannotAdmitAConnect()
{
Assert.Throws<ArgumentOutOfRangeException>(() =>
new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = 1, InitialSequence = uint.MaxValue }));
Assert.Throws<ArgumentOutOfRangeException>(() =>
new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = 1, MaxPendingReliablePackets = 0 }));
Assert.DoesNotThrow(() =>
new CultNetRudpSession(new CultNetRudpSessionOptions { ConnectionId = 1, InitialSequence = uint.MaxValue - 1, MaxPendingReliablePackets = 1 }));
using var socket = BindUdpSocket();
Assert.Throws<ArgumentOutOfRangeException>(() =>
new CultNetRudpSocketTransportServer(new CultNetRudpSocketTransportServerOptions
{
RuntimeId = "csharp-rudp-listener",
Socket = socket,
ConnectionId = 1,
InitialSequence = uint.MaxValue
}));
Assert.Throws<ArgumentOutOfRangeException>(() =>
new CultNetRudpSocketTransportServer(new CultNetRudpSocketTransportServerOptions
{
RuntimeId = "csharp-rudp-listener",
Socket = socket,
ConnectionId = 1,
MaxPendingReliablePackets = 0
}));
}
[Test]
public void RudpSocketTransport_CarriesCultNetSchemaMessages()
{

9
unity/org.gamecult.cultlib/README.md
View file

@ -17,6 +17,15 @@ Install version `1.0.57` through Unity Package Manager with:
https://github.com/GameCult/CultLib.git?path=/unity/org.gamecult.cultlib#cultlib-unity-v1.0.57
```
The package depends on `org.gamecult.cultmath` (0.3.0 or later) and does not
carry its own `CultMath.dll`. Unity Package Manager resolves git packages only
from the project manifest, so add CultMath beside CultLib in
`Packages/manifest.json`:
```text
"org.gamecult.cultmath": "https://github.com/GameCult/CultLib.git?path=/packages/cultmath/unity/org.gamecult.cultmath#cultmath-unity-v0.3.0"
```
CultMesh chooses mapped memory for reachable same-machine bodies and advertised
QUIC for remote realtime state. Applications register the packaged
`CultMeshNativeQuicRealtimeTransportConnector`; they do not load or configure

1
unity/org.gamecult.cultlib/Runtime/GameCult.CultLib.asmdef
View file

@ -8,6 +8,7 @@
"overrideReferences": true,
"precompiledReferences": [
"ConcurrentCollections.dll",
"CultMath.dll",
"GameCult.Caching.dll",
"GameCult.Caching.MessagePack.dll",
"GameCult.Logging.dll",

3
unity/org.gamecult.cultlib/package.json
View file

@ -3,6 +3,9 @@
"displayName": "CultLib",
"version": "1.0.60",
"unity": "2021.3",
"dependencies": {
"org.gamecult.cultmath": "0.3.0"
},
"description": "CultCache, CultNet, and CultMesh managed runtime assemblies for Unity.",
"license": "MIT",
"author": {