From ce0d61fc715207910e0a4048921944d02642514a Mon Sep 17 00:00:00 2001 From: Meta Vens Date: Tue, 29 Sep 2026 23:47:14 +0200 Subject: [PATCH 01/16] Ack map pass 2: re-map on 3bf1c0c, split session generation into Cuts 1b and 1c Applies the Q-A1/Q-A2 rulings (one public meaning: Acknowledged means delivered). Adds Cut 1b (every session ending drops what it owed; flush is bound to its session) and Cut 1c (how a new generation is recognised, blocked on new Q-A6). Probes at 3bf1c0c reproduce G1, G1b, G2 and G3 in all four runtimes. Kotlin trails as Cut K pending Q-A5. Co-Authored-By: Claude Opus 5.5 --- docs/cultnet-rudp-ack-cut.md | 1333 ++++++++++++++++++---------------- 1 file changed, 721 insertions(+), 612 deletions(-) diff --git a/docs/cultnet-rudp-ack-cut.md b/docs/cultnet-rudp-ack-cut.md index f21b2006..892110e1 100644 --- a/docs/cultnet-rudp-ack-cut.md +++ b/docs/cultnet-rudp-ack-cut.md @@ -1,9 +1,9 @@ -# CultNet RUDP: Acknowledged means delivered, and the receiver owns ordered delivery with one watermark +# CultNet RUDP: Acknowledged means delivered, one watermark owns ordered delivery, and a write belongs to its session -Status: cut map, Imagination pass 1, 2026-09-30, committed by Self. +Status: cut map, Imagination pass 2, 2026-09-30, committed by Self. Pass 2 re-maps every cut against +`main` at `3bf1c0c` and applies the operator rulings below. -**Rulings (operator, 2026-09-30).** These supersede the two-state proposal in the text below. Cuts 4 and 5 -need re-mapping to them before Hands. +**Rulings (operator, 2026-09-30).** - **Q-A1 (a):** Delivered is derived at the sender from a correct receiver's cumulative watermark. The wire does not change. - **Q-A2:** "I think this is telling us we don't actually need both received and delivered." @@ -11,120 +11,102 @@ need re-mapping to them before Hands. - Receipts go Pending, then Acknowledged (delivered), or Invalidated. No public Received state exists. - Retransmission still stops on each packet's own ack, so resending stays bounded. - Flush waits for Acknowledged. -- **Open:** Q-A3 (expiring reliable traffic next to ordered traffic), Q-A4 (porting receipts), Q-A5 (Kotlin). -- **Added to Cut 1's scope** (from stray-packet Soul pass 3, pre-existing): a restarted client on the same - endpoint and payload is locked out by repeated-Connect answers. A reconnect on the same transport treats a new - server session's frames as duplicates. Session generation belongs to the sequence space this cut owns. -- **Also in Cut 1's scope** (stray-packet Soul pass 4): the ending paths disagree about what a session owed. - - In Rust, a peer's Disconnect invalidates receipts. - - A local `disconnect()` or a timeout carries owed writes into the next session. There, a receipt from the old - session can become Acknowledged (probe R1). - - TS, Python and C# also carry owed writes across a peer Disconnect. - - One rule: a write belongs to the session it was issued in. +- **Scope added from the stray-packet Soul passes 3 and 4:** session generation. A restarted client on the + same endpoint and payload is locked out. A reconnect on the same transport treats a new server session's + frames as duplicates. The ending paths disagree about owed writes. One rule: a write belongs to the + session it was issued in. Pass 2 maps this as **Cut 1b** (the ending rule) and **Cut 1c** (how a new + generation is recognised), separate from Cut 1 (§8 says why). +- **Open:** Q-A3 (expiring reliable traffic next to ordered traffic), Q-A4 (porting receipts), Q-A5 + (Kotlin), and a new **Q-A6** (how a new generation is recognised; blocks Cut 1c only). -- **Body.** `origin/hands/rudp-stray-packets` at `d85877a7` ("the session no - longer keeps an initial sequence nothing reads"). Every `file:line` is against - it unless it names another revision. - - `main` (`ae8576d8`) has not touched any RUDP file since the branch forked at - `60663f95`. `git diff --stat` over the four transport files and - `packages/cultmesh-kotlin` is empty. - - Hands is still working on that branch. The line numbers are good until it - moves again; the anchors (function names) are the durable reference. -- **Base of every cut.** `main` after `hands/rudp-stray-packets` merges. This - campaign does not start before that merge, and it does not rebase that branch. +- **Body.** CultLib `main` at `3bf1c0c` ("Merge RUDP stray-packet hardening across all four runtimes"). + Every `file:line` is against it unless it names another revision. + - Since pass 1's base `d85877a7`, only `rudp.rs` and `rudp.ts` moved among the transport files. + `CultNetTransport.cs`, `transport.py`, `CultMesh.kt`, `cultmesh-rs`, `cultmesh-ts`, `peer.ts`, the two + TS publishers and `docs/cultnet-transport-parity.md` are byte-identical to `d85877a7`. + - `rudp.rs` moved +6 lines from `:283` and +12 from `:596`. `rudp.ts` moved only after `:824`. + - Odin pins `3bf1c0c` (`Odin crates/odin-core/Cargo.toml:14-17`). +- **Base of every cut.** `main` at `3bf1c0c` or later. - **Scope.** The RUDP session and socket transports in all five runtimes: C# - `src/GameCult.Networking/CultNetTransport.cs`, Rust - `packages/cultnet-rs/src/rudp.rs` (plus `packages/cultmesh-rs`), TypeScript - `packages/cultnet-ts/src/rudp.ts`, Python - `packages/cultnet-py/src/cultnet_py/transport.py`, and Kotlin - `packages/cultmesh-kotlin/.../CultMesh.kt`. Kotlin was not in the brief. It is - a fifth RUDP implementation with the same ordered-delivery code - (`CultMesh.kt:2939-2978`) and the same codec. **Q-A5** decides whether it is in. -- **Wire.** This map changes no byte on the wire (§1). + `src/GameCult.Networking/CultNetTransport.cs`, Rust `packages/cultnet-rs/src/rudp.rs` (plus + `packages/cultmesh-rs`), TypeScript `packages/cultnet-ts/src/rudp.ts` (plus the two hand-rolled + publishers), Python `packages/cultnet-py/src/cultnet_py/transport.py`, and Kotlin + `packages/cultmesh-kotlin/src/main/kotlin/org/gamecult/cultmesh/CultMesh.kt`. **Q-A5** decides whether + Kotlin is in. Until it is ruled, every cut is mapped for four runtimes and Kotlin trails (§ Sequencing). +- **Wire.** This map changes no byte on the wire (§1). Cut 1c changes the default initial sequence, which + is a value, not a format. ## The ends (operator, 2026-09-30) -- A frame the peer has buffered but not delivered must not look delivered. - Today an ordered frame that arrives behind a gap is held and acknowledged; if - the session ends it is lost, and the sender saw "Acknowledged". -- Resending stays bounded. The sender stops retransmitting when the peer has - **received** the frame, exactly as today. Delivery-only acknowledgement was - rejected: "I don't like the possibility of unbounded resending on an - unresponsive peer application. TCP sounds like it has the right idea." -- Two meanings: **Received** (the frame is in the peer's session) and - **Delivered** (handed to the peer's application). Unordered and unreliable - channels deliver on receipt. Receipts go Pending, then Received, then - Delivered, plus the existing Invalidated. -- A session that ends with frames held leaves their receipts at Received, - visibly not Delivered. +- A frame the peer has buffered but not delivered must not look delivered. Today an ordered frame that + arrives behind a gap is held and acknowledged; if the session ends it is lost, and the sender saw + "Acknowledged". +- Resending stays bounded. The sender stops retransmitting a packet when the peer acknowledges that packet, + exactly as today. Delivery-only acknowledgement was rejected: "I don't like the possibility of unbounded + resending on an unresponsive peer application. TCP sounds like it has the right idea." +- One public meaning: **Acknowledged means delivered** to the peer's application boundary (§2). Unordered + reliable frames are delivered on receipt. Receipts go Pending, then Acknowledged, or Invalidated. +- A write belongs to the session it was issued in. When that session ends, a write not yet Acknowledged is + Invalidated. It is never carried into, retransmitted in, or acknowledged by a later session. ## The finding that shapes this map -Self's proposal puts a second acknowledgement on the wire: a per-channel -"delivered through N" watermark beside the received ack. Probing the receivers -changed the picture in two ways. +Self's first proposal put a second acknowledgement on the wire: a per-channel "delivered through N" +watermark beside the received ack. Probing the receivers changed the picture in two ways. -**1. The ordered receivers are broken in every runtime, and the break is -bigger than the session-end case.** A held frame is not only lost at session -end. In a live session it can be stranded forever, or dropped, while the sender -sees it acknowledged. The per-channel `next` pointer -(`CultNetTransport.cs:918`, `rudp.rs:194`, `rudp.ts:160`, `transport.py:424`, -`CultMesh.kt:2643`) has been patched twice (`436654de` added the -first-use initialisation, `4335ab10` added the skip over other channels' -sequences) and is still wrong: +**1. The ordered receivers are broken in every runtime, and the break is bigger than the session-end case.** +A held frame is not only lost at session end. In a live session it can be stranded forever, or dropped, +while the sender sees it acknowledged. The per-channel `next` pointer (`CultNetTransport.cs:918`, +`rudp.rs:196`, `rudp.ts:160`, `transport.py:424`, `CultMesh.kt:2643`) has been patched twice (`436654de` +added the first-use initialisation, `4335ab10` added the skip over other channels' sequences) and is still +wrong: -- **P1: a gap filled by another channel strands the ordered channel.** Ordered - `o1`, reliable `u` on another channel (lost), `o2`, `o3`. When `u` arrives, - only `u`'s channel is drained. `o2` and `o3` are never delivered, and the - sender's pending set is empty. The same holds with two ordered channels - (P1b). -- **P6: a channel first used after other traffic loses frames.** `C1`, `C2` on - a new channel, then `X` on another. `X` arrives, then `C2`, then `C1`. The - channel initialises its `next` from `min(highest+1, frame)`, so `C2` is - delivered first and `C1` is **dropped as old** (`frame.sequence < next`). It - was acknowledged. -- Both reproduce at the branch tip, at Odin's pin `a8aeddac` and at Muninn's - pin `c2a9a6e5`, in every runtime probed. +- **P1: a gap filled by another channel strands the ordered channel.** Ordered `o1`, reliable `u` on + another channel (lost), `o2`, `o3`. When `u` arrives, only `u`'s channel is drained. `o2` and `o3` are + never delivered, and the sender's pending set is empty. The same holds with two ordered channels (P1b). +- **P6: a channel first used after other traffic loses frames.** `C1`, `C2` on a new channel, then `X` on + another. `X` arrives, then `C2`, then `C1`. The channel initialises its `next` from + `min(highest+1, frame)`, so `C2` is delivered first and `C1` is **dropped as old** + (`frame.sequence < next`). It was acknowledged. +- Both reproduce at `3bf1c0c` in C#, Rust, TS and Python (pass 2), and did at `d85877a7`, Odin's old pin + `a8aeddac` and Muninn's pin `c2a9a6e5` (pass 1). -**2. Once the receiver is correct, "Delivered" is not new information.** A -conforming receiver holds an ordered frame for exactly one reason: some -reliable sequence below it has not arrived. When the last missing sequence -arrives, the same `Receive` call drains the held frames, before the -acknowledgement is built. So: +**2. Once the receiver is correct, "delivered" is not new information.** A conforming receiver holds an +ordered frame for exactly one reason: some reliable sequence below it has not arrived. When the last +missing sequence arrives, the same `Receive` call drains the held frames, before the acknowledgement is +built. So: -> An ordered frame is Delivered exactly when every reliable sequence up to its -> last one has been Received. +> An ordered frame is delivered exactly when every reliable sequence up to its last one has been +> acknowledged. -The sender already knows that. It is the sender's own **acknowledged-through** -watermark: one below the lowest sequence that is still pending or queued. -TCP's cumulative ACK means the same thing, "every byte below N is in the -receive buffer", and it sits beside SACK, which means Received. Here the -sender can compute the cumulative value itself, because every sequence is -acknowledged individually and removed from its pending set. +The sender already knows that. It is the sender's own **acknowledged-through** watermark: one below the +lowest sequence that is still pending, queued or abandoned. TCP's cumulative ACK means the same thing, +"every byte below N is in the receive buffer", and it sits beside SACK. Here the sender can compute the +cumulative value itself, because every sequence is acknowledged individually and removed from its pending +set. The operator ruled this (Q-A1 a), and then ruled that the per-packet ack is not a public meaning at +all (Q-A2): it only stops retransmission. -A delivered watermark on the wire would be a second writer of a value the -first one already determines. Old receivers would never send it, so the sender -would need the derived rule as a fallback anyway. **This map derives Delivered -at the sender and changes no wire byte.** The explicit watermark was probed -and is available as an additive option, but it is not recommended. **Q-A1** -puts it to the operator, because it departs from Self's proposal. +The derivation is sound only if nothing the receiver acknowledged is later dropped while the session +lives, and nothing one session acknowledges is credited to another. The probes found these places where +that fails: -The derivation is sound only if nothing the receiver acknowledged is later -dropped while the session lives. The probes found four places where it is: - -| # | Where an acknowledged frame is lost while the session lives | Runtimes | Fixed by | +| # | Where an acknowledged frame is lost, or credited to the wrong session | Runtimes | Fixed by | |---|---|---|---| | P1, P1b | A cross-channel gap strands the ordered channel | all five | Cut 1 | | P6 | Late channel initialisation drops the earlier frame | all five | Cut 1 | | P5 | A reliable fragment set is evicted after its fragments were acknowledged | Rust, TS, Python (C# and Kotlin have no eviction) | Cut 2 | | P8 | A retransmit more than 4,096 sequences behind the receiver's highest counts as a duplicate: acknowledged and dropped | all five | Cut 3 (the sender never lets its span reach it) | +| G2 | After `connect()` again on the same transport, a new server session's frames are duplicates of the old one's sequences: acknowledged and dropped | C#, Rust, TS, Python | Cut 1c | +| G3 | A write left owed when a session ends is retransmitted into the next session and delivered there, and in Rust its receipt reads `Acknowledged` | Rust (local `disconnect()`, timeout); C#, TS, Python (every ending) | Cut 1b | -And one place where the sender loses track without the receiver's help: +And places where the sender loses track, or is locked out, without the receiver's help: -| # | Sender-side lie | Runtime | Fixed by | +| # | Sender-side lie or lockout | Runtime | Fixed by | |---|---|---|---| +| G1 | A restarted client on the same endpoint and Connect payload gets an Ack, never an Accept, and stays unconnected | Rust hub, C# listener and C# server mode, TS and Python server mode (Rust server mode resets on every Connect instead) | Cut 1c | +| G1b | C# and Python server-mode transports drop a Connect from any endpoint but the first, so a client that restarts on a new port is locked out for good | C#, Python | Cut 1c | | P2 | An expired reliable packet leaves a permanent hole. Ordered frames after it are held forever, and `flush_reliable` succeeds (outstanding 0) | Rust (the only runtime with reliable expiry) | Cut 4 reports it honestly; **Q-A3** decides the real fix | | P9 | A receipt for an expiring send reads `Acknowledged` once the packet expires unacknowledged | Rust | Cut 4 | | P10 | `send_reliable` with a full window queues the frame, then returns an error ("receipts require a non-empty reliable packet set"), so a caller that retries sends it twice | Rust | Cut 4 | @@ -132,643 +114,770 @@ And one place where the sender loses track without the receiver's help: ## Probe evidence -Everything ran on Yggdrasil through `ygg-verify.sh`, from local-only probe -branches in the throwaway worktrees `F:\Projects\CultLib-imag-ack` and -`-imag-ack-old`. Each probe commit also survives on the Yggdrasil mirror as -`refs/verify/`. +Everything ran on Yggdrasil through `ygg-verify.sh`, from local-only probe branches in throwaway +worktrees. Each probe commit survives on the Yggdrasil mirror as `refs/verify/`. -- `de2bdfe8`, `09318d13`, `03eac6ff`: Rust session and socket probes on the - branch at `ca18dfac` (`packages/cultnet-rs/tests/probe_ack.rs`). -- `b70a69ca`: the C#, TS and Python probes (`ProbeAckTests.cs`, - `probe-ack.test.ts`, `probe_ack.py`) on `ca18dfac`. -- `2320a4e5`: the C# probes rebased on `d85877a7`. - - At `ca18dfac`, `GameCult.Networking` did not compile (`CultNetTransport.cs(2707)`: `ReplacedPeerReason` not in scope). `afd52e97` on the branch fixes it. -- `7fe5d726`: P5 and P8 in C#, TS and Python. -- `f0e7136c`: the Rust probes at Odin's pin `a8aeddac`. -- `bf8fc826`: all four runtimes' probes at Muninn's pin `c2a9a6e5`. -- `db7d2886`: **the prototype**, the Rust receiver rebuilt on one watermark - plus sender-derived receipt states (about +130/−125 in `rudp.rs`). +Pass 1 (base `d85877a7` and the pins; mirror `CultLib-imag-ack`): -| # | Probe | Branch tip (all runtimes probed) | Pins `a8aeddac` / `c2a9a6e5` | Prototype `db7d2886` | +- `de2bdfe8`, `09318d13`, `03eac6ff`: Rust session and socket probes (`packages/cultnet-rs/tests/probe_ack.rs`). +- `b70a69ca`, `2320a4e5`, `7fe5d726`: the C#, TS and Python probes (`ProbeAckTests.cs`, + `probe-ack.test.ts`, `probe_ack.py`). +- `f0e7136c`: the Rust probes at Odin's old pin `a8aeddac`. `bf8fc826`: all four at Muninn's pin `c2a9a6e5`. +- `db7d2886`: **the prototype**, the Rust receiver rebuilt on one watermark plus sender-derived receipt + states (about +130/−125 in `rudp.rs`). Its `Received`/`Delivered` receipt states are superseded by Q-A2; + its receiver is what Cut 1 builds. + +Pass 2 (base `3bf1c0c`; mirror `CultLib-imag-ack2`): + +- `a383e27d`, then `1519a20a` (the C# server-mode G1 probe acknowledges the Accept before the restart, so + it tests a restart and not a lost Accept): the pass-1 probe files restored unchanged, plus + `packages/cultnet-rs/tests/probe_gen.rs`, `packages/cultnet-ts/test/probe-gen.test.ts`, + `packages/cultnet-py/tests/probe_gen.py` and `tests/GameCult.Networking.Tests/ProbeGenTests.cs`. +- Rust and C# G1 run over real loopback UDP (hub, listener, server-mode transport), with the restarted + client bound to the first client's exact address. Python G1 runs over UDP against the server-mode + transport. TS G1 calls `acceptConnect` on the connected session, which is what the socket transport does + for a same-endpoint Connect (`rudp.ts:1034-1049` calls it without a reset, and `rudp.ts:249-251` answers + it as a repeat). G2 and G3 are session-level: the socket transports' `connect()` is `create_connect` and + nothing else (`rudp.rs:1874-1883`, `CultNetTransport.cs:1836-1846`, `rudp.ts:891-897`, + `transport.py:994-998`). + +| # | Probe | Pass 1 (`d85877a7`, pins) | Pass 2 (`3bf1c0c`) | Prototype `db7d2886` | |---|---|---|---|---| -| P1 | `o1`, reliable `u` lost, `o2`, `o3`; then `u`, then `o3` | Delivered `[o1, u]`. `o2`, `o3` never. Sender pending `[]`. Rust, C#, TS, Python. | Same in Rust (both pins), C#, TS, Python (`c2a9a6e5`) | `[o1, u, o2, o3]` | -| P1b | `A1`, `B1` (second ordered channel) lost, `A2`, `A3` | `[A1, B1]`, all four | Same | `[A1, B1, A2, A3]` | -| P6 | `C1`, `C2` on a new channel, then `X`; arrival `X`, `C2`, `C1` | `[C2]`. `C1` dropped. All four. | Same | `[C1, C2, X]` | -| P2 | Rust: reliable media with expiry 1 ms, lost; then `s1`, `s2` on `schema` | Nothing delivered. Sender outstanding 0, expired 1, no resends. `flush_reliable` would return Ok. | Same | Unchanged (held behind the hole). Cut 4 makes the sender say so. | -| P3 | `s1` lost, `s2` arrives and is acknowledged | `s2` not delivered, but acknowledged: sender pending `[s1]` only. A reset then loses `s2`. | Same | Receipt for `s2`: `Received`; after `s1` arrives, 2 frames delivered and `Delivered` | -| P4 | Rust: `s0` delivered, `s1` lost, then a 5 MiB ordered frame in 60 kB fragments, pumped with acks | `RUDP ordered hold buffer is full`: the session is refused. After the gap it would have delivered. With no gap, delivered. | (probe predates the fix; not evidence) | The prototype held every ordered frame, so a 5 MiB frame was refused **even with no gap**: hold only what is not yet deliverable (Cut 1) | -| P5 | 3-fragment reliable frame, 2 fragments arrive and are acked; 70 lossy unreliable fragment sets; last fragment arrives | Rust, TS, Python: reliable frame never delivered, sender pending `[]` | Same (Rust) | Unchanged (Cut 2) | -| P7 | An Ack with flag bit `0x10`, a nonzero sequence field, or a 5-byte payload | Decoded and applied (ack cleared the pending sequence) in Rust, C#, TS, Python | Same at both pins | n/a | -| P8 | Reliable `g` lost while 4,200+ later sequences are sent and acked; then `g` arrives | `g` not delivered, and acknowledged: the sender stops. Rust, C#, TS, Python. | n/a | Unchanged (Cut 3) | -| P9 | Rust socket: `send_reliable("media")` with expiry 1 ms, never read; a ping triggers the purge | `Pending`, then `Acknowledged` | n/a | `Pending` (the abandoned sequence is never acknowledged) | -| P10 | Rust socket: 40 `schema` sends, then `send_reliable` | Error "receipts require a non-empty reliable packet set", after the frame was queued | n/a | Not addressed by the prototype (Cut 4) | +| P1 | `o1`, reliable `u` lost, `o2`, `o3`; then `u`, then `o3` | Delivered `[o1, u]`; `o2`, `o3` never; sender pending `[]`. Rust, C#, TS, Python; same at both pins | Same, all four | `[o1, u, o2, o3]` | +| P1b | `A1`, `B1` (second ordered channel) lost, `A2`, `A3` | `[A1, B1]`, all four | Same, all four | `[A1, B1, A2, A3]` | +| P6 | `C1`, `C2` on a new channel, then `X`; arrival `X`, `C2`, `C1` | `[C2]`; `C1` dropped. All four | Same, all four | `[C1, C2, X]` | +| P2 | Rust: reliable media with expiry 1 ms, lost; then `s1`, `s2` on `schema` | Nothing delivered. Sender outstanding 0, expired 1, no resends | Same | Unchanged (held behind the hole). Cut 4 makes the sender say so | +| P3 | `s1` lost, `s2` arrives and is acknowledged | `s2` not delivered, but acknowledged: sender pending `[s1]` only | Same | Receipt for `s2` stays short of delivered until `s1` arrives | +| P4 | Rust: `s0` delivered, `s1` lost, then a 5 MiB ordered frame in 60 kB fragments | `RUDP ordered hold buffer is full`: the session is refused. With no gap, delivered | Same | The prototype held every ordered frame, so a 5 MiB frame was refused **even with no gap**: hold only what is not yet deliverable (Cut 1) | +| P5 | 3-fragment reliable frame, 2 fragments acked; 70 lossy fragment sets; last fragment arrives | Rust, TS, Python: never delivered, sender pending `[]` | Same, Rust, TS, Python | Unchanged (Cut 2) | +| P7 | An Ack with flag bit `0x10`, a nonzero sequence field, or a 5-byte payload | Decoded and applied in Rust, C#, TS, Python | Same, all four | n/a | +| P8 | Reliable `g` lost while 4,200+ later sequences are sent and acked; then `g` arrives | `g` not delivered, and acknowledged. All four | Same, all four | Unchanged (Cut 3) | +| P9 | Rust socket: `send_reliable("media")` with expiry 1 ms, never read | `Pending`, then `Acknowledged` | Same | `Pending` | +| P10 | Rust socket: 40 `schema` sends, then `send_reliable` | Error "receipts require a non-empty reliable packet set", after the frame was queued | Same (the probe's `?` fails on that error) | Not addressed (Cut 4) | +| G1 | Client A connects, sends one ordered frame (acknowledging the Accept), and exits. Client B, a fresh session with the same initial sequence, binds A's exact address and sends the same Connect payload five times | n/a | **Locked out.** Rust hub (payload `"svc"` and empty): 5 Acks, 0 Accepts, B unconnected, the hub still holds A's generation. C# listener: same, no `PeerDisconnected`. C# server mode: 5 Acks. Python server mode: 5 Acks. TS session: the answer is an Ack. Rust server mode (control): B gets an Accept and its frame is delivered, because it resets on every Connect | Not probed | +| G1b | C# and Python server mode: a third client on a new port sends a Connect | n/a | No reply; the datagram is counted dropped | Not probed | +| G2 | Client connects to server 1 (initial sequence 1) and receives `old1..3`. Server 1 is gone; the client connects again on the same session to server 2 (initial sequence 1), which sends `new1`, `new2` | n/a | **`new1`, `new2` never delivered, and acknowledged** (server 2's pending set empties). C#, Rust, TS, Python | Not probed | +| G3 | Client sends one ordered write that is lost; the session ends; the client connects to a new server; the client's resends run | n/a | Owed after the end, and **delivered in the next session**: C#, TS, Python on peer Disconnect, local disconnect and timeout; Rust on local disconnect and timeout. Rust on peer Disconnect: owed 0, nothing delivered (`5f1e883d`) | Not probed | +| G3R | Rust socket: `send_reliable`, the write is lost, `disconnect()`, `connect()` on the same transport to a new peer | n/a | Receipt `Pending` after the disconnect; the write is delivered to the new peer; **receipt `Acknowledged`** (the Soul R1 shape) | Not probed | -**Suites on the prototype `db7d2886`** (baseline `03eac6ff`, the probe commit -without the prototype, was all green): +**Suites on the prototype `db7d2886`** (pass 1; baseline `03eac6ff` was all green): -- `cultnet-rs`: 110 + 4 + 24 pass; `tests/cultnet.rs` 51 pass and 2 fail. - Both failures are test shape, not mechanism: +- `cultnet-rs`: 110 + 4 + 24 pass; `tests/cultnet.rs` 51 pass and 2 fail. Both failures are test shape, + not mechanism: - `rudp_session_suppresses_duplicates_and_delivers_reliable_ordered_payloads_in_sequence` - (`tests/cultnet.rs:1000`) hands the receiver an Accept with sequence 91 from a - sender whose data starts at 1. That handshake is impossible. Under a - watermark seeded by the Accept, every data sequence is "already below" it. - The test must use a consistent handshake. - - `rudp_ordered_channel_tracking_is_bounded` (`:2655`) sends 256 ordered - channels with no gap. The prototype keeps state only for channels with held - frames, so nothing is refused. The bound becomes "channels with held - frames", and the test must hold frames behind a gap. -- `cultmesh-rs` did not compile: `lib.rs:892` matches the status enum - exhaustively and does not cover `Received`. Cut 4 rewrites that call site. -- The probe suite: P1, P1b, P6 and P3b are fixed as the table shows. + (`tests/cultnet.rs:1000`) hands the receiver an Accept with sequence 91 from a sender whose data starts + at 1. That handshake is impossible. Under a watermark seeded by the Accept, every data sequence is + "already below" it. The test must use a consistent handshake. + - `rudp_ordered_channel_tracking_is_bounded` (`:2655`) sends 256 ordered channels with no gap. The + prototype keeps state only for channels with held frames, so nothing is refused. The bound becomes + "channels with held frames", and the test must hold frames behind a gap. +- `cultmesh-rs` did not compile only because the prototype added a status variant. Under Q-A2 the status + enum keeps its three variants, so that break does not recur. -**Wire compatibility (P7).** Every decoder ignores flag bits 4-7 and byte 35. -Every session ignores an Ack's sequence field and payload -(`CultNetTransport.cs:1177`, `rudp.rs:572-586`, `rudp.ts:397`, -`transport.py:612`, `CultMesh.kt:2738`). Every decoder rejects an unknown -packet type (`CultNetTransport.cs` `Enum.IsDefined`, `rudp.rs` -`packet_type_from_code`, `rudp.ts:1290`, `transport.py:1212`, -`CultMesh.kt` `fromCode`) and an unknown version byte. So: +**Wire compatibility (P7).** Every decoder ignores flag bits 4-7 and byte 35. Every session ignores an +Ack's sequence field and payload (`CultNetTransport.cs:1177`, `rudp.rs:578-592`, `rudp.ts:397`, +`transport.py:612`, `CultMesh.kt:2738`). Every decoder rejects an unknown packet type +(`CultNetTransport.cs` `Enum.IsDefined`, `rudp.rs` `packet_type_from_code`, `rudp.ts:1313`, +`transport.py:1212`, `CultMesh.kt` `fromCode`) and an unknown version byte. So: -- An additive watermark fits in an Ack: flag bit 4 set, the watermark in the - Ack's unused sequence field, zero extra bytes. Old peers apply the ack and - ignore the rest, proven at both pins. -- A new packet type or a version bump is not additive: old peers refuse the - datagram, and since the stray-packet branch they count it as dropped. -- No capability negotiation is needed under this map, because nothing on the - wire changes. +- An additive watermark would fit in an Ack: flag bit 4 set, the value in the Ack's unused sequence field, + zero extra bytes. Old peers apply the ack and ignore the rest. Q-A1 ruled it out; it stays the known + place to put a FORWARD-TSN notice if Q-A3 is ever B. +- A new packet type or a version bump is not additive: old peers refuse the datagram and count it dropped. +- No capability negotiation is needed under this map, because nothing on the wire changes. ## Authority map -- **Owner of ordered delivery (receiver): the session's contiguous - received-through watermark `R`.** - - Every reliable sequence up to `R` has been received. - - A held ordered frame whose first sequence is at most `R + 1` is delivered, - in sequence order, in the same call that advanced `R`. - - `R` is seeded by the handshake. `AcceptConnect` seeds it with the Connect's - sequence, and receiving the Accept seeds it with the Accept's sequence. A - session connected without a handshake (Rust `assume_connected`, - `rudp.rs:241`, test-only today) seeds it from its first reliable packet. - - It advances in `RememberReceived` and resets in `ResetPeerState`. -- **Owner of Received (sender): the pending and queued sets**, as today. A - sequence leaves them when an ack names it. Retransmission stops there. -- **Owner of Delivered (sender): the acknowledged-through watermark `A`**, one - below the lowest sequence that is pending, queued or abandoned. An ordered - send is Delivered when `A` reaches its last sequence. A reliable unordered - send is Delivered when it is Received. -- **Inputs.** Received sequences (receiver); acks and expiry (sender). Nothing - else. -- **Outputs.** Delivered frames, in order (receiver). Receipt status and flush - completion (sender). +- **Owner of ordered delivery (receiver): the session's contiguous received-through watermark `R`.** + - Every reliable sequence up to `R` has been received since the peer state was last reset. + - A held ordered frame whose first sequence is at most `R + 1` is delivered, in sequence order, in the + same call that advanced `R`. + - `R` is seeded by the handshake. `AcceptConnect` seeds it with the Connect's sequence, and receiving + the Accept seeds it with the Accept's sequence. A session connected without a handshake (Rust + `assume_connected`, `rudp.rs:243`, test-only today) seeds it from its first reliable packet. + - It advances in `RememberReceived` and resets with the rest of the peer state (`ResetPeerState`). +- **Owner of retransmission (sender): the pending and queued sets**, as today. A sequence leaves them when + an ack names it. Retransmission stops there. This is not a public meaning. +- **Owner of Acknowledged (sender): the acknowledged-through watermark `A`**, one below the lowest sequence + that is pending, queued or abandoned in the current generation. An ordered send is Acknowledged when `A` + reaches its last sequence. A reliable unordered send is Acknowledged when every one of its sequences is + acked. +- **Owner of the session boundary: the session's generation.** One primitive ends a generation: the + session stops being connected, every write not yet Acknowledged is Invalidated and dropped from pending + and queued, and the generation counter advances. Every ending path calls it: a peer's Disconnect, a + local disconnect, a timeout and a refused packet (Cut 1b), and from Cut 1c a client's `connect()` on a + session that already had a peer, and a server accepting a Connect that does not repeat the current one. + - What was learned from the peer (`R`, the received set, held frames) is forgotten only where both sides + start fresh: a refusal (as today) and, from Cut 1c, a non-repeated handshake. Forgetting it at a + one-sided ending (a timeout the peer did not see) would let the peer's retransmits be delivered twice. +- **Owner of "is this Connect a repeat": the session**, from the Connect's sequence (Cut 1c; Q-A6). +- **Inputs.** Received sequences and handshake packets (receiver); acks, expiry and ending events + (sender). Nothing else. +- **Outputs.** Delivered frames, in order (receiver). Receipt status and flush completion (sender). - **Derived, not owners.** - - The per-channel `next` map is **deleted**, not demoted: - `_orderedNextSequenceByChannel` (`CultNetTransport.cs:918`), - `ordered_next_sequence_by_channel` (`rudp.rs:194`), - `#orderedNextSequenceByChannel` (`rudp.ts:160`), - `_ordered_next_sequence_by_channel` (`transport.py:424`), - `orderedNextSequenceByChannel` (`CultMesh.kt:2643`). - - `expectedSequenceIfUninitialized` and its equivalents are deleted - (`CultNetTransport.cs:1151`, `rudp.rs:534`, `rudp.ts:375`, - `transport.py:595`, `CultMesh.kt:2728`). - - `PendingOrderedFrame.NextSequence` is deleted. - - `OutstandingReliablePacketCount` stays, and means "not yet Received". -- **Forbidden writers.** No path delivers an ordered frame except the drain - after `R` advances. No receive path drops a frame whose sequence it - acknowledged while the session lives. No sender path reports Delivered - except through `A`. -- **Shared paths.** Accept, reliable data, fragments, duplicates and - retransmits all advance `R` through `RememberReceived` and then call the - drain. Direct sends, fragmented sends, queued promotion and resends all feed - the same pending and queued sets that `A` reads. -- **Deletion line.** `DeliverOrdered`, `DrainOrdered` and - `SkipReceivedNonChannelSequences` and their equivalents go before the drain - is added: `CultNetTransport.cs:1628-1700`, `rudp.rs:1126-1234`, - `rudp.ts:737-800`, `transport.py:868-920`, `CultMesh.kt:2939-2978`. + - The per-channel `next` map is **deleted**, not demoted: `_orderedNextSequenceByChannel` + (`CultNetTransport.cs:918`), `ordered_next_sequence_by_channel` (`rudp.rs:196`), + `#orderedNextSequenceByChannel` (`rudp.ts:160`), `_ordered_next_sequence_by_channel` + (`transport.py:424`), `orderedNextSequenceByChannel` (`CultMesh.kt:2643`). + - `expectedSequenceIfUninitialized` and its equivalents are deleted (`CultNetTransport.cs:1151`, + `rudp.rs:540`, `rudp.ts:375`, `transport.py:595`, `CultMesh.kt:2728`). + - `PendingOrderedFrame.NextSequence` is deleted (`rudp.rs:160-163` and equivalents). + - `OutstandingReliablePacketCount` stays, and means "not yet acked in this generation". It is a + retransmission count, not a delivery claim. + - Rust `session_scope: Uuid` (`rudp.rs:135`, `:179`, `:206`, rotated at `:345` and `:601`) is no longer an + owner. It is replaced by the generation (Cut 1b) and then by receipts resolved at their source + (Cut 4). + - TS `#generation`, `#endedGeneration` and `#endGeneration` in the socket transport (`rudp.ts:830-831`, + `:937-940`) are no longer owners; the session's generation replaces them. + - `DisconnectReason` / `disconnect_reason` / `#endedReason` are report-only: why the last generation + ended. They no longer gate flush. +- **Forbidden writers.** + - No path delivers an ordered frame except the drain after `R` advances. + - No receive path drops a frame whose sequence it acknowledged while the generation lives. + - No sender path reports Acknowledged except through `A` (or the unordered rule). + - No ending path leaves a write owed. No path retransmits a write into a generation after the one it was + issued in. + - No transport decides whether a Connect repeats: not by payload (Rust hub `rudp.rs:1637-1647`, C# + listener `CultNetTransport.cs:2670-2688`), not by `Connected` (C# `:2226-2236`, Python + `transport.py:1047-1053`, TS `rudp.ts:249-251`, `:1031-1049`), and not by resetting on every Connect + (Rust server mode `rudp.rs:2105-2116`). +- **Shared paths.** Accept, reliable data, fragments, duplicates and retransmits all advance `R` through + `RememberReceived` and then call the drain. Direct sends, fragmented sends, queued promotion and resends + all feed the pending and queued sets that `A` reads. Every ending path goes through the one generation + primitive. +- **Deletion line.** `DeliverOrdered`, `DrainOrdered` and `SkipReceivedNonChannelSequences` and their + equivalents go before the drain is added: `CultNetTransport.cs:1628-1700`, `rudp.rs:1138-1246`, + `rudp.ts:737-800`, `transport.py:868-920`, `CultMesh.kt:2939-2978`. The transport-level repeat decisions + above go before the session's is added. ## The design, question by question -### 1. Where Delivered comes from +### 1. Where Acknowledged comes from (Q-A1 a, ruled) - Derived at the sender from `A` (§ The finding). No wire change. - **Why it is sound after Cuts 1-3.** A conforming receiver: - - drops nothing it acknowledged while the session lives (P1, P6: Cut 1; P5: - Cut 2; P8: made unreachable by Cut 3); + - drops nothing it acknowledged while the session lives (P1, P6: Cut 1; P5: Cut 2; P8: made unreachable + by Cut 3); - holds an ordered frame only while a sequence below it is missing (Cut 1); - - drains held frames in the call that fills the last gap, before it - acknowledges (Cut 1). - - So every sequence up to `s` acknowledged means `s` was delivered. -- **Mixed versions.** A new sender with an old receiver can be wrong only when - that receiver strands a frame (P1, P6). That needs reliable traffic on more - than one channel of one session. The C#, TS, Python and Kotlin socket - transports send reliably only on `schema` (`CultNetTransport.cs:2372-2376`, - `rudp.ts:1233-1241`), so they cannot trigger it. Rust sends reliable - `media`/`audio` too (`rudp.rs` `channel_send_options`). The Eyes survey found - no live session that mixes those with `schema`: Muninn's media session - carries no `schema` traffic (`Muninn crates/muninn-daemon/src/main.rs:3099-3158`), - and neither does Sleipnir's HID stream (`Odin crates/sleipnir-daemon/src/main.rs:2217-2272`). - The old receiver loses such frames today in any case. Bumping the pin is the - fix. -- **Rejected: a wire watermark (Q-A1 B).** It is additive (P7), but it is a - second authority for a value the first one determines. Old receivers never - send it, so the derived rule is needed as a fallback anyway. + - drains held frames in the call that fills the last gap, before it acknowledges (Cut 1). + - So every sequence up to `s` acked means `s` was delivered. +- **And after Cuts 1b and 1c,** an ack can only be credited to the generation that issued the write + (G3, G2). +- **Mixed versions.** A new sender with an old receiver can be wrong only when that receiver strands a + frame (P1, P6). That needs reliable traffic on more than one channel of one session. The C#, TS, Python + and Kotlin socket transports send reliably only on `schema` (`CultNetTransport.cs:2372-2376`, + `rudp.ts:1254-1262`), so they cannot trigger it. Rust sends reliable `media`/`audio` too + (`rudp.rs:2523-2562` `channel_send_options`). The pass-1 Eyes survey found no live session that mixes + those with `schema`: Muninn's media session carries no `schema` traffic, and neither does Sleipnir's HID + stream. The old receiver loses such frames today in any case. Bumping the pin is the fix. +- **Every Odin-bound publisher's receiver is Odin.** Acknowledged means delivered for Idunn health, + presence and catalog publishing only once Odin pins a CultLib with Cuts 1-3. ### 2. What "delivered to the application" means -- **Delivered means the frame left the session's hold and was handed to the - transport's application boundary.** That boundary is: - - C#: enqueued in the socket transport's `_deliveredFrames` (read by - `TryReceiveOnce`), or returned in `CultNetRudpReceiveResult.Delivered` from - a bare session; - - Rust: pushed to `delivered_frames` (`rudp.rs:2137`) or the hub's - `pending_events`, or handed to the document server's sink - (`rudp_document_server.rs:343-360`), which acknowledges only after the sink - accepted; - - TS: the `"frame"` event was emitted (`rudp.ts:1055-1061`); +- **Delivered means the frame left the session's hold and was handed to the transport's application + boundary.** That boundary is: + - C#: enqueued in the socket transport's `_deliveredFrames` (read by `TryReceiveOnce`), or returned in + `CultNetRudpReceiveResult.Delivered` from a bare session; + - Rust: pushed to `delivered_frames` (`rudp.rs:2147`) or the hub's `pending_events`, or handed to the + document server's sink (`cultmesh-rs/src/rudp_document_server.rs:343-360`), which acknowledges only + after the sink accepted; + - TS: the `"frame"` event was emitted (`rudp.ts:1076-1082`); - Python and Kotlin: the delivered queue. -- **It is not "consumed".** The operator rejected application-driven - acknowledgement, and TCP's cumulative ACK has the same boundary: in the - receive buffer, readable, not read. -- A listener or sink that throws or rejects has still been delivered to. The - handler-rejection paths from the stray-packet branch end the session, which - is the application's answer, not the transport's. +- **It is not "consumed".** The operator rejected application-driven acknowledgement, and TCP's cumulative + ACK has the same boundary: in the receive buffer, readable, not read. +- A listener or sink that throws or rejects has still been delivered to. The handler-rejection paths from + the stray-packet work end the session, which is the application's answer, not the transport's. -### 3. The public API +### 3. The public API (Q-A2, ruled: one meaning) - **Rust receipts** (the only runtime that has them): - - `CultNetRudpReliableSendStatus` becomes `Pending`, `Received`, `Delivered`, - `Invalidated` (`rudp.rs:144-148`). `Acknowledged` is removed, not aliased. - Every caller must choose, and every caller today means Delivered. - - The receipt records whether its send was ordered (`rudp.rs:132-135`). - - After the session ends without a reset: `Pending` becomes `Invalidated` - (as today, `rudp.rs:1919-1928`). `Received` stays `Received`, which the - ruling asks for. `Delivered` stays. - - After a reset (`reset_peer_state`: a new Connect generation, or - `end_refused_session`), the session keeps one frozen snapshot of the ended - epoch: its scope, `A`, and its still-unacknowledged sequences, at most the - reliable window plus the queue. A receipt from that epoch resolves against - the snapshot, so `Received` stays `Received` and `Delivered` stays - `Delivered`. Older epochs are `Invalidated`. - - Today every receipt, delivered or not, reads `Invalidated` after a - reset, because the scope rotates (`rudp.rs:338-339`). - - `send_reliable` receipts cover queued packets (P10). The receipt lists every - sequence the send assigned, not only the ones admitted to the wire. - - `send_reliable` refuses a channel with reliable expiry (P9). The stale - comment at `rudp.rs:1903-1904` says this lineage has no expiry, which is - false since `bac5be88`. Restore the guard it describes. - - New: `CultNetRudpSocketTransportConnection::wait_delivered(&receipt, timeout) -> Result<()>`. - It pumps receive and resends like `flush_reliable`. It fails at once when - the session ended, naming the frozen status. Today four places hand-roll - that loop: `cultmesh-rs/src/lib.rs:806-838` and `:870-900`, - `CodexConnector src/idunn_health.rs:727-745` and - `Ghostlight crates/ghostlight-dungeon/src/idunn_health.rs:1126-1145`. -- **Flush: one wait, and it means Delivered** (**Q-A2**). - - When every sent sequence is Received, `A` is the highest sequence sent, so - every send is Delivered. The two waits coincide, except behind an abandoned - sequence. - - Rust `flush_reliable` (`rudp.rs:1985`) therefore fails at once when an - ordered send follows the session's first abandoned sequence. That case - never becomes Delivered, and today it reports success (P2). - - C# `FlushReliable` (`CultNetTransport.cs:1914`), TS `flush` - (`rudp.ts:906`), Python `flush_reliable` (`transport.py:1101`) and Kotlin - `flushReliable` (`CultMesh.kt:3186`) keep their code. Their contract text - changes to "every reliable frame sent has been delivered to the peer's - application". These runtimes have no expiry. - - A second "flush until received" variant would differ only in the - abandoned-sequence case, where it would report success on frames that - will never be delivered. It has no consumer. Not added. -- **Defaults.** Plain flush waits for Delivered. No parameter. -- **Compatibility.** No C#, TS, Python or Kotlin signature changes. Rust: - removing `Acknowledged` breaks the external callers at their next bump. Each - is a one-line change to `Delivered`, or a switch to `wait_delivered`. The - pinned callers are CodexConnector (`e171eca`) and Ghostlight (`85f7024`). + - `CultNetRudpReliableSendStatus` keeps its three variants, `Pending`, `Acknowledged`, `Invalidated` + (`rudp.rs:146-150`). No variant is added, removed or renamed. `Acknowledged` changes meaning from + "every packet was acked" to "delivered". + - The visible difference: an ordered send behind a missing sequence reads `Pending` until the gap + fills (P3), where today it reads `Acknowledged`. + - Every external caller already means delivered, so none changes code: `cultmesh-rs/src/lib.rs:832` + and `:892`, `tests/rudp_stray_packets.rs:309`, `cultmesh-rs/tests/rudp_document_server.rs:408,415`, + CodexConnector `src/idunn_health.rs:727-745`, Ghostlight + `crates/ghostlight-dungeon/src/idunn_health.rs:1126-1145`. + - **A receipt is resolved once, by its session.** It becomes `Acknowledged` when `A` passes its last + sequence (ordered) or when all its sequences are acked (unordered). It becomes `Invalidated` when its + generation ends first. After that it never changes. + - Mechanism: the receipt carries its sequences, whether it was ordered, and a shared status cell + (`Arc`). The session keeps the unresolved cells, at most one per outstanding send, and + writes each once. `reliable_send_status(&receipt)` (`rudp.rs:1931-1936`) reads the cell; its + signature does not change. + - This deletes the session scope (`rudp.rs:135`, `:179`, `:206`, `:283-285`, `:291-293`, `:345`, + `:601`) and needs no history of ended generations. Today a delivered write reads `Invalidated` after + any reset, because the scope rotates (`rudp.rs:291-293`). The pass-1 design needed a frozen snapshot + of one ended epoch and still misreported older ones. + - `PartialEq` on the receipt compares its sequences, which are unique within a session object because + issued sequences survive resets. + - `send_reliable` receipts cover queued packets (P10). The receipt lists every sequence the send + assigned, not only the ones admitted to the wire (`send_many`, `rudp.rs:456-528`, returns only the + admitted ones; `non_expiring_reliable_receipt`, `:305-318`, refuses an empty set). + - `send_reliable` refuses a channel with reliable expiry (P9). The comment at `rudp.rs:1915-1916` says + this lineage has no expiry, which is false since `bac5be88`. Restore the guard it describes. + - New: `CultNetRudpSocketTransportConnection::wait_acknowledged(&receipt, timeout) -> Result<()>`. It is + the ack wait: it pumps receive and resends like `flush_reliable` and returns when the receipt is + `Acknowledged`. It fails at once when the receipt is `Invalidated`, naming the end reason. It replaces + the hand-rolled loops at `cultmesh-rs/src/lib.rs:806-838` and `:870-900`; CodexConnector and + Ghostlight may adopt it at their next bump, and their existing polling stays correct without it. +- **Flush: one wait, and it means every send of this generation is Acknowledged.** + - A flush belongs to the generation it started in. It fails when that generation ends, whatever comes + after. Rust binds it to the scope today (`rudp.rs:2001-2014`) and TS to its transport generation + (`rudp.ts:911-935`). C# (`CultNetTransport.cs:1914-1931`) and Python (`transport.py:1101-1124`) check + only the disconnect reason, which the next Connect clears (`CultNetTransport.cs:1843`, `:2235`; + `transport.py:997`, `:1054`). Once Cut 1b drops owed writes at every ending, "outstanding is 0" after a + reconnect would read as success, so every runtime binds flush to the session's generation in Cut 1b. + - When every sent sequence is acked, `A` is the highest sequence sent, so every send is Acknowledged. + The only exception is behind an abandoned sequence (Rust expiry). There `flush_reliable` + (`rudp.rs:1992-2040`) fails at once when an ordered send follows the generation's first abandoned + sequence: that send can never be Acknowledged, and today flush reports success (P2). + - C# `FlushReliable`, TS `flush`, Python `flush_reliable` and Kotlin `flushReliable` + (`CultMesh.kt:3186`) keep their success condition. Their contract text becomes "every reliable frame + sent in this session has been delivered to the peer's application". These runtimes have no expiry. +- **Defaults.** Plain flush. No parameter. No second flush. +- **Compatibility.** No signature changes in any runtime except additions (`wait_acknowledged`; TS + `connectAndWait`, §4). The Rust enum is unchanged. No downstream caller has to change code at its next + bump. ### 4. Consumers, each assigned -Surveyed across `F:\Projects` (the HEAD of each repo is recorded in the Eyes -report). Every consumer that waits wants **Delivered**. None wants Received -alone. +Surveyed across `F:\Projects` in pass 1; the rows that moved are re-checked at the revision named. Every +consumer that waits wants delivered, which is now just Acknowledged. -| Consumer | What it waits on today | Assigned | Change | -|---|---|---|---| -| `cultmesh-rs/src/lib.rs:781-838` `request_raw_snapshot_from_rudp_catalog` (Idunn `src/drivers.rs:5527` uses it) | `Acknowledged`, then the response | Delivered (the response proves it anyway) | Cut 4: `wait_delivered` | -| `cultmesh-rs/src/lib.rs:870-900` `publish_cultnet_message_to_rudp_catalog` (Odin `odin-daemon/src/main.rs:616-658`) | `Acknowledged`, then disconnect. The doc at `:869` claims "application admission". | Delivered. That is true at the document server, which acknowledges after its sink accepted (`rudp_document_server.rs:343-372`). | Cut 4: `wait_delivered`; the doc says Delivered | -| `cultmesh-rs/src/lib.rs:918-938` `publish_cultnet_messages_to_rudp_catalog` (Ghostlight `mesh.rs:323`) | `flush_reliable`, then drop. **Several ordered frames, then close.** | Delivered | None; flush now means it | -| `cultmesh-ts/src/index.ts:6070-6098` `publishRudpDocumentOnce` (Heimdall, StreamPixels, Stonks, weksa, Bifrost, VoidBot, Hermodr) | `peer.flush` (`cultnet-ts/src/peer.ts:109-110`), then close | Delivered | None | -| `cultnet-ts/src/idunn-odin-presence-publisher.ts:100-127` | a hand-rolled session loop that waits for `outstandingReliablePacketCount === 0`, then closes. It never resends, and queued packets are never sent. | Delivered | Cut 5: use the socket transport and `flush` | -| `cultnet-ts/src/signed-daemon-health.ts:113-160` (Hermodr, Odin `idunn-rudp.cjs`) | a hand-rolled loop: sends only the packets `sendMany` admitted, once; waits up to 500 ms for **any** ack packet, swallows the timeout, closes | Delivered | Cut 5: use the socket transport and `flush`. Today a lost datagram is reported as published, and a payload over 32 packets is truncated. | -| `cultnet-ts/src/operation-service.ts` | the operation response | none (the response is the proof) | none | -| C# `FlushReliable` callers: Gjallar `Program.cs:3055-3080` (two ordered frames, flush, dispose) | flush | Delivered | None | -| Heimdall `src/idunn-rudp-health.ts:75-86` (vendored CultLib `b6b1d9c`) | first ack packet | Delivered | Heimdall's own cut at its bump (follow-up) | -| CodexConnector `idunn_health.rs:727-745`, Ghostlight `idunn_health.rs:1126-1145` | `Acknowledged` | Delivered | At their next pin bump | -| Muninn `main.rs:9446-9458`, Sleipnir `main.rs:880-904` (both copies) | nothing (send and drop) | none today | Follow-up: fire-and-forget health can be silently lost | -| Kotlin `flushReliable` | no callers | Delivered | Contract text only | +| Consumer | What it waits on today | Change | +|---|---|---| +| `cultmesh-rs/src/lib.rs:781-838` `request_raw_snapshot_from_rudp_catalog` (Idunn `src/drivers.rs:5527` uses it) | a hand-rolled loop until `Acknowledged`, then the response | Cut 4: `wait_acknowledged` | +| `cultmesh-rs/src/lib.rs:870-900` `publish_cultnet_message_to_rudp_catalog` (Odin `odin-daemon`) | a hand-rolled loop until `Acknowledged`, then disconnect. The doc at `:869` claims "application admission" | Cut 4: `wait_acknowledged`; the doc says delivered to the document server's sink (`rudp_document_server.rs:343-372`) | +| `cultmesh-rs/src/lib.rs:918-938` `publish_cultnet_messages_to_rudp_catalog` (Ghostlight `mesh.rs:323`) | `flush_reliable`, then drop. **Several ordered frames, then close** | None; flush now means it | +| `cultmesh-ts/src/index.ts:6070-6098` `publishRudpDocumentOnce` (Heimdall, StreamPixels, Stonks, weksa, Bifrost, VoidBot, Hermodr) | `peer.flush` (`cultnet-ts/src/peer.ts:109-110`), then close | None. Cut 5 moves its connect-and-wait helper (`index.ts:6603-6650`) into cultnet-ts | +| `cultnet-ts/src/idunn-odin-presence-publisher.ts:89-215` `publishDocument`, behind `createIdunnRuntimePresencePublisher`. **StreamPixels' service runs this**: `apps/service/scripts/idunn-entry.mjs` calls `createIdunnRuntimePresencePublisher` from its vendored CultLib `30ee8b9`, where the file is byte-identical to `3bf1c0c` | a hand-rolled session loop (`:186-208`) until `outstandingReliablePacketCount === 0`. It does resend (`:205`). It drops `readyToSend` (`:198`), so promoted queued packets wait for the resend timer. It pins `initialSequence: 1` (`:103`) and never sends a Disconnect, so Odin's hub keeps each publish's peer until it times out (each publish binds a new port, so each is a new peer) | Cut 5: the socket transport, `connectAndWait`, `send("schema")`, `flush`, `close` (which sends the Disconnect, `rudp.ts:956-969`) | +| `cultnet-ts/src/signed-daemon-health.ts:104-162` `publishSignedDaemonHealth` (Hermodr, Odin `idunn-rudp.cjs`) | a hand-rolled loop: sends the packets `sendMany` admitted, waits up to 500 ms for the **first** Ack packet of any content (`:146-153`), swallows the timeout, closes. Resends run inside that wait (`:303`) but promoted queued packets are never sent | Cut 5, as above. Today a write not acked within 500 ms, or any payload over 32 packets, is reported published | +| `cultnet-ts/src/operation-service.ts` | the operation response | none (the response is the proof) | +| C# `FlushReliable` callers: Gjallar `Program.cs:3055-3080` (two ordered frames, flush, dispose) | flush | None | +| Heimdall `src/idunn-rudp-health.ts:55-87` (vendored CultLib `b6b1d9c`, Heimdall `d5acc94`) | the first Ack packet | Heimdall's own cut at its bump: call cultnet-ts's publisher after Cut 5 | +| CodexConnector `idunn_health.rs:727-745`, Ghostlight `idunn_health.rs:1126-1145` | `Acknowledged` | None required. Optional `wait_acknowledged` at their next bump | +| Muninn `main.rs:9446-9458`, Sleipnir `main.rs:880-904` | nothing (send and drop) | Follow-up: fire-and-forget health can be silently lost | +| Kotlin `flushReliable` | no callers | Contract text only | ### 5. The hold buffer and the 4 MiB cap -- **It belongs here**, because the held frame is exactly the state this map - names. -- **Mechanism (P4).** Rust caps held ordered frames at 1,024 frames and 4 MiB - (`rudp.rs:47-48`, checked at `:1169-1175`). A refusal ends the session. The - default payload limit is 16 MiB (`rudp.rs:33`). With one lost packet under a - large ordered transfer, a legal 5 MiB frame is refused and kills the session. - - The sender's window counts pending packets, not span or bytes - (`rudp.rs:898-930`). While one packet stays lost, acknowledged packets keep - freeing slots, so the held data behind the gap grows without limit. -- **Fix: a sender-side flow window, TCP's receive window without a wire - field** (Cut 3). A reliable packet is admitted to the wire only while: - - its sequence is at most 1,023 above the lowest unacknowledged sequence; - and - - the payload bytes of sequences above the lowest unacknowledged one stay - within 4 MiB, with at least one packet always admissible. +- **It belongs here**, because the held frame is exactly the state this map names. +- **Mechanism (P4).** Rust caps held ordered frames at 1,024 frames and 4 MiB (`rudp.rs:47-48`, checked + at `:1181-1187`). A refusal ends the session. The default payload limit is 16 MiB (`rudp.rs:33`). With + one lost packet under a large ordered transfer, a legal 5 MiB frame is refused and kills the session. + - The sender's window counts pending packets, not span or bytes (`rudp.rs:910-942`). While one packet + stays lost, acked packets keep freeing slots, so the held data behind the gap grows without limit. +- **Fix: a sender-side flow window, TCP's receive window without a wire field** (Cut 3). A reliable + packet is admitted to the wire only while: + - its sequence is at most 1,023 above the lowest unacked sequence; and + - the payload bytes of sequences above the lowest unacked one stay within 4 MiB, with at least one + packet always admissible. - Otherwise it waits in the existing queue. - **What this buys:** - - a conforming sender never makes a Rust receiver hold more than 4 MiB, so - the cap stops refusing legal traffic; - - a conforming sender never gets 1,024 sequences ahead of the receiver's - highest, so the Rust far-ahead refusal (`rudp.rs:46`) never drops its - packets; - - a conforming sender never leaves a sequence 4,096 behind the receiver's - highest, so P8 cannot happen. The receive-history window stays 4,096 in - every runtime (`CultNetTransport.cs:884`, `rudp.rs:35`, `rudp.ts:20`, - `transport.py:404`, `CultMesh.kt:2625`). -- **What is kept.** The cap and the far-ahead refusal stay. They protect - against hostile peers. -- **Rejected:** - - raising the cap to the payload limit or above: that is 16-64 MiB per - hostile session instead of 4; - - lowering the ordered payload limit to 4 MiB: that is an API regression for - large snapshots. -- **Cost.** Normal flow never has a span beyond the 32-packet window, so the - new limits bind only while a packet is lost. -- **Old senders** (the pinned and vendored ones) can still trip the Rust cap - until they bump. -- **Not in scope.** C#, TS, Python and Kotlin receivers have no hold cap at - all. They hold without limit for a hostile peer. Follow-up. + - a conforming sender never makes a Rust receiver hold more than 4 MiB, so the cap stops refusing legal + traffic; + - a conforming sender never gets 1,024 sequences ahead of the receiver's highest, so the Rust far-ahead + refusal (`rudp.rs:46`, checked at `:626-630`) never drops its packets; + - a conforming sender never leaves a sequence 4,096 behind the receiver's highest, so P8 cannot happen. + The receive-history window stays 4,096 in every runtime (`CultNetTransport.cs:884`, `rudp.rs:35`, + `rudp.ts:20`, `transport.py:404`, `CultMesh.kt:2625`). +- **What is kept.** The cap and the far-ahead refusal stay. They protect against hostile peers. +- **Rejected:** raising the cap to the payload limit or above (16-64 MiB per hostile session instead of + 4); lowering the ordered payload limit to 4 MiB (an API regression for large snapshots). +- **Cost.** Normal flow never has a span beyond the 32-packet window, so the new limits bind only while a + packet is lost. +- **Old senders** (the pinned and vendored ones) can still trip the Rust cap until they bump. +- **Not in scope.** C#, TS, Python and Kotlin receivers have no hold cap. Follow-up. ### 6. Reliable expiry (P2) -- Only Rust has it (`rudp.rs:39`, `:886-896`). An expired packet's sequence - never reaches the receiver. `R` stops below it, and so did the old `next` - pointer. Ordered frames after it are held for the rest of the session. -- **Today's users.** Only Muninn's reliable `audio` - (`Muninn main.rs:3230-3241`), and its session carries no ordered traffic. - `hands/media-fec-cut3` deletes the `audio` profile entry and the `"audio"` - send arm. The `media` channel keeps a caller-chosen reliable-with-expiry - mode. +- Only Rust has it (`rudp.rs:39`, `:898-908`). An expired packet's sequence never reaches the receiver. + `R` stops below it, and so did the old `next` pointer. Ordered frames after it are held for the rest of + the session. +- **Today's users.** Only Muninn's reliable `audio`, and its session carries no ordered traffic. + `hands/media-fec-cut3` (unmerged, `4b85a173`) deletes the `audio` profile entry and the `"audio"` send + arm. The `media` channel keeps a caller-chosen reliable-with-expiry mode. - **This map makes it honest (Cut 4):** - - the sender records its first abandoned sequence; + - the sender records its generation's first abandoned sequence; - `A` never passes it; - - ordered receipts after it stay `Received`; + - ordered receipts after it stay `Pending`, and become `Invalidated` when the generation ends; - `flush_reliable` fails at once. -- **The real fix is a fork (Q-A3).** The literature's answer is PR-SCTP's - FORWARD-TSN (RFC 3758): the sender tells the receiver to move its cumulative - point past abandoned sequences. That is a wire addition, and every runtime's - receiver would have to learn it. +- **The real fix is a fork (Q-A3).** The literature's answer is PR-SCTP's FORWARD-TSN (RFC 3758): the + sender tells the receiver to move its cumulative point past abandoned sequences. -### 7. Interaction with the stray-packet branch +### 7. Interaction with the merged stray-packet work -The ack cuts build on these branch behaviours and do not alter them: +The ack cuts build on these behaviours and do not alter them: -- An ended session fails its flush (`CultNetTransport.cs:1924-1931`, - `rudp.rs:1987-2011`, `rudp.ts:906-916`, `transport.py:1101-1124`). -- A refusing session ends, and its goodbye is built after the reset - (`EndRefused`, `end_refused_session` `rudp.rs:358`). The refused sequence is - never acknowledged, so a refusal never becomes a false Received. -- Rust keeps issued sequences on reset. Cut 4's frozen epoch snapshot relies - on that: an old receipt's sequences can never collide with a new epoch's. -- One Accept per session in C#, TS and Python, and `AnswerRepeatedConnect`. A - repeated Connect's sequence is at most `R` and counts as a duplicate. -- The TS operation service expiry and handler rejection are above the - transport, and do not change. -- One thing the branch left: the Rust socket transport in server mode still - calls `reset_peer_state()` on every Connect (`rudp.rs:2099`), even a repeated - one. Under Cut 4 that rotates the epoch. That makes it visible, but it is the - branch's parity item, not this map's. It is recorded as a follow-up. +- A refusing session ends, and its goodbye is built after the reset (`EndRefused` + `CultNetTransport.cs:1315-1319`, `end_refused_session` `rudp.rs:364-367`). The refused sequence is never + acked, so a refusal never becomes a false Acknowledged. +- Issued sequences survive a reset in every runtime (`rudp.rs:341-343`, `CultNetTransport.cs:1292-1308`, + `rudp.ts:214-227`, `transport.py:458-471`). Cut 1c relies on it: a client's second Connect always + carries a new sequence. +- One Accept per generation, and a repeated Connect answered with the Accept still owed, or an Ack. Cut 1c + keeps the answer and moves the decision of what repeats into the session. +- An ended session fails its flush. Cut 1b generalises the TS and Rust binding to the generation into all + four runtimes. +- `5f1e883d` made a peer's Disconnect end Rust receipts (`rudp.rs:596-615`) and removed the + transport-level Pending→Invalidated override. Cut 1b makes the other ending paths agree with it. +- The TS operation service's expiry and handler rejection are above the transport and do not change. + +### 8. Session generation: why it is two cuts, and how a new one is recognised + +- **Why not inside Cut 1.** Cut 1 is a receiver change with a deletion line: one watermark replaces five + per-channel state machines. Its falsifiers are P1, P1b and P6. The generation work touches the ending + paths, the handshake, flush and receipts, and its falsifiers are G1-G3. Both touch `ResetPeerState` and + the Accept path, which is a merge cost, not a shared owner. Keeping them apart lets Soul falsify each + against its own probes. +- **Why two cuts.** The ending rule (Cut 1b) is ruled and needs no mechanism choice: every ending path + drops what it owed. How a new generation is recognised (Cut 1c) needs one (Q-A6). The two are separable + because 1b does not forget peer state: the peer may not know the session ended, and forgetting what was + received would let its retransmits be delivered twice. +- **What must hold after both.** A write is Acknowledged or Invalidated within the generation it was + issued in. A restarted client is admitted. A new server session's frames are delivered after a + reconnect. A retransmitted Connect is still a repeat. +- **Recognising a new generation (Cut 1c, Q-A6 recommendation A).** The Connect's sequence identifies the + generation, and the initial sequence is random by default, as TCP's initial sequence number is + (RFC 9293 §3.4.1, RFC 6528). + - A Connect repeats exactly when it carries the sequence of the Connect that started the current + generation. Its retransmits do; a client's second `connect()` cannot, because issued sequences + survive resets; a restarted process almost never does, because it drew a fresh random initial + sequence. + - A stale duplicate of the accepted Connect, delayed in the network, still repeats: same sequence. + - Anything else ends the current generation and starts a new one, at both ends: the client forgets peer + state when it sends a Connect on a session that had a peer, and the server when it accepts a Connect + that does not repeat. + - A client honours an Accept only while its Connect is pending, and only if the Accept's ack field names + that Connect. A stale Accept from an earlier generation cannot seed `R`. + - The initial sequence is drawn from [1, 2^31), which leaves at least 2^31 sequences before the + exhaustion error (`rudp.rs:944-962`) in every runtime. An explicit `initial_sequence` still works, for + tests. + - The Connect payload stays application evidence (the provider layer's `clientSessionId`, + `cultnet-rs/src/provider_session.rs:70-79`), carried to the application as today. It no longer decides + anything in the transport. ## Cuts -Every cut is on CultLib, branch `hands/rudp-delivered` from `main` after -`hands/rudp-stray-packets` merges, in a worktree Self creates. Heavy -verification runs on Yggdrasil. Each cut starts by committing its probes as -failing tests named for the rule, in every runtime it touches. +Every cut is on CultLib, branch `hands/rudp-delivered` from `main` at `3bf1c0c` or later, in a worktree +Self creates. Heavy verification runs on Yggdrasil. Each cut starts by committing its probes as failing +tests named for the rule, in every runtime it touches. The pass-1 and pass-2 probe files +(`refs/verify/1519a20a` on the `CultLib-imag-ack2` mirror) are the starting point; they print, so each +must become an asserting test. + +Kotlin: every cut below is mapped for C#, Rust, TS and Python. If Q-A5 is A, a trailing Cut K applies +Cuts 1, 1b, 1c and 3 to `CultMesh.kt` at the anchors given, on the verification host Q-A5 picks. **Negative checks that stay green in every cut:** - the cross-runtime reorder, drop and fragment proofs in the TS interop harness (`docs/cultnet-transport-parity.md:227-248`); -- the stray-packet tests (`rudp_stray_packets.rs`, and their C#, TS and Python - equivalents); +- the stray-packet tests (`packages/cultnet-rs/tests/rudp_stray_packets.rs`, and their C#, TS and Python + equivalents), except the ones each cut names as encoding the old rule; - the codec fixture vectors: no wire byte changes. -### Cut 1. One watermark owns ordered delivery (receiver, all five runtimes) +### Cut 1. One watermark owns ordered delivery (receiver) — ready -- **First.** Failing tests P1, P1b and P6 in each runtime. Rust's are in - `probe_ack.rs` at `db7d2886`, and the others are in the probe commits above. +- **First.** Failing tests P1, P1b and P6 in each runtime. - **Deletes first:** - - the per-channel `next` map and its resets: - - `CultNetTransport.cs:918`, `:1292-1310`; - - `rudp.rs:194`, `:221`, `:348`; - - `rudp.ts:160`, `:214-225`; - - `transport.py:424`, `:458-469`; - - `CultMesh.kt:2643`; - - `expectedSequenceIfUninitialized`: - `CultNetTransport.cs:1151-1153`, `rudp.rs:534-537`, `rudp.ts:375-377`, - `transport.py:595-599`, `CultMesh.kt:2728`; - - `DeliverOrdered`, `DrainOrdered` and `SkipReceivedNonChannelSequences` - (the deletion line above); - - `PendingOrderedFrame.NextSequence`. + - the per-channel `next` map and its resets: `CultNetTransport.cs:918`, `:1305`; `rudp.rs:196`, `:223`, + `:354`; `rudp.ts:160`, `:223`; `transport.py:424`, `:468`; `CultMesh.kt:2643`; + - `expectedSequenceIfUninitialized`: `CultNetTransport.cs:1151-1153`, `rudp.rs:540-543` (used at + `:679`), `rudp.ts:375-377` (used at `:450`), `transport.py:595-599`, `CultMesh.kt:2728`; + - `DeliverOrdered`, `DrainOrdered` and `SkipReceivedNonChannelSequences`: `CultNetTransport.cs:1628-1700`, + `rudp.rs:1138-1246`, `rudp.ts:737-800`, `transport.py:868-920`, `CultMesh.kt:2939-2978`; + - `PendingOrderedFrame.NextSequence` (`rudp.rs:160-163` and equivalents). - **Adds:** - - `R`, seeded in `AcceptConnect` (`CultNetTransport.cs:1012`, `rudp.rs:377`, - `rudp.ts:242`, `transport.py:485`, `CultMesh.kt:2667`) and on Accept - receipt (`CultNetTransport.cs:1155`, `rudp.rs:539`, `rudp.ts:379`, - `transport.py:601`, `CultMesh.kt:2730`); - - `R` advances in `RememberReceived` (`CultNetTransport.cs:1518`, - `rudp.rs:961`, `rudp.ts:619`, `transport.py:809`, `CultMesh.kt:2891`); - - a reassembled ordered frame is delivered at once when its first sequence - is at most `R + 1` and its channel holds nothing below it. Otherwise it is - held. The caps apply only to what is held: the prototype held everything - and so refused a 5 MiB frame with no gap; - - after every reliable receipt (data, fragment, duplicate, Accept), drain - every channel's held frames whose first sequence is at most `R + 1`, in - sequence order; - - Rust only: the 64-channel cap (`rudp.rs:42`) counts channels with held - frames. -- **Tests changed:** `tests/cultnet.rs:1000` (consistent handshake) and - `:2655` (hold frames behind a gap). The equivalents in the other runtimes are - whatever the change breaks. Each must be read before it is edited, because a - test that encodes `min(highest+1, frame)` encodes P6. + - `R`, seeded in `AcceptConnect` (`CultNetTransport.cs:1012`, `rudp.rs:383`, `rudp.ts:242`, + `transport.py:485`, `CultMesh.kt:2667`) and on Accept receipt (`CultNetTransport.cs:1155`, + `rudp.rs:544`, `rudp.ts:379`, `transport.py:601`, `CultMesh.kt:2730`); + - `R` advances in `RememberReceived` (`CultNetTransport.cs:1518`, `rudp.rs:973`, `rudp.ts:619`, + `transport.py:809`, `CultMesh.kt:2891`) and resets in `ResetPeerState`; + - a reassembled ordered frame is delivered at once when its first sequence is at most `R + 1` and its + channel holds nothing below it. Otherwise it is held. The caps apply only to what is held: the + prototype held everything and so refused a 5 MiB frame with no gap; + - after every reliable receipt (data, fragment, duplicate, Accept), drain every channel's held frames + whose first sequence is at most `R + 1`, in sequence order; + - Rust only: the 64-channel cap (`rudp.rs:42`) counts channels with held frames. +- **Tests changed:** `packages/cultnet-rs/tests/cultnet.rs:1000` (consistent handshake) and `:2655` + (hold frames behind a gap). The equivalents in the other runtimes are whatever the change breaks. Each + must be read before it is edited, because a test that encodes `min(highest+1, frame)` encodes P6. - **Verification:** - P1, P1b and P6 pass in every runtime; - - the full suites: `cargo test` in `cultnet-rs` and `cultmesh-rs`, `dotnet - test tests/GameCult.Networking.Tests`, `npm test` in `cultnet-ts`, the - Python suite, and Kotlin per Q-A5; + - the full suites: `cargo test` in `cultnet-rs` and `cultmesh-rs`, `dotnet test + tests/GameCult.Networking.Tests`, `npm test` in `cultnet-ts`, the Python suite; - the TS interop harness across runtimes; - negative grep: `rg -n "orderedNextSequence|ordered_next_sequence|expectedSequenceIfUninitialized|expected_sequence_if_uninitialized|SkipReceivedNonChannel|skip_received_non_channel" src packages` returns nothing; - - mutation testing on the diff: cargo-mutants, Stryker.NET, StrykerJS, and - mutmut for Python. -- **Ledger.** Per runtime about −70 and +40. Across five, about −350 and +200. + - mutation testing on the diff: cargo-mutants, Stryker.NET, StrykerJS, and mutmut for Python. +- **Ledger.** Per runtime about −70 and +40. Across four, about −280 and +160. -### Cut 2. A reliable fragment set is never evicted (receiver: Rust, TS, Python) +### Cut 1b. A write belongs to the session it was issued in: every ending drops what it owed — ready + +- **First.** Failing tests G3 (each ending path: peer Disconnect, local disconnect, timeout, refusal) in + C#, TS and Python, and local disconnect and timeout in Rust; G3R in Rust; and in every runtime, a flush + started before a timeout or disconnect fails even after a later `connect()` succeeds. +- **The owner.** Each session gets a generation counter and one private primitive that ends the current + generation: not connected; pending and queued dropped; generation advanced. In Rust it also invalidates + the generation's receipts (by the scope today; Cut 4 moves that into the receipts). It does not touch + what was received from the peer (§8). +- **Deletes first:** + - Rust: the inline clearing in the Disconnect branch (`rudp.rs:596-615`) and the scope rotation in + `reset_peer_state` (`:345`), both replaced by the primitive; the receipt doc that says local endings + carry writes (`rudp.rs:128-133`); the flush's scope capture (`:2001-2014`), replaced by the generation; + - TS: the transport's `#generation`, `#endedGeneration`, `#endGeneration` (`rudp.ts:826-831`, + `:937-940`) and their uses in `connect` (`:895`), `flush` (`:913-920`), `checkTimeout` (`:949`), + `#endRefusedSession` (`:976`), the server Connect path (`:1037-1046`) and the disconnect result + (`:1084`). `#endedReason` stays, as the report of why; + - C# and Python: flush's reliance on the disconnect reason (`CultNetTransport.cs:1924-1929`, + `transport.py:1110-1116`). The reason stays, as a report. +- **Adds:** + - the primitive, called from every ending path: + - peer Disconnect: `CultNetTransport.cs:1190-1199`, `rudp.rs:596-615`, `rudp.ts:406-414`, + `transport.py:619-625`; + - local disconnect: `CreateDisconnect` `CultNetTransport.cs:1324-1328`, `create_disconnect` + `rudp.rs:771-781`, `createDisconnect` `rudp.ts:508-515`, `create_disconnect` `transport.py:695-697`; + - timeout, when it fires: `CultNetTransport.cs:1333-1348`, `rudp.rs:783-796`, `rudp.ts:517-526`, + `transport.py:699-706`; + - refusal, through `ResetPeerState` (`CultNetTransport.cs:1292-1308`, `rudp.rs:344-358`, + `rudp.ts:214-227`, `transport.py:458-471`); + - a read-only generation on each session, which each transport's flush captures at start and fails on + when it changes (`CultNetTransport.cs:1914-1945`, `rudp.rs:1992-2040`, `rudp.ts:911-935`, + `transport.py:1101-1124`). +- **Hubs.** The Rust hub and the C# listener already discard a peer's session at its end, so its owed + writes die with it. No change. +- **Tests changed:** `tests/rudp_stray_packets.rs:638-690` keeps passing unchanged (peer Disconnect already + obeys the rule). Tests that expect a write to survive a local disconnect or timeout encode the old rule; + read each before editing. +- **Verification:** + - G3 in every runtime: after every ending path, owed 0 and nothing delivered to the next session; + - G3R: the receipt reads `Invalidated` after `disconnect()`, and stays so after reconnecting; + - the flush test above in every runtime; + - negative: `rg -n "endedGeneration|#endGeneration" packages/cultnet-ts/src` returns nothing; a TS + reconnect from inside a `disconnect` listener still cannot make an ended flush succeed (the case the + TS generation pair was written for, `rudp.ts:826-829`); + - mutation testing on the diff. +- **Ledger.** About −40 and +45 across four runtimes. TS is net negative. + +### Cut 1c. A Connect's sequence says whether it repeats — blocked on Q-A6 + +Mapped for Q-A6's recommended answer A. Blocked on Cut 1b (it uses the ending primitive). + +- **First.** Failing tests G1 (the hub, the C# listener, and server mode in every runtime: a restarted + client on the same address and payload is admitted and its frames delivered), G1b (C# and Python server + mode admit a Connect from a new endpoint), G2 (after `connect()` on the same session, a new server + session's frames are delivered), and a stale Accept (its ack field does not name the pending Connect) is + ignored. +- **Deletes first:** + - the transport-level repeat decisions: Rust server mode's reset on every Connect + (`rudp.rs:2105-2116`); the Rust hub's payload comparison (`rudp.rs:1637-1647`); C# server mode's + `_session.Connected` branch (`CultNetTransport.cs:2226-2236`); the C# listener's payload comparison + (`CultNetTransport.cs:2670-2688`); TS's `movedEndpoint`/`repeated` computation (`rudp.ts:1011-1049`) + and the session's own `connected` shortcut (`rudp.ts:249-251`); Python's `connected` branch + (`transport.py:1047-1053`); + - C# and Python server mode's refusal of a Connect from a new endpoint (`CultNetTransport.cs:2212-2216`, + `transport.py:1042-1044`), for Connect packets only. Rust and TS already admit it. +- **Adds:** + - the session records the sequence of the Connect that started its generation, and decides: a Connect + repeats exactly when it is connected and the Connect carries that sequence. `AcceptConnect` answers a + repeat as `AnswerRepeatedConnect` does today, and otherwise ends the generation, resets peer state and + accepts (`CultNetTransport.cs:1012-1059`, `rudp.rs:383-435`, `rudp.ts:242-290`, + `transport.py:485-525`). `AnswerRepeatedConnect` stops being public; a public predicate + (`ConnectRepeats(packet)`) serves the hubs, which keep building a fresh session for a new generation and + emit Disconnected for the old one (`rudp.rs:1648-1690`, `CultNetTransport.cs:2690-2710`); + - `CreateConnect` on a session that has had a peer ends the generation and resets peer state first + (`CultNetTransport.cs:1001-1010`, `rudp.rs:369-381`, `rudp.ts:229-240`, `transport.py:473-483`); + - on Accept receipt, the session honours it only while its Connect is pending and only if the Accept's + ack field names that Connect (`CultNetTransport.cs:1155-1160`, `rudp.rs:544-557`, `rudp.ts:379-390`, + `transport.py:601-604`); + - the default initial sequence is random in [1, 2^31): Rust `CultNetRudpSessionOptions::default` + (`rudp.rs:104-113`), the transport constructors (`:1311`, `:1332`) and hub options (`:1377`); C# + `CultNetTransport.cs:773`, `:850`, `:2404`; TS `rudp.ts:172`; Python `transport.py:298`, `:323`; + - CultLib's own fixed `initial_sequence: 1` call sites take the default: `cultmesh-rs/src/lib.rs:964`, + `:1409`, `cultmesh-rs/src/rudp_document_server.rs:432`, `cultnet-rs/src/cultmesh.rs:259`, + `cultnet-ts/src/signed-daemon-health.ts:116`. +- **Tests changed** (they encode payload identity or a fixed initial sequence): + - `packages/cultnet-rs/tests/rudp_server_hub.rs:171-240` calls `connect()` again with the same payload + and expects no replacement. Under this rule that is a new generation. Its retransmit half is already + covered by `rudp_stray_packets.rs:487` (the same Connect packet twenty times), which stays; + - `tests/GameCult.Networking.Tests/NetworkingTests.cs:1507` replaces a peer by payload with two sessions + at the default initial sequence; it passes under a random default, but must say it relies on distinct + sequences; + - `packages/cultnet-ts/test/cultnet.test.ts:2225` (another endpoint replaces the peer) stays; + - any test that asserts a sequence value with the default initial sequence. +- **Verification:** + - G1, G1b, G2 and the stale-Accept test in every runtime; + - the retransmit tests stay green: `rudp_stray_packets.rs:487`, `:511`, `NetworkingTests.cs:1483`, + `cultnet.test.ts:2119`, `:2134`, `test_cultnet.py:1134`; + - negative grep: `rg -n "SequenceEqual\(connectPayload\)|connect_payload == packet.payload|movedEndpoint" src packages` + returns nothing; + - mutation testing on the diff. +- **Ledger.** About −60 and +70 across four runtimes. Ten call sites lose an explicit initial sequence. + +### Cut 2. A reliable fragment set is never evicted (receiver: Rust, TS, Python) — ready - **First.** Failing test P5 in each of the three. - **Change:** - - eviction picks the stalest set that holds no reliable fragment - (`rudp.rs:1015-1035`, `rudp.ts:679-692`, `transport.py:836-842`); - - if every pending set holds reliable fragments, the packet is refused and - the session ends, as every other refusal does. - - A conforming sender cannot reach that bound: its window keeps at most 32 - reliable packets in flight, and the bound is 64 sets. -- **Verification:** P5 delivers the reliable frame. `d3bc6e66`'s test (lossy - fragment sets keep being evicted) stays green. `fragment_sets_evicted` still - counts evictions. + - eviction picks the stalest set that holds no reliable fragment (`rudp.rs:1027-1047`, + `rudp.ts:679-692`, `transport.py:836-842`); + - if every pending set holds reliable fragments, the packet is refused and the session ends, as every + other refusal does; + - a conforming sender cannot reach that bound: its window keeps at most 32 reliable packets in flight, + and the bound is 64 sets. +- **Verification:** P5 delivers the reliable frame. `d3bc6e66`'s test (lossy fragment sets keep being + evicted) stays green. `fragment_sets_evicted` still counts evictions. - **Ledger.** About −3 and +12 per runtime. -### Cut 3. The sender's flow window: 1,023 sequences and 4 MiB above the lowest unacknowledged (sender, all five runtimes) +### Cut 3. The sender's flow window: 1,023 sequences and 4 MiB above the lowest unacked — ready -- **First.** Failing tests P8 (all runtimes) and P4 (Rust receiver, with a - sender from each runtime). +- **First.** Failing tests P8 (all runtimes) and P4 (Rust receiver, with a sender from each runtime). - **Change.** Admission (`AdmitReliablePackets` / `PromoteQueuedReliable` at - `CultNetTransport.cs:1439-1490`, `rudp.rs:898-930`, `rudp.ts:586-600`, - `transport.py:776-796`, `CultMesh.kt:2856-2872`) admits a packet only while: + `CultNetTransport.cs:1439-1490`, `rudp.rs:910-942`, `rudp.ts:586-600`, `transport.py:776-796`, + `CultMesh.kt:2856-2872`) admits a packet only while: - the window has a slot (as today); - - its sequence is at most the lowest unacknowledged sequence plus 1,023; - - the admitted payload bytes above the lowest unacknowledged sequence stay - within 4 MiB. - - The first packet is always admissible. Expired packets (Rust) do not count - as unacknowledged here: they are gone from the wire. -- **Contract.** `docs/cultnet-transport-parity.md` (after `:263`) states the - receive window: a receiver must hold 1,023 sequences and 4 MiB beyond its - contiguous watermark, and a conforming sender never exceeds that. + - its sequence is at most the lowest unacked sequence plus 1,023; + - the admitted payload bytes above the lowest unacked sequence stay within 4 MiB. + - The first packet is always admissible. Expired packets (Rust) do not count as unacked here: they are + gone from the wire. +- **Contract.** `docs/cultnet-transport-parity.md` (after `:263`) states the receive window: a receiver + must hold 1,023 sequences and 4 MiB beyond its contiguous watermark, and a conforming sender never + exceeds that. - **Verification:** - P8: `g` is delivered; - - P4: the 5 MiB frame is delivered after the gap fills, and the session is - never refused; - - the existing large-snapshot tests (`c398bad1`) and the fragment interop - stay green; - - the Rust benchmark or media probe shows no throughput change on a lossless - link. + - P4: the 5 MiB frame is delivered after the gap fills, and the session is never refused; + - the existing large-snapshot tests (`c398bad1`) and the fragment interop stay green; + - the Rust benchmark or media probe shows no throughput change on a lossless link. - **Ledger.** About +25 per runtime, and nothing removed. -### Cut 4. Rust receipts: Received and Delivered, honest expiry, honest queueing (Rust, cultmesh-rs) +### Cut 4. Rust receipts: Acknowledged means delivered, resolved once; honest expiry and queueing — blocked on Cuts 1 and 1b, and on Q-A3 -- **First.** Failing tests: P3b (Received, then Delivered), P9, P10, P2 - (`flush_reliable` must fail), and the frozen-epoch case (a receipt at - `Received` stays `Received` across `end_refused_session`). +- **First.** Failing tests: P3 (the receipt for `s2` reads `Pending` until `s1` arrives, then + `Acknowledged`), P9, P10, P2 (`flush_reliable` must fail), and: a receipt `Acknowledged` before any + ending stays `Acknowledged` after the ending and after a reconnect; a `Pending` one becomes + `Invalidated`. - **Deletes first:** - - `Acknowledged` (`rudp.rs:146`); - - the stale comment and the half guard at `rudp.rs:1903-1908`; - - the hand-rolled status loops at `cultmesh-rs/src/lib.rs:806-838` and - `:870-900`. + - the session scope (`rudp.rs:135`, `:179`, `:206`, `:283-285`, the check at `:291-293`, and what + Cut 1b left of the rotations); + - the stale comment and the half guard at `rudp.rs:1915-1920`; + - the hand-rolled status loops at `cultmesh-rs/src/lib.rs:806-838` and `:870-900`. - **Adds:** - - `Received` and `Delivered`; - - `ordered` on the receipt; + - the receipt's status cell and ordered flag; the session's unresolved-receipt list, resolved once + (Acknowledged through `A`, Invalidated by the ending primitive); - `acked_through` (`A`), reading pending, queued and first-abandoned; - - first-abandoned, recorded in `purge_expired_reliable` (`rudp.rs:886`); - - the receipt covers queued sequences: `send_many` (`:450`) reports every - sequence it assigned; - - `send_reliable` refuses a channel with expiry; - - the one-epoch frozen snapshot in `reset_peer_state` (`:338`); - - `wait_delivered`; - - `flush_reliable` fails fast after an abandoned sequence precedes an ordered - send. -- **Changes:** - - `cultmesh-rs/src/lib.rs:832` and `:892` use `wait_delivered`; - - the doc at `:869` says Delivered; - - `tests/rudp_stray_packets.rs:309` and `cultmesh-rs/tests/rudp_document_server.rs:408,415` change from `Acknowledged` to `Delivered`. + - first-abandoned, recorded in `purge_expired_reliable` (`rudp.rs:898-908`); + - the receipt covers queued sequences: `send_many` (`:456`) reports every sequence it assigned; + - `send_reliable` (`:1903-1930`) refuses a channel with expiry; + - `wait_acknowledged`; + - `flush_reliable` fails fast when an ordered send follows the first abandoned sequence. +- **Unchanged:** `CultNetRudpReliableSendStatus` (`:146-150`), `reliable_send_status`'s signature + (`:1931`), and every caller of `Acknowledged`: `tests/rudp_stray_packets.rs:309`, + `cultmesh-rs/tests/rudp_document_server.rs:408,415`, CodexConnector, Ghostlight. +- **Changes:** `cultmesh-rs/src/lib.rs:832` and `:892` become `wait_acknowledged`; the doc at `:869` says + delivered to the document server's sink. - **Verification:** - the new tests pass; - - `rg -n "Acknowledged" packages/cultnet-rs packages/cultmesh-rs` has no hit; + - `rg -n "session_scope" packages/cultnet-rs` has no hit; - both suites are green; - cargo-mutants on the diff. -- **Ledger.** About −60 and +110 (of which `wait_delivered` is about 35, and - it replaces two loops of about 30 each). +- **Ledger.** About −90 (two loops of about 30 each, the scope) and +90 (of which `wait_acknowledged` is + about 35). -### Cut 5. CultLib-internal consumers and the contract text (TS, docs) +### Cut 5. CultLib-internal TS publishers and the contract text (TS, docs) — blocked on Cut 3, and on Cut 1b in TS - **Changes:** - - `signed-daemon-health.ts:113-160` and - `idunn-odin-presence-publisher.ts:100-127` send through - `CultNetRudpSocketTransportConnection` and `flush`. Their hand-rolled - session loops are deleted. - - Flush contract text in all five runtimes: "every reliable frame sent has - been delivered to the peer's application". - - `docs/cultnet-transport-parity.md:236-240` (the control-packet guard - paragraph) is replaced by the watermark rule, and `:261-272` gains the - Received/Delivered meanings and the receive window. + - `idunn-odin-presence-publisher.ts:89-215` and `signed-daemon-health.ts:104-162` (with its helpers + `:244-252`, `:273-389`, except the exported `parseEndpoint` `:254-271`) send through + `CultNetRudpSocketTransportConnection`: `connectAndWait`, `send("schema")`, `flush`, `close`. Their + hand-rolled session loops are deleted. The presence publisher keeps rejecting a `cultnet.error.v0` + response, from the transport's `frame` events during the flush. + - `connectAndWait(payload, timeoutMs)` on the TS socket transport, matching C# `ConnectAndWait` + (`CultNetTransport.cs:1859-1880`). It is `cultmesh-ts/src/index.ts:6603-6650` `waitForRudpConnected` moved + down; `createRudpPeer` (`index.ts:6220-6245`) uses it and the cultmesh-ts copy is deleted. + - Flush contract text in all runtimes: "every reliable frame sent in this session has been delivered to + the peer's application". + - `docs/cultnet-transport-parity.md:236-240` (the control-packet guard paragraph) is replaced by the + watermark rule, and `:261-272` gains: Acknowledged means delivered; a write belongs to the session it + was issued in; how a repeated Connect is recognised (after Cut 1c); the receive window (after Cut 3). - **Verification:** - - `runtime-presence-health.test.ts` and `signed-daemon-health.test.ts` stay - green; + - `runtime-presence-health.test.ts` and `signed-daemon-health.test.ts` stay green; - a new test drops the first datagram and still publishes; - - a new test publishes a payload over 32 packets whole. -- **Ledger.** About −100 and +40. + - a new test publishes a payload over 32 packets whole; + - a new test shows the publisher's peer receives a Disconnect. +- **Ledger.** About −270 and +50. -## Subtraction ledger (estimate) +## Subtraction ledger (estimate, four runtimes) | Cut | Removed | Added | Other | |---|---|---|---| -| 1 | ~350 | ~200 | five per-channel state machines become one watermark | +| 1 | ~280 | ~160 | four per-channel state machines become one watermark | +| 1b | ~40 | ~45 | one ending primitive; TS's transport generation pair goes | +| 1c | ~60 | ~70 | four transport-level repeat decisions become one session predicate | | 2 | ~10 | ~35 | none | -| 3 | 0 | ~125 | a documented receive-window contract | -| 4 | ~60 | ~110 | one public method (`wait_delivered`), two enum variants for one | -| 5 | ~100 | ~40 | two hand-rolled RUDP loops gone | +| 3 | 0 | ~100 | a documented receive-window contract | +| 4 | ~90 | ~90 | one public method (`wait_acknowledged`); the session scope goes; no enum change | +| 5 | ~270 | ~50 | two hand-rolled RUDP loops and cultmesh-ts's connect wait go | -- **Source, net:** about −10. Tests are extra: about 5 × 6 rule tests. +- **Source, net:** about −195. Tests are extra: about 4 × 10 rule tests. - **Wire:** zero bytes. -- **Downstream:** each external `Acknowledged` caller needs a one-line change - at its next bump. -- **Prototype check:** the Rust receiver and status prototype is +129/−124 in - `rudp.rs` and includes throwaway public helpers. +- **Downstream:** no caller has to change code at its next bump. Callers that fixed `initial_sequence` + keep a deterministic Connect sequence, and with it the restart lockout, until they drop the value. +- **Kotlin (Q-A5 A):** Cuts 1, 1b, 1c and 3 again, about the same size as one runtime each. ## Operator questions Each question stands alone. -- **Q-A1. Where does Delivered come from?** - - **A.** Derive it at the sender. An ordered send is Delivered when every - reliable sequence up to its last one is Received. A reliable unordered send - is Delivered when Received. Nothing changes on the wire. - - **B.** Add a delivered watermark to the wire. The receiver sets Ack flag - bit `0x10` and puts "delivered through N" in the Ack's unused sequence - field. That is additive: old C#, Rust, TS and Python peers at the branch tip - and at both pins ignore it (P7). Senders use it when it is present, and - fall back to A for old peers. - - **Recommended: A.** - - Once Cuts 1-3 make the receiver correct, B's watermark always equals A's - derivation. - - B is a second authority for one value, plus a capability sniff, plus five - encoder and decoder changes. - - B would only tell the sender more than A if a receiver strands frames, - and that receiver is old, so it does not send B. - - **Depends on it.** Cut 4's shape. B adds a Cut 3b in all five runtimes (+~40 - each) and makes the parity codec fixtures change. -- **Q-A2. One flush, or a "received" flush beside the "delivered" one?** - - **A.** One flush. It means Delivered. - - **B.** Add `flushReceived` beside it. - - **Recommended: A.** After this map, all-Received means all-Delivered, except - behind an abandoned sequence. There a received-flush would report success - on frames that will never be delivered, and plain flush fails fast. No - consumer wants Received alone (§4). Receipts keep both states. - - **Depends on it.** B adds a method in five runtimes and a test each. -- **Q-A3. Reliable expiry next to ordered traffic (P2).** An expired reliable - packet leaves a permanent hole, and every ordered frame after it on the same - session is held until the session ends. - - **A.** Report it honestly (Cut 4: receipts stay Received and flush fails - fast). Document "do not send ordered traffic on a session that uses - reliable expiry". No live consumer does. - - **B.** PR-SCTP's FORWARD-TSN (RFC 3758). On expiry the sender sends an - abandonment notice, which rides an Ack-typed packet with flag `0x10` so - that old peers ignore it (P7). New receivers treat the sequence as received - with no frame, and acknowledge it. The sender resends the notice until it - is acknowledged. That is a new cut in all five receivers. Old receivers - never acknowledge it, so the resend needs its own bound. +**Ruled (2026-09-30), recorded so they are not re-asked:** + +- **Q-A1. Where does delivered come from?** (a): derived at the sender from the cumulative watermark. No + wire change. +- **Q-A2. One flush, or a "received" flush beside it?** One meaning: Acknowledged means delivered. + Receipts are Pending, Acknowledged or Invalidated. Retransmission stops on each packet's own ack. Flush + waits for Acknowledged. + +**Open:** + +- **Q-A3. Reliable expiry next to ordered traffic (P2).** An expired reliable packet leaves a permanent + hole, and every ordered frame after it on the same session is held until the session ends. + - **A.** Report it honestly (Cut 4): ordered receipts after the hole stay `Pending` and become + `Invalidated` when the session ends, and flush fails fast. Document "do not send ordered traffic on a + session that uses reliable expiry". No live consumer does. + - **B.** PR-SCTP's FORWARD-TSN (RFC 3758). On expiry the sender sends an abandonment notice, which rides + an Ack-typed packet with flag `0x10` so that old peers ignore it (P7). New receivers treat the sequence + as received with no frame, and acknowledge it. The sender resends the notice until it is acknowledged. + That is a new cut in every receiver. Old receivers never acknowledge it, so the resend needs its own + bound. - **C.** Refuse ordered sends on a session after its first expiry. - - **D.** Delete reliable expiry. Muninn's `audio` is its only user and Muninn - Q4 retires it. This subtracts, but CultLib would lose partial reliability, - which a reasonable consumer can expect. - - **Recommended: A now.** Record B as the complete answer for when a consumer - mixes the two. - - **Depends on it.** Whether Cut 4 carries `first_abandoned` alone (A) or a - new wire notice (B). + - **D.** Delete reliable expiry. Muninn's `audio` is its only user and `hands/media-fec-cut3` retires it. + This subtracts, but CultLib would lose partial reliability, which a reasonable consumer can expect. + - **Recommended: A now.** Record B as the complete answer for when a consumer mixes the two. + - **Blocks:** Cut 4 only. A keeps Cut 4 as mapped (`first_abandoned` alone). B adds a wire cut in every + receiver before Cut 4. C changes Cut 4's `send_many` guard. D replaces Cut 4's expiry half with a + deletion. Cuts 1, 1b, 1c, 2, 3 and 5 are unaffected. + - **Q-A4. Receipts in C#, TypeScript, Python and Kotlin?** - - Only Rust has per-send receipts. The other four offer flush and an - outstanding count. - - **A.** Port receipts with the same four states and a `WaitDelivered`, in - this campaign. + - Only Rust has per-send receipts. The other four offer flush and an outstanding count. + - **A.** Port receipts with the same three states (Pending, Acknowledged, Invalidated) and a + `WaitAcknowledged`, in this campaign. - **B.** Record it as a parity gap. Flush covers every surveyed consumer. - - **Recommended: B.** No consumer outside Rust waits per send. A reasonable - C# consumer might want it, but it is additive later and does not change the - wire. - - **Depends on it.** A adds a cut of about +80 per runtime. + - **Recommended: B.** No consumer outside Rust waits per send. A reasonable C# consumer might want it, but + it is additive later, changes no wire byte, and after Cut 4 the Rust design (a status cell resolved + once by the session) ports without a second design. + - **Blocks:** nothing mapped. A adds a cut of about +80 per runtime after Cut 4. + - **Q-A5. Is Kotlin in the campaign?** - - `cultmesh-kotlin` carries a fifth RUDP runtime with P1 and P6 (the same code, - `CultMesh.kt:2939-2978`). Its flush has no callers. The stray-packet branch - did not touch it. Its only build path is `build.ps1`, which needs Android - Studio's `kotlinc` and JBR on Windows. Yggdrasil has no Kotlin image in the - verify list. - - **A.** Include it in Cuts 1, 3 and 5, and verify on Starfire as a - Windows-only job, or add a Kotlin image to Yggdrasil first. - - **B.** Record it as parity debt, with the stray-packet gaps. Mark the parity - doc so that nobody reads Kotlin as conforming. - - **Recommended: A, with a Yggdrasil Kotlin image.** Wire parity is the - library's promise. A runtime that strands acknowledged frames should not - stay listed as conforming. - - **Depends on it.** Cut 1 and Cut 3 scope and their verification host. + - `cultmesh-kotlin` carries a fifth RUDP runtime with P1 and P6 (the same code, `CultMesh.kt:2939-2978`). + Its flush has no callers. The stray-packet work did not touch it, so none of those fixes reached it + (not probed in pass 2). Its only build path is + `build.ps1`, which needs Android Studio's `kotlinc` and JBR on Windows. Yggdrasil has no Kotlin image in + the verify list. + - **A.** Include it, as a trailing Cut K that applies Cuts 1, 1b, 1c and 3, verified on a Yggdrasil + Kotlin image added first (or on Starfire as a Windows-only job). + - **B.** Record it as parity debt, with the stray-packet gaps. Mark the parity doc so that nobody reads + Kotlin as conforming. + - **Recommended: A, with a Yggdrasil Kotlin image.** Wire parity is the library's promise. A runtime + that strands acknowledged frames should not stay listed as conforming. + - **Blocks:** Cut K only. Cuts 1, 1b, 1c and 3 proceed in four runtimes either way; pass 1 had Q-A5 + blocking their scope, and pass 2 removes that by making Kotlin trail. -Not forks, recorded so they are not re-asked: +- **Q-A6. How does a session recognise a new generation? (new in pass 2)** Today a server treats a + Connect from a connected endpoint as a repeat by payload (Rust hub, C# listener) or unconditionally (C#, + TS and Python server mode), so a restarted client with the same payload is locked out (G1), and a + client's second `connect()` is not seen as new, so the client cannot safely forget the old server + session's sequences (G2). The Connect's bytes are identical for a retransmit and for a restarted process + that starts at the same initial sequence, so something has to differ. + - **A.** The Connect's sequence identifies the generation, and the default initial sequence is random in + [1, 2^31), as TCP's initial sequence number is (RFC 9293 §3.4.1, RFC 6528). A Connect repeats exactly + when it carries the current generation's Connect sequence. No wire change. The default of a public + option changes (from 1, or 100 for the C# listener); an explicit value still works for tests, and a + caller that fixes it keeps the lockout. + - **B.** Proof of receipt. A byte-identical Connect repeats until the server has seen the client + acknowledge the Accept; after that any Connect is a new generation. No default changes. But a delayed + duplicate of the original Connect that arrives after that ack replaces a live session and forgets what + it received, so its retransmits are delivered twice. + - **C.** The application's Connect payload identifies the generation (today's hub rule), and every + client must put a per-process nonce in it, as the provider layer's `clientSessionId` does. Server mode + adopts the same comparison. This leaves G1 as a documented obligation on every caller, including the + ones that send an empty payload (`cultmesh-rs/src/lib.rs:973`, the TS publishers). + - **Recommended: A.** It is the literature's answer, it is robust to delayed duplicates (they carry the + accepted sequence), and it moves the decision into the session, deleting four transport-level + decisions. + - **Blocks:** Cut 1c only. Cut 1b (the ending rule) is ruled and does not depend on it. -- `Acknowledged` is removed, not aliased. The ruled states replace it. -- Delivered means handed to the transport's application boundary, not consumed - (the ruling). +**Not forks, recorded so they are not re-asked:** + +- `Acknowledged` keeps its name and now means delivered. No variant is added or removed. +- Delivered means handed to the transport's application boundary, not consumed. - The 4 MiB cap stays. The sender's window keeps legal traffic under it (§5). -- The receiver's 4,096 receive history stays. The sender's window keeps legal - traffic inside it. -- A receipt keeps Received across a session end or reset (the ruling). A reset - keeps one frozen epoch. +- The receiver's 4,096 receive history stays. The sender's window keeps legal traffic inside it. +- A receipt is resolved once. `Acknowledged` before a session ends stays `Acknowledged`; anything else + becomes `Invalidated` at the end. +- Session generation is its own two cuts, not part of Cut 1 (§8). ## Sequencing and blockers -1. **`hands/rudp-stray-packets` merges to `main`.** It is in Hands now. Every - cut here is based on that merge. -2. **The Odin CultLib pin bump** follows that merge. It is **not** blocked on - this campaign: Odin crashed on stray packets, and it should not wait. Odin - uses `publish_cultnet_message_to_rudp_catalog`, not the enum, so Cut 4 does - not break its build when it bumps again. -3. **Cut 1** is blocked on step 1 only. -4. **Cut 2 and Cut 3** are blocked on step 1 only. They are independent of Cut 1 - and of each other, and can run in parallel worktrees. Cut 3 and Cut 1 touch - different functions in the same files. Merge Cut 1 first so that Cut 3's P4 - test exercises the new hold path. -5. **Cut 4** is blocked on Cut 1, because Delivered is sound only against a - correct receiver. It is also blocked on Q-A1 and Q-A3. -6. **Cut 5** is blocked on Cut 3 in TypeScript, because the socket transport it - routes through must carry the window. +1. **Done.** `hands/rudp-stray-packets` merged to `main` at `3bf1c0c`. Odin pins `3bf1c0c`. +2. **Ready for Hands now** (base `main` at `3bf1c0c` or later; four runtimes; Kotlin trails per Q-A5): + - **Cut 1** (the watermark). + - **Cut 1b** (the ending rule). Same files as Cut 1 in `ResetPeerState` and the Disconnect branches: + land it after Cut 1 on the same branch, or in parallel and rebase the second. + - **Cut 2** and **Cut 3**, independent of Cut 1 and of each other, in parallel worktrees. Merge Cut 1 + before Cut 3 so that Cut 3's P4 test exercises the new hold path. +3. **Cut 1c** is blocked on Q-A6 and on Cut 1b. +4. **Cut 4** is blocked on Cut 1 (Acknowledged is sound only against a correct receiver), on Cut 1b (it + resolves receipts through the ending primitive), and on Q-A3. +5. **Cut 5** is blocked on Cut 3 in TS (the socket transport it routes through must carry the window) and + on Cut 1b in TS (its flush must be bound to the generation). It does not need Cut 1c, but Cut 5's + parity-doc text for repeated Connects waits for it. +6. **Odin's next pin bump** follows Cuts 1-3 (and 1b). Odin is the receiver for every Idunn health, + presence and catalog publisher, so this bump is what makes their Acknowledged mean delivered. Odin uses + `publish_cultnet_message_to_rudp_catalog` and no receipt type, so Cut 4 never breaks its build. 7. **Parallel campaigns:** - - **`hands/media-fec-cut3`** (Muninn's Cut 3 in CultLib) touches `rudp.rs` - only in the profile's `audio` entry (`~2215`) and the - `"media" | "audio"` arm of `channel_send_options` (`~2380`). Neither - overlaps Cuts 1-4 (the session internals and `send_reliable` - `:1891-1928`). It can merge in either order, and the second to land - rebases trivially. Once it lands, reliable expiry is only for `media` when - the caller chooses Reliable. - - **Muninn's own cuts** use lossy media plus parity. Its reliable `audio` - session has no ordered traffic, so P2 does not bite it. Its next CultLib - pin (the `pins/` commit at Cut 3's CultLib tag, Muninn Q2) picks up - whatever has landed. Nothing here blocks Muninn, and nothing in Muninn - blocks this. - - **CultCache publication Cut 4** (CultNetDatabase on the journal, - `docs/cultcache-publication-cut.md` §4) touches `CultNetDatabase.cs` - only, and needs in-order, exactly-once delivery on `schema`. The C# - socket transports carry one reliable channel (`CultNetTransport.cs:2372-2376`), - so P1 and P6 cannot occur on their sessions. There is no blocking - relation in either direction. -8. **Heimdall and StreamPixels vendored bumps** (`b6b1d9c`, `542ddd3`) come - after Cut 5, so that they pick up honest flushes and the TS window at once. - Heimdall's `idunn-rudp-health.ts` first-ack wait moves to `flush` at that - bump. -9. **CodexConnector (`e171eca`) and Ghostlight (`85f7024`)** change - `Acknowledged` to `Delivered` (or to `wait_delivered`) at their next bump - after Cut 4. + - **`hands/media-fec-cut3`** (unmerged, `4b85a173`) touches `rudp.rs` only in the imports, the + profile's `audio` entry (`~2374` at `3bf1c0c`) and the `"media" | "audio"` arm of + `channel_send_options` (`~2547`). None overlaps Cuts 1-4. Either order; the second rebases trivially. + Once it lands, reliable expiry is only for `media` when the caller chooses Reliable. + - **Muninn's own cuts** use lossy media plus parity. Its reliable `audio` session has no ordered + traffic, so P2 does not bite it. Nothing here blocks Muninn, and nothing in Muninn blocks this. + - **CultCache publication Cut 4** (`docs/cultcache-publication-cut.md` §4) touches `CultNetDatabase.cs` + only, and needs in-order, exactly-once delivery on `schema`. The C# socket transports carry one + reliable channel (`CultNetTransport.cs:2372-2376`), so P1 and P6 cannot occur on their sessions. No + blocking relation in either direction. +8. **Vendored TS bumps come after Cut 5**, so they pick up honest flushes, the TS window and the rewritten + publishers at once: + - **StreamPixels** vendors CultLib at `30ee8b9` (StreamPixels `origin/main`, gitlink bumped in + `44a8945`). Its service runs `createIdunnRuntimePresencePublisher` + (`apps/service/scripts/idunn-entry.mjs`), whose file is byte-identical at `30ee8b9` and `3bf1c0c`; + `30ee8b9` also predates the TS stray-packet hardening in `rudp.ts`. + - **Heimdall** vendors `b6b1d9c` and hand-rolls its own publisher (`src/idunn-rudp-health.ts:55-87`, + first-ack wait). At its bump it calls cultnet-ts's publisher instead. + - Bumping either after Cut 1c as well also gives them the restart fix; neither needs it, because each + publish binds a new port. +9. **CodexConnector** and **Ghostlight** need no code change at their next bump. They may switch their + status loops to `wait_acknowledged` after Cut 4. ## Follow-ups outside this campaign -- **Hold caps in C#, TS, Python and Kotlin.** Their ordered hold buffers and - per-channel maps are unbounded against a hostile peer. Rust has caps. -- **The Rust server-mode socket transport resets on every Connect** - (`rudp.rs:2099`), even a repeated one. This is a stray-packet parity item. -- **The document server's per-session payload budget never shrinks** - (`rudp_document_server.rs:321`, `:584`). After 4 MiB in one session it drops - data without acknowledging it, and the sender retransmits until it gives up. -- **Fire-and-forget health.** Muninn `main.rs:9446`, Sleipnir `main.rs:880-904` - and Heimdall's first-ack wait can lose a health record silently. -- **FORWARD-TSN** if Q-A3 is A and a consumer later mixes expiry with ordered - traffic. -- **The P4 probe at the pins** predates its fix, so it proves nothing there. If - anyone needs old-sender-to-new-receiver evidence for the cap, rerun - `09318d13`'s P4 against a pinned sender. +- **Hold caps in C#, TS, Python and Kotlin.** Their ordered hold buffers and per-channel maps are unbounded + against a hostile peer. Rust has caps. +- **The document server's per-session payload budget never shrinks** (`rudp_document_server.rs:321`, + `:584`). After 4 MiB in one session it drops data without acknowledging it, and the sender retransmits + until it gives up. +- **Fire-and-forget health.** Muninn `main.rs:9446`, Sleipnir `main.rs:880-904` and Heimdall's first-ack + wait can lose a health record silently. +- **FORWARD-TSN** if Q-A3 is A and a consumer later mixes expiry with ordered traffic. +- **Callers outside CultLib that fix `initial_sequence`** keep the restart lockout after Cut 1c until they + drop the value. Record it in their pin-bump notes. +- **The P4 probe at the old pins** predates its fix, so it proves nothing there. If anyone needs + old-sender-to-new-receiver evidence for the cap, rerun `09318d13`'s P4 against a pinned sender. From 7904e15ed39b72b0b5ce79443019e51bdc99ad44 Mon Sep 17 00:00:00 2001 From: Meta Vens Date: Tue, 29 Sep 2026 23:51:02 +0200 Subject: [PATCH 02/16] CultMath: phacelle, Johansen's Phacelle noise on 3D cells with an exact gradient Cut 2a-ii-b. phacelle(float3 p, float3 side, float offset, float normalization) returns CultPhasor { float4 cos; float4 sin }, each float4(gradient.xyz, value.w). - 4x4x4 cells, pcg3d-hashed jitter in [-0.5, 0.5), weight max(0, exp(-2 d^2) - 0.01111), normalization as upstream (lpmitchell/AdvancedTerrainErosion PhacelleNoise). The caller supplies the stripe wave vector; offset is in cycles. - The gradient is exact: weight derivatives and the normalization are included, not upstream's grad(cos) ~ -sin * side. - Cells whose per-axis lower bound is at least 2.25 have weight exactly 0 and are skipped. An internal overload switches the prune off so the test can pin bit equality without a second copy of the loop. - cultmath_phacelle and CultPhasor mirror it in both tracked CultMath.hlsl copies (the same blob). The mirror test compares the struct field by field; it now covers 34 functions. - PhacelleTests: value against an independent double-precision reference, quarter-cycle phase rotation, gradient against central differences on both sides of the normalization threshold (excluding the threshold and each cell's weight-support edge, where the exact gradient is discontinuous), stripes along the flow on the unit sphere (0.215 against the 0.25 bound), step-difference continuity across cell planes, and pruned == unpruned bit for bit. Co-Authored-By: Claude Opus 5.5 --- packages/cultmath/shaders/CultMath.hlsl | 86 ++++++ packages/cultmath/src/CultMath/CultPhasor.cs | 21 ++ .../cultmath/src/CultMath/CultPhasor.cs.meta | 2 + packages/cultmath/src/CultMath/math.cs | 98 +++++++ .../HlslSourceCompatibilityTests.cs | 3 +- .../tests/CultMath.Tests/PhacelleTests.cs | 275 ++++++++++++++++++ .../Shaders/CultMath.hlsl | 86 ++++++ 7 files changed, 570 insertions(+), 1 deletion(-) create mode 100644 packages/cultmath/src/CultMath/CultPhasor.cs create mode 100644 packages/cultmath/src/CultMath/CultPhasor.cs.meta create mode 100644 packages/cultmath/tests/CultMath.Tests/PhacelleTests.cs diff --git a/packages/cultmath/shaders/CultMath.hlsl b/packages/cultmath/shaders/CultMath.hlsl index 50c5a746..d0dd22ae 100644 --- a/packages/cultmath/shaders/CultMath.hlsl +++ b/packages/cultmath/shaders/CultMath.hlsl @@ -503,6 +503,92 @@ CultCellular cultmath_cellular(float3 p) return result; } +// Invariant 8's struct return shape for cultmath_phacelle, compared field by field and bit for bit +// against CultMath.CultPhasor, same field order. +struct CultPhasor +{ + float4 cos; + float4 sin; +}; + +// Johansen's Phacelle noise on 3D cells. See math.phacelle's comment (math.cs) for the stripe wave +// vector, the weight, the normalization, the exact gradient and why the prune is exact. Mirrors +// the C# body operation for operation; the C#-only internal overload that switches the prune off +// has no shader twin. +CultPhasor cultmath_phacelle(float3 p, float3 side, float offset, float normalization) +{ + float3 cell = floor(p); + float3 local = p - cell; + float phaseOffset = offset * CULTMATH_TAU; + float sumW = 0.0; + float sumCos = 0.0; + float sumSin = 0.0; + float3 gradW = float3(0.0, 0.0, 0.0); + float3 gradCos = float3(0.0, 0.0, 0.0); + float3 gradSin = float3(0.0, 0.0, 0.0); + + for (int dz = -1; dz <= 2; dz++) + { + float gz = max(0.0, abs(local.z - (float)dz) - 0.5); + for (int dy = -1; dy <= 2; dy++) + { + float gy = max(0.0, abs(local.y - (float)dy) - 0.5); + for (int dx = -1; dx <= 2; dx++) + { + float gx = max(0.0, abs(local.x - (float)dx) - 0.5); + if (gx * gx + gy * gy + gz * gz >= 2.25) + continue; + + float3 gridStep = float3(dx, dy, dz); + int3 hash = cultmath_pcg3d(int3(cell + gridStep)); + float3 jitter = float3( + ((uint)hash.x >> 8) * (1.0 / 16777216.0) - 0.5, + ((uint)hash.y >> 8) * (1.0 / 16777216.0) - 0.5, + ((uint)hash.z >> 8) * (1.0 / 16777216.0) - 0.5); + float3 v = local - gridStep - jitter; + float falloff = exp(-2.0 * dot(v, v)); + float w = max(0.0, falloff - 0.01111); + float3 dw = w > 0.0 ? (-4.0 * falloff) * v : float3(0.0, 0.0, 0.0); + float phase = dot(v, side) + phaseOffset; + float c = cos(phase); + float s = sin(phase); + + sumW += w; + sumCos += w * c; + sumSin += w * s; + gradW += dw; + gradCos += dw * c - (w * s) * side; + gradSin += dw * s + (w * c) * side; + } + } + } + + float invW = 1.0 / sumW; + float rawCos = sumCos * invW; + float rawSin = sumSin * invW; + float3 gradRawCos = (gradCos - rawCos * gradW) * invW; + float3 gradRawSin = (gradSin - rawSin * gradW) * invW; + float rawLength = sqrt(rawCos * rawCos + rawSin * rawSin); + float floorLength = 1.0 - normalization; + + CultPhasor result; + if (rawLength > floorLength) + { + float outCos = rawCos / rawLength; + float outSin = rawSin / rawLength; + float3 gradLength = (rawCos * gradRawCos + rawSin * gradRawSin) / rawLength; + result.cos = float4((gradRawCos - outCos * gradLength) / rawLength, outCos); + result.sin = float4((gradRawSin - outSin * gradLength) / rawLength, outSin); + } + else + { + result.cos = float4(gradRawCos / floorLength, rawCos / floorLength); + result.sin = float4(gradRawSin / floorLength, rawSin / floorLength); + } + + return result; +} + float cultmath_value_noise(float2 position) { float2 cell = floor(position); diff --git a/packages/cultmath/src/CultMath/CultPhasor.cs b/packages/cultmath/src/CultMath/CultPhasor.cs new file mode 100644 index 00000000..0a28faa8 --- /dev/null +++ b/packages/cultmath/src/CultMath/CultPhasor.cs @@ -0,0 +1,21 @@ +namespace CultMath; + +/// +/// The value-and-gradient result of +/// (invariant 8's struct return shape). Both fields use the float4(gradient.xyz, value.w) +/// layout of a bare value-and-gradient return. +/// +public struct CultPhasor +{ + /// (∇cos, cos): the normalized cosine wave and its gradient. + public float4 cos; + + /// (∇sin, sin): the normalized sine wave and its gradient. + public float4 sin; + + public CultPhasor(float4 cos, float4 sin) + { + this.cos = cos; + this.sin = sin; + } +} diff --git a/packages/cultmath/src/CultMath/CultPhasor.cs.meta b/packages/cultmath/src/CultMath/CultPhasor.cs.meta new file mode 100644 index 00000000..d586aa3f --- /dev/null +++ b/packages/cultmath/src/CultMath/CultPhasor.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 807ae2c8a758cf31896da12f1e5047bd diff --git a/packages/cultmath/src/CultMath/math.cs b/packages/cultmath/src/CultMath/math.cs index 23916b6b..430325ea 100644 --- a/packages/cultmath/src/CultMath/math.cs +++ b/packages/cultmath/src/CultMath/math.cs @@ -968,6 +968,104 @@ public static partial class math // dxc and C#, bit-exact on both sides. internal static float cellular_unit(int bits) => ((uint)bits >> 8) * (1.0f / 16777216.0f); + // Johansen's Phacelle noise (phase + cell), generalised to 3D cells with a caller-supplied + // stripe wave vector. Each of the 4x4x4 cells around p carries one jittered feature point + // (jitter in [-0.5, 0.5) per axis, from pcg3d over the integer cell coordinate, the same + // integer-hashing rule as cellular) and one cosine/sine wave that runs along `side` (the wave + // vector: its length is 2*pi times the stripe frequency, and the caller chooses which + // perpendicular it is; on a unit sphere that is cross(p_hat, flow)). The waves are blended with + // the bell weight max(0, exp(-2 d^2) - 0.01111) (exactly 0 at distance 1.5, the nearest a + // feature outside the 4x4x4 block can be in 2D, which is what keeps cell-grid lines out), and + // the blended phasor is normalized: its length is stretched to 1 wherever the raw length is at + // least 1 - normalization, and scaled by 1 / (1 - normalization) below that. `offset` is the + // phase in cycles (1 is a full turn). Value and normalization follow Johansen's + // PhacelleNoise as ported in lpmitchell/AdvancedTerrainErosion. + // + // The gradient is exact: it carries the weight derivative (grad w = -4 v exp(-2 |v|^2) where + // w > 0) and the normalization, not upstream's convention grad(cos) ~ -sin * side, which + // ignores the weight gradient (Asura probe: 16.8 deg off at the median, 97.8 at the 95th + // percentile; the exact form is 0.00 and 0.03). With P = sum(w * (cos, sin)) and W = sum(w), + // I = P / W has grad I = (grad P - I * grad W) / W; above the threshold the output is I / |I| + // with gradient (grad I - out * grad|I|) / |I|, below it grad I / (1 - normalization). The + // gradient is discontinuous where |I| crosses the threshold, and where a cell's weight leaves + // its support (distance 1.4998): there grad w jumps from -4 * 0.01111 * v (length about 0.067) to 0. The value is + // continuous at both. + // + // Pruning is exact: per axis, |local - g| - 0.5 (floored at 0) lower-bounds |v| for any jitter, + // so a cell whose summed square is at least 2.25 has weight exactly 0 (exp(-4.5) = 0.011109 < + // 0.01111) and contributes nothing to any sum. Only the internal overload can switch the prune + // off; PhacelleTests uses it to pin bit equality. About 14 of 64 cells survive on average. + public static CultPhasor phacelle(float3 p, float3 side, float offset, float normalization) => + phacelle(p, side, offset, normalization, true); + + internal static CultPhasor phacelle(float3 p, float3 side, float offset, float normalization, bool prune) + { + var cell = floor(p); + var local = p - cell; + var phaseOffset = offset * TAU; + var sumW = 0.0f; + var sumCos = 0.0f; + var sumSin = 0.0f; + var gradW = new float3(0.0f, 0.0f, 0.0f); + var gradCos = new float3(0.0f, 0.0f, 0.0f); + var gradSin = new float3(0.0f, 0.0f, 0.0f); + + for (var dz = -1; dz <= 2; dz++) + { + var gz = max(0.0f, abs(local.z - (float)dz) - 0.5f); + for (var dy = -1; dy <= 2; dy++) + { + var gy = max(0.0f, abs(local.y - (float)dy) - 0.5f); + for (var dx = -1; dx <= 2; dx++) + { + var gx = max(0.0f, abs(local.x - (float)dx) - 0.5f); + if (prune && gx * gx + gy * gy + gz * gz >= 2.25f) + continue; + + var gridStep = new float3(dx, dy, dz); + var hash = pcg3d(int3(cell + gridStep)); + var jitter = new float3(cellular_unit(hash.x) - 0.5f, cellular_unit(hash.y) - 0.5f, cellular_unit(hash.z) - 0.5f); + var v = local - gridStep - jitter; + var falloff = exp(-2.0f * dot(v, v)); + var w = max(0.0f, falloff - 0.01111f); + var dw = w > 0.0f ? (-4.0f * falloff) * v : new float3(0.0f, 0.0f, 0.0f); + var phase = dot(v, side) + phaseOffset; + var c = cos(phase); + var s = sin(phase); + + sumW += w; + sumCos += w * c; + sumSin += w * s; + gradW += dw; + gradCos += dw * c - (w * s) * side; + gradSin += dw * s + (w * c) * side; + } + } + } + + var invW = 1.0f / sumW; + var rawCos = sumCos * invW; + var rawSin = sumSin * invW; + var gradRawCos = (gradCos - rawCos * gradW) * invW; + var gradRawSin = (gradSin - rawSin * gradW) * invW; + var rawLength = sqrt(rawCos * rawCos + rawSin * rawSin); + var floorLength = 1.0f - normalization; + + if (rawLength > floorLength) + { + var outCos = rawCos / rawLength; + var outSin = rawSin / rawLength; + var gradLength = (rawCos * gradRawCos + rawSin * gradRawSin) / rawLength; + return new CultPhasor( + new float4((gradRawCos - outCos * gradLength) / rawLength, outCos), + new float4((gradRawSin - outSin * gradLength) / rawLength, outSin)); + } + + return new CultPhasor( + new float4(gradRawCos / floorLength, rawCos / floorLength), + new float4(gradRawSin / floorLength, rawSin / floorLength)); + } + public static float value_noise(float2 position) { var cell = floor(position); diff --git a/packages/cultmath/tests/CultMath.Tests/HlslSourceCompatibilityTests.cs b/packages/cultmath/tests/CultMath.Tests/HlslSourceCompatibilityTests.cs index be8a3df4..ad690759 100644 --- a/packages/cultmath/tests/CultMath.Tests/HlslSourceCompatibilityTests.cs +++ b/packages/cultmath/tests/CultMath.Tests/HlslSourceCompatibilityTests.cs @@ -30,6 +30,7 @@ public sealed class HlslSourceCompatibilityTests Assert.NotNull(ShaderFunction(shader, "cultmath_snoise_grad", typeof(float3))); Assert.NotNull(ShaderFunction(shader, "cultmath_fbm_grad", typeof(float3), typeof(int), typeof(float), typeof(float))); Assert.NotNull(ShaderFunction(shader, "cultmath_ridged_grad", typeof(float3), typeof(int), typeof(float), typeof(float))); + Assert.NotNull(ShaderFunction(shader, "cultmath_phacelle", typeof(float3), typeof(float3), typeof(float), typeof(float))); } // HLSL functions carry no access modifier, so they compile as private instance methods. @@ -130,7 +131,7 @@ public sealed class HlslSourceCompatibilityTests } Assert.True(mismatches.Count == 0, $"{mismatches.Count} mismatches:{Environment.NewLine}{string.Join(Environment.NewLine, mismatches.Take(12))}"); - Assert.Equal(33, compared.Count); + Assert.Equal(34, compared.Count); } /// diff --git a/packages/cultmath/tests/CultMath.Tests/PhacelleTests.cs b/packages/cultmath/tests/CultMath.Tests/PhacelleTests.cs new file mode 100644 index 00000000..15d1a89a --- /dev/null +++ b/packages/cultmath/tests/CultMath.Tests/PhacelleTests.cs @@ -0,0 +1,275 @@ +using System; +using CultMath; +using Xunit; + +namespace CultMath.Tests; + +public sealed class PhacelleTests +{ + // Every tolerance below is at least 5x the largest error the same test measures at its seed. + private const float GradientEps = 1.0e-3f; + private const float GradientTolerance = 5.0e-3f; + private const double ValueTolerance = 1.6e-5; + private const double ThresholdBand = 0.02; + + private static float3 RandomPoint(System.Random random, float extent = 3.0f) => new( + (random.NextSingle() * 2.0f - 1.0f) * extent, (random.NextSingle() * 2.0f - 1.0f) * extent, (random.NextSingle() * 2.0f - 1.0f) * extent); + + // A wave vector of length 2*pi times a stripe frequency in [0.5, 1.5], in a random direction. + private static float3 RandomSide(System.Random random) + { + float3 direction; + do + { + direction = new float3(random.NextSingle() - 0.5f, random.NextSingle() - 0.5f, random.NextSingle() - 0.5f); + } + while (math.length(direction) < 0.1f); + + return math.normalize(direction) * (math.TAU * (0.5f + random.NextSingle())); + } + + // ---- An independent double-precision evaluation: all 64 cells, no prune, no gradient. ---- + // + // It shares only the hash (pcg3d over the integer cell coordinate, top 24 bits) and the constants + // the function is specified by: weight max(0, exp(-2 d^2) - 0.01111), jitter in [-0.5, 0.5). + + private readonly record struct Reference(double Cos, double Sin, double RawLength, double SupportEdgeGap); + + private static Reference Evaluate(float3 p, float3 side, float offset, float normalization) + { + double fx = Math.Floor(p.x), fy = Math.Floor(p.y), fz = Math.Floor(p.z); + double lx = p.x - fx, ly = p.y - fy, lz = p.z - fz; + double sumW = 0.0, sumCos = 0.0, sumSin = 0.0, edgeGap = double.MaxValue; + var supportRadius = Math.Sqrt(Math.Log(1.0 / 0.01111) / 2.0); + for (var dz = -1; dz <= 2; dz++) + for (var dy = -1; dy <= 2; dy++) + for (var dx = -1; dx <= 2; dx++) + { + var hash = math.pcg3d(new int3((int)fx + dx, (int)fy + dy, (int)fz + dz)); + double vx = lx - dx - (math.cellular_unit(hash.x) - 0.5); + double vy = ly - dy - (math.cellular_unit(hash.y) - 0.5); + double vz = lz - dz - (math.cellular_unit(hash.z) - 0.5); + var d2 = vx * vx + vy * vy + vz * vz; + edgeGap = Math.Min(edgeGap, Math.Abs(Math.Sqrt(d2) - supportRadius)); + var w = Math.Max(0.0, Math.Exp(-2.0 * d2) - 0.01111); + var phase = vx * side.x + vy * side.y + vz * side.z + (double)offset * math.TAU; + sumW += w; + sumCos += w * Math.Cos(phase); + sumSin += w * Math.Sin(phase); + } + + double rawCos = sumCos / sumW, rawSin = sumSin / sumW; + var raw = Math.Sqrt(rawCos * rawCos + rawSin * rawSin); + var length = Math.Max(1.0 - normalization, raw); + return new Reference(rawCos / length, rawSin / length, raw, edgeGap); + } + + private static float3 CentralDifference(Func f, float3 p, float eps) => new( + (f(p + new float3(eps, 0.0f, 0.0f)) - f(p - new float3(eps, 0.0f, 0.0f))) / (2.0f * eps), + (f(p + new float3(0.0f, eps, 0.0f)) - f(p - new float3(0.0f, eps, 0.0f))) / (2.0f * eps), + (f(p + new float3(0.0f, 0.0f, eps)) - f(p - new float3(0.0f, 0.0f, eps))) / (2.0f * eps)); + + private static float MaxComponentError(float3 numeric, float4 analytic) => MathF.Max( + MathF.Abs(numeric.x - analytic.x), MathF.Max(MathF.Abs(numeric.y - analytic.y), MathF.Abs(numeric.z - analytic.z))); + + // ---- Value: the function is Johansen's, on 3D cells ---- + + [Theory] + [InlineData(0.0f)] + [InlineData(0.5f)] + [InlineData(0.9f)] + [InlineData(1.0f)] + public void ValueMatchesTheIndependentReferenceAndNormalizesAboveTheThreshold(float normalization) + { + var random = new System.Random(0x7ACE1 + (int)(normalization * 10.0f)); + var maxError = 0.0; + var above = 0; + var below = 0; + for (var i = 0; i < 3000; i++) + { + var p = RandomPoint(random, 40.0f); + var side = RandomSide(random); + var offset = random.NextSingle(); + var actual = math.phacelle(p, side, offset, normalization); + var expected = Evaluate(p, side, offset, normalization); + + maxError = Math.Max(maxError, Math.Max(Math.Abs(actual.cos.w - expected.Cos), Math.Abs(actual.sin.w - expected.Sin))); + if (expected.RawLength >= 1.0 - normalization + ThresholdBand) + { + // The output magnitude is 1 wherever the raw magnitude reaches 1 - normalization. + above++; + var magnitude = Math.Sqrt((double)actual.cos.w * actual.cos.w + (double)actual.sin.w * actual.sin.w); + Assert.True(Math.Abs(magnitude - 1.0) < ValueTolerance, $"magnitude {magnitude} above the threshold at {p}"); + } + else if (expected.RawLength <= 1.0 - normalization - ThresholdBand) + { + below++; + } + } + + Assert.True(maxError < ValueTolerance, $"max value error against the reference {maxError}"); + if (normalization > 0.0f) + Assert.True(above >= 500, $"only {above} points above the threshold"); + if (normalization is > 0.0f and < 1.0f) + Assert.True(below >= 50, $"only {below} points below the threshold"); + } + + [Fact] + public void ValueIsPeriodicInOffsetAndAQuarterCycleRotatesCosIntoSin() + { + // offset is in cycles: a quarter turn maps cos to -sin and sin to cos, and a whole turn is the + // identity. The weights and the normalization do not depend on the phase, so this is exact + // up to float rounding. + var random = new System.Random(0x7ACE2); + for (var i = 0; i < 200; i++) + { + var p = RandomPoint(random); + var side = RandomSide(random); + var offset = random.NextSingle(); + var basePhasor = math.phacelle(p, side, offset, 0.5f); + var quarter = math.phacelle(p, side, offset + 0.25f, 0.5f); + var whole = math.phacelle(p, side, offset + 1.0f, 0.5f); + + Assert.True(MathF.Abs(quarter.cos.w + basePhasor.sin.w) < 2.0e-5f); + Assert.True(MathF.Abs(quarter.sin.w - basePhasor.cos.w) < 2.0e-5f); + Assert.True(MathF.Abs(whole.cos.w - basePhasor.cos.w) < 2.0e-5f); + Assert.True(MathF.Abs(whole.sin.w - basePhasor.sin.w) < 2.0e-5f); + } + } + + // ---- Invariant 8: the gradient is exact ---- + + [Fact] + public void GradientOfCosAndSinMatchesCentralDifferencesOnBothSidesOfTheNormalizationThreshold() + { + // Above the threshold the output is I / |I|; below it, I / (1 - normalization). The gradient + // is discontinuous where |I| crosses the threshold, and at the edge of each cell's weight support + // (distance 1.4998, where max(0, ...) starts: a jump of about 4 d 0.01111 in grad w), so a band + // around each is excluded (by the + // reference's raw length, since the output alone cannot say how close it is). + const float normalization = 0.5f; + var random = new System.Random(0x7ACE3); + var above = 0; + var below = 0; + var maxError = 0.0f; + for (var attempts = 0; attempts < 20000 && (above < 150 || below < 150); attempts++) + { + var p = RandomPoint(random); + var side = RandomSide(random); + var offset = random.NextSingle(); + var reference = Evaluate(p, side, offset, normalization); + var distance = reference.RawLength - (1.0 - normalization); + if (Math.Abs(distance) < ThresholdBand || reference.SupportEdgeGap < 3.0 * GradientEps) + continue; + if (distance > 0.0 ? above >= 150 : below >= 150) + continue; + if (distance > 0.0) above++; else below++; + + var analytic = math.phacelle(p, side, offset, normalization); + var numericCos = CentralDifference(q => math.phacelle(q, side, offset, normalization).cos.w, p, GradientEps); + var numericSin = CentralDifference(q => math.phacelle(q, side, offset, normalization).sin.w, p, GradientEps); + maxError = MathF.Max(maxError, MathF.Max(MaxComponentError(numericCos, analytic.cos), MaxComponentError(numericSin, analytic.sin))); + } + + Assert.True(above >= 150 && below >= 150, $"{above} points above and {below} below the threshold"); + Assert.True(maxError < GradientTolerance, $"max gradient error against central differences {maxError}"); + } + + // ---- Stripes run along the flow ---- + + [Fact] + public void OnTheUnitSphereStripesRunAlongTheFlow() + { + // side = cross(p_hat, flow) * 2*pi is tangent to the sphere and perpendicular to the flow, so the + // cosine should change across the stripes (along side) and hardly at all along them (along + // flow). Asura's probe measured 0.212 for this ratio, against 0.202 for the planar upstream. + const float radiusInCells = 10.0f; + var random = new System.Random(0x7ACE4); + var reference = math.normalize(new float3(0.3f, 0.5f, 0.8f)); + var alongFlow = 0.0; + var acrossFlow = 0.0; + var used = 0; + while (used < 2000) + { + var hat = math.normalize(new float3(random.NextSingle() - 0.5f, random.NextSingle() - 0.5f, random.NextSingle() - 0.5f)); + var tangent = reference - hat * math.dot(hat, reference); + if (math.length(tangent) < 0.2f) + continue; + + var flow = math.normalize(tangent); + var sideDirection = math.cross(hat, flow); + var phasor = math.phacelle(hat * radiusInCells, sideDirection * math.TAU, 0.25f, 0.5f); + var gradient = new float3(phasor.cos.x, phasor.cos.y, phasor.cos.z); + alongFlow += Math.Abs(math.dot(gradient, flow)); + acrossFlow += Math.Abs(math.dot(gradient, sideDirection)); + used++; + } + + var ratio = alongFlow / acrossFlow; + Assert.True(ratio <= 0.25, $"mean|d cos/d flow| / mean|d cos/d side| = {ratio}"); + } + + // ---- Continuity ---- + + [Fact] + public void ValueStepDifferenceShrinksTenfoldPerTenfoldStepIncludingAcrossCellPlanes() + { + // Pairs straddle an integer plane on each axis (where the 4x4x4 window moves by one cell and a + // weight that did not vanish at the window edge would jump) and also start at random points. + // Lipschitz continuity means the largest difference over a step scales with the step. + var random = new System.Random(0x7ACE5); + var steps = new[] { 1.0e-3f, 1.0e-4f, 1.0e-5f }; + var maxDifference = new float[steps.Length]; + for (var i = 0; i < 3000; i++) + { + var p = RandomPoint(random); + var side = RandomSide(random); + var offset = random.NextSingle(); + var axis = i % 3; + if (i % 2 == 0) + p = SetAxis(p, axis, MathF.Round(GetAxis(p, axis))); + for (var k = 0; k < steps.Length; k++) + { + var low = math.phacelle(SetAxis(p, axis, GetAxis(p, axis) - steps[k] * 0.5f), side, offset, 0.5f); + var high = math.phacelle(SetAxis(p, axis, GetAxis(p, axis) + steps[k] * 0.5f), side, offset, 0.5f); + maxDifference[k] = MathF.Max(maxDifference[k], MathF.Max(MathF.Abs(low.cos.w - high.cos.w), MathF.Abs(low.sin.w - high.sin.w))); + } + } + + Assert.True(maxDifference[1] * 8.0f <= maxDifference[0], $"1e-3: {maxDifference[0]}, 1e-4: {maxDifference[1]}"); + Assert.True(maxDifference[2] * 8.0f <= maxDifference[1], $"1e-4: {maxDifference[1]}, 1e-5: {maxDifference[2]}"); + } + + private static float GetAxis(float3 v, int axis) => axis == 0 ? v.x : axis == 1 ? v.y : v.z; + + private static float3 SetAxis(float3 v, int axis, float value) => + axis == 0 ? new float3(value, v.y, v.z) : axis == 1 ? new float3(v.x, value, v.z) : new float3(v.x, v.y, value); + + // ---- Exact pruning ---- + + [Fact] + public void PrunedOutputIsBitEqualToTheUnprunedSum() + { + var random = new System.Random(0x7ACE6); + for (var i = 0; i < 20000; i++) + { + var p = RandomPoint(random, 50.0f); + var side = RandomSide(random); + var offset = random.NextSingle(); + var normalization = random.NextSingle(); + var pruned = math.phacelle(p, side, offset, normalization); + var unpruned = math.phacelle(p, side, offset, normalization, false); + + AssertBitEqual(unpruned.cos, pruned.cos); + AssertBitEqual(unpruned.sin, pruned.sin); + } + } + + private static void AssertBitEqual(float4 expected, float4 actual) + { + Assert.Equal(BitConverter.SingleToInt32Bits(expected.x), BitConverter.SingleToInt32Bits(actual.x)); + Assert.Equal(BitConverter.SingleToInt32Bits(expected.y), BitConverter.SingleToInt32Bits(actual.y)); + Assert.Equal(BitConverter.SingleToInt32Bits(expected.z), BitConverter.SingleToInt32Bits(actual.z)); + Assert.Equal(BitConverter.SingleToInt32Bits(expected.w), BitConverter.SingleToInt32Bits(actual.w)); + } +} diff --git a/packages/cultmath/unity/org.gamecult.cultmath/Shaders/CultMath.hlsl b/packages/cultmath/unity/org.gamecult.cultmath/Shaders/CultMath.hlsl index 50c5a746..d0dd22ae 100644 --- a/packages/cultmath/unity/org.gamecult.cultmath/Shaders/CultMath.hlsl +++ b/packages/cultmath/unity/org.gamecult.cultmath/Shaders/CultMath.hlsl @@ -503,6 +503,92 @@ CultCellular cultmath_cellular(float3 p) return result; } +// Invariant 8's struct return shape for cultmath_phacelle, compared field by field and bit for bit +// against CultMath.CultPhasor, same field order. +struct CultPhasor +{ + float4 cos; + float4 sin; +}; + +// Johansen's Phacelle noise on 3D cells. See math.phacelle's comment (math.cs) for the stripe wave +// vector, the weight, the normalization, the exact gradient and why the prune is exact. Mirrors +// the C# body operation for operation; the C#-only internal overload that switches the prune off +// has no shader twin. +CultPhasor cultmath_phacelle(float3 p, float3 side, float offset, float normalization) +{ + float3 cell = floor(p); + float3 local = p - cell; + float phaseOffset = offset * CULTMATH_TAU; + float sumW = 0.0; + float sumCos = 0.0; + float sumSin = 0.0; + float3 gradW = float3(0.0, 0.0, 0.0); + float3 gradCos = float3(0.0, 0.0, 0.0); + float3 gradSin = float3(0.0, 0.0, 0.0); + + for (int dz = -1; dz <= 2; dz++) + { + float gz = max(0.0, abs(local.z - (float)dz) - 0.5); + for (int dy = -1; dy <= 2; dy++) + { + float gy = max(0.0, abs(local.y - (float)dy) - 0.5); + for (int dx = -1; dx <= 2; dx++) + { + float gx = max(0.0, abs(local.x - (float)dx) - 0.5); + if (gx * gx + gy * gy + gz * gz >= 2.25) + continue; + + float3 gridStep = float3(dx, dy, dz); + int3 hash = cultmath_pcg3d(int3(cell + gridStep)); + float3 jitter = float3( + ((uint)hash.x >> 8) * (1.0 / 16777216.0) - 0.5, + ((uint)hash.y >> 8) * (1.0 / 16777216.0) - 0.5, + ((uint)hash.z >> 8) * (1.0 / 16777216.0) - 0.5); + float3 v = local - gridStep - jitter; + float falloff = exp(-2.0 * dot(v, v)); + float w = max(0.0, falloff - 0.01111); + float3 dw = w > 0.0 ? (-4.0 * falloff) * v : float3(0.0, 0.0, 0.0); + float phase = dot(v, side) + phaseOffset; + float c = cos(phase); + float s = sin(phase); + + sumW += w; + sumCos += w * c; + sumSin += w * s; + gradW += dw; + gradCos += dw * c - (w * s) * side; + gradSin += dw * s + (w * c) * side; + } + } + } + + float invW = 1.0 / sumW; + float rawCos = sumCos * invW; + float rawSin = sumSin * invW; + float3 gradRawCos = (gradCos - rawCos * gradW) * invW; + float3 gradRawSin = (gradSin - rawSin * gradW) * invW; + float rawLength = sqrt(rawCos * rawCos + rawSin * rawSin); + float floorLength = 1.0 - normalization; + + CultPhasor result; + if (rawLength > floorLength) + { + float outCos = rawCos / rawLength; + float outSin = rawSin / rawLength; + float3 gradLength = (rawCos * gradRawCos + rawSin * gradRawSin) / rawLength; + result.cos = float4((gradRawCos - outCos * gradLength) / rawLength, outCos); + result.sin = float4((gradRawSin - outSin * gradLength) / rawLength, outSin); + } + else + { + result.cos = float4(gradRawCos / floorLength, rawCos / floorLength); + result.sin = float4(gradRawSin / floorLength, rawSin / floorLength); + } + + return result; +} + float cultmath_value_noise(float2 position) { float2 cell = floor(position); From 9f18a432cba499a42da96f3fa1b3a465c4b7fdc9 Mon Sep 17 00:00:00 2001 From: Meta Vens Date: Tue, 29 Sep 2026 23:51:04 +0200 Subject: [PATCH 03/16] CultMath: document phacelle and record Johansen's MPL-2.0 provenance design.md names CultPhasor as the second struct-return consumer and describes phacelle (stripe wave vector, weight, normalization, exact gradient and its two discontinuity classes, exact pruning). THIRD-PARTY-NOTICES.md, in both the package root and the Unity package, cites Rune Skovbo Johansen's Phacelle Noise (MPL-2.0) and records that the 3D-cell generalisation, the caller-supplied side vector, the exact gradient and the exact pruning are new. Co-Authored-By: Claude Opus 5.5 --- packages/cultmath/THIRD-PARTY-NOTICES.md | 15 ++++++++++++ packages/cultmath/docs/design.md | 23 +++++++++++++++++-- .../THIRD-PARTY-NOTICES.md | 15 ++++++++++++ 3 files changed, 51 insertions(+), 2 deletions(-) diff --git a/packages/cultmath/THIRD-PARTY-NOTICES.md b/packages/cultmath/THIRD-PARTY-NOTICES.md index 1de744a7..73f2461a 100644 --- a/packages/cultmath/THIRD-PARTY-NOTICES.md +++ b/packages/cultmath/THIRD-PARTY-NOTICES.md @@ -19,3 +19,18 @@ permutation and scale as `snoise(float3)`. - Copyright 2011 Ashima Arts. - Upstream: - License: MIT. + +## Phacelle Noise + +`math.phacelle` and `cultmath_phacelle` follow Rune Skovbo Johansen's Phacelle +noise (Shadertoy `t3dyWl`; erosion filter `wXcfWn`; blog, March 2026), read +from the C# port `PhacelleNoise` in `lpmitchell/AdvancedTerrainErosion` +(`AdvancedTerrainErosion.cs`, converted to burstable C# by Luke Mitchell, +2026). The cell weight, the phase blend and the normalization follow it. The +3D-cell generalisation, the caller-supplied stripe wave vector, the exact +gradient and the exact pruning are new here. + +- Copyright (c) 2025 Rune Skovbo Johansen. +- Upstream: +- License: MPL-2.0 (). CultMath's Unity package + is MPL-2.0, so the licence is compatible. diff --git a/packages/cultmath/docs/design.md b/packages/cultmath/docs/design.md index e14927fe..0858ee8c 100644 --- a/packages/cultmath/docs/design.md +++ b/packages/cultmath/docs/design.md @@ -128,8 +128,8 @@ whose HLSL return type is a plain struct of `float`/`floatN`/`int`/`intN` fields is compared to its C# counterpart field by field, recursing into each field's own components, down to bit-for-bit scalars, matching each leaf's declaration path (field names, not just position) and declared type as well -as its value. `CultCellular` (`math.cs`, `cellular`) is the first and, for -now, only consumer. A struct return with a mismatched, reordered-by-name, or +as its value. `CultCellular` (`math.cs`, `cellular`) and `CultPhasor` +(`math.cs`, `phacelle`) are its consumers. A struct return with a mismatched, reordered-by-name, or retyped field does not get a second comparison path or a shape of its own; it fails the same walk that already handles a bare vector return. @@ -264,6 +264,25 @@ together, laid out as `float4(gradient.xyz, value.w)`, with no value-only twin which is the right domain for `pcg3d`/`pcg4d`'s O(1) hash inputs but would turn an unclamped octave count into a multi-billion-iteration loop for these two mirrors. +- `phacelle(float3 p, float3 side, float offset, float normalization)` is Rune + Skovbo Johansen's Phacelle noise ("phase + cell": a stripe pattern built by + blending one cosine/sine wave per jittered cell), generalised to 3D cells. It + returns `CultPhasor { cos, sin }`, each `(∇, value)`. `side` is the stripe + wave vector, whose length is 2π times the stripe frequency; in 3D the + perpendicular to the flow is not unique, so the caller chooses it (on a unit + sphere, `cross(p̂, flow)`). `offset` is the phase in cycles. It visits 4×4×4 + cells with `pcg3d`-hashed jitter in `[-0.5, 0.5)`, weights each wave by + `max(0, exp(-2d²) - 0.01111)`, and normalizes the blended phasor as upstream + does (length 1 where the raw length is at least `1 - normalization`, scaled + by `1 / (1 - normalization)` below). The gradient is exact, weight + derivatives and normalization included, rather than upstream's + `∇cos ≈ -sin·side`, which ignores the weight gradient. It is discontinuous + where the raw length crosses the normalization threshold and where a cell's + weight leaves its support (distance 1.4998); the value is continuous at both. + Cells whose per-axis lower bound `Σ max(0, |local - g| - 0.5)²` is at least + 2.25 have weight exactly 0 and are skipped, which changes no output bit; + `PhacelleTests` pins that against an internal overload with the prune + disabled. No 2D variant exists; nothing consumes one. Integer hashing uses the PCG hashes from Jarzynski and Olano, "Hash Functions for GPU Rendering" (JCGT 9(3), 2020): `pcg(uint)` is O'Neill's RXS-M-XS 32/32 diff --git a/packages/cultmath/unity/org.gamecult.cultmath/THIRD-PARTY-NOTICES.md b/packages/cultmath/unity/org.gamecult.cultmath/THIRD-PARTY-NOTICES.md index 1de744a7..73f2461a 100644 --- a/packages/cultmath/unity/org.gamecult.cultmath/THIRD-PARTY-NOTICES.md +++ b/packages/cultmath/unity/org.gamecult.cultmath/THIRD-PARTY-NOTICES.md @@ -19,3 +19,18 @@ permutation and scale as `snoise(float3)`. - Copyright 2011 Ashima Arts. - Upstream: - License: MIT. + +## Phacelle Noise + +`math.phacelle` and `cultmath_phacelle` follow Rune Skovbo Johansen's Phacelle +noise (Shadertoy `t3dyWl`; erosion filter `wXcfWn`; blog, March 2026), read +from the C# port `PhacelleNoise` in `lpmitchell/AdvancedTerrainErosion` +(`AdvancedTerrainErosion.cs`, converted to burstable C# by Luke Mitchell, +2026). The cell weight, the phase blend and the normalization follow it. The +3D-cell generalisation, the caller-supplied stripe wave vector, the exact +gradient and the exact pruning are new here. + +- Copyright (c) 2025 Rune Skovbo Johansen. +- Upstream: +- License: MPL-2.0 (). CultMath's Unity package + is MPL-2.0, so the licence is compatible. From e9469cb3a432ba410c4694fa9ac3797ae115b7ad Mon Sep 17 00:00:00 2001 From: Meta Vens Date: Tue, 29 Sep 2026 23:53:21 +0200 Subject: [PATCH 04/16] Ack map: record Q-A3 B, Q-A4 A, Q-A5 A and Q-A6 A Co-Authored-By: Claude Opus 5.5 --- docs/cultnet-rudp-ack-cut.md | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/docs/cultnet-rudp-ack-cut.md b/docs/cultnet-rudp-ack-cut.md index 892110e1..30d6089f 100644 --- a/docs/cultnet-rudp-ack-cut.md +++ b/docs/cultnet-rudp-ack-cut.md @@ -16,8 +16,18 @@ Status: cut map, Imagination pass 2, 2026-09-30, committed by Self. Pass 2 re-ma frames as duplicates. The ending paths disagree about owed writes. One rule: a write belongs to the session it was issued in. Pass 2 maps this as **Cut 1b** (the ending rule) and **Cut 1c** (how a new generation is recognised), separate from Cut 1 (§8 says why). -- **Open:** Q-A3 (expiring reliable traffic next to ordered traffic), Q-A4 (porting receipts), Q-A5 - (Kotlin), and a new **Q-A6** (how a new generation is recognised; blocks Cut 1c only). +- **Q-A3 B (ruled after pass 2):** FORWARD-TSN now (RFC 3758). On expiry the sender sends an + abandonment notice (Ack-typed, flag `0x10`, ignored by old peers); new receivers treat the sequence as + received with no frame and acknowledge it; the notice's resend has its own bound. This is a new wire + cut in every receiver, landing before Cut 4. The recommendation was A; the operator chose B. +- **Q-A4 A (ruled after pass 2):** port receipts (Pending, Acknowledged, Invalidated, `WaitAcknowledged`) + to C#, TypeScript, Python and Kotlin in this campaign, after Cut 4. The recommendation was B. +- **Q-A5 A (ruled after pass 2):** Kotlin is in, as trailing Cut K, verified on a Kotlin image added to + Yggdrasil first. +- **Q-A6 A (ruled after pass 2):** the Connect's sequence identifies the session; the default initial + sequence becomes random in [1, 2^31), as TCP's ISN. No wire change; the public default changes. +- **Pending:** the cut sections below still read these four as open. Imagination pass 3 maps the + FORWARD-TSN cut, the receipts port and the Kotlin image, and re-sequences Cuts 1c, 4 and K. - **Body.** CultLib `main` at `3bf1c0c` ("Merge RUDP stray-packet hardening across all four runtimes"). Every `file:line` is against it unless it names another revision. From 2a3cab1bd7ab5f2532f8a2ff32f328d057ae2de5 Mon Sep 17 00:00:00 2001 From: Meta Vens Date: Tue, 29 Sep 2026 23:58:43 +0200 Subject: [PATCH 05/16] CultMath tests: a dense sweep pins cultmath_phacelle against C# bit for bit The generic mirror comparison passes about a hundred inputs through the shader and missed a mirror whose prune cut was 2.0 instead of 2.25, because only a rare cell has a nonzero weight under that cut. Twenty thousand random arguments compare the shader text and math.phacelle field by field and kill that mutant. Co-Authored-By: Claude Opus 5.5 --- .../HlslSourceCompatibilityTests.cs | 29 +++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/packages/cultmath/tests/CultMath.Tests/HlslSourceCompatibilityTests.cs b/packages/cultmath/tests/CultMath.Tests/HlslSourceCompatibilityTests.cs index ad690759..2cf737df 100644 --- a/packages/cultmath/tests/CultMath.Tests/HlslSourceCompatibilityTests.cs +++ b/packages/cultmath/tests/CultMath.Tests/HlslSourceCompatibilityTests.cs @@ -134,6 +134,35 @@ public sealed class HlslSourceCompatibilityTests Assert.Equal(34, compared.Count); } + /// + /// The generic comparison above passes about a hundred inputs through cultmath_phacelle, and + /// its prune only changes an output for the rare cell whose weight is nonzero but whose lower + /// bound sits just under the 2.25 cut (Soul's F6 lesson for cellular): a mirror whose cut is 2.0 + /// survives a hundred inputs. A dense sweep of realistic arguments closes that gap. + /// + [Fact] + public void PhacelleMirrorMatchesCSharpBitForBitAcrossADenseSweep() + { + var (assembly, errors) = CompileShaderMirror(); + Assert.True(assembly is not null, string.Join(Environment.NewLine, errors)); + var shaderType = assembly!.GetType("CultMathHlsl.HlslShader")!; + var shader = Activator.CreateInstance(shaderType); + var mirror = ShaderFunction(shaderType, "cultmath_phacelle", typeof(float3), typeof(float3), typeof(float), typeof(float))!; + + var random = new System.Random(0x5EED2); + float Next(float extent) => (random.NextSingle() * 2.0f - 1.0f) * extent; + for (var i = 0; i < 20000; i++) + { + var p = new float3(Next(50.0f), Next(50.0f), Next(50.0f)); + var side = new float3(Next(9.0f), Next(9.0f), Next(9.0f)); + var offset = random.NextSingle(); + var normalization = random.NextSingle(); + var expected = Values(phacelle(p, side, offset, normalization)); + var actual = Values(mirror.Invoke(shader, new object[] { p, side, offset, normalization })!); + Assert.True(BitwiseEqual(expected, actual), $"mirror differs from C# at p = {p}, side = {side}"); + } + } + /// /// A hand-built pair of structs standing in for a mirror struct return and its C# counterpart /// (invariant 8's one struct return shape, e.g. ): same field shape, From 0acc3728ed521a847d589f8f0cd9e9202f64ca96 Mon Sep 17 00:00:00 2001 From: Meta Vens Date: Wed, 30 Sep 2026 00:34:27 +0200 Subject: [PATCH 06/16] CultMath: Phacelle port moves into MPL-2.0 files; prune cut pinned by boundary points MPL-2.0 is file-level and the Phacelle port sat inside MIT files. phacelle and CultPhasor now live in math.Phacelle.cs and CultPhasor.cs, and cultmath_phacelle in CultMath.Phacelle.hlsl (included by CultMath.hlsl, both tracked copies), each with the MPL header and Johansen / Luke Mitchell attribution. math stays one partial class, so math.phacelle is called exactly as before. The GameCult.Math package declares MIT AND MPL-2.0 and packs THIRD-PARTY-NOTICES.md and the new include; the Unity LICENSE.md points at CultLib; build-unity-package.ps1 requires the new include. The random sweeps never met a cell whose box bound sits just under the 2.25 cut while its weight is still nonzero, so cuts of 2.245, 2.24 and 2.2 survived. Six hard-coded points (found by an offline search) sit next to such cells; the C# pruned-vs-unpruned test and the HLSL mirror test both use them, with a witness check that re-derives the property from the hash. Docs: the prune keeps about 45 of 64 cells (14 is the nonzero-weight count), the support radius is 1.49998 (d^2 = 2.24995), normalization's range and saturation, W = 0 only for adversarial hashes, and the support-edge gradient seam size. Co-Authored-By: Claude Sonnet 5.5 --- packages/cultmath/README.md | 2 + packages/cultmath/THIRD-PARTY-NOTICES.md | 9 +- packages/cultmath/docs/design.md | 26 +++- .../cultmath/scripts/build-unity-package.ps1 | 4 +- .../cultmath/shaders/CultMath.Phacelle.hlsl | 99 ++++++++++++++ .../shaders/CultMath.Phacelle.hlsl.meta | 7 + packages/cultmath/shaders/CultMath.hlsl | 86 +----------- .../cultmath/src/CultMath/CultMath.csproj | 4 + packages/cultmath/src/CultMath/CultPhasor.cs | 6 + .../cultmath/src/CultMath/math.Phacelle.cs | 127 ++++++++++++++++++ .../src/CultMath/math.Phacelle.cs.meta | 2 + packages/cultmath/src/CultMath/math.cs | 98 -------------- .../tests/CultMath.Tests/HlslMirrorTests.cs | 15 ++- .../HlslSourceCompatibilityTests.cs | 41 +++++- .../tests/CultMath.Tests/PhacelleTests.cs | 67 ++++++++- .../unity/org.gamecult.cultmath/LICENSE.md | 4 +- .../unity/org.gamecult.cultmath/README.md | 2 +- .../Shaders/CultMath.Phacelle.hlsl | 99 ++++++++++++++ .../Shaders/CultMath.Phacelle.hlsl.meta | 7 + .../Shaders/CultMath.hlsl | 86 +----------- .../THIRD-PARTY-NOTICES.md | 9 +- 21 files changed, 515 insertions(+), 285 deletions(-) create mode 100644 packages/cultmath/shaders/CultMath.Phacelle.hlsl create mode 100644 packages/cultmath/shaders/CultMath.Phacelle.hlsl.meta create mode 100644 packages/cultmath/src/CultMath/math.Phacelle.cs create mode 100644 packages/cultmath/src/CultMath/math.Phacelle.cs.meta create mode 100644 packages/cultmath/unity/org.gamecult.cultmath/Shaders/CultMath.Phacelle.hlsl create mode 100644 packages/cultmath/unity/org.gamecult.cultmath/Shaders/CultMath.Phacelle.hlsl.meta diff --git a/packages/cultmath/README.md b/packages/cultmath/README.md index 13f17973..f70df445 100644 --- a/packages/cultmath/README.md +++ b/packages/cultmath/README.md @@ -33,6 +33,8 @@ shader semantics. - `shaders/CultMath.hlsl`, a canonical HLSL mirror include for shader-side parity. HLSL already owns `float2`, `float3`, and `float4`; the include exposes `cultmath_*` functions for shared semantics HLSL does not provide. + It includes `shaders/CultMath.Phacelle.hlsl` (MPL-2.0) from the same + directory, so ship both files. - `Voronoi.SampleTones`, a C# batch surface that calls the Rust `cultmath-core` native kernel when `cultmath_core` is available and falls back to the managed parity path otherwise. diff --git a/packages/cultmath/THIRD-PARTY-NOTICES.md b/packages/cultmath/THIRD-PARTY-NOTICES.md index 73f2461a..625d63a9 100644 --- a/packages/cultmath/THIRD-PARTY-NOTICES.md +++ b/packages/cultmath/THIRD-PARTY-NOTICES.md @@ -32,5 +32,10 @@ gradient and the exact pruning are new here. - Copyright (c) 2025 Rune Skovbo Johansen. - Upstream: -- License: MPL-2.0 (). CultMath's Unity package - is MPL-2.0, so the licence is compatible. +- License: MPL-2.0 (). MPL-2.0 is file-level, so it + covers exactly the files that carry the port and nothing else: + `src/CultMath/math.Phacelle.cs`, `src/CultMath/CultPhasor.cs`, + `shaders/CultMath.Phacelle.hlsl` and the Unity copy of that include. Each + carries the MPL-2.0 header and this attribution. The rest of CultMath is MIT, + so the `GameCult.Math` NuGet package declares `MIT AND MPL-2.0` and ships this + file. The Unity package as a whole is MPL-2.0. diff --git a/packages/cultmath/docs/design.md b/packages/cultmath/docs/design.md index 0858ee8c..0de8eb14 100644 --- a/packages/cultmath/docs/design.md +++ b/packages/cultmath/docs/design.md @@ -129,7 +129,7 @@ fields is compared to its C# counterpart field by field, recursing into each field's own components, down to bit-for-bit scalars, matching each leaf's declaration path (field names, not just position) and declared type as well as its value. `CultCellular` (`math.cs`, `cellular`) and `CultPhasor` -(`math.cs`, `phacelle`) are its consumers. A struct return with a mismatched, reordered-by-name, or +(`math.Phacelle.cs`, `phacelle`) are its consumers. A struct return with a mismatched, reordered-by-name, or retyped field does not get a second comparison path or a shape of its own; it fails the same walk that already handles a bare vector return. @@ -278,11 +278,27 @@ together, laid out as `float4(gradient.xyz, value.w)`, with no value-only twin derivatives and normalization included, rather than upstream's `∇cos ≈ -sin·side`, which ignores the weight gradient. It is discontinuous where the raw length crosses the normalization threshold and where a cell's - weight leaves its support (distance 1.4998); the value is continuous at both. + weight leaves its support (d² = 2.24995, distance 1.49998); the value is + continuous at both. At the support edge `∇w` jumps from `-4·0.01111·v` to 0, + a per-cell jump of length `0.0444·|v|` (0.0667 at `|v| = 1.5`); measured over + random points, the output gradient jumps by a median 1.5% of its length and + at most about 27%. `normalization` is meant to lie in `[0, 1]`: 1 or more + saturates (every output is stretched to length 1), a negative value scales + every output down by `1 / (1 - normalization)`, and the normalized gradient + scales as `1 / |I|` for the blended phasor `I`, so it grows where the blend + nearly cancels. The total weight `W` is positive for every `pcg3d` input (the + smallest found by hill-climbing is 0.056); `W = 0`, and a NaN output, needs + adversarial jitter no hash yields. Cells whose per-axis lower bound `Σ max(0, |local - g| - 0.5)²` is at least - 2.25 have weight exactly 0 and are skipped, which changes no output bit; - `PhacelleTests` pins that against an internal overload with the prune - disabled. No 2D variant exists; nothing consumes one. + 2.25 lie outside the support and have weight exactly 0, so they are skipped, + which changes no output bit. Cells whose bound sits just under 2.25 can still + carry a nonzero weight, so the cut cannot be lowered; `PhacelleTests` pins + the cut with points next to such cells, against an internal overload with the + prune disabled, and the HLSL mirror test pins it on the shader side. The prune + keeps about 45 of the 64 cells on average (a saving of about 30%); about 14 + have a nonzero weight. The port sits in its own MPL-2.0 files + (`math.Phacelle.cs`, `CultPhasor.cs`, `CultMath.Phacelle.hlsl`); see + `THIRD-PARTY-NOTICES.md`. No 2D variant exists; nothing consumes one. Integer hashing uses the PCG hashes from Jarzynski and Olano, "Hash Functions for GPU Rendering" (JCGT 9(3), 2020): `pcg(uint)` is O'Neill's RXS-M-XS 32/32 diff --git a/packages/cultmath/scripts/build-unity-package.ps1 b/packages/cultmath/scripts/build-unity-package.ps1 index 39e55813..228d735c 100644 --- a/packages/cultmath/scripts/build-unity-package.ps1 +++ b/packages/cultmath/scripts/build-unity-package.ps1 @@ -78,7 +78,9 @@ $requiredTemplateFiles = @( "Runtime\UnityBridge\UnityConversions.cs", "Runtime\UnityBridge\UnityConversions.cs.meta", "Shaders\CultMath.hlsl", - "Shaders\CultMath.hlsl.meta" + "Shaders\CultMath.hlsl.meta", + "Shaders\CultMath.Phacelle.hlsl", + "Shaders\CultMath.Phacelle.hlsl.meta" ) foreach ($relativePath in $requiredTemplateFiles) { if (-not (Test-Path -LiteralPath (Join-Path $templateRoot $relativePath) -PathType Leaf)) { diff --git a/packages/cultmath/shaders/CultMath.Phacelle.hlsl b/packages/cultmath/shaders/CultMath.Phacelle.hlsl new file mode 100644 index 00000000..1db0a2fc --- /dev/null +++ b/packages/cultmath/shaders/CultMath.Phacelle.hlsl @@ -0,0 +1,99 @@ +#ifndef CULTMATH_PHACELLE_HLSL +#define CULTMATH_PHACELLE_HLSL + +// This Source Code Form is subject to the terms of the Mozilla Public License, v. 2.0. If a copy of +// the MPL was not distributed with this file, You can obtain one at https://mozilla.org/MPL/2.0/. +// +// Phacelle noise. Copyright (c) 2025 Rune Skovbo Johansen (Shadertoy t3dyWl; MPL-2.0), read from the +// C# port PhacelleNoise in lpmitchell/AdvancedTerrainErosion, converted to burstable C# by Luke +// Mitchell (2026). Modified by GameCult: 3D cells, a caller-supplied stripe wave vector, an exact +// gradient and exact pruning. This file is an MPL-2.0 file; CultMath.hlsl, which includes it after +// its own definitions of CULTMATH_TAU and cultmath_pcg3d, is MIT (see THIRD-PARTY-NOTICES.md). + +// Invariant 8's struct return shape for cultmath_phacelle, compared field by field and bit for bit +// against CultMath.CultPhasor, same field order. +struct CultPhasor +{ + float4 cos; + float4 sin; +}; + +// Johansen's Phacelle noise on 3D cells. See math.phacelle's comment (math.Phacelle.cs) for the stripe wave +// vector, the weight, the normalization, the exact gradient and why the prune is exact. Mirrors +// the C# body operation for operation; the C#-only internal overload that switches the prune off +// has no shader twin. +CultPhasor cultmath_phacelle(float3 p, float3 side, float offset, float normalization) +{ + float3 cell = floor(p); + float3 local = p - cell; + float phaseOffset = offset * CULTMATH_TAU; + float sumW = 0.0; + float sumCos = 0.0; + float sumSin = 0.0; + float3 gradW = float3(0.0, 0.0, 0.0); + float3 gradCos = float3(0.0, 0.0, 0.0); + float3 gradSin = float3(0.0, 0.0, 0.0); + + for (int dz = -1; dz <= 2; dz++) + { + float gz = max(0.0, abs(local.z - (float)dz) - 0.5); + for (int dy = -1; dy <= 2; dy++) + { + float gy = max(0.0, abs(local.y - (float)dy) - 0.5); + for (int dx = -1; dx <= 2; dx++) + { + float gx = max(0.0, abs(local.x - (float)dx) - 0.5); + if (gx * gx + gy * gy + gz * gz >= 2.25) + continue; + + float3 gridStep = float3(dx, dy, dz); + int3 hash = cultmath_pcg3d(int3(cell + gridStep)); + float3 jitter = float3( + ((uint)hash.x >> 8) * (1.0 / 16777216.0) - 0.5, + ((uint)hash.y >> 8) * (1.0 / 16777216.0) - 0.5, + ((uint)hash.z >> 8) * (1.0 / 16777216.0) - 0.5); + float3 v = local - gridStep - jitter; + float falloff = exp(-2.0 * dot(v, v)); + float w = max(0.0, falloff - 0.01111); + float3 dw = w > 0.0 ? (-4.0 * falloff) * v : float3(0.0, 0.0, 0.0); + float phase = dot(v, side) + phaseOffset; + float c = cos(phase); + float s = sin(phase); + + sumW += w; + sumCos += w * c; + sumSin += w * s; + gradW += dw; + gradCos += dw * c - (w * s) * side; + gradSin += dw * s + (w * c) * side; + } + } + } + + float invW = 1.0 / sumW; + float rawCos = sumCos * invW; + float rawSin = sumSin * invW; + float3 gradRawCos = (gradCos - rawCos * gradW) * invW; + float3 gradRawSin = (gradSin - rawSin * gradW) * invW; + float rawLength = sqrt(rawCos * rawCos + rawSin * rawSin); + float floorLength = 1.0 - normalization; + + CultPhasor result; + if (rawLength > floorLength) + { + float outCos = rawCos / rawLength; + float outSin = rawSin / rawLength; + float3 gradLength = (rawCos * gradRawCos + rawSin * gradRawSin) / rawLength; + result.cos = float4((gradRawCos - outCos * gradLength) / rawLength, outCos); + result.sin = float4((gradRawSin - outSin * gradLength) / rawLength, outSin); + } + else + { + result.cos = float4(gradRawCos / floorLength, rawCos / floorLength); + result.sin = float4(gradRawSin / floorLength, rawSin / floorLength); + } + + return result; +} + +#endif diff --git a/packages/cultmath/shaders/CultMath.Phacelle.hlsl.meta b/packages/cultmath/shaders/CultMath.Phacelle.hlsl.meta new file mode 100644 index 00000000..69308465 --- /dev/null +++ b/packages/cultmath/shaders/CultMath.Phacelle.hlsl.meta @@ -0,0 +1,7 @@ +fileFormatVersion: 2 +guid: e07a3b95c1d84f2aa6b83d4c5f19a0d7 +ShaderIncludeImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/packages/cultmath/shaders/CultMath.hlsl b/packages/cultmath/shaders/CultMath.hlsl index d0dd22ae..af2d3741 100644 --- a/packages/cultmath/shaders/CultMath.hlsl +++ b/packages/cultmath/shaders/CultMath.hlsl @@ -503,91 +503,7 @@ CultCellular cultmath_cellular(float3 p) return result; } -// Invariant 8's struct return shape for cultmath_phacelle, compared field by field and bit for bit -// against CultMath.CultPhasor, same field order. -struct CultPhasor -{ - float4 cos; - float4 sin; -}; - -// Johansen's Phacelle noise on 3D cells. See math.phacelle's comment (math.cs) for the stripe wave -// vector, the weight, the normalization, the exact gradient and why the prune is exact. Mirrors -// the C# body operation for operation; the C#-only internal overload that switches the prune off -// has no shader twin. -CultPhasor cultmath_phacelle(float3 p, float3 side, float offset, float normalization) -{ - float3 cell = floor(p); - float3 local = p - cell; - float phaseOffset = offset * CULTMATH_TAU; - float sumW = 0.0; - float sumCos = 0.0; - float sumSin = 0.0; - float3 gradW = float3(0.0, 0.0, 0.0); - float3 gradCos = float3(0.0, 0.0, 0.0); - float3 gradSin = float3(0.0, 0.0, 0.0); - - for (int dz = -1; dz <= 2; dz++) - { - float gz = max(0.0, abs(local.z - (float)dz) - 0.5); - for (int dy = -1; dy <= 2; dy++) - { - float gy = max(0.0, abs(local.y - (float)dy) - 0.5); - for (int dx = -1; dx <= 2; dx++) - { - float gx = max(0.0, abs(local.x - (float)dx) - 0.5); - if (gx * gx + gy * gy + gz * gz >= 2.25) - continue; - - float3 gridStep = float3(dx, dy, dz); - int3 hash = cultmath_pcg3d(int3(cell + gridStep)); - float3 jitter = float3( - ((uint)hash.x >> 8) * (1.0 / 16777216.0) - 0.5, - ((uint)hash.y >> 8) * (1.0 / 16777216.0) - 0.5, - ((uint)hash.z >> 8) * (1.0 / 16777216.0) - 0.5); - float3 v = local - gridStep - jitter; - float falloff = exp(-2.0 * dot(v, v)); - float w = max(0.0, falloff - 0.01111); - float3 dw = w > 0.0 ? (-4.0 * falloff) * v : float3(0.0, 0.0, 0.0); - float phase = dot(v, side) + phaseOffset; - float c = cos(phase); - float s = sin(phase); - - sumW += w; - sumCos += w * c; - sumSin += w * s; - gradW += dw; - gradCos += dw * c - (w * s) * side; - gradSin += dw * s + (w * c) * side; - } - } - } - - float invW = 1.0 / sumW; - float rawCos = sumCos * invW; - float rawSin = sumSin * invW; - float3 gradRawCos = (gradCos - rawCos * gradW) * invW; - float3 gradRawSin = (gradSin - rawSin * gradW) * invW; - float rawLength = sqrt(rawCos * rawCos + rawSin * rawSin); - float floorLength = 1.0 - normalization; - - CultPhasor result; - if (rawLength > floorLength) - { - float outCos = rawCos / rawLength; - float outSin = rawSin / rawLength; - float3 gradLength = (rawCos * gradRawCos + rawSin * gradRawSin) / rawLength; - result.cos = float4((gradRawCos - outCos * gradLength) / rawLength, outCos); - result.sin = float4((gradRawSin - outSin * gradLength) / rawLength, outSin); - } - else - { - result.cos = float4(gradRawCos / floorLength, rawCos / floorLength); - result.sin = float4(gradRawSin / floorLength, rawSin / floorLength); - } - - return result; -} +#include "CultMath.Phacelle.hlsl" float cultmath_value_noise(float2 position) { diff --git a/packages/cultmath/src/CultMath/CultMath.csproj b/packages/cultmath/src/CultMath/CultMath.csproj index 7c253027..b4a6ce8b 100644 --- a/packages/cultmath/src/CultMath/CultMath.csproj +++ b/packages/cultmath/src/CultMath/CultMath.csproj @@ -8,6 +8,8 @@ CultMath CultMath GameCult.Math + + MIT AND MPL-2.0 0.2.4 Clean-room C# math shaped for HLSL-style renderer code. GameCult @@ -21,6 +23,8 @@ + + diff --git a/packages/cultmath/src/CultMath/CultPhasor.cs b/packages/cultmath/src/CultMath/CultPhasor.cs index 0a28faa8..fbc1f9d7 100644 --- a/packages/cultmath/src/CultMath/CultPhasor.cs +++ b/packages/cultmath/src/CultMath/CultPhasor.cs @@ -1,3 +1,9 @@ +// This Source Code Form is subject to the terms of the Mozilla Public License, v. 2.0. If a copy of +// the MPL was not distributed with this file, You can obtain one at https://mozilla.org/MPL/2.0/. +// +// The result type of math.phacelle (math.Phacelle.cs), Rune Skovbo Johansen's Phacelle noise +// (Copyright (c) 2025, MPL-2.0; see THIRD-PARTY-NOTICES.md). + namespace CultMath; /// diff --git a/packages/cultmath/src/CultMath/math.Phacelle.cs b/packages/cultmath/src/CultMath/math.Phacelle.cs new file mode 100644 index 00000000..25689737 --- /dev/null +++ b/packages/cultmath/src/CultMath/math.Phacelle.cs @@ -0,0 +1,127 @@ +// This Source Code Form is subject to the terms of the Mozilla Public License, v. 2.0. If a copy of +// the MPL was not distributed with this file, You can obtain one at https://mozilla.org/MPL/2.0/. +// +// Phacelle noise. Copyright (c) 2025 Rune Skovbo Johansen (Shadertoy t3dyWl; MPL-2.0), read from the +// C# port PhacelleNoise in lpmitchell/AdvancedTerrainErosion, converted to burstable C# by Luke +// Mitchell (2026). Modified by GameCult: 3D cells, a caller-supplied stripe wave vector, an exact +// gradient and exact pruning. This file and CultPhasor.cs are the MPL-2.0 files of CultMath; the +// rest of the package is MIT (see THIRD-PARTY-NOTICES.md). + +namespace CultMath; + +public static partial class math +{ + // Johansen's Phacelle noise (phase + cell), generalised to 3D cells with a caller-supplied + // stripe wave vector. Each of the 4x4x4 cells around p carries one jittered feature point + // (jitter in [-0.5, 0.5) per axis, from pcg3d over the integer cell coordinate, the same + // integer-hashing rule as cellular) and one cosine/sine wave that runs along `side` (the wave + // vector: its length is 2*pi times the stripe frequency, and the caller chooses which + // perpendicular it is; on a unit sphere that is cross(p_hat, flow)). The waves are blended with + // the bell weight max(0, exp(-2 d^2) - 0.01111) (exactly 0 at distance 1.5, the nearest a + // feature outside the 4x4x4 block can be in 2D, which is what keeps cell-grid lines out), and + // the blended phasor is normalized: its length is stretched to 1 wherever the raw length is at + // least 1 - normalization, and scaled by 1 / (1 - normalization) below that. `offset` is the + // phase in cycles (1 is a full turn). Value and normalization follow Johansen's + // PhacelleNoise as ported in lpmitchell/AdvancedTerrainErosion. + // + // `normalization` is meant to lie in [0, 1]. At 0 the output is the raw blend (a phasor of + // length at most 1); at 1 the floor is 0 and every output is stretched to length 1. Values of + // 1 or more saturate at that (the floor is never above the raw length), and a negative value + // scales every output down by 1 / (1 - normalization) < 1. The normalized gradient scales as + // 1 / |I| where I is the blended phasor (below), so it grows wherever the blend nearly + // cancels; a normalization near 1 lets that happen at every point. + // + // The gradient is exact: it carries the weight derivative (grad w = -4 v exp(-2 |v|^2) where + // w > 0) and the normalization, not upstream's convention grad(cos) ~ -sin * side, which + // ignores the weight gradient (Asura probe: 16.8 deg off at the median, 97.8 at the 95th + // percentile; the exact form is 0.00 and 0.03). With P = sum(w * (cos, sin)) and W = sum(w), + // I = P / W has grad I = (grad P - I * grad W) / W; above the threshold the output is I / |I| + // with gradient (grad I - out * grad|I|) / |I|, below it grad I / (1 - normalization). The + // value is continuous at both seams below; the gradient is not. + // * Where |I| crosses the threshold, the gradient jumps between the two branches. + // * Where a cell's weight leaves its support (d^2 = ln(1 / 0.01111) / 2 = 2.24995, distance + // 1.49998), grad w jumps from -4 * 0.01111 * v to 0: a per-cell jump of length + // 0.0444 * |v|, 0.0667 at |v| = 1.5. Measured over random points, the resulting jump in + // the output gradient is a median 1.5% of its length and at most about 27%. + // + // W is the sum of 64 weights and is positive for every input pcg3d produces (the smallest + // W found by hill-climbing the hash-driven cell contents is 0.056). It is 0, and the output + // NaN, only if all 64 weights vanish, which takes adversarial jitter no hash yields. + // + // Pruning is exact: per axis, |local - g| - 0.5 (floored at 0) lower-bounds |v| for any jitter, + // so a cell whose summed square is at least 2.25 lies outside the support (2.24995 < 2.25) and + // has weight exactly 0: it contributes nothing to any sum. Cells whose bound sits just under + // 2.25 can still have a nonzero weight, so the cut cannot be lowered. On average the prune + // keeps about 45 of the 64 cells (a saving of about 30%); only about 14 have a nonzero weight. + // Only the internal overload can switch the prune off; PhacelleTests uses it to pin bit equality. + public static CultPhasor phacelle(float3 p, float3 side, float offset, float normalization) => + phacelle(p, side, offset, normalization, true); + + internal static CultPhasor phacelle(float3 p, float3 side, float offset, float normalization, bool prune) + { + var cell = floor(p); + var local = p - cell; + var phaseOffset = offset * TAU; + var sumW = 0.0f; + var sumCos = 0.0f; + var sumSin = 0.0f; + var gradW = new float3(0.0f, 0.0f, 0.0f); + var gradCos = new float3(0.0f, 0.0f, 0.0f); + var gradSin = new float3(0.0f, 0.0f, 0.0f); + + for (var dz = -1; dz <= 2; dz++) + { + var gz = max(0.0f, abs(local.z - (float)dz) - 0.5f); + for (var dy = -1; dy <= 2; dy++) + { + var gy = max(0.0f, abs(local.y - (float)dy) - 0.5f); + for (var dx = -1; dx <= 2; dx++) + { + var gx = max(0.0f, abs(local.x - (float)dx) - 0.5f); + if (prune && gx * gx + gy * gy + gz * gz >= 2.25f) + continue; + + var gridStep = new float3(dx, dy, dz); + var hash = pcg3d(int3(cell + gridStep)); + var jitter = new float3(cellular_unit(hash.x) - 0.5f, cellular_unit(hash.y) - 0.5f, cellular_unit(hash.z) - 0.5f); + var v = local - gridStep - jitter; + var falloff = exp(-2.0f * dot(v, v)); + var w = max(0.0f, falloff - 0.01111f); + var dw = w > 0.0f ? (-4.0f * falloff) * v : new float3(0.0f, 0.0f, 0.0f); + var phase = dot(v, side) + phaseOffset; + var c = cos(phase); + var s = sin(phase); + + sumW += w; + sumCos += w * c; + sumSin += w * s; + gradW += dw; + gradCos += dw * c - (w * s) * side; + gradSin += dw * s + (w * c) * side; + } + } + } + + var invW = 1.0f / sumW; + var rawCos = sumCos * invW; + var rawSin = sumSin * invW; + var gradRawCos = (gradCos - rawCos * gradW) * invW; + var gradRawSin = (gradSin - rawSin * gradW) * invW; + var rawLength = sqrt(rawCos * rawCos + rawSin * rawSin); + var floorLength = 1.0f - normalization; + + if (rawLength > floorLength) + { + var outCos = rawCos / rawLength; + var outSin = rawSin / rawLength; + var gradLength = (rawCos * gradRawCos + rawSin * gradRawSin) / rawLength; + return new CultPhasor( + new float4((gradRawCos - outCos * gradLength) / rawLength, outCos), + new float4((gradRawSin - outSin * gradLength) / rawLength, outSin)); + } + + return new CultPhasor( + new float4(gradRawCos / floorLength, rawCos / floorLength), + new float4(gradRawSin / floorLength, rawSin / floorLength)); + } +} diff --git a/packages/cultmath/src/CultMath/math.Phacelle.cs.meta b/packages/cultmath/src/CultMath/math.Phacelle.cs.meta new file mode 100644 index 00000000..898abc4c --- /dev/null +++ b/packages/cultmath/src/CultMath/math.Phacelle.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 2c9d4e6b1a8f47d3b05e7a3c9d1f8e26 diff --git a/packages/cultmath/src/CultMath/math.cs b/packages/cultmath/src/CultMath/math.cs index 430325ea..23916b6b 100644 --- a/packages/cultmath/src/CultMath/math.cs +++ b/packages/cultmath/src/CultMath/math.cs @@ -968,104 +968,6 @@ public static partial class math // dxc and C#, bit-exact on both sides. internal static float cellular_unit(int bits) => ((uint)bits >> 8) * (1.0f / 16777216.0f); - // Johansen's Phacelle noise (phase + cell), generalised to 3D cells with a caller-supplied - // stripe wave vector. Each of the 4x4x4 cells around p carries one jittered feature point - // (jitter in [-0.5, 0.5) per axis, from pcg3d over the integer cell coordinate, the same - // integer-hashing rule as cellular) and one cosine/sine wave that runs along `side` (the wave - // vector: its length is 2*pi times the stripe frequency, and the caller chooses which - // perpendicular it is; on a unit sphere that is cross(p_hat, flow)). The waves are blended with - // the bell weight max(0, exp(-2 d^2) - 0.01111) (exactly 0 at distance 1.5, the nearest a - // feature outside the 4x4x4 block can be in 2D, which is what keeps cell-grid lines out), and - // the blended phasor is normalized: its length is stretched to 1 wherever the raw length is at - // least 1 - normalization, and scaled by 1 / (1 - normalization) below that. `offset` is the - // phase in cycles (1 is a full turn). Value and normalization follow Johansen's - // PhacelleNoise as ported in lpmitchell/AdvancedTerrainErosion. - // - // The gradient is exact: it carries the weight derivative (grad w = -4 v exp(-2 |v|^2) where - // w > 0) and the normalization, not upstream's convention grad(cos) ~ -sin * side, which - // ignores the weight gradient (Asura probe: 16.8 deg off at the median, 97.8 at the 95th - // percentile; the exact form is 0.00 and 0.03). With P = sum(w * (cos, sin)) and W = sum(w), - // I = P / W has grad I = (grad P - I * grad W) / W; above the threshold the output is I / |I| - // with gradient (grad I - out * grad|I|) / |I|, below it grad I / (1 - normalization). The - // gradient is discontinuous where |I| crosses the threshold, and where a cell's weight leaves - // its support (distance 1.4998): there grad w jumps from -4 * 0.01111 * v (length about 0.067) to 0. The value is - // continuous at both. - // - // Pruning is exact: per axis, |local - g| - 0.5 (floored at 0) lower-bounds |v| for any jitter, - // so a cell whose summed square is at least 2.25 has weight exactly 0 (exp(-4.5) = 0.011109 < - // 0.01111) and contributes nothing to any sum. Only the internal overload can switch the prune - // off; PhacelleTests uses it to pin bit equality. About 14 of 64 cells survive on average. - public static CultPhasor phacelle(float3 p, float3 side, float offset, float normalization) => - phacelle(p, side, offset, normalization, true); - - internal static CultPhasor phacelle(float3 p, float3 side, float offset, float normalization, bool prune) - { - var cell = floor(p); - var local = p - cell; - var phaseOffset = offset * TAU; - var sumW = 0.0f; - var sumCos = 0.0f; - var sumSin = 0.0f; - var gradW = new float3(0.0f, 0.0f, 0.0f); - var gradCos = new float3(0.0f, 0.0f, 0.0f); - var gradSin = new float3(0.0f, 0.0f, 0.0f); - - for (var dz = -1; dz <= 2; dz++) - { - var gz = max(0.0f, abs(local.z - (float)dz) - 0.5f); - for (var dy = -1; dy <= 2; dy++) - { - var gy = max(0.0f, abs(local.y - (float)dy) - 0.5f); - for (var dx = -1; dx <= 2; dx++) - { - var gx = max(0.0f, abs(local.x - (float)dx) - 0.5f); - if (prune && gx * gx + gy * gy + gz * gz >= 2.25f) - continue; - - var gridStep = new float3(dx, dy, dz); - var hash = pcg3d(int3(cell + gridStep)); - var jitter = new float3(cellular_unit(hash.x) - 0.5f, cellular_unit(hash.y) - 0.5f, cellular_unit(hash.z) - 0.5f); - var v = local - gridStep - jitter; - var falloff = exp(-2.0f * dot(v, v)); - var w = max(0.0f, falloff - 0.01111f); - var dw = w > 0.0f ? (-4.0f * falloff) * v : new float3(0.0f, 0.0f, 0.0f); - var phase = dot(v, side) + phaseOffset; - var c = cos(phase); - var s = sin(phase); - - sumW += w; - sumCos += w * c; - sumSin += w * s; - gradW += dw; - gradCos += dw * c - (w * s) * side; - gradSin += dw * s + (w * c) * side; - } - } - } - - var invW = 1.0f / sumW; - var rawCos = sumCos * invW; - var rawSin = sumSin * invW; - var gradRawCos = (gradCos - rawCos * gradW) * invW; - var gradRawSin = (gradSin - rawSin * gradW) * invW; - var rawLength = sqrt(rawCos * rawCos + rawSin * rawSin); - var floorLength = 1.0f - normalization; - - if (rawLength > floorLength) - { - var outCos = rawCos / rawLength; - var outSin = rawSin / rawLength; - var gradLength = (rawCos * gradRawCos + rawSin * gradRawSin) / rawLength; - return new CultPhasor( - new float4((gradRawCos - outCos * gradLength) / rawLength, outCos), - new float4((gradRawSin - outSin * gradLength) / rawLength, outSin)); - } - - return new CultPhasor( - new float4(gradRawCos / floorLength, rawCos / floorLength), - new float4(gradRawSin / floorLength, rawSin / floorLength)); - } - public static float value_noise(float2 position) { var cell = floor(position); diff --git a/packages/cultmath/tests/CultMath.Tests/HlslMirrorTests.cs b/packages/cultmath/tests/CultMath.Tests/HlslMirrorTests.cs index 2b695fcb..6b69a908 100644 --- a/packages/cultmath/tests/CultMath.Tests/HlslMirrorTests.cs +++ b/packages/cultmath/tests/CultMath.Tests/HlslMirrorTests.cs @@ -7,7 +7,7 @@ public sealed class HlslMirrorTests [Fact] public void HlslMirrorDoesNotPublishGeometryKernels() { - var source = File.ReadAllText(Path.Combine(GetCultMathRoot(), "shaders", "CultMath.hlsl")); + var source = HlslSourceCompatibilityTests.ReadShaderSource(GetCultMathRoot()); Assert.DoesNotContain("AdvancedErosionFilter.hlsl", source); Assert.DoesNotContain("Planetary.hlsl", source); Assert.DoesNotContain("spherical_erosion", source); @@ -17,7 +17,7 @@ public sealed class HlslMirrorTests [Fact] public void HlslMirrorPublishesCultMathPrimitives() { - var include = File.ReadAllText(Path.Combine(GetCultMathRoot(), "shaders", "CultMath.hlsl")); + var include = HlslSourceCompatibilityTests.ReadShaderSource(GetCultMathRoot()); var requiredSymbols = new[] { "cultmath_radians", @@ -62,6 +62,17 @@ public sealed class HlslMirrorTests Assert.DoesNotContain("cultmath_normalize", include); } + [Theory] + [InlineData("CultMath.hlsl")] + [InlineData("CultMath.Phacelle.hlsl")] + public void UnityPackageShaderIsIdenticalToTheCanonicalShader(string name) + { + var root = GetCultMathRoot(); + Assert.Equal( + File.ReadAllText(Path.Combine(root, "shaders", name)).ReplaceLineEndings("\n"), + File.ReadAllText(Path.Combine(root, "unity", "org.gamecult.cultmath", "Shaders", name)).ReplaceLineEndings("\n")); + } + private static string GetCultMathRoot() { var directory = new DirectoryInfo(AppContext.BaseDirectory); diff --git a/packages/cultmath/tests/CultMath.Tests/HlslSourceCompatibilityTests.cs b/packages/cultmath/tests/CultMath.Tests/HlslSourceCompatibilityTests.cs index 2cf737df..2cb8ae49 100644 --- a/packages/cultmath/tests/CultMath.Tests/HlslSourceCompatibilityTests.cs +++ b/packages/cultmath/tests/CultMath.Tests/HlslSourceCompatibilityTests.cs @@ -163,6 +163,33 @@ public sealed class HlslSourceCompatibilityTests } } + /// + /// The dense sweep almost never meets a cell whose box bound is just under the 2.25 cut and whose + /// weight is still nonzero: a mirror cut of 2.245, 2.24 or 2.2 survives it. These points sit next + /// to such cells (), so the mirror must match C# bit for + /// bit, and must match the unpruned C# sum (the shader has no prune switch, so that is the + /// independent statement that the cut is exact). + /// + [Fact] + public void PhacelleMirrorMatchesCSharpBitForBitNextToCellsJustInsideTheCut() + { + var (assembly, errors) = CompileShaderMirror(); + Assert.True(assembly is not null, string.Join(Environment.NewLine, errors)); + var shaderType = assembly!.GetType("CultMathHlsl.HlslShader")!; + var shader = Activator.CreateInstance(shaderType); + var mirror = ShaderFunction(shaderType, "cultmath_phacelle", typeof(float3), typeof(float3), typeof(float), typeof(float))!; + + foreach (var p in PhacelleTests.PruneCutPoints) + { + Assert.True(PhacelleTests.HasCellJustInsideTheCut(p), $"no cell just inside the cut at {p}"); + var actual = Values(mirror.Invoke(shader, new object[] { p, PhacelleTests.PruneCutSide, PhacelleTests.PruneCutOffset, PhacelleTests.PruneCutNormalization })!); + var pruned = Values(phacelle(p, PhacelleTests.PruneCutSide, PhacelleTests.PruneCutOffset, PhacelleTests.PruneCutNormalization)); + var unpruned = Values(phacelle(p, PhacelleTests.PruneCutSide, PhacelleTests.PruneCutOffset, PhacelleTests.PruneCutNormalization, false)); + Assert.True(BitwiseEqual(pruned, actual), $"mirror differs from C# at p = {p}"); + Assert.True(BitwiseEqual(unpruned, actual), $"mirror differs from the unpruned C# sum at p = {p}"); + } + } + /// /// A hand-built pair of structs standing in for a mirror struct return and its C# counterpart /// (invariant 8's one struct return shape, e.g. ): same field shape, @@ -283,6 +310,18 @@ public sealed class HlslSourceCompatibilityTests ? (float.IsNaN(a) && float.IsNaN(b)) || BitConverter.SingleToInt32Bits(a) == BitConverter.SingleToInt32Bits(b) : p.First.Value.Equals(p.Second.Value))); + /// + /// The text of CultMath.hlsl with each #include "name" replaced in place by the + /// included file from the same directory, as the shader compiler would resolve it. Only the + /// package's own includes exist, so a missing file is an error rather than a skip. + /// + internal static string ReadShaderSource(string cultMathRoot) + { + var shaders = Path.Combine(cultMathRoot, "shaders"); + return Regex.Replace(File.ReadAllText(Path.Combine(shaders, "CultMath.hlsl")), @"(?m)^#include ""([^""]+)""[ \t]*\r?$", + match => File.ReadAllText(Path.Combine(shaders, match.Groups[1].Value))); + } + /// The documented HLSL-to-C# transformations, and nothing else. internal static string TransformHlslToCSharp(string hlsl) { @@ -311,7 +350,7 @@ public sealed class HlslSourceCompatibilityTests private static (Assembly? Assembly, IReadOnlyList Errors) CompileShaderMirror() { - var hlsl = File.ReadAllText(Path.Combine(FindCultMathRoot(), "shaders", "CultMath.hlsl")); + var hlsl = ReadShaderSource(FindCultMathRoot()); var tree = CSharpSyntaxTree.ParseText(TransformHlslToCSharp(hlsl)); var references = ((string)AppContext.GetData("TRUSTED_PLATFORM_ASSEMBLIES")!) .Split(Path.PathSeparator) diff --git a/packages/cultmath/tests/CultMath.Tests/PhacelleTests.cs b/packages/cultmath/tests/CultMath.Tests/PhacelleTests.cs index 15d1a89a..b64f70b1 100644 --- a/packages/cultmath/tests/CultMath.Tests/PhacelleTests.cs +++ b/packages/cultmath/tests/CultMath.Tests/PhacelleTests.cs @@ -144,7 +144,7 @@ public sealed class PhacelleTests { // Above the threshold the output is I / |I|; below it, I / (1 - normalization). The gradient // is discontinuous where |I| crosses the threshold, and at the edge of each cell's weight support - // (distance 1.4998, where max(0, ...) starts: a jump of about 4 d 0.01111 in grad w), so a band + // (distance 1.49998, where max(0, ...) starts: a jump of 4 d 0.01111 in grad w), so a band // around each is excluded (by the // reference's raw length, since the output alone cannot say how close it is). const float normalization = 0.5f; @@ -265,6 +265,71 @@ public sealed class PhacelleTests } } + // Points next to a cell whose jitter sits at the far corner of its cube: the cell's box bound is + // just under the 2.25 cut (2.2451 to 2.25) while its real distance is still under the support + // radius (d^2 < 2.24995), so its weight is small but nonzero and a prune that cuts even slightly + // low (2.245, 2.24, 2.2) drops a contribution and changes output bits. Random points meet such + // a cell about once in 10^4 evaluations, which is why the random sweep above let those cuts + // through. Each was found once by an offline search (cells K in [-12, 12]^3 whose jitter on one + // axis lies within 0.0008 of the cube face; p is the feature moved 1.4996 out along that axis, + // so |p - feature| = 1.4996 and the bound is about 1.4988). The witness check below re-derives + // that property from the hash, so a change to pcg3d fails loudly instead of degenerating. + // Shared with the HLSL mirror test. + internal static readonly float3[] PruneCutPoints = + { + new(1.77850008f, -2.16020727f, -0.9991166f), + new(1.09346282f, -2.37882328f, -3.00082445f), + new(-6.39144945f, -3.99953747f, -0.610023975f), + new(-0.99940908f, 4.16302729f, 4.07277536f), + new(3.99912024f, 7.21890545f, 1.09053993f), + new(5.43959713f, 1.99937367f, -6.14929914f), + }; + + internal static readonly float3 PruneCutSide = new(2.1f, -3.7f, 1.3f); + internal const float PruneCutOffset = 0.3f; + internal const float PruneCutNormalization = 0.5f; + + // True when some cell of the 4x4x4 window has a box bound in [2.2451, 2.25) and a positive weight. + internal static bool HasCellJustInsideTheCut(float3 p) + { + double fx = Math.Floor(p.x), fy = Math.Floor(p.y), fz = Math.Floor(p.z); + double lx = p.x - fx, ly = p.y - fy, lz = p.z - fz; + for (var dz = -1; dz <= 2; dz++) + for (var dy = -1; dy <= 2; dy++) + for (var dx = -1; dx <= 2; dx++) + { + var gx = Math.Max(0.0, Math.Abs(lx - dx) - 0.5); + var gy = Math.Max(0.0, Math.Abs(ly - dy) - 0.5); + var gz = Math.Max(0.0, Math.Abs(lz - dz) - 0.5); + var bound = gx * gx + gy * gy + gz * gz; + if (bound < 2.2451 || bound >= 2.25) + continue; + + var hash = math.pcg3d(new int3((int)fx + dx, (int)fy + dy, (int)fz + dz)); + double vx = lx - dx - (math.cellular_unit(hash.x) - 0.5); + double vy = ly - dy - (math.cellular_unit(hash.y) - 0.5); + double vz = lz - dz - (math.cellular_unit(hash.z) - 0.5); + if (Math.Exp(-2.0 * (vx * vx + vy * vy + vz * vz)) > 0.01111) + return true; + } + + return false; + } + + [Fact] + public void PrunedOutputIsBitEqualToTheUnprunedSumNextToCellsJustInsideTheCut() + { + foreach (var p in PruneCutPoints) + { + Assert.True(HasCellJustInsideTheCut(p), $"no cell just inside the cut at {p}: the fixture no longer exercises the prune"); + var pruned = math.phacelle(p, PruneCutSide, PruneCutOffset, PruneCutNormalization); + var unpruned = math.phacelle(p, PruneCutSide, PruneCutOffset, PruneCutNormalization, false); + + AssertBitEqual(unpruned.cos, pruned.cos); + AssertBitEqual(unpruned.sin, pruned.sin); + } + } + private static void AssertBitEqual(float4 expected, float4 actual) { Assert.Equal(BitConverter.SingleToInt32Bits(expected.x), BitConverter.SingleToInt32Bits(actual.x)); diff --git a/packages/cultmath/unity/org.gamecult.cultmath/LICENSE.md b/packages/cultmath/unity/org.gamecult.cultmath/LICENSE.md index 5e50f2d6..7e18d22d 100644 --- a/packages/cultmath/unity/org.gamecult.cultmath/LICENSE.md +++ b/packages/cultmath/unity/org.gamecult.cultmath/LICENSE.md @@ -4,7 +4,7 @@ CultMath is licensed under the Mozilla Public License 2.0 (`MPL-2.0`). The complete license text is available from the Mozilla Foundation at . Source and modification history for the -precompiled assembly in this package are available in the GameCult/CultMath -repository at the corresponding package tag. +precompiled assembly in this package are available in the GameCult/CultLib +repository (`packages/cultmath`) at the corresponding package tag. Third-party code retains its original license. See `THIRD-PARTY-NOTICES.md`. diff --git a/packages/cultmath/unity/org.gamecult.cultmath/README.md b/packages/cultmath/unity/org.gamecult.cultmath/README.md index 20e1840c..4981c54c 100644 --- a/packages/cultmath/unity/org.gamecult.cultmath/README.md +++ b/packages/cultmath/unity/org.gamecult.cultmath/README.md @@ -2,7 +2,7 @@ This is CultMath's Unity 2021.3-compatible Git-UPM surface. It contains the precompiled `CultMath.dll`, portable symbols, the numeric -`Shaders/CultMath.hlsl` mirror, and the `CultMath.UnityBridge` assembly. +`Shaders/CultMath.hlsl` mirror (with its `CultMath.Phacelle.hlsl` include), and the `CultMath.UnityBridge` assembly. `CultMath.UnityBridge` is the only C# source Unity compiles here. It holds the UnityEngine conversions the engine-free core cannot declare: `ToCultMath()` on diff --git a/packages/cultmath/unity/org.gamecult.cultmath/Shaders/CultMath.Phacelle.hlsl b/packages/cultmath/unity/org.gamecult.cultmath/Shaders/CultMath.Phacelle.hlsl new file mode 100644 index 00000000..1db0a2fc --- /dev/null +++ b/packages/cultmath/unity/org.gamecult.cultmath/Shaders/CultMath.Phacelle.hlsl @@ -0,0 +1,99 @@ +#ifndef CULTMATH_PHACELLE_HLSL +#define CULTMATH_PHACELLE_HLSL + +// This Source Code Form is subject to the terms of the Mozilla Public License, v. 2.0. If a copy of +// the MPL was not distributed with this file, You can obtain one at https://mozilla.org/MPL/2.0/. +// +// Phacelle noise. Copyright (c) 2025 Rune Skovbo Johansen (Shadertoy t3dyWl; MPL-2.0), read from the +// C# port PhacelleNoise in lpmitchell/AdvancedTerrainErosion, converted to burstable C# by Luke +// Mitchell (2026). Modified by GameCult: 3D cells, a caller-supplied stripe wave vector, an exact +// gradient and exact pruning. This file is an MPL-2.0 file; CultMath.hlsl, which includes it after +// its own definitions of CULTMATH_TAU and cultmath_pcg3d, is MIT (see THIRD-PARTY-NOTICES.md). + +// Invariant 8's struct return shape for cultmath_phacelle, compared field by field and bit for bit +// against CultMath.CultPhasor, same field order. +struct CultPhasor +{ + float4 cos; + float4 sin; +}; + +// Johansen's Phacelle noise on 3D cells. See math.phacelle's comment (math.Phacelle.cs) for the stripe wave +// vector, the weight, the normalization, the exact gradient and why the prune is exact. Mirrors +// the C# body operation for operation; the C#-only internal overload that switches the prune off +// has no shader twin. +CultPhasor cultmath_phacelle(float3 p, float3 side, float offset, float normalization) +{ + float3 cell = floor(p); + float3 local = p - cell; + float phaseOffset = offset * CULTMATH_TAU; + float sumW = 0.0; + float sumCos = 0.0; + float sumSin = 0.0; + float3 gradW = float3(0.0, 0.0, 0.0); + float3 gradCos = float3(0.0, 0.0, 0.0); + float3 gradSin = float3(0.0, 0.0, 0.0); + + for (int dz = -1; dz <= 2; dz++) + { + float gz = max(0.0, abs(local.z - (float)dz) - 0.5); + for (int dy = -1; dy <= 2; dy++) + { + float gy = max(0.0, abs(local.y - (float)dy) - 0.5); + for (int dx = -1; dx <= 2; dx++) + { + float gx = max(0.0, abs(local.x - (float)dx) - 0.5); + if (gx * gx + gy * gy + gz * gz >= 2.25) + continue; + + float3 gridStep = float3(dx, dy, dz); + int3 hash = cultmath_pcg3d(int3(cell + gridStep)); + float3 jitter = float3( + ((uint)hash.x >> 8) * (1.0 / 16777216.0) - 0.5, + ((uint)hash.y >> 8) * (1.0 / 16777216.0) - 0.5, + ((uint)hash.z >> 8) * (1.0 / 16777216.0) - 0.5); + float3 v = local - gridStep - jitter; + float falloff = exp(-2.0 * dot(v, v)); + float w = max(0.0, falloff - 0.01111); + float3 dw = w > 0.0 ? (-4.0 * falloff) * v : float3(0.0, 0.0, 0.0); + float phase = dot(v, side) + phaseOffset; + float c = cos(phase); + float s = sin(phase); + + sumW += w; + sumCos += w * c; + sumSin += w * s; + gradW += dw; + gradCos += dw * c - (w * s) * side; + gradSin += dw * s + (w * c) * side; + } + } + } + + float invW = 1.0 / sumW; + float rawCos = sumCos * invW; + float rawSin = sumSin * invW; + float3 gradRawCos = (gradCos - rawCos * gradW) * invW; + float3 gradRawSin = (gradSin - rawSin * gradW) * invW; + float rawLength = sqrt(rawCos * rawCos + rawSin * rawSin); + float floorLength = 1.0 - normalization; + + CultPhasor result; + if (rawLength > floorLength) + { + float outCos = rawCos / rawLength; + float outSin = rawSin / rawLength; + float3 gradLength = (rawCos * gradRawCos + rawSin * gradRawSin) / rawLength; + result.cos = float4((gradRawCos - outCos * gradLength) / rawLength, outCos); + result.sin = float4((gradRawSin - outSin * gradLength) / rawLength, outSin); + } + else + { + result.cos = float4(gradRawCos / floorLength, rawCos / floorLength); + result.sin = float4(gradRawSin / floorLength, rawSin / floorLength); + } + + return result; +} + +#endif diff --git a/packages/cultmath/unity/org.gamecult.cultmath/Shaders/CultMath.Phacelle.hlsl.meta b/packages/cultmath/unity/org.gamecult.cultmath/Shaders/CultMath.Phacelle.hlsl.meta new file mode 100644 index 00000000..9c074cb5 --- /dev/null +++ b/packages/cultmath/unity/org.gamecult.cultmath/Shaders/CultMath.Phacelle.hlsl.meta @@ -0,0 +1,7 @@ +fileFormatVersion: 2 +guid: 5b1c7e0a94d24f6c8a3e2d17f60b9c48 +ShaderIncludeImporter: + externalObjects: {} + userData: + assetBundleName: + assetBundleVariant: diff --git a/packages/cultmath/unity/org.gamecult.cultmath/Shaders/CultMath.hlsl b/packages/cultmath/unity/org.gamecult.cultmath/Shaders/CultMath.hlsl index d0dd22ae..af2d3741 100644 --- a/packages/cultmath/unity/org.gamecult.cultmath/Shaders/CultMath.hlsl +++ b/packages/cultmath/unity/org.gamecult.cultmath/Shaders/CultMath.hlsl @@ -503,91 +503,7 @@ CultCellular cultmath_cellular(float3 p) return result; } -// Invariant 8's struct return shape for cultmath_phacelle, compared field by field and bit for bit -// against CultMath.CultPhasor, same field order. -struct CultPhasor -{ - float4 cos; - float4 sin; -}; - -// Johansen's Phacelle noise on 3D cells. See math.phacelle's comment (math.cs) for the stripe wave -// vector, the weight, the normalization, the exact gradient and why the prune is exact. Mirrors -// the C# body operation for operation; the C#-only internal overload that switches the prune off -// has no shader twin. -CultPhasor cultmath_phacelle(float3 p, float3 side, float offset, float normalization) -{ - float3 cell = floor(p); - float3 local = p - cell; - float phaseOffset = offset * CULTMATH_TAU; - float sumW = 0.0; - float sumCos = 0.0; - float sumSin = 0.0; - float3 gradW = float3(0.0, 0.0, 0.0); - float3 gradCos = float3(0.0, 0.0, 0.0); - float3 gradSin = float3(0.0, 0.0, 0.0); - - for (int dz = -1; dz <= 2; dz++) - { - float gz = max(0.0, abs(local.z - (float)dz) - 0.5); - for (int dy = -1; dy <= 2; dy++) - { - float gy = max(0.0, abs(local.y - (float)dy) - 0.5); - for (int dx = -1; dx <= 2; dx++) - { - float gx = max(0.0, abs(local.x - (float)dx) - 0.5); - if (gx * gx + gy * gy + gz * gz >= 2.25) - continue; - - float3 gridStep = float3(dx, dy, dz); - int3 hash = cultmath_pcg3d(int3(cell + gridStep)); - float3 jitter = float3( - ((uint)hash.x >> 8) * (1.0 / 16777216.0) - 0.5, - ((uint)hash.y >> 8) * (1.0 / 16777216.0) - 0.5, - ((uint)hash.z >> 8) * (1.0 / 16777216.0) - 0.5); - float3 v = local - gridStep - jitter; - float falloff = exp(-2.0 * dot(v, v)); - float w = max(0.0, falloff - 0.01111); - float3 dw = w > 0.0 ? (-4.0 * falloff) * v : float3(0.0, 0.0, 0.0); - float phase = dot(v, side) + phaseOffset; - float c = cos(phase); - float s = sin(phase); - - sumW += w; - sumCos += w * c; - sumSin += w * s; - gradW += dw; - gradCos += dw * c - (w * s) * side; - gradSin += dw * s + (w * c) * side; - } - } - } - - float invW = 1.0 / sumW; - float rawCos = sumCos * invW; - float rawSin = sumSin * invW; - float3 gradRawCos = (gradCos - rawCos * gradW) * invW; - float3 gradRawSin = (gradSin - rawSin * gradW) * invW; - float rawLength = sqrt(rawCos * rawCos + rawSin * rawSin); - float floorLength = 1.0 - normalization; - - CultPhasor result; - if (rawLength > floorLength) - { - float outCos = rawCos / rawLength; - float outSin = rawSin / rawLength; - float3 gradLength = (rawCos * gradRawCos + rawSin * gradRawSin) / rawLength; - result.cos = float4((gradRawCos - outCos * gradLength) / rawLength, outCos); - result.sin = float4((gradRawSin - outSin * gradLength) / rawLength, outSin); - } - else - { - result.cos = float4(gradRawCos / floorLength, rawCos / floorLength); - result.sin = float4(gradRawSin / floorLength, rawSin / floorLength); - } - - return result; -} +#include "CultMath.Phacelle.hlsl" float cultmath_value_noise(float2 position) { diff --git a/packages/cultmath/unity/org.gamecult.cultmath/THIRD-PARTY-NOTICES.md b/packages/cultmath/unity/org.gamecult.cultmath/THIRD-PARTY-NOTICES.md index 73f2461a..625d63a9 100644 --- a/packages/cultmath/unity/org.gamecult.cultmath/THIRD-PARTY-NOTICES.md +++ b/packages/cultmath/unity/org.gamecult.cultmath/THIRD-PARTY-NOTICES.md @@ -32,5 +32,10 @@ gradient and the exact pruning are new here. - Copyright (c) 2025 Rune Skovbo Johansen. - Upstream: -- License: MPL-2.0 (). CultMath's Unity package - is MPL-2.0, so the licence is compatible. +- License: MPL-2.0 (). MPL-2.0 is file-level, so it + covers exactly the files that carry the port and nothing else: + `src/CultMath/math.Phacelle.cs`, `src/CultMath/CultPhasor.cs`, + `shaders/CultMath.Phacelle.hlsl` and the Unity copy of that include. Each + carries the MPL-2.0 header and this attribution. The rest of CultMath is MIT, + so the `GameCult.Math` NuGet package declares `MIT AND MPL-2.0` and ships this + file. The Unity package as a whole is MPL-2.0. From 03b1df7cf5691cd248a81d4871ccb950aa0c8f73 Mon Sep 17 00:00:00 2001 From: Meta Vens Date: Wed, 30 Sep 2026 00:42:27 +0200 Subject: [PATCH 07/16] Ack map pass 3: FORWARD-TSN (Cut F), receipts port, Cut 1c ruled, Cut K, Cut D Maps the four 2026-09-30 rulings. Cut 2 merges only with Cut F (Soul: expiring reliable sets pin the bound). Adds Cut D (a failed send to one peer no longer aborts a document server poll) and C# Cut 2c. Opens Q-A7, Q-A8, Q-A9; Q-A7 and Q-A9 block Cut F. Co-Authored-By: Claude Opus 5.5 --- docs/cultnet-rudp-ack-cut.md | 1162 ++++++++++++++++++++++++++-------- 1 file changed, 906 insertions(+), 256 deletions(-) diff --git a/docs/cultnet-rudp-ack-cut.md b/docs/cultnet-rudp-ack-cut.md index 30d6089f..9f07341c 100644 --- a/docs/cultnet-rudp-ack-cut.md +++ b/docs/cultnet-rudp-ack-cut.md @@ -1,50 +1,64 @@ # CultNet RUDP: Acknowledged means delivered, one watermark owns ordered delivery, and a write belongs to its session -Status: cut map, Imagination pass 2, 2026-09-30, committed by Self. Pass 2 re-maps every cut against -`main` at `3bf1c0c` and applies the operator rulings below. +Status: cut map, Imagination pass 3, 2026-09-30. Pass 3 maps the four rulings below, re-maps Cuts 1c and +4, and adds Cut F (FORWARD-TSN), Cuts 6a-6c (receipts), Cut 7 (expiry, if Q-A8), Cut K (Kotlin), Cut 2c (the +C# fragment bound) and Cut D (a failed send to one peer stays that peer's loss). The text of Cuts 1, 1b, 2 +and 3 is pass 2's; Hands are executing Cuts 1 and 1b (`hands/ack-cut1`) and Cut 2 (`hands/ack-cut2`) from +it. One thing moves for them: Cut 2 merges only together with Cut F (the Cut 2 Soul finding below). **Rulings (operator, 2026-09-30).** -- **Q-A1 (a):** Delivered is derived at the sender from a correct receiver's cumulative watermark. The wire - does not change. +- **Q-A1 (a):** Delivered is derived at the sender from a correct receiver's cumulative watermark. The + acknowledgement wire does not change. (Cut F adds one notice to the wire; it carries abandonment, not + delivery.) - **Q-A2:** "I think this is telling us we don't actually need both received and delivered." - There is one public meaning: Acknowledged means delivered. - Receipts go Pending, then Acknowledged (delivered), or Invalidated. No public Received state exists. - Retransmission still stops on each packet's own ack, so resending stays bounded. - Flush waits for Acknowledged. -- **Scope added from the stray-packet Soul passes 3 and 4:** session generation. A restarted client on the - same endpoint and payload is locked out. A reconnect on the same transport treats a new server session's - frames as duplicates. The ending paths disagree about owed writes. One rule: a write belongs to the - session it was issued in. Pass 2 maps this as **Cut 1b** (the ending rule) and **Cut 1c** (how a new +- **Scope added from the stray-packet Soul passes 3 and 4:** session generation. One rule: a write belongs + to the session it was issued in. Mapped as **Cut 1b** (the ending rule) and **Cut 1c** (how a new generation is recognised), separate from Cut 1 (§8 says why). -- **Q-A3 B (ruled after pass 2):** FORWARD-TSN now (RFC 3758). On expiry the sender sends an - abandonment notice (Ack-typed, flag `0x10`, ignored by old peers); new receivers treat the sequence as - received with no frame and acknowledge it; the notice's resend has its own bound. This is a new wire - cut in every receiver, landing before Cut 4. The recommendation was A; the operator chose B. -- **Q-A4 A (ruled after pass 2):** port receipts (Pending, Acknowledged, Invalidated, `WaitAcknowledged`) - to C#, TypeScript, Python and Kotlin in this campaign, after Cut 4. The recommendation was B. -- **Q-A5 A (ruled after pass 2):** Kotlin is in, as trailing Cut K, verified on a Kotlin image added to - Yggdrasil first. -- **Q-A6 A (ruled after pass 2):** the Connect's sequence identifies the session; the default initial - sequence becomes random in [1, 2^31), as TCP's ISN. No wire change; the public default changes. -- **Pending:** the cut sections below still read these four as open. Imagination pass 3 maps the - FORWARD-TSN cut, the receipts port and the Kotlin image, and re-sequences Cuts 1c, 4 and K. +- **Q-A3 B:** FORWARD-TSN (RFC 3758) now. On expiry the sender sends an abandonment notice, Ack-typed with + flag `0x10` so old peers ignore it. New receivers treat the sequence as received with no frame and + acknowledge it. The notice's resend has its own bound. A new wire cut in every receiver, before Cut 4. + Mapped as **Cut F** (§6). The recommendation was A; the operator chose B. (Supersedes pass 2's Cut 4 + expiry half, which assumed A.) +- **Q-A4 A:** port per-send receipts (Pending, Acknowledged, Invalidated, and a wait for Acknowledged) to C#, + TypeScript, Python and Kotlin in this campaign, after Cut 4. Mapped as **Cuts 6a-6c** and **K4** (§9). + The recommendation was B. +- **Q-A5 A:** Kotlin is in, as trailing **Cut K**, verified on a Kotlin image added to Yggdrasil first + (Cut K0). +- **Q-A6 A:** the Connect's sequence identifies the session; the default initial sequence becomes random in + [1, 2^31), as TCP's ISN. No wire change; the public default changes. Cut 1c is no longer blocked on a + question. +- **Cut 2 Soul finding (2026-09-30):** Cut 2's never-evict rule refuses a session whose sender expires + fragmented reliable frames (the default Rust `media` channel). Pass 3 absorbs it: Cut F's receiver drops + every incomplete fragment set a notice covers; Cut 2 merges only together with Cut F; C# gets the bound + it never had (Cut 2c). What a receiver does with a sender that never sends notices is **Q-A9**. +- **Open:** Q-A7 (the notice names one sequence, not a cumulative point), Q-A8 (reliable expiry in the + four runtimes that lack it) and Q-A9 (a capability bit, so Cut 2's rule applies only to peers that send + notices). Each has a recommendation. Q-A7 and Q-A9 block Cut F and with it Cut 2's merge; Q-A8 blocks + only Cut 7. -- **Body.** CultLib `main` at `3bf1c0c` ("Merge RUDP stray-packet hardening across all four runtimes"). - Every `file:line` is against it unless it names another revision. - - Since pass 1's base `d85877a7`, only `rudp.rs` and `rudp.ts` moved among the transport files. - `CultNetTransport.cs`, `transport.py`, `CultMesh.kt`, `cultmesh-rs`, `cultmesh-ts`, `peer.ts`, the two - TS publishers and `docs/cultnet-transport-parity.md` are byte-identical to `d85877a7`. - - `rudp.rs` moved +6 lines from `:283` and +12 from `:596`. `rudp.ts` moved only after `:824`. - - Odin pins `3bf1c0c` (`Odin crates/odin-core/Cargo.toml:14-17`). -- **Base of every cut.** `main` at `3bf1c0c` or later. +- **Body.** CultLib `main` at `e9469cb3`. Every transport file is byte-identical to `3bf1c0c` there + (`git diff 3bf1c0c e9469cb3 -- src packages` is empty), so every `file:line` below is against both unless + it names another revision. + - Odin `origin/main` (`5c37860`) pins `3bf1c0ce` (`crates/odin-core/Cargo.toml:14-17`). A stale local + checkout at `379b826` still shows `a8aeddac`; read Odin's pin from `origin/main`. + - Pass 3 probes: `3e2c7efb` (FN) and `c6f1012c` (CAP) on the Yggdrasil mirror `probe3` (§ Probe + evidence, pass 3). +- **Base of every cut.** `main` at `e9469cb3` or later. - **Scope.** The RUDP session and socket transports in all five runtimes: C# `src/GameCult.Networking/CultNetTransport.cs`, Rust `packages/cultnet-rs/src/rudp.rs` (plus `packages/cultmesh-rs`), TypeScript `packages/cultnet-ts/src/rudp.ts` (plus the two hand-rolled - publishers), Python `packages/cultnet-py/src/cultnet_py/transport.py`, and Kotlin - `packages/cultmesh-kotlin/src/main/kotlin/org/gamecult/cultmesh/CultMesh.kt`. **Q-A5** decides whether - Kotlin is in. Until it is ruled, every cut is mapped for four runtimes and Kotlin trails (§ Sequencing). -- **Wire.** This map changes no byte on the wire (§1). Cut 1c changes the default initial sequence, which - is a value, not a format. + publishers and `cultmesh-ts`'s document server), Python `packages/cultnet-py/src/cultnet_py/transport.py` + (plus `cultmesh-py`'s server), and Kotlin + `packages/cultmesh-kotlin/src/main/kotlin/org/gamecult/cultmesh/CultMesh.kt` (Cut K). +- **Wire.** One addition, in Cut F: an Ack packet with flag bit 4 (`0x10`) is an abandonment notice for the + sequence in its sequence field. Every current decoder ignores the bit and every current session ignores + an Ack's sequence field (P7, re-probed as FN in pass 3 in all five runtimes). If Q-A9 is B, the same bit + on Connect and Accept advertises support (probe CAP: old peers ignore it there too). Nothing else changes + on the wire. Cut 1c changes the default initial sequence, which is a value, not a format. ## The ends (operator, 2026-09-30) @@ -117,10 +131,17 @@ And places where the sender loses track, or is locked out, without the receiver' |---|---|---|---| | G1 | A restarted client on the same endpoint and Connect payload gets an Ack, never an Accept, and stays unconnected | Rust hub, C# listener and C# server mode, TS and Python server mode (Rust server mode resets on every Connect instead) | Cut 1c | | G1b | C# and Python server-mode transports drop a Connect from any endpoint but the first, so a client that restarts on a new port is locked out for good | C#, Python | Cut 1c | -| P2 | An expired reliable packet leaves a permanent hole. Ordered frames after it are held forever, and `flush_reliable` succeeds (outstanding 0) | Rust (the only runtime with reliable expiry) | Cut 4 reports it honestly; **Q-A3** decides the real fix | +| P2 | An expired reliable packet leaves a permanent hole. Ordered frames after it are held forever, and `flush_reliable` succeeds (outstanding 0) | Rust (the only runtime with reliable expiry) | Cut F (FORWARD-TSN, Q-A3 B); Cut 4 reports it honestly where a receiver never acknowledges the notice | | P9 | A receipt for an expiring send reads `Acknowledged` once the packet expires unacknowledged | Rust | Cut 4 | | P10 | `send_reliable` with a full window queues the frame, then returns an error ("receipts require a non-empty reliable packet set"), so a caller that retries sends it twice | Rust | Cut 4 | +And two faults found by Soul passes during pass 3: + +| # | Fault | Where | Fixed by | +|---|---|---|---| +| D1 | One peer whose datagrams cannot be sent fails every poll for every peer: the first failed `send_to` aborts `maintain` and `poll_once` (Odin Soul pass) | cultmesh-rs document server, Rust hub, C# listener; the cultmesh-py server's thread dies | Cut D (§10) | +| F2 | After Cut 2, a sender with reliable expiry pins one reliable fragment set per lossy frame, and the 65th refuses the session (Cut 2 Soul pass, default Rust `media`) | Rust, TS, Python receivers with Cut 2 | Cut F releases the sets; Cut 2 merges with it; Q-A9 for senders without notices | + ## Probe evidence @@ -170,6 +191,23 @@ Pass 2 (base `3bf1c0c`; mirror `CultLib-imag-ack2`): | G3 | Client sends one ordered write that is lost; the session ends; the client connects to a new server; the client's resends run | n/a | Owed after the end, and **delivered in the next session**: C#, TS, Python on peer Disconnect, local disconnect and timeout; Rust on local disconnect and timeout. Rust on peer Disconnect: owed 0, nothing delivered (`5f1e883d`) | Not probed | | G3R | Rust socket: `send_reliable`, the write is lost, `disconnect()`, `connect()` on the same transport to a new peer | n/a | Receipt `Pending` after the disconnect; the write is delivered to the new peer; **receipt `Acknowledged`** (the Soul R1 shape) | Not probed | +Pass 3 (base `e9469cb3`, transport files identical to `3bf1c0c`; mirror `probe3`, commits `3e2c7efb` for FN +and `c6f1012c` for CAP): + +- `packages/cultnet-rs/tests/probe_fn.rs`, `packages/cultnet-ts/test/probe-fn.test.ts`, + `packages/cultnet-py/tests/probe_fn.py`, `tests/GameCult.Networking.Tests/ProbeFnTests.cs` and + `packages/cultmesh-kotlin/src/main/kotlin/org/gamecult/cultmesh/ProbeFn.kt`. +- Images: `eureka-verify-rust`, `mcr.microsoft.com/dotnet/sdk:10.0`, `node:24`, `python:3.12`, and for + Kotlin `eclipse-temurin:21-jdk` with `kotlin-compiler-2.2.21.zip` fetched in the job (the recipe Cut K0 + bakes into an image). + +| # | Probe | Result at `e9469cb3`, all five runtimes | +|---|---|---| +| FN | Sender `a` sends ordered `o1`, `o2`, `o3`; `o2` is lost, so the receiver `b` holds `o3`. `b` sends reliable `x`, which `a` receives. `a` then sends the Cut F notice exactly as specified: `create_ack()` encoded, flag byte `\|= 0x10`, sequence field = `o2`'s sequence. `b` decodes and receives it; then `o2` arrives late | **Old receivers ignore the notice and apply its acks.** It decodes as an Ack; `b`'s pending `[501]` (`x`) empties, so the header's acks were applied; nothing is delivered; `reply` is empty; `b` stays connected; `o2` is **not** marked received (not in `b`'s ack or mask). The late `o2` then delivers `[o2, o3]`: the hole was untouched. Identical in Rust, C#, TypeScript, Python and Kotlin | +| FN-T | Transport path (code, not run) | Every socket transport acks only a reliable, Accept or Data packet, or (TS, Python, Kotlin) one whose receive delivered a frame (`rudp.rs:2152-2155`, `:1713-1715`, `CultNetTransport.cs:2281-2283`, `:2787-2788`, `rudp.ts:1089-1091`, `transport.py:1086-1088`, K:3243-3245). An old receiver delivers nothing on a notice (FN), so the notice draws no reply datagram and is not counted dropped. Every transport already forwards a session's `reply` (`rudp.rs:2135-2137`, `:1708`, `CultNetTransport.cs:2277-2278`, `:2748-2749`, `rudp.ts:1067-1068`, `transport.py:1070-1071`, K:3229, `rudp_document_server.rs:334-336`, `cultmesh-ts/src/index.ts:5828-5830`, `operation-service.ts:186`, `cultmesh-py/.../server.py:246-247`), which is how a new receiver answers a notice | +| CAP | Q-A9 B's capability bit on old peers: `a`'s Connect and `b`'s Accept are each encoded, given flag bit 4, and decoded before delivery; then `a` sends one ordered frame (`probe_cap.rs`, `probe-cap.test.ts`, `probe_cap.py`, `ProbeCapTests.cs`, `ProbeCap.kt`) | **Old peers ignore the bit on the handshake.** Both sides connect, the frame is delivered, and the Connect is acknowledged (the sender's pending holds only the data frame). Identical in all five runtimes | +| K-img | Kotlin self-tests on Linux | `kotlinc-jvm 2.2.21` on OpenJDK 21.0.12 compiles `CultMesh.kt` plus a probe file, and `java -cp out.jar:kotlin-stdlib.jar org.gamecult.cultmesh.CultMeshKt` (the self-tests, K:3349-3373) exits 0 | + **Suites on the prototype `db7d2886`** (pass 1; baseline `03eac6ff` was all green): - `cultnet-rs`: 110 + 4 + 24 pass; `tests/cultnet.rs` 51 pass and 2 fail. Both failures are test shape, @@ -190,11 +228,13 @@ Ack's sequence field and payload (`CultNetTransport.cs:1177`, `rudp.rs:578-592`, (`CultNetTransport.cs` `Enum.IsDefined`, `rudp.rs` `packet_type_from_code`, `rudp.ts:1313`, `transport.py:1212`, `CultMesh.kt` `fromCode`) and an unknown version byte. So: -- An additive watermark would fit in an Ack: flag bit 4 set, the value in the Ack's unused sequence field, - zero extra bytes. Old peers apply the ack and ignore the rest. Q-A1 ruled it out; it stays the known - place to put a FORWARD-TSN notice if Q-A3 is ever B. +- An additive field fits in an Ack: flag bit 4 set, the value in the Ack's unused sequence field, zero + extra bytes. Old peers apply the ack and ignore the rest. Q-A1 ruled out a watermark there; Cut F puts + the FORWARD-TSN notice there (Q-A3 B), and pass 3's FN probe confirms the old-peer behaviour in all five + runtimes. - A new packet type or a version bump is not additive: old peers refuse the datagram and count it dropped. -- No capability negotiation is needed under this map, because nothing on the wire changes. +- Cut F's notice needs no negotiation to be safe: it reads as a plain ack to every current peer. Q-A9 asks + whether to advertise support anyway, for the receiver's sake (§6). ## Authority map @@ -209,8 +249,12 @@ Ack's sequence field and payload (`CultNetTransport.cs:1177`, `rudp.rs:578-592`, - It advances in `RememberReceived` and resets with the rest of the peer state (`ResetPeerState`). - **Owner of retransmission (sender): the pending and queued sets**, as today. A sequence leaves them when an ack names it. Retransmission stops there. This is not a public meaning. +- **Owner of abandonment (sender, Cut F): the abandoned set.** An expired sequence is announced by notices + until an ack names it or 16 sends pass; then it is stranded (`first_stranded`). The receiver's `Abandon` + branch is the only path that marks a sequence received without its packet, and it drops every incomplete + fragment set the sequence falls in. - **Owner of Acknowledged (sender): the acknowledged-through watermark `A`**, one below the lowest sequence - that is pending, queued or abandoned in the current generation. An ordered send is Acknowledged when `A` + that is pending, queued, awaiting its abandonment notice's ack, or stranded in the current generation. An ordered send is Acknowledged when `A` reaches its last sequence. A reliable unordered send is Acknowledged when every one of its sequences is acked. - **Owner of the session boundary: the session's generation.** One primitive ends a generation: the @@ -246,12 +290,16 @@ Ack's sequence field and payload (`CultNetTransport.cs:1177`, `rudp.rs:578-592`, - No path delivers an ordered frame except the drain after `R` advances. - No receive path drops a frame whose sequence it acknowledged while the generation lives. - No sender path reports Acknowledged except through `A` (or the unordered rule). - - No ending path leaves a write owed. No path retransmits a write into a generation after the one it was - issued in. + - No ending path leaves a write owed. No path retransmits a write, or sends a notice, into a generation + after the one it was issued in. + - No path marks a sequence received without a packet, except the `Abandon` branch (Cut F). + - No per-peer send failure ends a poll, a loop, a thread or another peer's session (Cut D). - No transport decides whether a Connect repeats: not by payload (Rust hub `rudp.rs:1637-1647`, C# listener `CultNetTransport.cs:2670-2688`), not by `Connected` (C# `:2226-2236`, Python - `transport.py:1047-1053`, TS `rudp.ts:249-251`, `:1031-1049`), and not by resetting on every Connect - (Rust server mode `rudp.rs:2105-2116`). + `transport.py:1047-1053`, TS `rudp.ts:249-251`, `:1031-1049`), not by server key or payload + (cultmesh-rs document server `rudp_document_server.rs:413-427`, cultmesh-ts document server + `index.ts:5794-5802`), and not by resetting on every Connect (Rust server mode `rudp.rs:2105-2116`, + cultmesh-py server `server.py:225-240`). - **Shared paths.** Accept, reliable data, fragments, duplicates and retransmits all advance `R` through `RememberReceived` and then call the drain. Direct sends, fragmented sends, queued promotion and resends all feed the pending and queued sets that `A` reads. Every ending path goes through the one generation @@ -303,7 +351,7 @@ Ack's sequence field and payload (`CultNetTransport.cs:1177`, `rudp.rs:578-592`, ### 3. The public API (Q-A2, ruled: one meaning) -- **Rust receipts** (the only runtime that has them): +- **Rust receipts** (the only runtime that has them today; §9 ports them to the other four): - `CultNetRudpReliableSendStatus` keeps its three variants, `Pending`, `Acknowledged`, `Invalidated` (`rudp.rs:146-150`). No variant is added, removed or renamed. `Acknowledged` changes meaning from "every packet was acked" to "delivered". @@ -344,15 +392,18 @@ Ack's sequence field and payload (`CultNetTransport.cs:1177`, `rudp.rs:578-592`, `transport.py:997`, `:1054`). Once Cut 1b drops owed writes at every ending, "outstanding is 0" after a reconnect would read as success, so every runtime binds flush to the session's generation in Cut 1b. - When every sent sequence is acked, `A` is the highest sequence sent, so every send is Acknowledged. - The only exception is behind an abandoned sequence (Rust expiry). There `flush_reliable` - (`rudp.rs:1992-2040`) fails at once when an ordered send follows the generation's first abandoned - sequence: that send can never be Acknowledged, and today flush reports success (P2). + Under Cut F an abandoned sequence is resolved by its notice's ack, so flush waits for that too. The + only exception is a stranded sequence (a notice the receiver never acknowledged). There + `flush_reliable` (`rudp.rs:1992-2040`) fails at once when an ordered send follows it: that send can + never be Acknowledged, and today flush reports success (P2). - C# `FlushReliable`, TS `flush`, Python `flush_reliable` and Kotlin `flushReliable` (`CultMesh.kt:3186`) keep their success condition. Their contract text becomes "every reliable frame - sent in this session has been delivered to the peer's application". These runtimes have no expiry. + sent in this session has been delivered to the peer's application". These runtimes have no expiry + unless Q-A8 gives them some (Cut 7). - **Defaults.** Plain flush. No parameter. No second flush. - **Compatibility.** No signature changes in any runtime except additions (`wait_acknowledged`; TS - `connectAndWait`, §4). The Rust enum is unchanged. No downstream caller has to change code at its next + `connectAndWait`, §4; the receipts of §9, where C# and Kotlin `SendSchema*` return a receipt instead of + nothing, which is source-compatible). The Rust enum is unchanged. No downstream caller has to change code at its next bump. ### 4. Consumers, each assigned @@ -366,7 +417,7 @@ consumer that waits wants delivered, which is now just Acknowledged. | `cultmesh-rs/src/lib.rs:870-900` `publish_cultnet_message_to_rudp_catalog` (Odin `odin-daemon`) | a hand-rolled loop until `Acknowledged`, then disconnect. The doc at `:869` claims "application admission" | Cut 4: `wait_acknowledged`; the doc says delivered to the document server's sink (`rudp_document_server.rs:343-372`) | | `cultmesh-rs/src/lib.rs:918-938` `publish_cultnet_messages_to_rudp_catalog` (Ghostlight `mesh.rs:323`) | `flush_reliable`, then drop. **Several ordered frames, then close** | None; flush now means it | | `cultmesh-ts/src/index.ts:6070-6098` `publishRudpDocumentOnce` (Heimdall, StreamPixels, Stonks, weksa, Bifrost, VoidBot, Hermodr) | `peer.flush` (`cultnet-ts/src/peer.ts:109-110`), then close | None. Cut 5 moves its connect-and-wait helper (`index.ts:6603-6650`) into cultnet-ts | -| `cultnet-ts/src/idunn-odin-presence-publisher.ts:89-215` `publishDocument`, behind `createIdunnRuntimePresencePublisher`. **StreamPixels' service runs this**: `apps/service/scripts/idunn-entry.mjs` calls `createIdunnRuntimePresencePublisher` from its vendored CultLib `30ee8b9`, where the file is byte-identical to `3bf1c0c` | a hand-rolled session loop (`:186-208`) until `outstandingReliablePacketCount === 0`. It does resend (`:205`). It drops `readyToSend` (`:198`), so promoted queued packets wait for the resend timer. It pins `initialSequence: 1` (`:103`) and never sends a Disconnect, so Odin's hub keeps each publish's peer until it times out (each publish binds a new port, so each is a new peer) | Cut 5: the socket transport, `connectAndWait`, `send("schema")`, `flush`, `close` (which sends the Disconnect, `rudp.ts:956-969`) | +| `cultnet-ts/src/idunn-odin-presence-publisher.ts:89-215` `publishDocument`, behind `createIdunnRuntimePresencePublisher`. **StreamPixels' service runs this**: `apps/service/scripts/idunn-entry.mjs` runs `deployment/idunn/runtime-presence.mjs`, which calls `createIdunnRuntimePresencePublisher` (`:46`) from its vendored CultLib `542ddd3` (StreamPixels `a49f7d8`; pass 2's `30ee8b9` is stale), where it lives under its older name `cultnet-ts/src/idunn-runtime-authority.ts:206` | a hand-rolled session loop (`:186-208`) until `outstandingReliablePacketCount === 0`. It does resend (`:205`). It drops `readyToSend` (`:198`), so promoted queued packets wait for the resend timer. It pins `initialSequence: 1` (`:103`) and never sends a Disconnect, so Odin's hub keeps each publish's peer until it times out (each publish binds a new port, so each is a new peer) | Cut 5: the socket transport, `connectAndWait`, `send("schema")`, `flush`, `close` (which sends the Disconnect, `rudp.ts:956-969`) | | `cultnet-ts/src/signed-daemon-health.ts:104-162` `publishSignedDaemonHealth` (Hermodr, Odin `idunn-rudp.cjs`) | a hand-rolled loop: sends the packets `sendMany` admitted, waits up to 500 ms for the **first** Ack packet of any content (`:146-153`), swallows the timeout, closes. Resends run inside that wait (`:303`) but promoted queued packets are never sent | Cut 5, as above. Today a write not acked within 500 ms, or any payload over 32 packets, is reported published | | `cultnet-ts/src/operation-service.ts` | the operation response | none (the response is the proof) | | C# `FlushReliable` callers: Gjallar `Program.cs:3055-3080` (two ordered frames, flush, dispose) | flush | None | @@ -405,21 +456,140 @@ consumer that waits wants delivered, which is now just Acknowledged. - **Old senders** (the pinned and vendored ones) can still trip the Rust cap until they bump. - **Not in scope.** C#, TS, Python and Kotlin receivers have no hold cap. Follow-up. -### 6. Reliable expiry (P2) +### 6. Reliable expiry and FORWARD-TSN (P2; Q-A3 B, ruled) -- Only Rust has it (`rudp.rs:39`, `:898-908`). An expired packet's sequence never reaches the receiver. - `R` stops below it, and so did the old `next` pointer. Ordered frames after it are held for the rest of - the session. -- **Today's users.** Only Muninn's reliable `audio`, and its session carries no ordered traffic. - `hands/media-fec-cut3` (unmerged, `4b85a173`) deletes the `audio` profile entry and the `"audio"` send - arm. The `media` channel keeps a caller-chosen reliable-with-expiry mode. -- **This map makes it honest (Cut 4):** - - the sender records its generation's first abandoned sequence; - - `A` never passes it; - - ordered receipts after it stay `Pending`, and become `Invalidated` when the generation ends; - - `flush_reliable` fails at once. -- **The real fix is a fork (Q-A3).** The literature's answer is PR-SCTP's FORWARD-TSN (RFC 3758): the - sender tells the receiver to move its cumulative point past abandoned sequences. +- **The hole.** Only Rust has reliable expiry (`rudp.rs:39`, `:898-908`; the Python README's claim of + expiry at `packages/cultnet-py/README.md:103-111` is false, `transport.py` has none). An expired packet's + sequence never reaches the receiver, `R` stops below it, and every ordered frame after it on the session + is held until the session ends. +- **Who expires.** Every Rust `media` sender that does not opt into lossy delivery: the profile's media + delivery defaults to Reliable (`channel_delivery`, `rudp.rs:2515-2521`) and its expiry to 75 ms + (`rudp.rs:39`, set on the client and hub transports at `:1303` and through the profile at `:1548-1549`). + Media frames fragment. Muninn's reliable `audio` also expires; `hands/media-fec-cut3` deletes that profile + entry and moves Muninn's media to lossy delivery, but the default stays reliable-with-expiry for every + other caller. + +**What RFC 3758 does, and what this map keeps.** + +- The sender treats an abandoned chunk as "finally acked and no longer outstanding" (§3.5 A2) and advances + an `Advanced.Peer.Ack.Point` over consecutive abandoned TSNs. When that point passes the peer's + cumulative ack it sends FORWARD TSN (C1-C3), and tries again whenever T3-rtx expires (A5). +- FORWARD TSN carries a **New Cumulative TSN** plus (stream, stream sequence) pairs. The receiver "MUST + consider any missing TSNs earlier than or equal to this value as received" (§3.2), advances its + cumulative point, releases the listed streams' reordering queues, and acks "as if a DATA chunk had been + received" (§3.6). A FORWARD TSN at or behind its cumulative point is out of date, changes nothing, and + still draws a SACK. +- PR-SCTP is negotiated: an endpoint that did not see Forward-TSN-Supported in the peer's INIT must never + send FORWARD TSN (§3.1, §3.3). + +**How CultNet adapts it.** + +- **The notice names one sequence, not a cumulative point.** SCTP's SACK is cumulative, so a cumulative + FORWARD TSN is confirmed by the next SACK. CultNet's ack is not: it names the highest received sequence + plus a 32-bit mask, or one exact older sequence (`create_ack_for_received`, `rudp.rs:751-758`). A + cumulative notice could not be confirmed by that ack, which would mean a second wire field. So each + abandoned sequence gets its own notice, and the receiver answers it with the ack a reliable data packet + with that sequence would draw. That ack removes the sequence from the sender's abandoned set exactly as it + removes a data packet from pending. **Q-A7** puts the cumulative alternative to the operator. +- **No stream pairs.** One watermark `R` owns ordered delivery for every channel (Cut 1), so advancing `R` + and draining releases every channel's held frames. There is nothing per channel to name. +- **Negotiation: only if Q-A9 is B.** The ruling puts the notice in an Ack so an old peer reads it as an + ack (FN), and bounds the resend: an old receiver never acks the notice, and the sender stops after a + fixed number of sends. The Cut 2 Soul finding adds a reason to negotiate after all (Q-A9): a *receiver* + needs to know whether its peer will ever send notices. If Q-A9 is B, the sender also sends notices only + to a peer that advertised the bit (RFC 3758 §3.3), and an old receiver's sequence is stranded at once + instead of after the bound. The abandoned sequence is then *stranded*: the Cut 4 honest + report applies to it (ordered sends after it stay Pending, become Invalidated at the end, and flush fails + fast). That is the behaviour ruling A would have given everywhere; under B only old receivers get it. + +**Wire.** Packet type `4` (ack), flag bit 4 (`0x10`, "abandon"), and no other flag bit. Sequence field: the +abandoned sequence. Ack and ack mask: the sender's current ack state, so the notice is also a genuine ack. +Channel `control`, fragment fields `0`, empty payload. Decoders map type 4 with bit 4 to a new packet type, +**`Abandon`** (Rust `CultNetRudpPacketType::Abandon`, C# `CultNetRudpPacketType.Abandon`, TS `"abandon"`, +Python `CultNetRudpPacketType.ABANDON`, Kotlin `CultNetRudpPacketType.Abandon`), and the encoder maps it +back. Bit 4 on any other type is ignored, as today. No public packet struct gains a field, and no +downstream repo constructs packets or matches the type exhaustively (surveyed: Muninn tests compare +`== Data`; Mimir `CultMeshMedia` compares `Connect` and `Accept`). + +**Golden** (added beside the existing fixture in every runtime and in `docs/cultnet-transport-parity.md` +after the canonical fixture, `:505-517`): + +```text +packetType=abandon (wire: type 4, flags 0x10) +connectionId=0x01020304 +sequence=0x0000002a +ack=0x00000029 +ackMask=0x80000001 +channelId=control +payload=(empty) +hex=434e52300004102b010203040000002a0000002980000001000000000000000000000700636f6e74726f6c +``` + +**Sender (Rust now; any runtime that gains expiry under Q-A8).** + +- An expired packet, pending or queued, leaves the pending and queued sets as today and enters an + **abandoned set**: sequence, sends so far, last sent. It is due at once. +- `due_resends` emits a fresh notice for each due entry (built from the current ack state, not stored), + once per resend delay. After `RUDP_ABANDON_NOTICE_SENDS = 16` sends without an ack the entry leaves the set + and the session records the lowest such sequence as **`first_stranded`**. Sixteen sends at the default + 250 ms is four seconds. At 50% loss in each direction a round trip fails with probability 3/4, and all + sixteen with about 1%; at 20% each way, about 10^-7. The bound is for deaf receivers, not bad links, and + a new receiver that misses every notice degrades to the honest report, never to a lie. +- Any ack naming the sequence removes it from the abandoned set. That ack means the receiver holds the + sequence, from the original packet arriving late or from the notice; either way it is no longer a hole. +- Notices take no send-window slot and no payload bytes. For Cut 3's span rule, an abandoned sequence with + an outstanding notice counts as unacked (the receiver's `R` is held below it); a stranded one does not + (an old receiver has no `R` to hold, and counting it would stop the session after 1,023 more sequences). +- `outstanding_reliable_packet_count` counts abandoned entries, so a flush waits for their notices. +- The session-ending primitive (Cut 1b) and `reset_peer_state` clear the abandoned set and + `first_stranded`: they belong to the generation. + +**Receiver (all five runtimes).** + +- `receive` handles `Abandon` after applying its acks, as RFC 3758 §3.6 does: + - outside `[highest − 4,095, highest + 1,024]`: far-ahead is ignored without remembering (as Rust data + is, `rudp.rs:626-641`), far-behind is a duplicate; + - every incomplete fragment set whose sequence span contains the sequence is dropped (a set's span is + `sequence − fragment_index` for any fragment it holds, for `fragment_count` sequences), whether or not + the sequence was received: that frame can never complete; + - if the sequence was not received, it is remembered through `RememberReceived`, which advances `R` + (Cut 1), and held frames up to `R + 1` drain into `delivered`; + - `reply` is `create_ack_for_received(sequence)`, whether or not anything changed (the out-of-date case + still acks). +- Transports send the reply as they already do, and skip their own ack for an `Abandon` packet + (`rudp.ts:1089`, `transport.py:1087`, K:3243 would otherwise ack it a second time when the drain + delivered). + +**Interaction with the other cuts.** + +- **Cut 1 (`R`).** The notice is the only receive path besides data and Accept that calls + `RememberReceived`; it shares the drain. A late original of an abandoned packet is then a duplicate: acked + and dropped, as PR-SCTP drops it. +- **Cut 1b (ending).** An abandoned entry is a write of its generation; the ending primitive drops it. No + notice is ever sent into a later generation. +- **Cut 1c (generations).** A stale notice from an earlier generation of the same sender carries a + sequence below the new generation's Connect (issued sequences survive resets), so it is a duplicate. A + stale notice from a dead process on the same endpoint and connection id could mark a sequence of the new + process's session, exactly as a stale data packet can be delivered into it today; the random initial + sequence makes both land inside the 5,120-sequence acceptance range with probability about 2^-18. +- **Cut 2 (fragment eviction), from the Cut 2 Soul pass.** Cut 2 never evicts a reliable fragment set and + refuses the session when every pending set is reliable. Its claim that a conforming sender cannot reach + the 64-set bound is **false for an expiring sender**: fragment 0 arrives and is acked, the deadline drops + the rest, and the set is pinned for the session. Soul reproduced the refusal on the 65th lossy default + `media` frame (`soul_probe_expiring_reliable_frames_kill_receiver`, `packages/cultnet-rs/tests/cultnet.rs:3192` + at probe `3be97b34`). Before Cut 2, eviction kept that session alive. + - **With a Cut F sender** the notice for each dropped fragment lands inside the set's span and the + receiver drops the set: the rule above, explicitly "every incomplete set whose span covers a notified + sequence". Dead sets then live one round trip, not the session. + - **With a sender that never sends notices** (every build before Cut F) the receiver cannot tell an + expiring set from a set whose missing fragment is still coming. **Q-A9** decides what the receiver does + then; its recommendation is PR-SCTP's own answer, a capability bit on Connect and Accept (RFC 3758 + §3.1, Forward-TSN-Supported): Cut 2's never-evict rule applies to a peer that advertised it, and the + pre-Cut-2 stalest-set eviction stays for a peer that did not. + - So Cut 2 does not merge alone: it merges together with Cut F, which carries the capability and the + gate (§ Sequencing). +- **Cut 4.** Reads `first_stranded` where pass 2 read `first_abandoned`, and counts outstanding notices as + not yet Acknowledged. ### 7. Interaction with the merged stray-packet work @@ -453,7 +623,7 @@ The ack cuts build on these behaviours and do not alter them: - **What must hold after both.** A write is Acknowledged or Invalidated within the generation it was issued in. A restarted client is admitted. A new server session's frames are delivered after a reconnect. A retransmitted Connect is still a repeat. -- **Recognising a new generation (Cut 1c, Q-A6 recommendation A).** The Connect's sequence identifies the +- **Recognising a new generation (Cut 1c, Q-A6 A, ruled).** The Connect's sequence identifies the generation, and the initial sequence is random by default, as TCP's initial sequence number is (RFC 9293 §3.4.1, RFC 6528). - A Connect repeats exactly when it carries the sequence of the Connect that started the current @@ -473,17 +643,99 @@ The ack cuts build on these behaviours and do not alter them: `cultnet-rs/src/provider_session.rs:70-79`), carried to the application as today. It no longer decides anything in the transport. +### 9. Receipts in C#, TypeScript, Python and Kotlin (Q-A4 A, ruled) + +- **One design, ported.** Cut 4 builds it in Rust: a receipt carries the sequences its send assigned, + whether it was ordered, and a status cell the session writes once. The session keeps the unresolved + receipts and resolves them after every ack it applies (Acknowledged: ordered when `A` reaches its last + sequence, unordered when every sequence is acked) and in the ending primitive (Invalidated). The four + ports copy that shape; none invents a second one. +- **What every runtime has to change first** (the receipts survey, `e9469cb3`): + - `SendMany`/`sendMany`/`send_many` returns only the packets admitted to the 32-packet window + (`CultNetTransport.cs:1439-1465`, `rudp.ts:586-592`, `transport.py:776-786`, K:2856-2862). A receipt + must be built from every sequence the send *assigned*, queued ones included, which is Rust's P10. + - `ApplyAcknowledgements` only deletes from the pending map (`CultNetTransport.cs:1503-1516`, + `rudp.ts:610-617`, `transport.py:803-807`, K:2882-2889). Resolution hooks in after it. + - Invalidation needs the ending primitive. C#, TS and Python get it in Cut 1b; Kotlin in K2. +- **Where the receipt lives.** On the single-peer socket transport connection, as in Rust + (`CultNetRudpSocketTransportConnection::send_reliable`, `rudp.rs:1903-1930`). The Rust hub has no + receipts, so neither do the C# listener (`CultNetRudpSocketTransportServer`, `:2487`), `cultmesh-ts`'s + document server or `cultmesh-py`'s server. Hub receipts are a follow-up for all five together. +- **Public API, per runtime.** Every runtime's RUDP surface is synchronous-with-timeout except TypeScript's, + which is Promise-and-event. Each port matches its own runtime: + +| | Send | Receipt | Status | Wait | +|---|---|---|---|---| +| Rust (Cut 4) | `send_reliable(&mut self, channel, payload) -> Result` | `packet_sequences()` | `reliable_send_status(&receipt) -> CultNetRudpReliableSendStatus` | `wait_acknowledged(&receipt, timeout) -> Result<()>` | +| C# (6a) | `CultNetRudpSendReceipt SendReliable(string channelId, byte[] payload)`; `SendSchema(...)` and `SendSchemaMessage(...)` return the receipt instead of `void` (source-compatible) | sealed class, `IReadOnlyList PacketSequences` | `receipt.Status` (`CultNetRudpSendStatus { Pending, Acknowledged, Invalidated }`), read without the transport | `void WaitAcknowledged(CultNetRudpSendReceipt receipt, TimeSpan? timeout = null)`, the `FlushReliable` loop (`:1914-1955`); throws `InvalidOperationException` naming the end reason when Invalidated, `TimeoutException` at the deadline | +| TypeScript (6b) | `sendReliable(channelId: string, payload: Uint8Array): CultNetRudpSendReceipt`. `send` keeps `void`, because the transport interface declares it (`transport.ts:132-142`) | `readonly packetSequences: readonly number[]` | `receipt.status: "pending" \| "acknowledged" \| "invalidated"` (string union, as packet types are) | `waitAcknowledged(receipt, timeoutMs = 30_000): Promise`, polling like `flush` (`rudp.ts:911-935`); rejects with the end reason when invalidated | +| Python (6c) | `send_reliable(self, channel_id: str, payload: bytes) -> CultNetRudpSendReceipt` | frozen-looking object, `packet_sequences: tuple[int, ...]` | `receipt.status` (`CultNetRudpSendStatus` enum: `PENDING`, `ACKNOWLEDGED`, `INVALIDATED`) | `wait_acknowledged(self, receipt, timeout_seconds: float = 30.0) -> None`, the `flush_reliable` loop (`:1101-1131`); raises `ConnectionError` when invalidated, `TimeoutError` at the deadline | +| Kotlin (K4) | `fun sendReliable(channelId: String, payload: ByteArray): CultNetRudpSendReceipt`; `sendSchema`/`sendSchemaMessage` return it | `val packetSequences: List` | `receipt.status` (`enum class CultNetRudpSendStatus { Pending, Acknowledged, Invalidated }`), `@Volatile` | `fun waitAcknowledged(receipt, timeoutMs: Long = 30_000, pollIntervalMs: Long = 5)`, the `flushReliable` loop (K:3186-3205); throws `IOException` when invalidated, `SocketTimeoutException` at the deadline | + + - The status is on the receipt, not a transport query, in the four ports: a C#, TS, Python or Kotlin + caller reading `receipt.Status` is the idiom there, and it is what Rust's cell already is underneath. + Rust keeps `reliable_send_status` because its signature is public and used (Cut 4, Unchanged). + - `SendReliable` refuses a channel whose send options are not reliable (as Rust's `send_reliable` does, + `rudp.rs:1903-1930`). If Q-A8 brings expiry to a runtime, it refuses an expiring channel too, as Rust's + does after Cut 4 (P9). + - A receipt from an ended session is Invalidated and stays so; `WaitAcknowledged` on it fails at once. +- **Thread safety.** C# resolves under `_pendingReliableGate` (`:915`) and publishes the status with a + volatile write; Kotlin marks the field `@Volatile`; TS and Python are single-threaded per transport. +- **Docs.** Each runtime's README RUDP section gains the three states and the wait + (`src/GameCult.Networking/README.md`, `packages/cultnet-ts/README.md:24-25`, + `packages/cultnet-py/README.md:103-111`, `packages/cultmesh-kotlin/README.md:229-305`), and the parity + doc's flush paragraph (`docs/cultnet-transport-parity.md:260-273`) names receipts in all five. + +### 10. A failed send to one peer is that peer's lost datagram (Cut D; from the Odin Soul pass) + +- **Finding.** `CultMeshRudpDocumentServer::poll_once` begins with `self.maintain()?` + (`cultmesh-rs/src/rudp_document_server.rs:264`). `maintain` sends every peer's due resends and returns on + the first failed `send_to` (`:392-398`, through `send_packet` `:628-632`). Every per-peer send in + `poll_once` does the same: the reply (`:334-336`), the Accept (`:451`), a snapshot response (`:586-588`), + the goodbye (`:597-604`) and the ack (`:366-373`). + - So one peer whose datagrams cannot be sent fails every poll for every peer. Its resends come due every + 50 ms (`:158`); a poll loop that pauses longer than that between attempts never gets past `maintain` + again while the peer lives, and a peer that keeps sending never idles out (`:384-388`). + - Odin `origin/main` (`crates/odin-daemon/src/main.rs:661-675`) propagates any `poll_once` error, so + today the daemon exits on the first one; the Odin Soul pass reports a branch that retries every 100 ms + and exits after 30 s of failures. Either way one bad peer takes Odin down. + - The same socket is **non-blocking** (`:230`), so `send_to` also fails with `WouldBlock` when the send + buffer is full: a burst of snapshot responses to one peer can fail a poll for everyone with no bad peer + at all. +- **Rule: a send failure is a lost datagram, attributed to its peer.** It is counted and the poll goes on. + RUDP already recovers lost datagrams: the reliable packet stays pending and is resent; an unreliable one + was allowed to be lost. The peer's session ends by the rules that already end sessions: idle timeout, + maximum lifetime, a refusal or a Disconnect. This is TCP's treatment of ICMP unreachables as soft errors + (RFC 1122 §4.2.3.9): report them, do not abort the connection on them. + - Only failures that are not about one peer stay errors: `recv_from` failing with anything but + `WouldBlock` or `ConnectionReset` (Windows reports an earlier ICMP port-unreachable on the *next* + receive; the socket transport already treats it as nothing, `rudp.rs:2059-2066`, the document server does + not, `:266-272`), and encode failures, which are the server's own bug. + - Rejected: ending the peer's session on its first failed send. `WouldBlock` is transient and not the + peer's fault; ending on it would drop healthy peers under load. +- **Same shape elsewhere** (multi-peer servers only; a single-peer transport's failed send is already only + that peer's): + +| Server | Where one peer's failed send stops the others | Change | +|---|---|---| +| cultmesh-rs document server | as above | Cut D | +| Rust hub `CultNetRudpServerHub` | `poll_resends` returns on the first failure (`rudp.rs:1762-1777`); the receive path's sends to one peer abort `receive_event_once` (`:1644`, `:1673`, `:1708`, `:1711`, `:1714`, `:1759`, through `send_packet` `:1799-1804`) | Cut D | +| C# `CultNetRudpSocketTransportServer` (and `RudpCultNetSchemaServer` over it) | `PollResends` walks every peer and the first `SocketException` from `SendTo` (`CultNetTransport.cs:2829-2834`) escapes the loop (`:2796-2805`); `TryReceiveOnce`'s sends are unguarded except the refusal goodbye (`:2687`, `:2704`, `:2749-2750`, `:2788`; guarded `:2739-2745`) | Cut D | +| cultmesh-py server | worse: an `OSError` from `sendto` (`server.py:302-307`) in `_poll_rudp_resends` (`:277-285`) or the receive path escapes `_rudp_loop` (`:203-…`, which catches only `recvfrom`'s errors), and the RUDP thread dies for every peer | Cut D | +| cultmesh-ts document server | none: Node's `socket.send` is asynchronous and reports failures on the socket's `error` event (`cultmesh-ts/src/index.ts:5858`), which `reportError` logs (`:5778-5785`) | none | +| Kotlin interop server | none that matters: a catch-all logs and continues (K:4025), and it is interop-only | none (Cut K1 reviews it) | + ## Cuts -Every cut is on CultLib, branch `hands/rudp-delivered` from `main` at `3bf1c0c` or later, in a worktree -Self creates. Heavy verification runs on Yggdrasil. Each cut starts by committing its probes as failing +Every cut is on CultLib, on its own branch (`hands/ack-cut`) from `main` at `e9469cb3` or later, in a +worktree Self creates. Heavy verification runs on Yggdrasil. Each cut starts by committing its probes as failing tests named for the rule, in every runtime it touches. The pass-1 and pass-2 probe files (`refs/verify/1519a20a` on the `CultLib-imag-ack2` mirror) are the starting point; they print, so each must become an asserting test. -Kotlin: every cut below is mapped for C#, Rust, TS and Python. If Q-A5 is A, a trailing Cut K applies -Cuts 1, 1b, 1c and 3 to `CultMesh.kt` at the anchors given, on the verification host Q-A5 picks. +Kotlin: every numbered cut below is mapped for C#, Rust, TS and Python. Cut K (Q-A5 A) brings Kotlin to +the same state on a Yggdrasil Kotlin image, in steps K0-K5. **Negative checks that stay green in every cut:** @@ -491,9 +743,9 @@ Cuts 1, 1b, 1c and 3 to `CultMesh.kt` at the anchors given, on the verification (`docs/cultnet-transport-parity.md:227-248`); - the stray-packet tests (`packages/cultnet-rs/tests/rudp_stray_packets.rs`, and their C#, TS and Python equivalents), except the ones each cut names as encoding the old rule; -- the codec fixture vectors: no wire byte changes. +- the codec fixture vectors: the existing fixture's bytes never change. Cut F adds one fixture. -### Cut 1. One watermark owns ordered delivery (receiver) — ready +### Cut 1. One watermark owns ordered delivery (receiver) — in flight (`hands/ack-cut1`) - **First.** Failing tests P1, P1b and P6 in each runtime. - **Deletes first:** @@ -529,7 +781,7 @@ Cuts 1, 1b, 1c and 3 to `CultMesh.kt` at the anchors given, on the verification - mutation testing on the diff: cargo-mutants, Stryker.NET, StrykerJS, and mutmut for Python. - **Ledger.** Per runtime about −70 and +40. Across four, about −280 and +160. -### Cut 1b. A write belongs to the session it was issued in: every ending drops what it owed — ready +### Cut 1b. A write belongs to the session it was issued in: every ending drops what it owed — in flight (`hands/ack-cut1`) - **First.** Failing tests G3 (each ending path: peer Disconnect, local disconnect, timeout, refusal) in C#, TS and Python, and local disconnect and timeout in Rust; G3R in Rust; and in every runtime, a flush @@ -576,62 +828,120 @@ Cuts 1, 1b, 1c and 3 to `CultMesh.kt` at the anchors given, on the verification - mutation testing on the diff. - **Ledger.** About −40 and +45 across four runtimes. TS is net negative. -### Cut 1c. A Connect's sequence says whether it repeats — blocked on Q-A6 +### Cut 1c. A Connect's sequence says whether it repeats — ready after Cut 1b (Q-A6 A, ruled) -Mapped for Q-A6's recommended answer A. Blocked on Cut 1b (it uses the ending primitive). +Pass 3 changes against pass 2: the ruling; three more servers that decide repeats themselves (the +cultmesh-rs and cultmesh-ts document servers and the cultmesh-py server); the second C# default of 100; +the exact default mechanism; the full caller survey; `cultmesh-rs/src/lib.rs:1409` removed from the +production list (it is inside `#[cfg(test)]`, which starts at `:1140`). -- **First.** Failing tests G1 (the hub, the C# listener, and server mode in every runtime: a restarted - client on the same address and payload is admitted and its frames delivered), G1b (C# and Python server - mode admit a Connect from a new endpoint), G2 (after `connect()` on the same session, a new server - session's frames are delivered), and a stale Accept (its ack field does not name the pending Connect) is - ignored. +- **Repo/branch:** CultLib `hands/ack-cut1c` from `main` after Cut 1b merges. It uses the ending primitive. +- **First.** Failing tests: + - G1: a restarted client on the same address, connection id and payload is admitted and its frames + delivered, against the Rust hub, the C# listener, server mode in every runtime, the cultmesh-rs + document server (same `(addr, connection_id)` key), the cultmesh-ts document server (same payload) and + the TS operation service; + - G1b: C# and Python server mode admit a Connect from a new endpoint; + - G2: after `connect()` on the same session, a new server session's frames are delivered; + - a stale Accept (its ack field does not name the pending Connect) is ignored; + - the cultmesh-py server and Rust server mode: a *retransmitted* Connect after data has flowed does not + reset the session (today both reset on every Connect). - **Deletes first:** - - the transport-level repeat decisions: Rust server mode's reset on every Connect - (`rudp.rs:2105-2116`); the Rust hub's payload comparison (`rudp.rs:1637-1647`); C# server mode's - `_session.Connected` branch (`CultNetTransport.cs:2226-2236`); the C# listener's payload comparison - (`CultNetTransport.cs:2670-2688`); TS's `movedEndpoint`/`repeated` computation (`rudp.ts:1011-1049`) - and the session's own `connected` shortcut (`rudp.ts:249-251`); Python's `connected` branch - (`transport.py:1047-1053`); + - the transport-level repeat decisions: Rust server mode's reset on every Connect (`rudp.rs:2105-2116`); + the Rust hub's payload comparison (`rudp.rs:1637-1647`); C# server mode's `_session.Connected` branch + (`CultNetTransport.cs:2226-2236`); the C# listener's payload comparison (`:2670-2688`); TS's + `movedEndpoint`/`repeated` computation (`rudp.ts:1011-1049`) and the session's `connected` shortcut + (`rudp.ts:249-251`); Python's `connected` branch (`transport.py:1047-1053`); + - **new in pass 3:** the cultmesh-rs document server's "same key is a repeat" arm + (`cultmesh-rs/src/rudp_document_server.rs:413-427`, and its comment at `:414-415` that asks a fresh + client to choose a fresh id); the cultmesh-ts document server's `connectPayload` comparison and field + (`cultmesh-ts/src/index.ts:5794-5802`, `:5806`); the cultmesh-py server's replace-on-every-Connect + (`cultmesh-py/src/cultmesh_py/server.py:225-240`); - C# and Python server mode's refusal of a Connect from a new endpoint (`CultNetTransport.cs:2212-2216`, - `transport.py:1042-1044`), for Connect packets only. Rust and TS already admit it. + `transport.py:1042-1044`), for Connect packets only. - **Adds:** - - the session records the sequence of the Connect that started its generation, and decides: a Connect - repeats exactly when it is connected and the Connect carries that sequence. `AcceptConnect` answers a - repeat as `AnswerRepeatedConnect` does today, and otherwise ends the generation, resets peer state and - accepts (`CultNetTransport.cs:1012-1059`, `rudp.rs:383-435`, `rudp.ts:242-290`, - `transport.py:485-525`). `AnswerRepeatedConnect` stops being public; a public predicate - (`ConnectRepeats(packet)`) serves the hubs, which keep building a fresh session for a new generation and - emit Disconnected for the old one (`rudp.rs:1648-1690`, `CultNetTransport.cs:2690-2710`); + - the session records the sequence of the Connect that started its generation. A Connect repeats exactly + when the session is connected and the Connect carries that sequence. `AcceptConnect` answers a repeat + as `AnswerRepeatedConnect` does today, and otherwise ends the generation, resets peer state and accepts + (`CultNetTransport.cs:1012-1059`, `rudp.rs:383-435`, `rudp.ts:242-290`, `transport.py:485-525`). + `AnswerRepeatedConnect` stops being public; a public predicate `ConnectRepeats(packet)` / + `connect_repeats` / `connectRepeats` serves servers that keep one session per peer and must report the + old one ended (the Rust hub `rudp.rs:1648-1690`, the C# listener `:2690-2710`, the cultmesh-rs document + server). Servers that only call `AcceptConnect` (TS operation service `operation-service.ts:167-175`, + cultmesh-ts document server, cultmesh-py server, server-mode transports) need nothing else: the session + decides; - `CreateConnect` on a session that has had a peer ends the generation and resets peer state first (`CultNetTransport.cs:1001-1010`, `rudp.rs:369-381`, `rudp.ts:229-240`, `transport.py:473-483`); - on Accept receipt, the session honours it only while its Connect is pending and only if the Accept's ack field names that Connect (`CultNetTransport.cs:1155-1160`, `rudp.rs:544-557`, `rudp.ts:379-390`, `transport.py:601-604`); - - the default initial sequence is random in [1, 2^31): Rust `CultNetRudpSessionOptions::default` - (`rudp.rs:104-113`), the transport constructors (`:1311`, `:1332`) and hub options (`:1377`); C# - `CultNetTransport.cs:773`, `:850`, `:2404`; TS `rudp.ts:172`; Python `transport.py:298`, `:323`; - - CultLib's own fixed `initial_sequence: 1` call sites take the default: `cultmesh-rs/src/lib.rs:964`, - `:1409`, `cultmesh-rs/src/rudp_document_server.rs:432`, `cultnet-rs/src/cultmesh.rs:259`, - `cultnet-ts/src/signed-daemon-health.ts:116`. -- **Tests changed** (they encode payload identity or a fixed initial sequence): + - **the default initial sequence is drawn from a CSPRNG in [1, 2^31)** wherever a default is produced. + The option types do not change (`u32`/`uint`/`number`/`int`), so every explicit value keeps working and + no caller changes code. One private helper per runtime: + - Rust: `rand::random_range(1..1u32 << 31)` (`rand` 0.9 is already a dependency, + `packages/cultnet-rs/Cargo.toml`), in `CultNetRudpSessionOptions::default` (`rudp.rs:108`), + `CultNetRudpSocketTransportOptions::client()` (`:1310`) and `::server()` (`:1329`), + `CultNetRudpServerHubOptions::new()` (`:1377`), and `CultMeshRudpSocketOptions::default` + (`cultnet-rs/src/cultmesh.rs:259`); + - C#: `RandomNumberGenerator.GetInt32(1, int.MaxValue)` as the property initializer at + `CultNetTransport.cs:773`, `:850`, **`:2404` (the listener's 100)**, **`CultNetRudpSchemaServer.cs:48` + (a second 100)** and `src/GameCult.Mesh/CultMesh.cs:547`; + - TypeScript: `crypto.getRandomValues` in the session constructor's `options.initialSequence ?? …` + (`rudp.ts:172`); `cultmesh-ts`'s optional `initialSequence` (`index.ts:4336`) already falls through to it; + - Python: `secrets.randbelow(2**31 - 1) + 1` as a `default_factory` at `transport.py:298`, `:323`, and + as the default of `client.py:396` and `cultmesh-py/src/cultmesh_py/facade.py:160`, `:194` (keyword + defaults become `None` meaning "draw one", since a Python default is evaluated once); + - Kotlin: in Cut K2; + - CultLib's own production pins take the default: `cultmesh-rs/src/lib.rs:964` (client), + `cultmesh-rs/src/rudp_document_server.rs:432` (server). The two TS publishers + (`signed-daemon-health.ts:116`, `idunn-odin-presence-publisher.ts:103`) lose theirs in whichever of + Cut 5 or Cut 1c lands second. Interop peers, the parity tool and tests keep explicit values on purpose. +- **Tests changed** (they encode payload identity or a fixed default): - `packages/cultnet-rs/tests/rudp_server_hub.rs:171-240` calls `connect()` again with the same payload - and expects no replacement. Under this rule that is a new generation. Its retransmit half is already - covered by `rudp_stray_packets.rs:487` (the same Connect packet twenty times), which stays; - - `tests/GameCult.Networking.Tests/NetworkingTests.cs:1507` replaces a peer by payload with two sessions - at the default initial sequence; it passes under a random default, but must say it relies on distinct - sequences; - - `packages/cultnet-ts/test/cultnet.test.ts:2225` (another endpoint replaces the peer) stays; - - any test that asserts a sequence value with the default initial sequence. + and expects no replacement; under this rule that is a new generation. Its retransmit half stays covered + by `rudp_stray_packets.rs:487`; + - `tests/GameCult.Networking.Tests/NetworkingTests.cs:1507` (replace a peer by payload) passes but must + say it relies on distinct sequences; `:700` asserts `PendingReliableSequences == {100}` against the + listener's default; + - asserts that may rely on a default Connect sequence of 1 (read each; pin `1` in the test if it does not + already): `cultnet-rs/tests/cultnet.rs:574`, `:820`, `:851-852`; `cultnet-ts/test/cultnet.test.ts:472`, + `:576`, `:592-593`, `:1536`; `cultnet-py/tests/test_cultnet.py:596`, `:764`, `:779-780`; + - `cultnet-ts/test/cultnet.test.ts:2225` (another endpoint replaces the peer) stays. - **Verification:** - - G1, G1b, G2 and the stale-Accept test in every runtime; + - G1 (every server above), G1b, G2, the stale Accept and the retransmitted-Connect tests; - the retransmit tests stay green: `rudp_stray_packets.rs:487`, `:511`, `NetworkingTests.cs:1483`, `cultnet.test.ts:2119`, `:2134`, `test_cultnet.py:1134`; - - negative grep: `rg -n "SequenceEqual\(connectPayload\)|connect_payload == packet.payload|movedEndpoint" src packages` + - a test per runtime that two default-constructed sessions draw different initial sequences in [1, 2^31); + - negative grep: `rg -n "SequenceEqual\(connectPayload\)|connect_payload == packet.payload|movedEndpoint|byteArraysEqual\(record.connectPayload" src packages` returns nothing; - mutation testing on the diff. -- **Ledger.** About −60 and +70 across four runtimes. Ten call sites lose an explicit initial sequence. +- **Ledger.** About −90 and +85 across four runtimes and the three servers. Two CultLib production pins go. -### Cut 2. A reliable fragment set is never evicted (receiver: Rust, TS, Python) — ready +**Pin-bump notes: callers outside CultLib that pin an initial sequence** (survey, 2026-09-30). A pinned +value keeps a deterministic Connect sequence. It matters on a **client that can restart on the same +endpoint and connection id** (the G1 lockout survives there); on a server or a one-shot client that binds a +fresh port per run it only forgoes the stale-packet protection. Recommendation at each repo's bump: delete +the value. + +| Repo@rev | Site | Value | Role | Consumes CultLib by | +|---|---|---|---|---| +| Muninn@`bd69598` | `crates/muninn-daemon/src/main.rs:5975`; `crates/sleipnir-daemon/src/main.rs:2223` | 1 | client (restart risk) | Cargo pin `c2a9a6e5` | +| Muninn@`bd69598` | `muninn-daemon/src/main.rs:780`, `:5426`; `sleipnir-daemon/src/main.rs:2265` | 1 | server-mode transports | same | +| Gjallar@`6a92976` | `src/Gjallar/Program.cs:431`, `:2955`, `:3027` | 1 | clients (restart risk) | sibling `../CultLib` | +| Mimir@`d88f2e6` | `src/Mimir.EveBrowserReference/Program.cs:682`, `src/Mimir.EveDashboard/Program.cs:801` | 1 | clients (restart risk) | sibling ProjectReference | +| Mimir@`d88f2e6` | `src/Mimir.CultMeshMedia/Program.cs:485` | `peers.Count + 1` | hand-rolled per-peer relay; its Connect handling (`:365`) is its own repeat decision | same | +| weksa@`d16cff5` | `scripts/weksa-rudp-command.cjs:88` (per-peer server loop), `:192` (client); `scripts/idunn-rudp-health.cjs:43` | 1 | server loop and clients | sibling CultLib | +| Bifrost@`42728a6` | `tools/persona-discord-rudp.mjs:9` | 100 | server-mode transport | sibling CultLib | +| Heimdall@`e8e2832` | `src/idunn-rudp-health.ts:65` | 1 | one-shot health client (replaced at its bump, § Sequencing) | submodule `b6b1d9c` | +| Stonks@`981c484`, Vili@`8773c15`, VoidBot@`46d891b` | `src/idunn-rudp.cjs:43`, `scripts/idunn-rudp.cjs:33`, `scripts/publish-idunn-rudp-health.cjs:64` | 1 | one-shot health clients | `file:../CultLib` / sibling path | + +- None in Odin `origin/main` (`5c37860`; Sleipnir now lives in Muninn), Idunn, Hermodr, CodexConnector, + Ghostlight, Huginn or Epiphany. StreamPixels' own code has none; its vendored CultLib (`542ddd3`, at + StreamPixels `a49f7d8`) carries CultLib's own old defaults and pins, which a bump replaces. +- Sibling-path consumers (Gjallar, Mimir, Stonks, Vili, VoidBot, weksa, Bifrost) take the random default + as soon as the local CultLib checkout moves; their pinned sites do not change behaviour until edited. + +### Cut 2. A reliable fragment set is never evicted (receiver: Rust, TS, Python) — in flight (`hands/ack-cut2`); merges with Cut F - **First.** Failing test P5 in each of the three. - **Change:** @@ -639,13 +949,41 @@ Mapped for Q-A6's recommended answer A. Blocked on Cut 1b (it uses the ending pr `rudp.ts:679-692`, `transport.py:836-842`); - if every pending set holds reliable fragments, the packet is refused and the session ends, as every other refusal does; - - a conforming sender cannot reach that bound: its window keeps at most 32 reliable packets in flight, - and the bound is 64 sets. + - ~~a conforming sender cannot reach that bound~~ (false for an expiring sender; see the pass 3 + correction below): its window keeps at most 32 reliable packets in flight, and the bound is 64 sets. - **Verification:** P5 delivers the reliable frame. `d3bc6e66`'s test (lossy fragment sets keep being evicted) stays green. `fragment_sets_evicted` still counts evictions. - **Ledger.** About −3 and +12 per runtime. +- **Pass 3 correction (from the Cut 2 Soul pass on `hands/ack-cut2` `c5026aa1`).** The third bullet above + is false for an expiring sender: the default Rust `media` channel expires reliable packets after 75 ms, + a frame's first fragment is delivered and acked, the rest are dropped by the deadline, and the set is + pinned; Soul's `soul_probe_expiring_reliable_frames_kill_receiver` (`packages/cultnet-rs/tests/cultnet.rs:3192` + at `3be97b34`) refuses the session on the 65th such frame, where eviction used to keep it alive. + - **What changes for the branch in flight:** nothing in its code. Its rule is right for every sender that + does not expire. What changes is when it merges: **Cut 2 merges only together with Cut F**, which + releases dead sets through notices and (Q-A9 B) keeps pre-Cut-2 eviction for a peer that never sends + notices. Soul's probe becomes Cut F's regression test. + - C# (`CultNetTransport.cs:921`) and Kotlin (K:2645) have no fragment-set bound at all. **Cut 2c** brings + C# to the same bound; Kotlin gets it in K1 and the gated rule in K3. -### Cut 3. The sender's flow window: 1,023 sequences and 4 MiB above the lowest unacked — ready +### Cut 2c. C# gets the fragment-set bound (with Cut 2's rule) — merges with Cuts 2 and F + +- **Why in this campaign.** Rust, TS and Python bound pending fragment sets at 64 (`d3bc6e66` lineage; + `rudp.rs:328-340`, `rudp.ts:167`, `transport.py:430`); C# keeps `_fragmentBuffers` + (`CultNetTransport.cs:921`) unbounded, so a hostile peer grows it without limit and the parity doc cannot + call C# conforming. A runtime missing a sibling's protection is a gap to fill. +- **Repo/branch:** CultLib `hands/ack-cut2c`, from `main`; merges together with Cuts 2 and F (the gate + lives in F). +- **First.** C# ports of `d3bc6e66`'s lossy-set test and Cut 2's P5 test. +- **Adds:** `MaxPendingFragmentSets` (default 64, a setter as Rust's `set_max_pending_fragment_sets`, + `rudp.rs:328-340`), a per-set last-touched ordinal, eviction of the stalest set in `Reassemble` + (`:1556`), `FragmentSetsEvicted` beside `OutstandingReliablePacketCount` (`:993`), and Cut 2's rule (a + reliable set is not evicted; if every set is reliable the session is refused through `EndRefused`), + gated as in Cut F. +- **Verification:** the two tests; `GameCult.Networking.Tests` green; Stryker.NET on the diff. +- **Ledger.** About +45 and 0. + +### Cut 3. The sender's flow window: 1,023 sequences and 4 MiB above the lowest unacked — ready (merge after Cut 1) - **First.** Failing tests P8 (all runtimes) and P4 (Rust receiver, with a sender from each runtime). - **Change.** Admission (`AdmitReliablePackets` / `PromoteQueuedReliable` at @@ -655,7 +993,8 @@ Mapped for Q-A6's recommended answer A. Blocked on Cut 1b (it uses the ending pr - its sequence is at most the lowest unacked sequence plus 1,023; - the admitted payload bytes above the lowest unacked sequence stay within 4 MiB. - The first packet is always admissible. Expired packets (Rust) do not count as unacked here: they are - gone from the wire. + gone from the wire. (Cut F refines this: an abandoned sequence counts while its notice is outstanding, + and stops counting once acknowledged or stranded.) - **Contract.** `docs/cultnet-transport-parity.md` (after `:263`) states the receive window: a receiver must hold 1,023 sequences and 4 MiB beyond its contiguous watermark, and a conforming sender never exceeds that. @@ -666,26 +1005,195 @@ Mapped for Q-A6's recommended answer A. Blocked on Cut 1b (it uses the ending pr - the Rust benchmark or media probe shows no throughput change on a lossless link. - **Ledger.** About +25 per runtime, and nothing removed. -### Cut 4. Rust receipts: Acknowledged means delivered, resolved once; honest expiry and queueing — blocked on Cuts 1 and 1b, and on Q-A3 +### Cut D. A failed send to one peer is that peer's lost datagram (Rust document server and hub, C# listener, cultmesh-py server) — ready -- **First.** Failing tests: P3 (the receipt for `s2` reads `Pending` until `s1` arrives, then - `Acknowledged`), P9, P10, P2 (`flush_reliable` must fail), and: a receipt `Acknowledged` before any - ending stays `Acknowledged` after the ending and after a reconnect; a `Pending` one becomes - `Invalidated`. +Design in §10. Independent of every other cut; it touches send sites, not session logic. + +- **Repo/branch:** CultLib `hands/ack-cutD` from `main` at `e9469cb3` or later. Merge cost only with Cut 1c + in the Rust hub's and C# listener's Connect arms. +- **First.** Failing tests, one per server, each with two peers where every send to peer X fails: + - cultmesh-rs document server: `poll_once` never returns `Err`; peer Y connects, publishes, is acked and + gets a snapshot response; `maintain` returns `Ok`; the failure count rises; X's session ends by idle + timeout, not by the failure. Also: `recv_from` returning `ConnectionReset` is `Idle`; + - Rust hub: `poll_resends` and `receive_event_once` return `Ok` and keep serving Y; + - C# listener: `PollResends` reaches Y after X throws; `TryReceiveOnce` serves Y; + - cultmesh-py server: the RUDP thread survives an `OSError` from `sendto` and keeps serving Y. + - **The failure seam.** The Rust servers own a `UdpSocket` directly, and a loopback peer's address + cannot be made to fail on demand. Add the narrowest seam at the one send function: a + `#[cfg(test)]` set of addresses whose sends fail with a synthetic `io::Error` in `send_packet` + (`rudp_document_server.rs:628-632`, `rudp.rs:1799-1804`), and the same as an internal test hook in C# + `SendPacket` (`CultNetTransport.cs:2829-2834`). Python patches `socket.sendto` in the test. A socket + port trait would be the full dependency-injection answer; it is not bought here, because the only + consumer is this test. +- **Changes:** + - cultmesh-rs `rudp_document_server.rs`: `send_packet` (`:628-632`) keeps encode errors as errors and turns + a `send_to` failure into a counted loss; every per-peer call site drops its `?` on the send + (`:335`, `:373`, `:397`, `:451`, `:587`, `:603`); a `send_failures()` accessor beside `packets_dropped()` + (`:246-248`); `recv_from` treats `ConnectionReset` like `WouldBlock` (`:266-272`), as the socket + transport does (`cultnet-rs/src/rudp.rs:2059-2066`); + - cultnet-rs hub: `send_packet` (`rudp.rs:1799-1804`) likewise, counted in `CultNetTransportStats` + as a new `send_failures` field; `poll_resends` (`:1762-1777`) and the receive path (`:1644`, `:1673`, + `:1708`, `:1711`, `:1714`, `:1759`) stop propagating it; + - C# `CultNetRudpSocketTransportServer`: `SendPacket` (`:2829-2834`) catches `SocketException`, counts it + in `CultNetTransportStats.SendFailures`, and returns; `PollResends` (`:2796-2805`) and `TryReceiveOnce`'s + sends (`:2687`, `:2704`, `:2749-2750`, `:2788`) need no other change; the refusal goodbye's own catch + (`:2739-2745`) becomes redundant and goes; + - cultmesh-py `server.py`: `_send_rudp_packet` (`:302-307`) catches `OSError` and counts it. +- **Not changed:** single-peer socket transports (a failed send already concerns only their one peer, and + their callers, flush among them, get the error); `cultmesh-ts`'s document server (Node reports send + failures on the socket's `error` event, `index.ts:5858`). +- **Authority map:** + - Owner: each peer's session owns its peer's fate (idle timeout, lifetime, refusal, Disconnect). A send + failure is an input to nothing but a counter. + - Forbidden writers: no per-peer send failure ends a poll, a loop, a thread or another peer's session. +- **Verification:** + - the four tests; the full `cultmesh-rs`, `cultnet-rs`, `GameCult.Networking.Tests` and `cultmesh-py` + suites; + - negative: `rg -n "send_packet\(.*\)\?;" packages/cultmesh-rs/src/rudp_document_server.rs` returns + nothing (the pattern matches only a propagated send; encode errors propagate inside `send_packet`); + - operator: an Odin build on this CultLib keeps serving presence while one peer's address is + unroutable. +- **Ledger.** About −15 and +40 (the counters, the seams); tests about +120. + +### Cut F. Abandonment notices (FORWARD-TSN, RFC 3758): every receiver honours one, Rust sends one — blocked on Cuts 1 and 3, on Q-A7 and Q-A9; merges together with Cut 2 + +Design in §6. Mapped for the recommended answers to Q-A7 (one sequence per notice) and Q-A9 (a capability +bit gates Cut 2's rule). + +- **Repo/branch:** CultLib `hands/ack-cutF` from `hands/ack-cut2` rebased on `main` after Cuts 1 and 3 + merge. Cut 1 gives the receiver `R` and the drain the notice feeds; Cut 2 is the fragment code the notice + drops sets from and the rule the capability gates; Cut 3 is the admission rule the notice amends. Cut 2 + and Cut F merge to `main` together (the Cut 2 Soul finding: Cut 2 alone refuses sessions with expiring + senders). Kotlin's half is Cut K3. +- **First.** Failing tests, in C#, Rust, TypeScript and Python: + - the golden: `Abandon` encodes to the §6 bytes and decodes back, beside the existing fixture + (`packages/cultnet-rs/tests/cultnet.rs:319`, `packages/cultnet-ts/test/cultnet.test.ts:311`, + `packages/cultnet-py/tests/test_cultnet.py:409`, `tests/GameCult.Networking.Tests/NetworkingTests.cs:366`); + - a type-4 packet with bit 4 decodes as `Abandon`; bit 4 on a Data packet is still ignored; + - receiver: a notice for a hole advances `R`, drains the held ordered frames behind it, and replies with + the ack for that sequence (FN, inverted: `[o3]` delivered by the notice, `o2` marked received); + - receiver: a notice for a received sequence changes nothing and still replies; + - receiver: a notice inside an incomplete fragment set's span drops the set; + - receiver: a notice more than 1,024 above the highest received is ignored with no reply; + - receiver: the late original after its notice is a duplicate: acked, not delivered; + - Rust sender: an expired pending or queued packet produces a notice at once and then once per resend + delay; an ack naming the sequence stops it; after 16 unacknowledged sends it stops and records + `first_stranded`; every ending clears both; + - Rust end to end (P2 inverted): reliable `media` with expiry is lost, then ordered `s1`, `s2` on + `schema`; after the notice both are delivered and `flush_reliable` succeeds; + - Rust, Cut 3's window: an outstanding notice holds the lowest-unacked point; a stranded sequence does not. + - **the Cut 2 regression**, from Soul's `soul_probe_expiring_reliable_frames_kill_receiver` + (`packages/cultnet-rs/tests/cultnet.rs:3192` at `3be97b34`): 100 default `media` frames, each losing + every fragment but the first. With a Cut F sender the receiver's pending sets return to zero after the + notices and the session lives; with a stand-in sender that never advertises the capability nor sends + notices (Q-A9 B), the receiver evicts the stalest set as before Cut 2 and the session lives; with a + capable peer that stops sending notices, the 65th pinned set still refuses (Cut 2's rule holds where it + is true); + - (Q-A9 B) a Connect or Accept with flag bit 4 records the peer as capable; one without does not; an old + peer's accept of a flagged Connect works (the pass 3 CAP probe, kept as a test). - **Deletes first:** - - the session scope (`rudp.rs:135`, `:179`, `:206`, `:283-285`, the check at `:291-293`, and what - Cut 1b left of the rotations); + - Rust: the plain drop in `purge_expired_reliable` (`rudp.rs:898-908`; the `retain` calls become a move + into the abandoned set; the expired counter stays); + - the doc on `reliable_expire_after_ms` (`rudp.rs:121-124`), which says the packet is dropped; + - `packages/cultnet-py/README.md:103-111`'s claim that the Python session has reliable expiry. It has + none; the claim is false today. +- **Adds, receiver (C#, Rust, TS, Python):** + - the `Abandon` packet type and its codec mapping: Rust enum `rudp.rs:51-59`, `encode_flags` + `:2466-2475`, `packet_type_to_code` `:2477`, `decode_rudp_packet` `:2448-2455`, `packet_type_from_code` + `:2571`; C# `CultNetRudpPacketType` (`CultNetTransport.cs:509`) and `CultNetRudpPacketCodec` `EncodeFlags` `CultNetTransport.cs:697`, + `Decode` `:676-690` (its `Enum.IsDefined` check runs on the wire byte, which stays 4); TS union + `rudp.ts:26` and the codec at `:1264-1320`; Python enum `transport.py:250`, code table `:342`, codec + `:1175-1240`; + - the `Abandon` branch in `receive`, after acks are applied and beside the Ack/Pong branch + (`rudp.rs:578-592`, `CultNetTransport.cs:1177-1187`, `rudp.ts:397`, `transport.py:612`): the range + check, the fragment-set drop over the fragment buffers (`rudp.rs:1001-1114`, `CultNetTransport.cs:1556`, + `rudp.ts:643`, `transport.py:825`), `RememberReceived` and the Cut 1 drain, and + `reply = create_ack_for_received(sequence)` (`rudp.rs:751-758`, `CultNetTransport.cs:1273-1279`, + `rudp.ts:492-497`, `transport.py:685`); + - transports skip their own ack for an `Abandon` packet where a delivering receive would draw one: + `rudp.ts:1089`, `transport.py:1087` (Rust and C# ack only reliable, Accept or Data and need nothing). +- **Adds, capability (Q-A9 B; all four runtimes):** + - `Connect` and `Accept` carry flag bit 4; the session records whether its peer's handshake packet had + it, and resets that with peer state. `create_connect` (`rudp.rs:369-381`, `CultNetTransport.cs:1001-1010`, + `rudp.ts:229-240`, `transport.py:473-483`) and `accept_connect` (`rudp.rs:383-411`, + `CultNetTransport.cs:1012-1059`, `rudp.ts:242-290`, `transport.py:485-525`) set it; the codec keeps bit 4 + on those types (today every decoder drops it); + - the fragment-set bound (Cut 2's eviction, `rudp.rs:1027-1047`, `rudp.ts:679-692`, `transport.py:836-842` + as Cut 2 leaves them) applies the never-evict rule to a capable peer and the pre-Cut-2 stalest-set + eviction to one that is not; + - the sender sends notices only to a capable peer; for another, an expired sequence is stranded at once. +- **Adds, sender (Rust):** + - `RUDP_ABANDON_NOTICE_SENDS = 16` beside `DEFAULT_MEDIA_RELIABLE_EXPIRE_AFTER_MS` (`rudp.rs:39`); + - the abandoned set and `first_stranded` on the session (fields beside `pending_reliable`, + `queued_reliable`, `rudp.rs:193-195`); + - `purge_expired_reliable` moves expired pending and queued sequences into the set; + - `due_resends` (`:797-807`) emits due notices after due packets and retires entries at the bound; + - `apply_acknowledgements` (`:964-971`) removes named sequences from the set; + - `outstanding_reliable_packet_count` (`:270-272`) counts the set; + - the ending primitive (Cut 1b) and `reset_peer_state` (`:344-358`) clear both; + - Cut 3's lowest-unacked point (in `admit_reliable_packets` / `promote_queued_reliable`, `:910-942` as + Cut 3 leaves them) includes the set and excludes `first_stranded`. +- **Docs.** `docs/cultnet-transport-parity.md`: the packet-type and flags rows of the header table + (`:486-487`) gain "`4` with flag bit 4 = abandon"; the golden goes after the canonical fixture + (`:505-517`); the bounded-send paragraph (`:260-273`) gains the notice rule and its resend bound. +- **Authority map:** + - Owner: the sender session's abandoned set decides which sequences are announced and for how long; the + receiver session decides what a notice changes. + - Inputs: expiry (sender); a decoded `Abandon` packet in the current generation (receiver). + - Outputs: notices (sender); a remembered sequence, drained frames and one ack (receiver). + - Derived: `first_stranded` is derived from the set's retirements and read by Cut 4; the expired counter + is report-only. + - Forbidden writers: nothing marks a sequence received without a packet except the `Abandon` branch; no + notice outlives its generation; no transport interprets bit 4. + - Shared paths: data, fragments, duplicates, Accept and notices all advance `R` through + `RememberReceived` and the one drain. + - Deletion line: the plain drop in `purge_expired_reliable` goes before the set is added. +- **Verification:** + - the tests above in four runtimes; + - FN re-run against the new build as a **negative** check: an old-receiver stand-in (a session that drops + `Abandon` packets) never acks, and the Rust sender strands the sequence after 16 sends; + - the TS interop harness gains one lane: a Rust peer sends reliable `media` with expiry whose first + packet the bridge drops, then ordered `schema` frames; the TypeScript receiver delivers the `schema` + frames (`packages/cultnet-ts/test/interop/cultnet-interop.test.ts`, beside the drop lanes); + - negative grep: `rg -n "first_abandoned" packages src` returns nothing (pass 2's name must not appear); + - mutation testing on the diff. +- **Ledger.** Receiver about +40 per runtime (four); Rust sender about +60 and −6; the golden about +25 + per runtime; docs +15 and −8 (the false Python claim). + +### Cut 4. Rust receipts: Acknowledged means delivered, resolved once; honest stranding and queueing — blocked on Cuts 1, 1b and F + +Re-mapped in pass 3 for Q-A3 B. Pass 2's `first_abandoned` (every expiry leaves a permanent hole) is +replaced by Cut F's `first_stranded` (only a notice the receiver never acknowledged leaves one). The rest is +pass 2's cut. + +- **Repo/branch:** CultLib `hands/ack-cut4` from `main` after Cuts 1, 1b and F merge. +- **First.** Failing tests: + - P3: the receipt for `s2` reads `Pending` until `s1` arrives, then `Acknowledged`; + - P9 (`send_reliable` on an expiring channel is refused) and P10 (a window-full `send_reliable` returns a + receipt covering its queued packets, and the frame is sent once); + - a receipt `Acknowledged` before any ending stays `Acknowledged` after the ending and after a reconnect; + a `Pending` one becomes `Invalidated`; + - **P2 with a new receiver:** reliable `media` with expiry is lost, then ordered `s1`, `s2`; their + receipts read `Pending` while the notice is outstanding and `Acknowledged` once it is acked; + `flush_reliable` succeeds; + - **P2 with a deaf receiver** (a stand-in session that drops `Abandon` packets): after 16 notice sends the + receipts for `s1` and `s2` stay `Pending`, `flush_reliable` fails at once naming the stranded sequence, + and the ending makes them `Invalidated`. +- **Deletes first:** + - the session scope (`rudp.rs:135`, `:179`, `:206`, `:283-285`, the check at `:291-293`, and what Cut 1b + left of the rotations); - the stale comment and the half guard at `rudp.rs:1915-1920`; - the hand-rolled status loops at `cultmesh-rs/src/lib.rs:806-838` and `:870-900`. - **Adds:** - the receipt's status cell and ordered flag; the session's unresolved-receipt list, resolved once (Acknowledged through `A`, Invalidated by the ending primitive); - - `acked_through` (`A`), reading pending, queued and first-abandoned; - - first-abandoned, recorded in `purge_expired_reliable` (`rudp.rs:898-908`); + - `acked_through` (`A`): one below the lowest sequence that is pending, queued, in Cut F's abandoned set, + or `first_stranded`; - the receipt covers queued sequences: `send_many` (`:456`) reports every sequence it assigned; - `send_reliable` (`:1903-1930`) refuses a channel with expiry; - `wait_acknowledged`; - - `flush_reliable` fails fast when an ordered send follows the first abandoned sequence. + - `flush_reliable` fails fast when an ordered send follows `first_stranded`, and otherwise waits while + notices are outstanding (Cut F counts them in `outstanding_reliable_packet_count`). - **Unchanged:** `CultNetRudpReliableSendStatus` (`:146-150`), `reliable_send_status`'s signature (`:1931`), and every caller of `Acknowledged`: `tests/rudp_stray_packets.rs:309`, `cultmesh-rs/tests/rudp_document_server.rs:408,415`, CodexConnector, Ghostlight. @@ -693,11 +1201,11 @@ Mapped for Q-A6's recommended answer A. Blocked on Cut 1b (it uses the ending pr delivered to the document server's sink. - **Verification:** - the new tests pass; - - `rg -n "session_scope" packages/cultnet-rs` has no hit; + - `rg -n "session_scope|first_abandoned" packages/cultnet-rs` has no hit; - both suites are green; - cargo-mutants on the diff. -- **Ledger.** About −90 (two loops of about 30 each, the scope) and +90 (of which `wait_acknowledged` is - about 35). +- **Ledger.** About −90 (two loops of about 30 each, the scope) and +85 (of which `wait_acknowledged` is + about 35). Five lines smaller than pass 2: the abandoned bookkeeping moved to Cut F. ### Cut 5. CultLib-internal TS publishers and the contract text (TS, docs) — blocked on Cut 3, and on Cut 1b in TS @@ -722,24 +1230,164 @@ Mapped for Q-A6's recommended answer A. Blocked on Cut 1b (it uses the ending pr - a new test shows the publisher's peer receives a Disconnect. - **Ledger.** About −270 and +50. +### Cuts 6a, 6b, 6c. Receipts in C#, TypeScript and Python — blocked on Cut 4, and on Cut 1b in each runtime -## Subtraction ledger (estimate, four runtimes) +Design and public API in §9. One cut per runtime, in parallel worktrees, so Soul can falsify each alone. +Kotlin's is Cut K4. + +- **Repo/branch:** CultLib `hands/ack-cut6a` (C#), `hands/ack-cut6b` (TS), `hands/ack-cut6c` (Python), each + from `main` after Cut 4 merges. Cut 4 is the reference; Cut 1b supplies each runtime's ending primitive. +- **First.** Failing tests, in each runtime: + - an ordered receipt reads Pending while a sequence below it is missing, then Acknowledged (P3); + - a send made while the window is full returns a receipt that covers its queued packets and becomes + Acknowledged; the frame is sent once (P10's shape); + - an Acknowledged receipt stays Acknowledged after the session ends and after a reconnect; a Pending one + becomes Invalidated and stays so; + - the wait returns on Acknowledged, fails at once on Invalidated with the end reason, and times out; + - `SendReliable` on an unreliable channel is refused. +- **Deletes first:** nothing. These runtimes have no receipt code to replace. +- **Adds, per runtime:** + - C# (6a): `CultNetRudpSendReceipt`, `CultNetRudpSendStatus`; in `CultNetRudpSession`, a send path that + records every sequence `SendMany` assigns (`:1075-1140`, before `AdmitReliablePackets` `:1439-1465` + splits them), the unresolved list, resolution after `ApplyAcknowledgements` (`:1503-1516`) and in the + Cut 1b primitive, and `A` over `_pendingReliable` and `_queuedReliable`; on the socket transport, + `SendReliable`, `WaitAcknowledged` (the `FlushReliable` loop, `:1914-1955`), and `SendSchema` + (`:1960`, `:1968`) and `SendSchemaMessage` (`:1976`) returning the receipt; + - TS (6b): `CultNetRudpSendReceipt`, the status union; session changes at `sendMany` (`rudp.ts:304-368`), + `#admitReliablePackets` (`:586-592`), `#applyAcknowledgements` (`:610-617`); transport + `sendReliable` and `waitAcknowledged` beside `send` (`:899-909`) and `flush` (`:911-935`); exported from + the package index; + - Python (6c): `CultNetRudpSendReceipt`, `CultNetRudpSendStatus`; session changes at `send_many` + (`transport.py:538-588`), `_admit_reliable_packets` (`:776-786`), `_apply_acknowledgements` + (`:803-807`); transport `send_reliable` and `wait_acknowledged` beside `send` (`:1000-1009`) and + `flush_reliable` (`:1101-1131`); exported from `cultnet_py`; + - README sections per §9, and the parity doc's flush paragraph. +- **Authority map:** the session owns each receipt's status and writes it once; the transport reads it. + Forbidden writers: no transport, flush or timer writes a status; no receipt resolves from a packet's own + ack when it is ordered (only through `A`). +- **Verification:** the tests above; each runtime's full suite; the Cut 4 behaviour table held side by side + with Rust's (same inputs, same states); negative: in each runtime, the only assignment to the status is + in the session's resolve and end paths (`rg` for the status field outside those two functions returns + nothing); mutation testing on each diff (Stryker.NET, StrykerJS, mutmut). +- **Ledger.** About +90 per runtime, nothing removed; tests about +80 per runtime. + +### Cut 7. Reliable expiry in C#, TypeScript and Python — blocked on Q-A8 and Cut F + +Mapped only for Q-A8's recommended answer A. Each runtime gains Rust's send option (a per-send reliable +expiry, and the profile's `media_reliable_expire_after_ms`, `rudp.rs:2492-2508`) and Cut F's sender half +(the abandoned set, notices at the resend delay, the 16-send bound, `first_stranded`), and its receipts +(Cut 6) refuse an expiring channel. Kotlin's is K5. About +70 per runtime. If Q-A8 is B, this cut is not +built and the parity doc records Rust as the only runtime with partial reliability. + +### Cut K. Kotlin catches up (Q-A5 A, ruled) — K0 ready now; K1-K5 trail the four-runtime cuts + +`CultMesh.kt` (6,332 lines, the only RUDP file) has none of the `3bf1c0c` stray-packet hardening, the P1 +and P6 receiver, no reset path and no repeated-Connect answer. K is split so each step is falsifiable on its +own; the anchors below are `e9469cb3`, where `CultMesh.kt` is identical to `3bf1c0c`. Tests follow the +file's existing pattern (private self-test functions called from `main`, K:3349-3373); moving them out of +the production source is a follow-up. + +**K0. The Kotlin verify image and a Linux build path — ready now.** +- **Stopgap tooling** (`C:\Users\Meta\.claude\skills\eureka\tools\stopgap\`, owned by Self, not a CultLib + commit; it dies with `ygg-verify.sh`): + - `kotlin.Dockerfile`: `FROM eclipse-temurin:21-jdk`; `apt-get install unzip curl`; fetch + `https://github.com/JetBrains/kotlin/releases/download/v2.2.21/kotlin-compiler-2.2.21.zip` (the + version CI pins, `.github/workflows/cultnet-interop.yml:20`) into `/opt/kotlinc`; `ENV + KOTLIN_HOME=/opt/kotlinc PATH=/opt/kotlinc/bin:$PATH`; Node 24 copied from the `node:24` image + (`COPY --from=node:24 /usr/local /usr/local`) for the interop lane; PowerShell 7 (`pwsh`, Microsoft's + Debian package) for `build.ps1`. Pass 3's probe proved the JDK-plus-compiler half: the self-tests exit 0 + (K-img). + - `ygg-verify.sh`: a `kotlin` image alias tagged by that Dockerfile's hash, and the remote build step + generalised from `eureka-verify-rust:*` to `eureka-verify-:*` built from `.Dockerfile` + (today it builds and copies only `rust.Dockerfile`). Edit it by writing a temp file and renaming it over + the old one (the script's own rake). +- **CultLib:** + - `packages/cultmesh-kotlin/build.ps1` becomes portable PowerShell 7: `kotlinc` or `kotlinc.bat` by + platform, `[IO.Path]::PathSeparator` for the classpath, `java` or `java.exe`. One build script stays the + one owner of the Kotlin build; no `build.sh` beside it. + - The interop harness calls `pwsh` off Windows instead of `powershell` + (`packages/cultnet-ts/test/interop/cultnet-interop.test.ts:2799-2809`); it already picks `java` off + Windows (`:31-36`). + - `docs/cultnet-transport-parity.md` says, beside the runtime table (`:35`) and the RUDP claims + (`:205-273`), that Kotlin lacks the `3bf1c0c` hardening and the ack cuts until Cut K lands, so nobody + reads it as conforming meanwhile. +- **Verification:** `ygg-verify.sh kotlin 'cd packages/cultmesh-kotlin && pwsh -File build.ps1'` + runs the self-tests green; the interop lane + (`node --test --test-name-pattern "TypeScript and Kotlin|Kotlin and TypeScript"`) runs green on the same + image; CI on `windows-latest` stays green with the same script. + +**K1. Stray-packet parity** (the `3bf1c0c` behaviours the other four have, each with its test ported from +the C# or TS equivalent): +- a malformed, foreign-id or refused datagram is dropped and counted, never thrown out of `receiveOnce` + (K:3223, K:3228, K:2724, K:2567), with a `packetsDropped` stat (`CultNetTransportStats`, K:774); +- the connection id is checked before the endpoint is claimed, and in server mode only a Connect claims it + (K:3217-3222); +- `resetPeerState` and `endRefusedSession`: the refusing session resets, builds its goodbye after the + reset, and a server forgets its endpoint; a refused packet's sequence is remembered only after + reassembly accepts it (today K:2758 remembers before `reassemble` can throw, K:2915-2922); +- `answerRepeatedConnect`: a repeated Connect queues nothing and is answered with the owed Accept or an Ack + (today every Connect makes a new tracked Accept, K:2667-2676, K:3224-3226, K:4005-4006); +- a Connect from another endpoint replaces the peer in server mode (today ignored, K:3220-3221); +- a peer's Disconnect drops owed writes (K:2745-2748); `connect` clears a stale disconnect reason (K:3106); +- `initialSequence` of `0xFFFFFFFF` is refused at construction (K:2631) and sequence exhaustion is checked + before allocation (K:2841, K:2698-2711); +- the decoder's negative declared length (K:3306-3317) is an ordinary malformed drop; +- the fragment-set bound of 64 (K:2645, K:2911-2937 has none): the stalest set is evicted. Cut 2's + never-evict rule and its capability gate come in K3 with Cut F, as in the other runtimes. + +**K2. Cuts 1, 1b and 1c.** +- Cut 1: delete `orderedNextSequenceByChannel` (K:2643), `expectedSequenceIfUninitialized` (K:2728), + `deliverOrdered`/`drainOrdered`/`skipReceivedNonChannelSequences` (K:2939-2978); seed `R` in + `acceptConnect` (K:2667) and on Accept (K:2730); advance in `rememberReceived` (K:2891). +- Cut 1b: the ending primitive from a peer's Disconnect (K:2745), `createDisconnect` (K:2809), + `checkTimeout` (K:2814-2819) and K1's refusal; `flushReliable` (K:3186-3205) binds to the generation. +- Cut 1c: random defaults at K:2600, K:2996, K:3078 and the server's 100 at K:3016 + (`java.security.SecureRandom`); the session's repeat decision replaces server mode's + every-Connect `acceptConnect` (K:3224-3226) and the interop server's (K:4005); `createConnect` (K:2660) + and the Accept check (K:2730). + +**K3. Cuts 2, 3 and F.** Admission at K:2856-2872 gains the flow window; Cut 2's reliable-set rule behind +the Q-A9 capability; the `Abandon` type and receiver branch (K:2556-2567, K:2738, K:3296-3325), the golden beside `rudpPacketCodecUsesDeterministicReliableOrderedFixture` +(K:5236), and the transport skips its own ack for `Abandon` (K:3243-3245). + +**K4. Receipts** (§9): `sendReliable`, `CultNetRudpSendReceipt`, `CultNetRudpSendStatus`, `waitAcknowledged`, +at `sendMany` (K:2685-2719), `admitReliablePackets` (K:2856-2862), `applyAcknowledgements` (K:2882-2889) +and the transport (K:3132-3205). + +**K5. Reliable expiry and the notice sender** — only if Q-A8 is A (as Cut 7). + +- **Verification, each step:** the Kotlin image's self-tests and interop lane; the step's ported tests; + a negative grep per step (K2: `rg -n "orderedNextSequenceByChannel|expectedSequenceIfUninitialized" + packages/cultmesh-kotlin` returns nothing). +- **Ledger.** K1 about +180 and −40; K2 about −70 and +90; K3 about +80; K4 about +90; K5 about +70. + + +## Subtraction ledger (estimate) | Cut | Removed | Added | Other | |---|---|---|---| | 1 | ~280 | ~160 | four per-channel state machines become one watermark | | 1b | ~40 | ~45 | one ending primitive; TS's transport generation pair goes | -| 1c | ~60 | ~70 | four transport-level repeat decisions become one session predicate | -| 2 | ~10 | ~35 | none | +| 1c | ~90 | ~85 | seven transport- and server-level repeat decisions become one session predicate; two CultLib production pins go | +| 2 | ~10 | ~35 | none; merges with Cut F | +| 2c | 0 | ~45 | C# gains the fragment-set bound the other runtimes have | | 3 | 0 | ~100 | a documented receive-window contract | -| 4 | ~90 | ~90 | one public method (`wait_acknowledged`); the session scope goes; no enum change | +| D | ~15 | ~40 | one failure counter in three servers; a test seam at each send function | +| F | ~15 | ~350 | **wire: one packet type (`Abandon` = Ack + flag bit 4) and one golden in every runtime; with Q-A9 B, the same bit on Connect and Accept**; the false Python expiry claim goes | +| 4 | ~90 | ~85 | one public method (`wait_acknowledged`); the session scope goes; no enum change | | 5 | ~270 | ~50 | two hand-rolled RUDP loops and cultmesh-ts's connect wait go | +| 6a-6c | 0 | ~270 | public receipt API in three runtimes (Q-A4 A) | +| 7 | 0 | ~210 | only if Q-A8 is A: partial reliability in three runtimes | +| K0-K5 | ~110 | ~510 | Kotlin reaches parity; one verify image; `build.ps1` portable | -- **Source, net:** about −195. Tests are extra: about 4 × 10 rule tests. -- **Wire:** zero bytes. -- **Downstream:** no caller has to change code at its next bump. Callers that fixed `initial_sequence` - keep a deterministic Connect sequence, and with it the restart lockout, until they drop the value. -- **Kotlin (Q-A5 A):** Cuts 1, 1b, 1c and 3 again, about the same size as one runtime each. +- **Source, net, four runtimes without Cut 7:** about +325. Pass 2 was about −195; the rulings bought + FORWARD-TSN with its capability gate (about +335 net) and receipts in three runtimes (about +270); Cut D + adds about +25 and Cut 2c about +45. Tests are extra: + about 4 × 25 rule tests. +- **Kotlin:** about +400 net, most of it the `3bf1c0c` hardening the other four already carry. +- **Wire:** one additive packet type, ignored by every current peer (FN). +- **Downstream:** no caller has to change code at its next bump. Callers that pin `initial_sequence` keep + a deterministic Connect sequence until they drop the value (Cut 1c, pin-bump notes). ## Operator questions @@ -748,146 +1396,148 @@ Each question stands alone. **Ruled (2026-09-30), recorded so they are not re-asked:** -- **Q-A1. Where does delivered come from?** (a): derived at the sender from the cumulative watermark. No - wire change. +- **Q-A1. Where does delivered come from?** (a): derived at the sender from the cumulative watermark. - **Q-A2. One flush, or a "received" flush beside it?** One meaning: Acknowledged means delivered. - Receipts are Pending, Acknowledged or Invalidated. Retransmission stops on each packet's own ack. Flush - waits for Acknowledged. +- **Q-A3. Reliable expiry next to ordered traffic (P2)?** B: FORWARD-TSN now (Cut F). The recommendation + was A (report honestly, document "don't mix"). +- **Q-A4. Receipts in C#, TypeScript, Python and Kotlin?** A: port them in this campaign, after Cut 4 + (Cuts 6a-6c, K4). The recommendation was B (record a parity gap). +- **Q-A5. Is Kotlin in the campaign?** A: yes, as trailing Cut K on a Yggdrasil Kotlin image (K0). +- **Q-A6. How does a session recognise a new generation?** A: the Connect's sequence identifies it; the + default initial sequence is random in [1, 2^31) (Cut 1c). **Open:** -- **Q-A3. Reliable expiry next to ordered traffic (P2).** An expired reliable packet leaves a permanent - hole, and every ordered frame after it on the same session is held until the session ends. - - **A.** Report it honestly (Cut 4): ordered receipts after the hole stay `Pending` and become - `Invalidated` when the session ends, and flush fails fast. Document "do not send ordered traffic on a - session that uses reliable expiry". No live consumer does. - - **B.** PR-SCTP's FORWARD-TSN (RFC 3758). On expiry the sender sends an abandonment notice, which rides - an Ack-typed packet with flag `0x10` so that old peers ignore it (P7). New receivers treat the sequence - as received with no frame, and acknowledge it. The sender resends the notice until it is acknowledged. - That is a new cut in every receiver. Old receivers never acknowledge it, so the resend needs its own - bound. - - **C.** Refuse ordered sends on a session after its first expiry. - - **D.** Delete reliable expiry. Muninn's `audio` is its only user and `hands/media-fec-cut3` retires it. - This subtracts, but CultLib would lose partial reliability, which a reasonable consumer can expect. - - **Recommended: A now.** Record B as the complete answer for when a consumer mixes the two. - - **Blocks:** Cut 4 only. A keeps Cut 4 as mapped (`first_abandoned` alone). B adds a wire cut in every - receiver before Cut 4. C changes Cut 4's `send_many` guard. D replaces Cut 4's expiry half with a - deletion. Cuts 1, 1b, 1c, 2, 3 and 5 are unaffected. +- **Q-A7. Does an abandonment notice name one sequence, or a new cumulative point? (new in pass 3)** + RFC 3758's FORWARD TSN carries a *New Cumulative TSN*: "every missing sequence up to here counts as + received". SCTP can do that because its SACK is cumulative too, so the next SACK confirms it. CultNet's + ack is not cumulative: it names the highest received sequence plus the 32 below it, or one exact older + sequence. So the two shapes cost different things: + - **A. One sequence per notice.** The receiver answers exactly as it would answer a data packet with + that sequence, and that ack stops the sender's notice as it stops a retransmit. No new ack field. The + cost is one small notice per abandoned packet per resend: an expired 20-fragment frame sends 20 + notices, again every resend delay until acked. + - **B. A cumulative point.** One notice covers every abandoned sequence up to it. But nothing on the + wire can confirm it: an ack naming the point might mean the original of that one packet arrived late, + while lower holes are still open. So B needs a second addition, a confirmation Ack (another flag bit) + carrying the receiver's watermark, which puts a cumulative watermark on the wire after all, the thing + Q-A1 kept off it. + - **Recommended: A.** It keeps the wire change to one flag, reuses the ack path unchanged, and the extra + notices only flow while packets are expiring, which is a lossy moment on one session. + - **Blocks:** Cut F (and K3). Nothing else. -- **Q-A4. Receipts in C#, TypeScript, Python and Kotlin?** - - Only Rust has per-send receipts. The other four offer flush and an outstanding count. - - **A.** Port receipts with the same three states (Pending, Acknowledged, Invalidated) and a - `WaitAcknowledged`, in this campaign. - - **B.** Record it as a parity gap. Flush covers every surveyed consumer. - - **Recommended: B.** No consumer outside Rust waits per send. A reasonable C# consumer might want it, but - it is additive later, changes no wire byte, and after Cut 4 the Rust design (a status cell resolved - once by the session) ports without a second design. - - **Blocks:** nothing mapped. A adds a cut of about +80 per runtime after Cut 4. +- **Q-A8. Should C#, TypeScript, Python and Kotlin gain reliable expiry? (new in pass 3)** Rust is the only + runtime that can send a reliable packet that gives up after a deadline (partial reliability, for live + media). Cut F teaches every runtime to *receive* the notice, because any of them can talk to a Rust + sender; only Rust can *send* one. A reasonable consumer of the C# or TS library can expect the same send + option. Today the profile field that configures it (`media_reliable_expire_after_ms`, + `rudp.rs:2492-2508`) exists only in Rust and cultmesh-rs. + - **A. Yes, in this campaign** (Cut 7 for C#, TS, Python; K5 for Kotlin): the send option, the profile + field's behaviour and Cut F's sender half. About +70 per runtime. + - **B. Not now.** The parity doc records Rust as the only runtime with partial reliability. + - **Recommended: A.** CultLib is judged by what a reasonable consumer expects (operator, 2026-09-30), and + after Cut F the sender half is small and already specified. It is independent of every other cut, so it + can go last. + - **Blocks:** Cut 7 and K5 only. -- **Q-A5. Is Kotlin in the campaign?** - - `cultmesh-kotlin` carries a fifth RUDP runtime with P1 and P6 (the same code, `CultMesh.kt:2939-2978`). - Its flush has no callers. The stray-packet work did not touch it, so none of those fixes reached it - (not probed in pass 2). Its only build path is - `build.ps1`, which needs Android Studio's `kotlinc` and JBR on Windows. Yggdrasil has no Kotlin image in - the verify list. - - **A.** Include it, as a trailing Cut K that applies Cuts 1, 1b, 1c and 3, verified on a Yggdrasil - Kotlin image added first (or on Starfire as a Windows-only job). - - **B.** Record it as parity debt, with the stray-packet gaps. Mark the parity doc so that nobody reads - Kotlin as conforming. - - **Recommended: A, with a Yggdrasil Kotlin image.** Wire parity is the library's promise. A runtime - that strands acknowledged frames should not stay listed as conforming. - - **Blocks:** Cut K only. Cuts 1, 1b, 1c and 3 proceed in four runtimes either way; pass 1 had Q-A5 - blocking their scope, and pass 2 removes that by making Kotlin trail. - -- **Q-A6. How does a session recognise a new generation? (new in pass 2)** Today a server treats a - Connect from a connected endpoint as a repeat by payload (Rust hub, C# listener) or unconditionally (C#, - TS and Python server mode), so a restarted client with the same payload is locked out (G1), and a - client's second `connect()` is not seen as new, so the client cannot safely forget the old server - session's sequences (G2). The Connect's bytes are identical for a retransmit and for a restarted process - that starts at the same initial sequence, so something has to differ. - - **A.** The Connect's sequence identifies the generation, and the default initial sequence is random in - [1, 2^31), as TCP's initial sequence number is (RFC 9293 §3.4.1, RFC 6528). A Connect repeats exactly - when it carries the current generation's Connect sequence. No wire change. The default of a public - option changes (from 1, or 100 for the C# listener); an explicit value still works for tests, and a - caller that fixes it keeps the lockout. - - **B.** Proof of receipt. A byte-identical Connect repeats until the server has seen the client - acknowledge the Accept; after that any Connect is a new generation. No default changes. But a delayed - duplicate of the original Connect that arrives after that ack replaces a live session and forgets what - it received, so its retransmits are delivered twice. - - **C.** The application's Connect payload identifies the generation (today's hub rule), and every - client must put a per-process nonce in it, as the provider layer's `clientSessionId` does. Server mode - adopts the same comparison. This leaves G1 as a documented obligation on every caller, including the - ones that send an empty payload (`cultmesh-rs/src/lib.rs:973`, the TS publishers). - - **Recommended: A.** It is the literature's answer, it is robust to delayed duplicates (they carry the - accepted sequence), and it moves the decision into the session, deleting four transport-level - decisions. - - **Blocks:** Cut 1c only. Cut 1b (the ending rule) is ruled and does not depend on it. +- **Q-A9. What does a receiver do with fragment sets from a sender that never sends abandonment + notices? (new in pass 3, from the Cut 2 Soul pass)** Cut 2 stops evicting reliable fragment sets, because + evicting one whose fragments were acknowledged loses a frame the sender saw acknowledged (P5). But a + sender with reliable expiry, which is every default Rust `media` sender, abandons the rest of a frame + after its first fragment was acknowledged, so the set can never complete; after 64 of them Cut 2 refuses + the session. Cut F's notices release those sets, but only a Cut F sender sends notices, and the receiver + cannot tell an old expiring sender's dead set from a live set still waiting for a retransmit. + - **A. No fallback; bump both ends together.** Cut 2's rule applies to every peer. Any session between a + Cut 2 receiver and a pre-F expiring sender dies after 64 lossy frames, so every such pair (Muninn's + media, Mimir's relay, browsers on vendored CultLib) must move in the same deploy. + - **B. A capability bit, as PR-SCTP negotiates (recommended).** Connect and Accept carry flag bit 4 + ("I send and honour abandonment notices", RFC 3758 §3.1 Forward-TSN-Supported). Old peers ignore it on + those packets (probe CAP, all five runtimes). Cut 2's never-evict rule applies to a peer that advertised + it; a peer that did not keeps today's stalest-set eviction, with today's P5 risk, which only its own + upgrade removes. The sender also stops sending notices to peers that cannot use them. Cost: one flag on + two packet types and one remembered bit per session. + - **C. Age-based release.** A reliable set untouched for some multiple of the resend delay is released + for every peer. It keeps sessions alive without a wire change, but it releases a live set whose missing + fragment is merely slow, which is the P5 lie Cut 2 exists to remove, now on a timer. + - **Recommended: B.** It is the literature's answer, it makes Cut 2's rule hold exactly where its premise + holds, and mixed versions keep working without coordinated deploys. + - **Blocks:** Cut F's capability half and therefore the merge of Cuts 2, 2c and F. Q-A7 is independent. **Not forks, recorded so they are not re-asked:** - `Acknowledged` keeps its name and now means delivered. No variant is added or removed. - Delivered means handed to the transport's application boundary, not consumed. -- The 4 MiB cap stays. The sender's window keeps legal traffic under it (§5). -- The receiver's 4,096 receive history stays. The sender's window keeps legal traffic inside it. -- A receipt is resolved once. `Acknowledged` before a session ends stays `Acknowledged`; anything else - becomes `Invalidated` at the end. +- The 4 MiB cap stays; the receiver's 4,096 receive history stays. +- A receipt is resolved once. - Session generation is its own two cuts, not part of Cut 1 (§8). +- The notice's resend bound is 16 sends at the resend delay (§6). It is a constant, not an option. +- Receipts live on the single-peer socket transport in every runtime, as in Rust; hub receipts are a + follow-up for all five together. +- A failed send to one peer is that datagram lost (Cut D, RFC 1122's soft errors); it does not end the + peer's session. ## Sequencing and blockers -1. **Done.** `hands/rudp-stray-packets` merged to `main` at `3bf1c0c`. Odin pins `3bf1c0c`. -2. **Ready for Hands now** (base `main` at `3bf1c0c` or later; four runtimes; Kotlin trails per Q-A5): - - **Cut 1** (the watermark). - - **Cut 1b** (the ending rule). Same files as Cut 1 in `ResetPeerState` and the Disconnect branches: - land it after Cut 1 on the same branch, or in parallel and rebase the second. - - **Cut 2** and **Cut 3**, independent of Cut 1 and of each other, in parallel worktrees. Merge Cut 1 - before Cut 3 so that Cut 3's P4 test exercises the new hold path. -3. **Cut 1c** is blocked on Q-A6 and on Cut 1b. -4. **Cut 4** is blocked on Cut 1 (Acknowledged is sound only against a correct receiver), on Cut 1b (it - resolves receipts through the ending primitive), and on Q-A3. -5. **Cut 5** is blocked on Cut 3 in TS (the socket transport it routes through must carry the window) and - on Cut 1b in TS (its flush must be bound to the generation). It does not need Cut 1c, but Cut 5's - parity-doc text for repeated Connects waits for it. -6. **Odin's next pin bump** follows Cuts 1-3 (and 1b). Odin is the receiver for every Idunn health, - presence and catalog publisher, so this bump is what makes their Acknowledged mean delivered. Odin uses - `publish_cultnet_message_to_rudp_catalog` and no receipt type, so Cut 4 never breaks its build. -7. **Parallel campaigns:** - - **`hands/media-fec-cut3`** (unmerged, `4b85a173`) touches `rudp.rs` only in the imports, the - profile's `audio` entry (`~2374` at `3bf1c0c`) and the `"media" | "audio"` arm of - `channel_send_options` (`~2547`). None overlaps Cuts 1-4. Either order; the second rebases trivially. - Once it lands, reliable expiry is only for `media` when the caller chooses Reliable. - - **Muninn's own cuts** use lossy media plus parity. Its reliable `audio` session has no ordered - traffic, so P2 does not bite it. Nothing here blocks Muninn, and nothing in Muninn blocks this. - - **CultCache publication Cut 4** (`docs/cultcache-publication-cut.md` §4) touches `CultNetDatabase.cs` - only, and needs in-order, exactly-once delivery on `schema`. The C# socket transports carry one - reliable channel (`CultNetTransport.cs:2372-2376`), so P1 and P6 cannot occur on their sessions. No - blocking relation in either direction. -8. **Vendored TS bumps come after Cut 5**, so they pick up honest flushes, the TS window and the rewritten - publishers at once: - - **StreamPixels** vendors CultLib at `30ee8b9` (StreamPixels `origin/main`, gitlink bumped in - `44a8945`). Its service runs `createIdunnRuntimePresencePublisher` - (`apps/service/scripts/idunn-entry.mjs`), whose file is byte-identical at `30ee8b9` and `3bf1c0c`; - `30ee8b9` also predates the TS stray-packet hardening in `rudp.ts`. - - **Heimdall** vendors `b6b1d9c` and hand-rolls its own publisher (`src/idunn-rudp-health.ts:55-87`, - first-ack wait). At its bump it calls cultnet-ts's publisher instead. - - Bumping either after Cut 1c as well also gives them the restart fix; neither needs it, because each - publish binds a new port. -9. **CodexConnector** and **Ghostlight** need no code change at their next bump. They may switch their - status loops to `wait_acknowledged` after Cut 4. +1. **Done.** `hands/rudp-stray-packets` merged at `3bf1c0c`. Odin `origin/main` pins `3bf1c0ce`. +2. **In flight** (pass 2 text, unchanged by pass 3): Cuts 1 and 1b on `hands/ack-cut1`, Cut 2 on + `hands/ack-cut2`. No ruling forces a change to their specs: + - Q-A3 B adds the abandoned set, but Cut F adds it and hooks it into Cut 1b's primitive; 1b need not know. + - Q-A4 A needs an ending primitive in C#, TS and Python, which 1b already builds. + - Q-A6 is Cut 1c's. + - **Cut 2's merge moves** (the Cut 2 Soul finding): its code stays as specified, but it refuses sessions + with expiring senders unless Cut F releases their dead sets, so it merges only together with Cut F + (and Cut 2c). Its spec's "a conforming sender cannot reach that bound" is corrected in place. +3. **Ready now:** Cut 3 (merge after Cut 1, as pass 2 said); **Cut D** (independent); **K0** (tooling and + `build.ps1`; independent). +4. **Cut 1c** after Cut 1b merges. +5. **Cut F** built on `hands/ack-cut2` after Cuts 1 and 3 merge, once Q-A7 and Q-A9 are ruled. **Cuts 2, + 2c and F merge to `main` together.** +6. **Cut 4** after Cuts 1, 1b and F. +7. **Cuts 6a, 6b, 6c** after Cut 4, in parallel. +8. **Cut 5** after Cut 3 and Cut 1b in TS, as pass 2 said. It may use 6b's `waitAcknowledged` if 6b lands + first; it does not need it (flush suffices). +9. **Cut 7** after Cut F, once Q-A8 is ruled A. +10. **Cut K** after K0: K1 anytime; K2 after Cuts 1, 1b, 1c; K3 after Cuts 3 and F; K4 after Cut 4 + (and Cuts 6 as the parity reference); K5 after Cut 7. +11. **Pin bumps:** + - **Odin** after Cuts 1, 1b and 3 (it is the receiver for every Idunn publisher, so this bump makes + their Acknowledged mean delivered) and Cut D (it is the document server's only production host). Cuts + 2 and F come together at whichever bump follows their merge. + - **Muninn** (and any consumer of the default reliable `media` channel) takes Cuts 2 and F together. + Under Q-A9 B its ends may move separately; under A they must move in one deploy. Its client and + Sleipnir pins of `initial_sequence: 1` should go at that bump (Cut 1c notes). + - **CodexConnector** and **Ghostlight** need no code change; they may adopt `wait_acknowledged` after + Cut 4. +12. **Vendored TS bumps after Cut 5** (they pick up honest flushes, the TS window and the rewritten + publishers at once): + - **StreamPixels** vendors CultLib `542ddd3` (StreamPixels `a49f7d8`; pass 2's `30ee8b9` is stale). + Its service runs `createIdunnRuntimePresencePublisher`, which that revision carries under its older + name (`cultnet-ts/src/idunn-runtime-authority.ts`, pin at `:333`). + - **Heimdall** vendors `b6b1d9c` and hand-rolls its publisher (`src/idunn-rudp-health.ts:55-87`, + `initialSequence: 1` at `:65`). At its bump it calls cultnet-ts's publisher instead. +13. **Parallel campaigns:** `hands/media-fec-cut3` touches `rudp.rs` only in the imports, the `audio` + profile entry and the `"media" | "audio"` send arm; no overlap with Cuts 1-4, D or F beyond a rebase. + Once it lands, reliable expiry is only `media` in Reliable mode. CultCache publication Cut 4 is + unrelated in either direction. ## Follow-ups outside this campaign - **Hold caps in C#, TS, Python and Kotlin.** Their ordered hold buffers and per-channel maps are unbounded - against a hostile peer. Rust has caps. + against a hostile peer. Rust has caps. The far-ahead refusal on data is Rust-only too. - **The document server's per-session payload budget never shrinks** (`rudp_document_server.rs:321`, - `:584`). After 4 MiB in one session it drops data without acknowledging it, and the sender retransmits - until it gives up. + `:584`). - **Fire-and-forget health.** Muninn `main.rs:9446`, Sleipnir `main.rs:880-904` and Heimdall's first-ack wait can lose a health record silently. -- **FORWARD-TSN** if Q-A3 is A and a consumer later mixes expiry with ordered traffic. -- **Callers outside CultLib that fix `initial_sequence`** keep the restart lockout after Cut 1c until they - drop the value. Record it in their pin-bump notes. -- **The P4 probe at the old pins** predates its fix, so it proves nothing there. If anyone needs - old-sender-to-new-receiver evidence for the cap, rerun `09318d13`'s P4 against a pinned sender. +- **Receipts on hubs** (Rust hub, C# listener, cultmesh-ts and cultmesh-py servers) in all five runtimes + together; and a peer-level `send`-and-wait in TS's `CultNetPeer` (`peer.ts:98-110`). +- **Kotlin self-tests live in the production source** (`CultMesh.kt` K:5236-5633, run from `main`). Move + them to a test source set once the Kotlin image exists. +- **Callers outside CultLib that pin `initial_sequence`** (Cut 1c's table): drop the value at each bump. +- **Mimir `CultMeshMedia`** hand-rolls a per-peer relay with its own Connect handling (`Program.cs:365`, + `:485`); it should call the session's repeat predicate after its CultLib moves. +- **`CultNetTransportStats` parity.** Cut D adds `send_failures` in Rust and C#; Kotlin has no + `packetsDropped` until K1. Record the stats shape in the parity doc once both land. +- **The P4 probe at the old pins** predates its fix; rerun `09318d13`'s P4 against a pinned sender if + old-sender evidence is needed. From 9b8877ced7ab2408c0760f5d07cdb6d536d17bfa Mon Sep 17 00:00:00 2001 From: Meta Vens Date: Wed, 30 Sep 2026 00:45:41 +0200 Subject: [PATCH 08/16] Ack map: record Q-A7 A, Q-A8 A, Q-A9 B Co-Authored-By: Claude Opus 5.5 --- docs/cultnet-rudp-ack-cut.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/docs/cultnet-rudp-ack-cut.md b/docs/cultnet-rudp-ack-cut.md index 9f07341c..dfdeef1f 100644 --- a/docs/cultnet-rudp-ack-cut.md +++ b/docs/cultnet-rudp-ack-cut.md @@ -35,10 +35,12 @@ it. One thing moves for them: Cut 2 merges only together with Cut F (the Cut 2 S fragmented reliable frames (the default Rust `media` channel). Pass 3 absorbs it: Cut F's receiver drops every incomplete fragment set a notice covers; Cut 2 merges only together with Cut F; C# gets the bound it never had (Cut 2c). What a receiver does with a sender that never sends notices is **Q-A9**. -- **Open:** Q-A7 (the notice names one sequence, not a cumulative point), Q-A8 (reliable expiry in the - four runtimes that lack it) and Q-A9 (a capability bit, so Cut 2's rule applies only to peers that send - notices). Each has a recommendation. Q-A7 and Q-A9 block Cut F and with it Cut 2's merge; Q-A8 blocks - only Cut 7. +- **Ruled after pass 3 (operator, 2026-09-30), each the recommendation:** + - **Q-A7 A:** a notice names one sequence, not a cumulative point. + - **Q-A8 A:** C#, TypeScript, Python and Kotlin gain reliable expiry (Cut 7, K5). + - **Q-A9 B:** a capability bit on Connect and Accept; Cut 2's never-evict rule applies only to peers + that advertise it, and other peers keep pre-Cut-2 eviction. + Cut F is unblocked once Cuts 1 and 3 merge; the text below mapped these answers already. - **Body.** CultLib `main` at `e9469cb3`. Every transport file is byte-identical to `3bf1c0c` there (`git diff 3bf1c0c e9469cb3 -- src packages` is empty), so every `file:line` below is against both unless From 0479304d0b84aec8623dfea0a88897c4a5f98557 Mon Sep 17 00:00:00 2001 From: Meta Vens Date: Wed, 30 Sep 2026 01:56:03 +0200 Subject: [PATCH 09/16] Geometry: reference members are CultRecordRef, not [CultReference] strings e420410f (D11) refuses a reference member whose CLR shape the cache cannot walk; many:false needs CultRecordRef. CultGeometryBuildRequest.DomainKey, CultGeometrySelectedCutManifest.RequestKey and CultGeometryChunkArtifact.CutKey were strings marked [CultReference], so registering the geometry catalog threw and 7 geometry tests failed on main. All three are record keys of another document, so the members become CultRecordRef (the attribute is dropped, the type declares the reference). The msgpack formatter writes a CultRecordRef as its key string, so the wire bytes and the fingerprint inputs are unchanged. Co-Authored-By: Claude Sonnet 5.5 --- .../CultGeometryDocuments.cs | 15 +++++------- .../GeometryDocumentTests.cs | 24 +++++++++++++++++-- 2 files changed, 28 insertions(+), 11 deletions(-) diff --git a/src/GameCult.Geometry/CultGeometryDocuments.cs b/src/GameCult.Geometry/CultGeometryDocuments.cs index 306b5849..0da8af90 100644 --- a/src/GameCult.Geometry/CultGeometryDocuments.cs +++ b/src/GameCult.Geometry/CultGeometryDocuments.cs @@ -153,8 +153,7 @@ namespace GameCult.Geometry public string RequestId { get; set; } = string.Empty; [Key(1)] - [CultReference(typeof(CultGeometryDomainDocument))] - public string DomainKey { get; set; } = string.Empty; + public CultRecordRef DomainKey { get; set; } [Key(2)] [CultIndex] @@ -200,7 +199,7 @@ namespace GameCult.Geometry { if (request == null) throw new ArgumentNullException(nameof(request)); return new CultRecordKey("geometry:request:" + StableHash( - request.DomainKey, + request.DomainKey.Key.Value, request.WorkerGroup, StableVector(request.CameraPosition), StableVector(request.FrustumMin), @@ -228,8 +227,7 @@ namespace GameCult.Geometry public string CutId { get; set; } = string.Empty; [Key(1)] - [CultReference(typeof(CultGeometryBuildRequest))] - public string RequestKey { get; set; } = string.Empty; + public CultRecordRef RequestKey { get; set; } [Key(2)] public string[] SelectedNodes { get; set; } = Array.Empty(); @@ -247,7 +245,7 @@ namespace GameCult.Geometry { if (cut == null) throw new ArgumentNullException(nameof(cut)); return new CultRecordKey("geometry:cut:" + StableHash( - cut.RequestKey, + cut.RequestKey.Key.Value, cut.CutId, StableArray(cut.SelectedNodes), StableArray(cut.DeferredChildRequests), @@ -313,8 +311,7 @@ namespace GameCult.Geometry public string ChunkId { get; set; } = string.Empty; [Key(1)] - [CultReference(typeof(CultGeometrySelectedCutManifest))] - public string CutKey { get; set; } = string.Empty; + public CultRecordRef CutKey { get; set; } [Key(2)] [CultIndex] @@ -357,7 +354,7 @@ namespace GameCult.Geometry { if (chunk == null) throw new ArgumentNullException(nameof(chunk)); return new CultRecordKey("geometry:chunk:" + StableHash( - chunk.CutKey, + chunk.CutKey.Key.Value, chunk.ChunkId, chunk.SelectedCutId, StableArray(chunk.SourceDomainKeys), diff --git a/tests/GameCult.Geometry.Tests/GeometryDocumentTests.cs b/tests/GameCult.Geometry.Tests/GeometryDocumentTests.cs index 8741f891..2576c295 100644 --- a/tests/GameCult.Geometry.Tests/GeometryDocumentTests.cs +++ b/tests/GameCult.Geometry.Tests/GeometryDocumentTests.cs @@ -104,7 +104,7 @@ namespace GameCult.Geometry.Tests var domainKey = CultGeometryDomainDocument.CreateRecordKey(domain); var request = new CultGeometryBuildRequest { - DomainKey = domainKey.Value, + DomainKey = new CultRecordRef(domainKey), WorkerGroup = "ragnarok-column-workers", CameraPosition = [36f, -42f, 30f], FrustumMin = [-75f, -75f, -30f], @@ -129,6 +129,26 @@ namespace GameCult.Geometry.Tests first.Value.Should().StartWith("geometry:request:"); } + // The geometry documents reference each other by record key. The reference members are + // CultRecordRef so the cache can walk them, and they stay a bare msgpack string on the wire. + [Test] + public void GeometryReferences_AreWalkableByTheCache_AndStayStringsOnTheWire() + { + var cache = new CultCache(); + var descriptor = cache.Registry.GetRequired(); + descriptor.DeclaredMembers.Should().Contain(m => m.MemberName == "DomainKey" && m.IsReference && !m.IsMany); + + var chunk = SampleChunk(); + var references = cache.Registry.GetRequired().ReferencesOf(chunk, "CutKey").ToArray(); + references.Should().ContainSingle().Which.Target.Value.Should().Be("geometry:cut:test"); + + var payload = CultDocumentMessagePackSerialization.SerializeUntyped(chunk, typeof(CultGeometryChunkArtifact)); + var reader = new MessagePackReader(payload); + reader.ReadArrayHeader(); + reader.ReadString().Should().Be(chunk.ChunkId); + reader.ReadString().Should().Be("geometry:cut:test"); + } + [Test] public void ChunkArtifact_RoundTrips_ThroughCultCacheMessagePack() { @@ -382,7 +402,7 @@ namespace GameCult.Geometry.Tests return new CultGeometryChunkArtifact { ChunkId = "chunk/ragnarok-column/column-00", - CutKey = "geometry:cut:test", + CutKey = new CultRecordRef(new CultRecordKey("geometry:cut:test")), SelectedCutId = "cut-test", BoundsMin = [-1f, -1f, 0f], BoundsMax = [1f, 1f, 1f], From ef8f76a0d4baa5483067c7f617f1d79ad26f4178 Mon Sep 17 00:00:00 2001 From: Meta Vens Date: Wed, 30 Sep 2026 02:15:47 +0200 Subject: [PATCH 10/16] Geometry: register the cache's CultRecordRef formatter with the MessagePack generator The MessagePack source generator emitted a broken formatter reference for the CultRecordRef reference members. Declaring CultRecordRefFormatter<> a known formatter makes the generated resolver defer to the cache's own. Co-Authored-By: Claude Sonnet 5.5 --- src/GameCult.Geometry/CultGeometryDocuments.cs | 4 ++++ src/GameCult.Geometry/GameCult.Geometry.csproj | 1 + 2 files changed, 5 insertions(+) diff --git a/src/GameCult.Geometry/CultGeometryDocuments.cs b/src/GameCult.Geometry/CultGeometryDocuments.cs index 0da8af90..5d9ddaa9 100644 --- a/src/GameCult.Geometry/CultGeometryDocuments.cs +++ b/src/GameCult.Geometry/CultGeometryDocuments.cs @@ -7,6 +7,10 @@ using GameCult.Caching; using MessagePack; using static GameCult.Geometry.CultGeometryDocuments; +// The generator emits a resolver for these [MessagePackObject] documents; without this it would try to +// generate a formatter for the CultRecordRef reference members instead of using the cache's. +[assembly: MessagePackKnownFormatter(typeof(CultRecordRefFormatter<>))] + namespace GameCult.Geometry { /// diff --git a/src/GameCult.Geometry/GameCult.Geometry.csproj b/src/GameCult.Geometry/GameCult.Geometry.csproj index c10c2ecf..c1588906 100644 --- a/src/GameCult.Geometry/GameCult.Geometry.csproj +++ b/src/GameCult.Geometry/GameCult.Geometry.csproj @@ -10,6 +10,7 @@ + From e35b0092042aecc902854cd4cfbcb8f48b0266fb Mon Sep 17 00:00:00 2001 From: Meta Vens Date: Wed, 30 Sep 2026 02:15:56 +0200 Subject: [PATCH 11/16] Geometry: fix the Caching.MessagePack project reference path The previous commit lost its backslashes. Co-Authored-By: Claude Sonnet 5.5 --- src/GameCult.Geometry/GameCult.Geometry.csproj | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/GameCult.Geometry/GameCult.Geometry.csproj b/src/GameCult.Geometry/GameCult.Geometry.csproj index c1588906..cc66cba9 100644 --- a/src/GameCult.Geometry/GameCult.Geometry.csproj +++ b/src/GameCult.Geometry/GameCult.Geometry.csproj @@ -10,7 +10,7 @@ - + From 9ea134aca948ae864e51638526421caa966ee947 Mon Sep 17 00:00:00 2001 From: Meta Vens Date: Wed, 30 Sep 2026 02:25:27 +0200 Subject: [PATCH 12/16] Geometry: name the cache formatter on each CultRecordRef member The MessagePack source generator cannot resolve CultRecordRef on its own and emitted a broken formatter reference; MessagePackKnownFormatter on the open generic did not change that. An explicit MessagePackFormatter per member is the same formatter the cache resolver uses, so the wire stays a key string. Co-Authored-By: Claude Sonnet 5.5 --- src/GameCult.Geometry/CultGeometryDocuments.cs | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/src/GameCult.Geometry/CultGeometryDocuments.cs b/src/GameCult.Geometry/CultGeometryDocuments.cs index 5d9ddaa9..1ce73bdb 100644 --- a/src/GameCult.Geometry/CultGeometryDocuments.cs +++ b/src/GameCult.Geometry/CultGeometryDocuments.cs @@ -7,10 +7,6 @@ using GameCult.Caching; using MessagePack; using static GameCult.Geometry.CultGeometryDocuments; -// The generator emits a resolver for these [MessagePackObject] documents; without this it would try to -// generate a formatter for the CultRecordRef reference members instead of using the cache's. -[assembly: MessagePackKnownFormatter(typeof(CultRecordRefFormatter<>))] - namespace GameCult.Geometry { /// @@ -157,6 +153,7 @@ namespace GameCult.Geometry public string RequestId { get; set; } = string.Empty; [Key(1)] + [MessagePackFormatter(typeof(CultRecordRefFormatter))] public CultRecordRef DomainKey { get; set; } [Key(2)] @@ -231,6 +228,7 @@ namespace GameCult.Geometry public string CutId { get; set; } = string.Empty; [Key(1)] + [MessagePackFormatter(typeof(CultRecordRefFormatter))] public CultRecordRef RequestKey { get; set; } [Key(2)] @@ -315,6 +313,7 @@ namespace GameCult.Geometry public string ChunkId { get; set; } = string.Empty; [Key(1)] + [MessagePackFormatter(typeof(CultRecordRefFormatter))] public CultRecordRef CutKey { get; set; } [Key(2)] From fd34d4573e757ba845e0c975c1197b814d7152bf Mon Sep 17 00:00:00 2001 From: Meta Vens Date: Wed, 30 Sep 2026 02:30:43 +0200 Subject: [PATCH 13/16] Geometry tests: import System.Linq for the reference walk test Co-Authored-By: Claude Sonnet 5.5 --- tests/GameCult.Geometry.Tests/GeometryDocumentTests.cs | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/GameCult.Geometry.Tests/GeometryDocumentTests.cs b/tests/GameCult.Geometry.Tests/GeometryDocumentTests.cs index 2576c295..b5f63775 100644 --- a/tests/GameCult.Geometry.Tests/GeometryDocumentTests.cs +++ b/tests/GameCult.Geometry.Tests/GeometryDocumentTests.cs @@ -10,6 +10,7 @@ using R3; using System; using System.Collections.Generic; using System.IO; +using System.Linq; using System.Text.Json; using System.Threading.Tasks; From 6a87e53c23d2f1d9689f74ec5d17482b5fc0f347 Mon Sep 17 00:00:00 2001 From: Meta Vens Date: Wed, 30 Sep 2026 02:38:41 +0200 Subject: [PATCH 14/16] Publication map: record Cut 4 Soul follow-ups (crash-safe log store prerequisite) Co-Authored-By: Claude Opus 5.5 --- docs/cultcache-publication-cut.md | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/docs/cultcache-publication-cut.md b/docs/cultcache-publication-cut.md index 06aa47db..ed281978 100644 --- a/docs/cultcache-publication-cut.md +++ b/docs/cultcache-publication-cut.md @@ -813,4 +813,16 @@ Recorded, not forked: a gap (`ApplyShardSnapshotResponseAsync` `:675-736`). It predates this campaign. - **AetheriaEve `Program.cs:3722`.** It reads a committed write's `InvalidOperationException` as a denial. Under Q-P5 A, Cut 4's log failures cannot reach it. Observer and `OnUpdate` exceptions still can, so the Cuts 1-2 - recording stands. \ No newline at end of file + recording stands.- **Cut 4 Soul, recorded 2026-09-30 (Self), not fixed in Cut 4:** + - **File log store is not crash-safe (hard prerequisite).** `CultNetFileShardMutationLogStore.Append` and + `CompactThrough` use `File.WriteAllBytes` (truncate, rewrite, no temp+rename, no fsync). A failure mid-write + truncates the log; every later append burns a sequence and a restart cannot construct the database. It also + rewrites the whole file per append, which Cut 4 now does under the cache gate (2.2 ms per write at 100 + entries, 33 ms at 950, a 2.49 s stall). An atomic, append-only store must land before any live daemon + enables durable shard logs. Today only AetheriaEve's offline tools do; the live Aetheria daemon passes + `enableDurableShardLogs: false`. + - **A slow log append blocks cache readers** (`CultCache.Get` takes the gate). Accepted with a note in the + store's XML doc. + - **Crash between the store write and the log append** leaves the log lagging the store, and nothing detects + it; a restarted primary reuses that sequence (A2's divergence reached by a plain kill). The spec called this + lag detectable; for a crash it is not. From 486244b60f0cb43ba7d409cf9f0748f873819d43 Mon Sep 17 00:00:00 2001 From: Meta Vens Date: Wed, 30 Sep 2026 02:46:29 +0200 Subject: [PATCH 15/16] Caching: JSON converter and MessagePack formatter for CultPhasor The Phacelle merge added the public CultMath value type CultPhasor without its Caching adapters, failing the every-public-type tests. Register it the way CultCellular is: {"cos","sin"} objects in JSON (which also gives the dictionary-key form) and a [cos, sin] array in MessagePack. The adapters are Caching code and stay MIT; only CultPhasor.cs itself is MPL-2.0. Co-Authored-By: Claude Sonnet 5.5 --- .../CultMathJson.cs | 11 +++++++++-- .../CultMathResolver.cs | 17 ++++++++++++++++- src/GameCult.Caching.MessagePack/README.md | 1 + .../GameCult.Caching.Tests/CultMathJsonTests.cs | 3 +++ .../CultMathSerializationTests.cs | 5 +++++ 5 files changed, 34 insertions(+), 3 deletions(-) diff --git a/src/GameCult.Caching.MessagePack/CultMathJson.cs b/src/GameCult.Caching.MessagePack/CultMathJson.cs index 438c6150..0e65bf22 100644 --- a/src/GameCult.Caching.MessagePack/CultMathJson.cs +++ b/src/GameCult.Caching.MessagePack/CultMathJson.cs @@ -14,7 +14,7 @@ namespace GameCult.Caching.MessagePack; /// The canonical System.Text.Json shape of every public CultMath value type, the JSON twin of /// . Vectors, quaternions and colors are objects keyed by their /// field names ({"x":1,"y":2}), matrices are arrays of row objects, rect is -/// {"min":..,"max":..}, and CultCellular is {"nearest":..,"edge":..,"id":..}. +/// {"min":..,"max":..}, CultCellular is {"nearest":..,"edge":..,"id":..}, and CultPhasor is {"cos":..,"sin":..}. /// Property names are fixed: PropertyNamingPolicy does not rename them, and /// PropertyNameCaseInsensitive makes reading case-insensitive. /// @@ -481,7 +481,7 @@ public static class CultMathJson Add(UIntScalar.Instance, new Fields(("state", Val())), a => new Random { state = a[0] }, (v, a) => a[0] = v.state), - // Compositions: matrices are arrays of row objects, rect is {min, max}, CultCellular is {nearest, edge, id}. + // Compositions: matrices are arrays of row objects, rect is {min, max}, CultCellular is {nearest, edge, id}, CultPhasor is {cos, sin}. Add(f, new Rows(2, Xy()), a => new float2x2(a[0], a[1], a[2], a[3]), (v, a) => { a[0] = v[0].x; a[1] = v[0].y; a[2] = v[1].x; a[3] = v[1].y; }), @@ -507,6 +507,13 @@ public static class CultMathJson a[4] = v.edge.x; a[5] = v.edge.y; a[6] = v.edge.z; a[7] = v.edge.w; a[8] = v.id; }), + Add(f, new Fields(("cos", Xyzw()), ("sin", Xyzw())), + a => new CultPhasor(new float4(a[0], a[1], a[2], a[3]), new float4(a[4], a[5], a[6], a[7])), + (v, a) => + { + a[0] = v.cos.x; a[1] = v.cos.y; a[2] = v.cos.z; a[3] = v.cos.w; + a[4] = v.sin.x; a[5] = v.sin.y; a[6] = v.sin.z; a[7] = v.sin.w; + }), }; var map = new Dictionary(); diff --git a/src/GameCult.Caching.MessagePack/CultMathResolver.cs b/src/GameCult.Caching.MessagePack/CultMathResolver.cs index 803b3873..8383a464 100644 --- a/src/GameCult.Caching.MessagePack/CultMathResolver.cs +++ b/src/GameCult.Caching.MessagePack/CultMathResolver.cs @@ -180,7 +180,7 @@ public sealed class CultMathResolver : IFormatterResolver return value; }), - // Compositions: matrices are rows, rect is [min, max], CultCellular is [nearest, edge, id]. + // Compositions: matrices are rows, rect is [min, max], CultCellular is [nearest, edge, id], CultPhasor is [cos, sin]. Shape((ref MessagePackWriter w, float2x2 v) => { w.WriteArrayHeader(2); PutFloat2(ref w, v[0]); PutFloat2(ref w, v[1]); }, (ref MessagePackReader r) => { @@ -221,6 +221,21 @@ public sealed class CultMathResolver : IFormatterResolver } return value; }), + Shape((ref MessagePackWriter w, CultPhasor v) => { w.WriteArrayHeader(2); PutFloat4(ref w, v.cos); PutFloat4(ref w, v.sin); }, + (ref MessagePackReader r) => + { + var value = default(CultPhasor); + for (int i = 0, n = r.ReadArrayHeader(); i < n; i++) + { + switch (i) + { + case 0: value.cos = GetFloat4(ref r); break; + case 1: value.sin = GetFloat4(ref r); break; + default: r.Skip(); break; + } + } + return value; + }), }; var map = new Dictionary(); diff --git a/src/GameCult.Caching.MessagePack/README.md b/src/GameCult.Caching.MessagePack/README.md index fdf91ab2..ccd8f568 100644 --- a/src/GameCult.Caching.MessagePack/README.md +++ b/src/GameCult.Caching.MessagePack/README.md @@ -93,6 +93,7 @@ registered ahead of these wins. | `float2x2`, `float3x3` | array of row vectors (row-major) | array of row objects | | `rect` | `[min, max]` of `float2` | `{"min":{..},"max":{..}}` | | `CultCellular` | `[nearest, edge, id]` | `{"nearest":{..},"edge":{..},"id":..}` | +| `CultPhasor` | `[cos, sin]` of `float4` | `{"cos":{..},"sin":{..}}` | Floats are MessagePack float32 (`0xCA`), doubles float64 (`0xCB`), integers the shortest MessagePack integer. diff --git a/tests/GameCult.Caching.Tests/CultMathJsonTests.cs b/tests/GameCult.Caching.Tests/CultMathJsonTests.cs index 87e5b742..90d739be 100644 --- a/tests/GameCult.Caching.Tests/CultMathJsonTests.cs +++ b/tests/GameCult.Caching.Tests/CultMathJsonTests.cs @@ -43,6 +43,8 @@ namespace GameCult.Caching.Tests Json(new rect(5f, 7f, -1f, -2f)).Should().Be("""{"min":{"x":-1,"y":-2},"max":{"x":5,"y":7}}"""); Json(new CultCellular(new float4(1f, 2f, 3f, 4f), new float4(5f, 6f, 7f, 8f), 0.25f)).Should().Be( """{"nearest":{"x":1,"y":2,"z":3,"w":4},"edge":{"x":5,"y":6,"z":7,"w":8},"id":0.25}"""); + Json(new CultPhasor(new float4(1f, 2f, 3f, 4f), new float4(5f, 6f, 7f, 8f))).Should().Be( + """{"cos":{"x":1,"y":2,"z":3,"w":4},"sin":{"x":5,"y":6,"z":7,"w":8}}"""); } [Test] @@ -66,6 +68,7 @@ namespace GameCult.Caching.Tests RoundTrip(new float3x3(1f, 2f, 3f, 4f, 5f, 6f, 7f, 8f, 9f)); RoundTrip(new rect(-1f, -2f, 5f, 7f)); RoundTrip(new CultCellular(new float4(1, 2, 3, 4), new float4(5, 6, 7, 8), 0.25f)); + RoundTrip(new CultPhasor(new float4(1, 2, 3, 4), new float4(5, 6, 7, 8))); } [Test] diff --git a/tests/GameCult.Caching.Tests/CultMathSerializationTests.cs b/tests/GameCult.Caching.Tests/CultMathSerializationTests.cs index 32531679..98f126af 100644 --- a/tests/GameCult.Caching.Tests/CultMathSerializationTests.cs +++ b/tests/GameCult.Caching.Tests/CultMathSerializationTests.cs @@ -101,6 +101,7 @@ namespace GameCult.Caching.Tests RoundTrip(new float3x3(1f, 2f, 3f, 4f, 5f, 6f, 7f, 8f, 9f)); RoundTrip(new rect(-1f, -2f, 5f, 7f)); RoundTrip(new CultCellular(new float4(1, 2, 3, 4), new float4(5, 6, 7, 8), 0.25f)); + RoundTrip(new CultPhasor(new float4(1, 2, 3, 4), new float4(5, 6, 7, 8))); } [Test] @@ -132,6 +133,10 @@ namespace GameCult.Caching.Tests "94CA3F800000CA40000000CA40400000CA40800000" + "94CA40A00000CA40C00000CA40E00000CA41000000" + "CA3E800000"); + Hex(new CultPhasor(new float4(1f, 2f, 3f, 4f), new float4(5f, 6f, 7f, 8f))).Should().Be( + "92" + + "94CA3F800000CA40000000CA40400000CA40800000" + + "94CA40A00000CA40C00000CA40E00000CA41000000"); } [Test] From a014bf1b5b354948e81b6e5f4e21208aeb9965d9 Mon Sep 17 00:00:00 2001 From: Meta Vens Date: Wed, 30 Sep 2026 05:28:27 +0200 Subject: [PATCH 16/16] Ack map: 1b merges only with 1c; R is seeded by the handshake (Cut 1 Soul) Co-Authored-By: Claude Opus 5.5 --- docs/cultnet-rudp-ack-cut.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/cultnet-rudp-ack-cut.md b/docs/cultnet-rudp-ack-cut.md index dfdeef1f..59e95e68 100644 --- a/docs/cultnet-rudp-ack-cut.md +++ b/docs/cultnet-rudp-ack-cut.md @@ -622,6 +622,12 @@ The ack cuts build on these behaviours and do not alter them: drops what it owed. How a new generation is recognised (Cut 1c) needs one (Q-A6). The two are separable because 1b does not forget peer state: the peer may not know the session ended, and forgetting what was received would let its retransmits be delivered twice. + **Correction (Cut 1/1b Soul, 2026-09-30): they are separable as specs, not as merges.** 1b drops owed + writes, which leaves holes the sender will never fill; the receiver keeps `R` across the ending, so if the + next handshake does not reset it, every later ordered frame is held and acked, and flush succeeds while + nothing is delivered. Only 1c's reset on a non-repeated handshake removes that. **1b merges only together + with 1c.** Soul also found `R` seeded by the first reliable packet rather than the handshake; ruled: + `R` is seeded only by the handshake, and reliable data before the Accept is neither acked nor delivered. - **What must hold after both.** A write is Acknowledged or Invalidated within the generation it was issued in. A restarted client is admitted. A new server session's frames are delivered after a reconnect. A retransmitted Connect is still a repeat.