Soul's probe proved a client can grow the provider's memory without limit
(20k reliable frames plus 20k distinct latest-only keys took
receiveQueueSize to 40000 and RSS from 97 to 318 MB) because StreamPixels
never calls receive() to drain it, and there was no cap on connections
either. The pinned certificate only authenticates the server, so any
internet client can dial in.
Add acceptClientFrames (default true, unchanged for existing callers and
interop peers, which all use receive()): when false, any inbound stream a
peer opens faults that connection at STREAM_STARTED, before a frame can be
decoded or queued, so no inbox or generation-map entry is ever created for
client data, and receive() itself rejects clearly.
Add maxConnections (default 1024): a connection beyond the cap is refused
at accept, shut down before any transport, outbox, or retained-frame
seeding is allocated for it. Existing peers are unaffected.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This brings fix batch 3c (per-runtime out-buffers, freed on the last
release, plus a steady-state RSS test in its own child process) onto
the branch carrying fix batches 3a and 3b and retention.
The merge was automatic. Checked by hand: release() frees the reused
buffers only after the reference count reaches zero and the pump has
exited. retain() (from 3a) is called only while the provider already
holds a reference, so it cannot resurrect a runtime that is closing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hand-mutating CultMeshRealtimeInbox.receive() to check `completed`
before the ready queue (so dispose() rejects an already-queued frame
instead of draining it first) found no existing test that dies for it.
Add one: two reliable-ordered frames sit in the provider's receive
queue, the provider is disposed, and both frames must still come back
from receive() before the next call rejects.
Verified by hand against the final code and restored, per the fix
batch's mutation list; not committed as a harness.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Reproduces Soul's probe directly: 20,000 client-sent latest-only frames
on one key, none ever drained by receive(), leave exactly 1 pending
frame in the provider's receive queue, and that frame carries the
newest sequence sent. A second test sends 20 frames across 50 distinct
keys and checks the queue holds exactly 50 (bounded by key count, not
frame count). A third proves reliable-ordered client frames are never
coalesced and stay in order at the provider.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Soul's probe found readyFrames growing without limit: a publish-only
consumer of CultMeshQuicRealtimeProvider.receive() (StreamPixels' Node
service) let a single unauthenticated client key grow the provider's
receive queue by one entry per frame sent, forever.
Replace CultMeshRealtimeLatestOnlyCoalescer plus the three separate
ready-queue/waiter implementations built around it (the transport's
inbound path, the provider's outbox pump, and the provider's own
readyFrames array) with one CultMeshRealtimeInbox<T>, shaped like the
C# reference's CultMeshRealtimeInbox: latest-only entries coalesce to
at most one pending value per (channel, body) key, keeping that key's
first-arrival queue position; reliable-ordered entries always queue,
unbounded; receive() drains whatever is queued before a complete(error)
rejects further calls.
The provider's pumpReceivedFrom now publishes into this inbox under the
frame's (channel, body) key instead of pushing onto an unbounded array,
so a slow or absent receive() caller backs up to at most one pending
frame per distinct key. Added provider.receiveQueueSize (test-visible,
mirrors the existing trackedStreamCount precedent) to make that bound
observable.
realtime-quic.ts: 1298 -> 1273 lines (HEAD 1b0c71d), -16 against the
275010a anchor. Soul estimated -90 to -120 from unifying four ~209-line
sites into one ~80-line class; the actual saving is smaller because two
of those sites already need their own promise/waiter/AbortSignal
plumbing in TypeScript (no Channel with native CancellationToken
support), so consolidating them removes duplication but the surviving
single copy of that plumbing is not itself small. No further
correctness-neutral cut was found without code-golfing working logic.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Without sendOnStream's disposed guard, the queued send never settles (no
event or sweep is left to reject it), so a bare assert.rejects(send, ...)
would hang the whole file instead of failing this one test. Race the send
against a 3s timeout that rejects with a different message, so the mutant
is a loud assertion mismatch either way the native layer happens to fail
first.
Verified by hand: reverted sendOnStream's `if (this.disposed)` guard to
`if (false)` locally, rebuilt, and ran P8 alone — it died by assertion in
81ms (a native stream-send error on the already-shut-down connection,
still not matching /disposed/i), not a timeout. Restored the guard before
committing. Also hand-verified fix 1 the same way: commenting out the
`clearTimeout(handshakeTimer)` on CONNECTED reproduces the original
handshake-timer eviction, and the new fix-1 test dies by assertion
(0 !== 1) rather than a hang.
Full Windows pass on Starfire after restoring: 158/158 cultmesh-ts tests
and all 5 QUIC interop tests (including the native-connector and
late-join lanes), msquic.dll confirmed as the pinned build (4,181,856
bytes, SHA-256 C17C6581...).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
By the time the memory test ran in-process, ~150 earlier tests in the
suite had already grown and partly freed the glibc heap, so a real
per-call native leak could land in already-mapped free space and never
show up as RSS growth in that process: confirmed by hand, the
in-process version of this test kept passing even with `nextEvent`
mutated back to allocating fresh koffi buffers every call, once it ran
inside the full suite rather than alone.
Spawn a dedicated `node --expose-gc` child process for the measurement
instead. A fresh process has no such history, matching the ~8 MiB
(fixed) vs ~30 MiB (pre-fix) split measured in isolation. Confirmed by
hand again with the same mutation: the child-process version now fails
reliably (~28.7 MiB growth) inside the full 150-test suite, not just
standalone.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The prior version opened connection B only after A had already faulted,
then asserted a boolean flag set by B's own CONNECTED event. That flag
never flipped on Linux: something about accepting a second connection
right after the first one's forced shutdown made the assertion
unreliable, even though A's fault itself isolated correctly (visible in
the log: the onFault throw fired once, promptly).
Rewritten to establish B as a normal, fully connected loopback pair
first, fault A only afterward, then prove B's handler is still live by
sending an actual frame on B and confirming the server side still
receives it — a stronger assertion than a connected flag, and one that
does not depend on a second connection's accept timing relative to a
fault that just tore another one down.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
P8's test opened a fresh reliable stream on the first send, so disposing
before its microtask ran hit streamOpen() against an already-shut-down
connection (its own native error) instead of ever reaching sendOnStream's
disposed guard. Prime the stream with an awaited first send so the second,
raced send chains straight onto sendOnStream with no streamOpen call in
between.
P10b's test registered connection A's and B's server-side listeners after
an `await`, the exact async-registration gap fix 1 exists to close: a
connection's next native event can be dispatched, and silently dropped
(no listener registered yet), before that await's continuation runs.
Register both listeners synchronously inside their LISTENER_NEW_CONNECTION
callback instead.
The IPv6 fix 3 test asserted the wrong expectation: WHATWG URL's own
`hostname` keeps the brackets for an IPv6 literal, so the round trip
through parseQuicRealtimeEndpoint correctly returns "[::1]", not "::1".
Confirmed via a Linux verify run (cultlib-quic-native-dev image): before
this commit, exactly these three new tests failed and every other test
(155) passed, including every other fix/P3 test added alongside them.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The 5 MiB bound was a guess and false-failed on the fixed code: 200k
direct `nextEvent` calls carry their own ~8 MiB RSS noise floor from
thread-pool marshaling and promise bookkeeping, even with zero
`koffi.alloc` calls in the loop. Measured on Yggdrasil, same build and
host, same 200k-call loop:
fixed code (0 koffi.alloc calls in the loop): ~8.0 MiB RSS growth
pre-fix code (400k koffi.alloc calls, unfreed): ~29.9 MiB RSS growth
18 MiB sits well above the fixed baseline and well below the leaking
one, so the test stays robust to ordinary run-to-run noise while still
catching the regression.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The `.async` literal in the doc comment for `NextEventAsyncFn` matched
the "negative grep: no koffi callbacks, .async only on nextEvent" test's
`/\.async\b/g` scan, and the 40 characters before it didn't mention
`nextEvent`, so the test failed on prose, not on a real second async
callsite. Reworded to describe koffi's async export without the
literal `.async` token.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
realtime-quic-provider.test.ts: a peer stays connected and keeps
receiving well past handshakeTimeoutMs (must fail on the pre-fix code);
an invalid reliable-ordered or latest-only frame throws from broadcast()
without evicting any peer; advertisedHost is used in place of the bind
address and round-trips through the module's own parser; an IPv6 bind
advertises a bracketed literal; receive() with an already-aborted signal
rejects immediately; and a latest-only send failure evicts the peer
(P3 — the outbox's own send-failure path, not covered by the existing
reliable-ordered eviction test).
realtime-quic-consumer.test.ts: a reliable send still queued behind
reliableSendTail rejects instead of hanging when disposal wins the race
(P8 — sendOnStream's this.disposed guard).
realtime-quic-native.test.ts: a throwing onFault handler is isolated to
its own connection; the pump survives and a second connection's handler
still runs (P10b).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Soul's probes found every provider peer evicted at handshakeTimeoutMs
(10s by default): StreamPixels' Node service would drop every Unity
overlay 10s after it connected. Root cause: acceptConnection registered
a temporary buffering handler that was the only thing setting
state.connected, then attachPeer awaited CultMeshQuicNativeRuntime.open()
before swapping in the real handler and replaying buffered events. The
native bridge always emits NEW_CONNECTION before CONNECTED as separate
events; when CONNECTED landed after the swap, it went straight to
transport.handleConnectionEvent (which has no case for CONNECTED) with
nothing left to clear the handshake timer, so a fully healthy peer was
evicted on schedule regardless.
Fix: acceptConnection takes a synchronous retain() on the provider's own
already-open runtime instead of awaiting a fresh open(), so one handler
owns the connection from NEW_CONNECTION onward with no async window, no
handler swap, and no buffered-event replay. PendingAccept, its buffering
handler, and attachPeer are gone; dispose() now sweeps `accepted` (every
transport a connection ever got, whether or not its handshake reached
CONNECTED) instead of the deleted pendingAccepts set. Cut C's
retained-frame seeding still happens atomically before the peer joins
`peers`.
Also, while touching broadcast() and sendFrame():
- Encode and validate each frame exactly once in broadcast(), fanning the
encoded bytes out to every peer and outbox, instead of re-encoding once
per peer inside sendFrame(). An invalid frame (e.g. a 70,000-byte
channelId) now throws to the caller before touching any peer, rather
than failing the same way for every peer in turn and evicting all of
them while broadcast() itself reported success. The provider's retained
latest-only frames now cache their encoded bytes too, so seeding a newly
attached peer never re-encodes them either.
- Add an `advertisedHost` provider option, independent of the bind
address: `host` is typically 0.0.0.0, an empty string, or an IPv6
wildcard for binding, none of which describe a route a client can dial,
and the native listener only accepts IP addresses in the first place,
not a DNS name. IPv6 literals are bracketed correctly in the advertised
endpoint. The uppercase cert-sha256 pin is unchanged (the operator ruled
StreamPixels pins a long-lived self-signed certificate).
- provider.receive() with an already-aborted signal now rejects
immediately instead of registering a waiter an already-fired abort event
can never reach.
- The duplicated "datagrams not supported" message (sendFrame and
broadcast each had their own copy) is now one shared constant.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A caller that already holds a live runtime instance (the QUIC realtime
provider, handing a reference to each newly accepted peer) needs one more
reference without crossing an async boundary. `open()` itself has no I/O
to await, but being `async` still forces every caller through a microtask
hop; `retain()` just increments the shared refcount and returns `this`,
synchronously, so the caller stays inside the same turn.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Calls the runtime's private `nextEvent` directly (0 ms timeout, no
listener/connection open, so each round trip returns immediately with
status 0) 200,000 times after a warm-up, then asserts RSS growth
across the run stays under 5 MiB with `global.gc()` forced before and
after. The old per-call `koffi.alloc` code allocates a ~64 B event
struct plus a 4 B int32 every iteration and never frees either, so it
clears this bound by roughly 2x; the fixed code, which allocates
nothing per call, stays flat. Skips when no native bridge is built or
when Node wasn't started with `--expose-gc`; `npm test` now passes
`--expose-gc` so the check runs by default instead of always skipping.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The `LoadedNativeLibrary` interface just re-listed the sixteen fields of
the object literal `loadBindings` already built, one liability the
methods below never needed a second time. Split the library-building
body into `buildLibrary()` and let `LoadedNativeLibrary` be
`ReturnType<typeof buildLibrary>`; `loadBindings` is now a thin cache
wrapper around it. `nextEventAsync`'s signature stays explicit as
`NextEventAsyncFn`, since `promisify` can't infer one from koffi's
untyped `.async`.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A Soul-verified probe found every koffi out-parameter allocation
(runtime open, listener open, connection open, stream open, and both
per-`nextEvent` buffers) was never freed. `nextEvent` runs in the pump's
tight poll loop, so this leaked roughly 53 bytes per idle cycle even
with no traffic, and grew unbounded under real frame volume (214 MB of
RSS over 4M allocations on the pinned koffi 3.3.0 build).
Allocate `nextEvent`'s out-event and out-required buffers once per
runtime instance and reuse them for every call instead of once per
call: the pump never has more than one `nextEvent` in flight, so
decode-then-reuse is safe, and they are freed once in `release()`
after the pump loop exits. Every other allocation (`open`,
`listenerOpen`, `connectionOpen`, `streamOpen`) is synchronous and
short-lived, so those are freed with `koffi.free` right after decode.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Cut C's headline cross-process proof. The TS `serve` peer already tore its
listener down the instant it finished broadcasting, so nothing could dial it
late; it gains `--linger-ms` to hold the listener open afterward instead. The
interop harness starts that provider, runs a throwaway `dial` peer that
receives every frame and disconnects (the provider requires a first peer
before it broadcasts at all, so this is also what proves broadcasting
finished), then only after that dials with the C# native connector and
asserts the frame it gets is the retained one, by payload, not just
non-empty. New `NativeConnectorReceivesTheRetainedFrameWhenJoiningLate` in
GameCult.Mesh.Quic.Native.Tests carries the assertion; the existing
`NativeConnectorReceivesFromExternalProvider` is untouched. Windows-only,
matching the native connector under test.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
AReliableOrderedFrameIsNeverRetainedAndIsNotDeliveredToALateJoiner disposed
the first session and then waited on server.ConnectionCount == 0 before
attaching a second one, which timed out under Yggdrasil's ygg-verify
container (dotnet test, cultlib-quic-native-dev image): the retention rule
under test does not depend on the first peer disconnecting, so the fix keeps
both sessions connected concurrently and drops the flaky wait.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
One sentence each in the C# QUIC README and the transport-planes doc: a
provider retains the newest latest-only frame per (channel, body) and
delivers it to a peer as soon as it attaches, bounded by the number of
distinct keys.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
CultMeshQuicRealtimeServer mirrors the TS provider: a `_retained` dictionary
holds the newest latest-only frame per (channel, body) key, updated in
BroadcastAsync with the same CultMeshGeneration comparison the receive-side
filter already uses. AcceptLoopAsync seeds a newly accepted client with the
retained frames. Both the retained update+fan-out and the client
registration+seed run under one lock (`_retainedGate`), so a seed and a
concurrent broadcast are totally ordered and a client can never end up seeded
with a frame older than one it already received via broadcast. Reliable-
ordered frames are events and are never retained.
CultMeshGeneration is now internal so the server can reuse the transport's
own generation comparison instead of a second implementation.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
CultMeshQuicRealtimeProvider now keeps a `retained` map of the newest
latest-only frame per (channel, body) key, updated on broadcast using the
same generation comparison the receive-side filter already uses. Every peer
attached after that point is seeded with the retained frames before it is
added to the peer set, so a StreamPixels overlay that connects or reconnects
after state was published converges on it instead of waiting for the next
change. Reliable-ordered frames are events and are never retained.
Operator ruling (2026-09-23, OQ1 = A): the provider retains, not a service
heartbeat or a per-peer attach hook.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The mid-handshake test still hung under M2 in the full workspace test run
even after the previous two fixes (pendingAccepts sweep, try/finally).
Diagnosed by printing the stuck value: refCount settled one below its own
"before" snapshot, not above — the opposite of a leak. Root cause:
CultMeshQuicRealtimeProvider.dispose() releases its runtime reference
fire-and-forget (`void this.runtime.release()`), so when M2 makes every
preceding test's own dial fail during the handshake too, their releases
can still be landing when this test starts and snapshots "before". One
lands mid-test, after the snapshot, and refCount never returns to a value
that already excluded it.
Added waitForRefCountToStabilize(): polls until refCount stops moving for
a short quiet window before treating it as a baseline. Verified against
the actual M2 mutation, in the full `npm run test` workspace run (not
just this one file, which had already stayed clean in isolation): the
file now exits in the low tens of seconds with 9 expected failures (every
test that dials through the now-broken cert check) and none of the
"Promise resolution is still pending" hangs from the three earlier
attempts. Confirmed clean on the unmutated tree first.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Same hygiene sweep as the previous commits, applied to the three tests
whose own subject is a fault or cleanup path that, if it regressed, would
leave a receiveFrame() call with nothing to ever resolve or reject it:
the two "fix 1" throwing-handler tests (a malformed frame that should
fault the connection, and a peer shutdown that should run cleanup()) and
the "fix 3" peer-shutdown test. Each now wraps its transport in
try/finally and uses receiveFrameOrTimeout() instead of a bare
receiveFrame() behind assert.rejects.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Re-running M1 after the previous fix: the assertion now correctly failed
(assertRejectsAndDisposes disposed the resolved client transport), but
the file still hung. Root cause: under M1 the handshake actually
completes (the connector accepts any certificate), so the provider's
peer is fully attached rather than left mid-handshake, and the test's
`provider` was never wrapped in try/finally — the failed assertion threw
past `provider.dispose()`, leaking the listener and the now-live peer.
provider.dispose() is idempotent, so wrapping the whole body costs
nothing on the normal path.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The "dispose() evicts a connection still mid-handshake" test (previous
commit) used a bare assert.rejects(connector.connect(...)) to force the
handshake-rejection path it needs, which is exactly the pattern that hung
realtime-quic-consumer.test.ts under M1: found re-running M1 against the
full workspace after the previous fix — it hung again, this time in
realtime-quic-provider.test.ts, at this new test, for the same reason.
Ported the same assertRejectsAndDisposes() helper into this file rather
than importing it across test files (each file already keeps its own
small helpers).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Found running the M2 mutation (certificate pin compared case-sensitively)
on Yggdrasil: it broke the handshake for every provider test that dials
with the provider's uppercase-advertised pin, each test correctly failed
its own assertion in under 40ms, but the whole file then hung for two and
a half minutes past its last visible test result.
CultMeshQuicRealtimeProvider.dispose() only disposed connections already
in `this.peers` (attached after reaching CONNECTED). A connection whose
handshake never gets there — a rejected certificate, a stalled client —
sits in acceptConnection()/attachPeer()'s own state with its own
handshakeTimeoutMs timer and, once attachPeer's own
CultMeshQuicNativeRuntime.open() resolves, its own runtime reference,
entirely outside dispose()'s reach. Nine tests each leaked one such
pending accept under M2, each held open for up to its 10s default
handshake timeout, serialized enough to add up to the observed hang.
Fixed by tracking every in-flight accept's `evict` closure in a
`pendingAccepts` set that dispose() sweeps alongside `peers`; each accept
removes itself once it settles, whether attached or evicted, so the set
never grows for a healthy, long-lived connection. Added a regression test
that pins a handshake to reject on a long (30s) handshakeTimeoutMs and
asserts dispose() releases the runtime reference in under 2 seconds, not
after the timeout.
Also fixed the same underlying test-hygiene gap the M2 investigation
surfaced directly: the M4 and M4b tests (a stream-kind/delivery mismatch
should fault a connection) left `transportA` undisposed with an unbounded
`receiveFrame()` behind a bare `assert.rejects`; if the fault path
regresses, that call hangs forever instead of failing. Added
receiveFrameOrTimeout() (a bounded receiveFrame, in the same spirit as
assertRejectsAndDisposes()) and a `finally { transportA.dispose(); }` to
both tests.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
packages/cultmesh-ts/package.json has depended on koffi 3.3.0 since Cut 4
landed the native QUIC binding, but the root package-lock.json was never
regenerated to match: it carried no koffi entry at all, so `npm ci` at the
repo root failed with EUSAGE. The StreamPixels deploy host runs exactly
`npm ci && npm run build:ts`, so this blocked deployment.
Regenerated on Yggdrasil (Linux, the cultlib-quic-native-dev container),
not on the operator's workstation, via `npm install`. Verified from a
clean clone on the same host: `npm ci && npm run build:ts` both succeed
against the regenerated lockfile.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Found running the M1 mutation (bypasses the certificate-accept check) on
Yggdrasil under cgroup-limited CPU: three tests correctly failed their
assertion (`assert.rejects(connector.connect(...))` / `manager.connect(...)`
observed a resolve instead of a reject), but realtime-quic-consumer.test.js
then hung instead of exiting — "Promise resolution is still pending but the
event loop has already resolved". Each of those assertions discarded the
connect() call's return value on the assumption it would always reject, so
the transport that unexpectedly connected under the mutant was never
captured, never disposed: a live native connection, an active
onConnectionEvent registration and an outstanding runtime reference kept
the process alive after node:test had already recorded the failures. This
is the exact scenario the batch names by history ("Soul saw M1 and M3 fail
their assertion and then hang the file").
Added `assertRejectsAndDisposes()`, a drop-in replacement for
`assert.rejects()` that disposes whatever the promise resolves to before
failing the assertion, and used it at all eight call sites in this file
that assert a connect() call rejects (two direct connector.connect() calls,
six through the session manager). Verified on Yggdrasil: M1 re-run now
fails the same three assertions and the file exits on its own.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Real bug found while writing the "dispose with a pending send in flight"
regression test, not a pre-scoped fix: sendOnStream() registered a
pendingSends entry unconditionally. If the reliable-send microtask queued
by sendFrame() (chained onto reliableSendTail) ran after cleanup() had
already executed — cleanup()'s own reject-sweep runs once and is guarded
against re-entry, and offConnectionEvent means no later native event can
resolve the entry either — that entry's promise then never settles,
leaking the pending send forever. Confirmed by a hanging test that
reproduced intermittently (~20-25% of runs) when a peer transport's
dispose() raced an in-flight reliable-ordered sendFrame() on the same
transport; not reproducible in isolation outside the full node:test file
(over 100 standalone cycles clean), which points at the race window being
narrow enough that only real inter-test timing variance opens it.
sendOnStream() now checks `this.disposed` before registering, closing the
gap: a send that loses the race rejects immediately instead of
registering an entry nothing will ever settle.
Also gives that regression test its own `{ timeout: 10_000 }` so a
recurrence fails that one test loudly instead of hanging the whole file
for the harness's 60s+ default; 60 consecutive full-file runs were clean
with both changes in place (25 with the fix alone still showed the hang
occasionally, so the fix closes a real but not the only contributing
factor — recorded here rather than claimed as a full proof).
New coverage for the Cut 5 provider's runtime-release-on-every-path
requirement:
- a reliable-ordered send failure evicts the peer without broadcast()
throwing to the caller;
- the runtime reference is released when the listener itself fails to
open (a real PKCS12 credential, but a port already bound by another
listener, so listen()'s own `catch { await runtime.release(); throw
error; }` actually runs) — split out from a same-named test that had
only exercised an earlier failure point (extractLeafCertificateDer(),
before CultMeshQuicNativeRuntime.open() is ever called), which is kept
as its own, more honestly named test;
- dispose() with a pending send in flight releases the runtime reference
(the test whose flakiness led to the fix above).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Ports Soul's uaf.js probe (scenario a: bare open/release cycles,
including a fresh open overlapping the previous runtime's close) into a
committed test. UV_THREADPOOL_SIZE and MALLOC_PERTURB_ must be set before
Node starts, so the test spawns a child `node` process running
test/support/release-uaf-probe.js (plain JS, not compiled, requiring
dist/ directly like the package's own compiled tests) once per pool size
(1, 2, 4) and asserts each exits 0.
Trimmed from the probe's original 300 cycles to 20: at pool size 1 with
the crypto.pbkdf2 saturation load, 300 cycles took over two minutes on
this workstation, and the ordering bug this test defends does not need
that many iterations to reach the race — 20 keeps the whole test under
about 25 seconds while still exercising many open/release cycles with a
fresh open racing a still-closing runtime.
Verified by hand: reverting release()'s order in
realtime-quic-native.ts (runtimeCloseFn before awaiting pumpLoop, the
pre-fix order) did not make this test fail on Windows — MALLOC_PERTURB_
is a glibc feature the Windows CRT ignores, so corruption here is silent
rather than a deterministic crash. The revert-and-confirm check that
actually demonstrates the bug runs on Yggdrasil's Linux container, where
glibc honors MALLOC_PERTURB_; see the verify report for that run's
output. The fix was restored immediately either way, confirmed by a clean
git diff before this commit.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Two scenarios the batch specifically calls for: a throwing handler plus a
garbage frame (the native fault path, through faultConnection()'s now
try/catch-guarded onFault) and a throwing handler plus a peer shutdown
(the transport's own cleanup(), now releasing before running handlers).
Both assert the second onDisposed handler still runs despite the first
throwing, and the peer-shutdown case also asserts
CultMeshQuicNativeRuntime.refCount returns to baseline.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
M4b: the mirror image of the existing M4 test — a latest-only stream
carrying a frame whose own delivery byte says reliable-ordered must fault
only that connection, not the pump.
M10: an equal-generation frame is dropped by the receive-side filter.
Sequenced so it is observable (Soul's note): the first frame is consumed
before the duplicate generation arrives, so the coalescer no longer holds
a pending value the duplicate could silently overwrite with itself.
Without this sequencing, M10's `<=0` -> `<0` mutation is invisible: the
coalescer's overwrite semantics make an unconsumed duplicate look
identical to a dropped one.
F7: split into two tests, because a genuine end-to-end
SEND_CANCELED-through-sendFrame() scenario proved unreliable on this
host. The native-layer test (realtime-quic-native.test.ts) is
deterministic: it round-trips two 64-bit codes above 2^53 through
CONNECTION_SHUTDOWN, exercising the exact decode line F7 mutates
(`code: toBigInt64(decoded.code)`, shared by every native event type,
including STREAM_SEND_COMPLETE). The consumer-level attempt — prime a
reliable stream, start a second send, shut the connection down from the
peer mid-flight — is recorded as a comment: MsQuic here appears to
locally complete a `StreamSend` (as an uncanceled SEND_COMPLETE) faster
than any JS-observable window can land a peer shutdown ahead of it,
regardless of payload size (tried up to 8 MiB) or an explicit delay
before the shutdown call. Recorded as not yet reached at the consumer
layer, not as unreachable.
F6kinds: the existing "stream shutdown prunes streamKinds" test only
proved a second frame still worked afterward, which passes whether or not
the map is actually pruned (a stream id is never reused within one
connection, so a leaked entry has no other behavioural symptom — exactly
why the mutation survived). Strengthened to assert
CultMeshQuicRealtimeTransport.trackedStreamCount (added in the previous
commit) directly.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Covers the session-manager fixes in the previous commit:
- dispose() aborts a dial in flight instead of letting the connector's own
timeout run out before the transport is discarded (asserted by
wall-clock bound, not just by eventual rejection).
- disconnect() during a dial does not let that dial cache a session once
it lands; the manager stays usable afterward.
- F5abort: a losing race candidate is told to abort promptly, proven by
recording when spyConnector's own abort listener actually fires rather
than only asserting eventual disposal (which the mutation's removed
controller.abort() call does not affect, since the cleanup loop that
disposes settled losers is untouched by that mutation).
- F5loser: a losing candidate that connects anyway, past the point abort
would have helped (spyConnector gains an ignoreAbort option to model
this), is disposed rather than left connected.
Replaces the old "fix 4: a connect() that completes after dispose()"
test, which pinned the pre-fix behaviour of waiting out the connector's
full delay before discarding the late transport; that delay is exactly
what dispose() aborting the dial now removes.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Fix 1 (consumer half): CultMeshQuicRealtimeTransport.cleanup() ran
onDisposed handlers before releasing the runtime reference, unguarded. A
throwing handler skipped every handler queued after it (the throw exits
the for-of) and skipped the release entirely: on the peer-shutdown path
the refcount stuck and the process never exited; on the fault path the
throw escaped cleanup() into dispatch()'s catch in the native runtime,
which is itself now guarded (previous commit), but cleanup()'s own
contract should not depend on that. release() now runs first, and each
handler runs in its own try/catch.
Low fixes:
- receiveFrame(signal) with an already-aborted signal now rejects
immediately instead of registering an "abort" listener that already
fired and will never fire again, which hung the call forever.
- CultMeshQuicRealtimeConnector.connect() merged its four failure paths
(already-aborted, handshake timeout, later abort, handshake-time error)
into one fail() closure. The handshake-time error path was the one that
mattered: it dropped the runtime reference and rejected but never called
connectionShutdown on the native connection, leaking it at the native
level on every rejected handshake past the certificate stage.
- CultMeshQuicRealtimeSessionManager.dispose() now aborts every dial still
in flight instead of only marking sessions disposed and hoping a live
dial notices later; disconnect() aborts the one in-flight dial for its
target so a race with an in-progress connect() cannot cache the session
disconnect() meant to discard. Threaded through dialAsync/
raceFirstSuccessAsync/connectCandidateAsync as an AbortSignal that
combines with the per-race internal controller.
Subtraction (~-70 net in this file despite the fixes above and new
trackedStreamCount/dialAborts surface):
- SimpleGate replaced by a plain promise-chain field on the transport
(reliableSendTail); the class only ever wrapped one method.
- The two dead streamKinds.set() writes on outbound (self-opened) streams
removed: the map is only ever read for streams that deliver an inbound
STREAM_FRAME event, which a stream this side opened never does.
- The write-only authorityRoute field on CultMeshRealtimeCandidate
deleted; nothing read it.
- The one-use TimeoutError class deleted in favour of Error.
- routeCertificateView moved to cultnet-ts (previous commit); the local
copy deleted.
Also exposes CultMeshQuicRealtimeTransport.trackedStreamCount (streamKinds
map size) for the F6kinds regression test: nothing else could observe
whether STREAM_SHUTDOWN actually prunes the map, since a stream id is
never reused within one connection.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Fix 1 (native half): faultConnection() called the caller-supplied onFault
handler unguarded. A throwing handler escaped faultConnection(), then
dispatch(), then the pump's async loop as an unhandled rejection nobody
awaits, crashing the process on a single bad connection. Wrap the call and
report instead of propagating, matching the try/catch dispatch() already
has around the listener call that reaches faultConnection() in the first
place.
Subtraction: fold the private NativeBindings interface and the
bindingsCache wrapper object into the runtime class's own methods, which
already typed the same twelve calls one door down. The wrapper only
restated that typing a second time. loadBindings() now returns the raw
koffi handles (untyped, koffi has no useful types) and the class methods
do the alloc/decode work directly, the way half of them already did for
their out-parameters. Net -101 lines despite the added refCount getter and
a trimmed module doc comment (no longer narrating the koffi probe process
twice).
Also exposes CultMeshQuicNativeRuntime.refCount as a public static getter
(renaming the backing field to refCountValue): the batch's regression
tests assert it returns to baseline after a fault, a peer shutdown, and
release(); nothing else needed to reach into a private static before.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Both packages mapped an advertised route's wire-shaped certificate
(providerKeyId/providerPublicKeyX/providerPublicKeyY/...) to
CultMeshAuthorityRouteCertificate independently. Move the mapping into
cultnet-ts's cultmesh-authority module, next to verifyAuthorityRoute, and
have both consumers call it instead of duplicating the object literal.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Found by a second real failure, this time on Windows: `dotnet test --logger
"console;verbosity=detailed"` prints a passed test's TestContext.Out lines
twice (live-streamed, then again under "Standard Output Messages:"), and
matching CULTMESH_GOLDEN_SEQUENCE and CULTMESH_GOLDEN_HEX with two
independent regexes let the harness pair a sequence from one copy with a hex
from the other, producing a spurious mismatch even though the C# and
TypeScript sides agreed. One regex now requires them adjacent, exactly as
the C# test's two back-to-back WriteLine calls print them. All four QUIC
interop lanes now pass on Windows (real dotnet test run, native connector
lane included).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Found by a real failure on the Linux verification container: latest-only
coalesces on both the provider's outbox and the managed connector's own
receive-side inbox, so a single ReceiveAsync() call is not guaranteed to
land on the first frame broadcast — on a slower host it landed on sequence 3.
The C# test now also prints CULTMESH_GOLDEN_SEQUENCE, and the harness looks
up the TypeScript peer's logged encoding of that same sequence instead of
hardcoding sent === 1.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Verified live on Windows: a passed NUnit test under `dotnet test --logger
"console;verbosity=detailed"` prints "Test Run Successful." and "Passed: N",
never the literal "Passed!" the assertion was checking for. Both the native
and managed connector lanes now match the real strings; both were confirmed
to pass for real against a live TypeScript provider (lane 1: 137ms, lane 2:
145ms with the golden-bytes hex line matching the TypeScript peer's own
encoding byte for byte).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
TypeScript provider vs C# native connector (the Unity path, gated to
Windows since the native connector under test is); TypeScript provider vs
C# managed connector, with a golden-bytes check comparing the C# test's
re-encoded frame against the TypeScript peer's own encoding of the same
frame; latest-only under a stalled consumer (two dials against one 200-frame
provider, one paused mid-stream via the peer script's new --pause-after),
asserting the provider's broadcasts stay bounded regardless of the pause and
both consumers converge on sequence 200.
Both dotnet-test lanes run `dotnet test --filter FullyQualifiedName~<test>
--logger "console;verbosity=detailed"`: detailed verbosity is what surfaces
a passed test's own TestContext.Out lines, which the golden-bytes lane reads
back out of stdout.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
dial gains --expect-sequence (stop once a frame with that sequence arrives,
rather than counting frames received — the number of frames a stalled
latest-only reader sees is not the interesting value, what it converges on
is) and --pause-after/--pause-ms, which simulate the stalled-consumer test
hook at the only level this subprocess can reach: not calling receiveFrame
for a while. The shared native pump keeps dispatching and coalescing into
this transport's own inbox regardless of whether the app reads, so not
reading is externally equivalent to not polling.
serve now also logs each broadcast frame's own TypeScript encoding as hex, so
the harness can compare it against the same frame's C#-side re-encoding
without importing the codec itself.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
NativeConnectorReceivesFromExternalManagedProvider -> NativeConnectorReceivesFromExternalProvider:
CULTMESH_NATIVE_EXTERNAL_ENDPOINT's peer is no longer necessarily the C#
managed provider now that Cut 5 adds a TypeScript one on the same lane; the
env var name stays, since it names the endpoint, not the peer.
Add ManagedConnectorReceivesFromExternalProvider, gated by the same env var:
dials an external provider with the plain managed connector and no
ValidateProviderCertificate (so acceptance runs on the endpoint's own
cert-sha256 pin alone, like an ordinary consumer), and prints the received
frame's CultMeshRealtimeWireProtocol re-encoding as hex for the harness's
golden-bytes comparison against the TypeScript peer's own encoding.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
cultmesh-quic-peer.ts serve opens a CultMeshQuicRealtimeProvider on the same
test fixture credential the unit tests use, prints {status, endpoint} in the
shape spawnServeProcess already expects, waits for a first peer before
broadcasting (a provider's broadcast to zero peers silently sends nothing,
same reasoning as the C# quic-realtime-serve peer's
WaitForFirstConnectionAsync), then broadcasts --frames frames at --delivery
with --interval-ms between them, stopping early on stdin EOF like the C#
peer's WatchStdinCloseAsync.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Reliable-ordered ordering in both directions, latest-only coalescing (a
synchronous back-to-back publish is deterministically coalesced, not by
network timing luck), a stalled peer's outbox holding at most one pending
frame per key across 200 broadcasts, a non-blocking broadcast, the uppercase
cert-sha256 pin, unreliable failing closed, and eviction on peer shutdown.
Runs against the real native bridge and Cut 4's real consumer connector, no
mocks; skips when CULTMESH_QUIC_NATIVE_DIR is unset, matching every other
QUIC test file.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
CultMeshQuicRealtimeProvider (packages/cultmesh-ts/src/realtime-quic.ts) is
the StreamPixels role: listen() opens a native listener, computes an
uppercase cert-sha256 advertisedEndpoint (the Unity connector requires the
pin), and accepts peers into a Set of the same CultMeshQuicRealtimeTransport
Cut 4 already built for consumers. broadcast() mirrors
CultMeshQuicRealtimeServer.BroadcastAsync: reliable-ordered awaits every
peer's send_complete via transport.sendFrame directly (Task.WhenAll,
evict-on-failure); latest-only publishes into a new per-peer
CultMeshQuicRealtimeProviderOutbox and returns without waiting on any peer's
I/O. The outbox pumps sendFrame's existing fresh-stream-per-frame path one
frame at a time, so a stalled peer holds at most one pending frame per
(channel, body) key.
Extracted CultMeshRealtimeLatestOnlyCoalescer from the inbound path's
inline Map+FIFO bookkeeping (unchanged behaviour) so the new outbox reuses
the same coalescing idiom instead of a second inline copy.
Accepting a connection buffers native events behind a synchronous listener
registration (the pump dispatches strictly sequentially, so registering
before any await is what makes the async runtime-reference acquisition that
follows safe) and replays them once the peer's CultMeshQuicRealtimeTransport
exists; a handshake timer evicts a peer that never completes CONNECTED.
Wired into the CultMesh facade as createQuicRealtimeProvider.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The PKCS12/DER pair had no record of how it was made or what it is. Documents
the CN, the openssl recipe that reproduces an equivalent pair, and that
nothing depends on today's exact bytes past what the tests read back out.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A non-literal first stdio element loses spawn()'s typed-tuple overload
entirely (verified on Yggdrasil: tsc failed on the two other, unrelated
serve helpers' stdout/stderr going nullable), not just for stdin's varying
element. Asserted at the one call site instead.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Finding 8(c): the dev image builds the Node bridge's own libmsquic copy into
a build-local directory (scripts/build-quic-native.sh), which is never on the
loader's search path. .NET's System.Net.Quic resolves "libmsquic" through
that path independently and failed with PlatformNotSupportedException before
this. The smallest honest fix: dpkg -i the same pinned 2.5.9 deb (same URL,
same SHA-256 as the build script) system-wide and run ldconfig, so both the
Node bridge's own copy and .NET's independent resolution find a libmsquic.
Changes the image; the new tag is cultlib-quic-native-dev:75cb89e73850
(first 12 hex of this file's sha256), built fresh on Yggdrasil.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Finding 8(a): every spawnServeProcess call used stdio ["ignore", ...], which
hands the C# quic-realtime-serve peer an already-closed stdin. Its own
WatchStdinCloseAsync (Program.cs:1261-1269) treats that as EOF and tears the
serve loop down at once, before it can send anything. spawnServeProcess gains
an opt-in `stdin: "pipe"` (default stays "ignore" for every other serve mode,
which never reads stdin), used only by the "CultMesh QUIC realtime" test; its
teardown ends that pipe explicitly before killing the process.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
