Three rules the bridge has always promised, none of which anything committed
could see.
The timeout was the structural one. Both scenarios end their waits with a close,
so the duration the bridge waits on governed nothing either of them observes: a
hard-coded 2000 ms survived the whole Linux matrix and Windows, and a host asking
for five seconds would have got two forever with nothing reporting it. The fix is
not another entry at another constant. A timeout's only observable is how long
the call takes, so `polltimeout` lets an idle runtime's poll be ended by its own
timeout and measures it, twice at different values, because no single constant
sits in both bands. The seam's hold moves behind that too: it parks only a call
the wait woke, so a poll whose own timeout expired is on its way out rather than
held there by the fixture and counted as the bridge's doing.
The refusal during a close (the header's section 4) had no scenario at all. A
call admitted after the closer has begun increments the in-flight count behind a
wait that has already read it. `latecall` holds the close open with the seam and
makes a call into it.
And the two-phase poll was written off as needing a listener, a credential and a
connection. It needs a closed loopback port: the failure always carries a reason,
which is an 87-byte payload, in about a millisecond. Moving the pop above the
copy crashes on both targets.
18 killed, 0 survived, 0 skipped on linux-x64; 16 killed, 0 survived on
win32-x64, where the two ThreadSanitizer entries are honestly skipped. Both runs
restored the source to the same sha256, b163cab7d48ebc93.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The mount was this machine's own path, so the one documented way to build and
check the Linux bridge needed editing before it would run anywhere else. It is
the repository root now, named as the shell names it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The image the bridge is built and mutated in existed nowhere in the repository,
and the recipe that stood in for it installed a C++ toolchain and no JavaScript
runtime — so the container could not run the harness that checks the bridge it
had just built. It is a file now, pinned to the same debian:13 digest as the
release workflow, carrying ninja, Node, setarch and MsQuic's four runtime
dependencies.
The harness configures with `-G Ninja` on linux-x64, which is what
scripts/build-quic-native.sh uses. CMake's Linux default is Makefiles, the image
carries ninja rather than make, and the default configured nothing at all: the
first run in the committed container died in cmake. It said so, because the
control now prints what the child said, which is the whole point of that change.
The seccomp requirement is stated where someone hitting it will read it — the
bridge's README, beside the recipe — rather than being knowledge the runner
assumes its caller already has.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Section 4 forbade any host call that begins after `cultmesh_quic_runtime_close`
starts, and promised in the same sentence that such a call is refused with -1;
section 6 repeated the promise for `cultmesh_quic_next_event`. The refusal lives
behind the runtime's own gate, so once the close has returned there is nothing
left to read it: the case the sentence describes is exactly the case where the
promise cannot hold.
The prohibition stands alone. The refusal is stated for what it is — a call that
races the start of a close, while the close is still running, is refused and
touches nothing — and no longer as a guarantee covering the call the section
forbids.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The closerace scenario pinned the order of the count and the notify but never
that a host call is counted at all. Deleting the call scope from
`cultmesh_quic_next_event` outright leaves every configuration green — the
pollers finish before the teardown reaches them — and so does inverting the
comparison that makes a positive timeout block, which empties the library before
the close even begins. A scenario that only sleeps and hopes cannot tell any of
that apart from the real thing.
So the bridge says it itself, in development builds only (section 7 of the
header, behind CULTMESH_QUIC_DEBUG_ASSERTS, absent from every shipped binary):
the peak number of host calls counted inside the library, the number counted
when the close began its wait, and a hold that parks a call inside the library
until the scenario releases it. A held call returns the 0 an empty queue would
have and touches the runtime no further, so a bridge that frees the runtime
under it is reported by the scenario rather than crashing inside it.
`holdclose` drives the quiesce through that seam: the calls are counted, they
were still counted when the close began, and the close did not return while they
were held. `closerace` keeps hunting the race and now also refuses to pass when
a poller left before the close began, which is a scenario that raced nothing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The bridge's close path had no committed test. Whether a close is safe against
host threads already inside the library is the one thing about it a host cannot
establish by reading, and every check of it so far lived in a session scratchpad
and is gone.
The scenario runner under tests/ drives that path and nothing else: it parks
pollers in cultmesh_quic_next_event, closes the runtime under them, and asserts
that the close is what woke them. It consumes the bridge through the public
header, as a host does. CULTMESH_QUIC_BUILD_TESTS builds it, off by default,
because a delivery build has no business producing an executable.
CULTMESH_QUIC_DEBUG_ASSERTS, also off by default, makes the bridge state the
quiesce invariant itself rather than leaving it to whatever a sanitizer happens
to catch. Deleting the quiesce wait is otherwise falsifiable only by luck:
AddressSanitizer saw it 4 times in 48 attempts with pollers blocked mid-copy of
large frames, and never on release. With the assertion and pollers blocked it
fails on the first close.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The README claimed a rebuild reproduces the committed Unity plugin DLL. It does
not. That plugin is 40,448 bytes, predates both the static CRT and the move off
Schannel, and still imports VCRUNTIME140.dll and MSVCP140.dll; the current source
builds to 302,080 bytes with no CRT import.
What /Brepro buys is that a build reproduces itself, and the README now says
that, along with the fact that the committed plugin stays the older build until
the Unity package build replaces it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Section 3 and cultmesh_quic_last_status both told a host that the portable answer
is the return code. What the mapping makes portable is the sign. The magnitude is
not: the same MsQuic failure carries different numbers on different platforms, a
TLS error being -12032 on win32-x64 and -1126 on linux-x64, and the 16-bit mask
gives QUIC_STATUS_USER_CANCELED and QUIC_STATUS_FILE_NOT_FOUND the same -1002 on
Windows.
The header now says the sign is what a host may branch on, that the magnitude is
diagnostic and platform-specific, and that switching on a particular code
identifies nothing reliably.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every error sentence widened its QUIC_STATUS straight to uint64_t. On Windows
that status is a signed HRESULT, so the sentence carried 0xFFFFFFFF80072B18 where
the status is 0x80072B18. The status is 32 bits on both platforms and now goes
through uint32_t on its way in.
Diagnostics only: the v1 managed connector matches on the wording of these
sentences, not on the hex.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A call scope used to drop the lock after decrementing the in-flight count and
notify afterwards. That left a window: the closer's predicate is already true
once the count reaches zero, so a scope preempted between the unlock and the
notify could come back to find the runtime torn down and signal a condition
variable that no longer exists. ThreadSanitizer reports it as a broadcast racing
the destroy.
The decrement and the notify now happen under one lock guard. The closer cannot
return from its wait until the scope releases the gate, so the condition
variable is alive for as long as anything can signal it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Q9 ruled A. The Schannel MsQuic refuses QUIC_CREDENTIAL_TYPE_CERTIFICATE_PKCS12
with QUIC_STATUS_NOT_SUPPORTED, and PKCS12 is the only credential type a
provider has on both platforms, so a Windows provider could not open a
listener against the runtime this repo was shipping. The same bridge
binary passes all twenty smoke checks and the v1 gate against the OpenSSL
build, which is what Soul verified and what this now downloads.
Two digests, not one. The zip is pinned by FEE9A664 and 19,179,817 bytes;
the msquic.dll inside it by C17C6581 and 4,181,856 bytes. The earlier
report and the cut map's first record put the zip's digest on the DLL,
which is a pin that checks the archive and then ships a file nobody looked
at. Sizes ride along with both, because a digest that fails tells you less
than a digest and a length that fail together.
The CRT is linked statically. A Node or Unity host without the Visual C++
redistributable cannot load a DLL that imports vcruntime140.dll, and
Windows reports that as "the specified module could not be found" while
naming the module that was found. Confirmed on the rebuild: msquic.dll,
CRYPT32, ntdll, KERNEL32, and no CRT import at all. The DLL goes from
83,456 to 302,080 bytes, which is the price of the host not needing
anything installed.
The Unity plugin's committed msquic.dll is still the Schannel one and this
does not touch it; the next build-unity-package.ps1 run is what replaces
it, and that is not in this batch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Event 7 was not one per frame. The bridge writes the stream's kind byte
itself, that send completed as an anonymous event 7, and a host counting
completions saw six for five frames with nothing to tell them apart. The
send request now says which send it was; the kind byte's completion is the
bridge's own business and is dropped. A send MsQuic released without
putting on the wire reports it: event 7 carries a send result in `code`,
completed or canceled, so an aborted send is accounted for rather than
silently missing. Measured: 32 sends queued under an abort, 32 event 7, 28
canceled and 4 completed.
The initiator of a shutdown learned nothing. MsQuic gives it only
shutdown-complete, and the bridge swallowed that, so a host could end its
own connection and never see the id end. Event 4 now has one emitter —
shutdown-complete — and the two initiated-by callbacks record why rather
than announcing it. Each connection id gets exactly one event 4, whoever
ended it, with the code its side passed, and it is emitted after the id
has stopped resolving, so a host told a connection is gone cannot still
resolve it on the next line.
Return codes inverted by platform. QUIC_STATUS is a negative HRESULT on
Windows and a positive errno on POSIX, so a Linux host checking for a
negative code saw every failure as success, against a README that promised
negative. A MsQuic refusal is now -(1000 + (status & 0xffff)) on both, and
cultmesh_quic_last_status hands over the raw bits for diagnostics. On
win32-x64 a garbage PKCS12 returns -12032 with last_status 0x80072B18.
The v1 exports move with it; the managed connector tests the value against
zero and formats it, so the contract it was shipped with holds.
Smaller ones, from the same pass: AppendTrace is v1-only and bounded,
because a long-lived v2 connection grew that string once per MsQuic event
for nothing to read; a listener configuration failure takes the connection
out of the map instead of leaving it naming a handle MsQuic is about to
close; and next_event refuses a null payload with a non-zero capacity
rather than treating the lie as a discard.
Soul's C consumer now compiles against the header and asserts its own
struct and its eleven bound prototypes against it, and the koffi smoke
checks its declarations and the thirteen exports the same way: 20 of 20,
seq clean, cancel clean, the v1 gate 3 passed 1 skipped as before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
FindStream, FindConnection and FindListener took the lock, dropped it, and
handed back a raw pointer. MsQuic's worker freed the object on
shutdown-complete and the host thread dereferenced it on its next send.
One release run in three crashed on each target under an ordinary
peer-driven stream teardown; ASan caught it three times out of three, with
the free inside DestroyStream on a worker and the read inside
stream_send_frame on the host thread. A null check cannot fix that: the
pointer is not null, it is dead.
The maps hold shared_ptr now and a lookup returns a reference the caller
owns for the length of its call. The worker's destroy takes the object out
of its map and drops the map's reference only. The MsQuic handle follows
the object: the destructor closes it exactly once, under the exchange
guard, so the handle is alive for as long as any holder is. A stream holds
its connection, which orders every stream handle's close before its
connection's without a rule anywhere saying so.
runtime_close quiesces. It marks the runtime closing, wakes every blocked
poller, and waits for every in-flight call to return before it closes a
handle or frees a byte; a call that arrives after that is refused with -1.
Waking a poller without waiting for it was how close returned a poller
into freed memory while looking like it supported the pattern.
Two things fall out of the model rather than being fixed on top of it: a
listener that fails partway through opening leaves no map entry and no
handle, because every failure path just drops the local reference, and a
stream is in the map before StreamStart, because the shutdown-complete
that StreamStart can reach on a worker has to find it.
Cut with it: Listener::stopped, V1Client::connection (an unread raw
pointer into the same hazard), and FindListener.
Event 7 is still one too many and the shutdown initiator still learns
nothing; that is the next commit. The rest of Soul's C consumer passes on
win32-x64, including race at 400 streams, which crashed before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Soul wrote a C consumer of this ABI from the spec text, because there was
nothing to include. The struct layout, the exports and the conventions
lived in the .cpp and in prose, which is three places for one contract and
no way to notice when they part.
include/cultmesh_quic_native.h is now the contract: the 64-byte event with
every field offset asserted, the exports with their signatures, the id and
return-code conventions, the object lifetime rules, and the threading
rules. The .cpp includes it and keeps none of its own copy. The README
points at it rather than restating it.
The lifetime and return-code sections describe where the bridge is going,
not where it is; the two commits after this one make them true.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There is no Linux MsQuic binary in any NuGet package, and the v2.5.9 GitHub
release carries only test bundles, so the Windows script's approach does not
transfer. What does exist is Microsoft's Debian 13 pool, which is the same
distribution Yggdrasil runs, and the msquic source tree at the matching tag for
the POSIX headers the NuGet omits.
`scripts/build-quic-native.sh` fetches the pinned `libmsquic_2.5.9_amd64.deb`
and the three headers `msquic.h`, `msquic_posix.h` and `quic_sal_stub.h`, and
verifies all four by SHA-256 the same way `build-quic-native.ps1:14` verifies
the NuGet. A digest that stops matching is a changed dependency, not a warning.
Two details the deb forced. It carries three library files and no licence text,
so the MIT licence is copied from the one this repo already committed for the
Windows plugin — the same text for both platforms. And it ships only
`libmsquic.so.2.5.9`, with no `libmsquic.so` for `-lmsquic` to resolve against,
so the script makes that link in the extracted tree rather than in the image.
The Linux CMake branch links `-fvisibility=hidden`, so only the twelve v2
exports leave the `.so` and the Windows-only v1 five are absent by construction
rather than by discipline, and sets `RPATH $ORIGIN` so the `libmsquic.so.2`
shipped beside the bridge resolves without being installed system-wide.
`build-quic-native.ps1`'s default output directory becomes
`artifacts/quic-native/win32-x64` so the two platforms' outputs sit beside each
other instead of on top of each other. Both existing callers —
`build-unity-package.ps1:80-82` and the Native.Tests csproj `:20-22` — pass an
explicit `-OutputDirectory`, so neither is affected.
Built and smoked in the workstation's Docker `debian:13` engine pinned by
digest: g++ 14.2.0, glibc 2.41, `libgamecult_mesh_quic_native.so` 85,456 bytes
with `NEEDED libmsquic.so.2` and `RUNPATH $ORIGIN`, and the same 15-check v2 FFI
smoke that runs on Windows passes under Node 24 on that image. That is a Linux
build on a Linux host and nothing more: Q6 ruled the release artifact comes from
the Actions job, and this one is the development loop.
A Linux host still needs MsQuic's own runtime dependencies present — libssl3t64,
libnuma1, libxdp1, libnl-route-3-200 — which is now written down, because the
first smoke run failed on a missing libxdp.so.1 and the reason was not obvious
from the error.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The bridge was one platform, one role and one host: Windows by construction
(`windows.h`, `__declspec`, `CryptHashCertificate2`), client-only, and shaped
around the managed connector that polls it every 2 ms. Nothing about MsQuic
required that; the shape came from having exactly one caller.
It now carries a C ABI v2 of twelve exports over a single per-runtime event
queue: open a runtime, open listeners and connections on it, open streams and
send frames, and take typed events out of one blocking wait. The event struct is
64 bytes described by layout rather than by a shared header, so a host declares
it in its own FFI and no header travels with the ABI. `next_event` keeps the v1
two-phase convention — return 2, leave the event at the head, report the size
the caller needs — because that is the one part of v1 that was already
host-agnostic.
The rule that shapes everything else is that nothing calls out. MsQuic's worker
threads only push onto the queue and wake that wait, so no host runtime is
entered from a thread it does not own, and the bridge needs to know nothing
about what is on the other side. That is what makes one binary serve both a
Node process through FFI and a Unity player through P/Invoke.
Trust moved out. A v2 client connection defers validation, emits the DER as
event 3 and answers MsQuic only when the host calls
`connection_certificate_complete`; if the host never answers, MsQuic's handshake
timeout closes the connection and event 4 says so. The C code no longer has an
opinion about certificates — except in the v1 shim, where it must.
The five v1 exports are unchanged in signature and behaviour and are now
`#ifdef _WIN32`, matching `CanConnect`'s own Windows gate. A v1 client is a
private v2 runtime holding one connection, and the pin comparison is the single
thing it still decides in C, because that is the contract the shipped Unity
package was built against. `windows.h`, `wincrypt.h` and `CryptHashCertificate2`
appear nowhere outside that block.
Two things the rewrite had to get right that the old shape never exercised.
The v1 pin must be attached before `ConnectionStart`, not after it returns: the
handshake runs on an MsQuic worker thread and reaches the certificate event
immediately on loopback, so a pin applied afterwards loses the race and the
connection silently takes the host path nobody is listening on. And teardown
must close the connection handles itself — `RegistrationClose` blocks until the
app has closed every child handle, and a silent shutdown delivers no shutdown
events at all, so leaning on the callbacks to do it deadlocks the caller
forever. Both were found by the tests hanging, and both now have a mutation that
reproduces them.
Verified: the four v1 tests in tests/GameCult.Mesh.Quic.Native.Tests pass
unchanged (3 passed, 1 skipped for want of an external endpoint), including the
wrong-pin refusal and the departed-client broadcast test. A 15-check FFI smoke
drives the whole v2 path — listener, deferred certificate, both sides connected,
latest-only stream, one frame with FIN, identical bytes out — and passes on
Windows x64 and on Linux x64. Ten source mutations, ten killed, two green
no-op controls.
One open fork, reported separately and not decided here: the pinned Windows
MsQuic is the Schannel build, which refuses
QUIC_CREDENTIAL_TYPE_CERTIFICATE_PKCS12 with QUIC_STATUS_NOT_SUPPORTED, so
`listener_open` cannot open a provider against it. The map assumed PKCS12 was
"one credential type for Schannel and OpenSSL"; the header declares it but the
Schannel provider does not implement it. The identical bridge binary passes the
full provider smoke against OpenSSL MsQuic 2.5.9, so this is a question about
which flavour Windows pins, not about the bridge.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
b85a828 changed x86_64/gamecult_mesh_quic_native.dll with no native
source change. The two copies differ in six bytes: the PE header
TimeDateStamp and the debug directory timestamp. The MSVC link stamps the
build time, so every -UpdateTemplate run rewrote the committed DLL.
/Brepro on compile and link replaces the timestamps with content hashes.
Two clean builds from an emptied build root now produce identical bytes,
and the DLL embeds no build path. The committed DLL changes once to the
reproducible form; after that the release byte check covers it like the
managed plugins.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
