Event 7 was not one per frame. The bridge writes the stream's kind byte
itself, that send completed as an anonymous event 7, and a host counting
completions saw six for five frames with nothing to tell them apart. The
send request now says which send it was; the kind byte's completion is the
bridge's own business and is dropped. A send MsQuic released without
putting on the wire reports it: event 7 carries a send result in `code`,
completed or canceled, so an aborted send is accounted for rather than
silently missing. Measured: 32 sends queued under an abort, 32 event 7, 28
canceled and 4 completed.
The initiator of a shutdown learned nothing. MsQuic gives it only
shutdown-complete, and the bridge swallowed that, so a host could end its
own connection and never see the id end. Event 4 now has one emitter —
shutdown-complete — and the two initiated-by callbacks record why rather
than announcing it. Each connection id gets exactly one event 4, whoever
ended it, with the code its side passed, and it is emitted after the id
has stopped resolving, so a host told a connection is gone cannot still
resolve it on the next line.
Return codes inverted by platform. QUIC_STATUS is a negative HRESULT on
Windows and a positive errno on POSIX, so a Linux host checking for a
negative code saw every failure as success, against a README that promised
negative. A MsQuic refusal is now -(1000 + (status & 0xffff)) on both, and
cultmesh_quic_last_status hands over the raw bits for diagnostics. On
win32-x64 a garbage PKCS12 returns -12032 with last_status 0x80072B18.
The v1 exports move with it; the managed connector tests the value against
zero and formats it, so the contract it was shipped with holds.
Smaller ones, from the same pass: AppendTrace is v1-only and bounded,
because a long-lived v2 connection grew that string once per MsQuic event
for nothing to read; a listener configuration failure takes the connection
out of the map instead of leaving it naming a handle MsQuic is about to
close; and next_event refuses a null payload with a non-zero capacity
rather than treating the lie as a discard.
Soul's C consumer now compiles against the header and asserts its own
struct and its eleven bound prototypes against it, and the koffi smoke
checks its declarations and the thirteen exports the same way: 20 of 20,
seq clean, cancel clean, the v1 gate 3 passed 1 skipped as before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
FindStream, FindConnection and FindListener took the lock, dropped it, and
handed back a raw pointer. MsQuic's worker freed the object on
shutdown-complete and the host thread dereferenced it on its next send.
One release run in three crashed on each target under an ordinary
peer-driven stream teardown; ASan caught it three times out of three, with
the free inside DestroyStream on a worker and the read inside
stream_send_frame on the host thread. A null check cannot fix that: the
pointer is not null, it is dead.
The maps hold shared_ptr now and a lookup returns a reference the caller
owns for the length of its call. The worker's destroy takes the object out
of its map and drops the map's reference only. The MsQuic handle follows
the object: the destructor closes it exactly once, under the exchange
guard, so the handle is alive for as long as any holder is. A stream holds
its connection, which orders every stream handle's close before its
connection's without a rule anywhere saying so.
runtime_close quiesces. It marks the runtime closing, wakes every blocked
poller, and waits for every in-flight call to return before it closes a
handle or frees a byte; a call that arrives after that is refused with -1.
Waking a poller without waiting for it was how close returned a poller
into freed memory while looking like it supported the pattern.
Two things fall out of the model rather than being fixed on top of it: a
listener that fails partway through opening leaves no map entry and no
handle, because every failure path just drops the local reference, and a
stream is in the map before StreamStart, because the shutdown-complete
that StreamStart can reach on a worker has to find it.
Cut with it: Listener::stopped, V1Client::connection (an unread raw
pointer into the same hazard), and FindListener.
Event 7 is still one too many and the shutdown initiator still learns
nothing; that is the next commit. The rest of Soul's C consumer passes on
win32-x64, including race at 400 streams, which crashed before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Soul wrote a C consumer of this ABI from the spec text, because there was
nothing to include. The struct layout, the exports and the conventions
lived in the .cpp and in prose, which is three places for one contract and
no way to notice when they part.
include/cultmesh_quic_native.h is now the contract: the 64-byte event with
every field offset asserted, the exports with their signatures, the id and
return-code conventions, the object lifetime rules, and the threading
rules. The .cpp includes it and keeps none of its own copy. The README
points at it rather than restating it.
The lifetime and return-code sections describe where the bridge is going,
not where it is; the two commits after this one make them true.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The bridge was one platform, one role and one host: Windows by construction
(`windows.h`, `__declspec`, `CryptHashCertificate2`), client-only, and shaped
around the managed connector that polls it every 2 ms. Nothing about MsQuic
required that; the shape came from having exactly one caller.
It now carries a C ABI v2 of twelve exports over a single per-runtime event
queue: open a runtime, open listeners and connections on it, open streams and
send frames, and take typed events out of one blocking wait. The event struct is
64 bytes described by layout rather than by a shared header, so a host declares
it in its own FFI and no header travels with the ABI. `next_event` keeps the v1
two-phase convention — return 2, leave the event at the head, report the size
the caller needs — because that is the one part of v1 that was already
host-agnostic.
The rule that shapes everything else is that nothing calls out. MsQuic's worker
threads only push onto the queue and wake that wait, so no host runtime is
entered from a thread it does not own, and the bridge needs to know nothing
about what is on the other side. That is what makes one binary serve both a
Node process through FFI and a Unity player through P/Invoke.
Trust moved out. A v2 client connection defers validation, emits the DER as
event 3 and answers MsQuic only when the host calls
`connection_certificate_complete`; if the host never answers, MsQuic's handshake
timeout closes the connection and event 4 says so. The C code no longer has an
opinion about certificates — except in the v1 shim, where it must.
The five v1 exports are unchanged in signature and behaviour and are now
`#ifdef _WIN32`, matching `CanConnect`'s own Windows gate. A v1 client is a
private v2 runtime holding one connection, and the pin comparison is the single
thing it still decides in C, because that is the contract the shipped Unity
package was built against. `windows.h`, `wincrypt.h` and `CryptHashCertificate2`
appear nowhere outside that block.
Two things the rewrite had to get right that the old shape never exercised.
The v1 pin must be attached before `ConnectionStart`, not after it returns: the
handshake runs on an MsQuic worker thread and reaches the certificate event
immediately on loopback, so a pin applied afterwards loses the race and the
connection silently takes the host path nobody is listening on. And teardown
must close the connection handles itself — `RegistrationClose` blocks until the
app has closed every child handle, and a silent shutdown delivers no shutdown
events at all, so leaning on the callbacks to do it deadlocks the caller
forever. Both were found by the tests hanging, and both now have a mutation that
reproduces them.
Verified: the four v1 tests in tests/GameCult.Mesh.Quic.Native.Tests pass
unchanged (3 passed, 1 skipped for want of an external endpoint), including the
wrong-pin refusal and the departed-client broadcast test. A 15-check FFI smoke
drives the whole v2 path — listener, deferred certificate, both sides connected,
latest-only stream, one frame with FIN, identical bytes out — and passes on
Windows x64 and on Linux x64. Ten source mutations, ten killed, two green
no-op controls.
One open fork, reported separately and not decided here: the pinned Windows
MsQuic is the Schannel build, which refuses
QUIC_CREDENTIAL_TYPE_CERTIFICATE_PKCS12 with QUIC_STATUS_NOT_SUPPORTED, so
`listener_open` cannot open a provider against it. The map assumed PKCS12 was
"one credential type for Schannel and OpenSSL"; the header declares it but the
Schannel provider does not implement it. The identical bridge binary passes the
full provider smoke against OpenSSL MsQuic 2.5.9, so this is a question about
which flavour Windows pins, not about the bridge.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>