CultMesh document servers (Rust, TS, Python) refuse a put whose served snapshot reply
could never fit the response bound, before storing it. Refusals carry fixed text per
variant, acknowledge nothing, go unreliable and unordered, and never echo peer input or
OS text; publishers and the snapshot client report a refusal instead of timing out. Send
failures follow Cut D; a reply the queue cannot take is answered, not dropped; a failed
send's goodbye names a fixed error code in every runtime. Python validates options as
Rust does and survives any unstorable put or malformed TCP message.
Soul passes 1-4 plus merge verdict; the final batch closes it. Verified on Yggdrasil at
8b8f8436 (main merged): every CultNet/CultMesh suite in Rust, TS, Python and C#, interop 27/27.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The snapshot fallback sent the caught error's message to the peer, and
errors from getCache or the registry went the same way, so a message the
application or a peer supplied could reach the wire. Its ack fields also
covered the refused request. The snapshot path now refuses per case with
fixed text (the snapshot source failed / the snapshot response is too
large / the snapshot response could not be queued) through refuse(),
which sends unreliable and unordered, so it needs no room in a full
queue, with ack fields that acknowledge nothing, as every runtime's
refusal does. onError keeps the detail.
Tests, with a raw peer that acknowledges nothing: a queue of 8 filled
exactly by an unacknowledged 8-fragment reply, then a refusal that still
arrives and acknowledges nothing; and a getCache failure carrying a
canary, refused with "the snapshot source failed".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
EndUnsendable put the SocketException's message on the wire, the
platform's wording. SendErrorCode names a permanent failure EMSGSIZE,
EINVAL or EAFNOSUPPORT, anything else UNKNOWN, so every runtime's
unsendable goodbye reads "packet could not be sent: CODE" with the same
codes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Queue-full refusal (findings 2 and 6). The refusal was built in
_send_rudp_schema_frame before _end_refused_peer reset the session, so
its ack fields covered the refused request (ack=102 mask=0x3). A reply
the queue cannot take now raises _ReplyUnqueueable; the loop ends the
peer through _end_refused_peer, which builds the refusal before the
reset, gives it ack fields built after it (acknowledging nothing, like
the goodbye and like Rust's refusal), sends it unreliable and unordered,
then the goodbye. A refusal fragment that can never be sent is not
dropped: the goodbye names its failure ("packet could not be sent:
EMSGSIZE"), where it used to say the session refused a packet.
Queue full to the brim (item 1). The tests fill a queue of 8 exactly with
a reply the peer never acknowledges, then show the next reply is answered
with response_unqueueable, acknowledging nothing, followed by the
goodbye; and that a refusal whose fragment cannot be sent ends the
session with a goodbye naming EMSGSIZE.
TCP (finding 8). A message whose handling raised (snapshot recordKeys as
an int, a shard-log afterSequence that is not a number, subscribe,
schema-catalog and verse requests with int fields) killed the
connection's thread with a traceback that could quote the peer. The TCP
handler now refuses such a message with fixed text (message_unhandled),
logs the error type only, and serves the connection's next message.
Daemon (finding 10). --port 99999 is an argparse error (exit 2), not a
traceback.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
end_unsendable_session put the OSError's text on the wire, which is the
platform's wording and can quote an address. send_error_code names a
permanent failure EMSGSIZE, EINVAL or EAFNOSUPPORT (Winsock equivalents
included), anything else UNKNOWN, as the Rust and TypeScript goodbyes
do; is_permanent_send_error is now "has a known code".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The queue-full test used an empty queue, so a reliable refusal fitted and
the reliable-refusal mutant lived. The new test fills a queue of 8
exactly with a reply the peer never acknowledges; the next reply cannot
be queued, and its refusal must still arrive, acknowledging nothing the
peer sent. A RawPeer helper drives the peer packet by packet; the gap
test uses it too.
A response too large for any datagram fails with the kernel's EMSGSIZE:
the rejection is ResponseSendFailed("EMSGSIZE"), the goodbye is exactly
"packet could not be sent: EMSGSIZE", and no refusal is sent.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
request_raw_snapshot_from_rudp_catalog put a catalog's error text into
its own error unfiltered. It now reports "refused the snapshot", naming
the refusal only when it is one of the fixed CultMesh refusal texts, as
the publisher does; both share recognised_refusal.
The publisher's doc claimed any refusing catalog is reported as a
refusal. That holds against a Rust catalog, which never acknowledges a
message it refuses; a catalog that acknowledges on receipt and refuses
afterwards (the TypeScript document server) is reported as having
admitted it. The doc now says so.
A fake catalog refusing with a canary is shared by the publisher test
and a new snapshot-client test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
end_unsendable_session put the io::Error's Display on the wire: the
platform's wording, which differs by OS and by which packet failed.
cultnet-rs gains send_error_code, the fixed name of a failed send's
cause (EMSGSIZE, EINVAL, EAFNOSUPPORT, or UNKNOWN), matching the codes
the TypeScript goodbye already carries; is_permanent_send_error is now
"has a known code". The goodbye reads "packet could not be sent: CODE".
ResponseSendFailed carried an ErrorKind, which lost the cause (EMSGSIZE
has no kind and read as Uncategorized). It now carries the same fixed
code, and the session ends where the send failed, over the real error,
so the goodbye and the caller's rejection name the same failure.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Only "nothing at the path" reads as an empty store; a zero-byte file is Undecodable in every
runtime; a dangling link is an I/O error. Errors name keys and schema ids only, never stored
values, and echo a header only in the cultcache.store.v<digits> shape. Single-file writes
report NotReplaced / ReplacedNotDurable / Rejected; an unlock failure after a successful write
no longer replaces the write's result.
Soul passes 1-3; batch 4 closes pass 3. Merged tree 9d9e76d4 verified on Yggdrasil: Rust
(cultcache, cultnet, cultmesh), cultcache-ts 44 including the four-runtime interop test,
cultnet-ts 198, cultcache-py 47, GameCult.Caching.Tests 346.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
receive() returns a reply only for a Ping, answered with a Pong, and a
Ping delivers no frame, so no rejection can follow an earlier send in the
same poll. The arm that carried such a failure into the rejection's end
was dead code; its mutant survived for that reason.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A reply sent earlier in the same poll could fail permanently and then be
dropped when a rejection returned first; and a refusal fragment that can
never be sent must end the session naming its error. Both now reach
end_session as the unsendable end. The test makes the refusal itself
unsendable: the peer's goodbye names the error, nothing is counted as a
lost datagram, and the other peer is served.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Send-failure semantics are Cut D's: a transient failure is counted and
never reported; a permanent one ends that peer's session.
Conflicts:
- cultmesh-ts index.ts: the two sendPackets. Cut D's
sendPacket(record, packet) is the one; the branch's reporting
sendPacket is deleted, and so is the options doc's "Every failed
datagram send reaches onError".
- cultmesh-py server.py: the rudp_send_failures doc is Cut D's. The
receive loop is Cut D's (handling guarded, permanent send errors
carried) with the branch's over_rudp=True. _send_rudp_schema_frame
sends in rudp_max_fragment_bytes fragments and returns Cut D's
permanent error from any fragment; a reply the queue cannot take no
longer vanishes into a count (below). _send_rudp_packet and the two
peer-ending helpers are Cut D's.
- cultmesh-py test imports: both create_rudp_schema_transport and
decode_rudp_packet.
Semantic resolution:
- cultmesh-rs: end_rejected_session and Cut D's end_refused_session and
end_unsendable_session collapse into one end_session(key, SessionEnd).
A refusal fragment that can never be sent is not dropped: the goodbye
names its error. The refusal stays unreliable, unordered, and
acknowledges nothing (its ack fields are built after the reset).
- cultmesh-rs: Cut D's "snapshot response could not be sent: {error}"
becomes CultMeshRudpRejectionReason::ResponseSendFailed(ErrorKind),
with a fixed REFUSAL_TEXTS entry. Its session ends as unsendable: the
peer gets the goodbye naming the error and no refusal.
- P3: a snapshot the reliable queue cannot take is answered.
cultmesh-py sends a small cultnet.error.v0 (response_unqueueable,
fixed text) unreliable and unordered, then the session ends as Cut D
ends any message the handling cannot serve. cultmesh-ts sends its
snapshot fallback error unreliable and unordered. cultmesh-rs already
answers through the refusal (ResponseQueueFailed); a test pins it.
- Tests follow: the branch's EMSGSIZE tests now expect Cut D's permanent
handling (the TS one is deleted, Cut D's own tests cover it), and the
Python queue test expects the answer instead of a count.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A peer's undecodable or unhandleable message ends only that peer's session and never
escapes Poll (the C# RUDP schema server's live crash). A send failure's goodbye belongs to
the generation that sent it. TS transient send failures are counted, not reported. The
interop harness finds the Rust peer through cargo metadata.
Soul passes 1-4; batch 5 closes pass 4's findings. Merged tree a892e117 verified on
Yggdrasil: GameCult.Networking 481, Mesh 264, cultnet-ts 198, interop 27/27 with
CARGO_TARGET_DIR set.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first canary put a string where a u32 goes, and rmp-serde refuses that by
marker without quoting it, so the canary passed with the serde text back.
A string where a unit enum goes is quoted as an unknown variant. The canary
now covers get, get_all, get_all_with_keys, put_envelope and prepare_entry.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The harness assumed that the Rust interop peer builds under
packages/cultnet-rs/target. When CARGO_TARGET_DIR or a cargo config's
build.target-dir is set, cargo builds elsewhere, and every Rust lane
failed with a spawn ENOENT. The path now comes from `cargo metadata`'s
target_directory, read once on first use.
The TypeScript peer imports @gamecult/cultcache-ts, and the Python peers
import msgpack. When either was missing, the lanes died with import
errors in a child's stderr or with timeouts. The first use of each peer
now checks its prerequisite and fails with a message that names it and
the command that provides it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With only 1 KiB fragments, a pre-check measuring a small multiple of a
fragment (Soul's min(payload, frag*2)) still passed a 70,000-byte send.
The fixture now also runs with 40,000-byte fragments. Any such multiple
of that is over the datagram limit, so the mutant dies.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The schema server's backstop ended every session with "session refused
a packet", including when the fault was the server's own: for example a
handler's response too large to send, or a refusal the handler did not
type. Decoding and handling are now two catches:
- A message that cannot be decoded is the peer's. The session ends with
the refusal reason.
- A handler fault ends the session with "server could not handle a
message".
The transport's EndRefusedPeer becomes EndPeerSession(peer, reason). It
reports whether it ended the session, so the log says when a response
that failed after its first fragment had already ended it over the send.
Before, the log reported that it ended a session that was already gone.
The comment said that ending the session matched cultmesh-py. It does
not: Python silently skips a malformed frame. The honest anchor is the
C# TCP schema server, whose ServePeerAsync ends the one peer whose
message fails to decode or handle. The comment now says so.
Pinned, each by a test that kills its mutant:
- a handler fault's goodbye carries the server's reason, not the refusal
(mutant: the refusal reason);
- the post-fragment-0 path logs that the session had already ended
(mutant: a constant outcome);
- EndPeerSession says no goodbye to a replaced peer's endpoint (mutant:
the Ended check dropped);
- a refused peer's frames still queued are never handled (mutant: the
queue purge dropped).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
3b9ffac2 claimed that the generation check subsumed the UnsendableReason
once-guard. It did not. A thread that reads the generation only after
the session has ended carries the ended session's own generation, and the
check lets it through. For example, the polling thread answers a Ping on a
session that a caller's send has just ended but that has not yet been
retired. When that Pong fails permanently, the thread says a second
goodbye and overwrites the first failure's reason (Soul pass 4, PR4).
EndUnsendableSession now also returns when the session has already ended.
Two tests:
- The polling thread answers a Ping on a session already ended, and the
Pong can never be sent. Exactly one goodbye goes out, and the first
failure's reason stands. The hook admits the Ping after the session
ended and before the peer is queued for retirement. The test kills the
mutant that drops the Ended check.
- A failed send's goodbye is sent under the gate. A polling thread
admitting a new Connect from the endpoint waits on the gate, so the old
generation's goodbye never follows the new session's Accept. The test
kills the mutant that sends the goodbye outside the gate.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The five-slot refusal ran a non-string key or schema id through String(),
quoting stored bytes. A string is named; anything else is described by its
type.
A push through a live link, and a pushAll through a dangling one, replace
the link with a regular file. Tests named for R3 pin that until R3 decides
whether writes resolve or refuse links.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The five-slot refusal echoed a non-string key or schema id with repr, quoting
stored bytes. A string is named; anything else is described by its type.
A push through a live link replaces the link with a regular file. A test
named for R3 pins that until R3 decides whether writes resolve or refuse
links.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The redb envelope decode (get by identity and pull_all) and every typed
document decode (get, get_all, get_all_with_keys, put_envelope, prepare_entry)
kept serde's text, which quotes stored values. They now keep CultCache's
context, which names the entry by type and key, and a fixed description of
the decoder failure's kind.
- A snapshot the store cannot hold (two types claiming one schema id) was
reported as NotReplaced, so a caller retrying on that kind looped forever.
It is CultCacheStoreWriteFailedKind::Rejected now, raised before any I/O.
- release_lock and the read-only snapshot state one unlock policy: an unlock
failure never replaces the result. Only the read-only path, whose API has an
observer, reports it. release_lock also names the case where closing does not
release the lock: a duplicate of the open file description, such as a child
forked without exec.
- README: reads refuse a dangling link, but writes do not follow links yet. A
write through a live link, and push_all through a dangling one, replace the
link. Two tests pin that behaviour, named for R3 to decide between resolving
and refusing. The write example classifies Rejected apart from NotReplaced.
- The shared-gate test waits 30 s and 60 s for the writer instead of 1 s, and
checks that the file on disk is untouched while the gate is held.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The refusal went reliable and ordered, but the session ends with it, so
it is never resent: a peer missing any earlier reliable packet from the
server held it behind that gap forever. It now goes unreliable and
unordered, so it is delivered on arrival whatever the peer is missing.
The test drops the server's snapshot reply before the refused put: the
refusal still arrives, followed by the goodbye.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The refusal sent to a rejected peer carried the rejection's Display
text, which for SinkRefused and SnapshotSourceFailed is the sink's or
source's own error. Odin's sink formats the peer-supplied document type
into its error, so a peer's input came back on the wire.
The refusal now carries fixed text per rejection variant
(REFUSAL_TEXTS). The rejection returned from poll_once still carries the
detail for the caller; the server itself logs nothing.
publish_cultnet_message_to_rudp_catalog no longer quotes a catalog's text
unfiltered: it reports "refused the put", quoting the text only when it
is one of the fixed refusals.
The test fixtures' sink and source errors carry a canary; the refused
peer's text is the fixed one and never contains it. A fake catalog that
refuses with a canary of its own shows the publisher's error stays
fixed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The RUDP option rules refused a TCP-only server (enable_rudp=False) for
RUDP limits it never uses. Rust checks those options only for the RUDP
server they configure, so Python now checks rudp_resend_delay_ms,
rudp_max_pending_reliable_packets, rudp_max_fragment_bytes and the empty
snapshot's fragment rule only when enable_rudp is true. The snapshot
limits still apply to every server.
The daemon turned --max-snapshot-documents 0 into a traceback. The server
still owns the rule; the daemon reports its refusal through
parser.error: exit status 2 and the rule named, no traceback.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A well-formed put still killed the RUDP thread: _handle_raw_put called
apply_raw_put_message unguarded, and cache.put_envelope raises for a
blank recordKey, a payload that is not MessagePack or does not decode to
its schema, a global document under another key, or anything a
document's own decoder or validator raises. Over TCP it ended the
connection.
The apply site now refuses any put the store cannot take to the peer
that sent it, as malformed_document_put, over RUDP and TCP alike, and
logs the error's type. Neither the refusal nor the log quotes the error,
which can quote the put. No new shape checks: the store decides what it
can take.
The test sends each shape over RUDP and over TCP and shows the RUDP
thread alive, the TCP connection serving its next message, and other
peers served.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Admission has sized only the served reply since a1065cab; three comments
and one fixture assertion still described the receipt bound.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cargo-mutants found no test of what CultCacheStoreWriteFailed says. Both
outcomes now assert the path and the outcome in the error text.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
with_shared_lock, with_exclusive_lock, try_with_exclusive_lock and the redb
store's with_database returned the unlock's error in place of the action's
result, so a write that had replaced the store could be reported as failed
and read by Odin as not replaced. Closing the lock file releases the lock
whether or not the explicit unlock succeeds, so all four now release through
one helper that returns the action's result. The unlock failure is not
surfaced anywhere: there is no diagnostic channel on these paths.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every single-file write now carries CultCacheStoreWriteFailed in its error
chain, above the cause. NotReplaced: creating, writing, syncing or renaming
the staging file failed, and the store file holds what it held before.
ReplacedNotDurable: the rename landed and the directory sync failed. Odin
could not tell these apart, so it kept every failed snapshot as a candidate.
The text names the path and outcome only.
fsync does not fail on a healthy filesystem, so the staging and directory
sync steps take a fault armed per thread by tests; builds outside tests never
fail there. The staging write (RLIMIT_FSIZE child) and the rename (a
directory at the store path) fail for real.
README: the write outcome, the dangling-link rule and the no-echo rule.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The registry method existed on this branch only to size a receipt at put
admission in cultmesh-ts, which no longer does. It never reached main.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Admission sized the larger of the snapshot reply and a receipt echoing
the put, so a put servable at the bound was refused, within 89-125 bytes
of it, for a receipt an application that sends none never sends. Whether
a receipt is sent is onDocumentPutRaw's to decide after it stores the
put, so admission cannot know it (ruling: Self, fork A option a).
Admission now sizes only the snapshot reply serving the document alone.
A receipt that cannot be sent takes the existing path: onError hears why
and the peer is told with a cultnet.error.v0 that it is not coming. This
removes the receipt bound of 9757ed54 and its test.
The new test pins the band at its boundary with puts whose echoed receipt
is over the limit: exactly at the limit is admitted and served, with no
receipt and with an echoed one, and one byte over is refused.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RequireSingleFileFormat echoed any header string. It now names one only in
the shape cultcache.store.v<digits> and otherwise gives its byte length.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An unsupported header is echoed only as cultcache.store.v<digits>; any other
is described by its byte length (a 2,914-character header was echoed whole).
pullAll's ENOENT catch wrapped the whole decode, including the legacy-repair
write, so a failed repair read as an empty store. It now covers only the read,
and only nothing at the path is absent: a dangling symbolic link is refused
instead of being replaced by the writer. A soft pushAll uses the same read.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An unsupported header is echoed only as cultcache.store.v<digits>; any other
is described by its byte length, since the bytes are the store's.
A read that found nothing through a dangling symbolic link returned an empty
store, and the writer then replaced the link. Both stores now share one read
rule: only nothing at the path is absent. JsonLinesBackingStore also drops
`Path.exists()`, which reported a symlink loop or a file-as-parent as absent.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
"cultcache.store.v" is a prefix of the "cultcache.store.v1" the refusal names,
so the unquoted check failed on a correct refusal.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A decode refusal kept serde's text, which quotes values read from the file
(`invalid type: string "..."`). The detail is now CultCache's own context and
record refusals, which name only a record's key and schema id, plus a fixed
description of the decoder failure's kind. rmp-serde reports no position, so
none is given. An unsupported header is echoed only in the shape
`cultcache.store.v<digits>`; any other is described by its length.
`is_absent` used `fs::metadata`, which follows links, and a read that found
nothing through a dangling link returned an empty store, so a writer then
renamed a regular file over the link and the store left its volume. Only
nothing at the path is absent now (`symlink_metadata`), and a read through a
dangling link is an io::Error.
Tests: canaries for a catalog and a legacy value and for unshaped headers,
the dangling link on every read with nothing written, and a
`cultcache.storeX` header that claims no format.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
