CultLib/contracts/cultnet/interop/write-receipt.md
soul 2aade7fdd3
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m24s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 21s
Docs and test comments: correct the Rust duplicate-key citation, say what the TS and Rust duplicate tests pin
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 16:25:50 +00:00

23 KiB

CultNet write receipt (specification)

Status: specification. No runtime implements it yet; the conformance follow-ups below do. Rules: Heimdall's ruling of 2026-11-08 (a write completes on its result frame; an ack or a return is not admission). Files here: cultnet.write-admitted.schema.json, cultnet.error.v1.schema.json, write-receipt-vectors.json. They move to contracts/cultnet/ when the first runtime implements them.

The problem

A put or delete today has two answers on the wire: nothing (admitted, or the server never read it) and cultnet.error.v0 (refused, with no id saying which write). A transport ack proves delivery. Clients that completed on the ack, or on a window with no refusal, reported refused writes as done (CultLib #55 to #58 bound this with a result window; this spec replaces the window with an answer).

Frames

cultnet.write_admitted.v0: { schemaVersion, messageId, operation: "put"|"delete", schemaId, recordKey }, every key written, none nil. Sent once a put or delete is admitted. The id echoes the write's messageId byte for byte; operation, schemaId, recordKey echo the write.

A write_admitted messageId is valid only if it contains at least one Unicode letter (\p{L}) or number (\p{N}) and no Default_Ignorable_Code_Point (DerivedCoreProperties). The compare is raw: the id is echoed byte for byte, so a decoder never trims or normalizes it (" a " is valid; U+2800 is not, and neither is a followed by U+200B). Unlike the Required fields rule, this is not a White_Space test. The letters, numbers and Default_Ignorable code points are those of Unicode 16.0 in every runtime, taken from tables embedded in each runtime and generated by scripts/gen-unicode-messageid-tables.py from DerivedGeneralCategory.txt (Lu Ll Lt Lm Lo Nd Nl No) and DerivedCoreProperties.txt; no runtime asks its platform's Unicode data, so a runtime's own Unicode version cannot change a result. Bumping the version is one coordinated change: regenerate the five tables and update the vectors that pin the version in the same commit. The rule is on the decoded string, which must be well-formed Unicode: a lone surrogate refuses. A runtime whose MessagePack reader rejects invalid UTF-8 refuses the frame earlier; byte-level strictness is a property of the reader and is not required of a decoder. A frame with more than one messageId key is malformed: a decoder that can see the duplicate (the C# and Rust decoders) refuses it; a decoder that collapses the map (Python, TypeScript, Kotlin) judges the value it ends up with, the last key, and refuses if that value is invalid, so no decoder echoes an invalid id (vectors admittedMessageIdDuplicates declare each policy and every runner asserts its own). A decoder refuses an invalid id the way it refuses an empty or ill-typed one; the rule is for write_admitted only and error.v1 is unchanged.

Where a received messageId decides, and what that relies on. Beyond echoing it, a received messageId is used in exactly two places. On a server, the remembered-write key (actor, messageId) with the content hash, the per-session limit and the replay of rule 5 are taken from the id of the write frame the server itself parsed. On a client, the write tracker attributes a write_admitted or error.v1 to a write by equality with the id of a write that client itself sent (a frame naming an id not in flight settles nothing), and attributes an id-less error by the count of unanswered writes, not by any id. Both rely on the receiver being the only parser of that frame: a decoder's messageId on a frame that carries more than one messageId key is not trusted for any authorization, dedupe, replay or ordering decision, and the map-collapsing runtimes (Python, TypeScript, Kotlin, and C#'s typed deserializer on write frames) cannot see the duplicate. If a frame is ever parsed by two runtimes in a chain (a relay, proxy or forwarder that re-reads it), or a consumer needs messageId for authorization, dedupe, replay or ordering beyond echo and equality with a self-sent id, all runtimes must first refuse a duplicate messageId key (a spec change plus byte-level duplicate detection in the runtimes that decode to a map) before that use lands. Today Rust refuses a repeated key on write frames and on write_admitted (serde's duplicate field messageId, and refuse_duplicate_keys for the raw-list frames such as document_put_raw), though its readable_message_id names the first key when it names an unreadable frame; C# does not on write frames (a duplicate-key write is deduped on its last id, while the id it names in a refusal for an unreadable frame is read from the first key), and the Python server (cultmesh_py/server.py) only echoes the id while the TypeScript and Kotlin code matches it by equality with a request it sent.

cultnet.error.v1: cultnet.error.v0 plus a required messageId (string or nil). The code enum is v0's plus limit_reached (see Remembered-write limits); cultnet.error.v1.schema.json is the authority for the closed set. Sent in place of error.v0 to a receipt-capable connection. messageId is the id of the request the error answers; nil only when the frame was unreadable or carried no id.

Why a new error version and not an extra key on v0: the v0 schema is closed (additionalProperties: false) and the TypeScript decoder (Ajv over the closed schemas) rejects unknown keys and unknown schemaVersions, and parseCultNetMessage throws on an unknown version; Rust's decoder errors on an unknown version too. Unknown-key tolerance of the Rust, Python and Kotlin error decoders was not checked, so none is assumed, so a surprise key or frame breaks an old client's whole receive path. Old clients are therefore never sent either new frame. This reverses the "old clients ignore it" assumption, which is false for today's decoders.

Capability: opting in

A connection is receipt-capable when its cultnet.hello.v0 (sent before its first write; the schema channel is reliable and ordered) lists both cultnet.write_admitted.v0 and cultnet.error.v1 in supportedMessageVersions. A server MUST NOT send either new frame to any other connection, and answers it exactly as before (no receipt, error.v0). A receipt-capable server lists both in its own hello, so a client can tell a legacy server from a receipt server that has not answered yet. A client that sends no hello gets legacy behaviour. A server advertises the two frames in its hello only if it implements the receipt rules end to end.

The capability changes framing only; it never authorizes. A hello authenticates nothing and grants nothing: who may write, and which keys, is decided by the server from the transport-authenticated actor exactly as without the capability. Declaring the capability never makes a refused write admitted, never widens an actor's authority, and never changes what a write does; it selects only which frame the answer arrives in.

Server rules

  1. One answer. For every put or delete on a receipt-capable connection the server sends exactly one of write_admitted or error.v1, carrying that write's messageId. Never both, never two different answers.
  2. After the commit point. write_admitted is sent only after the write is committed by the same rule that makes it visible to readers. Receipt of the bytes, parse, or authorisation alone is not admission.
  3. Id on every refusal. Every refusal of a write, whatever its code (write_not_authorized, message_invalid, unowned_schema, unsupported_schema_version, shard authority errors, ...), is an error.v1 whose messageId is the write's id. When the frame is readable enough to find the id, it is found; message_invalid for a missing or ill-typed field still carries the id. Nil only when no id can be read.
  4. Batches. There is no batch frame; a batch is N writes, each with its own messageId, each answered under rules 1 to 3. A failed batch names its member by that id. A server may not collapse a batch into one answer.
  5. Identity and idempotent resend. A messageId is unique per authenticated actor and session: the client mints it, never reuses it for different content in that session, and a server never accepts the id of one actor as the id of another. The server's dedupe key is (actor, messageId), where actor is the transport-authenticated identity, never a name the hello or the frame claims. Alongside the key it keeps the content hash of the write and the answer it gave. The content hash is SHA-256 of the MessagePack array [operation, schemaId, recordKey, payload], where payload is the raw document payload as a MessagePack bin for a put and nil for a delete (the vectors give worked hashes). Scope: at least the session. The key lives for the life of the session it was made in, and a server may bound how many writes it remembers per session (see Remembered-write limits; a server never forgets an id it promised to answer). A server MAY keep the key longer for the same actor, up to DEDUPE_RETENTION_MS (10 minutes), but a client MUST NOT rely on it: a reconnect is a new session, and a resend on the new session is a new write.
    • Same key, same content hash, answer was write_admitted: answer write_admitted again with the same body. The write is not applied twice.
    • Same key, same content hash, answer was a refusal: the same refusal again.
    • Same key, different content hash: refuse message_invalid (details field: "messageId") and never re-send the old receipt. Old receipts are for the old content; showing one for new content would report an unapplied write as admitted. The original write's state is unchanged.
    • Id size. A write's messageId is at most 256 bytes of UTF-8. A longer id is refused message_invalid (details.field: "messageId", the refusal's messageId the id as received, text fixed per Fixed texts) BEFORE anything is remembered: no table entry, no digest, no budget charge. Such a refusal is therefore not replayed from the table; a resend is refused afresh, identically.
    • What is remembered. Per entry a server keeps only fixed-size digests (SHA-256) of the messageId and of the content hash, plus the answer (admitted, or the refusal's code and details). Memory per entry is bounded independent of id length and payload size. Lookup compares digests.
    • Remembered-write limits. The per-session table holds at most 1024 entries. A server MAY also enforce a server-wide memory budget across all sessions (reference: 32 MiB of remembered entries). A NEW write (a key not already in the table) that would exceed either bound is refused limit_reached for that write alone, messageId the write's id, details all null; the write is not applied and the session continues. A resend of a remembered key is still answered from the table.
    • Limit refusals are remembered. A limit_reached refusal is remembered like any other: a resend of the same id and content hash replays limit_reached. No slot is freed mid-session, so an id refused at a bound never becomes admitted later in that session (a server at the cap keeps the refused digest pair in a bounded refusal set of at most 1024 per session, outside the entry budget; past it, a further new write is refused limit_reached without being remembered, and the session refuses every id it has not seen for the rest of its life, even if a server-wide budget has since freed, so a refusal that was not remembered is never followed by an admission of the same id). The client mints a new id on a new session.
    • Required fields. A put or delete whose document or schemaId is null or missing is refused message_invalid with details.field "document" or "schemaId" (the first missing in that order) and fixed text; the write is not applied. A put or delete that is present but names nothing is not "missing": the write is refused message_invalid exactly as a frame that does not read, with details.field "message", the fixed unreadable text and the write's id (vector error_v1_empty_schema_id_is_unreadable_and_names_the_write), and is never applied; a server must not hand it to its store. Names nothing means, on a put or delete, a schemaId or recordKey that is blank; on a put's document, also a storedAt that is blank, a payload of zero bytes (a MessagePack value is at least one byte, so an empty payload is no document), a sourceRuntimeId, sourceAgentId or sourceRole that is present and blank, and a tags entry that is blank. Blank means empty or made only of Unicode White_Space code points (the PropList property) and nothing else, in every runtime, whatever its platform's own whitespace test says. The set is exactly U+0009 to U+000D, U+0020, U+0085, U+00A0, U+1680, U+2000 to U+200A, U+2028, U+2029, U+202F, U+205F and U+3000. U+200B, U+200C, U+200D, U+2060, U+FEFF, U+180E and U+001C to U+001F are not White_Space (some platforms' whitespace tests accept U+001C to U+001F or U+180E, for example Java's Character.isWhitespace and older .NET, so a runtime must check the list, not its platform test): a string of them names something. The vectors list both sets (whiteSpaceCodePoints, notWhiteSpaceCodePoints) and a runtime checks every code point in every covered field. Absent or nil source ids and tags, and an empty tags list, are valid. The rule holds on every connection, receipt-capable or not (a legacy connection is refused with the error.v0 of the same code, details and text); the vectors' blankFields section lists each case.
    • Fixed texts. The error string of these refusals is fixed, so vectors can compare it: message_invalid: messageId exceeds 256 bytes, message_invalid: document is required, message_invalid: schemaId is required, message_invalid: the message could not be read (a frame that does not decode, or a write that names nothing; the text is the same for both and never carries a decoder's own wording), message_invalid: receipt sessions require the reliable channel, limit_reached: the session's remembered-write limit is full.
    • Reliable channel only. A write is accepted on a receipt session only on a channel that is reliable and not sequenced (reliable ordered and reliable unordered both qualify; a sequenced channel drops superseded packets, so a write on it can vanish without trace). A write arriving on any other channel is refused message_invalid with details.field: "channel" (the transport profile's word for it) and fixed text. Receipts and refusals travel the reliable channel; a write that could be lost without trace cannot be answered once. The server refuses a sequenced write whenever it can read the write's id: from the delivered frame, and from a superseded packet that carries its whole frame. A write sent as several fragments whose frame is superseded is dropped whole and may go unanswered; the client's unconfirmed outcome covers it. A repeat of a packet the server already took, or a packet too far ahead to be taken, draws no further answer (rule 1).
    • Past the session (or a server's longer retention, if it keeps one) a resend is a new write.
  6. Forwarded writes. A server that forwards a non-primary write answers the original peer only with what the primary answered: the primary's receipt becomes write_admitted with the original id, its refusal becomes error.v1 with the original id. If the forward gets no answer the server relays nothing: no receipt, no error, no synthesized refusal. The client's own rule below then reports the write as unconfirmed. A forwarder never turns an absent or ambiguous primary answer into an admission.
  7. Looks alike. An admitted and a refused write differ in schemaVersion; a receipt has no error, an error.v1 has no admission. A transport ack is neither.

Client rules: three outcomes

A write settles as exactly one of:

  • admitted: a write_admitted whose messageId is the write's.
  • refused: an error.v1 whose messageId is the write's, carrying its code and details as the typed error.
  • unconfirmed: nothing settled it. Reasons: window_elapsed (the result window ran from the send and passed with no answer; see Result window), session_ended (the peer closed the session, or it ended, before the write was answered; a write registered after the session ended settles this way when it is sent) and local_send_failed (the client's own send of this write failed). A local send failure is never reported as refused: no server answer exists, and the write may or may not have left the machine. Unconfirmed is not success and not refusal. APIs that return a value return the tri-state; APIs that throw throw a distinct refused error and a distinct unconfirmed error and return normally only for admitted. A caller that needs admission treats unconfirmed as not admitted. Resend with the same messageId and unchanged content, on the same session, is safe against a receipt-capable server (rule 5). After a reconnect the resend is a new write, and a legacy server may apply a resend twice: the caller resends only writes that are idempotent by record key (a put of the same content to the same key, a delete of a key), and reads the record back before resending any other.

Result window. Two named constants (also in the vectors): RECEIPT_RESULT_WINDOW_MS = 5000, the default against a receipt-capable server (its hello lists both frames); LEGACY_RESULT_WINDOW_MS = 500, used only as the fallback against a server whose hello does not list them (or that sent none). The window runs from the send. It ends when the write's age reaches it: an answer arriving at exactly that instant is late. A receipt-capable server that has not answered within its window is reported as unconfirmed like any other; clients log it as a server that missed its window.

Late answers. An answer for a write that already settled unconfirmed does not change what the caller was told, but the client records it on the write: a late write_admitted makes the write's recorded state admitted_late, a late refusal makes it refused_late. The write handle exposes that state, and a caller that got unconfirmed MUST consult it (and the record) before resending: it must not blind-resend a write that was admitted late. A resend on the same session with the same id and content is harmless on a receipt-capable server (rule 5); across a reconnect the caller follows the idempotence rule above.

Legacy answers. An error.v0, or an error.v1 with nil messageId, names no write. It is attributed to a write only when exactly one write on the connection is unanswered; it then refuses that write. With two or more unanswered writes it settles none of them and each ends unconfirmed at its window. A frame naming a messageId that is not in flight settles nothing.

Batches can commit partially. A batch is N independent writes with no atomicity: some members may be admitted while others are refused or unanswered, and nothing is rolled back. The client reports each member's outcome and a batch result is only their set: admitted if every member is admitted, refused if any member is refused (naming the members), otherwise unconfirmed (naming the unanswered members). The caller owns retry: it resends exactly the members that are not admitted, with their original ids.

Changes to what #55 to #58 shipped

Those clients complete successfully when the window ends with no refusal. Under this spec that case is unconfirmed. The TypeScript one-shot writer (#62) already reports it so: it resolves admitted only on a write_admitted carrying the put's id, and no other frame, a put echoed under the same id included, ever counts as admission. A caller whose server cannot answer a write yet (Odin, until it is receipt-capable) opts into the explicit send-only mode per call (sendOnly: true), which resolves the distinct outcome sent once the transport has acknowledged the put and never admitted. The Idunn presence publisher is send-only by its own function (sendPresenceDocument, not the sendOnly option): the same sent result, plus a rejection if a refusal naming the put (or naming nothing) arrives by the transport ack. Receipts and send-only are both opt-in, neither is the default. The C# and Rust clients of #55 to #58 keep the window-success until their conformance work lands, and callers of those must not read it as admission.

Vectors

write-receipt-vectors.json: constants (the named windows and retention), contentHash (worked dedupe hashes), dedupe (server replay cases for rule 5), wire (MessagePack hex plus the decoded message for each new frame; the bytes are spec-authored with msgpack-python and every runtime's first conformance PR re-captures them with its own encoder and must match), schemaInvalid (frames that must be refused by a validator, each with why), and blankFields (server input cases for the Required fields rule: which document fields make a put unreadable, which are accepted), admittedMessageId (the write_admitted id rule above: code-point lists, each marked refuse or accept; a lone surrogate, and a frame that repeats the key, are carried as whole frames in wireHex, because a lone surrogate is not valid UTF-8 and only some runtimes' strings can hold it; write-admitted-message-id-ranges.json lists every range of the embedded tables, and each runtime probes the first and last point of each range and the points either side), scenarios (event replays with the outcome each write must settle as; the clock is abstract; each case names its window constant). A conformant client passes every scenario; a conformant server's answers for the same inputs satisfy server rules 1 to 7.

Conformance status

Server class Advertises both frames Server-wide memory budget (MAY, Remembered-write limits)
CultNetDatabaseServer yes not implemented: only the per-session limit of 1024 applies
Shard write forwarder yes not implemented: only the per-session limit of 1024 applies
Rust RUDP document server yes implemented (32 MiB default, max_remembered_write_bytes)
C# RUDP schema server (CultNetRudpSchemaServer, src/GameCult.Networking/CultNetRudpSchemaServer.cs) no n/a

Advertising is per server class, not per transport. A class advertises both frames in its hello only if it implements the receipt rules end to end; the C# RUDP schema server stays "no" until it answers writes with receipts.

Cross-runtime check: packages/cultmesh-ts/test/interop/write-receipt-interop.test.ts (npm run test:interop in packages/cultmesh-ts) runs the TypeScript client (WriteTracker, the decoder, the one-shot writer) against the real Rust RUDP document server and the real CultNetDatabaseServer over loopback, one scenario set against both. Known gap it records as a todo: the Rust RUDP document server refuses every delete (write_not_authorized: the server does not take deletes), where this spec has an admitted delete receipted like a put.

Conformance follow-ups (not in this PR)

C# server and client, Rust, TypeScript, Python, Kotlin: emit and parse the two frames; hello capability; tri-state client outcome; replace the #55 to #58 window-success; re-capture the wire vectors in each runtime. Server write paths include the shard write forwarder (rule 6) and the Odin catalog.