Compare commits

...
Sign in to create a new pull request.
Author SHA1 Message Date
7665df33ae Merge pull request 'TS: write_admitted / error.v1 decoder, WriteTracker, vector replay runner' (#62) from libby/ts-write-receipt-client into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Failing after 5m10s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Successful in 31s
2026-11-08 19:52:37 +00:00
92b4cb7ce9 Merge pull request 'C#: database server answers writes with write_admitted / error.v1 (write-receipt spec)' (#61) from libby/cs-write-receipt-server into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-08 19:51:52 +00:00
36494dafb9 Merge pull request 'cultmesh-rs: RUDP document server answers writes with write_admitted / error.v1' (#60) from libby/rs-write-receipt-server into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-08 19:49:50 +00:00
5bf2428295 Merge pull request 'spec: cultnet write receipt (write_admitted, error id echo, admitted/refused/unconfirmed)' (#59) from libby/cultnet-write-receipt-spec into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-08 19:49:40 +00:00
Libby
1a228349f1 C#: a put or delete with an empty or blank schemaId or recordKey is refused message_invalid (as an unreadable frame, like the Rust server) and not applied
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m35s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 31s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 19:46:45 +00:00
Libby
621b86ca15 Spec #59: an empty or blank schemaId or recordKey is message_invalid (details.field message, the write's id), never applied; vector for it
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m55s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 46s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 19:45:19 +00:00
Libby
90d252735f cultmesh-rs: a refused reliable-sequenced write is received and acknowledged first, and a refused write counts as session activity
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 6m11s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 52s
Refusing before session.receive left the packet unacknowledged: the client resent it forever and every later ordered write stalled. The test now uses the same peer and proves a later ordered write is admitted.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 19:38:43 +00:00
Libby
34bf7aa0ee C#: test kills the schema id detail-value mutant
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m59s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 31s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 17:03:36 +00:00
Libby
40b34295e3 C#: receipt writes need a reliable, non-sequenced channel; a refusal without a record key or schema id carries a null detail value
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 17:01:59 +00:00
Libby
e4b67a9e0b cultmesh-rs: a receipt session accepts writes only on a reliable, non-sequenced channel; a test pins that a reconnect starts a fresh session
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 10m5s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 1m8s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 17:00:48 +00:00
Libby
48cf3b3323 C#: tests kill the replay-details and unreliable non-write mutants
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 16:45:29 +00:00
Libby
0a67b4d8aa C#: a remembered refusal keeps its code and field, never the write's strings, and replays in fixed text; a write on an unreliable channel of a receipt session is refused message_invalid (channel)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 16:42:22 +00:00
Libby
1a05df2524 cultmesh-rs: server-wide remembered-write budget is a maintained counter; a session whose refusal set overflowed refuses unseen ids after the budget frees
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m35s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 1m1s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 16:36:10 +00:00
Libby
c8f869e54e cultnet-rs, cultmesh-rs: null or missing document/schemaId refused naming the field; budget tests use literal bytes; unreliable-refusal local failure test
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m33s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 26s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 15:32:03 +00:00
Libby
d7b36416c6 cultmesh-rs: pin limit-refusal memory bound and legacy unreliable drop (mutant-driven tests)
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 15:30:36 +00:00
Libby
ef5cd88111 C#: v0 errors for snapshot, shard log and variant refusals carry fixed text, never an exception's message
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m35s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 36s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 15:13:58 +00:00
Libby
d654cb2b99 C#: tests kill the refusal-set bound and abandoned-rehash mutants
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 15:12:31 +00:00
Libby
1447ccd8dd cultnet-rs, cultmesh-rs: write_admitted in spec field order; server-wide remembered-write budget over digest entries; id size limit; limit refusals remembered; unreliable writes refused; local failure is its own goodbye
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m14s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 46s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 15:09:28 +00:00
Libby
c4d1efe000 C#: receipt ids capped at 256 bytes and held as digests; limit refusals remembered, no slot freed; required fields refused with fixed text; RUDP server advertises no receipt frames
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 15:07:23 +00:00
Libby
541d734675 C#: a forward is retryable only when provably not sent; unconfirmed otherwise; refusals carry fixed text
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
CultNetWriteNotSentException names the pre-send boundary (connect, hello, no endpoint,
or a send port that proves nothing was written). Every other forwarding failure keeps
the session claim. ForUnownedSchema and the routing refusal no longer carry exception text.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 15:02:56 +00:00
Libby
2e5df24e96 C#: loopback test for the unreadable-frame answer; Server.Port seam; the answer now reaches the wire
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
An undecodable frame is classed legacy by Decode, so the schema-side guard was unreachable
and a receipt peer was never answered. The guard moves to the legacy decode.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 14:58:17 +00:00
Libby
74b8a4cbf8 C#: drop the redundant Silent state of a write claim (a silent claim is an unanswered one)
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 14:53:05 +00:00
Libby
0182a6d40b cultnet-rs, cultmesh-rs: remembered-write cap refusal is error.v1 limit_reached; pin authorize-once with a counting sink
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m19s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 41s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 14:32:49 +00:00
Libby
ed5f9d30d1 C#: receipt server follows Heimdall's review of #61
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m30s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 21s
Look the id up first and authorize only on first sighting; remember refusals within the cap; never re-forward an id whose primary stayed silent; end the connection when an answer cannot be queued; fixed refusal texts; limit_reached cap refusal; commit-point doc on ApplyPut/Delete.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 14:32:39 +00:00
Libby
5f699c27fa cultmesh-rs: a put reusing a delete's id is refused (mutant-driven test)
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 14:27:54 +00:00
Libby
d4f440ac7c cultmesh-rs: receipt server looks the id up first, remembers refusals, answers deletes and unqueueable refusals
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 4m47s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
Follows Heimdall's second review of #60: replay by (session,id) before any
authorization; refusals stored with their hash; delete shares the dedupe
table and settles capability; a refusal the queue cannot take ends the
session; a peer's refusal frames and a receipt session's refused deletes are
not violations; the write limit has its own fixed text; the sink trait says
Ok means committed and visible.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 14:22:50 +00:00
Libby
b0ea18371f C#: database server answers writes with write_admitted / error.v1 (write-receipt spec)
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m46s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 36s
Opt-in per connection by the first hello (or first write); the server answers a
capable hello with its own. A put or delete commits, then gets write_admitted under
its id; every refusal is error.v1 with the id (an unreadable frame carries the id
when it still holds one). (actor, messageId) dedupe is bound to the content hash and
asked only after the write authority grants; same id other content is message_invalid.
The shard write forwarder says hello, waits for the primary's receipt, and the server
relays only what the primary answered under the original id.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 14:04:37 +00:00
Libby
8bd31a5d4a cultmesh-rs: receipt server follows Heimdall's review of #60
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 7m11s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 26s
A refused put, a full dedupe table and a reused id refuse the write, not the
session; only a protocol violation ends one. A frame that does not decode but
holds a messageId is refused with that id. The server answers a receipt-capable
hello with its own, listing both frames. The sink's authorization (new
authorize_raw_document, default admit) runs before the session's remembered
writes are consulted. A second hello does not change the capability: it is set
once, by the first hello or the first write. A receipt that cannot be queued, and
a write the payload budget would drop silently, end the session instead of
leaving the write unanswered. Refusals to a receipt-capable session are queued
reliably.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 13:56:17 +00:00
Libby
45e95984e5 WIP C# write-receipt server (not built, not tested): messages, content hash, per-peer dedupe state, receipt-aware wait
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 13:41:14 +00:00
Libby
9f73047c29 cultmesh-rs: RUDP document server answers writes with write_admitted / error.v1 (write-receipt spec)
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
cultnet-rs gains cultnet.write_admitted.v0, cultnet.error.v1 (v0 + messageId) and
write_content_hash. The server opts a session in only when its hello lists both frames
(framing only; admission rules are unchanged). On an opted-in session an admitted put is
answered with a receipt carrying its id after the sink accepts it, a refusal is an error.v1
naming the id, a resend of the same content is re-receipted without reaching the sink, and the
same id with other content is refused message_invalid (never the old receipt). Other sessions
are answered exactly as before.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 13:38:46 +00:00
22 changed files with 4677 additions and 156 deletions

View file

@ -75,6 +75,23 @@
"messageId": null
}
},
{
"name": "error_v1_empty_schema_id_is_unreadable_and_names_the_write",
"messagePackHex": "86AD736368656D6156657273696F6EB063756C746E65742E6572726F722E7631A56572726F72D9216D6573736167655F696E76616C69643A20756E7265616461626C65206672616D65AB726F7574696E6748696E74C0A4636F6465AF6D6573736167655F696E76616C6964A764657461696C7384A56669656C64A76D657373616765A576616C7565C0A461734F66C0A763757272656E74C0A96D6573736167654964A57075742D31",
"message": {
"schemaVersion": "cultnet.error.v1",
"error": "message_invalid: unreadable frame",
"routingHint": null,
"code": "message_invalid",
"details": {
"field": "message",
"value": null,
"asOf": null,
"current": null
},
"messageId": "put-1"
}
},
{
"name": "error_v1_limit_reached",
"messagePackHex": "86AD736368656D6156657273696F6EB063756C746E65742E6572726F722E7631A56572726F72D93B6C696D69745F726561636865643A207468652073657373696F6E27732072656D656D62657265642D7772697465206C696D69742069732066756C6CAB726F7574696E6748696E74C0A4636F6465AD6C696D69745F72656163686564A764657461696C7384A56669656C64C0A576616C7565C0A461734F66C0A763757272656E74C0A96D6573736167654964A87075742D31303235",

View file

@ -36,7 +36,7 @@ A connection is receipt-capable when its `cultnet.hello.v0` (sent before its fir
- **What is remembered.** Per entry a server keeps only fixed-size digests (SHA-256) of the `messageId` and of the content hash, plus the answer (`admitted`, or the refusal's code and details). Memory per entry is bounded independent of id length and payload size. Lookup compares digests.
- **Remembered-write limits.** The per-session table holds at most 1024 entries. A server MAY also enforce a server-wide memory budget across all sessions (reference: 32 MiB of remembered entries). A NEW write (a key not already in the table) that would exceed either bound is refused `limit_reached` for that write alone, `messageId` the write's id, `details` all null; the write is not applied and the session continues. A resend of a remembered key is still answered from the table.
- **Limit refusals are remembered.** A `limit_reached` refusal is remembered like any other: a resend of the same id and content hash replays `limit_reached`. No slot is freed mid-session, so an id refused at a bound never becomes admitted later in that session (a server at the cap keeps the refused digest pair in a bounded refusal set of at most 1024 per session, outside the entry budget; past it, a further new write is refused `limit_reached` without being remembered, and the session refuses every id it has not seen for the rest of its life, even if a server-wide budget has since freed, so a refusal that was not remembered is never followed by an admission of the same id). The client mints a new id on a new session.
- **Required fields.** A put or delete whose document or `schemaId` is null or missing is refused `message_invalid` with `details.field` `"document"` or `"schemaId"` (the first missing in that order) and fixed text; the write is not applied.
- **Required fields.** A put or delete whose document or `schemaId` is null or missing is refused `message_invalid` with `details.field` `"document"` or `"schemaId"` (the first missing in that order) and fixed text; the write is not applied. A `schemaId` or `recordKey` that is empty or only whitespace (present but names nothing) is not "missing": the write is refused `message_invalid` exactly as a frame that does not read, with `details.field` `"message"` and the write's id (vector `error_v1_empty_schema_id_is_unreadable_and_names_the_write`), and is never applied; a server must not hand it to its store.
- **Fixed texts.** The `error` string of these refusals is fixed, so vectors can compare it: `message_invalid: messageId exceeds 256 bytes`, `message_invalid: document is required`, `message_invalid: schemaId is required`, `message_invalid: receipt sessions require the reliable channel`, `limit_reached: the session's remembered-write limit is full`.
- **Reliable channel only.** A write is accepted on a receipt session only on a channel that is reliable and not sequenced (reliable ordered and reliable unordered both qualify; a sequenced channel drops superseded packets, so a write on it can vanish without trace). A write arriving on any other channel is refused `message_invalid` with `details.field: "channel"` (the transport profile's word for it) and fixed text. Receipts and refusals travel the reliable channel; a write that could be lost without trace cannot be answered once.
- Past the session (or a server's longer retention, if it keeps one) a resend is a new write.

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

View file

@ -249,6 +249,9 @@ pub enum CultNetErrorCode {
WriteNotAuthorized,
UnsupportedSchemaVersion,
MessageInvalid,
/// `cultnet.error.v1` only: the session's remembered-write limit is full. `cultnet.error.v0` neither
/// encodes nor decodes it, so its enum stays the schema's.
LimitReached,
}
impl CultNetErrorCode {
@ -263,6 +266,7 @@ impl CultNetErrorCode {
Self::WriteNotAuthorized => "write_not_authorized",
Self::UnsupportedSchemaVersion => "unsupported_schema_version",
Self::MessageInvalid => "message_invalid",
Self::LimitReached => "limit_reached",
}
}
@ -277,6 +281,7 @@ impl CultNetErrorCode {
"write_not_authorized" => Ok(Self::WriteNotAuthorized),
"unsupported_schema_version" => Ok(Self::UnsupportedSchemaVersion),
"message_invalid" => Ok(Self::MessageInvalid),
"limit_reached" => Ok(Self::LimitReached),
_ => Err(anyhow!("ErrorMessage.Code is not a recognized cultnet.error.v0 code")),
}
}
@ -359,6 +364,26 @@ pub enum CultNetMessage {
#[serde(default, skip_serializing_if = "Option::is_none")]
details: Option<CultNetErrorDetails>,
},
/// `cultnet.error.v1` (contracts/cultnet/interop/write-receipt.md): `cultnet.error.v0` plus the id of the
/// request it answers. Sent only to a connection that declared the write-receipt capability.
#[serde(skip_deserializing)]
#[serde(rename = "cultnet.error.v1", rename_all = "camelCase")]
ErrorV1 {
error: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
code: Option<CultNetErrorCode>,
#[serde(default, skip_serializing_if = "Option::is_none")]
details: Option<CultNetErrorDetails>,
message_id: Option<String>,
},
/// `cultnet.write_admitted.v0`: the receipt for one admitted put or delete, sent after its commit point.
#[serde(rename = "cultnet.write_admitted.v0", rename_all = "camelCase")]
WriteAdmitted {
message_id: String,
operation: String,
schema_id: String,
record_key: String,
},
#[serde(rename = "cultnet.sample.change_name.v0", rename_all = "camelCase")]
SampleChangeName { name: String },
#[serde(rename = "cultnet.sample.chat.v0", rename_all = "camelCase")]
@ -543,7 +568,9 @@ pub fn encode_cultnet_message_for_wire(
| CultNetMessage::SnapshotResponseRawV1 { .. }
| CultNetMessage::ContentChunkRequest { .. }
| CultNetMessage::ContentChunkResponse { .. }
| CultNetMessage::Error { .. } => encode_raw_cultnet_schema_message(message),
| CultNetMessage::Error { .. }
| CultNetMessage::ErrorV1 { .. }
| CultNetMessage::WriteAdmitted { .. } => encode_raw_cultnet_schema_message(message),
_ => Ok(cultcache_rs::decode_payload(&rmp_serde::to_vec(
&serde_json::to_value(message)?,
)?)?),
@ -573,7 +600,8 @@ pub fn decode_cultnet_message_from_slice(
| "cultnet.snapshot_response_raw.v0"
| "cultmesh.content_chunk_request.v1"
| "cultmesh.content_chunk_response.v1"
| "cultnet.error.v0",
| "cultnet.error.v0"
| "cultnet.error.v1",
) => parse_raw_cultnet_schema_message(&wire_value)
.with_context(|| received_version_label(&wire_value))?,
_ => cultcache_rs::decode_payload::<CultNetMessage>(bytes)
@ -645,6 +673,20 @@ fn validate_message(message: &CultNetMessage) -> Result<()> {
require_non_empty(session, "session")?;
}
CultNetMessage::Error { error, code: _, details: _ } => require_non_empty(error, "error")?,
CultNetMessage::ErrorV1 { error, message_id, .. } => {
require_non_empty(error, "error")?;
if let Some(message_id) = message_id {
require_non_empty(message_id, "messageId")?;
}
}
CultNetMessage::WriteAdmitted { message_id, operation, schema_id, record_key } => {
require_non_empty(message_id, "messageId")?;
if operation != "put" && operation != "delete" {
return Err(anyhow!("operation must be put or delete"));
}
require_non_empty(schema_id, "schemaId")?;
require_non_empty(record_key, "recordKey")?;
}
CultNetMessage::SampleChangeName { name } => require_non_empty(name, "name")?,
CultNetMessage::SampleChat { text } => require_non_empty(text, "text")?,
CultNetMessage::DocumentDelete {
@ -1021,7 +1063,7 @@ fn parse_raw_cultnet_schema_message(input: &rmpv::Value) -> Result<CultNetMessag
.to_vec(),
error: require_legacy_string(get("error"), "error")?,
}),
"cultnet.error.v0" => {
"cultnet.error.v0" | "cultnet.error.v1" => {
let code = get("code")
.filter(|value| !value.is_nil())
.map(|value| {
@ -1031,12 +1073,23 @@ fn parse_raw_cultnet_schema_message(input: &rmpv::Value) -> Result<CultNetMessag
.and_then(CultNetErrorCode::from_wire_str)
})
.transpose()?;
if schema_version == "cultnet.error.v0" && code == Some(CultNetErrorCode::LimitReached) {
return Err(anyhow!("ErrorMessage.Code is not a recognized cultnet.error.v0 code"));
}
let details = require_error_details(get("details"), "ErrorMessage.Details")?;
Ok(CultNetMessage::Error {
error: require_legacy_string(get("error"), "ErrorMessage.Error")?,
code,
details,
})
let error = require_legacy_string(get("error"), "ErrorMessage.Error")?;
if schema_version == "cultnet.error.v1" {
return Ok(CultNetMessage::ErrorV1 {
error,
code,
details,
message_id: match get("messageId") {
None | Some(rmpv::Value::Nil) => None,
other => Some(require_legacy_string(other, "messageId")?),
},
});
}
Ok(CultNetMessage::Error { error, code, details })
}
_ => Err(anyhow!(
"Unsupported raw cultnet.schema.v0 message"
@ -1090,7 +1143,18 @@ fn require_legacy_optional_u64(
}
fn encode_raw_cultnet_schema_message(message: &CultNetMessage) -> Result<rmpv::Value> {
if let CultNetMessage::Error { code: Some(CultNetErrorCode::LimitReached), .. } = message {
return Err(anyhow!("limit_reached is a cultnet.error.v1 code; cultnet.error.v0 cannot carry it"));
}
Ok(rmpv::Value::Map(match message {
// Spec field order (write-receipt.md), not serde_json's sorted map.
CultNetMessage::WriteAdmitted { message_id, operation, schema_id, record_key } => vec![
(rmpv::Value::from("schemaVersion"), rmpv::Value::from("cultnet.write_admitted.v0")),
(rmpv::Value::from("messageId"), rmpv::Value::from(message_id.as_str())),
(rmpv::Value::from("operation"), rmpv::Value::from(operation.as_str())),
(rmpv::Value::from("schemaId"), rmpv::Value::from(schema_id.as_str())),
(rmpv::Value::from("recordKey"), rmpv::Value::from(record_key.as_str())),
],
CultNetMessage::DocumentPutRaw {
message_id,
document,
@ -1255,6 +1319,36 @@ fn encode_raw_cultnet_schema_message(message: &CultNetMessage) -> Result<rmpv::V
.unwrap_or(rmpv::Value::Nil),
),
],
CultNetMessage::ErrorV1 {
error,
code,
details,
message_id,
} => vec![
(rmpv::Value::from("schemaVersion"), rmpv::Value::from("cultnet.error.v1")),
(rmpv::Value::from("error"), rmpv::Value::from(error.as_str())),
(rmpv::Value::from("routingHint"), rmpv::Value::Nil),
(
rmpv::Value::from("code"),
code.as_ref()
.map(|code| rmpv::Value::from(code.as_wire_str()))
.unwrap_or(rmpv::Value::Nil),
),
(
rmpv::Value::from("details"),
details
.as_ref()
.map(encode_error_details)
.unwrap_or(rmpv::Value::Nil),
),
(
rmpv::Value::from("messageId"),
message_id
.as_deref()
.map(rmpv::Value::from)
.unwrap_or(rmpv::Value::Nil),
),
],
_ => {
return Err(anyhow!(
"Message is not a raw cultnet.schema.v0 binary replication message"
@ -1884,3 +1978,71 @@ fn legacy_schema_descriptor(schema: &CultNetSchemaDescriptor) -> Result<rmpv::Va
),
]))
}
/// The content hash a write-receipt server binds to a `(actor, messageId)` dedupe key
/// (contracts/cultnet/interop/write-receipt.md, rule 5): SHA-256 of the MessagePack array
/// `[operation, schemaId, recordKey, payload]`, `payload` a `bin` for a put and nil for a delete.
pub fn write_content_hash(
operation: &str,
schema_id: &str,
record_key: &str,
payload: Option<&[u8]>,
) -> Result<[u8; 32]> {
use sha2::{Digest, Sha256};
let value = rmpv::Value::Array(vec![
rmpv::Value::from(operation),
rmpv::Value::from(schema_id),
rmpv::Value::from(record_key),
payload.map_or(rmpv::Value::Nil, |payload| rmpv::Value::Binary(payload.to_vec())),
]);
let mut bytes = Vec::new();
rmpv::encode::write_value(&mut bytes, &value)?;
Ok(Sha256::digest(&bytes).into())
}
/// SHA-256 of a write's `messageId`: what a receipt server remembers in place of the id itself, so an
/// entry's memory does not depend on the id's length (write-receipt.md, "What is remembered").
pub fn message_id_digest(message_id: &str) -> [u8; 32] {
use sha2::{Digest, Sha256};
Sha256::digest(message_id.as_bytes()).into()
}
/// The `messageId` of a schema frame that does not decode as a message (write-receipt.md, rule 3: a refusal
/// carries the write's id whenever the frame is readable enough to find it). `None` when the bytes are not
/// a MessagePack map or hold no non-empty string `messageId`.
pub fn readable_message_id(bytes: &[u8]) -> Option<String> {
let value = rmpv::decode::read_value(&mut &*bytes).ok()?;
let rmpv::Value::Map(entries) = value else {
return None;
};
entries.into_iter().find_map(|(key, value)| match (key.as_str(), value) {
(Some("messageId"), rmpv::Value::String(id)) => id.into_str().filter(|id| !id.is_empty()),
_ => None,
})
}
/// The first required field (`document`, then `schemaId`) that a put or delete frame lacks or holds as nil
/// (write-receipt.md, "Required fields"), with the write's operation (`"put"` or `"delete"`). `None` when the
/// bytes are not such a frame, or both fields are present. A put's `schemaId` is its document's.
pub fn missing_required_write_field(bytes: &[u8]) -> Option<(&'static str, &'static str)> {
let rmpv::Value::Map(entries) = rmpv::decode::read_value(&mut &*bytes).ok()? else {
return None;
};
let field = |entries: &[(rmpv::Value, rmpv::Value)], name: &str| {
entries
.iter()
.find(|(key, _)| key.as_str() == Some(name))
.map(|(_, value)| value.clone())
.filter(|value| !value.is_nil())
};
match field(&entries, "schemaVersion")?.as_str()? {
"cultnet.document_put_raw.v0" => match field(&entries, "document") {
Some(rmpv::Value::Map(document)) => {
field(&document, "schemaId").is_none().then_some(("put", "schemaId"))
}
_ => Some(("put", "document")),
},
"cultnet.document_delete.v0" => field(&entries, "schemaId").is_none().then_some(("delete", "schemaId")),
_ => None,
}
}

View file

@ -468,6 +468,14 @@ impl CultNetRudpSession {
self.create_disconnect(RUDP_REFUSED_PACKET_REASON.to_vec())
}
/// Ends a session because the server itself could not answer a write it owes an answer to (its answer
/// could not be encoded, queued or sent). The peer did nothing wrong: this goodbye is not the
/// "session refused a packet" of a protocol refusal.
pub fn end_local_failure_session(&mut self) -> CultNetRudpPacket {
self.reset_peer_state();
self.create_disconnect(RUDP_LOCAL_FAILURE_REASON.to_vec())
}
/// Ends a session that owes its peer a packet that can never be sent as
/// built (see `is_permanent_send_error`). Resending it would fail forever,
/// and dropping it would leave a reliable sequence the peer waits on for
@ -2609,6 +2617,8 @@ where
/// Sent to a peer whose session refused one of its packets, and reported to
/// the local caller as the disconnect reason.
const RUDP_REFUSED_PACKET_REASON: &[u8] = b"session refused a packet";
/// Goodbye reason of a session whose server could not answer a write (`end_local_failure_session`).
const RUDP_LOCAL_FAILURE_REASON: &[u8] = b"server could not answer a write";
const SESSION_TIMED_OUT_REASON: &[u8] = b"session timed out";
/// Prefix of the goodbye reason for a session that owed a packet which can
/// never be sent; the failure's fixed name (`send_error_code`) follows it.

View file

@ -0,0 +1,155 @@
//! Wire parity with contracts/cultnet/interop/write-receipt-vectors.json: the content hash, and the two new frames.
use cultnet_rs::{
CultNetErrorCode, CultNetErrorDetails, CultNetMessage, CultNetWireContract, decode_cultnet_message_from_slice,
encode_cultnet_message_to_vec, write_content_hash,
};
fn hex(bytes: &[u8]) -> String {
bytes.iter().map(|byte| format!("{byte:02x}")).collect()
}
#[test]
fn the_content_hash_matches_the_spec_vectors() {
let a = [0x81, 0xA2, 0x69, 0x64, 0xA1, 0x31];
let b = [0x81, 0xA2, 0x69, 0x64, 0xA1, 0x32];
let cases = [
(write_content_hash("put", "note.v0", "note:1", Some(&a)), "7051c10071123b6ade50fbd5075e3859cae4ea70ce358a24a0b0c6980090e95d"),
(write_content_hash("put", "note.v0", "note:1", Some(&b)), "58d3fc6b858d2605a68550c9d4811ccdca6b9db7a0c3bfebf304d279257f6f9f"),
(write_content_hash("put", "note.v0", "note:2", Some(&a)), "a248cd920e58d41bc6ccca9e108030b1d491239000047fef088122364a32b6f8"),
(write_content_hash("delete", "note.v0", "note:1", None), "232b0f96651b28aa8ab7673a6106944a9f6a8d66852b6e21291a23ed084b8b37"),
];
for (hash, expected) in cases {
assert_eq!(hex(&hash.unwrap()), expected);
}
}
#[test]
fn the_new_frames_decode_back_equal() {
let admitted = CultNetMessage::WriteAdmitted {
message_id: "put-1".into(),
operation: "put".into(),
schema_id: "note.v0".into(),
record_key: "note:1".into(),
};
let bytes = encode_cultnet_message_to_vec(&admitted, CultNetWireContract::CultNetSchemaV0).unwrap();
assert_eq!(
decode_cultnet_message_from_slice(&bytes, CultNetWireContract::CultNetSchemaV0).unwrap(),
admitted
);
// error.v1 with a nil id round-trips as nil, and the code enum is v0's.
let error = CultNetMessage::ErrorV1 {
error: "message_invalid: unreadable frame".into(),
code: Some(CultNetErrorCode::MessageInvalid),
details: None,
message_id: None,
};
let bytes = encode_cultnet_message_to_vec(&error, CultNetWireContract::CultNetSchemaV0).unwrap();
assert_eq!(
decode_cultnet_message_from_slice(&bytes, CultNetWireContract::CultNetSchemaV0).unwrap(),
error
);
}
fn unhex(text: &str) -> Vec<u8> {
(0..text.len()).step_by(2).map(|i| u8::from_str_radix(&text[i..i + 2], 16).unwrap()).collect()
}
/// The spec's wire vectors (write-receipt-vectors.json `wire`): the bytes decode to the vector's fields, not
/// merely to something that encodes back to itself.
#[test]
fn the_spec_wire_vectors_decode_to_their_expected_fields() {
let admitted = decode_cultnet_message_from_slice(
&unhex("85AD736368656D6156657273696F6EB963756C746E65742E77726974655F61646D69747465642E7630A96D6573736167654964A57075742D31A96F7065726174696F6EA3707574A8736368656D614964A76E6F74652E7630A97265636F72644B6579A66E6F74653A31"),
CultNetWireContract::CultNetSchemaV0,
)
.unwrap();
assert_eq!(
admitted,
CultNetMessage::WriteAdmitted {
message_id: "put-1".into(),
operation: "put".into(),
schema_id: "note.v0".into(),
record_key: "note:1".into(),
}
);
let refused = decode_cultnet_message_from_slice(
&unhex("86AD736368656D6156657273696F6EB063756C746E65742E6572726F722E7631A56572726F72D93677726974655F6E6F745F617574686F72697A65643A2074686520636174616C6F6720726566757365642074686520646F63756D656E74AB726F7574696E6748696E74C0A4636F6465B477726974655F6E6F745F617574686F72697A6564A764657461696C7384A56669656C64A97265636F72644B6579A576616C7565A66E6F74653A32A461734F66C0A763757272656E74C0A96D6573736167654964A57075742D32"),
CultNetWireContract::CultNetSchemaV0,
)
.unwrap();
let CultNetMessage::ErrorV1 { error, code, details, message_id } = refused else {
panic!("expected error.v1");
};
assert_eq!(error, "write_not_authorized: the catalog refused the document");
assert_eq!(code, Some(CultNetErrorCode::WriteNotAuthorized));
assert_eq!(message_id.as_deref(), Some("put-2"));
let details = details.unwrap();
assert_eq!(details.field.as_deref(), Some("recordKey"));
assert_eq!(details.value.as_deref(), Some("note:2"));
assert_eq!((details.as_of, details.current), (None, None));
}
/// Spec vector `error_v1_limit_reached`: the bytes the server must send, and what they decode to.
#[test]
fn error_v1_limit_reached_is_the_spec_vector() {
let vector = "86AD736368656D6156657273696F6EB063756C746E65742E6572726F722E7631A56572726F72D93B6C696D69745F726561636865643A207468652073657373696F6E27732072656D656D62657265642D7772697465206C696D69742069732066756C6CAB726F7574696E6748696E74C0A4636F6465AD6C696D69745F72656163686564A764657461696C7384A56669656C64C0A576616C7565C0A461734F66C0A763757272656E74C0A96D6573736167654964A87075742D31303235";
let message = CultNetMessage::ErrorV1 {
error: "limit_reached: the session's remembered-write limit is full".into(),
code: Some(CultNetErrorCode::LimitReached),
details: Some(CultNetErrorDetails::default()),
message_id: Some("put-1025".into()),
};
let bytes = encode_cultnet_message_to_vec(&message, CultNetWireContract::CultNetSchemaV0).unwrap();
assert_eq!(bytes, unhex(vector), "encoded bytes must equal the spec vector");
let CultNetMessage::ErrorV1 { error, code, details, message_id } =
decode_cultnet_message_from_slice(&unhex(vector), CultNetWireContract::CultNetSchemaV0).unwrap()
else {
panic!("expected error.v1");
};
assert_eq!(error, "limit_reached: the session's remembered-write limit is full");
assert_eq!(code, Some(CultNetErrorCode::LimitReached));
assert_eq!(message_id.as_deref(), Some("put-1025"));
assert_eq!(details, Some(CultNetErrorDetails::default()));
}
/// Spec vectors `write_admitted_put` and `write_admitted_delete`: the encoder emits the spec's field order.
#[test]
fn write_admitted_encodes_to_the_spec_vector_bytes() {
let cases = [
("put-1", "put", "85AD736368656D6156657273696F6EB963756C746E65742E77726974655F61646D69747465642E7630A96D6573736167654964A57075742D31A96F7065726174696F6EA3707574A8736368656D614964A76E6F74652E7630A97265636F72644B6579A66E6F74653A31"),
("del-1", "delete", "85AD736368656D6156657273696F6EB963756C746E65742E77726974655F61646D69747465642E7630A96D6573736167654964A564656C2D31A96F7065726174696F6EA664656C657465A8736368656D614964A76E6F74652E7630A97265636F72644B6579A66E6F74653A31"),
];
for (id, operation, vector) in cases {
let message = CultNetMessage::WriteAdmitted {
message_id: id.into(),
operation: operation.into(),
schema_id: "note.v0".into(),
record_key: "note:1".into(),
};
let bytes = encode_cultnet_message_to_vec(&message, CultNetWireContract::CultNetSchemaV0).unwrap();
assert_eq!(bytes, unhex(vector));
}
}
/// `limit_reached` is in the error.v1 enum only: a v0 error neither encodes nor decodes it.
#[test]
fn error_v0_refuses_the_v1_only_limit_reached_code() {
let v0 = CultNetMessage::Error {
error: "x".into(),
code: Some(CultNetErrorCode::LimitReached),
details: None,
};
assert!(encode_cultnet_message_to_vec(&v0, CultNetWireContract::CultNetSchemaV0).is_err());
let v1 = CultNetMessage::ErrorV1 {
error: "x".into(),
code: Some(CultNetErrorCode::LimitReached),
details: None,
message_id: None,
};
let v1_bytes = encode_cultnet_message_to_vec(&v1, CultNetWireContract::CultNetSchemaV0).unwrap();
let at = v1_bytes.windows(16).position(|w| w == b"cultnet.error.v1").unwrap();
let mut as_v0 = v1_bytes.clone();
as_v0[at + 15] = b'0';
assert!(decode_cultnet_message_from_slice(&as_v0, CultNetWireContract::CultNetSchemaV0).is_err());
}

View file

@ -1017,6 +1017,8 @@ namespace GameCult.Networking
/// <summary>
/// Applies a raw document mutation after checking shard authority. This is the host's own door (and replication's):
/// it asks no remote write authority. A peer's put goes through <see cref="CultNetDatabaseServer"/>, which does.
/// Returning normally means the put is committed and visible to readers: a receipt-capable peer is told
/// <c>write_admitted</c> on the strength of it, so a write that has only been queued must not return.
/// </summary>
/// <exception cref="CultNetUnownedSchemaException">No shard owns the schema and key; nothing was written.</exception>
/// <exception cref="CultNetShardLogException">The change committed and was published, but the shard log could not record it.</exception>
@ -1047,6 +1049,7 @@ namespace GameCult.Networking
/// <summary>
/// Applies a raw document delete after checking shard authority. This is the host's own door: it asks no remote
/// write authority. A peer's delete goes through <see cref="CultNetDatabaseServer"/>, which does.
/// Returning normally means the delete is committed and visible to readers (see <see cref="ApplyPutAsync(CultNetDocumentPutRawMessage)"/>).
/// </summary>
/// <exception cref="CultNetUnownedSchemaException">No shard owns the schema and key; nothing was written.</exception>
/// <exception cref="CultNetShardLogException">The change committed and was published, but the shard log could not record it.</exception>

View file

@ -22,6 +22,7 @@ namespace GameCult.Networking
private readonly Func<CultNetSnapshotRequestV1Message, CultNetServerPeer, Task> _snapshotHandlerV1;
private readonly Func<CultNetDocumentPutRawMessage, CultNetServerPeer, Task> _putHandler;
private readonly Func<CultNetDocumentDeleteMessage, CultNetServerPeer, Task> _deleteHandler;
private readonly Func<CultNetHelloMessage, CultNetServerPeer, Task> _helloHandler;
private readonly Func<CultNetShardCatalogRequestMessage, CultNetServerPeer, Task> _shardCatalogHandler;
private readonly Func<CultNetShardLogRequestMessage, CultNetServerPeer, Task> _shardLogHandler;
private readonly Func<CultNetDatabaseSubscribeMessage, CultNetServerPeer, Task> _subscribeHandler;
@ -45,6 +46,7 @@ namespace GameCult.Networking
_snapshotHandlerV1 = HandleSnapshotRequestV1Async;
_putHandler = HandlePutAsync;
_deleteHandler = HandleDeleteAsync;
_helloHandler = HandleHelloAsync;
_shardCatalogHandler = HandleShardCatalogRequestAsync;
_shardLogHandler = HandleShardLogRequestAsync;
_subscribeHandler = HandleSubscribeAsync;
@ -55,6 +57,7 @@ namespace GameCult.Networking
_server.OnCultNet(_snapshotHandlerV1);
_server.OnCultNet(_putHandler);
_server.OnCultNet(_deleteHandler);
_server.OnCultNet(_helloHandler);
_server.OnCultNet(_shardCatalogHandler);
_server.OnCultNet(_shardLogHandler);
_server.OnCultNet(_subscribeHandler);
@ -217,6 +220,7 @@ namespace GameCult.Networking
_server.RemoveCultNetMessageListener<CultNetSnapshotRequestV1Message>(_snapshotHandlerV1);
_server.RemoveCultNetMessageListener<CultNetDocumentPutRawMessage>(_putHandler);
_server.RemoveCultNetMessageListener<CultNetDocumentDeleteMessage>(_deleteHandler);
_server.RemoveCultNetMessageListener<CultNetHelloMessage>(_helloHandler);
_server.RemoveCultNetMessageListener<CultNetShardCatalogRequestMessage>(_shardCatalogHandler);
_server.RemoveCultNetMessageListener<CultNetShardLogRequestMessage>(_shardLogHandler);
_server.RemoveCultNetMessageListener<CultNetDatabaseSubscribeMessage>(_subscribeHandler);
@ -262,7 +266,7 @@ namespace GameCult.Networking
catch (Exception ex)
{
_server.Logger.LogError($"CultNet snapshot v1 request failed: {ex.Message}");
peer.SendCultNet(new CultNetErrorMessage { Error = ex.Message });
peer.SendCultNet(new CultNetErrorMessage { Error = "the snapshot request could not be answered" });
}
return Task.CompletedTask;
@ -282,7 +286,8 @@ namespace GameCult.Networking
}
catch (Exception ex)
{
peer.SendCultNet(new CultNetErrorMessage { Error = ex.Message });
_server.Logger.LogError($"CultNet shard log request failed: {ex.Message}");
peer.SendCultNet(new CultNetErrorMessage { Error = "the shard log request could not be answered" });
}
return Task.CompletedTask;
@ -323,74 +328,256 @@ namespace GameCult.Networking
};
}
private async Task HandlePutAsync(CultNetDocumentPutRawMessage message, CultNetServerPeer peer)
// A hello decides the connection's write framing if it is the first thing to (write-receipt.md, "Capability"). The
// server answers a receipt-capable hello with its own, so the client can tell a receipt server from a legacy one.
private Task HandleHelloAsync(CultNetHelloMessage hello, CultNetServerPeer peer)
{
if (peer.Transport != null && peer.Transport.SettleWriteFraming(hello))
{
Answer(peer, new CultNetHelloMessage
{
RuntimeId = "gamecult-networking-database-server",
RuntimeKind = "dotnet",
SupportedMessageVersions = [CultNetSchemaVersions.WriteAdmitted, CultNetSchemaVersions.ErrorV1]
});
}
return Task.CompletedTask;
}
// The write's receipt state, or null when its connection did not opt in (it is then answered exactly as before).
private CultNetReceiptedWrite? BeginReceipted(
CultNetServerPeer peer,
string operation,
string? messageId,
string? schemaId,
string? recordKey,
byte[]? payload)
{
var transport = peer.Transport;
if (transport == null) return null;
transport.SettleWriteFraming(null);
var session = transport.WriteReceipts;
return session == null
? null
: new CultNetReceiptedWrite(session, message => Answer(peer, message), operation, messageId, schemaId, recordKey, payload);
}
// Every frame a write can be answered with. Mock point for tests: the answer leaves through the peer's transport. An
// answer that cannot be queued ends the connection: the peer settles its write as session_ended, never as silence.
private void Answer(CultNetServerPeer peer, ICultNetSchemaMessage message)
{
try
{
await _database.ApplyPutAsync(message, Authorize(peer, CultNetRemoteWriteOperation.Put, message.ShardId, message.ShardEpoch))
.ConfigureAwait(false);
Send(peer, message);
}
catch (Exception ex)
{
_server.Logger.LogWarning($"CultNet write answer could not be sent; ending the connection: {ex.Message}");
peer.Transport?.End();
}
}
private static void Send(CultNetServerPeer peer, ICultNetSchemaMessage message)
{
switch (message)
{
case CultNetWriteAdmittedMessage admitted: peer.SendCultNet(admitted); break;
case CultNetErrorV1Message error: peer.SendCultNet(error); break;
case CultNetErrorMessage error: peer.SendCultNet(error); break;
case CultNetHelloMessage hello: peer.SendCultNet(hello); break;
default: throw new ArgumentException($"{message.GetType().Name} is not a write answer.", nameof(message));
}
}
// A refusal: error.v1 carrying the write's id for a receipt-capable connection, the v0 error for any other.
// An empty or blank schemaId or recordKey names no record: refused as a frame that does not read, exactly as the Rust
// server's decoder refuses it (write-receipt.md, "Required fields"), never handed to the store.
private static bool IsUnreadableWriteTarget(string? schemaId, string? recordKey) =>
string.IsNullOrWhiteSpace(schemaId) || string.IsNullOrWhiteSpace(recordKey);
private static CultNetErrorMessage UnreadableWriteTarget() =>
CultNetErrorMessage.ForMessageInvalid("message", "the message could not be read");
private static void Refuse(CultNetServerPeer peer, CultNetReceiptedWrite? write, CultNetErrorMessage error)
{
if (write != null) write.Refused(error);
else peer.SendCultNet(error);
}
// Rule 6: a forward is answered with what the primary answered, under the original id. A refusal the primary sent is
// relayed; silence or any other failure relays nothing, so the client settles it unconfirmed.
private void RelayForwardFailure(CultNetReceiptedWrite write, Exception failure)
{
if (failure is CultNetWriteRefusedException refused)
{
write.Refused(refused.Refusal);
return;
}
_server.Logger.LogWarning($"CultNet forwarded write was not answered by its primary: {failure.Message}");
// Only a forward that provably never left (not sent) may be forwarded again. Every other failure, silence or not, is
// unconfirmed: the primary may have applied it, so the id is never forwarded again on this session.
if (failure is CultNetWriteNotSentException) write.NotSent();
else write.LeftSilent();
}
private async Task HandlePutAsync(CultNetDocumentPutRawMessage message, CultNetServerPeer peer)
{
var write = BeginReceipted(peer, "put", message.MessageId, message.Document?.SchemaId, message.Document?.RecordKey, message.Document?.Payload);
var idRefusal = write?.IdRefusal;
if (idRefusal != null)
{
write!.Refused(idRefusal);
return;
}
var missing = message.Document == null ? "document" : message.Document.SchemaId == null ? "schemaId" : null;
if (missing != null)
{
Refuse(peer, write, CultNetErrorMessage.ForMessageInvalid(missing, $"{missing} is required"));
return;
}
if (IsUnreadableWriteTarget(message.Document!.SchemaId, message.Document.RecordKey))
{
Refuse(peer, write, UnreadableWriteTarget());
return;
}
if (write != null && !write.Begin()) return;
var authorize = Authorize(peer, CultNetRemoteWriteOperation.Put, message.ShardId, message.ShardEpoch);
try
{
await _database.ApplyPutAsync(message, authorize).ConfigureAwait(false);
write?.Admitted();
}
catch (CultNetRemoteWriteRefusedException ex)
{
_server.Logger.LogWarning($"CultNet raw put refused: {ex.Message}");
peer.SendCultNet(CultNetErrorMessage.ForRemoteWriteRefused(ex.Request, ex.Reason));
Refuse(peer, write, CultNetErrorMessage.ForRemoteWriteRefused(ex.Request, ex.Reason));
}
catch (CultNetShardAuthorityException ex)
{
if (!await TryForwardPutAsync(ex, message).ConfigureAwait(false))
bool forwarded;
try
{
forwarded = await TryForwardPutAsync(ex, message).ConfigureAwait(false);
}
catch (CultNetWriteUnconfirmedException) when (write == null)
{
// A peer that did not opt in is told nothing of a forward its primary left unanswered.
return;
}
catch (Exception failure) when (write != null)
{
RelayForwardFailure(write, failure);
return;
}
if (forwarded)
{
write?.Admitted();
}
else
{
_server.Logger.LogError($"CultNet raw put authority failed: {ex.Message}");
peer.SendCultNet(CreateRoutingError(ex));
Refuse(peer, write, CreateRoutingError(ex));
}
}
catch (CultNetUnownedSchemaException ex)
{
_server.Logger.LogWarning($"CultNet raw put refused: {ex.Message}");
peer.SendCultNet(CultNetErrorMessage.ForUnownedSchema(ex));
Refuse(peer, write, CultNetErrorMessage.ForUnownedSchema(ex));
}
catch (Exception ex) when (CommittedWithLogFailure(ex))
{
// The write committed and was published; only its shard log failed, so the peer is not told it was refused.
_server.Logger.LogWarning($"CultNet raw put committed but its shard log failed: {ex.Message}");
write?.Admitted();
}
catch (Exception ex)
{
_server.Logger.LogError($"CultNet raw put failed: {ex.Message}");
peer.SendCultNet(new CultNetErrorMessage { Error = ex.Message });
Refuse(peer, write, new CultNetErrorMessage { Error = CultNetWriteReceiptSession.WriteFailedText });
}
}
private async Task HandleDeleteAsync(CultNetDocumentDeleteMessage message, CultNetServerPeer peer)
{
var write = BeginReceipted(peer, "delete", message.MessageId, message.SchemaId, message.RecordKey, null);
var idRefusal = write?.IdRefusal;
if (idRefusal != null)
{
write!.Refused(idRefusal);
return;
}
if (message.SchemaId == null)
{
Refuse(peer, write, CultNetErrorMessage.ForMessageInvalid("schemaId", "schemaId is required"));
return;
}
if (IsUnreadableWriteTarget(message.SchemaId, message.RecordKey))
{
Refuse(peer, write, UnreadableWriteTarget());
return;
}
if (write != null && !write.Begin()) return;
var authorize = Authorize(peer, CultNetRemoteWriteOperation.Delete, message.ShardId, message.ShardEpoch);
try
{
await _database.ApplyDeleteAsync(message, Authorize(peer, CultNetRemoteWriteOperation.Delete, message.ShardId, message.ShardEpoch))
.ConfigureAwait(false);
await _database.ApplyDeleteAsync(message, authorize).ConfigureAwait(false);
write?.Admitted();
}
catch (CultNetRemoteWriteRefusedException ex)
{
_server.Logger.LogWarning($"CultNet raw delete refused: {ex.Message}");
peer.SendCultNet(CultNetErrorMessage.ForRemoteWriteRefused(ex.Request, ex.Reason));
Refuse(peer, write, CultNetErrorMessage.ForRemoteWriteRefused(ex.Request, ex.Reason));
}
catch (CultNetShardAuthorityException ex)
{
if (!await TryForwardDeleteAsync(ex, message).ConfigureAwait(false))
bool forwarded;
try
{
peer.SendCultNet(CreateRoutingError(ex));
forwarded = await TryForwardDeleteAsync(ex, message).ConfigureAwait(false);
}
catch (CultNetWriteUnconfirmedException) when (write == null)
{
return;
}
catch (Exception failure) when (write != null)
{
RelayForwardFailure(write, failure);
return;
}
if (forwarded)
{
write?.Admitted();
}
else
{
Refuse(peer, write, CreateRoutingError(ex));
}
}
catch (CultNetUnownedSchemaException ex)
{
_server.Logger.LogWarning($"CultNet raw delete refused: {ex.Message}");
peer.SendCultNet(CultNetErrorMessage.ForUnownedSchema(ex));
Refuse(peer, write, CultNetErrorMessage.ForUnownedSchema(ex));
}
catch (Exception ex) when (CommittedWithLogFailure(ex))
{
_server.Logger.LogWarning($"CultNet raw delete committed but its shard log failed: {ex.Message}");
write?.Admitted();
}
catch (Exception ex)
{
peer.SendCultNet(new CultNetErrorMessage { Error = ex.Message });
Refuse(peer, write, new CultNetErrorMessage { Error = CultNetWriteReceiptSession.WriteFailedText });
}
}
@ -588,7 +775,7 @@ namespace GameCult.Networking
var refusedKey = change.GetType().GetProperty("Key")?.GetValue(change) is CultRecordKey changed ? changed.Value : string.Empty;
lock (refused)
refused.Add(refusedKey);
sendError(new CultNetErrorMessage { Error = refusal.Message });
sendError(new CultNetErrorMessage { Error = "the record is a variant CultNet cannot carry" });
return null;
}
@ -666,11 +853,14 @@ namespace GameCult.Networking
exception is CultNetShardLogException ||
exception is AggregateException aggregate && aggregate.Flatten().InnerExceptions.Any(inner => inner is CultNetShardLogException);
// Fixed text: the routing refusal never carries an exception's own message (the shard and reason travel in the hint).
internal const string RoutingRefusalText = "the write belongs to another shard primary";
private static CultNetErrorMessage CreateRoutingError(CultNetShardAuthorityException exception)
{
return new CultNetErrorMessage
{
Error = exception.Message,
Error = RoutingRefusalText,
RoutingHint = new CultNetShardRoutingHint
{
Reason = exception.Reason,

View file

@ -27,7 +27,8 @@ namespace GameCult.Networking
/// <summary>
/// Sends one schema-v0 message through the transport. Fire-and-forget: an ack or a return is not admission;
/// read the result frame (<see cref="CultNetWriteCompletion"/>).
/// read the result frame (<see cref="CultNetWriteCompletion"/>). Throws <see cref="CultNetWriteNotSentException"/> only
/// when it can prove no byte was written; any other exception leaves the send ambiguous and a forwarded write unconfirmed.
/// </summary>
void SendCultNet<T>(T message)
where T : ICultNetSchemaMessage;

View file

@ -53,6 +53,8 @@ namespace GameCult.Networking
CultNetSchemaVersions.Verify => Deserialize<CultNetVerifyMessage>(payload),
CultNetSchemaVersions.LoginSuccess => Deserialize<CultNetLoginSuccessMessage>(payload),
CultNetSchemaVersions.Error => Deserialize<CultNetErrorMessage>(payload),
CultNetSchemaVersions.ErrorV1 => Deserialize<CultNetErrorV1Message>(payload),
CultNetSchemaVersions.WriteAdmitted => Deserialize<CultNetWriteAdmittedMessage>(payload),
CultNetSchemaVersions.SampleChangeName => Deserialize<CultNetSampleChangeNameMessage>(payload),
CultNetSchemaVersions.SampleChat => Deserialize<CultNetSampleChatMessage>(payload),
CultNetSchemaVersions.OperationRequest => Deserialize<CultNetOperationRequestMessage>(payload),

View file

@ -48,6 +48,14 @@ namespace GameCult.Networking
/// </summary>
public const string Error = "cultnet.error.v0";
/// <summary>
/// error v1 contract identifier: error v0 plus the id of the request it answers (write-receipt.md).
/// </summary>
public const string ErrorV1 = "cultnet.error.v1";
/// <summary>
/// write admitted receipt contract identifier (write-receipt.md).
/// </summary>
public const string WriteAdmitted = "cultnet.write_admitted.v0";
/// <summary>
/// sample change name contract identifier.
/// </summary>
public const string SampleChangeName = "cultnet.sample.change_name.v0";
@ -538,7 +546,7 @@ namespace GameCult.Networking
public static CultNetErrorMessage ForUnownedSchema(CultNetUnownedSchemaException ex) =>
new CultNetErrorMessage
{
Error = $"unowned_schema: {ex.Message}",
Error = "unowned_schema: no shard owns the schema and key",
Code = "unowned_schema",
Details = new CultNetErrorDetails { Field = "schemaId", Value = ex.SchemaId }
};
@ -621,6 +629,80 @@ namespace GameCult.Networking
};
}
/// <summary>
/// <c>cultnet.error.v1</c> (contracts/cultnet/interop/write-receipt.md): the v0 error plus <see cref="MessageId"/>,
/// the id of the request it answers. Sent only to a connection that declared the write-receipt capability.
/// </summary>
[MessagePackObject]
public class CultNetErrorV1Message : ICultNetSchemaMessage
{
/// <summary>Gets or sets the schema version.</summary>
[Key("schemaVersion")] public string SchemaVersion { get; set; } = CultNetSchemaVersions.ErrorV1;
/// <summary>Gets or sets the error.</summary>
[Key("error")] public string Error { get; set; } = string.Empty;
/// <summary>Gets or sets optional shard routing details for authority failures.</summary>
[Key("routingHint")] public CultNetShardRoutingHint? RoutingHint { get; set; }
/// <summary>Gets or sets the machine-readable refusal code (the v0 enum).</summary>
[Key("code")] public string? Code { get; set; }
/// <summary>Gets or sets the code-specific structured detail.</summary>
[Key("details")] public CultNetErrorDetails? Details { get; set; }
/// <summary>Gets or sets the id of the request this answers; null only when no id could be read.</summary>
[Key("messageId")] public string? MessageId { get; set; }
/// <summary>Builds the v1 form of <paramref name="error"/> answering <paramref name="messageId"/>.</summary>
public static CultNetErrorV1Message From(CultNetErrorMessage error, string? messageId)
{
if (error == null) throw new ArgumentNullException(nameof(error));
return new CultNetErrorV1Message
{
Error = error.Error,
RoutingHint = error.RoutingHint,
Code = error.Code,
Details = error.Details,
MessageId = string.IsNullOrEmpty(messageId) ? null : messageId
};
}
/// <summary>
/// The refusal of a write a session has no room to remember (write-receipt.md, rule 5): code <c>limit_reached</c>,
/// details present and all nil. Exists only as error.v1; v0 has no such code.
/// </summary>
public static CultNetErrorV1Message LimitReached(string? messageId) => new CultNetErrorV1Message
{
Error = "limit_reached: the session's remembered-write limit is full",
Code = "limit_reached",
Details = new CultNetErrorDetails(),
MessageId = string.IsNullOrEmpty(messageId) ? null : messageId
};
/// <summary>Gets the v0 form, dropping the id.</summary>
public CultNetErrorMessage ToV0() => new CultNetErrorMessage
{
Error = Error,
RoutingHint = RoutingHint,
Code = Code,
Details = Details
};
}
/// <summary>
/// <c>cultnet.write_admitted.v0</c>: the receipt for one admitted put or delete, sent after its commit point.
/// </summary>
[MessagePackObject]
public class CultNetWriteAdmittedMessage : ICultNetSchemaMessage
{
/// <summary>Gets or sets the schema version.</summary>
[Key("schemaVersion")] public string SchemaVersion { get; set; } = CultNetSchemaVersions.WriteAdmitted;
/// <summary>Gets or sets the messageId of the admitted write, echoed.</summary>
[Key("messageId")] public string MessageId { get; set; } = string.Empty;
/// <summary>Gets or sets <c>put</c> or <c>delete</c>.</summary>
[Key("operation")] public string Operation { get; set; } = string.Empty;
/// <summary>Gets or sets the schemaId of the admitted write, echoed.</summary>
[Key("schemaId")] public string SchemaId { get; set; } = string.Empty;
/// <summary>Gets or sets the recordKey of the admitted write, echoed.</summary>
[Key("recordKey")] public string RecordKey { get; set; } = string.Empty;
}
/// <summary>
/// Code-specific structured detail carried on <see cref="CultNetErrorMessage"/> (R-N). Fields are
/// populated only for the code that defines them; every field is otherwise omitted (nil on the wire).

View file

@ -10,12 +10,16 @@ namespace GameCult.Networking
public interface ICultNetShardWriteForwarder
{
/// <summary>
/// Forwards a raw document put to the shard owner.
/// Forwards a raw document put to the shard owner. Completes only when the owner answered that it admitted the write;
/// throws <see cref="CultNetWriteRefusedException"/> when it refused, <see cref="CultNetWriteNotSentException"/> when
/// nothing could have reached it (the only failure after which the id may be forwarded again), and any other exception
/// when the frame may have gone out and was not answered (unconfirmed, including <see cref="CultNetWriteUnconfirmedException"/>). The server relays to the original peer exactly what the owner answered.
/// </summary>
Task ForwardPutAsync(CultNetShardDescriptor shard, CultNetDocumentPutRawMessage message);
/// <summary>
/// Forwards a raw document delete to the shard owner.
/// Forwards a raw document delete to the shard owner; completes, refuses and goes unconfirmed as
/// <see cref="ForwardPutAsync"/> does.
/// </summary>
Task ForwardDeleteAsync(CultNetShardDescriptor shard, CultNetDocumentDeleteMessage message);
}
@ -70,10 +74,12 @@ namespace GameCult.Networking
public Func<ICultNetSchemaClient>? CreateClient { get; set; }
/// <summary>
/// Gets or sets how long after a forwarded write the primary's refusal may still arrive; a refusal fails the
/// forward with <see cref="CultNetWriteRefusedException"/>. Defaults to <see cref="CultNetWriteCompletion.DefaultResultWindow"/>.
/// Gets or sets how long the primary has to answer a forwarded write (from the send). Its receipt completes the forward,
/// its refusal fails it with <see cref="CultNetWriteRefusedException"/>, and silence fails it with
/// <see cref="CultNetWriteUnconfirmedException"/>. Defaults to <see cref="CultNetWriteCompletion.ReceiptResultWindow"/>; a
/// primary that predates write receipts never answers, so forwarding to one is unconfirmed after this window.
/// </summary>
public TimeSpan ResultWindow { get; set; } = CultNetWriteCompletion.DefaultResultWindow;
public TimeSpan ResultWindow { get; set; } = CultNetWriteCompletion.ReceiptResultWindow;
}
/// <summary>
@ -92,29 +98,29 @@ namespace GameCult.Networking
}
/// <summary>
/// Forwards a put and completes on the primary's result frame: a refusal throws
/// <see cref="CultNetWriteRefusedException"/>, the same failure path as an unreachable primary.
/// Forwards a put under its original message id and completes on the primary's result frame: a receipt returns, a refusal
/// throws <see cref="CultNetWriteRefusedException"/> and silence throws <see cref="CultNetWriteUnconfirmedException"/>.
/// </summary>
public async Task ForwardPutAsync(CultNetShardDescriptor shard, CultNetDocumentPutRawMessage message)
{
if (message == null) throw new ArgumentNullException(nameof(message));
var endpoint = ResolvePrimaryEndpoint(shard);
if (shard == null) throw new ArgumentNullException(nameof(shard));
message.ShardId ??= shard.ShardId;
message.ShardEpoch ??= shard.Epoch;
await SendAsync(endpoint, message).ConfigureAwait(false);
await SendAsync(shard, message, message.MessageId).ConfigureAwait(false);
}
/// <summary>
/// Forwards a delete and completes on the primary's result frame: a refusal throws
/// <see cref="CultNetWriteRefusedException"/>.
/// Forwards a delete under its original message id; completes, refuses and goes unconfirmed as
/// <see cref="ForwardPutAsync"/> does.
/// </summary>
public async Task ForwardDeleteAsync(CultNetShardDescriptor shard, CultNetDocumentDeleteMessage message)
{
if (message == null) throw new ArgumentNullException(nameof(message));
var endpoint = ResolvePrimaryEndpoint(shard);
if (shard == null) throw new ArgumentNullException(nameof(shard));
message.ShardId ??= shard.ShardId;
message.ShardEpoch ??= shard.Epoch;
await SendAsync(endpoint, message).ConfigureAwait(false);
await SendAsync(shard, message, message.MessageId).ConfigureAwait(false);
}
/// <summary>
@ -138,19 +144,50 @@ namespace GameCult.Networking
return (uri.Host, uri.IsDefaultPort ? 3075 : uri.Port);
}
private async Task SendAsync<T>(string endpoint, T message) where T : ICultNetSchemaMessage
// The write frame has not been handed to the transport until this returns: any failure here is CultNetWriteNotSentException,
// the one failure the caller may retry under the same id.
private async Task<ICultNetSchemaClient> ConnectBeforeSendAsync(CultNetShardDescriptor shard, string messageId)
{
var (host, port) = ParseEndpoint(endpoint);
using var client = CreateClient(endpoint);
client.Connect(host, port);
await WaitForConnectionAsync(client, endpoint).ConfigureAwait(false);
await CultNetWriteCompletion.SendAndAwaitResultAsync(
refuse =>
ICultNetSchemaClient? client = null;
try
{
var endpoint = ResolvePrimaryEndpoint(shard);
var (host, port) = ParseEndpoint(endpoint);
client = CreateClient(endpoint);
client.Connect(host, port);
await WaitForConnectionAsync(client, endpoint).ConfigureAwait(false);
// The hello comes first on the ordered schema channel: it opts this connection in to the primary's receipts.
client.SendCultNet(new CultNetHelloMessage
{
RuntimeId = "gamecult-networking-shard-forwarder",
RuntimeKind = "dotnet",
SupportedMessageVersions = [CultNetSchemaVersions.WriteAdmitted, CultNetSchemaVersions.ErrorV1]
});
return client;
}
catch (Exception failure)
{
client?.Dispose();
if (failure is CultNetWriteNotSentException) throw;
throw new CultNetWriteNotSentException(messageId, failure);
}
}
private async Task SendAsync<T>(CultNetShardDescriptor shard, T message, string messageId) where T : ICultNetSchemaMessage
{
using var client = await ConnectBeforeSendAsync(shard, messageId).ConfigureAwait(false);
// From the write's send on, every failure (timeout, reset, silence, an exception from the send itself) propagates as it
// is: the frame may have gone out, so the caller treats it as unconfirmed.
await CultNetWriteCompletion.SendAndAwaitReceiptAsync(
(admitted, refuse, refuseV1) =>
{
client.OnCultNet(admitted);
client.OnCultNet(refuse);
client.OnCultNet(refuseV1);
return null;
},
() => client.SendCultNet(message),
messageId,
_options.ResultWindow).ConfigureAwait(false);
}

View file

@ -3533,7 +3533,8 @@ namespace GameCult.Networking
/// </summary>
public sealed class LiteNetLibTransportConnection
{
private readonly NetPeer? _peer;
private readonly Action<byte[]>? _deliver;
private readonly Action? _end;
private readonly CultNetTransportStats _stats = new CultNetTransportStats();
/// <summary>
@ -3541,10 +3542,99 @@ namespace GameCult.Networking
/// </summary>
public LiteNetLibTransportConnection(NetPeer peer, CultNetTransportProfile? profile = null)
{
_peer = peer ?? throw new ArgumentNullException(nameof(peer));
if (peer == null) throw new ArgumentNullException(nameof(peer));
_deliver = payload => peer.Send(payload, DeliveryMethod.ReliableOrdered);
_end = peer.Disconnect;
Profile = profile ?? CultNetTransportProfiles.CreateLiteNetLib("csharp-litenetlib-peer");
}
// The mock point for what leaves the connection: the peer's reliable ordered send.
internal LiteNetLibTransportConnection(Action<byte[]> deliver, Action? end = null)
{
_deliver = deliver ?? throw new ArgumentNullException(nameof(deliver));
_end = end;
Profile = CultNetTransportProfiles.CreateLiteNetLib("csharp-litenetlib-peer");
}
/// <summary>
/// Ends the connection. A write that can be neither answered nor refused is answered by this goodbye: the peer sees
/// its session end and settles the write as unconfirmed (<c>session_ended</c>), never as silence.
/// </summary>
internal void End() => _end?.Invoke();
/// <summary>
/// Answers a schema frame that did not decode with a <c>message_invalid</c> error.v1 naming its id when it holds one;
/// when that answer cannot be sent the connection ends instead.
/// </summary>
internal void AnswerUnreadableSchemaFrame(byte[] payload)
{
try
{
SendSchema(CultNetErrorV1Message.From(
CultNetErrorMessage.ForMessageInvalid("message", "the message could not be read"),
CultNetReadableMessageId.From(payload)));
}
catch (Exception)
{
End();
}
}
/// <summary>
/// A put or delete that arrived on an unreliable or sequenced channel of a receipt session (write-receipt.md, Reliable channel only):
/// answered <c>message_invalid</c>, details <c>channel</c>, fixed text, carrying the write's id, and not applied. True when
/// the frame was such a write and has been answered here.
/// </summary>
internal bool RefuseUnreliableWrite(ICultNetSchemaMessage message, DeliveryMethod method)
{
if (WriteReceipts == null) return false;
if (method is DeliveryMethod.ReliableOrdered or DeliveryMethod.ReliableUnordered) return false;
var messageId = message switch
{
CultNetDocumentPutRawMessage put => put.MessageId,
CultNetDocumentDeleteMessage delete => delete.MessageId,
_ => null
};
if (messageId == null && message is not (CultNetDocumentPutRawMessage or CultNetDocumentDeleteMessage)) return false;
try
{
SendSchema(CultNetErrorV1Message.From(
CultNetErrorMessage.ForMessageInvalid("channel", "receipt sessions require the reliable channel"), messageId));
}
catch (Exception)
{
End();
}
return true;
}
private readonly object _writeFramingGate = new();
private bool _writeFramingSettled;
/// <summary>
/// The write-receipt state of a connection that opted in (write-receipt.md); null for every other connection, which is
/// answered exactly as before.
/// </summary>
internal CultNetWriteReceiptSession? WriteReceipts { get; private set; }
/// <summary>
/// Decides this connection's write framing once: by its first hello, or by its first write if no hello came first. A
/// later hello changes nothing. The capability changes framing only; it never authorizes. Returns true when this call
/// opted the connection in.
/// </summary>
internal bool SettleWriteFraming(CultNetHelloMessage? hello)
{
lock (_writeFramingGate)
{
if (_writeFramingSettled) return false;
_writeFramingSettled = true;
if (hello == null || !CultNetWriteReceiptCapability.Declares(hello)) return false;
WriteReceipts = new CultNetWriteReceiptSession();
return true;
}
}
/// <summary>
/// Initializes a receive-only channel-aware adapter for LiteNetLib payload classification.
/// </summary>
@ -3580,14 +3670,14 @@ namespace GameCult.Networking
public void Send(string channelId, byte[] payload)
{
if (payload == null) throw new ArgumentNullException(nameof(payload));
if (_peer == null) throw new InvalidOperationException("A LiteNetLib peer is required to send through this transport.");
if (_deliver == null) throw new InvalidOperationException("A LiteNetLib peer is required to send through this transport.");
if (!string.Equals(channelId, "schema", StringComparison.Ordinal)
&& !string.Equals(channelId, "legacy", StringComparison.Ordinal))
{
throw new InvalidOperationException($"litenetlib transport only supports schema and legacy channels, got \"{channelId}\".");
}
_peer.Send(payload, DeliveryMethod.ReliableOrdered);
_deliver(payload);
_stats.BytesSent += payload.Length;
_stats.FramesSent++;
}

View file

@ -24,6 +24,40 @@ namespace GameCult.Networking
public CultNetErrorMessage Refusal { get; }
}
/// <summary>
/// A write nothing answered within its window: neither admitted nor refused (write-receipt.md, "unconfirmed").
/// </summary>
public sealed class CultNetWriteUnconfirmedException : Exception
{
/// <summary>Creates the exception for the write <paramref name="messageId"/>.</summary>
public CultNetWriteUnconfirmedException(string messageId)
: base($"No admission or refusal arrived for write '{messageId}'.")
{
MessageId = messageId;
}
/// <summary>Gets the id of the unconfirmed write.</summary>
public string MessageId { get; }
}
/// <summary>
/// A forwarded write that provably never left this process: the connection failed, or the send failed before any byte was
/// written. It is the only forwarding failure after which the same id may be forwarded again. A send port that cannot prove
/// nothing was written must not throw this; every other failure of a forward is unconfirmed.
/// </summary>
public sealed class CultNetWriteNotSentException : Exception
{
/// <summary>Creates the exception for the write <paramref name="messageId"/>; <paramref name="cause"/> is for logs only.</summary>
public CultNetWriteNotSentException(string messageId, Exception? cause = null)
: base($"Write '{messageId}' was not sent.", cause)
{
MessageId = messageId;
}
/// <summary>Gets the id of the write that was not sent.</summary>
public string MessageId { get; }
}
/// <summary>
/// Completes a write on the peer's result frame, never on a send's return or a transport acknowledgement:
/// an ack proves delivery, not admission.
@ -33,6 +67,49 @@ namespace GameCult.Networking
/// <summary>How long after a send a refusal may still arrive when the caller names no window.</summary>
public static readonly TimeSpan DefaultResultWindow = TimeSpan.FromMilliseconds(500);
/// <summary>How long a receipt-capable peer has to answer a write (write-receipt.md, <c>RECEIPT_RESULT_WINDOW_MS</c>).</summary>
public static readonly TimeSpan ReceiptResultWindow = TimeSpan.FromMilliseconds(5000);
/// <summary>
/// Sends one write to a receipt-capable peer and waits <paramref name="resultWindow"/> for its answer: a
/// <c>write_admitted</c> naming <paramref name="messageId"/> returns, an <c>error.v1</c> naming it throws
/// <see cref="CultNetWriteRefusedException"/>, and an id-less error (v0, or v1 with no id) is attributed to this write
/// because it is the only one in flight on the connection. Silence throws <see cref="CultNetWriteUnconfirmedException"/>.
/// </summary>
public static async Task SendAndAwaitReceiptAsync(
Func<Action<CultNetWriteAdmittedMessage>, Action<CultNetErrorMessage>, Action<CultNetErrorV1Message>, IDisposable?> subscribe,
Action send,
string messageId,
TimeSpan resultWindow,
CancellationToken cancellationToken = default)
{
if (subscribe == null) throw new ArgumentNullException(nameof(subscribe));
if (send == null) throw new ArgumentNullException(nameof(send));
var answer = new TaskCompletionSource<CultNetErrorMessage?>(TaskCreationOptions.RunContinuationsAsynchronously);
using var registration = subscribe(
admitted =>
{
if (string.Equals(admitted.MessageId, messageId, StringComparison.Ordinal)) answer.TrySetResult(null);
},
error => answer.TrySetResult(error),
error =>
{
if (error.MessageId == null || string.Equals(error.MessageId, messageId, StringComparison.Ordinal))
answer.TrySetResult(error.ToV0());
});
send();
var window = Task.Delay(resultWindow, cancellationToken);
await Task.WhenAny(answer.Task, window).ConfigureAwait(false);
if (!answer.Task.IsCompleted)
{
await window.ConfigureAwait(false);
throw new CultNetWriteUnconfirmedException(messageId);
}
var refusal = await answer.Task.ConfigureAwait(false);
if (refusal != null) throw new CultNetWriteRefusedException(refusal);
}
/// <summary>
/// Sends one write and waits <paramref name="resultWindow"/> for the peer's answer. A refusal frame rejects with
/// <see cref="CultNetWriteRefusedException"/> whatever its order against the transport acknowledgement. A peer

View file

@ -0,0 +1,396 @@
using System;
using System.Collections.Generic;
using System.Security.Cryptography;
using GameCult.Caching;
using MessagePack;
namespace GameCult.Networking
{
/// <summary>
/// The content hash a write-receipt server binds to a (peer, messageId) dedupe key
/// (contracts/cultnet/interop/write-receipt.md, rule 5).
/// </summary>
public static class CultNetWriteContentHash
{
/// <summary>
/// SHA-256 of the MessagePack array <c>[operation, schemaId, recordKey, payload]</c>, <c>payload</c> a <c>bin</c>
/// for a put and nil for a delete.
/// </summary>
public static byte[] Compute(string operation, string schemaId, string recordKey, byte[]? payload)
{
var buffer = new System.Buffers.ArrayBufferWriter<byte>();
var writer = new MessagePackWriter(buffer);
writer.WriteArrayHeader(4);
writer.Write(operation);
writer.Write(schemaId);
writer.Write(recordKey);
if (payload == null) writer.WriteNil();
else writer.Write(payload);
writer.Flush();
using var sha = SHA256.Create();
return sha.ComputeHash(buffer.WrittenSpan.ToArray());
}
}
/// <summary>
/// Whether a peer's hello opted in to write receipts: it lists both <c>cultnet.write_admitted.v0</c> and
/// <c>cultnet.error.v1</c>. The capability changes framing only; it never authorizes.
/// </summary>
internal static class CultNetWriteReceiptCapability
{
public static bool Declares(CultNetHelloMessage hello)
{
var versions = hello.SupportedMessageVersions;
return versions != null
&& Array.IndexOf(versions, CultNetSchemaVersions.WriteAdmitted) >= 0
&& Array.IndexOf(versions, CultNetSchemaVersions.ErrorV1) >= 0;
}
}
/// <summary>A SHA-256 digest held by value: the fixed-size stand-in for an id or a content hash.</summary>
internal readonly struct CultNetDigest : IEquatable<CultNetDigest>
{
private readonly ulong _a, _b, _c, _d;
private CultNetDigest(ulong a, ulong b, ulong c, ulong d) { _a = a; _b = b; _c = c; _d = d; }
public static CultNetDigest Of(byte[] bytes)
{
using var sha = SHA256.Create();
var hash = sha.ComputeHash(bytes);
return new CultNetDigest(
BitConverter.ToUInt64(hash, 0), BitConverter.ToUInt64(hash, 8),
BitConverter.ToUInt64(hash, 16), BitConverter.ToUInt64(hash, 24));
}
public bool Equals(CultNetDigest other) => _a == other._a && _b == other._b && _c == other._c && _d == other._d;
public override bool Equals(object? obj) => obj is CultNetDigest other && Equals(other);
public override int GetHashCode() => unchecked((int)_a ^ (int)(_a >> 32));
public static bool operator ==(CultNetDigest left, CultNetDigest right) => left.Equals(right);
public static bool operator !=(CultNetDigest left, CultNetDigest right) => !left.Equals(right);
}
/// <summary>
/// What a session keeps of a refusal (write-receipt.md, What is remembered): its code, the field its details name, and its
/// routing hint (a shard descriptor the server owns). Never the write's strings or an authority's reason: the replay is built
/// from fixed text and the fields of the resend, which carries the same content.
/// </summary>
internal readonly struct CultNetRememberedRefusal
{
public string? Code { get; }
public bool HasDetails { get; }
public string? Field { get; }
public CultNetShardRoutingHint? Hint { get; }
public CultNetRememberedRefusal(string? code, bool hasDetails, string? field, CultNetShardRoutingHint? hint)
{
Code = code; HasDetails = hasDetails; Field = field; Hint = hint;
}
public static CultNetRememberedRefusal Of(CultNetErrorMessage error) =>
new(error.Code, error.Details != null, error.Details?.Field, error.RoutingHint);
private static string? NullIfEmpty(string value) => value.Length == 0 ? null : value;
/// <summary>The refusal, in fixed text, naming the write's own record key or schema id where its details do.</summary>
public CultNetErrorMessage ToError(string schemaId, string recordKey) => new()
{
Error = Code switch
{
"unowned_schema" => "unowned_schema: no shard owns the schema and key",
null => Hint != null ? CultNetDatabaseServer.RoutingRefusalText : CultNetWriteReceiptSession.WriteFailedText,
_ => Code
},
Code = Code,
RoutingHint = Hint,
Details = HasDetails
? new CultNetErrorDetails
{
Field = Field,
Value = Field switch { "recordKey" => NullIfEmpty(recordKey), "schemaId" => NullIfEmpty(schemaId), _ => null }
}
: null
};
}
/// <summary>
/// One receipt-capable peer's writes: message id to content hash and the answer given, refusals included, so a resend of the
/// same content is answered again without being decided again and the same id with other content is refused. Ids and content
/// hashes are held as fixed-size digests; the answer is held as its kind (and, for a refusal, its code and the field its details name),
/// never the strings of the write.
/// </summary>
internal sealed class CultNetWriteReceiptSession
{
/// <summary>The most writes a peer is remembered for, refused ones included; a peer at the limit has new writes refused.</summary>
internal const int MaxRememberedWrites = 1024;
/// <summary>The most ids refused at the limit that are remembered, so their resend is judged the same way.</summary>
internal const int MaxRememberedLimitRefusals = 1024;
/// <summary>The longest messageId a write may carry, in UTF-8 bytes (write-receipt.md, Id size).</summary>
internal const int MaxMessageIdBytes = 256;
/// <summary>What a write is refused with when it failed for a reason no typed refusal names. Never an exception's own text.</summary>
internal const string WriteFailedText = "the write could not be applied";
private readonly object _gate = new();
private readonly Dictionary<CultNetDigest, Entry> _writes = new();
private readonly Dictionary<CultNetDigest, CultNetDigest> _limitRefusals = new();
private sealed class Entry
{
public Entry(CultNetDigest hash) { Hash = hash; }
public CultNetDigest Hash { get; set; }
public bool Abandoned { get; set; }
public bool Admitted { get; set; }
public CultNetRememberedRefusal? Refusal { get; set; }
}
public enum Begin { New, ReplayAdmitted, ReplayRefusal, Unanswered, DifferentContent, Full }
private static CultNetDigest IdDigest(string messageId) => CultNetDigest.Of(System.Text.Encoding.UTF8.GetBytes(messageId));
/// <summary>
/// Looks <paramref name="messageId"/> up first. An id not seen is claimed for this content (<see cref="Begin.New"/>);
/// otherwise the outcome says what already holds it, and <paramref name="refusal"/> is the refusal to repeat.
/// </summary>
public Begin TryBegin(string messageId, byte[] hash, out CultNetRememberedRefusal? refusal)
{
var id = IdDigest(messageId);
var content = CultNetDigest.Of(hash);
lock (_gate)
{
refusal = null;
if (_writes.TryGetValue(id, out var entry))
{
if (entry.Abandoned)
{
// Not sent, so judged afresh in the slot it already holds; no slot is ever freed.
entry.Abandoned = false;
entry.Hash = content;
return Begin.New;
}
if (entry.Hash != content) return Begin.DifferentContent;
if (entry.Admitted) return Begin.ReplayAdmitted;
if (entry.Refusal != null)
{
refusal = entry.Refusal;
return Begin.ReplayRefusal;
}
// No answer yet: the first attempt is still being applied, or its primary left it unanswered for good.
return Begin.Unanswered;
}
if (_writes.Count >= MaxRememberedWrites)
{
if (_limitRefusals.TryGetValue(id, out var refused)) return refused == content ? Begin.Full : Begin.DifferentContent;
if (_limitRefusals.Count < MaxRememberedLimitRefusals) _limitRefusals[id] = content;
return Begin.Full;
}
_writes[id] = new Entry(content);
return Begin.New;
}
}
/// <summary>The write was admitted: remember it so a resend repeats the receipt.</summary>
public void Admitted(string messageId)
{
lock (_gate)
{
if (_writes.TryGetValue(IdDigest(messageId), out var entry)) entry.Admitted = true;
}
}
/// <summary>The write was refused: remember the refusal so a resend repeats it.</summary>
public void Refused(string messageId, CultNetRememberedRefusal refusal)
{
lock (_gate)
{
if (_writes.TryGetValue(IdDigest(messageId), out var entry)) entry.Refusal = refusal;
}
}
/// <summary>
/// The write never reached anyone who could have applied it: a repeat is judged afresh. The entry keeps its slot, so
/// forgetting a write never makes room for an id that was refused at the limit.
/// </summary>
public void Abandoned(string messageId)
{
lock (_gate)
{
if (_writes.TryGetValue(IdDigest(messageId), out var entry)) entry.Abandoned = true;
}
}
}
/// <summary>
/// The messageId of a schema frame that does not decode (write-receipt.md, rule 3: a refusal carries the write's id whenever
/// the frame is readable enough to find it).
/// </summary>
internal static class CultNetReadableMessageId
{
/// <summary>The frame's top-level string <c>messageId</c>, or null when the bytes are not a map holding one.</summary>
public static string? From(byte[] payload)
{
try
{
var reader = new MessagePackReader(payload);
var entries = reader.ReadMapHeader();
for (var index = 0; index < entries; index++)
{
var key = reader.NextMessagePackType == MessagePackType.String ? reader.ReadString() : null;
if (key == "messageId" && reader.NextMessagePackType == MessagePackType.String)
{
var id = reader.ReadString();
return string.IsNullOrEmpty(id) ? null : id;
}
if (key == null) reader.Skip();
reader.Skip();
}
}
catch (Exception)
{
// Not a readable map: no id.
}
return null;
}
}
/// <summary>
/// One write on a receipt-capable connection: its dedupe claim and its single answer (write-receipt.md, rules 1 to 5).
/// </summary>
internal sealed class CultNetReceiptedWrite
{
private readonly CultNetWriteReceiptSession _session;
private readonly Action<ICultNetSchemaMessage> _send;
private readonly string _operation;
private readonly string _messageId;
private readonly string _schemaId;
private readonly string _recordKey;
private readonly byte[]? _payload;
private bool _claimed;
private bool _answered;
public CultNetReceiptedWrite(
CultNetWriteReceiptSession session,
Action<ICultNetSchemaMessage> send,
string operation,
string? messageId,
string? schemaId,
string? recordKey,
byte[]? payload)
{
_session = session;
_send = send;
_operation = operation;
_messageId = messageId ?? string.Empty;
_schemaId = schemaId ?? string.Empty;
_recordKey = recordKey ?? string.Empty;
_payload = payload;
}
/// <summary>
/// Looks the write's id up before anything else is decided (rule 5). True when the id is new and the write should go on
/// to authorization and the commit; false when it was answered here (a replay, a reused id, a full table) or is owed no
/// further answer (still in flight, or left silent by its primary).
/// </summary>
public bool Begin()
{
var outcome = _session.TryBegin(_messageId, CultNetWriteContentHash.Compute(_operation, _schemaId, _recordKey, _payload), out var refusal);
switch (outcome)
{
case CultNetWriteReceiptSession.Begin.New:
_claimed = true;
return true;
case CultNetWriteReceiptSession.Begin.ReplayAdmitted:
_answered = true;
_send(AdmittedReceipt());
return false;
case CultNetWriteReceiptSession.Begin.ReplayRefusal:
_answered = true;
_send(CultNetErrorV1Message.From(refusal!.Value.ToError(_schemaId, _recordKey), _messageId));
return false;
case CultNetWriteReceiptSession.Begin.Unanswered:
// The first attempt is still being applied and will answer this id, or its primary left it unanswered:
// then the claim stays, so the id is never sent again on this session, and the write stays unconfirmed.
_answered = true;
return false;
case CultNetWriteReceiptSession.Begin.DifferentContent:
Refused(CultNetErrorMessage.ForMessageInvalid("messageId", "the message id was already used for different content"));
return false;
default:
_answered = true;
_send(CultNetErrorV1Message.LimitReached(_messageId));
return false;
}
}
private CultNetWriteAdmittedMessage AdmittedReceipt() => new()
{
MessageId = _messageId,
Operation = _operation,
SchemaId = _schemaId,
RecordKey = _recordKey
};
/// <summary>The write committed: remember the receipt and send it.</summary>
public void Admitted()
{
if (_answered) return;
_answered = true;
if (_claimed) _session.Admitted(_messageId);
_send(AdmittedReceipt());
}
/// <summary>The write was refused: remember the refusal (a resend repeats it) and send its one error.v1.</summary>
public void Refused(CultNetErrorMessage error)
{
if (_answered) return;
_answered = true;
if (_claimed)
{
// A remembered refusal is answered from what is remembered, so the replay is the refusal again.
var kept = CultNetRememberedRefusal.Of(error);
_session.Refused(_messageId, kept);
error = kept.ToError(_schemaId, _recordKey);
}
_send(CultNetErrorV1Message.From(error, _messageId));
}
/// <summary>
/// A forward that may have reached its primary and is unconfirmed (silence, reset, timeout, an ambiguous send): no answer is
/// owed or possible, and the claim stays so the id is never forwarded again on this session. Sends nothing.
/// </summary>
public void LeftSilent()
{
if (_answered) return;
_answered = true;
}
/// <summary>A forward that was provably not sent (connect failed, or the send failed before any byte was written): forget the claim, send nothing.</summary>
public void NotSent()
{
if (_answered) return;
_answered = true;
if (_claimed) _session.Abandoned(_messageId);
_claimed = false;
}
/// <summary>
/// Why the write's id cannot be used at all, before anything is remembered: it is missing (refused with no id) or longer
/// than <see cref="CultNetWriteReceiptSession.MaxMessageIdBytes"/> UTF-8 bytes. Null when the id is usable.
/// </summary>
public CultNetErrorMessage? IdRefusal =>
_messageId.Length == 0
? CultNetErrorMessage.ForMessageInvalid("messageId", "a write needs a messageId")
: System.Text.Encoding.UTF8.GetByteCount(_messageId) > CultNetWriteReceiptSession.MaxMessageIdBytes
? CultNetErrorMessage.ForMessageInvalid("messageId", "messageId exceeds 256 bytes")
: null;
}
}

View file

@ -23,7 +23,7 @@ namespace GameCult.Networking
private const string EmailPattern =
@"^([0-9a-zA-Z]([\+\-_\.][0-9a-zA-Z]+)*)+@(([0-9a-zA-Z][-\w]*[0-9a-zA-Z]*\.)+[a-zA-Z0-9]{2,17})$";
private const string UsernamePattern = @"^[A-Za-z0-9]+(?:[ _-][A-Za-z0-9]+)*$";
private const int ServerPort = 3075;
private const int DefaultPort = 3075;
private const float SessionTimeoutSeconds = 1800; // 30 minutes
private const float SessionRefreshThresholdSeconds = 300; // 5 minutes
private const int MaxConnectionAttemptsPerMinute = 30;
@ -63,6 +63,12 @@ namespace GameCult.Networking
/// </summary>
public bool LogSensitivePayloads { get; set; }
/// <summary>
/// Gets or sets the UDP port <see cref="Start"/> binds (default 3075; 0 asks the OS for a free one). After
/// <see cref="Start"/> it holds the port actually bound.
/// </summary>
public int Port { get; set; } = DefaultPort;
private float Time => (float)(_timer?.Elapsed.TotalSeconds ?? 0d);
/// <summary>
@ -74,7 +80,7 @@ namespace GameCult.Networking
{
TransportId = "litenetlib-server",
Host = "0.0.0.0",
Port = ServerPort
Port = Port
});
/// <summary>
@ -357,7 +363,8 @@ namespace GameCult.Networking
UnsyncedEvents = true,
NatPunchEnabled = true
};
_netManager.Start(ServerPort);
_netManager.Start(Port);
Port = _netManager.LocalPort;
listener.NetworkErrorEvent += (point, code) => Logger.LogInfo($"{point.Address}: Error {code}");
@ -408,11 +415,25 @@ namespace GameCult.Networking
{
var cultNetMessage = LiteNetLibTransportConnection.DecodeSchema(frame);
Logger.LogDebug($"Received CultNet schema message {cultNetMessage.SchemaVersion}");
if (transport.RefuseUnreliableWrite(cultNetMessage, method)) return;
await HandleCultNetSchemaMessageAsync(peer, user, cultNetMessage).ConfigureAwait(false);
return;
}
var message = LiteNetLibTransportConnection.DecodeLegacy(frame);
Message message;
try
{
message = LiteNetLibTransportConnection.DecodeLegacy(frame);
}
catch (Exception ex) when (transport.WriteReceipts != null)
{
// A frame that decodes on neither channel is classed legacy, so this is where an unreadable frame lands.
// A receipt-capable peer is told which frame was unreadable (by its id, if the frame still holds one).
Logger.LogWarning($"CultNet frame unreadable: {ex.Message}");
transport.AnswerUnreadableSchemaFrame(frame.Payload);
return;
}
if (LogSensitivePayloads)
{
Logger.LogDebug($"Received message: {MessagePackSerializer.ConvertToJson(new ReadOnlyMemory<byte>(frame.Payload))}");
@ -507,7 +528,7 @@ namespace GameCult.Networking
}
});
Logger.LogInfo($"Server started on port {ServerPort}.");
Logger.LogInfo($"Server started on port {Port}.");
}
private async Task HandleCultNetSchemaMessageAsync(NetPeer peer, User user, ICultNetSchemaMessage message)

View file

@ -418,6 +418,7 @@ namespace GameCult.Networking.Tests
CultNetDatabaseChangeKind.Added, VariantKey, schemaId, database.Shards[0], cache.Get<VariantWireFixture>(VariantKey)!, previousDocument: null);
Assert.That(Deliver(databaseServer, added, selection, refused, errors), Is.Null);
Assert.That(errors, Has.Count.EqualTo(1));
Assert.That(errors[0].Error, Is.EqualTo("the record is a variant CultNet cannot carry"));
var removed = new CultNetDatabaseChange<VariantWireFixture>(
CultNetDatabaseChangeKind.Removed, VariantKey, schemaId, database.Shards[0], document: null, previousDocument: cache.Get<VariantWireFixture>(VariantKey)!);

View file

@ -178,6 +178,46 @@ namespace GameCult.Networking.Tests
Assert.That(scene.Logger.Errors, Has.Count.EqualTo(1));
}
// The RUDP schema server implements no receipt rules, so it must not advertise them (write-receipt.md, Capability and the
// conformance table): whatever hello it sends lists neither frame.
[Test]
public async Task TheRudpSchemaServerNeverAdvertisesTheReceiptFrames()
{
using var server = new RudpCultNetSchemaServer(new RudpCultNetSchemaServerOptions
{
RuntimeId = "csharp-rudp-no-receipts",
ConnectionId = ConnectionId
});
var client = await Connect(server);
try
{
SendSchema(server, client, CultNetSchemaMessageSerialization.Serialize(new CultNetHelloMessage
{
RuntimeId = "receipt-client",
RuntimeKind = "test",
SupportedMessageVersions = [CultNetSchemaVersions.WriteAdmitted, CultNetSchemaVersions.ErrorV1]
}));
await server.PollAvailableAsync(16);
var hellos = new List<CultNetHelloMessage>();
foreach (var packet in Drain(client.Socket))
foreach (var frame in client.Session.Receive(packet, 1).Delivered)
if (CultNetSchemaMessageSerialization.Deserialize(frame.Payload) is CultNetHelloMessage hello)
hellos.Add(hello);
foreach (var hello in hellos)
{
var listed = hello.SupportedMessageVersions ?? Array.Empty<string>();
Assert.That(listed, Has.None.EqualTo(CultNetSchemaVersions.WriteAdmitted));
Assert.That(listed, Has.None.EqualTo(CultNetSchemaVersions.ErrorV1));
}
}
finally
{
client.Socket.Dispose();
}
}
private static List<CultNetErrorMessage> ErrorsReceivedBy(Client client)
{
var errors = new List<CultNetErrorMessage>();

View file

@ -0,0 +1,290 @@
#nullable enable
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using GameCult.Caching;
using LiteNetLib;
using NUnit.Framework;
using static GameCult.Networking.Tests.NetworkingTests;
namespace GameCult.Networking.Tests
{
// The receipt server on a real socket: Server.Start on an ephemeral loopback port and real LiteNetLib clients. The
// handler-level tests (CultNetWriteReceiptServerTests) hand frames to the handlers; only a running server decides what
// a frame that does not decode is answered with.
public sealed class CultNetWriteReceiptLoopbackTests
{
private static readonly TimeSpan Cap = TimeSpan.FromSeconds(10);
private static readonly CultRecordKey KeyA = new("tests:loopback:a");
private static readonly CultRecordKey KeyB = new("tests:loopback:b");
private const string ShardId = "loopback-shard";
private sealed class Client : IDisposable
{
private readonly NetManager _manager;
private readonly NetPeer _peer;
private readonly BlockingCollection<ICultNetSchemaMessage> _received = new();
public int Closed;
public Client(int port, string key)
{
var listener = new EventBasedNetListener();
var connected = new TaskCompletionSource<bool>(TaskCreationOptions.RunContinuationsAsynchronously);
var decoder = new LiteNetLibTransportConnection();
listener.PeerConnectedEvent += _ => connected.TrySetResult(true);
listener.PeerDisconnectedEvent += (_, _) => Interlocked.Increment(ref Closed);
listener.NetworkReceiveEvent += (_, reader, _, _) =>
_received.Add(LiteNetLibTransportConnection.DecodeSchema(decoder.Receive(reader.GetRemainingBytes())));
_manager = new NetManager(listener) { UnsyncedEvents = true };
_manager.Start(0);
_peer = _manager.Connect("127.0.0.1", port, key);
if (!connected.Task.Wait(Cap)) Assert.Fail("the client did not connect");
}
public void Send<T>(T message) where T : ICultNetSchemaMessage =>
_peer.Send(CultNetSchemaMessageSerialization.Serialize(message), DeliveryMethod.ReliableOrdered);
public void SendUnreliable<T>(T message) where T : ICultNetSchemaMessage =>
_peer.Send(CultNetSchemaMessageSerialization.Serialize(message), DeliveryMethod.Unreliable);
public void Send<T>(T message, DeliveryMethod method) where T : ICultNetSchemaMessage =>
_peer.Send(CultNetSchemaMessageSerialization.Serialize(message), method);
public void SendRaw(byte[] bytes) => _peer.Send(bytes, DeliveryMethod.ReliableOrdered);
public T Next<T>() where T : ICultNetSchemaMessage
{
if (!_received.TryTake(out var message, Cap)) Assert.Fail($"no {typeof(T).Name} arrived within {Cap}");
Assert.That(message, Is.InstanceOf<T>());
return (T)message!;
}
public void ExpectNothingMore(string because)
{
// A reliable ordered channel: anything the server sent before the probe's answer has already arrived.
Assert.That(_received.Count, Is.Zero, because);
}
public void Dispose()
{
_manager.Stop();
_received.Dispose();
}
}
private sealed class Authority : ICultNetRemoteWriteAuthority
{
public CultNetRemoteWriteDecision Decide(CultNetRemoteWriteRequest request) =>
request.RecordKey.Equals(KeyB) ? CultNetRemoteWriteDecision.Refuse("denied") : CultNetRemoteWriteDecision.Grant();
}
private sealed class Running : IDisposable
{
public Server Server = null!;
public CultCache Cache = null!;
public CultNetDatabase Database = null!;
public string SchemaId = string.Empty;
private CultNetDatabaseServer _databaseServer = null!;
public static Running Start()
{
var cache = new CultCache();
var schemaId = cache.Registry.GetRequired<NetworkSchemaNote>().SchemaId;
var database = new CultNetDatabase(cache, new CultNetDatabaseOptions
{
Shards = [new CultNetShardDescriptor(ShardId, "primary", epoch: 1, isPrimary: true, schemaIds: [schemaId])]
});
var server = new Server(cache, ServerSecurityOptions.Development()) { Port = 0 };
var databaseServer = new CultNetDatabaseServer(server, database, new CultNetDatabaseServerOptions
{
RemoteWriteAuthority = new Authority()
});
server.Start();
return new Running { Server = server, Cache = cache, Database = database, SchemaId = schemaId, _databaseServer = databaseServer };
}
public Client Connect() => new(Server.Port, ServerSecurityOptions.Development().ConnectionKey);
public CultNetDocumentPutRawMessage PutOf(string id, CultRecordKey key, string text)
{
var message = Database.Documents.CreateRawDocumentPutMessage(id, new CultRecordHandle<NetworkSchemaNote>(key),
new NetworkSchemaNote { Schema = "tests.networking_note.v1", Text = text });
message.ShardId = ShardId;
message.ShardEpoch = 1;
return message;
}
public void Dispose()
{
_databaseServer.Dispose();
Server.Dispose();
}
}
private static CultNetHelloMessage HelloOf(params string[] versions) =>
new() { RuntimeId = "loopback-client", RuntimeKind = "test", SupportedMessageVersions = versions };
private static readonly string[] BothFrames = [CultNetSchemaVersions.WriteAdmitted, CultNetSchemaVersions.ErrorV1];
[Test]
public void StartBindsTheRequestedPortAndReportsTheOneBound()
{
using var running = Running.Start();
Assert.That(running.Server.Port, Is.Not.Zero.And.Not.EqualTo(3075));
}
// contracts/cultnet/interop/write-receipt-vectors.json, error_v1_unreadable_frame_has_null_message_id: code
// message_invalid, details.field "message", messageId null.
[TestCase(new byte[] { 0xC1 }, TestName = "AnUnreadableFrameFromAReceiptPeerIsAnsweredWithMessageInvalidAndNoId_NeverUsedByte")]
[TestCase(new byte[] { 0x92, 0x01, 0x02 }, TestName = "AnUnreadableFrameFromAReceiptPeerIsAnsweredWithMessageInvalidAndNoId_NotAMap")]
public void AnUnreadableFrameFromAReceiptPeerIsAnsweredWithMessageInvalidAndNoId(byte[] garbage)
{
using var running = Running.Start();
using var client = running.Connect();
client.Send(HelloOf(BothFrames));
client.Next<CultNetHelloMessage>();
client.SendRaw(garbage);
var error = client.Next<CultNetErrorV1Message>();
Assert.That(error.Code, Is.EqualTo("message_invalid"));
Assert.That(error.Details!.Field, Is.EqualTo("message"));
Assert.That(error.MessageId, Is.Null);
}
[Test]
public async Task AReceiptPeerIsStillServedAfterAnUnreadableFrameAndALegacyPeerIsAnsweredAsBefore()
{
using var running = Running.Start();
using var receipt = running.Connect();
receipt.Send(HelloOf(BothFrames));
receipt.Next<CultNetHelloMessage>();
receipt.SendRaw(new byte[] { 0xC1 });
receipt.Next<CultNetErrorV1Message>();
receipt.Send(running.PutOf("r1", KeyA, "one"));
Assert.That(receipt.Next<CultNetWriteAdmittedMessage>().MessageId, Is.EqualTo("r1"));
using var legacy = running.Connect();
legacy.Send(HelloOf()); // a hello that lists neither frame is answered with nothing
legacy.SendRaw(new byte[] { 0xC1 }); // unanswered: were it answered, error.v1 would arrive ahead of the refusal
legacy.Send(running.PutOf("l1", KeyB, "two"));
var refusal = legacy.Next<CultNetErrorMessage>();
Assert.That(refusal, Is.TypeOf<CultNetErrorMessage>());
Assert.That(refusal.Code, Is.EqualTo("write_not_authorized"));
legacy.Send(running.PutOf("l2", KeyA, "three"));
await WaitFor(() => running.Cache.Get<NetworkSchemaNote>(KeyA)?.Text == "three");
legacy.ExpectNothingMore("a legacy peer is never sent a receipt or error.v1");
Assert.That(running.Cache.Get<NetworkSchemaNote>(KeyB), Is.Null);
}
// write-receipt.md, Reliable channel only: a write on an unreliable channel of a receipt session is refused
// message_invalid, details.field "channel", fixed text, carrying the write's id; it is not applied.
[Test]
public async Task AWriteOnAnUnreliableChannelOfAReceiptPeerIsRefusedAndNotApplied()
{
using var running = Running.Start();
using var client = running.Connect();
client.Send(HelloOf(BothFrames));
client.Next<CultNetHelloMessage>();
client.SendUnreliable(running.PutOf("u1", KeyA, "lost"));
var error = client.Next<CultNetErrorV1Message>();
Assert.That(error.Code, Is.EqualTo("message_invalid"));
Assert.That(error.Details!.Field, Is.EqualTo("channel"));
Assert.That(error.Error, Is.EqualTo("message_invalid: receipt sessions require the reliable channel"));
Assert.That(error.MessageId, Is.EqualTo("u1"));
// The session goes on, and the same id on the reliable channel is a new write, not a replayed refusal.
client.Send(running.PutOf("u1", KeyA, "kept"));
Assert.That(client.Next<CultNetWriteAdmittedMessage>().MessageId, Is.EqualTo("u1"));
await WaitFor(() => running.Cache.Get<NetworkSchemaNote>(KeyA)?.Text == "kept");
}
// write-receipt.md, Reliable channel only: reliable and not sequenced. Reliable ordered and unordered qualify; reliable
// sequenced does not, and nor does any unreliable method.
[TestCase(DeliveryMethod.ReliableSequenced, true)]
[TestCase(DeliveryMethod.Sequenced, true)]
[TestCase(DeliveryMethod.Unreliable, true)]
[TestCase(DeliveryMethod.ReliableUnordered, false)]
[TestCase(DeliveryMethod.ReliableOrdered, false)]
public async Task OnlyAReliableNonSequencedChannelCarriesAWriteOfAReceiptPeer(DeliveryMethod method, bool refused)
{
using var running = Running.Start();
using var client = running.Connect();
client.Send(HelloOf(BothFrames));
client.Next<CultNetHelloMessage>();
client.Send(running.PutOf("c1", KeyA, "text"), method);
if (refused)
{
var error = client.Next<CultNetErrorV1Message>();
Assert.That(error.Details!.Field, Is.EqualTo("channel"));
Assert.That(error.MessageId, Is.EqualTo("c1"));
Assert.That(running.Cache.Get<NetworkSchemaNote>(KeyA), Is.Null);
}
else
{
Assert.That(client.Next<CultNetWriteAdmittedMessage>().MessageId, Is.EqualTo("c1"));
await WaitFor(() => running.Cache.Get<NetworkSchemaNote>(KeyA)?.Text == "text");
}
}
[Test]
public void ADeleteOnAnUnreliableChannelOfAReceiptPeerIsRefusedWithItsId()
{
using var running = Running.Start();
using var client = running.Connect();
client.Send(HelloOf(BothFrames));
client.Next<CultNetHelloMessage>();
client.SendUnreliable(new CultNetDocumentDeleteMessage
{
MessageId = "ud1", SchemaId = running.SchemaId, RecordKey = KeyA.Value, ShardId = ShardId, ShardEpoch = 1
});
var error = client.Next<CultNetErrorV1Message>();
Assert.That(error.Code, Is.EqualTo("message_invalid"));
Assert.That(error.Details!.Field, Is.EqualTo("channel"));
Assert.That(error.MessageId, Is.EqualTo("ud1"));
}
[Test]
public async Task OnlyWritesAreRefusedOnTheUnreliableChannelAndALegacyPeerIsLeftAsBefore()
{
using var running = Running.Start();
using var receipt = running.Connect();
receipt.Send(HelloOf(BothFrames));
receipt.Next<CultNetHelloMessage>();
receipt.SendUnreliable(HelloOf(BothFrames)); // not a write: answered with nothing on the unreliable channel
receipt.Send(running.PutOf("r1", KeyA, "one"));
Assert.That(receipt.Next<CultNetWriteAdmittedMessage>().MessageId, Is.EqualTo("r1"), "nothing else was sent ahead of the receipt");
receipt.Send(running.PutOf("r2", KeyA, "two"));
Assert.That(receipt.Next<CultNetWriteAdmittedMessage>().MessageId, Is.EqualTo("r2"));
receipt.ExpectNothingMore("the unreliable hello was answered with nothing");
using var legacy = running.Connect();
legacy.Send(HelloOf());
legacy.SendUnreliable(running.PutOf("l1", KeyB, "two")); // refused by the authority, answered as v0 as before
Assert.That(legacy.Next<CultNetErrorMessage>().Code, Is.EqualTo("write_not_authorized"));
legacy.SendUnreliable(running.PutOf("l2", KeyA, "three"));
await WaitFor(() => running.Cache.Get<NetworkSchemaNote>(KeyA)?.Text == "three");
legacy.ExpectNothingMore("a legacy peer is never sent a receipt, error.v1 or a channel refusal");
}
private static async Task WaitFor(Func<bool> condition)
{
var deadline = DateTime.UtcNow + Cap;
while (!condition())
{
if (DateTime.UtcNow > deadline) Assert.Fail("the condition did not hold within the cap");
await Task.Delay(5);
}
}
}
}

File diff suppressed because it is too large Load diff

View file

@ -5189,7 +5189,9 @@ namespace GameCult.Networking.Tests
[Test]
public async Task CultNetSchemaWriteForwarder_Uses_SchemaClientPort()
{
var fake = new CapturingSchemaClient(_ => null);
var fake = new CapturingSchemaClient(message => message is CultNetDocumentDeleteMessage delete
? new CultNetWriteAdmittedMessage { MessageId = delete.MessageId, Operation = "delete", SchemaId = delete.SchemaId, RecordKey = delete.RecordKey }
: null);
var forwarder = new CultNetSchemaWriteForwarder(new CultNetSchemaWriteForwarderOptions
{
CreateClient = () => fake
@ -5203,11 +5205,16 @@ namespace GameCult.Networking.Tests
await forwarder.ForwardDeleteAsync(shard, new CultNetDocumentDeleteMessage
{
MessageId = "fwd-del",
SchemaId = "note.v0",
RecordKey = "note:1"
});
var sent = (CultNetDocumentDeleteMessage)fake.SentMessages.Single();
var hello = (CultNetHelloMessage)fake.SentMessages[0];
Assert.That(hello.SupportedMessageVersions, Is.EquivalentTo(new[] { CultNetSchemaVersions.WriteAdmitted, CultNetSchemaVersions.ErrorV1 }),
"the hello opts the forwarding connection in before the write");
var sent = (CultNetDocumentDeleteMessage)fake.SentMessages[1];
Assert.That(sent.MessageId, Is.EqualTo("fwd-del"), "the write goes under its original id");
Assert.That(fake.ConnectedHost, Is.EqualTo("primary.example.test"));
Assert.That(sent.ShardId, Is.EqualTo("notes"));
Assert.That(sent.ShardEpoch, Is.EqualTo(3));
@ -5241,7 +5248,7 @@ namespace GameCult.Networking.Tests
}
[Test]
public async Task CultNetSchemaWriteForwarder_DoesNotCompleteOnTheSendReturning()
public void CultNetSchemaWriteForwarder_SilenceIsUnconfirmedNotSuccess()
{
var fake = new CapturingSchemaClient(_ => null);
var forwarder = new CultNetSchemaWriteForwarder(new CultNetSchemaWriteForwarderOptions
@ -5254,11 +5261,51 @@ namespace GameCult.Networking.Tests
primaryEndpoints: ["cultnet://primary.example.test:4075"]);
var started = DateTime.UtcNow;
await forwarder.ForwardDeleteAsync(shard, new CultNetDocumentDeleteMessage { SchemaId = "note.v0", RecordKey = "note:1" });
var unconfirmed = Assert.ThrowsAsync<CultNetWriteUnconfirmedException>(() =>
forwarder.ForwardDeleteAsync(shard, new CultNetDocumentDeleteMessage { MessageId = "fwd-del", SchemaId = "note.v0", RecordKey = "note:1" }));
Assert.That(unconfirmed!.MessageId, Is.EqualTo("fwd-del"));
Assert.That(DateTime.UtcNow - started, Is.GreaterThanOrEqualTo(TimeSpan.FromMilliseconds(380)));
}
[Test]
public void CultNetSchemaWriteForwarder_AnswersNamingAnotherWriteSettleNothing()
{
var fake = new CapturingSchemaClient(_ => null);
var forwarder = new CultNetSchemaWriteForwarder(new CultNetSchemaWriteForwarderOptions
{
CreateClient = () => fake,
ResultWindow = TimeSpan.FromMilliseconds(300)
});
var shard = new CultNetShardDescriptor(
"notes", "primary", 3, isPrimary: false,
primaryEndpoints: ["cultnet://primary.example.test:4075"]);
fake.OnSend = () =>
{
fake.Deliver(new CultNetWriteAdmittedMessage { MessageId = "someone-else", Operation = "delete", SchemaId = "note.v0", RecordKey = "note:1" });
fake.Deliver(new CultNetErrorV1Message { Error = "refused", Code = "write_not_authorized", MessageId = "someone-else" });
};
Assert.ThrowsAsync<CultNetWriteUnconfirmedException>(() =>
forwarder.ForwardDeleteAsync(shard, new CultNetDocumentDeleteMessage { MessageId = "fwd-del", SchemaId = "note.v0", RecordKey = "note:1" }));
}
[Test]
public void CultNetSchemaWriteForwarder_RejectsWithTheErrorV1NamingItsWrite()
{
var fake = new CapturingSchemaClient(_ => null);
var forwarder = new CultNetSchemaWriteForwarder(new CultNetSchemaWriteForwarderOptions { CreateClient = () => fake });
var shard = new CultNetShardDescriptor(
"notes", "primary", 3, isPrimary: false,
primaryEndpoints: ["cultnet://primary.example.test:4075"]);
fake.OnSend = () => fake.Deliver(new CultNetErrorV1Message { Error = "refused", Code = "write_not_authorized", MessageId = "fwd-del" });
var refused = Assert.ThrowsAsync<CultNetWriteRefusedException>(() =>
forwarder.ForwardDeleteAsync(shard, new CultNetDocumentDeleteMessage { MessageId = "fwd-del", SchemaId = "note.v0", RecordKey = "note:1" }));
Assert.That(refused!.Code, Is.EqualTo("write_not_authorized"));
}
[Test]
public void CultMeshVerseCatalog_Finds_CompatibleTransferTargets()
{