A record's HoldsIds was decided by its admission and trusted afterwards, so a
document changed after it was committed, or loaded and changed in place,
flushed under the header its admission-time content needed (Soul P7, P10).
HoldsIds now has one owner, the record's stored form, read when asked:
- a plain record: CultElementIds.Holds over its document, a scan that reads
and refuses nothing;
- a variant: the same scan over each override value, decoded with the codec
the record was built with (the cache's, or the attached cache's at load);
- a variant built with no codec (a store that loaded it with no cache) cannot
see its values and counts as holding an id, so a store never writes an
unmarked header over ids it could not see.
Cut: the admission path that set HoldsIds on plain records, on variants from
their resolved copies, and on flattened records; the internal setter and its
backing flag; IdPlan.HeldIds and IdPlan.HoldsIds. The Mesh typed write reads
Holds off the document it writes. Admission keeps its own id counting
(IdPlan.Count) for IdsInMemoryOnly.
Tests: an id added to a committed or loaded deck after admission marks a
PushAll v3, and taking it away unmarks it; a variant read without a codec
counts as holding ids while the same record read with one does not.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Cut 4a: brings the publication tests and the door table from
hands/cultnet-publish-commits at 28248ebe. The reorder buffer, drainer and
prediction gate die in 4b.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Adds the pass 4 probes, the journal-floor rule, and the re-taken Cuts 4-6, salvage and ledger.
Records Q-P5 A: a primary that cannot log a committed change throws CultNetShardLogException.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One conflict, at the end of Admit in CultCache.cs. The branch had added the
IdsInMemoryOnly marking of a load's minted ids before Apply; main had turned
the thread's hold into a Hold carrying Changes and Failures and run the
journals right after Apply. Both are kept, in that order: the marking touches
only the records and runs no user code, then Apply mints the Sequences, the
hold records the changes, and the journals run under the gate.
Id planning and Project do not collide. Ids are planned and applied first,
Resolve (variants, then Project, then the name and index rules) reads the
filled documents, land runs, and only then Apply mints. A Project getter that
throws is caught by the same handler that undoes the id plans, so a refused
admission changes no document and mints nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The audio parity validator now has a refusal row for every field in its
identity and timing disjunctions, and both parity validators have tests for
the inclusive edges: 256 shards, the video block that would make 257, and a
deadline equal to the base presentation time. cargo-mutants over the diff
had left these operators standing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rmp_serde refuses a sequence visitor that stops early, so the prefix-only header read the
previous commit introduced could not decode any store. store_header is now one visitor: the
first slot is the header (None when it is not a string, the legacy envelope array), the rest
is walked, and a truncated file is an error.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ExistingHeader read the whole snapshot and swallowed every failure into "no header": a
truncated v3 file got a v1 raw write and an unknown v9 file was overwritten, where
TypeScript, Python and Rust refuse. CultDocumentMessagePackSerialization.ReadStoreHeader
reads the prefix header, refuses a file that is not one complete MessagePack value or whose
format this runtime cannot read, and the raw write keeps a whole marked file marked.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A CultStoredDocument built by a caller and handed straight to a store never passed an
admission, so HoldsIds stayed false and its ids landed under a v1/v4 header. An admission
still decides it (it counts the ids it fills); a record that never had one reads it off the
document it stores. CacheBackingStore.HeaderFor documents the rule for third-party stores.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
store_header returned None for any bytes it could not walk, so push_all read a truncated
variant store as legacy and overwrote it v1. It now proves the file is one complete
MessagePack value first (an error otherwise, the rewrite refuses and leaves the file), then
reads the header from the prefix; the reader's own header read no longer walks to the end.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A stray, malformed, foreign-id or refused datagram is dropped and counted,
never fatal; a refused reliable frame ends the peer's session instead of
being acked and lost; flush fails once its session ends; one reliable
Accept per session; the TS operation service expires idle sessions and
survives a throwing handler. Four Soul passes; session-generation and
reconnect semantics move to the ack cut.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Also adds a swept-after-handler test for the operation service and states
the Rust receipt comment exactly: only a peer Disconnect invalidates receipts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The behaviour was already a refusal that leaves the file untouched; now a test fails if a rewrite
ever reads the disk header less strictly than a load does.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
push_all reads the file and reuses store_header and readable_store_format, the same pair a load
uses. The 96-byte prefix parser it replaces misread a str16 header and any str8 header past the
prefix as v1 and overwrote a store it could not read. Pins both encodings refused with the file
untouched.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CacheBackingStore.HeaderFor(holdsIds, existingHeader, wholeStore, directoryStore) replaces FormatFor,
DirectoryFormatFor and the writers' private marker logic. A writer that rewrites the whole store
decides by content; a writer that touches part of it keeps a header already marked. Flush,
unconditional and conditional commit, the directory store and CultMesh's single-file write all call
it. A typed CultMesh write replaces the file with a document it can see, so it decides by content;
a raw payload is opaque and keeps a marked file marked.
A record's HoldsIds is what its stored form persists: a load keeps a variant's delta as read, so
the ids a load mints in an override exist only in memory and no longer mark the store. A
conditional commit onto the file no longer reads this cache's view of what the file held, so a
stale view cannot over-mark it. IdPlan.HoldsIds is derived from HeldIds and the fills.
Reverts InternalsVisibleTo GameCult.Caching.MessagePack and the public DirectoryFormatV4/V5
constants: the directory store reaches the decision through the protected member it inherits.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The repeated-Connect answer applies its acknowledgement field in every runtime;
no test failed when a runtime stopped doing so. The TS replaced-peer test now
lets client A receive its Accept before A sends.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Disconnect left the session's pending writes and scope alone, and the
transport's Invalidated status leaned on disconnect_reason, which connect()
clears. After a reconnect the receipt returned to Pending and could become
Acknowledged in the new session. The session now rotates its scope and drops
what it still owed on Disconnect, so the session is the one owner of
Invalidated and the transport override is gone.
A flush records the session scope it started in and fails when a Connect that
replaces the peer rotates it, instead of resolving on the emptied queue.
Tests: Disconnect-then-reconnect receipt, stale end reason after a new Connect
is accepted, flush across a replaced peer, and a repeated Connect's ack field.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A server-mode transport treated a Connect from a new endpoint as a repeated
Connect once the session was connected: the new client got an Ack and never an
Accept, and the old session never ended. Only a Connect from the accepted
endpoint repeats; another endpoint resets the session and gets its Accept.
A flush recorded nothing about the session it started in, so a disconnect
listener that reconnected cleared the end reason before the 5 ms poll saw it.
Sessions now carry a generation; a flush fails if its generation ended.
The operation service's idle sweep deleted a session whose handler was still
running, losing the response. A session with a handler running, or one that
finished a handler within the idle window, is not idle.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Imagination pass 1. Probes found acknowledged ordered frames lost in live
sessions in every runtime; the map derives Delivered at the sender from a
correct receiver watermark. Operator questions Q-A1..Q-A5 open.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
push_all overwrote a v9, variant-v2 or variant store with a v1 file, dropping its variants; TypeScript and Python
refuse. The rewrite header now comes from the file's own prefix and shares one readable-format rule with the
decoder. TypeScript pullAll resets its header when the file is gone, empty or legacy, as Python does.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A flattened variant's HoldsIds is read off the document re-resolved against the post-batch base, not carried from the
variant or from the document the flatten was planned against. The conditional commit onto a file decides by the same
rule as the unconditional commit (ids in the records the file will hold, ids only this cache minted excepted, a marked
file staying marked). The directory manifest's header choice moves beside FormatFor. Pins the three-batch flatten
probe on the commit and flush paths, the conditional commit, the last id leaving, an id-bearing override that is not
the last, and a failed CultMesh write giving its ids back.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The hub test counts replaced sessions, since a fresh session repeats the same
Accept sequence and the distinct-sequence count could not see it. The TS test
requires an Ack, not a new Accept, once the Accept was acknowledged. A peer
Disconnect is pinned as an ended session for flush and receipts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
AddJournal runs under the gate right after Apply, once per admission that
changed something, before observers. Soul: merge with follow-ups; the
dispose and catch-path fixes landed. Cut 4 must decide what a throwing
journal means for replicas and whether a journal needs the change origin.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FlushReliable throws for writes an ended session forgot. A server-mode
transport and the listener answer a Connect from the peer they already hold
with AnswerRepeatedConnect: the Accept still owed, or an Ack, queueing
nothing. The listener replaced the peer session on every Connect, reusing
sequences and never reporting PeerDisconnected; it now replaces only on a
different connect payload and reports the old peer, as the Rust hub does.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
flush now rejects for writes an ended session forgot. acceptConnect on a
connected session delegates to the new answerRepeatedConnect, so a repeated
Connect re-sends the Accept still owed, or an Ack once it was acknowledged,
and queues nothing; the operation service no longer builds a session per
Connect. The operation service dropped sessions never, so 64 abandoned
clients locked every new one out: a session idle for sessionIdleTimeoutMs
(default 30 s, the document server's session_idle_timeout) is now dropped.
A request whose handler rejects ends its session like a transport refusal,
so the invoker learns promptly and a retransmit is not acknowledged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
flush_reliable raised nothing for writes the ended session had forgotten; it
now raises ConnectionError with the reason. Server mode answered every
repeated Connect by queueing a fresh reliable Accept; a Connect from the peer
already accepted is now answered with the Accept still owed and queues
nothing, as the Rust hub does.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A session that refused a packet, or was disconnected, forgot its
unacknowledged writes, so flush_reliable returned success for writes the peer
never acknowledged. The transport now fails the flush with the reason and
reads such writes as Invalidated. Reset no longer reissues sequence numbers or
fragment ids: a late duplicate Connect in server mode reused sequences the
peer had seen. end_refused_session owns the reset-then-goodbye ordering for
the hub, the transport and the document server, which used to end a refusing
session silently.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
AddJournal's Disposable.Create counted a refused dispose (from inside a journal
or another cache's hold) as done, leaving the journal registered forever. A
private registration now flips only after the removal succeeded. Held's catch
path dropped the hold's journal failures; it now throws an AggregateException
with the body's exception first. The store-composition contract states exactly
when the second getter read runs on a conditional commit. Adds the direct test
that a load is journaled.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A block with every data shard present passes through the code as a no-op
and yields no recovered records, so the early return in both recovery
functions did nothing the code did not already do. Mutation showed both
survive removal. The audio payload-length rule is now pinned by its
message, since the codec's own shard-size error would otherwise mask its
removal.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds the policy fields that must be non-zero, the inclusive shard limit at
the decoder for video and audio parity, admission statistics, seeded jitter
bounds, exact seeded loss, and arrival order for datagrams due together.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pins the block-boundary chunk and last-chunk length refusals in video
recovery, the audio recovery refusals (packets one either side of the block,
another session, wrong length, parity from different blocks), the remaining
audio block identity rules (session, stream, timebase, duration), and the
data-parity alternation inside a block.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cultnet_impair is a deterministic bidirectional UDP proxy for pressure
testing real CultNet RUDP sessions with seeded loss, burst, duplicate,
reorder, jitter and stall schedules. It is ported from the Odin donor
(80adfd3) as an example, not library surface. The decision policy (Profile,
Scheduler, seeded RNG) is split from the sockets into policy.rs, which the
proxy and the tests both compile, so a schedule has one owner. The ten
donor profiles live in tests/impairment/.
tests/impairment_policy.rs keeps the donor's six deterministic policy tests
(the burst test now pins the exact drops rather than a count) and adds
profile loading and rejection. tests/media_fec_loss.rs sends protected video
frames through those schedules with no sockets and asserts the guarantee: a
block recovers exactly when at most m of its records were lost, recovered
chunks equal what was sent, and a burst that the send order spreads under m
per block loses no frame. The recovered-frame ratios are printed, not
asserted; tests/impairment/loss-matrix.txt is that output.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The codec owns the erasure math for both ends: protect_video_frame and
protect_audio_block on the producer, recover_video_block and
recover_audio_block on the consumer. Each block is systematic Reed-Solomon
over GF(2^8) (reed-solomon-erasure 6, default features), named on every
parity record by the scheme id rs-gf256-v1. Any <= m erasures recover
exactly; more report BeyondRepair. Video frames split into ceil(n/16)
near-equal blocks with m = max(2, ceil(k/4)); audio is 4+2. A frame's
records come back in send order, rotating one record from each block so a
burst lands as one loss per block.
Deleted first: gamecult.media_video_parity_shard.v2 and its XOR-stripe
validator, and the reliable "audio" channel (profile entry, send-options
arm, constant, tests). Audio now rides "media" with parity.
Added: gamecult.media_video_parity_shard.v3 (block geometry, scheme id,
shard length), gamecult.media_audio_parity_shard.v1, the AudioParity wire
variant with validation, and the parity record key now names the block.
Every shard is bounded at MEDIA_FEC_MAX_SHARD_PAYLOAD_BYTES (720) so its
wrapped record fits MEDIA_FEC_MAX_WIRE_BYTES (1431), measured with
identifiers of up to 32 bytes.
Tests pin: exhaustive recovery for every shipped (k, m) and 4+2, m+1
refusal, known-answer parity bytes for both paths, the datagram bound on
every record kind, send order, deadline carriage and decoder validation.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Attaching a store with no file never calls Loaded, so it did not exercise the
rule; a commit removing a key the cache does not hold admits nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A disposed R3 subscription ignores OnNext, so a no-op removal from the cache's
observer array cannot be seen by what a receiver gets; the test now checks the
cache stops holding the subscriber. An admission that changed nothing (a store
attached with no records) is not journaled.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fragment-id-0 test is caught by the session-refusal branch, so a frame that
fails inside the handler needs its own test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TS pushAll wrote the header its instance last read (v1 when fresh) without reading the
file. It now reads the on-disk header first, as push and delete do through pullAll. Python
resets its remembered header when the file is gone or empty, so the disk decides.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
