CultElementIds.Holds is the one fact a store marks its header by; a test now
drives it directly over a dictionary value, a list of lists, a list under an
unset element and an element, and over unset ids that hold nothing, so a
mutant that skips any of those branches fails.
C# ReadStoreHeader refused a first-slot string without the cultcache.store.
prefix and then ran RequireSingleFileFormat, which refuses every string that
is not v1, v2 or v3 anyway. The prefix check is gone; a non-header string is
refused as a format this runtime does not read.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Soul's mutant deleting the empty-file arm of write_all_unlocked survived: no
test rewrote a zero-byte file. Without the arm the empty file reaches
store_header and is refused as truncated. The test pushes onto a zero-byte
file and expects a v1 store holding the pushed record.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A rewrite read the header of the file it replaces but not whether the file was
a store: C# ReadStoreHeader skipped one value and ignored bytes after it, and
took a non-array value or an array whose first slot is a string that is not a
store header for a headerless legacy file (so did the C# Mesh raw write, which
uses it); TypeScript's pushAll took any non-array value or non-header first slot
for v1; Rust ignored bytes after the array.
The rule, one per runtime: a file is a store on rewrite only when it is exactly
one complete MessagePack array. Bytes after it, a value that is not an array,
and a first slot that is a string but not a header this runtime reads are
refused, and the file is left untouched. An empty file and a legacy envelope
array (first slot not a string, empty included) carry no header and are still
rewritten.
- C#: ReadStoreHeader refuses non-array values, trailing bytes and a non-header
first string, with a NotSupportedException naming which.
- TypeScript: storeHeader (store-format.ts) decides the rewrite header;
@msgpack/msgpack's decode already refuses trailing bytes.
- Rust: store_header(bytes, whole) walks with a cursor; the rewrite path asks
for the whole file, the read paths keep reading as before.
- Python: push_all already reads the whole file (unpackb refuses trailing
bytes, the legacy decoder refuses anything but a list of maps); pinned by a
test.
Tests in each runtime run the same file matrix (truncated, trailing, scalar,
map, string first; empty, empty array, legacy), plus a C# unconditional commit
over a store with trailing bytes and the C# Mesh raw write.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SingleFileBackingStore.PushAll replaced the file without reading it, so a
truncated store or one in a format this runtime does not read was clobbered,
where Rust, TypeScript and Python refuse. PushAll now reads the header of a
non-empty file under the lock before it writes; a file it cannot read throws
and is left as it was. An empty file and a legacy envelope array carry no
header and are still replaced.
The header is read by the store's codec: SingleFileBackingStore gains a
protected virtual ReadStoreHeader (default: read the whole snapshot), and the
MessagePack store overrides it with
CultDocumentMessagePackSerialization.ReadStoreHeader. An unconditional commit
reads the file it rewrites through the same check.
Doc text: last-writer-wins now says it holds only over a file the writer can
read (CultCache.cs PushAll comment, cultcache-persistence-format.md
"Conditional commit is the only safe multi-process write" and writer rule 4,
cultcache-store-composition.md "A plain flush and an unconditional commit are
last-writer-wins").
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A record's HoldsIds was decided by its admission and trusted afterwards, so a
document changed after it was committed, or loaded and changed in place,
flushed under the header its admission-time content needed (Soul P7, P10).
HoldsIds now has one owner, the record's stored form, read when asked:
- a plain record: CultElementIds.Holds over its document, a scan that reads
and refuses nothing;
- a variant: the same scan over each override value, decoded with the codec
the record was built with (the cache's, or the attached cache's at load);
- a variant built with no codec (a store that loaded it with no cache) cannot
see its values and counts as holding an id, so a store never writes an
unmarked header over ids it could not see.
Cut: the admission path that set HoldsIds on plain records, on variants from
their resolved copies, and on flattened records; the internal setter and its
backing flag; IdPlan.HeldIds and IdPlan.HoldsIds. The Mesh typed write reads
Holds off the document it writes. Admission keeps its own id counting
(IdPlan.Count) for IdsInMemoryOnly.
Tests: an id added to a committed or loaded deck after admission marks a
PushAll v3, and taking it away unmarks it; a variant read without a codec
counts as holding ids while the same record read with one does not.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One conflict, at the end of Admit in CultCache.cs. The branch had added the
IdsInMemoryOnly marking of a load's minted ids before Apply; main had turned
the thread's hold into a Hold carrying Changes and Failures and run the
journals right after Apply. Both are kept, in that order: the marking touches
only the records and runs no user code, then Apply mints the Sequences, the
hold records the changes, and the journals run under the gate.
Id planning and Project do not collide. Ids are planned and applied first,
Resolve (variants, then Project, then the name and index rules) reads the
filled documents, land runs, and only then Apply mints. A Project getter that
throws is caught by the same handler that undoes the id plans, so a refused
admission changes no document and mints nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rmp_serde refuses a sequence visitor that stops early, so the prefix-only header read the
previous commit introduced could not decode any store. store_header is now one visitor: the
first slot is the header (None when it is not a string, the legacy envelope array), the rest
is walked, and a truncated file is an error.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ExistingHeader read the whole snapshot and swallowed every failure into "no header": a
truncated v3 file got a v1 raw write and an unknown v9 file was overwritten, where
TypeScript, Python and Rust refuse. CultDocumentMessagePackSerialization.ReadStoreHeader
reads the prefix header, refuses a file that is not one complete MessagePack value or whose
format this runtime cannot read, and the raw write keeps a whole marked file marked.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A CultStoredDocument built by a caller and handed straight to a store never passed an
admission, so HoldsIds stayed false and its ids landed under a v1/v4 header. An admission
still decides it (it counts the ids it fills); a record that never had one reads it off the
document it stores. CacheBackingStore.HeaderFor documents the rule for third-party stores.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
store_header returned None for any bytes it could not walk, so push_all read a truncated
variant store as legacy and overwrote it v1. It now proves the file is one complete
MessagePack value first (an error otherwise, the rewrite refuses and leaves the file), then
reads the header from the prefix; the reader's own header read no longer walks to the end.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A stray, malformed, foreign-id or refused datagram is dropped and counted,
never fatal; a refused reliable frame ends the peer's session instead of
being acked and lost; flush fails once its session ends; one reliable
Accept per session; the TS operation service expires idle sessions and
survives a throwing handler. Four Soul passes; session-generation and
reconnect semantics move to the ack cut.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Also adds a swept-after-handler test for the operation service and states
the Rust receipt comment exactly: only a peer Disconnect invalidates receipts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The behaviour was already a refusal that leaves the file untouched; now a test fails if a rewrite
ever reads the disk header less strictly than a load does.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
push_all reads the file and reuses store_header and readable_store_format, the same pair a load
uses. The 96-byte prefix parser it replaces misread a str16 header and any str8 header past the
prefix as v1 and overwrote a store it could not read. Pins both encodings refused with the file
untouched.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CacheBackingStore.HeaderFor(holdsIds, existingHeader, wholeStore, directoryStore) replaces FormatFor,
DirectoryFormatFor and the writers' private marker logic. A writer that rewrites the whole store
decides by content; a writer that touches part of it keeps a header already marked. Flush,
unconditional and conditional commit, the directory store and CultMesh's single-file write all call
it. A typed CultMesh write replaces the file with a document it can see, so it decides by content;
a raw payload is opaque and keeps a marked file marked.
A record's HoldsIds is what its stored form persists: a load keeps a variant's delta as read, so
the ids a load mints in an override exist only in memory and no longer mark the store. A
conditional commit onto the file no longer reads this cache's view of what the file held, so a
stale view cannot over-mark it. IdPlan.HoldsIds is derived from HeldIds and the fills.
Reverts InternalsVisibleTo GameCult.Caching.MessagePack and the public DirectoryFormatV4/V5
constants: the directory store reaches the decision through the protected member it inherits.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The repeated-Connect answer applies its acknowledgement field in every runtime;
no test failed when a runtime stopped doing so. The TS replaced-peer test now
lets client A receive its Accept before A sends.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Disconnect left the session's pending writes and scope alone, and the
transport's Invalidated status leaned on disconnect_reason, which connect()
clears. After a reconnect the receipt returned to Pending and could become
Acknowledged in the new session. The session now rotates its scope and drops
what it still owed on Disconnect, so the session is the one owner of
Invalidated and the transport override is gone.
A flush records the session scope it started in and fails when a Connect that
replaces the peer rotates it, instead of resolving on the emptied queue.
Tests: Disconnect-then-reconnect receipt, stale end reason after a new Connect
is accepted, flush across a replaced peer, and a repeated Connect's ack field.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A server-mode transport treated a Connect from a new endpoint as a repeated
Connect once the session was connected: the new client got an Ack and never an
Accept, and the old session never ended. Only a Connect from the accepted
endpoint repeats; another endpoint resets the session and gets its Accept.
A flush recorded nothing about the session it started in, so a disconnect
listener that reconnected cleared the end reason before the 5 ms poll saw it.
Sessions now carry a generation; a flush fails if its generation ended.
The operation service's idle sweep deleted a session whose handler was still
running, losing the response. A session with a handler running, or one that
finished a handler within the idle window, is not idle.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Imagination pass 1. Probes found acknowledged ordered frames lost in live
sessions in every runtime; the map derives Delivered at the sender from a
correct receiver watermark. Operator questions Q-A1..Q-A5 open.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
push_all overwrote a v9, variant-v2 or variant store with a v1 file, dropping its variants; TypeScript and Python
refuse. The rewrite header now comes from the file's own prefix and shares one readable-format rule with the
decoder. TypeScript pullAll resets its header when the file is gone, empty or legacy, as Python does.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A flattened variant's HoldsIds is read off the document re-resolved against the post-batch base, not carried from the
variant or from the document the flatten was planned against. The conditional commit onto a file decides by the same
rule as the unconditional commit (ids in the records the file will hold, ids only this cache minted excepted, a marked
file staying marked). The directory manifest's header choice moves beside FormatFor. Pins the three-batch flatten
probe on the commit and flush paths, the conditional commit, the last id leaving, an id-bearing override that is not
the last, and a failed CultMesh write giving its ids back.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The hub test counts replaced sessions, since a fresh session repeats the same
Accept sequence and the distinct-sequence count could not see it. The TS test
requires an Ack, not a new Accept, once the Accept was acknowledged. A peer
Disconnect is pinned as an ended session for flush and receipts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
AddJournal runs under the gate right after Apply, once per admission that
changed something, before observers. Soul: merge with follow-ups; the
dispose and catch-path fixes landed. Cut 4 must decide what a throwing
journal means for replicas and whether a journal needs the change origin.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FlushReliable throws for writes an ended session forgot. A server-mode
transport and the listener answer a Connect from the peer they already hold
with AnswerRepeatedConnect: the Accept still owed, or an Ack, queueing
nothing. The listener replaced the peer session on every Connect, reusing
sequences and never reporting PeerDisconnected; it now replaces only on a
different connect payload and reports the old peer, as the Rust hub does.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
flush now rejects for writes an ended session forgot. acceptConnect on a
connected session delegates to the new answerRepeatedConnect, so a repeated
Connect re-sends the Accept still owed, or an Ack once it was acknowledged,
and queues nothing; the operation service no longer builds a session per
Connect. The operation service dropped sessions never, so 64 abandoned
clients locked every new one out: a session idle for sessionIdleTimeoutMs
(default 30 s, the document server's session_idle_timeout) is now dropped.
A request whose handler rejects ends its session like a transport refusal,
so the invoker learns promptly and a retransmit is not acknowledged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
flush_reliable raised nothing for writes the ended session had forgotten; it
now raises ConnectionError with the reason. Server mode answered every
repeated Connect by queueing a fresh reliable Accept; a Connect from the peer
already accepted is now answered with the Accept still owed and queues
nothing, as the Rust hub does.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A session that refused a packet, or was disconnected, forgot its
unacknowledged writes, so flush_reliable returned success for writes the peer
never acknowledged. The transport now fails the flush with the reason and
reads such writes as Invalidated. Reset no longer reissues sequence numbers or
fragment ids: a late duplicate Connect in server mode reused sequences the
peer had seen. end_refused_session owns the reset-then-goodbye ordering for
the hub, the transport and the document server, which used to end a refusing
session silently.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
AddJournal's Disposable.Create counted a refused dispose (from inside a journal
or another cache's hold) as done, leaving the journal registered forever. A
private registration now flips only after the removal succeeded. Held's catch
path dropped the hold's journal failures; it now throws an AggregateException
with the body's exception first. The store-composition contract states exactly
when the second getter read runs on a conditional commit. Adds the direct test
that a load is journaled.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Attaching a store with no file never calls Loaded, so it did not exercise the
rule; a commit removing a key the cache does not hold admits nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A disposed R3 subscription ignores OnNext, so a no-op removal from the cache's
observer array cannot be seen by what a receiver gets; the test now checks the
cache stops holding the subscriber. An admission that changed nothing (a store
attached with no records) is not journaled.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fragment-id-0 test is caught by the session-refusal branch, so a frame that
fails inside the handler needs its own test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TS pushAll wrote the header its instance last read (v1 when fresh) without reading the
file. It now reads the on-disk header first, as push and delete do through pullAll. Python
resets its remembered header when the file is gone or empty, so the disk decides.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CultNetRudpPacketCodec.Decode did checked((int)length), so a datagram declaring
0x80000000 threw OverflowException, which no receive loop catches; the length
is now compared as a long and takes the ordinary malformed-packet drop.
Server mode claimed its endpoint on the first datagram of any type; only a
Connect claims it now. A session that refuses a packet has already recorded
its reliable sequence, so the socket transport and the listener end it: the
session forgets the peer (so the goodbye does not acknowledge the refused
frame), the peer gets a Disconnect, the listener drops the peer and raises
PeerDisconnected, and a server-mode transport forgets its endpoint.
CultNetRudpSession gains ResetPeerState, and the session and the listener
reject an InitialSequence of uint.MaxValue.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A store is written v3 (v5 for a directory) only when a record in it holds a non-empty
element id. CultElementIds.Plan reports that as IdPlan.HoldsIds; the admission stamps it
on the stored record and every store reads it. The descriptor's CarriesElementIds is
deleted. A file already marked on disk stays marked. CultPersistedStoreSnapshot.FormatFor
is the one header decision.
Plan refuses one element object sitting twice in a list at write, naming the record and
list, instead of writing both ids and refusing the store at load.
CultMesh single-file write mints, refuses and marks as a cache write does; the read applies
the load-time plan, so a write and a load refuse the same things.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CultCache.AddJournal registers an ordered admission journal. It is called under
the gate right after Apply, once per admission that changed something, with the
admission's changes (dependents included) in Sequence order, before any
observer. A journal that throws joins the hold's failures: the admission stands,
the other journals run, the change is published and the writer hears of it
afterwards. Any hold entered while a journal runs throws. A hold now carries its
own changes and failures. Order stays data; no delivery thread or scheduler is
added.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
