C#: a flush refuses to overwrite a store file whose header it cannot read

SingleFileBackingStore.PushAll replaced the file without reading it, so a
truncated store or one in a format this runtime does not read was clobbered,
where Rust, TypeScript and Python refuse. PushAll now reads the header of a
non-empty file under the lock before it writes; a file it cannot read throws
and is left as it was. An empty file and a legacy envelope array carry no
header and are still replaced.

The header is read by the store's codec: SingleFileBackingStore gains a
protected virtual ReadStoreHeader (default: read the whole snapshot), and the
MessagePack store overrides it with
CultDocumentMessagePackSerialization.ReadStoreHeader. An unconditional commit
reads the file it rewrites through the same check.

Doc text: last-writer-wins now says it holds only over a file the writer can
read (CultCache.cs PushAll comment, cultcache-persistence-format.md
"Conditional commit is the only safe multi-process write" and writer rule 4,
cultcache-store-composition.md "A plain flush and an unconditional commit are
last-writer-wins").

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

src/GameCult.Caching.MessagePack/CultDocumentMessagePackSerialization.cs

@@ -563,6 +563,8 @@ public class SingleFileMessagePackBackingStore : SingleFileBackingStore
         return snapshot;
     }
 
+    protected override string? ReadStoreHeader(byte[] data) => CultDocumentMessagePackSerialization.ReadStoreHeader(data);
+
     protected override byte[] SerializePayload(object document)
     {
         return CultDocumentMessagePackSerialization.SerializeUntyped(document, document.GetType(), Registry);

src/GameCult.Caching/Contracts/cultcache-persistence-format.md

@@ -57,7 +57,9 @@ an unconditional commit are last-writer-wins and write the same bytes: a single
 file is replaced by the writer's whole view, a directory manifest takes the
 writer's staged keys over the current manifest. An unconditional commit also
 persists writes staged earlier and leaves the store clean. Processes sharing a
-store must all use conditional commit.
+store must all use conditional commit. The last writer wins only over a file it
+can read: a single-file flush or commit refuses a file whose header it cannot
+read (truncated, or a format it does not read) and leaves the file as it was.
 
 A store that fails to hydrate on open (corrupt bytes, unresolvable schema)
 makes the open throw and is left byte-identical; it is never silently
@@ -434,7 +436,8 @@ The v1 concurrent single-file policy is:
 3. Writers re-read the current snapshot after taking the lock.
 4. Writers using conditional commit evaluate their conditions against the
    latest snapshot and write nothing if one fails. A plain flush and an
-   unconditional commit compare nothing and are last-writer-wins.
+   unconditional commit compare nothing and are last-writer-wins over a file
+   they can read; one whose header they cannot read is refused untouched.
 5. Writers write a temp file, flush it, atomically replace the `.cc` file, and
    release the lock.
 

src/GameCult.Caching/Contracts/cultcache-store-composition.md

@@ -239,7 +239,9 @@ replaces; writers bump a minted timestamp by one tick when it is not later.
 single-file stores, per key for directory stores.** Both run
 under the same lock, so two writers never interleave bytes, but they compare
 nothing. Only a conditional commit (`Expect` or `ExpectUnchanged`) protects
-against another writer; processes sharing a store must all use it.
+against another writer; processes sharing a store must all use it. A writer
+wins only over a file it can read: a single-file store refuses to replace a file
+whose header it cannot read, and leaves it as it was.
 
 - A single-file store writes this cache's whole view of the file: a record
   another writer added since this cache last pulled is gone.

src/GameCult.Caching/CultCache.cs

@@ -3417,6 +3417,15 @@ namespace GameCult.Caching
         protected FileInfo FileInfo { get; }
         protected abstract byte[] SerializeSnapshot(CultPersistedStoreSnapshot snapshot);
         protected abstract CultPersistedStoreSnapshot DeserializeSnapshot(byte[] data);
+
+        /// <summary>
+        /// The format header of a non-empty store file this store is about to replace: null for a legacy file that carries none,
+        /// the header otherwise. It throws for bytes that are not exactly one store this runtime reads, so a rewrite never
+        /// overwrites a file it cannot see. The default reads the whole snapshot; a codec that can read the header alone
+        /// overrides it.
+        /// </summary>
+        protected virtual string? ReadStoreHeader(byte[] data) => DeserializeSnapshot(data).FormatVersion;
+
         protected abstract byte[] SerializePayload(object document);
         protected abstract object DeserializePayload(Type documentType, byte[] payload);
 
@@ -3484,7 +3493,8 @@ namespace GameCult.Caching
         }
 
         // A plain flush writes this store's whole view under the lock: two writers never interleave bytes, but the last
-        // one wins. Processes sharing a store must use conditional commit.
+        // one wins over any file this runtime can read. Processes sharing a store must use conditional commit. A file it
+        // cannot read (truncated, trailing bytes, not a store, a format it does not read) is refused and left as it is.
         public override void PushAll()
         {
             ThrowIfReadOnly();
@@ -3492,6 +3502,8 @@ namespace GameCult.Caching
             {
                 using (AcquireLock(wait: true))
                 {
+                    if (ReadDisk() is { Length: > 0 } onDisk)
+                        ReadStoreHeader(onDisk);
                     WriteSnapshot(
                         Entries.Values.Select(entry => ToPersistedRecord(entry, SerializePayload)),
                         Entries.Values.Select(entry => entry.Descriptor.ToCatalogEntry()),
@@ -3517,7 +3529,7 @@ namespace GameCult.Caching
             if (fileLock == null)
                 return CultCommitOutcome.Contended;
 
-            var disk = ReadSnapshot() ?? new CultPersistedStoreSnapshot();
+            var disk = ReadSnapshot(rewriting: true) ?? new CultPersistedStoreSnapshot();
             if (!request.ConditionsHold(disk.Records, Entries.Values))
                 return CultCommitOutcome.Mismatch;
 
@@ -3586,14 +3598,25 @@ namespace GameCult.Caching
             Judging!(arriving, departing);
         }
 
-        private CultPersistedStoreSnapshot? ReadSnapshot()
+        // A commit rewrites the file it read, so it reads it as a rewrite does: the header proves the bytes are one store first.
+        private CultPersistedStoreSnapshot? ReadSnapshot(bool rewriting = false)
+        {
+            var bytes = ReadDisk();
+            if (bytes == null)
+                return null;
+            if (rewriting && bytes.Length > 0)
+                ReadStoreHeader(bytes);
+            return DeserializeSnapshot(bytes);
+        }
+
+        private byte[]? ReadDisk()
         {
             FileInfo.Refresh();
             if (!FileInfo.Exists)
                 return null;
             try
             {
-                return DeserializeSnapshot(ReadAllBytesShared(FileInfo.FullName));
+                return ReadAllBytesShared(FileInfo.FullName);
             }
             catch (FileNotFoundException)
             {

tests/GameCult.Caching.Tests/ElementIdTests.cs

@@ -785,6 +785,38 @@ namespace GameCult.Caching.Tests
             Assert.That(HeaderOf(path), Is.EqualTo("cultcache.store.v3"), "read without one, it cannot be seen, so it is marked");
         }
 
+        // A flush replaces the whole file, but never one whose header it cannot read (Rust, TypeScript and Python refuse the same):
+        // it throws and the file is left as it was. An empty file and a legacy envelope array carry no header and are replaced.
+        [TestCase("truncated", false)]
+        [TestCase("v9", false)]
+        [TestCase("empty", true)]
+        [TestCase("legacy", true)]
+        public void AFlushRefusesAFileWhoseHeaderItCannotReadAndLeavesItUntouched(string file, bool replaced)
+        {
+            var path = PathOf($"flush-over-{file}.cc");
+            using var cache = Open(path);
+            cache.Commit(batch => batch.Upsert(typeof(IdDeck), Deck("d"), new CultRecordKey("d")));
+            var whole = File.ReadAllBytes(path);
+            var bytes = file switch
+            {
+                "truncated" => whole[..^1],
+                "v9" => MessagePackSerializer.Serialize(new object[] { "cultcache.store.v9", Array.Empty<object>(), Array.Empty<object>() }),
+                "empty" => Array.Empty<byte>(),
+                _ => MessagePackSerializer.Serialize(new object[] { new Dictionary<string, object> { ["key"] = "old", ["type"] = "old", ["storedAt"] = "then" } })
+            };
+            File.WriteAllBytes(path, bytes);
+
+            if (replaced)
+            {
+                cache.BackingStores[0].PushAll();
+                Assert.That(HeaderOf(path), Is.EqualTo("cultcache.store.v3"), "the file is this store's view now");
+                return;
+            }
+
+            Assert.Throws<NotSupportedException>(() => cache.BackingStores[0].PushAll());
+            Assert.That(File.ReadAllBytes(path), Is.EqualTo(bytes), "the file was rewritten");
+        }
+
         [Test]
         public void ADirectoryStoreIsMarkedByTheIdsItWritesAndAnEmptyDeckDoesNotMarkIt()
         {