cultcache-py: delete and is_writable share one judge; pin reload replacement and the consent guards

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Libby 2026-11-12 03:28:35 +00:00
commit 269f0ad6a9
3 changed files with 95 additions and 11 deletions

View file

@ -9,7 +9,7 @@
### Breaking
- A record whose payload carries a slot past its reader's declared slots, at any depth (the document, a nested record, a list element, a union arm, a unit arm), is read, held read-only, and every write or removal of its key (`put`, `put_global`, `update`, `put_envelope`, `put_envelopes`, `delete`, `delete_global`) is refused with `SchemaConflictError` before any store is touched (`cultlib-gaps:ruling:extra-trailing-slots`). Reading dropped those slots silently and the next write shed them. `SchemaConflictError.unread_slots` is an `UnreadSlots(type_name, slot_count, declared_count, occurrences, depth0_only)` and the message names the type and both counts, never a slot value. A depth-0 mark is consented, and the record writable, only when the record is stored under an id the document lists in `compatible_schema_ids` and that id's catalog entry has slots past the document's, covering the mark's slot count (`cultlib-gaps:ruling:slot-mark-consent`); a nested mark, or a mark under the document's own id, keeps the record read-only whatever is declared. Missing trailing slots keep their declared defaults. An untouched marked record keeps its stored bytes through every other write. Added `CultCache.is_writable(document, key)` (advisory; the write still decides: it also answers False for a foreign-carried key and for a key held as a type the document does not own), `DocumentDefinition.decode_payload_marked`, and `UnreadSlots`. A document with a custom `decode` (no slot codec) never marks. A store holding a variant record is still unreadable here. Implies a minor bump while the version is 0.x (0.3.0 to 0.4.0): code that wrote back a record from a newer writer now gets `SchemaConflictError`.
- A record whose payload carries a slot past its reader's declared slots, at any depth (the document, a nested record, a list element, a union arm, a unit arm), is read, held read-only, and every write or removal of its key (`put`, `put_global`, `update`, `put_envelope`, `put_envelopes`, `delete`, `delete_global`) is refused with `SchemaConflictError` before any store is touched (`cultlib-gaps:ruling:extra-trailing-slots`). Reading dropped those slots silently and the next write shed them. `SchemaConflictError.unread_slots` is an `UnreadSlots(type_name, slot_count, declared_count, occurrences, depth0_only)` and the message names the type and both counts, never a slot value. A depth-0 mark is consented, and the record writable, only when the record is stored under an id the document lists in `compatible_schema_ids` and that id's catalog entry has slots past the document's, covering the mark's slot count (`cultlib-gaps:ruling:slot-mark-consent`); a nested mark, or a mark under the document's own id, keeps the record read-only whatever is declared. Missing trailing slots keep their declared defaults. An untouched marked record keeps its stored bytes through every other write. Added `CultCache.is_writable(document, key)` (advisory; the write still decides: it also answers False for a foreign-carried key, for a key held as a type the document does not own, and for a global document at a non-global key; `delete` asks the same judges, so removing a key held as another type is refused with `SchemaConflictError`, where it returned silently and removed nothing), `DocumentDefinition.decode_payload_marked`, and `UnreadSlots`. A document with a custom `decode` (no slot codec) never marks. A store holding a variant record is still unreadable here. Implies a minor bump while the version is 0.x (0.3.0 to 0.4.0): code that wrote back a record from a newer writer now gets `SchemaConflictError`.
- A cache holds one record per key whatever its type (`variants:ruling:cache-refuses-retype`). A `put`, `put_envelope`, `put_envelopes` or `put_global` of a second type under a key the cache holds, or under a key a store carries foreign, is refused with `SchemaConflictError` naming the key, even with no store, and changes nothing; retype a key by deleting it, then putting the new type. A write may replace another registered type's record only when the written type owns its schema id (declares it or lists it compatible) and both types share one home store. A load that reads one key from two stores, whatever the types, is refused with `CultCacheError` naming the key and both types, and admits nothing. A store that holds a record no registered type claims now loads and is carried: the cache does not serve it, the store keeps its bytes through every write, `delete` or `put` at its key is refused with `SchemaConflictError`, and registering a type that declares its schema id claims it at the next load. It raised `CultCacheError` "Unknown persisted document type". Two global types share the key `__global__`, so a second global type in one cache is now refused, and a store that already holds two global types at `__global__` (a JSONL store written by caches that never pulled could produce one) no longer loads: the pull raises `CultCacheError` and admits nothing. C# refuses a second persisted global on pull the same way (`cultcache-store-composition.md`, "Globals never invented"); repair such a file by deleting one of the two records with the runtime that wrote it.
- Added `foreign_records()` and the frozen `ForeignRecord(key, schema_id, schema_name, stored_at)`, sorted by key in UTF-16 code-unit order. `stores.retyped` and `stores.written_owns` lose their underscore: the cache judge and the single-file store's commit guard ask one predicate.

View file

@ -638,15 +638,23 @@ class CultCache:
record this document does not own. Advisory (the write still decides): it answers at call time, and a key nothing holds is
writable."""
self._assert_registered(document)
entry = document.catalog_entry()
probe = CultCacheEnvelope.create(key=key, type=document.type, payload=b"", schema_id=entry.schema_id, catalog_entry=entry)
try:
self._refuse_unread([key])
self._refuse_retype(document, [probe])
except SchemaConflictError:
self._refuse_at(document, key)
except (SchemaConflictError, CultCacheError):
return False
return True
def _refuse_at(self, document: DocumentDefinition[Any], key: str) -> None:
"""The judges a write or removal of the key by this document passes before any store is touched, asked by `is_writable` and
by `delete`: a global document writes only its global key, a record held read-only is never replaced, and a key held as
another type's record or carried foreign is refused (`_refuse_retype`)."""
if document.global_document and key != GLOBAL_KEY:
raise CultCacheError(f"Global document {document.type} must use key {GLOBAL_KEY}")
self._refuse_unread([key])
entry = document.catalog_entry()
probe = CultCacheEnvelope.create(key=key, type=document.type, payload=b"", schema_id=entry.schema_id, catalog_entry=entry)
self._refuse_retype(document, [probe])
def _refuse_unread(self, keys: list[str]) -> None:
"""The one check every write and removal asks before any store is touched."""
for key in keys:
@ -719,11 +727,7 @@ class CultCache:
def _delete(self, document: DocumentDefinition[Any], key: str) -> None:
self._assert_registered(document)
self._refuse_unread([key])
if key not in self._state.envelopes.get(document.type, {}):
carried = next((record for record in self._state.foreign if record.key == key), None)
if carried is not None:
raise self._foreign_refusal(carried)
self._refuse_at(document, key)
store = self._store_for_type(document.type)
legacy_key = self._state.legacy_global_keys.get(document.type) if key == GLOBAL_KEY else None
if store is not None:

View file

@ -8,6 +8,7 @@ import unittest
from pathlib import Path
import msgpack
from cultcache_py.cache import CultCacheError
from cultcache_py import (
CultCache,
@ -340,6 +341,85 @@ class StoreRowTests(unittest.TestCase):
cache.pull_all_backing_stores()
self.assertFalse(cache.is_writable(declaring_document, "consented"))
def test_a_removal_is_refused_where_is_writable_says_no(self) -> None:
path, cache = self._store("extra-slot-store.cc")
before = path.read_bytes()
other = define_database_entry_type("tests.other", [database_entry_field("a", 0, codec=t.string)], name="a")
cache2 = _open(self.tmp / "extra-slot-store.cc", surface_document, slot_unit_document, other)
self.assertFalse(cache2.is_writable(other, "clean"))
with self.assertRaises(SchemaConflictError):
cache2.delete(other, "clean")
self.assertEqual(path.read_bytes(), before)
self.assertIsNotNone(cache2.get(surface_document, "clean"))
self.assertTrue(cache2.is_writable(other, "nothing-here"))
def test_a_global_document_is_not_writable_at_a_key_other_than_the_global_one(self) -> None:
solo = define_database_entry_type("tests.solo", [database_entry_field("a", 0, codec=t.string)], global_document=True)
cache = CultCache.builder().register_document_type(solo).build()
self.assertFalse(cache.is_writable(solo, "elsewhere"))
with self.assertRaises(CultCacheError):
cache.put(solo, "elsewhere", {"a": "x"})
with self.assertRaises(CultCacheError):
cache.delete(solo, "elsewhere")
self.assertTrue(cache.is_writable(solo, CultCache.GLOBAL_KEY))
def test_a_reload_replaces_the_marks_wholesale(self) -> None:
path, cache = self._store("extra-slot-store.cc")
self.assertFalse(cache.is_writable(surface_document, "marked"))
# The store is repaired by its owner: the marked record is rewritten clean. The reload must forget the old mark.
[snapshot] = list(msgpack.Unpacker(open(path, "rb"), raw=False))
clean = next(r for r in snapshot[2] if r[0] == "clean")
for record in snapshot[2]:
if record[0] in ("marked", "unit-arm"):
record[3] = clean[3] if record[0] == "marked" else record[3]
snapshot[2] = [r for r in snapshot[2] if r[0] != "unit-arm"]
path.write_bytes(msgpack.packb(snapshot, use_bin_type=True))
cache.pull_all_backing_stores()
self.assertTrue(cache.is_writable(surface_document, "marked"))
self.assertEqual(cache._state.unread, {})
def test_a_listed_id_whose_catalog_reports_no_extra_slot_does_not_consent(self) -> None:
wide = next(e for e in SingleFileMessagePackBackingStore(VECTORS / "extra-slot-consent.cc").pull_all() if e.key == "consented")
same = dataclasses.replace(wide, catalog_entry=declaring_document.catalog_entry()) # the stored entry has no slot past the reader's
class Held:
def pull_all(self):
return [same]
def push(self, envelope): ...
def push_all(self, envelopes): ...
def delete(self, type, key): ...
cache = CultCache.builder().register_document_type(declaring_document).add_generic_store(Held()).build()
cache.pull_all_backing_stores()
self.assertFalse(cache.is_writable(declaring_document, "consented"))
def test_a_record_stored_under_the_readers_own_id_is_never_consented(self) -> None:
own = dataclasses.replace(
next(e for e in SingleFileMessagePackBackingStore(VECTORS / "extra-slot-consent.cc").pull_all() if e.key == "consented"),
schema_id="tests.union_surface.v2",
)
wider = dataclasses.replace(own.catalog_entry, schema_id="tests.union_surface.v2", compatible_schema_ids=("tests.union_surface.v2",))
listing = dataclasses.replace(
declaring_document, schema_id="tests.union_surface.v2", compatible_schema_ids=("tests.union_surface.wide", "tests.union_surface.v2")
)
class Held:
def pull_all(self):
return [dataclasses.replace(own, catalog_entry=wider)]
def push(self, envelope): ...
def push_all(self, envelopes): ...
def delete(self, type, key): ...
cache = CultCache.builder().register_document_type(listing).add_generic_store(Held()).build()
cache.pull_all_backing_stores()
self.assertFalse(cache.is_writable(listing, "consented"))
def test_writing_another_record_and_flushing_carries_the_marked_one_unchanged(self) -> None:
path, cache = self._store("extra-slot-store.cc")
original = {e.key: e for e in SingleFileMessagePackBackingStore(path).pull_all()}