GameCult/CultLib
Watch
9
0
Fork
You've already forked CultLib
 0
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions
CultLib/packages/cultcache-py/tests/test_cultcache.py

1591 lines
85 KiB
Python
Raw Normal View History

Split the Python runtime into cultcache-py, cultnet-py, cultmesh-py One distribution named cultcache-py carried three libraries. Rust and TypeScript layer them as cultcache < cultnet < cultmesh; Python now does the same, one package per layer, module names unchanged so every `from cultmesh_py import ...` keeps working. Two imports ran against that layering and are resolved by moving code to where the dependency direction says it lives, not by adding a shim: - cultnet_py.interop_peer imported the Verse/peer catalog, authority lease, stream, and simulation-fact contracts from cultmesh_py. They are wire contracts a bare peer has to speak without owning a node, which is why cultnet-rs and cultnet-ts keep them in the net layer. They now live in cultnet_py.cultmesh_contracts; cultmesh_py re-exports them. - cultcache_py.benchmark imported cultnet_py. It measures framed CultNet parse beside cache paths, so it and compare_csharp move to cultnet_py. verify checks all three packages and moves to cultmesh_py. The single 6,693-line test module is split by what each test touches: a test using any cultmesh_py name lands in cultmesh-py, else cultnet_py in cultnet-py, else cultcache-py. Each suite runs with only its layer and the layers below on the path, so a test that reaches up fails to import rather than quietly binding the packages back together. 11 / 53 / 78 = the same 142. Workflows install all three; the publish job is one matrix over the three packages keyed by tag prefix. The TS test harnesses put all three src directories on PYTHONPATH. cultcache-py bumps to 0.2.0 because it no longer contains cultnet_py or cultmesh_py; the new packages start at 0.1.0. Nothing was ever published, so no PyPI consumer moves. cultcache-py 11 tests, path: cultcache-py/src cultnet-py 53 tests, path: + cultnet-py/src cultmesh-py 78 tests, path: + cultmesh-py/src python -m cultmesh_py.verify -> ok, all five checks cultnet_py.interop_peer imports with cultmesh_py absent Not run here: the cultnet-ts interop ring and cultcache-ts store matrix need dotnet/kotlin peers; their PYTHONPATH change is four lines. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-10 13:10:47 +02:00
from __future__ import annotations
Add Python CultCache wire parity package
2026-06-13 20:35:58 +01:00
cultcache-py: only a missing store reads as empty; an empty file is refused `Path.exists()` returns False for ELOOP and ENOTDIR, so a symlink loop or a file where the parent directory should be read as an empty store. The read now catches FileNotFoundError only. An existing zero-byte file is refused with ValueError, like any other undecodable store, matching the Rust and C# runtimes. `[0x90]` stays an empty legacy store. Refusal text no longer points at a variant-aware runtime: an unsupported header names the format found, and a v1 record with a fifth slot says the file is not a valid store. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 17:04:39 +02:00
import os
Add Python CultCache wire parity package
2026-06-13 20:35:58 +01:00
import tempfile
Serialize the Python cache behind one reentrant lock per instance The Python cache took no lock, so "refused write changes neither store nor cache" and the one-home rule held only for one caller at a time. A put paused between its store push and its apply let a concurrent attach route the type to a new home (the record stayed in the old store), and let a second global put reach disk first so cache and disk disagreed. CultCache now holds a threading.RLock around attach, pull, registration, every write and delete (globals and update included), name/index resolution and snapshots, matching the C# per-cache gate and the TS serial queue. Single-dict reads stay unlocked. No public API change. Tests: test_concurrent_put_and_attach_cannot_land_record_in_non_home_store and test_concurrent_global_puts_cannot_both_reach_disk force the interleaving with a store that pauses inside push; both fail on 4dc7f3a and pass here. The contract's Python cells drop the one-caller caveat. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 12:03:45 +02:00
import threading
Add Python CultCache wire parity package
2026-06-13 20:35:58 +01:00
import unittest
Refuse duplicate global records while staging a pull A load is all-or-nothing. TypeScript checked for a second record of a global type inside #applyHydratedEntry, which runs after #resetHydratedState, so a refused pull left the cache cleared and partly reloaded. The check now runs in the staging pass beside the home check, before anything is cleared; the apply step's pull-only branch is gone. Python already validated before swapping, but its check only fired when the __global__ record arrived second, so a store holding __global__ then another key of a global type loaded both. Any second record of a global type is now refused regardless of order. Rust has no global document types and replaces its entry map only after staging; it had neither flaw. pullWithDuplicateGlobalLeavesCacheUnchanged (TS) and test_pull_with_duplicate_global_leaves_cache_unchanged (Python) fail on 9ab1706 and pass here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 00:57:09 +02:00
from dataclasses import asdict, dataclass, replace
Add Python CultCache wire parity package
2026-06-13 20:35:58 +01:00
from pathlib import Path
Refuse re-entrant Python cache mutations and adopt legacy-key globals on load Attach, pull, registration, writes and deletes raise when started on a thread already inside a mutation of the same cache (a store call, decoder, encoder, extractor or updater), instead of running inside it and corrupting the state being built. Reads stay reentrant. Internal delegation (put_global, update, delete_global, register_registry, add_backing_store) goes through private helpers so it is not re-entry. A pull adopts a global persisted under a key other than __global__ as that type's global without writing. The first write or delete of it removes the legacy record in the same call; a failed removal takes the new record back out. Two globals of one type on disk are still refused. The lock and unlocked-read comments now state the per-call guarantee. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 12:27:55 +02:00
from cultcache_py.backing_store import CultCacheEnvelope
Give every cultcache-py document type one home store Cut 5 (Python). stores_by_type maps each type to one store and generic_stores becomes a single generic_store. add_backing_store raises on a type already claimed and add_generic_store on a second generic store; a refused registration claims nothing. _stores_for_type and _all_specific_stores are deleted. _store_for_type returns the home store and raises when stores are attached but none is home to the type. It returns None only for a cache with no stores, which the contract keeps as an in-memory cache. put, put_envelope, put_envelopes and delete each write one store; pull_all_backing_stores pulls each distinct store once. Tests: test_second_generic_store_rejected, test_type_claimed_twice_rejected, test_types_route_to_home_store. No bytes change. Version 0.2.0 -> 0.3.0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 00:37:31 +02:00
from cultcache_py.cache import CultCacheError
Add Python CultCache wire parity package
2026-06-13 20:35:58 +01:00
from cultcache_py import (
CultCache,
JsonLinesBackingStore,
SingleFileMessagePackBackingStore,
Python: StoreUnreadableError for a store file it cannot read; a store is exactly three slots The refusal is a ValueError subclass with the cause chained. A store with more than three slots is refused instead of read. The readability test walks every vector in the folder, asserts the error type and cause, the header a replaced file carries and what it holds. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 04:15:04 +02:00
StoreUnreadableError,
Add Python CultCache wire parity package
2026-06-13 20:35:58 +01:00
define_database_entry_type,
define_document_type,
)
Split the Python runtime into cultcache-py, cultnet-py, cultmesh-py One distribution named cultcache-py carried three libraries. Rust and TypeScript layer them as cultcache < cultnet < cultmesh; Python now does the same, one package per layer, module names unchanged so every `from cultmesh_py import ...` keeps working. Two imports ran against that layering and are resolved by moving code to where the dependency direction says it lives, not by adding a shim: - cultnet_py.interop_peer imported the Verse/peer catalog, authority lease, stream, and simulation-fact contracts from cultmesh_py. They are wire contracts a bare peer has to speak without owning a node, which is why cultnet-rs and cultnet-ts keep them in the net layer. They now live in cultnet_py.cultmesh_contracts; cultmesh_py re-exports them. - cultcache_py.benchmark imported cultnet_py. It measures framed CultNet parse beside cache paths, so it and compare_csharp move to cultnet_py. verify checks all three packages and moves to cultmesh_py. The single 6,693-line test module is split by what each test touches: a test using any cultmesh_py name lands in cultmesh-py, else cultnet_py in cultnet-py, else cultcache-py. Each suite runs with only its layer and the layers below on the path, so a test that reaches up fails to import rather than quietly binding the packages back together. 11 / 53 / 78 = the same 142. Workflows install all three; the publish job is one matrix over the three packages keyed by tag prefix. The TS test harnesses put all three src directories on PYTHONPATH. cultcache-py bumps to 0.2.0 because it no longer contains cultnet_py or cultmesh_py; the new packages start at 0.1.0. Nothing was ever published, so no PyPI consumer moves. cultcache-py 11 tests, path: cultcache-py/src cultnet-py 53 tests, path: + cultnet-py/src cultmesh-py 78 tests, path: + cultmesh-py/src python -m cultmesh_py.verify -> ok, all five checks cultnet_py.interop_peer imports with cultmesh_py absent Not run here: the cultnet-ts interop ring and cultcache-ts store matrix need dotnet/kotlin peers; their PYTHONPATH change is four lines. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-10 13:10:47 +02:00
from cultcache_py.interop import read_note, write_note
Add Python RUDP socket transport
2026-06-14 23:18:08 +01:00
Add Python CultCache wire parity package
2026-06-13 20:35:58 +01:00
@dataclass
class Item:
name: str
category: str
value: int
def item_doc():
return define_document_type(
"item",
encode=lambda item: asdict(item),
decode=lambda raw: Item(**raw),
name="name",
indexes={"category": "category"},
)
class CultCacheTests(unittest.TestCase):
def test_round_trips_registered_documents(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
document = item_doc()
cache = (
CultCache.builder()
.register_document_type(document)
.add_generic_store(JsonLinesBackingStore(Path(tmp) / "cache.jsonl"))
.build()
)
cache.pull_all_backing_stores()
cache.put(document, "item:potion", Item(name="Potion", category="Consumable", value=50))
loaded = (
CultCache.builder()
.register_document_type(document)
.add_generic_store(JsonLinesBackingStore(Path(tmp) / "cache.jsonl"))
.build()
)
loaded.pull_all_backing_stores()
self.assertEqual(loaded.get_required(document, "item:potion").value, 50)
self.assertEqual(loaded.get_key_by_name(document, "Potion"), "item:potion")
self.assertEqual(loaded.get_by_index(document, "category", "Consumable").name, "Potion")
def test_global_document_uses_singleton_key(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
settings = define_document_type("settings", global_document=True)
cache = (
CultCache.builder()
.register_document_type(settings)
.add_generic_store(JsonLinesBackingStore(Path(tmp) / "cache.jsonl"))
.build()
)
cache.pull_all_backing_stores()
cache.put_global(settings, {"theme": "ash"})
self.assertEqual(cache.get_required_global(settings)["theme"], "ash")
def test_update_and_delete(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
document = item_doc()
cache = (
CultCache.builder()
.register_document_type(document)
.add_generic_store(JsonLinesBackingStore(Path(tmp) / "cache.jsonl"))
.build()
)
cache.pull_all_backing_stores()
cache.put(document, "item:potion", Item(name="Potion", category="Consumable", value=50))
cache.update(document, "item:potion", lambda item: Item(item.name, item.category, item.value + 5))
self.assertEqual(cache.get_required(document, "item:potion").value, 55)
cache.delete(document, "item:potion")
self.assertIsNone(cache.get(document, "item:potion"))
Optimize Python CultCache incremental lookups
2026-06-14 01:37:35 +01:00
def test_incremental_lookup_updates_replace_stale_name_and_index_entries(self) -> None:
document = item_doc()
cache = CultCache()
cache.register_document_type(document)
cache.put(document, "item:potion", Item(name="Potion", category="Consumable", value=50))
cache.put(document, "item:potion", Item(name="Elixir", category="Rare", value=80))
self.assertIsNone(cache.get_key_by_name(document, "Potion"))
self.assertIsNone(cache.get_key_by_index(document, "category", "Consumable"))
self.assertEqual(cache.get_key_by_name(document, "Elixir"), "item:potion")
self.assertEqual(cache.get_by_index(document, "category", "Rare").name, "Elixir")
def test_incremental_lookup_delete_restores_duplicate_owner(self) -> None:
document = item_doc()
cache = CultCache()
cache.register_document_type(document)
cache.put(document, "item:first", Item(name="Potion", category="Consumable", value=50))
cache.put(document, "item:second", Item(name="Potion", category="Consumable", value=60))
self.assertEqual(cache.get_key_by_name(document, "Potion"), "item:second")
self.assertEqual(cache.get_key_by_index(document, "category", "Consumable"), "item:second")
cache.delete(document, "item:second")
self.assertEqual(cache.get_key_by_name(document, "Potion"), "item:first")
self.assertEqual(cache.get_key_by_index(document, "category", "Consumable"), "item:first")
Add Python CultCache wire parity package
2026-06-13 20:35:58 +01:00
def test_database_entry_formatter_uses_slot_indexed_messagepack_array(self) -> None:
try:
import msgpack # type: ignore
except ModuleNotFoundError:
self.skipTest("msgpack optional dependency is not installed")
@dataclass
class Settings:
theme: str
retries: int = 0
document = define_database_entry_type(
"settings",
[
("theme", 0),
("retries", 2, 0),
],
cls=Settings,
)
payload = document.encode_payload(Settings(theme="ash", retries=3))
self.assertEqual(msgpack.unpackb(payload, raw=False), ["ash", None, 3])
self.assertEqual(document.decode_payload(msgpack.packb(["ash"], use_bin_type=True)).retries, 0)
Cache Python document schema catalogs
2026-06-14 01:43:14 +01:00
def test_document_catalog_entry_is_cached_after_first_derivation(self) -> None:
document = define_database_entry_type(
"settings",
[
("theme", 0),
("retries", 2, 0),
],
)
first = document.catalog_entry()
second = document.catalog_entry()
self.assertIs(first, second)
self.assertEqual([member.slot for member in first.members], [0, 2])
Add Python CultCache wire parity package
2026-06-13 20:35:58 +01:00
def test_messagepack_store_writes_v1_snapshot(self) -> None:
try:
import msgpack # type: ignore
except ModuleNotFoundError:
self.skipTest("msgpack optional dependency is not installed")
with tempfile.TemporaryDirectory() as tmp:
document = define_database_entry_type(
"settings",
[("theme", 0)],
schema_id="settings",
schema_name="settings",
schema_version="settings.v1",
)
store_path = Path(tmp) / "cache.msgpack"
cache = (
CultCache.builder()
.register_document_type(document)
.add_generic_store(SingleFileMessagePackBackingStore(store_path))
.build()
)
cache.pull_all_backing_stores()
cache.put(document, "app", {"theme": "ash"})
raw = msgpack.unpackb(store_path.read_bytes(), raw=False)
self.assertEqual(raw[0], "cultcache.store.v1")
self.assertEqual(raw[1][0][0], "settings")
self.assertEqual(raw[2][0][0], "app")
self.assertEqual(raw[2][0][1], "settings")
Tests: the recovery tests assert the refusal Rust, TypeScript and Python each had a test that a record missing its catalog entry is recovered from its payload; each now asserts the store is unreadable. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 05:58:47 +02:00
def test_messagepack_store_refuses_a_record_whose_schema_the_catalog_does_not_publish(self) -> None:
Recover schema-stamped cache records across runtimes
2026-06-27 14:34:53 +01:00
try:
import msgpack # type: ignore
except ModuleNotFoundError:
self.skipTest("msgpack optional dependency is not installed")
with tempfile.TemporaryDirectory() as tmp:
document = define_database_entry_type(
"runtime-policy",
[
("schema_version", 0),
("name", 1),
("value", 2),
],
schema_id="schema-current",
schema_name="tests.schema_stamped_entry",
schema_version="tests.schema_stamped_entry.v1",
)
store_path = Path(tmp) / "missing-catalog.msgpack"
store_path.write_bytes(msgpack.packb([
"cultcache.store.v1",
[],
[[
"record-1",
"sha256:stale-schema-id-from-cold-record",
"2026-06-25T12:00:00Z",
msgpack.packb([
"tests.schema_stamped_entry.v1",
"schema-stamped",
"still readable",
], use_bin_type=True),
]],
], use_bin_type=True))
cache = (
CultCache.builder()
.register_document_type(document)
.add_generic_store(SingleFileMessagePackBackingStore(store_path))
.build()
)
Tests: the recovery tests assert the refusal Rust, TypeScript and Python each had a test that a record missing its catalog entry is recovered from its payload; each now asserts the store is unreadable. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 05:58:47 +02:00
# The payload opens with a schema version, and the catalog is the store's own description: no recovery.
with self.assertRaises(StoreUnreadableError):
cache.pull_all_backing_stores()
Recover schema-stamped cache records across runtimes
2026-06-27 14:34:53 +01:00
Python: a record is stamped with its registered schema id and the catalog publishes it; cultnet no longer infers a schema from a payload put_envelope, put_envelopes and load stamp the registered schema's id, and the store writer publishes every record's id, so a replicated record under a compatible id no longer leaves a store nothing can open. cultnet-py resolves a raw record by the schema ids its documents list and nothing else. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 06:54:18 +02:00
def _foreign_id_document(self):
return define_database_entry_type(
"tests.foreign-id",
[("name", 0)],
schema_id="tests.foreign-id.current",
schema_name="tests.foreign-id",
schema_version="tests.foreign_id.v1",
compatible_schema_ids=["tests.foreign-id.current", "tests.foreign-id.older"],
)
def _open_foreign_id_store(self, document, path: Path) -> CultCache:
cache = CultCache.builder().register_document_type(document).add_generic_store(
SingleFileMessagePackBackingStore(path)
).build()
cache.pull_all_backing_stores()
return cache
# A record put under an id its schema only lists as compatible is stamped with the registered id, and the store the write
# leaves opens and takes the next write.
def test_put_envelope_under_a_compatible_schema_id_writes_a_store_that_reopens(self) -> None:
import msgpack # type: ignore
from cultcache_py import CultCacheSchemaCatalogEntry
document = self._foreign_id_document()
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.msgpack"
cache = self._open_foreign_id_store(document, path)
payload = document.encode_payload({"name": "foreign"})
foreign_entry = CultCacheSchemaCatalogEntry(
schema_id="tests.foreign-id.older",
schema_name="tests.foreign-id",
schema_version="tests.foreign_id.v1",
content_hash="tests.foreign-id.older",
canonical_schema_json="",
compatible_schema_ids=("tests.foreign-id.older",),
members=(),
)
cache.put_envelope(
document,
CultCacheEnvelope(
key="foreign",
type="tests.foreign-id",
payload=payload,
stored_at="2026-09-30T00:00:00Z",
schema_id="tests.foreign-id.older",
catalog_entry=foreign_entry,
),
)
stored = msgpack.unpackb(path.read_bytes(), raw=False)
self.assertEqual([entry[0] for entry in stored[1]], ["tests.foreign-id.current"])
self.assertEqual(stored[2][0][1], "tests.foreign-id.current")
reopened = self._open_foreign_id_store(document, path)
self.assertEqual(reopened.get_required(document, "foreign")["name"], "foreign")
reopened.put(document, "next", {"name": "next"})
self.assertEqual(self._open_foreign_id_store(document, path).get_required(document, "next")["name"], "next")
Python and TypeScript: tests for the store's publishing rule and the admission stamp; cultnet-py keeps its payload-based resolution cultnet-py resolves a raw record whose schema id no document lists by the schema version its payload opens with, and three cultmesh-py tests (foreign UI-generated ids) depend on it, so that resolution is restored pending a ruling. The compatible-id replication test goes through the real apply_raw_document_record and the reopened store. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 07:28:17 +02:00
def test_the_store_publishes_a_records_schema_id_even_when_the_envelopes_catalog_entry_does_not(self) -> None:
import msgpack # type: ignore
from cultcache_py import CultCacheSchemaCatalogEntry
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.msgpack"
store = SingleFileMessagePackBackingStore(path)
store.push(CultCacheEnvelope(
key="k",
type="tests.store-publishes",
payload=msgpack.packb({"name": "k"}, use_bin_type=True),
stored_at="2026-09-30T00:00:00Z",
schema_id="tests.store-publishes.record",
catalog_entry=CultCacheSchemaCatalogEntry(
schema_id="tests.store-publishes.other",
schema_name="tests.store-publishes",
schema_version="tests.store_publishes.v1",
content_hash="tests.store-publishes.other",
canonical_schema_json="",
compatible_schema_ids=("tests.store-publishes.other",),
members=(),
),
))
stored = msgpack.unpackb(path.read_bytes(), raw=False)
self.assertEqual(stored[2][0][1], "tests.store-publishes.record")
self.assertEqual([entry[0] for entry in stored[1]], ["tests.store-publishes.record"])
self.assertEqual(len(SingleFileMessagePackBackingStore(path).pull_all()), 1)
Writers derive the catalog from the records they write; the merging is deleted Every writer (C# single-file and directory, TypeScript, Python; Rust already did) chooses, for each schema id a record carries, one entry that publishes it: the entry that owns the id (a registered descriptor, else an arrived entry), else one that lists it as compatible. A registered descriptor wins over an arrived entry with the same own id. Entries are written as chosen, with no union of compatible ids and no dependence on arrival order; an entry no record needs is not written. Two entries of one tier that share an own id and disagree on the schema name, records of different types under one id, or a record no chosen entry publishes refuse the write with a typed error (CultSchemaConflictException, SchemaConflictError) naming the id, the names and a record key, and the store is left as it was. CultSchemaCatalogEntry.MergeById and the Python and TypeScript merge paths are gone. A cache-held record unchanged since the file was read is rewritten under its registered id (C# single-file and directory), so an entry that lists an older id is not needed to publish it. Changelogs for cultnet-py (removed export) and cultcache-py. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 10:03:34 +02:00
# The catalog a write leaves is derived from its records: one entry per carried id, chosen and never merged, the same whatever
# order the records arrive in. A registered (supplied) entry wins over an arrived (read back) entry with the same own id.
@staticmethod
def _writer_entry(schema_id: str, name: str, content_hash: str, compatible: tuple[str, ...]):
Catalog writers merge entries that share an id; every reader resolves an id by its owner first Entries that share a schema id are one schema seen by different writers, so the writer lists every id any of them lists, in any order: C# single-file and directory (CultSchemaCatalogEntry.MergeById), TypeScript and Python. Rust synthesises one entry per id and has no such merge. An id that one entry owns and another lists as compatible names the owner in Rust, TypeScript, Python and the TS inspector, as C# already did (own-id-over-compatible-v3). Also: a directory manifest in an unreadable format is the typed refusal naming it, cultnet-py applies and reports a record under the local schema id, and Rust's error prints its cause once. Vectors for the Mesh layout discriminator: a nil and an integer at catalog slot 5. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 08:17:59 +02:00
from cultcache_py import CultCacheSchemaCatalogEntry
Writers derive the catalog from the records they write; the merging is deleted Every writer (C# single-file and directory, TypeScript, Python; Rust already did) chooses, for each schema id a record carries, one entry that publishes it: the entry that owns the id (a registered descriptor, else an arrived entry), else one that lists it as compatible. A registered descriptor wins over an arrived entry with the same own id. Entries are written as chosen, with no union of compatible ids and no dependence on arrival order; an entry no record needs is not written. Two entries of one tier that share an own id and disagree on the schema name, records of different types under one id, or a record no chosen entry publishes refuse the write with a typed error (CultSchemaConflictException, SchemaConflictError) naming the id, the names and a record key, and the store is left as it was. CultSchemaCatalogEntry.MergeById and the Python and TypeScript merge paths are gone. A cache-held record unchanged since the file was read is rewritten under its registered id (C# single-file and directory), so an entry that lists an older id is not needed to publish it. Changelogs for cultnet-py (removed export) and cultcache-py. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 10:03:34 +02:00
return CultCacheSchemaCatalogEntry(
schema_id=schema_id, schema_name=name, schema_version=name + ".v1", content_hash=content_hash,
canonical_schema_json="", compatible_schema_ids=compatible, members=(),
)
Catalog writers merge entries that share an id; every reader resolves an id by its owner first Entries that share a schema id are one schema seen by different writers, so the writer lists every id any of them lists, in any order: C# single-file and directory (CultSchemaCatalogEntry.MergeById), TypeScript and Python. Rust synthesises one entry per id and has no such merge. An id that one entry owns and another lists as compatible names the owner in Rust, TypeScript, Python and the TS inspector, as C# already did (own-id-over-compatible-v3). Also: a directory manifest in an unreadable format is the typed refusal naming it, cultnet-py applies and reports a record under the local schema id, and Rust's error prints its cause once. Vectors for the Mesh layout discriminator: a nil and an integer at catalog slot 5. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 08:17:59 +02:00
Writers derive the catalog from the records they write; the merging is deleted Every writer (C# single-file and directory, TypeScript, Python; Rust already did) chooses, for each schema id a record carries, one entry that publishes it: the entry that owns the id (a registered descriptor, else an arrived entry), else one that lists it as compatible. A registered descriptor wins over an arrived entry with the same own id. Entries are written as chosen, with no union of compatible ids and no dependence on arrival order; an entry no record needs is not written. Two entries of one tier that share an own id and disagree on the schema name, records of different types under one id, or a record no chosen entry publishes refuse the write with a typed error (CultSchemaConflictException, SchemaConflictError) naming the id, the names and a record key, and the store is left as it was. CultSchemaCatalogEntry.MergeById and the Python and TypeScript merge paths are gone. A cache-held record unchanged since the file was read is rewritten under its registered id (C# single-file and directory), so an entry that lists an older id is not needed to publish it. Changelogs for cultnet-py (removed export) and cultcache-py. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 10:03:34 +02:00
def _writer_record(self, key: str, type: str, schema_id: str, catalog_entry=None) -> CultCacheEnvelope:
import msgpack # type: ignore
Catalog writers merge entries that share an id; every reader resolves an id by its owner first Entries that share a schema id are one schema seen by different writers, so the writer lists every id any of them lists, in any order: C# single-file and directory (CultSchemaCatalogEntry.MergeById), TypeScript and Python. Rust synthesises one entry per id and has no such merge. An id that one entry owns and another lists as compatible names the owner in Rust, TypeScript, Python and the TS inspector, as C# already did (own-id-over-compatible-v3). Also: a directory manifest in an unreadable format is the typed refusal naming it, cultnet-py applies and reports a record under the local schema id, and Rust's error prints its cause once. Vectors for the Mesh layout discriminator: a nil and an integer at catalog slot 5. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 08:17:59 +02:00
Writers derive the catalog from the records they write; the merging is deleted Every writer (C# single-file and directory, TypeScript, Python; Rust already did) chooses, for each schema id a record carries, one entry that publishes it: the entry that owns the id (a registered descriptor, else an arrived entry), else one that lists it as compatible. A registered descriptor wins over an arrived entry with the same own id. Entries are written as chosen, with no union of compatible ids and no dependence on arrival order; an entry no record needs is not written. Two entries of one tier that share an own id and disagree on the schema name, records of different types under one id, or a record no chosen entry publishes refuse the write with a typed error (CultSchemaConflictException, SchemaConflictError) naming the id, the names and a record key, and the store is left as it was. CultSchemaCatalogEntry.MergeById and the Python and TypeScript merge paths are gone. A cache-held record unchanged since the file was read is rewritten under its registered id (C# single-file and directory), so an entry that lists an older id is not needed to publish it. Changelogs for cultnet-py (removed export) and cultcache-py. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 10:03:34 +02:00
return CultCacheEnvelope(
key=key, type=type, payload=msgpack.packb({"key": key}, use_bin_type=True),
stored_at="2026-09-30T00:00:00Z", schema_id=schema_id, catalog_entry=catalog_entry,
)
@staticmethod
def _raw_entry(entry) -> list:
return [entry.schema_id, entry.schema_name, entry.schema_version, entry.content_hash, "", list(entry.compatible_schema_ids), []]
def test_a_registered_entry_is_written_as_it_is_over_the_arrived_entries_with_the_same_own_id_in_either_order(self) -> None:
import msgpack # type: ignore
Tests: the arrived entries' hashes sort before the registered one's, so a writer that ignores the tiers picks the arrived one Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 10:16:53 +02:00
stale1 = self._writer_entry("tests.x", "tests.n", "a-stale-1", ("tests.x", "old"))
stale2 = self._writer_entry("tests.x", "tests.n", "a-stale-2", ("tests.x",))
Writers derive the catalog from the records they write; the merging is deleted Every writer (C# single-file and directory, TypeScript, Python; Rust already did) chooses, for each schema id a record carries, one entry that publishes it: the entry that owns the id (a registered descriptor, else an arrived entry), else one that lists it as compatible. A registered descriptor wins over an arrived entry with the same own id. Entries are written as chosen, with no union of compatible ids and no dependence on arrival order; an entry no record needs is not written. Two entries of one tier that share an own id and disagree on the schema name, records of different types under one id, or a record no chosen entry publishes refuse the write with a typed error (CultSchemaConflictException, SchemaConflictError) naming the id, the names and a record key, and the store is left as it was. CultSchemaCatalogEntry.MergeById and the Python and TypeScript merge paths are gone. A cache-held record unchanged since the file was read is rewritten under its registered id (C# single-file and directory), so an entry that lists an older id is not needed to publish it. Changelogs for cultnet-py (removed export) and cultcache-py. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 10:03:34 +02:00
registered = self._writer_record("a", "tests.n", "tests.x", self._writer_entry("tests.x", "tests.n", "fresh", ("tests.x",)))
for entries in ([stale1, stale2], [stale2, stale1]):
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.msgpack"
path.write_bytes(msgpack.packb([
"cultcache.store.v1", [self._raw_entry(entry) for entry in entries],
[["b", "tests.x", "2026-09-30T00:00:00Z", msgpack.packb({"b": 1}, use_bin_type=True)]],
], use_bin_type=True))
SingleFileMessagePackBackingStore(path).push(registered)
catalog = msgpack.unpackb(path.read_bytes(), raw=False)[1]
self.assertEqual(len(catalog), 1)
self.assertEqual(catalog[0][3], "fresh")
self.assertEqual(catalog[0][5], ["tests.x"], "written as chosen, no union with the arrived entries' ids")
def test_a_rename_with_a_stable_schema_id_a_whole_view_write_names_the_registered_schema(self) -> None:
import msgpack # type: ignore
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.msgpack"
path.write_bytes(msgpack.packb([
"cultcache.store.v1", [self._raw_entry(self._writer_entry("tests.x", "tests.old", "stale", ("tests.x",)))],
[["a", "tests.x", "2026-09-30T00:00:00Z", msgpack.packb({"a": 1}, use_bin_type=True)]],
], use_bin_type=True))
SingleFileMessagePackBackingStore(path).push_all(
TypeScript and Python writers: records are compared for type within a tier only A read-back record's type is its schema's name and a supplied one's is the document type, so records of different tiers are never compared; two supplied (or two arrived) records of different types under one id still refuse the write. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 10:13:44 +02:00
[self._writer_record("a", "tests.old", "tests.x", self._writer_entry("tests.x", "tests.new", "fresh", ("tests.x",)))]
Writers derive the catalog from the records they write; the merging is deleted Every writer (C# single-file and directory, TypeScript, Python; Rust already did) chooses, for each schema id a record carries, one entry that publishes it: the entry that owns the id (a registered descriptor, else an arrived entry), else one that lists it as compatible. A registered descriptor wins over an arrived entry with the same own id. Entries are written as chosen, with no union of compatible ids and no dependence on arrival order; an entry no record needs is not written. Two entries of one tier that share an own id and disagree on the schema name, records of different types under one id, or a record no chosen entry publishes refuse the write with a typed error (CultSchemaConflictException, SchemaConflictError) naming the id, the names and a record key, and the store is left as it was. CultSchemaCatalogEntry.MergeById and the Python and TypeScript merge paths are gone. A cache-held record unchanged since the file was read is rewritten under its registered id (C# single-file and directory), so an entry that lists an older id is not needed to publish it. Changelogs for cultnet-py (removed export) and cultcache-py. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 10:03:34 +02:00
)
catalog = msgpack.unpackb(path.read_bytes(), raw=False)[1]
self.assertEqual([(entry[1], entry[3]) for entry in catalog], [("tests.new", "fresh")])
def test_two_entries_of_one_tier_that_share_an_own_id_and_disagree_on_the_schema_name_refuse_the_write_typed(self) -> None:
from cultcache_py import SchemaConflictError
first = self._writer_record("b", "tests.first", "tests.x", self._writer_entry("tests.x", "tests.first", "h1", ("tests.x",)))
second = self._writer_record("a", "tests.first", "tests.x", self._writer_entry("tests.x", "tests.second", "h2", ("tests.x",)))
for order in ([first, second], [second, first]):
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.msgpack"
with self.assertRaises(SchemaConflictError) as refused:
SingleFileMessagePackBackingStore(path).push_all(order)
self.assertEqual(refused.exception.schema_id, "tests.x")
self.assertEqual(refused.exception.record_key, "a")
self.assertEqual(refused.exception.schema_names, ["tests.first", "tests.second"])
self.assertFalse(path.exists())
def test_records_of_different_types_under_one_schema_id_refuse_the_write_typed_and_no_record_changes_type(self) -> None:
from cultcache_py import SchemaConflictError
a = self._writer_record("a", "tests.n", "tests.y", self._writer_entry("tests.x", "tests.n", "h1", ("tests.x", "tests.y")))
b = self._writer_record("b", "tests.m", "tests.y", self._writer_entry("tests.y", "tests.m", "h2", ("tests.y",)))
Catalog writers merge entries that share an id; every reader resolves an id by its owner first Entries that share a schema id are one schema seen by different writers, so the writer lists every id any of them lists, in any order: C# single-file and directory (CultSchemaCatalogEntry.MergeById), TypeScript and Python. Rust synthesises one entry per id and has no such merge. An id that one entry owns and another lists as compatible names the owner in Rust, TypeScript, Python and the TS inspector, as C# already did (own-id-over-compatible-v3). Also: a directory manifest in an unreadable format is the typed refusal naming it, cultnet-py applies and reports a record under the local schema id, and Rust's error prints its cause once. Vectors for the Mesh layout discriminator: a nil and an integer at catalog slot 5. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 08:17:59 +02:00
for order in ([a, b], [b, a]):
Writers derive the catalog from the records they write; the merging is deleted Every writer (C# single-file and directory, TypeScript, Python; Rust already did) chooses, for each schema id a record carries, one entry that publishes it: the entry that owns the id (a registered descriptor, else an arrived entry), else one that lists it as compatible. A registered descriptor wins over an arrived entry with the same own id. Entries are written as chosen, with no union of compatible ids and no dependence on arrival order; an entry no record needs is not written. Two entries of one tier that share an own id and disagree on the schema name, records of different types under one id, or a record no chosen entry publishes refuse the write with a typed error (CultSchemaConflictException, SchemaConflictError) naming the id, the names and a record key, and the store is left as it was. CultSchemaCatalogEntry.MergeById and the Python and TypeScript merge paths are gone. A cache-held record unchanged since the file was read is rewritten under its registered id (C# single-file and directory), so an entry that lists an older id is not needed to publish it. Changelogs for cultnet-py (removed export) and cultcache-py. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 10:03:34 +02:00
with tempfile.TemporaryDirectory() as tmp:
with self.assertRaises(SchemaConflictError) as refused:
SingleFileMessagePackBackingStore(Path(tmp) / "store.msgpack").push_all(order)
self.assertEqual(refused.exception.schema_id, "tests.y")
self.assertEqual(refused.exception.schema_names, ["tests.m", "tests.n"])
def test_an_entry_that_owns_an_id_is_chosen_over_one_that_lists_it(self) -> None:
import msgpack # type: ignore
owner = self._writer_record("a", "tests.n", "tests.y", self._writer_entry("tests.y", "tests.n", "h1", ("tests.y",)))
lister = self._writer_record("b", "tests.n", "tests.x", self._writer_entry("tests.x", "tests.n", "h2", ("tests.x", "tests.y")))
for order in ([lister, owner], [owner, lister]):
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.msgpack"
SingleFileMessagePackBackingStore(path).push_all(order)
catalog = msgpack.unpackb(path.read_bytes(), raw=False)[1]
self.assertEqual(sorted((entry[0], entry[5]) for entry in catalog), [("tests.x", ["tests.x", "tests.y"]), ("tests.y", ["tests.y"])])
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.msgpack"
SingleFileMessagePackBackingStore(path).push_all([owner])
self.assertEqual([entry[0] for entry in msgpack.unpackb(path.read_bytes(), raw=False)[1]], ["tests.y"])
Writers: drop the same-tier dedupe refusal (unreachable), and pin tie order and the registered lister Two entries of one tier that share an own id are refused when the id is carried, so a refusal at dedupe could never fire. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 10:14:33 +02:00
def test_entries_of_one_tier_that_tie_are_taken_in_one_fixed_order_and_a_registered_lister_beats_an_arrived_one(self) -> None:
import msgpack # type: ignore
Python: a record resolves to its document by schema id, and the type check runs across tiers A record now resolves to the document that owns its schema id, else one that lists it as compatible, before the catalog name is tried; a schema renamed under a stable id opens, and an id that names one document is not overruled by a name that names another. The name resolves a record only when no local document has the id. The writer refuses a write that would change which type an existing record resolves to, unless the entries of both the existing records and the written ones own the id (a rename, the registered descriptor wins). Tests pin the arrived-overwrites-registered guard in both id orders, the content-hash tie-break against the compatible-list order, sorted id processing, the cross-tier refusal and the rename, and a cache whose document does not list the id a loaded record sits under. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 11:04:07 +02:00
# The content hash orders them against the order their compatible ids would give: h1 sorts first, its list sorts last.
one = self._writer_record("a", "tests.n", "tests.x", self._writer_entry("tests.x", "tests.n", "h2", ("tests.x",)))
two = self._writer_record("b", "tests.n", "tests.x", self._writer_entry("tests.x", "tests.n", "h1", ("tests.x", "tests.y")))
Writers: drop the same-tier dedupe refusal (unreachable), and pin tie order and the registered lister Two entries of one tier that share an own id are refused when the id is carried, so a refusal at dedupe could never fire. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 10:14:33 +02:00
for order in ([one, two], [two, one]):
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.msgpack"
SingleFileMessagePackBackingStore(path).push_all(order)
self.assertEqual([entry[3] for entry in msgpack.unpackb(path.read_bytes(), raw=False)[1]], ["h1"])
# z sits under old, which the arrived entry (id x) lists; the supplied entry (id r) lists it too: the registered one publishes it.
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.msgpack"
path.write_bytes(msgpack.packb([
"cultcache.store.v1", [self._raw_entry(self._writer_entry("tests.x", "tests.n", "stale", ("tests.x", "old")))],
[["z", "old", "2026-09-30T00:00:00Z", msgpack.packb({"z": 1}, use_bin_type=True)]],
], use_bin_type=True))
SingleFileMessagePackBackingStore(path).push(
self._writer_record("a", "tests.n", "tests.r", self._writer_entry("tests.r", "tests.n", "fresh", ("tests.r", "old")))
)
self.assertEqual([entry[0] for entry in msgpack.unpackb(path.read_bytes(), raw=False)[1]], ["tests.r"])
Python: a record resolves to its document by schema id, and the type check runs across tiers A record now resolves to the document that owns its schema id, else one that lists it as compatible, before the catalog name is tried; a schema renamed under a stable id opens, and an id that names one document is not overruled by a name that names another. The name resolves a record only when no local document has the id. The writer refuses a write that would change which type an existing record resolves to, unless the entries of both the existing records and the written ones own the id (a rename, the registered descriptor wins). Tests pin the arrived-overwrites-registered guard in both id orders, the content-hash tie-break against the compatible-list order, sorted id processing, the cross-tier refusal and the rename, and a cache whose document does not list the id a loaded record sits under. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 11:04:07 +02:00
# A registered entry that owns an id keeps it against an arrived entry that owns it too and lists the id another record sits
# under, whichever of the two ids sorts first: the arrived entry's list is not merged in, so the record under the listed id is
# published by no chosen entry.
def test_an_arrived_entry_that_owns_a_registered_id_does_not_replace_it_nor_publish_what_it_lists(self) -> None:
import msgpack # type: ignore
from cultcache_py import SchemaConflictError
for own_id, listed_id in (("tests.a", "tests.b"), ("tests.b", "tests.a")):
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.msgpack"
path.write_bytes(msgpack.packb([
"cultcache.store.v1", [self._raw_entry(self._writer_entry(own_id, "tests.arrived", "stale", (own_id, listed_id)))],
[["k2", listed_id, "2026-09-30T00:00:00Z", msgpack.packb({"k": 2}, use_bin_type=True)]],
], use_bin_type=True))
before = path.read_bytes()
with self.assertRaises(SchemaConflictError) as refused:
SingleFileMessagePackBackingStore(path).push(
self._writer_record("k1", "tests.registered", own_id, self._writer_entry(own_id, "tests.registered", "fresh", (own_id,)))
)
self.assertEqual(refused.exception.schema_id, listed_id)
self.assertEqual(refused.exception.record_key, "k2")
self.assertEqual(path.read_bytes(), before, "the file is left as it was")
# Ids are taken in sorted order, so which entry survives, and so which id the refusal names, does not depend on the order the
# records arrive in.
def test_the_refusal_names_the_same_id_whatever_order_the_records_arrive_in(self) -> None:
from cultcache_py import SchemaConflictError
one = self._writer_record("r1", "tests.n", "tests.y", self._writer_entry("tests.x", "tests.n", "h1", ("tests.x", "tests.y")))
two = self._writer_record("r2", "tests.n", "tests.z", self._writer_entry("tests.x", "tests.n", "h2", ("tests.x", "tests.z")))
for order in ([one, two], [two, one]):
with tempfile.TemporaryDirectory() as tmp:
with self.assertRaises(SchemaConflictError) as refused:
SingleFileMessagePackBackingStore(Path(tmp) / "store.msgpack").push_all(order)
self.assertEqual(refused.exception.schema_id, "tests.y")
self.assertEqual(refused.exception.record_key, "r1")
# A write may not retype records already in the file: across tiers the id keeps the type its records resolve to, unless the
# entries of both tiers own it, which is a rename under a stable id and the registered descriptor wins.
def test_a_write_that_would_change_the_type_an_existing_record_resolves_to_refuses_typed_and_leaves_the_file(self) -> None:
import msgpack # type: ignore
from cultcache_py import SchemaConflictError
cases = [
("registered entry lists the id, arrived entry owns it",
self._writer_entry("tests.x", "tests.arrived", "h1", ("tests.x",)),
self._writer_record("a", "tests.registered", "tests.x", self._writer_entry("tests.r", "tests.registered", "h2", ("tests.r", "tests.x")))),
("registered entry owns the id, arrived entry lists it",
self._writer_entry("tests.r", "tests.arrived", "h1", ("tests.r", "tests.x")),
self._writer_record("a", "tests.registered", "tests.x", self._writer_entry("tests.x", "tests.registered", "h2", ("tests.x",)))),
]
payload = msgpack.packb({"z": 1}, use_bin_type=True)
with tempfile.TemporaryDirectory() as tmp:
for name, catalog, registered in cases:
path = Path(tmp) / (name.split(",")[0].replace(" ", "-") + ".msgpack")
path.write_bytes(msgpack.packb(
["cultcache.store.v1", [self._raw_entry(catalog)], [["z", "tests.x", "2026-09-30T00:00:00Z", payload]]], use_bin_type=True))
before = path.read_bytes()
with self.assertRaises(SchemaConflictError, msg=name) as refused:
SingleFileMessagePackBackingStore(path).push(registered)
self.assertEqual(refused.exception.schema_id, "tests.x", name)
self.assertEqual(sorted(refused.exception.schema_names), ["tests.arrived", "tests.registered"], name)
self.assertEqual(path.read_bytes(), before, name)
# The same registered record renames the schema when both tiers own the id: it writes, and the reader finds every record.
path = Path(tmp) / "rename.msgpack"
path.write_bytes(msgpack.packb([
"cultcache.store.v1", [self._raw_entry(self._writer_entry("tests.x", "tests.arrived", "h1", ("tests.x",)))],
[["z", "tests.x", "2026-09-30T00:00:00Z", payload]],
], use_bin_type=True))
SingleFileMessagePackBackingStore(path).push(
self._writer_record("a", "tests.registered", "tests.x", self._writer_entry("tests.x", "tests.registered", "h2", ("tests.x",)))
)
self.assertEqual([(entry[0], entry[1]) for entry in msgpack.unpackb(path.read_bytes(), raw=False)[1]], [("tests.x", "tests.registered")])
self.assertEqual(len(SingleFileMessagePackBackingStore(path).pull_all()), 2)
Writers derive the catalog from the records they write; the merging is deleted Every writer (C# single-file and directory, TypeScript, Python; Rust already did) chooses, for each schema id a record carries, one entry that publishes it: the entry that owns the id (a registered descriptor, else an arrived entry), else one that lists it as compatible. A registered descriptor wins over an arrived entry with the same own id. Entries are written as chosen, with no union of compatible ids and no dependence on arrival order; an entry no record needs is not written. Two entries of one tier that share an own id and disagree on the schema name, records of different types under one id, or a record no chosen entry publishes refuse the write with a typed error (CultSchemaConflictException, SchemaConflictError) naming the id, the names and a record key, and the store is left as it was. CultSchemaCatalogEntry.MergeById and the Python and TypeScript merge paths are gone. A cache-held record unchanged since the file was read is rewritten under its registered id (C# single-file and directory), so an entry that lists an older id is not needed to publish it. Changelogs for cultnet-py (removed export) and cultcache-py. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 10:03:34 +02:00
def test_a_record_no_chosen_entry_publishes_refuses_the_write_typed_and_leaves_the_file(self) -> None:
import msgpack # type: ignore
from cultcache_py import SchemaConflictError
Catalog writers merge entries that share an id; every reader resolves an id by its owner first Entries that share a schema id are one schema seen by different writers, so the writer lists every id any of them lists, in any order: C# single-file and directory (CultSchemaCatalogEntry.MergeById), TypeScript and Python. Rust synthesises one entry per id and has no such merge. An id that one entry owns and another lists as compatible names the owner in Rust, TypeScript, Python and the TS inspector, as C# already did (own-id-over-compatible-v3). Also: a directory manifest in an unreadable format is the typed refusal naming it, cultnet-py applies and reports a record under the local schema id, and Rust's error prints its cause once. Vectors for the Mesh layout discriminator: a nil and an integer at catalog slot 5. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 08:17:59 +02:00
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.msgpack"
Writers derive the catalog from the records they write; the merging is deleted Every writer (C# single-file and directory, TypeScript, Python; Rust already did) chooses, for each schema id a record carries, one entry that publishes it: the entry that owns the id (a registered descriptor, else an arrived entry), else one that lists it as compatible. A registered descriptor wins over an arrived entry with the same own id. Entries are written as chosen, with no union of compatible ids and no dependence on arrival order; an entry no record needs is not written. Two entries of one tier that share an own id and disagree on the schema name, records of different types under one id, or a record no chosen entry publishes refuse the write with a typed error (CultSchemaConflictException, SchemaConflictError) naming the id, the names and a record key, and the store is left as it was. CultSchemaCatalogEntry.MergeById and the Python and TypeScript merge paths are gone. A cache-held record unchanged since the file was read is rewritten under its registered id (C# single-file and directory), so an entry that lists an older id is not needed to publish it. Changelogs for cultnet-py (removed export) and cultcache-py. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 10:03:34 +02:00
path.write_bytes(msgpack.packb([
"cultcache.store.v1", [self._raw_entry(self._writer_entry("tests.x", "tests.legacy", "stale", ("tests.x", "old")))],
[["z", "old", "2026-09-30T00:00:00Z", msgpack.packb({"z": 1}, use_bin_type=True)]],
], use_bin_type=True))
before = path.read_bytes()
with self.assertRaises(SchemaConflictError) as refused:
SingleFileMessagePackBackingStore(path).push(
self._writer_record("a", "tests.legacy", "tests.x", self._writer_entry("tests.x", "tests.legacy", "fresh", ("tests.x",)))
)
self.assertEqual(refused.exception.schema_id, "old")
self.assertEqual(refused.exception.record_key, "z")
self.assertEqual(path.read_bytes(), before)
Catalog writers merge entries that share an id; every reader resolves an id by its owner first Entries that share a schema id are one schema seen by different writers, so the writer lists every id any of them lists, in any order: C# single-file and directory (CultSchemaCatalogEntry.MergeById), TypeScript and Python. Rust synthesises one entry per id and has no such merge. An id that one entry owns and another lists as compatible names the owner in Rust, TypeScript, Python and the TS inspector, as C# already did (own-id-over-compatible-v3). Also: a directory manifest in an unreadable format is the typed refusal naming it, cultnet-py applies and reports a record under the local schema id, and Rust's error prints its cause once. Vectors for the Mesh layout discriminator: a nil and an integer at catalog slot 5. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 08:17:59 +02:00
def test_an_id_an_entry_owns_names_that_entry_not_one_that_lists_it_as_compatible(self) -> None:
One local schema id in cultmesh-py; catalog writers merge only the same schema; a compatible entry before its owner reads as the owner cultnet-py's resolve_document_and_schema_id_for_raw_record is replaced by resolve_document_for_raw_record, which resolves the document only; cultmesh-py keys previous values by the local schema id, so a snapshot under a compatible id or the schema name reports an update with its previous value. Writers (C#, TypeScript, Python) merge same-id entries only when their schema names match; another schema's entry is dropped or the record gets a default entry under its own id, and a record's own id is claimed before an entry that lists it as compatible. The C# single-file writer prefers the entry being written now, as the directory writer does, so name, content hash and members come from the registered schema and the compatible ids are the union. Vector: compatible-before-owner-v3. A cultnet-py test's schema path no longer depends on the working directory. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 09:01:00 +02:00
for vector in ("own-id-over-compatible-v3.bin", "compatible-before-owner-v3.bin"):
store = SingleFileMessagePackBackingStore(self._C2A_VECTORS / "readability" / vector)
self.assertEqual([envelope.type for envelope in store.pull_all()], ["vectors.item", "vectors.item"], vector)
Python: a record is stamped with its registered schema id and the catalog publishes it; cultnet no longer infers a schema from a payload put_envelope, put_envelopes and load stamp the registered schema's id, and the store writer publishes every record's id, so a replicated record under a compatible id no longer leaves a store nothing can open. cultnet-py resolves a raw record by the schema ids its documents list and nothing else. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 06:54:18 +02:00
def test_a_record_loaded_under_a_compatible_schema_id_is_written_back_under_the_id_its_catalog_entry_carries(self) -> None:
import msgpack # type: ignore
document = self._foreign_id_document()
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.msgpack"
path.write_bytes(msgpack.packb([
"cultcache.store.v1",
[["tests.foreign-id.current", "tests.foreign-id", "tests.foreign_id.v1", "tests.foreign-id.current", "",
["tests.foreign-id.current", "tests.foreign-id.older"], []]],
[["old", "tests.foreign-id.older", "2026-09-30T00:00:00Z", document.encode_payload({"name": "old"})]],
], use_bin_type=True))
cache = self._open_foreign_id_store(document, path)
Python and TypeScript: tests for the store's publishing rule and the admission stamp; cultnet-py keeps its payload-based resolution cultnet-py resolves a raw record whose schema id no document lists by the schema version its payload opens with, and three cultmesh-py tests (foreign UI-generated ids) depend on it, so that resolution is restored pending a ruling. The compatible-id replication test goes through the real apply_raw_document_record and the reopened store. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 07:28:17 +02:00
self.assertEqual(cache.get_required_envelope(document, "old").schema_id, "tests.foreign-id.current")
Python: a record is stamped with its registered schema id and the catalog publishes it; cultnet no longer infers a schema from a payload put_envelope, put_envelopes and load stamp the registered schema's id, and the store writer publishes every record's id, so a replicated record under a compatible id no longer leaves a store nothing can open. cultnet-py resolves a raw record by the schema ids its documents list and nothing else. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 06:54:18 +02:00
cache.put(document, "next", {"name": "next"})
reopened = self._open_foreign_id_store(document, path)
self.assertEqual(reopened.get_required(document, "old")["name"], "old")
self.assertEqual(reopened.get_required(document, "next")["name"], "next")
Python: a record resolves to its document by schema id, and the type check runs across tiers A record now resolves to the document that owns its schema id, else one that lists it as compatible, before the catalog name is tried; a schema renamed under a stable id opens, and an id that names one document is not overruled by a name that names another. The name resolves a record only when no local document has the id. The writer refuses a write that would change which type an existing record resolves to, unless the entries of both the existing records and the written ones own the id (a rename, the registered descriptor wins). Tests pin the arrived-overwrites-registered guard in both id orders, the content-hash tie-break against the compatible-list order, sorted id processing, the cross-tier refusal and the rename, and a cache whose document does not list the id a loaded record sits under. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 11:04:07 +02:00
# The cache restamps a record on load, so a store's writer never has to: a record loaded under an id the registered document
# only lists is held, and written back by a whole write, under the id its catalog entry carries. A document that does not list
# the id a record sits under cannot have that record published by a write that leaves it in place: the put is refused, typed.
def test_a_cache_that_does_not_list_the_id_a_loaded_record_sits_under_is_refused_a_put_and_the_file_is_left(self) -> None:
import msgpack # type: ignore
from cultcache_py import SchemaConflictError
document = define_database_entry_type(
"tests.foreign-id", [("name", 0)], schema_id="tests.foreign-id.current",
schema_name="tests.foreign-id", schema_version="tests.foreign_id.v1",
)
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.msgpack"
path.write_bytes(msgpack.packb([
"cultcache.store.v1",
[["tests.foreign-id.current", "tests.foreign-id", "tests.foreign_id.v1", "tests.foreign-id.current", "",
["tests.foreign-id.current", "tests.foreign-id.older"], []]],
[["old", "tests.foreign-id.older", "2026-09-30T00:00:00Z", document.encode_payload({"name": "old"})]],
], use_bin_type=True))
before = path.read_bytes()
cache = self._open_foreign_id_store(document, path)
self.assertEqual(cache.get_required_envelope(document, "old").schema_id, "tests.foreign-id.current")
with self.assertRaises(SchemaConflictError) as refused:
cache.put(document, "next", {"name": "next"})
self.assertEqual(refused.exception.schema_id, "tests.foreign-id.older")
self.assertEqual(refused.exception.record_key, "old")
self.assertEqual(path.read_bytes(), before)
# A record resolves to a local document by its schema id, never by the name its catalog entry carries: a schema renamed under
# a stable id opens, and an id that names one document is not overruled by a name that names another.
def _store_of(self, path: Path, catalog: list, records: list) -> None:
import msgpack # type: ignore
path.write_bytes(msgpack.packb(["cultcache.store.v1", catalog, records], use_bin_type=True))
def test_a_schema_renamed_under_a_stable_id_opens_by_its_id(self) -> None:
document = define_database_entry_type(
"tests.old", [("name", 0)], schema_id="tests.stable", schema_name="tests.old", schema_version="tests.old.v1",
)
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.msgpack"
self._store_of(
path,
[["tests.stable", "tests.renamed", "tests.renamed.v1", "renamed", "", ["tests.stable"], []]],
[["a", "tests.stable", "2026-09-30T00:00:00Z", document.encode_payload({"name": "a"})]],
)
cache = self._open_foreign_id_store(document, path)
self.assertEqual(cache.get_required(document, "a")["name"], "a")
def test_a_schema_id_that_names_one_document_is_not_overruled_by_a_name_that_names_another(self) -> None:
first = define_database_entry_type("tests.a", [("name", 0)], schema_id="id.a", schema_name="tests.a", schema_version="tests.a.v1")
second = define_database_entry_type(
"tests.b", [("name", 0)], schema_id="id.b", schema_name="tests.b", schema_version="tests.b.v1", compatible_schema_ids=["id.b", "id.b.older"],
)
with tempfile.TemporaryDirectory() as tmp:
for schema_id in ("id.b", "id.b.older"):
path = Path(tmp) / (schema_id + ".msgpack")
# The entry that publishes the id carries the other document's name.
self._store_of(
path,
[["id.b", "tests.a", "tests.b.v1", "h", "", ["id.b", "id.b.older"], []]],
[["k", schema_id, "2026-09-30T00:00:00Z", second.encode_payload({"name": "k"})]],
)
cache = CultCache.builder().register_document_type(first).register_document_type(second).add_generic_store(
SingleFileMessagePackBackingStore(path)
).build()
cache.pull_all_backing_stores()
self.assertEqual(cache.get_required(second, "k")["name"], "k", schema_id)
self.assertIsNone(cache.get(first, "k"), schema_id)
Python: one registered document carries each schema id; a second claimant is refused typed register_document_type refuses a document whose own or compatible ids overlap another registered document's, raising SchemaConflictError (empty record key, both schema names, a message naming both document types), as TypeScript does. Before, the record resolved to whichever document the dict held first (Soul F4). With one carrier per id, resolution by id is a single pass. SchemaConflictError takes an optional message for this use. The test that pinned owner-over-lister resolution inside one cache becomes a refusal test in both orders; shared compatible ids and a shared own id (neither declaring its own) are refused in both orders. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 12:35:52 +02:00
def _assert_second_claim_refused(self, first, second, schema_id: str) -> None:
Python: a document listing an id registers beside the one owning it; registration refuses typed Owner beats lister, as in C#. A document owns its schema id; others may list it as compatible beside its owner, in either registration order (batch 9 refused any overlap, which forbade keeping v1 beside a v2 that declares v1's id; Soul F8). A record resolves to the document owning its id, else to the one document listing it. Several listers and no owner name no single document, and the load is refused with SchemaConflictError naming the id, the record key and the listers: refusing at registration would depend on whether the owner had registered yet. Every registration refusal is now SchemaConflictError (Soul F10): a second owner of one schema id, a second document with one schema name, and a second definition of one document type. Each names the new document's id and both schema names and document types. Before, the last two raised CultCacheError. Tests: an owner and a lister register in both orders and each reads its own records, and a lister alone reads the owner's id; two listers register in both orders, a record under their id is refused typed without its owner and is the owner's with it, registered before or after them; two owners, one schema name and one document type twice are each refused as a schema conflict. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 15:31:25 +02:00
"""A registration refusal is typed, names the id and both schema names and document types, and the cache keeps what it
had."""
Python: one registered document carries each schema id; a second claimant is refused typed register_document_type refuses a document whose own or compatible ids overlap another registered document's, raising SchemaConflictError (empty record key, both schema names, a message naming both document types), as TypeScript does. Before, the record resolved to whichever document the dict held first (Soul F4). With one carrier per id, resolution by id is a single pass. SchemaConflictError takes an optional message for this use. The test that pinned owner-over-lister resolution inside one cache becomes a refusal test in both orders; shared compatible ids and a shared own id (neither declaring its own) are refused in both orders. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 12:35:52 +02:00
from cultcache_py import SchemaConflictError
cache = CultCache()
cache.register_document_type(first)
with self.assertRaises(SchemaConflictError) as refused:
cache.register_document_type(second)
self.assertEqual(refused.exception.schema_id, schema_id)
self.assertEqual(refused.exception.schema_names, [first.catalog_entry().schema_name, second.catalog_entry().schema_name])
self.assertIn(first.type, str(refused.exception))
self.assertIn(second.type, str(refused.exception))
Python: a document listing an id registers beside the one owning it; registration refuses typed Owner beats lister, as in C#. A document owns its schema id; others may list it as compatible beside its owner, in either registration order (batch 9 refused any overlap, which forbade keeping v1 beside a v2 that declares v1's id; Soul F8). A record resolves to the document owning its id, else to the one document listing it. Several listers and no owner name no single document, and the load is refused with SchemaConflictError naming the id, the record key and the listers: refusing at registration would depend on whether the owner had registered yet. Every registration refusal is now SchemaConflictError (Soul F10): a second owner of one schema id, a second document with one schema name, and a second definition of one document type. Each names the new document's id and both schema names and document types. Before, the last two raised CultCacheError. Tests: an owner and a lister register in both orders and each reads its own records, and a lister alone reads the owner's id; two listers register in both orders, a record under their id is refused typed without its owner and is the owner's with it, registered before or after them; two owners, one schema name and one document type twice are each refused as a schema conflict. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 15:31:25 +02:00
if second is not first:
with self.assertRaisesRegex(CultCacheError, "not registered"):
cache.put(second, "k", {"name": "k"})
def _pull(self, documents: list, catalog: list, records: list) -> CultCache:
tmp = tempfile.mkdtemp()
path = Path(tmp) / "store.msgpack"
self._store_of(path, catalog, records)
builder = CultCache.builder()
for document in documents:
builder = builder.register_document_type(document)
cache = builder.add_generic_store(SingleFileMessagePackBackingStore(path)).build()
cache.pull_all_backing_stores()
return cache
Python: one registered document carries each schema id; a second claimant is refused typed register_document_type refuses a document whose own or compatible ids overlap another registered document's, raising SchemaConflictError (empty record key, both schema names, a message naming both document types), as TypeScript does. Before, the record resolved to whichever document the dict held first (Soul F4). With one carrier per id, resolution by id is a single pass. SchemaConflictError takes an optional message for this use. The test that pinned owner-over-lister resolution inside one cache becomes a refusal test in both orders; shared compatible ids and a shared own id (neither declaring its own) are refused in both orders. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 12:35:52 +02:00
Python: a document listing an id registers beside the one owning it; registration refuses typed Owner beats lister, as in C#. A document owns its schema id; others may list it as compatible beside its owner, in either registration order (batch 9 refused any overlap, which forbade keeping v1 beside a v2 that declares v1's id; Soul F8). A record resolves to the document owning its id, else to the one document listing it. Several listers and no owner name no single document, and the load is refused with SchemaConflictError naming the id, the record key and the listers: refusing at registration would depend on whether the owner had registered yet. Every registration refusal is now SchemaConflictError (Soul F10): a second owner of one schema id, a second document with one schema name, and a second definition of one document type. Each names the new document's id and both schema names and document types. Before, the last two raised CultCacheError. Tests: an owner and a lister register in both orders and each reads its own records, and a lister alone reads the owner's id; two listers register in both orders, a record under their id is refused typed without its owner and is the owner's with it, registered before or after them; two owners, one schema name and one document type twice are each refused as a schema conflict. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 15:31:25 +02:00
def test_a_document_listing_an_id_another_owns_registers_beside_it_in_either_order_and_the_owner_wins(self) -> None:
owner = define_database_entry_type("tests.owner", [("name", 0)], schema_id="id.owned", schema_name="tests.owner", schema_version="tests.owner.v1")
Python test: a document that owns an id is resolved before one registered earlier that lists it Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 11:14:38 +02:00
lister = define_database_entry_type(
"tests.lister", [("name", 0)], schema_id="id.lister", schema_name="tests.lister", schema_version="tests.lister.v1",
compatible_schema_ids=["id.lister", "id.owned"],
)
Python: a document listing an id registers beside the one owning it; registration refuses typed Owner beats lister, as in C#. A document owns its schema id; others may list it as compatible beside its owner, in either registration order (batch 9 refused any overlap, which forbade keeping v1 beside a v2 that declares v1's id; Soul F8). A record resolves to the document owning its id, else to the one document listing it. Several listers and no owner name no single document, and the load is refused with SchemaConflictError naming the id, the record key and the listers: refusing at registration would depend on whether the owner had registered yet. Every registration refusal is now SchemaConflictError (Soul F10): a second owner of one schema id, a second document with one schema name, and a second definition of one document type. Each names the new document's id and both schema names and document types. Before, the last two raised CultCacheError. Tests: an owner and a lister register in both orders and each reads its own records, and a lister alone reads the owner's id; two listers register in both orders, a record under their id is refused typed without its owner and is the owner's with it, registered before or after them; two owners, one schema name and one document type twice are each refused as a schema conflict. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 15:31:25 +02:00
catalog = [
["id.owned", "tests.owner", "tests.owner.v1", "h1", "", ["id.owned"], []],
["id.lister", "tests.lister", "tests.lister.v1", "h2", "", ["id.lister", "id.owned"], []],
]
records = [
["o", "id.owned", "2026-09-30T00:00:00Z", owner.encode_payload({"name": "o"})],
["l", "id.lister", "2026-09-30T00:00:00Z", lister.encode_payload({"name": "l"})],
]
for order in ([owner, lister], [lister, owner]):
cache = self._pull(order, catalog, records)
self.assertEqual(cache.get_required(owner, "o")["name"], "o")
self.assertIsNone(cache.get(lister, "o"))
self.assertEqual(cache.get_required(lister, "l")["name"], "l")
self.assertEqual(self._pull([lister], catalog, records[:1]).get_required(lister, "o")["name"], "o")
def test_two_documents_listing_one_id_register_and_a_record_under_it_needs_an_owner(self) -> None:
from cultcache_py import SchemaConflictError
Python: one registered document carries each schema id; a second claimant is refused typed register_document_type refuses a document whose own or compatible ids overlap another registered document's, raising SchemaConflictError (empty record key, both schema names, a message naming both document types), as TypeScript does. Before, the record resolved to whichever document the dict held first (Soul F4). With one carrier per id, resolution by id is a single pass. SchemaConflictError takes an optional message for this use. The test that pinned owner-over-lister resolution inside one cache becomes a refusal test in both orders; shared compatible ids and a shared own id (neither declaring its own) are refused in both orders. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 12:35:52 +02:00
a = define_database_entry_type(
"tests.list-a", [("name", 0)], schema_id="id.a", schema_name="tests.list-a", compatible_schema_ids=["id.a", "id.shared"],
)
b = define_database_entry_type(
"tests.list-b", [("name", 0)], schema_id="id.b", schema_name="tests.list-b", compatible_schema_ids=["id.b", "id.shared"],
)
Python: a document listing an id registers beside the one owning it; registration refuses typed Owner beats lister, as in C#. A document owns its schema id; others may list it as compatible beside its owner, in either registration order (batch 9 refused any overlap, which forbade keeping v1 beside a v2 that declares v1's id; Soul F8). A record resolves to the document owning its id, else to the one document listing it. Several listers and no owner name no single document, and the load is refused with SchemaConflictError naming the id, the record key and the listers: refusing at registration would depend on whether the owner had registered yet. Every registration refusal is now SchemaConflictError (Soul F10): a second owner of one schema id, a second document with one schema name, and a second definition of one document type. Each names the new document's id and both schema names and document types. Before, the last two raised CultCacheError. Tests: an owner and a lister register in both orders and each reads its own records, and a lister alone reads the owner's id; two listers register in both orders, a record under their id is refused typed without its owner and is the owner's with it, registered before or after them; two owners, one schema name and one document type twice are each refused as a schema conflict. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 15:31:25 +02:00
owner = define_database_entry_type("tests.shared", [("name", 0)], schema_id="id.shared", schema_name="tests.shared")
catalog = [["id.shared", "tests.shared", "tests.shared.v1", "h", "", ["id.shared"], []]]
records = [["k", "id.shared", "2026-09-30T00:00:00Z", owner.encode_payload({"name": "k"})]]
for listers in ([a, b], [b, a]):
with self.assertRaises(SchemaConflictError) as refused:
self._pull(listers, catalog, records)
self.assertEqual(refused.exception.schema_id, "id.shared")
self.assertEqual(refused.exception.record_key, "k")
self.assertEqual(refused.exception.schema_names, ["tests.list-a", "tests.list-b"])
for documents in ([*listers, owner], [owner, *listers]):
cache = self._pull(documents, catalog, records)
self.assertEqual(cache.get_required(owner, "k")["name"], "k")
Python: one registered document carries each schema id; a second claimant is refused typed register_document_type refuses a document whose own or compatible ids overlap another registered document's, raising SchemaConflictError (empty record key, both schema names, a message naming both document types), as TypeScript does. Before, the record resolved to whichever document the dict held first (Soul F4). With one carrier per id, resolution by id is a single pass. SchemaConflictError takes an optional message for this use. The test that pinned owner-over-lister resolution inside one cache becomes a refusal test in both orders; shared compatible ids and a shared own id (neither declaring its own) are refused in both orders. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 12:35:52 +02:00
def test_two_documents_owning_one_schema_id_are_refused_in_either_order(self) -> None:
# Each declares an older id and not its own, so the id they share is only ever their own.
a = define_database_entry_type("tests.own-a", [("name", 0)], schema_id="id.same", schema_name="tests.own-a", compatible_schema_ids=["id.a-older"])
b = define_database_entry_type("tests.own-b", [("name", 0)], schema_id="id.same", schema_name="tests.own-b", compatible_schema_ids=["id.b-older"])
self._assert_second_claim_refused(a, b, "id.same")
self._assert_second_claim_refused(b, a, "id.same")
Python test: a document that owns an id is resolved before one registered earlier that lists it Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 11:14:38 +02:00
Python: a document listing an id registers beside the one owning it; registration refuses typed Owner beats lister, as in C#. A document owns its schema id; others may list it as compatible beside its owner, in either registration order (batch 9 refused any overlap, which forbade keeping v1 beside a v2 that declares v1's id; Soul F8). A record resolves to the document owning its id, else to the one document listing it. Several listers and no owner name no single document, and the load is refused with SchemaConflictError naming the id, the record key and the listers: refusing at registration would depend on whether the owner had registered yet. Every registration refusal is now SchemaConflictError (Soul F10): a second owner of one schema id, a second document with one schema name, and a second definition of one document type. Each names the new document's id and both schema names and document types. Before, the last two raised CultCacheError. Tests: an owner and a lister register in both orders and each reads its own records, and a lister alone reads the owner's id; two listers register in both orders, a record under their id is refused typed without its owner and is the owner's with it, registered before or after them; two owners, one schema name and one document type twice are each refused as a schema conflict. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 15:31:25 +02:00
def test_every_registration_refusal_is_a_schema_conflict(self) -> None:
first = define_database_entry_type("tests.named", [("name", 0)], schema_id="id.first", schema_name="tests.named")
same_name = define_database_entry_type("tests.renamed", [("name", 0)], schema_id="id.second", schema_name="tests.named")
self._assert_second_claim_refused(first, same_name, "id.second")
self._assert_second_claim_refused(first, first, "id.first")
Add Python CultCache wire parity package
2026-06-13 20:35:58 +01:00
def test_interop_cli_helpers_round_trip_v1_store(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
store_path = str(Path(tmp) / "cache.msgpack")
written = write_note(store_path, "python-test")
loaded = read_note(store_path)
self.assertEqual(loaded["documentId"], written["documentId"])
self.assertEqual(loaded["authorRuntimeId"], "python-test")
self.assertIn("interop", loaded["tags"])
Give every cultcache-py document type one home store Cut 5 (Python). stores_by_type maps each type to one store and generic_stores becomes a single generic_store. add_backing_store raises on a type already claimed and add_generic_store on a second generic store; a refused registration claims nothing. _stores_for_type and _all_specific_stores are deleted. _store_for_type returns the home store and raises when stores are attached but none is home to the type. It returns None only for a cache with no stores, which the contract keeps as an in-memory cache. put, put_envelope, put_envelopes and delete each write one store; pull_all_backing_stores pulls each distinct store once. Tests: test_second_generic_store_rejected, test_type_claimed_twice_rejected, test_types_route_to_home_store. No bytes change. Version 0.2.0 -> 0.3.0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 00:37:31 +02:00
def test_second_generic_store_rejected(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
cache = CultCache()
cache.add_generic_store(SingleFileMessagePackBackingStore(Path(tmp) / "a.cc"))
with self.assertRaisesRegex(CultCacheError, "second generic store"):
cache.add_generic_store(SingleFileMessagePackBackingStore(Path(tmp) / "b.cc"))
def test_type_claimed_twice_rejected(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
cache = CultCache()
cache.add_backing_store(SingleFileMessagePackBackingStore(Path(tmp) / "a.cc"), ["item"])
with self.assertRaisesRegex(CultCacheError, "already routed"):
cache.add_backing_store(SingleFileMessagePackBackingStore(Path(tmp) / "b.cc"), ["settings", "item"])
# A refused registration claims nothing, so "settings" is still free.
cache.add_backing_store(SingleFileMessagePackBackingStore(Path(tmp) / "c.cc"), ["settings"])
def test_types_route_to_home_store(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
item = item_doc()
settings = define_database_entry_type("settings", [("theme", 0)])
generic_path = Path(tmp) / "generic.cc"
settings_path = Path(tmp) / "settings.cc"
def build() -> CultCache:
return (
CultCache.builder()
.register_document_type(item)
.register_document_type(settings)
.add_backing_store(SingleFileMessagePackBackingStore(settings_path), ["settings"])
.add_generic_store(SingleFileMessagePackBackingStore(generic_path))
.build()
)
cache = build()
cache.put(item, "item:potion", Item(name="Potion", category="Consumable", value=50))
cache.put(settings, "app", {"theme": "ash"})
generic = SingleFileMessagePackBackingStore(generic_path).pull_all()
routed = SingleFileMessagePackBackingStore(settings_path).pull_all()
self.assertEqual([(e.key, e.type) for e in generic], [("item:potion", "item")])
self.assertEqual([(e.key, e.type) for e in routed], [("app", "settings")])
cache.delete(settings, "app")
self.assertEqual(SingleFileMessagePackBackingStore(settings_path).pull_all(), [])
self.assertEqual(len(SingleFileMessagePackBackingStore(generic_path).pull_all()), 1)
reloaded = build()
reloaded.pull_all_backing_stores()
self.assertEqual(reloaded.get_required(item, "item:potion").value, 50)
self.assertIsNone(reloaded.get(settings, "app"))
Keep cultcache-py record homes stable and refuse strays at load Conform cultcache-py to the store-composition contract, following C#: - add_backing_store and add_generic_store raise, attaching nothing, when the new store would move the home of any document type the cache already holds. - add_backing_store(store, []) attaches the generic store (same as TypeScript, Rust and C#'s untyped attach) instead of silently doing nothing. - pull_all_backing_stores stages every store's records and replaces the cache's view only when all loaded. A record delivered by a store that is not its home raises naming the key and both stores; nothing is admitted and no store is written. The zero-store in-memory test passes on 6bb41f3 as well (Python already conformed); the other new tests fail there. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 00:50:48 +02:00
def test_attach_that_would_move_a_held_record_home_is_refused(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
settings = define_database_entry_type("settings", [("theme", 0)])
generic_path = Path(tmp) / "generic.cc"
settings_path = Path(tmp) / "settings.cc"
cache = (
CultCache.builder()
.register_document_type(settings)
.add_generic_store(SingleFileMessagePackBackingStore(generic_path))
.build()
)
cache.put(settings, "app", {"theme": "v1-generic"})
with self.assertRaisesRegex(
CultCacheError, "would move settings from the generic store to the store routed to settings"
):
cache.add_backing_store(SingleFileMessagePackBackingStore(settings_path), ["settings"])
# Nothing attached: the next write still lands in the generic store.
cache.put(settings, "app", {"theme": "v2-generic"})
generic = SingleFileMessagePackBackingStore(generic_path).pull_all()
self.assertEqual([(e.key, e.type) for e in generic], [("app", "settings")])
self.assertFalse(settings_path.exists())
def test_load_from_a_store_that_is_not_the_record_home_is_refused(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
settings = define_database_entry_type("settings", [("theme", 0)])
generic_path = Path(tmp) / "generic.cc"
settings_path = Path(tmp) / "settings.cc"
writer = CultCache.builder().register_document_type(settings).add_generic_store(
SingleFileMessagePackBackingStore(generic_path)
).build()
writer.put(settings, "app", {"theme": "stray"})
generic_bytes = generic_path.read_bytes()
cache = (
CultCache.builder()
.register_document_type(settings)
.add_backing_store(SingleFileMessagePackBackingStore(settings_path), ["settings"])
.add_generic_store(SingleFileMessagePackBackingStore(generic_path))
.build()
)
Validate a Python CultCache write whole before touching a store put, put_envelope and put_envelopes now compute the new records' name and index keys and resolve the home store before any push; applying to the cache afterwards runs no extractor. Each held record keeps the lookup keys it produced at admission, so overwrite and delete remove lookups without calling user extractors. put_envelope and put_envelopes refuse an empty key and a global under any key but __global__. Per-type dicts are created only after the store accepts. register_document_type, register_name_lookup and register_index derive every lookup under the candidate extractors first and install extractors and lookups together, so a raising extractor installs nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 11:48:59 +02:00
# The cache already holds a record, in its home store, before the refused load.
cache.put(settings, "held", {"theme": "home"})
before = cache.snapshot_envelopes()
settings_bytes = settings_path.read_bytes()
Keep cultcache-py record homes stable and refuse strays at load Conform cultcache-py to the store-composition contract, following C#: - add_backing_store and add_generic_store raise, attaching nothing, when the new store would move the home of any document type the cache already holds. - add_backing_store(store, []) attaches the generic store (same as TypeScript, Rust and C#'s untyped attach) instead of silently doing nothing. - pull_all_backing_stores stages every store's records and replaces the cache's view only when all loaded. A record delivered by a store that is not its home raises naming the key and both stores; nothing is admitted and no store is written. The zero-store in-memory test passes on 6bb41f3 as well (Python already conformed); the other new tests fail there. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 00:50:48 +02:00
with self.assertRaisesRegex(
CultCacheError,
"settings record app was loaded from the generic store, but its home is the store routed to settings",
):
cache.pull_all_backing_stores()
self.assertIsNone(cache.get(settings, "app"))
Validate a Python CultCache write whole before touching a store put, put_envelope and put_envelopes now compute the new records' name and index keys and resolve the home store before any push; applying to the cache afterwards runs no extractor. Each held record keeps the lookup keys it produced at admission, so overwrite and delete remove lookups without calling user extractors. put_envelope and put_envelopes refuse an empty key and a global under any key but __global__. Per-type dicts are created only after the store accepts. register_document_type, register_name_lookup and register_index derive every lookup under the candidate extractors first and install extractors and lookups together, so a raising extractor installs nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 11:48:59 +02:00
self.assertEqual(cache.snapshot_envelopes(), before)
self.assertEqual(cache.get_required(settings, "held"), {"theme": "home"})
Keep cultcache-py record homes stable and refuse strays at load Conform cultcache-py to the store-composition contract, following C#: - add_backing_store and add_generic_store raise, attaching nothing, when the new store would move the home of any document type the cache already holds. - add_backing_store(store, []) attaches the generic store (same as TypeScript, Rust and C#'s untyped attach) instead of silently doing nothing. - pull_all_backing_stores stages every store's records and replaces the cache's view only when all loaded. A record delivered by a store that is not its home raises naming the key and both stores; nothing is admitted and no store is written. The zero-store in-memory test passes on 6bb41f3 as well (Python already conformed); the other new tests fail there. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 00:50:48 +02:00
self.assertEqual(generic_path.read_bytes(), generic_bytes)
Validate a Python CultCache write whole before touching a store put, put_envelope and put_envelopes now compute the new records' name and index keys and resolve the home store before any push; applying to the cache afterwards runs no extractor. Each held record keeps the lookup keys it produced at admission, so overwrite and delete remove lookups without calling user extractors. put_envelope and put_envelopes refuse an empty key and a global under any key but __global__. Per-type dicts are created only after the store accepts. register_document_type, register_name_lookup and register_index derive every lookup under the candidate extractors first and install extractors and lookups together, so a raising extractor installs nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 11:48:59 +02:00
self.assertEqual(settings_path.read_bytes(), settings_bytes)
Keep cultcache-py record homes stable and refuse strays at load Conform cultcache-py to the store-composition contract, following C#: - add_backing_store and add_generic_store raise, attaching nothing, when the new store would move the home of any document type the cache already holds. - add_backing_store(store, []) attaches the generic store (same as TypeScript, Rust and C#'s untyped attach) instead of silently doing nothing. - pull_all_backing_stores stages every store's records and replaces the cache's view only when all loaded. A record delivered by a store that is not its home raises naming the key and both stores; nothing is admitted and no store is written. The zero-store in-memory test passes on 6bb41f3 as well (Python already conformed); the other new tests fail there. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 00:50:48 +02:00
Refuse duplicate global records while staging a pull A load is all-or-nothing. TypeScript checked for a second record of a global type inside #applyHydratedEntry, which runs after #resetHydratedState, so a refused pull left the cache cleared and partly reloaded. The check now runs in the staging pass beside the home check, before anything is cleared; the apply step's pull-only branch is gone. Python already validated before swapping, but its check only fired when the __global__ record arrived second, so a store holding __global__ then another key of a global type loaded both. Any second record of a global type is now refused regardless of order. Rust has no global document types and replaces its entry map only after staging; it had neither flaw. pullWithDuplicateGlobalLeavesCacheUnchanged (TS) and test_pull_with_duplicate_global_leaves_cache_unchanged (Python) fail on 9ab1706 and pass here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 00:57:09 +02:00
def test_pull_with_duplicate_global_leaves_cache_unchanged(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
item = item_doc()
settings = define_document_type("settings", global_document=True)
store_path = Path(tmp) / "generic.cc"
cache = (
CultCache.builder()
.register_document_type(item)
.register_document_type(settings)
.add_generic_store(SingleFileMessagePackBackingStore(store_path))
.build()
)
cache.put(item, "item:potion", Item(name="Potion", category="Consumable", value=50))
cache.put_global(settings, {"theme": "ash"})
before = cache.snapshot_envelopes()
# A second record of the global type lands in the store file behind the cache's back.
stray = replace(cache.get_required_envelope(settings, CultCache.GLOBAL_KEY), key="stray")
SingleFileMessagePackBackingStore(store_path).push(stray)
with self.assertRaisesRegex(CultCacheError, "Duplicate global document for type: settings"):
cache.pull_all_backing_stores()
self.assertEqual(cache.snapshot_envelopes(), before)
self.assertEqual(cache.get_required(item, "item:potion").value, 50)
self.assertEqual(cache.get_key_by_name(item, "Potion"), "item:potion")
self.assertEqual(cache.get_required_global(settings)["theme"], "ash")
Build the whole next state before a pull swaps it in A pull is all-or-nothing, but TS cleared the cache and then ran user name and index accessors while re-applying entries, and Python swapped values in and then rebuilt lookups. An accessor that threw left the cache partly loaded. TS: the hydrated maps (entries, type entry ids, name and index lookups, global keys) are one HydratedState. The pull builds a fresh one, running every check and accessor, and replaces #hydrated in one assignment at the end. The reset-then-apply sequence, #resetHydratedState, the staged global-key map and the three per-map getOrCreate helpers are deleted; apply/remove take the target state. The duplicate-global check now lives only in apply. Name and index lookup rebuilds build a local map before assigning. Python: the pull derives names and indexes into local dicts and assigns values, envelopes, names and indexes together; _rebuild_indexes delegates to the same pure _derive_lookups. Rust already stages into a local map and assigns self.entries last, with no derived lookups; unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 01:03:30 +02:00
def test_pull_with_throwing_index_accessor_leaves_cache_unchanged(self) -> None:
def category(item: Item) -> str:
if item.name == "Bomb":
raise ValueError("category accessor refuses Bomb")
return item.category
with tempfile.TemporaryDirectory() as tmp:
item = define_document_type(
"item",
encode=lambda value: asdict(value),
decode=lambda raw: Item(**raw),
name="name",
indexes={"category": category},
)
settings = define_document_type("settings", global_document=True)
store_path = Path(tmp) / "generic.cc"
cache = (
CultCache.builder()
.register_document_type(item)
.register_document_type(settings)
.add_generic_store(SingleFileMessagePackBackingStore(store_path))
.build()
)
cache.put(item, "item:potion", Item(name="Potion", category="Consumable", value=50))
cache.put_global(settings, {"theme": "ash"})
before_envelopes = cache.snapshot_envelopes()
before_values = cache.snapshot()
# A record whose index accessor throws lands in the store file behind the cache's back.
bomb = replace(
cache.get_required_envelope(item, "item:potion"),
key="item:bomb",
payload=item.encode_payload(Item(name="Bomb", category="Explosive", value=1)),
)
SingleFileMessagePackBackingStore(store_path).push(bomb)
with self.assertRaisesRegex(ValueError, "category accessor refuses Bomb"):
cache.pull_all_backing_stores()
self.assertEqual(cache.snapshot_envelopes(), before_envelopes)
self.assertEqual(cache.snapshot(), before_values)
self.assertEqual(cache.get_required(item, "item:potion").value, 50)
self.assertIsNone(cache.get(item, "item:bomb"))
self.assertEqual(cache.get_key_by_name(item, "Potion"), "item:potion")
self.assertIsNone(cache.get_key_by_name(item, "Bomb"))
self.assertEqual(cache.get_key_by_index(item, "category", "Consumable"), "item:potion")
self.assertEqual(cache.get_required_global(settings)["theme"], "ash")
Validate a Python CultCache write whole before touching a store put, put_envelope and put_envelopes now compute the new records' name and index keys and resolve the home store before any push; applying to the cache afterwards runs no extractor. Each held record keeps the lookup keys it produced at admission, so overwrite and delete remove lookups without calling user extractors. put_envelope and put_envelopes refuse an empty key and a global under any key but __global__. Per-type dicts are created only after the store accepts. register_document_type, register_name_lookup and register_index derive every lookup under the candidate extractors first and install extractors and lookups together, so a raising extractor installs nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 11:48:59 +02:00
def _store_state(self, path: Path) -> list[tuple[str, str, bytes]]:
return sorted((e.type, e.key, e.payload) for e in SingleFileMessagePackBackingStore(path).pull_all())
def test_put_with_throwing_name_extractor_changes_neither_store_nor_cache(self) -> None:
def name(value: dict) -> str:
if value["n"] == "bad":
raise ValueError("extractor refuses bad")
return value["n"]
with tempfile.TemporaryDirectory() as tmp:
item = define_document_type("item", name=name)
path = Path(tmp) / "generic.cc"
cache = CultCache.builder().register_document_type(item).add_generic_store(
SingleFileMessagePackBackingStore(path)
).build()
cache.put(item, "k", {"n": "good"})
store_before, cache_before = self._store_state(path), cache.snapshot_envelopes()
with self.assertRaisesRegex(ValueError, "extractor refuses bad"):
cache.put(item, "k", {"n": "bad"})
self.assertEqual(self._store_state(path), store_before)
self.assertEqual(cache.snapshot_envelopes(), cache_before)
self.assertEqual(cache.get_required(item, "k"), {"n": "good"})
self.assertEqual(cache.get_key_by_name(item, "good"), "k")
def test_overwrite_under_a_registered_extractor_that_now_raises_changes_neither_store_nor_cache(self) -> None:
refusing = {"on": False}
def category(value: dict) -> str:
if refusing["on"]:
raise ValueError("extractor refuses now")
return value["c"]
with tempfile.TemporaryDirectory() as tmp:
item = define_document_type("item")
path = Path(tmp) / "generic.cc"
cache = CultCache.builder().register_document_type(item).add_generic_store(
SingleFileMessagePackBackingStore(path)
).build()
cache.put(item, "k", {"c": "old"})
cache.register_index(item, "category", category)
refusing["on"] = True
store_before, cache_before = self._store_state(path), cache.snapshot_envelopes()
with self.assertRaisesRegex(ValueError, "extractor refuses now"):
cache.put(item, "k", {"c": "new"})
self.assertEqual(self._store_state(path), store_before)
self.assertEqual(cache.snapshot_envelopes(), cache_before)
self.assertEqual(cache.get_key_by_index(item, "category", "old"), "k")
# Deleting runs no extractor: it uses the lookup keys stored when the record was admitted.
cache.delete(item, "k")
self.assertIsNone(cache.get_key_by_index(item, "category", "old"))
def test_registering_an_extractor_that_raises_on_a_held_value_installs_nothing(self) -> None:
def name(value: dict) -> str:
if value["n"] == "bad":
raise ValueError("extractor refuses bad")
return value["n"]
with tempfile.TemporaryDirectory() as tmp:
item = define_document_type("item")
path = Path(tmp) / "generic.cc"
cache = CultCache.builder().register_document_type(item).add_generic_store(
SingleFileMessagePackBackingStore(path)
).build()
cache.put(item, "old", {"n": "bad"})
with self.assertRaisesRegex(ValueError, "extractor refuses bad"):
cache.register_name_lookup(item, name)
cache.put(item, "old", {"n": "fine"})
self.assertEqual(cache.get_required(item, "old"), {"n": "fine"})
self.assertIsNone(cache.get_key_by_name(item, "fine"))
self.assertEqual([item.decode_payload(e.payload) for e in SingleFileMessagePackBackingStore(path).pull_all()], [{"n": "fine"}])
def test_put_envelopes_with_throwing_extractor_changes_neither_store_nor_cache(self) -> None:
def name(value: dict) -> str:
if value["n"] == "bad":
raise ValueError("extractor refuses bad")
return value["n"]
with tempfile.TemporaryDirectory() as tmp:
item = define_document_type("item", name=name)
path = Path(tmp) / "generic.cc"
cache = CultCache.builder().register_document_type(item).add_generic_store(
SingleFileMessagePackBackingStore(path)
).build()
cache.put(item, "seed", {"n": "seed"})
seed = cache.get_required_envelope(item, "seed")
first = replace(seed, key="a", payload=item.encode_payload({"n": "a"}))
second = replace(seed, key="b", payload=item.encode_payload({"n": "bad"}))
store_before, cache_before = self._store_state(path), cache.snapshot_envelopes()
with self.assertRaisesRegex(ValueError, "extractor refuses bad"):
cache.put_envelopes(item, [first, second])
self.assertEqual(self._store_state(path), store_before)
self.assertEqual(cache.snapshot_envelopes(), cache_before)
self.assertIsNone(cache.get_key_by_name(item, "a"))
def test_global_replace_with_failing_store_push_keeps_old_global(self) -> None:
class FailingPushStore(SingleFileMessagePackBackingStore):
fail = False
def push(self, envelope):
if self.fail:
raise OSError("disk full")
super().push(envelope)
with tempfile.TemporaryDirectory() as tmp:
settings = define_document_type("settings", global_document=True)
path = Path(tmp) / "generic.cc"
store = FailingPushStore(path)
cache = CultCache.builder().register_document_type(settings).add_generic_store(store).build()
cache.put_global(settings, {"theme": "base"})
store_before, cache_before = self._store_state(path), cache.snapshot_envelopes()
store.fail = True
with self.assertRaisesRegex(OSError, "disk full"):
cache.put_global(settings, {"theme": "new"})
with self.assertRaisesRegex(CultCacheError, "must use key __global__"):
cache.put(settings, "other", {"theme": "new"})
self.assertEqual(self._store_state(path), store_before)
self.assertEqual(cache.snapshot_envelopes(), cache_before)
self.assertEqual(cache.get_required_global(settings), {"theme": "base"})
Serialize the Python cache behind one reentrant lock per instance The Python cache took no lock, so "refused write changes neither store nor cache" and the one-home rule held only for one caller at a time. A put paused between its store push and its apply let a concurrent attach route the type to a new home (the record stayed in the old store), and let a second global put reach disk first so cache and disk disagreed. CultCache now holds a threading.RLock around attach, pull, registration, every write and delete (globals and update included), name/index resolution and snapshots, matching the C# per-cache gate and the TS serial queue. Single-dict reads stay unlocked. No public API change. Tests: test_concurrent_put_and_attach_cannot_land_record_in_non_home_store and test_concurrent_global_puts_cannot_both_reach_disk force the interleaving with a store that pauses inside push; both fail on 4dc7f3a and pass here. The contract's Python cells drop the one-caller caveat. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 12:03:45 +02:00
@staticmethod
def _pausing_store(path: Path, barrier: threading.Barrier, resume: threading.Event):
"""A store whose first push writes, meets the other thread at the barrier, then waits until that
thread's call has returned (or a timeout: a locked cache makes the other thread wait on us)."""
class PausingStore(SingleFileMessagePackBackingStore):
pushes = 0
paused = False
other_reached_store_while_paused = False
def push(self, envelope):
self.other_reached_store_while_paused |= self.paused
self.pushes += 1
super().push(envelope)
if self.pushes == 1:
self.paused = True
barrier.wait(timeout=5)
resume.wait(timeout=0.5)
self.paused = False
return PausingStore(path)
@staticmethod
def _run_racing(first, second, resume: threading.Event) -> list[BaseException]:
errors: list[BaseException] = []
def run(action, signal):
try:
action()
except BaseException as error: # noqa: BLE001 - reported to the test
errors.append(error)
finally:
if signal:
resume.set()
threads = [threading.Thread(target=run, args=(first, False)), threading.Thread(target=run, args=(second, True))]
for thread in threads:
thread.start()
for thread in threads:
thread.join(timeout=10)
return errors
def test_concurrent_put_and_attach_cannot_land_record_in_non_home_store(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
settings = define_database_entry_type("settings", [("theme", 0)])
generic_path, settings_path = Path(tmp) / "generic.cc", Path(tmp) / "settings.cc"
barrier, resume = threading.Barrier(2), threading.Event()
generic = self._pausing_store(generic_path, barrier, resume)
cache = CultCache.builder().register_document_type(settings).add_generic_store(generic).build()
def attach() -> None:
barrier.wait(timeout=5)
cache.add_backing_store(SingleFileMessagePackBackingStore(settings_path), ["settings"])
errors = self._run_racing(lambda: cache.put(settings, "app", {"theme": "t"}), attach, resume)
# Either the attach was refused (the record is held, so its home cannot move) or it won;
# in every case each held record must be in the store that is its home.
home = SingleFileMessagePackBackingStore(generic_path if errors else settings_path)
self.assertEqual([(e.key, e.type) for e in home.pull_all()], [("app", "settings")])
self.assertTrue(all(isinstance(error, CultCacheError) for error in errors), errors)
def test_concurrent_global_puts_cannot_both_reach_disk(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
settings = define_document_type("settings", global_document=True)
path = Path(tmp) / "generic.cc"
barrier, resume = threading.Barrier(2), threading.Event()
store = self._pausing_store(path, barrier, resume)
cache = CultCache.builder().register_document_type(settings).add_generic_store(store).build()
def second() -> None:
barrier.wait(timeout=5)
cache.put_global(settings, {"theme": "second"})
errors = self._run_racing(lambda: cache.put_global(settings, {"theme": "first"}), second, resume)
self.assertEqual(errors, [])
# While the first write was between its store push and its apply, the second never reached disk.
self.assertFalse(store.other_reached_store_while_paused)
on_disk = SingleFileMessagePackBackingStore(path).pull_all()
self.assertEqual(len(on_disk), 1)
self.assertEqual(on_disk[0].payload, cache.get_required_envelope(settings, CultCache.GLOBAL_KEY).payload)
self.assertEqual(cache.get_required_global(settings), {"theme": "second"})
Refuse re-entrant Python cache mutations and adopt legacy-key globals on load Attach, pull, registration, writes and deletes raise when started on a thread already inside a mutation of the same cache (a store call, decoder, encoder, extractor or updater), instead of running inside it and corrupting the state being built. Reads stay reentrant. Internal delegation (put_global, update, delete_global, register_registry, add_backing_store) goes through private helpers so it is not re-entry. A pull adopts a global persisted under a key other than __global__ as that type's global without writing. The first write or delete of it removes the legacy record in the same call; a failed removal takes the new record back out. Two globals of one type on disk are still refused. The lock and unlocked-read comments now state the per-call guarantee. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 12:27:55 +02:00
@staticmethod
def _envelope(document, key: str, value) -> CultCacheEnvelope:
catalog_entry = document.catalog_entry()
return CultCacheEnvelope.create(
key=key,
type=document.type,
payload=document.encode_payload(value),
schema_id=catalog_entry.schema_id,
catalog_entry=catalog_entry,
)
def test_legacy_key_global_loads_without_writing_and_first_write_leaves_one_global(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
settings = define_document_type("settings", global_document=True)
path = Path(tmp) / "generic.cc"
def build() -> CultCache:
return CultCache.builder().register_document_type(settings).add_generic_store(
SingleFileMessagePackBackingStore(path)
).build()
SingleFileMessagePackBackingStore(path).push(self._envelope(settings, "legacy", {"theme": "old"}))
stored_bytes = path.read_bytes()
cache = build()
cache.pull_all_backing_stores()
self.assertEqual(cache.get_global(settings), {"theme": "old"})
self.assertEqual(cache.get(settings, CultCache.GLOBAL_KEY), {"theme": "old"})
self.assertEqual(path.read_bytes(), stored_bytes)
cache.put_global(settings, {"theme": "new"})
self.assertEqual([e.key for e in SingleFileMessagePackBackingStore(path).pull_all()], [CultCache.GLOBAL_KEY])
reloaded = build()
reloaded.pull_all_backing_stores()
self.assertEqual(reloaded.get_required_global(settings), {"theme": "new"})
def test_deleting_a_legacy_key_global_removes_the_legacy_record(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
settings = define_document_type("settings", global_document=True)
path = Path(tmp) / "generic.cc"
SingleFileMessagePackBackingStore(path).push(self._envelope(settings, "legacy", {"theme": "old"}))
cache = CultCache.builder().register_document_type(settings).add_generic_store(
SingleFileMessagePackBackingStore(path)
).build()
cache.pull_all_backing_stores()
cache.delete_global(settings)
self.assertEqual(SingleFileMessagePackBackingStore(path).pull_all(), [])
self.assertIsNone(cache.get_global(settings))
def test_two_globals_of_one_type_on_disk_are_refused_under_any_keys(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
settings = define_document_type("settings", global_document=True)
for keys in (["one", "two"], ["legacy", CultCache.GLOBAL_KEY]):
path = Path(tmp) / f"{keys[0]}.cc"
store = SingleFileMessagePackBackingStore(path)
store.push_all([self._envelope(settings, key, {"theme": key}) for key in keys])
cache = CultCache.builder().register_document_type(settings).add_generic_store(store).build()
with self.assertRaisesRegex(CultCacheError, "Duplicate global document for type: settings"):
cache.pull_all_backing_stores()
self.assertIsNone(cache.get_global(settings))
def test_decoder_writing_during_pull_raises_and_changes_neither_store_nor_cache(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "generic.cc"
holder: dict[str, CultCache] = {}
def decode(raw):
holder["cache"].put(item, "side", {"name": "side"})
return raw
item = define_document_type("item", decode=decode, encode=lambda value: value)
store = SingleFileMessagePackBackingStore(path)
store.push(self._envelope(item, "a", {"name": "a"}))
stored_bytes = path.read_bytes()
cache = holder["cache"] = CultCache.builder().register_document_type(item).add_generic_store(store).build()
with self.assertRaisesRegex(CultCacheError, "re-entrant put"):
cache.pull_all_backing_stores()
self.assertEqual(path.read_bytes(), stored_bytes)
self.assertEqual(cache.snapshot_envelopes(), [])
self.assertIsNone(cache.get(item, "side"))
# The refusal released the cache: an ordinary write still works.
cache.put(item, "b", {"name": "b"})
self.assertEqual(cache.get(item, "b"), {"name": "b"})
def test_index_extractor_writing_during_register_index_raises_and_installs_nothing(self) -> None:
thing = define_document_type("thing")
cache = CultCache.builder().register_document_type(thing).build()
cache.put(thing, "k1", {"n": "one", "cat": "x"})
def category(value: dict) -> str:
cache.put(thing, "k2", {"n": "two", "cat": "y"})
return value["cat"]
with self.assertRaisesRegex(CultCacheError, "re-entrant put"):
cache.register_index(thing, "cat", category)
self.assertIsNone(cache.get(thing, "k2"))
self.assertNotIn("thing", cache._state.index_extractors)
self.assertIsNone(cache.get_key_by_index(thing, "cat", "x"))
Validate a Python CultCache write whole before touching a store put, put_envelope and put_envelopes now compute the new records' name and index keys and resolve the home store before any push; applying to the cache afterwards runs no extractor. Each held record keeps the lookup keys it produced at admission, so overwrite and delete remove lookups without calling user extractors. put_envelope and put_envelopes refuse an empty key and a global under any key but __global__. Per-type dicts are created only after the store accepts. register_document_type, register_name_lookup and register_index derive every lookup under the candidate extractors first and install extractors and lookups together, so a raising extractor installs nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 11:48:59 +02:00
def test_put_envelope_refuses_a_global_under_another_key(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
settings = define_document_type("settings", global_document=True)
path = Path(tmp) / "generic.cc"
cache = CultCache.builder().register_document_type(settings).add_generic_store(
SingleFileMessagePackBackingStore(path)
).build()
cache.put_global(settings, {"theme": "base"})
store_before, cache_before = self._store_state(path), cache.snapshot_envelopes()
stray = replace(cache.get_required_envelope(settings, CultCache.GLOBAL_KEY), key="stray")
with self.assertRaisesRegex(CultCacheError, "must use key __global__"):
cache.put_envelope(settings, stray)
with self.assertRaisesRegex(CultCacheError, "must be non-empty"):
cache.put_envelope(settings, replace(stray, key=""))
self.assertEqual(self._store_state(path), store_before)
self.assertEqual(cache.snapshot_envelopes(), cache_before)
def test_refused_put_leaves_no_empty_type_entry(self) -> None:
def refuse(value: dict) -> dict:
raise ValueError("encode refuses")
with tempfile.TemporaryDirectory() as tmp:
other = define_document_type("other", encode=refuse)
path = Path(tmp) / "generic.cc"
cache = CultCache.builder().register_document_type(other).add_generic_store(
SingleFileMessagePackBackingStore(path)
).build()
with self.assertRaisesRegex(ValueError, "encode refuses"):
cache.put(other, "x", {"a": 1})
self.assertEqual(cache.snapshot(), {})
self.assertFalse(path.exists())
Keep cultcache-py record homes stable and refuse strays at load Conform cultcache-py to the store-composition contract, following C#: - add_backing_store and add_generic_store raise, attaching nothing, when the new store would move the home of any document type the cache already holds. - add_backing_store(store, []) attaches the generic store (same as TypeScript, Rust and C#'s untyped attach) instead of silently doing nothing. - pull_all_backing_stores stages every store's records and replaces the cache's view only when all loaded. A record delivered by a store that is not its home raises naming the key and both stores; nothing is admitted and no store is written. The zero-store in-memory test passes on 6bb41f3 as well (Python already conformed); the other new tests fail there. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 00:50:48 +02:00
def test_attach_with_empty_type_list_is_the_generic_store(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
settings = define_database_entry_type("settings", [("theme", 0)])
store_path = Path(tmp) / "generic.cc"
cache = CultCache.builder().register_document_type(settings).build()
cache.add_backing_store(SingleFileMessagePackBackingStore(store_path), [])
cache.put(settings, "y", {"theme": "t"})
self.assertEqual(
[(e.key, e.type) for e in SingleFileMessagePackBackingStore(store_path).pull_all()],
[("y", "settings")],
)
with self.assertRaisesRegex(CultCacheError, "second generic store"):
cache.add_backing_store(SingleFileMessagePackBackingStore(Path(tmp) / "b.cc"), [])
def test_zero_store_cache_writes_and_deletes_in_memory(self) -> None:
settings = define_database_entry_type("settings", [("theme", 0)])
cache = CultCache.builder().register_document_type(settings).build()
cache.put(settings, "x", {"theme": "mem"})
self.assertEqual(cache.get_required(settings, "x"), {"theme": "mem"})
cache.delete(settings, "x")
self.assertIsNone(cache.get(settings, "x"))
Add Python C# cache benchmark parity receipt
2026-06-14 01:30:50 +01:00
def test_cache_put_without_backing_store_keeps_in_memory_value(self) -> None:
document = define_database_entry_type(
"bench.memory_item",
[
("name", 0),
("value", 1, 0),
],
)
cache = CultCache()
cache.register_document_type(document)
cache.put(document, "item:one", {"name": "one", "value": 1})
self.assertEqual(cache.get(document, "item:one"), {"name": "one", "value": 1})
self.assertEqual(cache.get_required_envelope(document, "item:one").key, "item:one")
Bound the Python fragment reassembly map cultcache-py removed a fragment buffer only on successful reassembly, the same shape cultnet-ts had: a set that loses one fragment is kept for the life of the session and the map grows under any loss on a fragmenting channel. Cap at 64 to match the other runtimes, evict the oldest stranded set (dict insertion order) rather than refuse the payload, and expose fragment_sets_evicted so the drop is observable. The fragment id already wrapped correctly here; only the Rust allocator saturated. PYTHONPATH=src python -m unittest tests.test_cultcache -> 142 run, 1 pre-existing error (interop fixture path is cwd-relative, same on main) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-10 11:42:17 +02:00
Python: refuse unknown store headers and extra record slots by name Delete the raw_record[:4] slice and the legacy-envelope fallthrough for unknown cultcache.store.* headers. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 02:04:34 +02:00
# Shared refusal vectors: tests/vectors/document-variants-c0, read by every runtime's tests.
_VECTORS = Path(__file__).resolve().parents[3] / "tests" / "vectors" / "document-variants-c0"
_ITEM_SCHEMA_ID = "sha256:88d3fdf0a927acf3b163940d8f8c7fe62b3316542ce771a67ec8bc038f594788"
def _pull_vector(self, name: str) -> list[CultCacheEnvelope]:
with tempfile.TemporaryDirectory() as tmp:
store_path = Path(tmp) / "store.msgpack"
store_path.write_bytes((self._VECTORS / name).read_bytes())
return SingleFileMessagePackBackingStore(store_path).pull_all()
def test_single_file_refuses_unknown_header_by_name(self) -> None:
with self.assertRaises(ValueError) as caught:
self._pull_vector("unknown-header.msgpack")
self.assertIn("cultcache.store.v9", str(caught.exception))
def test_single_file_refuses_extra_record_slot_naming_the_record(self) -> None:
with self.assertRaises(ValueError) as caught:
self._pull_vector("extra-slot-full-payload.msgpack")
self.assertIn("item:anvil", str(caught.exception))
self.assertIn(self._ITEM_SCHEMA_ID, str(caught.exception))
def test_single_file_refuses_variant_store_by_version_or_record(self) -> None:
with self.assertRaises(ValueError) as caught:
self._pull_vector("variant-v2.msgpack")
message = str(caught.exception)
self.assertTrue("cultcache.store.v2" in message or "item:anvil-big" in message, message)
Prove TypeScript, Python and Rust refuse the C#-written variant store by name Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 04:21:59 +02:00
def test_single_file_refuses_the_csharp_written_variant_store_by_version_or_record(self) -> None:
with self.assertRaises(ValueError) as caught:
self._pull_vector("../document-variants-c1/variant-store.msgpack")
message = str(caught.exception)
self.assertTrue("cultcache.store.v2" in message or "laser-big" in message, message)
Python: refuse unknown store headers and extra record slots by name Delete the raw_record[:4] slice and the legacy-envelope fallthrough for unknown cultcache.store.* headers. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 02:04:34 +02:00
def test_v1_store_written_at_the_base_commit_still_reads_byte_for_byte(self) -> None:
envelopes = self._pull_vector("v1-base.msgpack")
self.assertEqual([(e.key, e.type) for e in envelopes], [("alpha", "vectors.item"), ("beta", "vectors.item")])
self.assertEqual(envelopes[0].payload, b"\x92\xa5alpha\x01")
self.assertEqual(envelopes[1].payload, b"\x92\xa4beta\x02")
cultcache-py: only a missing store reads as empty; an empty file is refused `Path.exists()` returns False for ELOOP and ENOTDIR, so a symlink loop or a file where the parent directory should be read as an empty store. The read now catches FileNotFoundError only. An existing zero-byte file is refused with ValueError, like any other undecodable store, matching the Rust and C# runtimes. `[0x90]` stays an empty legacy store. Refusal text no longer points at a variant-aware runtime: an unsupported header names the format found, and a v1 record with a fifth slot says the file is not a valid store. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 17:04:39 +02:00
def test_single_file_refusals_say_what_was_found_without_blaming_variants(self) -> None:
with self.assertRaises(ValueError) as unsupported:
self._pull_vector("unknown-header.msgpack")
self.assertIn("not one this runtime reads", str(unsupported.exception))
self.assertNotIn("variant", str(unsupported.exception))
with self.assertRaises(ValueError) as invalid:
self._pull_vector("extra-slot-full-payload.msgpack")
self.assertIn("not a valid store", str(invalid.exception))
self.assertNotIn("variant", str(invalid.exception))
def test_single_file_only_a_missing_store_reads_as_empty(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
self.assertEqual(SingleFileMessagePackBackingStore(Path(tmp) / "store.cc").pull_all(), [])
# An existing empty file is not a store: no CultCache writer leaves one.
empty = Path(tmp) / "empty.cc"
empty.write_bytes(b"")
with self.assertRaises(ValueError):
SingleFileMessagePackBackingStore(empty).pull_all()
self.assertEqual(empty.read_bytes(), b"")
# An empty array is the legacy envelope array with no envelopes: no header, no format claimed.
legacy_empty = Path(tmp) / "legacy-empty.cc"
legacy_empty.write_bytes(b"\x90")
self.assertEqual(SingleFileMessagePackBackingStore(legacy_empty).pull_all(), [])
cultcache-py: refusals echo a header only in its known shape; a dangling link is not absent An unsupported header is echoed only as cultcache.store.v<digits>; any other is described by its byte length, since the bytes are the store's. A read that found nothing through a dangling symbolic link returned an empty store, and the writer then replaced the link. Both stores now share one read rule: only nothing at the path is absent. JsonLinesBackingStore also drops `Path.exists()`, which reported a symlink loop or a file-as-parent as absent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 18:35:19 +02:00
def test_single_file_refusals_never_echo_a_value_from_the_store(self) -> None:
import msgpack # type: ignore
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.cc"
cases = [
(["cultcache.store.v12", [], []], True),
(["cultcache.store.SECRET-HEADER", [], []], False),
(["cultcache.store.v12SECRET", [], []], False),
(["cultcache.store.v", [], []], False),
(["cultcache.store.v1", ["SECRET-CATALOG-TEXT"], []], None),
]
for value, echoed in cases:
path.write_bytes(msgpack.packb(value, use_bin_type=True))
with self.assertRaises(ValueError) as caught:
SingleFileMessagePackBackingStore(path).pull_all()
message = str(caught.exception)
if echoed:
self.assertIn(repr(value[0]), message)
else:
self.assertNotIn("SECRET", message)
if echoed is False:
self.assertNotIn(repr(value[0]), message)
self.assertIn(f"of {len(value[0])} bytes", message)
Python: pin what the unreadable-format and extra-slot refusals say and what the runtime reads Mutation survivors on the merge's refusal text (the two readable formats, the unrecognised-header description, the record-slot refusal) had no test pinning them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 20:02:15 +02:00
def test_single_file_refusals_say_exactly_what_was_found_and_what_is_read(self) -> None:
import msgpack # type: ignore
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.cc"
reads = "it reads 'cultcache.store.v1' and 'cultcache.store.v3' only"
for header, found in (
("cultcache.store.v12", "'cultcache.store.v12'"),
("cultcache.store.SECRET", "an unrecognised cultcache.store.* header of 22 bytes"),
("cultcache.store.v", "an unrecognised cultcache.store.* header of 17 bytes"),
TS and Python: pin the refusal text (v3 named, ASCII digits only); TS comments say a zero-byte file is refused Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 21:05:38 +02:00
# Digits are ASCII only: two Arabic-Indic digits are four bytes and are not the known shape.
("cultcache.store.v١٢", "an unrecognised cultcache.store.* header of 21 bytes"),
Python: pin what the unreadable-format and extra-slot refusals say and what the runtime reads Mutation survivors on the merge's refusal text (the two readable formats, the unrecognised-header description, the record-slot refusal) had no test pinning them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 20:02:15 +02:00
):
path.write_bytes(msgpack.packb([header, [], []], use_bin_type=True))
with self.assertRaises(ValueError) as caught:
SingleFileMessagePackBackingStore(path).pull_all()
self.assertIn(f"CultCache store format {found} is not one this runtime reads; {reads}", str(caught.exception))
Python: fix the payload byte literal in the refusal-text test (the previous commit did not parse) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 20:04:16 +02:00
path.write_bytes(msgpack.packb(["cultcache.store.v1", [], [["k", "s", "t", b"\x90", "extra"]]], use_bin_type=True))
Python: pin what the unreadable-format and extra-slot refusals say and what the runtime reads Mutation survivors on the merge's refusal text (the two readable formats, the unrecognised-header description, the record-slot refusal) had no test pinning them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 20:02:15 +02:00
with self.assertRaises(ValueError) as caught:
SingleFileMessagePackBackingStore(path).pull_all()
self.assertIn(
"CultCache record 'k' (schema 's') has 5 slots, more than the 4 of a cultcache.store.v1 record, so this is not a valid store",
str(caught.exception),
)
cultcache-py: a record refusal names only a string key; pin link writes for R3 The five-slot refusal echoed a non-string key or schema id with repr, quoting stored bytes. A string is named; anything else is described by its type. A push through a live link replaces the link with a regular file. A test named for R3 pins that until R3 decides whether writes resolve or refuse links. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 19:54:51 +02:00
def test_single_file_record_refusal_names_only_a_string_key(self) -> None:
import msgpack # type: ignore
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.cc"
for key, schema in ((["SECRET-IN-LIST"], "s"), (b"SECRET-BYTES", "s"), ("k", {"SECRET": 1})):
path.write_bytes(msgpack.packb(
["cultcache.store.v1", [], [[key, schema, "t", b"\x90", "extra"]]], use_bin_type=True))
with self.assertRaises(ValueError) as caught:
SingleFileMessagePackBackingStore(path).pull_all()
self.assertNotIn("SECRET", str(caught.exception))
self.assertIn("not a valid store", str(caught.exception))
# What a write does through a symbolic link today. R3 decides whether a write resolves
# the link or refuses it; this pins the current behaviour so that change is deliberate.
@unittest.skipIf(os.name != "posix", "symbolic links need POSIX here")
def test_r3_decides_resolve_or_refuse_a_push_through_a_live_link_replaces_the_link(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
volume = Path(tmp) / "volume"
volume.mkdir()
target = volume / "store.cc"
envelope = CultCacheEnvelope(key="a", type="t", payload=b"\x90", stored_at="2026-09-30T00:00:00Z")
SingleFileMessagePackBackingStore(target).push(envelope)
before = target.read_bytes()
link = Path(tmp) / "store.cc"
link.symlink_to(target)
SingleFileMessagePackBackingStore(link).push(replace(envelope, key="b"))
self.assertFalse(link.is_symlink())
self.assertTrue(link.is_file())
self.assertEqual(target.read_bytes(), before)
cultcache-py: refusals echo a header only in its known shape; a dangling link is not absent An unsupported header is echoed only as cultcache.store.v<digits>; any other is described by its byte length, since the bytes are the store's. A read that found nothing through a dangling symbolic link returned an empty store, and the writer then replaced the link. Both stores now share one read rule: only nothing at the path is absent. JsonLinesBackingStore also drops `Path.exists()`, which reported a symlink loop or a file-as-parent as absent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 18:35:19 +02:00
@unittest.skipIf(os.name != "posix", "symbolic links need POSIX here")
def test_a_dangling_symlink_at_a_store_path_is_an_os_error_and_nothing_is_written(self) -> None:
for store_type in (SingleFileMessagePackBackingStore, JsonLinesBackingStore):
with tempfile.TemporaryDirectory() as tmp:
volume = Path(tmp) / "volume"
volume.mkdir()
path = Path(tmp) / "store.cc"
path.symlink_to(volume / "store.cc")
store = store_type(path)
with self.assertRaises(FileNotFoundError):
store.pull_all()
with self.assertRaises(FileNotFoundError):
store.push(CultCacheEnvelope(key="k", type="t", payload=b"\x90", stored_at="2026-09-30T00:00:00Z"))
self.assertTrue(path.is_symlink())
self.assertEqual(list(volume.iterdir()), [])
@unittest.skipIf(os.name != "posix", "symlink loops and file-as-parent errors are POSIX errno cases")
def test_json_lines_store_that_cannot_be_reached_is_an_os_error(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
self.assertEqual(JsonLinesBackingStore(Path(tmp) / "missing.jsonl").pull_all(), [])
loop = Path(tmp) / "loop.jsonl"
loop.symlink_to(loop)
parent = Path(tmp) / "body"
parent.write_bytes(b"not a directory")
for path in (loop, parent / "store.jsonl"):
with self.assertRaises(OSError) as caught:
JsonLinesBackingStore(path).pull_all()
self.assertNotIsInstance(caught.exception, FileNotFoundError)
cultcache-py: only a missing store reads as empty; an empty file is refused `Path.exists()` returns False for ELOOP and ENOTDIR, so a symlink loop or a file where the parent directory should be read as an empty store. The read now catches FileNotFoundError only. An existing zero-byte file is refused with ValueError, like any other undecodable store, matching the Rust and C# runtimes. `[0x90]` stays an empty legacy store. Refusal text no longer points at a variant-aware runtime: an unsupported header names the format found, and a v1 record with a fifth slot says the file is not a valid store. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 17:04:39 +02:00
@unittest.skipIf(os.name != "posix", "symlink loops and file-as-parent errors are POSIX errno cases")
def test_single_file_store_that_cannot_be_reached_is_an_os_error(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
loop = Path(tmp) / "loop.cc"
loop.symlink_to(loop)
parent = Path(tmp) / "body"
parent.write_bytes(b"not a directory")
for path in (loop, parent / "store.cc"):
with self.assertRaises(OSError) as caught:
SingleFileMessagePackBackingStore(path).pull_all()
self.assertNotIsInstance(caught.exception, FileNotFoundError)
Python: refuse unknown store headers and extra record slots by name Delete the raw_record[:4] slice and the legacy-envelope fallthrough for unknown cultcache.store.* headers. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 02:04:34 +02:00
TypeScript, Python and Rust read the v3 header and keep it on rewrite A store that can hold element ids declares cultcache.store.v3. These runtimes pass payload bytes through, so they read it and write back the header they found; they do not decide when a document carries ids. Vector: tests/vectors/document-variants-c2a/v3-base.msgpack. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 18:27:17 +02:00
# The element-id marker: tests/vectors/document-variants-c2a/v3-base.msgpack is v1-base with its header replaced.
_C2A_VECTORS = Path(__file__).resolve().parents[3] / "tests" / "vectors" / "document-variants-c2a"
def _header_after_rewrite(self, vector: Path) -> str:
import msgpack
with tempfile.TemporaryDirectory() as tmp:
store_path = Path(tmp) / "store.msgpack"
store_path.write_bytes(vector.read_bytes())
store = SingleFileMessagePackBackingStore(store_path)
envelopes = store.pull_all()
self.assertEqual([e.key for e in envelopes], ["alpha", "beta"])
store.push(envelopes[0])
return msgpack.unpackb(store_path.read_bytes(), raw=False)[0]
def test_single_file_reads_a_v3_element_id_store_and_a_rewrite_keeps_the_marker(self) -> None:
self.assertEqual(self._header_after_rewrite(self._C2A_VECTORS / "v3-base.msgpack"), "cultcache.store.v3")
def test_single_file_rewrite_of_a_v1_store_stays_v1(self) -> None:
self.assertEqual(self._header_after_rewrite(self._VECTORS / "v1-base.msgpack"), "cultcache.store.v1")
TypeScript and Python: a whole-store flush applies the header the file on disk carries TS pushAll wrote the header its instance last read (v1 when fresh) without reading the file. It now reads the on-disk header first, as push and delete do through pullAll. Python resets its remembered header when the file is gone or empty, so the disk decides. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:44:16 +02:00
def test_single_file_flush_after_the_marked_file_is_gone_writes_unmarked(self) -> None:
import msgpack
with tempfile.TemporaryDirectory() as tmp:
store_path = Path(tmp) / "store.msgpack"
store_path.write_bytes((self._C2A_VECTORS / "v3-base.msgpack").read_bytes())
store = SingleFileMessagePackBackingStore(store_path)
envelopes = store.pull_all()
store_path.unlink()
store.push_all(envelopes)
self.assertEqual(msgpack.unpackb(store_path.read_bytes(), raw=False)[0], "cultcache.store.v1")
Rust: a whole-store flush reads the on-disk header first and refuses one it cannot read push_all overwrote a v9, variant-v2 or variant store with a v1 file, dropping its variants; TypeScript and Python refuse. The rewrite header now comes from the file's own prefix and shares one readable-format rule with the decoder. TypeScript pullAll resets its header when the file is gone, empty or legacy, as Python does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 21:20:11 +02:00
def test_single_file_push_after_the_marked_file_is_gone_writes_unmarked(self) -> None:
import msgpack
with tempfile.TemporaryDirectory() as tmp:
store_path = Path(tmp) / "store.msgpack"
store_path.write_bytes((self._C2A_VECTORS / "v3-base.msgpack").read_bytes())
store = SingleFileMessagePackBackingStore(store_path)
envelopes = store.pull_all()
store_path.unlink()
store.push(envelopes[0])
self.assertEqual(msgpack.unpackb(store_path.read_bytes(), raw=False)[0], "cultcache.store.v1")
TypeScript and Python: pin pushAll/push_all onto a store whose header the runtime cannot read The behaviour was already a refusal that leaves the file untouched; now a test fails if a rewrite ever reads the disk header less strictly than a load does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 21:56:08 +02:00
def test_single_file_push_all_refuses_a_store_whose_header_it_cannot_read_and_leaves_it_untouched(self) -> None:
envelopes = SingleFileMessagePackBackingStore(self._C2A_VECTORS / "v3-base.msgpack").pull_all()
vectors = self._VECTORS.parent
for vector in (
self._VECTORS / "unknown-header.msgpack",
self._VECTORS / "variant-v2.msgpack",
vectors / "document-variants-c1" / "variant-store.msgpack",
):
with tempfile.TemporaryDirectory() as tmp:
store_path = Path(tmp) / "store.msgpack"
store_path.write_bytes(vector.read_bytes())
before = store_path.read_bytes()
Python: StoreUnreadableError for a store file it cannot read; a store is exactly three slots The refusal is a ValueError subclass with the cause chained. A store with more than three slots is refused instead of read. The readability test walks every vector in the folder, asserts the error type and cause, the header a replaced file carries and what it holds. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 04:15:04 +02:00
with self.assertRaisesRegex(StoreUnreadableError, "is not readable"):
TypeScript and Python: pin pushAll/push_all onto a store whose header the runtime cannot read The behaviour was already a refusal that leaves the file untouched; now a test fails if a rewrite ever reads the disk header less strictly than a load does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 21:56:08 +02:00
SingleFileMessagePackBackingStore(store_path).push_all(envelopes)
self.assertEqual(store_path.read_bytes(), before, f"{vector.name} was rewritten")
Store readability: one reader decides open, flush, commit and CultMesh writes A rewrite replaces a file only when the runtime's own reader opens it. C# drops ReadStoreHeader (a second, header-only verdict that let a flush replace files the reader refuses) and asks DeserializeSnapshot from open, flush and commit; the reader now requires exactly one array, so C# refuses bytes after the store on read as TS and Python do. A zero-byte file is an empty store for all three paths. CultMesh.WriteSingleFileDocument reads the file it replaces through the Mesh reader before writing. Rust reads and rewrites through one classification (trailing bytes refused on read; a non-store first slot must be a legacy envelope array the reader accepts). TypeScript checks a legacy array with pullAll's own decoder before replacing it. Python already conformed. Shared byte vectors and per-runtime verdicts live in tests/vectors/document-variants-c2a/readability; every runtime's test reads them. Holds gains second-dictionary-value and second-list-item cases. The persistence format contract now marks v3 by content and states the readability rule. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 01:59:28 +02:00
# A file is replaced by a rewrite exactly when this runtime's own reader opens it: one verdict per file, asked by
# pull_all, push_all, push and delete alike. The bytes and every runtime's verdict:
# tests/vectors/document-variants-c2a/readability.
def test_single_file_replaces_a_file_exactly_when_it_reads(self) -> None:
Python: StoreUnreadableError for a store file it cannot read; a store is exactly three slots The refusal is a ValueError subclass with the cause chained. A store with more than three slots is refused instead of read. The readability test walks every vector in the folder, asserts the error type and cause, the header a replaced file carries and what it holds. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 04:15:04 +02:00
import msgpack # type: ignore
Store readability: one reader decides open, flush, commit and CultMesh writes A rewrite replaces a file only when the runtime's own reader opens it. C# drops ReadStoreHeader (a second, header-only verdict that let a flush replace files the reader refuses) and asks DeserializeSnapshot from open, flush and commit; the reader now requires exactly one array, so C# refuses bytes after the store on read as TS and Python do. A zero-byte file is an empty store for all three paths. CultMesh.WriteSingleFileDocument reads the file it replaces through the Mesh reader before writing. Rust reads and rewrites through one classification (trailing bytes refused on read; a non-store first slot must be a legacy envelope array the reader accepts). TypeScript checks a legacy array with pullAll's own decoder before replacing it. Python already conformed. Shared byte vectors and per-runtime verdicts live in tests/vectors/document-variants-c2a/readability; every runtime's test reads them. Holds gains second-dictionary-value and second-list-item cases. The persistence format contract now marks v3 by content and states the readability rule. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 01:59:28 +02:00
root = self._C2A_VECTORS / "readability"
rows = [
line.split()
for line in (root / "manifest.txt").read_text(encoding="utf-8").splitlines()
if line and not line.startswith("#")
]
Python: StoreUnreadableError for a store file it cannot read; a store is exactly three slots The refusal is a ValueError subclass with the cause chained. A store with more than three slots is refused instead of read. The readability test walks every vector in the folder, asserts the error type and cause, the header a replaced file carries and what it holds. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 04:15:04 +02:00
# Every vector in the folder has a manifest row.
self.assertEqual(
sorted(path.name for path in root.glob("*.bin")),
sorted(vector for vector, *_ in rows if not vector.startswith("..")),
)
Every runtime rewrites only a file that is exactly one store A rewrite read the header of the file it replaces but not whether the file was a store: C# ReadStoreHeader skipped one value and ignored bytes after it, and took a non-array value or an array whose first slot is a string that is not a store header for a headerless legacy file (so did the C# Mesh raw write, which uses it); TypeScript's pushAll took any non-array value or non-header first slot for v1; Rust ignored bytes after the array. The rule, one per runtime: a file is a store on rewrite only when it is exactly one complete MessagePack array. Bytes after it, a value that is not an array, and a first slot that is a string but not a header this runtime reads are refused, and the file is left untouched. An empty file and a legacy envelope array (first slot not a string, empty included) carry no header and are still rewritten. - C#: ReadStoreHeader refuses non-array values, trailing bytes and a non-header first string, with a NotSupportedException naming which. - TypeScript: storeHeader (store-format.ts) decides the rewrite header; @msgpack/msgpack's decode already refuses trailing bytes. - Rust: store_header(bytes, whole) walks with a cursor; the rewrite path asks for the whole file, the read paths keep reading as before. - Python: push_all already reads the whole file (unpackb refuses trailing bytes, the legacy decoder refuses anything but a list of maps); pinned by a test. Tests in each runtime run the same file matrix (truncated, trailing, scalar, map, string first; empty, empty array, legacy), plus a C# unconditional commit over a store with trailing bytes and the C# Mesh raw write. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 00:01:58 +02:00
envelopes = SingleFileMessagePackBackingStore(self._C2A_VECTORS / "v3-base.msgpack").pull_all()
Store readability: one reader decides open, flush, commit and CultMesh writes A rewrite replaces a file only when the runtime's own reader opens it. C# drops ReadStoreHeader (a second, header-only verdict that let a flush replace files the reader refuses) and asks DeserializeSnapshot from open, flush and commit; the reader now requires exactly one array, so C# refuses bytes after the store on read as TS and Python do. A zero-byte file is an empty store for all three paths. CultMesh.WriteSingleFileDocument reads the file it replaces through the Mesh reader before writing. Rust reads and rewrites through one classification (trailing bytes refused on read; a non-store first slot must be a legacy envelope array the reader accepts). TypeScript checks a legacy array with pullAll's own decoder before replacing it. Python already conformed. Shared byte vectors and per-runtime verdicts live in tests/vectors/document-variants-c2a/readability; every runtime's test reads them. Holds gains second-dictionary-value and second-list-item cases. The persistence format contract now marks v3 by content and states the readability rule. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 01:59:28 +02:00
for vector, _, _, _, python in rows:
content = (root / vector).read_bytes()
for operation in ("pull_all", "push_all", "push", "delete"):
with tempfile.TemporaryDirectory() as tmp:
store_path = Path(tmp) / "store.msgpack"
store_path.write_bytes(content)
store = SingleFileMessagePackBackingStore(store_path)
run = {
"pull_all": store.pull_all,
"push_all": lambda: store.push_all(envelopes),
"push": lambda: store.push(envelopes[0]),
"delete": lambda: store.delete(envelopes[0].type, envelopes[0].key),
}[operation]
if python == "reads":
run()
Python: StoreUnreadableError for a store file it cannot read; a store is exactly three slots The refusal is a ValueError subclass with the cause chained. A store with more than three slots is refused instead of read. The readability test walks every vector in the folder, asserts the error type and cause, the header a replaced file carries and what it holds. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 04:15:04 +02:00
if operation == "pull_all":
continue
# The replaced file is a store: it carries the header its old content decides (a v3 store keeps its
# marker, all else is v1), and it holds what the operation wrote.
Python: a record is stamped with its registered schema id and the catalog publishes it; cultnet no longer infers a schema from a payload put_envelope, put_envelopes and load stamp the registered schema's id, and the store writer publishes every record's id, so a replicated record under a compatible id no longer leaves a store nothing can open. cultnet-py resolves a raw record by the schema ids its documents list and nothing else. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 06:54:18 +02:00
expected = "cultcache.store.v3" if "v3" in vector else "cultcache.store.v1"
Python: StoreUnreadableError for a store file it cannot read; a store is exactly three slots The refusal is a ValueError subclass with the cause chained. A store with more than three slots is refused instead of read. The readability test walks every vector in the folder, asserts the error type and cause, the header a replaced file carries and what it holds. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 04:15:04 +02:00
self.assertEqual(msgpack.unpackb(store_path.read_bytes(), raw=False)[0], expected, f"{vector} {operation}")
keys = [envelope.key for envelope in SingleFileMessagePackBackingStore(store_path).pull_all()]
if operation == "push_all":
self.assertEqual(sorted(set(keys) & {e.key for e in envelopes}), sorted(e.key for e in envelopes), f"{vector} {operation}")
elif operation == "push":
self.assertIn(envelopes[0].key, keys, f"{vector} {operation}")
else:
self.assertNotIn(envelopes[0].key, keys, f"{vector} {operation}")
Store readability: one reader decides open, flush, commit and CultMesh writes A rewrite replaces a file only when the runtime's own reader opens it. C# drops ReadStoreHeader (a second, header-only verdict that let a flush replace files the reader refuses) and asks DeserializeSnapshot from open, flush and commit; the reader now requires exactly one array, so C# refuses bytes after the store on read as TS and Python do. A zero-byte file is an empty store for all three paths. CultMesh.WriteSingleFileDocument reads the file it replaces through the Mesh reader before writing. Rust reads and rewrites through one classification (trailing bytes refused on read; a non-store first slot must be a legacy envelope array the reader accepts). TypeScript checks a legacy array with pullAll's own decoder before replacing it. Python already conformed. Shared byte vectors and per-runtime verdicts live in tests/vectors/document-variants-c2a/readability; every runtime's test reads them. Holds gains second-dictionary-value and second-list-item cases. The persistence format contract now marks v3 by content and states the readability rule. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 01:59:28 +02:00
else:
Python: StoreUnreadableError for a store file it cannot read; a store is exactly three slots The refusal is a ValueError subclass with the cause chained. A store with more than three slots is refused instead of read. The readability test walks every vector in the folder, asserts the error type and cause, the header a replaced file carries and what it holds. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 04:15:04 +02:00
with self.assertRaises(StoreUnreadableError, msg=f"{vector} {operation}") as refused:
Store readability: one reader decides open, flush, commit and CultMesh writes A rewrite replaces a file only when the runtime's own reader opens it. C# drops ReadStoreHeader (a second, header-only verdict that let a flush replace files the reader refuses) and asks DeserializeSnapshot from open, flush and commit; the reader now requires exactly one array, so C# refuses bytes after the store on read as TS and Python do. A zero-byte file is an empty store for all three paths. CultMesh.WriteSingleFileDocument reads the file it replaces through the Mesh reader before writing. Rust reads and rewrites through one classification (trailing bytes refused on read; a non-store first slot must be a legacy envelope array the reader accepts). TypeScript checks a legacy array with pullAll's own decoder before replacing it. Python already conformed. Shared byte vectors and per-runtime verdicts live in tests/vectors/document-variants-c2a/readability; every runtime's test reads them. Holds gains second-dictionary-value and second-list-item cases. The persistence format contract now marks v3 by content and states the readability rule. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 01:59:28 +02:00
run()
Python: StoreUnreadableError for a store file it cannot read; a store is exactly three slots The refusal is a ValueError subclass with the cause chained. A store with more than three slots is refused instead of read. The readability test walks every vector in the folder, asserts the error type and cause, the header a replaced file carries and what it holds. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 04:15:04 +02:00
self.assertIsNotNone(refused.exception.__cause__, f"{vector} {operation}")
Store readability: one reader decides open, flush, commit and CultMesh writes A rewrite replaces a file only when the runtime's own reader opens it. C# drops ReadStoreHeader (a second, header-only verdict that let a flush replace files the reader refuses) and asks DeserializeSnapshot from open, flush and commit; the reader now requires exactly one array, so C# refuses bytes after the store on read as TS and Python do. A zero-byte file is an empty store for all three paths. CultMesh.WriteSingleFileDocument reads the file it replaces through the Mesh reader before writing. Rust reads and rewrites through one classification (trailing bytes refused on read; a non-store first slot must be a legacy envelope array the reader accepts). TypeScript checks a legacy array with pullAll's own decoder before replacing it. Python already conformed. Shared byte vectors and per-runtime verdicts live in tests/vectors/document-variants-c2a/readability; every runtime's test reads them. Holds gains second-dictionary-value and second-list-item cases. The persistence format contract now marks v3 by content and states the readability rule. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 01:59:28 +02:00
self.assertEqual(store_path.read_bytes(), content, f"{vector} {operation} rewrote a file it cannot read")
Every runtime rewrites only a file that is exactly one store A rewrite read the header of the file it replaces but not whether the file was a store: C# ReadStoreHeader skipped one value and ignored bytes after it, and took a non-array value or an array whose first slot is a string that is not a store header for a headerless legacy file (so did the C# Mesh raw write, which uses it); TypeScript's pushAll took any non-array value or non-header first slot for v1; Rust ignored bytes after the array. The rule, one per runtime: a file is a store on rewrite only when it is exactly one complete MessagePack array. Bytes after it, a value that is not an array, and a first slot that is a string but not a header this runtime reads are refused, and the file is left untouched. An empty file and a legacy envelope array (first slot not a string, empty included) carry no header and are still rewritten. - C#: ReadStoreHeader refuses non-array values, trailing bytes and a non-header first string, with a NotSupportedException naming which. - TypeScript: storeHeader (store-format.ts) decides the rewrite header; @msgpack/msgpack's decode already refuses trailing bytes. - Rust: store_header(bytes, whole) walks with a cursor; the rewrite path asks for the whole file, the read paths keep reading as before. - Python: push_all already reads the whole file (unpackb refuses trailing bytes, the legacy decoder refuses anything but a list of maps); pinned by a test. Tests in each runtime run the same file matrix (truncated, trailing, scalar, map, string first; empty, empty array, legacy), plus a C# unconditional commit over a store with trailing bytes and the C# Mesh raw write. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 00:01:58 +02:00
TypeScript, Python and Rust read the v3 header and keep it on rewrite A store that can hold element ids declares cultcache.store.v3. These runtimes pass payload bytes through, so they read it and write back the header they found; they do not decide when a document carries ids. Vector: tests/vectors/document-variants-c2a/v3-base.msgpack. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 18:27:17 +02:00
Add Python CultCache wire parity package
2026-06-13 20:35:58 +01:00
if __name__ == "__main__":
unittest.main()
Reference in a new issue Copy permalink