GameCult/CultLib
Watch
9
0
Fork
You've already forked CultLib
 0
Code Issues Pull requests 2 Projects Releases Packages Wiki Activity Actions 1

cultcache-py, cultnet-py: walk 12 vector rows with the key read from the refusal, pin two-globals load refusal and replication retype refusal, document both

Browse source 
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Libby 2026-11-11 23:20:58 +00:00
commit e6f047fd92
4 changed files with 73 additions and 13 deletions
Download patch file Download diff file Expand all files Collapse all files

2
packages/cultcache-py/CHANGELOG.md
View file

@ -9,7 +9,7 @@
### Breaking
- A cache holds one record per key whatever its type (`variants:ruling:cache-refuses-retype`). A `put`, `put_envelope`, `put_envelopes` or `put_global` of a second type under a key the cache holds, or under a key a store carries foreign, is refused with `SchemaConflictError` naming the key, even with no store, and changes nothing; retype a key by deleting it, then putting the new type. A write may replace another registered type's record only when the written type owns its schema id (declares it or lists it compatible) and both types share one home store. A load that reads one key from two stores, whatever the types, is refused with `CultCacheError` naming the key and both types, and admits nothing. A store that holds a record no registered type claims now loads and is carried: the cache does not serve it, the store keeps its bytes through every write, `delete` or `put` at its key is refused with `SchemaConflictError`, and registering a type that declares its schema id claims it at the next load. It raised `CultCacheError` "Unknown persisted document type". Two global types share the key `__global__`, so a second global type in one cache is now refused.
- A cache holds one record per key whatever its type (`variants:ruling:cache-refuses-retype`). A `put`, `put_envelope`, `put_envelopes` or `put_global` of a second type under a key the cache holds, or under a key a store carries foreign, is refused with `SchemaConflictError` naming the key, even with no store, and changes nothing; retype a key by deleting it, then putting the new type. A write may replace another registered type's record only when the written type owns its schema id (declares it or lists it compatible) and both types share one home store. A load that reads one key from two stores, whatever the types, is refused with `CultCacheError` naming the key and both types, and admits nothing. A store that holds a record no registered type claims now loads and is carried: the cache does not serve it, the store keeps its bytes through every write, `delete` or `put` at its key is refused with `SchemaConflictError`, and registering a type that declares its schema id claims it at the next load. It raised `CultCacheError` "Unknown persisted document type". Two global types share the key `__global__`, so a second global type in one cache is now refused, and a store that already holds two global types at `__global__` (a JSONL store or a cache with no store could produce one) no longer loads: the pull raises `CultCacheError` and admits nothing. C# refuses a second persisted global on pull the same way (`cultcache-store-composition.md`, "Globals never invented"); repair such a file by deleting one of the two records with the runtime that wrote it.
- Added `foreign_records()` and the frozen `ForeignRecord(key, schema_id, schema_name, stored_at)`, sorted by key in UTF-16 code-unit order. `stores.retyped` and `stores.written_owns` lose their underscore: the cache judge and the single-file store's commit guard ask one predicate.
- A single-file store holding two records under one key is unreadable: the v1 reader and the legacy envelope-array reader refuse it with `StoreUnreadableError` (a key is compared by its exact code points, no case folding or normalisation). A catalog slot whose canonical schema is not a string is refused the same way. A store that loaded before can now be refused: repair it with the runtime that wrote the duplicate.

52
packages/cultcache-py/tests/test_key_identity_vector.py
View file

@ -1,5 +1,6 @@
from __future__ import annotations
import re
import shutil
import tempfile
import unittest
@ -7,8 +8,11 @@ from pathlib import Path
import msgpack
from cultcache_py.cache import CultCacheError
from cultcache_py import (
CultCache,
JsonLinesBackingStore,
SchemaConflictError,
SingleFileMessagePackBackingStore,
database_entry_field,
@ -36,18 +40,15 @@ DOCUMENTS = {
}
LABEL_OF = {document.type: label for label, document in DOCUMENTS.items()}
def file_records(path: Path) -> list[list[str]]:
return sorted([envelope.key, envelope.type] for envelope in SingleFileMessagePackBackingStore(path).pull_all())
def stored(path: Path, key: str):
for envelope in SingleFileMessagePackBackingStore(path).pull_all():
if envelope.key == key:
return envelope.schema_id, envelope.stored_at, envelope.payload
return None
class KeyIdentityVectorTests(unittest.TestCase):
def test_rows(self) -> None:
rows = msgpack.unpackb((VECTORS / "rows.msgpack").read_bytes(), raw=False)
@ -56,7 +57,7 @@ class KeyIdentityVectorTests(unittest.TestCase):
with self.subTest(row=label), tempfile.TemporaryDirectory() as tmp:
self.walk(Path(tmp), registered, ops, expect)
walked += 1
self.assertEqual(walked, 11)
self.assertEqual(walked, 12)
def walk(self, tmp: Path, registered: list[str], ops: list, expect: list) -> None:
refusal, held, file, foreign, kept = expect
@ -111,7 +112,14 @@ class KeyIdentityVectorTests(unittest.TestCase):
with self.assertRaises((SchemaConflictError, ValueError, RuntimeError)) as caught:
perform(ops[-1])
if ops[-1][0] == "attach":
self.assertIn(key, str(caught.exception))
# The key is read out of the refusal's own text (it opens "Record 'key' is held by") and occurs only as a
# delimited token, never inside a schema name or other word.
message = str(caught.exception)
named = re.match(r"Record '([^']*)' is held by", message)
self.assertIsNotNone(named, message)
self.assertEqual(named.group(1), key)
self.assertEqual(len(re.findall(rf"(?<![A-Za-z0-9_]){re.escape(key)}(?![A-Za-z0-9_])", message)), 1)
self.assertEqual(message.count(key), 1)
for path, before in state["attached"]:
self.assertEqual(path.read_bytes(), before)
else:
@ -138,7 +146,6 @@ class KeyIdentityVectorTests(unittest.TestCase):
self.assertIsNotNone(stored(vector, key))
self.assertEqual(stored(path, key), stored(vector, key))
def alpha_beta(*names: str, store=None) -> CultCache:
made = CultCache.builder()
for name in names:
@ -147,8 +154,32 @@ def alpha_beta(*names: str, store=None) -> CultCache:
made.add_generic_store(store)
return made.build()
class KeyIdentityTests(unittest.TestCase):
def test_two_global_types_are_refused_in_one_cache_and_a_store_holding_both_does_not_load(self) -> None:
# C# refuses a second persisted global on pull (cultcache-store-composition.md, "Globals never invented"); the Python
# JSONL store and a cache with no store used to let two global types share __global__, so such a file can exist.
ga = define_database_entry_type(
"tests.ki_ga", [database_entry_field("text", 0, codec=t.string)], schema_id="sha256:" + "1" * 64, global_document=True
)
gb = define_database_entry_type(
"tests.ki_gb", [database_entry_field("text", 0, codec=t.string)], schema_id="sha256:" + "2" * 64, global_document=True
)
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.jsonl"
for document in (ga, gb): # one cache per type that never pulled: how a file got two globals
writer = CultCache.builder().register_document_type(document).add_generic_store(JsonLinesBackingStore(path)).build()
writer.put_global(document, {"text": document.type})
both = CultCache.builder().register_document_type(ga).register_document_type(gb).add_generic_store(JsonLinesBackingStore(path)).build()
with self.assertRaises(CultCacheError) as caught:
both.pull_all_backing_stores()
self.assertIn("'__global__' is held by both", str(caught.exception))
self.assertEqual(both.snapshot(), {})
memory = CultCache.builder().register_document_type(ga).register_document_type(gb).build()
memory.put_global(ga, {"text": "a"})
with self.assertRaises(SchemaConflictError):
memory.put_global(gb, {"text": "b"})
self.assertEqual(memory.get_required_global(ga), {"text": "a"})
def test_every_write_path_refuses_a_second_type_and_never_echoes_the_payload(self) -> None:
alpha, beta = DOCUMENTS["alpha"], DOCUMENTS["beta"]
for name in ("put", "put_envelope", "put_envelopes"):
@ -182,9 +213,9 @@ class KeyIdentityTests(unittest.TestCase):
cache.add_backing_store(SingleFileMessagePackBackingStore(two), [DOCUMENTS["beta"].type])
with self.assertRaises(Exception) as caught:
cache.pull_all_backing_stores()
self.assertIn("'k'", str(caught.exception))
self.assertEqual(cache.get(DOCUMENTS["alpha"], "k"), {"text": "a"})
self.assertIsNone(cache.get(DOCUMENTS["beta"], "k"))
self.assertIn("'qx'", str(caught.exception))
self.assertEqual(cache.get(DOCUMENTS["alpha"], "qx"), {"text": "a"})
self.assertIsNone(cache.get(DOCUMENTS["beta"], "qx"))
def test_a_foreign_record_is_listed_and_its_bytes_survive_writes_and_removals_elsewhere(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
@ -221,7 +252,6 @@ class KeyIdentityTests(unittest.TestCase):
cache.pull_all_backing_stores()
self.assertEqual(cache.foreign_records(), [])
class OwnershipAndStoreKindTests(unittest.TestCase):
"""The cache judge answers on its own where the store has no commit guard (JSONL) or the records live in two stores."""

1
packages/cultnet-py/CHANGELOG.md
View file

@ -2,6 +2,7 @@
## Unreleased (breaking; version 0.x, so the next minor)
- `apply_raw_document_record` and `apply_raw_snapshot` raise `SchemaConflictError` when a remote record is of another type than the one the cache holds at its key (`cultcache-py` holds one record per key whatever its type); the cache is unchanged, and the caller decides whether to drop the record or the peer. `cultmesh-py`'s server answers such a put with `message_invalid`; its client's `read_next_change` lets the error out.
- `CultNetDatabaseChange` carries `commit_id` and `commit_count` (`commitId`/`commitCount`; None outside a multi-record commit, as C# writes nil): `from_wire` refuses an empty or non-string id and a count that is not an integer of at least 2, `to_wire` writes them when set, and the published `database_change_raw` schema lists both. Python applies no batch. The `simulation_fact` document's `shard`, `frame`, `subject` and `claim_kind` aliases are `roles` (many facts share each); the catalog bytes do not move.
- Decodes the write receipt frames of `contracts/cultnet/interop/write-receipt.md`: `parse_message` validates `cultnet.write_admitted.v0` and `cultnet.error.v1` against their closed schemas (unknown keys, empty or non-string ids, an unknown operation or code are refused; `error.v1` requires `messageId`, string or null; `limit_reached` is `error.v1` only). Added `CULTNET_ERROR_V1_CODES`, `CULTNET_WRITE_RECEIPT_MESSAGE_VERSIONS`, `write_content_hash` and `hello_advertises_write_receipts`; `hello` takes `supported_message_versions`. `tests/test_write_receipt.py` replays the vectors' `contentHash`, `wire` and `schemaInvalid` sections. Not implemented here: the client write tracker (`scenarios`) and the server rules (`dedupe`, `blankFields`); this package has no receipt server and no write client.
- Added `message_invalid` to `CULTNET_ERROR_CODES` and `message_invalid_error(field, reason)`, byte-identical to the C# `ForMessageInvalid`. `cultmesh-py` refuses a malformed put or delete with it instead of the out-of-schema `malformed_document_put` and `malformed_document_delete`, which are removed.

31
packages/cultnet-py/tests/test_cultnet.py
View file

@ -14,7 +14,7 @@ from pathlib import Path
from typing import Callable
from uuid import uuid4
import msgpack
from cultcache_py import CultCache, define_database_entry_type
from cultcache_py import CultCache, SchemaConflictError, define_database_entry_type
from cultnet_py.benchmark import run_benchmark
from cultnet_py.compare_csharp import (
DEFAULT_PARITY_THRESHOLD,
@ -1942,6 +1942,35 @@ class CultNetTests(unittest.TestCase):
self.assertEqual(raised.exception.response["messageId"], "subscription-error")
thread.join(2.0)
def test_replication_refuses_a_remote_record_of_another_type_at_a_held_key_and_changes_nothing(self) -> None:
# variants:ruling:cache-refuses-retype: a cache holds one record per key whatever its type, so a peer that reuses a
# key across types is refused where the record is applied; replication does not catch it.
first = define_database_entry_type("replica.first", [("name", 0), ("category", 1), ("value", 2, 0)], cls=Item)
second = define_database_entry_type("replica.second", [("name", 0), ("category", 1), ("value", 2, 0)], cls=Item)
cache = CultCache()
cache.register_document_type(first)
cache.register_document_type(second)
cache.put(first, "shared", Item("held", "", 0))
def raw(document):
return {
"schemaId": document.catalog_entry().schema_id,
"recordKey": "shared",
"storedAt": "2026-06-13T00:00:00Z",
"payloadEncoding": "messagepack",
"payload": document.encode_payload(Item("remote", "", 0)),
}
with self.assertRaises(SchemaConflictError):
apply_raw_document_record(cache, schema_document_map([first, second]), raw(second))
snapshot = {"schemaVersion": "cultnet.snapshot_response_raw.v0", "messageId": "s", "documents": [raw(second)]}
with self.assertRaises(SchemaConflictError):
apply_raw_snapshot(cache, [first, second], snapshot)
self.assertEqual(cache.get_required(first, "shared").name, "held")
self.assertIsNone(cache.get(second, "shared"))
apply_raw_document_record(cache, schema_document_map([first, second]), raw(first)) # the same type still replaces
self.assertEqual(cache.get_required(first, "shared").name, "remote")
def test_cultnet_replication_helpers_apply_raw_snapshot_and_shard_log(self) -> None:
document = define_database_entry_type(
"replica.item",