Compare commits

...
Sign in to create a new pull request.
Author SHA1 Message Date
a8c2825745 Slot tolerance: a record with unread slots is held read-only, not shed (#124) 2026-11-12 02:45:08 +00:00
c371832736 Rust cache: one record per key, foreign carry, key-identity vector walked (#123) 2026-11-12 01:05:46 +00:00
a0184785a2 Python cache: one record per key, foreign carry, key-identity vector walked (#122) 2026-11-12 00:45:11 +00:00
7a63290469 TypeScript cache: one record per key, foreign carry, key-identity vector walked (#121) 2026-11-12 00:03:19 +00:00
6d1684e1c2 Key-identity vector: retype refusal, foreign carry, key in two stores (C# rows, 12) (#120) 2026-11-11 23:42:09 +00:00
871d81807b Portability CI: use the runner image's Node and Python, venv, job timeout 60 (#119) 2026-11-11 21:55:30 +00:00
d3025d3c19 Fork card: null-answerer vectors for Revision and Retraction (additive) (#118) 2026-11-11 18:30:21 +00:00
3a51bbabad Fork card: decider_count, answerer_count, nullable answerer (reader views) (#117) 2026-11-11 18:12:00 +00:00
19b95ba2ab Walk rows lows: Rust @directory error kind and delete, stale known-gaps (#116) 2026-11-11 18:11:55 +00:00
af9142539e Fork card: raised_at, required nullable instant outside options_hash (#115) 2026-11-11 18:03:41 +00:00
f4eb2a04d1 Readability manifest: legacy-catalog c0 rows and directory-at-path, walked by all four runtimes (#114) 2026-11-11 17:25:05 +00:00
8beb972c68 cultnet-ts test: reconnecting client sends only after its own handshake (#113) 2026-11-11 16:05:36 +00:00
5fa7057ed1 Duplicate-key residuals: changelogs, retype by schema id, Rust CAS retype (#112) 2026-11-11 15:58:39 +00:00
0ebc7d63e8 Variants cut doc: C4 records Nibu's ruling on where hull families live (#111) 2026-11-11 15:08:01 +00:00
4246a37284 Single-file readers refuse duplicate keys in all four runtimes; c0 bad-canonical row (#110) 2026-11-11 15:07:48 +00:00
b88c0a8ee2 CI: one portability run per change; quic-native only for native paths (#109)
All checks were successful
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Successful in 36s
2026-11-11 14:24:23 +00:00
f2bb0dd1dd cultcache-ts: tests pin the single-put judge's alias and key-pair order (#108)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Successful in 25m21s
2026-11-11 13:56:35 +00:00
7c2e514941 Unique-index vector: three-holder normal-form and astral-alias rows 34-37; walk counts raised in all four runtimes (#107)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-11 13:41:10 +00:00
17caa0c874 cultcache-rs: judge_against names the least violation by UTF-16 order; walk refusals.msgpack (#102)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-11 12:16:57 +00:00
1211e8c6dc cultcache-ts: a load and an index registration name the least violation of the resulting state; walk refusals.msgpack (#103)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-11 12:00:00 +00:00
3deb270458 cultcache-py: a unique-index refusal names the least violation of the resulting state; walk refusals.msgpack (#101)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-11 11:48:38 +00:00
7225835d11 org.gamecult.cultlib 3.0.0: plugins carry CultElementId (#106)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-11 11:42:43 +00:00
f1edfa458d CultMath 0.5.1 (math.fbm), Unity package rebuilt (#105)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-11 11:42:34 +00:00
a57397aa7f Unique-index vector: group identity, distinct values, normal-form keys, single-put alias order (rows 26-33) (#104)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-11 11:42:27 +00:00
Libby
8eb77546eb org.gamecult.cultlib 3.0.0: plugins regenerated by build-unity-package.ps1 -UpdateTemplate (CultElementId, element ids, write sets)
All checks were successful
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Successful in 24m7s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 1m27s
The native QUIC bridge DLL is not restaged: its source is unchanged since 2.0.1 and the rebuild is not byte-stable.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-11 11:36:06 +00:00
Libby
6e0e2feb74 org.gamecult.cultlib 3.0.0: release prep (version, changelog heading, install pin)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-11 11:21:25 +00:00
Libby
3cb5a1753a CultMath 0.5.1: Unity plugin DLL and pdb rebuilt on Windows at 0.5.1
All checks were successful
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Successful in 28m45s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 26s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-11 11:06:16 +00:00
063835f679 Unique-index vector: holder identity and replaced set are ordinal (rows 23-25) (#100)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-11 10:47:13 +00:00
Libby
8adf014c66 CultMath 0.5.1: version, changelog (fbm), cultlib dependency pin
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-11 10:42:09 +00:00
23d254a501 CultCache: a unique-index refusal names the least violation; shared refusals vector (#99)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
2026-11-11 10:05:32 +00:00
7002c252ed Element ids: the existing-member route and one-way v3/v5 rollback, documented (#98)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
2026-11-11 08:53:13 +00:00
a6861adadb Unique-index leftovers: TS registerIndex refuses a role, commit vector in TS/Rust, nil document in the change schema (#97)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-11 08:43:52 +00:00
5e610b1b6e CI: run cultcache-py, cultcache-rs and GameCult.Caching.Tests on push and PR (#87)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-11 08:12:07 +00:00
447b310b4d cultcache-py: load order, role-alias register_index refusal, C# two-change commit vector (#96)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-11 08:05:39 +00:00
43aa451fb9 cultcache-rs: #[cultcache(index)] is unique at commit and load; re-registration re-judges held records (#95)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-11 07:43:22 +00:00
a7c22e6ee5 cultmesh-browser test: make the unallocatable-size refusal certain on hosts that overcommit (#93)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
2026-11-11 07:35:33 +00:00
ba25b43369 cultcache-py, cultnet-py: indexes means unique, roles names selection aliases; commitId/commitCount on change messages (#94)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
2026-11-11 06:59:08 +00:00
31d86e2a50 cultcache-ts and cultnet-ts: indexes means unique, roles names selection aliases; commitId/commitCount on change messages (#92)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Failing after 14m50s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Successful in 51s
2026-11-11 05:13:08 +00:00
abd1a92eb2 Replica commit hardening: Reconcile sends its diff as one tagged commit; straggler guard inside the lock; real-server and budget pins (#91)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-11 04:57:59 +00:00
7760219b7c CultCache: [CultIndex] is unique at commit and load; [CultRole] names a selection role; replicas apply a commit's changes as one batch (#90)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Failing after 20m15s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Successful in 48s
2026-11-11 03:48:08 +00:00
de8a062ab9 Fork card: stale answer over an oversize re-read is Refused(stale_fork), never counted; refusal text is informational (#89)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Failing after 17m31s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Successful in 57s
2026-11-11 02:10:37 +00:00
f6c9bf5f97 Fork card: fork_too_large (422) for a list whose first entry, or a read whose entry, is past the reply cap (#88)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-11 01:29:35 +00:00
9d6ee72eb3 Fork card: order is a unique rank per Mind; duplicate-order and order-0 vectors (#86)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-11 00:33:36 +00:00
44149afb7b cultcache-ts: opening a legacy store never rewrites it (#85)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-10 23:56:57 +00:00
e57fbaee90 Merge pull request 'tools/sprite-silhouette: sprite alpha to CSS shape-outside polygon' (#8) from libby/sprite-silhouette into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
2026-11-10 23:14:06 +00:00
0c19eddb77 Fork card: a stripped read serves note: null (#84)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-10 20:59:52 +00:00
43db7af238 Fork card: no fork hidden by fork_unknown (#83)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-10 20:51:18 +00:00
eb8939011e Fork card: Read visibility line agrees with ruling (c) (#82)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-10 20:50:47 +00:00
512e0e249f Fork card: read visibility applied at the door (#81)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-10 20:49:43 +00:00
89755b08b5 Fork card: deciders holds at most 64 (#80)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-10 20:03:58 +00:00
3d70e3395d Fork card M5: the desk envelope's names (#79)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-10 19:59:07 +00:00
7d2cde047f Fork card M5: decider list sent per call (#78)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-10 19:27:52 +00:00
ef648d1a66 Fork card: stored Refused answers carry their 4xx status (#77)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-10 19:25:54 +00:00
6231ecad5d Fork desk digests: confirmed on Huginn main f0ca504 (#76)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-10 18:30:43 +00:00
e12ee63840 Fork desk digests: door-form legal_retraction golden (#75)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-10 18:19:26 +00:00
976f8c3bf2 Fork desk r4: refused retraction target, counted_answer_id, body digest vectors (#74)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-10 18:14:42 +00:00
soul
de00274cd0 Merge remote-tracking branch 'origin/main' into libby/sprite-silhouette
All checks were successful
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Successful in 21m53s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 1m19s
2026-11-10 17:40:26 +00:00
a499c58108 CI: concurrency groups, no cancel on tag/dispatch (#71)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-10 17:40:13 +00:00
8ca1d18454 Fork card: cap depends at 8 lines of 1000 bytes (#73)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Failing after 18m46s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Successful in 52s
2026-11-10 17:16:26 +00:00
e6a279248c CultNet governance contracts: vote API and fork-desk card (#72)
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
2026-11-10 17:01:43 +00:00
55bf180647 Merge pull request 'CI: Windows jobs workflow_dispatch only, PowerShell on Linux, glibc comment' (#68) from libby/runner-label-docker into main
All checks were successful
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Successful in 15m24s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Successful in 46s
2026-11-09 22:21:02 +00:00
soul
356687b717 cultmesh-ts test: result window outlasts the transport ack in the RUDP catalog publish test
All checks were successful
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Successful in 18m17s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 51s
The put's flush is bounded by the result window, so a 25 ms flush and 100 ms window made a slow
ack on a loaded forge runner surface as local_send_failed instead of the window_elapsed the test
checks. Drop the flush override and give the window room.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 22:01:25 +00:00
Libby
a2bb0bced4 cultnet-ts test: wait for loopback datagrams instead of a fixed 100 ms sleep
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 11m15s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 1m7s
"server mode admits a pinned client that restarts on a new port" asserted the Accept arrived after sleep(100) and failed on the forge runner (ok(accept) undefined, run 894). It now polls for the packet or frame it asserts on, up to 2 s; no assertion changed.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 20:19:03 +00:00
Libby
ef82811f2e CI: fetch tags in cultmesh-portability checkout
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 13m26s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 36s
scripts/test-declared-dependency-ranges.mjs asserts the clone has release tags; actions/checkout fetches none by default.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 20:03:46 +00:00
Libby
a56ae56b7d CI: install the Rust toolchain on forge Linux runners
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 12m0s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
The act-latest image has no cargo; the cross-runtime CultCache store tests build the Rust interop example and failed with "cargo: not found" once the RUDP flake and pwsh were out of the way.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 19:50:50 +00:00
0fc0eb45ee Merge branch 'main' into libby/runner-label-docker
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 10m54s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
2026-11-09 19:32:08 +00:00
aff69c2914 Merge pull request 'cultmesh-ts test: queued provider registers observed from creation (RUDP replaced-generation flake)' (#70) from libby/rudp-queue-test-unhandled-rejection into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Failing after 6m51s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Successful in 16s
2026-11-09 19:32:05 +00:00
soul
073efd06e7 ci: rerun 3 (RUDP flake evidence)
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 8m23s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 41s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 16:53:27 +00:00
88545542bb Merge pull request 'write_admitted messageId: letter or number, no Default_Ignorable, raw compare' (#69) from libby/write-admitted-messageid-rule into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Failing after 6m48s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Successful in 21s
2026-11-09 16:49:35 +00:00
soul
c9cdbc7edc ci: rerun 2 (RUDP flake evidence)
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 8m38s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 16:29:33 +00:00
soul
2aade7fdd3 Docs and test comments: correct the Rust duplicate-key citation, say what the TS and Rust duplicate tests pin
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m24s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 21s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 16:25:50 +00:00
soul
27d9fbca94 write-receipt.md: where a received messageId decides, and the single-parser assumption it relies on
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 16:07:11 +00:00
soul
e08d5c47f4 GameCult.Networking: a write_admitted frame with more than one messageId key is malformed
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 16:05:21 +00:00
soul
cc5a9cec9b tests first: repeated messageId key, each runtime's declared policy (C# failing)
Vectors admittedMessageIdDuplicates carry [invalid,valid], [valid,invalid], [valid,valid]. Decoders that see the
duplicate (C#, Rust) must refuse all three; map-collapsing decoders (Python, TS, Kotlin) judge the last key.
Python, TS, Kotlin and Rust already behave as declared; C# fails: it accepted [valid,valid].

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 16:04:58 +00:00
soul
a04014e0b4 ci: rerun 1 (RUDP flake evidence)
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 12m3s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 15:50:27 +00:00
soul
b55521d50d cultmesh-ts test: observe queued provider registers from creation so a client timeout is not an unhandled rejection
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m59s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 51s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 15:50:10 +00:00
soul
5679417a82 CI: back to ubuntu-latest, install PowerShell where a Linux job needs pwsh
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 9m13s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 47s
The forge has two labels: docker (node 22, py 3.11) and ubuntu-latest (act-latest, node 24,
py 3.12); neither carries pwsh. Windows legs stay workflow_dispatch only.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 15:26:27 +00:00
soul
a97861945c write-receipt.md: the messageId rule is on the decoded string; byte-level UTF-8 strictness is the reader's, not required
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 32s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Failing after 30s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 15:13:09 +00:00
soul
6c130581a1 cultmesh-kotlin: UTF-8 strictness scoped to the write_admitted frame, not the shared MessagePack reader
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 15:13:01 +00:00
soul
99273f468c cultnet-ts: refuse a write_admitted messageId that is not well-formed Unicode (lone surrogate)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 15:12:07 +00:00
soul
2f372eb2f1 GameCult.Networking: check every messageId key of a write_admitted frame, not only the first
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 15:11:53 +00:00
soul
f378d4fdb0 tests first: duplicate messageId, ill-formed id, scoped UTF-8 strictness, every table range pinned (failing)
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
Failing before the fix: C# 1 (duplicate messageId, second is U+200B), TypeScript 1 (a + lone surrogate),
Kotlin 1 (a document payload string with a lone surrogate no longer decodes). Python and Rust already pass
the new vectors. The range probes read the generated write-admitted-message-id-ranges.json and pass on the
correct tables; they exist to fail when any range is deleted or narrowed.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 15:11:15 +00:00
soul
8598fd3d12 cultnet-rs: write_admitted messageId from the generated Unicode 16.0 tables (letters and numbers now 16.0, not 15.1)
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 15:05:00 +00:00
soul
c2d2c1d126 GameCult.Networking: write_admitted messageId from embedded Unicode 16.0 tables, not .NET's
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 15:05:00 +00:00
soul
c4f6e76839 cultmesh-kotlin: write_admitted messageId from embedded Unicode 16.0 tables, not the JDK's
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 15:04:59 +00:00
soul
87f53498ed cultnet-py: write_admitted messageId from embedded Unicode 16.0 tables, not unicodedata
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 15:04:59 +00:00
soul
de25b50abd cultnet-ts: write_admitted messageId from embedded Unicode 16.0 tables, not the regex engine's
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 15:04:59 +00:00
soul
3d37fda496 write_admitted messageId: pin Unicode 16.0, one generator for the embedded tables
scripts/gen-unicode-messageid-tables.py emits each runtime's Letter-or-Number and Default_Ignorable tables from the two UCD 16.0 files; --check fails on a stale or hand-edited table.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 15:04:59 +00:00
soul
cc931932fc tests first: Unicode 16.0 pin vectors (failing)
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 6m15s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 43s
Failing before the fix: Rust 1 (16100), Python 2 (16100, 10D40), Kotlin 4 (16100, 10D40, 2EBF0, 2EE5D).
TypeScript and C# already pass: their platform data is 16.0 or later.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 15:00:54 +00:00
soul
ba548d192d cultnet-rs: refuse a write_admitted messageId without a letter or number or with a Default_Ignorable code point
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
Raw compare. std is_numeric for N; embedded letter (Unicode 15.1) and Default_Ignorable (16.0) range tables.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 14:49:50 +00:00
soul
d546ef9c7f GameCult.Networking: refuse a write_admitted messageId without a letter or number, with a Default_Ignorable code point, or with invalid UTF-8
Raw compare. .NET Unicode categories for L/N, an embedded Default_Ignorable range table (Unicode 16.0).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 14:49:50 +00:00
soul
d1a4ebdd68 cultmesh-kotlin: refuse a write_admitted messageId without a letter or number or with a Default_Ignorable code point
Raw compare. JDK Character categories for L/N, an embedded Default_Ignorable range table (Unicode 16.0).
The MessagePack string reader now refuses invalid UTF-8 instead of substituting U+FFFD.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 14:45:37 +00:00
soul
688562c316 cultnet-py: refuse a write_admitted messageId without a letter or number or with a Default_Ignorable code point
Raw compare. unicodedata for L/N, an embedded Default_Ignorable range table (Unicode 16.0).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 14:43:26 +00:00
soul
bf55c10199 vector: write_admitted messageId refuses U+FEFF before a letter (kills a trim-before-check decoder)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 14:42:44 +00:00
soul
4c1537885d cultnet-ts: refuse a write_admitted messageId without a letter or number or with a Default_Ignorable code point
Raw compare, no trim. The platform's regex property tables carry both properties; no table embedded.
D800 vector now driven from wireHex.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 14:42:01 +00:00
soul
70f8135d2c vector: write_admitted messageId accepts a NUL between letters (rule: letter present, no Default_Ignorable)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 14:41:11 +00:00
soul
4571a0d4e7 tests first: write_admitted messageId rule vectors (failing)
Pins Heimdall's rule: a write_admitted messageId needs a Unicode letter or number and no
Default_Ignorable_Code_Point, compared raw. Shared vectors plus a runner test in TS, Python,
Kotlin, C# and Rust. No decoder changed; the refuse vectors fail.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 14:39:51 +00:00
325da81428 Merge pull request 'TS write-receipt client against the real Rust and C# receipt servers (interop harness)' (#67) from libby/write-receipt-ts-interop into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Failing after 7m52s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Successful in 59s
2026-11-09 14:16:00 +00:00
e4f550ef78 Merge pull request 'cultnet-py: decode write_admitted.v0 and error.v1 against the write-receipt vectors' (#66) from libby/write-receipt-py-interop into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-09 14:15:59 +00:00
59365cea35 Merge pull request 'cultmesh-kotlin: write receipt frames, hash, tracker, vector replay' (#65) from libby/write-receipt-kt-interop into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-09 14:15:58 +00:00
5ba3e9e2aa Merge pull request 'Rust RUDP server: pin blank schemaId/recordKey refusal, decide sequenced refusal per frame' (#63) from libby/rs-receipt-key-tests-per-frame into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-09 14:15:49 +00:00
69703358c0 Merge pull request 'write-receipt: one unreadable text, one blank-field rule (both servers)' (#64) from libby/write-receipt-text-blank-rules into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-09 14:15:48 +00:00
soul
96c5fb1ca4 interop peer: C# store holds the spec's admitted schema-id names (compatible ids)
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 6m32s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 37s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 14:12:38 +00:00
soul
b8e40dc372 interop test: pace C# relay connections under the server's per-IP per-minute limit
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 14:10:42 +00:00
soul
1704fb5060 cultmesh-ts: test:interop builds cultcache-ts and cultnet-ts (with contracts) first
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 14:05:57 +00:00
Libby
041a769475 CI: quic-native glibc comment no longer names ubuntu-latest as the runner
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / docker reactive semantics and leases (pull_request) Failing after 4m48s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 32s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 13:40:39 +00:00
Libby
fc0f83a385 CI: Windows jobs are workflow_dispatch only (no Windows runner exists)
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
CultMesh portability and performance / docker reactive semantics and leases (pull_request) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 13:32:41 +00:00
Libby
3e6ec8567c CI: forge runner label is docker, not ubuntu-latest
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / docker reactive semantics and leases (pull_request) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 13:30:05 +00:00
Libby
8c31933e10 cultnet-py: the write-receipt vector sections include the file's counts
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 7m35s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 51s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 10:38:29 +00:00
Libby
2f3b30ab85 Merge remote-tracking branch 'origin/libby/write-receipt-text-blank-rules' into libby/rs-receipt-key-tests-per-frame
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 8m39s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 1m0s
2026-11-09 10:37:49 +00:00
Libby
819df6a666 Merge remote-tracking branch 'origin/libby/write-receipt-text-blank-rules' into libby/write-receipt-ts-interop
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 10m47s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 45s
2026-11-09 10:37:49 +00:00
Libby
e490828278 Merge remote-tracking branch 'origin/libby/write-receipt-text-blank-rules' into libby/write-receipt-py-interop 2026-11-09 10:37:49 +00:00
Libby
6ca9c455b7 cultmesh-kotlin: sendFailed vectors, a tracker that ends with its session, White_Space blank
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 6m53s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 1m12s
The replay runner handles the sendFailed event, fails on an event key it does
not know and on a vector count the file did not declare. WriteTracker settles a
write sent after sessionEnd as session_ended. Source ids on a raw document
record are dropped when blank by Unicode White_Space (isBlankText), not Kotlin's
isNullOrBlank, which keeps U+0085 and drops U+001C to U+001F; the vector file's
JSON reader skips JSON whitespace only.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 10:37:48 +00:00
Libby
1136080aaf Merge remote-tracking branch 'origin/libby/write-receipt-text-blank-rules' into libby/write-receipt-kt-interop 2026-11-09 10:34:56 +00:00
Libby
f0c0f936ba write-receipt: seven tracker vectors, a tracker that ends with its session, runners that count
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 7m39s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 22s
New scenario vectors: id-less error.v1 with one and with two writes unanswered,
sendFailed, an id-less error after the window, a receipt one ms inside the window
and at the exact boundary (late), a send after sessionEnd. The last exposed a
TS tracker leaving such a write pending forever; it now settles session_ended.
The file declares counts, and the TS replay runner fails on a count it did not
run or an event key scenarios.semantics does not name.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 10:34:49 +00:00
Libby
b082101c78 cultnet-ts: a blank text in a raw frame is Unicode White_Space, not trim()
validateBinaryMessageRoot, requireNonEmptyString and requireOptionalStringArray
used String.prototype.trim(), which takes U+FEFF and misses U+0085, so the TS
decoder and the servers disagreed on what names nothing (write-receipt rule 5).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 10:22:15 +00:00
Libby
636b387cf1 write-receipt spec: a repeated or unreceived sequenced packet draws no second answer
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m30s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 16s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 09:13:31 +00:00
Libby
6c441d1cda Rust RUDP server: answer a sequenced write once, not once per send
A repeat of a packet the session already took, and a packet too far ahead to be
remembered, delivered no frame and so drew the superseded-write refusal again on
every send, breaking write-receipt rule 1. The refusal now applies only to the
first packet the session takes for a sequence; CultNetRudpSession::has_received
exposes that test.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 09:13:18 +00:00
Libby
34cceedf98 Rust RUDP server: pin the superseded-write refusal against a releasing packet and the half-space bound
cargo-mutants on the previous commit missed two mutants: `&&` to `||` in the
superseded-packet check (a packet that also releases another write hid it) and
`<` to `<=` at the half-space bound of sequence_at_or_before. A test and two
assertions kill them; the rerun catches 14 of 15 (1 unviable).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 09:13:18 +00:00
Libby
4e89f07bff Rust RUDP server: answer a superseded sequenced write; order frame and packet by serial arithmetic
Superseded packet: a reliable-sequenced packet that arrives after a newer one
delivers no frame, yet is acknowledged and never resent, so its write went
unanswered. The test (red first: only "new" was refused, "old" got nothing)
showed the drop happens in the session below frame decoding, but an unfragmented
packet carries its whole frame, so the server can read the id from the packet
payload. Chosen: refuse it from the packet payload. A superseded fragment of a
larger write has no readable id in the packet, so write-receipt.md now says that
write may go unanswered and the client's unconfirmed outcome covers it.

Ordering: frame.sequence <= packet.sequence is replaced by sequence_at_or_before,
serial arithmetic modulo 2^32. Deviation from the brief: cultnet-rs'
at_or_before carries a 4096-sequence receive window made for duplicate detection;
a frame's first fragment may lie further back than that, so reusing it would
leave such a sequenced write unrefused and applied. The new test pins that, and a
hand mutant with the window fails it. A wrapped sequence cannot reach this
compare through the session today (it is judged a duplicate), so the unit test
pins the helper, not an end-to-end wrap.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 09:13:18 +00:00
Libby
bd2aff8ed0 Rust RUDP server: decide the sequenced-channel refusal per frame
A sequenced packet's arrival can release held ordered frames in the same delivery; refuse only the packet's own frame.
heimdall-low-refuse-sequenced-per-frame.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 09:13:04 +00:00
Libby
4f6ee0db83 Rust RUDP server: pin the refusal of blank schemaId/recordKey on receipt sessions
heimdall-low-rust-keys-untested.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 09:13:04 +00:00
Libby
04207007d6 interop test: read the store under the record key the blank-field vector set
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 3m0s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 46s
A vector that sets recordKey (zero_width_space_record_key_is_a_name) replaces the harness key, so the admitted case waited for
blank:<name> and timed out against the Rust server, and the refused record-key cases counted applications under a key the server
never saw. Use the key the document carried.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 07:45:35 +00:00
Libby
ec8ecd2fb0 Merge write-receipt-text-blank-rules (3b1b7697) into libby/write-receipt-py-interop
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 7m32s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 46s
2026-11-09 07:38:23 +00:00
Libby
80f1cfa385 Merge write-receipt-text-blank-rules (3b1b7697) into libby/write-receipt-kt-interop
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 7m5s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 26s
2026-11-09 07:38:23 +00:00
Libby
4b5f343a44 Merge write-receipt-text-blank-rules (3b1b7697) into libby/write-receipt-ts-interop 2026-11-09 07:38:23 +00:00
Libby
3b1b769762 write-receipt spec: name the platforms that disagree accurately
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m3s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 26s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 22:27:17 +00:00
Libby
52204c68c9 write-receipt vectors: U+001C to U+001F and padded names are not blank
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 22:26:28 +00:00
Libby
140eca5138 cultmesh-ts: TS write-receipt client against the real Rust and C# receipt servers (test:interop)
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m56s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 31s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 22:06:55 +00:00
Libby
e4d2446dbe C# InteropPeer: receipt-serve (CultNetDatabaseServer on an ephemeral port) and receipt-relay (LiteNetLib byte shim for the TS client)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 22:06:55 +00:00
Libby
56b0c1679c cultmesh-rs: receipt_interop_server example, the real RUDP document server on an ephemeral port for the interop harness
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 22:06:55 +00:00
Libby
f6c61b1ebf cultnet-py: decode cultnet.write_admitted.v0 and cultnet.error.v1 against the write-receipt vectors
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m36s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 31s
Strict closed-schema decode, content hash helper, hello capability; vectors contentHash, wire and schemaInvalid replayed.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 22:06:08 +00:00
Libby
cbb336f501 cultmesh-kotlin: decode write_admitted.v0 and error.v1, content hash, WriteTracker, vector replay
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 6m5s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 26s
Strict decode of the two receipt frames (and error.v0), hello capability test,
the dedupe content hash and the client settle rules, ported from cultnet-ts.
write-receipt-vectors.json replays 38/38 (constants, contentHash, wire,
schemaInvalid, scenarios, v0 limit_reached); dedupe and blankFields are server
inputs and not run.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 22:03:27 +00:00
Libby
db5e8834f3 write-receipt: blank is exactly Unicode White_Space; C# server checks the listed set in one place
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 22:02:49 +00:00
Libby
ccbe392b28 write-receipt vectors and tests: blank is Unicode White_Space, every code point checked
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 22:02:10 +00:00
Libby
98989f90ea C# tests: a nil payload names nothing (mutant survivor)
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 6m30s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 36s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 21:57:38 +00:00
Libby
bc9a1842f5 C# database server: refuse a put that names nothing, one unreadable text
IsUnreadableDocument follows the spec's Required fields rule (blank storedAt, empty payload, blank source ids and tags join blank schemaId/recordKey); ForUnreadableMessage is the single owner of the fixed text. Tests replay the blankFields vectors on receipt and legacy connections.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 21:26:49 +00:00
Libby
e99e9d4bc0 write-receipt spec: one unreadable text, one blank-field rule; Rust server tests pin both
The unreadable-frame text is 'message_invalid: the message could not be read' (the servers' wording; the Rust wire never carried decoder text, only its local rejection display). A put that names nothing (blank schemaId/recordKey/storedAt, empty payload, blank present source ids or tags) is refused as unreadable on every connection; the vectors' blankFields section lists the cases and the Rust server tests replay them.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 21:26:49 +00:00
7665df33ae Merge pull request 'TS: write_admitted / error.v1 decoder, WriteTracker, vector replay runner' (#62) from libby/ts-write-receipt-client into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Failing after 5m10s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Successful in 31s
2026-11-08 19:52:37 +00:00
92b4cb7ce9 Merge pull request 'C#: database server answers writes with write_admitted / error.v1 (write-receipt spec)' (#61) from libby/cs-write-receipt-server into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-08 19:51:52 +00:00
36494dafb9 Merge pull request 'cultmesh-rs: RUDP document server answers writes with write_admitted / error.v1' (#60) from libby/rs-write-receipt-server into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-08 19:49:50 +00:00
5bf2428295 Merge pull request 'spec: cultnet write receipt (write_admitted, error id echo, admitted/refused/unconfirmed)' (#59) from libby/cultnet-write-receipt-spec into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-08 19:49:40 +00:00
Libby
1a228349f1 C#: a put or delete with an empty or blank schemaId or recordKey is refused message_invalid (as an unreadable frame, like the Rust server) and not applied
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m35s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 31s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 19:46:45 +00:00
Libby
621b86ca15 Spec #59: an empty or blank schemaId or recordKey is message_invalid (details.field message, the write's id), never applied; vector for it
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m55s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 46s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 19:45:19 +00:00
Libby
90d252735f cultmesh-rs: a refused reliable-sequenced write is received and acknowledged first, and a refused write counts as session activity
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 6m11s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 52s
Refusing before session.receive left the packet unacknowledged: the client resent it forever and every later ordered write stalled. The test now uses the same peer and proves a later ordered write is admitted.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 19:38:43 +00:00
Libby
dc1ae51e92 cultmesh-ts: publishRudpDocumentOnce builds the put before connecting, so a put that cannot be built keeps its cause on both paths
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 6m13s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 31s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 19:37:08 +00:00
Libby
0fbf2baea6 cultnet-ts, cultmesh-ts: Heimdall's five Lows on send-only (spec wording, error.v1 ack-time refusal tests, build errors keep their cause, presence changelog, ack-cut anchors)
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 19:32:49 +00:00
Libby
89ff3d2cbd cultnet-ts test: the Odin presence publisher is send-only, so a refusal after the transport ack is not awaited
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 19:26:25 +00:00
Libby
206dbe142b cultmesh-ts: sendOnly tests use one fake server per publish (the reused one threw on the second connect and hung the run); spec names send-only and the presence publisher
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
The hang was the fake server, not the writer: it is bound to one connection id, and two publishes in one test sent a second connect it rejected from inside the socket handler. The unacknowledged-send test also scripted an admit, whose reply piggybacks the put's ack.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 19:24:52 +00:00
Libby
0caef4d564 WIP (unverified): publishRudpDocumentOnce sendOnly + presence publisher send-only; tests written, cultmesh-ts receipt-client run hung
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 19:24:52 +00:00
Libby
c89c4ee39a cultmesh-ts: test kills the flush-cap mutant
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m58s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 1m18s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 17:07:54 +00:00
Libby
af1ac72359 cultmesh-ts, cultnet-ts: only write_admitted admits a put (no echo admission); the send is bounded by the result window and a timeout is local_send_failed
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 17:05:42 +00:00
Libby
293b4e83bc cultmesh-ts, cultnet-ts: an unanswered put is unconfirmed on every server, session_ended before a result is an error, a failed local send is its own reason
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 17:03:39 +00:00
Libby
a67faad27c cultnet-ts: tests for two tracker mutants (send expires other windows; non-answer frames settle nothing)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 17:03:39 +00:00
Libby
6eea66c57f cultnet-ts, cultmesh-ts: write receipt client (frames, WriteTracker, replay runner, opt-in writer)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 17:03:39 +00:00
Libby
34bf7aa0ee C#: test kills the schema id detail-value mutant
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m59s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 31s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 17:03:36 +00:00
Libby
40b34295e3 C#: receipt writes need a reliable, non-sequenced channel; a refusal without a record key or schema id carries a null detail value
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 17:01:59 +00:00
Libby
e4b67a9e0b cultmesh-rs: a receipt session accepts writes only on a reliable, non-sequenced channel; a test pins that a reconnect starts a fresh session
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 10m5s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 1m8s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 17:00:48 +00:00
Libby
74adad916a Spec #59: receipt writes need a reliable, non-sequenced channel; error.v1 schema description names limit_reached; #55-#58 paragraph states which clients still window-succeed
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 9m8s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 1m14s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 16:59:39 +00:00
Libby
48cf3b3323 C#: tests kill the replay-details and unreliable non-write mutants
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 16:45:29 +00:00
Libby
0a67b4d8aa C#: a remembered refusal keeps its code and field, never the write's strings, and replays in fixed text; a write on an unreliable channel of a receipt session is refused message_invalid (channel)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 16:42:22 +00:00
Libby
1a05df2524 cultmesh-rs: server-wide remembered-write budget is a maintained counter; a session whose refusal set overflowed refuses unseen ids after the budget frees
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m35s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 1m1s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 16:36:10 +00:00
Libby
2c5203dc82 Spec #59: local_send_failed is its own unconfirmed reason
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m31s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 31s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 16:34:52 +00:00
Libby
3443e1dc93 Spec #59: error.v1 code enum names limit_reached; server-wide budget is a conformance column; an overflowed refusal set refuses unseen ids for the session
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 16:34:19 +00:00
Libby
c8f869e54e cultnet-rs, cultmesh-rs: null or missing document/schemaId refused naming the field; budget tests use literal bytes; unreliable-refusal local failure test
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m33s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 26s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 15:32:03 +00:00
Libby
d7b36416c6 cultmesh-rs: pin limit-refusal memory bound and legacy unreliable drop (mutant-driven tests)
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 15:30:36 +00:00
Libby
ef5cd88111 C#: v0 errors for snapshot, shard log and variant refusals carry fixed text, never an exception's message
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m35s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 36s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 15:13:58 +00:00
Libby
d654cb2b99 C#: tests kill the refusal-set bound and abandoned-rehash mutants
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 15:12:31 +00:00
Libby
1447ccd8dd cultnet-rs, cultmesh-rs: write_admitted in spec field order; server-wide remembered-write budget over digest entries; id size limit; limit refusals remembered; unreliable writes refused; local failure is its own goodbye
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m14s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 46s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 15:09:28 +00:00
Libby
c4d1efe000 C#: receipt ids capped at 256 bytes and held as digests; limit refusals remembered, no slot freed; required fields refused with fixed text; RUDP server advertises no receipt frames
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 15:07:23 +00:00
Libby
734087766a Spec #59: id size, digest memory, limit refusals remembered, required fields, reliable channel, local send failure, per-class conformance table
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 6m0s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 46s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 15:04:21 +00:00
Libby
541d734675 C#: a forward is retryable only when provably not sent; unconfirmed otherwise; refusals carry fixed text
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
CultNetWriteNotSentException names the pre-send boundary (connect, hello, no endpoint,
or a send port that proves nothing was written). Every other forwarding failure keeps
the session claim. ForUnownedSchema and the routing refusal no longer carry exception text.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 15:02:56 +00:00
Libby
b89a883700 spec: C# RUDP schema server is not receipt-capable; advertise only if end to end
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 14:58:51 +00:00
Libby
2e5df24e96 C#: loopback test for the unreadable-frame answer; Server.Port seam; the answer now reaches the wire
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
An undecodable frame is classed legacy by Decode, so the schema-side guard was unreachable
and a receipt peer was never answered. The guard moves to the legacy decode.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 14:58:17 +00:00
Libby
74b8a4cbf8 C#: drop the redundant Silent state of a write claim (a silent claim is an unanswered one)
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 14:53:05 +00:00
Libby
0182a6d40b cultnet-rs, cultmesh-rs: remembered-write cap refusal is error.v1 limit_reached; pin authorize-once with a counting sink
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m19s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 41s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 14:32:49 +00:00
Libby
ed5f9d30d1 C#: receipt server follows Heimdall's review of #61
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m30s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 21s
Look the id up first and authorize only on first sighting; remember refusals within the cap; never re-forward an id whose primary stayed silent; end the connection when an answer cannot be queued; fixed refusal texts; limit_reached cap refusal; commit-point doc on ApplyPut/Delete.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 14:32:39 +00:00
Libby
eafcdcdf7c cultnet: add limit_reached to error.v1 code enum, with captured vector
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 4m51s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 26s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 14:29:18 +00:00
Libby
5f699c27fa cultmesh-rs: a put reusing a delete's id is refused (mutant-driven test)
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 14:27:54 +00:00
Libby
d4f440ac7c cultmesh-rs: receipt server looks the id up first, remembers refusals, answers deletes and unqueueable refusals
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 4m47s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
Follows Heimdall's second review of #60: replay by (session,id) before any
authorization; refusals stored with their hash; delete shares the dedupe
table and settles capability; a refusal the queue cannot take ends the
session; a peer's refusal frames and a receipt session's refused deletes are
not violations; the write limit has its own fixed text; the sink trait says
Ok means committed and visible.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 14:22:50 +00:00
Libby
b0ea18371f C#: database server answers writes with write_admitted / error.v1 (write-receipt spec)
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m46s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 36s
Opt-in per connection by the first hello (or first write); the server answers a
capable hello with its own. A put or delete commits, then gets write_admitted under
its id; every refusal is error.v1 with the id (an unreadable frame carries the id
when it still holds one). (actor, messageId) dedupe is bound to the content hash and
asked only after the write authority grants; same id other content is message_invalid.
The shard write forwarder says hello, waits for the primary's receipt, and the server
relays only what the primary answered under the original id.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 14:04:37 +00:00
Libby
7a560b1984 spec: dedupe scope is at least the session; a resend after a reconnect is a new write
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m52s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 41s
Per Heimdall's review. The dedupe key lives for the life of its session, bounded per
session without forgetting an id the server promised to answer. A longer retention
is optional and never to be relied on. After a reconnect callers resend only writes
that are idempotent by record key and read back before resending any other.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 13:58:39 +00:00
Libby
8bd31a5d4a cultmesh-rs: receipt server follows Heimdall's review of #60
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 7m11s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 26s
A refused put, a full dedupe table and a reused id refuse the write, not the
session; only a protocol violation ends one. A frame that does not decode but
holds a messageId is refused with that id. The server answers a receipt-capable
hello with its own, listing both frames. The sink's authorization (new
authorize_raw_document, default admit) runs before the session's remembered
writes are consulted. A second hello does not change the capability: it is set
once, by the first hello or the first write. A receipt that cannot be queued, and
a write the payload budget would drop silently, end the session instead of
leaving the write unanswered. Refusals to a receipt-capable session are queued
reliably.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 13:56:17 +00:00
Libby
45e95984e5 WIP C# write-receipt server (not built, not tested): messages, content hash, per-peer dedupe state, receipt-aware wait
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 13:41:14 +00:00
Libby
9f73047c29 cultmesh-rs: RUDP document server answers writes with write_admitted / error.v1 (write-receipt spec)
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
cultnet-rs gains cultnet.write_admitted.v0, cultnet.error.v1 (v0 + messageId) and
write_content_hash. The server opts a session in only when its hello lists both frames
(framing only; admission rules are unchanged). On an opted-in session an admitted put is
answered with a receipt carrying its id after the sink accepts it, a refusal is an error.v1
naming the id, a resend of the same content is re-receipted without reaching the sink, and the
same id with other content is refused message_invalid (never the old receipt). Other sessions
are answered exactly as before.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 13:38:46 +00:00
Libby
ab7bc771e3 spec: write receipt per Heimdall's review (dedupe by actor+content hash, admitted-late, 5 s window, capability is framing only)
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 6m28s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 2m19s
- messageId unique per authenticated actor and session; dedupe key (actor, id) bound to SHA-256 of
  [operation, schemaId, recordKey, payload]; same id with other content is message_invalid, never the old receipt.
- A late answer after the window is recorded on the write (admitted_late / refused_late); no blind resend.
- A forwarder with no primary answer relays nothing; the client reports unconfirmed.
- RECEIPT_RESULT_WINDOW_MS 5000 (default), LEGACY_RESULT_WINDOW_MS 500 (fallback only), DEDUPE_RETENTION_MS as vector constants.
- The hello capability changes framing only and never authorizes.
- Batches can commit partially; the caller owns retry.
- Vectors: constants, worked content hashes, 7 dedupe cases, late-answer and window scenarios.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 13:31:49 +00:00
Libby
662ae048d8 spec: cultnet write receipt (write_admitted.v0, error.v1 id echo, three-outcome clients)
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
Specification and shared vectors only, no runtime code. A write is answered
exactly once with cultnet.write_admitted.v0 or cultnet.error.v1 carrying its
messageId; the new frames go only to connections whose hello lists them (today's
TS decoder rejects unknown keys and versions, so "old clients ignore it" is
false). Clients settle admitted | refused | unconfirmed; the result window is the
pre-receipt fallback and ends unconfirmed. Wire vectors, schema-invalid cases and
16 replay scenarios in write-receipt-vectors.json.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 13:28:46 +00:00
dc5abedffc Merge pull request 'C#: SubmitDocumentAsync and the shard write forwarder complete on the result frame' (#58) from libby/cs-client-ack-finding into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Failing after 7m52s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Successful in 31s
2026-11-08 13:24:39 +00:00
Libby
09e6a528b8 C#: SubmitDocumentAsync and the shard write forwarder complete on the result frame
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 6m41s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 32s
CultNetWriteCompletion.SendAndAwaitResultAsync registers a refusal handler before
the send, then waits a result window (default 500 ms; CultMeshClientOptions.
SubmitResultWindow, CultNetSchemaWriteForwarderOptions.ResultWindow). A
cultnet.error.v0 inside it rejects with the new CultNetWriteRefusedException
(Code: write_not_authorized, message_invalid, ...) whatever its order against the
send or transport ack; a send returning no longer completes the write. The
forwarder's refusal now throws on the same path as an unreachable primary, so
CultNetDatabaseServer.TryForwardPutAsync/DeleteAsync no longer report a refused
forward as done. Raw send primitives (C# schema client/session, Python, Kotlin)
carry the doc line: fire-and-forget, an ack or a return is not admission, read the
result frame.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 13:24:27 +00:00
1b3b625474 Merge pull request 'cultmesh-rs: single and batch publish complete on the result frame, not the transport ack' (#57) from libby/rs-client-ack-finding into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-08 13:18:41 +00:00
Libby
675a6494c1 cultmesh-rs: single and batch publish complete on the result frame, not the transport ack
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 6m10s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 1m12s
One loop now serves both: every put acknowledged, then a refusal frame (typed
cultnet.error.v0) fails the call in either order against the ack, and a success
frame or the end of the new result_window option (default 500 ms) completes it.
The batch no longer waits on flush_reliable alone, so it cannot hide a refusal.
Raw send primitives (CultNetRudpSocketTransportConnection::send/send_reliable/
flush_reliable, tcp_framed send) carry the doc line: fire-and-forget, an ack or
a return is not admission, read the result frame.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 13:18:40 +00:00
Libby
3c68d7d19c Merge remote-tracking branch 'origin/main' into libby/rs-client-ack-finding 2026-11-08 13:15:17 +00:00
3162e2ea66 Merge pull request 'cultnet-ts: the Odin presence publisher completes on the result frame, not the ack' (#56) from libby/ts-client-ack-finding-cultnet into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-08 13:15:08 +00:00
Libby
531fce567d cultnet-ts test: cancel the Odin stub's delayed send when the test ends
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 7m47s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 57s
The stub's 120 ms second packet fired after peer.close() and threw
ERR_SOCKET_DGRAM_NOT_RUNNING as an uncaught exception, failing whichever
publisher test was running when the timer landed (load-dependent). Timers are
now cleared in the test's after hook instead of swallowing the error.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 13:14:49 +00:00
Libby
0b326fa5ee WIP cultnet-ts: Odin presence publisher completes on the result frame, not the ack (passes alone; two publisher tests fail inside the parallel full run, unexplained)
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m32s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 30s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 11:36:23 +00:00
Libby
cdbe5889e0 merge main 2026-11-08 11:35:00 +00:00
ff1c92d2fe Merge pull request 'cultmesh-ts: a put completes on its result frame, not on the transport ack' (#55) from libby/ts-client-ack-finding-cultmesh into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Failing after 5m38s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Successful in 31s
2026-11-08 11:34:56 +00:00
Libby
7082c9488b cultmesh-ts: a put completes on its result frame, not on the transport ack
publishRudpDocumentOnce resolved when peer.flush returned (ack only) and read no frame, so a
refused put resolved. It now rejects with CultMeshRudpWriteRefusedError (the peer's typed
cultnet.error.v0: write_not_authorized, message_invalid) whether the refusal arrives before the ack,
after it, or within resultWindowMs (default 500 ms) after it; a success reply ends the wait early.
publishRudpDocumentAndWaitForReceipt rejects with the same typed error instead of timing out.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 11:34:55 +00:00
Libby
dc7eec7e6e merge main 2026-11-08 11:33:16 +00:00
d4e4a2c085 Merge pull request 'GameCult.Networking test: the RUDP schema client delivers a refused put's result frame whatever the order of the ack' (#54) from libby/cs-client-ack-is-not-admission into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-08 11:30:21 +00:00
8cfc41d665 Merge pull request 'cultmesh-py test: a put the server refused is reported refused after its reliable ack arrived' (#53) from libby/py-client-ack-is-not-admission into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-08 11:30:18 +00:00
6f2092171f Merge pull request 'cultmesh-rs test: a publish whose refusal arrives before the ack is refused' (#52) from libby/rs-client-ack-is-not-admission into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-08 11:30:15 +00:00
de1d6b3444 Merge pull request 'cultnet-ts test: the Odin presence publisher does not complete on the ack when the refusal arrives first' (#51) from libby/ts-client-ack-is-not-admission-cultnet into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-08 11:30:12 +00:00
822d660627 Merge pull request 'cultmesh-ts test: publishRudpDocumentAndWaitForReceipt reports a refused put as refused' (#50) from libby/ts-client-ack-is-not-admission into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-08 11:30:09 +00:00
Libby
c1d24146db GameCult.Networking test: the RUDP schema client delivers a refused put's result frame whatever the order of the ack
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 8m11s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 1m10s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 11:29:50 +00:00
Libby
c6da8fc54a cultmesh-py test: a put the server refused is reported refused after its reliable ack arrived
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 7m26s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 37s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 11:28:23 +00:00
Libby
9e734447f6 cultmesh-rs test (FAILS, findings): publish completes on the ack when the refusal follows it; the batch publish completes on the ack whatever the refusal
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 7m32s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 26s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 11:27:53 +00:00
Libby
f53930bc13 cultmesh-rs test: a publish whose refusal arrives before the ack is refused
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m16s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 51s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 11:27:48 +00:00
Libby
dfd090dc55 cultmesh-ts test (FAILS, a finding): publishRudpDocumentOnce completes on the ack of a refused put
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 8m38s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 1m18s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 11:27:07 +00:00
Libby
4f90ea8156 cultmesh-ts test: publishRudpDocumentAndWaitForReceipt reports a refused put as refused, whether the ack comes before or after the refusal
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 8m50s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 1m24s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 11:27:06 +00:00
Libby
7766348c75 cultnet-ts test (FAILS, a finding): the Odin presence publisher completes on the ack when the refusal follows it
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 11:26:22 +00:00
Libby
08cf3b34b5 cultnet-ts test: the Odin presence publisher does not complete on the ack when the refusal arrives first
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m59s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 26s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 11:26:22 +00:00
7486f0a326 Merge pull request 'cultmesh-rs: send the capped frame's answer reliably and end the session once it is acknowledged' (#48) from libby/rs-capped-answer-reliable into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-08 11:21:28 +00:00
Libby
b2077fa2bf cultmesh-rs: a Pong in the capped grace acknowledges nothing; the grace deadline saturates
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 8m57s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 1m41s
A Ping in the 2 s grace after the capped frame was answered with a Pong built from the
session's ack state, which includes the capped frame. Every reply while the session is
ending now zeroes its ack fields. The deadline no longer wraps at the end of the clock.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 11:21:08 +00:00
3915a4e5f5 Merge pull request 'GameCult.Networking: resend the capped frame's answer until acknowledged, then end the session' (#49) from libby/cs-capped-answer-reliable into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-08 11:10:54 +00:00
Libby
0b1a468ddd GameCult.Networking: resend the capped frame's answer until acknowledged, then end the session
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 8m47s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 1m13s
The schema server answered the 8th undecodable frame and ended the peer's session in the same
call; ending clears the session's pending reliable packets, so a lost datagram was never
resent. The session now stays until the peer has acknowledged every answer (a lost one is
resent by PollResends) or a 2 s grace runs out, serves nothing after the capped frame
meanwhile, and then sends the goodbye.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 11:10:48 +00:00
Libby
03141bc93b cultmesh-rs: send the capped frame's answer reliably and end the session once it is acknowledged
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
The capped (8th) refusal went out unreliable and unordered with the goodbye at once, so a
lost datagram left the peer without it. It is now queued like any response and resent until
the peer acknowledges it; the session ends then (or after a 2 s grace) and serves nothing
after the capped frame meanwhile. The answer, its resends and the poll that carried the
capped frame acknowledge nothing, as every refusal. A reliable queue that cannot take the
answer ends the session at once with the unreliable refusal, as before.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 11:04:22 +00:00
337168812e Merge pull request 'cultmesh-ts: resend the capped frame's answer until acknowledged, then end the session' (#47) from libby/ts-capped-answer-reliable into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
2026-11-08 10:59:27 +00:00
Libby
9e5edeca52 cultmesh-ts: resend the capped frame's answer until acknowledged, then end the session
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 7m6s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 51s
endCappedSession deleted the session right after the answer was sent, so the resend
timer never saw it and a lost datagram left the peer without the answer. The session
now stays until the peer has acknowledged every reply (a lost one is resent) or a 2 s
grace runs out, serves nothing after the capped frame meanwhile, and then sends the goodbye.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 10:59:23 +00:00
fe73795d89 Merge pull request 'cultmesh-py: answer an unreadable RUDP frame and end the session after 8 consecutive refusals' (#45) from libby/py-refused-frames-cap into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Failing after 6m3s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Successful in 57s
2026-11-08 09:54:41 +00:00
Libby
bf81695537 cultmesh-py: answer the capped frame reliably and end the session once the answer is acknowledged
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m17s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 52s
The 8th refusal was sent unreliable and unordered with only the first response,
then the goodbye at once, so a lost datagram left the peer without it. All answers
now go the normal reliable path; the session ends when the peer has acknowledged
them (a lost one is resent) or after a 2 s grace, and serves nothing after the
capped frame meanwhile.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 09:54:37 +00:00
b1d6ec098d Merge pull request 'cultmesh-ts: end a RUDP session after 8 consecutive unreadable frames or authority-refused deletes' (#44) from libby/ts-refused-frames-cap into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-08 09:52:26 +00:00
Libby
23c7ce3738 cultmesh-ts: count and answer a frame that decodes but is no message the parser takes
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 6m9s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 1m17s
A map with an unknown schemaVersion was rethrown: unanswered, uncounted toward the
session cap. It is now answered message_invalid, reported, and counted like any other
unreadable frame, so the cap holds for it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 09:52:22 +00:00
51d2d0cf46 Merge pull request 'GameCult.Networking: answer an undecodable RUDP frame message_invalid and end the session at 8 in a row' (#46) from libby/cs-refused-frames-cap into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-08 09:47:22 +00:00
Libby
89bbb277bd GameCult.Networking: answer an undecodable RUDP frame message_invalid and end the session at 8 in a row
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 7m10s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 1m20s
Heimdall's parity ruling. The schema server ended a peer's session on the first
frame it could not decode. It now answers message_invalid (field "message") and
keeps the session; RudpCultNetSchemaServer.MaxConsecutiveRefusedFrames (8)
consecutive undecodable frames end it after the last is answered, and any
decoded frame resets the run. Handler faults still end the session at once.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 09:47:08 +00:00
Libby
b4979cb65b cultmesh-py: answer an unreadable RUDP frame and end the session after 8 consecutive refusals
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
Heimdall's parity ruling. A frame that is no msgpack map used to be dropped
silently; it is now answered message_invalid (field "message") and the session
kept. MAX_CONSECUTIVE_REFUSED_FRAMES (8) consecutive unreadable frames, or
consecutive deletes the write authority refuses, end the session after the
capped frame is answered. The authority's decision is unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 09:43:41 +00:00
Libby
621e05d552 cultmesh-ts: end a RUDP session after 8 consecutive unreadable frames or authority-refused deletes
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
Heimdall's parity ruling. The refusal is still answered and the session kept
below the cap; the frame that reaches CULTMESH_RUDP_DOCUMENT_SERVER_MAX_CONSECUTIVE_REFUSED_FRAMES
is answered too, then the session ends. The write-authority decision is unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 09:41:18 +00:00
4828c41646 Merge pull request 'cultmesh-rs: keep the session through an unreadable frame or refused delete; end at 8 in a row' (#43) from libby/keep-session-invalid-cap into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
2026-11-08 09:36:55 +00:00
Libby
a0d9ed0cc5 cultmesh-rs: keep the session through an unreadable frame or refused delete; end it at 8 in a row
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 3m28s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 56s
Heimdall's parity ruling: the refusal is answered on the live session, and
MAX_CONSECUTIVE_REFUSED_FRAMES (8) consecutive undecodable frames (reset by
any decoded frame) or refused deletes (reset by any decoded non-delete) end
it. The sink gate is unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 09:36:41 +00:00
fb705cbc1d Merge pull request 'cultmesh-rs: RUDP document server refuses unreadable frames and deletes with typed errors' (#42) from libby/rs-rudp-typed-refusals into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-08 09:33:41 +00:00
Libby
34be64789e cultmesh-rs: RUDP document server refuses unreadable frames and deletes with typed errors
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 5m40s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 32s
An undecodable schema frame was dropped and a DocumentDelete ignored, both
silently. Now the first is refused message_invalid (details.field "message")
and the second write_not_authorized naming the record, each with a fixed
reason, ending the peer's session as the other refusals do. The sink gate is
unchanged. Both bytes are pinned against the shared error-vectors.json.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 09:33:27 +00:00
a95b43d48f Merge pull request 'cultnet error contract: one message_invalid code replaces malformed_document_put/delete' (#41) from libby/error-message-invalid into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Failing after 4m36s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Successful in 21s
2026-11-08 09:17:52 +00:00
Libby
a558f3ad3b cultmesh-py: pin details.field on the message_invalid refusals of a malformed or unstorable put
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Failing after 4m43s
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Successful in 51s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 09:14:12 +00:00
Libby
8f71bff69c cultnet error contract: one message_invalid code replaces malformed_document_put/delete
Q6-A of docs/cultnet-error-contract-cut.md. message_invalid joins the closed
cultnet.error.v0 code enum (both schema copies), with details.field naming the
request field at fault ("message" when the whole frame cannot be read).
C# ForMessageInvalid, Rust CultNetErrorCode::MessageInvalid, TS
cultNetMessageInvalidError and Python message_invalid_error all write the same
captured bytes (error-vectors.json plus the C# and Rust cases). cultmesh-py and
cultmesh-ts refuse malformed puts and deletes with it; the C# subscription
servers use it for the missing subscriptionId.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-08 09:14:12 +00:00
fd119ea36c Merge pull request 'cultmesh-ts: write authority decide timeout counts as a typed refusal' (#40) from libby/ts-decide-timeout into main
Some checks failed
CultMesh QUIC native / Build win32-x64 (push) Has been skipped
CultMesh QUIC native / Build linux-x64 (push) Has been skipped
CultMesh QUIC native / Release native binaries (push) Has been skipped
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (push) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (push) Has been cancelled
2026-11-08 09:13:11 +00:00
Libby
86faf53f86 tools/sprite-silhouette: README sample uses the real void/neutral size
Some checks failed
CultMesh QUIC native / Build win32-x64 (pull_request) Has been skipped
CultMesh QUIC native / Build linux-x64 (pull_request) Has been skipped
CultMesh QUIC native / Release native binaries (pull_request) Has been skipped
CultMesh portability and performance / ubuntu-latest one-command multiplayer onboarding (pull_request) Has been cancelled
CultMesh portability and performance / windows-latest one-command multiplayer onboarding (pull_request) Has been cancelled
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled
CultMesh portability and performance / windows-latest reactive semantics and leases (pull_request) Has been cancelled
CultNet cross-runtime interop / C# / TypeScript / Rust / Python / Kotlin (pull_request) Has been cancelled
CultMesh QUIC native / Verify committed binaries match SHA256SUMS (pull_request) Has been cancelled
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-05 10:50:08 +00:00
Libby
62c734f5b2 tools/sprite-silhouette: sprite alpha silhouette to a CSS shape-outside polygon
Zero-dependency Node tool: PNG alpha, threshold 0.35, outer contour of the largest
8-connected shape, Douglas-Peucker to a pixel tolerance, percent coordinates keyed by
cast-art pose id. Ten tests, including every pose in the shared catalogue.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-05 10:49:52 +00:00
266 changed files with 53146 additions and 1243 deletions

View file

@ -1,50 +1,121 @@
name: CultMesh portability and performance name: CultMesh portability and performance
# The forge runs one job at a time and this workflow holds the slot 14 to 30 minutes, so it runs once per change:
# on the pull request (never again on the push that merges it; a break that merges from a stale base waits for the
# schedule), skipped for design-note-only changes, and on a
# daily schedule so main is still exercised when nobody opens a pull request.
on: on:
push:
branches:
- main
- "codex/**"
pull_request: pull_request:
paths-ignore:
# Not `**/*.md` or `docs/**`: docs/start/README.md is read by the first-call tests of every runtime.
- "docs/*.md"
- "research/**"
- ".voidbot/**"
schedule:
- cron: "17 3 * * *"
workflow_dispatch: workflow_dispatch:
concurrency:
# A newer push or PR update replaces the older run on the same ref. Tag pushes and
# workflow_dispatch runs are never cancelled: a tag run carries a release.
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
jobs: jobs:
reactive-and-resource-lifetime: reactive-and-resource-lifetime:
name: ${{ matrix.os }} reactive semantics and leases name: ${{ matrix.os }} reactive semantics and leases
runs-on: ${{ matrix.os }} runs-on: ${{ matrix.os }}
timeout-minutes: 30 # The job dies at the lower of this value and the forge runner's own cap, which Eureka is raising from 30 to 60 minutes
# (run 1378 carried 60 and was still cancelled at 30:02); the two must both say 60 before a full run can finish.
timeout-minutes: 60
strategy: strategy:
fail-fast: false fail-fast: false
matrix: matrix:
# The forge runner is Ubuntu-only: Windows legs run on workflow_dispatch (GitHub or a Windows runner), Ubuntu gates push and PR. # The forge runner (label `ubuntu-latest`) is Linux-only and no Windows runner exists: the Windows leg runs on workflow_dispatch only (GitHub or a Windows runner), the ubuntu-latest leg gates push and PR. Windows push/PR triggers return in the PR that registers a Windows runner.
os: ${{ fromJSON(github.event_name == 'workflow_dispatch' && '["windows-latest", "ubuntu-latest"]' || '["ubuntu-latest"]') }} os: ${{ fromJSON(github.event_name == 'workflow_dispatch' && '["windows-latest", "ubuntu-latest"]' || '["ubuntu-latest"]') }}
steps: steps:
- name: Check out CultLib - name: Check out CultLib
uses: actions/checkout@v4 uses: actions/checkout@v4
with:
# The declared-dependency-range test builds the admitted releases from their tags.
fetch-depth: 0
fetch-tags: true
- name: Set up .NET - name: Set up .NET
uses: actions/setup-dotnet@v4 uses: actions/setup-dotnet@v4
with: with:
dotnet-version: 10.0.x dotnet-version: 10.0.x
# The forge runner image (catthehacker/ubuntu:act-latest) carries Node 24 and Python 3.12 on PATH, and these two
# setup actions downloaded them again from github.com anyway (about 2.7 and 2.8 minutes, run 639) because the
# image's tool cache is not the one they read. They run only where the image does not carry the tools (the
# workflow_dispatch Windows leg); the check below fails a Linux image that stops carrying them.
- name: Set up Node.js - name: Set up Node.js
if: runner.os != 'Linux'
uses: actions/setup-node@v4 uses: actions/setup-node@v4
with: with:
node-version: 24 node-version: 24
cache: npm cache: npm
- name: Set up Python - name: Set up Python
if: runner.os != 'Linux'
uses: actions/setup-python@v5 uses: actions/setup-python@v5
with: with:
python-version: "3.12" python-version: "3.12"
- name: Check the image carries Node 24 and Python 3.12
if: runner.os == 'Linux'
shell: bash
run: |
set -euo pipefail
node --version | grep -E '^v24\.'
python --version | grep -E '^Python 3\.12\.'
# The image's Python is distro-managed (PEP 668): pip refuses to install into it, so the Linux leg installs the
# CultLib packages into a venv under RUNNER_TEMP that later steps find first on PATH.
- name: Create the Python environment
if: runner.os == 'Linux'
shell: bash
run: |
set -euo pipefail
python -m venv "$RUNNER_TEMP/venv"
echo "$RUNNER_TEMP/venv/bin" >> "$GITHUB_PATH"
- name: Install TypeScript workspace dependencies - name: Install TypeScript workspace dependencies
run: npm install --no-package-lock run: npm install --no-package-lock
- name: Install Python CultLib packages - name: Install Python CultLib packages
run: python -m pip install -e packages/cultcache-py -e packages/cultnet-py -e packages/cultmesh-py run: python -m pip install -e packages/cultcache-py -e packages/cultnet-py -e packages/cultmesh-py
# The forge runner images carry no cargo either (the cross-runtime store tests build the Rust interop example); GitHub's hosted images do.
- name: Install Rust toolchain
if: runner.os == 'Linux'
shell: bash
run: |
set -euo pipefail
if ! command -v cargo >/dev/null && [ ! -x "$HOME/.cargo/bin/cargo" ]; then
curl -fsSL https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain stable
fi
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
# The forge runner images carry no pwsh; GitHub's Windows and hosted images do.
- name: Install PowerShell
if: runner.os == 'Linux'
shell: bash
run: |
set -euo pipefail
if command -v pwsh >/dev/null; then pwsh -Version; exit 0; fi
SUDO=""; [ "$(id -u)" -eq 0 ] || SUDO="sudo"
PWSH_VERSION=7.4.6
curl -fsSL "https://github.com/PowerShell/PowerShell/releases/download/v${PWSH_VERSION}/powershell-${PWSH_VERSION}-linux-x64.tar.gz" -o "${RUNNER_TEMP:-/tmp}/powershell.tar.gz"
$SUDO mkdir -p /opt/microsoft/powershell/7
$SUDO tar -xzf "${RUNNER_TEMP:-/tmp}/powershell.tar.gz" -C /opt/microsoft/powershell/7
$SUDO chmod +x /opt/microsoft/powershell/7/pwsh
$SUDO ln -sf /opt/microsoft/powershell/7/pwsh /usr/local/bin/pwsh
echo "DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1" >> "$GITHUB_ENV"
DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1 pwsh -Version
- name: Verify explicit reactive scheduling across runtimes - name: Verify explicit reactive scheduling across runtimes
shell: pwsh shell: pwsh
run: ./scripts/verify-reactive-document-scaling.ps1 run: ./scripts/verify-reactive-document-scaling.ps1
@ -60,8 +131,8 @@ jobs:
--logger "console;verbosity=minimal" --logger "console;verbosity=minimal"
# The TypeScript side of the CultMesh authority rules. This job is the # The TypeScript side of the CultMesh authority rules. This job is the
# only one that runs on both Windows and Linux for main, codex/** and # only one that runs on Linux for main, codex/** and pull requests (and on
# pull requests: cultnet-interop.yml is windows-latest and main-only, so # Windows too under workflow_dispatch): cultnet-interop.yml is windows-latest and dispatch-only, so
# until these steps existed the shared verifier's tests never ran on # until these steps existed the shared verifier's tests never ran on
# Linux at all, and the browser client's never ran in CI on any OS. # Linux at all, and the browser client's never ran in CI on any OS.
# packages/cultnet-ts verifies both vector files (content.test.ts judges both # packages/cultnet-ts verifies both vector files (content.test.ts judges both
@ -77,6 +148,26 @@ jobs:
- name: Verify the CultCache document owner with the TypeScript cache - name: Verify the CultCache document owner with the TypeScript cache
run: npm run test --workspace packages/cultcache-ts run: npm run test --workspace packages/cultcache-ts
# The Python cache's unit suite ran only in cultnet-interop.yml (windows-latest, dispatch-only), so on Linux
# it ran nowhere. It sits after the TypeScript cache's test so the built cultcache-ts store holder exists for the
# cross-runtime store-lock test, which skips without it.
- name: Verify the CultCache document owner with the Python cache
run: python -m unittest discover -s packages/cultcache-py/tests
# cargo test for packages/cultcache-rs ran in no workflow: the Rust cache's own unit and doc tests were unwatched
# (only the interop example built, inside the C# store tests). The target directory is under RUNNER_TEMP, away from the checkout.
- name: Verify the CultCache document owner with the Rust cache
shell: bash
run: CARGO_TARGET_DIR="$RUNNER_TEMP/cultcache-rs-target" cargo test --manifest-path packages/cultcache-rs/Cargo.toml
# tests/GameCult.Caching.Tests, the C# reference cache's own suite (about 1,260 tests, under a minute), ran in no
# forge workflow; only the Mesh tests above exercised the reference.
# The forge runner is globalization-invariant, so ADeterministicIdIsTheSameUnderEveryCulture... cannot construct a
# culture and throws CultureNotFoundException; it is excluded here and still runs wherever ICU is present
# (variants:follow_up:culture-test-invariant-image).
- name: Verify the CultCache document owner with the C# reference
run: dotnet test tests/GameCult.Caching.Tests/GameCult.Caching.Tests.csproj --filter "FullyQualifiedName!~ADeterministicIdIsTheSameUnderEveryCulture" --logger "console;verbosity=minimal"
# Packs the four TypeScript packages and installs them into a clean consumer, so a package that builds but does # Packs the four TypeScript packages and installs them into a clean consumer, so a package that builds but does
# not install or load fails here. # not install or load fails here.
- name: Verify the packed TypeScript packages install and load from a clean consumer - name: Verify the packed TypeScript packages install and load from a clean consumer

View file

@ -1,7 +1,7 @@
name: CultNet cross-runtime interop name: CultNet cross-runtime interop
# workflow_dispatch only: this job is Windows-only (pwsh Kotlin build, setup steps), and the forge # workflow_dispatch only: this job is Windows-only (pwsh Kotlin build, setup steps), and the forge
# runner is Ubuntu-only, so a push or pull_request trigger would queue a job that can never run. # runner is Ubuntu-only (no Windows runner exists; push and pull_request return in the PR that registers one), so a push or pull_request trigger would queue a job that can never run.
# Run it from GitHub or any Windows runner. # Run it from GitHub or any Windows runner.
on: on:
workflow_dispatch: workflow_dispatch:

View file

@ -9,15 +9,29 @@ name: CultMesh QUIC native
# from the pinned debian:13 container below, never from a local or Yggdrasil # from the pinned debian:13 container below, never from a local or Yggdrasil
# build. # build.
# Only a change to the committed binaries, their manifest or this workflow can change what verify-committed checks,
# so a branch push or pull request runs it for those paths alone; a tag always runs it.
on: on:
push: push:
branches: branches:
- main - main
paths:
- "packages/cultmesh-ts/native/**"
- ".github/workflows/quic-native.yml"
tags: tags:
- "cultmesh-quic-native-v*" - "cultmesh-quic-native-v*"
pull_request: pull_request:
paths:
- "packages/cultmesh-ts/native/**"
- ".github/workflows/quic-native.yml"
workflow_dispatch: workflow_dispatch:
concurrency:
# A newer push or PR update replaces the older run on the same ref. Tag pushes and
# workflow_dispatch runs are never cancelled: a tag run carries a release.
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions: permissions:
contents: read contents: read
@ -49,11 +63,12 @@ jobs:
# workflow_dispatch and a cultmesh-quic-native-v* tag run the two builds. # workflow_dispatch and a cultmesh-quic-native-v* tag run the two builds.
# A dry (workflow_dispatch) run uploads artifacts and creates no release. # A dry (workflow_dispatch) run uploads artifacts and creates no release.
# No Windows runner is registered on the forge, so this job is workflow_dispatch only
# (run it on GitHub or a Windows runner). The tag trigger returns in the PR that registers one.
# Until then a tag push skips the release job, which needs this build.
build-win32-x64: build-win32-x64:
name: Build win32-x64 name: Build win32-x64
if: >- if: github.event_name == 'workflow_dispatch'
github.event_name == 'workflow_dispatch' ||
(github.event_name == 'push' && startsWith(github.ref, 'refs/tags/cultmesh-quic-native-v'))
runs-on: windows-latest runs-on: windows-latest
timeout-minutes: 20 timeout-minutes: 20
steps: steps:
@ -92,8 +107,8 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
# Q6 A: this pinned Debian 13 container is the only permitted Linux # Q6 A: this pinned Debian 13 container is the only permitted Linux
# release builder. The digest is the pin; the `debian:13` tag is only a # release builder. The digest is the pin; the `debian:13` tag is only a
# label. The artifact links against this image's glibc (2.41); a bare # label. The artifact links against this image's glibc (2.41);
# ubuntu-latest runner carries glibc 2.39, which is not the deploy host's # a bare runner's OS image has its own glibc (a GitHub ubuntu-latest image carried 2.39), which is not the deploy host's
# OS (docs/typescript-quic-realtime-cut.md:2192-2197). # OS (docs/typescript-quic-realtime-cut.md:2192-2197).
container: container:
image: debian:13@sha256:f324c7ff54321e8d9c588493a20244965938ce0aa50bbd1022d38010e9ffc4b1 image: debian:13@sha256:f324c7ff54321e8d9c588493a20244965938ce0aa50bbd1022d38010e9ffc4b1

View file

@ -114,7 +114,10 @@ copies), the C# `CultNetErrorMessage` factory, the Rust `CultNetErrorCode`, and
`selection_invalid`, `cursor_stale`, `cursor_invalid`, `reference_outside_target`, `variant_unsupported`, `selection_invalid`, `cursor_stale`, `cursor_invalid`, `reference_outside_target`, `variant_unsupported`,
`unowned_schema` (a write, or a remote put or delete, for a schema and key no shard owns), `unowned_schema` (a write, or a remote put or delete, for a schema and key no shard owns),
`write_not_authorized` (a peer's put or delete its server's remote write authority did not grant) and `write_not_authorized` (a peer's put or delete its server's remote write authority did not grant) and
`unsupported_schema_version` (a message whose `schemaVersion` the peer does not speak; `details` names `schemaVersion` and the offered value). `unsupported_schema_version` (a message whose `schemaVersion` the peer does not speak; `details` names `schemaVersion` and the offered value) and
`limit_reached` (a receipt-capable server's per-session remembered-write table is full, so a new write is refused; `details` all null) and
`message_invalid` (a request that lacks a required field or carries an ill-typed one: a document-put or delete, or a subscription without an id; `details.field` names the request field, or `message` when the whole frame cannot be read).
A malformed put or delete is `message_invalid`, not a per-operation code: the peer knows which operation it sent.
## Live Invariants ## Live Invariants
@ -137,6 +140,15 @@ copies), the C# `CultNetErrorMessage` factory, the Rust `CultNetErrorCode`, and
- A shard snapshot represents a log sequence. Applying it replaces the local - A shard snapshot represents a log sequence. Applying it replaces the local
shard view and advances the replica cursor to that sequence. shard view and advances the replica cursor to that sequence.
- Realtime subscriptions publish domain changes, not storage envelopes. - Realtime subscriptions publish domain changes, not storage envelopes.
- A subscription's `database_change_raw` messages may carry `commitId` and `commitCount`: the changes of one cache
commit that this subscription delivers (`commitCount` of them, at least two). Both fields are optional and additive; a
peer that does not know them ignores them and applies each change alone. A replica that knows them applies the
commit's changes as one commit, so the replica judges the result (a swap of two unique-index values) and not the state
between records. A replica whose cache refuses a commit is faulted, never silent. `commitId` is opaque and valid only
within the subscription. The C# subscription server and database server tag commits; other runtimes adopt next.
- The C# writer sets every key of a `database_change_raw` message, nil where unset: a removed change carries `document`
nil, an added or updated one `recordKey` and `schemaId` nil. The schema accepts those nils; an added or updated change
still needs a document, and a removed one a `recordKey` and `schemaId`.
- Client-owned input documents may be predicted locally only inside explicit - Client-owned input documents may be predicted locally only inside explicit
`CultNetClientAuthorityScope` declarations. `CultNetClientAuthorityScope` declarations.
- Predicted input state is not authoritative shared state. The shard log still - Predicted input state is not authoritative shared state. The shard log still

View file

@ -10,15 +10,29 @@
"messageId": { "type": "string", "minLength": 1 }, "messageId": { "type": "string", "minLength": 1 },
"subscriptionId": { "type": "string", "minLength": 1 }, "subscriptionId": { "type": "string", "minLength": 1 },
"changeKind": { "enum": ["added", "updated", "removed"] }, "changeKind": { "enum": ["added", "updated", "removed"] },
"document": { "$ref": "./cultnet.raw-document-record.schema.json" }, "document": {
"recordKey": { "type": "string", "minLength": 1 }, "oneOf": [{ "type": "null" }, { "$ref": "./cultnet.raw-document-record.schema.json" }]
"schemaId": { "type": "string", "minLength": 1 } },
"recordKey": { "type": ["string", "null"], "minLength": 1 },
"schemaId": { "type": ["string", "null"], "minLength": 1 },
"commitId": { "type": ["string", "null"], "minLength": 1 },
"commitCount": { "type": ["integer", "null"], "minimum": 2 }
}, },
"allOf": [ "oneOf": [
{ {
"if": { "properties": { "changeKind": { "const": "removed" } } }, "properties": {
"then": { "required": ["recordKey", "schemaId"] }, "changeKind": { "const": "removed" },
"else": { "required": ["document"] } "recordKey": { "type": "string" },
"schemaId": { "type": "string" }
},
"required": ["recordKey", "schemaId"]
},
{
"properties": {
"changeKind": { "enum": ["added", "updated"] },
"document": { "$ref": "./cultnet.raw-document-record.schema.json" }
},
"required": ["document"]
} }
] ]
} }

View file

@ -28,7 +28,7 @@
"code": { "code": {
"description": "Machine-readable refusal code (R-N). Additive: a v0 peer that does not know this field ignores it. The set is a closed enum: Rust and TypeScript refuse a message whose code is not listed, so adding a code is one coordinated change across every runtime (this schema, the C# factory, the Rust CultNetErrorCode, and a captured-bytes case for each).", "description": "Machine-readable refusal code (R-N). Additive: a v0 peer that does not know this field ignores it. The set is a closed enum: Rust and TypeScript refuse a message whose code is not listed, so adding a code is one coordinated change across every runtime (this schema, the C# factory, the Rust CultNetErrorCode, and a captured-bytes case for each).",
"type": ["string", "null"], "type": ["string", "null"],
"enum": [null, "selection_invalid", "cursor_stale", "cursor_invalid", "reference_outside_target", "variant_unsupported", "unowned_schema", "write_not_authorized", "unsupported_schema_version"] "enum": [null, "selection_invalid", "cursor_stale", "cursor_invalid", "reference_outside_target", "variant_unsupported", "unowned_schema", "write_not_authorized", "unsupported_schema_version", "message_invalid"]
}, },
"details": { "details": {
"description": "Code-specific structured detail (R-N). Every field is populated only for the code that defines it; the others are null, not omitted.", "description": "Code-specific structured detail (R-N). Every field is populated only for the code that defines it; the others are null, not omitted.",
@ -36,7 +36,7 @@
"additionalProperties": false, "additionalProperties": false,
"required": ["field", "value", "asOf", "current"], "required": ["field", "value", "asOf", "current"],
"properties": { "properties": {
"field": { "description": "selection_invalid: the selection field that failed. variant_unsupported: \"subscriptionId\", when a subscription was ended. unowned_schema: \"schemaId\". write_not_authorized: \"recordKey\". unsupported_schema_version: \"schemaVersion\".", "type": ["string", "null"] }, "field": { "description": "selection_invalid: the selection field that failed. variant_unsupported: \"subscriptionId\", when a subscription was ended. unowned_schema: \"schemaId\". write_not_authorized: \"recordKey\". unsupported_schema_version: \"schemaVersion\". message_invalid: the request field that is missing or ill-typed (\"message\" when the whole frame cannot be read).", "type": ["string", "null"] },
"value": { "description": "selection_invalid: the offending value, when there is one string worth naming. variant_unsupported: the id of the subscription that ended. unowned_schema: the schema id no shard owns. write_not_authorized: the key of the refused write. unsupported_schema_version: the schemaVersion the peer does not speak.", "type": ["string", "null"] }, "value": { "description": "selection_invalid: the offending value, when there is one string worth naming. variant_unsupported: the id of the subscription that ended. unowned_schema: the schema id no shard owns. write_not_authorized: the key of the refused write. unsupported_schema_version: the schemaVersion the peer does not speak.", "type": ["string", "null"] },
"asOf": { "description": "cursor_stale: the cursor's own minted asOf.", "type": ["integer", "null"], "minimum": 0 }, "asOf": { "description": "cursor_stale: the cursor's own minted asOf.", "type": ["integer", "null"], "minimum": 0 },
"current": { "description": "cursor_stale: the answering server's current asOf.", "type": ["integer", "null"], "minimum": 0 } "current": { "description": "cursor_stale: the answering server's current asOf.", "type": ["integer", "null"], "minimum": 0 }

File diff suppressed because one or more lines are too long

View file

@ -0,0 +1,206 @@
# CultNet fork-desk contract v0 (specification)
Status: specification, drafted for Heimdall (Bifrost receiver), Huginn (the second door) and Void (Ratatoskr desk). No runtime implements it yet.
Rule source: `/opt/commons/handoff/fork-desk-answer-binding-20261110.md` (Heimdall; sections 1.1 to 1.5, 1.3a to 1.3d), amended by the Persona briefs of 2026-11-10 (Eureka's fit check). Where this page differs from that file it says so under "Differences from the binding file".
Format source: the fork desk artifact in `/opt/commons/handoff/fork-desk-original/` (ruling `eureka-substrate:ruling:fork-desk-nibu-format`).
Files here: `fork-card.schema.json` (one schema, a `$defs` entry per document), `fork-card-vectors.json`. Conformance: `tests/GameCult.Networking.Tests/GovernanceVectorTests.cs` (C#) and `packages/cultnet-ts/test/governance-vectors.test.ts` (TypeScript), both generated against vectors written by `scripts/gen-governance-vectors.py`. Each suite computes every hash with its own code and replays every scenario through a reference judge written from this page.
All documents are CultNet map-mode: every key written, nil where absent, closed schemas. Field names are snake_case as in the binding file and the mind's records; `schemaVersion` keeps CultNet's spelling (open choice 1). The bytes of every document are pinned in the vectors as `messagePackHex`.
## Documents
| schema id | what | who writes it |
|---|---|---|
| `gamecult.fork.card.v0` | the fork as shown | the publisher (a Mind's card projection) |
| `gamecult.fork.answer_request.v0` | the answer a decider sends | the client; Bifrost forwards it unchanged to Huginn's second door |
| `gamecult.desk_answer.v0` | the stored answer with one tagged outcome | Huginn only |
| `gamecult.fork.answer_result.v0` | what the sender gets back for a stored answer | Bifrost, from the stored `desk_answer` |
| `gamecult.fork.refusal.v0` | a body refused at the door, nothing stored (but `fork_too_large` on a stale answer, which sits over a stored `Refused(stale_fork)`) | Bifrost |
| `gamecult.fork.record.v0` | a fork's answers plus the derived counted answer | a projection of the stored `desk_answer`s |
| `gamecult.fork.ruling_cite.v0` | the citation a ruling carries | the Mind admitting the ruling |
| `gamecult.fork.list_request.v0` | list a Mind's forks (Transport) | the desk |
| `gamecult.fork.list.v0` | a page of entries, counts and a stamp | Bifrost, projected from Huginn |
| `gamecult.fork.read_request.v0` | read one fork (Transport) | the desk |
| `gamecult.fork.entry.v0` | one card with its record | Bifrost, projected from Huginn |
| `gamecult.fork.unavailable.v0` | the outcome is not confirmed (the second door or Huginn did not answer); retry with the same `request_id` | Bifrost |
### card
`{schemaVersion, fork_id, title, context, options[{id, text}], recommendation, depends[], leans, flag, raised_in, raised_at, deciders[], decider_count, status, ruling, ruled_from[], order, options_hash}`.
- `fork_id`: the mind's `question` key, `<campaign>:question:<label>`.
- **Hashed (the shown members):** `title`, `context`, `options` (each `{id, text}`, in shown order), `recommendation` (an option id, **never nil**: the stored `question` requires `recommended`), `depends` (strings, each one of the stored question's `depends` lines verbatim, at most 8, each at most 1000 UTF-8 bytes (the stored `Line`), in shown order, empty when none; the stored kind holds prose lines, so a line may read like a reference such as `campaign:kind:label` but is hashed as the text it is).
- **Not hashed:** `fork_id` (bound beside the hash), `leans` (an option id or nil), `flag`, `raised_in`, `raised_at` (the instant the question was first admitted, an RFC3339 instant string as `answered_at`, or nil when the mind holds no admission time, as for a card projected before the field; the key is always written; it never moves when the question is edited; age is the client's: now minus `raised_at`), `deciders`, `decider_count` (the number of deciders of the call, set beside `deciders`; neither it nor the emptiness of `deciders` enters the hash, so the hash a non-decider recomputes is a decider's), `status` (`open` or `closed`), `ruling` (the ruling id when a ruling closed the fork; nil when the card is open, and nil on a closed card whose question was withdrawn or otherwise resolved with no ruling), `ruled_from` (answer ids a ruling has been admitted from; non-empty only on a closed card), `order`, and `options_hash` itself.
- **`deciders` is read-only data.** The list is set outside the Mind, through Bifrost's admin path, and **Bifrost sends it per call**: Huginn keeps no copy (M5). The card projection copies the list the call carried. **The per-call list holds one to 64 ids** (Huginn's bound; vectors `deciders_at_the_cap_of_64`, `deciders_65`); the per-call list has no schema here, so the bound is stated for the cut that builds it. **`decider_count` is the size of that list, an integer 1 to 64, always written** (vectors `decider_count_missing`, `_zero`, `_65`, `_string`, `_null`). A fork with no decider is still refused, now as `decider_count` 0. **`deciders` holds that list for a decider and is empty for a reader who is not one: empty means "the ids are withheld from this reader", never "no deciders"** (vector `withheld_deciders`). Rule R-card: when `deciders` is non-empty it holds exactly `decider_count` ids; a visible list is whole (vectors `decider_count_fewer_than_the_ids`, `decider_count_more_than_the_ids_shown`). A question's own `audience` may only narrow it. A Mind never writes it, so a Mind cannot add itself and answer its own fork.
- Rules a schema cannot say, each with a negative vector: a `depends` line is at most 1000 UTF-8 bytes (the schema bounds 8 lines and 1000 characters; bytes need the rule); option ids are unique; `order` is unique among a Mind's cards (a list with two cards on one `order` is refused, naming both fork ids); `recommendation` and `leans` name an option; deciders are unique; an open card names no `ruling` and its `ruled_from` is empty (a ruling that answers a fork closes it); `ruled_from` is non-empty only when `ruling` is set; a closed card may name no ruling (the question was withdrawn, superseded or deferred: step 6 reads `status`, not the ruling); `options_hash` equals the hash of the shown members; R-card (above); R-answer-result: `answer_result.answer.answerer` is never nil, being the sender's own stamped account (vector `answer_result_with_a_nil_answerer`); R-record, R-entry and R-list ("Reader views" below).
**Reader views.** A card and its record are shown in one of two views. The **decider view** carries the ids: `deciders` whole, every `answerer` named, and the counts equal to the ids. The **non-decider view** withholds them: `deciders` empty, every `answerer` nil, the counts kept (`decider_count`; `answerer_count` on the record). Who is a decider is Huginn's decision, not this contract's: a reader is a decider exactly when the call's subject is byte-equal to one of the call's deciders (`eureka-substrate:ruling:desk-read-visibility-withheld-ids`); Huginn's door withholds, Bifrost cannot (the card is opaque to it). There is no viewer flag on any document: the view is read off whether `deciders` is empty. This contract carries both views and refuses a view that leaks: R-record: if every `answerer` is named, `answerer_count` is the number of distinct ids; if all are nil and there is at least one answer, `answerer_count` is between 1 and the number of answers; with no answers it is 0 (vectors `answerer_count_disagrees_with_the_ids`, `answerer_count_counts_answers_not_people`, `withheld_count_over_the_answers`, `withheld_count_zero_over_answers`, `withheld_count_over_no_answers`); R-entry, the leak rule: an entry with `deciders` non-empty names every answerer, an entry with `deciders` empty names none (vectors `decider_card_with_withheld_answers`, `withheld_card_with_named_answers`, `withheld_card_with_a_named_answerer_among_nils`); R-list: all entries of one page agree on the view (vector `a_page_mixes_views`). The view does not move the page: `non_decider_page` has the counts, order, cursor and stamp of the decider page it twins, and the vectors' no-leak check finds no id string in a non-decider vector's bytes. `options_hash` is the same in both views, and the bound-answer and `stale_fork` machinery is untouched.
### Options hash (canonical bytes)
`options_hash` = lowercase hex SHA-256 of the UTF-8 bytes of the following, byte for byte:
1. Build an object with exactly the five members `title`, `context`, `options`, `recommendation`, `depends`. Each option is an object with exactly `id` and `text`.
2. **NFC-normalise every string, object keys included.** No trimming, no newline rewriting, no case folding. A string with a lone surrogate is not a card (it has no UTF-8). Normalisation stability makes NFC stable across Unicode versions for already-assigned code points; a runtime's own normaliser is trusted and the vectors include a decomposed pair and astral code points (open choice 2).
3. Serialise as RFC 8785 (JCS): no whitespace; object keys sorted by UTF-16 code unit, which for these ASCII keys is `context, depends, options, recommendation, title` and, inside an option, `id, text`; array order is as shown (option order and `depends` order are part of the hash).
4. Strings are written in double quotes; only `"` and `\` are backslash-escaped, U+0008, U+0009, U+000A, U+000C, U+000D as `\b \t \n \f \r`, every other code point below U+0020 as `\u00xx` in lowercase hex; everything else is written raw as UTF-8 (U+007F, U+2028, `<`, `&` and astral code points included).
5. **Numbers and nulls:** the member set contains none. There is no number to canonicalise (a number anywhere in the five members is a defect) and `recommendation` is never nil, so no nil is hashed.
`fork_id` is not in the hash and is bound beside it in its own field, so one hash cannot be replayed under another fork. The publisher computes the hash and stores it in the card; the client recomputes it from what it rendered and refuses to send on a mismatch; the receiver recomputes from its stored fork, never from the client.
Vectors `optionsHash`: the seven of the binding file (`heimdall_A..G`), computed by the Python generator, by the C# suite and by the TypeScript suite independently, plus text, title, context and `depends`-order mutations, a case showing the unshown members do not move the hash, escapes and markup, and astral ids. Each suite also mutates every shown member, an option id included, and asserts the hash changes.
### answer_request
`{schemaVersion, fork_id, options_hash, answer, retracted, retracts, answered_at, request_id}`. **`request_id`** is set by the client, a lowercase D-format UUID, one per intended answer, kept across retries; the first stored answer that carries it binds it to the stamped `answerer`. "Same body" means every field equal, `answered_at` included, so a client resends identical bytes. A **retry with the same `request_id` and the same body returns the stored outcome unchanged** (a `Refused` one included; a replayed `stale_fork` carries the **current** card, because a stored `desk_answer` has no card field and no storage is added for one) and **never adds a Revision**; the same `request_id` with a different body is `request_id_conflict` (409, a refusal, nothing stored). The same id from another account is another request, judged on its own: Y reusing X's `request_id` with a body identical to X's gets **Y's own new answer, with its own answer id, stamped Y**; X's stored outcome (a `Refused` one included) is never returned in its place, and Y's request neither conflicts with it nor replays it. An `of` that names the counted answer is allowed, because answer ids are not secret between deciders (vectors V14: an off-list X's `Refused`, then a decider Y's identical body is its own `Counted`; X's `Counted`, then Y's identical body is its own `Revision` whose `of` names X's answer, and X's later retry still returns X's answer). Both suites assert it on every scenario: a response's `answerer` is the step's account, and an id first bound by another account never yields that account's answer id or document. **`retracted` is part of the body**: the same `request_id` with `retracted: true` and then `false` (or the reverse) and every other field equal is `request_id_conflict`, for the illegal forms as for any other (vectors `R1_retracted_true_then_false_...`, `R1_retracted_false_then_true_...`). Each illegal form's own retry, byte for byte, replays its stored `bad_retraction` (vectors `R1_a_retraction_with_an_answer_...`, `R1_a_retraction_with_no_target_...`, `R1_a_plain_answer_with_a_target_...`). A body refused at step 1 or 2 binds nothing, so its id is free to reuse; so a target-less retraction resent with `retracted: false` (no answer, no retraction) is `body_invalid` at step 2, not a conflict. The lookup comes after steps 1 and 2 and before the fork is looked up, so a retry gets its stored outcome even if the card has since changed. `answer` is `{choice, note}` (at least one non-nil) or nil on a retraction. `choice` is an option id; `note` is the person's words, verbatim, 1 to 4000 **UTF-8 bytes**, not blank. A schema counts characters, not bytes, so its `maxLength: 4000` (code points) is only the looser bound and both suites enforce the byte limit as a rule: 4000 ASCII bytes pass; 1000 astral characters (4000 bytes) pass and 1001 (4004) fail; `e` plus a combining acute is 3 bytes, so 1333 of them and an `x` pass and 1334 fail; 1999 two-byte characters plus three ASCII (4001 bytes) fail. A note over the limit is `body_invalid`. `answered_at` is the client's claim. **A request never carries `answerer` or `authority`**: both are stamped by Huginn (`answerer` from the verified subject the receiver passes; `authority` from the door's configuration). A body that names either key at any depth is refused whole, never stripped (`body_forbidden_field`); any other unknown key is `body_unknown_field`. The request type Bifrost sends over the second door is this document; it has no field for either key, so a client cannot supply them by construction, and Bifrost's scan is the first line, not the only one. How Huginn receives the verified subject is the door's authenticated context, not a body field (open choice 3). In map-mode `retracted` is always written (the binding file's optional `retracted` becomes `false`).
### desk_answer (the stored kind)
`{schemaVersion, answer_id, fork_id, outcome, of, retracts, code, options_hash, choice, note, answered_at, received_at, claim_in_future, answerer, authority, request_id}`, key `<campaign>:desk_answer:<question-label>.n<seq>`; `answer_id` is `n<seq>`, assigned by Huginn per question, and **an answer is named by its answer id** everywhere (`retracts`, `of`, `ruled_from`).
The stamped `authority` field of a stored answer is an **attribution**: the role the verified account holds (`Operator`, `Steward`, `Member`), copied from Bifrost's state. A stored answer confers no authority. Authority exists only where a ruling cites a counted answer ("Operator authority" below); the field keeps the binding file's spelling (open choice 8). One tagged `outcome`, each variant carrying only its own link:
- `Counted`: the answer that counts. `of`, `retracts`, `code` nil.
- `Revision`: a later passing answer, kept, not counted while the first stands. `of` names the counted answer at the time.
- `Retraction`: the stored form of `retracted: true` plus `retracts`. `retracts` names the answer retracted; `choice` and `note` nil. The illegal pair (a retraction with no target, or a target with no retraction) cannot be written.
- `Refused`: any refusal from steps 4 to 7 or R1 and R2, with its `code`, stored too. `choice` and `note` keep what the body carried, and so does `retracts`: a refused body that named a target (a retraction of any refusal code, or a plain answer carrying one) projects it as `n<seq>`, never nil; a refused body that named none has `retracts` nil. `of` is nil.
`received_at` is Huginn's admission time; "first" means first by admission order (the `n<seq>` ordinal), never by `answered_at`. `claim_in_future` is true when `answered_at` is more than five minutes after `received_at`: accepted but flagged. It is a function of the two stored values, so the writer compares against the `received_at` it stores, at the precision it stores it (a flag decided against an unrounded clock and a stored second-truncated `received_at` disagrees with the document it sits in). **Not stored, and no answer id:** a body refused at step 1 or 2 (Bifrost; it never reaches Huginn), and a body naming an unknown fork (step 3; Huginn answers it from its stored forks, but with no question there is nothing to key a record under). Each gets a `refusal` and is dropped. A malformed body is therefore never stored, and neither is an unknown fork. Every refusal from step 4 on is stored. **`unavailable`** (503) is not a refusal and not in the refusal set: Bifrost answers it when the second door or Huginn did not answer after the auth steps and the body shape passed (a bad body is still refused first). **It says the outcome is NOT CONFIRMED, never that nothing was stored**: a timeout after the request reached Huginn may have stored it. It carries no answer id. The desk retries the same request with the same `request_id` and learns the outcome: the stored one if Huginn had it, a fresh one if not. It covers **calls 1 and 2 as well as the answer POST** when Huginn cannot be reached (Transport; a read has no outcome to confirm, so a retry is plain). `retryAfterSeconds` is nil when Bifrost has no hint.
**The record.** `record` is `{schemaVersion, fork_id, options_hash, counted_answer_id, answerer_count, answers[]}`. `answerer_count` is an integer of at least 0: the number of distinct answerer ids over every stored answer of the fork, refusals included (a refusal stores the stamped answerer too), and it is the same number in both views (vectors `answerer_count_missing`, `_negative`, `_null`). `answers[].answerer` is a string or nil (nil only in the non-decider view).
**The mind's record and this document.** Huginn keeps the answer as its own record kind (`epiphany.pipeline.desk_answer.v2`, key `<campaign>:desk_answer:<question-label>.n<seq>`, the outcome as one externally tagged value, the fork as `fork`). The second door projects it to this document and nothing else crosses: `answer_id` is the key's `n<seq>` part, `fork_id` is the stored `fork`, the tagged outcome flattens into `outcome`, `of`, `retracts` and `code`, and the six refusal codes are spelt as in the schema (snake_case; the record's enum spells them in the mind's own PascalCase). On the wire an answer is named by its `n<seq>` alone, always inside the fork whose `fork_id` the document carries; a runtime that holds the full record key converts at the door, in one place. Huginn's stored `desk_answer` keeps a non-nil `answerer`; nil exists only in a record projected to a non-decider. `body_sha256` and any other digest Huginn keeps to decide "same body" stay in the record: Huginn alone decides a replay or a `request_id_conflict` and Bifrost never compares bodies, so no digest is a field of this document. A digest is a valid way to decide it only if it covers `fork_id`, `options_hash`, `answer.choice`, `answer.note`, `retracted`, `retracts` and `answered_at` (every field of the request but `request_id`), is computed by Huginn and never read from a client.
**Huginn's digest, pinned in the vectors (`bodyDigests`; informative, not a wire document).** SHA-256 of one compact JSON object as `serde_json::to_string` writes it (UTF-8, no spaces), lowercase hex, keys in this order, `null` for a missing value, strings verbatim: `fork`, `options_hash`, `answered_at`, `choice`, `note`, `retracts` (the **full stored key** `<campaign>:desk_answer:<label>.n<seq>`: the door converts the wire's `n<seq>` before the digest), then, **only for the three illegal forms** (R1: a retraction carrying an answer, a retraction naming no target, a plain answer carrying a target), `illegal_retraction: true` and `retracted: true|false`. `request_id` is not in it. A plain answer and a legal retraction carry neither trailing key, so their digests are the ones Huginn main pins (`8a4d5b2e`, `9f807538`) and do not move. `9f807538` is pinned for a target no door produces (`Q1.1`); the golden for the real legal retraction is the door's form, the wire's `n1` converted to `Q1.n1` (`84576840`, vector `legal_retraction_door_form`, also main-valid). Source: Eureka's `desk-body-digest-order-20261110.md`, computed with `sha256sum`; the generator, the C# suite and the TypeScript suite recompute every digest from the input fields. **Status:** the three `retracted` goldens (`935cab1a`, `82b24ccc`, `34ef86ed`, and `79eee623` for form (i) with a null target) are the form of `cut-desk-digest-retracted.r2`, merged as Huginn #30. Eureka reports that these four and the door-form legal retraction (`84576840`) reproduce byte for byte on Huginn main `f0ca504` (the #30 fix); that run is hers, reported to us, and was not rerun here. Before #30, Huginn main (`7ea11b4`) wrote `illegal_retraction` alone, so forms (i) and (iii) collided there (`69074cc6`, pinned as `illegal_i_and_iii_alike_on_main`). The wire contract does not wait on it: the conflict scenarios are pinned in `scenarios`.
### Counting rules (in order; the first failure refuses with that code)
| step | rule | refusal | stored |
|---|---|---|---|
| 1 | verified session or bearer | `unauthenticated` (401) | no |
| 2 | closed body; no `answerer`/`authority` at any depth; type-correct; not (no `answer` and not a retraction) | `body_forbidden_field`, `body_unknown_field`, `body_invalid` (400) | no |
| 3 | `fork_id` is a stored fork | `fork_unknown` (404) | no |
| 4 | `options_hash` equals the stored fork's hash | `stale_fork` (409) | **yes**; the result carries the current card |
| 5 | the stamped `answerer` is on `deciders` | `not_a_decider` (403) | yes |
| 6 | the fork is open (`status: open`) | `fork_closed` (409) | yes |
| 7 | `choice`, if present, is an option id | `choice_invalid` (400) | yes |
| 8 | first passing answer for `(fork_id, options_hash)` is `Counted`; later passing answers are `Revision`s naming the counted one | | yes |
| 9 | a Matrix edit, redaction or reaction changes nothing | | no event is an answer |
**Retraction** (`retracted: true`, `retracts: <answer id>`, `answer: nil`, `options_hash` still carried, so step 4 applies) passes steps 1 to 6, then:
- R1 form: it carries no `answer`, it names a `Counted` or `Revision` answer of this fork under this `options_hash`; else `bad_retraction` (400). A retraction naming an answer already retracted is `already_retracted` (409). `retracts` on a body that is not a retraction is `bad_retraction`.
- R2 who: the original answerer or any other decider; step 5 already requires a decider.
- R3 is dropped and its label stays vacant, so R4 keeps its name in the binding file. A ruling that answers a fork closes it (Eureka and Heimdall, 2026-11-10), so a ruling cannot cite an answer on an open fork (that is the stale-click gap, and a card with `ruled_from` set and `status: open` is invalid), and a retraction after a ruling meets step 6: refused `fork_closed`, stored. There is no `ruled` code (vector `V10a_a_ruling_that_closed_the_fork_makes_a_retraction_fork_closed`).
- R4 counted answer: retracting the counted answer removes it from the count and the next passing answer in admission order counts again. Retracting an answer that was not counted changes nothing. After every answer is retracted, a fresh answer is `Counted`. Records stay append-only.
**The counted answer is derived**, not stored: the first `Counted` or `Revision` answer under the card's current `options_hash` that no `Retraction` names. When the card changes, answers under the old hash stop counting and the next answer under the new hash is `Counted` (vectors `a_changed_card_...`). The `record` projection carries it as `counted_answer_id`.
### Operator authority
A ruling may carry `authority: Operator` only if it cites (`ruling_cite.answer_id`) a stored `desk_answer` whose outcome is `Counted` and whose stamped attribution (the answer's `authority` field) is `Operator` (or a stamped relay of her words from a mapped door, which this contract does not model). The citation is what gives a ruling authority: an attribution on an answer is a fact about the account, and authority exists only where a ruling cites a counted answer. The cited answer id is what `ruled_from` lists on the closed card. Negative vectors (`rulingCites`): no citation, a `Revision`, a `Refused` answer, a Counted answer by a non-operator, an answer that does not exist. Other authorities need no citation.
## Transport
Source: the desk's client needs (Ratatoskr PR #24 section 12, `ratatoskr-desk-spec-20261110.md`) and Heimdall's binding file, `/opt/commons/handoff/fork-desk-answer-binding-20261110.md`, section 1.3c, which is the source for the second door. **Everything in this section is spec text for Bifrost to build; no route, scope or audience below exists in Bifrost today.** The routes (with the `{mind}` segment), the scope names `desk.read` and `desk.answer`, the audience `bifrost-desk` and the auth order are Heimdall's rulings (his file, sections 4.1 to 4.3a); this page writes them as documents and vectors.
**Authorities.** Cards are projected **read-only from Huginn**: Bifrost never authors a card, a ruling or a status, and the desk never writes one. Bifrost verifies the bearer, resolves the actor from its `sub` (a `UserAccount.Id`), and for an answer forwards a typed `DeskAnswer` on the peer-checked second door to Huginn, then returns **Huginn's outcome unchanged**: the stored `desk_answer` in an `answer_result`, or its refusal. The wire of the second door is **pending**: Eureka's cut spec follows her record PR, so this page defines the documents the desk sees and not the bytes between Bifrost and Huginn (open choice 3).
| call | route (proposal) | request | response | token |
|---|---|---|---|---|
| 1 list | `GET /api/desk/{mind}/forks?status=&cursor=&limit=` | `list_request`: `mind` from the path, the rest from the query string | `list` | audience `bifrost-desk`, scope `desk.read` |
| 2 read | `GET /api/desk/{mind}/forks/{fork_id}` (the id percent-encoded) | `read_request`, its `mind` and `fork_id` from the path | `entry`, or `refusal` `fork_unknown` (404) or `fork_too_large` (422) | `desk.read` |
| 3 answer | `POST /api/desk/{mind}/answers` | `answer_request` as the body; `fork_id` stays in the body, the Mind is the route's and is no body field | `answer_result` (200 for `Counted`, `Revision` and `Retraction`; a stored `Refused` answer keeps the `answer_result` body and takes its code's status from the counting-rules table: `stale_fork` 409, `not_a_decider` 403, `fork_closed` 409, `choice_invalid` 400, `bad_retraction` 400, `already_retracted` 409, as the vectors' `results` pin; a replay returns the same status as the first call; the desk reads `outcome` and `code` from the body), or a `refusal` (`fork_unknown`, or `fork_too_large` when a stale answer's card re-read is past the cap), or `unavailable` | audience `bifrost-desk`, scope `desk.answer` |
**The `{mind}` segment** (Heimdall's rulings M1 to M5; vectors `transport.mindRoutes`).
- **M1.** `{mind}` is looked up in a **fixed configured list** of Mind ids, each mapped in configuration to that Mind's socket on the peer-checked door. The segment is percent-decoded once and compared, ordinally, with the list's ids: no case folding, trimming or Unicode normalisation, and a decoding failure is a miss. It is **never joined into a path, a socket name or a query**, so a value off the list opens nothing (`../x`, `a/b`, `a%2fb`, `eureka%00`, a 4 KB string, `GAMECULT-CHAT` for `gamecult-chat`, a trailing space, bytes that are not UTF-8: all unknown, 404, no socket opened).
- **M2.** An unknown Mind gets **the same refusal as an unknown fork**: `fork_unknown`, 404, the same bytes (the document's `error` names neither the Mind nor the fork). The list of Minds is not probeable. Auth is judged first, so a caller without a valid token, scope or Active membership learns nothing about which Minds exist.
- **M3.** On the POST the route's Mind must hold `fork_id`: step 3 is judged against **that Mind's** stored forks, so a `fork_id` the route's Mind does not hold is `fork_unknown` and nothing is stored, in either Mind. A fork cannot be answered through another Mind's route. A body that names `mind` is `body_unknown_field`: the Mind is the route's.
- **M4.** Scopes are **not per Mind**: `desk.read` and `desk.answer` open the routes of every configured Mind; every reader sees every fork's card and outcome (a non-decider sees no account id, "Reader views"), and which forks a caller may answer is the decider list.
- **M5. The decider list is sent per call.** Bifrost holds the admin-set list and sends it with **every answer, read and list request** to Huginn, as a typed field on the same peer-checked door; Huginn keeps no copy of it. Because it travels with read and list too, a projected card's `deciders` is the list of that call for a decider and empty for a reader who is not one, with `decider_count` travelling with both views, and step 5 judges an answer against the list the same call carried. It comes only from Bifrost's admin-set state: never from a Mind and never from any request body a client controls: no HTTP field, header or query parameter on any desk route sets or extends it, and a body naming `deciders` is `body_unknown_field` (scenario `M5_...`). A request carrying it from a Mind's own door or session is refused. **This is Bifrost and Huginn work and no vector here can test it** (the typed field and the door's wire are pending, open choice 3); the rule is stated so that the cut that builds the request has it. (Heimdall's ruling, #minds 2026-11-10 19:26Z; it replaces the earlier reading of a list Huginn receives once and holds.) **Envelope names:** the desk's envelope on that door uses the names this page uses, `mind`, `authority`, `subject` and `deciders`, with no conversion at the door (Eureka, #minds 2026-11-10 19:58Z); `deciders` is a per-call list of short ids at the envelope's top level beside `subject`, never inside the request body, and Huginn keeps no copy.
Calls 1 and 2 are enough for a read-only desk. Every response is the pinned MessagePack document or its JSON form with the same keys (open choice 1 of `vote-api.md`: the media type is unsettled).
**Auth, in order, per route.** (1) A bearer, else `unauthenticated` (401). (2) The token's audience is `bifrost-desk`: **the Matrix Authentication Service token chat holds today is refused, and so is any `votes.*` token** (audience `bifrost-votes`), both as `unauthenticated`. (3) The scope the route needs, else `insufficient_scope` (403): `desk.read` for calls 1 and 2, `desk.answer` for call 3, and neither implies the other. (4) The account behind `sub` is an Active member, else `forbidden` (403), as in the vote API; scope is judged first. (5) For call 3 the body is then shaped (steps 1 and 2 above), so **a non-Active member who sends a bad body, including one naming `answerer` or `authority`, gets `forbidden`, not a body refusal**; a body naming either key at any depth is `body_forbidden_field` only for an Active member. (6) Only then is Huginn asked, so `unavailable` never hides an auth or body refusal. A `votes.*` token touches none of the three. Heimdall's file leaves the audience and the claims check open; `insufficient_scope`, `forbidden`, `request_invalid` (400, a list or read request that fails its document; an unknown key, `authority` included, is `request_invalid` here, whatever Huginn's own decode does with it) and `request_id_conflict` are added to the door refusals, and so is `fork_too_large` (422, list and read; see List and Read). **`request_id_conflict` is answer-only**: the idempotency key exists only on the answer. A door `request_id_conflict` on a read or list is a door fault and reaches the client as 503 `unavailable`, never as `request_id_conflict` (Persona, 2026-11-11, agreed with Heimdall; vectors `transport.doorFaults`, each carrying the client response's bytes). The `unavailable` text's "same request_id" retry hint is for answers; reads and lists carry no request_id..
**List.** `list_request`: `{mind, status, cursor, limit}`. `mind` is the route's `{mind}`, a Mind id from the configured list. `status` is `open`, `recorded` or nil for both; `recorded` means a closed card. Cards come **newest first by `order`**. `order` is the card's unique rank per Mind: an integer from 0 that never changes once the card exists, later questions only appending (Huginn derives it from the Mind's questions sorted by admission ordinal, then key). A Mind never holds two cards of one `order`; a list or card set that does is refused, and the refusal names both fork ids. 0 is legal, but no cursor names it (`c0` fails the cursor pattern): a page ending at order 1 returns `c1`, and the next page returns the order-0 card alone with `next_cursor` nil. `limit` is 0 to 50; **`limit` 0 returns no entries, only the counts and the stamp**, which is what "N open" on the Mind's header needs. `cursor` is nil or `c<N>`: the cards whose `order` is below N; a client treats it as opaque and sends back the `next_cursor` it was given. `list`: `{mind, stamp, open_count, recorded_count, forks[entry], next_cursor}`. The counts cover every card the viewer may see, whatever `status` and `cursor` asked for. `stamp` is `s<N>`, N the receiver's last-seen Huginn admission ordinal for the Mind's desk documents: the same stamp means the same state, so a desk polls `limit` 0 and fetches bodies only when the stamp moves. `next_cursor` is nil on the last page. **A page may hold fewer than `limit` entries** when the next entry would push the encoded response past the receiver's response budget (Huginn: 65536 bytes); it then holds at least one, `next_cursor` is `c<order of the last entry returned>` so the next page resumes below it, and the counts and the stamp are those of the full list, unchanged. **`fork_too_large`** (HTTP 422) refuses a page whose **first** entry alone is past the budget: 422 because the request is well formed and authorised but names a resource the list cannot carry, and 413 is for a request body that is too large, which this is not; retrying returns the same refusal. Its `error` names the fork id and its `order`, for example `fork_too_large: fork <fork_id> (order <n>) alone is past the response budget`. **A refusal's `error` text is informational and not pinned**: the contract pins the code, the HTTP status, that the text names the fork id (and the order on a list), and the `c<order>` continuation on a list; the vectors' wording is a sample, and the judges check those four things, not the bytes of the text. The client continues the list with cursor `c<n>`, the cards strictly below that order, which skips this fork; it is then readable only by id (the read route returns the whole entry, and refuses with the same code only when that entry alone is past the cap, see Read); `c<n>` is valid only when n is at least 1, so for order 0 there is nothing below and the client has reached the end. `limit` 0 returns no entries and never refuses. (Ruled by Persona, 2026-11-11.) An `entry` is `{card, record}`: the card and its `record` (the answers with their outcomes and the derived `counted_answer_id`).
**Read.** `read_request` is `{mind, fork_id}`; the response is the `entry` for one fork of that Mind. Its `card.options_hash` is **computed by the server** from the five shown members as in "Options hash" and the client recomputes it from what it rendered; a card whose hash is not the hash of its members is no card (negative vector `card_hash_wrong`). The record's `fork_id` and `options_hash` are the card's, and `counted_answer_id` is derived, not copied (negative vectors for each). Every `desk.read` holder sees every fork's card and outcome in a configured Mind; only a decider's notes depend on the caller, going to callers in the admin-set `deciders` list, which Bifrost enforces at the door (ruling `desk-read-visibility`, option (c)); `fork_unknown` is only for a fork id that does not exist under the route's Mind (an unknown Mind answers the same, M2), so no fork is ever hidden by it, and a caller without `desk.read` is refused by the auth order above (`insufficient_scope`) before any lookup. **Huginn withholds account ids from a reader who is not a decider and returns counts ("Reader views"); Bifrost still strips notes.** Every `desk.read` holder sees cards and outcomes; notes (a decider's own words) only go to callers in the admin-set `deciders` list. The read and list shapes are unchanged and carry no audience field (`eureka-substrate:question:desk-read-visibility`, Heimdall's option (c)). **A read is refused `fork_too_large` (422) when the whole entry is past the reply cap** (Huginn's daemon caps every reply at 65536 bytes; the cap, not the page budget, is what a read meets). Its `error` names the fork id and has no order and no cursor to carry, because there is nothing to continue with (sample: `fork_too_large: fork <fork_id> alone is past the response budget`) (vector `transport.readTooLarge`; the cap is in no vector). **The answer route meets the same cap:** when an answer is `stale_fork`, Bifrost re-reads the current card to attach it, and if that read is past the cap the answer is refused `fork_too_large` (422) with the read form of the text, the same document as a read. Huginn has already stored the answer as `Refused(stale_fork)`, as for any stale answer, and it is never counted; the client is told `fork_too_large` instead of receiving the current card, and a replay of the same `request_id` gets the same 422 while the fork stays oversize. It is never `unavailable` (vector `transport.answerTooLarge`). A stripped read serves a decider's `note` as `note: null`, never omitted (map-mode: every key is written); `desk_answer.note` is already nullable in the schema, so a served record validates as it stands.
**Answer.** The body is the `answer_request` and nothing else a client may set: `fork_id`, `options_hash`, `answer`, `answered_at`, `request_id` (plus `retracted` and `retracts`, written `false` and nil unless the call retracts). The response is the stored outcome: `Counted`, `Revision`, `Retraction`, or `Refused` with its code, and the **`counted_answer_id`** the fork has once this call is applied (`n<seq>` or nil; derived as in "The counted answer is derived", the same value the `record` projection carries, so the desk needs no second read). A replay returns the stored outcome with the `counted_answer_id` current at the retry, not the one it had when first stored. **A `stale_fork` refusal carries the current card** in `answer_result.card`, so the desk can show the new fork without another read. Nothing is stored for a step 1 or 2 refusal, an unknown fork (or an unknown Mind, or a fork another Mind holds) or a `request_id_conflict`. `unavailable` promises nothing either way.
**Vectors (`transport` in `fork-card-vectors.json`).** `listRequest`, `readRequest` and `entry` (valid and invalid, with bytes); `listInvalid` (including `two_cards_share_one_order`, whose `refusal` is the exact text both judges must produce: `order 4 is shared by <first fork id> and <second fork id>`); `listScenarios` (six scenarios: over one Mind's five cards, paged open-only, recorded-only, all and `limit` 0; an empty Mind; and `order_zero_is_legal_but_no_cursor_names_it`, four cards of orders 3 to 0 paged by 3, whose first page ends at order 1 with `c1` and whose second page is the order-0 card with `next_cursor` nil; each response with its bytes, the cursor of page two being page one's `next_cursor`); `listShortPages` (three scenarios: a page that stops short and resumes below its last entry; `fork_too_large` on a first entry of order 3, continued with `c3`; and a short page ending at order 1 followed by `fork_too_large` on order 0, where the client is at the end; each step carries its bytes, and the receiver's budget is in no vector, so the judges check a short page as a nonempty strict prefix of the unbudgeted page and a refusal as naming the entry the page would open with); `doorFaults` (three cases: a door `request_id_conflict` is `unavailable` 503 on read and list and itself 409 on the answer); `readTooLarge` (one case: a read whose entry is past the cap, refused `fork_too_large` 422 with the read-form text and its bytes); `answerTooLarge` (one case: a stale answer whose card re-read is past the cap, the same 422 document over a stored `Refused(stale_fork)`: `outcome` `Refused`, `code` `stale_fork`, `counted` false); `authCases` (twenty-two cases: the MAS token, three `votes.*` tokens, a desk token with the other route's scope, a non-Active member on each route, and Huginn down on each route, with auth and membership judged first). Both suites replay the list requests through a reference judge and the auth cases through another. `mindRoutes` (a configured list of two Minds with their sockets and forks; 57 cases over the three routes: bad and percent-encoded segments, unknown Mind against unknown fork with `sameResponse` pairs of equal bytes, the POST on a Mind that does not hold the fork, scopes that are not per Mind, and auth judged before the Mind lookup) is replayed by both suites through a reference judge that reports the refusal, the socket it would open and the Mind that stored an answer. Top-level `unavailable` pins the 503 document. Scenarios pin: an `unavailable` where Huginn never had the request, retried into a `Counted` answer; an `unavailable` where Huginn had stored it, whose retry returns the stored outcome and adds no Revision; a replayed `Refused` with its card; `request_id_conflict`; the same id from another account; and a non-Active member with a bad body, with the door down, getting `forbidden` first. A bad body is refused before the door is asked.
**Bifrost must still build:** all three routes; the `bifrost-desk` audience and the `desk.read` and `desk.answer` scopes on the OIDC token it issues to chat; the configured Mind list and its sockets (M1); the projection of cards and records from Huginn, the stamp, the cursor and the counts; the `DeskAnswer` call on the second door and the pass-through of its outcome; the decider list, sent with every answer, read and list request from the admin-set list only (M5); the `unavailable` answer. **Huginn must still build** the second door's wire, the per-call decider list it receives with each answer, read and list request, keeping no copy (M5), and the read side the projection uses (Eureka's cut spec, after her record PR). **The desk may build today** against the vectors: nothing in them needs a live service.
## Old format, the mind's record, and what maps to what
The old desk (`index.html`) read `questions/<id>` and wrote `answers/<id>` in a shared store. The mind's `question` record is the source in the new design; the card is its projection.
| old desk | mind `question` record | card |
|---|---|---|
| `qid`, `id` | key `campaign:question:label` | `fork_id` |
| `title` | `title` | `title` |
| `body` | `question` (the long text) | `context` |
| `depends`: a free string | `depends`: up to 8 prose lines (`huginn schema question`) | `depends`: the lines verbatim |
| `options[{key, label, text}]` | `options[{label, text}]` | `options[{id, text}]` |
| option `key` | option `label` (a slug, the id) | option `id` |
| option `label` (a human title) | none | **dropped**: not stored, not hashed; shown text is `text` |
| `rec` (a key) | `recommended` (a label) | `recommendation` (an id), never nil |
| `lean` ("Self leans" mark) | none | `leans` (an id or nil), not hashed |
| `flag` (the provisional line) | none | `flag` (a string or nil), not hashed |
| `status` open or closed, `ruling` | derived from the resolution (status is never written): any resolution of the question closes it; the ruling is the one that answered it | `status`, `ruling`, both a projection (`ruling` is nil on an open card and on a card closed with no ruling) |
| `order` | none (unique rank per Mind, from 0) | `order` |
| none | `raised_in`, `asked_on` | `raised_in`; `asked_on` is not carried |
| none | the question's admission time (no timestamp in the old format; `asked_on` is a date and stays uncarried) | `raised_at`, an instant or nil, not hashed |
| none | none | `deciders` (outside the Mind), `ruled_from`, `options_hash` |
| none | none | `decider_count`, `answerer_count` (counts that stand in for withheld ids) |
| answer `{choice: key or null, note, question, updatedAt}`, one mutable document per question | none | `answer_request` then an append-only `desk_answer` |
| `note`: her words verbatim | | `answer.note`, verbatim |
| "Clear pick": `choice` back to null so the note stands alone | | an `answer` with `choice` nil and `note` set, as the first answer or as a `Revision` |
| "Changing an answer is fine until it's admitted as a ruling" | | a later answer is a `Revision`; `Retraction` withdraws one; neither changes who counted first, and a ruling that answers the fork closes it, after which no answer or retraction is taken (`fork_closed`) |
Differences that need Void's choice: the old page **overwrites** one answer per question and the new store **appends**, so the booth cannot show "your answer" as one mutable field (it shows the latest answer by the person, plus its history); the old page needs no identity and the new one stamps `answerer` from a verified session; the old `label` has nowhere to live unless the card's `text` carries it.
## Transport note for Void: Matrix event versus how the mind carries questions
How a mind carries a question today: a typed `question` record in Huginn (`huginn view`, `huginn query`); the Persona relays it to the operator in chat, and the operator's reply comes back as chat text that Self reads and admits as a `ruling` in her words. Identity is whoever the chat attributes the message to; nothing binds an answer to the question text she saw.
Option A, **a Matrix event in the Mind's room with the answer as a reply.** Gains: it sits where people already are, needs no sign-in beyond Matrix, and the thread shows the exchange. Loses: the sender is an mxid, not a Bifrost account, so `answerer` and `authority` have no verified source; an edit or redaction changes the visible message but not the decision (binding rule 9), so the thread and the record can disagree; there is nowhere to carry `options_hash`, so a stale card cannot be refused; and a reply is free text, so `choice` is lost.
Option B, **Bifrost receives the answer and Huginn stores it** (the design in this page; Ratatoskr PR #24 also withdraws the Matrix carrier). Gains: a verified account, a bound hash, refusals that are stored and visible, one counted answer, retractions. Loses: the booth needs Bifrost sign-in, which does not reach chat yet (Bifrost PR #9 and #37 design it), so until then the desk is read-only; and the exchange is no longer in the room.
**Recommendation: B, with Matrix as presentation only.** The Mind may post a card as a listing and a link into the desk; no Matrix event is ever an answer (rule 9). Confidence: high on the rule (it is Heimdall's and Eureka's already), moderate on the interim, where read-only until sign-in is a real cost; if the operator needs to answer before sign-in reaches chat, the honest stopgap is the old path (chat text, Self admits a ruling in her words) rather than a Matrix reply the record cannot bind.
## Differences from the binding file
- Spelling: every field is as the binding file names it (`fork_id`, `options_hash`, `answered_at`, `retracted`, `retracts`, `answer.choice`, `answer.note`, `recommendation`, `depends`, option `id`), except that `schemaVersion` keeps CultNet's camelCase and `answer_id` replaces the earlier `receipt id` (Persona brief).
- Map-mode writes every key: `retracted` is always present; nothing is "optional".
- `Refused` stores every refusal from steps 4 to 7 and R1 and R2 (his section 1.3b); only steps 1 and 2 and `fork_unknown` are not stored. The binding file does not say whether `fork_unknown` is stored; it cannot be keyed without a question, so it is not (open choice 4).
- The binding file's `choice_valid` has no named refusal code; this page names it `choice_invalid`. `fork_known` is `fork_unknown`. A type-invalid body is `body_invalid`.
- `ruled` (his R3 and V10) is dropped: a ruling that answers a fork closes it, so the case is `fork_closed` (V4, V10a). His file drops it too.
- Heimdall's section 1.3e (Q1 to Q6, V12 to V18) is the source for `request_id`; this page matches it, with two notes. Q1's open item is settled by his ruling: a replayed `stale_fork` returns the current card. In V14 a decider's identical body is a `Revision` whose `of` necessarily names the first account's counted answer id (the record shows it to every decider), so "never contains X's answer_id" holds for Y's answer id and document, not for `of` (Heimdall: `of` naming the counted answer is allowed); the vector with an off-list first account pins the case with a `Counted`, which has no `of`.
- `request_id` (on the request and the stored `desk_answer`) and `request_id_conflict` are this page's: his file has no idempotency key, and a retry after `unavailable` needs one. Eureka's `desk_answer` kind and Huginn's request type must carry it.
- Transport: the routes, the `{mind}` segment, the scopes, the audience and the auth order are his rulings (4.1 to 4.3a); the documents (list, read, `unavailable`) are this page's. The note limit is 4000 UTF-8 bytes here, as his 1.1 leaves the unit unsaid.
- The recommendation is never nil. The binding file's earlier vectors D and E used a nil recommendation; its updated D and E use `"a"`, as here.
- `raised_at` (2026-11-11) is the desk-feed addition (Bifrost PR 67): a required key, an instant or nil, outside `options_hash` (the five hashed members and every digest are unchanged; vectors `raised_at_nil`, `raised_at_epoch_integer`, `raised_at_date_only`, `raised_at_missing`). The card is spec-authored and no runtime deploys it, so the schema id stays `gamecult.fork.card.v0`.
- Reader views (2026-11-11) are the desk-feed addition behind Huginn PR 56 and `eureka-substrate:ruling:desk-read-visibility-withheld-ids`: the card gains `decider_count` and `deciders` may be empty; the record gains `answerer_count` and `answerer` may be nil, the counts beside the collections they summarise (not on the list envelope, which neither the read entry nor the `stale_fork` card travels in). `options_hash` and every digest are unchanged. The schema ids stay `gamecult.fork.card.v0` and `gamecult.fork.record.v0` for the same reason as `raised_at`.
- Hash vectors: A, B, C, D, E, F and G of the binding file are reproduced digest for digest by the Python generator, the C# suite and the TypeScript suite (the generator prints `same` for each; none differs).
## Open choices
1. Case of field names: snake_case for the fork documents (as the binding file) beside camelCase for the vote documents (as Bifrost's serializer). One convention would be tidier.
2. NFC across runtimes relies on each runtime's normaliser; a pinned table, as `write-receipt.md` does for letters, would remove the reliance. The risk is only for code points assigned after a runtime's Unicode version.
3. How Huginn learns the verified subject: the second door's authenticated context (the peer credential names Bifrost the service, not the member), so the subject must travel beside the request. This contract has no envelope for it.
4. Whether `fork_unknown` and shape refusals should leave an audit record somewhere other than a `desk_answer`.
5. Whether `leans` and `flag`, which the booth shows, should join the hash. The binding file's five members do not include them; a changed lean would not make an answer stale. `raised_at` is immutable, so it was kept out of the hash for the same reason it need not be in: it can never make a shown card stale.
6. What the person sees of a retracted counted answer when the next answer counts: this contract only says the count moves.
7. `received_at` is not in the binding file's stored fields; it is kept for `claim_in_future` and could be dropped if the `n<seq>` answer ordinal alone decided answer order.
8. The stamped `authority` field is an attribution, not an authority (above); renaming it `attribution` would say so, at the price of a new key the door must also refuse in a body.
9. The claims Bifrost verifies are PR #24's open item 5. (The audience `bifrost-desk`, the three routes and the scope names are ruled by Heimdall.)
10. The second door's wire between Bifrost and Huginn (pending Eureka's cut spec). `unavailable` is a document of its own, outside the refusal set, so a desk can retry it; calls 1 and 2 use it too when Huginn is down (vectors `huginn_down_*`).
11. While a known Mind's door is down its routes answer `unavailable` (503) and an unknown Mind answers `fork_unknown` (404), so the two are distinguishable for that period. M2 makes them identical only when every door is up; making them identical when one is down means answering `unavailable` for any `{mind}` not on the list, which tells a client nothing about which Minds exist but also hides a typo. Heimdall's to rule; the vectors do not pin it.
12. A fork's answers are unbounded in number, so a fork's entry can grow past the reply cap and become unreadable (`fork_too_large` on read, and on list when it is the first entry of a page). Whether the contract should cap the answers per fork is for Persona and Heimdall to rule; nothing here decides it.
13. The read form of `fork_too_large` is a target, not current behaviour: Huginn's door answers an over-cap read as `Refused(ResponseTooLarge{bytes, limit})`, which carries no fork id and is not `fork_too_large`. Bifrost maps it to `fork_too_large` (422) using the route's `fork_id`; until it does, a read past the cap reaches the client as 503 `unavailable`. The same holds for the answer route: a `stale_fork` answer whose card re-read is past the cap is a Bifrost target too (`fork_too_large` 422 over a stored `Refused(stale_fork)`), and until it is built the client sees 503 `unavailable`.
14. What a non-decider can still infer: the card's `status` and `order`, the `authority` role on each answer (a role, not an id, and the ruling leaves it whole), and `answerer_count`. Whether a role plus a count is too much to show is Eureka's ruling to revisit.

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,107 @@
# CultNet vote API v0 (specification)
Status: specification, drafted for Heimdall (Bifrost) and a Ratatoskr voting module. No server implements it yet.
Files here: `vote-api.schema.json` (one schema, a `$defs` entry per document), `vote-api-vectors.json`. Conformance suites: `tests/GameCult.Networking.Tests/GovernanceVectorTests.cs` (C#) and `packages/cultnet-ts/test/governance-vectors.test.ts` (TypeScript). The vectors are written by `scripts/gen-governance-vectors.py`.
Facts it rests on: `/opt/commons/handoff/bifrost-browser-voting-facts.md` (Bifrost `416f11a`; `416f11a..276f611` changes four files: a Standard-close concurrency fix in `MotionGovernanceService.cs`, two test files and `docs/deploy-main-to-live.md`; none touches the sealed API, so the facts hold at `276f611`) and Bifrost PR #37 (`docs/ratatoskr-voting-auth.md`, see "PR #37" below).
## Scope
**Which API covers which.** This contract covers **sealed-ballot motions only** (`gamecult.sealed_ballot.v0`). **Standard motions are not covered here.** Bifrost PR #37 also serves them (`PUT /api/votes/motions/{id}/ballot`, a receipt `{motionId, choice, revision, castAtUtc, receiptId}`), and that API is #37's own: this page defines no document for it, and a Standard motion id sent to any exchange below is answered `not_found`. Today Standard motions have no JSON read or cast API at all, no Mind path, and no receipt (open choice 1). The contract names four exchanges and one refusal document:
| exchange | request | response |
|---|---|---|
| motion list | `gamecult.vote.motion_list_request.v0` | `gamecult.vote.motion_list.v0` |
| motion read | `gamecult.vote.motion_read_request.v0` | `gamecult.vote.motion.v0` |
| cast (commit or reveal) | `gamecult.vote.cast.v0` | `gamecult.vote.receipt.v0` |
| any refusal | | `gamecult.vote.refusal.v0` |
Every document is CultNet map-mode: all keys are written, nil where absent, and the schema is closed (`additionalProperties: false`). Instants are RFC 3339 UTC with a `Z` and up to seven fractional digits. Ids are lowercase `D`-format UUIDs. A motion states `highStakes`, a boolean fixed when the motion opens and never changed after (Bifrost PR #37, D2): it is the property that makes a commit demand a fresh sign-in (`reauthentication_required`). The bytes of every document are pinned by `messagePackHex` in the vectors (shortest integers, shortest strings, keys in the order the schema lists them); JSON with the same keys is the xenos-boundary form of the same documents (open choice 4: the wire format).
## Documents
- **motion_list_request**: `{schemaVersion, phase}`; `phase` is `Commit`, `Reveal`, `Closed` or nil for all.
- **motion_list**: `{schemaVersion, motions: [{id, title, round, phase, commitClosesAtUtc, closesAtUtc, highStakes}]}`. **Heimdall must add this endpoint**: Bifrost has no list route for sealed motions today, only fetch-by-id. No pagination in v0 (open choice 2).
- **motion_read_request**: `{schemaVersion, motionId}`.
- **motion**: Bifrost's `SealedMotionState` with its names: `{schemaVersion, id, title, summary, round, priorMotionId, ballotSchema, phase, candidates (two or more), commitClosesAtUtc, closesAtUtc, highStakes, yourVoterId, yourCommitment (a receipt or nil), announcement}`. It never carries a tally, a turnout, or another voter's commitment or ballot (a `tally` key is a negative vector). `yourVoterId` is the voter's `UserAccount.Id`, the `voterUserAccountId` of the commitment.
- **cast**: `{schemaVersion, motionId, verb, commitment, ballot}` with exactly one of two shapes: `verb: "commit"` with a 64-hex `commitment` and `ballot: nil`, or `verb: "reveal"` with `commitment: nil` and a `ballot`. One document for both verbs; today's two routes (`/commit`, `/reveal`) may stay, taking this document as the body, with a `verb` that must match the route (open choice 3).
- **ballot** (inside a reveal): `{schema: "gamecult.sealed_ballot.v0", motionId, round, candidate, reason, salt, persona}`. `candidate` and `reason` are single-line and **already trimmed** (no Unicode White_Space at either end, using the White_Space list of `write-receipt.md`); the contract refuses an untrimmed value rather than trimming it, so the bytes hashed are the bytes sent. **This is a behaviour change Bifrost must make:** it trims `candidate` and `reason` silently on reveal today (facts file section 3c, `SealedBallotService.cs:154, 385-387`), so an untrimmed reveal is accepted and hashed trimmed; under this contract it is `invalid` (open choice 5). `reason` is 1 to 280 characters counted in UTF-16 code units, as Bifrost counts. A JSON Schema `maxLength` counts code points, so the schema alone accepts up to 280 astral characters; both suites add the rule (vectors `cast_reveal_reason_140_astral_280_units` valid, `ballot_reason_141_astral_282_units` refused). `salt` is 32 to 128 lowercase hex. `persona` is `""` for a member and a printable-ASCII, space-free id of at most 64 characters for a Mind.
- **receipt**: Bifrost's `SealedBallotReceipt`: `{schemaVersion, motionId, voterUserAccountId, commitment, revision (>= 1), committedAtUtc, revealed}`. The receipt for a reveal has `revealed: true` and **echoes nothing of the ballot** (no candidate, reason or salt; negative vectors for each).
- **refusal**: `{schemaVersion, code, error, retryAfterSeconds}`; the `code` set is closed:
| code | HTTP | trigger |
|---|---|---|
| `unauthenticated` | 401 | no bearer, or one Heimdall does not know (the API must answer this, not redirect to a sign-in page) |
| `reauthentication_required` | 401 | a commit on a `highStakes` motion with no sign-in fresh enough (#37 D2: `prompt=login`, `max_age`); the voter signs in again at the top level and retries |
| `forbidden` | 403 | authenticated but not an Active member, or a Mind key used on a route it may not call |
| `not_found` | 404 | unknown motion id, a Standard motion id, or a reveal with no prior commit |
| `wrong_phase` | 409 | commit outside Commit, reveal outside Reveal |
| `contended` | 409 | a reveal raced the close fence: send it again |
| `invalid` | 400 | the body fails the schema or an in-document rule (untrimmed text, candidate not in the motion, `round` or `motionId` mismatch, `persona` wrong for the credential) |
| `commitment_mismatch` | 400 | a reveal whose canonical bytes do not hash to the stored commitment |
| `payload_too_large` | 413 | body over 4096 bytes |
| `rate_limited` | 429 | cast rate exceeded, per account; `retryAfterSeconds >= 1`, mirrored in `Retry-After` |
| `held` | 503 | the Ratatoskr voting brake is set (#37 D5): nothing is read or stored while it is |
Each code has a vector (`refusals`), and every failure of the schema has a negative vector (`invalid`).
**Not on Bifrost's member API yet.** The **413 body cap of 4096 bytes** and the **429 per-account cast limit** are not on Bifrost's member sealed API today (facts file sections 3c and 5): both exist only on the Mind intake route (4096 bytes; 30 requests a minute per source address). #37 proposes 30 casts per 10 minutes per account for the member routes (its D6). `reauthentication_required` and `held` are #37 proposals (D2, D5) and exist nowhere in code. All four are spec text until Heimdall builds them.
**Two refusals Bifrost must split first.** Bifrost returns a reveal whose bytes do not hash to the stored commitment as a plain `400` `Invalid` ("The ballot does not match the stored commitment."), the same status and error type as every other bad body, so `commitment_mismatch` and `invalid` are one today. It returns a contended reveal (a race with the close fence, `SealedBallotService.cs:186-189`) as `WrongPhase` `409`, the same type as a reveal outside the Reveal phase, so `contended` and `wrong_phase` are one today. A client cannot tell "send it again" from "too late", or "wrong ballot" from "malformed body", until Bifrost splits each pair.
## Commitment
`commitment = lowercase hex SHA-256 of the UTF-8 bytes of`
gamecult.sealed_ballot.v0 \n {motionId} \n {voterUserAccountId} \n {persona} \n {round} \n {candidate} \n {reason} \n {salt}
with `\n` U+000A, ids in `D` format, `round` in decimal, `persona` empty for a member. This is Bifrost's formula as the code has it (`SealedBallotService.cs:154, 385-387`); `docs/sealed-ballot-motion.md:22-23` omits `{persona}` and is stale. The vectors give the canonical bytes (`canonicalHex`) and the digest for a member, a Mind, non-ASCII text, and a decomposed-versus-precomposed pair (no normalisation: they are different ballots). `pairs` pins what a reveal must match: the right ballot matches; a changed reason, another voter id, or a dropped persona does not, and the answer is `commitment_mismatch`.
## Receipts the voter can check later
A cast returns the receipt. The voter checks it later in two places that must agree byte for byte with it: `motion.yourCommitment` on a motion read (a commit receipt is present from the commit; the revision counts re-commits), and, after close, the `commitment` the result lists for them. The contract adds no receipt id or signature: the receipt is the commitment, its revision and the time (open choice 6).
## Personas
A Persona votes only on sealed motions, through a Mind key (Bifrost binds an Ed25519 public key 1:1 to an Active member account). The ballot's `persona` is the bound persona id and the commitment covers it, so a Mind's ballot cannot be replayed as a member's (`pairs`: dropping the persona does not match). The Mind-intake route today takes a signed `MindEnvelope` anonymously; this contract does not model that envelope, nonce or signature (open choice 7). Standard motions have no Mind path.
## Authentication
On the wire the credential is an opaque `Authorization: Bearer` header. The token route, claims, scopes and audience belong to Heimdall and Bifrost and are not designed here. A request without a usable bearer is answered `unauthenticated` as a typed refusal.
**Auth per route, now and after #37.** Routes are Bifrost's; this contract fixes the documents, not the paths (open choice 3).
| exchange | today (`/api/sealed-motions`, facts file 3c) | after #37 (`/api/votes`) |
|---|---|---|
| motion list | no route | bearer, scope `votes.read` |
| motion read | session cookie + Active member | bearer, scope `votes.read` |
| commit, reveal | session cookie + Active member | bearer, scope `votes.cast`, audience `bifrost-votes`; a high-stakes commit also needs a fresh sign-in |
| Mind cast | anonymous; the Ed25519 signature on a `MindEnvelope` is the credential | unchanged: Minds never use `/api/votes` (#37) |
| open a motion, Mind-key registry | session cookie + moderator | unchanged, cookie only |
Today a member route has **no bearer scheme**: it accepts the host-only `__Host-bifrost_session` cookie and answers an unauthenticated call with a redirect to the sign-in page, not a 401. After #37 `/api/votes` is bearer-only and answers a typed 401. The wire header stays an opaque bearer either way; only Bifrost knows which of the two it is checking.
## PR #37 (Bifrost `docs/ratatoskr-voting-auth.md`, head `447a7cb`)
Checked 2026-11-10 by an Eyes pass over the PR and the schema; conflicts for Heimdall:
1. **Standard motions.** The PR serves them (`PUT /api/votes/motions/{id}/ballot {choice, comment?}`, a receipt `{motionId, choice, revision, castAtUtc, receiptId}`); the contract is sealed-only and answers `not_found` for a Standard id.
2. **Verb model.** The PR keeps `POST .../commit` and `.../reveal` and adds `GET .../result`; the contract has one `cast` document and no result document.
3. **List and read as GET.** The PR names GET routes; the contract's request documents carry a `phase` filter and a `motionId`. The mapping (query string and path) is undefined until Heimdall fixes the routes.
4. **Refusals.** The PR lists 401/403/404/409/429 with `WWW-Authenticate`; the contract adds 400 (`invalid`, `commitment_mismatch`) and 413, and a typed body.
5. **Minds.** The PR says Minds never use `/api/votes` and keeps the anonymous signed `/mind` route; the contract carries `persona` in the ballot and a `forbidden` case for a Mind key on a route it may not call. The PR's token is never a Persona's.
6. **Authority.** The contract's `unauthenticated` says a bearer "Heimdall does not know"; the PR makes Bifrost the OIDC issuer (a second client, scopes `votes.read` and `votes.cast`, audience `bifrost-votes`) and notes that Bifrost OIDC is not live (its D7).
7. **Rate limits.** The PR keys per account (30 casts per 10 minutes); the contract now says per account too, and neither exists on the member API yet (above).
8. **Wire format.** The PR's errors are JSON; the contract's vectors are MessagePack. Nothing settles the media type.
9. **The PR's seven open decisions** (consent for `votes.cast`, re-authentication, no refresh token, grant revocation, a Ratatoskr brake, rate numbers, MAS rollout order) assume nothing the contract contradicts. D2 (re-authentication at cast) is answered yes for sealed commit in this contract: `reauthentication_required`, with `highStakes` fixed at open. D5 (a brake) is `held`.
## Open choices for Heimdall
1. Standard motions in v0 or a v1 (the PR already serves them).
2. Pagination and filters on the list.
3. One `cast` route or the two existing ones; `verb` redundancy.
4. Wire format: MessagePack documents with a JSON export, or JSON only (the PR is JSON).
5. Refuse untrimmed `candidate` and `reason` (this contract) or keep trimming server-side.
6. Whether a receipt should carry an id or a server signature so a voter can prove it to a third party.
7. The Mind path: keep the signed envelope on its own route, or take Mind keys as a bearer credential.
8. `reason` length in UTF-16 code units (as Bifrost) or code points.
9. Whether the result document (`GET .../result`) joins this contract.

View file

@ -0,0 +1,420 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://github.com/GameCult/CultLib/contracts/cultnet/governance/vote-api.schema.json",
"title": "CultNet vote API v0 (sealed-ballot motions)",
"description": "Typed documents for listing, reading and casting on sealed-ballot motions, and the receipt a cast returns. Map-mode: every key is written, nil where absent. Spec: vote-api.md. Vectors: vote-api-vectors.json.",
"$defs": {
"uuid": {
"type": "string",
"pattern": "^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$"
},
"instant": {
"type": "string",
"pattern": "^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}(\\.[0-9]{1,7})?Z$"
},
"hex64": {
"type": "string",
"pattern": "^[0-9a-f]{64}$"
},
"trimmedLine": {
"type": "string",
"minLength": 1,
"pattern": "^(?![\\t-\\r \\u0085   - 

   ])[^\\n\\r\\u0085

]+(?<![\\t-\\r \\u0085   - 

   ])$"
},
"ballot": {
"type": "object",
"additionalProperties": false,
"required": [
"schema",
"motionId",
"round",
"candidate",
"reason",
"salt",
"persona"
],
"properties": {
"schema": {
"const": "gamecult.sealed_ballot.v0"
},
"motionId": {
"$ref": "#/$defs/uuid"
},
"round": {
"type": "integer",
"minimum": 1
},
"candidate": {
"$ref": "#/$defs/trimmedLine"
},
"reason": {
"type": "string",
"minLength": 1,
"maxLength": 280,
"pattern": "^(?![\\t-\\r \\u0085   - 

   ])[^\\n\\r\\u0085

]+(?<![\\t-\\r \\u0085   - 

   ])$"
},
"salt": {
"type": "string",
"pattern": "^[0-9a-f]{32,128}$"
},
"persona": {
"type": "string",
"pattern": "^[!-~]{0,64}$"
}
}
},
"motionListRequest": {
"type": "object",
"additionalProperties": false,
"required": [
"schemaVersion",
"phase"
],
"properties": {
"schemaVersion": {
"const": "gamecult.vote.motion_list_request.v0"
},
"phase": {
"oneOf": [
{
"enum": [
"Commit",
"Reveal",
"Closed"
]
},
{
"type": "null"
}
]
}
}
},
"motionListItem": {
"type": "object",
"additionalProperties": false,
"required": [
"id",
"title",
"round",
"phase",
"commitClosesAtUtc",
"closesAtUtc",
"highStakes"
],
"properties": {
"id": {
"$ref": "#/$defs/uuid"
},
"title": {
"type": "string",
"minLength": 1
},
"round": {
"type": "integer",
"minimum": 1
},
"phase": {
"enum": [
"Commit",
"Reveal",
"Closed"
]
},
"commitClosesAtUtc": {
"$ref": "#/$defs/instant"
},
"closesAtUtc": {
"$ref": "#/$defs/instant"
},
"highStakes": {
"type": "boolean",
"description": "Fixed when the motion opens and never changed after: a high-stakes sealed motion needs a fresh sign-in at commit (Bifrost PR #37, D2)."
}
}
},
"motionList": {
"type": "object",
"additionalProperties": false,
"required": [
"schemaVersion",
"motions"
],
"properties": {
"schemaVersion": {
"const": "gamecult.vote.motion_list.v0"
},
"motions": {
"type": "array",
"items": {
"$ref": "#/$defs/motionListItem"
}
}
}
},
"motionReadRequest": {
"type": "object",
"additionalProperties": false,
"required": [
"schemaVersion",
"motionId"
],
"properties": {
"schemaVersion": {
"const": "gamecult.vote.motion_read_request.v0"
},
"motionId": {
"$ref": "#/$defs/uuid"
}
}
},
"receipt": {
"type": "object",
"additionalProperties": false,
"required": [
"schemaVersion",
"motionId",
"voterUserAccountId",
"commitment",
"revision",
"committedAtUtc",
"revealed"
],
"properties": {
"schemaVersion": {
"const": "gamecult.vote.receipt.v0"
},
"motionId": {
"$ref": "#/$defs/uuid"
},
"voterUserAccountId": {
"$ref": "#/$defs/uuid"
},
"commitment": {
"$ref": "#/$defs/hex64"
},
"revision": {
"type": "integer",
"minimum": 1
},
"committedAtUtc": {
"$ref": "#/$defs/instant"
},
"revealed": {
"type": "boolean"
}
}
},
"motion": {
"type": "object",
"additionalProperties": false,
"required": [
"schemaVersion",
"id",
"title",
"summary",
"round",
"priorMotionId",
"ballotSchema",
"phase",
"candidates",
"commitClosesAtUtc",
"closesAtUtc",
"highStakes",
"yourVoterId",
"yourCommitment",
"announcement"
],
"properties": {
"schemaVersion": {
"const": "gamecult.vote.motion.v0"
},
"id": {
"$ref": "#/$defs/uuid"
},
"title": {
"type": "string",
"minLength": 1
},
"summary": {
"type": "string"
},
"round": {
"type": "integer",
"minimum": 1
},
"priorMotionId": {
"oneOf": [
{
"$ref": "#/$defs/uuid"
},
{
"type": "null"
}
]
},
"ballotSchema": {
"const": "gamecult.sealed_ballot.v0"
},
"phase": {
"enum": [
"Commit",
"Reveal",
"Closed"
]
},
"candidates": {
"type": "array",
"minItems": 2,
"items": {
"$ref": "#/$defs/trimmedLine"
}
},
"commitClosesAtUtc": {
"$ref": "#/$defs/instant"
},
"closesAtUtc": {
"$ref": "#/$defs/instant"
},
"highStakes": {
"type": "boolean",
"description": "Fixed when the motion opens and never changed after: a high-stakes sealed motion needs a fresh sign-in at commit (Bifrost PR #37, D2)."
},
"yourVoterId": {
"$ref": "#/$defs/uuid"
},
"yourCommitment": {
"oneOf": [
{
"$ref": "#/$defs/receipt"
},
{
"type": "null"
}
]
},
"announcement": {
"oneOf": [
{
"type": "string"
},
{
"type": "null"
}
]
}
}
},
"commitCast": {
"type": "object",
"additionalProperties": false,
"required": [
"schemaVersion",
"motionId",
"verb",
"commitment",
"ballot"
],
"properties": {
"schemaVersion": {
"const": "gamecult.vote.cast.v0"
},
"motionId": {
"$ref": "#/$defs/uuid"
},
"verb": {
"const": "commit"
},
"commitment": {
"$ref": "#/$defs/hex64"
},
"ballot": {
"type": "null"
}
}
},
"revealCast": {
"type": "object",
"additionalProperties": false,
"required": [
"schemaVersion",
"motionId",
"verb",
"commitment",
"ballot"
],
"properties": {
"schemaVersion": {
"const": "gamecult.vote.cast.v0"
},
"motionId": {
"$ref": "#/$defs/uuid"
},
"verb": {
"const": "reveal"
},
"commitment": {
"type": "null"
},
"ballot": {
"$ref": "#/$defs/ballot"
}
}
},
"cast": {
"oneOf": [
{
"$ref": "#/$defs/commitCast"
},
{
"$ref": "#/$defs/revealCast"
}
]
},
"refusal": {
"type": "object",
"additionalProperties": false,
"required": [
"schemaVersion",
"code",
"error",
"retryAfterSeconds"
],
"properties": {
"schemaVersion": {
"const": "gamecult.vote.refusal.v0"
},
"code": {
"enum": [
"unauthenticated",
"forbidden",
"reauthentication_required",
"not_found",
"wrong_phase",
"contended",
"invalid",
"commitment_mismatch",
"payload_too_large",
"rate_limited",
"held"
]
},
"error": {
"type": "string",
"minLength": 1
},
"retryAfterSeconds": {
"oneOf": [
{
"type": "integer",
"minimum": 1
},
{
"type": "null"
}
]
}
}
}
}
}

View file

@ -0,0 +1,122 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://github.com/GameCult/CultLib/contracts/cultnet/interop/cultnet.error.v1.schema.json",
"title": "CultNet Error Message v1",
"description": "cultnet.error.v0 plus messageId (write-receipt.md). Sent in place of cultnet.error.v0 only to a connection that declared the write-receipt capability. v0 is unchanged: its schema is closed (additionalProperties false), so an extra key would make a v0 peer reject the frame instead of ignoring it. Map-mode: every key is written, messageId nil when the error answers no readable message. The code enum is the v0 enum plus limit_reached (write-receipt.md, Remembered-write limits).",
"type": "object",
"required": [
"schemaVersion",
"error",
"routingHint",
"code",
"details",
"messageId"
],
"additionalProperties": false,
"properties": {
"schemaVersion": {
"const": "cultnet.error.v1"
},
"error": {
"type": "string",
"minLength": 1
},
"routingHint": {
"description": "Optional shard routing detail for an authority failure. Null for every selection/cursor refusal code (R-N).",
"type": [
"object",
"null"
],
"additionalProperties": false,
"required": [
"shard",
"reason"
],
"properties": {
"shard": {
"type": [
"object",
"null"
]
},
"reason": {
"type": "string"
}
}
},
"code": {
"description": "Machine-readable refusal code (R-N). Additive: a v0 peer that does not know this field ignores it. The set is a closed enum: Rust and TypeScript refuse a message whose code is not listed, so adding a code is one coordinated change across every runtime (this schema, the C# factory, the Rust CultNetErrorCode, and a captured-bytes case for each). limit_reached: a receipt-capable server refuses a NEW write because its per-session remembered-write table (1024 entries) or its server-wide memory budget is full; messageId is the write's id.",
"type": [
"string",
"null"
],
"enum": [
null,
"selection_invalid",
"cursor_stale",
"cursor_invalid",
"reference_outside_target",
"variant_unsupported",
"unowned_schema",
"write_not_authorized",
"unsupported_schema_version",
"message_invalid",
"limit_reached"
]
},
"details": {
"description": "Code-specific structured detail (R-N). Every field is populated only for the code that defines it; the others are null, not omitted.",
"type": [
"object",
"null"
],
"additionalProperties": false,
"required": [
"field",
"value",
"asOf",
"current"
],
"properties": {
"field": {
"description": "selection_invalid: the selection field that failed. variant_unsupported: \"subscriptionId\", when a subscription was ended. unowned_schema: \"schemaId\". write_not_authorized: \"recordKey\". unsupported_schema_version: \"schemaVersion\". message_invalid: the request field that is missing or ill-typed (\"message\" when the whole frame cannot be read). limit_reached: no field; details stay all null. message_invalid also names \"messageId\" (over 256 bytes), \"document\", \"schemaId\" or \"channel\" (write on an unreliable channel).",
"type": [
"string",
"null"
]
},
"value": {
"description": "selection_invalid: the offending value, when there is one string worth naming. variant_unsupported: the id of the subscription that ended. unowned_schema: the schema id no shard owns. write_not_authorized: the key of the refused write. unsupported_schema_version: the schemaVersion the peer does not speak.",
"type": [
"string",
"null"
]
},
"asOf": {
"description": "cursor_stale: the cursor's own minted asOf.",
"type": [
"integer",
"null"
],
"minimum": 0
},
"current": {
"description": "cursor_stale: the answering server's current asOf.",
"type": [
"integer",
"null"
],
"minimum": 0
}
}
},
"messageId": {
"description": "The messageId of the write (or other request) this error answers, echoed byte for byte. Null only when the frame was unreadable or carried no messageId.",
"type": [
"string",
"null"
],
"minLength": 1
}
}
}

View file

@ -0,0 +1,16 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://github.com/GameCult/CultLib/contracts/cultnet/interop/cultnet.write-admitted.schema.json",
"title": "CultNet Write Admitted",
"description": "The receipt for one admitted document put or delete (write-receipt.md). Sent only after the write's commit point, only to a connection that declared the write-receipt capability, once per messageId. Map-mode: every key is written, none is nil. A receipt never carries an error, and an error.v1 never carries admission.",
"type": "object",
"required": ["schemaVersion", "messageId", "operation", "schemaId", "recordKey"],
"additionalProperties": false,
"properties": {
"schemaVersion": { "const": "cultnet.write_admitted.v0" },
"messageId": { "description": "The messageId of the put or delete this admits, echoed byte for byte.", "type": "string", "minLength": 1 },
"operation": { "enum": ["put", "delete"] },
"schemaId": { "description": "The schemaId of the admitted write, echoed.", "type": "string", "minLength": 1 },
"recordKey": { "description": "The recordKey of the admitted write, echoed.", "type": "string", "minLength": 1 }
}
}

View file

@ -145,6 +145,22 @@
"current": null "current": null
} }
} }
},
{
"name": "message_invalid",
"messagePackHex": "85AD736368656D6156657273696F6EB063756C746E65742E6572726F722E7630A56572726F72D93E6D6573736167655F696E76616C69643A2052617720707574206D65737361676573206D75737420636F6E7461696E206120646F63756D656E74206D61702EAB726F7574696E6748696E74C0A4636F6465AF6D6573736167655F696E76616C6964A764657461696C7384A56669656C64A8646F63756D656E74A576616C7565C0A461734F66C0A763757272656E74C0",
"message": {
"schemaVersion": "cultnet.error.v0",
"error": "message_invalid: Raw put messages must contain a document map.",
"routingHint": null,
"code": "message_invalid",
"details": {
"field": "document",
"value": null,
"asOf": null,
"current": null
}
}
} }
] ]
} }

File diff suppressed because one or more lines are too long

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,89 @@
# CultNet write receipt (specification)
Status: specification. No runtime implements it yet; the conformance follow-ups below do.
Rules: Heimdall's ruling of 2026-11-08 (a write completes on its result frame; an ack or a return is not admission).
Files here: `cultnet.write-admitted.schema.json`, `cultnet.error.v1.schema.json`, `write-receipt-vectors.json`. They move to `contracts/cultnet/` when the first runtime implements them.
## The problem
A put or delete today has two answers on the wire: nothing (admitted, or the server never read it) and `cultnet.error.v0` (refused, with no id saying which write). A transport ack proves delivery. Clients that completed on the ack, or on a window with no refusal, reported refused writes as done (CultLib #55 to #58 bound this with a result window; this spec replaces the window with an answer).
## Frames
**`cultnet.write_admitted.v0`**: `{ schemaVersion, messageId, operation: "put"|"delete", schemaId, recordKey }`, every key written, none nil. Sent once a put or delete is admitted. The id echoes the write's `messageId` byte for byte; `operation`, `schemaId`, `recordKey` echo the write.
**A `write_admitted` `messageId` is valid only if it contains at least one Unicode letter (`\p{L}`) or number (`\p{N}`) and no Default_Ignorable_Code_Point (DerivedCoreProperties).** The compare is raw: the id is echoed byte for byte, so a decoder never trims or normalizes it (`" a "` is valid; U+2800 is not, and neither is `a` followed by U+200B). Unlike the Required fields rule, this is not a White_Space test. The letters, numbers and Default_Ignorable code points are those of **Unicode 16.0** in every runtime, taken from tables embedded in each runtime and generated by `scripts/gen-unicode-messageid-tables.py` from `DerivedGeneralCategory.txt` (Lu Ll Lt Lm Lo Nd Nl No) and `DerivedCoreProperties.txt`; no runtime asks its platform's Unicode data, so a runtime's own Unicode version cannot change a result. Bumping the version is one coordinated change: regenerate the five tables and update the vectors that pin the version in the same commit. The rule is on the decoded string, which must be well-formed Unicode: a lone surrogate refuses. A runtime whose MessagePack reader rejects invalid UTF-8 refuses the frame earlier; byte-level strictness is a property of the reader and is not required of a decoder. A frame with more than one `messageId` key is malformed: a decoder that can see the duplicate (the C# and Rust decoders) refuses it; a decoder that collapses the map (Python, TypeScript, Kotlin) judges the value it ends up with, the last key, and refuses if that value is invalid, so no decoder echoes an invalid id (vectors `admittedMessageIdDuplicates` declare each policy and every runner asserts its own). A decoder refuses an invalid id the way it refuses an empty or ill-typed one; the rule is for `write_admitted` only and `error.v1` is unchanged.
**Where a received `messageId` decides, and what that relies on.** Beyond echoing it, a received `messageId` is used in exactly two places. On a server, the remembered-write key `(actor, messageId)` with the content hash, the per-session limit and the replay of rule 5 are taken from the id of the write frame the server itself parsed. On a client, the write tracker attributes a `write_admitted` or `error.v1` to a write by equality with the id of a write that client itself sent (a frame naming an id not in flight settles nothing), and attributes an id-less error by the count of unanswered writes, not by any id. Both rely on the receiver being the only parser of that frame: a decoder's `messageId` on a frame that carries more than one `messageId` key is not trusted for any authorization, dedupe, replay or ordering decision, and the map-collapsing runtimes (Python, TypeScript, Kotlin, and C#'s typed deserializer on write frames) cannot see the duplicate. If a frame is ever parsed by two runtimes in a chain (a relay, proxy or forwarder that re-reads it), or a consumer needs `messageId` for authorization, dedupe, replay or ordering beyond echo and equality with a self-sent id, all runtimes must first refuse a duplicate `messageId` key (a spec change plus byte-level duplicate detection in the runtimes that decode to a map) before that use lands. Today Rust refuses a repeated key on write frames and on `write_admitted` (serde's `duplicate field messageId`, and `refuse_duplicate_keys` for the raw-list frames such as `document_put_raw`), though its `readable_message_id` names the first key when it names an unreadable frame; C# does not on write frames (a duplicate-key write is deduped on its last id, while the id it names in a refusal for an unreadable frame is read from the first key), and the Python server (`cultmesh_py/server.py`) only echoes the id while the TypeScript and Kotlin code matches it by equality with a request it sent.
**`cultnet.error.v1`**: `cultnet.error.v0` plus a required `messageId` (string or nil). The code enum is v0's plus `limit_reached` (see Remembered-write limits); `cultnet.error.v1.schema.json` is the authority for the closed set. Sent in place of `error.v0` to a receipt-capable connection. `messageId` is the id of the request the error answers; nil only when the frame was unreadable or carried no id.
Why a new error version and not an extra key on v0: the v0 schema is closed (`additionalProperties: false`) and the TypeScript decoder (Ajv over the closed schemas) rejects unknown keys and unknown `schemaVersion`s, and `parseCultNetMessage` throws on an unknown version; Rust's decoder errors on an unknown version too. Unknown-key tolerance of the Rust, Python and Kotlin error decoders was not checked, so none is assumed, so a surprise key or frame breaks an old client's whole receive path. Old clients are therefore never sent either new frame. This reverses the "old clients ignore it" assumption, which is false for today's decoders.
## Capability: opting in
A connection is receipt-capable when its `cultnet.hello.v0` (sent before its first write; the schema channel is reliable and ordered) lists both `cultnet.write_admitted.v0` and `cultnet.error.v1` in `supportedMessageVersions`. A server MUST NOT send either new frame to any other connection, and answers it exactly as before (no receipt, `error.v0`). A receipt-capable server lists both in its own hello, so a client can tell a legacy server from a receipt server that has not answered yet. A client that sends no hello gets legacy behaviour. A server advertises the two frames in its hello only if it implements the receipt rules end to end.
**The capability changes framing only; it never authorizes.** A hello authenticates nothing and grants nothing: who may write, and which keys, is decided by the server from the transport-authenticated actor exactly as without the capability. Declaring the capability never makes a refused write admitted, never widens an actor's authority, and never changes what a write does; it selects only which frame the answer arrives in.
## Server rules
1. **One answer.** For every put or delete on a receipt-capable connection the server sends exactly one of `write_admitted` or `error.v1`, carrying that write's `messageId`. Never both, never two different answers.
2. **After the commit point.** `write_admitted` is sent only after the write is committed by the same rule that makes it visible to readers. Receipt of the bytes, parse, or authorisation alone is not admission.
3. **Id on every refusal.** Every refusal of a write, whatever its code (`write_not_authorized`, `message_invalid`, `unowned_schema`, `unsupported_schema_version`, shard authority errors, ...), is an `error.v1` whose `messageId` is the write's id. When the frame is readable enough to find the id, it is found; `message_invalid` for a missing or ill-typed field still carries the id. Nil only when no id can be read.
4. **Batches.** There is no batch frame; a batch is N writes, each with its own `messageId`, each answered under rules 1 to 3. A failed batch names its member by that id. A server may not collapse a batch into one answer.
5. **Identity and idempotent resend.** A `messageId` is unique per authenticated actor and session: the client mints it, never reuses it for different content in that session, and a server never accepts the id of one actor as the id of another. The server's dedupe key is `(actor, messageId)`, where *actor* is the transport-authenticated identity, never a name the hello or the frame claims. Alongside the key it keeps the **content hash** of the write and the answer it gave. The content hash is SHA-256 of the MessagePack array `[operation, schemaId, recordKey, payload]`, where `payload` is the raw document payload as a MessagePack `bin` for a put and nil for a delete (the vectors give worked hashes). **Scope: at least the session.** The key lives for the life of the session it was made in, and a server may bound how many writes it remembers per session (see Remembered-write limits; a server never forgets an id it promised to answer). A server MAY keep the key longer for the same actor, up to `DEDUPE_RETENTION_MS` (10 minutes), but a client MUST NOT rely on it: **a reconnect is a new session, and a resend on the new session is a new write.**
- Same key, same content hash, answer was `write_admitted`: answer `write_admitted` again with the same body. The write is not applied twice.
- Same key, same content hash, answer was a refusal: the same refusal again.
- **Same key, different content hash: refuse `message_invalid` (details `field: "messageId"`) and never re-send the old receipt.** Old receipts are for the old content; showing one for new content would report an unapplied write as admitted. The original write's state is unchanged.
- **Id size.** A write's `messageId` is at most 256 bytes of UTF-8. A longer id is refused `message_invalid` (`details.field: "messageId"`, the refusal's `messageId` the id as received, text fixed per Fixed texts) BEFORE anything is remembered: no table entry, no digest, no budget charge. Such a refusal is therefore not replayed from the table; a resend is refused afresh, identically.
- **What is remembered.** Per entry a server keeps only fixed-size digests (SHA-256) of the `messageId` and of the content hash, plus the answer (`admitted`, or the refusal's code and details). Memory per entry is bounded independent of id length and payload size. Lookup compares digests.
- **Remembered-write limits.** The per-session table holds at most 1024 entries. A server MAY also enforce a server-wide memory budget across all sessions (reference: 32 MiB of remembered entries). A NEW write (a key not already in the table) that would exceed either bound is refused `limit_reached` for that write alone, `messageId` the write's id, `details` all null; the write is not applied and the session continues. A resend of a remembered key is still answered from the table.
- **Limit refusals are remembered.** A `limit_reached` refusal is remembered like any other: a resend of the same id and content hash replays `limit_reached`. No slot is freed mid-session, so an id refused at a bound never becomes admitted later in that session (a server at the cap keeps the refused digest pair in a bounded refusal set of at most 1024 per session, outside the entry budget; past it, a further new write is refused `limit_reached` without being remembered, and the session refuses every id it has not seen for the rest of its life, even if a server-wide budget has since freed, so a refusal that was not remembered is never followed by an admission of the same id). The client mints a new id on a new session.
- **Required fields.** A put or delete whose document or `schemaId` is null or missing is refused `message_invalid` with `details.field` `"document"` or `"schemaId"` (the first missing in that order) and fixed text; the write is not applied. A put or delete that is present but names nothing is not "missing": the write is refused `message_invalid` exactly as a frame that does not read, with `details.field` `"message"`, the fixed unreadable text and the write's id (vector `error_v1_empty_schema_id_is_unreadable_and_names_the_write`), and is never applied; a server must not hand it to its store. Names nothing means, on a put or delete, a `schemaId` or `recordKey` that is blank; on a put's document, also a `storedAt` that is blank, a `payload` of zero bytes (a MessagePack value is at least one byte, so an empty payload is no document), a `sourceRuntimeId`, `sourceAgentId` or `sourceRole` that is present and blank, and a `tags` entry that is blank. **Blank** means empty or made only of Unicode White_Space code points (the PropList property) and nothing else, in every runtime, whatever its platform's own whitespace test says. The set is exactly U+0009 to U+000D, U+0020, U+0085, U+00A0, U+1680, U+2000 to U+200A, U+2028, U+2029, U+202F, U+205F and U+3000. U+200B, U+200C, U+200D, U+2060, U+FEFF, U+180E and U+001C to U+001F are not White_Space (some platforms' whitespace tests accept U+001C to U+001F or U+180E, for example Java's `Character.isWhitespace` and older .NET, so a runtime must check the list, not its platform test): a string of them names something. The vectors list both sets (`whiteSpaceCodePoints`, `notWhiteSpaceCodePoints`) and a runtime checks every code point in every covered field. Absent or nil source ids and tags, and an empty `tags` list, are valid. The rule holds on every connection, receipt-capable or not (a legacy connection is refused with the `error.v0` of the same code, details and text); the vectors' `blankFields` section lists each case.
- **Fixed texts.** The `error` string of these refusals is fixed, so vectors can compare it: `message_invalid: messageId exceeds 256 bytes`, `message_invalid: document is required`, `message_invalid: schemaId is required`, `message_invalid: the message could not be read` (a frame that does not decode, or a write that names nothing; the text is the same for both and never carries a decoder's own wording), `message_invalid: receipt sessions require the reliable channel`, `limit_reached: the session's remembered-write limit is full`.
- **Reliable channel only.** A write is accepted on a receipt session only on a channel that is reliable and not sequenced (reliable ordered and reliable unordered both qualify; a sequenced channel drops superseded packets, so a write on it can vanish without trace). A write arriving on any other channel is refused `message_invalid` with `details.field: "channel"` (the transport profile's word for it) and fixed text. Receipts and refusals travel the reliable channel; a write that could be lost without trace cannot be answered once. The server refuses a sequenced write whenever it can read the write's id: from the delivered frame, and from a superseded packet that carries its whole frame. A write sent as several fragments whose frame is superseded is dropped whole and may go unanswered; the client's unconfirmed outcome covers it. A repeat of a packet the server already took, or a packet too far ahead to be taken, draws no further answer (rule 1).
- Past the session (or a server's longer retention, if it keeps one) a resend is a new write.
6. **Forwarded writes.** A server that forwards a non-primary write answers the original peer only with what the primary answered: the primary's receipt becomes `write_admitted` with the original id, its refusal becomes `error.v1` with the original id. If the forward gets no answer the server relays nothing: no receipt, no error, no synthesized refusal. The client's own rule below then reports the write as unconfirmed. A forwarder never turns an absent or ambiguous primary answer into an admission.
7. **Looks alike.** An admitted and a refused write differ in `schemaVersion`; a receipt has no `error`, an `error.v1` has no admission. A transport ack is neither.
## Client rules: three outcomes
A write settles as exactly one of:
- **admitted**: a `write_admitted` whose `messageId` is the write's.
- **refused**: an `error.v1` whose `messageId` is the write's, carrying its `code` and `details` as the typed error.
- **unconfirmed**: nothing settled it. Reasons: `window_elapsed` (the result window ran from the *send* and passed with no answer; see Result window), `session_ended` (the peer closed the session, or it ended, before the write was answered; a write registered after the session ended settles this way when it is sent) and `local_send_failed` (the client's own send of this write failed). A local send failure is never reported as refused: no server answer exists, and the write may or may not have left the machine. Unconfirmed is not success and not refusal. APIs that return a value return the tri-state; APIs that throw throw a distinct refused error and a distinct unconfirmed error and return normally only for admitted. A caller that needs admission treats unconfirmed as not admitted. Resend with the same `messageId` and unchanged content, **on the same session**, is safe against a receipt-capable server (rule 5). After a reconnect the resend is a new write, and a legacy server may apply a resend twice: the caller resends only writes that are idempotent by record key (a put of the same content to the same key, a delete of a key), and reads the record back before resending any other.
**Result window.** Two named constants (also in the vectors): `RECEIPT_RESULT_WINDOW_MS = 5000`, the default against a receipt-capable server (its hello lists both frames); `LEGACY_RESULT_WINDOW_MS = 500`, used only as the fallback against a server whose hello does not list them (or that sent none). The window runs from the send. It ends when the write's age reaches it: an answer arriving at exactly that instant is late. A receipt-capable server that has not answered within its window is reported as unconfirmed like any other; clients log it as a server that missed its window.
**Late answers.** An answer for a write that already settled `unconfirmed` does not change what the caller was told, but the client records it on the write: a late `write_admitted` makes the write's recorded state `admitted_late`, a late refusal makes it `refused_late`. The write handle exposes that state, and a caller that got `unconfirmed` MUST consult it (and the record) before resending: it must not blind-resend a write that was admitted late. A resend on the same session with the same id and content is harmless on a receipt-capable server (rule 5); across a reconnect the caller follows the idempotence rule above.
**Legacy answers.** An `error.v0`, or an `error.v1` with nil `messageId`, names no write. It is attributed to a write only when exactly one write on the connection is unanswered; it then refuses that write. With two or more unanswered writes it settles none of them and each ends unconfirmed at its window. A frame naming a `messageId` that is not in flight settles nothing.
**Batches can commit partially.** A batch is N independent writes with no atomicity: some members may be admitted while others are refused or unanswered, and nothing is rolled back. The client reports each member's outcome and a batch result is only their set: admitted if every member is admitted, refused if any member is refused (naming the members), otherwise unconfirmed (naming the unanswered members). The caller owns retry: it resends exactly the members that are not admitted, with their original ids.
## Changes to what #55 to #58 shipped
Those clients complete *successfully* when the window ends with no refusal. Under this spec that case is `unconfirmed`. The TypeScript one-shot writer (#62) already reports it so: it resolves `admitted` only on a `write_admitted` carrying the put's id, and no other frame, a put echoed under the same id included, ever counts as admission. A caller whose server cannot answer a write yet (Odin, until it is receipt-capable) opts into the explicit send-only mode per call (`sendOnly: true`), which resolves the distinct outcome `sent` once the transport has acknowledged the put and never `admitted`. The Idunn presence publisher is send-only by its own function (`sendPresenceDocument`, not the `sendOnly` option): the same `sent` result, plus a rejection if a refusal naming the put (or naming nothing) arrives by the transport ack. Receipts and send-only are both opt-in, neither is the default. The C# and Rust clients of #55 to #58 keep the window-success until their conformance work lands, and callers of those must not read it as admission.
## Vectors
`write-receipt-vectors.json`: `constants` (the named windows and retention), `contentHash` (worked dedupe hashes), `dedupe` (server replay cases for rule 5), `wire` (MessagePack hex plus the decoded message for each new frame; the bytes are spec-authored with msgpack-python and every runtime's first conformance PR re-captures them with its own encoder and must match), `schemaInvalid` (frames that must be refused by a validator, each with why), and `blankFields` (server input cases for the Required fields rule: which document fields make a put unreadable, which are accepted), `admittedMessageId` (the `write_admitted` id rule above: code-point lists, each marked refuse or accept; a lone surrogate, and a frame that repeats the key, are carried as whole frames in `wireHex`, because a lone surrogate is not valid UTF-8 and only some runtimes' strings can hold it; `write-admitted-message-id-ranges.json` lists every range of the embedded tables, and each runtime probes the first and last point of each range and the points either side), `scenarios` (event replays with the outcome each write must settle as; the clock is abstract; each case names its window constant). A conformant client passes every scenario; a conformant server's answers for the same inputs satisfy server rules 1 to 7.
## Conformance status
| Server class | Advertises both frames | Server-wide memory budget (MAY, Remembered-write limits) |
|---|---|---|
| `CultNetDatabaseServer` | yes | not implemented: only the per-session limit of 1024 applies |
| Shard write forwarder | yes | not implemented: only the per-session limit of 1024 applies |
| Rust RUDP document server | yes | implemented (32 MiB default, `max_remembered_write_bytes`) |
| C# RUDP schema server (`CultNetRudpSchemaServer`, `src/GameCult.Networking/CultNetRudpSchemaServer.cs`) | no | n/a |
Advertising is per server class, not per transport. A class advertises both frames in its hello only if it implements the receipt rules end to end; the C# RUDP schema server stays "no" until it answers writes with receipts.
Cross-runtime check: `packages/cultmesh-ts/test/interop/write-receipt-interop.test.ts` (`npm run test:interop` in `packages/cultmesh-ts`) runs the TypeScript client (`WriteTracker`, the decoder, the one-shot writer) against the real Rust RUDP document server and the real `CultNetDatabaseServer` over loopback, one scenario set against both. Known gap it records as a todo: the Rust RUDP document server refuses every delete (`write_not_authorized: the server does not take deletes`), where this spec has an admitted delete receipted like a put.
## Conformance follow-ups (not in this PR)
C# server and client, Rust, TypeScript, Python, Kotlin: emit and parse the two frames; hello capability; tri-state client outcome; replace the #55 to #58 window-success; re-capture the wire vectors in each runtime. Server write paths include the shard write forwarder (rule 6) and the Odin catalog.

View file

@ -441,8 +441,8 @@ consumer that waits wants delivered, which is now just Acknowledged.
| `cultmesh-rs/src/lib.rs:781-838` `request_raw_snapshot_from_rudp_catalog` (Idunn `src/drivers.rs:5527` uses it) | a hand-rolled loop until `Acknowledged`, then the response | Cut 4: `wait_acknowledged` | | `cultmesh-rs/src/lib.rs:781-838` `request_raw_snapshot_from_rudp_catalog` (Idunn `src/drivers.rs:5527` uses it) | a hand-rolled loop until `Acknowledged`, then the response | Cut 4: `wait_acknowledged` |
| `cultmesh-rs/src/lib.rs:870-900` `publish_cultnet_message_to_rudp_catalog` (Odin `odin-daemon`) | a hand-rolled loop until `Acknowledged`, then disconnect. The doc at `:869` claims "application admission" | Cut 4: `wait_acknowledged`; the doc says delivered to the document server's sink (`rudp_document_server.rs:343-372`) | | `cultmesh-rs/src/lib.rs:870-900` `publish_cultnet_message_to_rudp_catalog` (Odin `odin-daemon`) | a hand-rolled loop until `Acknowledged`, then disconnect. The doc at `:869` claims "application admission" | Cut 4: `wait_acknowledged`; the doc says delivered to the document server's sink (`rudp_document_server.rs:343-372`) |
| `cultmesh-rs/src/lib.rs:918-938` `publish_cultnet_messages_to_rudp_catalog` (Ghostlight `mesh.rs:323`) | `flush_reliable`, then drop. **Several ordered frames, then close** | None; flush now means it | | `cultmesh-rs/src/lib.rs:918-938` `publish_cultnet_messages_to_rudp_catalog` (Ghostlight `mesh.rs:323`) | `flush_reliable`, then drop. **Several ordered frames, then close** | None; flush now means it |
| `cultmesh-ts/src/index.ts:6070-6098` `publishRudpDocumentOnce` (Heimdall, StreamPixels, Stonks, weksa, Bifrost, VoidBot, Hermodr) | `peer.flush` (`cultnet-ts/src/peer.ts:109-110`), then close | None. Cut 5 moves its connect-and-wait helper (`index.ts:6603-6650`) into cultnet-ts | | `cultmesh-ts/src/index.ts:6211-6303` `publishRudpDocumentOnce` (Heimdall, StreamPixels, Stonks, weksa, Bifrost, VoidBot, Hermodr) | `peer.flush` (`cultnet-ts/src/peer.ts:109-110`), then close | None. Cut 5 moves its connect-and-wait helper (`index.ts:6603-6650`) into cultnet-ts |
| `cultnet-ts/src/idunn-odin-presence-publisher.ts:89-215` `publishDocument`, behind `createIdunnRuntimePresencePublisher`. **StreamPixels' service runs this**: `apps/service/scripts/idunn-entry.mjs` runs `deployment/idunn/runtime-presence.mjs`, which calls `createIdunnRuntimePresencePublisher` (`:46`) from its vendored CultLib `542ddd3` (StreamPixels `a49f7d8`; pass 2's `30ee8b9` is stale), where it lives under its older name `cultnet-ts/src/idunn-runtime-authority.ts:206` | a hand-rolled session loop (`:186-208`) until `outstandingReliablePacketCount === 0`. It does resend (`:205`). It drops `readyToSend` (`:198`), so promoted queued packets wait for the resend timer. It pins `initialSequence: 1` (`:103`) and never sends a Disconnect, so Odin's hub keeps each publish's peer until it times out (each publish binds a new port, so each is a new peer) | Cut 5: the socket transport, `connectAndWait`, `send("schema")`, `flush`, `close` (which sends the Disconnect, `rudp.ts:956-969`) | | `cultnet-ts/src/idunn-odin-presence-publisher.ts:97-136` `sendPresenceDocument` (formerly `publishDocument`), behind `createIdunnRuntimePresencePublisher`. **StreamPixels' service runs this**: `apps/service/scripts/idunn-entry.mjs` runs `deployment/idunn/runtime-presence.mjs`, which calls `createIdunnRuntimePresencePublisher` (`:46`) from its vendored CultLib `542ddd3` (StreamPixels `a49f7d8`; pass 2's `30ee8b9` is stale), where it lives under its older name `cultnet-ts/src/idunn-runtime-authority.ts:206` | a hand-rolled session loop (`:186-208`) until `outstandingReliablePacketCount === 0`. It does resend (`:205`). It drops `readyToSend` (`:198`), so promoted queued packets wait for the resend timer. It pins `initialSequence: 1` (`:103`) and never sends a Disconnect, so Odin's hub keeps each publish's peer until it times out (each publish binds a new port, so each is a new peer) | Cut 5: the socket transport, `connectAndWait`, `send("schema")`, `flush`, `close` (which sends the Disconnect, `rudp.ts:956-969`) |
| `cultnet-ts/src/signed-daemon-health.ts:104-162` `publishSignedDaemonHealth` (Hermodr, Odin `idunn-rudp.cjs`) | a hand-rolled loop: sends the packets `sendMany` admitted, waits up to 500 ms for the **first** Ack packet of any content (`:146-153`), swallows the timeout, closes. Resends run inside that wait (`:303`) but promoted queued packets are never sent | Cut 5, as above. Today a write not acked within 500 ms, or any payload over 32 packets, is reported published | | `cultnet-ts/src/signed-daemon-health.ts:104-162` `publishSignedDaemonHealth` (Hermodr, Odin `idunn-rudp.cjs`) | a hand-rolled loop: sends the packets `sendMany` admitted, waits up to 500 ms for the **first** Ack packet of any content (`:146-153`), swallows the timeout, closes. Resends run inside that wait (`:303`) but promoted queued packets are never sent | Cut 5, as above. Today a write not acked within 500 ms, or any payload over 32 packets, is reported published |
| `cultnet-ts/src/operation-service.ts` | the operation response | none (the response is the proof) | | `cultnet-ts/src/operation-service.ts` | the operation response | none (the response is the proof) |
| C# `FlushReliable` callers: Gjallar `Program.cs:3055-3080` (two ordered frames, flush, dispose) | flush | None | | C# `FlushReliable` callers: Gjallar `Program.cs:3055-3080` (two ordered frames, flush, dispose) | flush | None |

View file

@ -701,6 +701,7 @@ checked by the operator in the editor.
- Run `MintElementIds` once over `GameData/Aetheria.cc`. - Run `MintElementIds` once over `GameData/Aetheria.cc`.
- AetherDb migrations skip variants or use the variant API (Q1a). - AetherDb migrations skip variants or use the variant API (Q1a).
- Re-author the families listed in section 6. - Re-author the families listed in section 6.
- Hull families live in each ship's own `.cc`, not in `GameData/Aetheria.cc`, because hulls are mod-owned packages (Nibu's ruling about her data, 2026-11-11, #minds).
### C5. CultNet deltas (C# reference) ### C5. CultNet deltas (C# reference)
@ -1608,7 +1609,7 @@ The one door per runtime, and who walks through it:
- **Shared paths.** cultnet-ts reaches the header decision only through `readSingleFileStore` (AUTH:4, :321) and checks only record count, schema id and type. cultnet-ts therefore needs a cultcache-ts release that exports it, which is the release that ships with this change; the manifests move to that version then (c0-ships-with-c2a), not before. - **Shared paths.** cultnet-ts reaches the header decision only through `readSingleFileStore` (AUTH:4, :321) and checks only record count, schema id and type. cultnet-ts therefore needs a cultcache-ts release that exports it, which is the release that ships with this change; the manifests move to that version then (c0-ships-with-c2a), not before.
- **Cut line.** `merge-r1` h3 (Rust) and h4 (TS, cultnet-ts); spec `cut-merge-r1.r3` authority map. - **Cut line.** `merge-r1` h3 (Rust) and h4 (TS, cultnet-ts); spec `cut-merge-r1.r3` authority map.
- **Pinned by.** `tests/vectors/document-variants-c2a/readability/manifest.txt`, walked by `StoreReadabilityTests.OpenReadsExactlyTheFilesTheVectorsSayItReads` (C#), `CultMeshSingleFileReadabilityTests.AWriteReplacesAFileExactlyWhenItReads` (Mesh), `a_file_is_replaced_exactly_when_it_reads` (RS:5132), "SingleFileMessagePackBackingStore replaces a file exactly when it reads" (TS test :2321) and `test_single_file_replaces_a_file_exactly_when_it_reads` (Python). AUTH cases: `packages/cultnet-ts/test/idunn-runtime-authority.test.ts`. Declared divergences in the manifest: `legacy-envelopes.bin` (C# refuses; the others read and replace it) and `variant-slot-v3.bin` (C# reads; the others refuse). - **Pinned by.** `tests/vectors/document-variants-c2a/readability/manifest.txt`, walked by `StoreReadabilityTests.OpenReadsExactlyTheFilesTheVectorsSayItReads` (C#), `CultMeshSingleFileReadabilityTests.AWriteReplacesAFileExactlyWhenItReads` (Mesh), `a_file_is_replaced_exactly_when_it_reads` (RS:5132), "SingleFileMessagePackBackingStore replaces a file exactly when it reads" (TS test :2321) and `test_single_file_replaces_a_file_exactly_when_it_reads` (Python). AUTH cases: `packages/cultnet-ts/test/idunn-runtime-authority.test.ts`. Declared divergences in the manifest: `legacy-envelopes.bin` (C# refuses; the others read and replace it) and `variant-slot-v3.bin` (C# reads; the others refuse).
- **Known gaps.** `variants:follow_up:c0-bad-canonical-in-manifest` (`current-catalog-bad-canonical.msgpack` is not in the manifest; TS reads it, Rust refuses). `variants:follow_up:legacy-vectors-all-runtimes` (the three `legacy-catalog-*` c0 vectors have a verdict only in C# tests; all four runtimes refuse them, probed). `variants:follow_up:range-check-in-ci` (`npm run test:pack:ts` builds every admitted tag and checks exports, but `.github/workflows/publish-packages.yml` does not run it). - **Known gaps.** `variants:follow_up:range-check-in-ci` (`npm run test:pack:ts` builds every admitted tag and checks exports, but `.github/workflows/publish-packages.yml` does not run it).
### B. Existence and links (what "nothing at the path" means) ### B. Existence and links (what "nothing at the path" means)
@ -1639,7 +1640,7 @@ The one door per runtime, and who walks through it:
- **Shared paths.** Open, commit, Mesh read and write. - **Shared paths.** Open, commit, Mesh read and write.
- **Cut line.** `mesh-single-file-reader` h1 (the C# clause). Rust, TS and Python were correct by construction of their reads. - **Cut line.** `mesh-single-file-reader` h1 (the C# clause). Rust, TS and Python were correct by construction of their reads.
- **Pinned by.** C#: `ADirectoryAtTheStorePathIsAnIoErrorOnOpenAndOnCommitNotAnEmptyStore` (`StoreReadabilityTests.cs:236`). Mesh: `ADirectoryAtThePathIsAnIoErrorOnReadAndWriteAndIsLeftAlone` (:149). Rust: `a_write_over_a_directory_stops_at_the_read_and_leaves_no_staging_file` RS:5726. - **Pinned by.** C#: `ADirectoryAtTheStorePathIsAnIoErrorOnOpenAndOnCommitNotAnEmptyStore` (`StoreReadabilityTests.cs:236`). Mesh: `ADirectoryAtThePathIsAnIoErrorOnReadAndWriteAndIsLeftAlone` (:149). Rust: `a_write_over_a_directory_stops_at_the_read_and_leaves_no_staging_file` RS:5726.
- **Known gaps.** TS and Python have no test for a directory at the store path. No follow-up is registered; the natural home is `write-set-vectors` with the other readability pins. - **Known gaps.** None open. The `@directory` manifest row is walked by all four runtimes: every operation including delete is refused and leaves the directory untouched; Rust pins an I/O failure rather than a typed unreadable store.
### E. Lock ### E. Lock

View file

@ -4,6 +4,8 @@ Data for the per-consumer sweep in docs/document-variants-cut.md, C2. Every obje
Method: a static scan of `F:\Projects\<consumer>` C# files for types marked `[CultDocument(`, walked through keyed members, base classes, dictionary values and `[Union]` subtypes. Read-only; no consumer was edited. Worktree copies (`Aetheria-catalog`, `CultLib-*`, and so on) are excluded. Struct elements are values and need no id. Non-C# consumers carry the id as an ordinary member and are not listed. The scan is text-based: a document registered without a literal `[CultDocument(` attribute, or a type whose members lack an integer `[Key]`, is missed, so the sweep pass should confirm each repo by registering its types once. Method: a static scan of `F:\Projects\<consumer>` C# files for types marked `[CultDocument(`, walked through keyed members, base classes, dictionary values and `[Union]` subtypes. Read-only; no consumer was edited. Worktree copies (`Aetheria-catalog`, `CultLib-*`, and so on) are excluded. Struct elements are values and need no id. Non-C# consumers carry the id as an ordinary member and are not listed. The scan is text-based: a document registered without a literal `[CultDocument(` attribute, or a type whose members lack an integer `[Key]`, is missed, so the sweep pass should confirm each repo by registering its types once.
A type that already has a readable string id keeps it and adds a trailing keyed string member marked `[CultElementId(nameof(ExistingId))]`; see the Element ids section of `src/GameCult.Caching/README.md`.
Lines: `Type file:line (via Owner.Member)`, paths relative to the consumer root. Lines: `Type file:line (via Owner.Member)`, paths relative to the consumer root.

View file

@ -25,5 +25,13 @@ closure through UPM rather than NuGet restore.
Install the Unity package from the immutable release tag: Install the Unity package from the immutable release tag:
```text ```text
https://github.com/GameCult/CultLib.git?path=/unity/org.gamecult.cultlib#cultlib-unity-v2.0.0 https://github.com/GameCult/CultLib.git?path=/unity/org.gamecult.cultlib#cultlib-unity-v3.0.0
``` ```
## Rolling back across the store format
A consumer that upgrades and writes a store holding an element id leaves a
`cultcache.store.v3` single-file store (`v5` directory store). Refusal by name depends on the store kind and the pin. A **directory** store (`v5`) is refused by every Unity pin from `cultlib-unity-v1.0.57` and `caching-unity-v1.2.0` on (the earliest tags of each line whose `DirectoryMessagePackBackingStore` reads only the `v4` stamp: "only ... is readable"). A **single-file** store (`v3`) is refused only from `cultlib-unity-v2.0.0` and `caching-unity-v2.0.0`: `cultlib-unity-v1.0.61` and `caching-unity-v1.4.0` read the header string and never compare it. The NuGet line (`cultlib-v1.0.32`, the last `cultlib-v` tag) refuses neither: its directory store writes `cultcache.store.v1.directory` and never checks the stamp on read. Rolling back a single-file store to any of those pins reads it silently, drops the ids, and can rewrite the elements without them. That is the dangerous case. There is
no general carry-back; restore from a copy taken before upgrading, and upgrade
readers before writers. Detail: the Store format and rollback section of
`src/GameCult.Caching/README.md`.

View file

@ -16,7 +16,7 @@ Cross-runtime parity has three layers:
Every store reader refuses, by name, a header or record slot it does not Every store reader refuses, by name, a header or record slot it does not
understand (vectors: `tests/vectors/document-variants-c0`). understand (vectors: `tests/vectors/document-variants-c0`).
Element ids: a store that can hold them declares `cultcache.store.v3`; TypeScript, Element ids: a store that can hold them declares `cultcache.store.v3`; TypeScript,
Python and Rust read it and keep the header, and readers older than ids refuse it Python and Rust read it and keep the header, and the C# runtime decides the mark from content, and these runtimes never drop it. A C# reader from `cultlib-unity-v2.0.0` on refuses a v3 single-file store it cannot read (earlier pins do not)
(vectors: `tests/vectors/document-variants-c2a`). (vectors: `tests/vectors/document-variants-c2a`).
Document variants: C# resolves them (`cultcache.store.v2`); every other runtime, Document variants: C# resolves them (`cultcache.store.v2`); every other runtime,
CultMesh single-file reads, the directory store and CultNet refuse a variant by name CultMesh single-file reads, the directory store and CultNet refuse a variant by name

View file

@ -44,7 +44,7 @@ using MessagePack;
public sealed class Item public sealed class Item
{ {
[Key(0)] [CultName] public string Name = string.Empty; [Key(0)] [CultName] public string Name = string.Empty;
[Key(1)] [CultIndex] public string Category = string.Empty; [Key(1)] public string Category = string.Empty;
[Key(2)] public int Value; [Key(2)] public int Value;
} }
@ -94,7 +94,6 @@ const itemDocument = defineDocumentType({
schemaVersion: "gamecult.first_call_item.v1", schemaVersion: "gamecult.first_call_item.v1",
schema: z.object({ name: z.string(), category: z.string(), value: z.number().int() }), schema: z.object({ name: z.string(), category: z.string(), value: z.number().int() }),
name: "name", name: "name",
indexes: { category: "category" },
}); });
const registry = defineDocumentRegistry(itemDocument); const registry = defineDocumentRegistry(itemDocument);
@ -168,8 +167,8 @@ fn first_call() -> anyhow::Result<()> {
## Python first call ## Python first call
A document is a definition with an encoder and decoder; `name=` and `indexes=` register the name lookup A document is a definition with an encoder and decoder; `name=` registers the name lookup, `indexes=` unique lookups and
and indexes. The source is `roles=` many-row selection aliases. The source is
[`packages/cultcache-py/tests/test_first_call.py`](../../packages/cultcache-py/tests/test_first_call.py), [`packages/cultcache-py/tests/test_first_call.py`](../../packages/cultcache-py/tests/test_first_call.py),
run by `python -m unittest tests.test_first_call` in `packages/cultcache-py` with `src` on `PYTHONPATH` run by `python -m unittest tests.test_first_call` in `packages/cultcache-py` with `src` on `PYTHONPATH`
and `msgpack` installed (the store needs it). A second test fails if the block below stops matching the and `msgpack` installed (the store needs it). A second test fails if the block below stops matching the
@ -195,7 +194,6 @@ item_doc = define_document_type(
encode=lambda item: asdict(item), encode=lambda item: asdict(item),
decode=lambda raw: Item(**raw), decode=lambda raw: Item(**raw),
name="name", name="name",
indexes={"category": "category"},
) )

View file

@ -33,9 +33,9 @@ Recovered from the legacy "Database Tools" window (Aetheria `d3db1730`,
**The attribute is not `[CultIndex]`.** `[CultIndex]` means unique (operator, 2026-09-30; **The attribute is not `[CultIndex]`.** `[CultIndex]` means unique (operator, 2026-09-30;
`CultDocumentContracts.cs:34`). Grouping is `CultInspectorGroupByAttribute`, in `CultDocumentContracts.cs:34`). Grouping is `CultInspectorGroupByAttribute`, in
`CultInspectorAttributes.cs`, read only by `CultInspectorModel`. It changes no schema id, `CultInspectorAttributes.cs`, read only by `CultInspectorModel`. It changes no schema id,
catalog entry, index, or store byte. The index cut's deletion of grouping-only `[CultIndex]` catalog entry, index, or store byte. The index cut converts the grouping and selection `[CultIndex]`
declarations (CultMesh, Geometry) adds no `[CultInspectorGroupBy]` in their place: those are declarations (CultMesh, Geometry) to `[CultRole]` rather than deleting them, and adds no
service stores, not Studio-authored catalogs. `[CultInspectorGroupBy]` in their place: those are service stores, not Studio-authored catalogs.
### Invariants ### Invariants

View file

@ -33,13 +33,16 @@ Probes: scratch tests in this worktree, removed. C# filtered runs on Starfire (o
| Probe | C# | TypeScript | Rust | Python | | Probe | C# | TypeScript | Rust | Python |
|---|---|---|---|---| |---|---|---|---|---|
| P1 cache {A,B}: B at k, then A at k | **C**: B's record replaced, k now A's id | C, and **both** records live at k | C, both live at k | C, both live at k | | P1 cache {A,B}: B at k, then A at k | **C**: B's record replaced, k now A's id | R `SchemaConflictError` (cultcache-ts: one record per key) | R `SchemaConflictError` naming k, A not written (leaf cut) | R `SchemaConflictError`, A's write refused, B kept (was C, both live) |
| P2 cache {A} only, store holds B at k, write A at k | R `CultSchemaConflictException` (B is foreign, carried) | R at load: "No schema is registered for persisted schema id" | R at load, same | R at load: "Unknown persisted document type" | | P2 cache {A} only, store holds B at k, write A at k | R `CultSchemaConflictException` (B is foreign, carried) | R `SchemaConflictError` at write (foreign loads, carried; cultcache-ts) | R `SchemaConflictError` (B loads as foreign, carried; leaf cut) | R `SchemaConflictError` (B is foreign, carried; the load admits it; was R at load) |
| P3 cacheless `WriteDocument<A>` over B, registry null / {A} / {A,B} | R / R / R | no cacheless write | no cacheless write | no cacheless write | | P3 cacheless `WriteDocument<A>` over B, registry null / {A} / {A,B} | R / R / R | no cacheless write | no cacheless write | no cacheless write |
| P4 cache {A,B}: remove k and upsert A at k in one batch | C (k is A's) | n/a | n/a | n/a | | P4 cache {A,B}: remove k and upsert A at k in one batch | C (k is A's) | n/a | n/a | n/a |
| P5 cache {A,B}: remove, then add in a second commit | C | n/a | n/a | n/a | | P5 cache {A,B}: remove, then add in a second commit | C | n/a | n/a | n/a |
| P6 N2 over an N1 record (undeclared name drift), cache {N2} / cacheless reg {N2} | C restamped / C restamped | not probed | not probed | C (cache) | | P6 N2 over an N1 record (undeclared name drift), cache {N2} / cacheless reg {N2} | C restamped / C restamped | not probed | not probed | C (cache) |
The C#-written probe the leaves are held to is `tests/vectors/key-identity` (12 rows: retype, foreign carry, key in two stores), walked by
`KeyIdentityVectorTests`; each leaf cut corrects its own column above.
What this establishes: What this establishes:
- C#: one record per key. The only path that retypes is the cache arm (P1): `CacheBackingStore.RefuseUndeclared` - C#: one record per key. The only path that retypes is the cache arm (P1): `CacheBackingStore.RefuseUndeclared`
@ -52,12 +55,22 @@ What this establishes:
- Retype as remove plus add already works in the cache (P4, P5); the cut adds no new primitive. - Retype as remove plus add already works in the cache (P4, P5); the cut adds no new primitive.
- Directory store: its own judge (`DirectoryMessagePackBackingStore.cs:287-306`) asks `Registry.Owns` for every key, - Directory store: its own judge (`DirectoryMessagePackBackingStore.cs:287-306`) asks `Registry.Owns` for every key,
upserts included. It must take the same per-key rule. upserts included. It must take the same per-key rule.
- TypeScript, Rust, Python: records are keyed by `(type, key)` (Rust `entries: BTreeMap<(String, String), _>`, - TypeScript and Python: records are keyed by `(type, key)` (TS `#entryId(definition.type, key)`, `cult-cache.ts:323`;
`lib.rs:2659`; TS `#entryId(definition.type, key)`, `cult-cache.ts:323`; Python sorts and stores by `(type, key)`). Python sorts and stores by `(type, key)`). A write of A at k never meets B's record: both live at k (P1). Neither carries
A write of A at k never meets B's record: both live at k (P1). None carries a foreign record; a store holding an a foreign record; a store holding an unregistered type does not load (P2). So neither has a "write meets another type's
unregistered type does not load (P2). So no leaf has a "write meets another type's record" path to fix. Bringing the record" path to fix: bringing the ruling to them first needs C#'s key identity (one record per key) and C#'s foreign
ruling to the leaves first needs C#'s key identity (one record per key) and C#'s foreign carry, which are parity changes carry, parity changes of their store model, not a write-ownership rule (follow-up `variants:follow_up:leaf-key-identity-parity`).
of the leaves' store model, not a write-ownership rule. Owned by the new follow-up `variants:follow_up:leaf-key-identity-parity`. - Rust is that follow-up's first leaf cut (`cut-leaf-key-identity-rs`). The cache keeps its `(type, key)` map; one judge
(`refuse_retype`) refuses another registered type at a key the cache holds, or any write at a key a store carries
foreign, on every write path, pull and SoA load, comparing types, not the ids envelopes claim. At a key the cache does not
hold (a fresh key, or one another writer filled after this cache's pull) the single-file store's commit guard judges by
the stored and claimed schema ids, so an envelope claiming another registered type's id is refused earlier, at every
envelope admission (`refuse_claimed_id`, the pull's own `resolve_registered_type` lookup); an id naming no registered type
passes, trusted as claimed: an envelope under such an id can replace another writer's record stored under it at a key
this cache does not hold, or be served as another type by a reader that registers the type the id names (Soul g8a, g8c,
open: which ids a Rust type owns is not declared to CultCache). Store-level writers called directly (`push`,
`insert_entry_if_absent`, compare-and-swap) trust the claimed id.
The cache carries an unregistered type's records as foreign and walks the vector's 12 rows.
- Interop (unproven): a Python store holding A and B at k fails C# decode first on payload encoding (Python's - Interop (unproven): a Python store holding A and B at k fails C# decode first on payload encoding (Python's
lambda-defined document wrote JSON payloads), so what C# does with two records at one key was not reached. lambda-defined document wrote JSON payloads), so what C# does with two records at one key was not reached.

View file

@ -9,7 +9,16 @@
### Breaking ### Breaking
- A single-file store written as `cultcache.store.v3` is read and written: a record that holds an element id (an owned list element with an `[CultElementId]` member, carried through every write) marks the store v3, and a v3 store is carried through a rewrite with its ids. A runtime from before this change refuses a v3 store by name. - A cache holds one record per key whatever its type (`variants:ruling:cache-refuses-retype`). A `put`, `put_envelope`, `put_envelopes` or `put_global` of a second type under a key the cache holds, or under a key a store carries foreign, is refused with `SchemaConflictError` naming the key, even with no store, and changes nothing; retype a key by deleting it, then putting the new type. A write may replace another registered type's record only when the written type owns its schema id (declares it or lists it compatible) and both types share one home store. A load that reads one key from two stores, whatever the types, is refused with `CultCacheError` naming the key and both types, and admits nothing. A store that holds a record no registered type claims now loads and is carried: the cache does not serve it, the store keeps its bytes through every write, `delete` or `put` at its key is refused with `SchemaConflictError`, and registering a type that declares its schema id claims it at the next load. It raised `CultCacheError` "Unknown persisted document type". Two global types share the key `__global__`, so a second global type in one cache is now refused, and a store that already holds two global types at `__global__` (a JSONL store written by caches that never pulled could produce one) no longer loads: the pull raises `CultCacheError` and admits nothing. C# refuses a second persisted global on pull the same way (`cultcache-store-composition.md`, "Globals never invented"); repair such a file by deleting one of the two records with the runtime that wrote it.
- Added `foreign_records()` and the frozen `ForeignRecord(key, schema_id, schema_name, stored_at)`, sorted by key in UTF-16 code-unit order. `stores.retyped` and `stores.written_owns` lose their underscore: the cache judge and the single-file store's commit guard ask one predicate.
- A single-file store holding two records under one key is unreadable: the v1 reader and the legacy envelope-array reader refuse it with `StoreUnreadableError` (a key is compared by its exact code points, no case folding or normalisation). A catalog slot whose canonical schema is not a string is refused the same way. A store that loaded before can now be refused: repair it with the runtime that wrote the duplicate.
- A `put` or `push_all` that would land a second type under a key the store holds is refused with `SchemaConflictError`, and the file is left as it was. Retype a key by deleting it, then putting the new type. A write replaces the record at a key only when the written type owns it: its catalog entry declares the stored schema id (its own, or one it lists compatible), so a renamed type that lists the old id compatible may replace the record, or it has the stored record's schema name. This runtime does not compare shapes, so a same-name type is never refused for dropping persisted slots as C# refuses it.
- Writes of the same type replace by key: the last record wins, in a batch as in the file.
- `indexes` are unique, as C# `[CultIndex]`: a write, load or `register_index` that leaves two records of a type holding one non-blank value under one alias is refused (`CultCacheError`) naming both keys and the alias, and writes nothing. Index text is the C# canonical text (`True`/`False`, integers as digits, floats as the exact expansion, blank values never indexed), so an empty or whitespace string is no longer a value. A new `roles` map names the many-row selection aliases (never indexed for lookup, same catalog `index_alias`). Classify every `indexes` key per the README before updating. The verdict is the contract, the text is informational.
- A single-file store written as `cultcache.store.v3` is read and written: a store the C# runtime marked v3 (it holds a record with an element id: an owned list element with a `[CultElementId]` member) keeps its header here, and is carried through every rewrite with its ids; this runtime never drops the mark. A runtime from before this change refuses a v3 store by name. Rolling back to an older release after a store has been written as v3 is one-way: there is no carry-back, so restore from a copy taken before upgrading.
- A write over a file the reader refuses (`StoreUnreadableError`, from `push`, `push_all` and `delete`) refuses and leaves the file as it is. A flush used to replace such a file. A write set cannot merge with a disk it cannot read, so the file is repaired or removed by its owner before the store writes again. - A write over a file the reader refuses (`StoreUnreadableError`, from `push`, `push_all` and `delete`) refuses and leaves the file as it is. A flush used to replace such a file. A write set cannot merge with a disk it cannot read, so the file is repaired or removed by its owner before the store writes again.
- Added `StoreUnreadableError` (a `ValueError`) for a store file this runtime cannot read, and `SchemaConflictError` (a `ValueError`) for a - Added `StoreUnreadableError` (a `ValueError`) for a store file this runtime cannot read, and `SchemaConflictError` (a `ValueError`) for a
write the catalog cannot describe. A store is exactly three slots; a schema is never recovered from a record's payload. write the catalog cannot describe. A store is exactly three slots; a schema is never recovered from a record's payload.

View file

@ -19,7 +19,11 @@ discovery, sessions, local server) build on it, mirroring `cultcache-rs` /
`[format, schemaCatalog, records]` `[format, schemaCatalog, records]`
- records store raw MessagePack payload bytes under schema-catalog identity - records store raw MessagePack payload bytes under schema-catalog identity
- payloads decode only through the registered document definition for that type - payloads decode only through the registered document definition for that type
- unknown persisted types fail closed - a cache holds one record per key whatever its type: a write of a second type at a held key is refused with
`SchemaConflictError` (in memory too), and two stores holding one key refuse the load
- a store record no registered type claims loads and is carried as foreign: `cache.foreign_records()` lists it, the store
keeps its bytes through every write, a write or removal at its key is refused, and registering a type that declares its
schema id claims it at the next load
- global documents are singleton-style per type - global documents are singleton-style per type
- one home store per document type: a store added with `add_backing_store` - one home store per document type: a store added with `add_backing_store`
owns its types, otherwise the generic store is the home; a type claimed by a owns its types, otherwise the generic store is the home; a type claimed by a
@ -73,7 +77,7 @@ store.
- `register_document_type(...)` - `register_document_type(...)`
- `register_registry(...)` - `register_registry(...)`
- `register_name_lookup(...)` - `register_name_lookup(...)`
- `register_index(...)` - `register_index(...)` (a unique index)
- `add_backing_store(...)` - `add_backing_store(...)`
- `add_generic_store(...)` - `add_generic_store(...)`
- `pull_all_backing_stores()` - `pull_all_backing_stores()`
@ -95,6 +99,33 @@ store.
- `delete_global(...)` - `delete_global(...)`
- `snapshot()` - `snapshot()`
## Indexes And Roles
`indexes=` are unique lookups (variants:ruling:unique-indexes; C# `[CultIndex]`): within one document type and one alias
no two records hold one non-blank value, judged on the result of every write (a batch is judged whole, so a swap inside
one `put_envelopes` lands) and of every load and `register_index`. A refusal is a `CultCacheError`,
`Records <k1> and <k2> (<schema name>) both hold unique index '<alias>'; an index value names one record.`, keys in
ordinal order, never the value; it writes nothing, and a refused load or registration installs nothing. The contract is
the verdict; the text is informational. Blank values (absent, empty, or only C# `White_Space` characters) are never
indexed and never conflict. Two types may share an alias and a value. Values compare as the C# canonical text: strings as
is (never trimmed), `True`/`False`, integers as digits, a non-integral float as its exact decimal expansion, NaN and
infinities no value.
`roles=` name selection aliases (C# `[CultRole]`): many records may share a value, nothing is indexed or refused,
`get_by_index` on a role answers `None`, and the member's catalog `index_alias` is the same either way, so catalog bytes
do not move. An alias is declared in one of the two. `CultMeshDatabase.watch_by_index` reads `indexes` only.
Migrating an existing `indexes` key, per key, before bumping:
1. Nothing looks records up by the key (`get_by_index`, `get_key_by_index`, `node.get_by_index`, `watch_by_index`): move it to `roles`.
2. Something looks it up: it stays unique only when two live records can never hold one non-blank value (the record key
is derived from that value alone, or it is an outside identity: email, sku, external id). Enums, statuses, kinds,
categories, parent or foreign ids, shared hashes and timestamps are roles; select them with `get_all` plus a filter.
3. Open each real `.cc` once with the new definitions. A refusal names both keys and the alias and writes nothing.
Misclassifying: a role declared unique makes a store that holds a shared value refuse to open until the key is
reclassified; a unique key declared a role answers no lookups and stops refusing duplicates.
## Persistence Model ## Persistence Model
Backing stores persist envelopes, not caller domain objects. The cache decodes payloads through registered document definitions and rejects unknown type discriminators. This keeps Python's dynamic runtime from turning persistence into an open polymorphic sewer with a cheerful docstring. Backing stores persist envelopes, not caller domain objects. The cache decodes payloads through registered document definitions and rejects unknown type discriminators. This keeps Python's dynamic runtime from turning persistence into an open polymorphic sewer with a cheerful docstring.
@ -112,6 +143,10 @@ used by the TypeScript, Rust, and C# runtimes:
Large corpora should use a sharded store or a real database. A single snapshot file is a scalpel, not a forklift. Large corpora should use a sharded store or a real database. A single snapshot file is a scalpel, not a forklift.
## Store format and rollback
A store whose elements hold an id is marked `cultcache.store.v3` by the C# runtime. This runtime reads and writes a v3 single-file store, keeps the header it read and carries the ids through a rewrite; it never drops the mark, so it has no carry-back. A cultcache-py from before element ids refuses a v3 store by name, so rolling back to it after a store has been written as v3 is one-way: restore from a copy taken before upgrading, and upgrade every reader before any writer. (The C# line is different: CultLib 1.x does not refuse; see the GameCult.Caching README.)
## DatabaseEntry Slot Contract ## DatabaseEntry Slot Contract
For cross-runtime cache entries, use `define_database_entry_type(...)` instead For cross-runtime cache entries, use `define_database_entry_type(...)` instead

View file

@ -1,7 +1,7 @@
from .backing_store import BackingStore, CultCacheEnvelope from .backing_store import BackingStore, CultCacheEnvelope
from .backing_store import CultCacheSchemaCatalogEntry, CultCacheSchemaCatalogMember from .backing_store import CultCacheSchemaCatalogEntry, CultCacheSchemaCatalogMember
from . import leaves from . import leaves
from .cache import CultCache, CultCacheBuilder from .cache import CultCache, CultCacheBuilder, ForeignRecord
from .documents import ( from .documents import (
MAX_PAYLOAD_DEPTH, MAX_PAYLOAD_DEPTH,
MAX_SLOT_KEY, MAX_SLOT_KEY,
@ -29,6 +29,7 @@ __all__ = [
"CultCacheSchemaCatalogMember", "CultCacheSchemaCatalogMember",
"DatabaseEntryField", "DatabaseEntryField",
"DocumentDefinition", "DocumentDefinition",
"ForeignRecord",
"JsonLinesBackingStore", "JsonLinesBackingStore",
"MAX_PAYLOAD_DEPTH", "MAX_PAYLOAD_DEPTH",
"MAX_SLOT_KEY", "MAX_SLOT_KEY",

View file

@ -1,13 +1,15 @@
from __future__ import annotations from __future__ import annotations
import functools import functools
import math
import threading import threading
from decimal import Decimal
from dataclasses import dataclass, field, replace from dataclasses import dataclass, field, replace
from typing import Any, Callable, Generic, TypeVar from typing import Any, Callable, Generic, Iterable, TypeVar
from .backing_store import BackingStore, CultCacheEnvelope from .backing_store import BackingStore, CultCacheEnvelope
from .documents import DocumentDefinition, extract_value from .documents import DocumentDefinition, extract_value
from .stores import SchemaConflictError from .stores import SchemaConflictError, retyped, written_owns
T = TypeVar("T") T = TypeVar("T")
F = TypeVar("F", bound=Callable[..., Any]) F = TypeVar("F", bound=Callable[..., Any])
@ -52,6 +54,52 @@ class CultCacheError(RuntimeError):
pass pass
# Blank is no value: empty or only C# White_Space characters (the 25 chars string.IsNullOrWhiteSpace is true for).
# An explicit set, not str.isspace(): Python also counts U+001C-001F, which C# does not, and omits nothing C# counts.
_BLANK_INDEX_CHARS = frozenset(
chr(code)
for code in (
*range(0x09, 0x0E),
0x20,
0x85,
0xA0,
0x1680,
*range(0x2000, 0x200B),
0x2028,
0x2029,
0x202F,
0x205F,
0x3000,
)
)
def _utf16_order(text: str) -> bytes:
"""Sorts as C# StringComparer.Ordinal does: by UTF-16 code unit, not code point."""
return text.encode("utf-16-be", "surrogatepass")
def index_text(value: Any) -> str | None:
"""The canonical text of an index value, as C# IndexValue and RenderCanonicalNumber, or None when it names no
value: a string is as is (never trimmed), a boolean is `True`/`False`, an integer is its digits, a non-integral
finite float is its exact decimal expansion, NaN and infinities and blank strings are no value."""
if value is None:
return None
if isinstance(value, bool): # before int: bool is an int subclass
return "True" if value else "False"
if isinstance(value, int):
return str(value)
if isinstance(value, float):
if not math.isfinite(value):
return None
if value.is_integer():
return str(int(value)) # -0.0 is 0
text = format(Decimal(value), "f") # Decimal(float) is exact
return text.rstrip("0").rstrip(".") if "." in text else text
text = value if isinstance(value, str) else str(value)
return None if all(char in _BLANK_INDEX_CHARS for char in text) else text
@dataclass @dataclass
class _State: class _State:
documents: dict[str, DocumentDefinition[Any]] = field(default_factory=dict) documents: dict[str, DocumentDefinition[Any]] = field(default_factory=dict)
@ -72,6 +120,20 @@ class _State:
# of that global removes it from the home store. Remove once no store holds a global under a key # of that global removes it from the home store. Remove once no store holds a global under a key
# other than GLOBAL_KEY. # other than GLOBAL_KEY.
legacy_global_keys: dict[str, str] = field(default_factory=dict) legacy_global_keys: dict[str, str] = field(default_factory=dict)
# The records the last pull read that no registered type claims, sorted by key. Derived from the pull and swapped with the
# rest of the state, written nowhere else; it decides only refusals at its keys. A store keeps their bytes.
foreign: list["ForeignRecord"] = field(default_factory=list)
@dataclass(frozen=True)
class ForeignRecord:
"""A record a store holds that no registered type claims: the cache does not serve it, refuses a write or a removal at its
key, and a store keeps its bytes. Registering a type that declares its schema id claims it at the next load."""
key: str
schema_id: str
schema_name: str
stored_at: str
@dataclass(frozen=True) @dataclass(frozen=True)
@ -186,6 +248,8 @@ class CultCache:
@_mutating @_mutating
def register_index(self, document: DocumentDefinition[Any], index: str, extractor: str | Any) -> None: def register_index(self, document: DocumentDefinition[Any], index: str, extractor: str | Any) -> None:
self._assert_registered(document) self._assert_registered(document)
if index in document.roles:
raise CultCacheError(f"Index '{index}' of {document.type} is declared as a role; a role is never a unique index.")
index_extractors = {type: dict(indexes) for type, indexes in self._state.index_extractors.items()} index_extractors = {type: dict(indexes) for type, indexes in self._state.index_extractors.items()}
index_extractors.setdefault(document.type, {})[index] = extractor index_extractors.setdefault(document.type, {})[index] = extractor
self._install_lookups(self._state.name_extractors, index_extractors) self._install_lookups(self._state.name_extractors, index_extractors)
@ -195,7 +259,11 @@ class CultCache:
) -> None: ) -> None:
"""Derives every lookup under the candidate extractors, then installs extractors and lookups """Derives every lookup under the candidate extractors, then installs extractors and lookups
together. An extractor that raises on a held value installs nothing.""" together. An extractor that raises on a held value installs nothing."""
names, indexes, lookup_keys = self._derive_lookups(self._state.values, name_extractors, index_extractors) names, indexes, lookup_keys = self._derive_lookups(
[(type, key, value) for type, values in self._state.values.items() for key, value in values.items()],
name_extractors,
index_extractors,
)
( (
self._state.name_extractors, self._state.name_extractors,
self._state.index_extractors, self._state.index_extractors,
@ -263,19 +331,28 @@ class CultCache:
structures, running every check and every user extractor; the cache's state is replaced structures, running every check and every user extractor; the cache's state is replaced
only once that build finishes, so a refused load admits nothing.""" only once that build finishes, so a refused load admits nothing."""
loaded_values: dict[str, dict[str, Any]] = {} loaded_values: dict[str, dict[str, Any]] = {}
read_values: dict[tuple[str, str], Any] = {} # (type, key) -> value, one per key, the last read wins; the judge is order-free
loaded_envelopes: dict[str, dict[str, CultCacheEnvelope]] = {} loaded_envelopes: dict[str, dict[str, CultCacheEnvelope]] = {}
loaded_legacy_global_keys: dict[str, str] = {} loaded_legacy_global_keys: dict[str, str] = {}
routes, generic = self._state.stores_by_type, self._state.generic_store routes, generic = self._state.stores_by_type, self._state.generic_store
seen_globals: set[str] = set() seen_globals: set[str] = set()
foreign: list[ForeignRecord] = []
type_at_key: dict[str, tuple[str, int]] = {} # one record per key across every store: its type (or schema name) and store
stores: list[BackingStore] = [] stores: list[BackingStore] = []
for candidate in [*self._state.stores_by_type.values(), self._state.generic_store]: for candidate in [*self._state.stores_by_type.values(), self._state.generic_store]:
if candidate is not None and all(candidate is not store for store in stores): if candidate is not None and all(candidate is not store for store in stores):
stores.append(candidate) stores.append(candidate)
for store in stores: for position, store in enumerate(stores):
for envelope in store.pull_all(): for envelope in store.pull_all():
document = self._resolve_document_for_envelope(envelope) document = self._resolve_document_for_envelope(envelope)
if document is None: if document is None:
raise CultCacheError(f"Unknown persisted document type: {envelope.type}") entry = envelope.catalog_entry
held_name = entry.schema_name if entry is not None else envelope.type
self._claim_key(type_at_key, envelope.key, held_name, position)
foreign.append(
ForeignRecord(envelope.key, envelope.schema_id or envelope.type, held_name, envelope.stored_at)
)
continue
if envelope.type != document.type: if envelope.type != document.type:
envelope = CultCacheEnvelope( envelope = CultCacheEnvelope(
key=envelope.key, key=envelope.key,
@ -298,12 +375,14 @@ class CultCache:
if envelope.key != GLOBAL_KEY: if envelope.key != GLOBAL_KEY:
loaded_legacy_global_keys[envelope.type] = envelope.key loaded_legacy_global_keys[envelope.type] = envelope.key
envelope = replace(envelope, key=GLOBAL_KEY) envelope = replace(envelope, key=GLOBAL_KEY)
self._claim_key(type_at_key, envelope.key, envelope.type, position)
envelope = self._stamped(document, envelope) envelope = self._stamped(document, envelope)
value = document.decode_payload(envelope.payload) value = document.decode_payload(envelope.payload)
loaded_values.setdefault(envelope.type, {})[envelope.key] = value loaded_values.setdefault(envelope.type, {})[envelope.key] = value
read_values[(envelope.type, envelope.key)] = value
loaded_envelopes.setdefault(envelope.type, {})[envelope.key] = envelope loaded_envelopes.setdefault(envelope.type, {})[envelope.key] = envelope
loaded_names, loaded_indexes, loaded_lookup_keys = self._derive_lookups( loaded_names, loaded_indexes, loaded_lookup_keys = self._derive_lookups(
loaded_values, self._state.name_extractors, self._state.index_extractors [(type, key, value) for (type, key), value in read_values.items()], self._state.name_extractors, self._state.index_extractors
) )
# Plain attribute assignments: nothing fallible runs once the swap begins. # Plain attribute assignments: nothing fallible runs once the swap begins.
( (
@ -313,7 +392,28 @@ class CultCache:
self._state.indexes, self._state.indexes,
self._state.lookup_keys, self._state.lookup_keys,
self._state.legacy_global_keys, self._state.legacy_global_keys,
) = (loaded_values, loaded_envelopes, loaded_names, loaded_indexes, loaded_lookup_keys, loaded_legacy_global_keys) self._state.foreign,
) = (
loaded_values,
loaded_envelopes,
loaded_names,
loaded_indexes,
loaded_lookup_keys,
loaded_legacy_global_keys,
sorted(foreign, key=lambda record: _utf16_order(record.key)),
)
@staticmethod
def _claim_key(type_at_key: dict[str, tuple[str, int]], key: str, type: str, store: int) -> None:
"""A key holds one record whatever its type, across every store a load reads. The refusal names the key and both types,
never a payload."""
held, held_store = type_at_key.setdefault(key, (type, store))
if held != type or held_store != store:
raise CultCacheError(f"Record {key!r} is held by both {held} and {type}: a key holds one record; the load admitted nothing.")
def foreign_records(self) -> list[ForeignRecord]:
"""The records the last load read that no registered type claims, sorted by key (UTF-16 code-unit order)."""
return list(self._state.foreign)
def get(self, document: DocumentDefinition[T], key: str) -> T | None: def get(self, document: DocumentDefinition[T], key: str) -> T | None:
self._assert_registered(document) self._assert_registered(document)
@ -432,12 +532,21 @@ class CultCache:
"""Validates every record (home and name and index extractors included) before the store is """Validates every record (home and name and index extractors included) before the store is
touched; once the store accepts, applying to the cache has no fallible step.""" touched; once the store accepts, applying to the cache has no fallible step."""
type = document.type type = document.type
# A batch naming a key twice lands the last record for it, as C# CultCacheBatch stages one operation per key. The
# judge, the store and the apply all read this one list.
records = list({envelope.key: (envelope, value) for envelope, value in records}.values())
name_extractor = self._state.name_extractors.get(type) name_extractor = self._state.name_extractors.get(type)
index_extractors = self._state.index_extractors.get(type, {}) index_extractors = self._state.index_extractors.get(type, {})
admitted = [ admitted = [
(envelope, value, self._lookup_keys_for(value, name_extractor, index_extractors)) (envelope, value, self._lookup_keys_for(value, name_extractor, index_extractors))
for envelope, value in records for envelope, value in records
] ]
displaced = self._refuse_retype(document, [envelope for envelope, _ in records])
self._refuse_index_sharing(
[(type, envelope.key, keys) for envelope, _, keys in admitted],
self._state.indexes,
{(type, envelope.key) for envelope, _ in records},
)
store = self._store_for_type(type) store = self._store_for_type(type)
legacy_key = self._state.legacy_global_keys.get(type) legacy_key = self._state.legacy_global_keys.get(type)
if store is not None: if store is not None:
@ -451,6 +560,8 @@ class CultCache:
values = self._state.values.setdefault(type, {}) values = self._state.values.setdefault(type, {})
envelopes = self._state.envelopes.setdefault(type, {}) envelopes = self._state.envelopes.setdefault(type, {})
lookup_keys = self._state.lookup_keys.setdefault(type, {}) lookup_keys = self._state.lookup_keys.setdefault(type, {})
for other, key in displaced:
self._drop(other, key)
for envelope, value, keys in admitted: for envelope, value, keys in admitted:
self._remove_lookups(type, envelope.key) self._remove_lookups(type, envelope.key)
values[envelope.key] = value values[envelope.key] = value
@ -461,6 +572,39 @@ class CultCache:
for index, index_value in keys.indexes.items(): for index, index_value in keys.indexes.items():
self._state.indexes[(type, index, index_value)] = envelope.key self._state.indexes[(type, index, index_value)] = envelope.key
def _refuse_retype(self, document: DocumentDefinition[Any], written: list[CultCacheEnvelope]) -> list[tuple[str, str]]:
"""The one key-identity judge (variants:ruling:cache-refuses-retype): a key holds one record whatever its type, so a write
may not land at a key this cache holds another type's record at, unless the written type owns that record's schema id
(the single-file store's commit guard asks the same question of the file), and may not land at a key a store carries
foreign. Returns the (type, key) records an owning write replaces; refuses, changing nothing, otherwise."""
replaced: list[tuple[str, str]] = []
foreign = {record.key: record for record in self._state.foreign}
for envelope in written:
carried = foreign.get(envelope.key)
if carried is not None:
raise self._foreign_refusal(carried)
for other, held_envelopes in self._state.envelopes.items():
held = held_envelopes.get(envelope.key)
if other == document.type or held is None:
continue
if not written_owns(held, envelope) or self._store_for_type(other) is not self._store_for_type(document.type):
raise retyped(held, envelope)
replaced.append((other, envelope.key))
return replaced
@staticmethod
def _foreign_refusal(record: ForeignRecord) -> SchemaConflictError:
return SchemaConflictError(
record.schema_id,
[record.schema_name],
record.key,
message=(
f"Record {record.key!r} is held by the store as a {record.schema_name} (schema id {record.schema_id!r}) that no "
"registered type claims; a write or removal there would replace what this cache cannot read. Register the type "
"that owns the schema id, or remove the record with the runtime that wrote it."
),
)
@staticmethod @staticmethod
def _retire_legacy_global(store: BackingStore, type: str, legacy_key: str) -> None: def _retire_legacy_global(store: BackingStore, type: str, legacy_key: str) -> None:
"""The first write of a global loaded from a legacy key removes that record in the same write. If """The first write of a global loaded from a legacy key removes that record in the same write. If
@ -500,19 +644,27 @@ class CultCache:
def _delete(self, document: DocumentDefinition[Any], key: str) -> None: def _delete(self, document: DocumentDefinition[Any], key: str) -> None:
self._assert_registered(document) self._assert_registered(document)
if key not in self._state.envelopes.get(document.type, {}):
carried = next((record for record in self._state.foreign if record.key == key), None)
if carried is not None:
raise self._foreign_refusal(carried)
store = self._store_for_type(document.type) store = self._store_for_type(document.type)
legacy_key = self._state.legacy_global_keys.get(document.type) if key == GLOBAL_KEY else None legacy_key = self._state.legacy_global_keys.get(document.type) if key == GLOBAL_KEY else None
if store is not None: if store is not None:
store.delete(document.type, key if legacy_key is None else legacy_key) store.delete(document.type, key if legacy_key is None else legacy_key)
if legacy_key is not None: if legacy_key is not None:
del self._state.legacy_global_keys[document.type] del self._state.legacy_global_keys[document.type]
self._remove_lookups(document.type, key) self._drop(document.type, key)
def _drop(self, type: str, key: str) -> None:
"""Removes the held record and its lookups from the cache's maps; the store is the caller's."""
self._remove_lookups(type, key)
for by_type in (self._state.values, self._state.envelopes, self._state.lookup_keys): for by_type in (self._state.values, self._state.envelopes, self._state.lookup_keys):
held = by_type.get(document.type) held = by_type.get(type)
if held is not None: if held is not None:
held.pop(key, None) held.pop(key, None)
if not held: if not held:
by_type.pop(document.type, None) by_type.pop(type, None)
@_mutating @_mutating
def delete_global(self, document: DocumentDefinition[Any]) -> None: def delete_global(self, document: DocumentDefinition[Any]) -> None:
@ -552,36 +704,76 @@ class CultCache:
name = None if name_extractor is None else extract_value(value, name_extractor) name = None if name_extractor is None else extract_value(value, name_extractor)
indexes: dict[str, str] = {} indexes: dict[str, str] = {}
for index, extractor in index_extractors.items(): for index, extractor in index_extractors.items():
index_value = extract_value(value, extractor) index_value = index_text(extract_value(value, extractor))
if index_value is not None: if index_value is not None:
indexes[index] = str(index_value) indexes[index] = index_value
return _LookupKeys(None if name is None else str(name), indexes) return _LookupKeys(None if name is None else str(name), indexes)
def _derive_lookups( def _derive_lookups(
self, self,
values_by_type: dict[str, dict[str, Any]], records: list[tuple[str, str, Any]],
name_extractors: dict[str, str | Any], name_extractors: dict[str, str | Any],
index_extractors: dict[str, dict[str, str | Any]], index_extractors: dict[str, dict[str, str | Any]],
) -> tuple[_Names, _Indexes, _LookupKeysByType]: ) -> tuple[_Names, _Indexes, _LookupKeysByType]:
"""Builds name and index lookups for the given values in fresh dicts, touching no cache state.""" """Builds name and index lookups for the given (type, key, value) records, judged whole and order-free, in fresh dicts, touching no cache state."""
names: _Names = {} names: _Names = {}
indexes: _Indexes = {} indexes: _Indexes = {}
lookup_keys: _LookupKeysByType = {} lookup_keys: _LookupKeysByType = {}
for type, values in values_by_type.items(): landing: list[tuple[str, str, _LookupKeys]] = []
name_extractor = name_extractors.get(type) for type, key, value in records:
type_index_extractors = index_extractors.get(type, {}) keys = self._lookup_keys_for(value, name_extractors.get(type), index_extractors.get(type, {}))
for key, value in values.items(): lookup_keys.setdefault(type, {})[key] = keys
keys = self._lookup_keys_for(value, name_extractor, type_index_extractors) landing.append((type, key, keys))
lookup_keys.setdefault(type, {})[key] = keys # Judged whole and order-free, over every type at once and on the file alone (never against what the cache held): a held value can
# have been mutated in place since it was admitted, and another process may have moved a value between records.
self._refuse_index_sharing(landing, {}, ())
for type in lookup_keys:
for key, keys in lookup_keys.get(type, {}).items():
if keys.name is not None: if keys.name is not None:
names[(type, keys.name)] = key names[(type, keys.name)] = key
for index, index_value in keys.indexes.items(): for index, index_value in keys.indexes.items():
indexes[(type, index, index_value)] = key indexes[(type, index, index_value)] = key
return names, indexes, lookup_keys return names, indexes, lookup_keys
def _refuse_index_sharing(
self,
landing: list[tuple[str, str, _LookupKeys]],
held: dict[tuple[str, str, str], str],
replaced: Iterable[tuple[str, str]],
) -> None:
"""The one index rule (variants:ruling:unique-indexes): within one type and one unique alias, no two records hold one
non-blank value, judged on the state the admission would leave and never on staging, file or pull order. The holders
of a value are the landing keys projecting it plus the held key unless this admission replaces it. Of every
(type, alias, value) with two or more distinct holders, the refusal names the least by (alias, its least key, its
second-least key, schema name), each compared by UTF-16 code unit, never the value."""
released = set(replaced)
landed: dict[tuple[str, str, str], list[str]] = {}
for type, key, keys in landing:
for index, index_value in keys.indexes.items():
landed.setdefault((type, index, index_value), []).append(key)
violations: list[tuple[bytes, bytes, bytes, bytes, str, str, str, str]] = []
for (type, index, index_value), keys in landed.items():
held_key = held.get((type, index, index_value))
if held_key is not None and (type, held_key) not in released:
keys = [*keys, held_key]
distinct = sorted(keys, key=_utf16_order)
if len(distinct) > 1:
document = self._state.documents.get(type)
schema_name = (document.schema_name if document is not None else None) or type
violations.append(
(_utf16_order(index), _utf16_order(distinct[0]), _utf16_order(distinct[1]), _utf16_order(schema_name),
index, distinct[0], distinct[1], schema_name)
)
if violations:
*_, index, first, second, schema_name = min(violations, key=lambda violation: violation[:4])
raise CultCacheError(
f"Records {first} and {second} ({schema_name}) both hold unique index '{index}'; "
"an index value names one record."
)
def _remove_lookups(self, type: str, key: str) -> None: def _remove_lookups(self, type: str, key: str) -> None:
"""Removes the lookups the held record produced at admission, handing a name or index value to """Removes the lookups the held record produced at admission, handing a name to another held
another held record that produced it too. Reads stored lookup keys only; runs no extractor.""" record that produced it too (an index value has one holder). Reads stored lookup keys only; runs no extractor."""
held = self._state.lookup_keys.get(type, {}) held = self._state.lookup_keys.get(type, {})
keys = held.get(key) keys = held.get(key)
if keys is None: if keys is None:
@ -596,9 +788,6 @@ class CultCache:
if self._state.indexes.get(lookup_key) != key: if self._state.indexes.get(lookup_key) != key:
continue continue
del self._state.indexes[lookup_key] del self._state.indexes[lookup_key]
for candidate_key, candidate in held.items():
if candidate_key != key and candidate.indexes.get(index) == index_value:
self._state.indexes[lookup_key] = candidate_key
def _assert_registered(self, document: DocumentDefinition[Any]) -> None: def _assert_registered(self, document: DocumentDefinition[Any]) -> None:
if self._state.documents.get(document.type) is not document: if self._state.documents.get(document.type) is not document:

View file

@ -157,6 +157,11 @@ class DocumentDefinition(Generic[T]):
global_document: bool = False global_document: bool = False
name: str | Extractor[T] | None = None name: str | Extractor[T] | None = None
indexes: dict[str, str | Extractor[T]] = field(default_factory=dict) indexes: dict[str, str | Extractor[T]] = field(default_factory=dict)
"""Unique lookups (C# `[CultIndex]`): no two records of the type hold one non-blank value, judged at every commit and
load. `get_by_index` answers the one holder."""
roles: dict[str, str | Extractor[T]] = field(default_factory=dict)
"""Selection aliases (C# `[CultRole]`): many records may share a value. Never indexed for lookup, never refuse; the
member's catalog `index_alias` is the same either way."""
payload_encoder: Callable[[Any], bytes] = _json_payload_encoder payload_encoder: Callable[[Any], bytes] = _json_payload_encoder
payload_decoder: Callable[[bytes], Any] = _json_payload_decoder payload_decoder: Callable[[bytes], Any] = _json_payload_decoder
schema_id: str | None = None schema_id: str | None = None
@ -190,6 +195,14 @@ class DocumentDefinition(Generic[T]):
self.validate(value) self.validate(value)
return value return value
def __post_init__(self) -> None:
# Judged here so every constructor passes it: the factories, the dataclass constructor and dataclasses.replace.
for alias in self.roles:
if alias in self.indexes:
raise ValueError(
f"Document type {self.type} declares '{alias}' as both an index and a role; an index already names its role."
)
def catalog_entry(self) -> CultCacheSchemaCatalogEntry: def catalog_entry(self) -> CultCacheSchemaCatalogEntry:
if self._catalog_entry is not None: if self._catalog_entry is not None:
return self._catalog_entry return self._catalog_entry
@ -241,6 +254,7 @@ def define_document_type(
global_document: bool = False, global_document: bool = False,
name: str | Extractor[T] | None = None, name: str | Extractor[T] | None = None,
indexes: dict[str, str | Extractor[T]] | None = None, indexes: dict[str, str | Extractor[T]] | None = None,
roles: dict[str, str | Extractor[T]] | None = None,
payload_encoder: Callable[[Any], bytes] = _json_payload_encoder, payload_encoder: Callable[[Any], bytes] = _json_payload_encoder,
payload_decoder: Callable[[bytes], Any] = _json_payload_decoder, payload_decoder: Callable[[bytes], Any] = _json_payload_decoder,
schema_id: str | None = None, schema_id: str | None = None,
@ -259,6 +273,7 @@ def define_document_type(
global_document=global_document, global_document=global_document,
name=name, name=name,
indexes=indexes or {}, indexes=indexes or {},
roles=roles or {},
payload_encoder=payload_encoder, payload_encoder=payload_encoder,
payload_decoder=payload_decoder, payload_decoder=payload_decoder,
schema_id=schema_id, schema_id=schema_id,
@ -280,6 +295,7 @@ def define_database_entry_type(
global_document: bool = False, global_document: bool = False,
name: str | Extractor[T] | None = None, name: str | Extractor[T] | None = None,
indexes: dict[str, str | Extractor[T]] | None = None, indexes: dict[str, str | Extractor[T]] | None = None,
roles: dict[str, str | Extractor[T]] | None = None,
schema_id: str | None = None, schema_id: str | None = None,
schema_name: str | None = None, schema_name: str | None = None,
schema_version: str | None = None, schema_version: str | None = None,
@ -296,7 +312,7 @@ def define_database_entry_type(
normalized = _normalize_fields(type, fields) normalized = _normalize_fields(type, fields)
member_name_lookup = name if isinstance(name, str) else None member_name_lookup = name if isinstance(name, str) else None
index_lookup = indexes or {} index_lookup = {**(roles or {}), **(indexes or {})}
def members() -> tuple[CultCacheSchemaCatalogMember, ...]: def members() -> tuple[CultCacheSchemaCatalogMember, ...]:
return tuple( return tuple(
@ -323,6 +339,7 @@ def define_database_entry_type(
global_document=global_document, global_document=global_document,
name=name, name=name,
indexes=indexes, indexes=indexes,
roles=roles,
payload_encoder=_pack_tree, payload_encoder=_pack_tree,
payload_decoder=_messagepack_payload_decoder, payload_decoder=_messagepack_payload_decoder,
schema_id=schema_id, schema_id=schema_id,

View file

@ -245,11 +245,44 @@ def _derive_catalog(records: list[tuple[CultCacheEnvelope, bool]]) -> list[CultC
return [entry for entry, _ in chosen.values()] return [entry for entry, _ in chosen.values()]
def retyped(held: CultCacheEnvelope, written: CultCacheEnvelope) -> SchemaConflictError:
"""The refusal of a write that would put a second type under a key the store already holds (C# CacheBackingStore.Retyped)."""
return SchemaConflictError(
_schema_id_for(held),
[held.type],
held.key,
message=(
f"Record '{held.key}' is a {held.type} (schema id '{_schema_id_for(held)}'); writing a {written.type} there would "
"retype the key. Remove the record first, in the same batch or an earlier commit, then add the other type's."
),
)
def written_owns(held: CultCacheEnvelope, written: CultCacheEnvelope) -> bool:
"""Whether a write of `written` may replace `held` at its key (C# WrittenOwns): the written type's entry declares the held
record's schema id (its own id or one it lists compatible), or it is the type of the same schema name. The same-name half
stands in for C#'s "reads the published entry whole": this runtime does not compare shapes, so it never refuses a
same-name type for dropping slots."""
stored_id = _schema_id_for(held)
entry = written.catalog_entry if written.catalog_entry is not None else _default_catalog_entry(written)
return stored_id == entry.schema_id or stored_id in entry.compatible_schema_ids or held.type == written.type
def _encode_snapshot( def _encode_snapshot(
supplied: list[CultCacheEnvelope], arrived: list[CultCacheEnvelope], format_version: str supplied: list[CultCacheEnvelope], arrived: list[CultCacheEnvelope], format_version: str
) -> list[Any]: ) -> list[Any]:
replaced = {(envelope.type, envelope.key) for envelope in supplied} # A key names one record whatever its type, so a supplied record replaces the arrived one by key alone, and only of the
kept = [envelope for envelope in arrived if (envelope.type, envelope.key) not in replaced] # type that owns it: retyping a key is a removal and an add, so a record another type owns under the key refuses the write.
held_by_key = {envelope.key: envelope for envelope in arrived}
landing: dict[str, CultCacheEnvelope] = {}
for envelope in supplied:
for held in (landing.get(envelope.key), held_by_key.get(envelope.key)):
if held is not None and not written_owns(held, envelope):
raise retyped(held, envelope)
landing[envelope.key] = envelope
supplied = list(landing.values())
replaced = set(landing)
kept = [envelope for envelope in arrived if envelope.key not in replaced]
envelopes = [*kept, *supplied] envelopes = [*kept, *supplied]
catalog = [ catalog = [
_encode_catalog_entry(entry) _encode_catalog_entry(entry)
@ -400,6 +433,7 @@ def _decode_snapshot(decoded: Any) -> tuple[str, list[CultCacheEnvelope]] | None
catalog_by_schema_id.setdefault(compatible_schema_id, entry) catalog_by_schema_id.setdefault(compatible_schema_id, entry)
envelopes: list[CultCacheEnvelope] = [] envelopes: list[CultCacheEnvelope] = []
seen: set[str] = set()
for raw_record in decoded[2]: for raw_record in decoded[2]:
if not isinstance(raw_record, list) or len(raw_record) < 4: if not isinstance(raw_record, list) or len(raw_record) < 4:
raise ValueError("CultCache persisted records must be MessagePack arrays") raise ValueError("CultCache persisted records must be MessagePack arrays")
@ -412,6 +446,9 @@ def _decode_snapshot(decoded: Any) -> tuple[str, list[CultCacheEnvelope]] | None
key, schema_id, stored_at, payload = raw_record key, schema_id, stored_at, payload = raw_record
if not isinstance(key, str) or not key: if not isinstance(key, str) or not key:
raise ValueError("CultCache persisted records must declare a key") raise ValueError("CultCache persisted records must declare a key")
if key in seen:
raise ValueError(f"CultCache record {key!r} appears more than once")
seen.add(key)
if not isinstance(schema_id, str) or not schema_id: if not isinstance(schema_id, str) or not schema_id:
raise ValueError("CultCache persisted records must declare a schema id") raise ValueError("CultCache persisted records must declare a schema id")
if not isinstance(stored_at, str) or not stored_at: if not isinstance(stored_at, str) or not stored_at:
@ -437,9 +474,13 @@ def _decode_legacy_envelopes(decoded: Any) -> list[CultCacheEnvelope]:
if not isinstance(decoded, list): if not isinstance(decoded, list):
raise ValueError("CultCache MessagePack store is not a recognized snapshot") raise ValueError("CultCache MessagePack store is not a recognized snapshot")
envelopes: list[CultCacheEnvelope] = [] envelopes: list[CultCacheEnvelope] = []
seen: set[Any] = set()
for item in decoded: for item in decoded:
if not isinstance(item, dict): if not isinstance(item, dict):
raise ValueError("CultCache legacy envelopes must be MessagePack maps") raise ValueError("CultCache legacy envelopes must be MessagePack maps")
if item.get("key") in seen:
raise ValueError(f"CultCache record {item['key']!r} appears more than once")
seen.add(item.get("key"))
try: try:
payload = item["payload"] payload = item["payload"]
envelopes.append( envelopes.append(
@ -484,12 +525,14 @@ def _decode_catalog_entry(value: Any) -> CultCacheSchemaCatalogEntry:
raise ValueError(f'CultCache schema catalog entry "{schema_id}" has invalid compatible schema ids') raise ValueError(f'CultCache schema catalog entry "{schema_id}" has invalid compatible schema ids')
if not isinstance(members, list): if not isinstance(members, list):
raise ValueError(f'CultCache schema catalog entry "{schema_id}" has invalid members') raise ValueError(f'CultCache schema catalog entry "{schema_id}" has invalid members')
if not isinstance(canonical_schema_json, str):
raise ValueError(f'CultCache schema catalog entry "{schema_id}" has an invalid canonical schema')
return CultCacheSchemaCatalogEntry( return CultCacheSchemaCatalogEntry(
schema_id=schema_id, schema_id=schema_id,
schema_name=schema_name, schema_name=schema_name,
schema_version=schema_version if isinstance(schema_version, str) and schema_version else f"{schema_name}.v1", schema_version=schema_version if isinstance(schema_version, str) and schema_version else f"{schema_name}.v1",
content_hash=content_hash if isinstance(content_hash, str) and content_hash else schema_id, content_hash=content_hash if isinstance(content_hash, str) and content_hash else schema_id,
canonical_schema_json=canonical_schema_json if isinstance(canonical_schema_json, str) else "", canonical_schema_json=canonical_schema_json,
compatible_schema_ids=tuple(compatible_schema_ids), compatible_schema_ids=tuple(compatible_schema_ids),
members=tuple(_decode_catalog_member(member) for member in members), members=tuple(_decode_catalog_member(member) for member in members),
) )

View file

@ -103,20 +103,20 @@ class CultCacheTests(unittest.TestCase):
self.assertEqual(cache.get_key_by_name(document, "Elixir"), "item:potion") self.assertEqual(cache.get_key_by_name(document, "Elixir"), "item:potion")
self.assertEqual(cache.get_by_index(document, "category", "Rare").name, "Elixir") self.assertEqual(cache.get_by_index(document, "category", "Rare").name, "Elixir")
def test_incremental_lookup_delete_restores_duplicate_owner(self) -> None: def test_incremental_lookup_delete_restores_duplicate_name_owner(self) -> None:
document = item_doc() document = item_doc()
cache = CultCache() cache = CultCache()
cache.register_document_type(document) cache.register_document_type(document)
cache.put(document, "item:first", Item(name="Potion", category="Consumable", value=50)) cache.put(document, "item:first", Item(name="Potion", category="Consumable", value=50))
cache.put(document, "item:second", Item(name="Potion", category="Consumable", value=60)) cache.put(document, "item:second", Item(name="Potion", category="Rare", value=60))
self.assertEqual(cache.get_key_by_name(document, "Potion"), "item:second") self.assertEqual(cache.get_key_by_name(document, "Potion"), "item:second")
self.assertEqual(cache.get_key_by_index(document, "category", "Consumable"), "item:second")
cache.delete(document, "item:second") cache.delete(document, "item:second")
self.assertEqual(cache.get_key_by_name(document, "Potion"), "item:first") self.assertEqual(cache.get_key_by_name(document, "Potion"), "item:first")
self.assertEqual(cache.get_key_by_index(document, "category", "Consumable"), "item:first") self.assertEqual(cache.get_key_by_index(document, "category", "Consumable"), "item:first")
self.assertIsNone(cache.get_key_by_index(document, "category", "Rare"))
def test_database_entry_formatter_uses_slot_indexed_messagepack_array(self) -> None: def test_database_entry_formatter_uses_slot_indexed_messagepack_array(self) -> None:
try: try:
@ -1121,6 +1121,25 @@ class CultCacheTests(unittest.TestCase):
catalog_entry=catalog_entry, catalog_entry=catalog_entry,
) )
def test_pulling_a_synthetic_pre_v1_envelope_array_leaves_the_file_bytes_identical(self) -> None:
try:
import msgpack # type: ignore
except ImportError:
self.skipTest("msgpack optional dependency is not installed")
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "legacy.cc"
legacy = msgpack.packb(
[{"key": "note:hello", "type": "note", "payload": {"title": "Hello"}, "storedAt": "2026-10-20T00:00:00Z"}],
use_bin_type=True,
)
path.write_bytes(legacy)
envelopes = SingleFileMessagePackBackingStore(path).pull_all()
self.assertEqual([e.key for e in envelopes], ["note:hello"])
self.assertEqual(path.read_bytes(), legacy)
self.assertEqual(sorted(p.name for p in Path(tmp).iterdir()), ["legacy.cc"])
def test_legacy_key_global_loads_without_writing_and_first_write_leaves_one_global(self) -> None: def test_legacy_key_global_loads_without_writing_and_first_write_leaves_one_global(self) -> None:
with tempfile.TemporaryDirectory() as tmp: with tempfile.TemporaryDirectory() as tmp:
settings = define_document_type("settings", global_document=True) settings = define_document_type("settings", global_document=True)
@ -1548,15 +1567,19 @@ class CultCacheTests(unittest.TestCase):
# Every vector in the folder has a manifest row. # Every vector in the folder has a manifest row.
self.assertEqual( self.assertEqual(
sorted(path.name for path in root.glob("*.bin")), sorted(path.name for path in root.glob("*.bin")),
sorted(vector for vector, *_ in rows if not vector.startswith("..")), sorted(vector for vector, *_ in rows if not vector.startswith(("..", "@"))),
) )
envelopes = SingleFileMessagePackBackingStore(self._C2A_VECTORS / "v3-base.msgpack").pull_all() envelopes = SingleFileMessagePackBackingStore(self._C2A_VECTORS / "v3-base.msgpack").pull_all()
for vector, _, _, _, python in rows: for vector, _, _, _, python in rows:
content = (root / vector).read_bytes() # "@directory": the walker makes an empty directory at the store path; it is refused and left untouched.
content = None if vector == "@directory" else (root / vector).read_bytes()
for operation in ("pull_all", "push_all", "push", "delete"): for operation in ("pull_all", "push_all", "push", "delete"):
with tempfile.TemporaryDirectory() as tmp: with tempfile.TemporaryDirectory() as tmp:
store_path = Path(tmp) / "store.msgpack" store_path = Path(tmp) / "store.msgpack"
store_path.write_bytes(content) if content is None:
store_path.mkdir()
else:
store_path.write_bytes(content)
store = SingleFileMessagePackBackingStore(store_path) store = SingleFileMessagePackBackingStore(store_path)
run = { run = {
"pull_all": store.pull_all, "pull_all": store.pull_all,
@ -1579,12 +1602,129 @@ class CultCacheTests(unittest.TestCase):
self.assertIn(envelopes[0].key, keys, f"{vector} {operation}") self.assertIn(envelopes[0].key, keys, f"{vector} {operation}")
else: else:
self.assertNotIn(envelopes[0].key, keys, f"{vector} {operation}") self.assertNotIn(envelopes[0].key, keys, f"{vector} {operation}")
elif content is None:
# A path that cannot be reached as a file stays an OSError (_read_store): not a store that failed to decode.
with self.assertRaises(OSError, msg=f"{vector} {operation}"):
run()
self.assertTrue(store_path.is_dir() and not any(store_path.iterdir()), f"{vector} {operation} touched the directory")
else: else:
with self.assertRaises(StoreUnreadableError, msg=f"{vector} {operation}") as refused: with self.assertRaises(StoreUnreadableError, msg=f"{vector} {operation}") as refused:
run() run()
self.assertIsNotNone(refused.exception.__cause__, f"{vector} {operation}") self.assertIsNotNone(refused.exception.__cause__, f"{vector} {operation}")
self.assertEqual(store_path.read_bytes(), content, f"{vector} {operation} rewrote a file it cannot read") self.assertEqual(store_path.read_bytes(), content, f"{vector} {operation} rewrote a file it cannot read")
@staticmethod
def _rec(type: str, key: str, second: int = 0) -> CultCacheEnvelope:
return CultCacheEnvelope(key=key, type=type, payload=b"\x90", stored_at=f"2026-09-30T00:00:0{second}Z")
def test_a_supplied_record_replaces_an_arrived_one_of_the_same_type_by_key(self) -> None:
import msgpack # type: ignore
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.msgpack"
store = SingleFileMessagePackBackingStore(path)
store.push(self._rec("first", "k"))
store.push(self._rec("first", "k", 1))
store.push_all([self._rec("first", "k", 2), self._rec("first", "k", 3)])
records = msgpack.unpackb(path.read_bytes(), raw=False)[2]
self.assertEqual([record[0] for record in records], ["k"])
self.assertEqual([(e.type, e.stored_at) for e in store.pull_all()], [("first", "2026-09-30T00:00:03Z")])
def test_a_write_that_would_retype_a_key_is_refused_and_leaves_the_file(self) -> None:
from cultcache_py import SchemaConflictError
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.msgpack"
store = SingleFileMessagePackBackingStore(path)
store.push(self._rec("first", "k"))
content = path.read_bytes()
for run in (
lambda: store.push(self._rec("second", "k", 1)),
lambda: store.push_all([self._rec("other", "j"), self._rec("second", "k", 1)]),
):
with self.assertRaises(SchemaConflictError) as refused:
run()
self.assertEqual(refused.exception.record_key, "k")
self.assertEqual(refused.exception.schema_names, ["first"])
self.assertIn("would retype the key", str(refused.exception))
self.assertEqual(path.read_bytes(), content)
# Retyping is a removal and an add.
store.delete("first", "k")
store.push(self._rec("second", "k", 1))
self.assertEqual([e.type for e in SingleFileMessagePackBackingStore(path).pull_all()], ["second"])
def test_a_retype_is_judged_by_the_stored_schema_id_the_written_type_declares_not_by_name(self) -> None:
from cultcache_py import SchemaConflictError
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.msgpack"
store = SingleFileMessagePackBackingStore(path)
store.push(self._writer_record("k", "old", "id-old", self._writer_entry("id-old", "old", "h1", ("id-old",))))
content = path.read_bytes()
# A renamed type that lists the stored id compatible owns the record, so its write replaces it.
renamed = self._writer_record("k", "new", "id-new", self._writer_entry("id-new", "new", "h2", ("id-new", "id-old")))
# A type that lists nothing of the stored id and has another name does not.
stranger = self._writer_record("k", "stranger", "id-x", self._writer_entry("id-x", "stranger", "h3", ("id-x",)))
for run in (lambda: store.push(stranger), lambda: store.push_all([stranger])):
with self.assertRaises(SchemaConflictError):
run()
self.assertEqual(path.read_bytes(), content)
store.push(renamed)
self.assertEqual([(e.type, e.schema_id) for e in SingleFileMessagePackBackingStore(path).pull_all()], [("new", "id-new")])
def test_a_batch_naming_one_key_under_two_types_is_refused_whole(self) -> None:
from cultcache_py import SchemaConflictError
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.msgpack"
store = SingleFileMessagePackBackingStore(path)
with self.assertRaises(SchemaConflictError):
store.push_all([self._rec("first", "k"), self._rec("second", "k", 1)])
self.assertFalse(path.exists())
def test_a_cache_put_of_a_second_type_under_a_key_is_refused_and_persists_nothing_new(self) -> None:
from cultcache_py import SchemaConflictError
other = define_document_type("other", encode=lambda item: asdict(item), decode=lambda raw: Item(**raw))
document = item_doc()
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.msgpack"
def build() -> CultCache:
cache = (
CultCache.builder()
.register_document_type(document)
.register_document_type(other)
.add_generic_store(SingleFileMessagePackBackingStore(path))
.build()
)
cache.pull_all_backing_stores()
return cache
cache = build()
cache.put(document, "shared", Item(name="A", category="c", value=1))
content = path.read_bytes()
with self.assertRaises(SchemaConflictError):
cache.put(other, "shared", Item(name="B", category="c", value=2))
self.assertEqual(path.read_bytes(), content)
self.assertIsNone(cache.get(other, "shared"))
reloaded = build()
self.assertEqual(reloaded.get_required(document, "shared").value, 1)
self.assertIsNone(reloaded.get(other, "shared"))
cache.delete(document, "shared")
cache.put(other, "shared", Item(name="B", category="c", value=2))
self.assertEqual(build().get_required(other, "shared").value, 2)
def test_a_catalog_entry_with_a_non_string_canonical_schema_is_unreadable(self) -> None:
# Direct probe of the c0 vector; its manifest row is added by the TS cut.
vector = self._C2A_VECTORS.parent / "document-variants-c0" / "current-catalog-bad-canonical.msgpack"
content = vector.read_bytes()
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.msgpack"
path.write_bytes(content)
with self.assertRaises(StoreUnreadableError):
SingleFileMessagePackBackingStore(path).pull_all()
self.assertEqual(path.read_bytes(), content)
if __name__ == "__main__": if __name__ == "__main__":

View file

@ -24,7 +24,6 @@ item_doc = define_document_type(
encode=lambda item: asdict(item), encode=lambda item: asdict(item),
decode=lambda raw: Item(**raw), decode=lambda raw: Item(**raw),
name="name", name="name",
indexes={"category": "category"},
) )

View file

@ -0,0 +1,316 @@
from __future__ import annotations
import re
import shutil
import tempfile
import unittest
from pathlib import Path
import msgpack
from cultcache_py.cache import CultCacheError
from cultcache_py import (
CultCache,
JsonLinesBackingStore,
SchemaConflictError,
SingleFileMessagePackBackingStore,
database_entry_field,
define_database_entry_type,
leaves as t,
)
# variants:ruling:cache-refuses-retype, write-ownership-by-written-type, unheld-removal-refused-at-commit. The vector in
# tests/vectors/key-identity is written only by the C# reference; this walks every row against the Python cache.
VECTORS = Path(__file__).resolve().parents[3] / "tests" / "vectors" / "key-identity"
IDS = {
"alpha": "sha256:b20703a98f49c253898404ab50ba2c630720f0b8a5e1a41614d5650be22799fe",
"beta": "sha256:dee4b727cf2ee04e61ab635ff0c5f3dca7345a411dc5f1018e583ab16009cf94",
"gamma": "sha256:fbc26e351e00c9e7185e8de171c7486aba707c229c949f0aae6b889288af5f6d",
}
DOCUMENTS = {
label: define_database_entry_type(
f"tests.ki_{label}",
[database_entry_field("text", 0, codec=t.string)],
schema_id=schema_id,
schema_name=f"tests.ki_{label}",
)
for label, schema_id in IDS.items()
}
LABEL_OF = {document.type: label for label, document in DOCUMENTS.items()}
def file_records(path: Path) -> list[list[str]]:
return sorted([envelope.key, envelope.type] for envelope in SingleFileMessagePackBackingStore(path).pull_all())
def stored(path: Path, key: str):
for envelope in SingleFileMessagePackBackingStore(path).pull_all():
if envelope.key == key:
return envelope.schema_id, envelope.stored_at, envelope.payload
return None
class KeyIdentityVectorTests(unittest.TestCase):
def test_rows(self) -> None:
rows = msgpack.unpackb((VECTORS / "rows.msgpack").read_bytes(), raw=False)
walked = 0
for label, registered, ops, expect in rows:
with self.subTest(row=label), tempfile.TemporaryDirectory() as tmp:
self.walk(Path(tmp), registered, ops, expect)
walked += 1
self.assertEqual(walked, 12)
def walk(self, tmp: Path, registered: list[str], ops: list, expect: list) -> None:
refusal, held, file, foreign, kept = expect
state: dict = {"cache": None, "path": None, "before": None, "attached": []}
def builder() -> CultCache.builder:
made = CultCache.builder()
for name in registered:
made.register_document_type(DOCUMENTS[name])
return made
def open_cache(target) -> None:
if target == "memory":
state["cache"], state["path"] = builder().build(), None
return
path = tmp / "store.cc"
if target is not None and target != path:
shutil.copyfile(VECTORS / target, path)
state["before"] = path.read_bytes()
state["path"] = path
made = builder().add_generic_store(SingleFileMessagePackBackingStore(path))
state["cache"] = made.build()
state["cache"].pull_all_backing_stores()
def perform(op: list) -> None:
cache = state["cache"]
if op[0] == "open":
open_cache(op[1])
elif op[0] == "attach":
path = tmp / f"attached-{len(state['attached'])}.cc"
shutil.copyfile(VECTORS / op[1], path)
state["attached"].append((path, path.read_bytes()))
cache.add_backing_store(
SingleFileMessagePackBackingStore(path), [DOCUMENTS[name].type for name in op[2]]
)
cache.pull_all_backing_stores()
elif op[0] == "reopen":
open_cache(state["path"]) # the same file, no fresh copy
elif op[0] == "put":
cache.put(DOCUMENTS[op[1]], op[2], {"text": op[3]})
elif op[0] == "delete":
cache.delete(DOCUMENTS[op[1]], op[2])
else:
self.fail(f"unknown op {op[0]}")
for op in ops[:-1]:
perform(op)
if refusal is None:
perform(ops[-1])
else:
key = refusal[0]
with self.assertRaises((SchemaConflictError, ValueError, RuntimeError)) as caught:
perform(ops[-1])
if ops[-1][0] == "attach":
# The key is read out of the refusal's own text (it opens "Record 'key' is held by") and occurs only as a
# delimited token, never inside a schema name or other word.
message = str(caught.exception)
named = re.match(r"Record '([^']*)' is held by", message)
self.assertIsNotNone(named, message)
self.assertEqual(named.group(1), key)
self.assertEqual(len(re.findall(rf"(?<![A-Za-z0-9_]){re.escape(key)}(?![A-Za-z0-9_])", message)), 1)
self.assertEqual(message.count(key), 1)
for path, before in state["attached"]:
self.assertEqual(path.read_bytes(), before)
else:
self.assertIsInstance(caught.exception, SchemaConflictError)
self.assertEqual(caught.exception.record_key, key)
self.assertIn(key, str(caught.exception))
cache = state["cache"]
got_held = sorted(
[LABEL_OF[type], key, value["text"]]
for type, values in cache.snapshot().items()
for key, value in values.items()
)
self.assertEqual(got_held, held)
path = state["path"]
if file is None:
self.assertIsNone(path)
else:
self.assertEqual(file_records(path), file)
self.assertEqual([record.key for record in cache.foreign_records()], foreign)
if kept:
vector = tmp / "vector-before.cc"
vector.write_bytes(state["before"])
for key in kept:
self.assertIsNotNone(stored(vector, key))
self.assertEqual(stored(path, key), stored(vector, key))
def alpha_beta(*names: str, store=None) -> CultCache:
made = CultCache.builder()
for name in names:
made.register_document_type(DOCUMENTS[name])
if store is not None:
made.add_generic_store(store)
return made.build()
class KeyIdentityTests(unittest.TestCase):
def test_two_global_types_are_refused_in_one_cache_and_a_store_holding_both_does_not_load(self) -> None:
# C# refuses a second persisted global on pull (cultcache-store-composition.md, "Globals never invented"); the Python
# JSONL store and a cache with no store used to let two global types share __global__, so such a file can exist.
ga = define_database_entry_type(
"tests.ki_ga", [database_entry_field("text", 0, codec=t.string)], schema_id="sha256:" + "1" * 64, global_document=True
)
gb = define_database_entry_type(
"tests.ki_gb", [database_entry_field("text", 0, codec=t.string)], schema_id="sha256:" + "2" * 64, global_document=True
)
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.jsonl"
for document in (ga, gb): # one cache per type that never pulled: how a file got two globals
writer = CultCache.builder().register_document_type(document).add_generic_store(JsonLinesBackingStore(path)).build()
writer.put_global(document, {"text": document.type})
both = CultCache.builder().register_document_type(ga).register_document_type(gb).add_generic_store(JsonLinesBackingStore(path)).build()
with self.assertRaises(CultCacheError) as caught:
both.pull_all_backing_stores()
self.assertIn("'__global__' is held by both", str(caught.exception))
self.assertEqual(both.snapshot(), {})
memory = CultCache.builder().register_document_type(ga).register_document_type(gb).build()
memory.put_global(ga, {"text": "a"})
with self.assertRaises(SchemaConflictError):
memory.put_global(gb, {"text": "b"})
self.assertEqual(memory.get_required_global(ga), {"text": "a"})
def test_every_write_path_refuses_a_second_type_and_never_echoes_the_payload(self) -> None:
alpha, beta = DOCUMENTS["alpha"], DOCUMENTS["beta"]
for name in ("put", "put_envelope", "put_envelopes"):
with self.subTest(path=name):
cache = alpha_beta("alpha", "beta")
cache.put(alpha, "k", {"text": "held"})
envelope = cache.get_envelope(alpha, "k")
other = type(envelope).create(
key="k", type=beta.type, payload=beta.encode_payload({"text": "CANARY"}), schema_id=IDS["beta"]
)
writes = {
"put": lambda: cache.put(beta, "k", {"text": "CANARY"}),
"put_envelope": lambda: cache.put_envelope(beta, other),
"put_envelopes": lambda: cache.put_envelopes(beta, [other]),
}
with self.assertRaises(SchemaConflictError) as caught:
writes[name]()
self.assertEqual(caught.exception.record_key, "k")
self.assertNotIn("CANARY", str(caught.exception))
self.assertEqual(cache.get(alpha, "k"), {"text": "held"})
self.assertIsNone(cache.get(beta, "k"))
def test_two_routed_stores_holding_one_key_refuse_the_pull_and_keep_the_earlier_state(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
one, two = Path(tmp) / "one.cc", Path(tmp) / "two.cc"
shutil.copyfile(VECTORS / "catalog-store.cc", one)
shutil.copyfile(VECTORS / "run-store.cc", two)
cache = alpha_beta("alpha", "beta")
cache.add_backing_store(SingleFileMessagePackBackingStore(one), [DOCUMENTS["alpha"].type])
cache.pull_all_backing_stores()
cache.add_backing_store(SingleFileMessagePackBackingStore(two), [DOCUMENTS["beta"].type])
with self.assertRaises(Exception) as caught:
cache.pull_all_backing_stores()
self.assertIn("'qx'", str(caught.exception))
self.assertEqual(cache.get(DOCUMENTS["alpha"], "qx"), {"text": "a"})
self.assertIsNone(cache.get(DOCUMENTS["beta"], "qx"))
def test_a_foreign_record_is_listed_and_its_bytes_survive_writes_and_removals_elsewhere(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.cc"
shutil.copyfile(VECTORS / "foreign-store.cc", path)
before = stored(path, "k")
cache = alpha_beta("alpha", store=SingleFileMessagePackBackingStore(path))
cache.pull_all_backing_stores()
[record] = cache.foreign_records()
self.assertEqual((record.key, record.schema_id, record.schema_name), ("k", IDS["gamma"], "tests.ki_gamma"))
self.assertEqual(record.stored_at, before[1])
cache.put(DOCUMENTS["alpha"], "m", {"text": "m"})
cache.delete(DOCUMENTS["alpha"], "j")
self.assertEqual(stored(path, "k"), before)
self.assertEqual(file_records(path), [["k", "tests.ki_gamma"], ["m", "tests.ki_alpha"]])
claimed = alpha_beta("alpha", "gamma", store=SingleFileMessagePackBackingStore(path))
claimed.pull_all_backing_stores()
self.assertEqual(claimed.foreign_records(), [])
self.assertEqual(claimed.get(DOCUMENTS["gamma"], "k"), {"text": "g"})
def test_the_foreign_list_is_sorted_by_utf16_code_unit_and_replaced_by_the_next_pull(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.cc"
store = SingleFileMessagePackBackingStore(path)
gamma = DOCUMENTS["gamma"]
writer = alpha_beta("gamma", store=store)
for key in ("\U00010400", "A", "b", "B"):
writer.put(gamma, key, {"text": "x"})
cache = alpha_beta("alpha", store=SingleFileMessagePackBackingStore(path))
cache.pull_all_backing_stores()
self.assertEqual([record.key for record in cache.foreign_records()], ["B", "b", "\U00010400", "A"])
for key in ("B", "b", "\U00010400", "A"):
writer.delete(gamma, key)
cache.pull_all_backing_stores()
self.assertEqual(cache.foreign_records(), [])
class OwnershipAndStoreKindTests(unittest.TestCase):
"""The cache judge answers on its own where the store has no commit guard (JSONL) or the records live in two stores."""
def test_a_jsonl_store_gets_no_help_from_a_disk_judge(self) -> None:
from cultcache_py import JsonLinesBackingStore
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.jsonl"
gamma = DOCUMENTS["gamma"]
alpha_beta("gamma", store=JsonLinesBackingStore(path)).put(gamma, "k", {"text": "g"})
before = path.read_bytes()
cache = alpha_beta("alpha", store=JsonLinesBackingStore(path))
cache.pull_all_backing_stores()
self.assertEqual([record.key for record in cache.foreign_records()], ["k"])
with self.assertRaises(SchemaConflictError):
cache.put(DOCUMENTS["alpha"], "k", {"text": "x"})
with self.assertRaises(SchemaConflictError):
cache.delete(DOCUMENTS["alpha"], "k")
self.assertEqual(path.read_bytes(), before)
cache.put(DOCUMENTS["alpha"], "j", {"text": "a"}) # the foreign line is carried through a write elsewhere
self.assertIn(b"tests.ki_gamma", path.read_bytes())
def test_a_foreign_key_in_two_stores_refuses_the_pull(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
one, two = Path(tmp) / "one.cc", Path(tmp) / "two.cc"
for path in (one, two):
alpha_beta("gamma", store=SingleFileMessagePackBackingStore(path)).put(DOCUMENTS["gamma"], "k", {"text": "g"})
cache = alpha_beta("alpha")
cache.add_backing_store(SingleFileMessagePackBackingStore(one), [DOCUMENTS["alpha"].type])
cache.add_backing_store(SingleFileMessagePackBackingStore(two), [DOCUMENTS["beta"].type])
with self.assertRaises(Exception) as caught:
cache.pull_all_backing_stores()
self.assertIn("'k'", str(caught.exception))
def test_a_type_that_owns_the_held_schema_id_replaces_the_record_and_the_key_keeps_one_type(self) -> None:
successor = define_database_entry_type(
"tests.ki_alpha2",
[database_entry_field("text", 0, codec=t.string)],
schema_id="sha256:" + "a" * 64,
schema_name="tests.ki_alpha2",
compatible_schema_ids=[IDS["alpha"]],
)
for routed in (False, True):
with self.subTest(routed=routed), tempfile.TemporaryDirectory() as tmp:
one, two = Path(tmp) / "one.cc", Path(tmp) / "two.cc"
made = CultCache.builder().register_document_type(DOCUMENTS["alpha"]).register_document_type(successor)
if routed:
made.add_backing_store(SingleFileMessagePackBackingStore(one), [DOCUMENTS["alpha"].type])
made.add_backing_store(SingleFileMessagePackBackingStore(two), [successor.type])
else:
made.add_generic_store(SingleFileMessagePackBackingStore(one))
cache = made.build()
cache.put(DOCUMENTS["alpha"], "k", {"text": "old"})
if routed:
with self.assertRaises(SchemaConflictError): # one key would live in two stores
cache.put(successor, "k", {"text": "new"})
self.assertEqual(cache.get(DOCUMENTS["alpha"], "k"), {"text": "old"})
else:
cache.put(successor, "k", {"text": "new"})
self.assertIsNone(cache.get(DOCUMENTS["alpha"], "k"))
self.assertEqual(cache.get(successor, "k"), {"text": "new"})
self.assertEqual(file_records(one), [["k", "tests.ki_alpha2"]])

View file

@ -0,0 +1,664 @@
from __future__ import annotations
import itertools
import shutil
import tempfile
import unittest
from dataclasses import asdict, dataclass
from decimal import Decimal
from pathlib import Path
import msgpack
from cultcache_py import (
CultCache,
CultCacheEnvelope,
SingleFileMessagePackBackingStore,
define_database_entry_type,
define_document_type,
leaves as t,
database_entry_field,
)
from cultcache_py.cache import CultCacheError, index_text
# variants:ruling:unique-indexes. `indexes` are unique within one type, `roles` are selection aliases. The vectors in
# tests/vectors/unique-index are written by the C# reference; the rest of the behaviour is pinned here.
VECTORS = Path(__file__).resolve().parents[3] / "tests" / "vectors" / "unique-index"
@dataclass
class Item:
sku: object = None
serial: object = None
group: object = None
def refusal(key_a: str, key_b: str, schema_name: str, alias: str) -> str:
return f"Records {key_a} and {key_b} ({schema_name}) both hold unique index '{alias}'; an index value names one record."
def item_doc(type: str = "item", *, indexes=None, roles=None):
return define_document_type(
type,
encode=asdict,
decode=lambda raw: Item(**raw),
indexes={"sku": "sku", "serial": "serial"} if indexes is None else indexes,
roles={"group": "group"} if roles is None else roles,
)
def envelope(document, key: str, item: Item) -> CultCacheEnvelope:
return CultCacheEnvelope.create(key=key, type=document.type, payload=document.encode_payload(item))
def cache_of(*documents, store=None) -> CultCache:
builder = CultCache.builder()
for document in documents:
builder.register_document_type(document)
if store is not None:
builder.add_generic_store(store)
return builder.build()
class RecordingStore:
"""A store that counts what reaches it; the judge must refuse before any store call."""
def __init__(self) -> None:
self.pushed: list[str] = []
def pull_all(self):
return []
def push(self, envelope) -> None:
self.pushed.append(envelope.key)
def push_all(self, envelopes) -> None:
self.pushed.extend(envelope.key for envelope in envelopes)
def delete(self, type, key) -> None:
pass
class IndexTextTests(unittest.TestCase):
def test_canonical_text(self) -> None:
cases = [
("S-1", "S-1"),
(" a ", " a "),
("", ""),
("\x1f", "\x1f"),
("\x1c", "\x1c"),
("\x1d", "\x1d"),
("\x1e", "\x1e"),
("᠎", "᠎"),
("​", "​"),
(True, "True"),
(False, "False"),
(7, "7"),
(-7, "-7"),
(9223372036854775807, "9223372036854775807"),
(18446744073709551615, "18446744073709551615"),
(0.0, "0"),
(-0.0, "0"),
(2.0, "2"),
(-3.0, "-3"),
(1e21, "1000000000000000000000"),
(0.1, "0.1000000000000000055511151231257827021181583404541015625"),
(-0.5, "-0.5"),
(2.5e-07, "0.00000024999999999999998868702795647156467140348468092270195484161376953125"),
]
for value, text in cases:
with self.subTest(value=value):
self.assertEqual(index_text(value), text)
self.assertIsInstance(index_text(value), str)
def test_no_value(self) -> None:
blank = ["", " ", "\t", "\n", "\x0b", "\x0c", "\r", "\x85", "\xa0", " ", "
", "
", " ",
" ", " ", "  \x85 ", *map(chr, range(0x2000, 0x200B))]
for value in [None, float("nan"), float("inf"), float("-inf"), *blank]:
with self.subTest(value=repr(value)):
self.assertIsNone(index_text(value))
def test_blank_set_is_exactly_the_csharp_set(self) -> None:
csharp = {*range(0x09, 0x0E), 0x20, 0x85, 0xA0, 0x1680, *range(0x2000, 0x200B), 0x2028, 0x2029, 0x202F, 0x205F, 0x3000}
self.assertEqual(len(csharp), 25)
got = {code for code in range(0x110000) if 0xD800 > code or code > 0xDFFF if index_text(chr(code)) is None}
self.assertEqual(got, csharp)
# Python's isspace is the wrong test: it also counts U+001C-001F.
self.assertTrue("\x1f".isspace())
self.assertEqual(index_text("\x1f"), "\x1f")
def test_floats_render_exactly_not_through_str(self) -> None:
for value in (0.1, 1e-7, 123456789.123456789, 5e-324, 1.7976931348623157e308):
with self.subTest(value=value):
text = index_text(value)
self.assertNotIn("e", text.lower())
self.assertEqual(Decimal(text), Decimal(value))
self.assertFalse(text.endswith("0") and "." in text)
def test_bool_is_not_an_integer(self) -> None:
self.assertEqual(index_text(True), "True")
self.assertNotEqual(index_text(True), "1")
class UniqueIndexTests(unittest.TestCase):
def test_a_write_of_a_held_value_at_another_key_is_refused_by_every_write_path(self) -> None:
for alias in ("sku", "serial"): # two unique indexes: each is judged
other = "serial" if alias == "sku" else "sku"
paths = {
"put": lambda cache, d, k, i: cache.put(d, k, i),
"put_envelope": lambda cache, d, k, i: cache.put_envelope(d, envelope(d, k, i)),
"put_envelopes": lambda cache, d, k, i: cache.put_envelopes(d, [envelope(d, k, i)]),
"update": lambda cache, d, k, i: cache.update(d, k, lambda _held: i),
}
for name, write in paths.items():
with self.subTest(alias=alias, path=name):
document = item_doc()
store = RecordingStore()
cache = cache_of(document, store=store)
cache.put(document, "item:b", Item(sku="CANARY-b", serial="canary-b"))
cache.put(document, "item:a", Item(sku="CANARY-a", serial="canary-a"))
before = cache.snapshot()
pushed = list(store.pushed)
taken = "CANARY-b" if alias == "sku" else "canary-b"
landing = Item(**{alias: taken, other: "fresh"})
with self.assertRaises(CultCacheError) as caught:
write(cache, document, "item:a", landing)
self.assertEqual(str(caught.exception), refusal("item:a", "item:b", "item", alias))
self.assertNotIn("anary", str(caught.exception))
self.assertEqual(cache.snapshot(), before)
self.assertEqual(store.pushed, pushed)
self.assertEqual(cache.get_key_by_index(document, alias, taken), "item:b")
self.assertIsNone(cache.get_key_by_index(document, other, "fresh"))
def test_a_record_rewriting_its_own_key_keeps_its_value(self) -> None:
document = item_doc()
cache = cache_of(document)
cache.put(document, "item:a", Item(sku="S", serial="1", group="g"))
cache.put(document, "item:a", Item(sku="S", serial="2", group="h"))
self.assertEqual(cache.get_key_by_index(document, "sku", "S"), "item:a")
self.assertIsNone(cache.get_key_by_index(document, "serial", "1"))
self.assertEqual(cache.get_key_by_index(document, "serial", "2"), "item:a")
def test_the_refusal_names_the_schema_not_the_type(self) -> None:
document = define_document_type(
"t.item", indexes={"sku": "sku"}, schema_name="schema.Item", schema_id="schema.Item.v1"
)
cache = cache_of(document)
cache.put(document, "k:1", {"sku": "S"})
with self.assertRaises(CultCacheError) as caught:
cache.put(document, "k:2", {"sku": "S"})
self.assertEqual(str(caught.exception), refusal("k:1", "k:2", "schema.Item", "sku"))
def test_the_two_smallest_keys_are_named_in_ordinal_order(self) -> None:
document = item_doc()
cache = cache_of(document)
cache.put(document, "item:m", Item(sku="S"))
with self.assertRaises(CultCacheError) as caught:
cache.put_envelopes(document, [envelope(document, "item:z", Item(sku="S")), envelope(document, "item:A", Item(sku="S"))])
self.assertEqual(str(caught.exception), refusal("item:A", "item:m", "item", "sku"))
def test_put_envelopes_holding_one_value_twice_is_refused_whole(self) -> None:
document = item_doc()
store = RecordingStore()
cache = cache_of(document, store=store)
with self.assertRaises(CultCacheError) as caught:
cache.put_envelopes(
document,
[
envelope(document, "item:b", Item(sku="S")),
envelope(document, "item:c", Item(sku="T")),
envelope(document, "item:a", Item(sku="S")),
],
)
self.assertEqual(str(caught.exception), refusal("item:a", "item:b", "item", "sku"))
self.assertEqual(cache.snapshot(), {})
self.assertEqual(store.pushed, [])
self.assertIsNone(cache.get_key_by_index(document, "sku", "T"))
def test_verdict_does_not_depend_on_batch_order(self) -> None:
document = item_doc()
rows = [("item:a", Item(sku="S", serial="1")), ("item:b", Item(sku="T", serial="1")), ("item:c", Item(sku="U"))]
for order in itertools.permutations(rows):
with self.subTest(order=[key for key, _ in order]):
cache = cache_of(document)
with self.assertRaises(CultCacheError) as caught:
cache.put_envelopes(document, [envelope(document, key, item) for key, item in order])
self.assertEqual(str(caught.exception), refusal("item:a", "item:b", "item", "serial"))
self.assertEqual(cache.snapshot(), {})
def test_put_envelopes_swapping_two_records_values_lands(self) -> None:
document = item_doc()
cache = cache_of(document)
cache.put(document, "item:a", Item(sku="S", serial="1"))
cache.put(document, "item:b", Item(sku="T", serial="2"))
cache.put_envelopes(
document,
[envelope(document, "item:a", Item(sku="T", serial="2")), envelope(document, "item:b", Item(sku="S", serial="1"))],
)
self.assertEqual(cache.get_key_by_index(document, "sku", "S"), "item:b")
self.assertEqual(cache.get_key_by_index(document, "sku", "T"), "item:a")
self.assertEqual(cache.get_key_by_index(document, "serial", "1"), "item:b")
self.assertEqual(cache.get_key_by_index(document, "serial", "2"), "item:a")
def test_a_batch_may_take_a_value_the_batch_releases(self) -> None:
document = item_doc()
cache = cache_of(document)
cache.put(document, "item:a", Item(sku="S"))
cache.put_envelopes(document, [envelope(document, "item:a", Item(sku="T")), envelope(document, "item:b", Item(sku="S"))])
self.assertEqual(cache.get_key_by_index(document, "sku", "S"), "item:b")
self.assertEqual(cache.get_key_by_index(document, "sku", "T"), "item:a")
def test_delete_frees_the_value(self) -> None:
document = item_doc()
cache = cache_of(document)
cache.put(document, "item:a", Item(sku="S"))
cache.delete(document, "item:a")
self.assertIsNone(cache.get_key_by_index(document, "sku", "S"))
cache.put(document, "item:b", Item(sku="S"))
self.assertEqual(cache.get_key_by_index(document, "sku", "S"), "item:b")
def test_blank_values_never_conflict(self) -> None:
document = item_doc()
cache = cache_of(document)
blanks = ["", " ", "\x85", " ", None]
for number, blank in enumerate(blanks):
cache.put(document, f"item:blank{number}", Item(sku=blank, serial=blank))
cache.put_envelopes(document, [envelope(document, "item:more", Item(sku=" ")), envelope(document, "item:more2", Item(sku=""))])
for blank in ("", " ", "\x85"):
self.assertIsNone(cache.get_key_by_index(document, "sku", blank))
# U+001F and U+FEFF are values.
cache.put(document, "item:us", Item(sku="\x1f"))
cache.put(document, "item:bom", Item(sku=""))
self.assertEqual(cache.get_key_by_index(document, "sku", "\x1f"), "item:us")
self.assertEqual(cache.get_key_by_index(document, "sku", ""), "item:bom")
with self.assertRaises(CultCacheError):
cache.put(document, "item:us2", Item(sku="\x1f"))
def test_values_compare_as_the_canonical_text(self) -> None:
document = item_doc()
cache = cache_of(document)
cache.put(document, "item:a", Item(sku=1))
for same in (1, 1.0, "1"):
with self.subTest(same=same), self.assertRaises(CultCacheError):
cache.put(document, "item:b", Item(sku=same))
cache.put(document, "item:true", Item(sku=True))
self.assertEqual(cache.get_key_by_index(document, "sku", "True"), "item:true")
self.assertEqual(cache.get_key_by_index(document, "sku", "1"), "item:a")
cache.put(document, "item:f", Item(sku=0.1))
self.assertEqual(cache.get_key_by_index(document, "sku", "0.1000000000000000055511151231257827021181583404541015625"), "item:f")
def test_two_types_may_share_an_alias_and_a_value(self) -> None:
one, two = item_doc("one"), item_doc("two")
cache = cache_of(one, two)
cache.put(one, "k:1", Item(sku="S", serial="1"))
cache.put(two, "k:2", Item(sku="S", serial="1"))
self.assertEqual(cache.get_key_by_index(one, "sku", "S"), "k:1")
self.assertEqual(cache.get_key_by_index(two, "sku", "S"), "k:2")
cache.delete(one, "k:1")
self.assertEqual(cache.get_key_by_index(two, "sku", "S"), "k:2")
def test_roles_may_share_values_and_are_not_looked_up(self) -> None:
document = item_doc()
cache = cache_of(document)
cache.put(document, "item:a", Item(sku="1", group="g"))
cache.put(document, "item:b", Item(sku="2", group="g"))
self.assertIsNone(cache.get_key_by_index(document, "group", "g"))
self.assertIsNone(cache.get_by_index(document, "group", "g"))
def test_an_alias_in_both_indexes_and_roles_is_refused(self) -> None:
with self.assertRaisesRegex(ValueError, "both an index and a role"):
define_document_type("bad", indexes={"a": "a"}, roles={"a": "a"})
with self.assertRaisesRegex(ValueError, "both an index and a role"):
define_database_entry_type("bad", [("a", 0)], indexes={"a": "a"}, roles={"a": "a"})
define_document_type("fine", indexes={"a": "a"}, roles={"b": "a"})
def test_register_index_over_held_duplicates_installs_nothing(self) -> None:
document = item_doc(indexes={}, roles={})
cache = cache_of(document)
cache.put(document, "item:b", Item(sku="S", serial="1"))
cache.put(document, "item:a", Item(sku="S", serial="2"))
with self.assertRaises(CultCacheError) as caught:
cache.register_index(document, "sku", "sku")
self.assertEqual(str(caught.exception), refusal("item:a", "item:b", "item", "sku"))
self.assertIsNone(cache.get_key_by_index(document, "sku", "S"))
cache.register_index(document, "serial", "serial") # an installed alias is not poisoned by the refused one
self.assertEqual(cache.get_key_by_index(document, "serial", "2"), "item:a")
# Nothing was installed, so a write sharing the refused alias's value is still admitted.
cache.put(document, "item:c", Item(sku="S", serial="3"))
def test_registration_names_an_installed_alias_a_caller_mutated_into_a_collision(self) -> None:
document = item_doc(indexes={"sku": "sku"}, roles={})
cache = cache_of(document)
cache.put(document, "item:x", Item(sku="1", serial="s1"))
cache.put(document, "item:y", Item(sku="2", serial="s2"))
cache.get_required(document, "item:y").sku = "1" # the cache hands out its held value
with self.assertRaises(CultCacheError) as caught:
cache.register_index(document, "serial", "serial")
self.assertEqual(str(caught.exception), refusal("item:x", "item:y", "item", "sku"))
def test_derive_names_the_ordinal_first_alias_then_value(self) -> None:
document = item_doc(indexes={}, roles={})
cache = cache_of(document)
cache.put(document, "item:1", Item(sku="a", serial="p", group="x"))
cache.put(document, "item:2", Item(sku="a", serial="p", group="x"))
cache.put(document, "item:3", Item(sku="b", serial="q", group="y"))
cache.put(document, "item:4", Item(sku="b", serial="q", group="y"))
with self.assertRaises(CultCacheError) as caught:
cache.register_index(document, "serial", "serial")
self.assertEqual(str(caught.exception), refusal("item:1", "item:2", "item", "serial"))
def test_a_document_in_two_unique_indexes_loads_and_registers_through_every_path(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "items.cc"
document = item_doc()
cache = cache_of(document, store=SingleFileMessagePackBackingStore(path))
cache.put(document, "item:a", Item(sku="S", serial="1"))
cache.put(document, "item:b", Item(sku="T", serial="2"))
# a store written by a looser definition, then opened by this one
loose = item_doc(indexes={}, roles={})
writer = cache_of(loose, store=SingleFileMessagePackBackingStore(path))
writer.pull_all_backing_stores()
writer.put(loose, "item:c", Item(sku="U", serial="2"))
before = path.read_bytes()
reader = cache_of(document, store=SingleFileMessagePackBackingStore(path))
with self.assertRaises(CultCacheError) as caught:
reader.pull_all_backing_stores()
self.assertEqual(str(caught.exception), refusal("item:b", "item:c", "item", "serial"))
self.assertEqual(path.read_bytes(), before)
self.assertEqual(reader.snapshot(), {})
def test_a_refused_load_keeps_the_held_state(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "items.cc"
loose = item_doc(indexes={}, roles={})
writer = cache_of(loose, store=SingleFileMessagePackBackingStore(path))
writer.put(loose, "item:a", Item(sku="S"))
writer.put(loose, "item:b", Item(sku="S"))
document = item_doc()
cache = cache_of(document, store=SingleFileMessagePackBackingStore(path))
with self.assertRaises(CultCacheError):
cache.pull_all_backing_stores()
self.assertIsNone(cache.get_key_by_index(document, "sku", "S"))
self.assertEqual(cache.get_all(document), [])
class SimulationFactTests(unittest.TestCase):
def test_simulation_fact_catalog_entry_is_unchanged_by_indexes_to_roles(self) -> None:
import re
from cultnet_py.cultmesh_contracts import SIMULATION_FACT_DOCUMENT_TYPE, simulation_fact_document as document
aliases = {"shard": "shard_id", "frame": "frame", "subject": "subject_id", "claim_kind": "claim_kind"}
self.assertEqual(document.roles, aliases)
self.assertEqual(document.indexes, {})
entry = document.catalog_entry()
snake = lambda name: re.sub(r"(?<!^)(?=[A-Z])", "_", name).lower() # noqa: E731
twin = define_database_entry_type(
SIMULATION_FACT_DOCUMENT_TYPE,
[(snake(m.member_name), m.slot) for m in entry.members],
name="fact_id",
indexes=aliases,
schema_name=entry.schema_name,
schema_version=entry.schema_version,
)
self.assertEqual(entry.canonical_schema_json, twin.catalog_entry().canonical_schema_json)
self.assertEqual(entry.content_hash, twin.catalog_entry().content_hash)
self.assertEqual(entry.members, twin.catalog_entry().members)
self.assertEqual({m.index_alias for m in entry.members if m.index_alias}, set(aliases))
def test_facts_sharing_every_role_value_are_all_admitted(self) -> None:
from cultnet_py.cultmesh_contracts import CultMeshSimulationFact, simulation_fact_document as document
cache = cache_of(document)
for number in range(3):
fact = CultMeshSimulationFact.from_candidate(
{"shardId": "s", "shardEpoch": 1, "frame": 4, "subjectId": "x", "claimKind": "k", "claimHash": f"h{number}",
"witnessCount": 1, "supportWeight": 1.0, "totalWeight": 1.0, "confidence": 1.0}
)
cache.put(document, f"simulation:{number}", fact)
self.assertEqual(len(cache.get_all(document)), 3)
self.assertIsNone(cache.get_key_by_index(document, "shard", "s"))
class VectorTests(unittest.TestCase):
def test_values_rows(self) -> None:
document = item_doc(indexes={"sku": "sku"}, roles={})
for label, kind, value, text in msgpack.unpackb((VECTORS / "values.msgpack").read_bytes(), raw=False):
with self.subTest(label=label):
self.assertEqual(index_text(value), text)
cache = cache_of(document)
cache.put(document, "item:1", Item(sku=value))
if text is None:
cache.put(document, "item:2", Item(sku=value)) # blank: two records commit
self.assertEqual(len(cache.get_all(document)), 2)
else:
self.assertEqual(cache.get_key_by_index(document, "sku", text), "item:1")
with self.assertRaises(CultCacheError):
cache.put(document, "item:2", Item(sku=value))
def test_store_verdicts(self) -> None:
unit = define_database_entry_type
outcomes = msgpack.unpackb((VECTORS / "outcomes.msgpack").read_bytes(), raw=False)
self.assertEqual(len(outcomes), 3)
for name, alias, verdict, key_a, key_b in outcomes:
for roles_first in (False, True): # the verdict does not depend on declaration order
with self.subTest(file=name, alias=alias, roles_first=roles_first), tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / name
shutil.copyfile(VECTORS / name, path)
before = path.read_bytes()
fields = [database_entry_field("sku", 0, codec=t.string), database_entry_field("category", 1, codec=t.string)]
declared = {"indexes": {alias: "sku"} if alias else {}, "roles": {} if alias else {"Sku": "sku"}}
if roles_first:
declared = dict(reversed(list(declared.items())))
document = unit(
"tests.unique_item",
fields,
schema_id="sha256:93bb890de83069061af696b23f94fd32fb3c2dad3c91ee645e8add56085f1882",
schema_name="tests.unique_item",
**declared,
)
cache = cache_of(document, store=SingleFileMessagePackBackingStore(path))
if verdict == "open":
cache.pull_all_backing_stores()
self.assertGreaterEqual(len(cache.get_all(document)), 2)
else:
with self.assertRaises(CultCacheError) as caught:
cache.pull_all_backing_stores()
self.assertEqual(str(caught.exception), refusal(key_a, key_b, "tests.unique_item", alias))
self.assertNotIn("S-1", str(caught.exception))
self.assertEqual(cache.snapshot(), {})
self.assertEqual(path.read_bytes(), before)
def test_refusals(self) -> None:
ids = {
"tests.unique_item": "sha256:93bb890de83069061af696b23f94fd32fb3c2dad3c91ee645e8add56085f1882",
"tests.uq_pair": "sha256:2538cceafb6dcbce22ef11b932ecb804af5f96bc3aaedff692a52d3640433e60",
"tests.uq_alias": "sha256:77a77a394ebd8bf71e14b9b6f2cea4978c25d4e705fd12364f660edbf6c63b1f",
"tests.uq_wide": "sha256:b79eecd11eed5e456902f095d2748da7f59125c4c8972b0da7dddc81d07fc03f",
"tests.uq_alpha": "sha256:53aeaf2ea9b25e85ade6b15fc04c7653e7b206c87e258f1aa74dd32a840a1389",
"tests.uq_beta": "sha256:45ed62a64c91f6d815d3ec975c4a4a1b2ce07b6e7262a32d828673cf10899586",
}
aliases = {
"tests.uq_pair": {"Sku": "sku", "Alt": "second"},
"tests.uq_alias": {"Sku": "sku", "alt": "second"},
"tests.uq_wide": {"\uff21": "sku", "\U00010400": "second"}, # U+FF21 and U+10400: code-unit order puts the astral alias first
}
documents = {
name: define_database_entry_type(
name,
[database_entry_field("sku", 0, codec=t.string), database_entry_field("second", 1, codec=t.string)],
indexes=aliases.get(name, {"Sku": "sku"}),
roles={},
schema_id=schema_id,
schema_name=name,
)
for name, schema_id in ids.items()
}
rows = msgpack.unpackb((VECTORS / "refusals.msgpack").read_bytes(), raw=False)
walked = 0
for label, ops, expected in rows:
if any(op[0] == "commit" and len({schema for _, schema, _, _ in op[1]}) > 1 for op in ops):
continue # put_envelopes takes one type: no batch across types (rows 3 and 26)
with self.subTest(label=label), tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "store.cc"
state: dict[str, CultCache] = {}
def perform(op) -> None:
if op[0] == "open":
if op[1] is not None:
shutil.copyfile(VECTORS / op[1], path)
state["cache"] = cache_of(*documents.values(), store=SingleFileMessagePackBackingStore(path))
state["cache"].pull_all_backing_stores()
elif op[0] == "pull":
shutil.copyfile(VECTORS / op[1], path)
state["cache"].pull_all_backing_stores()
else:
document = documents[op[1][0][1]]
state["cache"].put_envelopes(
document, [envelope(document, key, {"sku": sku, "second": second}) for key, _, sku, second in op[1]]
)
for op in ops[:-1]:
perform(op)
if expected is None:
perform(ops[-1])
else:
last = ops[-1]
if last[0] == "open":
before = (VECTORS / last[1]).read_bytes()
elif last[0] == "pull":
before = (VECTORS / last[1]).read_bytes()
else:
before = path.read_bytes() if path.exists() else b""
k1, k2, schema, alias = expected
with self.assertRaises(CultCacheError) as caught:
perform(last)
self.assertEqual(str(caught.exception), refusal(k1, k2, schema, alias))
self.assertEqual(path.read_bytes() if path.exists() else b"", before)
walked += 1
self.assertEqual(walked, 35)
class ListStore(RecordingStore):
"""A store whose contents the test swaps between loads: another process rewrote the file."""
def __init__(self, envelopes=()) -> None:
super().__init__()
self.envelopes = list(envelopes)
def pull_all(self):
return list(self.envelopes)
class BatchAndOrderTests(unittest.TestCase):
def test_a_batch_naming_a_key_twice_lands_the_last_record_and_keeps_every_held_lookup(self) -> None:
# C# CultCacheBatch stages one operation per key, the last one named.
document = item_doc()
store = RecordingStore()
cache = cache_of(document, store=store)
cache.put(document, "item:h", Item(sku="S"))
store.pushed.clear()
cache.put_envelopes(
document, [envelope(document, "item:k", Item(sku="S")), envelope(document, "item:k", Item(sku="T"))]
)
self.assertEqual(store.pushed, ["item:k"])
self.assertEqual(cache.get_required(document, "item:k").sku, "T")
self.assertEqual(cache.get_key_by_index(document, "sku", "S"), "item:h")
self.assertEqual(cache.get_key_by_index(document, "sku", "T"), "item:k")
with self.assertRaises(CultCacheError) as caught:
cache.put(document, "item:z", Item(sku="S"))
self.assertEqual(str(caught.exception), refusal("item:h", "item:z", "item", "sku"))
def test_a_batch_naming_a_key_twice_is_judged_on_the_record_that_lands(self) -> None:
document = item_doc()
cache = cache_of(document)
cache.put(document, "item:h", Item(sku="S"))
# the first record for k would collide with h; only the last lands, so the batch is admitted
cache.put_envelopes(
document, [envelope(document, "item:k", Item(sku="S")), envelope(document, "item:k", Item(sku="T"))]
)
# and the reverse order lands the colliding record, which is refused
with self.assertRaises(CultCacheError):
cache.put_envelopes(
document, [envelope(document, "item:j", Item(sku="U")), envelope(document, "item:j", Item(sku="S"))]
)
self.assertNotIn("item:j", cache.snapshot().get("item", {}))
def test_a_load_judges_every_type_at_once_aliases_in_ordinal_order(self) -> None:
first = item_doc("first", indexes={"zeta": "sku"}, roles={})
second = item_doc("second", indexes={"alpha": "sku"}, roles={})
loose_first, loose_second = item_doc("first", indexes={}, roles={}), item_doc("second", indexes={}, roles={})
store = ListStore(
[envelope(loose_first, "f:1", Item(sku="S")), envelope(loose_first, "f:2", Item(sku="S"))]
+ [envelope(loose_second, "s:1", Item(sku="S")), envelope(loose_second, "s:2", Item(sku="S"))]
)
cache = cache_of(first, second, store=store)
with self.assertRaises(CultCacheError) as caught:
cache.pull_all_backing_stores()
self.assertEqual(str(caught.exception), refusal("s:1", "s:2", "second", "alpha"))
def test_aliases_sort_by_utf16_code_unit_not_code_point(self) -> None:
astral, bmp = "k\U0001F600", "k\uff01" # code points: bmp < astral; UTF-16 units: D83D < FF01
document = item_doc(indexes={bmp: "sku", astral: "serial"}, roles={})
cache = cache_of(document)
rows = [Item(sku="S", serial="T"), Item(sku="S", serial="T")]
with self.assertRaises(CultCacheError) as caught:
cache.put_envelopes(document, [envelope(document, f"item:{n}", row) for n, row in enumerate(rows)])
self.assertEqual(str(caught.exception), refusal("item:0", "item:1", "item", astral))
def test_a_load_names_the_least_violation_whatever_order_the_store_yields(self) -> None:
alpha, beta = item_doc("alpha", indexes={"k": "sku"}, roles={}), item_doc("beta", indexes={"k": "sku"}, roles={})
loose_alpha, loose_beta = item_doc("alpha", indexes={}, roles={}), item_doc("beta", indexes={}, roles={})
store = ListStore(
[
envelope(loose_alpha, "a:0", Item(sku="Q")),
envelope(loose_beta, "b:1", Item(sku="S")),
envelope(loose_beta, "b:2", Item(sku="S")),
envelope(loose_alpha, "a:1", Item(sku="P")),
envelope(loose_alpha, "a:2", Item(sku="P")),
]
)
cache = cache_of(alpha, beta, store=store)
with self.assertRaises(CultCacheError) as caught:
cache.pull_all_backing_stores()
self.assertEqual(str(caught.exception), refusal("a:1", "a:2", "alpha", "k"))
def test_register_index_refuses_an_alias_the_definition_declares_as_a_role(self) -> None:
document = item_doc()
cache = cache_of(document)
with self.assertRaises(CultCacheError):
cache.register_index(document, "group", "group")
cache.put_envelopes(
document, [envelope(document, "item:1", Item(group="guide")), envelope(document, "item:2", Item(group="guide"))]
)
class LoadFreshnessTests(unittest.TestCase):
def test_a_load_is_judged_on_the_file_alone_not_against_the_held_map(self) -> None:
document = item_doc()
store = ListStore([envelope(document, "item:b", Item(sku="S")), envelope(document, "item:c", Item(sku="T"))])
cache = cache_of(document, store=store)
cache.pull_all_backing_stores()
self.assertEqual(cache.get_key_by_index(document, "sku", "S"), "item:b")
# another process moved S from b to a, and b to T's old place
store.envelopes = [envelope(document, "item:a", Item(sku="S")), envelope(document, "item:b", Item(sku="U")), store.envelopes[1]]
cache.pull_all_backing_stores()
self.assertEqual(cache.get_key_by_index(document, "sku", "S"), "item:a")
self.assertEqual(cache.get_key_by_index(document, "sku", "U"), "item:b")
class DefinitionTests(unittest.TestCase):
def test_every_constructor_refuses_an_alias_in_both_indexes_and_roles(self) -> None:
from dataclasses import replace
from cultcache_py import DocumentDefinition
with self.assertRaisesRegex(ValueError, "both an index and a role"):
DocumentDefinition(type="item", indexes={"sku": "sku"}, roles={"sku": "sku"})
with self.assertRaisesRegex(ValueError, "both an index and a role"):
replace(item_doc(roles={}), roles={"sku": "sku"})
if __name__ == "__main__":
unittest.main()

View file

@ -8,12 +8,22 @@ only in git history.
### Breaking ### Breaking
- A single-file store written as `cultcache.store.v3` is read and written: a record that holds an element id (an owned list element with an `[CultElementId]` member, carried through every write) marks the store v3, and a v3 store is carried through a rewrite with its ids. A runtime from before this change refuses a v3 store by name. - A single-file store written as `cultcache.store.v3` is read and written: a store the C# runtime marked v3 (it holds a record with an element id: an owned list element with a `[CultElementId]` member) keeps its header here, and is carried through every rewrite with its ids; this runtime never drops the mark. A runtime from before this change refuses a v3 store by name. Rolling back to an older release after a store has been written as v3 is one-way: there is no carry-back, so restore from a copy taken before upgrading.
- A write over a file the reader refuses (a zero-byte file, a truncated store, a stamp this runtime does not read) refuses with `CultCacheStoreUnreadable` and leaves the file as it is. A whole-store write used to replace such a file. This reverses 0.3.1, where `push_all` replaced a file with no readable stamp (healing it); the stamp rule now has one owner, the reader every write asks under the lock, and it accepts v1 and v3 (0.3.1 refused v3). - A write over a file the reader refuses (a zero-byte file, a truncated store, a stamp this runtime does not read) refuses with `CultCacheStoreUnreadable` and leaves the file as it is. A whole-store write used to replace such a file. This reverses 0.3.1, where `push_all` replaced a file with no readable stamp (healing it); the stamp rule now has one owner, the reader every write asks under the lock, and it accepts v1 and v3 (0.3.1 refused v3).
- `#[cultcache(index)]` means unique, as in C# `RefuseIndexSharing`: within one type and one alias no two records hold one non-blank value. A put, batch, push or load that would make two records hold a value is refused, naming both keys and the alias, and a store that already holds a duplicate does not open. A grouping index is no longer expressible; a consumer that relied on shared values must drop the `index` and look up by another way.
- `bool` index and lookup text renders `True` / `False`, as C# does, where it rendered `true` / `false`.
- `DatabaseEntry::UNIQUE_INDEXES` replaces `cult_indexes` as the one index declaration and changes type (its members are the unique aliases); a hand implementation overriding `cult_indexes` no longer compiles.
- `register_entry_type` over a type that already holds records re-judges them under the new declaration (unique indexes and the global rule) and refuses, changing nothing, when they violate it. A declaration with an index also refuses when a held record cannot be decoded as the new type; a declaration without one does not read the records, so it does not.
- A single-file store holds one record per key. The v1 and legacy readers refuse a store with two records under one key (`CultCacheStoreUnreadable`, the record `Undecodable`): the store does not open, and a write over it refuses and leaves the file. A consumer that holds such a file must repair it with the runtime that wrote it before opening it here.
- Every single-file writer (`push`, `push_all`, `insert_entry_if_absent`, the compare-exchange and snapshot-append family, and the cache's puts and batches) refuses to put a second type under a key the store holds, with `SchemaConflictError` naming the key and both types, writing nothing. Retyping a key is a delete, then an add (one batch may hold both). A write owns the held record when its schema id equals the id the held record is stored under, as C# `WrittenOwns` asks; a type renamed under a stable id therefore replaces its own record, and a write of another type name under another id is refused (`push` of the same type name under another id still replaces the record, as before; `compare_and_swap_entry`, and a `push_all` batch holding one type under two ids, refuse it). Registered types here declare no compatible ids, so equality is the whole test. Writers used to leave two records under the key.
- `compare_and_swap_entry` over a single file refuses an in-place retype (expected `A/k`, replacement `B/k` under another schema id) with the same `SchemaConflictError`, as `compare_exchange` does, where it used to retype the record and return `true`. A lost race still returns `false` first.
- A cache holds one record per key. `put`, `put_envelope`, `put_raw_envelope` and a batch of a second type under a key the cache holds, or a key a store carries foreign, are refused with `SchemaConflictError` even with no store (a memory-only cache used to hold both), whatever schema id the envelope claims: another registered type's record is never owned. `put_envelope`, `put_raw_envelope`, `put_prepared_batch` and `load_soa` also refuse, at any key, an envelope whose schema id names another type registered in this cache (the id a pull would resolve the record by). An id this cache does not register is trusted as the envelope claims it: at a key this cache does not hold, the single-file store decides ownership by that id, so such an envelope can replace another writer's record stored under it, and a reader registering the type the id names serves the record as that type. Store-level writers called directly (`push`, `insert_entry_if_absent`, compare-and-swap) still trust the claimed id. A refused write or pull changes nothing. Two stores holding one key refuse the pull, naming the key and both types, and so does one store holding two records at a key that resolve to one type. A store holding a record no registered type claims now loads (it used to refuse with "No schema is registered for persisted entry type") and the record is carried, through `put_prepared_batch` too, which lays foreign records back byte for byte; `delete` at a foreign key is refused. `load_soa` refuses a second type at a key or a record at a foreign key, and still refuses an unregistered type. The redb stores still key by `(type, key)`, and their own writers (`push`, `insert_entry_if_absent`, compare-and-swap) still put a second type at a held key: a cache now refuses to pull such a redb store with `SchemaConflictError`, where it used to load both records.
- A write of the same type under a held key replaces the record by key, and in one `push_all` the last given of several entries under a key and a schema id is the record, whatever their type names. The redb stores are unchanged and still key by type and key.
### Added ### Added
- Lookups, matching the C# `GetByName`, `GetByIndex` and `GetGlobal`. `#[cultcache(name)]` and `#[cultcache(index)]` / `#[cultcache(index = "alias")]` on `DatabaseEntry` fields and `#[cultcache(global)]` on the struct give `CultCache::get_by_name`, `get_by_index` and `get_global`. An empty name and a blank index value are not found; records of one type sharing a name or value find the one with the smallest key. A second record of a global type under another key is refused on every write path and on load with `CultCacheSecondGlobal`. Two `name` members, or two members with one index alias, are refused at compile time. A member's text comes from `CultLookupValue` (`String`, `Option` of one, integers, `bool`, `char`). - `CultCache::foreign_records()` and `ForeignRecord { key, schema_id, schema_name, stored_at }`: the records the last pull found that no registered type claims, sorted by key.
- Lookups, matching the C# `GetByName`, `GetByIndex` and `GetGlobal`. `#[cultcache(name)]` and `#[cultcache(index)]` / `#[cultcache(index = "alias")]` on `DatabaseEntry` fields and `#[cultcache(global)]` on the struct give `CultCache::get_by_name`, `get_by_index` and `get_global`. An empty name and a blank index value are not found; records of one type sharing a name find the one with the smallest key (an index value is unique, below). A second record of a global type under another key is refused on every write path and on load with `CultCacheSecondGlobal`. Two `name` members, or two members with one index alias, are refused at compile time. A member's text comes from `CultLookupValue` (`String`, `Option` of one, integers, `bool`, `char`).
- `SchemaConflictError`, the typed refusal of a write the catalog cannot describe: records of different types under one schema id, or an arrived schema and a registered one that share an id and disagree on the schema name. A whole-store write (`push_all`) refuses with it, names the schema id, the names involved and a record key, writes nothing and leaves the file as it is. A caller tells it apart with `error.downcast_ref::<SchemaConflictError>()`. - `SchemaConflictError`, the typed refusal of a write the catalog cannot describe: records of different types under one schema id, or an arrived schema and a registered one that share an id and disagree on the schema name. A whole-store write (`push_all`) refuses with it, names the schema id, the names involved and a record key, writes nothing and leaves the file as it is. A caller tells it apart with `error.downcast_ref::<SchemaConflictError>()`.
## [0.3.1] ## [0.3.1]

View file

@ -21,8 +21,14 @@ CultCache is a domain cache with persistence adapters.
- `CultCache` is the query and mutation surface. - `CultCache` is the query and mutation surface.
- Domain structs implement `DatabaseEntry`. - Domain structs implement `DatabaseEntry`.
- Entries are stored behind a `type::key` identity, so multiple entry types - A cache holds one record per key whatever its type. Records are held per
can share a logical key without colliding. `(type, key)`, but a write of a second type under a key the cache holds, or a
key a store carries foreign, is refused with `SchemaConflictError` naming the
key (no store needed); retyping a key is a delete, then an add. A store
record no registered type claims loads: the cache does not serve it, lists it
in `foreign_records()`, lays it back byte for byte through every write
including `put_prepared_batch`, and refuses a write or delete at its key.
Registering a type that claims its schema id picks it up at the next pull.
- Each payload is encoded directly from the known `DatabaseEntry` type into - Each payload is encoded directly from the known `DatabaseEntry` type into
MessagePack bytes. MessagePack bytes.
- Backing stores are adapters, not the public data model. - Backing stores are adapters, not the public data model.
@ -394,6 +400,10 @@ fn outcome(error: &anyhow::Error) {
} }
``` ```
## Store format and rollback
A store whose elements hold an id is marked `cultcache.store.v3` by the C# runtime. This runtime reads and writes a v3 single-file store, keeps the header it read and carries the ids through a rewrite; it never drops the mark, so it has no carry-back. A cultcache-rs from before element ids refuses a v3 store by name, so rolling back to it after a store has been written as v3 is one-way: restore from a copy taken before upgrading, and upgrade every reader before any writer. (The C# line is different: CultLib 1.x does not refuse; see the GameCult.Caching README.)
## Near-Term Ergonomic Improvements ## Near-Term Ergonomic Improvements
1. **Derive macro** 1. **Derive macro**
@ -412,7 +422,9 @@ fn outcome(error: &anyhow::Error) {
3. **Name and index lookups** 3. **Name and index lookups**
- `cache.get_by_name::<T>("Potion")` - `cache.get_by_name::<T>("Potion")`
- `cache.get_by_index::<T>("faction", "Lucent")` - `cache.get_by_index::<T>("callsign", "Lucent-7")`: `#[cultcache(index)]` is unique. Within one type no two
records hold one non-blank value of the alias; a put, batch or load that would make two is refused, naming both
keys and the alias, and writes nothing. A grouping (faction, kind, status) is not an index: filter `get_all`.
- generated extractors instead of reflection strings - generated extractors instead of reflection strings
4. **Schema/version metadata** 4. **Schema/version metadata**
@ -452,7 +464,7 @@ That is the Rust version of the original trick:
- C# refuses to store objects outside the `DatabaseEntry` root type. - C# refuses to store objects outside the `DatabaseEntry` root type.
- Runtime-visible `DatabaseEntry` subclasses get dedicated formatters. - Runtime-visible `DatabaseEntry` subclasses get dedicated formatters.
- Unknown or unregistered types are rejected. - A write of an unregistered type is rejected; a stored record of an unregistered type is carried as foreign, not decoded.
- MessagePack payload bytes deserialize through known concrete formatters, not - MessagePack payload bytes deserialize through known concrete formatters, not
an open-world type loader or generic structured-value bridge. an open-world type loader or generic structured-value bridge.

View file

@ -69,23 +69,16 @@ fn database_entry_codec(input: DeriveInput) -> syn::Result<proc_macro2::TokenStr
} }
} }
}); });
let index_members = slots.iter().filter_map(|slot| slot.index.as_ref().map(|alias| (alias, &slot.ident))); let index_members: Vec<_> = slots.iter().filter_map(|slot| slot.index.as_ref().map(|alias| (alias, &slot.ident))).collect();
let index_pushes = index_members.map(|(alias, member)| { let index_rows = index_members.iter().map(|(alias, member)| {
quote! { quote! {
if let ::std::option::Option::Some(value) = ::cultcache_rs::CultLookupValue::lookup_value(&self.#member) { (#alias, (|record: &Self| ::cultcache_rs::CultLookupValue::lookup_value(&record.#member))
held.push((#alias, value)); as fn(&Self) -> ::std::option::Option<::std::string::String>)
}
} }
}); });
let index_pushes: Vec<_> = index_pushes.collect(); let index_rows: Vec<_> = index_rows.collect();
let indexes_fn = (!index_pushes.is_empty()).then(|| { let unique_indexes = (!index_rows.is_empty()).then(|| {
quote! { quote! { const UNIQUE_INDEXES: &'static [::cultcache_rs::CultIndexMember<Self>] = &[#(#index_rows),*]; }
fn cult_indexes(&self) -> ::std::vec::Vec<(&'static str, ::std::string::String)> {
let mut held = ::std::vec::Vec::new();
#(#index_pushes)*
held
}
}
}); });
Ok(quote! { Ok(quote! {
@ -93,8 +86,8 @@ fn database_entry_codec(input: DeriveInput) -> syn::Result<proc_macro2::TokenStr
const TYPE: &'static str = #entry_type; const TYPE: &'static str = #entry_type;
const SCHEMA_NAME: &'static str = #schema_name; const SCHEMA_NAME: &'static str = #schema_name;
const GLOBAL: bool = #global; const GLOBAL: bool = #global;
#unique_indexes
#name_fn #name_fn
#indexes_fn
} }
#codec #codec
@ -443,7 +436,8 @@ struct CultCacheSlot {
bytes: bool, bytes: bool,
/// `#[cultcache(name)]`: the member `get_by_name` finds the record by. /// `#[cultcache(name)]`: the member `get_by_name` finds the record by.
name: bool, name: bool,
/// `#[cultcache(index)]` or `#[cultcache(index = "alias")]`: the alias `get_by_index` finds the record by. /// `#[cultcache(index)]` or `#[cultcache(index = "alias")]`: a unique lookup. Within one type no two records hold one
/// non-blank value of the alias; `get_by_index` finds the record by it.
index: Option<String>, index: Option<String>,
} }
@ -941,15 +935,15 @@ mod tests {
); );
assert!(tokens.contains("constGLOBAL:bool=true"), "{tokens}"); assert!(tokens.contains("constGLOBAL:bool=true"), "{tokens}");
assert!(tokens.contains("fncult_name"), "{tokens}"); assert!(tokens.contains("fncult_name"), "{tokens}");
assert!(tokens.contains("(\"code\",value)"), "{tokens}"); assert!(tokens.contains("(\"code\","), "{tokens}");
assert!(tokens.contains("(\"sku\",value)"), "{tokens}"); assert!(tokens.contains("(\"sku\","), "{tokens}");
} }
#[test] #[test]
fn a_record_without_lookups_generates_none_of_them() { fn a_record_without_lookups_generates_none_of_them() {
let tokens = entry_tokens(r#"#[cultcache(type = "t")] struct S { #[cultcache(key = 0)] a: String }"#); let tokens = entry_tokens(r#"#[cultcache(type = "t")] struct S { #[cultcache(key = 0)] a: String }"#);
assert!(tokens.contains("constGLOBAL:bool=false"), "{tokens}"); assert!(tokens.contains("constGLOBAL:bool=false"), "{tokens}");
assert!(!tokens.contains("fncult_name") && !tokens.contains("fncult_indexes"), "{tokens}"); assert!(!tokens.contains("fncult_name") && !tokens.contains("UNIQUE_INDEXES"), "{tokens}");
} }
// ResolveIndexAlias: a blank alias is the member's name. // ResolveIndexAlias: a blank alias is the member's name.
@ -958,7 +952,7 @@ mod tests {
let tokens = entry_tokens( let tokens = entry_tokens(
r#"#[cultcache(type = "t")] struct S { #[cultcache(key = 0, index = " ")] code: String }"#, r#"#[cultcache(type = "t")] struct S { #[cultcache(key = 0, index = " ")] code: String }"#,
); );
assert!(tokens.contains("(\"code\",value)"), "{tokens}"); assert!(tokens.contains("(\"code\","), "{tokens}");
} }
#[test] #[test]

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,283 @@
//! `tests/vectors/key-identity`, written only by the C# reference: a cache holds one record per key whatever its type, and a
//! store holding a record no registered type claims loads, lists it as foreign and keeps its bytes
//! (`variants:ruling:cache-refuses-retype`, `write-ownership-by-written-type`, `unheld-removal-refused-at-commit`).
//! Every row is walked against this cache: each put row twice, once as `put` and once as a one-entry `put_prepared_batch`.
use std::path::{Path, PathBuf};
use cultcache_rs::{
CacheBackingStore, CultCache, DatabaseEntry, SchemaConflictError, SingleFileMessagePackBackingStore,
};
use serde_json::Value;
// The ids C# writes in the store catalogs and on every record (the vector's README).
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
#[cultcache(type = "sha256:b20703a98f49c253898404ab50ba2c630720f0b8a5e1a41614d5650be22799fe", schema = "tests.ki_alpha")]
struct Alpha {
#[cultcache(key = 0)]
text: String,
}
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
#[cultcache(type = "sha256:dee4b727cf2ee04e61ab635ff0c5f3dca7345a411dc5f1018e583ab16009cf94", schema = "tests.ki_beta")]
struct Beta {
#[cultcache(key = 0)]
text: String,
}
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
#[cultcache(type = "sha256:fbc26e351e00c9e7185e8de171c7486aba707c229c949f0aae6b889288af5f6d", schema = "tests.ki_gamma")]
struct Gamma {
#[cultcache(key = 0)]
text: String,
}
fn vectors() -> PathBuf {
PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../tests/vectors/key-identity")
}
/// `[refusal key or nil, held [type, key, text], file [key, schema name] or nil, foreign keys, kept foreign keys]`.
type Expect = (Option<Vec<String>>, Vec<(String, String, String)>, Option<Vec<(String, String)>>, Vec<String>, Vec<String>);
type Row = (String, Vec<String>, Vec<Vec<Value>>, Expect);
fn text(value: &Value) -> &str {
value.as_str().unwrap()
}
fn register(cache: &mut CultCache, names: &[String]) {
for name in names {
match name.as_str() {
"alpha" => drop(cache.register_entry_type::<Alpha>().unwrap()),
"beta" => drop(cache.register_entry_type::<Beta>().unwrap()),
"gamma" => drop(cache.register_entry_type::<Gamma>().unwrap()),
other => panic!("unknown type {other}"),
}
}
}
fn short(entry_type: &str) -> &'static str {
match entry_type {
t if t == Alpha::TYPE => "alpha",
t if t == Beta::TYPE => "beta",
t if t == Gamma::TYPE => "gamma",
other => panic!("unknown entry type {other}"),
}
}
fn put(cache: &mut CultCache, as_batch: bool, kind: &str, key: &str, body: &str) -> anyhow::Result<()> {
macro_rules! write {
($t:ident) => {{
let value = $t { text: body.to_string() };
if as_batch {
let (entry, _) = cache.prepare_entry(key, &value)?;
cache.put_prepared_batch(vec![entry])
} else {
cache.put(key, &value).map(drop)
}
}};
}
match kind {
"alpha" => write!(Alpha),
"beta" => write!(Beta),
"gamma" => write!(Gamma),
other => panic!("unknown type {other}"),
}
}
fn delete(cache: &mut CultCache, kind: &str, key: &str) -> anyhow::Result<bool> {
match kind {
"alpha" => cache.delete::<Alpha>(key),
"beta" => cache.delete::<Beta>(key),
"gamma" => cache.delete::<Gamma>(key),
other => panic!("unknown type {other}"),
}
}
fn open_cache(registered: &[String], path: Option<&Path>) -> anyhow::Result<CultCache> {
let mut cache = CultCache::new();
register(&mut cache, registered);
if let Some(path) = path {
cache.add_generic_backing_store(SingleFileMessagePackBackingStore::new(path))?;
cache.pull_all_backing_stores()?;
}
Ok(cache)
}
/// The records a store file holds, read by a fresh store: (key, schema name, schema id, stored-at, payload).
fn file_records(path: &Path) -> Vec<(String, String, Option<String>, String, Vec<u8>)> {
let mut records: Vec<_> = SingleFileMessagePackBackingStore::new(path)
.pull_all()
.unwrap()
.into_iter()
.map(|entry| (entry.key, entry.r#type, entry.schema_id, entry.stored_at, entry.payload))
.collect();
records.sort_by(|left, right| left.0.encode_utf16().cmp(right.0.encode_utf16()));
records
}
fn walk(row: &Row, as_batch: bool) -> Result<(), String> {
let (label, registered, ops, expect) = row;
let dir = tempfile::tempdir().unwrap();
let mut path: Option<PathBuf> = None;
let mut before = Vec::new();
let mut cache: Option<CultCache> = None;
let mut refused: Option<anyhow::Error> = None;
let mut refused_key: Option<String> = None;
let mut shared_key: Option<String> = None;
let mut wrote = false;
for (index, op) in ops.iter().enumerate() {
let last = index == ops.len() - 1;
let result: anyhow::Result<()> = match text(&op[0]) {
"open" => {
drop(cache.take());
match &op[1] {
Value::String(file) if file == "memory" => open_cache(registered, None).map(|opened| cache = Some(opened)),
file => {
let fresh = dir.path().join("store.cc");
if let Value::String(file) = file {
std::fs::copy(vectors().join(file), &fresh).unwrap();
}
before = if fresh.exists() { file_records(&fresh) } else { Vec::new() };
let opened = open_cache(registered, Some(&fresh));
path = Some(fresh);
opened.map(|opened| cache = Some(opened))
}
}
}
"reopen" => {
drop(cache.take());
open_cache(registered, path.as_deref()).map(|opened| cache = Some(opened))
}
"attach" => {
let file = text(&op[1]);
let attached = dir.path().join(file);
std::fs::copy(vectors().join(file), &attached).unwrap();
let held: Vec<String> = cache.as_ref().unwrap().snapshot().into_iter().map(|entry| entry.key).collect();
let shared: Vec<String> = file_records(&attached).into_iter().map(|record| record.0).filter(|key| held.contains(key)).collect();
let home: Vec<String> = op[2].as_array().unwrap().iter().map(|name| short_to_type(text(name)).to_string()).collect();
let cache = cache.as_mut().unwrap();
let attach = cache
.add_backing_store(SingleFileMessagePackBackingStore::new(&attached), home)
.and_then(|()| cache.pull_all_backing_stores());
if attach.is_err() && last && shared.len() == 1 {
refused_key = Some(shared[0].clone());
shared_key = refused_key.clone();
assert_eq!(std::fs::read(&attached).unwrap(), std::fs::read(vectors().join(file)).unwrap(), "{label}: a refused attach leaves the store as it was");
}
attach
}
"put" => {
wrote = true;
put(cache.as_mut().unwrap(), as_batch, text(&op[1]), text(&op[2]), text(&op[3]))
}
"delete" => delete(cache.as_mut().unwrap(), text(&op[1]), text(&op[2])).map(drop),
other => panic!("{label}: unknown op {other}"),
};
match result {
Ok(()) => {}
Err(error) if last => refused = Some(error),
Err(error) => return Err(format!("{label}: op {index} failed: {error:#}")),
}
}
if !wrote && as_batch {
return Ok(()); // not a write row: the batch variant is the same walk
}
let Some(cache) = cache.as_ref() else {
return Err(format!("{label} (batch {as_batch}): the cache did not open: {:#}", refused.unwrap()));
};
let last = ops.last().unwrap();
if let Some(error) = &refused {
if let Some(conflict) = error.downcast_ref::<SchemaConflictError>() {
refused_key = Some(conflict.record_key.clone());
let named = if text(&last[0]) == "attach" { shared_key.clone().unwrap() } else { text(&last[2]).to_string() };
assert_eq!(named, conflict.record_key, "{label}: the refusal names the key written, removed or shared");
}
let key = refused_key.clone().unwrap_or_default();
let message = format!("{error:#}");
let (all, delimited) = occurrences(&message, &key);
assert!(!key.is_empty() && delimited > 0, "{label}: the refusal names the key as a token: {message}");
if text(&last[0]) == "attach" {
assert_eq!(all, delimited, "{label}: the key occurs in the refusal only as a delimited token: {message}");
}
}
let mut held: Vec<(String, String, String)> = cache
.snapshot()
.into_iter()
.map(|entry| {
let body = match short(&entry.r#type) {
"alpha" => cache.get::<Alpha>(&entry.key).unwrap().unwrap().text,
"beta" => cache.get::<Beta>(&entry.key).unwrap().unwrap().text,
_ => cache.get::<Gamma>(&entry.key).unwrap().unwrap().text,
};
(short(&entry.r#type).to_string(), entry.key, body)
})
.collect();
held.sort_by(|a, b| (a.0.as_str(), a.1.encode_utf16().collect::<Vec<_>>()).cmp(&(b.0.as_str(), b.1.encode_utf16().collect::<Vec<_>>())));
let foreign: Vec<String> = cache.foreign_records().into_iter().map(|record| record.key).collect();
let (file, kept) = match &path {
None => (None, Vec::new()),
Some(path) => {
let after = file_records(path);
let kept = foreign
.iter()
.filter(|key| {
let original = before.iter().find(|record| &record.0 == *key);
let now = after.iter().find(|record| &record.0 == *key);
original.is_some() && original.map(|r| (&r.2, &r.3, &r.4)) == now.map(|r| (&r.2, &r.3, &r.4))
})
.cloned()
.collect();
// This writer names a catalog entry by the record's type, which for these types is the id; C# names it by the
// schema name. Which type the record at a key is, is the verdict, so the id reads as its schema name.
let name = |described: String| match described.as_str() {
t if t == Alpha::TYPE => Alpha::SCHEMA_NAME.to_string(),
t if t == Beta::TYPE => Beta::SCHEMA_NAME.to_string(),
t if t == Gamma::TYPE => Gamma::SCHEMA_NAME.to_string(),
_ => described,
};
(Some(after.into_iter().map(|record| (record.0, name(record.1))).collect::<Vec<_>>()), kept)
}
};
let observed: Expect = (refused_key.map(|key| vec![key]), held, file, foreign, kept);
if &observed == expect { Ok(()) } else { Err(format!("{label} (batch {as_batch}): committed {expect:?} but this cache answers {observed:?}")) }
}
/// How often `key` occurs in `message`, and how often as a delimited token (not a piece of a longer word, schema name or id).
fn occurrences(message: &str, key: &str) -> (usize, usize) {
let word = |c: Option<char>| c.is_some_and(|c| c.is_ascii_alphanumeric() || c == '_');
let found: Vec<usize> = message.match_indices(key).map(|(at, _)| at).collect();
let delimited = found
.iter()
.filter(|at| !word(message[..**at].chars().next_back()) && !word(message[**at + key.len()..].chars().next()))
.count();
(found.len(), delimited)
}
fn short_to_type(name: &str) -> &'static str {
match name {
"alpha" => Alpha::TYPE,
"beta" => Beta::TYPE,
"gamma" => Gamma::TYPE,
other => panic!("unknown type {other}"),
}
}
#[test]
fn key_identity_rows_hold_their_verdicts() {
let bytes = std::fs::read(vectors().join("rows.msgpack")).unwrap();
let rows: Vec<Row> = rmp_serde::from_slice(&bytes).unwrap();
let mut walked = 0;
let mut failed = Vec::new();
for row in &rows {
let mut ok = true;
for as_batch in [false, true] {
if let Err(message) = walk(row, as_batch) {
failed.push(message);
ok = false;
}
}
walked += usize::from(ok);
}
assert!(failed.is_empty(), "{}", failed.join("\n"));
assert_eq!(walked, 12, "rows walked");
}

View file

@ -11,7 +11,7 @@ use cultcache_rs::{
struct Item { struct Item {
#[cultcache(key = 0, name)] #[cultcache(key = 0, name)]
name: String, name: String,
#[cultcache(key = 1, index)] #[cultcache(key = 1)]
category: String, category: String,
#[cultcache(key = 2, index = "sku")] #[cultcache(key = 2, index = "sku")]
code: Option<String>, code: Option<String>,
@ -30,7 +30,7 @@ struct Gadget {
#[cultcache(key = 0, name)] #[cultcache(key = 0, name)]
name: String, name: String,
#[cultcache(key = 1, index)] #[cultcache(key = 1, index)]
category: String, serial: String,
} }
/// Sorts after `Item` and has its exact shape: a lookup that ran past the end of its own type would answer from it. /// Sorts after `Item` and has its exact shape: a lookup that ran past the end of its own type would answer from it.
@ -39,7 +39,7 @@ struct Gadget {
struct Zed { struct Zed {
#[cultcache(key = 0, name)] #[cultcache(key = 0, name)]
name: String, name: String,
#[cultcache(key = 1, index)] #[cultcache(key = 1)]
category: String, category: String,
#[cultcache(key = 2, index = "sku")] #[cultcache(key = 2, index = "sku")]
code: Option<String>, code: Option<String>,
@ -48,7 +48,7 @@ struct Zed {
} }
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)] #[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
#[cultcache(type = "settings", global)] #[cultcache(type = "settings", schema = "tests.settings", global)]
struct Settings { struct Settings {
#[cultcache(key = 0)] #[cultcache(key = 0)]
value: String, value: String,
@ -71,7 +71,7 @@ struct Numbered {
#[cultcache(key = 0, name)] #[cultcache(key = 0, name)]
id: u32, id: u32,
#[cultcache(key = 1, index)] #[cultcache(key = 1, index)]
owner: u64, badge: u64,
} }
fn cache() -> CultCache { fn cache() -> CultCache {
@ -110,7 +110,6 @@ fn a_record_is_found_by_name_and_by_index_after_a_reopen() {
let reopened = reopened(&path); let reopened = reopened(&path);
let by_name = reopened.get_by_name::<Item>("Potion").unwrap().unwrap(); let by_name = reopened.get_by_name::<Item>("Potion").unwrap().unwrap();
assert_eq!(by_name.value, 50); assert_eq!(by_name.value, 50);
assert_eq!(reopened.get_by_index::<Item>("category", "Consumable").unwrap(), Some(by_name.clone()));
assert_eq!(reopened.get_by_index::<Item>("sku", "P-1").unwrap(), Some(by_name)); assert_eq!(reopened.get_by_index::<Item>("sku", "P-1").unwrap(), Some(by_name));
} }
@ -121,31 +120,32 @@ fn a_missing_name_or_value_is_none_and_an_unregistered_type_is_refused() {
assert_eq!(cache.get_by_name::<Item>("B").unwrap(), None); assert_eq!(cache.get_by_name::<Item>("B").unwrap(), None);
assert_eq!(cache.get_by_name::<Item>("a").unwrap(), None, "names compare ordinally"); assert_eq!(cache.get_by_name::<Item>("a").unwrap(), None, "names compare ordinally");
assert_eq!(cache.get_by_index::<Item>("category", "other").unwrap(), None); assert_eq!(cache.get_by_index::<Item>("sku", "other").unwrap(), None);
assert_eq!(cache.get_by_index::<Item>("nosuchalias", "cat").unwrap(), None); assert_eq!(cache.get_by_index::<Item>("nosuchalias", "a-1").unwrap(), None);
assert_eq!(cache.get_by_index::<Item>("category", "cat").unwrap(), None, "category is a grouping, not an index");
assert_eq!(cache.get_by_index::<Item>("code", "a-1").unwrap(), None, "the alias is sku, not the member name"); assert_eq!(cache.get_by_index::<Item>("code", "a-1").unwrap(), None, "the alias is sku, not the member name");
let bare = CultCache::new(); let bare = CultCache::new();
assert!(bare.get_by_name::<Item>("A").is_err()); assert!(bare.get_by_name::<Item>("A").is_err());
assert!(bare.get_by_index::<Item>("category", "cat").is_err()); assert!(bare.get_by_index::<Item>("sku", "a-1").is_err());
assert!(bare.get_global::<Settings>().is_err()); assert!(bare.get_global::<Settings>().is_err());
} }
#[test] #[test]
fn an_index_without_an_alias_is_named_by_its_member() { fn an_index_without_an_alias_is_named_by_its_member() {
let mut cache = cache(); let mut cache = cache();
cache.put("item:a", &item("A", "cat", None)).unwrap(); cache.put("numbered:1", &Numbered { id: 1, badge: 7 }).unwrap();
assert!(cache.get_by_index::<Item>("category", "cat").unwrap().is_some()); assert!(cache.get_by_index::<Numbered>("badge", "7").unwrap().is_some());
} }
// A blank name is no name; a blank index value is not indexed; a nil member holds no value. // A blank name is no name; a blank index value is not indexed; a nil member holds no value.
#[test] #[test]
fn blank_and_nil_values_are_not_found() { fn blank_and_nil_values_are_not_found() {
let mut cache = cache(); let mut cache = cache();
cache.put("item:blank", &item("", " ", None)).unwrap(); cache.put("item:blank", &item("", " ", Some(" "))).unwrap();
assert_eq!(cache.get_by_name::<Item>("").unwrap(), None); assert_eq!(cache.get_by_name::<Item>("").unwrap(), None);
assert_eq!(cache.get_by_index::<Item>("category", " ").unwrap(), None); assert_eq!(cache.get_by_index::<Item>("sku", " ").unwrap(), None);
assert_eq!(cache.get_by_index::<Item>("category", "").unwrap(), None); assert_eq!(cache.get_by_index::<Item>("sku", "").unwrap(), None);
assert_eq!(cache.get_by_index::<Item>("sku", "").unwrap(), None); assert_eq!(cache.get_by_index::<Item>("sku", "").unwrap(), None);
// Only emptiness voids a name: whitespace is a name, as in C# (Length > 0). // Only emptiness voids a name: whitespace is a name, as in C# (Length > 0).
@ -156,68 +156,41 @@ fn blank_and_nil_values_are_not_found() {
#[test] #[test]
fn lookups_never_cross_types() { fn lookups_never_cross_types() {
let mut cache = cache(); let mut cache = cache();
cache.put("item:a", &item("same", "same", None)).unwrap(); cache.put("item:a", &item("same", "same", Some("same"))).unwrap();
cache.put("gadget:a", &Gadget { name: "same".into(), category: "same".into() }).unwrap(); cache.put("gadget:a", &Gadget { name: "same".into(), serial: "same".into() }).unwrap();
cache.put("zed:a", &Zed { name: "same".into(), category: "same".into(), code: None, value: 0 }).unwrap(); cache.put("zed:a", &Zed { name: "same".into(), category: "same".into(), code: None, value: 0 }).unwrap();
assert_eq!(cache.get_by_name::<Gadget>("same").unwrap().unwrap().name, "same"); assert_eq!(cache.get_by_name::<Gadget>("same").unwrap().unwrap().name, "same");
cache.delete::<Item>("item:a").unwrap(); cache.delete::<Item>("item:a").unwrap();
assert_eq!(cache.get_by_name::<Item>("same").unwrap(), None); assert_eq!(cache.get_by_name::<Item>("same").unwrap(), None);
assert_eq!(cache.get_by_index::<Item>("category", "same").unwrap(), None); assert_eq!(cache.get_by_index::<Item>("sku", "same").unwrap(), None);
assert!(cache.get_by_name::<Gadget>("same").unwrap().is_some()); assert!(cache.get_by_name::<Gadget>("same").unwrap().is_some());
assert!(cache.get_by_index::<Gadget>("category", "same").unwrap().is_some()); assert!(cache.get_by_index::<Gadget>("serial", "same").unwrap().is_some());
} }
#[test] #[test]
fn a_name_and_index_follow_a_rewritten_record_and_leave_a_removed_one() { fn a_name_and_index_follow_a_rewritten_record_and_leave_a_removed_one() {
let mut cache = cache(); let mut cache = cache();
cache.put("item:a", &item("old", "c1", None)).unwrap(); cache.put("item:a", &item("old", "cat", Some("c1"))).unwrap();
cache.put("item:a", &item("new", "c2", None)).unwrap(); cache.put("item:a", &item("new", "cat", Some("c2"))).unwrap();
assert_eq!(cache.get_by_name::<Item>("old").unwrap(), None, "a stale name is not found"); assert_eq!(cache.get_by_name::<Item>("old").unwrap(), None, "a stale name is not found");
assert_eq!(cache.get_by_index::<Item>("category", "c1").unwrap(), None, "a stale value is not found"); assert_eq!(cache.get_by_index::<Item>("sku", "c1").unwrap(), None, "a stale value is not found");
assert!(cache.get_by_name::<Item>("new").unwrap().is_some()); assert!(cache.get_by_name::<Item>("new").unwrap().is_some());
assert!(cache.get_by_index::<Item>("category", "c2").unwrap().is_some()); assert!(cache.get_by_index::<Item>("sku", "c2").unwrap().is_some());
cache.delete::<Item>("item:a").unwrap(); cache.delete::<Item>("item:a").unwrap();
assert_eq!(cache.get_by_name::<Item>("new").unwrap(), None); assert_eq!(cache.get_by_name::<Item>("new").unwrap(), None);
assert_eq!(cache.get_by_index::<Item>("category", "c2").unwrap(), None); assert_eq!(cache.get_by_index::<Item>("sku", "c2").unwrap(), None);
} }
#[test] #[test]
fn non_string_members_are_found_by_their_text() { fn non_string_members_are_found_by_their_text() {
let mut cache = cache(); let mut cache = cache();
let owner = 161_u64; let badge = 161_u64;
cache.put("numbered:1", &Numbered { id: 42, owner }).unwrap(); cache.put("numbered:1", &Numbered { id: 42, badge }).unwrap();
assert!(cache.get_by_name::<Numbered>("42").unwrap().is_some()); assert!(cache.get_by_name::<Numbered>("42").unwrap().is_some());
assert!(cache.get_by_index::<Numbered>("owner", "161").unwrap().is_some()); assert!(cache.get_by_index::<Numbered>("badge", "161").unwrap().is_some());
}
// VariantTests.RemovingTheHolderADuplicateValueFindsLeavesTheOtherFindable: records sharing a value find the one with
// the smallest key, whatever the write order, live and after a reopen; removing the holder finds the other.
#[test]
fn records_sharing_a_value_find_the_smallest_key_and_removing_it_finds_the_next() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("shared.cc");
{
let mut cache = stored_cache(&path);
cache.put("item:p2", &item("p2", "x", None)).unwrap();
cache.put("item:p1", &item("p1", "x", None)).unwrap();
let found = |cache: &CultCache| cache.get_by_index::<Item>("category", "x").unwrap().map(|found| found.name);
assert_eq!(found(&cache).as_deref(), Some("p1"), "the smallest key, though written second");
assert_eq!(reopened(&path).get_by_index::<Item>("category", "x").unwrap().unwrap().name, "p1");
cache.delete::<Item>("item:p1").unwrap();
assert_eq!(found(&cache).as_deref(), Some("p2"), "removing the holder finds the other one");
cache.put("item:p1", &item("p1", "x", None)).unwrap();
assert_eq!(found(&cache).as_deref(), Some("p1"));
cache.delete::<Item>("item:p2").unwrap();
assert_eq!(found(&cache).as_deref(), Some("p1"), "removing the other changes nothing");
cache.delete::<Item>("item:p1").unwrap();
assert_eq!(found(&cache), None);
}
assert_eq!(reopened(&path).get_by_index::<Item>("category", "x").unwrap(), None);
} }
#[test] #[test]
@ -244,6 +217,71 @@ fn a_global_is_none_until_written_and_survives_a_reopen() {
assert_eq!(reopened(&path).get_global::<Settings>().unwrap(), Some(settings("durable authority"))); assert_eq!(reopened(&path).get_global::<Settings>().unwrap(), Some(settings("durable authority")));
} }
/// `Settings` as an ordinary type: the same type id and schema, no global rule.
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
#[cultcache(type = "settings", schema = "tests.settings")]
struct LooseSettings {
#[cultcache(key = 0)]
value: String,
}
// Declaring a held type global judges the records it holds: two are already a second, so the registration is
// refused and the type stays ordinary.
#[test]
fn registering_a_type_global_over_two_held_records_is_refused_and_changes_nothing() {
let mut cache = CultCache::new();
cache.register_entry_type::<LooseSettings>().unwrap();
cache.put("a", &LooseSettings { value: "one".into() }).unwrap();
cache.put("b", &LooseSettings { value: "two".into() }).unwrap();
let refusal = cache.register_entry_type::<Settings>().unwrap_err();
assert!(refusal.downcast_ref::<CultCacheSecondGlobal>().is_some(), "{refusal:#}");
cache.put("c", &LooseSettings { value: "three".into() }).unwrap();
assert_eq!(cache.get_all_with_keys::<LooseSettings>().unwrap().len(), 3, "still ordinary");
}
// One held record may become the global; a second is then refused as a second global.
#[test]
fn registering_a_type_global_over_one_held_record_is_admitted() {
let mut cache = CultCache::new();
cache.register_entry_type::<LooseSettings>().unwrap();
cache.put("a", &LooseSettings { value: "one".into() }).unwrap();
cache.register_entry_type::<Settings>().unwrap();
let refusal = cache.put("b", &Settings { value: "two".into() }).unwrap_err();
assert!(refusal.downcast_ref::<CultCacheSecondGlobal>().is_some(), "{refusal:#}");
}
/// A global with an indexed member, and the same type and schema declaring that member as an integer.
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
#[cultcache(type = "indexed_global", schema = "tests.indexed_global", global)]
struct IndexedGlobal {
#[cultcache(key = 0, index = "V")]
v: String,
}
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
#[cultcache(type = "indexed_global", schema = "tests.indexed_global", global)]
struct RetypedGlobal {
#[cultcache(key = 0, index = "V")]
v: i64,
}
// A refused re-registration over an already-global type leaves it global: the rule keeps refusing a second record.
#[test]
fn a_refused_reregistration_leaves_a_global_type_global() {
let mut cache = CultCache::new();
cache.register_entry_type::<IndexedGlobal>().unwrap();
cache.put("a", &IndexedGlobal { v: "x".into() }).unwrap();
assert!(cache.register_entry_type::<RetypedGlobal>().is_err(), "the held text cannot be decoded as an integer");
let refusal = cache.put("b", &IndexedGlobal { v: "y".into() }).unwrap_err();
assert!(refusal.downcast_ref::<CultCacheSecondGlobal>().is_some(), "{refusal:#}");
assert_eq!(cache.get_global::<IndexedGlobal>().unwrap(), Some(IndexedGlobal { v: "x".into() }));
assert_eq!(cache.get_by_index::<IndexedGlobal>("V", "x").unwrap(), cache.get::<IndexedGlobal>("a").unwrap());
}
#[test] #[test]
fn a_type_that_is_not_a_global_has_no_global() { fn a_type_that_is_not_a_global_has_no_global() {
let mut cache = cache(); let mut cache = cache();

View file

@ -0,0 +1,775 @@
//! `#[cultcache(index)]` is unique (`variants:ruling:unique-indexes`; C# `RefuseIndexSharing`): within one type and one
//! alias no two records hold one non-blank value, judged on the result of each admission, on every write path and every
//! load. The refusal text is informational; the contract is the verdict, so most pins assert the verdict and the
//! shared vectors in `tests/vectors/unique-index` pin the text and the canonical index text.
use std::path::{Path, PathBuf};
use cultcache_rs::{
CultCache, CultCacheEnvelope, CultCacheSoaTable, DatabaseEntry, SingleFileMessagePackBackingStore,
};
use serde::Deserialize;
/// The vectors' `tests.unique_item`: Sku is unique, Category a grouping.
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
#[cultcache(type = "unique_item", schema = "tests.unique_item")]
struct Unique {
#[cultcache(key = 0, index = "Sku")]
sku: String,
#[cultcache(key = 1)]
category: String,
}
/// The same type with no index: a role, which writes the duplicates `Unique` refuses.
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
#[cultcache(type = "unique_item", schema = "tests.unique_item")]
struct Twin {
#[cultcache(key = 0)]
sku: String,
#[cultcache(key = 1)]
category: String,
}
/// Two unique indexes declared out of alphabetical order, so no ordering is an accident of declaration.
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
#[cultcache(type = "multi", schema = "Multi")]
struct Multi {
#[cultcache(key = 0, index = "B")]
b: Option<String>,
#[cultcache(key = 1, index = "A")]
a: Option<String>,
}
/// Another type sharing the alias `Sku`: types never conflict.
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
#[cultcache(type = "other", schema = "Other")]
struct Other {
#[cultcache(key = 0, index = "Sku")]
sku: String,
}
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
#[cultcache(type = "plain", schema = "Plain")]
struct Plain {
#[cultcache(key = 0)]
value: String,
}
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
#[cultcache(type = "vs", schema = "VS")]
struct VString {
#[cultcache(key = 0, index = "V")]
v: String,
}
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
#[cultcache(type = "vi", schema = "VI")]
struct VInt {
#[cultcache(key = 0, index = "V")]
v: i64,
}
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
#[cultcache(type = "vu", schema = "VU")]
struct VUint {
#[cultcache(key = 0, index = "V")]
v: u64,
}
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
#[cultcache(type = "vb", schema = "VB")]
struct VBool {
#[cultcache(key = 0, index = "V")]
v: bool,
}
/// The refusals vector's other schemas: `Sku` is unique in each; `Alt` is a second unique index in `uq_pair` (alias
/// `Alt`) and `uq_alias` (alias `alt`, sorting after `Sku` only under an ordinal compare); alpha and beta are plain.
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
#[cultcache(type = "uq_pair", schema = "tests.uq_pair")]
struct UqPair {
#[cultcache(key = 0, index = "Sku")]
sku: String,
#[cultcache(key = 1, index = "Alt")]
alt: String,
}
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
#[cultcache(type = "uq_alias", schema = "tests.uq_alias")]
struct UqAlias {
#[cultcache(key = 0, index = "Sku")]
sku: String,
#[cultcache(key = 1, index = "alt")]
alt: String,
}
/// Aliases U+FF21 and U+10400: UTF-16 puts the astral alias first (D801 DC00 < FF21), code-point order the other way.
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
#[cultcache(type = "uq_wide", schema = "tests.uq_wide")]
struct UqWide {
#[cultcache(key = 0, index = "\u{FF21}")]
sku: String,
#[cultcache(key = 1, index = "\u{10400}")]
alt: String,
}
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
#[cultcache(type = "uq_alpha", schema = "tests.uq_alpha")]
struct UqAlpha {
#[cultcache(key = 0, index = "Sku")]
sku: String,
#[cultcache(key = 1)]
second: String,
}
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
#[cultcache(type = "uq_beta", schema = "tests.uq_beta")]
struct UqBeta {
#[cultcache(key = 0, index = "Sku")]
sku: String,
#[cultcache(key = 1)]
second: String,
}
fn unique(sku: &str) -> Unique {
Unique { sku: sku.into(), category: "c".into() }
}
fn multi(a: Option<&str>, b: Option<&str>) -> Multi {
Multi { a: a.map(Into::into), b: b.map(Into::into) }
}
fn refusal(k1: &str, k2: &str, schema: &str, alias: &str) -> String {
format!("Records {k1} and {k2} ({schema}) both hold unique index '{alias}'; an index value names one record.")
}
fn cache_at(path: &Path) -> CultCache {
let mut cache = CultCache::new();
cache.register_entry_type::<Unique>().unwrap();
cache.register_entry_type::<Multi>().unwrap();
cache.register_entry_type::<Other>().unwrap();
cache.register_entry_type::<Plain>().unwrap();
cache.add_generic_backing_store(SingleFileMessagePackBackingStore::new(path)).unwrap();
cache.pull_all_backing_stores().unwrap();
cache
}
fn memory_cache() -> CultCache {
let mut cache = CultCache::new();
cache.register_entry_type::<Unique>().unwrap();
cache.register_entry_type::<Multi>().unwrap();
cache.register_entry_type::<Other>().unwrap();
cache
}
fn twin_cache_at(path: &Path) -> CultCache {
let mut cache = CultCache::new();
cache.register_entry_type::<Twin>().unwrap();
cache.add_generic_backing_store(SingleFileMessagePackBackingStore::new(path)).unwrap();
cache.pull_all_backing_stores().unwrap();
cache
}
fn error_text(result: anyhow::Result<impl std::fmt::Debug>) -> String {
format!("{:#}", result.unwrap_err())
}
fn skus(cache: &CultCache) -> Vec<(String, String)> {
cache.get_all_with_keys::<Unique>().unwrap().into_iter().map(|(key, value)| (key, value.sku)).collect()
}
#[test]
fn the_derive_emits_the_unique_aliases() {
fn aliases<T: DatabaseEntry>() -> Vec<&'static str> {
T::UNIQUE_INDEXES.iter().map(|(alias, _)| *alias).collect()
}
assert_eq!(aliases::<Unique>(), ["Sku"]);
assert_eq!(aliases::<Multi>(), ["B", "A"]);
assert!(aliases::<Plain>().is_empty());
}
#[test]
fn a_put_of_a_held_value_at_another_key_is_refused_and_changes_nothing() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("u.cc");
let mut cache = cache_at(&path);
cache.put("item:b", &unique("S-1")).unwrap();
let bytes = std::fs::read(&path).unwrap();
let text = error_text(cache.put("item:a", &unique("S-1")));
assert_eq!(text, refusal("item:a", "item:b", "tests.unique_item", "Sku"));
assert!(!text.contains("S-1"), "the refusal never names the value");
assert_eq!(std::fs::read(&path).unwrap(), bytes, "the store is untouched");
assert_eq!(cache.get::<Unique>("item:a").unwrap(), None);
assert_eq!(cache.get_by_index::<Unique>("Sku", "S-1").unwrap().unwrap().sku, "S-1");
assert_eq!(cache.get_by_index::<Unique>("Sku", "S-1").unwrap(), cache.get::<Unique>("item:b").unwrap());
}
// The old bug: write b then a sharing a value used to answer whichever key sorted first.
#[test]
fn the_first_holder_keeps_the_value_whatever_the_key_order() {
let mut cache = memory_cache();
cache.put("item:z", &unique("S")).unwrap();
assert!(cache.put("item:a", &unique("S")).is_err());
assert_eq!(cache.get_by_index::<Unique>("Sku", "S").unwrap().unwrap(), unique("S"));
assert_eq!(skus(&cache), vec![("item:z".to_string(), "S".to_string())]);
}
#[test]
fn a_record_rewriting_its_own_key_keeps_its_value_and_a_change_releases_the_old_one() {
let mut cache = memory_cache();
cache.put("k", &unique("S")).unwrap();
cache.put("k", &Unique { sku: "S".into(), category: "other".into() }).unwrap();
assert!(cache.get_by_index::<Unique>("Sku", "S").unwrap().is_some());
cache.put("k", &unique("T")).unwrap();
assert_eq!(cache.get_by_index::<Unique>("Sku", "S").unwrap(), None, "a stale value is not found");
cache.put("j", &unique("S")).unwrap();
assert_eq!(cache.get_by_index::<Unique>("Sku", "S").unwrap().unwrap().sku, "S");
assert_eq!(cache.get_by_index::<Unique>("Sku", "T").unwrap().unwrap().sku, "T");
// Updating through `update` is the same rule.
assert!(cache.update::<Unique, _>("k", |_| unique("S")).is_err());
assert_eq!(cache.get::<Unique>("k").unwrap().unwrap().sku, "T");
}
#[test]
fn delete_releases_a_value() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("d.cc");
let mut cache = cache_at(&path);
cache.put("item:a", &unique("S")).unwrap();
assert!(cache.put("item:b", &unique("S")).is_err());
assert!(cache.delete::<Unique>("item:a").unwrap());
assert_eq!(cache.get_by_index::<Unique>("Sku", "S").unwrap(), None);
cache.put("item:b", &unique("S")).unwrap();
assert!(cache.get_by_index::<Unique>("Sku", "S").unwrap().is_some());
assert!(!cache.delete::<Unique>("item:a").unwrap(), "deleting an absent key releases nothing");
assert!(cache.get_by_index::<Unique>("Sku", "S").unwrap().is_some());
assert!(cache_at(&path).get_by_index::<Unique>("Sku", "S").unwrap().is_some());
}
#[test]
fn types_sharing_an_alias_and_a_value_do_not_conflict() {
let mut cache = memory_cache();
cache.put("a", &unique("S")).unwrap();
cache.put("oa", &Other { sku: "S".into() }).unwrap();
cache.put("ob", &Other { sku: "T".into() }).unwrap();
assert!(cache.put("ob", &Other { sku: "S".into() }).is_err());
assert_eq!(cache.get_by_index::<Unique>("Sku", "S").unwrap().unwrap(), unique("S"));
assert_eq!(cache.get_by_index::<Other>("Sku", "S").unwrap().unwrap(), Other { sku: "S".into() });
}
#[test]
fn an_alias_the_type_does_not_declare_finds_nothing() {
let mut cache = memory_cache();
cache.put("a", &unique("S")).unwrap();
assert_eq!(cache.get_by_index::<Unique>("sku", "S").unwrap(), None);
assert_eq!(cache.get_by_index::<Unique>("Category", "c").unwrap(), None);
}
#[test]
fn blank_and_none_values_never_conflict() {
let mut cache = memory_cache();
for (key, sku) in [("a", " "), ("b", ""), ("c", "\u{85}"), ("d", "\t\n\u{a0}\u{3000}")] {
cache.put(key, &unique(sku)).unwrap();
}
cache.put("m1", &multi(None, None)).unwrap();
cache.put("m2", &multi(None, Some(" "))).unwrap();
assert_eq!(cache.get_by_index::<Unique>("Sku", " ").unwrap(), None);
// U+FEFF, U+001F, U+200B, U+180E are values, never trimmed.
cache.put("e", &unique("\u{feff}")).unwrap();
assert!(cache.put("f", &unique("\u{feff}")).is_err());
cache.put("g", &unique(" S ")).unwrap();
cache.put("h", &unique("S")).unwrap();
assert_eq!(cache.get_by_index::<Unique>("Sku", " S ").unwrap().unwrap().sku, " S ");
assert_eq!(cache.get_by_index::<Unique>("Sku", "S").unwrap().unwrap().sku, "S");
}
// C# `string.IsNullOrWhiteSpace` over every code unit: exactly these 25 are blank (probed at 7760219b, dotnet 10).
#[test]
fn the_blank_set_is_exactly_the_csharp_white_space() {
let blank: Vec<u32> = (0x9..=0xD)
.chain([0x20, 0x85, 0xA0, 0x1680])
.chain(0x2000..=0x200A)
.chain([0x2028, 0x2029, 0x202F, 0x205F, 0x3000])
.collect();
assert_eq!(blank.len(), 25);
let mut cache = memory_cache();
let mut found = Vec::new();
for unit in (0..=0xFFFF_u32).filter(|unit| char::from_u32(*unit).is_some()) {
let text = char::from_u32(unit).unwrap().to_string();
cache.put("a", &unique(&text)).unwrap();
let second = cache.put("b", &unique(&text));
if second.is_ok() {
found.push(unit);
cache.delete::<Unique>("b").unwrap();
assert_eq!(cache.get_by_index::<Unique>("Sku", &text).unwrap(), None, "U+{unit:04X} blank is never found");
} else {
assert_eq!(cache.get_by_index::<Unique>("Sku", &text).unwrap().unwrap().sku, text);
}
}
assert_eq!(found, blank);
}
fn staged<T: DatabaseEntry>(cache: &CultCache, rows: &[(&str, T)]) -> Vec<CultCacheEnvelope> {
rows.iter().map(|(key, value)| cache.prepare_entry(*key, value).unwrap().0).collect()
}
// Two unique indexes, collision on the second-declared, through every write path.
#[test]
fn every_write_path_refuses_and_changes_nothing() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("paths.cc");
let mut cache = cache_at(&path);
cache.put("m:held", &multi(Some("a1"), Some("b1"))).unwrap();
let bytes = std::fs::read(&path).unwrap();
let before = cache.snapshot();
let clash_b = multi(Some("a2"), Some("b1"));
let clash_a = multi(Some("a1"), Some("b2"));
for (clash, alias) in [(&clash_b, "B"), (&clash_a, "A")] {
let pick = |text: &str| {
let (first, second) = if "m:new" < "m:held" { ("m:new", "m:held") } else { ("m:held", "m:new") };
assert_eq!(text, refusal(first, second, "Multi", alias));
};
pick(&error_text(cache.put("m:new", clash)));
pick(&error_text(cache.update::<Multi, _>("m:new", |_| clash.clone())));
let envelope = staged(&cache, &[("m:new", clash.clone())]).remove(0);
pick(&error_text(cache.put_envelope::<Multi>(envelope.clone())));
pick(&error_text(cache.put_raw_envelope(envelope.clone())));
pick(&error_text(cache.put_prepared_batch(vec![envelope])));
assert_eq!(std::fs::read(&path).unwrap(), bytes);
assert_eq!(cache.snapshot(), before);
assert_eq!(cache.get_by_index::<Multi>("B", "b2").unwrap(), None);
}
// A clean row through every path lands and is found by both aliases.
let fine = multi(Some("a9"), Some("b9"));
cache.put("m:1", &fine).unwrap();
let envelope = staged(&cache, &[("m:2", multi(Some("a8"), Some("b8")))]).remove(0);
cache.put_envelope::<Multi>(envelope).unwrap();
let envelope = staged(&cache, &[("m:3", multi(Some("a7"), Some("b7")))]).remove(0);
cache.put_raw_envelope(envelope).unwrap();
cache.put_prepared_batch(staged(&cache, &[("m:4", multi(Some("a6"), Some("b6")))])).unwrap();
for (alias, value, key) in [("A", "a9", 1), ("B", "b8", 2), ("A", "a7", 3), ("B", "b6", 4)] {
let found = cache.get_by_index::<Multi>(alias, value).unwrap().unwrap();
assert_eq!(Some(found), cache.get::<Multi>(&format!("m:{key}")).unwrap());
}
// Held values survive a reopen of the store: the load installs the same holders.
let mut reopened = cache_at(&path);
assert!(reopened.put("m:x", &multi(None, Some("b9"))).is_err());
assert!(reopened.put("m:x", &multi(Some("a6"), None)).is_err());
assert!(reopened.put("m:x", &multi(Some("zz"), Some("yy"))).is_ok());
}
#[test]
fn a_batch_is_judged_whole() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("batch.cc");
let mut cache = cache_at(&path);
cache.put("a", &unique("X")).unwrap();
cache.put("b", &unique("Y")).unwrap();
// A swap inside one batch lands.
cache.put_prepared_batch(staged(&cache, &[("a", unique("Y")), ("b", unique("X"))])).unwrap();
assert_eq!(cache.get_by_index::<Unique>("Sku", "Y").unwrap(), cache.get::<Unique>("a").unwrap());
assert_eq!(cache.get_by_index::<Unique>("Sku", "X").unwrap(), cache.get::<Unique>("b").unwrap());
assert_eq!(cache_at(&path).get_by_index::<Unique>("Sku", "X").unwrap().unwrap().sku, "X");
// A batch taking a value a record being replaced just released lands; one holding a value twice is refused whole.
cache.put_prepared_batch(staged(&cache, &[("a", unique("Z")), ("c", unique("Y"))])).unwrap();
let bytes = std::fs::read(&path).unwrap();
let before = cache.snapshot();
let text = error_text(cache.put_prepared_batch(staged(&cache, &[("p", unique("N")), ("q", unique("N")), ("r", unique("W"))])));
assert_eq!(text, refusal("p", "q", "tests.unique_item", "Sku"));
assert_eq!(std::fs::read(&path).unwrap(), bytes);
assert_eq!(cache.snapshot(), before);
// A key staged twice lands as its last envelope.
cache.put_prepared_batch(staged(&cache, &[("d", unique("D1")), ("d", unique("D2")), ("e", unique("D1"))])).unwrap();
assert_eq!(cache.get_by_index::<Unique>("Sku", "D2").unwrap(), cache.get::<Unique>("d").unwrap());
assert_eq!(cache.get_by_index::<Unique>("Sku", "D1").unwrap(), cache.get::<Unique>("e").unwrap());
}
#[test]
fn a_load_of_a_duplicate_image_is_refused_and_keeps_the_prior_view() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("load.cc");
let mut cache = cache_at(&path);
cache.put("item:keep", &unique("K")).unwrap();
let mut twin = twin_cache_at(&path);
twin.put("item:b", &Twin { sku: "S".into(), category: "c".into() }).unwrap();
twin.put("item:a", &Twin { sku: "S".into(), category: "c".into() }).unwrap();
let bytes = std::fs::read(&path).unwrap();
let before = cache.snapshot();
assert_eq!(error_text(cache.pull_all_backing_stores()), refusal("item:a", "item:b", "tests.unique_item", "Sku"));
assert_eq!(cache.snapshot(), before, "a refused pull admits nothing");
assert_eq!(cache.get_by_index::<Unique>("Sku", "K").unwrap().unwrap().sku, "K");
assert_eq!(std::fs::read(&path).unwrap(), bytes);
// load_soa of the same image through a cache with no store.
let table: CultCacheSoaTable = twin.snapshot_soa();
let mut soa = memory_cache();
soa.put("item:keep", &unique("K")).unwrap();
let before = soa.snapshot();
assert_eq!(error_text(soa.load_soa(table)), refusal("item:a", "item:b", "tests.unique_item", "Sku"));
assert_eq!(soa.snapshot(), before);
assert!(soa.get_by_index::<Unique>("Sku", "K").unwrap().is_some());
// A clean image loads and replaces the holders with the image's.
let mut clean = twin_cache_at(&dir.path().join("clean.cc"));
clean.put("item:x", &Twin { sku: "X".into(), category: "c".into() }).unwrap();
soa.load_soa(clean.snapshot_soa()).unwrap();
assert_eq!(soa.get_by_index::<Unique>("Sku", "K").unwrap(), None);
assert!(soa.get_by_index::<Unique>("Sku", "X").unwrap().is_some());
soa.put("item:y", &unique("K")).unwrap();
}
#[test]
fn a_multi_index_store_with_a_duplicate_on_the_second_alias_refuses_to_open() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("multi.cc");
{
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
#[cultcache(type = "multi", schema = "Multi")]
struct Role {
#[cultcache(key = 0)]
b: Option<String>,
#[cultcache(key = 1)]
a: Option<String>,
}
let mut cache = CultCache::new();
cache.register_entry_type::<Role>().unwrap();
cache.add_generic_backing_store(SingleFileMessagePackBackingStore::new(&path)).unwrap();
cache.pull_all_backing_stores().unwrap();
cache.put("m:1", &Role { b: Some("b".into()), a: Some("a1".into()) }).unwrap();
cache.put("m:2", &Role { b: Some("b".into()), a: Some("a2".into()) }).unwrap();
}
let mut cache = CultCache::new();
cache.register_entry_type::<Multi>().unwrap();
cache.add_generic_backing_store(SingleFileMessagePackBackingStore::new(&path)).unwrap();
assert_eq!(error_text(cache.pull_all_backing_stores()), refusal("m:1", "m:2", "Multi", "B"));
}
#[test]
fn a_bool_index_is_found_by_the_csharp_text() {
let mut cache = CultCache::new();
cache.register_entry_type::<VBool>().unwrap();
cache.put("t", &VBool { v: true }).unwrap();
assert!(cache.put("u", &VBool { v: true }).is_err());
cache.put("f", &VBool { v: false }).unwrap();
assert_eq!(cache.get_by_index::<VBool>("V", "True").unwrap().unwrap().v, true);
assert_eq!(cache.get_by_index::<VBool>("V", "False").unwrap().unwrap().v, false);
assert_eq!(cache.get_by_index::<VBool>("V", "true").unwrap(), None);
assert_eq!(cache.get_by_index::<VBool>("V", "false").unwrap(), None);
}
// ---- shared vectors, written by the C# reference ----
#[allow(dead_code)]
#[derive(Debug, Deserialize)]
#[serde(untagged)]
enum Cell {
Nil,
Bool(bool),
Int(i64),
Uint(u64),
Double(f64),
Text(String),
}
fn vectors() -> PathBuf {
PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../tests/vectors/unique-index")
}
type ValueRow = (String, String, Cell, Option<String>);
#[test]
fn vectors_values_give_the_csharp_index_text() {
let bytes = std::fs::read(vectors().join("values.msgpack")).unwrap();
let rows: Vec<ValueRow> = rmp_serde::from_slice(&bytes).unwrap();
assert!(!rows.is_empty());
let mut checked = 0;
for (label, kind, value, text) in rows {
let mut cache = CultCache::new();
cache.register_entry_type::<VString>().unwrap();
cache.register_entry_type::<VInt>().unwrap();
cache.register_entry_type::<VUint>().unwrap();
cache.register_entry_type::<VBool>().unwrap();
// `double` rows are skipped: CultLookupValue has no float impl, so Rust cannot index a double member.
match (kind.as_str(), value) {
("string", Cell::Text(held)) => {
cache.put("one", &VString { v: held.clone() }).unwrap();
match text {
Some(text) => {
assert_eq!(cache.get_by_index::<VString>("V", &text).unwrap().unwrap().v, held, "{label}");
assert!(cache.put("two", &VString { v: held }).is_err(), "{label}");
}
None => {
cache.put("two", &VString { v: held.clone() }).unwrap();
assert_eq!(cache.get_by_index::<VString>("V", &held).unwrap(), None, "{label}");
}
}
}
("int64", Cell::Int(held)) => {
cache.put("one", &VInt { v: held }).unwrap();
assert_eq!(cache.get_by_index::<VInt>("V", &text.unwrap()).unwrap().unwrap().v, held, "{label}");
assert!(cache.put("two", &VInt { v: held }).is_err(), "{label}");
}
("uint64", Cell::Uint(held)) => {
cache.put("one", &VUint { v: held }).unwrap();
assert_eq!(cache.get_by_index::<VUint>("V", &text.unwrap()).unwrap().unwrap().v, held, "{label}");
assert!(cache.put("two", &VUint { v: held }).is_err(), "{label}");
}
("uint64", Cell::Int(held)) => {
cache.put("one", &VUint { v: held as u64 }).unwrap();
assert_eq!(cache.get_by_index::<VUint>("V", &text.unwrap()).unwrap().unwrap().v, held as u64, "{label}");
}
("int64", Cell::Uint(held)) => {
cache.put("one", &VInt { v: held as i64 }).unwrap();
assert_eq!(cache.get_by_index::<VInt>("V", &text.unwrap()).unwrap().unwrap().v, held as i64, "{label}");
}
("bool", Cell::Bool(held)) => {
cache.put("one", &VBool { v: held }).unwrap();
assert_eq!(cache.get_by_index::<VBool>("V", &text.unwrap()).unwrap().unwrap().v, held, "{label}");
}
("double", Cell::Double(_) | Cell::Int(_) | Cell::Uint(_) | Cell::Nil) => continue,
(kind, value) => panic!("{label}: unexpected vector row {kind} {value:?}"),
}
checked += 1;
}
assert!(checked > 10, "the vectors were read, not skipped: {checked}");
}
type OutcomeRow = (String, Option<String>, String, Option<String>, Option<String>);
#[test]
fn vectors_stores_hold_their_verdicts() {
let bytes = std::fs::read(vectors().join("outcomes.msgpack")).unwrap();
let rows: Vec<OutcomeRow> = rmp_serde::from_slice(&bytes).unwrap();
assert!(rows.len() >= 3);
let dir = tempfile::tempdir().unwrap();
for (file, alias, verdict, k1, k2) in rows {
let path = dir.path().join(&file);
std::fs::copy(vectors().join(&file), &path).unwrap();
let before = std::fs::read(&path).unwrap();
let attach = |cache: &mut CultCache| {
cache.add_generic_backing_store(SingleFileMessagePackBackingStore::new(&path)).unwrap();
cache.pull_all_backing_stores()
};
let mut cache = CultCache::new();
let opened = match alias.as_deref() {
Some("Sku") => {
cache.register_entry_type::<Unique>().unwrap();
attach(&mut cache)
}
None => {
cache.register_entry_type::<Twin>().unwrap();
attach(&mut cache)
}
Some(other) => panic!("unexpected vector alias {other}"),
};
match verdict.as_str() {
"open" => opened.unwrap_or_else(|error| panic!("{file} / {alias:?} should open: {error:#}")),
"refused" => assert_eq!(
error_text(opened),
refusal(&k1.unwrap(), &k2.unwrap(), "tests.unique_item", "Sku"),
"{file}"
),
other => panic!("unexpected verdict {other}"),
}
assert_eq!(std::fs::read(&path).unwrap(), before, "{file} is byte-identical");
}
}
/// One op of a refusals row: `["open", file-or-nil]`, `["pull", file]` or `["commit", [[key, schema, sku, second], ...]]`.
#[derive(Deserialize)]
#[serde(untagged)]
enum Arg {
Records(Vec<(String, String, String, String)>),
File(Option<String>),
}
type RefusalRow = (String, Vec<(String, Arg)>, Option<(String, String, String, String)>);
/// Rows `[label, ops, refusal]`: the pair a refusal names is a function of the state the admission would leave
/// (`tests/vectors/unique-index/README.md`, RULE), whatever the staging, file or pull order, key case or alias case.
#[test]
fn vectors_refusals_hold_the_rule() {
let bytes = std::fs::read(vectors().join("refusals.msgpack")).unwrap();
let rows: Vec<RefusalRow> = rmp_serde::from_slice(&bytes).unwrap();
let dir = tempfile::tempdir().unwrap();
let mut walked = 0;
for (index, (label, ops, refused)) in rows.into_iter().enumerate() {
let path = dir.path().join(format!("row-{index}.cc"));
let mut cache: Option<CultCache> = None;
let mut failure = None;
for (at, (kind, arg)) in ops.iter().enumerate() {
let last = at + 1 == ops.len();
let mut before = path.exists().then(|| std::fs::read(&path).unwrap());
let result = match (kind.as_str(), arg) {
("open", Arg::File(file)) => {
if let Some(file) = file {
std::fs::copy(vectors().join(file), &path).unwrap();
}
let mut opened = CultCache::new();
opened.register_entry_type::<Unique>().unwrap();
opened.register_entry_type::<UqPair>().unwrap();
opened.register_entry_type::<UqAlias>().unwrap();
opened.register_entry_type::<UqWide>().unwrap();
opened.register_entry_type::<UqAlpha>().unwrap();
opened.register_entry_type::<UqBeta>().unwrap();
opened.add_generic_backing_store(SingleFileMessagePackBackingStore::new(&path)).unwrap();
let pulled = opened.pull_all_backing_stores();
cache = Some(opened);
pulled
}
("pull", Arg::File(Some(file))) => {
std::fs::copy(vectors().join(file), &path).unwrap();
before = Some(std::fs::read(&path).unwrap());
cache.as_mut().unwrap().pull_all_backing_stores()
}
("commit", Arg::Records(records)) => {
let cache = cache.as_mut().unwrap();
let staged: Vec<_> = records
.iter()
.map(|(key, schema, sku, second)| {
let (sku, second) = (sku.clone(), second.clone());
match schema.as_str() {
"tests.unique_item" => cache.prepare_entry(key, &Unique { sku, category: second }).map(|(envelope, _)| envelope),
"tests.uq_pair" => cache.prepare_entry(key, &UqPair { sku, alt: second }).map(|(envelope, _)| envelope),
"tests.uq_alias" => cache.prepare_entry(key, &UqAlias { sku, alt: second }).map(|(envelope, _)| envelope),
"tests.uq_wide" => cache.prepare_entry(key, &UqWide { sku, alt: second }).map(|(envelope, _)| envelope),
"tests.uq_alpha" => cache.prepare_entry(key, &UqAlpha { sku, second }).map(|(envelope, _)| envelope),
"tests.uq_beta" => cache.prepare_entry(key, &UqBeta { sku, second }).map(|(envelope, _)| envelope),
other => panic!("{label}: unexpected schema {other}"),
}
.unwrap()
})
.collect();
cache.put_prepared_batch(staged)
}
other => panic!("{label}: unexpected op {:?}", other.0),
};
if last {
failure = Some((result, before));
} else {
result.unwrap_or_else(|error| panic!("{label}: op {at} should succeed: {error:#}"));
}
}
let (result, before) = failure.unwrap();
match refused {
None => result.unwrap_or_else(|error| panic!("{label}: should succeed: {error:#}")),
Some((k1, k2, schema, alias)) => {
let error = result.err().unwrap_or_else(|| panic!("{label}: should refuse"));
assert_eq!(format!("{error:#}"), refusal(&k1, &k2, &schema, &alias), "{label}");
if let Some(before) = before {
assert_eq!(std::fs::read(&path).unwrap(), before, "{label}: the file is byte-identical");
}
}
}
walked += 1;
}
assert_eq!(walked, 37);
}
// A pull replaces the holder table with the stored image's: a value another writer changed is released, not kept.
#[test]
fn a_second_pull_replaces_the_holders_with_the_stored_image() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("pull.cc");
let mut cache = cache_at(&path);
cache.put("a", &unique("S")).unwrap();
cache_at(&path).put("a", &unique("T")).unwrap();
cache.pull_all_backing_stores().unwrap();
assert_eq!(cache.get_by_index::<Unique>("Sku", "S").unwrap(), None, "the replaced value is no longer held");
assert_eq!(cache.get_by_index::<Unique>("Sku", "T").unwrap(), cache.get::<Unique>("a").unwrap());
cache.put("b", &unique("S")).unwrap();
assert!(cache.put("c", &unique("T")).is_err());
}
// The projector decides what held records hold: declaring an index for a type that already has records derives
// their holders, or refuses the registration when they collide.
#[test]
fn registering_an_index_over_held_records_derives_their_holders() {
let mut cache = CultCache::new();
cache.register_entry_type::<Twin>().unwrap();
cache.put("a", &Twin { sku: "S".into(), category: "x".into() }).unwrap();
cache.put("b", &Twin { sku: "T".into(), category: "x".into() }).unwrap();
cache.register_entry_type::<Unique>().unwrap();
assert_eq!(cache.get_by_index::<Unique>("Sku", "S").unwrap(), cache.get::<Unique>("a").unwrap());
assert!(cache.get_by_index::<Unique>("Sku", "S").unwrap().is_some());
assert!(cache.put("c", &unique("S")).is_err(), "a held value is refused");
cache.put("c", &unique("U")).unwrap();
}
#[test]
fn registering_an_index_over_colliding_held_records_is_refused_and_changes_nothing() {
let mut cache = CultCache::new();
cache.register_entry_type::<Twin>().unwrap();
cache.put("a", &Twin { sku: "S".into(), category: "x".into() }).unwrap();
cache.put("b", &Twin { sku: "S".into(), category: "y".into() }).unwrap();
let text = error_text(cache.register_entry_type::<Unique>());
assert_eq!(text, refusal("a", "b", "tests.unique_item", "Sku"));
// Still the unindexed declaration: duplicates keep landing, nothing is looked up.
cache.put("c", &Twin { sku: "S".into(), category: "z".into() }).unwrap();
assert_eq!(cache.get_all_with_keys::<Twin>().unwrap().len(), 3);
assert_eq!(cache.get_by_index::<Unique>("Sku", "S").unwrap(), None);
}
/// The same type and schema declaring Sku as an integer: the held text cannot be decoded as one.
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
#[cultcache(type = "unique_item", schema = "tests.unique_item")]
struct Retyped {
#[cultcache(key = 0, index = "Sku")]
sku: i64,
}
/// A refused re-registration leaves the previous declaration in force: it keeps judging, so a later duplicate is refused.
#[test]
fn a_refused_reregistration_keeps_the_previous_index_judging() {
let mut cache = CultCache::new();
cache.register_entry_type::<Unique>().unwrap();
cache.put("a", &unique("S")).unwrap();
cache.put("b", &unique("T")).unwrap();
// A different declaration over the same type (Category is the unique alias) collides on held records.
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
#[cultcache(type = "unique_item", schema = "tests.unique_item")]
struct ByCat {
#[cultcache(key = 0)]
sku: String,
#[cultcache(key = 1, index = "Cat")]
category: String,
}
assert!(cache.register_entry_type::<ByCat>().is_err(), "a and b both hold category c");
assert!(cache.put("z", &unique("S")).is_err(), "the previous index still judges");
assert_eq!(cache.get_by_index::<Unique>("Sku", "S").unwrap(), cache.get::<Unique>("a").unwrap());
cache.put("z", &unique("U")).unwrap();
}
// A held record the new declaration cannot decode refuses the registration; the unindexed declaration stays.
#[test]
fn registering_an_index_over_an_undecodable_held_record_is_refused_and_changes_nothing() {
let mut cache = CultCache::new();
cache.register_entry_type::<Twin>().unwrap();
cache.put("a", &Twin { sku: "S".into(), category: "x".into() }).unwrap();
assert!(cache.register_entry_type::<Retyped>().is_err());
cache.put("b", &Twin { sku: "S".into(), category: "y".into() }).unwrap();
assert_eq!(cache.get_all_with_keys::<Twin>().unwrap().len(), 2, "no index judges the duplicate");
}
// Dropping the declaration releases the holders.
#[test]
fn registering_without_the_index_releases_the_holders() {
let mut cache = CultCache::new();
cache.register_entry_type::<Unique>().unwrap();
cache.put("a", &unique("S")).unwrap();
cache.register_entry_type::<Twin>().unwrap();
assert_eq!(cache.get_by_index::<Unique>("Sku", "S").unwrap(), None, "no declaration, no holders");
cache.put("b", &Twin { sku: "S".into(), category: "x".into() }).unwrap();
assert!(cache.register_entry_type::<Unique>().is_err(), "the duplicate now written is a held violation");
}

View file

@ -3,13 +3,42 @@
All notable changes to `@gamecult/cultcache-ts` are recorded here, following All notable changes to `@gamecult/cultcache-ts` are recorded here, following
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/). [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
## [Unreleased]
### Breaking
- A single-file store marked `cultcache.store.v3` (it holds an element id, written by the C# runtime) is read, and a rewrite keeps the mark and the ids; this runtime does not decide when a document carries ids. A runtime from before this change refuses a v3 store by name, so rolling back to an older release after a store has been written as v3 is one-way: there is no carry-back, so restore from a copy taken before upgrading.
- `indexes` are unique, as C# `[CultIndex]`: a write or a load that leaves two records of a type holding one non-blank
value under one alias is refused naming both keys and the alias, and writes nothing. Index text is the C# canonical
text (booleans `True`/`False`, integers as digits, doubles as the exact expansion, blank values never indexed), so
an empty string is no longer a value. `registerIndex` over held duplicates installs nothing. A new `roles` map names
the many-row selection aliases (never indexed for lookup, same catalog `indexAlias`). The per-key migration rule is
in the README; classify every key before the CultLib your tools load is updated. The refusal verdict is the contract, its text is informational.
- The single-file reader (v1 and the legacy envelope array) refuses a store that holds two records under one key as `StoreUnreadableError`; a store that opened before may now be refused. Repair it by removing the extra record with the runtime that wrote it.
- A `put`, `push` or `pushAll` that would land a second type under a held key (against the file, or within one batch) is refused with `SchemaConflictError`, and nothing is written. The written type may replace the record when its catalog entry carries the stored schema id as its own or lists it compatible (a type renamed under a stable id restamps the record, as C# does); otherwise retyping a key is a delete, then a put. The C# allowance for an undeclared older id of the same schema name that reads whole is not ported: a same-named type keeps passing, including one that drops persisted slots, which C# refuses.
- Same-type writes replace by key; in one `pushAll` batch the last entry for a key wins.
- A store whose canonical-schema catalog slot is not a string is refused as `StoreUnreadableError`.
- The cache holds one record per key (variants:ruling:cache-refuses-retype). A `put` or `putEnvelope` that lands a second type under a key the cache holds, or under a key a store carries foreign, is refused with `SchemaConflictError` naming the key, even with no store (a memory-only cache used to hold both types at the key). A type that owns the held record's schema id (as for the store) replaces it, unless the held record is in a store other than the written type's home: that rename is refused with `SchemaConflictError` (remove the record first). Two stores holding one key, registered or foreign, refuse the load and admit nothing. A store's `delete` removes the record whose stored schema id the removal's type owns, whatever the arrived record's `type` string.
- A store holding a record no registered type claims now loads (it used to throw "No schema is registered..."): the cache does not serve the record, lists it in `foreignRecords()`, keeps its bytes through every write, and refuses a `put` or a `delete` at its key with `SchemaConflictError` (variants:ruling:unheld-removal-refused-at-commit). Registering a type that claims its schema id serves it at the next `pullAllBackingStores`.
### Added
- `CultCache.foreignRecords()`: the `{key, schemaId, schemaName, storedAt}` of every record the last pull carried and no registered type claims, sorted by key ordinal.
### Changed
- Opening a legacy (pre-v1) store is a pure read: payloads that were not bytes are normalized in memory and the
file is left as it was, so a codec that refuses the content cannot lose the old bytes. The next push, delete or
flush rewrites the file as v1, merging what the open read.
- `SingleFileStoreRead` no longer carries `repairedLegacyPayload`; no consumer used it.
## [0.15.0] ## [0.15.0]
0.14.1 and 0.14.2 were never tagged or published; this entry carries them with the Unreleased work, and `cultcache-ts-v0.14.0` is the previous release. 0.14.1 and 0.14.2 were never tagged or published; this entry carries them with the Unreleased work, and `cultcache-ts-v0.14.0` is the previous release.
### Breaking ### Breaking
- A single-file store written as `cultcache.store.v3` is read and written: a record that holds an element id (an owned list element with an `[CultElementId]` member, carried through every write) marks the store v3, and a v3 store is carried through a rewrite with its ids. A runtime from before this change refuses a v3 store by name. - A single-file store written as `cultcache.store.v3` is read and written: a store the C# runtime marked v3 (it holds a record with an element id: an owned list element with a `[CultElementId]` member) keeps its header here, and is carried through every rewrite with its ids; this runtime never drops the mark. A runtime from before this change refuses a v3 store by name.
- A single-file store refuses a zero-byte file as `StoreUnreadableError`; only a path with nothing at it is an empty store. A header this runtime does not read is refused, and the refusal echoes the header only as `cultcache.store.v<digits>`. - A single-file store refuses a zero-byte file as `StoreUnreadableError`; only a path with nothing at it is an empty store. A header this runtime does not read is refused, and the refusal echoes the header only as `cultcache.store.v<digits>`.
- A write over a file the reader refuses (`StoreUnreadableError`, from `push`, `pushAll` and `delete`) refuses and leaves the file as it is. A flush used to replace such a file. A write set cannot merge with a disk it cannot read, so the file is repaired or removed by its owner before the store writes again. A full `pushAll` over a file with no readable stamp does not replace it: the reader refuses it. The stamp rule has one owner, `refuseUnreadFormat`, asked only by the one reader that every write consults under the lock. The reader judges the stamp from the header before it decodes the body, so a newer store is refused by name whether or not its body is whole. - A write over a file the reader refuses (`StoreUnreadableError`, from `push`, `pushAll` and `delete`) refuses and leaves the file as it is. A flush used to replace such a file. A write set cannot merge with a disk it cannot read, so the file is repaired or removed by its owner before the store writes again. A full `pushAll` over a file with no readable stamp does not replace it: the reader refuses it. The stamp rule has one owner, `refuseUnreadFormat`, asked only by the one reader that every write consults under the lock. The reader judges the stamp from the header before it decodes the body, so a newer store is refused by name whether or not its body is whole.
- A dangling symbolic link at the store path is an I/O error with code `ENOENT`, not an empty store. - A dangling symbolic link at the store path is an I/O error with code `ENOENT`, not an empty store.
@ -41,8 +70,6 @@ All notable changes to `@gamecult/cultcache-ts` are recorded here, following
- The Windows replace is a POSIX-semantics rename first (it lands while C# and Rust readers - The Windows replace is a POSIX-semantics rename first (it lands while C# and Rust readers
hold the store open), falling back to a write-through `MoveFileExW`, with 25 attempts 20 ms hold the store open), falling back to a write-through `MoveFileExW`, with 25 attempts 20 ms
apart on errors 5, 32 and 33. Writers wait up to 30 s for the lock. apart on errors 5, 32 and 33. Writers wait up to 30 s for the lock.
- Opening a legacy store that needs healing rewrites it under the lock, from a read taken
under the lock.
### Dependencies ### Dependencies

View file

@ -63,6 +63,10 @@ const payload = encodeDocumentPayload(noteDocument, note); // DocumentPayload
Browsers import the same names from `@gamecult/cultcache-ts/document`, which holds no Node built-ins. The whole cache without the single-file store is `@gamecult/cultcache-ts/browser`, which is also what bundlers resolve for the package root (`browser` condition); bring your own `CacheBackingStore`. Browsers import the same names from `@gamecult/cultcache-ts/document`, which holds no Node built-ins. The whole cache without the single-file store is `@gamecult/cultcache-ts/browser`, which is also what bundlers resolve for the package root (`browser` condition); bring your own `CacheBackingStore`.
## Store format and rollback
A store whose elements hold an id is marked `cultcache.store.v3` by the C# runtime. This runtime reads a v3 store, keeps the header it read and carries the ids through a rewrite; it never drops the mark, so it has no carry-back. A cultcache-ts from before element ids refuses a v3 store by name, so rolling back to it after a store has been written as v3 is one-way: restore from a copy taken before upgrading, and upgrade every reader before any writer. (The C# line is different: CultLib 1.x does not refuse; see the GameCult.Caching README.)
## `.cc` Inspection ## `.cc` Inspection
CultCache files may use the `.cc` extension. The bytes are still the canonical CultCache files may use the `.cc` extension. The bytes are still the canonical
@ -93,12 +97,16 @@ const itemDocument = defineDocumentType({
schemaVersion: "item.v1", schemaVersion: "item.v1",
schema: z.object({ schema: z.object({
name: z.string(), name: z.string(),
sku: z.string(),
category: z.string(), category: z.string(),
value: z.number().int(), value: z.number().int(),
}), }),
name: "name", name: "name",
indexes: { indexes: {
category: "category", sku: "sku", // unique: a sku names one item
},
roles: {
category: "category", // a selection alias: many items share a category
}, },
}); });
@ -122,6 +130,7 @@ await cache.pullAllBackingStores();
await cache.put(itemDocument, "item:potion", { await cache.put(itemDocument, "item:potion", {
name: "Potion", name: "Potion",
sku: "POT-001",
category: "Consumable", category: "Consumable",
value: 50, value: 50,
}); });
@ -132,7 +141,7 @@ await cache.putGlobal(settingsDocument, {
}); });
const potion = cache.getByName(itemDocument, "Potion"); const potion = cache.getByName(itemDocument, "Potion");
const consumable = cache.getByIndex(itemDocument, "category", "Consumable"); const bySku = cache.getByIndex(itemDocument, "sku", "POT-001");
const settings = cache.getRequiredGlobal(settingsDocument); const settings = cache.getRequiredGlobal(settingsDocument);
``` ```
@ -194,7 +203,7 @@ A document type has exactly one home store
## Name, Index, and Global Semantics ## Name, Index, and Global Semantics
`name` and `indexes` can be declared on the document definition: `name`, `indexes` and `roles` can be declared on the document definition:
```ts ```ts
const playerDocument = defineDocumentType({ const playerDocument = defineDocumentType({
@ -202,17 +211,80 @@ const playerDocument = defineDocumentType({
schema: playerSchema, schema: playerSchema,
name: "displayName", name: "displayName",
indexes: { indexes: {
faction: "faction", email: "email", // unique lookup
email: "email", },
roles: {
faction: "faction", // selection alias, many players share one
}, },
}); });
``` ```
`indexes` are unique (variants:ruling:unique-indexes; C# `[CultIndex]`, Rust `#[cultcache(index)]`). Within one type
and one alias no two records hold one non-blank value, judged on the result of every write and of every load; a
refusal reads `Records <k1> and <k2> (<schema name>) both hold unique index '<alias>'; an index value names one record.`
(keys in ordinal order, never the value), and writes nothing. The contract is the verdict, which stores and writes are
refused; the text is informational. On a load TS judges record by record in file order and names the first collision
it meets, where C# judges the whole store, so the two runtimes can name different pairs of one store. A blank value (absent, empty, or only whitespace) is
never indexed and never conflicts. Two types may share an alias and a value. Index values are compared as the C#
reference renders them: strings as is (never trimmed), `True`/`False`, integers as digits, a non-integral number as
its exact decimal expansion; NaN and infinities have no value. `roles` name selection and reference aliases (the
member's catalog `indexAlias`, so the catalog bytes are the same either way); they imply nothing about uniqueness and
are never indexed for lookup, so `getByIndex` on a role answers `undefined`. An alias is declared in one of the two.
### One record per key, and foreign records
A cache holds one record per key whatever its type (variants:ruling:cache-refuses-retype). A write of a type that does not
own the held record's schema id (it carries that id as its own or lists it compatible) is refused with
`SchemaConflictError` naming the key, with or without a store; retyping a key is `delete` then `put`. A store that holds a
record no registered type claims still loads: the cache does not serve it, `foreignRecords()` lists it (`key`,
`schemaId`, `schemaName`, `storedAt`, by key ordinal), every write keeps its bytes, and a `put` or `delete` at its key is
refused. Registering a type that claims its schema id serves it at the next `pullAllBackingStores`. Two stores that hold
one key refuse the load and admit nothing.
### Migrating an existing `indexes` key
Apply per key, and do it BEFORE the CultLib checkout your tools load is rebuilt or updated. A consumer that loads
CultLib at run time (Bifrost reads `$VOIDBOT_CULTLIB_ROOT`, else the sibling `../CultLib` dist) has no pin to hold: its
next start after the checkout moves is the bump. A consumer that pins a version applies this before it bumps the pin.
Moving an `indexes` key to `roles` is safe on the old CultLib too when the definition lists `members` explicitly (an old
codec definition without `members` derives `indexAlias` from `indexes` only, so a moved key would lose its catalog alias),
so the consumer edit lands first:
1. Nothing looks records up by the key (`getByIndex`, `getKeyByIndex`, `getIdByIndex`; Python `get_by_index`,
`get_key_by_index`, `node.get_by_index`, `watch_by_index`; Rust `get_by_index`): it is a role. Move it to `roles`;
where a definition lists `members` explicitly, delete it from `indexes` and keep the member's `indexAlias` (the
catalog is the same).
2. Something looks it up: it stays unique only when two live records can never hold one non-blank value, because the
record key is derived from that value alone, or the value is an outside identity (email, sku, external id).
Enums, statuses, kinds, categories, parent or foreign ids, shared hashes and timestamps are roles; a lookup on
one already answered an arbitrary record, so move it to a selection (`any_of`) or `getAll` plus a filter.
3. Before the checkout moves, open each real `.cc` once with the new definitions. A refusal names both keys and the alias and
writes nothing.
Misclassifying costs differently each way. A role declared unique: the second record holding a shared value is
refused at commit, and a store already holding one refuses to open, so that store is down until the key is
reclassified (loud, names both keys and the alias). A unique key declared a role: lookups by it answer nothing and
duplicates stop being refused.
The Bifrost TS stores (at 781e6ea) call no lookup anywhere, so by rule 1 every key is a role and no key becomes unique.
All six definitions list `members` explicitly, each with its `indexAlias`, so the edit is deleting each `indexes` block:
the catalog is unchanged and the edit is safe on the CultLib it loads today. It must land before that checkout is
updated:
| Definition | Keys | Verdict |
|---|---|---|
| `update-request` (`tools/agent-transport.mjs`) | `targetRepoName`, `targetAgentIdentity`, `status` | role: many requests per repo and agent; status is an enum |
| `crossing_receipt` (`tools/bifrost-crossing-documents.mjs`) | `commandId`, `crossingKind`, `status` | role: receipts are `crossing_<commandId>_<activityKind>`, several per command; the rest are enums |
| release authority (`tools/bifrost-repository-release-authority.mjs`) | `repositoryFullName`, `upstreamRef`, `commitSha`, `status` | role: the authority id is derived from repo, ref and sha, so one sha under two refs is two records |
| `topic` (`tools/governance-threads.mjs`) | `jurisdictionRepoName`, `jurisdictionAgentIdentity`, `status` | role: many topics per jurisdiction |
| `comment` (`tools/governance-threads.mjs`) | `topicId`, `authorId`, `stance` | role: many comments per topic; stance is an enum |
| `advertisement` (`tools/provider-advertisement.mjs`) | `serviceName` | role: one record holds it today, but nothing looks it up |
You can also register them later if you feel like changing the machine after boot: You can also register them later if you feel like changing the machine after boot:
```ts ```ts
await cache.registerNameLookup(playerDocument, "displayName"); await cache.registerNameLookup(playerDocument, "displayName");
await cache.registerIndex(playerDocument, "faction", "faction"); await cache.registerIndex(playerDocument, "email", "email");
``` ```
Registration, `addBackingStore`, `pullAllBackingStores` and every write and Registration, `addBackingStore`, `pullAllBackingStores` and every write and
@ -357,7 +429,7 @@ That is the TypeScript version of the original CultCache trick:
- the top-level MessagePack value is `[formatVersion, schemaCatalog, records]` - the top-level MessagePack value is `[formatVersion, schemaCatalog, records]`
- each record is `[key, schemaId, storedAt, payload]` - each record is `[key, schemaId, storedAt, payload]`
- each catalog entry is `[schemaId, schemaName, schemaVersion, contentHash, canonicalSchemaJson, compatibleSchemaIds, members]` - each catalog entry is `[schemaId, schemaName, schemaVersion, contentHash, canonicalSchemaJson, compatibleSchemaIds, members]`
- it still reads the older TS/Rust-style envelope array and rewrites repaired legacy payloads as v1 snapshots - it still reads the older TS/Rust-style envelope array (object payloads are normalized in memory; opening never rewrites the file) and replaces it with a v1 snapshot on the next push, delete or flush
- it queues local writes in order, then takes the shared `<file>.lock` (`flock` on Unix, `LockFileEx` on Windows, via koffi, loaded on the first write) for each one, so C#, Rust and TypeScript writers exclude each other; the read a write merges from is taken under that lock - it queues local writes in order, then takes the shared `<file>.lock` (`flock` on Unix, `LockFileEx` on Windows, via koffi, loaded on the first write) for each one, so C#, Rust and TypeScript writers exclude each other; the read a write merges from is taken under that lock
- it fsyncs the staged file and, on Unix, the directory after the replace; on Windows it replaces with a POSIX-semantics rename and falls back to a write-through `MoveFileExW`, retrying briefly on sharing violations - it fsyncs the staged file and, on Unix, the directory after the replace; on Windows it replaces with a POSIX-semantics rename and falls back to a write-through `MoveFileExW`, retrying briefly on sharing violations
- it persists raw MessagePack payload bytes inside schema-addressed records - it persists raw MessagePack payload bytes inside schema-addressed records

View file

@ -2,7 +2,7 @@
// (node:fs, koffi). Bring your own CacheBackingStore, or none. // (node:fs, koffi). Bring your own CacheBackingStore, or none.
export * as codecs from "./codec"; export * as codecs from "./codec";
export type { Codec, SlotsCodec } from "./codec"; export type { Codec, SlotsCodec } from "./codec";
export { CultCache, CultCacheBuilder } from "./cult-cache"; export { CultCache, CultCacheBuilder, type CultCacheForeignRecord } from "./cult-cache";
export { inspectCultCacheBytes } from "./cult-cache-inspector"; export { inspectCultCacheBytes } from "./cult-cache-inspector";
export { export {
decodeDocumentPayload, decodeDocumentPayload,

View file

@ -1,4 +1,5 @@
import { import {
declaresRole,
decodeDocumentPayload, decodeDocumentPayload,
documentPayloadBytes, documentPayloadBytes,
documentPayloadFromBytes, documentPayloadFromBytes,
@ -8,7 +9,7 @@ import {
} from "./document"; } from "./document";
import { resolveDocumentFormatter } from "./document-formatter"; import { resolveDocumentFormatter } from "./document-formatter";
import type { DocumentPayload } from "./document"; import type { DocumentPayload } from "./document";
import { SchemaConflictError } from "./store-format"; import { SchemaConflictError, schemaIdForEnvelope, writtenOwns } from "./store-format";
import type { import type {
AnyCultCacheDocumentDefinition, AnyCultCacheDocumentDefinition,
CacheBackingStore, CacheBackingStore,
@ -46,6 +47,7 @@ type EntryLookups = {
}; };
type HydratedEntry = CultCacheEnvelope & { type HydratedEntry = CultCacheEnvelope & {
catalogEntry: NonNullable<CultCacheEnvelope["catalogEntry"]>;
value: unknown; value: unknown;
lookups: EntryLookups; lookups: EntryLookups;
}; };
@ -62,6 +64,17 @@ type HydratedState = {
// delete of that global removes it from the home store. Remove once no store holds a global under // delete of that global removes it from the home store. Remove once no store holds a global under
// a key other than GLOBAL_KEY. // a key other than GLOBAL_KEY.
readonly legacyGlobals: Map<string, CultCacheEnvelope>; readonly legacyGlobals: Map<string, CultCacheEnvelope>;
// The records the last pull found that no registered type claims. Derived from the pull and replaced with the state; it
// decides nothing but the refusal of a write or a removal at its keys.
readonly foreign: CultCacheForeignRecord[];
};
/** A record a store carries that no registered type claims: the cache does not serve it, keeps its bytes, and refuses a write or a removal at its key. */
export type CultCacheForeignRecord = {
readonly key: string;
readonly schemaId: string;
readonly schemaName: string;
readonly storedAt: string;
}; };
// Lookups for every held record of one type under candidate accessors, derived before anything is installed. // Lookups for every held record of one type under candidate accessors, derived before anything is installed.
@ -79,6 +92,7 @@ function emptyHydratedState(): HydratedState {
indexToKeyMaps: new Map(), indexToKeyMaps: new Map(),
globalKeys: new Map(), globalKeys: new Map(),
legacyGlobals: new Map(), legacyGlobals: new Map(),
foreign: [],
}; };
} }
@ -224,7 +238,7 @@ export class CultCache {
return this.#serial(() => { return this.#serial(() => {
const registered = this.#requireDefinition(definition); const registered = this.#requireDefinition(definition);
const nameAccessor = this.#compileAccessor(definition.type, "name", accessor); const nameAccessor = this.#compileAccessor(definition.type, "name", accessor);
const derived = this.#deriveTypeLookups(definition.type, nameAccessor, registered.indexAccessors); const derived = this.#deriveTypeLookups(definition.type, registered.catalogEntry.schemaName, nameAccessor, registered.indexAccessors);
this.#installAccessors(registered, nameAccessor, registered.indexAccessors, derived); this.#installAccessors(registered, nameAccessor, registered.indexAccessors, derived);
return definition; return definition;
}); });
@ -240,10 +254,14 @@ export class CultCache {
throw new Error(`CultCache index names for type "${definition.type}" must be non-empty.`); throw new Error(`CultCache index names for type "${definition.type}" must be non-empty.`);
} }
if (declaresRole(definition, indexName)) {
throw new Error(`Index "${indexName}" of ${definition.type} is declared as a role; a role is never a unique index.`);
}
const registered = this.#requireDefinition(definition); const registered = this.#requireDefinition(definition);
const indexAccessors = new Map(registered.indexAccessors); const indexAccessors = new Map(registered.indexAccessors);
indexAccessors.set(indexName, this.#compileAccessor(definition.type, `index "${indexName}"`, accessor)); indexAccessors.set(indexName, this.#compileAccessor(definition.type, `index "${indexName}"`, accessor));
const derived = this.#deriveTypeLookups(definition.type, registered.nameAccessor, indexAccessors); const derived = this.#deriveTypeLookups(definition.type, registered.catalogEntry.schemaName, registered.nameAccessor, indexAccessors);
this.#installAccessors(registered, registered.nameAccessor, indexAccessors, derived); this.#installAccessors(registered, registered.nameAccessor, indexAccessors, derived);
return definition; return definition;
}); });
@ -274,11 +292,14 @@ export class CultCache {
for (const entry of entries) { for (const entry of entries) {
const registered = this.#resolveDefinitionForEnvelope(entry); const registered = this.#resolveDefinitionForEnvelope(entry);
if (!registered) { if (!registered) {
throw new Error( // A store may hold what this cache has no type for; it is carried, not served, and not a reason to refuse the load.
entry.schemaId next.foreign.push({
? `No schema is registered for persisted schema id "${entry.schemaId}".` key: entry.key,
: `No schema is registered for persisted document type "${entry.type}".`, schemaId: schemaIdForEnvelope(entry),
); schemaName: entry.catalogEntry?.schemaName ?? entry.type,
storedAt: entry.storedAt,
});
continue;
} }
const type = registered.definition.type; const type = registered.definition.type;
@ -307,10 +328,37 @@ export class CultCache {
} }
} }
const hydrated = this.#admitEntry(next, registered, admitted, value); const hydrated = this.#admitEntry(next, registered, admitted, value);
const other = this.#heldUnderOtherType(next, type, hydrated.key);
if (other !== undefined) {
throw new Error(`Record ${hydrated.key} is held by two stores, as type "${other.type}" and as type "${type}".`);
}
this.#installEntry(next, registered, hydrated); this.#installEntry(next, registered, hydrated);
} }
} }
const foreignByKey = new Map<string, string>();
for (const record of next.foreign) {
const first = foreignByKey.get(record.key);
if (first !== undefined) {
throw new Error(`Record ${record.key} is held by two stores, as schema "${first}" and as schema "${record.schemaName}".`);
}
foreignByKey.set(record.key, record.schemaName);
const other = [...this.#definitions.keys()].map((type) => next.entries.get(this.#entryId(type, record.key))).find((held) => held !== undefined);
if (other !== undefined) {
throw new Error(`Record ${record.key} is held by two stores, as type "${other.type}" and as schema "${record.schemaName}".`);
}
}
// Judged whole, once: the refusal names the least violation of the state this load would leave, whatever the
// order of the stores and of the entries in them.
refuseLeastIndexSharing(function* () {
for (const entry of next.entries.values()) {
for (const [alias, value] of entry.lookups.indexes) {
yield { type: entry.type, schemaName: entry.catalogEntry.schemaName, alias, value, key: entry.key };
}
}
}());
this.#hydrated = next; this.#hydrated = next;
}); });
} }
@ -418,6 +466,7 @@ export class CultCache {
return key ? this.get(definition, key) : undefined; return key ? this.get(definition, key) : undefined;
} }
// A unique index: one key per non-blank value. A roles alias is never indexed and answers undefined.
getKeyByIndex<TDefinition extends AnyCultCacheDocumentDefinition>( getKeyByIndex<TDefinition extends AnyCultCacheDocumentDefinition>(
definition: TDefinition, definition: TDefinition,
indexName: string, indexName: string,
@ -536,6 +585,11 @@ export class CultCache {
}); });
} }
// What the last pull found that no registered type claims, by key ordinal. A later registration claims a record at the next pull.
foreignRecords(): readonly CultCacheForeignRecord[] {
return [...this.#hydrated.foreign].sort((left, right) => compareOrdinal(left.key, right.key));
}
snapshot(): CultCacheEnvelope[] { snapshot(): CultCacheEnvelope[] {
return [...this.#hydrated.entries.values()].map((entry) => this.#toEnvelope(entry)); return [...this.#hydrated.entries.values()].map((entry) => this.#toEnvelope(entry));
} }
@ -579,7 +633,7 @@ export class CultCache {
if (existingBySchemaName && existingBySchemaName !== registered) { if (existingBySchemaName && existingBySchemaName !== registered) {
throw refuse(existingBySchemaName, registered.catalogEntry, `schema name "${registered.catalogEntry.schemaName}"`); throw refuse(existingBySchemaName, registered.catalogEntry, `schema name "${registered.catalogEntry.schemaName}"`);
} }
const derived = this.#deriveTypeLookups(definition.type, nameAccessor, indexAccessors); const derived = this.#deriveTypeLookups(definition.type, registered.catalogEntry.schemaName, nameAccessor, indexAccessors);
// Every check and accessor has run; install. // Every check and accessor has run; install.
this.#definitions.set(definition.type, registered); this.#definitions.set(definition.type, registered);
@ -645,16 +699,73 @@ export class CultCache {
// is touched; once the store accepts, applying to the cache has no fallible step. // is touched; once the store accepts, applying to the cache has no fallible step.
async #writeNow(registered: RegisteredDefinition, entry: CultCacheEnvelope, value: unknown): Promise<void> { async #writeNow(registered: RegisteredDefinition, entry: CultCacheEnvelope, value: unknown): Promise<void> {
const home = this.#homeStore(entry.type); const home = this.#homeStore(entry.type);
const retyped = this.#refuseRetype(entry);
if (retyped !== undefined && this.#homeStore(retyped.type) !== home) {
// The write owns the held record's id and would replace it, but the held record is in another store; the push
// reaches only this type's home, so the old record would stay in its own file and load as a second holder.
throw new SchemaConflictError(
schemaIdForEnvelope(entry),
[retyped.type, entry.type],
entry.key,
`CultCache holds record "${entry.key}" as type "${retyped.type}" in another store than the home of type "${entry.type}", so the rename cannot replace it; remove the record first.`,
);
}
const hydrated = this.#admitEntry(this.#hydrated, registered, entry, value); const hydrated = this.#admitEntry(this.#hydrated, registered, entry, value);
this.#refuseIndexSharing(this.#hydrated, entry.type, registered.catalogEntry.schemaName, entry.key, hydrated.lookups);
const legacy = this.#hydrated.legacyGlobals.get(entry.type); const legacy = this.#hydrated.legacyGlobals.get(entry.type);
await home?.push(entry); await home?.push(entry);
if (home && legacy) { if (home && legacy) {
await this.#retireLegacyGlobal(home, legacy, entry); await this.#retireLegacyGlobal(home, legacy, entry);
} }
this.#hydrated.legacyGlobals.delete(entry.type); this.#hydrated.legacyGlobals.delete(entry.type);
if (retyped !== undefined) {
// The record the write restamped (it owns the held id) is the same record: one key, one held type.
this.#removeEntry(this.#hydrated, this.#definitions.get(retyped.type)!, retyped);
}
this.#installEntry(this.#hydrated, registered, hydrated); this.#installEntry(this.#hydrated, registered, hydrated);
} }
// The one key-identity judge of the cache: may this write land at its key, given what the cache holds there under
// every registered type and the keys its stores carry foreign. The store judges the file with the same predicate
// (writtenOwns); this judges the cache, which a store-less cache has no other judge for. Returns the held record of
// another type the write owns and so replaces, if any. A refusal names the key and schemas, never a payload.
#refuseRetype(entry: CultCacheEnvelope): HydratedEntry | undefined {
this.#refuseForeignKey(entry.key, entry.type, "written");
const held = this.#heldUnderOtherType(this.#hydrated, entry.type, entry.key);
if (held === undefined || writtenOwns(entry, schemaIdForEnvelope(held))) {
return held;
}
throw new SchemaConflictError(
schemaIdForEnvelope(entry),
[held.type, entry.type],
entry.key,
`CultCache holds record "${entry.key}" as type "${held.type}", so type "${entry.type}" cannot be written under the same key; remove the record first.`,
);
}
#refuseForeignKey(key: string, type: string, what: "written" | "removed"): void {
const foreign = this.#hydrated.foreign.find((record) => record.key === key);
if (foreign !== undefined) {
throw new SchemaConflictError(
foreign.schemaId,
[foreign.schemaName],
key,
`CultCache store carries record "${key}" as schema "${foreign.schemaName}" (${foreign.schemaId}), which no registered type claims, so type "${type}" cannot be ${what} at that key.`,
);
}
}
// The record another registered type holds at the key, if any.
#heldUnderOtherType(state: HydratedState, type: string, key: string): HydratedEntry | undefined {
for (const other of this.#definitions.keys()) {
const held = other === type ? undefined : state.entries.get(this.#entryId(other, key));
if (held !== undefined) {
return held;
}
}
return undefined;
}
// The first write of a global loaded from a legacy key removes that record in the same write. If the // The first write of a global loaded from a legacy key removes that record in the same write. If the
// removal fails, the new record is taken back out, so the store keeps only the legacy record. // removal fails, the new record is taken back out, so the store keeps only the legacy record.
async #retireLegacyGlobal(home: CacheBackingStore, legacy: CultCacheEnvelope, written: CultCacheEnvelope): Promise<void> { async #retireLegacyGlobal(home: CacheBackingStore, legacy: CultCacheEnvelope, written: CultCacheEnvelope): Promise<void> {
@ -673,6 +784,9 @@ export class CultCache {
const registered = this.#requireDefinition(definition); const registered = this.#requireDefinition(definition);
const entry = this.#hydrated.entries.get(this.#entryId(definition.type, key)); const entry = this.#hydrated.entries.get(this.#entryId(definition.type, key));
if (!entry) { if (!entry) {
// variants:ruling:unheld-removal-refused-at-commit: a key a store carries foreign is refused, a key held under
// another type or by nobody is the existing false.
this.#refuseForeignKey(key, definition.type, "removed");
return false; return false;
} }
@ -795,11 +909,13 @@ export class CultCache {
switch (typeof value) { switch (typeof value) {
case "string": case "string":
return value; return isBlankIndexText(value) ? undefined : value;
case "number":
case "boolean": case "boolean":
return value ? "True" : "False";
case "bigint": case "bigint":
return String(value); return value.toString();
case "number":
return canonicalNumberText(value);
default: default:
throw new Error(`CultCache ${label} accessor for type "${type}" produced a non-scalar value.`); throw new Error(`CultCache ${label} accessor for type "${type}" produced a non-scalar value.`);
} }
@ -894,6 +1010,7 @@ export class CultCache {
} }
} }
const lookups = this.#lookupsFor(registered.nameAccessor, registered.indexAccessors, value);
return { return {
...entry, ...entry,
payload: this.#cloneBytes(entry.payload), payload: this.#cloneBytes(entry.payload),
@ -902,10 +1019,22 @@ export class CultCache {
schemaId: registered.catalogEntry.schemaId, schemaId: registered.catalogEntry.schemaId,
catalogEntry: registered.catalogEntry, catalogEntry: registered.catalogEntry,
value, value,
lookups: this.#lookupsFor(registered.nameAccessor, registered.indexAccessors, value), lookups,
}; };
} }
// The one index rule (variants:ruling:unique-indexes): within one type and one unique alias, no two records hold one
// non-blank value. The held key at the landing record's own key is its previous self and keeps its value. The message
// names both keys, ordinal, and the alias, never the value.
#refuseIndexSharing(state: HydratedState, type: string, schemaName: string, key: string, lookups: EntryLookups): void {
for (const alias of [...lookups.indexes.keys()].sort(compareOrdinal)) {
const held = state.indexToKeyMaps.get(this.#indexId(type, alias))?.get(lookups.indexes.get(alias)!);
if (held !== undefined && held !== key) {
throw indexSharingRefusal(held, key, schemaName, alias);
}
}
}
// Infallible half of admission: replaces any held record at the same key and installs the admitted one. // Infallible half of admission: replaces any held record at the same key and installs the admitted one.
#installEntry(state: HydratedState, registered: RegisteredDefinition, hydrated: HydratedEntry): void { #installEntry(state: HydratedState, registered: RegisteredDefinition, hydrated: HydratedEntry): void {
const entryId = this.#entryId(hydrated.type, hydrated.key); const entryId = this.#entryId(hydrated.type, hydrated.key);
@ -966,6 +1095,7 @@ export class CultCache {
// Runs the candidate accessors over every held record of the type into local maps. // Runs the candidate accessors over every held record of the type into local maps.
#deriveTypeLookups( #deriveTypeLookups(
type: string, type: string,
schemaName: string,
nameAccessor: Accessor | undefined, nameAccessor: Accessor | undefined,
indexAccessors: ReadonlyMap<string, Accessor>, indexAccessors: ReadonlyMap<string, Accessor>,
): DerivedTypeLookups { ): DerivedTypeLookups {
@ -983,10 +1113,18 @@ export class CultCache {
nameMap?.set(lookups.name, entry.key); nameMap?.set(lookups.name, entry.key);
} }
for (const [indexName, indexValue] of lookups.indexes) { for (const [indexName, indexValue] of lookups.indexes) {
indexMaps.get(indexName)?.set(indexValue, entry.key); const map = indexMaps.get(indexName)!;
if (!map.has(indexValue)) map.set(indexValue, entry.key);
} }
} }
// Judged whole, by the same judge as a load: the least violation of the state the registration would leave.
refuseLeastIndexSharing(function* () {
for (const [entry, lookups] of entryLookups) {
for (const [alias, value] of lookups.indexes) yield { type, schemaName, alias, value, key: entry.key };
}
}());
return { entryLookups, nameMap, indexMaps }; return { entryLookups, nameMap, indexMaps };
} }
@ -1057,3 +1195,70 @@ export class CultCache {
return documentPayloadFromBytes(new Uint8Array(documentPayloadBytes(payload))); return documentPayloadFromBytes(new Uint8Array(documentPayloadBytes(payload)));
} }
} }
function compareOrdinal(a: string, b: string): number {
return a < b ? -1 : a > b ? 1 : 0;
}
// The whole-state judge (variants:question:unique-index-refusal-pair-rule): over every (type, alias, value) held at two
// or more distinct keys, throws for the one least by (alias, least key, second-least key, schema name), each compared by
// UTF-16 code unit. A function of the state, never of the order the rows arrive in.
function refuseLeastIndexSharing(rows: Iterable<{ type: string; schemaName: string; alias: string; value: string; key: string }>): void {
const groups = new Map<string, { schemaName: string; alias: string; keys: string[] }>();
for (const { type, schemaName, alias, value, key } of rows) {
const id = JSON.stringify([type, alias, value]);
const group = groups.get(id);
if (group) group.keys.push(key);
else groups.set(id, { schemaName, alias, keys: [key] });
}
let least: { schemaName: string; alias: string; first: string; second: string } | undefined;
for (const { schemaName, alias, keys } of groups.values()) {
if (keys.length < 2) continue;
keys.sort(compareOrdinal);
const candidate = { schemaName, alias, first: keys[0], second: keys[1] };
if (!least || compareLeast(candidate, least) < 0) least = candidate;
}
if (least) throw indexSharingRefusal(least.first, least.second, least.schemaName, least.alias);
}
function compareLeast(
a: { schemaName: string; alias: string; first: string; second: string },
b: { schemaName: string; alias: string; first: string; second: string },
): number {
return compareOrdinal(a.alias, b.alias) || compareOrdinal(a.first, b.first) || compareOrdinal(a.second, b.second) || compareOrdinal(a.schemaName, b.schemaName);
}
function indexSharingRefusal(keyA: string, keyB: string, schemaName: string, alias: string): Error {
const [first, second] = compareOrdinal(keyA, keyB) <= 0 ? [keyA, keyB] : [keyB, keyA];
return new Error(`Records ${first} and ${second} (${schemaName}) both hold unique index '${alias}'; an index value names one record.`);
}
// Blank is no value: empty or only C# White_Space characters. Not JavaScript's \s, which counts U+FEFF and omits U+0085.
const BLANK_INDEX_TEXT = /^[\t-\r \u0085\u00A0\u1680\u2000-\u200A\u2028\u2029\u202F\u205F\u3000]*$/;
function isBlankIndexText(value: string): boolean {
return BLANK_INDEX_TEXT.test(value);
}
// The canonical text of a number, as C# RenderCanonicalNumber: integers as their digits (-0 is 0), a non-integral finite
// double as its exact decimal expansion, NaN and infinities as no value.
function canonicalNumberText(value: number): string | undefined {
if (!Number.isFinite(value)) {
return undefined;
}
if (Number.isInteger(value)) {
return BigInt(value).toString();
}
// value = mantissa * 2^exponent with exponent < 0, so value = mantissa * 5^k / 10^k for k = -exponent.
const view = new DataView(new ArrayBuffer(8));
view.setFloat64(0, value);
const bits = view.getBigUint64(0);
const biasedExponent = Number((bits >> 52n) & 0x7ffn);
const fraction = bits & 0xfffffffffffffn;
const mantissa = biasedExponent === 0 ? fraction : fraction | (1n << 52n);
const k = -((biasedExponent === 0 ? 1 : biasedExponent) - 1075);
const digits = (mantissa * 5n ** BigInt(k)).toString().padStart(k + 1, "0");
const text = `${digits.slice(0, digits.length - k)}.${digits.slice(digits.length - k)}`.replace(/0+$/, "").replace(/\.$/, "");
return value < 0 ? `-${text}` : text;
}

View file

@ -32,23 +32,22 @@ export function defineDocumentType<TSchema extends CultCacheSchema>(
throw new Error(`CultCache document type "${definition.type}" declares an invalid name accessor.`); throw new Error(`CultCache document type "${definition.type}" declares an invalid name accessor.`);
} }
if (Array.isArray(definition.indexes)) { const declared = new Set<string>();
const seen = new Set<string>(); for (const [kind, set] of [["index", definition.indexes], ["role", definition.roles]] as const) {
for (const index of definition.indexes) { const names = new Set<string>();
if (!index.name || index.name.trim().length === 0) { for (const name of Array.isArray(set) ? set.map((entry) => entry.name) : set ? Object.keys(set) : []) {
throw new Error(`CultCache document type "${definition.type}" declares an index with an empty name.`);
}
if (seen.has(index.name)) {
throw new Error(`CultCache document type "${definition.type}" declares duplicate index "${index.name}".`);
}
seen.add(index.name);
}
} else if (definition.indexes) {
for (const name of Object.keys(definition.indexes)) {
if (!name || name.trim().length === 0) { if (!name || name.trim().length === 0) {
throw new Error(`CultCache document type "${definition.type}" declares an index with an empty name.`); throw new Error(`CultCache document type "${definition.type}" declares ${kind === "index" ? "an index" : "a role"} with an empty name.`);
} }
if (names.has(name)) {
throw new Error(`CultCache document type "${definition.type}" declares duplicate ${kind} "${name}".`);
}
if (kind === "role" && declared.has(name)) {
throw new Error(`CultCache document type "${definition.type}" declares "${name}" as both an index and a role; an index already names its role.`);
}
names.add(name);
} }
for (const name of names) declared.add(name);
} }
if (definition.members) { if (definition.members) {
@ -132,16 +131,25 @@ export function encodeDocumentPayload(definition: AnyCultCacheDocumentDefinition
return documentPayloadFromBytes(bytes); return documentPayloadFromBytes(bytes);
} }
// Whether the definition declares `name` as a role alias (either `roles` form).
export function declaresRole(definition: AnyCultCacheDocumentDefinition, name: string): boolean {
const roles = definition.roles;
return Array.isArray(roles) ? roles.some((entry) => entry.name === name) : roles ? Object.hasOwn(roles, name) : false;
}
// The schema members of a document: the ones it declares, else those its codec derives, else none. // The schema members of a document: the ones it declares, else those its codec derives, else none.
export function resolveDocumentMembers(definition: AnyCultCacheDocumentDefinition): readonly CultCacheSchemaCatalogMember[] { export function resolveDocumentMembers(definition: AnyCultCacheDocumentDefinition): readonly CultCacheSchemaCatalogMember[] {
if (definition.members || !definition.codec) { if (definition.members || !definition.codec) {
return definition.members ?? []; return definition.members ?? [];
} }
const codec = definition.codec; const codec = definition.codec;
const indexAlias = (slotName: string) => const indexAlias = (slotName: string) => {
!Array.isArray(definition.indexes) && definition.indexes for (const set of [definition.indexes, definition.roles]) {
? Object.entries(definition.indexes).find(([, accessor]) => accessor === slotName)?.[0] ?? null const alias = !Array.isArray(set) && set ? Object.entries(set).find(([, accessor]) => accessor === slotName)?.[0] : undefined;
: null; if (alias !== undefined) return alias;
}
return null;
};
return codec.members.map((member, index) => ({ return codec.members.map((member, index) => ({
...member, ...member,
isName: (definition.name as unknown) === codec.names[index], isName: (definition.name as unknown) === codec.names[index],

View file

@ -7,7 +7,7 @@ import { z } from "zod";
import { documentPayloadBytes, documentPayloadFromBytes } from "./document"; import { documentPayloadBytes, documentPayloadFromBytes } from "./document";
import { commitStoreFile, withStoreWriteLock } from "./store-file"; import { commitStoreFile, withStoreWriteLock } from "./store-file";
import { STORE_FORMAT_VERSION, SchemaConflictError, type StoreFormat, StoreUnreadableError, isStoreSnapshot, refuseUnreadStoreBytes, requireStoreSlots, requireV1RecordSlots } from "./store-format"; import { STORE_FORMAT_VERSION, SchemaConflictError, type StoreFormat, StoreUnreadableError, isStoreSnapshot, refuseUnreadStoreBytes, requireStoreSlots, requireV1RecordSlots, schemaIdForEnvelope, writtenOwns } from "./store-format";
import type { import type {
CacheBackingStore, CacheBackingStore,
CultCacheEnvelope, CultCacheEnvelope,
@ -47,21 +47,11 @@ export class SingleFileMessagePackBackingStore implements CacheBackingStore {
} }
async pullAll(): Promise<CultCacheEnvelope[]> { async pullAll(): Promise<CultCacheEnvelope[]> {
// The disk decides the header: a file that is gone or legacy is not marked. // A pure read: the disk decides the header, and a legacy file is normalized in memory only. The file is rewritten
this.#format = STORE_FORMAT_VERSION; // as v1 by the next push, delete or flush, which merges what this read returns; opening never replaces bytes the
// consumer's codec has not yet accepted.
const disk = this.#readDisk(); const disk = this.#readDisk();
this.#format = disk.format; this.#format = disk.format;
if (disk.repairedLegacyPayload) {
// The heal decides from its own read, taken under the store lock.
await this.#enqueue(async () => {
const again = this.#readDisk();
this.#format = again.format;
if (again.repairedLegacyPayload) {
await this.#writeAll([], again.envelopes);
}
});
}
return disk.envelopes; return disk.envelopes;
} }
@ -70,9 +60,8 @@ export class SingleFileMessagePackBackingStore implements CacheBackingStore {
const disk = this.#readDisk(); const disk = this.#readDisk();
this.#format = disk.format; this.#format = disk.format;
const existing = disk.envelopes; const existing = disk.envelopes;
const filtered = existing.filter( requireOneTypePerKey(this.filePath, [entry], existing);
(candidate) => !(candidate.type === entry.type && candidate.key === entry.key), const filtered = existing.filter((candidate) => candidate.key !== entry.key);
);
await this.#writeAll([entry], filtered); await this.#writeAll([entry], filtered);
}); });
} }
@ -82,8 +71,10 @@ export class SingleFileMessagePackBackingStore implements CacheBackingStore {
const disk = this.#readDisk(); const disk = this.#readDisk();
this.#format = disk.format; this.#format = disk.format;
const existing = disk.envelopes; const existing = disk.envelopes;
// The record the file stores at the key is the one removed when the removal's schema owns its stored id: the
// arrived record's type is its schema's name, which a type of another name does not carry.
const filtered = existing.filter( const filtered = existing.filter(
(candidate) => !(candidate.type === entry.type && candidate.key === entry.key), (candidate) => !(candidate.key === entry.key && (schemaIdForEnvelope(candidate) === schemaIdForEnvelope(entry) || writtenOwns(entry, schemaIdForEnvelope(candidate)))),
); );
await this.#writeAll([], filtered); await this.#writeAll([], filtered);
}); });
@ -113,7 +104,9 @@ export class SingleFileMessagePackBackingStore implements CacheBackingStore {
this.#format = STORE_FORMAT_VERSION; this.#format = STORE_FORMAT_VERSION;
} }
await this.#writeAll(entries, []); requireOneTypePerKey(this.filePath, entries, []);
// A key is held once: of same-type entries sharing a key in one batch, the last is kept.
await this.#writeAll([...new Map(entries.map((entry) => [entry.key, entry])).values()], []);
}); });
} }
@ -147,6 +140,28 @@ export class SingleFileMessagePackBackingStore implements CacheBackingStore {
} }
} }
// A store holds a key once, and the reader refuses a file that holds it twice, so a write that would put a second type under a
// key (against a record on disk, or within its own batch) is refused before anything is written. Retyping a record is a delete
// of the old type followed by a put of the new one.
function requireOneTypePerKey(path: string, supplied: CultCacheEnvelope[], held: CultCacheEnvelope[]): void {
const typeByKey = new Map<string, CultCacheEnvelope>();
for (const entry of held) {
typeByKey.set(entry.key, entry);
}
for (const entry of supplied) {
const prior = typeByKey.get(entry.key);
if (prior !== undefined && prior.type !== entry.type && !writtenOwns(entry, schemaIdForEnvelope(prior))) {
throw new SchemaConflictError(
schemaIdForEnvelope(entry),
[prior.type, entry.type],
entry.key,
`CultCache store ${path} holds record "${entry.key}" as type "${prior.type}", so type "${entry.type}" cannot be written under the same key; remove the record first.`,
);
}
typeByKey.set(entry.key, entry);
}
}
/** /**
* The one reader of a CultCache single-file store, for this package's store and for any consumer that must read a * The one reader of a CultCache single-file store, for this package's store and for any consumer that must read a
* store file (an authority bundle, say) rather than decode it itself. A path with nothing at it is an absent, empty, * store file (an authority bundle, say) rather than decode it itself. A path with nothing at it is an absent, empty,
@ -158,7 +173,7 @@ export class SingleFileMessagePackBackingStore implements CacheBackingStore {
export function readSingleFileStore(path: string): SingleFileStoreRead { export function readSingleFileStore(path: string): SingleFileStoreRead {
const data = readStoreBytes(path); const data = readStoreBytes(path);
if (data === undefined) { if (data === undefined) {
return { absent: true, format: STORE_FORMAT_VERSION, envelopes: [], repairedLegacyPayload: false }; return { absent: true, format: STORE_FORMAT_VERSION, envelopes: [] };
} }
try { try {
@ -214,8 +229,9 @@ type DecodedSnapshot = {
}; };
function encodeSnapshot(supplied: CultCacheEnvelope[], arrived: CultCacheEnvelope[], format: StoreFormat): unknown[] { function encodeSnapshot(supplied: CultCacheEnvelope[], arrived: CultCacheEnvelope[], format: StoreFormat): unknown[] {
const replaced = new Set(supplied.map((entry) => `${entry.type}::${entry.key}`)); // A key is held once: a supplied record replaces the arrived record under its key whatever the type.
const kept = arrived.filter((entry) => !replaced.has(`${entry.type}::${entry.key}`)); const replaced = new Set(supplied.map((entry) => entry.key));
const kept = arrived.filter((entry) => !replaced.has(entry.key));
const entries = [...kept, ...supplied]; const entries = [...kept, ...supplied];
const catalog = catalogEntriesFor([ const catalog = catalogEntriesFor([
...kept.map((envelope) => ({ envelope, registered: false })), ...kept.map((envelope) => ({ envelope, registered: false })),
@ -361,10 +377,6 @@ function defaultEntryFor(entry: CultCacheEnvelope): CultCacheSchemaCatalogEntry
}; };
} }
function schemaIdForEnvelope(entry: CultCacheEnvelope): string {
return entry.schemaId ?? entry.type;
}
function encodeCatalogEntry(entry: CultCacheSchemaCatalogEntry): unknown[] { function encodeCatalogEntry(entry: CultCacheSchemaCatalogEntry): unknown[] {
return [ return [
entry.schemaId, entry.schemaId,
@ -395,8 +407,6 @@ export type SingleFileStoreRead = {
absent: boolean; absent: boolean;
format: StoreFormat; format: StoreFormat;
envelopes: CultCacheEnvelope[]; envelopes: CultCacheEnvelope[];
// A legacy file whose payloads were not bytes: reading it repairs them, and the store rewrites the file.
repairedLegacyPayload: boolean;
}; };
function decodeStoreFile(data: Uint8Array): Omit<SingleFileStoreRead, "absent"> { function decodeStoreFile(data: Uint8Array): Omit<SingleFileStoreRead, "absent"> {
@ -420,7 +430,7 @@ function decodeStoreFile(data: Uint8Array): Omit<SingleFileStoreRead, "absent">
storedAt: record.storedAt, storedAt: record.storedAt,
}; };
}); });
return { format: snapshot.format, envelopes, repairedLegacyPayload: false }; return { format: snapshot.format, envelopes };
} }
const legacy = decodeLegacyEnvelopeArray(decoded); const legacy = decodeLegacyEnvelopeArray(decoded);
@ -428,19 +438,9 @@ function decodeStoreFile(data: Uint8Array): Omit<SingleFileStoreRead, "absent">
throw new Error("CultCache file is not a recognized CultCache MessagePack store."); throw new Error("CultCache file is not a recognized CultCache MessagePack store.");
} }
let repairedLegacyPayload = false; requireUniqueKeys(legacy);
const normalized = legacy.map((entry) => { const normalized = legacy.map((entry) => ({ ...entry, payload: normalizePayload(entry.payload) }));
const payload = normalizePayload(entry.payload); return { format: STORE_FORMAT_VERSION, envelopes: envelopeArraySchema.parse(normalized).map((entry) => ({ ...entry, payload: documentPayloadFromBytes(entry.payload) })) as CultCacheEnvelope[] };
if (payload !== entry.payload) {
repairedLegacyPayload = true;
}
return {
...entry,
payload,
};
});
return { format: STORE_FORMAT_VERSION, envelopes: envelopeArraySchema.parse(normalized).map((entry) => ({ ...entry, payload: documentPayloadFromBytes(entry.payload) })) as CultCacheEnvelope[], repairedLegacyPayload };
} }
function decodeSnapshot(decoded: unknown): DecodedSnapshot | undefined { function decodeSnapshot(decoded: unknown): DecodedSnapshot | undefined {
@ -472,9 +472,21 @@ function decodeSnapshot(decoded: unknown): DecodedSnapshot | undefined {
} }
const records = recordsRaw.map(decodeRecord); const records = recordsRaw.map(decodeRecord);
requireUniqueKeys(records);
return { format: decoded[0], catalogBySchemaId, records }; return { format: decoded[0], catalogBySchemaId, records };
} }
// A store holds a key once, whatever the type: a repeat is refused in a v1 snapshot and in a legacy envelope array alike.
function requireUniqueKeys(records: Array<{ key: string }>): void {
const seen = new Set<string>();
for (const { key } of records) {
if (seen.has(key)) {
throw new Error(`CultCache record "${key}" appears more than once.`);
}
seen.add(key);
}
}
function decodeCatalogEntry(value: unknown): CultCacheSchemaCatalogEntry { function decodeCatalogEntry(value: unknown): CultCacheSchemaCatalogEntry {
if (!Array.isArray(value)) { if (!Array.isArray(value)) {
throw new Error("CultCache schema catalog entries must be MessagePack arrays."); throw new Error("CultCache schema catalog entries must be MessagePack arrays.");
@ -502,12 +514,16 @@ function decodeCatalogEntry(value: unknown): CultCacheSchemaCatalogEntry {
throw new Error(`CultCache schema catalog entry "${schemaId}" has invalid members.`); throw new Error(`CultCache schema catalog entry "${schemaId}" has invalid members.`);
} }
if (typeof canonicalSchemaJson !== "string") {
throw new Error(`CultCache schema catalog entry "${schemaId}" has an invalid canonical schema.`);
}
return { return {
schemaId, schemaId,
schemaName, schemaName,
schemaVersion: isNonEmptyString(schemaVersion) ? schemaVersion : `${schemaName}.v1`, schemaVersion: isNonEmptyString(schemaVersion) ? schemaVersion : `${schemaName}.v1`,
contentHash: isNonEmptyString(contentHash) ? contentHash : schemaId, contentHash: isNonEmptyString(contentHash) ? contentHash : schemaId,
canonicalSchemaJson: typeof canonicalSchemaJson === "string" ? canonicalSchemaJson : "", canonicalSchemaJson,
compatibleSchemaIds, compatibleSchemaIds,
members: members.map(decodeCatalogMember), members: members.map(decodeCatalogMember),
}; };

View file

@ -1,5 +1,7 @@
import { decode } from "@msgpack/msgpack"; import { decode } from "@msgpack/msgpack";
import type { CultCacheEnvelope } from "./types";
export const STORE_FORMAT_VERSION = "cultcache.store.v1"; export const STORE_FORMAT_VERSION = "cultcache.store.v1";
/** /**
@ -151,3 +153,19 @@ export function requireStoreSlots(decoded: unknown[]): void {
throw new Error(`CultCache store has ${decoded.length} top-level slots; this runtime reads ${STORE_SLOTS} (header, schema catalog, records).`); throw new Error(`CultCache store has ${decoded.length} top-level slots; this runtime reads ${STORE_SLOTS} (header, schema catalog, records).`);
} }
} }
/** The schema id a record is stored under: the one it carries, else its type. */
export function schemaIdForEnvelope(entry: CultCacheEnvelope): string {
return entry.schemaId ?? entry.type;
}
/**
* As C# WrittenOwns' declared half: the written type owns the stored schema id when its catalog entry carries that id as its own
* or lists it compatible, so a type renamed under a stable id restamps the record it already holds. (The C# half that lets an
* undeclared older id of the same schema name through when it reads whole is not ported; a same-named type keeps passing.)
* The one predicate of the key-identity rule: the store judges the file with it and the cache judges what it holds.
*/
export function writtenOwns(written: CultCacheEnvelope, storedId: string): boolean {
const entry = written.catalogEntry;
return entry !== undefined && (entry.schemaId === storedId || (entry.compatibleSchemaIds ?? []).includes(storedId));
}

View file

@ -59,9 +59,21 @@ export interface CultCacheDocumentDefinition<TSchema extends CultCacheSchema = C
members?: readonly CultCacheSchemaCatalogMember[]; members?: readonly CultCacheSchemaCatalogMember[];
global?: boolean; global?: boolean;
name?: CultCacheDocumentAccessor<InferCultCacheSchemaValue<TSchema>>; name?: CultCacheDocumentAccessor<InferCultCacheSchemaValue<TSchema>>;
/**
* Unique lookup aliases. A non-blank value names one record of the type: a write that would leave two records holding
* one value, and a store that already holds two, is refused naming both keys and the alias. Blank values (absent,
* empty, or only whitespace) are never indexed. `getByIndex` and `getKeyByIndex` answer from these.
*/
indexes?: indexes?:
| Readonly<Record<string, CultCacheDocumentAccessor<InferCultCacheSchemaValue<TSchema>>>> | Readonly<Record<string, CultCacheDocumentAccessor<InferCultCacheSchemaValue<TSchema>>>>
| readonly CultCacheDocumentIndexDefinition<InferCultCacheSchemaValue<TSchema>>[]; | readonly CultCacheDocumentIndexDefinition<InferCultCacheSchemaValue<TSchema>>[];
/**
* Selection and reference aliases: the member's catalog `indexAlias`. A role implies nothing about uniqueness, many
* records may share a value, and it is never indexed for lookup. An alias is declared in `indexes` or `roles`, not both.
*/
roles?:
| Readonly<Record<string, CultCacheDocumentAccessor<InferCultCacheSchemaValue<TSchema>>>>
| readonly CultCacheDocumentIndexDefinition<InferCultCacheSchemaValue<TSchema>>[];
formatter?: CultCacheDocumentFormatter<InferCultCacheSchemaValue<TSchema>>; formatter?: CultCacheDocumentFormatter<InferCultCacheSchemaValue<TSchema>>;
codec?: SlotsCodec<InferCultCacheSchemaValue<TSchema> & object>; codec?: SlotsCodec<InferCultCacheSchemaValue<TSchema> & object>;
} }

View file

@ -14,7 +14,7 @@ import { z } from "zod";
import { CultCache } from "../src/cult-cache"; import { CultCache } from "../src/cult-cache";
import { inspectCultCacheBytes } from "../src/cult-cache-inspector"; import { inspectCultCacheBytes } from "../src/cult-cache-inspector";
import { defineDocumentRegistry, defineDocumentType, documentPayloadBytes, documentPayloadFromBytes } from "../src/document"; import { defineDocumentRegistry, defineDocumentType, documentPayloadBytes, documentPayloadFromBytes } from "../src/document";
import { SingleFileMessagePackBackingStore, readStoreBytes } from "../src/single-file-messagepack-backing-store"; import { SingleFileMessagePackBackingStore, readSingleFileStore, readStoreBytes } from "../src/single-file-messagepack-backing-store";
import { SchemaConflictError, StoreUnreadableError, refuseUnreadStoreBytes } from "../src/store-format"; import { SchemaConflictError, StoreUnreadableError, refuseUnreadStoreBytes } from "../src/store-format";
import type { AnyCultCacheDocumentDefinition, CacheBackingStore, CultCacheEnvelope, CultCacheSchema } from "../src/types"; import type { AnyCultCacheDocumentDefinition, CacheBackingStore, CultCacheEnvelope, CultCacheSchema } from "../src/types";
@ -143,6 +143,63 @@ test("CultCache supports registry bootstrap, global documents, and lookup by nam
}); });
}); });
test("a put that would land a second type under a persisted key is refused, the cache and the file are unchanged, and the store reloads", async () => {
const alpha = defineDocumentType({ type: "alpha", schema: z.object({ v: z.number() }) });
const beta = defineDocumentType({ type: "beta", schema: z.object({ v: z.number() }) });
const storePath = join(await mkdtemp(join(tmpdir(), "cultcache-ts-")), "cache.msgpack");
const registry = defineDocumentRegistry(alpha, beta);
const open = () => CultCache.builder().withRegistry(registry).withGenericStore(new SingleFileMessagePackBackingStore(storePath)).build();
const cache = open();
await cache.put(alpha, "shared", { v: 1 });
const before = [...(await readFile(storePath))];
await assert.rejects(() => cache.put(beta, "shared", { v: 2 }), SchemaConflictError);
assert.deepEqual([...(await readFile(storePath))], before);
assert.deepEqual(cache.get(alpha, "shared"), { v: 1 });
assert.equal(cache.get(beta, "shared"), undefined);
const reloaded = open();
await reloaded.pullAllBackingStores();
assert.deepEqual(reloaded.get(alpha, "shared"), { v: 1 });
assert.equal(await cache.delete(alpha, "shared"), true);
await cache.put(beta, "shared", { v: 2 });
const retyped = open();
await retyped.pullAllBackingStores();
assert.equal(retyped.get(alpha, "shared"), undefined);
assert.deepEqual(retyped.get(beta, "shared"), { v: 2 });
});
test("a put by a type renamed under the stored id, which lists that id compatible, restamps the record; an unrelated type is refused", async () => {
const old = defineDocumentType({ type: "ren.old", schema: z.object({ v: z.number() }), schemaId: "ren.id.v1", schemaName: "ren.old" });
const renamed = defineDocumentType({
type: "ren.new",
schema: z.object({ v: z.number() }),
schemaId: "ren.id.v2",
schemaName: "ren.new",
compatibleSchemaIds: ["ren.id.v2", "ren.id.v1"],
});
const other = defineDocumentType({ type: "ren.other", schema: z.object({ v: z.number() }), schemaId: "ren.id.other", schemaName: "ren.other" });
const storePath = join(await mkdtemp(join(tmpdir(), "cultcache-ts-")), "cache.msgpack");
const open = (...types: (typeof old)[]) =>
types
.reduce((builder, type) => builder.withDocumentType(type), CultCache.builder())
.withGenericStore(new SingleFileMessagePackBackingStore(storePath))
.build();
await open(old).put(old, "k", { v: 1 });
const cache = open(renamed, other as unknown as typeof old);
await cache.pullAllBackingStores();
assert.deepEqual(cache.get(renamed, "k"), { v: 1 });
await assert.rejects(() => cache.put(other, "k", { v: 9 }), SchemaConflictError);
await cache.put(renamed, "k", { v: 2 });
const reloaded = open(renamed);
await reloaded.pullAllBackingStores();
assert.deepEqual(reloaded.get(renamed, "k"), { v: 2 });
assert.equal(readSingleFileStore(storePath).envelopes.filter((envelope) => envelope.key === "k").length, 1);
});
test("CultCache can register name and index lookups after entries already exist", async () => { test("CultCache can register name and index lookups after entries already exist", async () => {
const noteDocument = defineDocumentType({ const noteDocument = defineDocumentType({
type: "note", type: "note",
@ -209,10 +266,10 @@ test("CultCache fails closed on unknown or illegal persisted polymorphic state",
])) ]))
.build(); .build();
await assert.rejects( // A record no registered type claims is carried, not a reason to refuse the load.
async () => cacheWithUnknownType.pullAllBackingStores(), await cacheWithUnknownType.pullAllBackingStores();
/No schema is registered for persisted document type "forbidden"\./, assert.deepEqual(cacheWithUnknownType.snapshot(), []);
); assert.deepEqual(cacheWithUnknownType.foreignRecords().map((record) => [record.key, record.schemaName]), [["mystery", "forbidden"]]);
const cacheWithDuplicateGlobal = CultCache.builder() const cacheWithDuplicateGlobal = CultCache.builder()
.withDocumentType(settingsDocument) .withDocumentType(settingsDocument)
@ -504,46 +561,56 @@ test("SingleFileMessagePackBackingStore refuses a record whose schema the catalo
await assert.rejects(() => cache.pullAllBackingStores(), StoreUnreadableError); await assert.rejects(() => cache.pullAllBackingStores(), StoreUnreadableError);
}); });
test("SingleFileMessagePackBackingStore heals legacy envelopes whose payload was persisted as an object", async () => { const legacyNoteType = defineDocumentType({
const noteDocument = defineDocumentType({ type: "note",
type: "note", schema: z.object({ title: z.string(), body: z.string().optional() }),
schema: z.object({ });
title: z.string(),
body: z.string(),
}),
});
// SYNTHETIC pre-v1 file (no real one is known): an envelope array with object payloads.
async function legacyNoteStore(second: unknown): Promise<{ storePath: string; bytes: Uint8Array }> {
const storePath = join(await mkdtemp(join(tmpdir(), "cultcache-ts-")), "legacy.msgpack"); const storePath = join(await mkdtemp(join(tmpdir(), "cultcache-ts-")), "legacy.msgpack");
const envelopePayload = { const bytes = encode([
title: "Hello", { key: "note:hello", type: "note", payload: { title: "Hello", body: "world" }, storedAt: "2026-10-20T00:00:00Z" },
body: "world", { key: "note:other", type: "note", payload: second, storedAt: "2026-10-20T00:00:00Z" },
}; ]);
await writeFile(storePath, bytes);
await writeFile( return { storePath, bytes };
storePath, }
encode([
{
key: "note:hello",
type: "note",
payload: envelopePayload,
storedAt: new Date().toISOString(),
},
]),
);
test("a codec that refuses a synthetic pre-v1 store's content leaves the file byte-identical", async () => {
const { storePath, bytes } = await legacyNoteStore({ title: 42 });
const cache = CultCache.builder() const cache = CultCache.builder()
.withDocumentType(noteDocument) .withDocumentType(legacyNoteType)
.withGenericStore(new SingleFileMessagePackBackingStore(storePath))
.build();
await assert.rejects(() => cache.pullAllBackingStores());
assert.deepEqual([...(await readFile(storePath))], [...bytes]);
assert.deepEqual((await readdir(dirname(storePath))).sort(), ["legacy.msgpack"]);
});
test("a synthetic pre-v1 store is read untouched and becomes v1 only when the consumer writes, keeping its other entries", async () => {
const { storePath, bytes } = await legacyNoteStore({ title: "Other" });
const cache = CultCache.builder()
.withDocumentType(legacyNoteType)
.withGenericStore(new SingleFileMessagePackBackingStore(storePath)) .withGenericStore(new SingleFileMessagePackBackingStore(storePath))
.build(); .build();
await cache.pullAllBackingStores(); await cache.pullAllBackingStores();
assert.deepEqual(cache.getRequired(noteDocument, "note:hello"), envelopePayload); assert.deepEqual(cache.getRequired(legacyNoteType, "note:hello"), { title: "Hello", body: "world" });
assert.deepEqual([...(await readFile(storePath))], [...bytes], "reading does not rewrite the file");
const rewritten = decode(await readFile(storePath)) as unknown[]; await cache.put(legacyNoteType, "note:new", { title: "New" });
assert.equal(rewritten[0], "cultcache.store.v1"); assert.equal((decode(await readFile(storePath)) as unknown[])[0], "cultcache.store.v1");
const records = rewritten[2] as unknown[][];
assert.equal(records.length, 1); const reopened = CultCache.builder()
assert.ok(records[0]?.[3] instanceof Uint8Array); .withDocumentType(legacyNoteType)
.withGenericStore(new SingleFileMessagePackBackingStore(storePath))
.build();
await reopened.pullAllBackingStores();
assert.deepEqual(reopened.getRequired(legacyNoteType, "note:hello"), { title: "Hello", body: "world" });
assert.deepEqual(reopened.getRequired(legacyNoteType, "note:other"), { title: "Other" });
assert.deepEqual(reopened.getRequired(legacyNoteType, "note:new"), { title: "New" });
}); });
test("CultCache rejects a second generic backing store", async () => { test("CultCache rejects a second generic backing store", async () => {
@ -2515,17 +2582,24 @@ test("SingleFileMessagePackBackingStore replaces a file exactly when it reads",
.filter((line) => line.length > 0 && !line.startsWith("#")) .filter((line) => line.length > 0 && !line.startsWith("#"))
.map((line) => line.split(/\s+/u)); .map((line) => line.split(/\s+/u));
// Every vector in the folder has a manifest row. // Every vector in the folder has a manifest row.
const listed = rows.map(([vector]) => vector!).filter((vector) => !vector.startsWith("..")).sort(); const listed = rows.map(([vector]) => vector!).filter((vector) => !vector.startsWith("..") && !vector.startsWith("@")).sort();
assert.deepEqual((await readdir(root)).filter((name) => name.endsWith(".bin")).sort(), listed); assert.deepEqual((await readdir(root)).filter((name) => name.endsWith(".bin")).sort(), listed);
const dir = await mkdtemp(join(tmpdir(), "cultcache-readability-")); const dir = await mkdtemp(join(tmpdir(), "cultcache-readability-"));
const seed = await copyVector(dir, join(c2aVectors, "v3-base.msgpack"), "seed.msgpack"); const seed = await copyVector(dir, join(c2aVectors, "v3-base.msgpack"), "seed.msgpack");
const envelopes = await new SingleFileMessagePackBackingStore(seed).pullAll(); const envelopes = await new SingleFileMessagePackBackingStore(seed).pullAll();
for (const [vector, , , typescript] of rows) { for (const [vector, , , typescript] of rows) {
const reads = typescript === "reads"; const reads = typescript === "reads";
const bytes = await readFile(join(root, vector)); // "@directory": the walker makes an empty directory at the store path; the runtime refuses to open or replace it and leaves it be.
const isDirectory = vector === "@directory";
const bytes = isDirectory ? Buffer.alloc(0) : await readFile(join(root, vector));
for (const operation of ["pullAll", "softPushAll", "pushAll", "push", "delete"] as const) { for (const operation of ["pullAll", "softPushAll", "pushAll", "push", "delete"] as const) {
const file = join(dir, `${vector.replace(/[^a-z0-9.-]/giu, "_")}.${operation}.msgpack`); const file = join(dir, `${vector.replace(/[^a-z0-9.-]/giu, "_")}.${operation}.msgpack`);
await writeFile(file, bytes); if (isDirectory) {
assert.equal(typescript, "refuses", "a directory is never a store");
await mkdir(file);
} else {
await writeFile(file, bytes);
}
const store = new SingleFileMessagePackBackingStore(file); const store = new SingleFileMessagePackBackingStore(file);
const run = () => const run = () =>
operation === "pullAll" operation === "pullAll"
@ -2560,6 +2634,10 @@ test("SingleFileMessagePackBackingStore replaces a file exactly when it reads",
} else { } else {
assert.ok(!keys.includes(envelopes[0]!.key), `${vector} ${operation}`); assert.ok(!keys.includes(envelopes[0]!.key), `${vector} ${operation}`);
} }
} else if (isDirectory) {
await assert.rejects(run, (error) => !(error instanceof StoreUnreadableError), `${vector} ${operation}: an I/O failure, not a typed refusal`);
assert.ok((await lstat(file)).isDirectory(), `${vector} ${operation} replaced the directory`);
assert.deepEqual(await readdir(file), [], `${vector} ${operation} wrote into the directory`);
} else { } else {
await assert.rejects( await assert.rejects(
run, run,
@ -2572,6 +2650,27 @@ test("SingleFileMessagePackBackingStore replaces a file exactly when it reads",
} }
}); });
// One key is held once: a push whose type does not own the stored schema id is refused, so the writer cannot emit a store its own
// reader refuses; one whose type owns it (its own id, or one it lists compatible, as C# WrittenOwns) restamps the record in place.
test("SingleFileMessagePackBackingStore push asks whether the written type owns the stored id, not what the types are called", async () => {
// One cache holds one record per key, so each envelope is made in a cache of its own.
const owner = CultCache.builder().withDocumentType(ownsId).build();
const lister = CultCache.builder().withDocumentType(listsId).build();
await owner.put(ownsId, "k", { name: "owner" });
await lister.put(listsId, "k", { name: "lister" });
const file = join(await mkdtemp(join(tmpdir(), "cultcache-push-key-")), "store.msgpack");
const store = new SingleFileMessagePackBackingStore(file);
const idsOf = async () => ((decode(await readFile(file)) as unknown[][])[2] as unknown[][]).map((record) => [record[0], record[1]]);
await store.push(lister.getRequiredEnvelope(listsId, "k"));
await assert.rejects(() => store.push(owner.getRequiredEnvelope(ownsId, "k")), SchemaConflictError);
assert.deepEqual(await idsOf(), [["k", "id.lister"]]);
await store.pushAll([lister.getRequiredEnvelope(listsId, "k")]);
await store.push(lister.getRequiredEnvelope(listsId, "k"));
assert.equal((await store.pullAll()).length, 1);
});
// Owner beats lister: a definition that owns an id and one that lists it as compatible register together in either order. A record // Owner beats lister: a definition that owns an id and one that lists it as compatible register together in either order. A record
// under the id is the owner's, and a lister's only when nothing owns it; two listers and no owner name no single definition. // under the id is the owner's, and a lister's only when nothing owns it; two listers and no owner name no single definition.
const ownsId = defineDocumentType({ type: "tests.owns", schema: z.object({ name: z.string() }), schemaId: "id.owned", schemaName: "tests.owns" }); const ownsId = defineDocumentType({ type: "tests.owns", schema: z.object({ name: z.string() }), schemaId: "id.owned", schemaName: "tests.owns" });

View file

@ -24,7 +24,6 @@ const itemDocument = defineDocumentType({
schemaVersion: "gamecult.first_call_item.v1", schemaVersion: "gamecult.first_call_item.v1",
schema: z.object({ name: z.string(), category: z.string(), value: z.number().int() }), schema: z.object({ name: z.string(), category: z.string(), value: z.number().int() }),
name: "name", name: "name",
indexes: { category: "category" },
}); });
const registry = defineDocumentRegistry(itemDocument); const registry = defineDocumentRegistry(itemDocument);

View file

@ -0,0 +1,377 @@
import assert from "node:assert/strict";
import { existsSync } from "node:fs";
import { copyFile, mkdtemp, readFile, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
import { test } from "node:test";
import { decode, encode } from "@msgpack/msgpack";
import { z } from "zod";
import { CultCache } from "../src/cult-cache";
import { defineDocumentType, documentPayloadBytes, documentPayloadFromBytes } from "../src/document";
import { SingleFileMessagePackBackingStore, readSingleFileStore } from "../src/single-file-messagepack-backing-store";
import { SchemaConflictError } from "../src/store-format";
import type { AnyCultCacheDocumentDefinition, CultCacheEnvelope } from "../src/types";
// variants:ruling:cache-refuses-retype, write-ownership-by-written-type, unheld-removal-refused-at-commit. The vectors in
// tests/vectors/key-identity are written by the C# reference (see its README for the rule and the row shapes); this walks
// every row against the TS cache. A leaf keeps no verdict of its own.
function findVectors(from: string): string {
for (let dir = from; ; dir = resolve(dir, "..")) {
if (existsSync(join(dir, "tests", "vectors", "key-identity"))) return join(dir, "tests", "vectors", "key-identity");
if (resolve(dir, "..") === dir) throw new Error("tests/vectors/key-identity not found above the test.");
}
}
const vectors = findVectors(__dirname);
// One string slot [0: Text]; the schema ids are the ones C# wrote (README). The type is the schema name, the name a record
// read back from a store carries as its type (the store's delete matches on it).
function kiType(type: string, id: string) {
const schemaName = `tests.ki_${type}`;
return defineDocumentType({
type: schemaName,
schemaId: `sha256:${id}`,
schemaName,
schemaVersion: `${schemaName}.v1`,
schema: z.object({ text: z.string() }),
formatter: {
encode: (value: { text: string }) => encode([value.text]),
decode: (payload: Uint8Array) => ({ text: (decode(payload) as [string])[0] }),
},
});
}
const types: Record<string, AnyCultCacheDocumentDefinition> = {
alpha: kiType("alpha", "b20703a98f49c253898404ab50ba2c630720f0b8a5e1a41614d5650be22799fe"),
beta: kiType("beta", "dee4b727cf2ee04e61ab635ff0c5f3dca7345a411dc5f1018e583ab16009cf94"),
gamma: kiType("gamma", "fbc26e351e00c9e7185e8de171c7486aba707c229c949f0aae6b889288af5f6d"),
};
type Op = [string, ...unknown[]];
type Row = [string, string[], Op[], [[string] | null, [string, string, string][], [string, string][] | null, string[], string[]]];
const compare = (left: string, right: string) => (left < right ? -1 : left > right ? 1 : 0);
const textOf = (envelope: CultCacheEnvelope) => (decode(documentPayloadBytes(envelope.payload)) as [string])[0];
const stored = (path: string) => new Map(readSingleFileStore(path).envelopes.map((e) => [e.key, e]));
const sameRecord = (left: CultCacheEnvelope | undefined, right: CultCacheEnvelope | undefined) =>
left !== undefined && right !== undefined && left.schemaId === right.schemaId && left.storedAt === right.storedAt &&
Buffer.compare(Buffer.from(documentPayloadBytes(left.payload)), Buffer.from(documentPayloadBytes(right.payload))) === 0;
test("vectors rows.msgpack: every row's refusal, held records, file, foreign list and kept bytes", async () => {
const rows = decode(await readFile(join(vectors, "rows.msgpack"))) as Row[];
assert.equal(rows.length, 12);
let walked = 0;
const failures: string[] = [];
for (const [label, registered, ops, expect] of rows) {
const dir = await mkdtemp(join(tmpdir(), "ki-"));
try {
const [refusal, held, file, foreign, kept] = expect;
let counter = 0;
let path: string | undefined;
let cache!: CultCache;
let before: Map<string, CultCacheEnvelope> | undefined;
const attached: string[] = [];
const build = (store: string | undefined) => {
let builder = CultCache.builder();
for (const name of registered) builder = builder.withDocumentType(types[name]!);
if (store !== undefined) builder = builder.withGenericStore(new SingleFileMessagePackBackingStore(store));
return builder.build();
};
const fresh = async (name: string) => {
const copy = join(dir, `${counter++}-${name}`);
await copyFile(join(vectors, name), copy);
return copy;
};
let refused: unknown;
let failed = false;
let beforeBytes: Buffer | undefined;
let beforeCache: CultCacheEnvelope[] = [];
let beforeAttached: Buffer[] = [];
for (const [index, op] of ops.entries()) {
const last = index === ops.length - 1;
if (last) {
beforeCache = cache?.snapshot() ?? [];
beforeAttached = await Promise.all(attached.map((copy) => readFile(copy)));
beforeBytes = path !== undefined ? await readFile(path) : undefined;
}
try {
switch (op[0]) {
case "open": {
const arg = op[1] as string | null;
path = arg === "memory" ? undefined : arg === null ? join(dir, "fresh.cc") : await fresh(arg);
if (path !== undefined && arg !== null) before = stored(path);
cache = build(path);
await cache.pullAllBackingStores();
break;
}
case "attach": {
const copy = await fresh(op[1] as string);
attached.push(copy);
await cache.addBackingStore(new SingleFileMessagePackBackingStore(copy), ...(op[2] as string[]).map((name) => types[name]!));
await cache.pullAllBackingStores();
break;
}
case "reopen":
cache = build(path);
await cache.pullAllBackingStores();
break;
case "put":
await cache.put(types[op[1] as string]!, op[2] as string, { text: op[3] as string } as never);
break;
case "delete":
await cache.delete(types[op[1] as string]!, op[2] as string);
break;
default:
assert.fail(`${label}: unknown op ${op[0]}`);
}
} catch (error) {
if (!last) throw error;
refused = error;
failed = true;
}
}
if (refusal === null) {
assert.equal(failed, false, `${label}: the last op lands (${String(refused)})`);
} else {
assert.equal(failed, true, `${label}: the last op is refused`);
const [key] = refusal;
if (label === "key-in-two-stores-refused") {
// The key is read out of the refusal as a delimited token: the message opens "Record <key> is held by" and the key
// occurs nowhere else in it (not inside a schema name).
const message = (refused as Error).message;
const named = /^Record (\S+) is held by/u.exec(message);
assert.equal(named?.[1], key, `${label}: the refusal opens with the key`);
assert.equal(message.split(/[^A-Za-z0-9_.]+/u).filter((token) => token === key).length, 1, `${label}: the key occurs only as a delimited token`);
assert.deepEqual(cache.snapshot(), beforeCache, `${label}: the earlier pulled state survives`);
assert.deepEqual(await Promise.all(attached.slice(0, beforeAttached.length).map((copy) => readFile(copy))), beforeAttached, `${label}: the attached files are byte-identical`);
} else {
assert.ok(refused instanceof SchemaConflictError, `${label}: a typed schema conflict, got ${String(refused)}`);
assert.equal((refused as SchemaConflictError).recordKey, key, `${label}: names the key`);
assert.deepEqual(cache.snapshot(), beforeCache, `${label}: the cache is unchanged`);
if (path !== undefined) assert.deepEqual(await readFile(path), beforeBytes, `${label}: the store is byte-identical`);
}
}
assert.deepEqual(
cache.snapshot()
.map((e) => [e.type.replace("tests.ki_", ""), e.key, textOf(e)] as [string, string, string])
.sort((l, r) => compare(l[0], r[0]) || compare(l[1], r[1])),
held,
`${label}: held`,
);
if (path === undefined) {
assert.equal(file, null, `${label}: no store`);
} else {
assert.deepEqual(
[...stored(path).values()]
.map((e) => [e.key, e.catalogEntry?.schemaName ?? e.type] as [string, string])
.sort((l, r) => compare(l[0], r[0])),
file,
`${label}: file`,
);
if (before !== undefined) {
const after = stored(path);
assert.deepEqual(foreign.filter((key) => sameRecord(before!.get(key), after.get(key))), kept, `${label}: kept`);
}
}
assert.deepEqual(cache.foreignRecords().map((record: { key: string }) => record.key), foreign, `${label}: foreign`);
walked++;
} catch (error) {
failures.push(`${label}: ${error instanceof Error ? error.message.split("\n")[0] : String(error)}`);
} finally {
await rm(dir, { recursive: true, force: true });
}
}
assert.deepEqual(failures, [], "every row holds its verdict");
assert.equal(walked, 12);
});
// Beyond the vector: what the walker's string-typed rows cannot say.
const CANARY = "CANARY-ki-77c1";
const plain = (type: string, schemaId: string, compatibleSchemaIds?: string[]) =>
defineDocumentType({ type, schemaId, schemaName: type, schemaVersion: `${type}.v1`, schema: z.object({ v: z.string() }), ...(compatibleSchemaIds ? { compatibleSchemaIds } : {}) });
test("a refusal names the key and schemas and never echoes the written value", async () => {
const alpha = plain("cn.alpha", "cn.alpha.v1");
const beta = plain("cn.beta", "cn.beta.v1");
const cache = CultCache.builder().withDocumentType(alpha).withDocumentType(beta).build();
await cache.put(alpha, "k", { v: "first" });
await assert.rejects(cache.put(beta, "k", { v: CANARY }), (error: Error) => {
assert.ok(error instanceof SchemaConflictError && error.recordKey === "k");
assert.ok(!error.message.includes(CANARY));
return true;
});
assert.deepEqual(cache.get(alpha, "k"), { v: "first" });
assert.equal(cache.get(beta, "k"), undefined);
await assert.rejects(cache.putEnvelope(beta, { key: "k", type: "cn.beta", payload: documentPayloadFromBytes(encode({ v: CANARY })), storedAt: "2026-01-01T00:00:00Z" }), SchemaConflictError);
assert.equal(cache.snapshot().length, 1);
});
test("a type that owns the held record's id replaces it: one record at the key, no second type held", async () => {
const old = plain("cn.old", "cn.id.v1");
const renamed = plain("cn.new", "cn.id.v2", ["cn.id.v2", "cn.id.v1"]);
const cache = CultCache.builder().withDocumentType(old).withDocumentType(renamed).build();
await cache.put(old, "k", { v: "a" });
await cache.put(renamed, "k", { v: "b" });
assert.deepEqual(cache.snapshot().map((e) => [e.type, e.key]), [["cn.new", "k"]]);
assert.equal(cache.get(old, "k"), undefined);
});
test("a store carrying an unregistered record loads, lists it sorted, keeps it through writes of other keys, and a later registration claims it", async () => {
const dir = await mkdtemp(join(tmpdir(), "ki-"));
try {
const path = join(dir, "s.cc");
await copyFile(join(vectors, "foreign-store.cc"), path);
const open = (...names: string[]) => {
let builder = CultCache.builder();
for (const name of names) builder = builder.withDocumentType(types[name]!);
return builder.withGenericStore(new SingleFileMessagePackBackingStore(path)).build();
};
const cache = open("alpha");
await cache.pullAllBackingStores();
const record = stored(path).get("k")!;
assert.deepEqual(cache.foreignRecords(), [
{ key: "k", schemaId: "sha256:fbc26e351e00c9e7185e8de171c7486aba707c229c949f0aae6b889288af5f6d", schemaName: "tests.ki_gamma", storedAt: record.storedAt },
]);
await cache.put(types.alpha!, "z", { text: "z" } as never);
await cache.delete(types.alpha!, "j");
assert.ok(sameRecord(stored(path).get("k"), record), "the foreign record's stored bytes survive");
assert.equal(await cache.delete(types.alpha!, "nobody"), false);
const claimed = open("alpha", "gamma");
await claimed.pullAllBackingStores();
assert.deepEqual(claimed.foreignRecords(), []);
assert.equal(claimed.snapshot().length, 2);
} finally {
await rm(dir, { recursive: true, force: true });
}
});
test("a foreign key one store carries and another holds under a registered type refuses the load and admits nothing", async () => {
const dir = await mkdtemp(join(tmpdir(), "ki-"));
try {
const foreign = join(dir, "foreign.cc");
const catalog = join(dir, "catalog.cc");
await copyFile(join(vectors, "catalog-store.cc"), catalog);
const writer = CultCache.builder().withDocumentType(types.gamma!).withGenericStore(new SingleFileMessagePackBackingStore(foreign)).build();
await writer.put(types.gamma!, "qx", { text: "g" } as never);
const cache = CultCache.builder().withDocumentType(types.alpha!).build();
await cache.addBackingStore(new SingleFileMessagePackBackingStore(catalog), types.alpha!);
await cache.pullAllBackingStores();
await cache.addBackingStore(new SingleFileMessagePackBackingStore(foreign));
await assert.rejects(cache.pullAllBackingStores(), /held by two stores/u);
assert.deepEqual(cache.snapshot().map((e) => e.key), ["qx"], "the earlier pulled state survives");
assert.deepEqual(cache.foreignRecords(), []);
} finally {
await rm(dir, { recursive: true, force: true });
}
});
// The cache judge answers for what the cache holds whatever the store is: a store that never judges (it only records
// what it is asked) is asked nothing at a foreign key, and foreign records list by key ordinal.
test("a foreign key is refused by the cache before any store is asked, and foreign records list sorted", async () => {
const calls: string[] = [];
const record = (key: string, schemaId: string): CultCacheEnvelope => ({
key,
type: "unclaimed",
schemaId,
storedAt: "2026-01-01T00:00:00Z",
payload: documentPayloadFromBytes(encode(["x"])),
});
const store = {
pullAll: async () => [record("m", "unclaimed.m.v1"), record("b", "unclaimed.b.v1")],
push: async (entry: CultCacheEnvelope) => void calls.push(`push ${entry.key}`),
delete: async (entry: CultCacheEnvelope) => void calls.push(`delete ${entry.key}`),
};
const cache = CultCache.builder().withDocumentType(types.alpha!).withGenericStore(store).build();
await cache.pullAllBackingStores();
assert.deepEqual(cache.foreignRecords().map((r) => [r.key, r.schemaId, r.schemaName]), [["b", "unclaimed.b.v1", "unclaimed"], ["m", "unclaimed.m.v1", "unclaimed"]]);
await assert.rejects(cache.put(types.alpha!, "m", { text: CANARY } as never), (error: Error) => {
assert.ok(error instanceof SchemaConflictError && error.recordKey === "m");
assert.ok(!error.message.includes(CANARY));
return true;
});
await assert.rejects(cache.delete(types.alpha!, "b"), (error: Error) => error instanceof SchemaConflictError && error.recordKey === "b");
assert.equal(await cache.delete(types.alpha!, "nobody"), false);
assert.deepEqual(calls, []);
});
// The store's delete matches the record the file stores by its schema id, not by the type string: a record read back is
// typed by its schema's name, which a type of another name does not carry.
test("a delete removes the record the file stores although the type differs from the schema name, and the key retypes across a reload", async () => {
const dir = await mkdtemp(join(tmpdir(), "ki-"));
try {
const path = join(dir, "s.cc");
const named = (type: string, schemaName: string, schemaId: string) =>
defineDocumentType({ type, schemaId, schemaName, schemaVersion: `${schemaName}.v1`, schema: z.object({ v: z.string() }) });
const first = named("app.first", "wire.first", "wire.first.v1");
const second = named("app.second", "wire.second", "wire.second.v1");
const open = () => CultCache.builder().withDocumentType(first).withDocumentType(second).withGenericStore(new SingleFileMessagePackBackingStore(path)).build();
const writer = open();
await writer.put(first, "k", { v: "a" });
const reader = open();
await reader.pullAllBackingStores();
assert.equal(await reader.delete(first, "k"), true);
assert.deepEqual([...stored(path).keys()], [], "the record is gone from the file");
await reader.put(second, "k", { v: "b" });
assert.deepEqual([...stored(path).values()].map((e) => [e.key, e.catalogEntry?.schemaName]), [["k", "wire.second"]]);
const again = open();
await again.pullAllBackingStores();
assert.deepEqual(again.snapshot().map((e) => [e.type, e.key]), [["app.second", "k"]]);
} finally {
await rm(dir, { recursive: true, force: true });
}
});
test("an owned rename whose two types home in different stores is refused, and neither store changes", async () => {
const dir = await mkdtemp(join(tmpdir(), "ki-"));
try {
const oldPath = join(dir, "old.cc");
const newPath = join(dir, "new.cc");
const old = plain("rn.old", "rn.id.v1");
const renamed = plain("rn.new", "rn.id.v2", ["rn.id.v2", "rn.id.v1"]);
const cache = CultCache.builder().withDocumentType(old).withDocumentType(renamed).build();
await cache.addBackingStore(new SingleFileMessagePackBackingStore(oldPath), old);
await cache.addBackingStore(new SingleFileMessagePackBackingStore(newPath), renamed);
await cache.put(old, "k", { v: "a" });
const before = await readFile(oldPath);
await assert.rejects(cache.put(renamed, "k", { v: CANARY }), (error: Error) => {
assert.ok(error instanceof SchemaConflictError && error.recordKey === "k");
assert.ok(!error.message.includes(CANARY));
return true;
});
assert.deepEqual(await readFile(oldPath), before);
assert.equal(existsSync(newPath), false, "the new home was never written");
assert.deepEqual(cache.snapshot().map((e) => [e.type, e.key]), [["rn.old", "k"]]);
// Same home: the rename replaces.
const together = CultCache.builder().withDocumentType(old).withDocumentType(renamed).withGenericStore(new SingleFileMessagePackBackingStore(join(dir, "one.cc"))).build();
await together.put(old, "k", { v: "a" });
await together.put(renamed, "k", { v: "b" });
assert.deepEqual(together.snapshot().map((e) => [e.type, e.key]), [["rn.new", "k"]]);
} finally {
await rm(dir, { recursive: true, force: true });
}
});
test("two stores each carrying an unregistered record at one key refuse the load and admit nothing", async () => {
const dir = await mkdtemp(join(tmpdir(), "ki-"));
try {
const paths = [join(dir, "a.cc"), join(dir, "b.cc")];
for (const path of paths) {
const writer = CultCache.builder().withDocumentType(types.gamma!).withGenericStore(new SingleFileMessagePackBackingStore(path)).build();
await writer.put(types.gamma!, "k", { text: "g" } as never);
}
// Neither store's records are claimed: gamma is not registered here.
const cache = CultCache.builder().withDocumentType(types.alpha!).withDocumentType(types.beta!).build();
await cache.addBackingStore(new SingleFileMessagePackBackingStore(paths[0]!), types.alpha!);
await cache.addBackingStore(new SingleFileMessagePackBackingStore(paths[1]!), types.beta!);
await assert.rejects(cache.pullAllBackingStores(), /^Error: Record k is held by two stores/u);
assert.deepEqual(cache.foreignRecords(), []);
assert.deepEqual(cache.snapshot(), []);
} finally {
await rm(dir, { recursive: true, force: true });
}
});

View file

@ -12,6 +12,7 @@ import { decode, encode } from "@msgpack/msgpack";
import { documentPayloadBytes, documentPayloadFromBytes } from "../src/document"; import { documentPayloadBytes, documentPayloadFromBytes } from "../src/document";
import { SingleFileMessagePackBackingStore } from "../src/single-file-messagepack-backing-store"; import { SingleFileMessagePackBackingStore } from "../src/single-file-messagepack-backing-store";
import { SchemaConflictError, StoreUnreadableError } from "../src/store-format";
import type { CultCacheEnvelope } from "../src/types"; import type { CultCacheEnvelope } from "../src/types";
import { import {
@ -471,44 +472,70 @@ test("the store lock is released when the operation throws", async () => {
} }
}); });
// SYNTHETIC pre-v1 vector: no real file of this shape is known to exist. A top-level array of {key,type,payload,storedAt}
// whose payload is not binary (an object, a Node Buffer JSON, an int array) is what the removed heal-on-open rewrote.
const legacyObjectPayload = (): Uint8Array => const legacyObjectPayload = (): Uint8Array =>
encode([{ key: "note:hello", type: "note", payload: { title: "Hello" }, storedAt: "2026-10-20T00:00:00Z" }]); encode([{ key: "note:hello", type: "note", payload: { title: "Hello" }, storedAt: "2026-10-20T00:00:00Z" }]);
test("an open that heals a legacy store waits for the store lock and heals from its own read", async () => { test("opening a synthetic pre-v1 store with object payloads returns them normalized and leaves the file untouched, even while another holder has the lock", async () => {
const directory = await scratch(); const directory = await scratch();
try { try {
const path = join(directory, "legacy.cc"); const path = join(directory, "legacy.cc");
const legacy = legacyObjectPayload(); const legacy = legacyObjectPayload();
await writeFile(path, legacy); await writeFile(path, legacy);
const store = new SingleFileMessagePackBackingStore(path); const store = new SingleFileMessagePackBackingStore(path);
let healed: Promise<unknown> | undefined;
let settled = false;
await withStoreWriteLock(path, async () => { await withStoreWriteLock(path, async () => {
healed = store.pullAll().then(() => { const [lockHeld] = await Promise.all([readdir(directory)]);
settled = true; const envelopes = await within(store.pullAll(), 5000, "an open does not wait for the write lock");
}); assert.deepEqual(envelopes.map((entry) => entry.key), ["note:hello"]);
await sleepFor(300); assert.deepEqual(decode(documentPayloadBytes(envelopes[0].payload)), { title: "Hello" });
assert.equal(settled, false, "the heal must wait for the lock"); assert.deepEqual([...(await readFile(path))], [...legacy]);
assert.deepEqual([...(await readFile(path))], [...legacy], "the file keeps its legacy bytes while the lock is held elsewhere"); assert.deepEqual((await readdir(directory)).sort(), lockHeld.sort(), "an open adds no lock or temp file");
}); });
await within(healed!, 5000, "the heal"); assert.deepEqual([...(await readFile(path))], [...legacy]);
assert.equal((decode(await readFile(path)) as unknown[])[0], "cultcache.store.v1"); assert.deepEqual((await readdir(directory)).filter((name) => name !== "legacy.cc" && name !== "legacy.cc.lock"), []);
} finally { } finally {
await rm(directory, { recursive: true, force: true }); await rm(directory, { recursive: true, force: true });
} }
}); });
test("push and delete over a legacy store that needs healing finish instead of waiting on their own lock", async () => { test("opening a synthetic pre-v1 store with Buffer-JSON and int-array payloads normalizes them in memory and leaves the file untouched", async () => {
const directory = await scratch();
try {
const path = join(directory, "legacy.cc");
const inner = encode({ title: "Hello" });
const legacy = encode([
{ key: "a", type: "note", payload: { type: "Buffer", data: [...inner] }, storedAt: "2026-10-20T00:00:00Z" },
{ key: "b", type: "note", payload: [...inner], storedAt: "2026-10-20T00:00:00Z" },
]);
await writeFile(path, legacy);
const envelopes = await new SingleFileMessagePackBackingStore(path).pullAll();
assert.deepEqual(envelopes.map((entry) => entry.key), ["a", "b"]);
for (const envelope of envelopes) {
assert.deepEqual(decode(documentPayloadBytes(envelope.payload)), { title: "Hello" });
}
assert.deepEqual([...(await readFile(path))], [...legacy]);
assert.deepEqual(await readdir(directory), ["legacy.cc"]);
} finally {
await rm(directory, { recursive: true, force: true });
}
});
test("push and delete over a synthetic pre-v1 store write v1 and keep the other legacy entries, normalized payloads included", async () => {
const directory = await scratch(); const directory = await scratch();
try { try {
const path = join(directory, "legacy.cc"); const path = join(directory, "legacy.cc");
await writeFile(path, legacyObjectPayload()); await writeFile(path, legacyObjectPayload());
const store = new SingleFileMessagePackBackingStore(path); const store = new SingleFileMessagePackBackingStore(path);
await within(store.push(entryOf("a")), 5000, "push"); await within(store.push(entryOf("a")), 5000, "push");
assert.equal((decode(await readFile(path)) as unknown[])[0], "cultcache.store.v1");
assert.deepEqual((await store.pullAll()).map((entry) => entry.key).sort(), ["a", "note:hello"]);
await writeFile(path, legacyObjectPayload()); await writeFile(path, legacyObjectPayload());
await within(store.delete(entryOf("a")), 5000, "delete"); await within(store.delete(entryOf("a")), 5000, "delete");
const keys = (await store.pullAll()).map((entry) => entry.key); assert.equal((decode(await readFile(path)) as unknown[])[0], "cultcache.store.v1");
assert.deepEqual(keys, ["note:hello"]); const kept = await store.pullAll();
assert.deepEqual(kept.map((entry) => entry.key), ["note:hello"]);
assert.deepEqual(decode(documentPayloadBytes(kept[0].payload)), { title: "Hello" });
} finally { } finally {
await rm(directory, { recursive: true, force: true }); await rm(directory, { recursive: true, force: true });
} }
@ -545,15 +572,84 @@ const snapshotOf = async (store: SingleFileMessagePackBackingStore): Promise<str
.map((entry) => `${entry.type}/${entry.key}:${[...documentPayloadBytes(entry.payload)].at(-1)}`) .map((entry) => `${entry.type}/${entry.key}:${[...documentPayloadBytes(entry.payload)].at(-1)}`)
.sort(); .sort();
test("push replaces the record at the same type and key and keeps every other", async () => { test("push replaces the record under the same key and type and keeps every other; retyping is delete then push", async () => {
const directory = await scratch(); const directory = await scratch();
try { try {
const store = new SingleFileMessagePackBackingStore(join(directory, "store.cc")); const store = new SingleFileMessagePackBackingStore(join(directory, "store.cc"));
await store.pushAll([entryOf("a", "t", 1), entryOf("b", "t", 2), entryOf("a", "u", 3)]); await store.pushAll([entryOf("a", "t", 1), entryOf("b", "t", 2), entryOf("c", "u", 3)]);
await store.push(entryOf("a", "t", 9)); await store.push(entryOf("a", "t", 9));
assert.deepEqual(await snapshotOf(store), ["t/a:9", "t/b:2", "u/a:3"]); assert.deepEqual(await snapshotOf(store), ["t/a:9", "t/b:2", "u/c:3"]);
await store.push(entryOf("c", "t", 4)); await store.push(entryOf("d", "t", 4));
assert.deepEqual(await snapshotOf(store), ["t/a:9", "t/b:2", "t/c:4", "u/a:3"]); assert.deepEqual(await snapshotOf(store), ["t/a:9", "t/b:2", "t/d:4", "u/c:3"]);
await store.delete(entryOf("a", "t"));
await store.push(entryOf("a", "u", 5));
assert.deepEqual(await snapshotOf(store), ["t/b:2", "t/d:4", "u/a:5", "u/c:3"]);
} finally {
await rm(directory, { recursive: true, force: true });
}
});
test("a push that would land a second type under an existing key is refused and the file is byte for byte unchanged", async () => {
const directory = await scratch();
try {
const path = join(directory, "store.cc");
const store = new SingleFileMessagePackBackingStore(path);
await store.pushAll([entryOf("k", "t", 1), entryOf("m", "t", 2)]);
const before = [...(await readFile(path))];
await assert.rejects(
() => store.push(entryOf("k", "u", 9)),
(error: unknown) => error instanceof SchemaConflictError && error.recordKey === "k" && /holds record "k" as type "t"/u.test(error.message),
);
assert.deepEqual([...(await readFile(path))], before);
assert.deepEqual(await snapshotOf(new SingleFileMessagePackBackingStore(path)), ["t/k:1", "t/m:2"]);
} finally {
await rm(directory, { recursive: true, force: true });
}
});
test("a pushAll batch that carries one key under two types is refused and writes nothing", async () => {
const directory = await scratch();
try {
const path = join(directory, "store.cc");
const store = new SingleFileMessagePackBackingStore(path);
await assert.rejects(() => store.pushAll([entryOf("k", "t", 1), entryOf("k", "u", 2)]), SchemaConflictError);
assert.equal((await readdir(directory)).includes("store.cc"), false);
await store.pushAll([entryOf("a", "t", 1)]);
const before = [...(await readFile(path))];
await assert.rejects(() => store.pushAll([entryOf("k", "t", 1), entryOf("k", "u", 2)]), SchemaConflictError);
assert.deepEqual([...(await readFile(path))], before);
} finally {
await rm(directory, { recursive: true, force: true });
}
});
test("a pushAll batch that carries one key twice under one type keeps the last and the file reads back with one record", async () => {
const directory = await scratch();
try {
const path = join(directory, "store.cc");
await new SingleFileMessagePackBackingStore(path).pushAll([entryOf("k", "t", 1), entryOf("a", "t", 3), entryOf("k", "t", 2)]);
assert.deepEqual(await snapshotOf(new SingleFileMessagePackBackingStore(path)), ["t/a:3", "t/k:2"]);
} finally {
await rm(directory, { recursive: true, force: true });
}
});
test("a legacy envelope array that holds a key under two types is refused on open, as a v1 store is", async () => {
const directory = await scratch();
try {
const path = join(directory, "legacy.cc");
const bytes = encode([
{ key: "a", type: "t", payload: { n: 1 }, storedAt: "2026-10-20T00:00:00Z" },
{ key: "a", type: "u", payload: { n: 2 }, storedAt: "2026-10-20T00:00:00Z" },
]);
await writeFile(path, bytes);
const store = new SingleFileMessagePackBackingStore(path);
await assert.rejects(
() => store.pullAll(),
(error: unknown) => error instanceof StoreUnreadableError && /"a" appears more than once/u.test(error.message),
);
await assert.rejects(() => store.push(entryOf("b")), StoreUnreadableError);
assert.deepEqual([...(await readFile(path))], [...bytes]);
} finally { } finally {
await rm(directory, { recursive: true, force: true }); await rm(directory, { recursive: true, force: true });
} }
@ -563,11 +659,12 @@ test("delete removes only the record at that type and key", async () => {
const directory = await scratch(); const directory = await scratch();
try { try {
const store = new SingleFileMessagePackBackingStore(join(directory, "store.cc")); const store = new SingleFileMessagePackBackingStore(join(directory, "store.cc"));
await store.pushAll([entryOf("a", "t", 1), entryOf("b", "t", 2), entryOf("a", "u", 3)]); await store.pushAll([entryOf("a", "t", 1), entryOf("b", "t", 2), entryOf("c", "u", 3)]);
await store.delete(entryOf("a", "t")); await store.delete(entryOf("a", "t"));
assert.deepEqual(await snapshotOf(store), ["t/b:2", "u/a:3"]); assert.deepEqual(await snapshotOf(store), ["t/b:2", "u/c:3"]);
await store.delete(entryOf("zzz", "t")); await store.delete(entryOf("zzz", "t"));
assert.deepEqual(await snapshotOf(store), ["t/b:2", "u/a:3"]); await store.delete(entryOf("c", "t"));
assert.deepEqual(await snapshotOf(store), ["t/b:2", "u/c:3"]);
} finally { } finally {
await rm(directory, { recursive: true, force: true }); await rm(directory, { recursive: true, force: true });
} }
@ -697,7 +794,7 @@ test("a store file of an unrecognized shape is refused, and a store naming a mis
} }
}); });
test("opening a store that needs no healing writes nothing: a missing store leaves no lock file, a legacy store with byte payloads keeps its bytes", async () => { test("opening writes nothing: a missing store leaves no lock file, a legacy store with byte payloads keeps its bytes", async () => {
const directory = await scratch(); const directory = await scratch();
try { try {
const absent = join(directory, "nested", "absent.cc"); const absent = join(directory, "nested", "absent.cc");

View file

@ -0,0 +1,583 @@
import assert from "node:assert/strict";
import { existsSync } from "node:fs";
import { copyFile, mkdtemp, readFile, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
import { test } from "node:test";
import { decode } from "@msgpack/msgpack";
import { z } from "zod";
import { codecs } from "../src";
import { CultCache } from "../src/cult-cache";
import { defineDocumentType, resolveDocumentMembers } from "../src/document";
import { SingleFileMessagePackBackingStore } from "../src/single-file-messagepack-backing-store";
import type { AnyCultCacheDocumentDefinition } from "../src/types";
// variants:ruling:unique-indexes. `indexes` are unique within one type, `roles` are selection aliases. The vectors in
// tests/vectors/unique-index are written by the C# reference; the rest of the behaviour is pinned here.
function findVectors(from: string): string {
for (let dir = from; ; dir = resolve(dir, "..")) {
if (existsSync(join(dir, "tests", "vectors", "unique-index"))) return join(dir, "tests", "vectors", "unique-index");
if (resolve(dir, "..") === dir) throw new Error("tests/vectors/unique-index not found above the test.");
}
}
const vectors = findVectors(__dirname);
const CANARY = "CANARY-uidx-5e1b";
const skuSchema = z.object({ sku: z.any().optional(), group: z.string().optional() });
function item(type: string, where: "indexes" | "roles" | "none" = "indexes", schemaName = `tests.${type}`) {
return defineDocumentType({
type,
schemaId: `${schemaName}.v1`,
schemaName,
schemaVersion: `${schemaName}.v1`,
schema: skuSchema,
...(where === "none" ? {} : { [where]: { sku: "sku" } }),
});
}
// A definition whose payload is never read back, so a value of any scalar kind (a bigint included) can be held and indexed.
function scalar(type: string) {
const held: unknown[] = [];
return defineDocumentType({
type,
schemaId: `tests.${type}.v1`,
schemaName: `tests.${type}`,
schemaVersion: `tests.${type}.v1`,
schema: skuSchema,
formatter: {
encode: (value: unknown) => new Uint8Array([held.push(value) - 1]),
decode: (payload: Uint8Array) => held[payload[0]] as never,
},
indexes: { sku: "sku" },
});
}
function memory(...definitions: AnyCultCacheDocumentDefinition[]) {
let builder = CultCache.builder();
for (const definition of definitions) builder = builder.withDocumentType(definition);
return builder.build();
}
function refusal(keyA: string, keyB: string, schemaName: string, alias: string) {
return `Records ${keyA} and ${keyB} (${schemaName}) both hold unique index '${alias}'; an index value names one record.`;
}
test("a put of a held value at another key is refused naming both keys and the alias, and nothing moves", async () => {
const dir = await mkdtemp(join(tmpdir(), "uidx-"));
try {
const file = join(dir, "items.cc");
const document = item("item");
const cache = CultCache.builder().withDocumentType(document).withGenericStore(new SingleFileMessagePackBackingStore(file)).build();
await cache.pullAllBackingStores();
await cache.put(document, "zeta", { sku: CANARY });
const before = await readFile(file);
const snapshot = cache.snapshot();
await assert.rejects(cache.put(document, "alpha", { sku: CANARY }), (error: Error) => {
assert.equal(error.message, refusal("alpha", "zeta", "tests.item", "sku"));
assert.ok(!error.message.includes(CANARY), "the value is never echoed");
return true;
});
assert.deepEqual(await readFile(file), before, "the store is byte-identical");
assert.deepEqual(cache.snapshot(), snapshot);
assert.equal(cache.get(document, "alpha"), undefined);
assert.equal(cache.getKeyByIndex(document, "sku", CANARY), "zeta");
} finally {
await rm(dir, { recursive: true, force: true });
}
});
test("rewriting a record's own key with its own value lands", async () => {
const document = item("item");
const cache = memory(document);
await cache.put(document, "a", { sku: "S", group: "one" });
await cache.put(document, "a", { sku: "S", group: "two" });
assert.equal(cache.getKeyByIndex(document, "sku", "S"), "a");
assert.equal(cache.get(document, "a")?.group, "two");
});
test("a removed or changed holder releases its value", async () => {
const document = item("item");
const cache = memory(document);
await cache.put(document, "b", { sku: "S" });
await assert.rejects(cache.put(document, "a", { sku: "S" }), /Records a and b/u);
assert.equal(cache.getKeyByIndex(document, "sku", "S"), "b", "the refused write did not take the value");
await cache.delete(document, "b");
assert.equal(cache.getKeyByIndex(document, "sku", "S"), undefined);
await cache.put(document, "a", { sku: "S" });
assert.equal(cache.getKeyByIndex(document, "sku", "S"), "a");
await cache.put(document, "a", { sku: "T" });
await cache.put(document, "b", { sku: "S" });
assert.equal(cache.getKeyByIndex(document, "sku", "S"), "b");
assert.equal(cache.getKeyByIndex(document, "sku", "T"), "a");
});
// The C# White_Space set, and nothing else, is blank. Every UTF-16 code unit is judged, so a rule written for a few
// characters (or for JavaScript's \s, which counts U+FEFF and omits U+0085) cannot pass.
const CSHARP_WHITE_SPACE = (code: number) =>
(code >= 0x09 && code <= 0x0d) || code === 0x20 || code === 0x85 || code === 0xa0 || code === 0x1680 ||
(code >= 0x2000 && code <= 0x200a) || code === 0x2028 || code === 0x2029 || code === 0x202f || code === 0x205f || code === 0x3000;
test("blank is exactly the C# White_Space set: every code unit, alone, repeated and padded", async () => {
const document = item("item");
const cache = memory(document);
const disagreements: string[] = [];
for (let code = 0; code <= 0xffff; code++) {
if (code >= 0xd800 && code <= 0xdfff) continue; // lone surrogates do not survive MessagePack text
const character = String.fromCharCode(code);
await cache.put(document, "x", { sku: character });
let blank = true;
try {
await cache.put(document, "y", { sku: character });
} catch {
blank = false;
}
if (blank !== CSHARP_WHITE_SPACE(code)) disagreements.push(code.toString(16));
await cache.delete(document, "x");
await cache.delete(document, "y");
}
assert.deepEqual(disagreements, []);
// Not trimmed: a padded value is the value, distinct from its trimmed form.
await cache.put(document, "x", { sku: " a " });
await cache.put(document, "y", { sku: "a" });
assert.equal(cache.getKeyByIndex(document, "sku", " a "), "x");
assert.equal(cache.getKeyByIndex(document, "sku", "a"), "y");
await assert.rejects(cache.put(document, "z", { sku: " a " }), /Records x and z/u);
});
test("blank values are never indexed and never conflict: empty, spaces, absent", async () => {
const document = item("item");
const cache = memory(document);
for (const [key, sku] of [["a", ""], ["b", " "], ["c", "\u0085"], ["d", undefined], ["e", " "]] as const) {
await cache.put(document, key, { sku });
}
assert.equal(cache.getKeyByIndex(document, "sku", ""), undefined);
assert.equal(cache.getKeyByIndex(document, "sku", " "), undefined);
// U+FEFF and U+001F are values.
await cache.put(document, "f", { sku: "" });
await assert.rejects(cache.put(document, "g", { sku: "" }), /Records f and g/u);
await cache.put(document, "h", { sku: "\u001F" });
await assert.rejects(cache.put(document, "i", { sku: "\u001F" }), /Records h and i/u);
});
test("two types may share an alias and a value", async () => {
const first = item("first");
const second = item("second");
const cache = memory(first, second);
await cache.put(first, "a", { sku: "S" });
await cache.put(second, "b", { sku: "S" });
assert.equal(cache.getKeyByIndex(first, "sku", "S"), "a");
assert.equal(cache.getKeyByIndex(second, "sku", "S"), "b");
});
test("roles may share values and are not looked up", async () => {
const document = item("item", "roles");
const cache = memory(document);
await cache.put(document, "a", { sku: "S" });
await cache.put(document, "b", { sku: "S" });
await cache.put(document, "c", { sku: "S" });
assert.equal(cache.getKeyByIndex(document, "sku", "S"), undefined);
assert.equal(cache.getByIndex(document, "sku", "S"), undefined);
});
test("an alias declared in indexes and roles is refused at definition; each map is checked like indexes", () => {
const base = { type: "bad", schema: skuSchema, schemaId: "tests.bad.v1", schemaName: "tests.bad", schemaVersion: "tests.bad.v1" };
assert.throws(
() => defineDocumentType({ ...base, indexes: { sku: "sku" }, roles: { sku: "sku" } }),
{ message: 'CultCache document type "bad" declares "sku" as both an index and a role; an index already names its role.' },
);
assert.throws(() => defineDocumentType({ ...base, roles: { " ": "sku" } }), /declares a role with an empty name/u);
assert.throws(
() => defineDocumentType({ ...base, roles: [{ name: "r", accessor: "sku" }, { name: "r", accessor: "group" }] }),
/declares duplicate role "r"/u,
);
assert.doesNotThrow(() => defineDocumentType({ ...base, indexes: { sku: "sku" }, roles: { group: "group" } }));
});
test("a codec definition's catalog entry is identical with an alias under indexes or roles", async () => {
const codec = codecs.slots<{ sku: string; group: string }>({ 0: ["sku", codecs.string], 1: ["group", codecs.string] }, "T.Item");
const make = (where: "indexes" | "roles") =>
defineDocumentType({
type: "codec_item",
schemaId: "tests.codec_item.v1",
schemaName: "tests.codec_item",
schemaVersion: "tests.codec_item.v1",
schema: z.object({ sku: z.string(), group: z.string() }),
codec,
[where]: { sku: "sku" },
});
const catalog = async (definition: AnyCultCacheDocumentDefinition) => {
const cache = memory(definition);
await cache.put(definition, "a", { sku: "S", group: "g" } as never);
return cache.snapshot()[0].catalogEntry!;
};
const members = (definition: AnyCultCacheDocumentDefinition) => resolveDocumentMembers(definition);
assert.deepEqual(members(make("indexes")), members(make("roles")));
assert.equal(members(make("roles")).find((member) => member.memberName === "Sku")?.indexAlias, "sku");
const [unique, role] = [await catalog(make("indexes")), await catalog(make("roles"))];
assert.equal(unique.canonicalSchemaJson, role.canonicalSchemaJson);
assert.equal(unique.contentHash, role.contentHash);
assert.equal(unique.schemaId, role.schemaId);
});
test("vectors values.msgpack: every row's canonical text, blank rows no value", async () => {
const rows = decode(await readFile(join(vectors, "values.msgpack")), { useBigInt64: true }) as [string, string, unknown, string | null][];
assert.ok(rows.length >= 20);
for (const [label, kind, value, text] of rows) {
const document = scalar(`value_${label.replaceAll(/[^a-z0-9]/gu, "_")}`);
const cache = memory(document);
void kind;
const normalized = value;
await cache.put(document, "one", { sku: normalized });
if (text === null) {
await cache.put(document, "two", { sku: normalized });
assert.equal(cache.get(document, "two") !== undefined, true, `${label}: a blank value commits twice`);
for (const probe of ["", " ", "NaN", "0", "Infinity", "-Infinity", "null", "undefined"]) {
assert.equal(cache.getKeyByIndex(document, "sku", probe), undefined, `${label}: found by '${probe}'`);
}
} else {
assert.equal(cache.getKeyByIndex(document, "sku", text), "one", `${label}: found by its text`);
await assert.rejects(cache.put(document, "two", { sku: normalized }), /Records one and two/u, label);
}
}
});
// An independent oracle for the double rule: the text must denote exactly the double, so N / 10^d == mantissa * 2^exponent
// in exact integer arithmetic, with no trailing zero and no bare point. A rule written to a few spellings cannot pass.
function exactlyDenotes(text: string, value: number): boolean {
const match = /^(-?)(\d+)(?:\.(\d*[1-9]))?$/u.exec(text);
if (!match) return false;
const decimals = match[3]?.length ?? 0;
const numerator = BigInt(match[2] + (match[3] ?? ""));
const view = new DataView(new ArrayBuffer(8));
view.setFloat64(0, Math.abs(value));
const bits = view.getBigUint64(0);
const biased = Number((bits >> 52n) & 0x7ffn);
const mantissa = biased === 0 ? bits & 0xfffffffffffffn : (bits & 0xfffffffffffffn) | (1n << 52n);
const exponent = (biased === 0 ? 1 : biased) - 1075;
const lhs = numerator * (exponent >= 0 ? 1n : 2n ** BigInt(-exponent));
const rhs = mantissa * (exponent >= 0 ? 2n ** BigInt(exponent) : 1n) * 10n ** BigInt(decimals);
return (match[1] === "-") === (value < 0) && lhs === rhs;
}
test("a number's index text is its exact value: integers as digits, doubles as the exact expansion, NaN and infinities none", async () => {
const document = scalar("number");
const cache = memory(document);
const probes = [0.1, -0.1, 2.5e-7, 1.5, -123.456, 5e-324, 2.2250738585072014e-308, 1e-10, 123456789.123456789, Number.MIN_VALUE * 3,
...Array.from({ length: 200 }, (_, i) => Math.sin(i + 1) * 10 ** ((i % 40) - 20)), 1e21 + 2 ** 20 + 0.5 * 0];
// Round trip through lookups: the exact expansion of each probe is found, and it denotes the probe exactly.
for (const value of probes) {
if (Number.isInteger(value)) continue;
await cache.put(document, "k", { sku: value });
const candidates = candidateTexts(value);
const found = candidates.filter((candidate) => cache.getKeyByIndex(document, "sku", candidate) === "k");
assert.equal(found.length, 1, `${value}: exactly one spelling is its index text`);
assert.ok(exactlyDenotes(found[0], value), `${value}: ${found[0]} denotes it exactly`);
}
await cache.put(document, "k", { sku: 1e21 });
assert.equal(cache.getKeyByIndex(document, "sku", "1000000000000000000000"), "k");
await cache.put(document, "k", { sku: -0 });
assert.equal(cache.getKeyByIndex(document, "sku", "0"), "k");
await cache.put(document, "k", { sku: 12n });
assert.equal(cache.getKeyByIndex(document, "sku", "12"), "k");
await cache.put(document, "k", { sku: true });
assert.equal(cache.getKeyByIndex(document, "sku", "True"), "k");
await cache.put(document, "k", { sku: false });
assert.equal(cache.getKeyByIndex(document, "sku", "False"), "k");
for (const value of [Number.NaN, Number.POSITIVE_INFINITY, Number.NEGATIVE_INFINITY]) {
await cache.put(document, "p", { sku: value });
await cache.put(document, "q", { sku: value });
}
});
// Spellings a lookup could be probed with: the exact decimal expansion computed by an arbitrary-precision route.
function candidateTexts(value: number): string[] {
const view = new DataView(new ArrayBuffer(8));
view.setFloat64(0, Math.abs(value));
const bits = view.getBigUint64(0);
const biased = Number((bits >> 52n) & 0x7ffn);
const mantissa = biased === 0 ? bits & 0xfffffffffffffn : (bits & 0xfffffffffffffn) | (1n << 52n);
const k = 1075 - (biased === 0 ? 1 : biased);
const scaled = (mantissa * 5n ** BigInt(k)).toString().padStart(k + 1, "0");
const point = scaled.length - k;
const text = `${scaled.slice(0, point)}.${scaled.slice(point)}`.replace(/0+$/u, "").replace(/\.$/u, "");
return [(value < 0 ? "-" : "") + text, String(value), value.toString()].filter((spelling, i, all) => all.indexOf(spelling) === i);
}
function twoDefinitions(schemaId: string, where: "indexes" | "roles") {
return defineDocumentType({
type: "unique_item",
schemaId,
schemaName: "tests.unique_item",
schemaVersion: "tests.unique_item.v1",
schema: z.object({ sku: z.string(), category: z.string() }),
codec: codecs.slots<{ sku: string; category: string }>({ 0: ["sku", codecs.string], 1: ["category", codecs.string] }, "UqVecItem"),
[where]: { Sku: "sku" },
});
}
test("vectors duplicate-store.cc and blank-store.cc hold their verdicts", async () => {
const outcomes = decode(await readFile(join(vectors, "outcomes.msgpack"))) as [string, string | null, string, string | null, string | null][];
assert.equal(outcomes.length, 3);
const dir = await mkdtemp(join(tmpdir(), "uidx-vec-"));
try {
for (const [name, alias, outcome, key1, key2] of outcomes) {
const file = join(dir, `${outcomes.indexOf(outcomes.find((row) => row[0] === name && row[1] === alias)!)}-${name}`);
await copyFile(join(vectors, name), file);
const before = await readFile(file);
const [envelope] = await new SingleFileMessagePackBackingStore(file).pullAll();
const document = twoDefinitions(envelope.schemaId!, alias === null ? "roles" : "indexes");
const cache = CultCache.builder().withDocumentType(document).withGenericStore(new SingleFileMessagePackBackingStore(file)).build();
const stateBefore = cache.snapshot();
if (outcome === "open") {
await cache.pullAllBackingStores();
assert.equal(cache.snapshot().length, name === "blank-store.cc" ? 2 : 4);
if (alias === null) assert.equal(cache.getKeyByIndex(document, "Sku", "S-1"), undefined);
} else {
await assert.rejects(cache.pullAllBackingStores(), (error: Error) => {
assert.equal(error.message, refusal(key1!, key2!, "tests.unique_item", alias!));
assert.ok(!error.message.includes("S-1"));
return true;
});
assert.deepEqual(cache.snapshot(), stateBefore, "a refused load admits nothing");
}
assert.deepEqual(await readFile(file), before, `${name} is untouched`);
}
} finally {
await rm(dir, { recursive: true, force: true });
}
});
// refusals.msgpack: rows [label, ops, refusal] written by the C# reference. TypeScript has no batch commit
// (variants:idea:unique-index-runtimes.ts-batch-commit), so it performs every row whose commits each hold one record:
// 5, 8-13, 19-22, 24, 25, 27, 29, 31, 32, 33, 35 and 37 of the README (20).
const REFUSAL_SCHEMAS: [string, string, string, Record<string, "sku" | "second">][] = [
["unique_item", "tests.unique_item", "sha256:93bb890de83069061af696b23f94fd32fb3c2dad3c91ee645e8add56085f1882", { Sku: "sku" }],
["uq_pair", "tests.uq_pair", "sha256:2538cceafb6dcbce22ef11b932ecb804af5f96bc3aaedff692a52d3640433e60", { Sku: "sku", Alt: "second" }],
["uq_alias", "tests.uq_alias", "sha256:77a77a394ebd8bf71e14b9b6f2cea4978c25d4e705fd12364f660edbf6c63b1f", { Sku: "sku", alt: "second" }],
["uq_wide", "tests.uq_wide", "sha256:b79eecd11eed5e456902f095d2748da7f59125c4c8972b0da7dddc81d07fc03f", { "\uFF21": "sku", "\u{10400}": "second" }],
["uq_alpha", "tests.uq_alpha", "sha256:53aeaf2ea9b25e85ade6b15fc04c7653e7b206c87e258f1aa74dd32a840a1389", { Sku: "sku" }],
["uq_beta", "tests.uq_beta", "sha256:45ed62a64c91f6d815d3ec975c4a4a1b2ce07b6e7262a32d828673cf10899586", { Sku: "sku" }],
];
type RefusalOp = ["open", string | null] | ["pull", string] | ["commit", [string, string, string, string][]];
test("vectors refusals.msgpack: every row TS can perform", async () => {
const rows = decode(await readFile(join(vectors, "refusals.msgpack"))) as [string, RefusalOp[], [string, string, string, string] | null][];
const definitions = new Map(
REFUSAL_SCHEMAS.map(([type, schemaName, schemaId, indexes]) => [
schemaName,
defineDocumentType({
type,
schemaId,
schemaName,
schemaVersion: `${schemaName}.v1`,
schema: z.object({ sku: z.string(), second: z.string() }),
codec: codecs.slots<{ sku: string; second: string }>({ 0: ["sku", codecs.string], 1: ["second", codecs.string] }, `Uq${type}`),
indexes,
}),
]),
);
const dir = await mkdtemp(join(tmpdir(), "uidx-refusals-"));
let walked = 0;
try {
for (const [label, ops, refused] of rows) {
if (ops.some((op) => op[0] === "commit" && op[1].length > 1)) continue;
const file = join(dir, `${walked}.cc`);
let cache: CultCache | undefined;
const perform = async (op: RefusalOp) => {
if (op[0] === "open") {
if (op[1] !== null) await copyFile(join(vectors, op[1]), file);
let builder = CultCache.builder().withGenericStore(new SingleFileMessagePackBackingStore(file));
for (const definition of definitions.values()) builder = builder.withDocumentType(definition);
cache = builder.build();
await cache.pullAllBackingStores();
} else if (op[0] === "pull") {
await copyFile(join(vectors, op[1]), file);
await cache!.pullAllBackingStores();
} else {
for (const [key, schema, sku, second] of op[1]) await cache!.put(definitions.get(schema)!, key, { sku, second });
}
};
for (const op of ops.slice(0, -1)) await perform(op);
const last = ops[ops.length - 1];
if (refused === null) {
await perform(last);
} else {
const [k1, k2, schema, alias] = refused;
const before = last[0] === "commit" || (last[0] === "open" && last[1] === null)
? (existsSync(file) ? await readFile(file) : Buffer.alloc(0))
: await readFile(join(vectors, last[1]!));
await assert.rejects(perform(last), { message: refusal(k1, k2, schema, alias) }, label);
assert.deepEqual(existsSync(file) ? await readFile(file) : Buffer.alloc(0), before, `${label}: the file is untouched`);
}
walked++;
}
} finally {
await rm(dir, { recursive: true, force: true });
}
assert.equal(walked, 20);
});
test("a refused load leaves the cache's prior state as it was, and a later good load still works", async () => {
const dir = await mkdtemp(join(tmpdir(), "uidx-load-"));
try {
const bad = join(dir, "bad.cc");
await copyFile(join(vectors, "duplicate-store.cc"), bad);
const [envelope] = await new SingleFileMessagePackBackingStore(bad).pullAll();
const document = twoDefinitions(envelope.schemaId!, "indexes");
const cache = CultCache.builder().withDocumentType(document).withGenericStore(new SingleFileMessagePackBackingStore(bad)).build();
await assert.rejects(cache.pullAllBackingStores(), /Records item:a and item:b/u);
assert.equal(cache.get(document, "item:a"), undefined);
assert.equal(cache.getKeyByIndex(document, "Sku", "S-1"), undefined);
} finally {
await rm(dir, { recursive: true, force: true });
}
});
test("registerIndex refuses a name the definition declares as a role", async () => {
const document = item("item", "roles");
const cache = memory(document);
await cache.put(document, "a", { sku: "S", group: "g" });
await cache.put(document, "b", { sku: "S", group: "h" });
await assert.rejects(cache.registerIndex(document, "sku", "sku"), { message: /declared as a role; a role is never a unique index/u });
assert.equal(cache.getKeyByIndex(document, "sku", "S"), undefined);
await cache.put(document, "c", { sku: "S", group: "i" });
});
test("registerIndex over held duplicates installs nothing", async () => {
const document = item("item", "none");
const cache = memory(document);
await cache.put(document, "a", { sku: "S", group: "g" });
await cache.put(document, "b", { sku: "S", group: "h" });
await assert.rejects(cache.registerIndex(document, "sku", "sku"), { message: refusal("a", "b", "tests.item", "sku") });
assert.equal(cache.getKeyByIndex(document, "sku", "S"), undefined, "no map was installed");
await cache.put(document, "c", { sku: "S", group: "i" });
await cache.registerIndex(document, "group", "group");
assert.equal(cache.getKeyByIndex(document, "group", "h"), "b");
await assert.rejects(cache.put(document, "d", { sku: "x", group: "h" }), /Records b and d/u);
});
test("registerIndex over distinct held values installs the lookup", async () => {
const document = item("item", "none");
const cache = memory(document);
await cache.put(document, "a", { sku: "S" });
await cache.registerIndex(document, "sku", "sku");
assert.equal(cache.getKeyByIndex(document, "sku", "S"), "a");
});
// ---- a type with several unique indexes: the second alias is judged, in alias order, in both judges ----
const pairSchema = z.object({ a: z.string().optional(), b: z.string().optional() });
function pair(where: { indexes?: Record<string, "a" | "b">; roles?: Record<string, "a" | "b"> }) {
return defineDocumentType({
type: "pair",
schemaId: "tests.pair.v1",
schemaName: "tests.pair",
schemaVersion: "tests.pair.v1",
schema: pairSchema,
...where,
});
}
// Declared b first, so insertion order is the opposite of alias order.
const BOTH_UNIQUE = { indexes: { b: "b", a: "a" } as Record<string, "a" | "b"> };
test("a collision on the second unique alias is refused, with a value on the first alias that does not collide", async () => {
const document = pair(BOTH_UNIQUE);
const cache = memory(document);
await cache.put(document, "k1", { a: "x1", b: "shared" });
await assert.rejects(cache.put(document, "k2", { a: "x2", b: "shared" }), { message: refusal("k1", "k2", "tests.pair", "b") });
await assert.rejects(cache.put(document, "k2", { a: "x1", b: "y" }), { message: refusal("k1", "k2", "tests.pair", "a") });
assert.equal(cache.get(document, "k2"), undefined);
});
test("when two aliases collide the first alias in ordinal order is named, and its keys are in ordinal order", async () => {
const document = pair(BOTH_UNIQUE);
const cache = memory(document);
await cache.put(document, "m", { a: "same-a", b: "same-b" });
await assert.rejects(cache.put(document, "c", { a: "same-a", b: "same-b" }), { message: refusal("c", "m", "tests.pair", "a") });
// The colliding holders differ per alias: a is held by m, b by n; alias a is still the one named.
await cache.put(document, "n", { a: "other-a", b: "other-b" });
await assert.rejects(cache.put(document, "z", { a: "same-a", b: "other-b" }), { message: refusal("m", "z", "tests.pair", "a") });
});
test("a load judges every unique alias: a store collision on the second alias refuses the open", async () => {
const dir = await mkdtemp(join(tmpdir(), "uidx-pair-"));
try {
for (const [name, records, expected] of [
["second-only", [["k1", { a: "1", b: "s" }], ["k2", { a: "2", b: "s" }]], refusal("k1", "k2", "tests.pair", "b")],
["both-aliases", [["m", { a: "s", b: "s" }], ["c", { a: "s", b: "s" }]], refusal("c", "m", "tests.pair", "a")],
] as const) {
const file = join(dir, `${name}.cc`);
const roles = pair({ roles: { b: "b", a: "a" } });
const writer = CultCache.builder().withDocumentType(roles).withGenericStore(new SingleFileMessagePackBackingStore(file)).build();
for (const [key, value] of records) await writer.put(roles, key, value);
const before = await readFile(file);
const unique = pair(BOTH_UNIQUE);
const reader = CultCache.builder().withDocumentType(unique).withGenericStore(new SingleFileMessagePackBackingStore(file)).build();
await assert.rejects(reader.pullAllBackingStores(), { message: expected }, name);
assert.deepEqual(reader.snapshot(), [], "a refused load admits nothing");
assert.deepEqual(await readFile(file), before);
const rolesReader = CultCache.builder().withDocumentType(roles).withGenericStore(new SingleFileMessagePackBackingStore(file)).build();
await rolesReader.pullAllBackingStores();
assert.equal(rolesReader.snapshot().length, 2, `${name}: the same store opens under roles`);
}
} finally {
await rm(dir, { recursive: true, force: true });
}
});
test("registerIndex of a second unique alias judges it, naming the two smallest keys whatever order they were written in", async () => {
const document = pair({ indexes: { a: "a" } });
const cache = memory(document);
await cache.put(document, "z", { a: "1", b: "s" });
await cache.put(document, "m", { a: "2", b: "s" });
await cache.put(document, "q", { a: "3", b: "s" });
await assert.rejects(cache.registerIndex(document, "b", "b"), { message: refusal("m", "q", "tests.pair", "b") });
assert.equal(cache.getKeyByIndex(document, "a", "1"), "z", "the alias already installed is intact");
assert.equal(cache.getKeyByIndex(document, "b", "s"), undefined);
});
// ---- the single-put judge's two orders, which no pair-order vector row reaches from TS (a vector row's second record
// arrives in a batch that TS cannot perform): aliases by UTF-16 code unit, then the key pair by UTF-16 code unit ----
test("a put colliding on two aliases names the one least by UTF-16 code unit, not by code point", async () => {
// U+10400 is the surrogate pair D801 DC00, so it sorts before U+FF21 by code unit and after it by code point.
const document = pair({ indexes: { "A": "a", "\u{10400}": "b" } });
const cache = memory(document);
await cache.put(document, "h1", { a: "s1", b: "t1" });
await cache.put(document, "h2", { a: "s2", b: "t2" });
await assert.rejects(cache.put(document, "p", { a: "s1", b: "t2" }), { message: refusal("h2", "p", "tests.pair", "\u{10400}") });
assert.equal(cache.get(document, "p"), undefined);
});
test("a put names its key pair by UTF-16 code unit, not by normalised form", async () => {
// "e" U+0302 is the NFD form of U+00EA: equal when normalised, and less by code unit.
const document = item("item");
const cache = memory(document);
await cache.put(document, "ê", { sku: "v" });
await assert.rejects(cache.put(document, "ê", { sku: "v" }), { message: refusal("ê", "ê", "tests.item", "sku") });
assert.equal(cache.get(document, "ê"), undefined);
});
test("a put colliding on two aliases orders them by UTF-16 code unit, not by normalised form", async () => {
// U+00EA normalises to e U+0302, which is less than e U+0303 when normalised and greater by code unit.
const document = pair({ indexes: { "ê": "a", "ẽ": "b" } });
const cache = memory(document);
await cache.put(document, "h1", { a: "s1", b: "t1" });
await cache.put(document, "h2", { a: "s2", b: "t2" });
await assert.rejects(cache.put(document, "p", { a: "s1", b: "t2" }), { message: refusal("h2", "p", "tests.pair", "ẽ") });
});

View file

@ -10,7 +10,7 @@
<PackageId>GameCult.Math</PackageId> <PackageId>GameCult.Math</PackageId>
<!-- MIT overall; math.Phacelle.cs and CultPhasor.cs (and the two Phacelle HLSL files) are MPL-2.0; math.Cycles.cs and CultMath.Cycles.hlsl are Apache-2.0. --> <!-- MIT overall; math.Phacelle.cs and CultPhasor.cs (and the two Phacelle HLSL files) are MPL-2.0; math.Cycles.cs and CultMath.Cycles.hlsl are Apache-2.0. -->
<PackageLicenseExpression>MIT AND MPL-2.0 AND Apache-2.0</PackageLicenseExpression> <PackageLicenseExpression>MIT AND MPL-2.0 AND Apache-2.0</PackageLicenseExpression>
<Version>0.5.0</Version> <Version>0.5.1</Version>
<Description>Clean-room C# math shaped for HLSL-style renderer code.</Description> <Description>Clean-room C# math shaped for HLSL-style renderer code.</Description>
<Authors>GameCult</Authors> <Authors>GameCult</Authors>
<PackageReadmeFile>README.md</PackageReadmeFile> <PackageReadmeFile>README.md</PackageReadmeFile>

View file

@ -5,7 +5,7 @@ All notable changes to this package are documented in this file.
Earlier releases (0.2.0-0.2.3) predate this package's changelog and are not Earlier releases (0.2.0-0.2.3) predate this package's changelog and are not
backfilled here; see `docs/semver-policy.md` for why. backfilled here; see `docs/semver-policy.md` for why.
## [Unreleased] ## [0.5.1] - 2026-11-11
### Added ### Added

View file

@ -1,7 +1,7 @@
{ {
"name": "org.gamecult.cultmath", "name": "org.gamecult.cultmath",
"displayName": "CultMath", "displayName": "CultMath",
"version": "0.5.0", "version": "0.5.1",
"unity": "2021.3", "unity": "2021.3",
"description": "Precompiled CultMath numeric primitives and matching HLSL semantics for GameCult Unity runtimes.", "description": "Precompiled CultMath numeric primitives and matching HLSL semantics for GameCult Unity runtimes.",
"license": "MIT AND MPL-2.0 AND Apache-2.0", "license": "MIT AND MPL-2.0 AND Apache-2.0",

View file

@ -567,8 +567,14 @@ test("a size the runtime cannot allocate is too-large in both exports, not a Ran
const count = 2 ** 15; const count = 2 ** 15;
const huge = { ...manifest, sizeBytes: count * int32Max, chunks: Array.from({ length: count }, (_, index) => ({ ...manifest.chunks[0]!, offset: index * int32Max, sizeBytes: int32Max })) }; const huge = { ...manifest, sizeBytes: count * int32Max, chunks: Array.from({ length: count }, (_, index) => ({ ...manifest.chunks[0]!, offset: index * int32Max, sizeBytes: int32Max })) };
const server = double(() => { throw new Error("no request expected"); }); const server = double(() => { throw new Error("no request expected"); });
await assert.rejects(fetchVerifiedContent(ENDPOINT, huge, { fetch: server.fetch, maxBytes: size }), codeOf("too-large")); // a host with overcommit hands out 64 TiB lazily, so the refusal is made certain: the runtime cannot allocate it
await assert.rejects(fetchVerifiedChunk(ENDPOINT, manifest.chunks[0]!.chunkHash!, size, { fetch: server.fetch, maxBytes: size }), codeOf("too-large")); const native = globalThis.Uint8Array;
const refusing = new Proxy(native, { construct: (target, args, receiver) => { if (typeof args[0] === "number" && args[0] > 2 ** 40) throw new RangeError("Array buffer allocation failed"); return Reflect.construct(target, args, receiver); } });
globalThis.Uint8Array = refusing;
try {
await assert.rejects(fetchVerifiedContent(ENDPOINT, huge, { fetch: server.fetch, maxBytes: size }), codeOf("too-large"));
await assert.rejects(fetchVerifiedChunk(ENDPOINT, manifest.chunks[0]!.chunkHash!, size, { fetch: server.fetch, maxBytes: size }), codeOf("too-large"));
} finally { globalThis.Uint8Array = native; }
assert.equal(server.calls.length, 0); assert.equal(server.calls.length, 0);
}); });

View file

@ -474,3 +474,14 @@ powershell -ExecutionPolicy Bypass -File .\packages\cultmesh-kotlin\build.ps1
The build writes `artifacts/cultmesh-kotlin/cultmesh-kotlin.jar`. The build writes `artifacts/cultmesh-kotlin/cultmesh-kotlin.jar`.
It also runs the built-in RUDP packet fixture and localhost UDP socket It also runs the built-in RUDP packet fixture and localhost UDP socket
self-test. self-test.
## Write Receipt
`WriteReceipt.kt` decodes `cultnet.write_admitted.v0` and `cultnet.error.v1` (and `error.v0`) strictly against their
closed schemas (`decodeWriteAdmitted`, `decodeErrorFrame`), computes the dedupe `writeContentHash`, tests a hello with
`helloAdvertisesWriteReceipts`, and settles sent writes as admitted, refused or unconfirmed in `WriteTracker`
(pure state machine, abstract clock; the caller owns transport and timers). Kotlin has no write client yet, so nothing
sends a put and waits on this. `WriteReceiptVectors.kt` replays
`contracts/cultnet/interop/write-receipt-vectors.json` (constants, contentHash, wire, schemaInvalid, scenarios; the
server-side `dedupe` and `blankFields` sections are not run); `build.ps1` runs it after the self-test, or run
`CultMeshKt write-receipt-vectors <path>`.

View file

@ -26,4 +26,7 @@ $sources = Get-ChildItem $src -Recurse -Filter *.kt | Select-Object -ExpandPrope
if ($LASTEXITCODE -ne 0) { throw "kotlinc failed with exit code $LASTEXITCODE" } if ($LASTEXITCODE -ne 0) { throw "kotlinc failed with exit code $LASTEXITCODE" }
& $java -cp "$jarPath;$kotlinStdlib" org.gamecult.cultmesh.CultMeshKt & $java -cp "$jarPath;$kotlinStdlib" org.gamecult.cultmesh.CultMeshKt
if ($LASTEXITCODE -ne 0) { throw "cultmesh-kotlin self-test failed with exit code $LASTEXITCODE" } if ($LASTEXITCODE -ne 0) { throw "cultmesh-kotlin self-test failed with exit code $LASTEXITCODE" }
$vectors = Join-Path $repoRoot "contracts\cultnet\interop\write-receipt-vectors.json"
& $java -cp "$jarPath;$kotlinStdlib" org.gamecult.cultmesh.CultMeshKt write-receipt-vectors $vectors
if ($LASTEXITCODE -ne 0) { throw "write-receipt vector replay failed with exit code $LASTEXITCODE" }
Write-Host "Built $jarPath" Write-Host "Built $jarPath"

View file

@ -19,6 +19,7 @@ import java.net.Socket
import java.net.SocketTimeoutException import java.net.SocketTimeoutException
import java.net.URI import java.net.URI
import java.nio.ByteBuffer import java.nio.ByteBuffer
import java.nio.charset.CodingErrorAction
import java.nio.charset.StandardCharsets import java.nio.charset.StandardCharsets
import java.security.SecureRandom import java.security.SecureRandom
import java.security.MessageDigest import java.security.MessageDigest
@ -808,6 +809,7 @@ interface CultNetSchemaMessageTransport : AutoCloseable {
val profile: CultNetTransportProfile val profile: CultNetTransportProfile
val stats: CultNetTransportStats val stats: CultNetTransportStats
/** Fire-and-forget: an ack or a return is not admission; read the result frame. */
fun sendSchemaMessage(message: CultNetMessage) fun sendSchemaMessage(message: CultNetMessage)
fun receiveSchemaMessage(timeoutMs: Long = 1_000, pollIntervalMs: Long = 5): CultNetMessage? fun receiveSchemaMessage(timeoutMs: Long = 1_000, pollIntervalMs: Long = 5): CultNetMessage?
@ -1099,9 +1101,9 @@ data class CultNetRawDocumentRecord(
"payloadEncoding" to payloadEncoding, "payloadEncoding" to payloadEncoding,
"payload" to payload.copyOf(), "payload" to payload.copyOf(),
) )
if (!sourceRuntimeId.isNullOrBlank()) wire["sourceRuntimeId"] = sourceRuntimeId if (!isBlankText(sourceRuntimeId)) wire["sourceRuntimeId"] = sourceRuntimeId
if (!sourceAgentId.isNullOrBlank()) wire["sourceAgentId"] = sourceAgentId if (!isBlankText(sourceAgentId)) wire["sourceAgentId"] = sourceAgentId
if (!sourceRole.isNullOrBlank()) wire["sourceRole"] = sourceRole if (!isBlankText(sourceRole)) wire["sourceRole"] = sourceRole
if (tags.isNotEmpty()) wire["tags"] = tags if (tags.isNotEmpty()) wire["tags"] = tags
return wire return wire
} }
@ -2363,6 +2365,9 @@ fun parseCultNetMessage(payload: ByteArray): CultNetMessage {
if (schemaVersion !is String || schemaVersion.isBlank()) { if (schemaVersion !is String || schemaVersion.isBlank()) {
throw IOException("CultNet schema-v0 messages must declare schemaVersion") throw IOException("CultNet schema-v0 messages must declare schemaVersion")
} }
// A write_admitted id is echoed byte for byte, so that frame alone is read again with invalid UTF-8 refused; document payload
// strings elsewhere keep decoding as before.
if (schemaVersion == WRITE_ADMITTED_VERSION) MessagePackReader(payload, strictUtf8 = true).readAny()
val body = linkedMapOf<String, Any?>() val body = linkedMapOf<String, Any?>()
for ((key, value) in decoded) { for ((key, value) in decoded) {
if (key !is String) throw IOException("CultNet schema-v0 message keys must be strings") if (key !is String) throw IOException("CultNet schema-v0 message keys must be strings")
@ -3167,13 +3172,16 @@ class CultNetRudpSocketTransportConnection(
return connected return connected
} }
/** Fire-and-forget: an ack or a return is not admission; read the result frame. */
fun send(channelId: String, payload: ByteArray) { fun send(channelId: String, payload: ByteArray) {
session.sendMany(channelId, payload, channelSendOptions(channelId, nowMs()), maxFragmentBytes).forEach { sendPacket(it) } session.sendMany(channelId, payload, channelSendOptions(channelId, nowMs()), maxFragmentBytes).forEach { sendPacket(it) }
framesSent += 1 framesSent += 1
} }
/** Fire-and-forget: an ack or a return is not admission; read the result frame. */
fun send(channelId: String, payload: String) = send(channelId, payload.toByteArray(StandardCharsets.UTF_8)) fun send(channelId: String, payload: String) = send(channelId, payload.toByteArray(StandardCharsets.UTF_8))
/** Fire-and-forget: an ack or a return is not admission; read the result frame. */
fun sendSchema(payload: ByteArray) = send("schema", payload) fun sendSchema(payload: ByteArray) = send("schema", payload)
fun sendSchema(payload: String) = sendSchema(payload.toByteArray(StandardCharsets.UTF_8)) fun sendSchema(payload: String) = sendSchema(payload.toByteArray(StandardCharsets.UTF_8))
@ -3409,6 +3417,7 @@ fun main(args: Array<String>) {
rudpClientBindsLoopbackOnlyForALoopbackEndpoint() rudpClientBindsLoopbackOnlyForALoopbackEndpoint()
rudpSocketTransportHandshakesAndCarriesReliableOrderedSchemaFrames() rudpSocketTransportHandshakesAndCarriesReliableOrderedSchemaFrames()
rudpSocketTransportCarriesFragmentedReliableOrderedSchemaFrames() rudpSocketTransportCarriesFragmentedReliableOrderedSchemaFrames()
rawDocumentRecordDropsOnlyWhiteSpaceSourceIds()
return return
} }
@ -3420,6 +3429,7 @@ fun main(args: Array<String>) {
"rudp-dial-message-once" -> rudpDialSchemaMessageOnce(options) "rudp-dial-message-once" -> rudpDialSchemaMessageOnce(options)
"serve" -> interopServe(options) "serve" -> interopServe(options)
"dial" -> interopDial(options) "dial" -> interopDial(options)
"write-receipt-vectors" -> if (!replayWriteReceiptVectors(args.getOrNull(1) ?: error("usage: write-receipt-vectors <vectors.json>"))) kotlin.system.exitProcess(1)
else -> error("Unknown mode ${args[0]}") else -> error("Unknown mode ${args[0]}")
} }
} }
@ -6145,7 +6155,8 @@ private fun requireSchemaResponse(message: CultNetMessage?, expectedSchemaVersio
return response return response
} }
class MessagePackReader(payload: ByteArray) { /** [strictUtf8] makes a str whose bytes are not UTF-8 throw instead of becoming U+FFFD; only the write_admitted frame asks for it. */
class MessagePackReader(payload: ByteArray, private val strictUtf8: Boolean = false) {
private val input = DataInputStream(ByteArrayInputStream(payload)) private val input = DataInputStream(ByteArrayInputStream(payload))
private var pushed = -1 private var pushed = -1
@ -6300,7 +6311,10 @@ class MessagePackReader(payload: ByteArray) {
code == 0xdb -> input.readInt() code == 0xdb -> input.readInt()
else -> throw IOException("expected string") else -> throw IOException("expected string")
} }
return String(readPayload(length), StandardCharsets.UTF_8) val bytes = readPayload(length)
if (!strictUtf8) return String(bytes, StandardCharsets.UTF_8)
return StandardCharsets.UTF_8.newDecoder().onMalformedInput(CodingErrorAction.REPORT).onUnmappableCharacter(CodingErrorAction.REPORT)
.decode(ByteBuffer.wrap(bytes)).toString()
} }
private fun readBinaryAfterCode(code: Int): ByteArray { private fun readBinaryAfterCode(code: Int): ByteArray {

View file

@ -0,0 +1,129 @@
// generated, Unicode 16.0.0, by scripts/gen-unicode-messageid-tables.py (inputs sha256 24a07ef9fabf71e3); do not edit
package org.gamecult.cultmesh
// (low, high) pairs, flattened and sorted.
internal val LETTER_OR_NUMBER: IntArray = intArrayOf(
0x30, 0x39, 0x41, 0x5A, 0x61, 0x7A, 0xAA, 0xAA, 0xB2, 0xB3, 0xB5, 0xB5, 0xB9, 0xBA, 0xBC, 0xBE, 0xC0, 0xD6, 0xD8,
0xF6, 0xF8, 0x2C1, 0x2C6, 0x2D1, 0x2E0, 0x2E4, 0x2EC, 0x2EC, 0x2EE, 0x2EE, 0x370, 0x374, 0x376, 0x377, 0x37A,
0x37D, 0x37F, 0x37F, 0x386, 0x386, 0x388, 0x38A, 0x38C, 0x38C, 0x38E, 0x3A1, 0x3A3, 0x3F5, 0x3F7, 0x481, 0x48A,
0x52F, 0x531, 0x556, 0x559, 0x559, 0x560, 0x588, 0x5D0, 0x5EA, 0x5EF, 0x5F2, 0x620, 0x64A, 0x660, 0x669, 0x66E,
0x66F, 0x671, 0x6D3, 0x6D5, 0x6D5, 0x6E5, 0x6E6, 0x6EE, 0x6FC, 0x6FF, 0x6FF, 0x710, 0x710, 0x712, 0x72F, 0x74D,
0x7A5, 0x7B1, 0x7B1, 0x7C0, 0x7EA, 0x7F4, 0x7F5, 0x7FA, 0x7FA, 0x800, 0x815, 0x81A, 0x81A, 0x824, 0x824, 0x828,
0x828, 0x840, 0x858, 0x860, 0x86A, 0x870, 0x887, 0x889, 0x88E, 0x8A0, 0x8C9, 0x904, 0x939, 0x93D, 0x93D, 0x950,
0x950, 0x958, 0x961, 0x966, 0x96F, 0x971, 0x980, 0x985, 0x98C, 0x98F, 0x990, 0x993, 0x9A8, 0x9AA, 0x9B0, 0x9B2,
0x9B2, 0x9B6, 0x9B9, 0x9BD, 0x9BD, 0x9CE, 0x9CE, 0x9DC, 0x9DD, 0x9DF, 0x9E1, 0x9E6, 0x9F1, 0x9F4, 0x9F9, 0x9FC,
0x9FC, 0xA05, 0xA0A, 0xA0F, 0xA10, 0xA13, 0xA28, 0xA2A, 0xA30, 0xA32, 0xA33, 0xA35, 0xA36, 0xA38, 0xA39, 0xA59,
0xA5C, 0xA5E, 0xA5E, 0xA66, 0xA6F, 0xA72, 0xA74, 0xA85, 0xA8D, 0xA8F, 0xA91, 0xA93, 0xAA8, 0xAAA, 0xAB0, 0xAB2,
0xAB3, 0xAB5, 0xAB9, 0xABD, 0xABD, 0xAD0, 0xAD0, 0xAE0, 0xAE1, 0xAE6, 0xAEF, 0xAF9, 0xAF9, 0xB05, 0xB0C, 0xB0F,
0xB10, 0xB13, 0xB28, 0xB2A, 0xB30, 0xB32, 0xB33, 0xB35, 0xB39, 0xB3D, 0xB3D, 0xB5C, 0xB5D, 0xB5F, 0xB61, 0xB66,
0xB6F, 0xB71, 0xB77, 0xB83, 0xB83, 0xB85, 0xB8A, 0xB8E, 0xB90, 0xB92, 0xB95, 0xB99, 0xB9A, 0xB9C, 0xB9C, 0xB9E,
0xB9F, 0xBA3, 0xBA4, 0xBA8, 0xBAA, 0xBAE, 0xBB9, 0xBD0, 0xBD0, 0xBE6, 0xBF2, 0xC05, 0xC0C, 0xC0E, 0xC10, 0xC12,
0xC28, 0xC2A, 0xC39, 0xC3D, 0xC3D, 0xC58, 0xC5A, 0xC5D, 0xC5D, 0xC60, 0xC61, 0xC66, 0xC6F, 0xC78, 0xC7E, 0xC80,
0xC80, 0xC85, 0xC8C, 0xC8E, 0xC90, 0xC92, 0xCA8, 0xCAA, 0xCB3, 0xCB5, 0xCB9, 0xCBD, 0xCBD, 0xCDD, 0xCDE, 0xCE0,
0xCE1, 0xCE6, 0xCEF, 0xCF1, 0xCF2, 0xD04, 0xD0C, 0xD0E, 0xD10, 0xD12, 0xD3A, 0xD3D, 0xD3D, 0xD4E, 0xD4E, 0xD54,
0xD56, 0xD58, 0xD61, 0xD66, 0xD78, 0xD7A, 0xD7F, 0xD85, 0xD96, 0xD9A, 0xDB1, 0xDB3, 0xDBB, 0xDBD, 0xDBD, 0xDC0,
0xDC6, 0xDE6, 0xDEF, 0xE01, 0xE30, 0xE32, 0xE33, 0xE40, 0xE46, 0xE50, 0xE59, 0xE81, 0xE82, 0xE84, 0xE84, 0xE86,
0xE8A, 0xE8C, 0xEA3, 0xEA5, 0xEA5, 0xEA7, 0xEB0, 0xEB2, 0xEB3, 0xEBD, 0xEBD, 0xEC0, 0xEC4, 0xEC6, 0xEC6, 0xED0,
0xED9, 0xEDC, 0xEDF, 0xF00, 0xF00, 0xF20, 0xF33, 0xF40, 0xF47, 0xF49, 0xF6C, 0xF88, 0xF8C, 0x1000, 0x102A,
0x103F, 0x1049, 0x1050, 0x1055, 0x105A, 0x105D, 0x1061, 0x1061, 0x1065, 0x1066, 0x106E, 0x1070, 0x1075, 0x1081,
0x108E, 0x108E, 0x1090, 0x1099, 0x10A0, 0x10C5, 0x10C7, 0x10C7, 0x10CD, 0x10CD, 0x10D0, 0x10FA, 0x10FC, 0x1248,
0x124A, 0x124D, 0x1250, 0x1256, 0x1258, 0x1258, 0x125A, 0x125D, 0x1260, 0x1288, 0x128A, 0x128D, 0x1290, 0x12B0,
0x12B2, 0x12B5, 0x12B8, 0x12BE, 0x12C0, 0x12C0, 0x12C2, 0x12C5, 0x12C8, 0x12D6, 0x12D8, 0x1310, 0x1312, 0x1315,
0x1318, 0x135A, 0x1369, 0x137C, 0x1380, 0x138F, 0x13A0, 0x13F5, 0x13F8, 0x13FD, 0x1401, 0x166C, 0x166F, 0x167F,
0x1681, 0x169A, 0x16A0, 0x16EA, 0x16EE, 0x16F8, 0x1700, 0x1711, 0x171F, 0x1731, 0x1740, 0x1751, 0x1760, 0x176C,
0x176E, 0x1770, 0x1780, 0x17B3, 0x17D7, 0x17D7, 0x17DC, 0x17DC, 0x17E0, 0x17E9, 0x17F0, 0x17F9, 0x1810, 0x1819,
0x1820, 0x1878, 0x1880, 0x1884, 0x1887, 0x18A8, 0x18AA, 0x18AA, 0x18B0, 0x18F5, 0x1900, 0x191E, 0x1946, 0x196D,
0x1970, 0x1974, 0x1980, 0x19AB, 0x19B0, 0x19C9, 0x19D0, 0x19DA, 0x1A00, 0x1A16, 0x1A20, 0x1A54, 0x1A80, 0x1A89,
0x1A90, 0x1A99, 0x1AA7, 0x1AA7, 0x1B05, 0x1B33, 0x1B45, 0x1B4C, 0x1B50, 0x1B59, 0x1B83, 0x1BA0, 0x1BAE, 0x1BE5,
0x1C00, 0x1C23, 0x1C40, 0x1C49, 0x1C4D, 0x1C7D, 0x1C80, 0x1C8A, 0x1C90, 0x1CBA, 0x1CBD, 0x1CBF, 0x1CE9, 0x1CEC,
0x1CEE, 0x1CF3, 0x1CF5, 0x1CF6, 0x1CFA, 0x1CFA, 0x1D00, 0x1DBF, 0x1E00, 0x1F15, 0x1F18, 0x1F1D, 0x1F20, 0x1F45,
0x1F48, 0x1F4D, 0x1F50, 0x1F57, 0x1F59, 0x1F59, 0x1F5B, 0x1F5B, 0x1F5D, 0x1F5D, 0x1F5F, 0x1F7D, 0x1F80, 0x1FB4,
0x1FB6, 0x1FBC, 0x1FBE, 0x1FBE, 0x1FC2, 0x1FC4, 0x1FC6, 0x1FCC, 0x1FD0, 0x1FD3, 0x1FD6, 0x1FDB, 0x1FE0, 0x1FEC,
0x1FF2, 0x1FF4, 0x1FF6, 0x1FFC, 0x2070, 0x2071, 0x2074, 0x2079, 0x207F, 0x2089, 0x2090, 0x209C, 0x2102, 0x2102,
0x2107, 0x2107, 0x210A, 0x2113, 0x2115, 0x2115, 0x2119, 0x211D, 0x2124, 0x2124, 0x2126, 0x2126, 0x2128, 0x2128,
0x212A, 0x212D, 0x212F, 0x2139, 0x213C, 0x213F, 0x2145, 0x2149, 0x214E, 0x214E, 0x2150, 0x2189, 0x2460, 0x249B,
0x24EA, 0x24FF, 0x2776, 0x2793, 0x2C00, 0x2CE4, 0x2CEB, 0x2CEE, 0x2CF2, 0x2CF3, 0x2CFD, 0x2CFD, 0x2D00, 0x2D25,
0x2D27, 0x2D27, 0x2D2D, 0x2D2D, 0x2D30, 0x2D67, 0x2D6F, 0x2D6F, 0x2D80, 0x2D96, 0x2DA0, 0x2DA6, 0x2DA8, 0x2DAE,
0x2DB0, 0x2DB6, 0x2DB8, 0x2DBE, 0x2DC0, 0x2DC6, 0x2DC8, 0x2DCE, 0x2DD0, 0x2DD6, 0x2DD8, 0x2DDE, 0x2E2F, 0x2E2F,
0x3005, 0x3007, 0x3021, 0x3029, 0x3031, 0x3035, 0x3038, 0x303C, 0x3041, 0x3096, 0x309D, 0x309F, 0x30A1, 0x30FA,
0x30FC, 0x30FF, 0x3105, 0x312F, 0x3131, 0x318E, 0x3192, 0x3195, 0x31A0, 0x31BF, 0x31F0, 0x31FF, 0x3220, 0x3229,
0x3248, 0x324F, 0x3251, 0x325F, 0x3280, 0x3289, 0x32B1, 0x32BF, 0x3400, 0x4DBF, 0x4E00, 0xA48C, 0xA4D0, 0xA4FD,
0xA500, 0xA60C, 0xA610, 0xA62B, 0xA640, 0xA66E, 0xA67F, 0xA69D, 0xA6A0, 0xA6EF, 0xA717, 0xA71F, 0xA722, 0xA788,
0xA78B, 0xA7CD, 0xA7D0, 0xA7D1, 0xA7D3, 0xA7D3, 0xA7D5, 0xA7DC, 0xA7F2, 0xA801, 0xA803, 0xA805, 0xA807, 0xA80A,
0xA80C, 0xA822, 0xA830, 0xA835, 0xA840, 0xA873, 0xA882, 0xA8B3, 0xA8D0, 0xA8D9, 0xA8F2, 0xA8F7, 0xA8FB, 0xA8FB,
0xA8FD, 0xA8FE, 0xA900, 0xA925, 0xA930, 0xA946, 0xA960, 0xA97C, 0xA984, 0xA9B2, 0xA9CF, 0xA9D9, 0xA9E0, 0xA9E4,
0xA9E6, 0xA9FE, 0xAA00, 0xAA28, 0xAA40, 0xAA42, 0xAA44, 0xAA4B, 0xAA50, 0xAA59, 0xAA60, 0xAA76, 0xAA7A, 0xAA7A,
0xAA7E, 0xAAAF, 0xAAB1, 0xAAB1, 0xAAB5, 0xAAB6, 0xAAB9, 0xAABD, 0xAAC0, 0xAAC0, 0xAAC2, 0xAAC2, 0xAADB, 0xAADD,
0xAAE0, 0xAAEA, 0xAAF2, 0xAAF4, 0xAB01, 0xAB06, 0xAB09, 0xAB0E, 0xAB11, 0xAB16, 0xAB20, 0xAB26, 0xAB28, 0xAB2E,
0xAB30, 0xAB5A, 0xAB5C, 0xAB69, 0xAB70, 0xABE2, 0xABF0, 0xABF9, 0xAC00, 0xD7A3, 0xD7B0, 0xD7C6, 0xD7CB, 0xD7FB,
0xF900, 0xFA6D, 0xFA70, 0xFAD9, 0xFB00, 0xFB06, 0xFB13, 0xFB17, 0xFB1D, 0xFB1D, 0xFB1F, 0xFB28, 0xFB2A, 0xFB36,
0xFB38, 0xFB3C, 0xFB3E, 0xFB3E, 0xFB40, 0xFB41, 0xFB43, 0xFB44, 0xFB46, 0xFBB1, 0xFBD3, 0xFD3D, 0xFD50, 0xFD8F,
0xFD92, 0xFDC7, 0xFDF0, 0xFDFB, 0xFE70, 0xFE74, 0xFE76, 0xFEFC, 0xFF10, 0xFF19, 0xFF21, 0xFF3A, 0xFF41, 0xFF5A,
0xFF66, 0xFFBE, 0xFFC2, 0xFFC7, 0xFFCA, 0xFFCF, 0xFFD2, 0xFFD7, 0xFFDA, 0xFFDC, 0x10000, 0x1000B, 0x1000D,
0x10026, 0x10028, 0x1003A, 0x1003C, 0x1003D, 0x1003F, 0x1004D, 0x10050, 0x1005D, 0x10080, 0x100FA, 0x10107,
0x10133, 0x10140, 0x10178, 0x1018A, 0x1018B, 0x10280, 0x1029C, 0x102A0, 0x102D0, 0x102E1, 0x102FB, 0x10300,
0x10323, 0x1032D, 0x1034A, 0x10350, 0x10375, 0x10380, 0x1039D, 0x103A0, 0x103C3, 0x103C8, 0x103CF, 0x103D1,
0x103D5, 0x10400, 0x1049D, 0x104A0, 0x104A9, 0x104B0, 0x104D3, 0x104D8, 0x104FB, 0x10500, 0x10527, 0x10530,
0x10563, 0x10570, 0x1057A, 0x1057C, 0x1058A, 0x1058C, 0x10592, 0x10594, 0x10595, 0x10597, 0x105A1, 0x105A3,
0x105B1, 0x105B3, 0x105B9, 0x105BB, 0x105BC, 0x105C0, 0x105F3, 0x10600, 0x10736, 0x10740, 0x10755, 0x10760,
0x10767, 0x10780, 0x10785, 0x10787, 0x107B0, 0x107B2, 0x107BA, 0x10800, 0x10805, 0x10808, 0x10808, 0x1080A,
0x10835, 0x10837, 0x10838, 0x1083C, 0x1083C, 0x1083F, 0x10855, 0x10858, 0x10876, 0x10879, 0x1089E, 0x108A7,
0x108AF, 0x108E0, 0x108F2, 0x108F4, 0x108F5, 0x108FB, 0x1091B, 0x10920, 0x10939, 0x10980, 0x109B7, 0x109BC,
0x109CF, 0x109D2, 0x10A00, 0x10A10, 0x10A13, 0x10A15, 0x10A17, 0x10A19, 0x10A35, 0x10A40, 0x10A48, 0x10A60,
0x10A7E, 0x10A80, 0x10A9F, 0x10AC0, 0x10AC7, 0x10AC9, 0x10AE4, 0x10AEB, 0x10AEF, 0x10B00, 0x10B35, 0x10B40,
0x10B55, 0x10B58, 0x10B72, 0x10B78, 0x10B91, 0x10BA9, 0x10BAF, 0x10C00, 0x10C48, 0x10C80, 0x10CB2, 0x10CC0,
0x10CF2, 0x10CFA, 0x10D23, 0x10D30, 0x10D39, 0x10D40, 0x10D65, 0x10D6F, 0x10D85, 0x10E60, 0x10E7E, 0x10E80,
0x10EA9, 0x10EB0, 0x10EB1, 0x10EC2, 0x10EC4, 0x10F00, 0x10F27, 0x10F30, 0x10F45, 0x10F51, 0x10F54, 0x10F70,
0x10F81, 0x10FB0, 0x10FCB, 0x10FE0, 0x10FF6, 0x11003, 0x11037, 0x11052, 0x1106F, 0x11071, 0x11072, 0x11075,
0x11075, 0x11083, 0x110AF, 0x110D0, 0x110E8, 0x110F0, 0x110F9, 0x11103, 0x11126, 0x11136, 0x1113F, 0x11144,
0x11144, 0x11147, 0x11147, 0x11150, 0x11172, 0x11176, 0x11176, 0x11183, 0x111B2, 0x111C1, 0x111C4, 0x111D0,
0x111DA, 0x111DC, 0x111DC, 0x111E1, 0x111F4, 0x11200, 0x11211, 0x11213, 0x1122B, 0x1123F, 0x11240, 0x11280,
0x11286, 0x11288, 0x11288, 0x1128A, 0x1128D, 0x1128F, 0x1129D, 0x1129F, 0x112A8, 0x112B0, 0x112DE, 0x112F0,
0x112F9, 0x11305, 0x1130C, 0x1130F, 0x11310, 0x11313, 0x11328, 0x1132A, 0x11330, 0x11332, 0x11333, 0x11335,
0x11339, 0x1133D, 0x1133D, 0x11350, 0x11350, 0x1135D, 0x11361, 0x11380, 0x11389, 0x1138B, 0x1138B, 0x1138E,
0x1138E, 0x11390, 0x113B5, 0x113B7, 0x113B7, 0x113D1, 0x113D1, 0x113D3, 0x113D3, 0x11400, 0x11434, 0x11447,
0x1144A, 0x11450, 0x11459, 0x1145F, 0x11461, 0x11480, 0x114AF, 0x114C4, 0x114C5, 0x114C7, 0x114C7, 0x114D0,
0x114D9, 0x11580, 0x115AE, 0x115D8, 0x115DB, 0x11600, 0x1162F, 0x11644, 0x11644, 0x11650, 0x11659, 0x11680,
0x116AA, 0x116B8, 0x116B8, 0x116C0, 0x116C9, 0x116D0, 0x116E3, 0x11700, 0x1171A, 0x11730, 0x1173B, 0x11740,
0x11746, 0x11800, 0x1182B, 0x118A0, 0x118F2, 0x118FF, 0x11906, 0x11909, 0x11909, 0x1190C, 0x11913, 0x11915,
0x11916, 0x11918, 0x1192F, 0x1193F, 0x1193F, 0x11941, 0x11941, 0x11950, 0x11959, 0x119A0, 0x119A7, 0x119AA,
0x119D0, 0x119E1, 0x119E1, 0x119E3, 0x119E3, 0x11A00, 0x11A00, 0x11A0B, 0x11A32, 0x11A3A, 0x11A3A, 0x11A50,
0x11A50, 0x11A5C, 0x11A89, 0x11A9D, 0x11A9D, 0x11AB0, 0x11AF8, 0x11BC0, 0x11BE0, 0x11BF0, 0x11BF9, 0x11C00,
0x11C08, 0x11C0A, 0x11C2E, 0x11C40, 0x11C40, 0x11C50, 0x11C6C, 0x11C72, 0x11C8F, 0x11D00, 0x11D06, 0x11D08,
0x11D09, 0x11D0B, 0x11D30, 0x11D46, 0x11D46, 0x11D50, 0x11D59, 0x11D60, 0x11D65, 0x11D67, 0x11D68, 0x11D6A,
0x11D89, 0x11D98, 0x11D98, 0x11DA0, 0x11DA9, 0x11EE0, 0x11EF2, 0x11F02, 0x11F02, 0x11F04, 0x11F10, 0x11F12,
0x11F33, 0x11F50, 0x11F59, 0x11FB0, 0x11FB0, 0x11FC0, 0x11FD4, 0x12000, 0x12399, 0x12400, 0x1246E, 0x12480,
0x12543, 0x12F90, 0x12FF0, 0x13000, 0x1342F, 0x13441, 0x13446, 0x13460, 0x143FA, 0x14400, 0x14646, 0x16100,
0x1611D, 0x16130, 0x16139, 0x16800, 0x16A38, 0x16A40, 0x16A5E, 0x16A60, 0x16A69, 0x16A70, 0x16ABE, 0x16AC0,
0x16AC9, 0x16AD0, 0x16AED, 0x16B00, 0x16B2F, 0x16B40, 0x16B43, 0x16B50, 0x16B59, 0x16B5B, 0x16B61, 0x16B63,
0x16B77, 0x16B7D, 0x16B8F, 0x16D40, 0x16D6C, 0x16D70, 0x16D79, 0x16E40, 0x16E96, 0x16F00, 0x16F4A, 0x16F50,
0x16F50, 0x16F93, 0x16F9F, 0x16FE0, 0x16FE1, 0x16FE3, 0x16FE3, 0x17000, 0x187F7, 0x18800, 0x18CD5, 0x18CFF,
0x18D08, 0x1AFF0, 0x1AFF3, 0x1AFF5, 0x1AFFB, 0x1AFFD, 0x1AFFE, 0x1B000, 0x1B122, 0x1B132, 0x1B132, 0x1B150,
0x1B152, 0x1B155, 0x1B155, 0x1B164, 0x1B167, 0x1B170, 0x1B2FB, 0x1BC00, 0x1BC6A, 0x1BC70, 0x1BC7C, 0x1BC80,
0x1BC88, 0x1BC90, 0x1BC99, 0x1CCF0, 0x1CCF9, 0x1D2C0, 0x1D2D3, 0x1D2E0, 0x1D2F3, 0x1D360, 0x1D378, 0x1D400,
0x1D454, 0x1D456, 0x1D49C, 0x1D49E, 0x1D49F, 0x1D4A2, 0x1D4A2, 0x1D4A5, 0x1D4A6, 0x1D4A9, 0x1D4AC, 0x1D4AE,
0x1D4B9, 0x1D4BB, 0x1D4BB, 0x1D4BD, 0x1D4C3, 0x1D4C5, 0x1D505, 0x1D507, 0x1D50A, 0x1D50D, 0x1D514, 0x1D516,
0x1D51C, 0x1D51E, 0x1D539, 0x1D53B, 0x1D53E, 0x1D540, 0x1D544, 0x1D546, 0x1D546, 0x1D54A, 0x1D550, 0x1D552,
0x1D6A5, 0x1D6A8, 0x1D6C0, 0x1D6C2, 0x1D6DA, 0x1D6DC, 0x1D6FA, 0x1D6FC, 0x1D714, 0x1D716, 0x1D734, 0x1D736,
0x1D74E, 0x1D750, 0x1D76E, 0x1D770, 0x1D788, 0x1D78A, 0x1D7A8, 0x1D7AA, 0x1D7C2, 0x1D7C4, 0x1D7CB, 0x1D7CE,
0x1D7FF, 0x1DF00, 0x1DF1E, 0x1DF25, 0x1DF2A, 0x1E030, 0x1E06D, 0x1E100, 0x1E12C, 0x1E137, 0x1E13D, 0x1E140,
0x1E149, 0x1E14E, 0x1E14E, 0x1E290, 0x1E2AD, 0x1E2C0, 0x1E2EB, 0x1E2F0, 0x1E2F9, 0x1E4D0, 0x1E4EB, 0x1E4F0,
0x1E4F9, 0x1E5D0, 0x1E5ED, 0x1E5F0, 0x1E5FA, 0x1E7E0, 0x1E7E6, 0x1E7E8, 0x1E7EB, 0x1E7ED, 0x1E7EE, 0x1E7F0,
0x1E7FE, 0x1E800, 0x1E8C4, 0x1E8C7, 0x1E8CF, 0x1E900, 0x1E943, 0x1E94B, 0x1E94B, 0x1E950, 0x1E959, 0x1EC71,
0x1ECAB, 0x1ECAD, 0x1ECAF, 0x1ECB1, 0x1ECB4, 0x1ED01, 0x1ED2D, 0x1ED2F, 0x1ED3D, 0x1EE00, 0x1EE03, 0x1EE05,
0x1EE1F, 0x1EE21, 0x1EE22, 0x1EE24, 0x1EE24, 0x1EE27, 0x1EE27, 0x1EE29, 0x1EE32, 0x1EE34, 0x1EE37, 0x1EE39,
0x1EE39, 0x1EE3B, 0x1EE3B, 0x1EE42, 0x1EE42, 0x1EE47, 0x1EE47, 0x1EE49, 0x1EE49, 0x1EE4B, 0x1EE4B, 0x1EE4D,
0x1EE4F, 0x1EE51, 0x1EE52, 0x1EE54, 0x1EE54, 0x1EE57, 0x1EE57, 0x1EE59, 0x1EE59, 0x1EE5B, 0x1EE5B, 0x1EE5D,
0x1EE5D, 0x1EE5F, 0x1EE5F, 0x1EE61, 0x1EE62, 0x1EE64, 0x1EE64, 0x1EE67, 0x1EE6A, 0x1EE6C, 0x1EE72, 0x1EE74,
0x1EE77, 0x1EE79, 0x1EE7C, 0x1EE7E, 0x1EE7E, 0x1EE80, 0x1EE89, 0x1EE8B, 0x1EE9B, 0x1EEA1, 0x1EEA3, 0x1EEA5,
0x1EEA9, 0x1EEAB, 0x1EEBB, 0x1F100, 0x1F10C, 0x1FBF0, 0x1FBF9, 0x20000, 0x2A6DF, 0x2A700, 0x2B739, 0x2B740,
0x2B81D, 0x2B820, 0x2CEA1, 0x2CEB0, 0x2EBE0, 0x2EBF0, 0x2EE5D, 0x2F800, 0x2FA1D, 0x30000, 0x3134A, 0x31350,
0x323AF,
)
internal val DEFAULT_IGNORABLE: IntArray = intArrayOf(
0xAD, 0xAD, 0x34F, 0x34F, 0x61C, 0x61C, 0x115F, 0x1160, 0x17B4, 0x17B5, 0x180B, 0x180F, 0x200B, 0x200F, 0x202A,
0x202E, 0x2060, 0x206F, 0x3164, 0x3164, 0xFE00, 0xFE0F, 0xFEFF, 0xFEFF, 0xFFA0, 0xFFA0, 0xFFF0, 0xFFF8, 0x1BCA0,
0x1BCA3, 0x1D173, 0x1D17A, 0xE0000, 0xE0FFF,
)

View file

@ -0,0 +1,286 @@
package org.gamecult.cultmesh
// The write receipt (contracts/cultnet/interop/write-receipt.md): the two frames, their strict decode, the dedupe
// content hash, the hello capability test and the client settle rules. Mirrors packages/cultnet-ts (contracts.ts,
// write-tracker.ts) and packages/cultnet-rs (write_content_hash). The replay of write-receipt-vectors.json lives in
// WriteReceiptVectors.kt. Server rules (dedupe, limits, blank fields) are not a decoder's business and are not here.
import java.io.IOException
import java.security.MessageDigest
const val WRITE_ADMITTED_VERSION = "cultnet.write_admitted.v0"
const val ERROR_V1_VERSION = "cultnet.error.v1"
const val ERROR_V0_VERSION = "cultnet.error.v0"
/** A connection is receipt-capable only when its hello lists both frames. */
val CULTNET_WRITE_RECEIPT_MESSAGE_VERSIONS: List<String> = listOf(WRITE_ADMITTED_VERSION, ERROR_V1_VERSION)
const val RECEIPT_RESULT_WINDOW_MS = 5000L
const val LEGACY_RESULT_WINDOW_MS = 500L
fun resultWindowMs(serverAdvertisesReceipts: Boolean): Long =
if (serverAdvertisesReceipts) RECEIPT_RESULT_WINDOW_MS else LEGACY_RESULT_WINDOW_MS
/** The closed refusal-code set of error.v0; error.v1 adds `limit_reached`. */
val CULTNET_ERROR_V0_CODES: Set<String> = setOf(
"selection_invalid", "cursor_stale", "cursor_invalid", "reference_outside_target", "variant_unsupported",
"unowned_schema", "write_not_authorized", "unsupported_schema_version", "message_invalid",
)
val CULTNET_ERROR_V1_CODES: Set<String> = CULTNET_ERROR_V0_CODES + "limit_reached"
fun helloAdvertisesWriteReceipts(hello: CultNetMessage): Boolean {
val versions = hello.body["supportedMessageVersions"] as? List<*> ?: return false
return CULTNET_WRITE_RECEIPT_MESSAGE_VERSIONS.all { it in versions }
}
data class CultNetWriteAdmitted(val messageId: String, val operation: String, val schemaId: String, val recordKey: String) {
fun toMessage() = CultNetMessage(
WRITE_ADMITTED_VERSION,
linkedMapOf("messageId" to messageId, "operation" to operation, "schemaId" to schemaId, "recordKey" to recordKey),
)
}
data class CultNetErrorDetails(val field: String?, val value: String?, val asOf: Long?, val current: Long?)
/** error.v0 (`messageId` always null, `version` 0) or error.v1 (`version` 1). */
data class CultNetErrorFrame(
val version: Int,
val error: String,
val routingHint: Map<String, Any?>?,
val code: String?,
val details: CultNetErrorDetails?,
val messageId: String?,
) {
fun toMessage(): CultNetMessage {
val body = linkedMapOf<String, Any?>(
"error" to error,
"routingHint" to routingHint,
"code" to code,
"details" to details?.let {
linkedMapOf("field" to it.field, "value" to it.value, "asOf" to it.asOf, "current" to it.current)
},
)
if (version == 1) body["messageId"] = messageId
return CultNetMessage(if (version == 1) ERROR_V1_VERSION else ERROR_V0_VERSION, body)
}
}
/**
* Strict decode of a write_admitted.v0 message against its closed schema: exactly five keys, a non-empty string each,
* operation put|delete.
*/
fun decodeWriteAdmitted(message: CultNetMessage): CultNetWriteAdmitted {
if (message.schemaVersion != WRITE_ADMITTED_VERSION) throw IOException("not a $WRITE_ADMITTED_VERSION message")
closedKeys(message, setOf("messageId", "operation", "schemaId", "recordKey"))
val operation = nonEmpty(message, "operation")
if (operation != "put" && operation != "delete") throw IOException("operation must be put or delete")
return CultNetWriteAdmitted(admittedMessageId(message), operation, nonEmpty(message, "schemaId"), nonEmpty(message, "recordKey"))
}
// write-receipt.md: a letter (Lu Ll Lt Lm Lo) or number (Nd Nl No) somewhere, no Default_Ignorable code point anywhere, as of
// Unicode 16.0, compared raw (never trimmed). The tables are embedded (UnicodeMessageIdTables.kt, generated by
// scripts/gen-unicode-messageid-tables.py), not the JDK's, so the running JVM's Unicode version changes no result; the other
// four runtimes embed the same generated tables.
private fun inRanges(ranges: IntArray, point: Int): Boolean {
var low = 0
var high = ranges.size / 2 - 1
while (low <= high) {
val mid = (low + high) ushr 1
when {
point < ranges[2 * mid] -> high = mid - 1
point > ranges[2 * mid + 1] -> low = mid + 1
else -> return true
}
}
return false
}
private fun admittedMessageId(message: CultNetMessage): String {
val id = nonEmpty(message, "messageId")
var hasLetterOrNumber = false
id.codePoints().forEach { cp ->
if (inRanges(DEFAULT_IGNORABLE, cp)) throw IOException("messageId holds a Default_Ignorable code point")
if (inRanges(LETTER_OR_NUMBER, cp)) hasLetterOrNumber = true
}
if (!hasLetterOrNumber) throw IOException("messageId must contain a letter or number")
return id
}
/** Strict decode of error.v0 (no messageId, no limit_reached) or error.v1 (messageId required, string or nil, never empty). */
fun decodeErrorFrame(message: CultNetMessage): CultNetErrorFrame {
val version = when (message.schemaVersion) {
ERROR_V1_VERSION -> 1
ERROR_V0_VERSION -> 0
else -> throw IOException("not an error message")
}
closedKeys(message, setOf("error", "routingHint", "code", "details") + if (version == 1) setOf("messageId") else emptySet())
val error = nonEmpty(message, "error")
val codeValue = message.body["code"]
if (codeValue != null && (codeValue !is String || codeValue !in (if (version == 1) CULTNET_ERROR_V1_CODES else CULTNET_ERROR_V0_CODES))) {
throw IOException("unknown error code")
}
var messageId: String? = null
if (version == 1) {
val id = message.body["messageId"]
if (id != null && (id !is String || id.isEmpty())) throw IOException("messageId must be a non-empty string or nil")
messageId = id as String?
}
return CultNetErrorFrame(version, error, routingHintOf(message.body["routingHint"]), codeValue as String?, detailsOf(message.body["details"]), messageId)
}
private fun closedKeys(message: CultNetMessage, required: Set<String>) {
if (!message.body.keys.containsAll(required)) throw IOException("${message.schemaVersion} is missing ${required - message.body.keys}")
val unknown = message.body.keys - required
if (unknown.isNotEmpty()) throw IOException("${message.schemaVersion} carries unknown keys $unknown")
}
private fun nonEmpty(message: CultNetMessage, key: String): String {
val value = message.body[key]
if (value !is String || value.isEmpty()) throw IOException("$key must be a non-empty string")
return value
}
private fun routingHintOf(value: Any?): Map<String, Any?>? {
if (value == null) return null
if (value !is Map<*, *> || value.keys != setOf("shard", "reason")) throw IOException("routingHint must be nil or {shard, reason}")
if (value["shard"] != null && value["shard"] !is Map<*, *>) throw IOException("routingHint.shard must be an object or nil")
if (value["reason"] !is String) throw IOException("routingHint.reason must be a string")
@Suppress("UNCHECKED_CAST")
return value as Map<String, Any?>
}
private fun detailsOf(value: Any?): CultNetErrorDetails? {
if (value == null) return null
if (value !is Map<*, *> || value.keys != setOf("field", "value", "asOf", "current")) {
throw IOException("details must be nil or {field, value, asOf, current}")
}
fun text(key: String): String? = value[key].let { if (it == null || it is String) it as String? else throw IOException("details.$key must be a string or nil") }
fun cursor(key: String): Long? = value[key].let {
if (it == null) null else if (it is Long && it >= 0) it else throw IOException("details.$key must be a non-negative integer or nil")
}
return CultNetErrorDetails(text("field"), text("value"), cursor("asOf"), cursor("current"))
}
/** sha256 of the MessagePack array [operation, schemaId, recordKey, payload]: payload a bin for a put, nil for a delete. Lowercase hex. */
fun writeContentHash(operation: String, schemaId: String, recordKey: String, payload: ByteArray?): String {
val writer = MessagePackWriter().array(4).string(operation).string(schemaId).string(recordKey)
if (payload == null) writer.nil() else writer.binary(payload)
return MessageDigest.getInstance("SHA-256").digest(writer.toByteArray()).joinToString("") { "%02x".format(it) }
}
/** Why nothing settled a write. */
enum class WriteUnconfirmedReason(val wire: String) { WINDOW_ELAPSED("window_elapsed"), SESSION_ENDED("session_ended"), LOCAL_SEND_FAILED("local_send_failed") }
/** What a late answer is recorded as on a write that already settled unconfirmed. */
enum class WriteRecorded(val wire: String) { ADMITTED_LATE("admitted_late"), REFUSED_LATE("refused_late") }
/** What the caller was told. Settled once; never changed by a later frame. */
sealed class WriteSettlement {
object Admitted : WriteSettlement()
data class Refused(val code: String?, val error: String, val response: CultNetErrorFrame) : WriteSettlement()
data class Unconfirmed(val reason: WriteUnconfirmedReason) : WriteSettlement()
}
/**
* The client half of the receipt rules: a pure state machine with an abstract clock, one decision, how each sent write
* settles. Transport, timers and decoding belong to the caller; `frame` takes a message and decodes it strictly.
*/
class WriteTracker(private val windowMs: Long, private val onSettle: (String, WriteSettlement) -> Unit = { _, _ -> }) {
private class Entry(val sentAtMs: Long) {
var settlement: WriteSettlement? = null
var recorded: WriteRecorded? = null
}
private val writes = linkedMapOf<String, Entry>()
private var ended = false
/** Registers a write sent at [atMs]. Re-sending a known id is the same write and does not restart its window. */
fun send(messageId: String, atMs: Long) {
tick(atMs)
if (messageId in writes) return
val entry = Entry(atMs)
writes[messageId] = entry
// On a session that already ended nothing can answer this write.
if (ended) settle(messageId, entry, WriteSettlement.Unconfirmed(WriteUnconfirmedReason.SESSION_ENDED))
}
/** Feeds one received message. Only write_admitted, error.v1 and error.v0 answer; anything else is ignored. */
fun frame(message: CultNetMessage, atMs: Long) {
tick(atMs)
when (message.schemaVersion) {
WRITE_ADMITTED_VERSION -> {
answer(decodeWriteAdmitted(message).messageId, WriteSettlement.Admitted, WriteRecorded.ADMITTED_LATE)
return
}
ERROR_V1_VERSION, ERROR_V0_VERSION -> {
val error = decodeErrorFrame(message)
val refusal = WriteSettlement.Refused(error.code, error.error, error)
if (error.messageId != null) {
answer(error.messageId, refusal, WriteRecorded.REFUSED_LATE)
return
}
// An error that names no write is attributable only when exactly one write is unanswered.
val unanswered = writes.filterValues { it.settlement == null }
if (unanswered.size == 1) settle(unanswered.keys.first(), unanswered.values.first(), refusal)
}
else -> return
}
}
/** Every unanswered write whose window, counted from its own send, has run settles unconfirmed. */
fun tick(atMs: Long) {
for ((id, entry) in writes) {
if (entry.settlement == null && atMs - entry.sentAtMs >= windowMs) settle(id, entry, WriteSettlement.Unconfirmed(WriteUnconfirmedReason.WINDOW_ELAPSED))
}
}
/** The session ended: every unanswered write settles unconfirmed, and so does any write sent afterwards. */
fun sessionEnd(atMs: Long) {
tick(atMs)
ended = true
for ((id, entry) in writes) if (entry.settlement == null) settle(id, entry, WriteSettlement.Unconfirmed(WriteUnconfirmedReason.SESSION_ENDED))
}
/** This client's own send of [messageId] failed: that write settles unconfirmed; the others are untouched. */
fun sendFailed(messageId: String, atMs: Long) {
val entry = writes[messageId]
if (entry != null && entry.settlement == null) settle(messageId, entry, WriteSettlement.Unconfirmed(WriteUnconfirmedReason.LOCAL_SEND_FAILED))
tick(atMs)
}
/** The settlement told to the caller, or null while unanswered or unknown. */
fun settlement(messageId: String): WriteSettlement? = writes[messageId]?.settlement
/** The late answer recorded on an unconfirmed write. */
fun recorded(messageId: String): WriteRecorded? = writes[messageId]?.recorded
private fun answer(messageId: String, settlement: WriteSettlement, late: WriteRecorded) {
val entry = writes[messageId] ?: return // names a write that is not in flight: settles nothing
val current = entry.settlement
if (current == null) settle(messageId, entry, settlement)
else if (current is WriteSettlement.Unconfirmed && entry.recorded == null) entry.recorded = late
}
private fun settle(messageId: String, entry: Entry, settlement: WriteSettlement) {
entry.settlement = settlement
onSettle(messageId, settlement)
}
}
/** Unicode White_Space (PropList), the one definition of "blank" in write-receipt.md rule 5; Kotlin's own isBlank takes U+001C to U+001F. */
private val WHITE_SPACE_CODE_POINTS: Set<Int> = (0x0009..0x000D).toSet() + 0x0020 + 0x0085 + 0x00A0 + 0x1680 +
(0x2000..0x200A).toSet() + 0x2028 + 0x2029 + 0x202F + 0x205F + 0x3000
/** True for null, empty, or only White_Space code points. */
fun isBlankText(text: String?): Boolean = text == null || text.codePoints().allMatch { it in WHITE_SPACE_CODE_POINTS }
/** Self-test: blank is White_Space, not Kotlin's isBlank (rule 5): U+001C to U+001F name something, U+00A0 and U+0085 do not. */
fun rawDocumentRecordDropsOnlyWhiteSpaceSourceIds() {
fun wireKeys(id: String) = CultNetRawDocumentRecord("note.v0", "note:1", "2026-11-08T00:00:00Z", payload = byteArrayOf(1), sourceRuntimeId = id, sourceAgentId = id, sourceRole = id).toWireMap().keys
for (blank in listOf("", "\u0085", "\u00a0\u2003", "\u3000", "\t\r\n ")) {
check(isBlankText(blank) && "sourceRuntimeId" !in wireKeys(blank)) { "blank ${blank.map { it.code }} must be dropped" }
}
for (named in listOf("\u001c", "\u001f", "\u200b", "\ufeff", "\u180e", "a")) {
check(!isBlankText(named) && "sourceRuntimeId" in wireKeys(named) && "sourceRole" in wireKeys(named)) { "${named.map { it.code }} names something and must be kept" }
}
}

View file

@ -0,0 +1,310 @@
package org.gamecult.cultmesh
// Replays contracts/cultnet/interop/write-receipt-vectors.json against this runtime: constants, contentHash, every
// `wire` frame (decode to the vector's fields, re-encode to its bytes), every `schemaInvalid` frame (must be refused),
// and every `scenarios` event replay through WriteTracker. `dedupe` and `blankFields` are server inputs and are not
// run here. Mode: `java -cp ... org.gamecult.cultmesh.CultMeshKt write-receipt-vectors <vectors.json>`.
import java.io.File
import java.io.IOException
// The event actions scenarios.semantics names; anything else is refused, not skipped.
private val EVENT_KEYS = listOf("send", "frame", "tick", "sessionEnd", "sendFailed", "transportAck")
fun replayWriteReceiptVectors(path: String): Boolean {
@Suppress("UNCHECKED_CAST")
val vectors = JsonReader(File(path).readText(Charsets.UTF_8)).readDocument() as Map<String, Any?>
val results = linkedMapOf<String, List<String>>()
fun check(name: String, run: () -> List<String>) {
results[name] = try { run() } catch (e: Throwable) { listOf("threw: $e") }
}
val constants = obj(vectors["constants"])
check("constants") {
listOfNotNull(
"RECEIPT window".takeIf { constants["RECEIPT_RESULT_WINDOW_MS"] != RECEIPT_RESULT_WINDOW_MS },
"LEGACY window".takeIf { constants["LEGACY_RESULT_WINDOW_MS"] != LEGACY_RESULT_WINDOW_MS },
)
}
for (case in list(obj(vectors["contentHash"])["cases"])) {
val c = obj(case)
check("contentHash/${c["name"]}") {
val got = writeContentHash(c["operation"] as String, c["schemaId"] as String, c["recordKey"] as String, (c["payloadHex"] as String?)?.let(::unhex))
if (got == c["sha256"]) emptyList() else listOf("$got != ${c["sha256"]}")
}
}
for (case in list(vectors["wire"])) {
val c = obj(case)
check("wire/${c["name"]}") {
val bytes = unhex(c["messagePackHex"] as String)
val problems = mutableListOf<String>()
val message = decodeReceiptFrame(parseCultNetMessage(bytes))
if (message.toWireMap() != c["message"]) problems += "decoded ${message.toWireMap()} != ${c["message"]}"
val again = message.toBytes().joinToString("") { "%02X".format(it) }
if (again != (c["messagePackHex"] as String).uppercase()) problems += "re-encoded bytes $again != captured ${c["messagePackHex"]}"
problems
}
}
// admittedMessageId: a write_admitted messageId needs a letter or number and no Default_Ignorable code point, compared
// raw. The frame is built with this runtime's own encoder; a case with `wireHex` (the lone surrogate, invalid UTF-8)
// supplies the whole frame.
for (case in list(obj(vectors["admittedMessageId"])["cases"])) {
val c = obj(case)
check("admittedMessageId/${c["name"]}") {
val id = list(c["codePoints"]).joinToString("") { String(Character.toChars((it as String).toInt(16))) }.repeat(((c["repeat"] as Long?) ?: 1L).toInt())
val bytes = (c["wireHex"] as String?)?.let(::unhex) ?: MessagePackWriter().value(linkedMapOf(
"schemaVersion" to WRITE_ADMITTED_VERSION, "messageId" to id, "operation" to "put", "schemaId" to "note.v0", "recordKey" to "note:1",
)).toByteArray()
val decoded = try { decodeReceiptFrame(parseCultNetMessage(bytes)) } catch (_: IOException) { null }
when {
c["expect"] == "refuse" -> if (decoded == null) emptyList() else listOf("accepted an id that must be refused (${c["why"]})")
decoded == null -> listOf("refused an id that must be accepted (${c["why"]})")
decoded.toWireMap()["messageId"] != id -> listOf("id not echoed raw: ${decoded.toWireMap()["messageId"]}")
else -> emptyList()
}
}
}
// The generated ranges file is the independent copy of the embedded tables: first, last and the points either side of every
// range, so deleting or narrowing any range fails here.
check("unicodeTableRanges") {
@Suppress("UNCHECKED_CAST")
val ranges = JsonReader(File(File(path).parentFile, "write-admitted-message-id-ranges.json").readText(Charsets.UTF_8)).readDocument() as Map<String, Any?>
fun pairs(key: String) = list(ranges[key]).map { list(it).map { n -> (n as Long).toInt() } }
val letters = pairs("letterOrNumber")
val ignorable = pairs("defaultIgnorable")
val problems = mutableListOf<String>()
if (letters.size <= 700 || ignorable.size != 17) problems += "ranges file holds ${letters.size} and ${ignorable.size} ranges"
fun accepted(id: String): Boolean = try {
decodeReceiptFrame(parseCultNetMessage(MessagePackWriter().value(linkedMapOf(
"schemaVersion" to WRITE_ADMITTED_VERSION, "messageId" to id, "operation" to "put", "schemaId" to "note.v0", "recordKey" to "note:1",
)).toByteArray()))
true
} catch (_: IOException) { false }
for ((low, high) in letters) for ((point, want) in listOf(low to true, high to true, low - 1 to false, high + 1 to false)) {
if (accepted(String(Character.toChars(point))) != want) problems += "U+${point.toString(16)} alone should be ${if (want) "accepted" else "refused"}"
}
for ((low, high) in ignorable) for ((point, want) in listOf(low to false, high to false, low - 1 to true, high + 1 to true)) {
if (accepted("a" + String(Character.toChars(point))) != want) problems += "a + U+${point.toString(16)} should be ${if (want) "accepted" else "refused"}"
}
problems
}
// A document payload string with a lone surrogate (ED A0 80) is not a write_admitted id: the shared MessagePack reader keeps
// reading it. Strictness is scoped to the write_admitted frame, never to the reader.
check("documentPayloadStringWithLoneSurrogateStillDecodes") {
val text = MessagePackReader(unhex("A3EDA080")).readString()
if (text.isNotEmpty()) emptyList() else listOf("decoded to an empty string")
}
// admittedMessageIdDuplicates: Kotlin decodes the frame to a map, so it cannot see a repeated key: it judges the last one.
for (case in list(obj(vectors["admittedMessageIdDuplicates"])["cases"])) {
val c = obj(case)
check("admittedMessageIdDuplicates/${c["name"]}") {
val decoded = try { decodeReceiptFrame(parseCultNetMessage(unhex(c["wireHex"] as String))) } catch (_: IOException) { null }
when {
c["lastKeyWins"] == "refuse" -> if (decoded == null) emptyList() else listOf("accepted a frame whose last messageId key is invalid")
decoded == null -> listOf("refused a frame whose last messageId key is valid")
decoded.toWireMap()["messageId"] != c["lastKeyWinsEcho"] -> listOf("echoed ${decoded.toWireMap()["messageId"]}, wanted ${c["lastKeyWinsEcho"]}")
else -> emptyList()
}
}
}
for (case in list(vectors["schemaInvalid"])) {
val c = obj(case)
check("schemaInvalid/${c["name"]}") {
try {
decodeReceiptFrame(parseCultNetMessage(unhex(c["messagePackHex"] as String)))
listOf("accepted a frame that must be refused (${c["why"]})")
} catch (_: IOException) {
emptyList()
}
}
}
// Not a vector: `limit_reached` is in error.v1's enum only (write-receipt.md), as the TS and Rust suites also pin.
check("error_v0_refuses_the_v1_only_limit_reached_code") {
val body = linkedMapOf<String, Any?>("error" to "x", "routingHint" to null, "code" to "limit_reached", "details" to null)
val v1 = decodeErrorFrame(CultNetMessage(ERROR_V1_VERSION, body + ("messageId" to null))).code
try {
decodeErrorFrame(CultNetMessage(ERROR_V0_VERSION, body))
listOf("error.v0 accepted limit_reached")
} catch (_: IOException) {
if (v1 == "limit_reached") emptyList() else listOf("error.v1 lost limit_reached")
}
}
for (case in list(obj(vectors["scenarios"])["cases"])) {
val scenario = obj(case)
check("scenario/${scenario["name"]}") {
val problems = mutableListOf<String>()
// The fake server states its capability the only way a real one does: in its hello.
val hello = cultNetHello(
"fake-server", "fake",
supportedMessageVersions = if (scenario["server"] == "receipt") CULTNET_WRITE_RECEIPT_MESSAGE_VERSIONS else emptyList(),
)
val windowMs = resultWindowMs(helloAdvertisesWriteReceipts(hello))
if (windowMs != constants[scenario["window"] as String]) problems += "window $windowMs != ${scenario["window"]}"
val told = linkedMapOf<String, WriteSettlement>()
val tracker = WriteTracker(windowMs) { id, settlement ->
if (id in told) problems += "$id settled twice"
told[id] = settlement
}
for (event in list(scenario["events"]).map(::obj).sortedBy { it["atMs"] as Long }) {
val at = event["atMs"] as Long
val actions = event.keys - "atMs"
if (actions.size != 1 || actions.first() !in EVENT_KEYS) { problems += "event at $at has keys $actions, wanted atMs and exactly one of $EVENT_KEYS"; continue }
when {
"send" in event -> tracker.send(event["send"] as String, at)
// A frame reaches the client as bytes: the fake server encodes it, the client decodes it.
"frame" in event -> tracker.frame(parseCultNetMessage(MessagePackWriter().value(event["frame"]).toByteArray()), at)
"tick" in event -> tracker.tick(at)
"sessionEnd" in event -> tracker.sessionEnd(at)
"sendFailed" in event -> tracker.sendFailed(event["sendFailed"] as String, at)
// transportAck: delivery is not an answer; the tracker has no such input and must not need one.
}
}
val expect = obj(scenario["expect"])
for ((id, wantAny) in expect) {
val want = obj(wantAny)
val got = told[id]
if (got == null) { problems += "$id: never settled, wanted ${want["outcome"]}"; continue }
val (outcome, code, reason) = when (got) {
is WriteSettlement.Admitted -> Triple("admitted", null, null)
is WriteSettlement.Refused -> Triple("refused", got.code, null)
is WriteSettlement.Unconfirmed -> Triple("unconfirmed", null, got.reason.wire)
}
if (outcome != want["outcome"]) problems += "$id: told $outcome, wanted ${want["outcome"]}"
if ("code" in want && code != want["code"]) problems += "$id: code $code, wanted ${want["code"]}"
if ("reason" in want && reason != want["reason"]) problems += "$id: reason $reason, wanted ${want["reason"]}"
if (tracker.recorded(id)?.wire != want["recorded"]) problems += "$id: recorded ${tracker.recorded(id)?.wire}, wanted ${want["recorded"]}"
}
for (id in told.keys) if (id !in expect) problems += "$id: settled but the scenario expects nothing for it"
problems
}
}
// A runner that read fewer vectors than the file declares has silently skipped some.
check("counts") {
val declared = obj(vectors["counts"])
fun ran(prefix: String) = results.keys.count { it.startsWith(prefix) }
val loaded = mapOf(
"wire" to list(vectors["wire"]).size, "schemaInvalid" to list(vectors["schemaInvalid"]).size,
"contentHashCases" to list(obj(vectors["contentHash"])["cases"]).size, "scenarioCases" to list(obj(vectors["scenarios"])["cases"]).size,
"admittedMessageIdCases" to list(obj(vectors["admittedMessageId"])["cases"]).size,
)
val executed = mapOf("wire" to ran("wire/"), "schemaInvalid" to ran("schemaInvalid/"), "contentHashCases" to ran("contentHash/"), "scenarioCases" to ran("scenario/"), "admittedMessageIdCases" to ran("admittedMessageId/"))
loaded.keys.filter { loaded[it] != (declared[it] as Long).toInt() || executed[it] != loaded[it] }
.map { "$it: declared ${declared[it]}, loaded ${loaded[it]}, ran ${executed[it]}" }
}
for ((name, problems) in results) {
println(if (problems.isEmpty()) "PASS $name" else "FAIL $name")
problems.forEach { println(" $it") }
}
val failed = results.values.count { it.isNotEmpty() }
println("${results.size - failed}/${results.size} passed")
return failed == 0
}
/** Strict decode of whichever receipt-family frame this is, returned as the message its typed form encodes; anything else is refused. */
private fun decodeReceiptFrame(message: CultNetMessage): CultNetMessage = when (message.schemaVersion) {
WRITE_ADMITTED_VERSION -> decodeWriteAdmitted(message).toMessage()
ERROR_V1_VERSION, ERROR_V0_VERSION -> decodeErrorFrame(message).toMessage()
else -> throw IOException("not a receipt frame: ${message.schemaVersion}")
}
@Suppress("UNCHECKED_CAST")
private fun obj(value: Any?): Map<String, Any?> = value as Map<String, Any?>
private fun list(value: Any?): List<Any?> = value as List<Any?>
private fun unhex(text: String): ByteArray = ByteArray(text.length / 2) { text.substring(it * 2, it * 2 + 2).toInt(16).toByte() }
/** A small JSON reader for the vector file: objects, arrays, strings, integers, true/false/null. */
private class JsonReader(private val text: String) {
private var at = 0
fun readDocument(): Any? = readValue().also { skipSpace(); if (at != text.length) throw IOException("trailing JSON") }
private fun skipSpace() { while (at < text.length && text[at] in " \t\r\n") at++ }
private fun readValue(): Any? {
skipSpace()
return when (val c = text[at]) {
'{' -> {
at++
val map = linkedMapOf<String, Any?>()
skipSpace()
if (text[at] == '}') { at++; return map }
do {
skipSpace()
val key = readString()
skipSpace(); expect(':')
map[key] = readValue()
skipSpace()
} while (closes('}'))
map
}
'[' -> {
at++
val items = mutableListOf<Any?>()
skipSpace()
if (text[at] == ']') { at++; return items }
do {
items += readValue()
skipSpace()
} while (closes(']'))
items
}
'"' -> readString()
't' -> literal("true", true)
'f' -> literal("false", false)
'n' -> literal("null", null)
else -> {
if (c != '-' && !c.isDigit()) throw IOException("unexpected '$c' at $at")
val start = at++
while (at < text.length && text[at].isDigit()) at++
text.substring(start, at).toLong()
}
}
}
/** After an element: false when [close] ends the container, true after a comma. */
private fun closes(close: Char): Boolean {
val c = text[at++]
if (c == close) return false
if (c != ',') throw IOException("expected , or $close at ${at - 1}")
return true
}
private fun literal(word: String, value: Any?): Any? {
if (!text.startsWith(word, at)) throw IOException("bad literal at $at")
at += word.length
return value
}
private fun expect(c: Char) { if (text[at++] != c) throw IOException("expected $c at ${at - 1}") }
private fun readString(): String {
expect('"')
val out = StringBuilder()
while (true) {
val c = text[at++]
when (c) {
'"' -> return out.toString()
'\\' -> when (val e = text[at++]) {
'n' -> out.append('\n'); 't' -> out.append('\t'); 'r' -> out.append('\r')
'b' -> out.append('\b'); 'f' -> out.append('\u000c')
'u' -> { out.append(text.substring(at, at + 4).toInt(16).toChar()); at += 4 }
'"', '\\', '/' -> out.append(e)
else -> throw IOException("bad escape \\$e")
}
else -> out.append(c)
}
}
}
}

View file

@ -43,9 +43,10 @@ from cultmesh_py import (
note_doc = define_database_entry_type( note_doc = define_database_entry_type(
"mesh.note", "mesh.note",
[("body", 0), ("name", 1), ("kind", 2)], [("body", 0), ("name", 1), ("kind", 2), ("slug", 3)],
name="name", name="name",
indexes={"kind": "kind"}, indexes={"slug": "slug"}, # unique: no two notes hold one slug; watch_by_index and get_key_by_index look it up
roles={"kind": "kind"}, # many notes share a kind: a selection alias, never a lookup
) )
node = CultMesh.start_node( node = CultMesh.start_node(
"mesh.cc", "mesh.cc",
@ -55,13 +56,13 @@ node = CultMesh.start_node(
node.database.register_document(note_doc) node.database.register_document(note_doc)
unsubscribe = node.database.watch_record(note_doc, "note:1", lambda change: print(change.change_kind)) unsubscribe = node.database.watch_record(note_doc, "note:1", lambda change: print(change.change_kind))
unsubscribe_named = node.database.watch_by_name(note_doc, "intro", lambda change: print(change.record_key)) unsubscribe_named = node.database.watch_by_name(note_doc, "intro", lambda change: print(change.record_key))
unsubscribe_kind = node.database.watch_by_index(note_doc, "kind", "guide", lambda change: print(change.record_key)) unsubscribe_slug = node.database.watch_by_index(note_doc, "slug", "intro-guide", lambda change: print(change.record_key))
node.database.put(note_doc, "note:1", {"body": "hello", "name": "intro", "kind": "guide"}) node.database.put(note_doc, "note:1", {"body": "hello", "name": "intro", "kind": "guide", "slug": "intro-guide"})
live_note = node.database.get_required(note_doc, "note:1") live_note = node.database.get_required(note_doc, "note:1")
put_message = node.database.put_raw_message( put_message = node.database.put_raw_message(
note_doc, note_doc,
"note:2", "note:2",
{"body": "wire me", "name": "wire", "kind": "interop"}, {"body": "wire me", "name": "wire", "kind": "interop", "slug": "wire-me"},
shard_id="interop", shard_id="interop",
shard_epoch=1, shard_epoch=1,
) )

View file

@ -10,7 +10,7 @@ from pathlib import Path
from typing import Any, Callable from typing import Any, Callable
from cultcache_py import CultCache, CultCacheEnvelope, SingleFileMessagePackBackingStore from cultcache_py import CultCache, CultCacheEnvelope, SingleFileMessagePackBackingStore
from cultcache_py.cache import CultCacheError from cultcache_py.cache import CultCacheError, index_text
from cultcache_py.documents import DocumentDefinition, extract_value from cultcache_py.documents import DocumentDefinition, extract_value
from cultnet_py import ( from cultnet_py import (
CultNetAppliedRecord, CultNetAppliedRecord,
@ -520,7 +520,7 @@ class CultMeshDatabase:
raise ValueError(f"Document type {document.type!r} has no index {index!r}") raise ValueError(f"Document type {document.type!r} has no index {index!r}")
def filtered(change: CultMeshDatabaseChange) -> None: def filtered(change: CultMeshDatabaseChange) -> None:
if self._change_matches_extractor(change, extractor, value): if self._change_matches_extractor(change, extractor, value, index_text):
callback(self._change_as_document(change, requested)) callback(self._change_as_document(change, requested))
return self.watch(filtered, document=document) return self.watch(filtered, document=document)
@ -1295,10 +1295,11 @@ class CultMeshDatabase:
change: CultMeshDatabaseChange, change: CultMeshDatabaseChange,
extractor: str | Callable[[Any], str | int | float | bool | None], extractor: str | Callable[[Any], str | int | float | bool | None],
expected: str, expected: str,
render: Callable[[Any], str | None] = str,
) -> bool: ) -> bool:
return ( return (
self._value_matches_extractor(change.value, extractor, expected) self._value_matches_extractor(change.value, extractor, expected, render)
or self._value_matches_extractor(change.previous_value, extractor, expected) or self._value_matches_extractor(change.previous_value, extractor, expected, render)
) )
def _value_matches_extractor( def _value_matches_extractor(
@ -1306,11 +1307,12 @@ class CultMeshDatabase:
value: Any | None, value: Any | None,
extractor: str | Callable[[Any], str | int | float | bool | None], extractor: str | Callable[[Any], str | int | float | bool | None],
expected: str, expected: str,
render: Callable[[Any], str | None] = str,
) -> bool: ) -> bool:
if value is None: if value is None:
return False return False
actual = extract_value(value, extractor) actual = extract_value(value, extractor)
return actual is not None and str(actual) == expected return actual is not None and render(actual) == expected
def _previous_values_for_snapshot_response(self, response: dict[str, Any]) -> dict[tuple[str, str], Any]: def _previous_values_for_snapshot_response(self, response: dict[str, Any]) -> dict[tuple[str, str], Any]:
if response.get("schemaVersion") != "cultnet.snapshot_response_raw.v0": if response.get("schemaVersion") != "cultnet.snapshot_response_raw.v0":

View file

@ -12,6 +12,7 @@ import msgpack
from cultnet_py import ( from cultnet_py import (
error_message, error_message,
message_invalid_error,
unsupported_schema_version_error, unsupported_schema_version_error,
CultNetRawSnapshotResponse, CultNetRawSnapshotResponse,
CultNetSimulationObservationHub, CultNetSimulationObservationHub,
@ -58,6 +59,16 @@ from cultnet_py.cultmesh_contracts import (
) )
_LOGGER = logging.getLogger(__name__) _LOGGER = logging.getLogger(__name__)
# How many consecutive refused frames of one kind a RUDP session may send before it is ended: undecodable
# frames (reset by any frame that decodes) and deletes the write authority refuses (reset by any decoded
# frame that is not a refused delete). Fewer are answered with a typed refusal and the session is kept; the
# frame that reaches this number is answered too, then the session ends. The same number is the cap in every
# runtime.
MAX_CONSECUTIVE_REFUSED_FRAMES = 8
# How long a session whose capped frame was answered waits for the peer to acknowledge the answer before it ends anyway.
_RUDP_CAPPED_ANSWER_GRACE_MS = 2000
_EPHEMERAL_PAIR_ATTEMPTS = 64 _EPHEMERAL_PAIR_ATTEMPTS = 64
@ -89,6 +100,10 @@ class _DatabaseSubscription:
class _RudpPeerConnection: class _RudpPeerConnection:
session: CultNetRudpSession session: CultNetRudpSession
subscriptions: dict[str, _DatabaseSubscription] = field(default_factory=dict) subscriptions: dict[str, _DatabaseSubscription] = field(default_factory=dict)
undecodable_frames_in_a_row: int = 0
refused_deletes_in_a_row: int = 0
# Set once the peer's capped frame has been answered: the session ends when the answers are acknowledged.
ending_deadline_ms: int | None = None
@dataclass @dataclass
@ -345,6 +360,7 @@ class CultMeshLocalServer:
if packet.connection_id != self.rudp_connection_id: if packet.connection_id != self.rudp_connection_id:
continue continue
self._end_answered_peers(rudp_socket, peers)
now_ms = _now_ms() now_ms = _now_ms()
peer = peers.get(remote_addr) peer = peers.get(remote_addr)
if packet.packet_type == CultNetRudpPacketType.CONNECT: if packet.packet_type == CultNetRudpPacketType.CONNECT:
@ -393,18 +409,31 @@ class CultMeshLocalServer:
for delivered in result.delivered: for delivered in result.delivered:
if delivered.channel_id != "schema": if delivered.channel_id != "schema":
continue continue
if peer.ending_deadline_ms is not None:
# The capped frame was answered; nothing after it is served while the answer is delivered.
break
try: try:
message = msgpack.unpackb(delivered.payload, raw=False) message = msgpack.unpackb(delivered.payload, raw=False)
except (msgpack.ExtraData, msgpack.FormatError, msgpack.StackError, ValueError): except (msgpack.ExtraData, msgpack.FormatError, msgpack.StackError, ValueError):
continue message = None
if not isinstance(message, dict): if isinstance(message, dict):
continue peer.undecodable_frames_in_a_row = 0
responses = self._handle_connection_message( responses = self._handle_connection_message(
message, message,
peer.subscriptions, peer.subscriptions,
over_rudp=True, over_rudp=True,
peer=CultMeshServerPeer(f"{remote_addr[0]}:{remote_addr[1]}", "rudp"), peer=CultMeshServerPeer(f"{remote_addr[0]}:{remote_addr[1]}", "rudp"),
) )
refused_delete = message.get("schemaVersion") == "cultnet.document_delete.v0" and any(
response.get("code") == "write_not_authorized" for response in responses
)
peer.refused_deletes_in_a_row = peer.refused_deletes_in_a_row + 1 if refused_delete else 0
refused, run = refused_delete, peer.refused_deletes_in_a_row
else:
# A frame that is no CultNet message is answered, not dropped, and the session kept.
peer.undecodable_frames_in_a_row += 1
refused, run = True, peer.undecodable_frames_in_a_row
responses = [self._message_invalid({}, "message", "CultMesh RUDP server could not read the message.")]
for response in responses: for response in responses:
if unsendable is None: if unsendable is None:
unsendable = self._send_rudp_schema_frame( unsendable = self._send_rudp_schema_frame(
@ -414,6 +443,11 @@ class CultMeshLocalServer:
msgpack.packb(response, use_bin_type=True), msgpack.packb(response, use_bin_type=True),
message_id=str(response.get("messageId") or ""), message_id=str(response.get("messageId") or ""),
) )
if refused and run >= MAX_CONSECUTIVE_REFUSED_FRAMES:
# The capped frame is answered like any other, reliably. The session ends once the peer
# has acknowledged every answer (or the grace runs out): see _end_answered_peers.
peer.ending_deadline_ms = now_ms + _RUDP_CAPPED_ANSWER_GRACE_MS
break
except Exception as error: # noqa: BLE001 - one peer's message never ends the thread except Exception as error: # noqa: BLE001 - one peer's message never ends the thread
# Whatever a peer's message makes the handling raise is that peer's failure. # Whatever a peer's message makes the handling raise is that peer's failure.
# The session recorded the frame's sequence, so it cannot be kept: a retransmit # The session recorded the frame's sequence, so it cannot be kept: a retransmit
@ -444,11 +478,26 @@ class CultMeshLocalServer:
if unsendable is not None: if unsendable is not None:
self._end_unsendable_peer(rudp_socket, peers, remote_addr, unsendable) self._end_unsendable_peer(rudp_socket, peers, remote_addr, unsendable)
def _end_answered_peers(
self,
rudp_socket: socket.socket,
peers: dict[tuple[str, int], _RudpPeerConnection],
) -> None:
"""Ends each session whose capped frame was answered once the peer has acknowledged every answer (a lost
answer is resent first), or when the grace has run out."""
now_ms = _now_ms()
for remote_addr, peer in list(peers.items()):
if peer.ending_deadline_ms is None:
continue
if peer.session.outstanding_reliable_packet_count == 0 or now_ms >= peer.ending_deadline_ms:
self._end_refused_peer(rudp_socket, peers, remote_addr)
def _poll_rudp_resends( def _poll_rudp_resends(
self, self,
rudp_socket: socket.socket, rudp_socket: socket.socket,
peers: dict[tuple[str, int], _RudpPeerConnection], peers: dict[tuple[str, int], _RudpPeerConnection],
) -> None: ) -> None:
self._end_answered_peers(rudp_socket, peers)
now_ms = _now_ms() now_ms = _now_ms()
for remote_addr, peer in list(peers.items()): for remote_addr, peer in list(peers.items()):
for packet in peer.session.due_resends(now_ms): for packet in peer.session.due_resends(now_ms):
@ -602,12 +651,8 @@ class CultMeshLocalServer:
) -> list[dict[str, Any]]: ) -> list[dict[str, Any]]:
document_record = message.get("document") document_record = message.get("document")
if not isinstance(document_record, dict): if not isinstance(document_record, dict):
return [self._error_response( return [self._message_invalid(message, "document", "Raw put messages must contain a document map.")]
"Raw put messages must contain a document map.", malformed_shard = self._malformed_shard_fields(message)
message_id=str(message.get("messageId") or ""),
code="malformed_document_put",
)]
malformed_shard = self._malformed_shard_fields(message, "malformed_document_put")
if malformed_shard is not None: if malformed_shard is not None:
return [malformed_shard] return [malformed_shard]
# Refusal precedes every other judgment of the put. The node resolves the schema first, because the # Refusal precedes every other judgment of the put. The node resolves the schema first, because the
@ -634,11 +679,7 @@ class CultMeshLocalServer:
# raises. It is refused to the peer that sent it; no thread and no other peer is # raises. It is refused to the peer that sent it; no thread and no other peer is
# affected. Neither the refusal nor the log quotes the error, which can quote the put. # affected. Neither the refusal nor the log quotes the error, which can quote the put.
_LOGGER.warning("CultMesh server refused a raw put the store could not take (%s).", type(error).__name__) _LOGGER.warning("CultMesh server refused a raw put the store could not take (%s).", type(error).__name__)
return [self._error_response( return [self._message_invalid(message, "document", "Raw put document could not be stored.")]
"Raw put document could not be stored.",
message_id=str(message.get("messageId") or ""),
code="malformed_document_put",
)]
if change is None: if change is None:
return [self._error_response( return [self._error_response(
"Raw put message did not apply to a registered document.", "Raw put message did not apply to a registered document.",
@ -663,12 +704,10 @@ class CultMeshLocalServer:
schema_id = str(message.get("schemaId") or "") schema_id = str(message.get("schemaId") or "")
record_key = str(message.get("recordKey") or "") record_key = str(message.get("recordKey") or "")
if not schema_id or not record_key: if not schema_id or not record_key:
return [self._error_response( return [self._message_invalid(
"Raw delete messages must contain schemaId and recordKey.", message, "schemaId" if not schema_id else "recordKey", "Raw delete messages must contain schemaId and recordKey.",
message_id=str(message.get("messageId") or ""),
code="malformed_document_delete",
)] )]
malformed_shard = self._malformed_shard_fields(message, "malformed_document_delete") malformed_shard = self._malformed_shard_fields(message)
if malformed_shard is not None: if malformed_shard is not None:
return [malformed_shard] return [malformed_shard]
target = self.node.database.raw_delete_target(message) or (schema_id, record_key) target = self.node.database.raw_delete_target(message) or (schema_id, record_key)
@ -684,7 +723,13 @@ class CultMeshLocalServer:
)] )]
return self._database_delete_notifications(message, subscriptions, resolved_schema_id=change.schema_id) return self._database_delete_notifications(message, subscriptions, resolved_schema_id=change.schema_id)
def _malformed_shard_fields(self, message: dict[str, Any], code: str) -> dict[str, Any] | None: def _message_invalid(self, message: dict[str, Any], field: str, reason: str) -> dict[str, Any]:
"""The typed `message_invalid` refusal naming the request field at fault, under the request's messageId."""
response = message_invalid_error(field, reason).to_wire()
response["messageId"] = str(message.get("messageId") or "")
return response
def _malformed_shard_fields(self, message: dict[str, Any]) -> dict[str, Any] | None:
"""The protocol error for a shardId that is not a string or a shardEpoch that is not an int (a bool is """The protocol error for a shardId that is not a string or a shardEpoch that is not an int (a bool is
not), else None. It precedes the gate, so the authority only ever judges a write whose shard it can not), else None. It precedes the gate, so the authority only ever judges a write whose shard it can
read, and the store never files a write under a shard the authority did not see.""" read, and the store never files a write under a shard the authority did not see."""
@ -694,10 +739,10 @@ class CultMeshLocalServer:
shard_epoch is None or (isinstance(shard_epoch, int) and not isinstance(shard_epoch, bool)) shard_epoch is None or (isinstance(shard_epoch, int) and not isinstance(shard_epoch, bool))
): ):
return None return None
return self._error_response( return self._message_invalid(
message,
"shardId" if not (shard_id is None or isinstance(shard_id, str)) else "shardEpoch",
"shardId must be a string and shardEpoch an integer.", "shardId must be a string and shardEpoch an integer.",
message_id=str(message.get("messageId") or ""),
code=code,
) )
def _refuse_unauthorized_write( def _refuse_unauthorized_write(
@ -929,11 +974,7 @@ class CultMeshLocalServer:
and isinstance(document_record.get("recordKey"), str) and isinstance(document_record.get("recordKey"), str)
and isinstance(document_record.get("payload"), (bytes, bytearray)) and isinstance(document_record.get("payload"), (bytes, bytearray))
): ):
return self._error_response( return self._message_invalid(message, "document", "Raw put documents must carry a string recordKey and a binary payload.")
"Raw put documents must carry a string recordKey and a binary payload.",
message_id=str(message.get("messageId") or ""),
code="malformed_document_put",
)
if self.max_snapshot_bytes is None and not over_rudp: if self.max_snapshot_bytes is None and not over_rudp:
return None return None
alone = self.node.database.raw_put_served_alone(message, message_id=_SHORTEST_SERVED_MESSAGE_ID) alone = self.node.database.raw_put_served_alone(message, message_id=_SHORTEST_SERVED_MESSAGE_ID)

View file

@ -0,0 +1,52 @@
"""Heimdall's invariant: no client treats a reliable-transport ack as write admission. A put completes only on
its result frame (write_not_authorized, or the success reply).
The Python write path is a subscription over RUDP: `send` queues the put, `read_next` returns the result frame
and raises the peer's refusal. The server here refuses the put; its reliable ack of the put arrives (the test
waits for it) and the client still reports the refusal, never success. The transport's `receive` yields data
frames only, so the order of the ack against the refusal cannot change what `read_next` reports.
"""
from __future__ import annotations
import unittest
from cultcache_py import define_database_entry_type
from cultmesh_py import CultMesh, create_node
from cultmesh_py.server import CultMeshLocalServer
from cultnet_py import CultNetDatabaseSubscription, CultNetPeerError, create_rudp_schema_transport
from test_remote_write_authority import KEY_A, NOTE, SCHEMA_ID, put_of
class ClientWriteCompletionTests(unittest.TestCase):
def test_a_put_the_server_refused_is_reported_refused_after_its_ack_arrived(self) -> None:
node = create_node(runtime_id="completion-server")
node.database.register_document(NOTE)
server = CultMeshLocalServer(node=node, port=0, write_authority=None)
server.start()
try:
subscription = CultNetDatabaseSubscription(
host="127.0.0.1",
port=server.port,
timeout_seconds=4.0,
subscription_id="completion",
schema_ids=[SCHEMA_ID],
include_snapshot=False,
create_transport=lambda: create_rudp_schema_transport(
host="127.0.0.1", port=server.port, connection_id=server.rudp_connection_id, timeout_seconds=4.0
),
)
with subscription as sub:
sub.send(put_of(KEY_A, "peer"))
sub._transport.flush_reliable(2.0) # the put's reliable ack has arrived
with self.assertRaises(CultNetPeerError) as raised:
sub.read_next()
finally:
server.stop()
self.assertEqual(raised.exception.response["code"], "write_not_authorized")
self.assertIsNone(server.node.cache.get(NOTE, KEY_A))
if __name__ == "__main__":
unittest.main()

View file

@ -24,7 +24,9 @@ from cultcache_py import (
define_document_type, define_document_type,
) )
from cultcache_py.interop import INTEROP_SCHEMA_VERSION, interop_note_document from cultcache_py.interop import INTEROP_SCHEMA_VERSION, interop_note_document
from cultmesh_py.server import MAX_CONSECUTIVE_REFUSED_FRAMES
from cultmesh_py.verify import verify from cultmesh_py.verify import verify
from cultmesh_py.write_authority import CultMeshRemoteWriteDecision
from cultnet_py import ( from cultnet_py import (
compute_simulation_claim_hash, compute_simulation_claim_hash,
CultNetClientAuthorityScope, CultNetClientAuthorityScope,
@ -4883,11 +4885,146 @@ class CultMeshTests(unittest.TestCase):
for message_id, refusal in refusals: for message_id, refusal in refusals:
self.assertEqual(refusal["schemaVersion"], "cultnet.error.v0", message_id) self.assertEqual(refusal["schemaVersion"], "cultnet.error.v0", message_id)
self.assertEqual(refusal["messageId"], message_id) self.assertEqual(refusal["messageId"], message_id)
self.assertEqual(refusal["code"], "malformed_document_put", message_id) self.assertEqual(refusal["code"], "message_invalid", message_id)
self.assertEqual(refusal["details"]["field"], "document", message_id)
self.assertEqual(after["messageId"], "after") self.assertEqual(after["messageId"], "after")
self.assertEqual(after["documents"], []) self.assertEqual(after["documents"], [])
self.assertTrue(alive) self.assertTrue(alive)
def _rudp_open(self, server: Any) -> Any:
return create_rudp_schema_transport(
host="127.0.0.1",
port=server.port,
connection_id=server.rudp_connection_id,
timeout_seconds=4.0,
max_fragment_bytes=1200,
)
def _rudp_answer(self, transport: Any, payload: bytes) -> dict[str, Any]:
transport.send("schema", payload)
return msgpack.unpackb(transport.receive(timeout_seconds=4.0).payload, raw=False)
def _assert_rudp_session_ended(self, transport: Any, server: Any) -> None:
transport.send("schema", msgpack.packb(snapshot_request(message_id="after-cap").to_wire(), use_bin_type=True))
with self.assertRaises(Exception):
transport.receive(timeout_seconds=1.0)
self.assertTrue(server._rudp_thread.is_alive())
def test_cultmesh_local_server_answers_unreadable_frames_and_ends_the_session_at_eight_in_a_row(self) -> None:
# An undecodable frame (not a msgpack map) is answered message_invalid and the session kept;
# eight in a row end it (the eighth is answered too); any decoded frame resets the run.
self.assertEqual(MAX_CONSECUTIVE_REFUSED_FRAMES, 8)
node = CultMesh.create_node(runtime_id="mesh-rudp-unreadable")
server = CultMesh.serve_node(node, write_authority=CultMeshGrantAllWrites())
garbage = b"\xc1"
snapshot = msgpack.packb(snapshot_request(message_id="ok").to_wire(), use_bin_type=True)
transport = self._rudp_open(server)
try:
for _ in range(2):
for _ in range(MAX_CONSECUTIVE_REFUSED_FRAMES - 1):
reply = self._rudp_answer(transport, garbage)
self.assertEqual(reply["code"], "message_invalid")
self.assertEqual(reply["details"]["field"], "message")
self.assertEqual(self._rudp_answer(transport, snapshot)["messageId"], "ok")
for _ in range(MAX_CONSECUTIVE_REFUSED_FRAMES - 1):
self._rudp_answer(transport, garbage)
transport.send("schema", garbage)
eighth = msgpack.unpackb(transport.receive(timeout_seconds=4.0).payload, raw=False)
self.assertEqual(eighth["code"], "message_invalid")
self._assert_rudp_session_ended(transport, server)
finally:
transport.close()
server.stop()
def test_cultmesh_local_server_resends_a_lost_capped_answer_before_it_ends_the_session(self) -> None:
# The eighth refusal is answered reliably: the datagram carrying it is lost on the wire, and the
# peer still gets the answer (by resend) before the goodbye. An unreliable final answer, or a
# goodbye sent at once, would leave the peer without it.
node = CultMesh.create_node(runtime_id="mesh-rudp-lost-answer")
server = CultMesh.serve_node(node, write_authority=CultMeshGrantAllWrites())
send = server._send_rudp_packet
dropped: list[Any] = []
armed = threading.Event()
def lossy(rudp_socket: Any, remote_addr: Any, packet: Any) -> Any:
if armed.is_set() and not dropped and packet.packet_type == CultNetRudpPacketType.DATA and b"message_invalid" in (packet.payload or b""):
dropped.append(packet)
return None
return send(rudp_socket, remote_addr, packet)
server._send_rudp_packet = lossy
transport = self._rudp_open(server)
try:
for _ in range(MAX_CONSECUTIVE_REFUSED_FRAMES - 1):
self._rudp_answer(transport, b"\xc1")
armed.set()
reply = self._rudp_answer(transport, b"\xc1")
self.assertEqual(len(dropped), 1, "fixture: the first transmission of the last answer was lost")
self.assertEqual(reply["code"], "message_invalid")
self._assert_rudp_session_ended(transport, server)
finally:
transport.close()
server.stop()
def test_cultmesh_local_server_serves_nothing_after_the_capped_frame(self) -> None:
node = CultMesh.create_node(runtime_id="mesh-rudp-after-cap")
server = CultMesh.serve_node(node, write_authority=CultMeshGrantAllWrites())
transport = self._rudp_open(server)
try:
for _ in range(MAX_CONSECUTIVE_REFUSED_FRAMES - 1):
self._rudp_answer(transport, b"\xc1")
transport.send("schema", b"\xc1")
transport.send("schema", msgpack.packb(snapshot_request(message_id="behind-the-cap").to_wire(), use_bin_type=True))
first = msgpack.unpackb(transport.receive(timeout_seconds=4.0).payload, raw=False)
self.assertEqual(first["code"], "message_invalid")
with self.assertRaises(Exception):
transport.receive(timeout_seconds=1.0)
finally:
transport.close()
server.stop()
def test_cultmesh_local_server_ends_the_session_at_eight_refused_deletes_in_a_row(self) -> None:
class GrantOnlyKeyB:
def decide(self, request: Any) -> Any:
return CultMeshRemoteWriteDecision.grant() if request.record_key == "note:b" else CultMeshRemoteWriteDecision.refuse("no")
document = self._rudp_bound_note()
node = CultMesh.create_node(runtime_id="mesh-rudp-deletes")
node.database.register_document(document)
server = CultMesh.serve_node(node, write_authority=GrantOnlyKeyB())
def delete(key: str) -> bytes:
return msgpack.packb({
"schemaVersion": "cultnet.document_delete.v0",
"messageId": f"del-{key}",
"schemaId": document.schema_id,
"recordKey": key,
}, use_bin_type=True)
snapshot = msgpack.packb(snapshot_request(message_id="ok").to_wire(), use_bin_type=True)
transport = self._rudp_open(server)
try:
def refused_run(count: int) -> None:
for _ in range(count):
reply = self._rudp_answer(transport, delete("note:a"))
self.assertEqual(reply["code"], "write_not_authorized")
self.assertEqual(reply["details"]["value"], "note:a")
refused_run(MAX_CONSECUTIVE_REFUSED_FRAMES - 1)
# A granted delete resets the run (nothing is stored under it, so the store answers that).
self.assertEqual(self._rudp_answer(transport, delete("note:b"))["code"], "unregistered_document_delete")
refused_run(MAX_CONSECUTIVE_REFUSED_FRAMES - 1)
# So does any decoded non-delete frame.
self.assertEqual(self._rudp_answer(transport, snapshot)["messageId"], "ok")
refused_run(MAX_CONSECUTIVE_REFUSED_FRAMES - 1)
transport.send("schema", delete("note:a"))
eighth = msgpack.unpackb(transport.receive(timeout_seconds=4.0).payload, raw=False)
self.assertEqual(eighth["code"], "write_not_authorized")
self._assert_rudp_session_ended(transport, server)
finally:
transport.close()
server.stop()
def test_cultmesh_local_server_refuses_a_put_its_store_cannot_take_and_serves_on(self) -> None: def test_cultmesh_local_server_refuses_a_put_its_store_cannot_take_and_serves_on(self) -> None:
# Well-formed puts the store refuses: a blank key, a payload that is not MessagePack, a # Well-formed puts the store refuses: a blank key, a payload that is not MessagePack, a
# payload its schema cannot decode, a global document under another key. Each is refused # payload its schema cannot decode, a global document under another key. Each is refused
@ -4942,7 +5079,8 @@ class CultMeshTests(unittest.TestCase):
for message_id, refusal in rudp_refusals + tcp_refusals: for message_id, refusal in rudp_refusals + tcp_refusals:
self.assertEqual(refusal["schemaVersion"], "cultnet.error.v0", message_id) self.assertEqual(refusal["schemaVersion"], "cultnet.error.v0", message_id)
self.assertEqual(refusal["messageId"], message_id) self.assertEqual(refusal["messageId"], message_id)
self.assertEqual(refusal["code"], "malformed_document_put", message_id) self.assertEqual(refusal["code"], "message_invalid", message_id)
self.assertEqual(refusal["details"]["field"], "document", message_id)
self.assertNotIn("note:ok", refusal["error"], message_id) self.assertNotIn("note:ok", refusal["error"], message_id)
self.assertEqual(len(tcp_refusals), 5) self.assertEqual(len(tcp_refusals), 5)
self.assertEqual(rudp_after["schemaVersion"], "cultnet.hello.v0") self.assertEqual(rudp_after["schemaVersion"], "cultnet.hello.v0")
@ -5056,9 +5194,11 @@ class CultMeshTests(unittest.TestCase):
self.assertEqual(unsupported["details"]["value"], "cultnet.nope.v0") self.assertEqual(unsupported["details"]["value"], "cultnet.nope.v0")
self.assertIn("unsupported_schema_version", unsupported["error"]) self.assertIn("unsupported_schema_version", unsupported["error"])
self.assertEqual(malformed_put["messageId"], "bad-put") self.assertEqual(malformed_put["messageId"], "bad-put")
self.assertEqual(malformed_put["code"], "malformed_document_put") self.assertEqual(malformed_put["code"], "message_invalid")
self.assertEqual(malformed_put["details"]["field"], "document")
self.assertEqual(malformed_delete["messageId"], "bad-delete") self.assertEqual(malformed_delete["messageId"], "bad-delete")
self.assertEqual(malformed_delete["code"], "malformed_document_delete") self.assertEqual(malformed_delete["code"], "message_invalid")
self.assertEqual(malformed_delete["details"]["field"], "recordKey")
self.assertEqual(unsupported_v1["code"], "unsupported_schema_version") self.assertEqual(unsupported_v1["code"], "unsupported_schema_version")
self.assertEqual(unsupported_v1["details"]["value"], "cultnet.snapshot_request.v1") self.assertEqual(unsupported_v1["details"]["value"], "cultnet.snapshot_request.v1")

View file

@ -311,15 +311,12 @@ class RemoteWriteAuthorityTests(unittest.TestCase):
self.seed(server, KEY_B, "server") self.seed(server, KEY_B, "server")
shards_before = server.node.database.shard_ids() shards_before = server.node.database.shard_ids()
for field, value in (("shardId", {"not": "a string"}), ("shardId", 5), ("shardEpoch", "7"), ("shardEpoch", True), ("shardEpoch", 1.5)): for field, value in (("shardId", {"not": "a string"}), ("shardId", 5), ("shardEpoch", "7"), ("shardEpoch", True), ("shardEpoch", 1.5)):
for operation, code in ( for operation in ("put", "delete"):
("put", "malformed_document_put"),
("delete", "malformed_document_delete"),
):
with self.subTest(field=field, value=value, operation=operation): with self.subTest(field=field, value=value, operation=operation):
message = put_of(KEY_A, "x") if operation == "put" else delete_of(KEY_B) message = put_of(KEY_A, "x") if operation == "put" else delete_of(KEY_B)
message[field] = value message[field] = value
replies = self.send(server, message) replies = self.send(server, message)
self.assertEqual([reply["code"] for reply in replies], [code]) self.assertEqual([(reply["code"], reply["details"]["field"]) for reply in replies], [("message_invalid", field)])
self.assertEqual(authority.requests, [], "the authority was never asked") self.assertEqual(authority.requests, [], "the authority was never asked")
self.assertIsNone(self.text_of(server, KEY_A), "nothing was written") self.assertIsNone(self.text_of(server, KEY_A), "nothing was written")
self.assertEqual(self.text_of(server, KEY_B), "server", "nothing was deleted") self.assertEqual(self.text_of(server, KEY_B), "server", "nothing was deleted")

View file

@ -0,0 +1,84 @@
from __future__ import annotations
import re
import unittest
from pathlib import Path
from cultcache_py import define_database_entry_type
from cultcache_py.cache import CultCacheError
from cultmesh_py import CultMesh
# variants:ruling:unique-indexes. `indexes` are unique lookups, `roles` are selection aliases. A watch by index reads the
# lookup's own canonical text (cultcache_py.cache.index_text), never a second rendering of the value.
class WatchByIndexTests(unittest.TestCase):
def test_the_watch_and_the_lookup_agree_on_the_text_of_a_value(self) -> None:
document = define_database_entry_type("mesh.priced", [("price", 0)], indexes={"price": "price"})
node = CultMesh.create_node(runtime_id="mesh-watch-text")
node.database.register_document(document)
cases = [
(2.0, "2", "2.0"),
(0.1, "0.1000000000000000055511151231257827021181583404541015625", "0.1"),
(-0.0, "0", "-0.0"),
]
for index, (value, text, str_text) in enumerate(cases):
with self.subTest(value=value):
key = f"item:{index}"
heard: list[str] = []
misheard: list[str] = []
node.database.watch_by_index(document, "price", text, lambda change: heard.append(change.record_key))
node.database.watch_by_index(document, "price", str_text, lambda change: misheard.append(change.record_key))
node.database.put(document, key, {"price": value})
self.assertEqual(node.database.get_by_index(document, "price", text), {"price": value})
self.assertEqual(heard, [key])
if str_text != text:
self.assertEqual(misheard, [])
self.assertIsNone(node.database.get_by_index(document, "price", str_text))
def test_a_watch_hears_a_record_leaving_a_value_by_the_canonical_text(self) -> None:
document = define_database_entry_type("mesh.leaving", [("price", 0)], indexes={"price": "price"})
node = CultMesh.create_node(runtime_id="mesh-watch-leaving")
node.database.register_document(document)
node.database.put(document, "item:1", {"price": 2.0})
heard: list[str] = []
node.database.watch_by_index(document, "price", "2", lambda change: heard.append(change.record_key))
node.database.put(document, "item:1", {"price": 3.0})
self.assertEqual(heard, ["item:1"])
def test_a_blank_value_is_never_indexed_so_never_watched(self) -> None:
document = define_database_entry_type("mesh.blank", [("sku", 0)], indexes={"sku": "sku"})
node = CultMesh.create_node(runtime_id="mesh-watch-blank")
node.database.register_document(document)
heard: list[str] = []
node.database.watch_by_index(document, "sku", " ", lambda change: heard.append(change.record_key))
node.database.put(document, "item:1", {"sku": " "})
self.assertEqual(heard, [])
def test_a_role_alias_is_not_watchable_by_index(self) -> None:
document = define_database_entry_type("mesh.roles", [("kind", 0)], roles={"kind": "kind"})
node = CultMesh.create_node(runtime_id="mesh-watch-role")
node.database.register_document(document)
with self.assertRaisesRegex(ValueError, "has no index"):
node.database.watch_by_index(document, "kind", "guide", lambda _: None)
class ReadmeTests(unittest.TestCase):
def test_the_readme_note_document_admits_many_notes_of_one_kind(self) -> None:
readme = (Path(__file__).parents[1] / "README.md").read_text(encoding="utf-8")
declaration = re.search(r"^note_doc = define_database_entry_type\(\n.*?^\)\n", readme, re.S | re.M)
self.assertIsNotNone(declaration)
scope: dict = {"define_database_entry_type": define_database_entry_type}
exec(declaration.group(0), scope) # noqa: S102 - the README's own declaration
note_doc = scope["note_doc"]
node = CultMesh.create_node(runtime_id="mesh-readme")
node.database.register_document(note_doc)
node.database.put(note_doc, "note:1", {"body": "a", "name": "one", "kind": "guide", "slug": "one"})
node.database.put(note_doc, "note:2", {"body": "b", "name": "two", "kind": "guide", "slug": "two"})
self.assertEqual(node.database.get_by_index(note_doc, "slug", "two")["body"], "b")
with self.assertRaises(CultCacheError):
node.database.put(note_doc, "note:3", {"body": "c", "name": "three", "kind": "guide", "slug": "one"})
if __name__ == "__main__":
unittest.main()

View file

@ -0,0 +1,77 @@
//! The real Rust RUDP document server on an ephemeral loopback port, for the cross-runtime write-receipt interop
//! harness (packages/cultmesh-ts/test/interop/write-receipt-interop.test.ts). It is the production server with a
//! recording sink and nothing else: stdout carries one JSON line per fact the harness asserts on, and the server runs
//! until stdin closes.
//!
//! {"event":"listening","port":N}
//! {"event":"applied","op":"put","messageId":"...","schemaId":"...","recordKey":"..."}
//!
//! Record keys that start with `refuse:` are refused by the sink (authorize and commit), which is how the harness
//! asks for a refused write.
use anyhow::Result;
use cultmesh_rs::{
CultMeshRudpDocumentServer, CultMeshRudpRawDocumentReceipt, CultMeshRudpRawDocumentSink,
CultMeshRudpSnapshotQuery, CultMeshSystemClock,
};
use cultnet_rs::CultNetRawDocumentRecord;
use serde_json::json;
use std::io::Read;
use std::net::UdpSocket;
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
use std::thread;
use std::time::Duration;
const REFUSED_PREFIX: &str = "refuse:";
struct RecordingSink;
impl CultMeshRudpRawDocumentSink for RecordingSink {
fn authorize_raw_document(&mut self, receipt: &CultMeshRudpRawDocumentReceipt) -> Result<()> {
if receipt.document.record_key.starts_with(REFUSED_PREFIX) {
anyhow::bail!("not authorized");
}
Ok(())
}
fn accept_raw_document(&mut self, receipt: CultMeshRudpRawDocumentReceipt) -> Result<()> {
if receipt.document.record_key.starts_with(REFUSED_PREFIX) {
anyhow::bail!("not authorized");
}
println!(
"{}",
json!({
"event": "applied",
"op": "put",
"messageId": receipt.message_id,
"schemaId": receipt.document.schema_id,
"recordKey": receipt.document.record_key,
})
);
Ok(())
}
}
fn main() -> Result<()> {
let source = |_: &CultMeshRudpSnapshotQuery| -> Result<Vec<CultNetRawDocumentRecord>> { Ok(Vec::new()) };
let mut server = CultMeshRudpDocumentServer::new(
UdpSocket::bind("127.0.0.1:0")?,
RecordingSink,
source,
CultMeshSystemClock::default(),
Default::default(),
)?;
println!("{}", json!({ "event": "listening", "port": server.local_addr()?.port() }));
let stop = Arc::new(AtomicBool::new(false));
let watcher = Arc::clone(&stop);
thread::spawn(move || {
let mut sink = Vec::new();
let _ = std::io::stdin().read_to_end(&mut sink);
watcher.store(true, Ordering::SeqCst);
});
while !stop.load(Ordering::SeqCst) {
server.poll_once()?;
thread::sleep(Duration::from_millis(1));
}
Ok(())
}

View file

@ -82,7 +82,10 @@ pub struct CultMeshRudpDocumentPublishOptions {
pub target: SocketAddr, pub target: SocketAddr,
pub runtime_id: String, pub runtime_id: String,
pub connect_timeout: Duration, pub connect_timeout: Duration,
/// How long the acknowledgement of every put may take.
pub flush_timeout: Duration, pub flush_timeout: Duration,
/// How long after the acknowledgement a refusal may still arrive; a success frame ends the wait early.
pub result_window: Duration,
pub poll_interval: Duration, pub poll_interval: Duration,
pub resend_delay_ms: u64, pub resend_delay_ms: u64,
pub source_agent_id: Option<String>, pub source_agent_id: Option<String>,
@ -149,6 +152,7 @@ impl Default for CultMeshRudpDocumentPublishOptions {
runtime_id: "cultmesh-rudp-document-publisher".to_string(), runtime_id: "cultmesh-rudp-document-publisher".to_string(),
connect_timeout: Duration::from_secs(3), connect_timeout: Duration::from_secs(3),
flush_timeout: Duration::from_millis(300), flush_timeout: Duration::from_millis(300),
result_window: Duration::from_millis(500),
poll_interval: Duration::from_millis(10), poll_interval: Duration::from_millis(10),
resend_delay_ms: 50, resend_delay_ms: 50,
source_agent_id: None, source_agent_id: None,
@ -882,77 +886,30 @@ fn unix_millis() -> u128 {
/// Publish one raw CultNet schema message to a RUDP catalog. /// Publish one raw CultNet schema message to a RUDP catalog.
/// ///
/// Each call owns a fresh connection epoch and returns only after the catalog /// A put completes on the catalog's result frame, never on the transport
/// acknowledges reliable application admission. Against a Rust CultMesh /// acknowledgement: an ack proves delivery, not admission. A refusal (a
/// catalog, which never acknowledges a message it refuses, a refusal is /// `cultnet.error.v0` frame) returns an error in either order against the ack,
/// returned as an error, naming the refusal when it is one of the fixed texts /// naming the refusal when it is one of the fixed texts a catalog sends. A
/// such a catalog sends. A catalog that acknowledges a message on receipt and /// catalog that admits a put sends no success frame, so after the ack the call
/// refuses it afterwards (the TypeScript document server does) is reported as /// waits `result_window` for a refusal and reports the put as admitted when
/// having admitted it: the acknowledgement arrives first. /// none comes (interim until the catalog sends a success frame; a success
/// frame ends the wait early). Each call owns a fresh connection epoch.
pub fn publish_cultnet_message_to_rudp_catalog( pub fn publish_cultnet_message_to_rudp_catalog(
message: &cultnet_rs::CultNetMessage, message: &cultnet_rs::CultNetMessage,
options: CultMeshRudpDocumentPublishOptions, options: CultMeshRudpDocumentPublishOptions,
) -> Result<()> { ) -> Result<()> {
if options.runtime_id.trim().is_empty() { publish_cultnet_messages_to_rudp_catalog(std::slice::from_ref(message), options)
anyhow::bail!("runtime_id must be non-empty");
}
let mut client = connect_rudp_client(
options.target,
&options.runtime_id,
options.connect_timeout,
options.poll_interval,
options.resend_delay_ms,
"cultmesh-rudp-document-publisher",
)?;
let payload = encode_cultnet_message_to_vec(message, CultNetWireContract::CultNetSchemaV0)?;
let receipt = client.send_reliable("schema", payload)?;
let flush_deadline = Instant::now() + options.flush_timeout;
while Instant::now() < flush_deadline {
if let Some(frame) = client.receive_once()? {
if frame.channel_id == "schema" {
if let Ok(cultnet_rs::CultNetMessage::Error { error, .. }) =
decode_cultnet_message_from_slice(
&frame.payload,
CultNetWireContract::CultNetSchemaV0,
)
{
let _ = client.disconnect(b"document-refused".to_vec());
anyhow::bail!(
"CultMesh RUDP catalog {} refused the put{}",
options.target,
recognised_refusal(&error)
);
}
}
}
match client.reliable_send_status(&receipt) {
cultnet_rs::CultNetRudpReliableSendStatus::Acknowledged => {
let _ = client.disconnect(b"document-acknowledged".to_vec());
return Ok(());
}
cultnet_rs::CultNetRudpReliableSendStatus::Invalidated => {
anyhow::bail!("CultMesh RUDP document receipt was invalidated")
}
cultnet_rs::CultNetRudpReliableSendStatus::Pending => {}
}
client.poll_resends()?;
thread::sleep(options.poll_interval);
}
let _ = client.disconnect(b"document-unacknowledged".to_vec());
anyhow::bail!(
"timed out waiting for reliable acknowledgement from CultMesh RUDP catalog {}",
options.target
)
} }
/// Publish a batch of raw CultNet schema messages to a RUDP catalog. /// Publish a batch of raw CultNet schema messages to a RUDP catalog on one
/// connection.
/// ///
/// Unlike [`publish_cultnet_message_to_rudp_catalog`], which polls for /// Completes under the same rule as [`publish_cultnet_message_to_rudp_catalog`]:
/// per-message application-level acknowledgement, this sends every message on /// every message acknowledged, then no refusal frame within `result_window`
/// one connection and waits only for a transport-level `flush_reliable`. It /// (or one success frame per message). A refusal of any message in the batch
/// trades per-receipt confirmation for batch throughput; callers that need a /// fails the whole call; the batch cannot hide it. The catalog's answers do not
/// stronger delivery guarantee should use the singular publish path instead. /// name the message they answer, so a failed batch does not say which message
/// was refused or whether the others were admitted.
pub fn publish_cultnet_messages_to_rudp_catalog( pub fn publish_cultnet_messages_to_rudp_catalog(
messages: &[cultnet_rs::CultNetMessage], messages: &[cultnet_rs::CultNetMessage],
options: CultMeshRudpDocumentPublishOptions, options: CultMeshRudpDocumentPublishOptions,
@ -969,11 +926,62 @@ pub fn publish_cultnet_messages_to_rudp_catalog(
"cultmesh-rudp-document-publisher", "cultmesh-rudp-document-publisher",
)?; )?;
let mut receipts = Vec::with_capacity(messages.len());
for message in messages { for message in messages {
let payload = encode_cultnet_message_to_vec(message, CultNetWireContract::CultNetSchemaV0)?; let payload = encode_cultnet_message_to_vec(message, CultNetWireContract::CultNetSchemaV0)?;
client.send("schema", payload)?; receipts.push(client.send_reliable("schema", payload)?);
}
let ack_deadline = Instant::now() + options.flush_timeout;
let mut acknowledged_at: Option<Instant> = None;
let mut successes = 0;
loop {
while let Some(frame) = client.receive_once()? {
if frame.channel_id != "schema" {
continue;
}
match decode_cultnet_message_from_slice(&frame.payload, CultNetWireContract::CultNetSchemaV0) {
Ok(cultnet_rs::CultNetMessage::Error { error, .. }) => {
let _ = client.disconnect(b"document-refused".to_vec());
anyhow::bail!(
"CultMesh RUDP catalog {} refused the put{}",
options.target,
recognised_refusal(&error)
);
}
Ok(_) => successes += 1,
Err(_) => {}
}
}
if acknowledged_at.is_none() {
let mut all_acknowledged = true;
for receipt in &receipts {
match client.reliable_send_status(receipt) {
cultnet_rs::CultNetRudpReliableSendStatus::Acknowledged => {}
cultnet_rs::CultNetRudpReliableSendStatus::Invalidated => {
anyhow::bail!("CultMesh RUDP document receipt was invalidated")
}
cultnet_rs::CultNetRudpReliableSendStatus::Pending => all_acknowledged = false,
}
}
if all_acknowledged {
acknowledged_at = Some(Instant::now());
} else if Instant::now() >= ack_deadline {
let _ = client.disconnect(b"document-unacknowledged".to_vec());
anyhow::bail!(
"timed out waiting for reliable acknowledgement from CultMesh RUDP catalog {}",
options.target
)
}
}
if let Some(acknowledged_at) = acknowledged_at {
if successes >= messages.len() || acknowledged_at.elapsed() >= options.result_window {
let _ = client.disconnect(b"document-completed".to_vec());
return Ok(());
}
}
client.poll_resends()?;
thread::sleep(options.poll_interval);
} }
client.flush_reliable(options.flush_timeout)
} }
fn connect_rudp_client( fn connect_rudp_client(
@ -1451,6 +1459,141 @@ mod tests {
Ok((target, stop, catalog)) Ok((target, stop, catalog))
} }
/// A catalog that acknowledges every reliable message it receives and refuses it with a typed
/// write_not_authorized, the acknowledgement either before the refusal or after it (the second packet
/// follows the first by 100 ms). The refusal carries no acknowledgement of its own.
/// What the stub catalog sends for a put, and in which order against the transport ack.
#[derive(Clone, Copy, PartialEq, Eq)]
enum CatalogReply {
RefusalThenAck,
AckThenRefusal,
AckThenSuccess,
}
fn acknowledging_refusing_catalog(
reply: CatalogReply,
) -> Result<(
SocketAddr,
std::sync::Arc<std::sync::atomic::AtomicBool>,
thread::JoinHandle<Result<()>>,
)> {
use cultnet_rs::{
CultNetRudpPacketType, CultNetRudpSendOptions, CultNetRudpSession,
CultNetRudpSessionOptions, decode_rudp_packet, encode_rudp_packet,
};
let socket = UdpSocket::bind("127.0.0.1:0")?;
socket.set_read_timeout(Some(Duration::from_millis(5)))?;
let target = socket.local_addr()?;
let stop = std::sync::Arc::new(std::sync::atomic::AtomicBool::new(false));
let catalog_stop = stop.clone();
let catalog = thread::spawn(move || -> Result<()> {
let mut session: Option<CultNetRudpSession> = None;
let mut wire = vec![0_u8; 65_535];
while !catalog_stop.load(std::sync::atomic::Ordering::SeqCst) {
let Ok((bytes, peer)) = socket.recv_from(&mut wire) else {
continue;
};
let Ok(packet) = decode_rudp_packet(&wire[..bytes]) else {
continue;
};
if packet.packet_type == CultNetRudpPacketType::Connect {
let mut accepted = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: packet.connection_id,
initial_sequence: None,
resend_delay_ms: 10_000,
max_pending_reliable_packets: Some(64),
});
let accept = accepted.accept_connect(&packet, 0, Vec::new())?;
socket.send_to(&encode_rudp_packet(&accept)?, peer)?;
session = Some(accepted);
continue;
}
let Some(session) = session.as_mut() else {
continue;
};
let Ok(received) = session.receive(&packet, 1) else {
continue;
};
if !packet.reliable || received.delivered.is_empty() {
continue;
}
let ack = session.create_ack_for_received(packet.sequence);
let refusal = encode_cultnet_message_to_vec(
&if reply == CatalogReply::AckThenSuccess {
cultnet_rs::CultNetMessage::SnapshotResponseRaw {
message_id: "completion".into(),
documents: Vec::new(),
}
} else {
cultnet_rs::CultNetMessage::Error {
error: "write_not_authorized: the catalog refused the document".into(),
code: Some(cultnet_rs::CultNetErrorCode::WriteNotAuthorized),
details: None,
}
},
CultNetWireContract::CultNetSchemaV0,
)?;
let options = CultNetRudpSendOptions {
reliable: false,
ordered: false,
sequenced: false,
now_ms: 2,
reliable_expire_after_ms: None,
};
let mut refusals = session.send_many("schema", refusal, options, Some(1200))?;
for refusal in &mut refusals {
refusal.ack = 0;
refusal.ack_mask = 0;
}
let send_refusals = |socket: &UdpSocket| -> Result<()> {
for refusal in &refusals {
socket.send_to(&encode_rudp_packet(refusal)?, peer)?;
}
Ok(())
};
if reply == CatalogReply::RefusalThenAck {
send_refusals(&socket)?;
thread::sleep(Duration::from_millis(100));
socket.send_to(&encode_rudp_packet(&ack)?, peer)?;
} else {
socket.send_to(&encode_rudp_packet(&ack)?, peer)?;
thread::sleep(Duration::from_millis(100));
send_refusals(&socket)?;
}
}
Ok(())
});
Ok((target, stop, catalog))
}
fn publish_options(target: SocketAddr) -> CultMeshRudpDocumentPublishOptions {
CultMeshRudpDocumentPublishOptions {
target,
connect_timeout: Duration::from_secs(2),
flush_timeout: Duration::from_secs(3),
poll_interval: Duration::from_millis(2),
resend_delay_ms: 5,
..CultMeshRudpDocumentPublishOptions::default()
}
}
fn refused_note() -> cultnet_rs::CultNetMessage {
cultnet_rs::CultNetMessage::DocumentPutRaw {
message_id: "completion".into(),
document: cultnet_rs::CultNetRawDocumentRecord {
schema_id: "cultmesh.test.raw.v0".into(),
record_key: "tests:completion:a".into(),
stored_at: "2026-11-08T00:00:00Z".into(),
payload_encoding: cultnet_rs::CultNetRawPayloadEncoding::Messagepack,
payload: vec![0xc0],
source_runtime_id: None,
source_agent_id: None,
source_role: None,
tags: None,
},
}
}
/// A catalog's refusal text is quoted only when it is one of the fixed /// A catalog's refusal text is quoted only when it is one of the fixed
/// refusals a CultMesh catalog sends. Any other text, which could carry /// refusals a CultMesh catalog sends. Any other text, which could carry
/// anything, stays out of the publisher's error. /// anything, stays out of the publisher's error.
@ -1901,4 +2044,63 @@ mod tests {
assert_eq!(lease.bytes(), &[1, 2, 3]); assert_eq!(lease.bytes(), &[1, 2, 3]);
Ok(()) Ok(())
} }
// Heimdall's invariant: no client treats a reliable-transport ack as write admission. The catalog here
// acknowledges the put and refuses it, the refusal arriving before or after the ack.
#[test]
fn a_publish_whose_refusal_arrives_before_the_ack_is_refused() -> Result<()> {
let (target, stop, catalog) = acknowledging_refusing_catalog(CatalogReply::RefusalThenAck)?;
let result = publish_cultnet_message_to_rudp_catalog(&refused_note(), publish_options(target));
stop.store(true, std::sync::atomic::Ordering::SeqCst);
catalog.join().expect("catalog thread should not panic")?;
let error = result.expect_err("a refused put is never reported as admitted");
assert!(error.to_string().contains("refused the put"), "{error:#}");
Ok(())
}
#[test]
fn a_publish_whose_ack_arrives_before_the_refusal_is_refused() -> Result<()> {
let (target, stop, catalog) = acknowledging_refusing_catalog(CatalogReply::AckThenRefusal)?;
let result = publish_cultnet_message_to_rudp_catalog(&refused_note(), publish_options(target));
stop.store(true, std::sync::atomic::Ordering::SeqCst);
catalog.join().expect("catalog thread should not panic")?;
let error = result.expect_err("a refused put is never reported as admitted");
assert!(error.to_string().contains("refused the put"), "{error:#}");
Ok(())
}
#[test]
fn a_batch_publish_is_not_complete_on_the_ack_of_refused_puts() -> Result<()> {
for reply in [CatalogReply::AckThenRefusal, CatalogReply::RefusalThenAck] {
let (target, stop, catalog) = acknowledging_refusing_catalog(reply)?;
let result = publish_cultnet_messages_to_rudp_catalog(&[refused_note()], publish_options(target));
stop.store(true, std::sync::atomic::Ordering::SeqCst);
catalog.join().expect("catalog thread should not panic")?;
assert!(result.is_err(), "a refused put was reported as published");
}
Ok(())
}
#[test]
fn a_success_frame_completes_the_publish_without_waiting_out_the_result_window() -> Result<()> {
for batch in [false, true] {
let (target, stop, catalog) = acknowledging_refusing_catalog(CatalogReply::AckThenSuccess)?;
let options = CultMeshRudpDocumentPublishOptions {
result_window: Duration::from_secs(30),
..publish_options(target)
};
let started = Instant::now();
let result = if batch {
publish_cultnet_messages_to_rudp_catalog(&[refused_note()], options)
} else {
publish_cultnet_message_to_rudp_catalog(&refused_note(), options)
};
stop.store(true, std::sync::atomic::Ordering::SeqCst);
catalog.join().expect("catalog thread should not panic")?;
result?;
assert!(started.elapsed() < Duration::from_secs(10), "batch={batch}: waited out the window");
}
Ok(())
}
} }

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

View file

@ -5,17 +5,23 @@ All notable changes to `@gamecult/cultmesh-ts` are recorded here, following
## Unreleased ## Unreleased
### Added
- `publishRudpDocumentOnce` takes `writeReceipts: true`: it sends a hello listing `cultnet.write_admitted.v0` and `cultnet.error.v1` and, against a server that lists both back, resolves only on that write's `write_admitted`, rejects with `CultMeshRudpWriteRefusedError` (now also carrying an `error.v1`) on its refusal and with the new `CultMeshRudpWriteUnconfirmedError` (`reason`: `window_elapsed` after 5000 ms, `session_ended`, or `local_send_failed`) when nothing settles it. Without the option, or against a server that did not advertise, there is no hello and the window is 500 ms; a put nothing settled in it is unconfirmed too, no longer "published" (**behaviour change**: a server that applies a put but answers nothing now yields `CultMeshRudpWriteUnconfirmedError`, as the spec requires). A legacy server's echo of the put's id is not admission: only `write_admitted` is. The flush that sends the put is bounded by the result window, and a send not acknowledged by then is `local_send_failed`. `resultWindowMs` overrides either window. `publishRudpDocumentAndWaitForReceipt` is not changed.
### Breaking ### Breaking
- `publishRudpDocumentOnce` no longer treats a window with no refusal as published: a put nothing settled in its result window (500 ms, or 5000 ms against a receipt-capable server) rejects `CultMeshRudpWriteUnconfirmedError`, and a legacy server's echo of the put's id is not admission. It now resolves `"admitted"` (only on a `write_admitted`) or `"sent"`. **Migration:** a caller whose server cannot answer a write yet (for example Odin, until it is receipt-capable) passes `sendOnly: true` per call and treats `"sent"` as "sent", never as admitted. Receipts stay opt-in (`writeReceipts: true`); neither mode is the default.
- The package is renamed `cultmesh-ts` to `@gamecult/cultmesh-ts` (the repo already scopes `@gamecult/cultcache-ts`), so the unscoped name cannot be claimed by someone else. Import from `@gamecult/cultmesh-ts` and its subpaths; the API is unchanged. Depends on `@gamecult/cultnet-ts`. - The package is renamed `cultmesh-ts` to `@gamecult/cultmesh-ts` (the repo already scopes `@gamecult/cultcache-ts`), so the unscoped name cannot be claimed by someone else. Import from `@gamecult/cultmesh-ts` and its subpaths; the API is unchanged. Depends on `@gamecult/cultnet-ts`.
- `asSchemaAlias` is removed from `CultMeshDocumentHandle`, `CultMeshCollectionHandle` and their bound forms. Read the handle by the schema you mean. - `asSchemaAlias` is removed from `CultMeshDocumentHandle`, `CultMeshCollectionHandle` and their bound forms. Read the handle by the schema you mean.
- `CultMeshDocumentCatalog` (`document`, `tryDocument`, `latest`, `replace`, `submitPrediction`, `authoritativeWriter`, `predictionWriter`, `watch`) and `CultMeshCollectionCatalog` (`collection`, `tryCollection`, `latest`, `watchChanges`) take `<TDocument = never, TSchema>` and yield `CultMeshCatalogValue<TDocument, TSchema>`: an explicit type argument wins, a CultCache document definition names its own value type, and a bare schema descriptor yields `unknown`. A value type is no longer inferred from a value or an expected result. The `parse` option is removed from these calls, and a lookup by type answers with that type's handle or none, not an alias of a compatible schema. - `CultMeshDocumentCatalog` (`document`, `tryDocument`, `latest`, `replace`, `submitPrediction`, `authoritativeWriter`, `predictionWriter`, `watch`) and `CultMeshCollectionCatalog` (`collection`, `tryCollection`, `latest`, `watchChanges`) take `<TDocument = never, TSchema>` and yield `CultMeshCatalogValue<TDocument, TSchema>`: an explicit type argument wins, a CultCache document definition names its own value type, and a bare schema descriptor yields `unknown`. A value type is no longer inferred from a value or an expected result. The `parse` option is removed from these calls, and a lookup by type answers with that type's handle or none, not an alias of a compatible schema.
- `createRudpDocumentServer` takes `writeAuthorityTimeoutMs` (default `CULTMESH_RUDP_DOCUMENT_SERVER_DEFAULT_DECIDE_TIMEOUT_MS`, 5000; a value outside 1..2147483647 throws `RangeError`): a write authority whose `decide()` has not settled by then refuses the write like one that throws (`write_not_authorized`, and `onError` hears of it), so a hung authority no longer holds up the session's later frames. - `createRudpDocumentServer` takes `writeAuthorityTimeoutMs` (default `CULTMESH_RUDP_DOCUMENT_SERVER_DEFAULT_DECIDE_TIMEOUT_MS`, 5000; a value outside 1..2147483647 throws `RangeError`): a write authority whose `decide()` has not settled by then refuses the write like one that throws (`write_not_authorized`, and `onError` hears of it), so a hung authority no longer holds up the session's later frames.
- `createRudpDocumentServer` refuses every peer put and delete unless it is given a `writeAuthority`. A refused write is answered with the typed `write_not_authorized` naming the record key (the bytes pinned in `error-vectors.json`), and `onDocumentPutRaw` is never called; pass `writeAuthority: new CultMeshGrantAllWrites()` to keep the old accept-everything behaviour. The authority is asked before the put is sized or decoded and before any handler runs, with the canonical schema id and the peer's address (the schema id is resolved to its canonical form first, which is the timing residual below); one that throws, rejects or answers anything but `granted: true` refuses. A frame the server cannot read is judged as the write it looks like (a put when it does not decode at all), so an ungranted peer gets `write_not_authorized` for it too; a granted peer gets `onError` and a `malformed_document_put` or `malformed_document_delete` refusal. A put carrying a wrong-typed `shardId` or `shardEpoch` is judged with null shard fields and, once granted, refused with `malformed_document_put`. A refusal's record key and reason are clipped (64 and 96 UTF-8 bytes) so the refusal is at most 309 bytes; a `maxPayloadBytes` below that cannot carry it, and the peer then hears nothing (the failure goes to `onError`). Residual: ungranted peers can still distinguish registered from unregistered schemas by timing. A peer `document_delete` now reaches the new `onDocumentDeleteRaw` once granted (without it, a text refusal) instead of the unsupported-request error. - `createRudpDocumentServer` refuses every peer put and delete unless it is given a `writeAuthority`. A refused write is answered with the typed `write_not_authorized` naming the record key (the bytes pinned in `error-vectors.json`), and `onDocumentPutRaw` is never called; pass `writeAuthority: new CultMeshGrantAllWrites()` to keep the old accept-everything behaviour. The authority is asked before the put is sized or decoded and before any handler runs, with the canonical schema id and the peer's address (the schema id is resolved to its canonical form first, which is the timing residual below); one that throws, rejects or answers anything but `granted: true` refuses. A frame the server cannot read is judged as the write it looks like (a put when it does not decode at all), so an ungranted peer gets `write_not_authorized` for it too; a granted peer gets `onError` and a typed `message_invalid` refusal (`details.field` is `message`). A put carrying a wrong-typed `shardId` or `shardEpoch` is judged with null shard fields and, once granted, refused with `message_invalid` naming that field. A refusal's record key and reason are clipped (64 and 96 UTF-8 bytes) so the refusal is at most 309 bytes; a `maxPayloadBytes` below that cannot carry it, and the peer then hears nothing (the failure goes to `onError`). Residual: ungranted peers can still distinguish registered from unregistered schemas by timing. A peer `document_delete` now reaches the new `onDocumentDeleteRaw` once granted (without it, a text refusal) instead of the unsupported-request error.
- Adding a document or collection to a catalog under a schema id or schema name another type already holds is refused with `SchemaConflictError`. - Adding a document or collection to a catalog under a schema id or schema name another type already holds is refused with `SchemaConflictError`.
### Added ### Added
- `createRudpDocumentServer` ends a session after `CULTMESH_RUDP_DOCUMENT_SERVER_MAX_CONSECUTIVE_REFUSED_FRAMES` (8) consecutive undecodable frames or consecutive deletes the write authority refuses; fewer are answered and the session is kept. Any decoded frame resets the undecodable run, any decoded non-refused-delete frame resets the delete run. The same cap holds in every runtime.
- `CultMeshCatalogValue<TDocument, TSchema>`, exported. - `CultMeshCatalogValue<TDocument, TSchema>`, exported.
- `documentById` and `tryDocumentById` on `CultMeshDocumentCatalog`, which address one document by its document id when a catalog holds several documents of one type. - `documentById` and `tryDocumentById` on `CultMeshDocumentCatalog`, which address one document by its document id when a catalog holds several documents of one type.

View file

@ -24,7 +24,8 @@
"build": "node ../../node_modules/typescript/bin/tsc -p tsconfig.json", "build": "node ../../node_modules/typescript/bin/tsc -p tsconfig.json",
"prepack": "node ../../node_modules/typescript/bin/tsc -p tsconfig.json", "prepack": "node ../../node_modules/typescript/bin/tsc -p tsconfig.json",
"typecheck": "node ../../node_modules/typescript/bin/tsc -p tsconfig.json --pretty false", "typecheck": "node ../../node_modules/typescript/bin/tsc -p tsconfig.json --pretty false",
"test": "node ../../node_modules/typescript/bin/tsc -p tsconfig.json && node ../../node_modules/typescript/bin/tsc -p tsconfig.test.json && node --expose-gc --test dist-test/test/*.test.js" "test": "node ../../node_modules/typescript/bin/tsc -p tsconfig.json && node ../../node_modules/typescript/bin/tsc -p tsconfig.test.json && node --expose-gc --test dist-test/test/*.test.js",
"test:interop": "npm --prefix ../.. run build --workspace @gamecult/cultcache-ts --workspace @gamecult/cultnet-ts && node ../../node_modules/typescript/bin/tsc -p tsconfig.json && node ../../node_modules/typescript/bin/tsc -p tsconfig.test.json && node --test --test-force-exit dist-test/test/interop/write-receipt-interop.test.js"
}, },
"keywords": [ "keywords": [
"cultmesh", "cultmesh",

View file

@ -20,6 +20,11 @@ import {
CultNetRudpSocketTransportConnection, CultNetRudpSocketTransportConnection,
rudpClientBindHost, rudpClientBindHost,
CultNetSchemaCatalog, CultNetSchemaCatalog,
CULTNET_WRITE_RECEIPT_MESSAGE_VERSIONS,
WriteTracker,
helloAdvertisesWriteReceipts,
resultWindowMs,
cultNetMessageInvalidError,
cultNetTextError, cultNetTextError,
cultNetWriteNotAuthorizedError, cultNetWriteNotAuthorizedError,
CultNetShardCatalog, CultNetShardCatalog,
@ -43,6 +48,8 @@ import {
type CultNetSchemaCatalogOptions, type CultNetSchemaCatalogOptions,
type CultNetSnapshotResponseRawMessage, type CultNetSnapshotResponseRawMessage,
type CultNetWireContract, type CultNetWireContract,
type WriteSettlement,
type WriteUnconfirmedReason,
} from "@gamecult/cultnet-ts"; } from "@gamecult/cultnet-ts";
import { import {
@ -4120,6 +4127,16 @@ export interface CultMeshRudpDocumentServerOptions extends CultMeshRudpSocketOpt
/** A write authority that has not answered in this long has refused the write. */ /** A write authority that has not answered in this long has refused the write. */
export const CULTMESH_RUDP_DOCUMENT_SERVER_DEFAULT_DECIDE_TIMEOUT_MS = 5_000; export const CULTMESH_RUDP_DOCUMENT_SERVER_DEFAULT_DECIDE_TIMEOUT_MS = 5_000;
/**
* How many consecutive refused frames of one kind a `createRudpDocumentServer` session may send before it is
* ended: undecodable frames (reset by any frame that decodes) and deletes the write authority refuses (reset by any
* decoded frame that is not a refused delete). Fewer are answered with a typed refusal and the session is kept; the
* frame that reaches this number is answered too, then the session ends. The same number is the cap in every runtime.
*/
export const CULTMESH_RUDP_DOCUMENT_SERVER_MAX_CONSECUTIVE_REFUSED_FRAMES = 8;
/** How long a session whose capped frame was answered waits for the peer to acknowledge the answers before it ends anyway. */
const CULTMESH_RUDP_DOCUMENT_SERVER_CAPPED_ANSWER_GRACE_MS = 2000;
/** Replies larger than one fragment are split into fragments of at most this many bytes. */ /** Replies larger than one fragment are split into fragments of at most this many bytes. */
export const CULTMESH_RUDP_DOCUMENT_SERVER_DEFAULT_MAX_FRAGMENT_BYTES = 1200; export const CULTMESH_RUDP_DOCUMENT_SERVER_DEFAULT_MAX_FRAGMENT_BYTES = 1200;
@ -4172,7 +4189,64 @@ export interface CultMeshRudpDocumentServer {
close(): void; close(): void;
} }
/**
* A write the peer refused: the typed `cultnet.error.v0` it answered with (`write_not_authorized`,
* `message_invalid`, ...), whichever of the transport ack and the result frame came first.
*/
export class CultMeshRudpWriteRefusedError extends Error {
readonly code: string | undefined;
readonly response: Extract<CultNetMessage, { schemaVersion: "cultnet.error.v0" | "cultnet.error.v1" }>;
constructor(response: Extract<CultNetMessage, { schemaVersion: "cultnet.error.v0" | "cultnet.error.v1" }>) {
super(response.error);
this.name = "CultMeshRudpWriteRefusedError";
this.code = response.code ?? undefined;
this.response = response;
}
}
/**
* Nothing settled the write (write-receipt.md, "unconfirmed"): the result window ran out with no answer, the
* session ended first, or the client's own send failed. Not success and not refusal; a caller that needs admission treats it as not admitted.
*/
export class CultMeshRudpWriteUnconfirmedError extends Error {
readonly reason: WriteUnconfirmedReason;
readonly messageId: string;
constructor(messageId: string, reason: WriteUnconfirmedReason) {
super(`Write ${messageId} was not confirmed: ${reason}.`);
this.name = "CultMeshRudpWriteUnconfirmedError";
this.reason = reason;
this.messageId = messageId;
}
}
export interface CultMeshRudpDocumentPublishOptions extends CultMeshRudpPeerOptions { export interface CultMeshRudpDocumentPublishOptions extends CultMeshRudpPeerOptions {
/**
* How long, counted from the send, the put may wait for its result frame. A reliable-transport ack is not
* admission; the outcome is the peer's result frame. Default: 5000 ms against a server whose hello lists the
* write-receipt frames, 500 ms otherwise. A put nothing settled within the window is unconfirmed, never
* published, whatever the server advertised: only a `write_admitted` is admission. The send itself is also
* bounded by the window (`flushTimeoutMs` is the upper bound); a send not acknowledged by then is
* unconfirmed as `local_send_failed`.
*/
resultWindowMs?: number;
/**
* Declare the write-receipt capability: send a hello listing `cultnet.write_admitted.v0` and `cultnet.error.v1`
* before the put. A server that lists both back is receipt-capable: the put then resolves only on its
* `write_admitted`, rejects with `CultMeshRudpWriteRefusedError` on its `error.v1`, and rejects with
* `CultMeshRudpWriteUnconfirmedError` when nothing settles it. Unset, no hello is sent and the server answers
* as it always did.
*/
writeReceipts?: boolean;
/** With `writeReceipts`, how long to wait for the server's hello before treating it as legacy. Default 500 ms. */
helloWaitMs?: number;
/**
* Send the put and report only that it was sent: no hello, no wait for a result frame. Resolves `"sent"` once the
* transport has acknowledged the put (`flushTimeoutMs`), rejects `CultMeshRudpWriteUnconfirmedError`
* (`local_send_failed`) if it cannot be sent, and never resolves `"admitted"`: whether the server applied or
* refused it is unknown. For callers whose server cannot answer a write yet. Opt-in per call; not combinable with
* `writeReceipts`.
*/
sendOnly?: boolean;
messageId?: string; messageId?: string;
sourceRuntimeId?: string; sourceRuntimeId?: string;
sourceAgentId?: string; sourceAgentId?: string;
@ -5536,6 +5610,11 @@ export class CultMesh {
closed: boolean; closed: boolean;
/** A packet it owed could never be sent, so the session ended. */ /** A packet it owed could never be sent, so the session ended. */
unsendable: boolean; unsendable: boolean;
/** Consecutive undecodable frames, and consecutive deletes the authority refused. */
unreadableInARow: number;
refusedDeletesInARow: number;
/** Set once the peer's capped frame is answered: the session ends when the answers are acknowledged, or at this time. */
endingDeadlineMs?: number;
work: Promise<void>; work: Promise<void>;
}; };
const sessions = new Map<string, SessionRecord>(); const sessions = new Map<string, SessionRecord>();
@ -5590,6 +5669,8 @@ export class CultMesh {
connectPayload, connectPayload,
closed: false, closed: false,
unsendable: false, unsendable: false,
unreadableInARow: 0,
refusedDeletesInARow: 0,
session: new CultNetRudpSession({ session: new CultNetRudpSession({
connectionId, connectionId,
initialSequence: options.initialSequence, initialSequence: options.initialSequence,
@ -5654,7 +5735,8 @@ export class CultMesh {
record: SessionRecord, record: SessionRecord,
payload: Uint8Array, payload: Uint8Array,
): Promise<void> { ): Promise<void> {
if (record.closed) return; // Nothing after the capped frame is served while its answer is delivered.
if (record.closed || record.endingDeadlineMs !== undefined) return;
let decoded: unknown; let decoded: unknown;
let message: CultNetMessage; let message: CultNetMessage;
try { try {
@ -5664,12 +5746,21 @@ export class CultMesh {
// A frame the server cannot read is judged as the write it looks like (a put when it does not even decode), so // A frame the server cannot read is judged as the write it looks like (a put when it does not even decode), so
// an ungranted peer hears the same refusal for it as for any other write, not a parse error. // an ungranted peer hears the same refusal for it as for any other write, not a parse error.
const write = readableWrite(decoded); const write = readableWrite(decoded);
if (decoded !== undefined && !write) throw error; record.unreadableInARow += 1;
if (await writeRefused(record, write?.operation ?? "put", write?.schemaId ?? "", write?.recordKey ?? "", null, null)) return; if (decoded !== undefined && !write) {
reportError(error); // It decodes but is no message the parser takes (an unknown schemaVersion, a non-map) and is no write the
sendSchemaMessage(record, malformedWriteError(write?.operation ?? "put", "CultMesh RUDP document server could not read the write.")); // authority could judge: answered and counted like any other unreadable frame.
reportError(error);
sendSchemaMessage(record, cultNetMessageInvalidError("message", "CultMesh RUDP document server could not read the message."));
} else if (!await writeRefused(record, write?.operation ?? "put", write?.schemaId ?? "", write?.recordKey ?? "", null, null)) {
reportError(error);
sendSchemaMessage(record, cultNetMessageInvalidError("message", "CultMesh RUDP document server could not read the write."));
}
if (record.unreadableInARow >= CULTMESH_RUDP_DOCUMENT_SERVER_MAX_CONSECUTIVE_REFUSED_FRAMES) endCappedSession(record);
return; return;
} }
record.unreadableInARow = 0;
if (message.schemaVersion !== "cultnet.document_delete.v0") record.refusedDeletesInARow = 0;
switch (message.schemaVersion) { switch (message.schemaVersion) {
case "cultnet.snapshot_request.v0": { case "cultnet.snapshot_request.v0": {
if (!options.getCache) { if (!options.getCache) {
@ -5716,7 +5807,10 @@ export class CultMesh {
malformedShard ? null : (shardEpoch as number | undefined) ?? null, malformedShard ? null : (shardEpoch as number | undefined) ?? null,
)) return; )) return;
if (malformedShard) { if (malformedShard) {
sendSchemaMessage(record, malformedWriteError("put", "Raw put messages must carry a string shardId and an integer shardEpoch.")); sendSchemaMessage(record, cultNetMessageInvalidError(
typeof shardId !== "undefined" && typeof shardId !== "string" ? "shardId" : "shardEpoch",
"Raw put messages must carry a string shardId and an integer shardEpoch.",
));
return; return;
} }
} }
@ -5740,7 +5834,12 @@ export class CultMesh {
} }
return; return;
case "cultnet.document_delete.v0": { case "cultnet.document_delete.v0": {
if (await writeRefused(record, "delete", message.schemaId, message.recordKey, null, null)) return; if (await writeRefused(record, "delete", message.schemaId, message.recordKey, null, null)) {
record.refusedDeletesInARow += 1;
if (record.refusedDeletesInARow >= CULTMESH_RUDP_DOCUMENT_SERVER_MAX_CONSECUTIVE_REFUSED_FRAMES) endCappedSession(record);
return;
}
record.refusedDeletesInARow = 0;
if (!options.onDocumentDeleteRaw) { if (!options.onDocumentDeleteRaw) {
sendSchemaMessage(record, cultNetTextError("CultMesh RUDP document server has no delete handler.")); sendSchemaMessage(record, cultNetTextError("CultMesh RUDP document server has no delete handler."));
return; return;
@ -5785,12 +5884,6 @@ export class CultMesh {
return undefined; return undefined;
} }
// Not a `code`: malformed_document_put/delete are outside the schema's closed code enum, which TS and Rust peers
// refuse to decode (docs/cultnet-error-contract-cut.md), so the name leads the text of a codeless error instead.
function malformedWriteError(operation: CultMeshRemoteWriteOperation, text: string): CultNetMessage {
return cultNetTextError(`malformed_document_${operation}: ${text}`);
}
/** /**
* Asks the write authority about a put or delete, before anything of the write is sized, resolved or decoded, and * Asks the write authority about a put or delete, before anything of the write is sized, resolved or decoded, and
* answers a refusal with the typed `write_not_authorized` naming the record. True when the write is refused. * answers a refusal with the typed `write_not_authorized` naming the record. True when the write is refused.
@ -5897,6 +5990,17 @@ export class CultMesh {
sendRudpDatagram(socket, goodbye, record.remote.port, record.remote.address, () => {}); sendRudpDatagram(socket, goodbye, record.remote.port, record.remote.address, () => {});
} }
/**
* A session that reached the cap of consecutive refused frames: its last refusal went out like any reply
* (reliable, so a lost datagram is resent), and the session ends once the peer has acknowledged every reply or
* the grace runs out. The resend timer ends it.
*/
function endCappedSession(record: SessionRecord): void {
const key = `${record.remote.address}:${record.remote.port}`;
if (sessions.get(key) !== record) return;
record.endingDeadlineMs ??= Date.now() + CULTMESH_RUDP_DOCUMENT_SERVER_CAPPED_ANSWER_GRACE_MS;
}
/** The session owes its peer a packet that can never be sent as built. */ /** The session owes its peer a packet that can never be sent as built. */
function endUnsendableSession(record: SessionRecord, error: Error): void { function endUnsendableSession(record: SessionRecord, error: Error): void {
if (record.unsendable) return; if (record.unsendable) return;
@ -6030,6 +6134,11 @@ export class CultMesh {
closeSession(record); closeSession(record);
continue; continue;
} }
if (record.endingDeadlineMs !== undefined
&& (record.session.outstandingReliablePacketCount === 0 || nowMs >= record.endingDeadlineMs)) {
endRefusedSession(key, record);
continue;
}
for (const packet of record.session.dueResends(nowMs)) { for (const packet of record.session.dueResends(nowMs)) {
sendPacket(record, packet); sendPacket(record, packet);
} }
@ -6107,25 +6216,88 @@ export class CultMesh {
recordKey: string, recordKey: string,
value: CultCacheDocumentValue<TDefinition>, value: CultCacheDocumentValue<TDefinition>,
options: CultMeshRudpDocumentPublishOptions = {}, options: CultMeshRudpDocumentPublishOptions = {},
): Promise<void> { ): Promise<"admitted" | "sent"> {
requireNonEmpty(runtimeId, "runtimeId"); requireNonEmpty(runtimeId, "runtimeId");
requireNonEmpty(recordKey, "recordKey"); requireNonEmpty(recordKey, "recordKey");
if (options.sendOnly && options.writeReceipts) throw new TypeError("sendOnly and writeReceipts cannot be combined.");
const registry = new CultNetDocumentRegistry([binding]); const registry = new CultNetDocumentRegistry([binding]);
const messageId = options.messageId ?? `${runtimeId}:${binding.definition.type}:${recordKey}`;
// Built before any connection and outside every try: a put that cannot be built is the caller's error, with its cause.
const put = registry.createRawDocumentPutMessage(
binding,
messageId,
recordKey,
value,
{
sourceRuntimeId: options.sourceRuntimeId ?? runtimeId,
sourceAgentId: options.sourceAgentId,
sourceRole: options.sourceRole,
tags: options.tags,
},
);
const peer = await CultMesh.createRudpPeer(runtimeId, connectionId, endpoint, options); const peer = await CultMesh.createRudpPeer(runtimeId, connectionId, endpoint, options);
try { try {
peer.send(registry.createRawDocumentPutMessage( if (options.sendOnly) {
binding, try {
options.messageId ?? `${runtimeId}:${binding.definition.type}:${recordKey}`, peer.send(put);
recordKey, await peer.flush(options.flushTimeoutMs ?? 30_000);
value, } catch {
{ throw new CultMeshRudpWriteUnconfirmedError(messageId, "local_send_failed");
sourceRuntimeId: options.sourceRuntimeId ?? runtimeId, }
sourceAgentId: options.sourceAgentId, return "sent";
sourceRole: options.sourceRole, }
tags: options.tags, // A transport ack is not admission: the put completes on its result frame, and the WriteTracker alone decides
}, // how it settles (write-receipt.md, "Client rules"). This function only maps the settlement to resolve/throw.
)); let receiptCapable = false;
await peer.flush(options.flushTimeoutMs ?? 30_000); let tracker: WriteTracker | undefined;
let settlement: WriteSettlement | undefined;
let wake: (() => void) | undefined;
let serverHello: (() => void) | undefined;
peer.on("message", (message: CultNetMessage) => {
if (message.schemaVersion === "cultnet.hello.v0") {
receiptCapable = helloAdvertisesWriteReceipts(message);
serverHello?.();
} else {
tracker?.frame(message, Date.now());
}
});
peer.on("close", () => tracker?.sessionEnd(Date.now()));
if (options.writeReceipts) {
const answered = new Promise<void>((resolve) => { serverHello = resolve; });
peer.sendHello({
schemaVersion: "cultnet.hello.v0", runtimeId, runtimeKind: "cultmesh-ts",
supportedMessageVersions: [...CULTNET_WRITE_RECEIPT_MESSAGE_VERSIONS],
});
let helloTimer: NodeJS.Timeout | undefined;
await Promise.race([answered, new Promise<void>((resolve) => { helloTimer = setTimeout(resolve, options.helloWaitMs ?? 500); })]);
clearTimeout(helloTimer);
}
const writeTracker = new WriteTracker({
windowMs: options.resultWindowMs ?? resultWindowMs(receiptCapable),
onSettle: (_id, result) => { settlement = result; wake?.(); },
});
tracker = writeTracker;
writeTracker.send(messageId, Date.now());
try {
peer.send(put);
// The flush may not outlast the result window: a send nobody acknowledged by then is a local send failure.
const windowRemaining = writeTracker.nextDeadlineMs(Date.now()) ?? 0;
await peer.flush(Math.min(options.flushTimeoutMs ?? 30_000, Math.max(windowRemaining, 1)));
} catch {
// Our own send failed or was never acknowledged: the put may or may not have left. Never a refusal.
writeTracker.sendFailed(messageId, Date.now());
}
while (!settlement) {
const remaining = writeTracker.nextDeadlineMs(Date.now()) ?? 0;
await new Promise<void>((resolve) => {
const timer = setTimeout(resolve, remaining);
wake = () => { clearTimeout(timer); resolve(); };
});
writeTracker.tick(Date.now());
}
if (settlement.outcome === "refused") throw new CultMeshRudpWriteRefusedError(settlement.response);
if (settlement.outcome === "unconfirmed") throw new CultMeshRudpWriteUnconfirmedError(messageId, settlement.reason);
return "admitted";
} finally { } finally {
peer.close(); peer.close();
} }
@ -6144,6 +6316,7 @@ export class CultMesh {
const receipt = new Promise<CultCacheDocumentValue<TReceiptDefinition>>((resolve, reject) => { const receipt = new Promise<CultCacheDocumentValue<TReceiptDefinition>>((resolve, reject) => {
const timeout = setTimeout(() => { cleanup(); reject(new Error(`Timed out waiting for RUDP application receipt for ${recordKey}.`)); }, options.receiptTimeoutMs ?? 5_000); const timeout = setTimeout(() => { cleanup(); reject(new Error(`Timed out waiting for RUDP application receipt for ${recordKey}.`)); }, options.receiptTimeoutMs ?? 5_000);
const onMessage = (message: CultNetMessage): void => { const onMessage = (message: CultNetMessage): void => {
if (message.schemaVersion === "cultnet.error.v0") { cleanup(); reject(new CultMeshRudpWriteRefusedError(message)); return; }
if (message.schemaVersion !== "cultnet.document_put_raw.v0" || message.messageId !== messageId || message.document.schemaId !== receiptBinding.definition.schemaId) return; if (message.schemaVersion !== "cultnet.document_put_raw.v0" || message.messageId !== messageId || message.document.schemaId !== receiptBinding.definition.schemaId) return;
try { try {
const decoded = decodeDocumentPayload(receiptBinding.definition, message.document.payload); const decoded = decodeDocumentPayload(receiptBinding.definition, message.document.payload);

View file

@ -0,0 +1,120 @@
// Heimdall's invariant: no client may treat a reliable-transport ack as write admission. A put completes only
// on its result frame (write_not_authorized, or the success reply). Each test serves a client a peer that
// acknowledges the put and refuses it, the ack arriving before the refusal and after it.
import assert from "node:assert/strict";
import { createSocket, type Socket } from "node:dgram";
import test from "node:test";
import { z } from "zod";
import { defineDocumentType } from "@gamecult/cultcache-ts";
import {
CultNetDocumentRegistry,
CultNetRudpSession,
cultNetWriteNotAuthorizedError,
decodeRudpPacket,
defineCultNetDocumentBinding,
encodeCultNetMessageForWire,
encodeRudpPacket,
} from "@gamecult/cultnet-ts";
import { encodeWireValue } from "@gamecult/cultnet-ts/wire";
import { CultMesh, CultMeshGrantAllWrites, CultMeshRudpWriteRefusedError, CultMeshRudpWriteUnconfirmedError } from "../src/index";
const isTheRefusal = (error: unknown): boolean =>
error instanceof CultMeshRudpWriteRefusedError && error.code === "write_not_authorized";
const note = defineDocumentType({
type: "cultmesh.completion_note",
schemaId: "cultmesh.completion_note.v0",
schema: z.object({ noteId: z.string(), text: z.string() }),
name: "noteId",
});
const binding = defineCultNetDocumentBinding({ definition: note });
const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
type Order = "ack-then-refusal" | "refusal-then-ack";
/** A peer that acknowledges every reliable put and refuses it with write_not_authorized, in the given order. */
async function refusingPeer(connectionId: number, order: Order) {
const socket: Socket = createSocket("udp4");
await new Promise<void>((resolve) => socket.bind(0, "127.0.0.1", resolve));
const session = new CultNetRudpSession({ connectionId, resendDelayMs: 50 });
const timers = new Set<NodeJS.Timeout>();
let closed = false;
const send = (packet: Parameters<typeof encodeRudpPacket>[0], port: number, address: string) => {
if (!closed) socket.send(encodeRudpPacket(packet), port, address);
};
const later = (action: () => void) => { const timer = setTimeout(() => { timers.delete(timer); action(); }, 120); timers.add(timer); };
socket.on("message", (wire, remote) => {
const packet = decodeRudpPacket(wire);
const now = Date.now();
if (packet.packetType === "connect") {
send(session.acceptConnect(packet, now), remote.port, remote.address);
return;
}
const result = session.receive(packet, now);
if (result.reply) send(result.reply, remote.port, remote.address);
for (const frame of result.delivered) {
if (frame.channelId !== "schema") continue;
const ack = session.createAckForReceived(packet.sequence);
const refusal = session.sendMany(
"schema",
encodeWireValue(encodeCultNetMessageForWire(cultNetWriteNotAuthorizedError("tests:completion:a"), "cultnet.schema.v0")),
{ reliable: true, ordered: true, nowMs: now },
);
if (order === "ack-then-refusal") {
send(ack, remote.port, remote.address);
later(() => refusal.forEach((p) => send(p, remote.port, remote.address)));
} else {
// The refusal carries no acknowledgement of its own; the ack follows it.
refusal.forEach((p) => send({ ...p, ack: 0, ackMask: 0 }, remote.port, remote.address));
later(() => send(ack, remote.port, remote.address));
}
}
if (packet.reliable) send(session.createAckForReceived(packet.sequence), remote.port, remote.address);
});
return { endpoint: `rudp://127.0.0.1:${socket.address().port}`, close: () => { closed = true; timers.forEach(clearTimeout); socket.close(); } };
}
const orders: Order[] = ["ack-then-refusal", "refusal-then-ack"];
let nextConnection = 0x10205000;
for (const order of orders) {
test(`publishRudpDocumentOnce does not complete on the ack of a refused put (${order})`, async () => {
const peer = await refusingPeer(nextConnection, order);
try {
await assert.rejects(
CultMesh.publishRudpDocumentOnce("completion-once", nextConnection++, peer.endpoint, binding, "tests:completion:a", { noteId: "a", text: "x" }, { resendDelayMs: 50, resendPollMs: 5, flushTimeoutMs: 2_000 }),
isTheRefusal,
);
} finally { peer.close(); }
});
test(`publishRudpDocumentAndWaitForReceipt reports a refused put as refused, never as success (${order})`, async () => {
const peer = await refusingPeer(nextConnection, order);
try {
await assert.rejects(
CultMesh.publishRudpDocumentAndWaitForReceipt("completion-receipt", nextConnection++, peer.endpoint, binding, "tests:completion:a", { noteId: "a", text: "x" }, binding, { resendDelayMs: 50, resendPollMs: 5, receiptTimeoutMs: 800 }),
isTheRefusal,
);
} finally { peer.close(); }
await sleep(10);
});
}
test("a put the server applied but did not answer is unconfirmed, never published", async () => {
const connectionId = nextConnection++;
const puts: string[] = [];
const server = CultMesh.createRudpDocumentServer("completion-admit", connectionId, {
documents: new CultNetDocumentRegistry([binding]),
writeAuthority: new CultMeshGrantAllWrites(),
bindHost: "127.0.0.1", bindPort: 0, resendDelayMs: 50, resendPollMs: 5,
onDocumentPutRaw: (put) => { puts.push(put.recordKey); },
});
await server.start();
try {
await assert.rejects(
CultMesh.publishRudpDocumentOnce("completion-admit-client", connectionId, `rudp://127.0.0.1:${server.bind.port}`, binding, "tests:completion:a", { noteId: "a", text: "x" }, { resendDelayMs: 50, resendPollMs: 5, resultWindowMs: 200 }),
(error: unknown) => error instanceof CultMeshRudpWriteUnconfirmedError && error.reason === "window_elapsed",
);
assert.deepEqual(puts, ["tests:completion:a"], "the server did apply it; this server just does not answer");
} finally { server.close(); }
});

View file

@ -21,7 +21,7 @@ import {
type CultNetMessage, type CultNetMessage,
type CultNetSnapshotResponseRawMessage, type CultNetSnapshotResponseRawMessage,
} from "@gamecult/cultnet-ts"; } from "@gamecult/cultnet-ts";
import { CultMesh, CultMeshGrantAllWrites, CultMeshRudpUnservableDocumentError } from "../src/index"; import { CultMesh, CultMeshGrantAllWrites, CultMeshRudpUnservableDocumentError, CultMeshRudpWriteUnconfirmedError } from "../src/index";
// A record's payload as the bytes a peer would put on the wire. // A record's payload as the bytes a peer would put on the wire.
const payloadOf = (value: unknown) => documentPayloadFromBytes(encodeWireValue(value)); const payloadOf = (value: unknown) => documentPayloadFromBytes(encodeWireValue(value));
@ -2968,25 +2968,31 @@ test("CultMesh TS publishes one registered document to a RUDP catalog", async ()
try { try {
await server.start(); await server.start();
await CultMesh.publishRudpDocumentOnce( // This server answers a put with nothing, so the writer cannot know the put landed: unconfirmed, though it did.
"cultmesh-ts-muninn", await assert.rejects(
connectionId, CultMesh.publishRudpDocumentOnce(
`rudp://127.0.0.1:${server.bind.port}`, "cultmesh-ts-muninn",
defineCultNetDocumentBinding({ definition: noteDocument }), connectionId,
"note:odin", `rudp://127.0.0.1:${server.bind.port}`,
{ defineCultNetDocumentBinding({ definition: noteDocument }),
noteId: "note:odin", "note:odin",
body: "pay respects once", {
}, noteId: "note:odin",
{ body: "pay respects once",
resendDelayMs: 25, },
resendPollMs: 5, {
maxFragmentBytes: 1024, resendDelayMs: 25,
maxPendingReliablePackets: 16, resendPollMs: 5,
connectTimeoutMs: 1_000, maxFragmentBytes: 1024,
flushTimeoutMs: 25, maxPendingReliablePackets: 16,
sourceRole: "test-provider", connectTimeoutMs: 1_000,
}, // The flush is bounded by the result window, so the window must outlast the transport ack on a loaded
// runner: a short one turns a slow ack into local_send_failed, which is not the path this test checks.
resultWindowMs: 750,
sourceRole: "test-provider",
},
),
(error: unknown) => error instanceof CultMeshRudpWriteUnconfirmedError && error.reason === "window_elapsed",
); );
await waitFor(() => received !== undefined, "published RUDP document"); await waitFor(() => received !== undefined, "published RUDP document");

View file

@ -0,0 +1,434 @@
// The TypeScript write-receipt client (cultnet-ts WriteTracker and decoder, cultmesh-ts one-shot writer) against the two real
// receipt servers over loopback, as child processes on ephemeral ports:
//
// rust packages/cultmesh-rs/examples/receipt_interop_server.rs, the RUDP document server, reached over RUDP.
// csharp tests/GameCult.Networking.InteropPeer `receipt-serve`, CultNetDatabaseServer on Server (LiteNetLib). TypeScript has no
// LiteNetLib client, so a second child, `receipt-relay`, carries bytes between the test and the server (4-byte
// big-endian length, then the CultNet frame: CultNetPeer's own stream framing). It decodes and judges nothing.
//
// Every scenario runs against both targets; its outcome is read from the WriteTracker (admitted, refused, unconfirmed), and the
// server's own stdout facts say what was committed. Contract: contracts/cultnet/interop/write-receipt.md and its vectors.
//
// Needs cargo and dotnet; run it with `npm run test:interop` in packages/cultmesh-ts (CARGO and DOTNET override the commands;
// RECEIPT_INTEROP_PREBUILT=1 skips the two builds).
import assert from "node:assert/strict";
import { execFile, spawn, spawnSync, type ChildProcess } from "node:child_process";
import { readFileSync } from "node:fs";
import { join, resolve } from "node:path";
import { Duplex } from "node:stream";
import { after, before, beforeEach, describe, test } from "node:test";
import { promisify } from "node:util";
import { encode } from "@msgpack/msgpack";
import { z } from "zod";
import { defineDocumentType } from "@gamecult/cultcache-ts";
import {
CULTNET_WRITE_RECEIPT_MESSAGE_VERSIONS,
CultNetPeer,
WriteTracker,
defineCultNetDocumentBinding,
helloAdvertisesWriteReceipts,
resultWindowMs,
type CultNetErrorV1Message,
type CultNetHelloMessage,
type CultNetMessage,
type WriteSettlement,
} from "@gamecult/cultnet-ts";
import { CultMesh, CultMeshRudpWriteRefusedError, CultMeshRudpWriteUnconfirmedError } from "../../src/index";
const execFileAsync = promisify(execFile);
// dist-test/test/interop -> the repository root
const repoRoot = resolve(__dirname, "..", "..", "..", "..", "..");
const cargo = process.env.CARGO ?? "cargo";
const dotnet = process.env.DOTNET ?? "dotnet";
const rustManifestDir = join(repoRoot, "packages", "cultmesh-rs");
const csharpProject = join(repoRoot, "tests", "GameCult.Networking.InteropPeer", "GameCult.Networking.InteropPeer.csproj");
const csharpDll = join(repoRoot, "bin", "GameCult.Networking.InteropPeer", "Debug", "net10.0", "GameCult.Networking.InteropPeer.dll");
// Set when the server binaries were built beforehand (the builds themselves cannot start under a tight address-space cap).
const prebuilt = process.env.RECEIPT_INTEROP_PREBUILT === "1";
const vectors = JSON.parse(readFileSync(join(repoRoot, "contracts", "cultnet", "interop", "write-receipt-vectors.json"), "utf8"));
const SCHEMA_ID = "receipt.interop_note.v1";
const UNREADABLE = "message_invalid: the message could not be read";
const sleep = (ms: number) => new Promise<void>((done) => setTimeout(done, ms));
async function waitFor(predicate: () => boolean, what: string, ms = 5_000): Promise<void> {
const startedAt = Date.now();
while (!predicate()) {
if (Date.now() - startedAt > ms) throw new Error(`timed out waiting for ${what}`);
await sleep(5);
}
}
// ---- the servers -------------------------------------------------------------------------------------------------------
interface Fact { event: string; recordKey?: string; messageId?: string; [key: string]: unknown }
/** A running server child and the JSON facts it prints on stdout. */
class ServerProcess {
readonly facts: Fact[] = [];
stderr = "";
port = 0;
constructor(readonly child: ChildProcess) {
let pending = "";
child.stdout!.on("data", (chunk: Buffer) => {
pending += chunk.toString("utf8");
for (let at = pending.indexOf("\n"); at >= 0; at = pending.indexOf("\n")) {
const line = pending.slice(0, at).trim();
pending = pending.slice(at + 1);
if (line) this.facts.push(JSON.parse(line) as Fact);
}
});
child.stderr!.on("data", (chunk: Buffer) => { this.stderr += chunk.toString("utf8"); });
}
/** How many times `event` was printed for `recordKey`. */
count(event: string, recordKey: string): number {
return this.facts.filter((fact) => fact.event === event && fact.recordKey === recordKey).length;
}
async stop(): Promise<void> {
this.child.stdin!.end();
await new Promise<void>((done) => {
const killer = setTimeout(() => this.child.kill("SIGKILL"), 5_000);
this.child.once("exit", () => { clearTimeout(killer); done(); });
if (this.child.exitCode !== null) done();
});
}
}
async function launch(command: string, args: string[]): Promise<ServerProcess> {
const server = new ServerProcess(spawn(command, args, { stdio: ["pipe", "pipe", "pipe"] }));
await waitFor(() => server.facts.some((fact) => fact.event === "listening") || server.child.exitCode !== null, "the server to listen", 60_000);
const listening = server.facts.find((fact) => fact.event === "listening");
if (!listening) throw new Error(`the server exited before listening:\n${server.stderr}`);
server.port = listening.port as number;
return server;
}
/** One connection to a server: the peer that speaks CultNet, and how to close it. */
interface Connection { peer: CultNetPeer; close(): void }
interface Target {
name: string;
/** Shard fields the server's database requires on a write, if any. */
shard?: { shardId: string; shardEpoch: number };
/** A document payload this target's store reads. */
payload(text: string): Uint8Array;
start(): Promise<ServerProcess>;
connect(server: ServerProcess): Promise<Connection>;
}
let nextConnectionId = 0x20206000;
const rust: Target = {
name: "rust",
payload: (text) => encode({ text }),
async start() {
if (!prebuilt) await execFileAsync(cargo, ["build", "--example", "receipt_interop_server", "--message-format", "short"], { cwd: rustManifestDir });
const metadata = spawnSync(cargo, ["metadata", "--format-version", "1", "--no-deps"], { cwd: rustManifestDir, encoding: "utf8" });
if (metadata.status !== 0) throw new Error(`cargo metadata failed: ${metadata.stderr}`);
const exe = join(JSON.parse(metadata.stdout).target_directory, "debug", "examples", `receipt_interop_server${process.platform === "win32" ? ".exe" : ""}`);
return launch(exe, []);
},
async connect(server) {
const peer = await CultMesh.createRudpPeer("receipt-interop", nextConnectionId++, { host: "127.0.0.1", port: server.port, uri: `rudp://127.0.0.1:${server.port}` });
return { peer, close: () => peer.close() };
},
};
// The C# server admits 30 connection requests per IP per minute (Server.MaxConnectionAttemptsPerMinute) and answers the rest with
// no hello. Every scenario wants a fresh session, so before each C# test the harness waits until the last minute holds at most
// CONNECTIONS_PER_MINUTE connections, leaving room for the two a test may open. The server's limit stays as it is.
const CONNECTIONS_PER_MINUTE = 24;
const connectedAt: number[] = [];
async function awaitConnectionBudget(): Promise<void> {
for (;;) {
const now = Date.now();
while (connectedAt.length > 0 && now - connectedAt[0] >= 61_000) connectedAt.shift();
if (connectedAt.length <= CONNECTIONS_PER_MINUTE) return;
await sleep(connectedAt[0] + 61_000 - now);
}
}
const csharp: Target = {
name: "csharp",
shard: { shardId: "receipt-interop-shard", shardEpoch: 1 },
payload: (text) => encode([text]),
async start() {
if (!prebuilt) await execFileAsync(dotnet, ["build", csharpProject, "-nologo", "-v", "q"], { cwd: repoRoot, maxBuffer: 64 * 1024 * 1024 });
return launch(dotnet, [csharpDll, "receipt-serve"]);
},
async connect(server) {
connectedAt.push(Date.now());
const relay = spawn(dotnet, [csharpDll, "receipt-relay", "--port", String(server.port)], { stdio: ["pipe", "pipe", "ignore"] });
const peer = new CultNetPeer(Duplex.from({ readable: relay.stdout!, writable: relay.stdin! }), { wireContract: "cultnet.schema.v0" });
return { peer, close: () => { relay.stdin!.end(); relay.kill(); } };
},
};
// ---- a client session: one connection, the frames it saw, one WriteTracker per write -----------------------------------------
class Session {
readonly frames: CultNetMessage[] = [];
readonly invalid: Error[] = [];
private readonly trackers = new Set<WriteTracker>();
private serverReceipts = false;
private constructor(readonly target: Target, private readonly connection: Connection) {
connection.peer.on("message", (message: CultNetMessage) => {
if (message.schemaVersion === "cultnet.hello.v0") this.serverReceipts = helloAdvertisesWriteReceipts(message);
this.frames.push(message);
for (const tracker of this.trackers) tracker.frame(message, Date.now());
});
connection.peer.on("invalidMessage", (error: Error) => this.invalid.push(error));
connection.peer.on("close", () => { for (const tracker of this.trackers) tracker.sessionEnd(Date.now()); });
}
static async open(target: Target, server: ServerProcess): Promise<Session> {
return new Session(target, await target.connect(server));
}
/** Declares the receipt capability and waits for the server's own hello. */
async hello(versions: readonly string[] = CULTNET_WRITE_RECEIPT_MESSAGE_VERSIONS): Promise<CultNetHelloMessage> {
this.connection.peer.sendHello({ schemaVersion: "cultnet.hello.v0", runtimeId: "receipt-interop", runtimeKind: "cultmesh-ts", supportedMessageVersions: [...versions] });
await waitFor(() => this.frames.some((frame) => frame.schemaVersion === "cultnet.hello.v0"), "the server's hello");
return this.frames.find((frame) => frame.schemaVersion === "cultnet.hello.v0") as CultNetHelloMessage;
}
put(messageId: string, recordKey: string, text: string, document: Record<string, unknown> = {}): CultNetMessage {
return this.shard({
schemaVersion: "cultnet.document_put_raw.v0",
messageId,
document: {
schemaId: SCHEMA_ID, recordKey, storedAt: new Date().toISOString(), payloadEncoding: "messagepack",
payload: this.target.payload(text), ...document,
},
});
}
delete(messageId: string, recordKey: string): CultNetMessage {
return this.shard({ schemaVersion: "cultnet.document_delete.v0", messageId, schemaId: SCHEMA_ID, recordKey });
}
private shard(message: unknown): CultNetMessage {
return { ...(message as object), ...this.target.shard } as CultNetMessage;
}
/**
* Sends one write and settles it with its own tracker, windowed by what the server advertised (5000 ms against a receipt
* server, 500 ms otherwise). Only the tracker decides the outcome.
*/
async write(message: CultNetMessage): Promise<WriteSettlement> {
const messageId = (message as { messageId: string }).messageId;
const tracker = new WriteTracker({ windowMs: resultWindowMs(this.serverReceipts) });
this.trackers.add(tracker);
tracker.send(messageId, Date.now());
this.connection.peer.send(message);
for (;;) {
tracker.tick(Date.now());
const state = tracker.state(messageId)!;
if (state.outcome !== "pending") return state;
await sleep(Math.min(20, tracker.nextDeadlineMs(Date.now()) ?? 20));
}
}
/** Frames of `schemaVersion` that name `messageId`. */
answers(messageId: string, schemaVersion: string): CultNetMessage[] {
return this.frames.filter((frame) => frame.schemaVersion === schemaVersion && (frame as { messageId?: string }).messageId === messageId);
}
close(): void { this.connection.close(); }
}
function refusal(settlement: WriteSettlement): CultNetErrorV1Message {
assert.equal(settlement.outcome, "refused", JSON.stringify(settlement));
return (settlement as Extract<WriteSettlement, { outcome: "refused" }>).response as CultNetErrorV1Message;
}
// ---- the scenarios, run against each server -----------------------------------------------------------------------------------
for (const target of [rust, csharp]) {
describe(`write receipt, TypeScript client against the real ${target.name} server`, () => {
let server: ServerProcess;
let session: Session | undefined;
before(async () => { server = await target.start(); }, { timeout: 180_000 });
after(async () => { session?.close(); await server?.stop(); });
beforeEach(async () => { if (target === csharp) await awaitConnectionBudget(); }, { timeout: 90_000 });
/** A fresh receipt session (hello sent, the server's hello read) for a scenario. */
async function receiptSession(): Promise<Session> {
session?.close();
session = await Session.open(target, server);
const hello = await session.hello();
assert.ok(helloAdvertisesWriteReceipts(hello), "the server's hello lists both receipt frames");
return session;
}
test("hello listing both frames: the server answers with its own hello listing both, then a put is receipted after it", { timeout: 30_000 }, async () => {
const s = await receiptSession();
assert.equal(s.frames[0].schemaVersion, "cultnet.hello.v0");
const settlement = await s.write(s.put("hello-1", "hello:a", "one"));
assert.deepEqual(settlement, { outcome: "admitted" });
assert.deepEqual(s.frames.map((frame) => frame.schemaVersion), ["cultnet.hello.v0", "cultnet.write_admitted.v0"]);
});
test("a hello listing only one of the two frames does not opt in: no receipt, and the put is applied", { timeout: 30_000 }, async () => {
for (const listed of CULTNET_WRITE_RECEIPT_MESSAGE_VERSIONS) {
session?.close();
session = await Session.open(target, server);
session.hello([listed]).catch(() => undefined); // a server may stay silent to a hello that does not opt in
await sleep(300);
const settlement = await session.write(session.put(`half-${listed}`, `half:${listed}`, "one"));
assert.deepEqual(settlement, { outcome: "unconfirmed", reason: "window_elapsed" }, `listing only ${listed}`);
assert.ok(!session.frames.some((frame) => frame.schemaVersion === "cultnet.write_admitted.v0" || frame.schemaVersion === "cultnet.error.v1"));
await waitFor(() => server.count("applied", `half:${listed}`) === 1, "the put to be applied");
}
});
test("put admitted: the receipt echoes the write and the store has it once", { timeout: 30_000 }, async () => {
const s = await receiptSession();
const settlement = await s.write(s.put("put-1", "put:a", "one"));
assert.deepEqual(settlement, { outcome: "admitted" });
assert.deepEqual(s.answers("put-1", "cultnet.write_admitted.v0"), [
{ schemaVersion: "cultnet.write_admitted.v0", messageId: "put-1", operation: "put", schemaId: SCHEMA_ID, recordKey: "put:a" },
]);
await waitFor(() => server.count("applied", "put:a") === 1, "the put to be applied");
});
// The Rust RUDP document server has no delete: its sink only accepts documents, and every delete on a receipt session is
// answered with a refusal (rudp_document_server.rs, deletes_on_a_receipt_session_are_refused_one_by_one). The spec
// (Server rules 1 to 3) has a delete admitted and receipted like a put.
test("delete admitted: the receipt says operation delete and the store drops the record", {
timeout: 30_000,
todo: target.name === "rust" ? "the Rust RUDP document server refuses every delete; there is no delete port in CultMeshRudpRawDocumentSink" : false,
}, async () => {
const s = await receiptSession();
assert.deepEqual(await s.write(s.put("del-put", "del:a", "one")), { outcome: "admitted" });
assert.deepEqual(await s.write(s.delete("del-1", "del:a")), { outcome: "admitted" });
assert.deepEqual(s.answers("del-1", "cultnet.write_admitted.v0"), [
{ schemaVersion: "cultnet.write_admitted.v0", messageId: "del-1", operation: "delete", schemaId: SCHEMA_ID, recordKey: "del:a" },
]);
await waitFor(() => server.count("removed", "del:a") === 1, "the delete to be applied");
});
test("refused write: an error.v1 carrying the write's id, and nothing is applied", { timeout: 30_000 }, async () => {
const s = await receiptSession();
const response = refusal(await s.write(s.put("deny-1", "refuse:a", "one")));
assert.equal(response.schemaVersion, "cultnet.error.v1");
assert.equal(response.messageId, "deny-1");
assert.equal(response.code, "write_not_authorized");
// The session is kept: the next write on it is admitted.
assert.deepEqual(await s.write(s.put("deny-2", "deny:b", "one")), { outcome: "admitted" });
assert.equal(server.count("applied", "refuse:a"), 0);
});
test("resend of the same id and content on the same session: re-receipted, applied once", { timeout: 30_000 }, async () => {
const s = await receiptSession();
assert.deepEqual(await s.write(s.put("resend-1", "resend:a", "one")), { outcome: "admitted" });
assert.deepEqual(await s.write(s.put("resend-1", "resend:a", "one")), { outcome: "admitted" });
const receipts = s.answers("resend-1", "cultnet.write_admitted.v0");
assert.equal(receipts.length, 2);
assert.deepEqual(receipts[0], receipts[1]);
await sleep(200); // a second application would be printed by now
assert.equal(server.count("applied", "resend:a"), 1);
});
test("same id with different content: message_invalid naming the id, never a receipt, the first write untouched", { timeout: 30_000 }, async () => {
const s = await receiptSession();
assert.deepEqual(await s.write(s.put("clash-1", "clash:a", "one")), { outcome: "admitted" });
const response = refusal(await s.write(s.put("clash-1", "clash:a", "other")));
assert.equal(response.code, "message_invalid");
assert.equal(response.messageId, "clash-1");
assert.equal(response.details?.field, "messageId");
assert.equal(s.answers("clash-1", "cultnet.write_admitted.v0").length, 1, "no receipt for the changed write");
await sleep(200);
assert.equal(server.count("applied", "clash:a"), 1);
});
// Each case of the vectors' blankFields section: a document that names nothing is refused as an unreadable frame, with the
// write's id and the fixed text, and never applied; a document that is merely missing optional fields is admitted.
for (const blank of vectors.blankFields.cases as { name: string; set: Record<string, unknown>; outcome: "admitted" | "refused" }[]) {
test(`blank-field vector ${blank.name}: ${blank.outcome}`, { timeout: 30_000 }, async () => {
const s = await receiptSession();
const { payloadHex, ...fields } = blank.set;
// A case that sets recordKey replaces the harness key, so the store is read under the key the server actually saw.
const recordKey = typeof fields.recordKey === "string" ? fields.recordKey : `blank:${blank.name}`;
const message = s.put(`blank-${blank.name}`, recordKey, "one", fields);
const sent = (message as unknown as { document: Record<string, unknown> }).document;
for (const [key, value] of Object.entries(fields)) if (value === null) delete sent[key]; // nil means absent on the wire here
if (payloadHex === "") sent.payload = new Uint8Array();
const settlement = await s.write(message);
if (blank.outcome === "admitted") {
assert.deepEqual(settlement, { outcome: "admitted" });
await waitFor(() => server.count("applied", recordKey) === 1, "the put to be applied");
return;
}
const response = refusal(settlement);
assert.equal(response.code, "message_invalid");
assert.equal(response.error, UNREADABLE);
assert.equal(response.details?.field, "message");
assert.equal(response.messageId, `blank-${blank.name}`);
await sleep(200);
assert.equal(server.count("applied", recordKey), 0);
});
}
test("a delete that names nothing is refused as unreadable with its id", { timeout: 30_000 }, async () => {
const s = await receiptSession();
const message = s.delete("blank-del", "blank:del");
(message as unknown as { schemaId: string }).schemaId = "";
const response = refusal(await s.write(message));
assert.equal(response.code, "message_invalid");
assert.equal(response.error, UNREADABLE);
assert.equal(response.details?.field, "message");
assert.equal(response.messageId, "blank-del");
});
test("legacy client (no hello): no receipt frames, the write is applied, and its refusal is the old error.v0", { timeout: 30_000 }, async () => {
session?.close();
session = await Session.open(target, server);
assert.deepEqual(await session.write(session.put("legacy-1", "legacy:a", "one")), { outcome: "unconfirmed", reason: "window_elapsed" });
await waitFor(() => server.count("applied", "legacy:a") === 1, "the put to be applied");
const refused = (await session.write(session.put("legacy-2", "refuse:legacy", "one"))) as Extract<WriteSettlement, { outcome: "refused" }>;
assert.equal(refused.outcome, "refused");
assert.equal(refused.response.schemaVersion, "cultnet.error.v0");
assert.deepEqual(session.frames.map((frame) => frame.schemaVersion), ["cultnet.error.v0"]);
assert.equal(session.invalid.length, 0);
});
});
}
// ---- the one-shot writer, which is the real cultmesh-ts client path, against the Rust RUDP server ----------------------------
// (The C# server speaks LiteNetLib, not RUDP: no RUDP client path reaches it.)
describe("write receipt, cultmesh-ts publishRudpDocumentOnce against the real rust server", () => {
const note = defineDocumentType({ type: "receipt.interop_note", schemaId: SCHEMA_ID, schema: z.object({ text: z.string() }), name: "text" });
const binding = defineCultNetDocumentBinding({ definition: note });
let server: ServerProcess;
before(async () => { server = await rust.start(); }, { timeout: 180_000 });
after(async () => { await server?.stop(); });
const endpoint = () => ({ host: "127.0.0.1", port: server.port, uri: `rudp://127.0.0.1:${server.port}` });
test("writeReceipts: the put resolves admitted on the receipt, after the server applied it", { timeout: 30_000 }, async () => {
const outcome = await CultMesh.publishRudpDocumentOnce("receipt-interop", nextConnectionId++, endpoint(), binding, "once:a", { text: "one" }, { writeReceipts: true, messageId: "once-1" });
assert.equal(outcome, "admitted");
await waitFor(() => server.count("applied", "once:a") === 1, "the put to be applied");
});
test("writeReceipts: a refused put rejects with the server's error.v1 naming the put", { timeout: 30_000 }, async () => {
await assert.rejects(
CultMesh.publishRudpDocumentOnce("receipt-interop", nextConnectionId++, endpoint(), binding, "refuse:once", { text: "one" }, { writeReceipts: true, messageId: "once-2" }),
(error: unknown) => {
assert.ok(error instanceof CultMeshRudpWriteRefusedError);
assert.equal(error.response.schemaVersion, "cultnet.error.v1");
assert.equal((error.response as CultNetErrorV1Message).messageId, "once-2");
return true;
},
);
});
test("without writeReceipts the server stays silent: the put is unconfirmed, not admitted, though the server applied it", { timeout: 30_000 }, async () => {
await assert.rejects(
CultMesh.publishRudpDocumentOnce("receipt-interop", nextConnectionId++, endpoint(), binding, "once:legacy", { text: "one" }, { messageId: "once-3" }),
(error: unknown) => error instanceof CultMeshRudpWriteUnconfirmedError && error.reason === "window_elapsed",
);
await waitFor(() => server.count("applied", "once:legacy") === 1, "the put to be applied");
});
});

View file

@ -287,18 +287,21 @@ test("a replaced RUDP generation drops operations queued behind an active handle
runtimeId: identity.serviceInstanceId, runtimeId: identity.serviceInstanceId,
connectionId, connectionId,
sessionToken: "signed-provider-token", sessionToken: "signed-provider-token",
operationTimeoutMs: 100, operationTimeoutMs: 1, // expires long before the test ends: the settle-by-timeout path is always exercised
socketFactory: () => socket, socketFactory: () => socket,
}).connect(identity, new AbortController().signal); }).connect(identity, new AbortController().signal);
try { try {
// The handler is held open and the closed generation never answers, so these two settle only by the
// client's operation timeout or by connection.close(). Observe them from the start: whichever fires
// first is a legitimate end of the call, and this test asserts on server-side handler calls, not on it.
const first = connection.register( const first = connection.register(
{ identity, requestedLeaseDurationMs: 10_000 }, { identity, requestedLeaseDurationMs: 10_000 },
new AbortController().signal, new AbortController().signal,
); ).then(() => undefined, () => undefined);
const second = connection.register( const second = connection.register(
{ identity, requestedLeaseDurationMs: 10_000 }, { identity, requestedLeaseDurationMs: 10_000 },
new AbortController().signal, new AbortController().signal,
); ).then(() => undefined, () => undefined);
await waitFor(() => handlerCalls === 1, "first queued provider operation"); await waitFor(() => handlerCalls === 1, "first queued provider operation");
const replacement = new CultNetRudpSession({ connectionId }); const replacement = new CultNetRudpSession({ connectionId });
@ -318,7 +321,7 @@ test("a replaced RUDP generation drops operations queued behind an active handle
assert.equal(handlerCalls, 1, "transport-queued work from the closed generation must be dropped"); assert.equal(handlerCalls, 1, "transport-queued work from the closed generation must be dropped");
connection.close(); connection.close();
await Promise.allSettled([first, second]); await Promise.all([first, second]);
} finally { } finally {
connection.close(); connection.close();
server.close(); server.close();

View file

@ -1,11 +1,13 @@
// A peer's put or delete is applied only when the server's write authority grants it. Ports the behaviours of // A peer's put or delete is applied only when the server's write authority grants it. Ports the behaviours of
// cultmesh-py test_remote_write_authority.py (no forwarding: this server does not forward non-primary writes). // cultmesh-py test_remote_write_authority.py (no forwarding: this server does not forward non-primary writes).
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { createSocket, type Socket } from "node:dgram";
import { readFileSync } from "node:fs"; import { readFileSync } from "node:fs";
import { join } from "node:path"; import { join } from "node:path";
import test from "node:test"; import test from "node:test";
import { z } from "zod"; import { z } from "zod";
import { documentPayloadFromBytes, defineDocumentType } from "@gamecult/cultcache-ts"; import { documentPayloadFromBytes, defineDocumentType } from "@gamecult/cultcache-ts";
import { decodeRudpPacket } from "@gamecult/cultnet-ts";
import { encodeWireValue } from "@gamecult/cultnet-ts/wire"; import { encodeWireValue } from "@gamecult/cultnet-ts/wire";
import { import {
CultNetDocumentRegistry, CultNetDocumentRegistry,
@ -16,6 +18,7 @@ import {
} from "@gamecult/cultnet-ts"; } from "@gamecult/cultnet-ts";
import { import {
CultMesh, CultMesh,
CULTMESH_RUDP_DOCUMENT_SERVER_MAX_CONSECUTIVE_REFUSED_FRAMES,
CultMeshGrantAllWrites, CultMeshGrantAllWrites,
type CultMeshRemoteWriteAuthority, type CultMeshRemoteWriteAuthority,
type CultMeshRemoteWriteDecision, type CultMeshRemoteWriteDecision,
@ -316,7 +319,10 @@ test("wrong-typed shard fields on a put are judged with null shard fields, then
for (const fields of bad) r.peer.send(put(KEY_A, fields)); for (const fields of bad) r.peer.send(put(KEY_A, fields));
await r.settle(); await r.settle();
assert.equal(r.replies.length, bad.length); assert.equal(r.replies.length, bad.length);
assert.ok(r.replies.every((reply) => reply.schemaVersion === "cultnet.error.v0" && reply.code === null && reply.error.startsWith("malformed_document_put:"))); assert.deepEqual(
r.replies.map((reply) => reply.schemaVersion === "cultnet.error.v0" ? [reply.code, reply.details?.field] : reply),
["shardId", "shardId", "shardEpoch", "shardEpoch", "shardEpoch"].map((field) => ["message_invalid", field]),
);
assert.equal(authority.requests.length, bad.length, "the authority is asked first"); assert.equal(authority.requests.length, bad.length, "the authority is asked first");
assert.ok(authority.requests.every((q) => q.shardId === null && q.shardEpoch === null), "with null shard fields"); assert.ok(authority.requests.every((q) => q.shardId === null && q.shardEpoch === null), "with null shard fields");
assert.equal(r.puts.length, 0); assert.equal(r.puts.length, 0);
@ -493,12 +499,187 @@ test("a granted peer's unreadable write reaches onError and gets a malformed ref
r.send(h.frame); r.send(h.frame);
await sleep(250); await sleep(250);
assert.equal(r.replies.length, 1, h.name); assert.equal(r.replies.length, 1, h.name);
const reply = r.replies[0] as { code: unknown; error: string }; const reply = r.replies[0] as { code: unknown; error: string; details: { field: string } };
assert.equal(reply.code, null, h.name); assert.equal(reply.code, "message_invalid", h.name);
assert.ok(reply.error.startsWith(`malformed_document_${h.operation}:`), h.name); assert.equal(reply.details.field, "message", h.name);
assert.equal(r.errors.length, 1, h.name); assert.equal(r.errors.length, 1, h.name);
assert.deepEqual([r.puts.length, r.deletes.length], [0, 0], h.name); assert.deepEqual([r.puts.length, r.deletes.length], [0, 0], h.name);
assert.deepEqual([authority.requests.length, authority.requests[0]!.operation, authority.requests[0]!.recordKey], [1, h.operation, h.recordKey], h.name); assert.deepEqual([authority.requests.length, authority.requests[0]!.operation, authority.requests[0]!.recordKey], [1, h.operation, h.recordKey], h.name);
} finally { r.close(); } } finally { r.close(); }
} }
}); });
const CAP = 8; // the cap every runtime shares; the exported constant must stay this number
test("the consecutive-refused-frames cap is the number every runtime shares", () => {
assert.equal(CULTMESH_RUDP_DOCUMENT_SERVER_MAX_CONSECUTIVE_REFUSED_FRAMES, CAP);
});
test("unreadable frames are answered and the session kept until eight in a row; any decoded frame resets the run", async () => {
const r = await rawRig(new Recording(grant));
try {
const batch = async (frames: number) => {
for (let i = 0; i < frames; i++) r.send(Uint8Array.from([0xc1]));
await sleep(400);
};
await batch(CAP - 1);
assert.equal(r.replies.length, CAP - 1, "each is answered");
r.send(encodeWireValue(put(KEY_A)));
await sleep(250);
assert.deepEqual(r.puts.map((p) => p.recordKey), [KEY_A], "the session was kept through seven");
await batch(CAP - 1);
assert.equal(r.replies.length, 2 * (CAP - 1), "the valid frame reset the run");
r.send(Uint8Array.from([0xc1]));
await sleep(300);
assert.equal(r.replies.length, 2 * (CAP - 1) + 1, "the eighth consecutive one is answered too");
assert.ok(r.replies.every((reply) => (reply as { code?: string }).code === "message_invalid"));
// The goodbye reached the peer: its client no longer takes a send.
assert.throws(() => r.send(encodeWireValue(put(KEY_B))), /before the session is connected/, "then the session ended");
assert.deepEqual(r.puts.map((p) => p.recordKey), [KEY_A]);
} finally { r.close(); }
});
test("deletes the authority refuses are answered and the session kept until eight in a row; any other decoded frame resets the run", async () => {
const authority = new Recording((request) => request.recordKey === KEY_B ? { granted: true } : { granted: false });
await using(await rig(authority), async (r) => {
const refuse = async (deletes: number) => {
for (let i = 0; i < deletes; i++) r.peer.send(del(KEY_A));
await sleep(400);
};
await refuse(CAP - 1);
assert.equal(r.replies.length, CAP - 1);
r.peer.send(del(KEY_B)); // granted: resets the run
await sleep(250);
assert.deepEqual(r.deletes.map((d) => d.recordKey), [KEY_B]);
await refuse(CAP - 1);
assert.equal(r.replies.length, 2 * (CAP - 1));
r.peer.send(put(KEY_B)); // a decoded non-delete resets too
await sleep(250);
await refuse(CAP - 1);
assert.equal(r.replies.length, 3 * (CAP - 1), "a put reset the run");
r.peer.send(del(KEY_A));
await sleep(300);
assert.equal(r.replies.length, 3 * (CAP - 1) + 1, "the eighth consecutive refused delete is answered too");
for (const reply of r.replies) assertRefused(reply, KEY_A);
assert.throws(() => r.peer.send(del(KEY_B)), /before the session is connected/, "then the session ended");
assert.equal(r.deletes.length, 1);
});
});
test("a frame that decodes but is no message the parser takes is answered, counted, and ended at eight in a row like any unreadable frame", async () => {
const r = await rawRig(new Recording(grant));
const unknown = () => encodeWireValue({ schemaVersion: "cultnet.not-a-message.v0", messageId: "x" });
try {
const batch = async (frames: number) => {
for (let i = 0; i < frames; i++) r.send(unknown());
await sleep(400);
};
await batch(CAP - 1);
assert.equal(r.replies.length, CAP - 1, "each is answered");
assert.equal(r.errors.length, CAP - 1, "and reported");
r.send(encodeWireValue(put(KEY_A)));
await sleep(250);
assert.deepEqual(r.puts.map((p) => p.recordKey), [KEY_A], "the session was kept through seven");
await batch(CAP - 1);
assert.equal(r.replies.length, 2 * (CAP - 1), "the decoded put reset the run");
r.send(unknown());
await sleep(300);
assert.equal(r.replies.length, 2 * (CAP - 1) + 1, "the eighth is answered too");
for (const reply of r.replies) {
assert.equal((reply as { code?: string }).code, "message_invalid");
assert.equal((reply as { details: { field: string } }).details.field, "message");
}
assert.throws(() => r.send(unknown()), /before the session is connected/, "then the session ended");
} finally { r.close(); }
});
/** A UDP relay between a client and a server that drops the first (or, with `dropEvery`, every) server datagram carrying `payloadContains` once armed. */
async function lossyRelay(serverPort: number, payloadContains: string, dropEvery = false) {
const dropped: unknown[] = [];
let armed = false;
let clientAddress: { port: number; address: string } | undefined;
const toClient: Socket = createSocket("udp4");
const toServer: Socket = createSocket("udp4");
toServer.on("message", (wire) => {
if (!clientAddress) return;
if (armed && (dropEvery || dropped.length === 0)) {
const packet = decodeRudpPacket(wire);
if (packet.packetType === "data" && Buffer.from(packet.payload ?? []).includes(payloadContains)) {
dropped.push(packet);
return;
}
}
toClient.send(wire, clientAddress.port, clientAddress.address);
});
toClient.on("message", (wire, remote) => {
clientAddress = remote;
toServer.send(wire, serverPort, "127.0.0.1");
});
await new Promise<void>((resolve) => toClient.bind(0, "127.0.0.1", resolve));
await new Promise<void>((resolve) => toServer.bind(0, "127.0.0.1", resolve));
return {
port: toClient.address().port,
dropped,
arm: () => { armed = true; },
close: () => { toClient.close(); toServer.close(); },
};
}
async function cappedOverRelay(dropEvery: boolean, body: (x: { relay: Awaited<ReturnType<typeof lossyRelay>>; client: { send(channel: string, payload: Uint8Array): void }; replies: CultNetMessage[] }) => Promise<void>) {
const connectionId = nextConnection++;
const server = CultMesh.createRudpDocumentServer("authority-lossy", connectionId, {
documents: new CultNetDocumentRegistry([defineCultNetDocumentBinding({ definition: note })]),
writeAuthority: new Recording(grant), bindHost: "127.0.0.1", bindPort: 0, resendDelayMs: 25, resendPollMs: 5,
onDocumentPutRaw: () => {}, onError: () => {},
});
await server.start();
const relay = await lossyRelay(server.bind.port, "message_invalid", dropEvery);
const client = await CultMesh.createRudpClient("lossy", connectionId, `rudp://127.0.0.1:${relay.port}`, { resendDelayMs: 25, resendPollMs: 5 });
try {
client.connect();
const started = Date.now();
while (!client.connected) {
if (Date.now() - started > 2_000) throw new Error("not connected");
await sleep(5);
}
const replies: CultNetMessage[] = [];
new CultNetPeer(client, { wireContract: "cultnet.schema.v0" }).on("message", (m) => replies.push(m));
for (let i = 0; i < CAP - 1; i++) client.send("schema", Uint8Array.from([0xc1]));
await sleep(300);
assert.equal(replies.length, CAP - 1);
relay.arm();
client.send("schema", Uint8Array.from([0xc1]));
await body({ relay, client, replies });
} finally { client.close(); server.close(); relay.close(); }
}
test("the capped frame's answer is resent when its datagram is lost, and the goodbye follows only after it arrived", async () => {
await cappedOverRelay(false, async ({ relay, client, replies }) => {
await sleep(600);
assert.equal(relay.dropped.length, 1, "fixture: the first transmission of the last answer was lost");
assert.equal(replies.length, CAP, "the lost answer was resent and arrived");
assert.equal((replies[CAP - 1] as { code?: string }).code, "message_invalid");
assert.throws(() => client.send("schema", Uint8Array.from([0xc1])), /before the session is connected/, "then the session ended");
});
});
test("a capped session whose answer is never acknowledged still ends, after the grace", async () => {
await cappedOverRelay(true, async ({ client, replies }) => {
await sleep(1200);
assert.equal(replies.length, CAP - 1, "fixture: the last answer never arrives");
client.send("schema", Uint8Array.from([0xc1])); // still connected: the server is waiting for the acknowledgement
await sleep(1500);
assert.throws(() => client.send("schema", Uint8Array.from([0xc1])), /before the session is connected/, "the grace ran out and the goodbye went out");
});
});
test("nothing behind the capped frame is served while its answer is delivered", async () => {
const r = await rawRig(new Recording(grant));
try {
for (let i = 0; i < CAP; i++) r.send(Uint8Array.from([0xc1]));
r.send(encodeWireValue(put(KEY_A)));
await sleep(500);
assert.equal(r.replies.length, CAP);
assert.deepEqual(r.puts, [], "the put behind the cap was not served");
} finally { r.close(); }
});

View file

@ -0,0 +1,243 @@
// Client side of contracts/cultnet/interop/write-receipt.md in the one-shot writer: with `writeReceipts` the put
// completes only on a write_admitted carrying its id; a refusal, a silent window and a legacy server each end as the
// spec says. Each test serves the client a fake RUDP peer that acknowledges the put and answers as scripted.
import assert from "node:assert/strict";
import { createSocket, type Socket } from "node:dgram";
import test from "node:test";
import { z } from "zod";
import { defineDocumentType } from "@gamecult/cultcache-ts";
import {
CULTNET_WRITE_RECEIPT_MESSAGE_VERSIONS,
CultNetRudpSession,
decodeRudpPacket,
defineCultNetDocumentBinding,
encodeCultNetMessageForWire,
encodeRudpPacket,
type CultNetMessage,
} from "@gamecult/cultnet-ts";
import { decodeWireValue, encodeWireValue } from "@gamecult/cultnet-ts/wire";
import { CultMesh, CultMeshRudpWriteRefusedError, CultMeshRudpWriteUnconfirmedError } from "../src/index";
const note = defineDocumentType({
type: "cultmesh.receipt_note",
schemaId: "cultmesh.receipt_note.v0",
schema: z.object({ noteId: z.string(), text: z.string() }),
name: "noteId",
});
const binding = defineCultNetDocumentBinding({ definition: note });
let nextConnection = 0x10206000;
interface Script {
/** Does the server's hello list both receipt frames? */
advertise: boolean;
/** What the server answers a put with, `delayMs` after it arrives. */
answer: "admit" | "refuse" | "wrong-id" | "silence" | "goodbye" | "echo";
delayMs?: number;
/** Acknowledge the reliable packets it receives? Default true. */
ack?: boolean;
}
async function fakeServer(connectionId: number, script: Script) {
const socket: Socket = createSocket("udp4");
await new Promise<void>((resolve) => socket.bind(0, "127.0.0.1", resolve));
const session = new CultNetRudpSession({ connectionId, resendDelayMs: 50 });
const timers = new Set<NodeJS.Timeout>();
const seen: string[] = [];
let closed = false;
const send = (packet: Parameters<typeof encodeRudpPacket>[0], port: number, address: string) => {
if (!closed) socket.send(encodeRudpPacket(packet), port, address);
};
const say = (message: CultNetMessage, port: number, address: string) => {
const packets = session.sendMany("schema", encodeWireValue(encodeCultNetMessageForWire(message)), { reliable: true, ordered: true, nowMs: Date.now() });
packets.forEach((p) => send(p, port, address));
};
socket.on("message", (wire, remote) => {
const packet = decodeRudpPacket(wire);
const now = Date.now();
if (packet.packetType === "connect") {
send(session.acceptConnect(packet, now), remote.port, remote.address);
return;
}
const result = session.receive(packet, now);
if (result.reply) send(result.reply, remote.port, remote.address);
for (const frame of result.delivered) {
if (frame.channelId !== "schema") continue;
const message = decodeWireValue(frame.payload) as { schemaVersion: string; messageId?: string };
seen.push(message.schemaVersion);
if (message.schemaVersion === "cultnet.hello.v0") {
say({
schemaVersion: "cultnet.hello.v0", runtimeId: "fake", runtimeKind: "fake",
...(script.advertise ? { supportedMessageVersions: [...CULTNET_WRITE_RECEIPT_MESSAGE_VERSIONS] } : {}),
}, remote.port, remote.address);
continue;
}
if (message.schemaVersion !== "cultnet.document_put_raw.v0" || script.answer === "silence") continue;
const timer = setTimeout(() => {
timers.delete(timer);
if (script.answer === "echo") {
say({ schemaVersion: "cultnet.document_put_raw.v0", messageId: message.messageId!, document: { schemaId: "cultmesh.receipt_note.v0", recordKey: "tests:receipt:a", storedAt: "t", payloadEncoding: "messagepack", payload: new Uint8Array([1]) } } as unknown as CultNetMessage, remote.port, remote.address);
} else if (script.answer === "goodbye") {
send(session.endUnsendableSession(new Error("scripted goodbye")), remote.port, remote.address);
} else if (script.answer === "refuse") {
say({ schemaVersion: "cultnet.error.v1", error: "write_not_authorized", routingHint: null, code: "write_not_authorized", details: null, messageId: message.messageId! }, remote.port, remote.address);
} else {
const messageId = script.answer === "wrong-id" ? "someone-else" : message.messageId!;
say({ schemaVersion: "cultnet.write_admitted.v0", messageId, operation: "put", schemaId: "cultmesh.receipt_note.v0", recordKey: "tests:receipt:a" }, remote.port, remote.address);
}
}, script.delayMs ?? 20);
timers.add(timer);
}
if (packet.reliable && script.ack !== false) send(session.createAckForReceived(packet.sequence), remote.port, remote.address);
});
return { endpoint: `rudp://127.0.0.1:${socket.address().port}`, seen, close: () => { closed = true; timers.forEach(clearTimeout); socket.close(); } };
}
const transport = { resendDelayMs: 50, resendPollMs: 5, flushTimeoutMs: 2_000 };
const publish = (server: { endpoint: string }, options: Record<string, unknown>) =>
CultMesh.publishRudpDocumentOnce("receipt-client", nextConnection++, server.endpoint, binding, "tests:receipt:a", { noteId: "a", text: "x" }, { ...transport, ...options });
test("a receipt-capable server's write_admitted completes the put, past the legacy 500 ms window", async () => {
const server = await fakeServer(nextConnection, { advertise: true, answer: "admit", delayMs: 700 });
try {
await publish(server, { writeReceipts: true });
assert.deepEqual(server.seen.slice(0, 2), ["cultnet.hello.v0", "cultnet.document_put_raw.v0"]);
} finally { server.close(); }
});
test("a receipt-capable server's error.v1 refuses the put with its code", async () => {
const server = await fakeServer(nextConnection, { advertise: true, answer: "refuse" });
try {
await assert.rejects(publish(server, { writeReceipts: true }), (error: unknown) =>
error instanceof CultMeshRudpWriteRefusedError && error.code === "write_not_authorized" && error.response.schemaVersion === "cultnet.error.v1");
} finally { server.close(); }
});
test("a receipt-capable server that stays silent leaves the put unconfirmed, never published", async () => {
const server = await fakeServer(nextConnection, { advertise: true, answer: "silence" });
try {
await assert.rejects(publish(server, { writeReceipts: true, resultWindowMs: 250 }), (error: unknown) =>
error instanceof CultMeshRudpWriteUnconfirmedError && error.reason === "window_elapsed");
} finally { server.close(); }
});
test("a receipt for another id settles nothing", async () => {
const server = await fakeServer(nextConnection, { advertise: true, answer: "wrong-id" });
try {
await assert.rejects(publish(server, { writeReceipts: true, resultWindowMs: 250 }), CultMeshRudpWriteUnconfirmedError);
} finally { server.close(); }
});
test("a server whose hello does not advertise leaves a silent put unconfirmed after the 500 ms window, never published", async () => {
const server = await fakeServer(nextConnection, { advertise: false, answer: "silence" });
const started = Date.now();
try {
await assert.rejects(publish(server, { writeReceipts: true }), (error: unknown) =>
error instanceof CultMeshRudpWriteUnconfirmedError && error.reason === "window_elapsed");
assert.ok(Date.now() - started >= 500, "the legacy window ran from the send");
} finally { server.close(); }
});
test("a session that ends before any result leaves the put unconfirmed (session_ended), against a legacy server too", async () => {
for (const advertise of [false, true]) {
const server = await fakeServer(nextConnection, { advertise, answer: "goodbye" });
try {
await assert.rejects(publish(server, { writeReceipts: true, resultWindowMs: 3_000 }), (error: unknown) =>
error instanceof CultMeshRudpWriteUnconfirmedError && error.reason === "session_ended");
} finally { server.close(); }
}
});
test("a put whose send is never acknowledged is unconfirmed as local_send_failed, not an untyped error", async () => {
const server = await fakeServer(nextConnection, { advertise: true, answer: "silence", ack: false });
const started = Date.now();
try {
await assert.rejects(publish(server, { writeReceipts: true, flushTimeoutMs: 300 }), (error: unknown) =>
error instanceof CultMeshRudpWriteUnconfirmedError && error.reason === "local_send_failed");
assert.ok(Date.now() - started < 2_500, "flushTimeoutMs bounds the send inside the 5 s window");
} finally { server.close(); }
});
test("the flush is bounded by the result window: a send nobody acknowledges is local_send_failed at the window, not at the flush timeout", async () => {
const server = await fakeServer(nextConnection, { advertise: true, answer: "silence", ack: false });
const started = Date.now();
try {
await assert.rejects(publish(server, { writeReceipts: true, resultWindowMs: 300, flushTimeoutMs: 20_000 }), (error: unknown) =>
error instanceof CultMeshRudpWriteUnconfirmedError && error.reason === "local_send_failed");
assert.ok(Date.now() - started < 3_000, "it did not wait out the 20 s flush timeout");
} finally { server.close(); }
});
test("a server that does not advertise receipts and echoes the put's id back has not admitted it", async () => {
const server = await fakeServer(nextConnection, { advertise: false, answer: "echo" });
try {
await assert.rejects(publish(server, { writeReceipts: true, resultWindowMs: 300 }), (error: unknown) =>
error instanceof CultMeshRudpWriteUnconfirmedError && error.reason === "window_elapsed");
} finally { server.close(); }
});
test("a silent server never yields admitted, in either mode: the default is unconfirmed, sendOnly is the distinct outcome sent", async () => {
const dflt = await fakeServer(nextConnection, { advertise: false, answer: "silence" });
try {
await assert.rejects(publish(dflt, { resultWindowMs: 200 }), CultMeshRudpWriteUnconfirmedError);
} finally { dflt.close(); }
const sendOnly = await fakeServer(nextConnection, { advertise: false, answer: "silence" });
try {
const outcome = await publish(sendOnly, { sendOnly: true });
assert.equal(outcome, "sent");
assert.notEqual(outcome, "admitted");
assert.deepEqual(sendOnly.seen, ["cultnet.document_put_raw.v0"], "sendOnly sends no hello");
} finally { sendOnly.close(); }
});
test("only a write_admitted resolves admitted, and sendOnly does not wait for or act on an answer", async () => {
const admitting = await fakeServer(nextConnection, { advertise: true, answer: "admit" });
try {
assert.equal(await publish(admitting, { writeReceipts: true }), "admitted");
} finally { admitting.close(); }
const refusing = await fakeServer(nextConnection, { advertise: true, answer: "refuse", delayMs: 1 });
try {
assert.equal(await publish(refusing, { sendOnly: true }), "sent", "a refusal is not awaited: sendOnly says only that the put was sent");
} finally { refusing.close(); }
});
test("sendOnly fails local_send_failed when the send is never acknowledged", async () => {
const server = await fakeServer(nextConnection, { advertise: true, answer: "silence", ack: false });
try {
await assert.rejects(publish(server, { sendOnly: true, flushTimeoutMs: 300 }), (error: unknown) =>
error instanceof CultMeshRudpWriteUnconfirmedError && error.reason === "local_send_failed");
} finally { server.close(); }
});
test("sendOnly keeps the cause of a put that cannot be built: it is not reported as local_send_failed", async () => {
const server = await fakeServer(nextConnection, { advertise: false, answer: "silence" });
try {
await assert.rejects(
CultMesh.publishRudpDocumentOnce("receipt-client", nextConnection++, server.endpoint, binding, "tests:receipt:a", { noteId: 1 } as unknown as { noteId: string; text: string }, { ...transport, sendOnly: true }),
(error: unknown) => !(error instanceof CultMeshRudpWriteUnconfirmedError) && error instanceof Error,
);
assert.deepEqual(server.seen, [], "nothing was sent");
} finally { server.close(); }
});
test("the receipt path keeps the cause of a put that cannot be built: it is not reported as local_send_failed", async () => {
const server = await fakeServer(nextConnection, { advertise: true, answer: "admit" });
try {
await assert.rejects(
CultMesh.publishRudpDocumentOnce("receipt-client", nextConnection++, server.endpoint, binding, "tests:receipt:a", { noteId: 1 } as unknown as { noteId: string; text: string }, { ...transport, writeReceipts: true }),
(error: unknown) => !(error instanceof CultMeshRudpWriteUnconfirmedError) && error instanceof Error,
);
assert.ok(!server.seen.includes("cultnet.document_put_raw.v0"), "no put was sent");
} finally { server.close(); }
});
test("sendOnly cannot be combined with writeReceipts", async () => {
await assert.rejects(publish({ endpoint: "rudp://127.0.0.1:9" }, { sendOnly: true, writeReceipts: true }), TypeError);
});
test("without writeReceipts no hello is sent and the receipt frames are never asked for", async () => {
const server = await fakeServer(nextConnection, { advertise: true, answer: "silence" });
try {
await assert.rejects(publish(server, { resultWindowMs: 100 }), CultMeshRudpWriteUnconfirmedError);
assert.deepEqual(server.seen, ["cultnet.document_put_raw.v0"]);
} finally { server.close(); }
});

View file

@ -2,6 +2,10 @@
## Unreleased (breaking; version 0.x, so the next minor) ## Unreleased (breaking; version 0.x, so the next minor)
- `apply_raw_document_record` and `apply_raw_snapshot` raise `SchemaConflictError` when a remote record is of another type than the one the cache holds at its key (`cultcache-py` holds one record per key whatever its type); the cache is unchanged, and the caller decides whether to drop the record or the peer. `cultmesh-py`'s server answers such a put with `message_invalid`; its client's `read_next_change` lets the error out.
- `CultNetDatabaseChange` carries `commit_id` and `commit_count` (`commitId`/`commitCount`; None outside a multi-record commit, as C# writes nil): `from_wire` refuses an empty or non-string id and a count that is not an integer of at least 2, `to_wire` writes them when set, and the published `database_change_raw` schema lists both. Python applies no batch. The `simulation_fact` document's `shard`, `frame`, `subject` and `claim_kind` aliases are `roles` (many facts share each); the catalog bytes do not move.
- Decodes the write receipt frames of `contracts/cultnet/interop/write-receipt.md`: `parse_message` validates `cultnet.write_admitted.v0` and `cultnet.error.v1` against their closed schemas (unknown keys, empty or non-string ids, an unknown operation or code are refused; `error.v1` requires `messageId`, string or null; `limit_reached` is `error.v1` only). Added `CULTNET_ERROR_V1_CODES`, `CULTNET_WRITE_RECEIPT_MESSAGE_VERSIONS`, `write_content_hash` and `hello_advertises_write_receipts`; `hello` takes `supported_message_versions`. `tests/test_write_receipt.py` replays the vectors' `contentHash`, `wire` and `schemaInvalid` sections. Not implemented here: the client write tracker (`scenarios`) and the server rules (`dedupe`, `blankFields`); this package has no receipt server and no write client.
- Added `message_invalid` to `CULTNET_ERROR_CODES` and `message_invalid_error(field, reason)`, byte-identical to the C# `ForMessageInvalid`. `cultmesh-py` refuses a malformed put or delete with it instead of the out-of-schema `malformed_document_put` and `malformed_document_delete`, which are removed.
- Added `CultNetRudpSocketTransportServer` (`cultnet_py.rudp_server`): one UDP socket serving many RUDP peers, one session per - Added `CultNetRudpSocketTransportServer` (`cultnet_py.rudp_server`): one UDP socket serving many RUDP peers, one session per
remote endpoint, ported from the C# `CultNetRudpSocketTransportServer` and held to its bytes by remote endpoint, ported from the C# `CultNetRudpSocketTransportServer` and held to its bytes by
`tests/vectors/rudp-multipeer-server/transcript.txt`. The caller drives it (`try_receive_once`, `poll_resends`). Like the C# `tests/vectors/rudp-multipeer-server/transcript.txt`. The caller drives it (`try_receive_once`, `poll_resends`). Like the C#

View file

@ -39,6 +39,8 @@ from .cultmesh_contracts import (
from .framing import decode_frame, encode_frame, read_frame, write_frame from .framing import decode_frame, encode_frame, read_frame, write_frame
from .messages import ( from .messages import (
CULTNET_ERROR_CODES, CULTNET_ERROR_CODES,
CULTNET_ERROR_V1_CODES,
CULTNET_WRITE_RECEIPT_MESSAGE_VERSIONS,
CultNetMessage, CultNetMessage,
CultNetWitnessArtifactBundle, CultNetWitnessArtifactBundle,
compute_simulation_claim_hash, compute_simulation_claim_hash,
@ -49,8 +51,11 @@ from .messages import (
document_put_raw, document_put_raw,
encode_witness_artifact_bundle_payload, encode_witness_artifact_bundle_payload,
error_message, error_message,
message_invalid_error,
unsupported_schema_version_error, unsupported_schema_version_error,
write_content_hash,
hello, hello,
hello_advertises_write_receipts,
login, login,
login_success, login_success,
parse_message, parse_message,
@ -175,6 +180,8 @@ __all__ = [
"verse_catalog_request", "verse_catalog_request",
"verse_from_wire", "verse_from_wire",
"CULTNET_ERROR_CODES", "CULTNET_ERROR_CODES",
"CULTNET_ERROR_V1_CODES",
"CULTNET_WRITE_RECEIPT_MESSAGE_VERSIONS",
"CultNetMessage", "CultNetMessage",
"CultNetWitnessArtifactBundle", "CultNetWitnessArtifactBundle",
"CultNetDatabaseSubscription", "CultNetDatabaseSubscription",
@ -262,8 +269,11 @@ __all__ = [
"encode_frame", "encode_frame",
"encode_rudp_packet", "encode_rudp_packet",
"error_message", "error_message",
"message_invalid_error",
"unsupported_schema_version_error", "unsupported_schema_version_error",
"hello", "hello",
"hello_advertises_write_receipts",
"write_content_hash",
"INTEROP_WIRE_CONTRACT", "INTEROP_WIRE_CONTRACT",
"login", "login",
"login_success", "login_success",

View file

@ -58,6 +58,7 @@ class CultNetRawClient:
create_transport: CultNetSchemaTransportFactory | None = None create_transport: CultNetSchemaTransportFactory | None = None
def send(self, message: CultNetMessage | dict[str, Any]) -> None: def send(self, message: CultNetMessage | dict[str, Any]) -> None:
"""Fire-and-forget: an ack or a return is not admission; read the result frame."""
wire = message.to_wire() if isinstance(message, CultNetMessage) else message wire = message.to_wire() if isinstance(message, CultNetMessage) else message
with self.open_transport() as transport: with self.open_transport() as transport:
transport.send("schema", msgpack.packb(wire, use_bin_type=True)) transport.send("schema", msgpack.packb(wire, use_bin_type=True))
@ -319,6 +320,7 @@ class CultNetDatabaseSubscription:
self._transport = None self._transport = None
def send(self, message: CultNetMessage | dict[str, Any]) -> None: def send(self, message: CultNetMessage | dict[str, Any]) -> None:
"""Fire-and-forget: an ack or a return is not admission; read the result frame."""
if self._transport is None: if self._transport is None:
raise RuntimeError("CultNet database subscription is not open") raise RuntimeError("CultNet database subscription is not open")
wire = message.to_wire() if isinstance(message, CultNetMessage) else message wire = message.to_wire() if isinstance(message, CultNetMessage) else message

View file

@ -692,7 +692,7 @@ simulation_fact_document: DocumentDefinition[CultMeshSimulationFact] = define_da
], ],
cls=CultMeshSimulationFact, cls=CultMeshSimulationFact,
name="fact_id", name="fact_id",
indexes={"shard": "shard_id", "frame": "frame", "subject": "subject_id", "claim_kind": "claim_kind"}, roles={"shard": "shard_id", "frame": "frame", "subject": "subject_id", "claim_kind": "claim_kind"},
schema_name=SIMULATION_FACT_DOCUMENT_TYPE, schema_name=SIMULATION_FACT_DOCUMENT_TYPE,
schema_version=SIMULATION_FACT_SCHEMA_VERSION, schema_version=SIMULATION_FACT_SCHEMA_VERSION,
) )

View file

@ -6,6 +6,8 @@ from typing import Any
import msgpack import msgpack
from .unicode_messageid_tables import DEFAULT_IGNORABLE, LETTER_OR_NUMBER
@dataclass(frozen=True) @dataclass(frozen=True)
class CultNetMessage: class CultNetMessage:
@ -29,7 +31,11 @@ def parse_message(payload: bytes) -> CultNetMessage:
body = dict(decoded) body = dict(decoded)
del body["schemaVersion"] del body["schemaVersion"]
if schema_version == "cultnet.error.v0": if schema_version == "cultnet.error.v0":
_validate_error_body(body) _validate_error_body(body, "cultnet.error.v0", CULTNET_ERROR_CODES)
elif schema_version == "cultnet.error.v1":
_validate_error_body(body, "cultnet.error.v1", CULTNET_ERROR_V1_CODES)
elif schema_version == "cultnet.write_admitted.v0":
_validate_write_admitted_body(body)
return CultNetMessage(schema_version=schema_version, body=body) return CultNetMessage(schema_version=schema_version, body=body)
@ -44,8 +50,16 @@ CULTNET_ERROR_CODES: tuple[str, ...] = (
"unowned_schema", "unowned_schema",
"write_not_authorized", "write_not_authorized",
"unsupported_schema_version", "unsupported_schema_version",
"message_invalid",
) )
# cultnet.error.v1 (contracts/cultnet/interop/cultnet.error.v1.schema.json): v0's codes plus limit_reached.
CULTNET_ERROR_V1_CODES: tuple[str, ...] = (*CULTNET_ERROR_CODES, "limit_reached")
# The two frames a receipt-capable hello lists in supportedMessageVersions; a peer lists both or neither
# (contracts/cultnet/interop/write-receipt.md, "Capability: opting in").
CULTNET_WRITE_RECEIPT_MESSAGE_VERSIONS: tuple[str, ...] = ("cultnet.write_admitted.v0", "cultnet.error.v1")
_ERROR_DETAILS_KEYS = ("field", "value", "asOf", "current") _ERROR_DETAILS_KEYS = ("field", "value", "asOf", "current")
@ -53,16 +67,80 @@ def _is_uint(value: Any) -> bool:
return isinstance(value, int) and not isinstance(value, bool) and value >= 0 return isinstance(value, int) and not isinstance(value, bool) and value >= 0
def _validate_error_body(body: dict[str, Any]) -> None: def _is_name(value: Any) -> bool:
"""Refuse a cultnet.error.v0 body that is not the full-key shape the schema requires.""" return isinstance(value, str) and value != ""
expected = ("error", "routingHint", "code", "details")
# write-receipt.md: a letter (Lu Ll Lt Lm Lo) or number (Nd Nl No) somewhere, no Default_Ignorable code point anywhere, as of
# Unicode 16.0, compared raw (never stripped). The tables are embedded (unicode_messageid_tables.py, generated by
# scripts/gen-unicode-messageid-tables.py), not unicodedata's, so the running Python's Unicode version changes no result;
# the other four runtimes embed the same generated tables.
def _in_ranges(ranges: tuple[int, ...], point: int) -> bool:
"""Whether [point] lies in the flattened, sorted (low, high) pairs."""
low, high = 0, len(ranges) // 2 - 1
while low <= high:
mid = (low + high) // 2
if point < ranges[2 * mid]:
high = mid - 1
elif point > ranges[2 * mid + 1]:
low = mid + 1
else:
return True
return False
def _is_admitted_message_id(value: Any) -> bool:
if not isinstance(value, str):
return False
has_letter_or_number = False
for ch in value:
point = ord(ch)
if _in_ranges(DEFAULT_IGNORABLE, point):
return False
has_letter_or_number = has_letter_or_number or _in_ranges(LETTER_OR_NUMBER, point)
return has_letter_or_number
def _validate_write_admitted_body(body: dict[str, Any]) -> None:
"""Refuse a cultnet.write_admitted.v0 body that is not exactly the five keys of its closed schema."""
expected = ("messageId", "operation", "schemaId", "recordKey")
if set(body) != set(expected): if set(body) != set(expected):
raise ValueError(f"cultnet.error.v0 must carry exactly {list(expected)}, got {sorted(body)}") raise ValueError(f"cultnet.write_admitted.v0 must carry exactly {list(expected)}, got {sorted(body)}")
for key in ("messageId", "schemaId", "recordKey"):
if not _is_name(body[key]):
raise ValueError(f"cultnet.write_admitted.v0 {key} must be a non-empty string")
if not _is_admitted_message_id(body["messageId"]):
raise ValueError("cultnet.write_admitted.v0 messageId must contain a letter or number and no Default_Ignorable code point")
if body["operation"] not in ("put", "delete"):
raise ValueError("cultnet.write_admitted.v0 operation must be put or delete")
def write_content_hash(operation: str, schema_id: str, record_key: str, payload: bytes | None) -> bytes:
"""SHA-256 of the MessagePack array [operation, schemaId, recordKey, payload] (write-receipt.md, rule 5).
payload is the raw document payload, packed as a MessagePack bin for a put and nil for a delete.
"""
return hashlib.sha256(msgpack.packb([operation, schema_id, record_key, payload], use_bin_type=True)).digest()
def hello_advertises_write_receipts(hello_message: CultNetMessage) -> bool:
"""True when a hello lists both receipt frames: the only thing that makes a connection receipt-capable."""
listed = hello_message.body.get("supportedMessageVersions")
return isinstance(listed, list) and all(version in listed for version in CULTNET_WRITE_RECEIPT_MESSAGE_VERSIONS)
def _validate_error_body(body: dict[str, Any], version: str, codes: tuple[str, ...]) -> None:
"""Refuse a cultnet.error body that is not the full-key shape its closed schema requires."""
expected = ("error", "routingHint", "code", "details") + (("messageId",) if version == "cultnet.error.v1" else ())
if set(body) != set(expected):
raise ValueError(f"{version} must carry exactly {list(expected)}, got {sorted(body)}")
if not isinstance(body["error"], str) or not body["error"]: if not isinstance(body["error"], str) or not body["error"]:
raise ValueError("cultnet.error.v0 error must be a non-empty string") raise ValueError(f"{version} error must be a non-empty string")
code = body["code"] code = body["code"]
if code is not None and (not isinstance(code, str) or code not in CULTNET_ERROR_CODES): if code is not None and (not isinstance(code, str) or code not in codes):
raise ValueError(f"cultnet.error.v0 code {code!r} is not a recognized refusal code") raise ValueError(f"{version} code {code!r} is not a recognized refusal code")
if "messageId" in body and body["messageId"] is not None and not _is_name(body["messageId"]):
raise ValueError(f"{version} messageId must be a non-empty string or null")
hint = body["routingHint"] hint = body["routingHint"]
if hint is not None: if hint is not None:
if ( if (
@ -71,17 +149,17 @@ def _validate_error_body(body: dict[str, Any]) -> None:
or not isinstance(hint["reason"], str) or not isinstance(hint["reason"], str)
or not (hint["shard"] is None or isinstance(hint["shard"], dict)) or not (hint["shard"] is None or isinstance(hint["shard"], dict))
): ):
raise ValueError("cultnet.error.v0 routingHint must be null or {shard, reason}") raise ValueError(f"{version} routingHint must be null or {{shard, reason}}")
details = body["details"] details = body["details"]
if details is not None: if details is not None:
if not isinstance(details, dict) or set(details) != set(_ERROR_DETAILS_KEYS): if not isinstance(details, dict) or set(details) != set(_ERROR_DETAILS_KEYS):
raise ValueError(f"cultnet.error.v0 details must be null or carry exactly {list(_ERROR_DETAILS_KEYS)}") raise ValueError(f"{version} details must be null or carry exactly {list(_ERROR_DETAILS_KEYS)}")
for key in ("field", "value"): for key in ("field", "value"):
if details[key] is not None and not isinstance(details[key], str): if details[key] is not None and not isinstance(details[key], str):
raise ValueError(f"cultnet.error.v0 details.{key} must be a string or null") raise ValueError(f"{version} details.{key} must be a string or null")
for key in ("asOf", "current"): for key in ("asOf", "current"):
if details[key] is not None and not _is_uint(details[key]): if details[key] is not None and not _is_uint(details[key]):
raise ValueError(f"cultnet.error.v0 details.{key} must be a non-negative integer or null") raise ValueError(f"{version} details.{key} must be a non-negative integer or null")
def hello( def hello(
@ -91,17 +169,18 @@ def hello(
supported_schema_versions: list[str] | None = None, supported_schema_versions: list[str] | None = None,
supported_mutation_contracts: list[dict[str, Any]] | None = None, supported_mutation_contracts: list[dict[str, Any]] | None = None,
transport_profiles: list[dict[str, Any]] | None = None, transport_profiles: list[dict[str, Any]] | None = None,
supported_message_versions: list[str] | None = None,
) -> CultNetMessage: ) -> CultNetMessage:
return CultNetMessage( body: dict[str, Any] = {
"cultnet.hello.v0", "runtimeId": runtime_id,
{ "supportedWireContracts": supported_wire_contracts or ["cultnet.schema.v0"],
"runtimeId": runtime_id, "supportedSchemaVersions": supported_schema_versions or [],
"supportedWireContracts": supported_wire_contracts or ["cultnet.schema.v0"], "supportedMutationContracts": supported_mutation_contracts or [],
"supportedSchemaVersions": supported_schema_versions or [], "transportProfiles": transport_profiles or [],
"supportedMutationContracts": supported_mutation_contracts or [], }
"transportProfiles": transport_profiles or [], if supported_message_versions is not None:
}, body["supportedMessageVersions"] = supported_message_versions
) return CultNetMessage("cultnet.hello.v0", body)
def login( def login(
@ -171,7 +250,7 @@ def error_message(
} }
if details is not None and set(details) - set(_ERROR_DETAILS_KEYS): if details is not None and set(details) - set(_ERROR_DETAILS_KEYS):
raise ValueError(f"cultnet.error.v0 details has unknown keys {sorted(set(details) - set(_ERROR_DETAILS_KEYS))}") raise ValueError(f"cultnet.error.v0 details has unknown keys {sorted(set(details) - set(_ERROR_DETAILS_KEYS))}")
_validate_error_body(body) _validate_error_body(body, "cultnet.error.v0", CULTNET_ERROR_CODES)
return CultNetMessage("cultnet.error.v0", body) return CultNetMessage("cultnet.error.v0", body)
@ -185,6 +264,11 @@ def unsupported_schema_version_error(schema_version: Any) -> CultNetMessage:
) )
def message_invalid_error(field: str, reason: str) -> CultNetMessage:
"""The refusal for a request that lacks a required field or carries an ill-typed one (C# ForMessageInvalid)."""
return error_message(f"message_invalid: {reason}", code="message_invalid", details={"field": field})
def login_success( def login_success(
*, *,
nonce: str, nonce: str,

View file

@ -22,6 +22,7 @@ CULTNET_ERROR_CODES = (
"unowned_schema", "unowned_schema",
"write_not_authorized", "write_not_authorized",
"unsupported_schema_version", "unsupported_schema_version",
"message_invalid",
) )
WIRE_MESSAGE_SCHEMA_VERSIONS = ( WIRE_MESSAGE_SCHEMA_VERSIONS = (
@ -428,9 +429,11 @@ def wire_message_schema_properties(schema_version: str) -> dict[str, Any]:
"messageId": common["messageId"], "messageId": common["messageId"],
"subscriptionId": common["subscriptionId"], "subscriptionId": common["subscriptionId"],
"changeKind": {"enum": ["added", "updated", "removed"]}, "changeKind": {"enum": ["added", "updated", "removed"]},
"document": {"$ref": "#/$defs/rawDocumentRecord"}, "document": {"oneOf": [{"type": "null"}, {"$ref": "#/$defs/rawDocumentRecord"}]},
"schemaId": common["schemaId"], "schemaId": {"type": ["string", "null"], "minLength": 1},
"recordKey": common["recordKey"], "recordKey": {"type": ["string", "null"], "minLength": 1},
"commitId": {"type": ["string", "null"], "minLength": 1},
"commitCount": {"type": ["integer", "null"], "minimum": 2},
}, },
"cultnet.shard_catalog_request.v0": { "cultnet.shard_catalog_request.v0": {
**common, **common,

View file

@ -15,6 +15,10 @@ class CultNetDatabaseChange:
raw_document: CultNetRawDocumentRecord | None = None raw_document: CultNetRawDocumentRecord | None = None
schema_id: str | None = None schema_id: str | None = None
record_key: str | None = None record_key: str | None = None
commit_id: str | None = None
"""Set on every change of one multi-record commit, the same for all of them; None outside one (C# writes nil)."""
commit_count: int | None = None
"""How many changes the commit holds (>= 2); None outside one. Carried data: Python applies no batch."""
@classmethod @classmethod
def from_wire(cls, message: dict[str, Any]) -> "CultNetDatabaseChange": def from_wire(cls, message: dict[str, Any]) -> "CultNetDatabaseChange":
@ -31,6 +35,12 @@ class CultNetDatabaseChange:
if raw_document is not None: if raw_document is not None:
schema_id = schema_id or raw_document.schema_id schema_id = schema_id or raw_document.schema_id
record_key = record_key or raw_document.record_key record_key = record_key or raw_document.record_key
commit_id = message.get("commitId")
if commit_id is not None and not (isinstance(commit_id, str) and commit_id):
raise ValueError(f"database change commitId must be a non-empty string, received {commit_id!r}")
commit_count = message.get("commitCount")
if commit_count is not None and (isinstance(commit_count, bool) or not isinstance(commit_count, int) or commit_count < 2):
raise ValueError(f"database change commitCount must be an integer of at least 2, received {commit_count!r}")
return cls( return cls(
message_id=str(message.get("messageId") or ""), message_id=str(message.get("messageId") or ""),
subscription_id=str(message.get("subscriptionId") or ""), subscription_id=str(message.get("subscriptionId") or ""),
@ -39,6 +49,8 @@ class CultNetDatabaseChange:
raw_document=raw_document, raw_document=raw_document,
schema_id=schema_id, schema_id=schema_id,
record_key=record_key, record_key=record_key,
commit_id=commit_id,
commit_count=commit_count,
) )
def to_wire(self) -> dict[str, Any]: def to_wire(self) -> dict[str, Any]:
@ -56,6 +68,10 @@ class CultNetDatabaseChange:
wire["schemaId"] = self.schema_id wire["schemaId"] = self.schema_id
if self.record_key is not None: if self.record_key is not None:
wire["recordKey"] = self.record_key wire["recordKey"] = self.record_key
if self.commit_id is not None:
wire["commitId"] = self.commit_id
if self.commit_count is not None:
wire["commitCount"] = self.commit_count
return wire return wire

View file

@ -1256,6 +1256,7 @@ class CultNetRudpSocketTransportConnection:
self._send_packet(self.session.create_connect(_now_ms(), payload)) self._send_packet(self.session.create_connect(_now_ms(), payload))
def send(self, channel_id: str, payload: bytes) -> None: def send(self, channel_id: str, payload: bytes) -> None:
"""Fire-and-forget: an ack or a return is not admission; read the result frame."""
packets = self.session.send_many( packets = self.session.send_many(
channel_id, channel_id,
payload, payload,
@ -1567,6 +1568,7 @@ class TcpFramedTransportConnection:
) )
def send(self, channel_id: str, payload: bytes) -> None: def send(self, channel_id: str, payload: bytes) -> None:
"""Fire-and-forget: an ack or a return is not admission; read the result frame."""
if channel_id != "schema": if channel_id != "schema":
raise ValueError(f"tcp_framed transport only supports the schema channel, got {channel_id!r}") raise ValueError(f"tcp_framed transport only supports the schema channel, got {channel_id!r}")
before = len(payload) + 4 before = len(payload) + 4

View file

@ -0,0 +1,127 @@
# generated, Unicode 16.0.0, by scripts/gen-unicode-messageid-tables.py (inputs sha256 24a07ef9fabf71e3); do not edit
# (low, high) pairs, flattened and sorted.
LETTER_OR_NUMBER: tuple[int, ...] = (
0x30, 0x39, 0x41, 0x5A, 0x61, 0x7A, 0xAA, 0xAA, 0xB2, 0xB3, 0xB5, 0xB5, 0xB9, 0xBA, 0xBC, 0xBE, 0xC0, 0xD6, 0xD8,
0xF6, 0xF8, 0x2C1, 0x2C6, 0x2D1, 0x2E0, 0x2E4, 0x2EC, 0x2EC, 0x2EE, 0x2EE, 0x370, 0x374, 0x376, 0x377, 0x37A,
0x37D, 0x37F, 0x37F, 0x386, 0x386, 0x388, 0x38A, 0x38C, 0x38C, 0x38E, 0x3A1, 0x3A3, 0x3F5, 0x3F7, 0x481, 0x48A,
0x52F, 0x531, 0x556, 0x559, 0x559, 0x560, 0x588, 0x5D0, 0x5EA, 0x5EF, 0x5F2, 0x620, 0x64A, 0x660, 0x669, 0x66E,
0x66F, 0x671, 0x6D3, 0x6D5, 0x6D5, 0x6E5, 0x6E6, 0x6EE, 0x6FC, 0x6FF, 0x6FF, 0x710, 0x710, 0x712, 0x72F, 0x74D,
0x7A5, 0x7B1, 0x7B1, 0x7C0, 0x7EA, 0x7F4, 0x7F5, 0x7FA, 0x7FA, 0x800, 0x815, 0x81A, 0x81A, 0x824, 0x824, 0x828,
0x828, 0x840, 0x858, 0x860, 0x86A, 0x870, 0x887, 0x889, 0x88E, 0x8A0, 0x8C9, 0x904, 0x939, 0x93D, 0x93D, 0x950,
0x950, 0x958, 0x961, 0x966, 0x96F, 0x971, 0x980, 0x985, 0x98C, 0x98F, 0x990, 0x993, 0x9A8, 0x9AA, 0x9B0, 0x9B2,
0x9B2, 0x9B6, 0x9B9, 0x9BD, 0x9BD, 0x9CE, 0x9CE, 0x9DC, 0x9DD, 0x9DF, 0x9E1, 0x9E6, 0x9F1, 0x9F4, 0x9F9, 0x9FC,
0x9FC, 0xA05, 0xA0A, 0xA0F, 0xA10, 0xA13, 0xA28, 0xA2A, 0xA30, 0xA32, 0xA33, 0xA35, 0xA36, 0xA38, 0xA39, 0xA59,
0xA5C, 0xA5E, 0xA5E, 0xA66, 0xA6F, 0xA72, 0xA74, 0xA85, 0xA8D, 0xA8F, 0xA91, 0xA93, 0xAA8, 0xAAA, 0xAB0, 0xAB2,
0xAB3, 0xAB5, 0xAB9, 0xABD, 0xABD, 0xAD0, 0xAD0, 0xAE0, 0xAE1, 0xAE6, 0xAEF, 0xAF9, 0xAF9, 0xB05, 0xB0C, 0xB0F,
0xB10, 0xB13, 0xB28, 0xB2A, 0xB30, 0xB32, 0xB33, 0xB35, 0xB39, 0xB3D, 0xB3D, 0xB5C, 0xB5D, 0xB5F, 0xB61, 0xB66,
0xB6F, 0xB71, 0xB77, 0xB83, 0xB83, 0xB85, 0xB8A, 0xB8E, 0xB90, 0xB92, 0xB95, 0xB99, 0xB9A, 0xB9C, 0xB9C, 0xB9E,
0xB9F, 0xBA3, 0xBA4, 0xBA8, 0xBAA, 0xBAE, 0xBB9, 0xBD0, 0xBD0, 0xBE6, 0xBF2, 0xC05, 0xC0C, 0xC0E, 0xC10, 0xC12,
0xC28, 0xC2A, 0xC39, 0xC3D, 0xC3D, 0xC58, 0xC5A, 0xC5D, 0xC5D, 0xC60, 0xC61, 0xC66, 0xC6F, 0xC78, 0xC7E, 0xC80,
0xC80, 0xC85, 0xC8C, 0xC8E, 0xC90, 0xC92, 0xCA8, 0xCAA, 0xCB3, 0xCB5, 0xCB9, 0xCBD, 0xCBD, 0xCDD, 0xCDE, 0xCE0,
0xCE1, 0xCE6, 0xCEF, 0xCF1, 0xCF2, 0xD04, 0xD0C, 0xD0E, 0xD10, 0xD12, 0xD3A, 0xD3D, 0xD3D, 0xD4E, 0xD4E, 0xD54,
0xD56, 0xD58, 0xD61, 0xD66, 0xD78, 0xD7A, 0xD7F, 0xD85, 0xD96, 0xD9A, 0xDB1, 0xDB3, 0xDBB, 0xDBD, 0xDBD, 0xDC0,
0xDC6, 0xDE6, 0xDEF, 0xE01, 0xE30, 0xE32, 0xE33, 0xE40, 0xE46, 0xE50, 0xE59, 0xE81, 0xE82, 0xE84, 0xE84, 0xE86,
0xE8A, 0xE8C, 0xEA3, 0xEA5, 0xEA5, 0xEA7, 0xEB0, 0xEB2, 0xEB3, 0xEBD, 0xEBD, 0xEC0, 0xEC4, 0xEC6, 0xEC6, 0xED0,
0xED9, 0xEDC, 0xEDF, 0xF00, 0xF00, 0xF20, 0xF33, 0xF40, 0xF47, 0xF49, 0xF6C, 0xF88, 0xF8C, 0x1000, 0x102A,
0x103F, 0x1049, 0x1050, 0x1055, 0x105A, 0x105D, 0x1061, 0x1061, 0x1065, 0x1066, 0x106E, 0x1070, 0x1075, 0x1081,
0x108E, 0x108E, 0x1090, 0x1099, 0x10A0, 0x10C5, 0x10C7, 0x10C7, 0x10CD, 0x10CD, 0x10D0, 0x10FA, 0x10FC, 0x1248,
0x124A, 0x124D, 0x1250, 0x1256, 0x1258, 0x1258, 0x125A, 0x125D, 0x1260, 0x1288, 0x128A, 0x128D, 0x1290, 0x12B0,
0x12B2, 0x12B5, 0x12B8, 0x12BE, 0x12C0, 0x12C0, 0x12C2, 0x12C5, 0x12C8, 0x12D6, 0x12D8, 0x1310, 0x1312, 0x1315,
0x1318, 0x135A, 0x1369, 0x137C, 0x1380, 0x138F, 0x13A0, 0x13F5, 0x13F8, 0x13FD, 0x1401, 0x166C, 0x166F, 0x167F,
0x1681, 0x169A, 0x16A0, 0x16EA, 0x16EE, 0x16F8, 0x1700, 0x1711, 0x171F, 0x1731, 0x1740, 0x1751, 0x1760, 0x176C,
0x176E, 0x1770, 0x1780, 0x17B3, 0x17D7, 0x17D7, 0x17DC, 0x17DC, 0x17E0, 0x17E9, 0x17F0, 0x17F9, 0x1810, 0x1819,
0x1820, 0x1878, 0x1880, 0x1884, 0x1887, 0x18A8, 0x18AA, 0x18AA, 0x18B0, 0x18F5, 0x1900, 0x191E, 0x1946, 0x196D,
0x1970, 0x1974, 0x1980, 0x19AB, 0x19B0, 0x19C9, 0x19D0, 0x19DA, 0x1A00, 0x1A16, 0x1A20, 0x1A54, 0x1A80, 0x1A89,
0x1A90, 0x1A99, 0x1AA7, 0x1AA7, 0x1B05, 0x1B33, 0x1B45, 0x1B4C, 0x1B50, 0x1B59, 0x1B83, 0x1BA0, 0x1BAE, 0x1BE5,
0x1C00, 0x1C23, 0x1C40, 0x1C49, 0x1C4D, 0x1C7D, 0x1C80, 0x1C8A, 0x1C90, 0x1CBA, 0x1CBD, 0x1CBF, 0x1CE9, 0x1CEC,
0x1CEE, 0x1CF3, 0x1CF5, 0x1CF6, 0x1CFA, 0x1CFA, 0x1D00, 0x1DBF, 0x1E00, 0x1F15, 0x1F18, 0x1F1D, 0x1F20, 0x1F45,
0x1F48, 0x1F4D, 0x1F50, 0x1F57, 0x1F59, 0x1F59, 0x1F5B, 0x1F5B, 0x1F5D, 0x1F5D, 0x1F5F, 0x1F7D, 0x1F80, 0x1FB4,
0x1FB6, 0x1FBC, 0x1FBE, 0x1FBE, 0x1FC2, 0x1FC4, 0x1FC6, 0x1FCC, 0x1FD0, 0x1FD3, 0x1FD6, 0x1FDB, 0x1FE0, 0x1FEC,
0x1FF2, 0x1FF4, 0x1FF6, 0x1FFC, 0x2070, 0x2071, 0x2074, 0x2079, 0x207F, 0x2089, 0x2090, 0x209C, 0x2102, 0x2102,
0x2107, 0x2107, 0x210A, 0x2113, 0x2115, 0x2115, 0x2119, 0x211D, 0x2124, 0x2124, 0x2126, 0x2126, 0x2128, 0x2128,
0x212A, 0x212D, 0x212F, 0x2139, 0x213C, 0x213F, 0x2145, 0x2149, 0x214E, 0x214E, 0x2150, 0x2189, 0x2460, 0x249B,
0x24EA, 0x24FF, 0x2776, 0x2793, 0x2C00, 0x2CE4, 0x2CEB, 0x2CEE, 0x2CF2, 0x2CF3, 0x2CFD, 0x2CFD, 0x2D00, 0x2D25,
0x2D27, 0x2D27, 0x2D2D, 0x2D2D, 0x2D30, 0x2D67, 0x2D6F, 0x2D6F, 0x2D80, 0x2D96, 0x2DA0, 0x2DA6, 0x2DA8, 0x2DAE,
0x2DB0, 0x2DB6, 0x2DB8, 0x2DBE, 0x2DC0, 0x2DC6, 0x2DC8, 0x2DCE, 0x2DD0, 0x2DD6, 0x2DD8, 0x2DDE, 0x2E2F, 0x2E2F,
0x3005, 0x3007, 0x3021, 0x3029, 0x3031, 0x3035, 0x3038, 0x303C, 0x3041, 0x3096, 0x309D, 0x309F, 0x30A1, 0x30FA,
0x30FC, 0x30FF, 0x3105, 0x312F, 0x3131, 0x318E, 0x3192, 0x3195, 0x31A0, 0x31BF, 0x31F0, 0x31FF, 0x3220, 0x3229,
0x3248, 0x324F, 0x3251, 0x325F, 0x3280, 0x3289, 0x32B1, 0x32BF, 0x3400, 0x4DBF, 0x4E00, 0xA48C, 0xA4D0, 0xA4FD,
0xA500, 0xA60C, 0xA610, 0xA62B, 0xA640, 0xA66E, 0xA67F, 0xA69D, 0xA6A0, 0xA6EF, 0xA717, 0xA71F, 0xA722, 0xA788,
0xA78B, 0xA7CD, 0xA7D0, 0xA7D1, 0xA7D3, 0xA7D3, 0xA7D5, 0xA7DC, 0xA7F2, 0xA801, 0xA803, 0xA805, 0xA807, 0xA80A,
0xA80C, 0xA822, 0xA830, 0xA835, 0xA840, 0xA873, 0xA882, 0xA8B3, 0xA8D0, 0xA8D9, 0xA8F2, 0xA8F7, 0xA8FB, 0xA8FB,
0xA8FD, 0xA8FE, 0xA900, 0xA925, 0xA930, 0xA946, 0xA960, 0xA97C, 0xA984, 0xA9B2, 0xA9CF, 0xA9D9, 0xA9E0, 0xA9E4,
0xA9E6, 0xA9FE, 0xAA00, 0xAA28, 0xAA40, 0xAA42, 0xAA44, 0xAA4B, 0xAA50, 0xAA59, 0xAA60, 0xAA76, 0xAA7A, 0xAA7A,
0xAA7E, 0xAAAF, 0xAAB1, 0xAAB1, 0xAAB5, 0xAAB6, 0xAAB9, 0xAABD, 0xAAC0, 0xAAC0, 0xAAC2, 0xAAC2, 0xAADB, 0xAADD,
0xAAE0, 0xAAEA, 0xAAF2, 0xAAF4, 0xAB01, 0xAB06, 0xAB09, 0xAB0E, 0xAB11, 0xAB16, 0xAB20, 0xAB26, 0xAB28, 0xAB2E,
0xAB30, 0xAB5A, 0xAB5C, 0xAB69, 0xAB70, 0xABE2, 0xABF0, 0xABF9, 0xAC00, 0xD7A3, 0xD7B0, 0xD7C6, 0xD7CB, 0xD7FB,
0xF900, 0xFA6D, 0xFA70, 0xFAD9, 0xFB00, 0xFB06, 0xFB13, 0xFB17, 0xFB1D, 0xFB1D, 0xFB1F, 0xFB28, 0xFB2A, 0xFB36,
0xFB38, 0xFB3C, 0xFB3E, 0xFB3E, 0xFB40, 0xFB41, 0xFB43, 0xFB44, 0xFB46, 0xFBB1, 0xFBD3, 0xFD3D, 0xFD50, 0xFD8F,
0xFD92, 0xFDC7, 0xFDF0, 0xFDFB, 0xFE70, 0xFE74, 0xFE76, 0xFEFC, 0xFF10, 0xFF19, 0xFF21, 0xFF3A, 0xFF41, 0xFF5A,
0xFF66, 0xFFBE, 0xFFC2, 0xFFC7, 0xFFCA, 0xFFCF, 0xFFD2, 0xFFD7, 0xFFDA, 0xFFDC, 0x10000, 0x1000B, 0x1000D,
0x10026, 0x10028, 0x1003A, 0x1003C, 0x1003D, 0x1003F, 0x1004D, 0x10050, 0x1005D, 0x10080, 0x100FA, 0x10107,
0x10133, 0x10140, 0x10178, 0x1018A, 0x1018B, 0x10280, 0x1029C, 0x102A0, 0x102D0, 0x102E1, 0x102FB, 0x10300,
0x10323, 0x1032D, 0x1034A, 0x10350, 0x10375, 0x10380, 0x1039D, 0x103A0, 0x103C3, 0x103C8, 0x103CF, 0x103D1,
0x103D5, 0x10400, 0x1049D, 0x104A0, 0x104A9, 0x104B0, 0x104D3, 0x104D8, 0x104FB, 0x10500, 0x10527, 0x10530,
0x10563, 0x10570, 0x1057A, 0x1057C, 0x1058A, 0x1058C, 0x10592, 0x10594, 0x10595, 0x10597, 0x105A1, 0x105A3,
0x105B1, 0x105B3, 0x105B9, 0x105BB, 0x105BC, 0x105C0, 0x105F3, 0x10600, 0x10736, 0x10740, 0x10755, 0x10760,
0x10767, 0x10780, 0x10785, 0x10787, 0x107B0, 0x107B2, 0x107BA, 0x10800, 0x10805, 0x10808, 0x10808, 0x1080A,
0x10835, 0x10837, 0x10838, 0x1083C, 0x1083C, 0x1083F, 0x10855, 0x10858, 0x10876, 0x10879, 0x1089E, 0x108A7,
0x108AF, 0x108E0, 0x108F2, 0x108F4, 0x108F5, 0x108FB, 0x1091B, 0x10920, 0x10939, 0x10980, 0x109B7, 0x109BC,
0x109CF, 0x109D2, 0x10A00, 0x10A10, 0x10A13, 0x10A15, 0x10A17, 0x10A19, 0x10A35, 0x10A40, 0x10A48, 0x10A60,
0x10A7E, 0x10A80, 0x10A9F, 0x10AC0, 0x10AC7, 0x10AC9, 0x10AE4, 0x10AEB, 0x10AEF, 0x10B00, 0x10B35, 0x10B40,
0x10B55, 0x10B58, 0x10B72, 0x10B78, 0x10B91, 0x10BA9, 0x10BAF, 0x10C00, 0x10C48, 0x10C80, 0x10CB2, 0x10CC0,
0x10CF2, 0x10CFA, 0x10D23, 0x10D30, 0x10D39, 0x10D40, 0x10D65, 0x10D6F, 0x10D85, 0x10E60, 0x10E7E, 0x10E80,
0x10EA9, 0x10EB0, 0x10EB1, 0x10EC2, 0x10EC4, 0x10F00, 0x10F27, 0x10F30, 0x10F45, 0x10F51, 0x10F54, 0x10F70,
0x10F81, 0x10FB0, 0x10FCB, 0x10FE0, 0x10FF6, 0x11003, 0x11037, 0x11052, 0x1106F, 0x11071, 0x11072, 0x11075,
0x11075, 0x11083, 0x110AF, 0x110D0, 0x110E8, 0x110F0, 0x110F9, 0x11103, 0x11126, 0x11136, 0x1113F, 0x11144,
0x11144, 0x11147, 0x11147, 0x11150, 0x11172, 0x11176, 0x11176, 0x11183, 0x111B2, 0x111C1, 0x111C4, 0x111D0,
0x111DA, 0x111DC, 0x111DC, 0x111E1, 0x111F4, 0x11200, 0x11211, 0x11213, 0x1122B, 0x1123F, 0x11240, 0x11280,
0x11286, 0x11288, 0x11288, 0x1128A, 0x1128D, 0x1128F, 0x1129D, 0x1129F, 0x112A8, 0x112B0, 0x112DE, 0x112F0,
0x112F9, 0x11305, 0x1130C, 0x1130F, 0x11310, 0x11313, 0x11328, 0x1132A, 0x11330, 0x11332, 0x11333, 0x11335,
0x11339, 0x1133D, 0x1133D, 0x11350, 0x11350, 0x1135D, 0x11361, 0x11380, 0x11389, 0x1138B, 0x1138B, 0x1138E,
0x1138E, 0x11390, 0x113B5, 0x113B7, 0x113B7, 0x113D1, 0x113D1, 0x113D3, 0x113D3, 0x11400, 0x11434, 0x11447,
0x1144A, 0x11450, 0x11459, 0x1145F, 0x11461, 0x11480, 0x114AF, 0x114C4, 0x114C5, 0x114C7, 0x114C7, 0x114D0,
0x114D9, 0x11580, 0x115AE, 0x115D8, 0x115DB, 0x11600, 0x1162F, 0x11644, 0x11644, 0x11650, 0x11659, 0x11680,
0x116AA, 0x116B8, 0x116B8, 0x116C0, 0x116C9, 0x116D0, 0x116E3, 0x11700, 0x1171A, 0x11730, 0x1173B, 0x11740,
0x11746, 0x11800, 0x1182B, 0x118A0, 0x118F2, 0x118FF, 0x11906, 0x11909, 0x11909, 0x1190C, 0x11913, 0x11915,
0x11916, 0x11918, 0x1192F, 0x1193F, 0x1193F, 0x11941, 0x11941, 0x11950, 0x11959, 0x119A0, 0x119A7, 0x119AA,
0x119D0, 0x119E1, 0x119E1, 0x119E3, 0x119E3, 0x11A00, 0x11A00, 0x11A0B, 0x11A32, 0x11A3A, 0x11A3A, 0x11A50,
0x11A50, 0x11A5C, 0x11A89, 0x11A9D, 0x11A9D, 0x11AB0, 0x11AF8, 0x11BC0, 0x11BE0, 0x11BF0, 0x11BF9, 0x11C00,
0x11C08, 0x11C0A, 0x11C2E, 0x11C40, 0x11C40, 0x11C50, 0x11C6C, 0x11C72, 0x11C8F, 0x11D00, 0x11D06, 0x11D08,
0x11D09, 0x11D0B, 0x11D30, 0x11D46, 0x11D46, 0x11D50, 0x11D59, 0x11D60, 0x11D65, 0x11D67, 0x11D68, 0x11D6A,
0x11D89, 0x11D98, 0x11D98, 0x11DA0, 0x11DA9, 0x11EE0, 0x11EF2, 0x11F02, 0x11F02, 0x11F04, 0x11F10, 0x11F12,
0x11F33, 0x11F50, 0x11F59, 0x11FB0, 0x11FB0, 0x11FC0, 0x11FD4, 0x12000, 0x12399, 0x12400, 0x1246E, 0x12480,
0x12543, 0x12F90, 0x12FF0, 0x13000, 0x1342F, 0x13441, 0x13446, 0x13460, 0x143FA, 0x14400, 0x14646, 0x16100,
0x1611D, 0x16130, 0x16139, 0x16800, 0x16A38, 0x16A40, 0x16A5E, 0x16A60, 0x16A69, 0x16A70, 0x16ABE, 0x16AC0,
0x16AC9, 0x16AD0, 0x16AED, 0x16B00, 0x16B2F, 0x16B40, 0x16B43, 0x16B50, 0x16B59, 0x16B5B, 0x16B61, 0x16B63,
0x16B77, 0x16B7D, 0x16B8F, 0x16D40, 0x16D6C, 0x16D70, 0x16D79, 0x16E40, 0x16E96, 0x16F00, 0x16F4A, 0x16F50,
0x16F50, 0x16F93, 0x16F9F, 0x16FE0, 0x16FE1, 0x16FE3, 0x16FE3, 0x17000, 0x187F7, 0x18800, 0x18CD5, 0x18CFF,
0x18D08, 0x1AFF0, 0x1AFF3, 0x1AFF5, 0x1AFFB, 0x1AFFD, 0x1AFFE, 0x1B000, 0x1B122, 0x1B132, 0x1B132, 0x1B150,
0x1B152, 0x1B155, 0x1B155, 0x1B164, 0x1B167, 0x1B170, 0x1B2FB, 0x1BC00, 0x1BC6A, 0x1BC70, 0x1BC7C, 0x1BC80,
0x1BC88, 0x1BC90, 0x1BC99, 0x1CCF0, 0x1CCF9, 0x1D2C0, 0x1D2D3, 0x1D2E0, 0x1D2F3, 0x1D360, 0x1D378, 0x1D400,
0x1D454, 0x1D456, 0x1D49C, 0x1D49E, 0x1D49F, 0x1D4A2, 0x1D4A2, 0x1D4A5, 0x1D4A6, 0x1D4A9, 0x1D4AC, 0x1D4AE,
0x1D4B9, 0x1D4BB, 0x1D4BB, 0x1D4BD, 0x1D4C3, 0x1D4C5, 0x1D505, 0x1D507, 0x1D50A, 0x1D50D, 0x1D514, 0x1D516,
0x1D51C, 0x1D51E, 0x1D539, 0x1D53B, 0x1D53E, 0x1D540, 0x1D544, 0x1D546, 0x1D546, 0x1D54A, 0x1D550, 0x1D552,
0x1D6A5, 0x1D6A8, 0x1D6C0, 0x1D6C2, 0x1D6DA, 0x1D6DC, 0x1D6FA, 0x1D6FC, 0x1D714, 0x1D716, 0x1D734, 0x1D736,
0x1D74E, 0x1D750, 0x1D76E, 0x1D770, 0x1D788, 0x1D78A, 0x1D7A8, 0x1D7AA, 0x1D7C2, 0x1D7C4, 0x1D7CB, 0x1D7CE,
0x1D7FF, 0x1DF00, 0x1DF1E, 0x1DF25, 0x1DF2A, 0x1E030, 0x1E06D, 0x1E100, 0x1E12C, 0x1E137, 0x1E13D, 0x1E140,
0x1E149, 0x1E14E, 0x1E14E, 0x1E290, 0x1E2AD, 0x1E2C0, 0x1E2EB, 0x1E2F0, 0x1E2F9, 0x1E4D0, 0x1E4EB, 0x1E4F0,
0x1E4F9, 0x1E5D0, 0x1E5ED, 0x1E5F0, 0x1E5FA, 0x1E7E0, 0x1E7E6, 0x1E7E8, 0x1E7EB, 0x1E7ED, 0x1E7EE, 0x1E7F0,
0x1E7FE, 0x1E800, 0x1E8C4, 0x1E8C7, 0x1E8CF, 0x1E900, 0x1E943, 0x1E94B, 0x1E94B, 0x1E950, 0x1E959, 0x1EC71,
0x1ECAB, 0x1ECAD, 0x1ECAF, 0x1ECB1, 0x1ECB4, 0x1ED01, 0x1ED2D, 0x1ED2F, 0x1ED3D, 0x1EE00, 0x1EE03, 0x1EE05,
0x1EE1F, 0x1EE21, 0x1EE22, 0x1EE24, 0x1EE24, 0x1EE27, 0x1EE27, 0x1EE29, 0x1EE32, 0x1EE34, 0x1EE37, 0x1EE39,
0x1EE39, 0x1EE3B, 0x1EE3B, 0x1EE42, 0x1EE42, 0x1EE47, 0x1EE47, 0x1EE49, 0x1EE49, 0x1EE4B, 0x1EE4B, 0x1EE4D,
0x1EE4F, 0x1EE51, 0x1EE52, 0x1EE54, 0x1EE54, 0x1EE57, 0x1EE57, 0x1EE59, 0x1EE59, 0x1EE5B, 0x1EE5B, 0x1EE5D,
0x1EE5D, 0x1EE5F, 0x1EE5F, 0x1EE61, 0x1EE62, 0x1EE64, 0x1EE64, 0x1EE67, 0x1EE6A, 0x1EE6C, 0x1EE72, 0x1EE74,
0x1EE77, 0x1EE79, 0x1EE7C, 0x1EE7E, 0x1EE7E, 0x1EE80, 0x1EE89, 0x1EE8B, 0x1EE9B, 0x1EEA1, 0x1EEA3, 0x1EEA5,
0x1EEA9, 0x1EEAB, 0x1EEBB, 0x1F100, 0x1F10C, 0x1FBF0, 0x1FBF9, 0x20000, 0x2A6DF, 0x2A700, 0x2B739, 0x2B740,
0x2B81D, 0x2B820, 0x2CEA1, 0x2CEB0, 0x2EBE0, 0x2EBF0, 0x2EE5D, 0x2F800, 0x2FA1D, 0x30000, 0x3134A, 0x31350,
0x323AF,
)
DEFAULT_IGNORABLE: tuple[int, ...] = (
0xAD, 0xAD, 0x34F, 0x34F, 0x61C, 0x61C, 0x115F, 0x1160, 0x17B4, 0x17B5, 0x180B, 0x180F, 0x200B, 0x200F, 0x202A,
0x202E, 0x2060, 0x206F, 0x3164, 0x3164, 0xFE00, 0xFE0F, 0xFEFF, 0xFEFF, 0xFFA0, 0xFFA0, 0xFFF0, 0xFFF8, 0x1BCA0,
0x1BCA3, 0x1D173, 0x1D17A, 0xE0000, 0xE0FFF,
)

View file

@ -13,7 +13,8 @@ from datetime import UTC, datetime
from pathlib import Path from pathlib import Path
from typing import Callable from typing import Callable
from uuid import uuid4 from uuid import uuid4
from cultcache_py import CultCache, define_database_entry_type import msgpack
from cultcache_py import CultCache, SchemaConflictError, define_database_entry_type
from cultnet_py.benchmark import run_benchmark from cultnet_py.benchmark import run_benchmark
from cultnet_py.compare_csharp import ( from cultnet_py.compare_csharp import (
DEFAULT_PARITY_THRESHOLD, DEFAULT_PARITY_THRESHOLD,
@ -1566,6 +1567,51 @@ class CultNetTests(unittest.TestCase):
self.assertEqual(delete_change.schema_id, "schema-note") self.assertEqual(delete_change.schema_id, "schema-note")
self.assertEqual(delete_change.to_wire()["recordKey"], "note:1") self.assertEqual(delete_change.to_wire()["recordKey"], "note:1")
def test_cultnet_database_change_commit_fields(self) -> None:
base = {
"schemaVersion": "cultnet.database_change_raw.v0",
"messageId": "change-commit",
"subscriptionId": "sub-1",
"changeKind": "removed",
"schemaId": "schema-note",
"recordKey": "note:1",
}
grouped = CultNetDatabaseChange.from_wire({**base, "commitId": "commit-7", "commitCount": 3})
self.assertEqual((grouped.commit_id, grouped.commit_count), ("commit-7", 3))
self.assertEqual(grouped.to_wire()["commitId"], "commit-7")
self.assertEqual(grouped.to_wire()["commitCount"], 3)
self.assertEqual(CultNetDatabaseChange.from_wire(grouped.to_wire()), grouped)
for absent in (base, {**base, "commitId": None, "commitCount": None}): # C# writes nil outside a commit
change = CultNetDatabaseChange.from_wire(absent)
self.assertIsNone(change.commit_id)
self.assertIsNone(change.commit_count)
self.assertNotIn("commitId", change.to_wire())
self.assertNotIn("commitCount", change.to_wire())
for field, bad in (
("commitId", ""),
("commitId", 7),
("commitCount", 1),
("commitCount", 0),
("commitCount", True),
("commitCount", 2.0),
("commitCount", "3"),
):
with self.subTest(field=field, bad=bad), self.assertRaisesRegex(ValueError, field):
CultNetDatabaseChange.from_wire({**base, field: bad})
def test_a_two_change_commit_written_by_csharp_parses(self) -> None:
# tests/vectors/database-change-commit/two-change-commit.msgpack: the two change messages of one commit, written by the
# C# serializer (CultNetReplicaCommitTests), so a rename or retyping of commitId/commitCount fails here.
vector = Path(__file__).resolve().parents[3] / "tests" / "vectors" / "database-change-commit" / "two-change-commit.msgpack"
messages = list(msgpack.Unpacker(io.BytesIO(vector.read_bytes()), raw=False))
self.assertEqual(len(messages), 2)
changes = [CultNetDatabaseChange.from_wire(message) for message in messages]
self.assertEqual([change.commit_count for change in changes], [2, 2])
self.assertEqual({change.commit_id for change in changes}, {"9f1c0f0e4b7a4c1f9d2e"})
self.assertEqual([change.to_wire()["commitCount"] for change in changes], [2, 2])
def test_cultnet_raw_put_helper_carries_message_id(self) -> None: def test_cultnet_raw_put_helper_carries_message_id(self) -> None:
put = document_put_raw( put = document_put_raw(
message_id="put-1", message_id="put-1",
@ -1896,6 +1942,35 @@ class CultNetTests(unittest.TestCase):
self.assertEqual(raised.exception.response["messageId"], "subscription-error") self.assertEqual(raised.exception.response["messageId"], "subscription-error")
thread.join(2.0) thread.join(2.0)
def test_replication_refuses_a_remote_record_of_another_type_at_a_held_key_and_changes_nothing(self) -> None:
# variants:ruling:cache-refuses-retype: a cache holds one record per key whatever its type, so a peer that reuses a
# key across types is refused where the record is applied; replication does not catch it.
first = define_database_entry_type("replica.first", [("name", 0), ("category", 1), ("value", 2, 0)], cls=Item)
second = define_database_entry_type("replica.second", [("name", 0), ("category", 1), ("value", 2, 0)], cls=Item)
cache = CultCache()
cache.register_document_type(first)
cache.register_document_type(second)
cache.put(first, "shared", Item("held", "", 0))
def raw(document):
return {
"schemaId": document.catalog_entry().schema_id,
"recordKey": "shared",
"storedAt": "2026-06-13T00:00:00Z",
"payloadEncoding": "messagepack",
"payload": document.encode_payload(Item("remote", "", 0)),
}
with self.assertRaises(SchemaConflictError):
apply_raw_document_record(cache, schema_document_map([first, second]), raw(second))
snapshot = {"schemaVersion": "cultnet.snapshot_response_raw.v0", "messageId": "s", "documents": [raw(second)]}
with self.assertRaises(SchemaConflictError):
apply_raw_snapshot(cache, [first, second], snapshot)
self.assertEqual(cache.get_required(first, "shared").name, "held")
self.assertIsNone(cache.get(second, "shared"))
apply_raw_document_record(cache, schema_document_map([first, second]), raw(first)) # the same type still replaces
self.assertEqual(cache.get_required(first, "shared").name, "remote")
def test_cultnet_replication_helpers_apply_raw_snapshot_and_shard_log(self) -> None: def test_cultnet_replication_helpers_apply_raw_snapshot_and_shard_log(self) -> None:
document = define_database_entry_type( document = define_database_entry_type(
"replica.item", "replica.item",
@ -2269,9 +2344,12 @@ class CultNetTests(unittest.TestCase):
change_schema = json.loads(by_version["cultnet.database_change_raw.v0"]["schemaJson"]) change_schema = json.loads(by_version["cultnet.database_change_raw.v0"]["schemaJson"])
self.assertEqual(change_schema["properties"]["changeKind"]["enum"], ["added", "updated", "removed"]) self.assertEqual(change_schema["properties"]["changeKind"]["enum"], ["added", "updated", "removed"])
self.assertIn("document", change_schema["properties"]) # The C# writer sets every key, nil where unset (a removed change's document, an added change's recordKey and schemaId).
self.assertIn("schemaId", change_schema["properties"]) self.assertEqual(change_schema["properties"]["document"]["oneOf"][0], {"type": "null"})
self.assertIn("recordKey", change_schema["properties"]) self.assertEqual(change_schema["properties"]["schemaId"], {"type": ["string", "null"], "minLength": 1})
self.assertEqual(change_schema["properties"]["recordKey"], {"type": ["string", "null"], "minLength": 1})
self.assertEqual(change_schema["properties"]["commitId"], {"type": ["string", "null"], "minLength": 1})
self.assertEqual(change_schema["properties"]["commitCount"], {"type": ["integer", "null"], "minimum": 2})
log_schema = json.loads(by_version["cultnet.shard_log_response.v0"]["schemaJson"]) log_schema = json.loads(by_version["cultnet.shard_log_response.v0"]["schemaJson"])
self.assertIn("entries", log_schema["required"]) self.assertIn("entries", log_schema["required"])

View file

@ -6,7 +6,7 @@ from pathlib import Path
import msgpack import msgpack
from cultnet_py import CULTNET_ERROR_CODES, error_message, parse_message, unsupported_schema_version_error from cultnet_py import CULTNET_ERROR_CODES, error_message, message_invalid_error, parse_message, unsupported_schema_version_error
# contracts/cultnet/interop/error-vectors.json: bytes captured from MessagePack C# 3.1.7 and pinned # contracts/cultnet/interop/error-vectors.json: bytes captured from MessagePack C# 3.1.7 and pinned
# by cultnet-rs/tests/error_message.rs (Rust encodes to exactly these bytes). # by cultnet-rs/tests/error_message.rs (Rust encodes to exactly these bytes).
@ -51,6 +51,10 @@ class CultNetErrorMessageTests(unittest.TestCase):
) )
self.assertEqual(selection.to_bytes(), by_name["selection_invalid_field_value"]) self.assertEqual(selection.to_bytes(), by_name["selection_invalid_field_value"])
self.assertEqual(unsupported_schema_version_error("cultnet.hello.v1").to_bytes(), by_name["unsupported_schema_version"]) self.assertEqual(unsupported_schema_version_error("cultnet.hello.v1").to_bytes(), by_name["unsupported_schema_version"])
self.assertEqual(
message_invalid_error("document", "Raw put messages must contain a document map.").to_bytes(),
by_name["message_invalid"],
)
def test_builder_writes_nil_keys_and_accepts_a_null_code(self) -> None: def test_builder_writes_nil_keys_and_accepts_a_null_code(self) -> None:
wire = error_message("boom").to_wire() wire = error_message("boom").to_wire()

View file

@ -0,0 +1,310 @@
from __future__ import annotations
import json
import unittest
from pathlib import Path
from typing import Any
import msgpack
from cultnet_py import (
CULTNET_ERROR_CODES,
CULTNET_ERROR_V1_CODES,
CULTNET_WRITE_RECEIPT_MESSAGE_VERSIONS,
error_message,
hello,
hello_advertises_write_receipts,
parse_message,
write_content_hash,
)
# Decoder-side conformance with contracts/cultnet/interop/write-receipt-vectors.json: `contentHash`, `wire` and
# `schemaInvalid`. `dedupe` and `blankFields` are server obligations and `scenarios` is the client write tracker; `counts` is the file's own tally;
# this runtime has neither a receipt server nor a write client, so it replays none of them.
_INTEROP = Path(__file__).resolve().parents[3] / "contracts/cultnet/interop"
_VECTORS = json.loads((_INTEROP / "write-receipt-vectors.json").read_text(encoding="utf-8"))
_ERROR_V1_SCHEMA = json.loads((_INTEROP / "cultnet.error.v1.schema.json").read_text(encoding="utf-8"))
def _pack(wire: dict[str, Any]) -> bytes:
return msgpack.packb(wire, use_bin_type=True)
def _admitted(**override: Any) -> dict[str, Any]:
return {
"schemaVersion": "cultnet.write_admitted.v0",
"messageId": "put-1",
"operation": "put",
"schemaId": "note.v0",
"recordKey": "note:1",
**override,
}
def _refused(**override: Any) -> dict[str, Any]:
return {
"schemaVersion": "cultnet.error.v1",
"error": "write_not_authorized: no",
"routingHint": None,
"code": "write_not_authorized",
"details": None,
"messageId": "put-1",
**override,
}
class WriteReceiptVectorTests(unittest.TestCase):
def test_vector_sections_are_the_ones_this_file_accounts_for(self) -> None:
self.assertEqual(
set(_VECTORS) - {"provenance", "specification"},
{"constants", "contentHash", "dedupe", "wire", "blankFields", "schemaInvalid", "scenarios", "counts", "admittedMessageId", "admittedMessageIdDuplicates"},
)
self.assertEqual(len(_VECTORS["contentHash"]["cases"]), 4)
self.assertEqual(len(_VECTORS["wire"]), 7)
self.assertEqual(len(_VECTORS["schemaInvalid"]), 7)
# The file declares how many of each it holds; a loader that drops entries disagrees with it.
counts = _VECTORS["counts"]
self.assertEqual(counts["wire"], len(_VECTORS["wire"]))
self.assertEqual(counts["schemaInvalid"], len(_VECTORS["schemaInvalid"]))
self.assertEqual(counts["contentHashCases"], len(_VECTORS["contentHash"]["cases"]))
self.assertEqual(counts["scenarioCases"], len(_VECTORS["scenarios"]["cases"]))
self.assertEqual(counts["admittedMessageIdCases"], len(_VECTORS["admittedMessageId"]["cases"]))
def test_admitted_message_id_vectors_are_refused_or_accepted_raw(self) -> None:
# `admittedMessageId`: a write_admitted messageId needs a letter or number and no Default_Ignorable code point,
# compared raw (never trimmed). A lone surrogate is not valid UTF-8, so its vector carries the whole frame.
cases = _VECTORS["admittedMessageId"]["cases"]
failures = []
for case in cases:
raw = (
bytes.fromhex(case["wireHex"])
if "wireHex" in case
else _pack(_admitted(messageId="".join(chr(int(cp, 16)) for cp in case["codePoints"]) * case.get("repeat", 1)))
)
try:
message = parse_message(raw)
except Exception: # noqa: BLE001 - any refusal counts
refused = True
else:
refused = False
wanted = "".join(chr(int(cp, 16)) for cp in case["codePoints"]) * case.get("repeat", 1)
if message.body["messageId"] != wanted:
failures.append(f"{case['name']}: id not echoed raw")
if refused != (case["expect"] == "refuse"):
failures.append(f"{case['name']}: expected {case['expect']}")
self.assertEqual(failures, [])
def test_a_repeated_message_id_key_follows_the_last_key_wins_policy(self) -> None:
# Python decodes the frame to a dict, so it cannot see a duplicate: it judges the last key (admittedMessageIdDuplicates).
section = _VECTORS["admittedMessageIdDuplicates"]
self.assertEqual(_VECTORS["counts"]["admittedMessageIdDuplicateCases"], len(section["cases"]))
for case in section["cases"]:
raw = bytes.fromhex(case["wireHex"])
if case["lastKeyWins"] == "refuse":
with self.assertRaises(ValueError, msg=case["name"]):
parse_message(raw)
else:
self.assertEqual(parse_message(raw).body["messageId"], case["lastKeyWinsEcho"], case["name"])
def test_every_range_of_the_unicode_tables_is_pinned_at_both_ends(self) -> None:
# write-admitted-message-id-ranges.json is the generated, independent copy: for each letter-or-number range its first and
# last point are accepted and the points either side refused; for each Default_Ignorable range "a" beside its first and
# last point is refused and beside the points either side accepted. Deleting or narrowing any embedded range fails here.
ranges = json.loads((_INTEROP / "write-admitted-message-id-ranges.json").read_text(encoding="utf-8"))
self.assertGreater(len(ranges["letterOrNumber"]), 700)
self.assertEqual(len(ranges["defaultIgnorable"]), 17)
def accepted(text: str) -> bool:
try:
parse_message(_pack(_admitted(messageId=text)))
return True
except ValueError:
return False
failures = []
for low, high in ranges["letterOrNumber"]:
for point, want in ((low, True), (high, True), (low - 1, False), (high + 1, False)):
if accepted(chr(point)) != want:
failures.append(f"U+{point:04X} alone should be {'accepted' if want else 'refused'}")
for low, high in ranges["defaultIgnorable"]:
for point, want in ((low, False), (high, False), (low - 1, True), (high + 1, True)):
if accepted("a" + chr(point)) != want:
failures.append(f"a + U+{point:04X} should be {'accepted' if want else 'refused'}")
self.assertEqual(failures, [])
def test_content_hash_matches_every_vector(self) -> None:
for case in _VECTORS["contentHash"]["cases"]:
payload = None if case["payloadHex"] is None else bytes.fromhex(case["payloadHex"])
digest = write_content_hash(case["operation"], case["schemaId"], case["recordKey"], payload)
self.assertEqual(digest.hex(), case["sha256"], case["name"])
def test_wire_vectors_decode_to_their_fields_in_order_and_reencode_byte_identical(self) -> None:
for vector in _VECTORS["wire"]:
raw = bytes.fromhex(vector["messagePackHex"])
message = parse_message(raw)
self.assertEqual(message.to_wire(), vector["message"], vector["name"])
self.assertEqual(list(message.to_wire()), list(vector["message"]), vector["name"])
self.assertEqual(message.to_bytes(), raw, vector["name"])
def test_schema_invalid_vectors_are_refused(self) -> None:
for vector in _VECTORS["schemaInvalid"]:
with self.assertRaises(ValueError, msg=vector["name"]):
parse_message(bytes.fromhex(vector["messagePackHex"]))
def test_a_payload_that_is_not_bytes_changes_the_hash(self) -> None:
# The payload is a bin, not an str and not a re-parsed document.
as_bin = write_content_hash("put", "note.v0", "note:1", b"\x81\xa2id\xa11")
self.assertNotEqual(as_bin, write_content_hash("put", "note.v0", "note:1", None))
self.assertEqual(len(as_bin), 32)
class ErrorV1Tests(unittest.TestCase):
def test_code_set_equals_the_schema_enum_and_v0_is_unchanged(self) -> None:
schema_codes = [c for c in _ERROR_V1_SCHEMA["properties"]["code"]["enum"] if c is not None]
self.assertEqual(sorted(CULTNET_ERROR_V1_CODES), sorted(schema_codes))
self.assertNotIn("limit_reached", CULTNET_ERROR_CODES)
self.assertEqual(CULTNET_ERROR_V1_CODES[-1], "limit_reached")
def test_every_v1_code_decodes_and_an_unknown_one_does_not(self) -> None:
for code in CULTNET_ERROR_V1_CODES:
self.assertEqual(parse_message(_pack(_refused(code=code))).body["code"], code)
parse_message(_pack(_refused(code=None)))
with self.assertRaises(ValueError):
parse_message(_pack(_refused(code="invented")))
with self.assertRaises(ValueError):
parse_message(_pack(_refused(code=7)))
def test_limit_reached_is_v1_only(self) -> None:
v0 = {k: v for k, v in _refused(code="limit_reached", schemaVersion="cultnet.error.v0").items() if k != "messageId"}
with self.assertRaises(ValueError):
parse_message(_pack(v0))
with self.assertRaises(ValueError):
error_message("x", code="limit_reached")
def test_message_id_is_required_and_is_a_nonempty_string_or_null(self) -> None:
self.assertEqual(parse_message(_pack(_refused(messageId="w-é 1"))).body["messageId"], "w-é 1")
self.assertIsNone(parse_message(_pack(_refused(messageId=None))).body["messageId"])
omitted = {k: v for k, v in _refused().items() if k != "messageId"}
with self.assertRaises(ValueError):
parse_message(_pack(omitted))
for bad in ("", 5, b"put-1", ["put-1"], True):
with self.assertRaises(ValueError, msg=repr(bad)):
parse_message(_pack(_refused(messageId=bad)))
def test_v0_stays_closed_to_message_id(self) -> None:
v0 = _refused(schemaVersion="cultnet.error.v0", code="message_invalid")
with self.assertRaises(ValueError):
parse_message(_pack(v0))
del v0["messageId"]
self.assertEqual(parse_message(_pack(v0)).schema_version, "cultnet.error.v0")
def test_v1_is_closed_to_unknown_keys_and_keeps_the_shape_rules_of_v0(self) -> None:
for bad in (
_refused(extra=1),
_refused(error=""),
_refused(error=3),
_refused(routingHint={"shard": None}),
_refused(routingHint={"shard": 1, "reason": "r"}),
_refused(routingHint={"shard": None, "reason": 1}),
_refused(details={"field": None, "value": None, "asOf": None}),
_refused(details={"field": 1, "value": None, "asOf": None, "current": None}),
_refused(details={"field": None, "value": None, "asOf": -1, "current": None}),
_refused(details={"field": None, "value": None, "asOf": None, "current": True}),
):
with self.assertRaises(ValueError, msg=str(bad)):
parse_message(_pack(bad))
for good in (
_refused(routingHint={"shard": None, "reason": "r"}),
_refused(details={"field": "messageId", "value": "x", "asOf": 1, "current": 2}),
):
parse_message(_pack(good))
def test_the_refusal_reason_names_its_version(self) -> None:
with self.assertRaisesRegex(ValueError, "cultnet.error.v1 code"):
parse_message(_pack(_refused(code="invented")))
with self.assertRaisesRegex(ValueError, r"routingHint must be null or \{shard, reason\}"):
parse_message(_pack(_refused(routingHint={"shard": None})))
class RefusalTextTests(unittest.TestCase):
"""A refusal names its frame version and the rule it broke; a caller reads this text in a log."""
def assertRefused(self, wire: dict[str, Any], pattern: str) -> None:
with self.assertRaisesRegex(ValueError, pattern):
parse_message(_pack(wire))
def test_error_frames_name_their_own_version(self) -> None:
v0 = {k: v for k, v in _refused(schemaVersion="cultnet.error.v0").items() if k != "messageId"}
self.assertRefused({**v0, "code": "nope"}, r"^cultnet\.error\.v0 code")
self.assertRefused({**v0, "extra": 1}, r"^cultnet\.error\.v0 must carry exactly")
self.assertRefused(_refused(extra=1), r"^cultnet\.error\.v1 must carry exactly")
self.assertRefused(_refused(error=""), r"^cultnet\.error\.v1 error must be")
self.assertRefused(_refused(messageId=""), r"^cultnet\.error\.v1 messageId must be")
self.assertRefused(_refused(details={"field": None}), r"^cultnet\.error\.v1 details must be null or carry exactly")
self.assertRefused(
_refused(details={"field": 1, "value": None, "asOf": None, "current": None}), r"^cultnet\.error\.v1 details\.field must be"
)
self.assertRefused(
_refused(details={"field": None, "value": None, "asOf": -1, "current": None}), r"^cultnet\.error\.v1 details\.asOf must be"
)
with self.assertRaisesRegex(ValueError, r"^cultnet\.error\.v0 code"):
error_message("x", code="bogus")
def test_write_admitted_names_the_broken_rule(self) -> None:
self.assertRefused(_admitted(extra=1), r"^cultnet\.write_admitted\.v0 must carry exactly")
self.assertRefused(_admitted(recordKey=""), r"^cultnet\.write_admitted\.v0 recordKey must be a non-empty string")
self.assertRefused(_admitted(operation="upsert"), r"^cultnet\.write_admitted\.v0 operation must be put or delete$")
class WriteAdmittedTests(unittest.TestCase):
def test_both_operations_decode(self) -> None:
for operation in ("put", "delete"):
self.assertEqual(parse_message(_pack(_admitted(operation=operation))).body["operation"], operation)
def test_every_key_is_required_and_no_other_is_allowed(self) -> None:
for key in ("messageId", "operation", "schemaId", "recordKey"):
with self.assertRaises(ValueError, msg=key):
parse_message(_pack({k: v for k, v in _admitted().items() if k != key}))
for extra in ("error", "code", "details"):
with self.assertRaises(ValueError, msg=extra):
parse_message(_pack(_admitted(**{extra: "x"})))
def test_each_string_is_nonempty_and_a_string(self) -> None:
for key in ("messageId", "schemaId", "recordKey"):
for bad in ("", None, 3, b"x"):
with self.assertRaises(ValueError, msg=f"{key}={bad!r}"):
parse_message(_pack(_admitted(**{key: bad})))
def test_operation_is_put_or_delete(self) -> None:
for bad in ("upsert", "PUT", "", None, 1):
with self.assertRaises(ValueError, msg=repr(bad)):
parse_message(_pack(_admitted(operation=bad)))
def test_message_id_is_echoed_byte_for_byte(self) -> None:
self.assertEqual(parse_message(_pack(_admitted(messageId=" w é 1 "))).body["messageId"], " w é 1 ")
class HelloCapabilityTests(unittest.TestCase):
def test_capability_needs_both_frames(self) -> None:
both = list(CULTNET_WRITE_RECEIPT_MESSAGE_VERSIONS)
self.assertEqual(both, ["cultnet.write_admitted.v0", "cultnet.error.v1"])
self.assertTrue(hello_advertises_write_receipts(hello(runtime_id="r", supported_message_versions=both)))
self.assertTrue(hello_advertises_write_receipts(hello(runtime_id="r", supported_message_versions=[*both, "x"])))
self.assertFalse(hello_advertises_write_receipts(hello(runtime_id="r", supported_message_versions=both[:1])))
self.assertFalse(hello_advertises_write_receipts(hello(runtime_id="r", supported_message_versions=both[1:])))
self.assertFalse(hello_advertises_write_receipts(hello(runtime_id="r", supported_message_versions=[])))
self.assertFalse(hello_advertises_write_receipts(hello(runtime_id="r")))
def test_a_non_list_listing_is_not_a_capability(self) -> None:
wire = hello(runtime_id="r").to_wire()
wire["supportedMessageVersions"] = "cultnet.write_admitted.v0 cultnet.error.v1"
self.assertFalse(hello_advertises_write_receipts(parse_message(_pack(wire))))
def test_hello_writes_the_listing_only_when_given_and_round_trips(self) -> None:
self.assertNotIn("supportedMessageVersions", hello(runtime_id="r").body)
listed = hello(runtime_id="r", supported_message_versions=list(CULTNET_WRITE_RECEIPT_MESSAGE_VERSIONS))
self.assertEqual(parse_message(listed.to_bytes()), listed)
if __name__ == "__main__":
unittest.main()

Some files were not shown because too many files have changed in this diff Show more