Commit graph CultLib/packages/cultcache-rs
Author SHA1 Message Date
Meta Vens
9c3c92634a Require an atomic push_all from every Rust backing store
The default CacheBackingStore::push_all deleted every row and then pushed
one by one, so a failing push left the store emptied or partial while
put_prepared_batch claimed all-or-nothing. The default is deleted:
push_all is a required method documented as one atomic replace, and a
compile_fail doctest pins that a store without it does not compile.
Every in-tree implementor (single-file, both redb stores, the test store,
Epiphany's runtime store) already supplies its own.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 12:28:07 +02:00
Meta Vens
4ed9871252 Make Rust CultCache batch and SoA loads validate before admitting
put_prepared_batch refuses an empty key or stored_at before any store is
touched. Its one-home check covers the staged records only, and the home store
receives only its own records (those this cache holds for it plus the staged
ones), so a single-home batch is no longer refused because other homes hold
records.

load_soa builds the next view in a local map, refusing an unregistered type or,
once a store is attached, a type with no home store; the view is replaced in one
step, so a refused load leaves the prior view intact.

load_envelope is deleted: it had no callers and admitted records into the view
without a store.

Tests: refuse-at-load and pull start from a non-empty cache and assert prior
state intact; atomic load_soa; single-home batch beside other homes' records;
batch with empty key or stored_at.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 11:57:44 +02:00
Meta Vens
03ff299233 Keep cultcache-rs record homes stable and refuse strays at load
Conform cultcache-rs to the store-composition contract, following C#:

- add_backing_store returns Err, attaching nothing, when the new store
  would move the home of any entry type the cache already holds.
- pull_all_backing_stores loads every store into a staged image and
  replaces the cache's entries only when all loaded. An entry delivered by
  a store that is not its home returns Err naming the key and both
  stores; nothing is admitted and no store is written.
- A cache with zero stores is in memory: put, put_envelope,
  put_raw_envelope, put_prepared_batch and delete no longer error. Once
  any store is attached, a type with no home still errors.

Tests for each rule fail on 6bb41f3.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 00:50:48 +02:00
Meta Vens
6bb41f39ea Give every cultcache-rs entry type one home store
Cut 5 (Rust). The four mirror loops in put, put_envelope, put_raw_envelope and
delete are deleted, and so is put_prepared_batch's route.len() != 1 branch.
resolve_route_index returns the single home store (the store that claims the
type, else the generic store) and replaces resolve_route_indices.

add_backing_store and add_generic_backing_store return Result and refuse a
second generic store or a type already claimed by another store, attaching
nothing when they refuse. Callers take the Result: the crate's tests, the
cultcache_interop example, cultnet-rs's tests and interop peer example, and
CultMeshNode::create_node in cultmesh-rs.

Tests: second_generic_store_is_rejected, type_claimed_twice_is_rejected;
type_specific_store_routes_before_generic_store stays, now attaching the routed
store before the generic one. No bytes change.

The store-composition contract's runtime table now records Rust, TypeScript
and Python as conforming to one home store per type, and the README routing
section drops its non-conformance note.

Version 0.1.0 -> 0.2.0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 00:38:41 +02:00
Meta Vens
486fe560af Correct stale store-routing, SoA, and generator docs
The README and the Rust README described mirrors as C# behavior; the parity
scope claimed the C# SoA table; the generator README described DatabaseEntry
formatters that no longer exist.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 16:01:32 +02:00
Meta Vens
c2a9a6e526 Windows atomic replace waits out a transient reader
Some checks failed
Publish packages / @gamecult/cultcache-ts to npm (push) Has been skipped
Publish packages / cultcache-py to PyPI (push) Has been cancelled
Publish packages / cultmesh-py to PyPI (push) Has been cancelled
Publish packages / cultnet-py to PyPI (push) Has been cancelled
A concurrent reader of the store's destination file (another process
opening it, an indexer, an antivirus scan) makes MoveFileExW fail with a
sharing or access error for as long as it holds the handle. That is
transient by construction; treating the first refusal as the outcome
killed Muninn serve on 2026-09-11 the moment a status CLI overlapped a
write. The replace retries such errors for up to half a second. The test
holds the destination open from another thread and expects the replace to
outlast it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-11 02:44:17 +02:00
Meta Vens
8cb3b7282b Publish the non-Rust packages and make the licensing uniform
Rust consumers pin a CultLib revision through cargo git dependencies, which is
how Ghostlight, Epiphany, CodexConnector and Odin already consume the core
stack. npm and pip cannot take a package from a subdirectory of a git repo at
all, which is why the TypeScript and Python consumers vendored forks instead:
there was nothing else to depend on. This gives them a versioned artifact.

Tagging cultcache-ts-v* or cultcache-py-v* builds, tests and publishes. The tag
version must match the manifest version or the workflow refuses rather than
shipping a mislabelled artifact. PyPI uses trusted publishing, so no long-lived
token is stored; npm needs an NPM_TOKEN secret.

cultcache-ts is now @gamecult/cultcache-ts. The scope is not cosmetic: it makes
the package unambiguously org-owned on a public registry, and it is what lets
CultLib hold the name rather than racing for an unscoped one. CultLib's own
cultmesh-ts and cultnet-ts and their imports move with it; all three packages
typecheck and cultcache-ts passes 12 of 13 tests, the exception being the
cross-runtime store test that shells out to Python, which cannot run on this
machine because its Python has no msgpack.

Licensing was inconsistent with the repo. cultcache-py declared itself "Private
GameCult infrastructure" and cultcache-rs and cultmesh-rs declared nothing,
inside a repo whose LICENSE is MIT and whose every other package says MIT. All
three now say MIT. That was load-bearing here: a package that calls itself
private is not one to push to a public registry.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 04:41:44 +02:00
Meta Vens
8f29ee568d Add the per-entry snapshot compare-exchange Odin consumes
A first pass at Odin's vendored CultCache compared only its uncommitted diff
against CultLib and concluded nothing else was missing. That was wrong: a
whole-file symbol comparison found CultCacheExpectedEnvelope and a per-entry
compare_exchange that CultLib lacks and Odin's crates import. Comparing a diff
tells you what changed recently, not what the fork uniquely holds.

Unlike the snapshot form this constrains only the identities it names, so an
unrelated concurrent write does not lose the race. Expecting `None` makes it a
create-once claim, which is what an admitted-generation owner needs.

The expectation-identity guard is carried over as written even though it is
unreachable: a condition whose envelope names a different identity is already
rejected by the value comparison above it. Left in place rather than quietly
dropped, since it costs nothing and the lookup it depends on may change.

cultcache-rs: 53 tests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 03:32:07 +02:00
Meta Vens
bac5be88b9 Restore reliable expiry and snapshot compare-exchange to the Rust core
Two capabilities existed only in Odin's vendored forks, which is why the forks
could not be deleted. Both are ported here with tests so Odin can consume
CultLib instead.

cultnet-rs gains reliable send expiry. A reliable packet may now carry a
deadline; past it the packet is dropped instead of retransmitted, and the count
is reported through session and transport stats. Live media tolerates loss but
not lateness -- a frame retransmitted after its moment has passed is worse than
a missing one -- so the media channel takes a 75ms default while state channels
never expire. Handshake packets never expire either: a dropped connect is not a
late frame.

The deadline is fixed when a send is admitted, so time spent waiting in the
backlog spends it exactly as time spent awaiting an acknowledgement does. This
lineage has a queued_reliable backlog that Odin's does not, and a queued packet
promoted after its moment had passed would be as useless as a retransmitted
one. Expiry runs before the window is measured, so an expiring stream is not
refused for a backlog it no longer owns.

cultcache-rs gains compare_exchange_snapshot and a nonblocking
try_compare_exchange_snapshot. CultLib had neither, despite carrying the larger
body -- the two lineages had diverged in both directions. Contention is returned
as data rather than waited on, so the caller keeps retry and scheduling
authority. This is the primitive an admitted-generation CAS needs.

No wire format changes. Expiry is a sender-side policy and compare-exchange is
local storage, so the other four runtimes stay in agreement without matching
these refusals. They are still worth carrying eventually, and the parity ledger
already tracks that this kind of gap does not announce itself.

cultnet-rs: 106 tests. cultcache-rs: 51 tests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 03:21:01 +02:00
Meta Vens
eb74ead837 Restore cross-runtime schema recovery and prove staging cleanup
Taking the standalone body as the CultCache base dropped a0b31b8, and it was a
real regression rather than a missing convenience. A record whose catalog entry
is absent -- written by another runtime, or older than the catalog it was
stamped against -- made decode_store_snapshot fail, and one such record failed
the entire pull. Recovering the type from the payload is what "across runtimes"
meant in that commit's title.

Restored in full: the payload is a MessagePack array whose first field is the
schema version, so the schema name is recoverable from the record itself; the
registry now also indexes schema name to entry type and refuses a name claimed
by a second type; and pull resolves a persisted type that is either a
registered type or a recovered schema name. The stale schema id is preserved on
the envelope rather than rewritten.

743eaa5's durability half needed no code. The standalone lineage reached the
same place independently through crash-atomic single-file snapshots and
abandoned staging cleanup, and its cleanup carries the same protections: only a
candidate named for this exact store plus a canonical UUID is removed, and
anything that is not a regular file is skipped. Untested, though, which is how
a careful deletion rule quietly becomes a careless one. Now asserted: an
abandoned canonical candidate is removed while a non-UUID lookalike, a bare
.tmp suffix, and another store's candidate all survive, and the committed
snapshot still reloads.

48 tests pass. This clears the cultcache half of the retained debt from b672fb7.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 00:45:48 +02:00
Meta Vens
b672fb7a37 Return the Rust CultCache engine to CultLib
packages/cultcache-rs forked from GameCult/cultcache-rs at the 5c69bfc
monorepo import and the two lineages then diverged in both directions. The
standalone repo took 28 commits of engine work CultLib never saw: typed SoA
projection, atomic heterogeneous batch publication, single-entry and
conditional batch CAS, a transactional keyed redb backend with serialized
cross-process opens, lockless provider snapshot reads, RAII read-only shared
snapshots, snapshot-fenced replace/append, crash-atomic single-file snapshots,
compaction fences, portable and canonical byte fields, and unified retained
storage authority. That body is 3,714 lines with 48 tests against CultLib's
1,554 with 18.

CultLib is the canonical home because wire parity conformance lives here and
nothing outside it is tested for cross-runtime agreement. A runtime-specific
core-stack repo cannot hold the engine every other runtime must agree with.

The standalone body lands as the base rather than merging 28 commits backward.
It is already proven against a CultLib-derived cultnet-rs: Ghostlight vendors
exactly this pairing and compiles. Here, cultcache-rs passes 46 tests and the
cultnet-rs RUDP gate passes 20 with no source change.

Retained debt, not carried forward: a0b31b8 recovered schema-stamped records
missing catalog entries (SchemaStamped, infer_schema_name,
infer_schema_version_from_payload) and 743eaa5 hardened snapshot replacement
(SnapshotReplacementStage, CommittedSnapshotDurabilityUncertain, stale
candidate cleanup, with_exclusive_lock_using_unlock). Their tests were
white-box against internals this body does not have. The standalone
independently made single-file snapshots crash atomic and cleaned abandoned
staging files, so the durability guarantee is probably held under other names
-- probably is not proven. Re-establish both as black-box tests against the
new API before the standalone repo is retired.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-03 22:56:59 +02:00
Meta Vens
743eaa5507 Harden Rust CultCache snapshot replacement 2026-07-19 00:49:52 +01:00
Meta Vens
a0b31b8f2a Recover schema-stamped cache records across runtimes 2026-06-27 14:34:53 +01:00
Meta Vens
5c69bfc581 Bring language packages into CultLib monorepo 2026-05-30 00:39:39 +01:00