From 4111fbb46849e10640d5c32f01afcb1cd1ba78b6 Mon Sep 17 00:00:00 2001 From: Meta Vens Date: Thu, 10 Sep 2026 01:18:45 +0200 Subject: [PATCH] Write down that a stored record's encoding is part of its contract Three byte-exact re-encode guards exist across three separate Idunn stores -- the runtime projection, the deployment brake, and the trust bindings -- each decoding a record, re-encoding it, and refusing on any byte difference. Idunn's control store does the same for its own transactions. Three independent instances is a house pattern, and it means changing how any persisted CultCache record serializes is never a flag flip. Records the measured behaviour of #[cultcache(key = N, bytes)] rather than the assumed one: reading stays compatible both ways, so the cost lands on reproduction and therefore on those guards, not on compatibility. A field with no live producer is free; a field with persisted records needs its store rewritten in the same change. Also records that Option> cannot take the attribute today, and that a shape-validated field nobody produces may be a fossil rather than an omission -- the estate has at least one other case of a live consumer outliving its producer. This was learned by nearly deploying an attribute flip that would have made Idunn refuse the projection store Odin reads. Co-Authored-By: Claude Opus 5 --- docs/persisted-record-encoding.md | 68 +++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) create mode 100644 docs/persisted-record-encoding.md diff --git a/docs/persisted-record-encoding.md b/docs/persisted-record-encoding.md new file mode 100644 index 00000000..b8e13f8a --- /dev/null +++ b/docs/persisted-record-encoding.md @@ -0,0 +1,68 @@ +# The encoding of a persisted CultCache record is part of its contract + +Changing how a stored record serializes is never a flag flip in this estate. +Consumers verify canonicality by **re-encoding what they decoded and comparing +bytes**, so a record that reads back correctly but re-encodes differently is +refused — and the refusal happens on live data, at startup or on the next read. + +Three guards, three separate stores, one idiom: + +| store | guard | refusal | +|---|---|---| +| Idunn runtime projection | `Idunn/src/drivers.rs:3611` | `service trust-anchor projection is noncanonical` | +| Idunn deployment brake | `Idunn/src/control_plane.rs` (`read_deployment_brake`) | `deployment brake is noncanonical or keyed by another authority` | +| Idunn trust bindings | `Idunn/src/provisioning.rs:546`, `:630` | `trust store contains a noncanonical or mismatched binding` | + +Idunn's control store applies the same rule to its own transactions. Assume the +guard exists and go looking for it before changing any persisted record's +serialization; three independent instances is a house pattern, not a +coincidence. + +## What this means for `#[cultcache(key = N, bytes)]` + +The attribute switches a `Vec` slot from a MessagePack array-of-integers +(serde's default) to a real `bin`. Measured, not assumed +(`packages/cultnet-rs/tests/byte_slot_migration.rs`): + +- **Reading stays compatible in both directions.** `serde_bytes::ByteBuf` + accepts an array marker, and a plain `Vec` accepts `bin`. No dual-read + path is needed, and reverting a binary is survivable. +- **Reproduction does not.** A record written before the flip can be read but + can never re-encode to the bytes it was stored as, and its `canonical_sha256` + moves with it. + +So the cost lands entirely on the guards above, not on readability. A field with +persisted records must flip the attribute **and** rewrite its store in the same +change, one target at a time, and never during a deployment — the projection +store is what the observer reads, and refusing it takes the observer down. + +A field with no live producer is free. Check for one before assuming a +migration: `IdunnAuthenticatedProviderHealthProjectionRecord` is constructed +only inside a test module, so its byte field costs nothing to convert. + +Size, for whether it is worth doing at all: 406 bytes as an array against 278 as +`bin` for a 256-byte payload, and 1234 against 851 for an 848-byte one. About +1.45x on every byte, which matters on a media path and is noise on a control +record. + +## `Option>` cannot take the attribute yet + +The derive emits `serde_bytes::Bytes::new(&self.field)`, which needs `&[u8]`, +and builds the field via `.into_vec()`, which yields `Vec`. Both are type +errors against an `Option>`, so the derive fails loudly rather than +collapsing "absent slot" into "unsigned". `IdunnDeploymentBrakeRecord.signature` +needs an `Option`-aware path in `cultcache-rs-derive` before it can move. + +## Fields validated for shape and produced by nothing + +`trust_binding_sha256` is declared on two records with different schemas +(`idunn.authenticated_provider_health_projection.v1` in cultnet-rs, +`idunn.authenticated_daemon_health_admission.v1` in odin-core), shape-validated +in both, and computed by nothing in CultLib, Idunn or Odin. + +Do not assume that means "omission". It may be a fossil — a contract whose +producer was deleted. The estate has form: Muninn's receiver runs a Reed-Solomon +audio FEC path against parity shards the Rust sender has never emitted, while an +acceptance ledger describes that FEC as having worked. Settling which requires +reading history, not source, and the answer changes whether the right move is to +implement the field or delete it.