Carry media payloads as bin
The three media payload fields take the derive's `bytes` attribute, so they serialize as MessagePack `bin` instead of an array of integers. An 848-byte payload goes from 1234 bytes on the wire to 851 — 1.45x off every media byte, on the one link measured to be starved. This is a correction to the previous commit's reasoning, not a new capability. That commit claimed the derive could not emit bin and left a test ignored against a hint that needed designing. The hint has existed all along: cultcache-rs-derive parses `bytes` at lib.rs:92, serializes through serde_bytes::Bytes at :135, and decodes through ByteBuf at :154 and :170. What is inert on a DatabaseEntry struct is `#[serde(with = "serde_bytes")]`, because the derive reads only its own attribute namespace — which is a different and much smaller claim than the one that was made. The ignored test is now un-ignored and passing, and the tolerant assertion it was propping up is exact again. Four fields elsewhere are missing `bytes` and are deliberately untouched here: IdunnDeploymentBrakeRecord.signature, IdunnDaemonHealthTrustBindingRecord .signer_public_key, GameCultServiceTrustAnchorRecord.signer_public_key, and IdunnAuthenticatedProviderHealthProjectionRecord.signature. Those carry stored state and signing history — six trust anchors live in the deployment projection on yggdrasil, and the brake store is byte-exact checked — so they need sequencing with a re-sign rather than a flag flip. Owned by the Swarm Migration work. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
a250ec69eb
commit
2ba404cc0b
2 changed files with 21 additions and 25 deletions
|
|
@ -18,6 +18,12 @@
|
|||
//! wire format: append new fields with `default` at the next free index, never
|
||||
//! reorder or renumber. `missing_video_chunk_keys` shows the pattern.
|
||||
//!
|
||||
//! Payloads carry `bytes` so they serialize as MessagePack `bin` rather than an
|
||||
//! array of integers, which is what the C#, TypeScript and Python runtimes
|
||||
//! expect and what keeps a 848-byte payload at 851 bytes instead of 1234. Note
|
||||
//! that this is the derive's own attribute — `#[serde(with = "serde_bytes")]`
|
||||
//! is inert here, because `DatabaseEntry` writes its own `Serialize`.
|
||||
//!
|
||||
//! # Timebase
|
||||
//!
|
||||
//! `pts_ticks`, `duration_ticks` and `deadline_ticks` are expressed in
|
||||
|
|
@ -76,7 +82,7 @@ pub struct GameCultMediaVideoAccessUnitRecord {
|
|||
pub chunk_index: u16,
|
||||
#[cultcache(key = 12)]
|
||||
pub chunk_count: u16,
|
||||
#[cultcache(key = 13)]
|
||||
#[cultcache(key = 13, bytes)]
|
||||
pub payload: Vec<u8>,
|
||||
}
|
||||
|
||||
|
|
@ -123,7 +129,7 @@ pub struct GameCultMediaVideoParityShardRecord {
|
|||
pub chunk_payload_bytes: u32,
|
||||
#[cultcache(key = 15)]
|
||||
pub last_chunk_payload_bytes: u32,
|
||||
#[cultcache(key = 16)]
|
||||
#[cultcache(key = 16, bytes)]
|
||||
pub payload: Vec<u8>,
|
||||
}
|
||||
|
||||
|
|
@ -153,7 +159,7 @@ pub struct GameCultMediaAudioPacketRecord {
|
|||
pub timebase_den: u32,
|
||||
#[cultcache(key = 8)]
|
||||
pub deadline_ticks: i64,
|
||||
#[cultcache(key = 9)]
|
||||
#[cultcache(key = 9, bytes)]
|
||||
pub payload: Vec<u8>,
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -67,33 +67,23 @@ fn video_access_unit_encodes_as_a_positional_array() {
|
|||
assert_eq!(slots[8].as_bool(), Some(true), "slot 8 is keyframe");
|
||||
assert!(slots[9].is_nil(), "an absent dependency stays nil, not 0");
|
||||
assert_eq!(slots[12].as_u64(), Some(5), "slot 12 is chunk_count");
|
||||
assert!(
|
||||
matches!(&slots[13], Value::Binary(_) | Value::Array(_)),
|
||||
"slot 13 is the payload; see payload_should_encode_as_messagepack_bin \
|
||||
for which of the two it ought to be"
|
||||
assert_eq!(
|
||||
slots[13].as_slice(),
|
||||
Some([0xDE, 0xAD, 0xBE, 0xEF].as_slice()),
|
||||
"slot 13 is the payload, carried as bin"
|
||||
);
|
||||
}
|
||||
|
||||
/// Ignored until `cultcache-rs-derive` grows a `bytes` hint.
|
||||
/// Payloads must reach the wire as MessagePack `bin`, not as an array of
|
||||
/// integers. The C#, TypeScript and Python runtimes all read byte fields as
|
||||
/// `bin`, so a payload serialized as a sequence is unreadable by the reference
|
||||
/// implementation, and costs 1.45x besides: 848 bytes encode to 1234 as an
|
||||
/// integer array against 851 as `bin`.
|
||||
///
|
||||
/// The derive writes its own `Serialize` and calls `serialize_element` per slot,
|
||||
/// so a `Vec<u8>` field goes out as a MessagePack array of integers. C#,
|
||||
/// TypeScript and Python all emit `bin` for byte fields, which makes Rust the
|
||||
/// divergent runtime and these records unreadable by the reference
|
||||
/// implementation. `#[serde(with = "serde_bytes")]` cannot fix it — the derive
|
||||
/// ignores serde attributes.
|
||||
///
|
||||
/// It also costs 1.45x on every media byte: an 848-byte payload encodes to 1234
|
||||
/// bytes as an integer array against 851 as `bin`.
|
||||
///
|
||||
/// The fix is not local to this file. Changing byte encoding globally would
|
||||
/// invalidate every signature in the Idunn authority chain, because signatures
|
||||
/// are taken over the whole positional serialization and canonical hashes over
|
||||
/// the same bytes. It needs an opt-in hint so these three media records can
|
||||
/// adopt `bin` without moving the signature-bearing ones. Owned by the Swarm
|
||||
/// Migration work; un-ignore this when the hint lands.
|
||||
/// This is what the derive's `bytes` attribute buys. `#[serde(with =
|
||||
/// "serde_bytes")]` will not do it — `DatabaseEntry` emits its own `Serialize`
|
||||
/// and reads only its own `cultcache` attribute namespace.
|
||||
#[test]
|
||||
#[ignore = "blocked on a `bytes` opt-in hint in cultcache-rs-derive"]
|
||||
fn payload_should_encode_as_messagepack_bin() {
|
||||
let encoded = rmp_serde::to_vec(&access_unit()).expect("encodes");
|
||||
let Value::Array(slots) = decode_as_value(&encoded) else {
|
||||
|
|
|
|||
Loading…
Reference in a new issue