Media FEC: enforce the datagram invariant at the producer, refuse overflowing audio blocks

The 720-byte shard cap stood in for "an encoded record fits one datagram" and
failed it at legal maximum-width identifiers. The invariant is now checked where
records are produced: protect_video_frame and protect_audio_block take the
producer's provenance, encode each record, and refuse the block when one exceeds
MediaFecPolicy::max_wire_bytes (default 1431, the IPv4/1500 MTU budget; smaller
paths pass less). The fixed cap is deleted; it contradicted a larger budget and
bounded nothing the decoder does not already hold in memory.

Audio block arithmetic lives in one function, audio_slot, used by the protect
path, the wire validator and recovery. The validator refuses a parity record
whose last packet id or pts overflows; recovery uses the same checked function
instead of adding.

Tests pin the policy 256-shard bounds, block identity on block_count and
block_data_start, the k=16 m=4 known answer, and the burst property
(worst always-recovering burst = blocks x smallest m for 1..=399 chunks), and
the wire behaviour of the media channel profile (delivery and expiry). Docs
state the rs-gf256-v1 construction and that audio playout gates on pts.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Meta Vens 2026-09-30 02:50:11 +02:00
commit a813c09b6d
6 changed files with 657 additions and 194 deletions

View file

@ -15,10 +15,35 @@
//! are repaired and any more are reported as [`MediaFecError::BeyondRepair`],
//! never as wrong bytes. The construction is named on every parity record by
//! [`MEDIA_FEC_SCHEME_RS_GF256_V1`] and backed by the `reed-solomon-erasure`
//! crate. The library is not the contract: `tests/media_fec.rs` pins the parity
//! bytes of a known input, so a library that changes the matrix fails there
//! instead of on the wire, and a decoder in another runtime has a vector to
//! meet.
//! crate. The library is not the contract: the construction below is, and
//! `tests/media_fec.rs` pins the parity bytes of known inputs, so a library that
//! changes the matrix fails there instead of on the wire, and a decoder in
//! another runtime has vectors to meet.
//!
//! # The `rs-gf256-v1` construction
//!
//! Enough to implement it in another runtime without this crate.
//!
//! * The field is GF(2^8) over the polynomial `x^8 + x^4 + x^3 + x^2 + 1`
//! (`0x11D`). Addition is XOR; multiplication is carry-less multiplication
//! reduced by `0x11D`.
//! * For a block of `k` data shards and `m` parity shards, let `n = k + m`
//! (`n <= 256`) and build the `n x k` Vandermonde matrix `V` whose entry in
//! row `r`, column `c` is `r^c`, where the row number `r` (0 through
//! `n - 1`) is itself read as a field element and `0^0 = 1`. The evaluation
//! points are the integers 0 through `n - 1`, not powers of a generator.
//! * The encoding matrix is `E = V * inverse(top)`, where `top` is the first
//! `k` rows of `V`. Its first `k` rows are the identity, so data shards are
//! sent unchanged; its last `m` rows generate parity.
//! * Every shard of a block has the same length, `shard_payload_bytes`. A
//! shorter final video chunk is zero-padded to that length before encoding.
//! Byte `j` of parity shard `p` is the XOR over data shards `i` of
//! `E[k + p][i] * data[i][j]`.
//! * Any `k` surviving shards recover the block: invert the `k x k` submatrix
//! of `E` for the surviving rows and multiply.
//!
//! The known-answer vectors in `tests/media_fec.rs` were checked against an
//! independent implementation of exactly this text.
//!
//! # What a block is
//!
@ -31,9 +56,28 @@
//! * Audio: `4` consecutive packets of one payload length, protected by `2`
//! parity shards.
//!
//! Every shard, data or parity, fits one datagram once wrapped for the wire
//! ([`MEDIA_FEC_MAX_SHARD_PAYLOAD_BYTES`]), so parity never reintroduces the
//! fragment amplification it exists to remove.
//! # One datagram
//!
//! Every record the protect functions return, data or parity, fits one
//! datagram once wrapped for the wire, so parity never reintroduces the
//! fragment amplification it exists to remove. The protect functions enforce
//! that against [`MediaFecPolicy::max_wire_bytes`]: they encode each record
//! with the caller's provenance and refuse the block if any is larger.
//! The default budget is [`MEDIA_FEC_MAX_WIRE_BYTES`], an IPv4 path with a
//! 1500-byte MTU. Pass a smaller budget when the path carries less: IPv6 over
//! 1500 leaves a UDP payload of 1452 bytes, and a WireGuard or Tailscale
//! tunnel about 1252, so subtract the 41 bytes of RUDP header and channel name
//! from the path's UDP payload limit.
//!
//! The budget is measured with the provenance given, so a producer whose
//! `stored_at` or identifiers grow later must pass the widest it will use.
//!
//! # Audio playout
//!
//! A recovered audio packet has the block's latest `deadline_ticks`, so a
//! packet recovered early in the block reports a deadline later than its own.
//! Audio consumers must gate playout on `pts_ticks`, which recovery derives
//! exactly, and use `deadline_ticks` only to give up on the block as a whole.
//!
//! # What this module does not own
//!
@ -51,8 +95,9 @@ use crate::media_stream_contracts::{
GameCultMediaVideoAccessUnitRecord, GameCultMediaVideoParityShardRecord,
};
use crate::media_stream_wire::{
GAMECULT_MEDIA_CHANNEL, GameCultMediaWireRecord, validate_audio_parity_record,
validate_audio_record, validate_video_parity_record, validate_video_record,
GAMECULT_MEDIA_CHANNEL, GameCultMediaWireRecord, MediaWireProvenance,
encode_media_wire_record, validate_audio_parity_record, validate_audio_record,
validate_video_parity_record, validate_video_record,
};
use crate::rudp::RUDP_FIXED_HEADER_BYTES;
@ -64,24 +109,15 @@ pub const MEDIA_FEC_SCHEME_RS_GF256_V1: &str = "rs-gf256-v1";
/// The largest UDP payload an IPv4 datagram carries on a 1500-byte MTU.
pub const MEDIA_IPV4_UDP_PAYLOAD_BYTES: usize = 1_472;
/// The most a wrapped media record may occupy so that the RUDP header and the
/// channel name still leave it in one datagram (1,431 bytes).
/// The default datagram budget: the most a wrapped media record may occupy so
/// that the RUDP header and the channel name still leave it in one IPv4
/// datagram on a 1500-byte MTU (1,431 bytes).
pub const MEDIA_FEC_MAX_WIRE_BYTES: usize =
MEDIA_IPV4_UDP_PAYLOAD_BYTES - RUDP_FIXED_HEADER_BYTES - GAMECULT_MEDIA_CHANNEL.len();
/// The largest shard payload the codec produces or accepts. What is left of
/// [`MEDIA_FEC_MAX_WIRE_BYTES`] must hold the record's fields and its envelope,
/// which repeats the stream and session identifiers three times over: a parity
/// record wraps to 705 bytes around its payload when the stream id, session id
/// and codec name are 32, 32 and 16 bytes and the frame id has 20 digits. This is the largest multiple of 16
/// that fits with identifiers of that size; the test
/// `every_record_at_the_maximum_shard_size_fits_one_datagram` measures it. A
/// producer with longer identifiers must cut its chunks smaller.
pub const MEDIA_FEC_MAX_SHARD_PAYLOAD_BYTES: usize = 720;
/// A producer's choice of `(k, m)`. [`MediaFecPolicy::STANDARD`] is the ruled
/// default; each record carries the `(k, m)` it was made with, so a decoder
/// needs no policy.
/// A producer's choice of `(k, m)` and datagram budget.
/// [`MediaFecPolicy::STANDARD`] is the ruled default; each record carries the
/// `(k, m)` it was made with, so a decoder needs no policy.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct MediaFecPolicy {
/// The most data chunks in one video block.
@ -95,16 +131,21 @@ pub struct MediaFecPolicy {
pub audio_data_shards: u16,
/// Parity shards per audio block.
pub audio_parity_shards: u16,
/// The most a wrapped record may occupy. The protect functions refuse a
/// block with any record over it. See the module docs for smaller paths.
pub max_wire_bytes: usize,
}
impl MediaFecPolicy {
/// Video `k <= 16` with `m = max(2, ceil(k / 4))`; audio 4+2.
/// Video `k <= 16` with `m = max(2, ceil(k / 4))`; audio 4+2; the IPv4
/// datagram budget.
pub const STANDARD: Self = Self {
video_max_block_data_shards: 16,
video_min_parity_shards: 2,
video_parity_divisor: 4,
audio_data_shards: 4,
audio_parity_shards: 2,
max_wire_bytes: MEDIA_FEC_MAX_WIRE_BYTES,
};
/// `m` for a video block of `data_shards` chunks.
@ -230,6 +271,40 @@ fn from_anyhow(error: anyhow::Error) -> MediaFecError {
MediaFecError::Invalid(error.to_string())
}
/// Refuses a record that would not fit one datagram once wrapped.
fn fits_datagram(
record: &GameCultMediaWireRecord,
provenance: MediaWireProvenance<'_>,
policy: &MediaFecPolicy,
) -> Result<(), MediaFecError> {
let wire = encode_media_wire_record(record, provenance).map_err(from_anyhow)?;
if wire.len() > policy.max_wire_bytes {
return Err(MediaFecError::invalid(format!(
"a {} record wraps to {} bytes, over the {}-byte datagram budget: cut chunks smaller or raise max_wire_bytes",
record.schema_id(),
wire.len(),
policy.max_wire_bytes
)));
}
Ok(())
}
/// Where slot `slot` of an audio block lands: its packet id and presentation
/// time, or `None` if either overflows. The one place block arithmetic lives;
/// the protect path, the wire validator and recovery all ask it.
pub(crate) fn audio_slot(
base_packet_id: u64,
base_pts_ticks: i64,
packet_duration_ticks: u32,
slot: u64,
) -> Option<(u64, i64)> {
let packet_id = base_packet_id.checked_add(slot)?;
let pts_ticks = i64::from(packet_duration_ticks)
.checked_mul(i64::try_from(slot).ok()?)?
.checked_add(base_pts_ticks)?;
Some((packet_id, pts_ticks))
}
// ---------------------------------------------------------------------------
// Video
// ---------------------------------------------------------------------------
@ -247,10 +322,12 @@ fn from_anyhow(error: anyhow::Error) -> MediaFecError {
/// Errors if `chunks` are not exactly one whole frame (mixed frames, a missing
/// or repeated chunk index, disagreeing framing), if a chunk is not shard
/// shaped (every chunk of a block but its last must be the same length, the
/// last no longer), or if a chunk exceeds [`MEDIA_FEC_MAX_SHARD_PAYLOAD_BYTES`].
/// last no longer), or if any resulting record, wrapped with `provenance`,
/// exceeds `policy.max_wire_bytes`.
pub fn protect_video_frame(
chunks: &[GameCultMediaVideoAccessUnitRecord],
policy: &MediaFecPolicy,
provenance: MediaWireProvenance<'_>,
) -> Result<Vec<GameCultMediaWireRecord>, MediaFecError> {
policy.validate()?;
let first = chunks
@ -273,12 +350,6 @@ pub fn protect_video_frame(
"a block never spans frames: chunks disagree on stream, session, frame or framing",
));
}
if chunk.payload.len() > MEDIA_FEC_MAX_SHARD_PAYLOAD_BYTES {
return Err(MediaFecError::invalid(format!(
"chunk payload of {} bytes exceeds the {MEDIA_FEC_MAX_SHARD_PAYLOAD_BYTES}-byte shard limit",
chunk.payload.len()
)));
}
if by_index.insert(chunk.chunk_index, chunk).is_some() {
return Err(MediaFecError::invalid(format!(
"chunk {} appears twice",
@ -347,7 +418,11 @@ pub fn protect_video_frame(
lanes.push(interleave(data_records.collect(), parity_records));
start += size;
}
Ok(rotate(lanes))
let records = rotate(lanes);
for record in &records {
fits_datagram(record, provenance, policy)?;
}
Ok(records)
}
fn same_frame(a: &GameCultMediaVideoAccessUnitRecord, b: &GameCultMediaVideoAccessUnitRecord) -> bool {
@ -508,9 +583,12 @@ fn same_block(a: &GameCultMediaVideoParityShardRecord, b: &GameCultMediaVideoPar
/// Protects one block of audio packets: exactly `policy.audio_data_shards`
/// packets with contiguous ids and presentation times, one payload length, and
/// one codec and timebase. Returns the parity records in `parity_index` order.
/// Errors if a parity record, wrapped with `provenance`, exceeds
/// `policy.max_wire_bytes`; the packets are smaller than their parity.
pub fn protect_audio_block(
packets: &[GameCultMediaAudioPacketRecord],
policy: &MediaFecPolicy,
provenance: MediaWireProvenance<'_>,
) -> Result<Vec<GameCultMediaAudioParityShardRecord>, MediaFecError> {
policy.validate()?;
if packets.len() != usize::from(policy.audio_data_shards) {
@ -524,18 +602,15 @@ pub fn protect_audio_block(
let shard_bytes = first.payload.len();
for (index, packet) in packets.iter().enumerate() {
validate_audio_record(packet).map_err(from_anyhow)?;
let expected_id = first.packet_id.checked_add(index as u64);
let expected_pts = i64::from(first.duration_ticks)
.checked_mul(index as i64)
.and_then(|offset| first.pts_ticks.checked_add(offset));
let expected =
audio_slot(first.packet_id, first.pts_ticks, first.duration_ticks, index as u64);
if packet.stream_id != first.stream_id
|| packet.session_id != first.session_id
|| packet.codec != first.codec
|| packet.timebase_num != first.timebase_num
|| packet.timebase_den != first.timebase_den
|| packet.duration_ticks != first.duration_ticks
|| Some(packet.packet_id) != expected_id
|| Some(packet.pts_ticks) != expected_pts
|| Some((packet.packet_id, packet.pts_ticks)) != expected
|| packet.payload.len() != shard_bytes
{
return Err(MediaFecError::invalid(
@ -543,14 +618,10 @@ pub fn protect_audio_block(
));
}
}
if shard_bytes > MEDIA_FEC_MAX_SHARD_PAYLOAD_BYTES {
return Err(MediaFecError::invalid(format!(
"audio payload of {shard_bytes} bytes exceeds the {MEDIA_FEC_MAX_SHARD_PAYLOAD_BYTES}-byte shard limit"
)));
}
let data: Vec<Vec<u8>> = packets.iter().map(|packet| packet.payload.clone()).collect();
let deadline_ticks = packets.iter().map(|packet| packet.deadline_ticks).max().unwrap_or(0);
Ok(parity_shards(&data, usize::from(policy.audio_parity_shards))?
let parity: Vec<GameCultMediaAudioParityShardRecord> =
parity_shards(&data, usize::from(policy.audio_parity_shards))?
.into_iter()
.enumerate()
.map(|(parity_index, payload)| GameCultMediaAudioParityShardRecord {
@ -570,13 +641,20 @@ pub fn protect_audio_block(
shard_payload_bytes: shard_bytes as u32,
payload,
})
.collect())
.collect();
// A parity record repeats every field of a packet and adds the scheme and
// the geometry, so it is the larger of the block's records: if it fits, the
// packets do.
for shard in &parity {
fits_datagram(&GameCultMediaWireRecord::AudioParity(shard.clone()), provenance, policy)?;
}
Ok(parity)
}
/// Rebuilds the packets an audio block lost, ordered by `packet_id`. A
/// recovered packet takes its id and presentation time from the block's
/// arithmetic and its deadline from the block's, the latest in the block.
/// Errors as [`recover_video_block`] does.
/// arithmetic and its deadline from the block's, the latest in the block:
/// gate playout on `pts_ticks`, not `deadline_ticks`. Errors as [`recover_video_block`] does.
pub fn recover_audio_block(
parity: &[GameCultMediaAudioParityShardRecord],
data: &[GameCultMediaAudioPacketRecord],
@ -615,21 +693,30 @@ pub fn recover_audio_block(
let missing: Vec<usize> = (0..k).filter(|&slot| shards[slot].is_none()).collect();
let recovered = recover_data_shards(k, shards)?;
Ok(missing
missing
.into_iter()
.map(|slot| GameCultMediaAudioPacketRecord {
stream_id: geometry.stream_id.clone(),
session_id: geometry.session_id.clone(),
packet_id: geometry.base_packet_id + slot as u64,
codec: geometry.codec.clone(),
pts_ticks: geometry.base_pts_ticks + i64::from(geometry.packet_duration_ticks) * slot as i64,
duration_ticks: geometry.packet_duration_ticks,
timebase_num: geometry.timebase_num,
timebase_den: geometry.timebase_den,
deadline_ticks: geometry.deadline_ticks,
payload: recovered[slot].clone(),
.map(|slot| {
let (packet_id, pts_ticks) = audio_slot(
geometry.base_packet_id,
geometry.base_pts_ticks,
geometry.packet_duration_ticks,
slot as u64,
)
.ok_or_else(|| MediaFecError::invalid("audio block runs past the packet id or pts range"))?;
Ok(GameCultMediaAudioPacketRecord {
stream_id: geometry.stream_id.clone(),
session_id: geometry.session_id.clone(),
packet_id,
codec: geometry.codec.clone(),
pts_ticks,
duration_ticks: geometry.packet_duration_ticks,
timebase_num: geometry.timebase_num,
timebase_den: geometry.timebase_den,
deadline_ticks: geometry.deadline_ticks,
payload: recovered[slot].clone(),
})
})
.collect())
.collect()
}
fn same_audio_block(a: &GameCultMediaAudioParityShardRecord, b: &GameCultMediaAudioParityShardRecord) -> bool {

View file

@ -28,7 +28,7 @@ use crate::contracts::{
CultNetMessage, CultNetRawDocumentRecord, CultNetRawPayloadEncoding, CultNetWireContract,
decode_cultnet_message_from_slice, encode_cultnet_message_to_vec,
};
use crate::media_fec::{MEDIA_FEC_MAX_SHARD_PAYLOAD_BYTES, MEDIA_FEC_SCHEME_RS_GF256_V1};
use crate::media_fec::{MEDIA_FEC_SCHEME_RS_GF256_V1, audio_slot};
use crate::media_stream_contracts::{
GAMECULT_MEDIA_AUDIO_PACKET_SCHEMA, GAMECULT_MEDIA_AUDIO_PARITY_SHARD_SCHEMA,
GAMECULT_MEDIA_RECEIVER_FEEDBACK_SCHEMA, GAMECULT_MEDIA_VIDEO_ACCESS_UNIT_SCHEMA,
@ -403,11 +403,9 @@ pub fn validate_video_parity_record(record: &GameCultMediaVideoParityShardRecord
"video parity media record block exceeds 256 shards"
));
}
if record.shard_payload_bytes == 0
|| record.shard_payload_bytes as usize > MEDIA_FEC_MAX_SHARD_PAYLOAD_BYTES
{
if record.shard_payload_bytes == 0 {
return Err(anyhow!(
"video parity media record shard_payload_bytes must be in 1..={MEDIA_FEC_MAX_SHARD_PAYLOAD_BYTES}"
"video parity media record shard_payload_bytes must be non-zero"
));
}
if record.last_chunk_payload_bytes == 0
@ -454,11 +452,21 @@ pub fn validate_audio_parity_record(record: &GameCultMediaAudioParityShardRecord
{
return Err(anyhow!("audio parity media record stripe metadata is invalid"));
}
if record.shard_payload_bytes == 0
|| record.shard_payload_bytes as usize > MEDIA_FEC_MAX_SHARD_PAYLOAD_BYTES
if audio_slot(
record.base_packet_id,
record.base_pts_ticks,
record.packet_duration_ticks,
u64::from(record.data_shard_count) - 1,
)
.is_none()
{
return Err(anyhow!(
"audio parity media record shard_payload_bytes must be in 1..={MEDIA_FEC_MAX_SHARD_PAYLOAD_BYTES}"
"audio parity media record block runs past the packet id or pts range"
));
}
if record.shard_payload_bytes == 0 {
return Err(anyhow!(
"audio parity media record shard_payload_bytes must be non-zero"
));
}
if record.payload.len() != record.shard_payload_bytes as usize {

View file

@ -90,3 +90,116 @@ fn the_other_channels_keep_their_fixed_delivery() {
assert_eq!(delivery_of("schema"), CultNetTransportDelivery::Reliable);
assert_eq!(delivery_of("realtime"), CultNetTransportDelivery::Unreliable);
}
// ---------------------------------------------------------------------------
// What the media channel puts on the wire
// ---------------------------------------------------------------------------
use std::thread::sleep;
use std::time::Duration;
use cultnet_rs::{
CultNetRudpPacket, CultNetRudpSession, CultNetRudpSessionOptions, decode_rudp_packet,
encode_rudp_packet,
};
/// A connected client transport and the raw socket standing in for its peer.
struct Link {
client: CultNetRudpSocketTransportConnection,
peer: UdpSocket,
}
impl Link {
fn open(delivery: Option<CultNetTransportDelivery>, expire_after_ms: Option<u64>) -> Self {
let peer = UdpSocket::bind("127.0.0.1:0").expect("binds");
peer.set_read_timeout(Some(Duration::from_millis(500))).unwrap();
let socket = UdpSocket::bind("127.0.0.1:0").expect("binds");
socket.set_read_timeout(Some(Duration::from_millis(300))).unwrap();
let mut options = CultNetRudpSocketTransportOptions::client(
"test-media",
socket,
peer.local_addr().unwrap(),
7,
);
options.media_delivery = delivery;
options.media_reliable_expire_after_ms = expire_after_ms;
options.resend_delay_ms = 10;
let mut client = CultNetRudpSocketTransportConnection::new(options).expect("opens");
client.connect(Vec::new()).expect("sends connect");
let mut wire = vec![0_u8; 2_048];
let (received, client_addr) = peer.recv_from(&mut wire).expect("the connect arrives");
let connect = decode_rudp_packet(&wire[..received]).expect("decodes");
let mut server = CultNetRudpSession::new(CultNetRudpSessionOptions {
connection_id: 7,
..CultNetRudpSessionOptions::default()
});
let accept = server.accept_connect(&connect, 0, Vec::new()).expect("accepts");
peer.send_to(&encode_rudp_packet(&accept).unwrap(), client_addr).unwrap();
client.receive_once().expect("takes the accept");
assert!(client.connected());
let mut link = Self { client, peer };
link.datagrams(Duration::from_millis(100));
link
}
/// Everything the peer receives within `window`.
fn datagrams(&mut self, window: Duration) -> Vec<CultNetRudpPacket> {
self.peer.set_read_timeout(Some(window)).unwrap();
let mut packets = Vec::new();
let mut wire = vec![0_u8; 2_048];
while let Ok((received, _)) = self.peer.recv_from(&mut wire) {
packets.push(decode_rudp_packet(&wire[..received]).expect("decodes"));
}
packets
}
fn send_media(&mut self, window: Duration) -> Vec<CultNetRudpPacket> {
self.client.send("media", vec![1, 2, 3]).expect("sends");
self.datagrams(window)
}
}
fn media_only(packets: Vec<CultNetRudpPacket>) -> Vec<CultNetRudpPacket> {
packets.into_iter().filter(|packet| packet.channel_id == "media").collect()
}
/// The channel the profile describes as reliable is sent reliable, unordered
/// and unsequenced; asking for an unreliable lane sends it unreliable.
#[test]
fn the_media_channel_is_sent_with_the_delivery_the_profile_advertises() {
let mut reliable = Link::open(None, None);
let sent = media_only(reliable.send_media(Duration::from_millis(100)));
assert_eq!(sent.len(), 1);
assert!(sent[0].reliable, "the default media lane is reliable");
assert!(!sent[0].ordered && !sent[0].sequenced, "media is neither ordered nor sequenced");
let mut unreliable = Link::open(Some(CultNetTransportDelivery::Unreliable), None);
let sent = media_only(unreliable.send_media(Duration::from_millis(100)));
assert_eq!(sent.len(), 1);
assert!(!sent[0].reliable, "an unreliable media lane is not retransmitted");
assert!(!sent[0].ordered && !sent[0].sequenced);
}
/// A reliable media packet stops being retransmitted once its expiry passes,
/// and keeps being retransmitted when it has none.
#[test]
fn the_media_channels_expiry_is_applied_to_its_packets() {
let mut expiring = Link::open(None, Some(20));
assert_eq!(media_only(expiring.send_media(Duration::from_millis(50))).len(), 1);
sleep(Duration::from_millis(150));
expiring.client.poll_resends().unwrap();
assert!(
media_only(expiring.datagrams(Duration::from_millis(150))).is_empty(),
"an expired media packet is dropped, not retransmitted"
);
let mut patient = Link::open(None, None);
assert_eq!(media_only(patient.send_media(Duration::from_millis(50))).len(), 1);
sleep(Duration::from_millis(150));
patient.client.poll_resends().unwrap();
assert!(
!media_only(patient.datagrams(Duration::from_millis(150))).is_empty(),
"without an expiry the unacknowledged packet is retransmitted"
);
}

View file

@ -8,7 +8,7 @@
use cultnet_rs::{
GameCultMediaAudioPacketRecord, GameCultMediaAudioParityShardRecord,
GameCultMediaVideoAccessUnitRecord, GameCultMediaVideoParityShardRecord,
GameCultMediaWireRecord, MEDIA_FEC_MAX_SHARD_PAYLOAD_BYTES, MEDIA_FEC_MAX_WIRE_BYTES,
GameCultMediaWireRecord, MEDIA_FEC_MAX_WIRE_BYTES,
MEDIA_FEC_SCHEME_RS_GF256_V1, MediaFecError, MediaFecPolicy, MediaWireProvenance,
decode_media_wire_record, encode_media_wire_record, protect_audio_block, protect_video_frame,
recover_audio_block, recover_video_block,
@ -141,7 +141,7 @@ fn a_frame_splits_into_ceil_n_over_16_near_equal_blocks() {
/// `m + 1` pattern to be refused.
fn assert_every_erasure_pattern_behaves(k: u16) {
let original = frame(u64::from(k), k, 37, 21);
let records = protect_video_frame(&original, &STANDARD).expect("protects");
let records = protect_video_frame(&original, &STANDARD, provenance()).expect("protects");
let m = usize::from(STANDARD.video_parity_shards(k));
let total = usize::from(k) + m;
assert_eq!(records.len(), total, "one block: k data and m parity records");
@ -193,7 +193,7 @@ fn every_le_m_erasure_recovers_exactly_and_m_plus_one_is_refused_for_every_video
#[test]
fn every_le_m_erasure_recovers_exactly_and_m_plus_one_is_refused_for_audio_4_plus_2() {
let original = audio_block(100, 41);
let parity = protect_audio_block(&original, &STANDARD).expect("protects");
let parity = protect_audio_block(&original, &STANDARD, provenance()).expect("protects");
assert_eq!(parity.len(), 2);
for erased in 0_u32..(1 << 6) {
let erasures = erased.count_ones();
@ -232,7 +232,7 @@ fn every_le_m_erasure_recovers_exactly_and_m_plus_one_is_refused_for_audio_4_plu
#[test]
fn recovery_reports_how_short_the_block_is() {
let original = frame(9, 8, 37, 37);
let (data, parity) = split(protect_video_frame(&original, &STANDARD).unwrap());
let (data, parity) = split(protect_video_frame(&original, &STANDARD, provenance()).unwrap());
// 8 data + 2 parity; keep 6 data and 1 parity: 7 present, 8 needed.
let outcome = recover_video_block(&parity[..1], &data[..6]);
assert_eq!(
@ -244,7 +244,7 @@ fn recovery_reports_how_short_the_block_is() {
#[test]
fn a_recovered_short_final_chunk_has_its_true_length() {
let original = frame(4, 5, 100, 7);
let (mut data, parity) = split(protect_video_frame(&original, &STANDARD).unwrap());
let (mut data, parity) = split(protect_video_frame(&original, &STANDARD, provenance()).unwrap());
let lost = data.pop().unwrap();
assert_eq!(lost.payload.len(), 7);
let recovered = recover_video_block(&parity, &data).unwrap();
@ -254,7 +254,7 @@ fn a_recovered_short_final_chunk_has_its_true_length() {
#[test]
fn duplicate_shards_neither_count_twice_nor_break_recovery() {
let original = frame(5, 4, 50, 50);
let (mut data, parity) = split(protect_video_frame(&original, &STANDARD).unwrap());
let (mut data, parity) = split(protect_video_frame(&original, &STANDARD, provenance()).unwrap());
let lost = data.remove(1);
data.push(data[0].clone());
let mut parity_twice = parity.clone();
@ -265,14 +265,14 @@ fn duplicate_shards_neither_count_twice_nor_break_recovery() {
#[test]
fn a_complete_block_recovers_nothing() {
let original = frame(6, 4, 50, 50);
let (data, parity) = split(protect_video_frame(&original, &STANDARD).unwrap());
let (data, parity) = split(protect_video_frame(&original, &STANDARD, provenance()).unwrap());
assert_eq!(recover_video_block(&parity[..1], &data).unwrap(), Vec::new());
}
#[test]
fn recovery_refuses_records_that_are_not_one_block() {
let original = frame(7, 32, 40, 40);
let (data, parity) = split(protect_video_frame(&original, &STANDARD).unwrap());
let (data, parity) = split(protect_video_frame(&original, &STANDARD, provenance()).unwrap());
let block_zero: Vec<_> = parity.iter().filter(|p| p.block_index == 0).cloned().collect();
let block_one: Vec<_> = parity.iter().filter(|p| p.block_index == 1).cloned().collect();
assert_eq!(block_zero.len(), 4);
@ -304,7 +304,7 @@ fn recovery_refuses_records_that_are_not_one_block() {
#[test]
fn a_final_chunk_of_the_wrong_length_is_refused() {
let original = frame(8, 5, 100, 7);
let (data, parity) = split(protect_video_frame(&original, &STANDARD).unwrap());
let (data, parity) = split(protect_video_frame(&original, &STANDARD, provenance()).unwrap());
let mut last = data.iter().find(|chunk| chunk.chunk_index == 4).unwrap().clone();
last.payload.push(0);
assert!(matches!(recover_video_block(&parity, &[last]), Err(MediaFecError::Invalid(_))));
@ -312,7 +312,7 @@ fn a_final_chunk_of_the_wrong_length_is_refused() {
#[test]
fn within_a_block_data_and_parity_alternate() {
let records = protect_video_frame(&frame(9, 4, 20, 20), &STANDARD).unwrap();
let records = protect_video_frame(&frame(9, 4, 20, 20), &STANDARD, provenance()).unwrap();
let kinds: String = records
.iter()
.map(|record| match record {
@ -327,7 +327,7 @@ fn within_a_block_data_and_parity_alternate() {
#[test]
fn audio_recovery_refuses_packets_that_are_not_from_the_block() {
let original = audio_block(100, 20);
let parity = protect_audio_block(&original, &STANDARD).unwrap();
let parity = protect_audio_block(&original, &STANDARD, provenance()).unwrap();
let next_block = audio_block(104, 20);
// One past the block's last packet, and one before its first.
for stray in [next_block[0].clone(), audio_block(96, 20)[3].clone()] {
@ -352,7 +352,7 @@ fn audio_recovery_refuses_packets_that_are_not_from_the_block() {
#[test]
fn a_frame_of_several_blocks_recovers_each_block_from_its_own_parity() {
let original = frame(11, 40, 60, 33);
let records = protect_video_frame(&original, &STANDARD).unwrap();
let records = protect_video_frame(&original, &STANDARD, provenance()).unwrap();
let sizes = STANDARD.video_block_sizes(40);
assert_eq!(sizes, vec![14, 13, 13]);
let (data, parity) = split(records);
@ -397,7 +397,7 @@ fn known_answer_audio_parity_bytes() {
for (i, packet) in block.iter_mut().enumerate() {
packet.payload = (0..37).map(|j| (i * 37 + j * 7 + 3) as u8).collect();
}
let parity = protect_audio_block(&block, &STANDARD).unwrap();
let parity = protect_audio_block(&block, &STANDARD, provenance()).unwrap();
assert_eq!(parity[0].fec_scheme, "rs-gf256-v1");
assert_eq!(
hex(&parity[0].payload),
@ -420,7 +420,7 @@ fn known_answer_video_parity_bytes_pad_a_short_final_chunk_with_zeros() {
let len = chunk.payload.len();
chunk.payload = (0..len).map(|j| (usize::from(chunk.chunk_index) * 11 + j * 3 + 1) as u8).collect();
}
let (_, parity) = split(protect_video_frame(&chunks, &STANDARD).unwrap());
let (_, parity) = split(protect_video_frame(&chunks, &STANDARD, provenance()).unwrap());
assert_eq!(parity.len(), 2);
assert_eq!(hex(&parity[0].payload), "1a11081f15", "parity 0");
assert_eq!(hex(&parity[1].payload), "1d18f3ce8b", "parity 1");
@ -432,13 +432,13 @@ fn known_answer_video_parity_bytes_pad_a_short_final_chunk_with_zeros() {
fn parity_is_a_pure_function_of_its_inputs() {
let original = frame(3, 20, 200, 90);
assert_eq!(
protect_video_frame(&original, &STANDARD).unwrap(),
protect_video_frame(&original, &STANDARD).unwrap()
protect_video_frame(&original, &STANDARD, provenance()).unwrap(),
protect_video_frame(&original, &STANDARD, provenance()).unwrap()
);
let block = audio_block(8, 64);
assert_eq!(
protect_audio_block(&block, &STANDARD).unwrap(),
protect_audio_block(&block, &STANDARD).unwrap()
protect_audio_block(&block, &STANDARD, provenance()).unwrap(),
protect_audio_block(&block, &STANDARD, provenance()).unwrap()
);
}
@ -462,71 +462,152 @@ fn the_datagram_budget_is_the_rudp_fragment_the_media_channel_leaves() {
assert_eq!(MEDIA_FEC_MAX_WIRE_BYTES, 1_431);
}
/// Identifiers are the caller's, so the bound is stated for stream and session
/// identifiers, and codec names, of up to this many bytes each.
const IDENTIFIER_BYTES: usize = 32;
/// A provenance whose every field is 32 bytes, the widest the tests claim.
fn widest_provenance() -> MediaWireProvenance<'static> {
let field = |prefix: &str| -> &'static str { Box::leak(long(prefix).into_boxed_str()) };
MediaWireProvenance {
stored_at: field("unix:"),
runtime_id: field("runtime-"),
role: field("role-"),
producer: field("producer-"),
}
}
#[test]
fn every_record_at_the_maximum_shard_size_fits_one_datagram() {
let long = |prefix: &str| format!("{prefix}{}", "x".repeat(IDENTIFIER_BYTES - prefix.len()));
let mut chunks = frame(u64::MAX - 1, 20, MEDIA_FEC_MAX_SHARD_PAYLOAD_BYTES, MEDIA_FEC_MAX_SHARD_PAYLOAD_BYTES);
fn long(prefix: &str) -> String {
format!("{prefix}{}", "x".repeat(32 - prefix.len()))
}
/// Twenty chunks (two blocks) of an odd frame with a dependency, every field
/// at the width MessagePack spends the most bytes on.
fn widest_frame(payload_bytes: usize) -> Vec<GameCultMediaVideoAccessUnitRecord> {
let mut chunks = frame(u64::MAX, 20, payload_bytes, payload_bytes);
for chunk in &mut chunks {
chunk.stream_id = long("stream-");
chunk.session_id = long("session-");
chunk.codec = long("codec-");
chunk.pts_ticks = i64::MIN;
chunk.deadline_ticks = i64::MAX;
chunk.duration_ticks = u32::MAX;
chunk.timebase_num = u32::MAX;
chunk.timebase_den = u32::MAX;
assert!(chunk.dependency_frame_id.is_some() && !chunk.keyframe);
}
let records = protect_video_frame(&chunks, &STANDARD).unwrap();
assert!(records.len() > chunks.len());
let mut largest = 0;
for record in &records {
let wire = encode_media_wire_record(record, provenance()).unwrap();
largest = largest.max(wire.len());
assert!(
wire.len() <= MEDIA_FEC_MAX_WIRE_BYTES,
"{} record is {} bytes, over {MEDIA_FEC_MAX_WIRE_BYTES}",
record.schema_id(),
wire.len()
);
}
chunks
}
let mut packets = audio_block(u64::MAX - 8, MEDIA_FEC_MAX_SHARD_PAYLOAD_BYTES);
for packet in &mut packets {
fn widest_audio(payload_bytes: usize) -> Vec<GameCultMediaAudioPacketRecord> {
let mut packets = audio_block(u64::MAX - 3, payload_bytes);
for (index, packet) in packets.iter_mut().enumerate() {
packet.stream_id = long("stream-");
packet.session_id = long("session-");
packet.codec = long("codec-");
packet.duration_ticks = u32::MAX;
packet.timebase_num = u32::MAX;
packet.timebase_den = u32::MAX;
packet.pts_ticks = i64::MIN + i64::from(u32::MAX) * index as i64;
packet.deadline_ticks = i64::MAX;
}
let mut audio: Vec<_> = packets.iter().cloned().map(GameCultMediaWireRecord::Audio).collect();
audio.extend(protect_audio_block(&packets, &STANDARD).unwrap().into_iter().map(GameCultMediaWireRecord::AudioParity));
for record in &audio {
let wire = encode_media_wire_record(record, provenance()).unwrap();
largest = largest.max(wire.len());
assert!(wire.len() <= MEDIA_FEC_MAX_WIRE_BYTES, "{} is {} bytes", record.schema_id(), wire.len());
}
// The limit is not slack: the largest record leaves less than a full extra
// identifier's worth of headroom.
packets
}
/// Fitting is monotonic in the payload, so the largest payload that fits is a
/// partition point.
fn largest_fitting_payload(fits: impl Fn(usize) -> bool) -> usize {
let payloads: Vec<usize> = (1..=MEDIA_FEC_MAX_WIRE_BYTES).collect();
let count = payloads.partition_point(|&payload| fits(payload));
assert!(count > 0, "no payload fits");
payloads[count - 1]
}
fn wire_len(record: &GameCultMediaWireRecord, provenance: MediaWireProvenance<'_>) -> usize {
encode_media_wire_record(record, provenance).unwrap().len()
}
fn is_over_budget(error: &MediaFecError) -> bool {
matches!(error, MediaFecError::Invalid(message) if message.contains("datagram budget"))
}
/// The invariant is "an encoded record fits one datagram", enforced where the
/// record is produced. At the widest legal identifiers the largest payload that
/// fits is found by search, the record it produces is exactly the budget, and
/// one byte more is refused.
#[test]
fn a_video_frame_is_protected_only_if_every_record_fits_the_datagram_budget() {
let widest = widest_provenance();
let fits = |payload: usize| protect_video_frame(&widest_frame(payload), &STANDARD, widest);
let largest_payload = largest_fitting_payload(|payload| fits(payload).is_ok());
assert!(
largest + 96 > MEDIA_FEC_MAX_WIRE_BYTES,
"the shard limit wastes {} bytes of datagram",
MEDIA_FEC_MAX_WIRE_BYTES - largest
largest_payload >= 600,
"maximum-width identifiers must still leave room for a useful chunk, got {largest_payload}"
);
let records = fits(largest_payload).unwrap();
let largest = records.iter().map(|record| wire_len(record, widest)).max().unwrap();
assert_eq!(largest, MEDIA_FEC_MAX_WIRE_BYTES, "the budget is used to the byte, not approximated");
let error = fits(largest_payload + 1).unwrap_err();
assert!(is_over_budget(&error), "{error}");
// The budget is the policy's, and it is inclusive.
let exact = MediaFecPolicy { max_wire_bytes: largest, ..STANDARD };
assert!(protect_video_frame(&widest_frame(largest_payload), &exact, widest).is_ok());
let one_short = MediaFecPolicy { max_wire_bytes: largest - 1, ..STANDARD };
let error = protect_video_frame(&widest_frame(largest_payload), &one_short, widest).unwrap_err();
assert!(is_over_budget(&error), "{error}");
}
#[test]
fn a_shard_over_the_limit_is_refused_at_the_producer_and_at_the_decoder() {
let too_big = MEDIA_FEC_MAX_SHARD_PAYLOAD_BYTES + 1;
let chunks = frame(1, 3, too_big, too_big);
assert!(matches!(protect_video_frame(&chunks, &STANDARD), Err(MediaFecError::Invalid(_))));
assert!(matches!(
protect_audio_block(&audio_block(1, too_big), &STANDARD),
Err(MediaFecError::Invalid(_))
));
fn an_audio_block_is_protected_only_if_every_record_fits_the_datagram_budget() {
let widest = widest_provenance();
let fits = |payload: usize| protect_audio_block(&widest_audio(payload), &STANDARD, widest);
let largest_payload = largest_fitting_payload(|payload| fits(payload).is_ok());
assert!(largest_payload >= 600, "got {largest_payload}");
let ok = frame(1, 3, MEDIA_FEC_MAX_SHARD_PAYLOAD_BYTES, MEDIA_FEC_MAX_SHARD_PAYLOAD_BYTES);
let (_, mut parity) = split(protect_video_frame(&ok, &STANDARD).unwrap());
parity[0].shard_payload_bytes = too_big as u32;
parity[0].payload = vec![0; too_big];
let wire = encode_media_wire_record(&GameCultMediaWireRecord::VideoParity(parity[0].clone()), provenance()).unwrap();
assert!(decode_media_wire_record(&wire).is_err());
let packets = widest_audio(largest_payload);
let parity = fits(largest_payload).unwrap();
let largest_parity = parity
.iter()
.map(|shard| wire_len(&GameCultMediaWireRecord::AudioParity(shard.clone()), widest))
.max()
.unwrap();
assert_eq!(largest_parity, MEDIA_FEC_MAX_WIRE_BYTES);
// The parity is the largest record of the block, which is why only parity
// is measured.
for packet in &packets {
assert!(wire_len(&GameCultMediaWireRecord::Audio(packet.clone()), widest) < largest_parity);
}
let error = fits(largest_payload + 1).unwrap_err();
assert!(is_over_budget(&error), "{error}");
let one_short = MediaFecPolicy { max_wire_bytes: largest_parity - 1, ..STANDARD };
let error = protect_audio_block(&packets, &one_short, widest).unwrap_err();
assert!(is_over_budget(&error), "{error}");
}
/// A path that carries less than the IPv4 default passes a smaller budget, and
/// a frame protected under the default is refused under it.
#[test]
fn a_smaller_datagram_budget_refuses_what_the_default_admits() {
let chunks = frame(2, 20, 700, 700);
assert!(protect_video_frame(&chunks, &STANDARD, provenance()).is_ok());
let tunnel = MediaFecPolicy { max_wire_bytes: 1_211, ..STANDARD };
let error = protect_video_frame(&chunks, &tunnel, provenance()).unwrap_err();
assert!(is_over_budget(&error), "{error}");
}
/// The record a shard produces is sized by the fields around it, so a shard of
/// any length the budget admits is admitted at the decoder too: a jumbo-frame
/// producer is not stopped by a fixed shard cap.
#[test]
fn a_larger_datagram_budget_admits_larger_shards() {
let jumbo = MediaFecPolicy { max_wire_bytes: 8_959, ..STANDARD };
let chunks = frame(2, 3, 4_000, 4_000);
let records = protect_video_frame(&chunks, &jumbo, provenance()).unwrap();
for record in &records {
let wire = encode_media_wire_record(record, provenance()).unwrap();
assert!(decode_media_wire_record(&wire).is_ok());
}
assert!(protect_video_frame(&chunks, &STANDARD, provenance()).is_err());
}
// ---------------------------------------------------------------------------
@ -537,37 +618,37 @@ fn a_shard_over_the_limit_is_refused_at_the_producer_and_at_the_decoder() {
fn a_block_never_spans_frames() {
let mut chunks = frame(20, 4, 30, 30);
chunks[2].frame_id = 21;
let error = protect_video_frame(&chunks, &STANDARD).unwrap_err();
let error = protect_video_frame(&chunks, &STANDARD, provenance()).unwrap_err();
assert!(matches!(error, MediaFecError::Invalid(_)), "{error}");
let mut chunks = frame(20, 4, 30, 30);
chunks[3].deadline_ticks += 1;
assert!(protect_video_frame(&chunks, &STANDARD).is_err(), "parity would misstate the deadline");
assert!(protect_video_frame(&chunks, &STANDARD, provenance()).is_err(), "parity would misstate the deadline");
let mut chunks = frame(20, 4, 30, 30);
chunks[1].session_id = "session-2".to_string();
assert!(protect_video_frame(&chunks, &STANDARD).is_err());
assert!(protect_video_frame(&chunks, &STANDARD, provenance()).is_err());
}
#[test]
fn a_frame_must_be_whole_and_shard_shaped() {
let mut chunks = frame(30, 4, 30, 30);
chunks.pop();
assert!(protect_video_frame(&chunks, &STANDARD).is_err(), "a missing chunk");
assert!(protect_video_frame(&chunks, &STANDARD, provenance()).is_err(), "a missing chunk");
let mut chunks = frame(30, 4, 30, 30);
chunks[3] = chunks[2].clone();
assert!(protect_video_frame(&chunks, &STANDARD).is_err(), "a repeated chunk");
assert!(protect_video_frame(&chunks, &STANDARD, provenance()).is_err(), "a repeated chunk");
let mut chunks = frame(30, 4, 30, 30);
chunks[1].payload.truncate(29);
assert!(protect_video_frame(&chunks, &STANDARD).is_err(), "a short chunk that is not last");
assert!(protect_video_frame(&chunks, &STANDARD, provenance()).is_err(), "a short chunk that is not last");
let mut chunks = frame(30, 4, 30, 30);
chunks[3].payload.push(0);
assert!(protect_video_frame(&chunks, &STANDARD).is_err(), "a last chunk longer than the shard");
assert!(protect_video_frame(&chunks, &STANDARD, provenance()).is_err(), "a last chunk longer than the shard");
assert!(protect_video_frame(&[], &STANDARD).is_err());
assert!(protect_video_frame(&[], &STANDARD, provenance()).is_err());
}
#[test]
@ -576,45 +657,45 @@ fn chunks_may_arrive_in_any_order() {
let mut shuffled = original.clone();
shuffled.reverse();
assert_eq!(
protect_video_frame(&original, &STANDARD).unwrap(),
protect_video_frame(&shuffled, &STANDARD).unwrap()
protect_video_frame(&original, &STANDARD, provenance()).unwrap(),
protect_video_frame(&shuffled, &STANDARD, provenance()).unwrap()
);
}
#[test]
fn an_audio_block_is_four_contiguous_equal_length_packets() {
assert!(protect_audio_block(&audio_block(1, 20)[..3], &STANDARD).is_err(), "three packets");
assert!(protect_audio_block(&audio_block(1, 20)[..3], &STANDARD, provenance()).is_err(), "three packets");
let mut packets = audio_block(1, 20);
packets[2].packet_id += 1;
assert!(protect_audio_block(&packets, &STANDARD).is_err(), "a gap in packet ids");
assert!(protect_audio_block(&packets, &STANDARD, provenance()).is_err(), "a gap in packet ids");
let mut packets = audio_block(1, 20);
packets[1].pts_ticks += 1;
assert!(protect_audio_block(&packets, &STANDARD).is_err(), "a gap in presentation time");
assert!(protect_audio_block(&packets, &STANDARD, provenance()).is_err(), "a gap in presentation time");
let mut packets = audio_block(1, 20);
packets[3].payload.push(0);
let error = protect_audio_block(&packets, &STANDARD).unwrap_err();
let error = protect_audio_block(&packets, &STANDARD, provenance()).unwrap_err();
assert!(error.to_string().contains("payload length"), "unequal payload lengths: {error}");
let mut packets = audio_block(1, 20);
packets[0].codec = "pcm".to_string();
assert!(protect_audio_block(&packets, &STANDARD).is_err(), "mixed codecs");
assert!(protect_audio_block(&packets, &STANDARD, provenance()).is_err(), "mixed codecs");
let mut packets = audio_block(1, 20);
packets[2].session_id = "session-2".to_string();
assert!(protect_audio_block(&packets, &STANDARD).is_err(), "mixed sessions");
assert!(protect_audio_block(&packets, &STANDARD, provenance()).is_err(), "mixed sessions");
let mut packets = audio_block(1, 20);
packets[2].stream_id = "other".to_string();
assert!(protect_audio_block(&packets, &STANDARD).is_err(), "mixed streams");
assert!(protect_audio_block(&packets, &STANDARD, provenance()).is_err(), "mixed streams");
let mut packets = audio_block(1, 20);
packets[1].timebase_den = 44_100;
assert!(protect_audio_block(&packets, &STANDARD).is_err(), "mixed timebases");
assert!(protect_audio_block(&packets, &STANDARD, provenance()).is_err(), "mixed timebases");
let mut packets = audio_block(1, 20);
packets[3].duration_ticks = 480;
assert!(protect_audio_block(&packets, &STANDARD).is_err(), "mixed durations");
assert!(protect_audio_block(&packets, &STANDARD, provenance()).is_err(), "mixed durations");
}
#[test]
fn parity_carries_its_frames_deadline_unchanged() {
let original = frame(40, 40, 50, 50);
let (_, parity) = split(protect_video_frame(&original, &STANDARD).unwrap());
let (_, parity) = split(protect_video_frame(&original, &STANDARD, provenance()).unwrap());
assert!(!parity.is_empty());
for shard in &parity {
assert_eq!(shard.deadline_ticks, original[0].deadline_ticks);
@ -628,7 +709,7 @@ fn parity_carries_its_frames_deadline_unchanged() {
fn audio_parity_carries_the_latest_deadline_in_its_block_and_recovery_uses_it() {
let mut packets = audio_block(50, 30);
packets[1].deadline_ticks = 99_999; // the block's latest, not its last
let parity = protect_audio_block(&packets, &STANDARD).unwrap();
let parity = protect_audio_block(&packets, &STANDARD, provenance()).unwrap();
assert!(parity.iter().all(|shard| shard.deadline_ticks == 99_999));
let held = vec![packets[0].clone(), packets[2].clone(), packets[3].clone()];
@ -650,7 +731,7 @@ fn audio_parity_carries_the_latest_deadline_in_its_block_and_recovery_uses_it()
fn send_order_is_a_permutation_of_the_frames_records() {
for n in [1_u16, 2, 15, 16, 17, 31, 32, 33, 47, 100] {
let original = frame(60, n, 20, 9);
let records = protect_video_frame(&original, &STANDARD).unwrap();
let records = protect_video_frame(&original, &STANDARD, provenance()).unwrap();
let sizes = STANDARD.video_block_sizes(n);
let expected_parity: usize = sizes.iter().map(|&k| usize::from(STANDARD.video_parity_shards(k))).sum();
assert_eq!(records.len(), usize::from(n) + expected_parity, "n = {n}");
@ -672,7 +753,7 @@ fn equal_blocks_send_one_record_per_block_before_any_block_repeats() {
let sizes = STANDARD.video_block_sizes(n);
let blocks = sizes.len();
assert!(sizes.windows(2).all(|pair| pair[0] == pair[1]), "n = {n} must split equally");
let records = protect_video_frame(&frame(70, n, 20, 20), &STANDARD).unwrap();
let records = protect_video_frame(&frame(70, n, 20, 20), &STANDARD, provenance()).unwrap();
let order: Vec<usize> = records.iter().map(|r| block_of(r, &sizes)).collect();
for window in order.windows(blocks) {
let mut seen = window.to_vec();
@ -691,7 +772,7 @@ fn unequal_blocks_keep_every_block_spaced_by_the_blocks_still_sending() {
for n in [17_u16, 19, 35, 40, 50, 100] {
let sizes = STANDARD.video_block_sizes(n);
assert!(sizes.windows(2).any(|pair| pair[0] != pair[1]), "n = {n} must split unequally");
let records = protect_video_frame(&frame(71, n, 20, 20), &STANDARD).unwrap();
let records = protect_video_frame(&frame(71, n, 20, 20), &STANDARD, provenance()).unwrap();
let order: Vec<usize> = records.iter().map(|r| block_of(r, &sizes)).collect();
for later in 0..order.len() {
let Some(earlier) = (0..later).rev().find(|&i| order[i] == order[later]) else { continue };
@ -716,7 +797,7 @@ fn unequal_blocks_keep_every_block_spaced_by_the_blocks_still_sending() {
fn a_frame_survives_the_wire_and_recovers_from_what_arrives() {
let original = frame(80, 20, 300, 111);
let sizes = STANDARD.video_block_sizes(20);
let records = protect_video_frame(&original, &STANDARD).unwrap();
let records = protect_video_frame(&original, &STANDARD, provenance()).unwrap();
// The network eats a burst of four consecutive records.
let arrived: Vec<GameCultMediaWireRecord> = records
@ -759,7 +840,7 @@ fn a_custom_policy_sizes_blocks_by_its_own_fields() {
assert_eq!(policy.video_block_sizes(20), vec![7, 7, 6]);
assert_eq!(policy.video_parity_shards(7), 4);
assert_eq!(policy.video_parity_shards(1), 1);
let (data, parity) = split(protect_video_frame(&frame(90, 20, 20, 20), &policy).unwrap());
let (data, parity) = split(protect_video_frame(&frame(90, 20, 20, 20), &policy, provenance()).unwrap());
assert_eq!(data.len(), 20);
assert_eq!(parity.len(), 4 + 4 + 3);
assert_eq!(MediaFecPolicy::default(), STANDARD);
@ -768,23 +849,23 @@ fn a_custom_policy_sizes_blocks_by_its_own_fields() {
#[test]
fn a_policy_that_cannot_form_a_block_is_refused() {
let zero = MediaFecPolicy { video_min_parity_shards: 0, ..STANDARD };
assert!(protect_video_frame(&frame(1, 2, 10, 10), &zero).is_err());
assert!(protect_video_frame(&frame(1, 2, 10, 10), &zero, provenance()).is_err());
let no_divisor = MediaFecPolicy { video_parity_divisor: 0, ..STANDARD };
assert!(protect_video_frame(&frame(1, 2, 10, 10), &no_divisor).is_err());
assert!(protect_video_frame(&frame(1, 2, 10, 10), &no_divisor, provenance()).is_err());
let no_audio = MediaFecPolicy { audio_data_shards: 0, ..STANDARD };
assert!(protect_audio_block(&audio_block(1, 10), &no_audio).is_err());
assert!(protect_audio_block(&audio_block(1, 10), &no_audio, provenance()).is_err());
let no_audio_parity = MediaFecPolicy { audio_parity_shards: 0, ..STANDARD };
assert!(protect_audio_block(&audio_block(1, 10), &no_audio_parity).is_err());
assert!(protect_audio_block(&audio_block(1, 10), &no_audio_parity, provenance()).is_err());
let too_wide = MediaFecPolicy { video_max_block_data_shards: 255, ..STANDARD };
assert!(protect_video_frame(&frame(1, 2, 10, 10), &too_wide).is_err());
assert!(protect_video_frame(&frame(1, 2, 10, 10), &too_wide, provenance()).is_err());
let audio_too_wide = MediaFecPolicy { audio_data_shards: 255, audio_parity_shards: 2, ..STANDARD };
assert!(protect_audio_block(&audio_block(1, 10), &audio_too_wide).is_err());
assert!(protect_audio_block(&audio_block(1, 10), &audio_too_wide, provenance()).is_err());
}
#[test]
fn parity_records_survive_the_positional_encoding_unchanged() {
let (_, parity) = split(protect_video_frame(&frame(5, 5, 40, 17), &STANDARD).unwrap());
let audio = protect_audio_block(&audio_block(5, 25), &STANDARD).unwrap();
let (_, parity) = split(protect_video_frame(&frame(5, 5, 40, 17), &STANDARD, provenance()).unwrap());
let audio = protect_audio_block(&audio_block(5, 25), &STANDARD, provenance()).unwrap();
for shard in parity {
let bytes = rmp_serde::to_vec(&shard).unwrap();
assert_eq!(rmp_serde::from_slice::<GameCultMediaVideoParityShardRecord>(&bytes).unwrap(), shard);
@ -794,3 +875,178 @@ fn parity_records_survive_the_positional_encoding_unchanged() {
assert_eq!(rmp_serde::from_slice::<GameCultMediaAudioParityShardRecord>(&bytes).unwrap(), shard);
}
}
// ---------------------------------------------------------------------------
// Rulings pinned: policy bounds, block identity, audio arithmetic, bursts
// ---------------------------------------------------------------------------
fn is_invalid_mentioning(error: &MediaFecError, needle: &str) -> bool {
matches!(error, MediaFecError::Invalid(message) if message.contains(needle))
}
fn audio_run(count: u64, payload_bytes: usize) -> Vec<GameCultMediaAudioPacketRecord> {
(0..count)
.map(|index| GameCultMediaAudioPacketRecord {
stream_id: "muninn.raven.av.rudp".to_string(),
session_id: "session-1".to_string(),
packet_id: 500 + index,
codec: "opus".to_string(),
pts_ticks: 960 * index as i64,
duration_ticks: 960,
timebase_num: 1,
timebase_den: 48_000,
deadline_ticks: 960 * index as i64 + 5_000,
payload: bytes(index as u32, payload_bytes),
})
.collect()
}
/// A block is at most 256 shards, data and parity together, and a block of
/// exactly 256 is admitted.
#[test]
fn a_policy_admits_256_shards_per_block_and_refuses_257() {
let audio_ok = MediaFecPolicy { audio_data_shards: 250, audio_parity_shards: 6, ..STANDARD };
let parity = protect_audio_block(&audio_run(250, 16), &audio_ok, provenance()).unwrap();
assert_eq!(parity.len(), 6);
let audio_wide = MediaFecPolicy { audio_data_shards: 250, audio_parity_shards: 7, ..STANDARD };
let error = protect_audio_block(&audio_run(250, 16), &audio_wide, provenance()).unwrap_err();
assert!(is_invalid_mentioning(&error, "256 shards"), "{error}");
// One video block of 250 chunks with 6 parity shards.
let video_ok = MediaFecPolicy {
video_max_block_data_shards: 250,
video_min_parity_shards: 6,
video_parity_divisor: 100,
..STANDARD
};
let records = protect_video_frame(&frame(1, 250, 4, 4), &video_ok, provenance()).unwrap();
assert_eq!(records.len(), 256);
let video_wide = MediaFecPolicy { video_min_parity_shards: 7, ..video_ok };
let error = protect_video_frame(&frame(1, 250, 4, 4), &video_wide, provenance()).unwrap_err();
assert!(is_invalid_mentioning(&error, "256 shards"), "{error}");
}
/// Two parity shards that differ only in where the block sits in the frame are
/// not one block, even when each is a well-formed record.
#[test]
fn parity_shards_that_differ_only_in_block_count_or_start_do_not_combine() {
let (_, parity) = split(protect_video_frame(&frame(7, 32, 40, 40), &STANDARD, provenance()).unwrap());
let block_zero: Vec<_> = parity.iter().filter(|p| p.block_index == 0).cloned().collect();
let mut more_blocks = block_zero[1].clone();
more_blocks.block_count += 1;
let error = recover_video_block(&[block_zero[0].clone(), more_blocks], &[]).unwrap_err();
assert!(is_invalid_mentioning(&error, "one block"), "block_count: {error}");
let mut shifted = block_zero[1].clone();
shifted.block_data_start += 1;
let error = recover_video_block(&[block_zero[0].clone(), shifted], &[]).unwrap_err();
assert!(is_invalid_mentioning(&error, "one block"), "block_data_start: {error}");
}
/// The largest packet id and pts a block may end on are admitted and recover
/// to exactly those values; one tick or id more is refused everywhere: at the
/// wire, in recovery, and at the producer.
#[test]
fn an_audio_block_that_would_overflow_its_last_id_or_pts_is_refused() {
// Id: the block's last packet is u64::MAX.
let base = u64::MAX - 3;
let packets = audio_block(base, 20);
let parity = protect_audio_block(&packets, &STANDARD, provenance()).unwrap();
let recovered = recover_audio_block(&parity, &packets[..3]).unwrap();
assert_eq!(recovered.len(), 1);
assert_eq!(recovered[0].packet_id, u64::MAX);
assert_eq!(recovered[0], packets[3]);
let mut past_end = parity.clone();
for shard in &mut past_end {
shard.base_packet_id += 1;
}
let error = recover_audio_block(&past_end, &[]).unwrap_err();
assert!(is_invalid_mentioning(&error, "packet id or pts range"), "{error}");
let wire = encode_media_wire_record(
&GameCultMediaWireRecord::AudioParity(past_end[0].clone()),
provenance(),
)
.unwrap();
assert!(decode_media_wire_record(&wire).unwrap_err().to_string().contains("packet id or pts range"));
let mut overflowing = audio_block(base, 20);
overflowing[0].packet_id += 1;
assert!(protect_audio_block(&overflowing, &STANDARD, provenance()).is_err());
// Pts: the block's last packet ends on i64::MAX.
let mut packets = audio_block(9, 20);
let first_pts = i64::MAX - 3 * 960;
for (index, packet) in packets.iter_mut().enumerate() {
packet.pts_ticks = first_pts + 960 * index as i64;
packet.deadline_ticks = i64::MAX;
}
let parity = protect_audio_block(&packets, &STANDARD, provenance()).unwrap();
let recovered = recover_audio_block(&parity, &packets[..3]).unwrap();
assert_eq!(recovered[0].pts_ticks, i64::MAX);
assert_eq!(recovered[0], packets[3]);
let mut past_end = parity.clone();
for shard in &mut past_end {
shard.base_pts_ticks += 1;
}
let error = recover_audio_block(&past_end, &[]).unwrap_err();
assert!(is_invalid_mentioning(&error, "packet id or pts range"), "{error}");
let mut overflowing = packets.clone();
overflowing[1].pts_ticks += 1;
assert!(protect_audio_block(&overflowing, &STANDARD, provenance()).is_err());
}
/// Sixteen 8-byte chunks, the widest standard video block (m = 4).
/// `payload[i][j] = (i * 29 + j * 13 + 5) mod 256`. The expected bytes come
/// from an independent implementation of the construction in the module docs.
#[test]
fn known_answer_video_parity_bytes_for_k16_m4() {
let mut chunks = frame(2, 16, 8, 8);
for chunk in &mut chunks {
let i = usize::from(chunk.chunk_index);
chunk.payload = (0..8).map(|j| (i * 29 + j * 13 + 5) as u8).collect();
}
let (_, parity) = split(protect_video_frame(&chunks, &STANDARD, provenance()).unwrap());
let hexes: Vec<String> = parity.iter().map(|shard| hex(&shard.payload)).collect();
assert_eq!(
hexes,
["5ed2f608c328adbe", "e46d011eabb600e2", "7c1168f2ba8a2343", "4f03d220d182f232"]
);
}
/// Blocks interleave, so a burst of consecutive losses costs each block a
/// share: the longest burst that leaves every frame whole, wherever it starts,
/// is the block count times the smallest parity count among the blocks.
#[test]
fn the_worst_burst_that_always_recovers_is_blocks_times_the_smallest_m() {
for chunk_count in 1..=399_u16 {
let sizes = STANDARD.video_block_sizes(chunk_count);
let parity_of: Vec<usize> =
sizes.iter().map(|&size| usize::from(STANDARD.video_parity_shards(size))).collect();
let records = protect_video_frame(&frame(u64::from(chunk_count), chunk_count, 1, 1), &STANDARD, provenance())
.unwrap();
let blocks: Vec<usize> = records.iter().map(|record| block_of(record, &sizes)).collect();
// The shortest burst, over every start, that costs some block more
// than its m; the burst one shorter always recovers.
let mut shortest_failing = usize::MAX;
for start in 0..blocks.len() {
let mut lost = vec![0_usize; sizes.len()];
for (offset, &block) in blocks[start..].iter().enumerate() {
lost[block] += 1;
if lost[block] > parity_of[block] {
shortest_failing = shortest_failing.min(offset + 1);
break;
}
}
}
let smallest_m = *parity_of.iter().min().unwrap();
assert_eq!(
shortest_failing - 1,
sizes.len() * smallest_m,
"{chunk_count} chunks in {} blocks",
sizes.len()
);
}
}

View file

@ -18,7 +18,7 @@ use std::path::Path;
use cultnet_rs::{
GameCultMediaVideoAccessUnitRecord, GameCultMediaWireRecord, MediaFecError, MediaFecPolicy,
protect_video_frame, recover_video_block,
MediaWireProvenance, protect_video_frame, recover_video_block,
};
use policy::{Profile, Scheduler};
@ -26,6 +26,15 @@ const POLICY: MediaFecPolicy = MediaFecPolicy::STANDARD;
const SEED: u64 = 20_260_930;
const FRAMES: u64 = 200;
fn provenance() -> MediaWireProvenance<'static> {
MediaWireProvenance {
stored_at: "unix:1700000000000",
runtime_id: "raven-muninn-primary",
role: "muninn.media",
producer: "muninn",
}
}
fn frame(frame_id: u64, chunk_count: u16) -> Vec<GameCultMediaVideoAccessUnitRecord> {
(0..chunk_count)
.map(|index| GameCultMediaVideoAccessUnitRecord {
@ -67,7 +76,7 @@ fn run(profile: &Profile, chunk_count: u16) -> Tally {
for frame_id in 0..FRAMES {
let original = frame(frame_id, chunk_count);
let records = protect_video_frame(&original, &POLICY).unwrap();
let records = protect_video_frame(&original, &POLICY, provenance()).unwrap();
let survivors: Vec<GameCultMediaWireRecord> = records
.into_iter()
.filter(|_| !scheduler.decide(0).drop)

View file

@ -342,10 +342,6 @@ fn a_malformed_video_parity_record_is_refused_with_its_reason() {
r.shard_payload_bytes = 0;
r.payload.clear();
})),
("shard_payload_bytes", Box::new(|r| {
r.shard_payload_bytes = 721;
r.payload = vec![0; 721];
})),
("last chunk length", Box::new(|r| r.last_chunk_payload_bytes = 0)),
("last chunk length", Box::new(|r| r.last_chunk_payload_bytes = 5)),
("declared shard length", Box::new(|r| r.payload.push(0))),
@ -375,9 +371,10 @@ fn a_malformed_audio_parity_record_is_refused_with_its_reason() {
("stripe", Box::new(|r| r.data_shard_count = 255)),
("timing", Box::new(|r| r.packet_duration_ticks = 0)),
("base_pts_ticks", Box::new(|r| r.deadline_ticks = -1)),
("shard_payload_bytes", Box::new(|r| {
r.shard_payload_bytes = 721;
r.payload = vec![0; 721];
("packet id or pts range", Box::new(|r| r.base_packet_id = u64::MAX - 2)),
("packet id or pts range", Box::new(|r| {
r.base_pts_ticks = i64::MAX - 960 * 3 + 1;
r.deadline_ticks = i64::MAX;
})),
("declared shard length", Box::new(|r| { r.payload.pop(); })),
("codec", Box::new(|r| r.codec = String::new())),
@ -394,22 +391,6 @@ fn a_malformed_audio_parity_record_is_refused_with_its_reason() {
}
}
/// The shard limit is inclusive: a shard of exactly the largest size is
/// admitted, one byte more is not.
#[test]
fn a_parity_shard_at_the_size_limit_is_admitted() {
use cultnet_rs::MEDIA_FEC_MAX_SHARD_PAYLOAD_BYTES as MAX;
let mut video = video_parity();
video.shard_payload_bytes = MAX as u32;
video.last_chunk_payload_bytes = MAX as u32;
video.payload = vec![0; MAX];
assert!(decodes(GameCultMediaWireRecord::VideoParity(video)).is_ok());
let mut audio = audio_parity();
audio.shard_payload_bytes = MAX as u32;
audio.payload = vec![0; MAX];
assert!(decodes(GameCultMediaWireRecord::AudioParity(audio)).is_ok());
}
/// The 256-shard ceiling and the deadline floor are inclusive on the side that
/// keeps a valid record valid.
#[test]
@ -429,4 +410,13 @@ fn records_on_the_edge_of_each_bound_are_admitted() {
let mut audio = audio_parity();
audio.deadline_ticks = audio.base_pts_ticks;
assert!(decodes(GameCultMediaWireRecord::AudioParity(audio)).is_ok(), "a deadline at the base pts");
let mut audio = audio_parity();
audio.base_packet_id = u64::MAX - 3;
assert!(decodes(GameCultMediaWireRecord::AudioParity(audio)).is_ok(), "the last packet id is u64::MAX");
let mut audio = audio_parity();
audio.base_pts_ticks = i64::MAX - 960 * 3;
audio.deadline_ticks = i64::MAX;
assert!(decodes(GameCultMediaWireRecord::AudioParity(audio)).is_ok(), "the last pts is i64::MAX");
}