diff --git a/docs/implementation-plan.md b/docs/implementation-plan.md index 6010b14..6e242a3 100644 --- a/docs/implementation-plan.md +++ b/docs/implementation-plan.md @@ -38,6 +38,10 @@ a named invariant that the native runtime cannot protect yet. Starfire and Nightwing both have USB-attached Moves. Muninn daemons publish source-local glowing-orb marker candidates and controller/IMU/button state; Odin owns discovery and schema projection, not pose fusion. +- `native/reservoir` exposes a `move_evidence` typed view and + `LocalcastMoveEvidenceBufferDescriptor` so Muninn Move witnesses can be + admitted as compute-upload buffers before Mimir's fusion pass and Fensalir's + GPU consumers touch them. - `mimir.move_controller_pose.v1` is Mimir's resolved wand pose contract. Mimir owns calibration, association, triangulation, IMU fusion, prediction, confidence, and latency accounting for those poses before Fensalir consumes diff --git a/docs/perfect-machine-module-library.md b/docs/perfect-machine-module-library.md index 1e30e44..a057446 100644 --- a/docs/perfect-machine-module-library.md +++ b/docs/perfect-machine-module-library.md @@ -14,7 +14,7 @@ job-interview answer for why it belongs in the body of the machine. | Alignment actuator | `Mimir.Runtime` + Faust | Converts sync state into bounded fractional-delay, gain, and sample-rate-offset controls; Faust moves samples. | `src/Mimir.Runtime/Synchronization/MimirAudioAlignmentActuator.cs` | | Audio actuator strategies | `Mimir.Runtime` + Faust | Integer baseline, Farrow fractional delay, variable ASRC, and hybrid delay/ASRC strategy profiles. | `src/Mimir.Runtime/Synchronization/MimirAudioActuatorConfiguration.cs` | | CultMesh contracts | `Mimir.Runtime` | Typed codebook, decoder, acoustic path, actuator, program, Move evidence, and Mimir-fused Move controller pose documents for distributed Mimir nodes. | `src/Mimir.Runtime/Synchronization/MimirCultMeshContracts.cs`, `src/Mimir.Runtime/Synchronization/MimirTrackingObservation.cs` | -| Move controller fusion | `Mimir.Runtime` + Fensalir | Mimir fuses Muninn marker candidates plus controller/IMU/button feeds into `mimir.move_controller_pose.v1`; Fensalir consumes the resolved pose stream for interactive environments. | `src/Mimir.Runtime/Synchronization/MimirTrackingObservation.cs` | +| Move controller fusion | `Mimir.Runtime` + native reservoir + Fensalir | Mimir admits Muninn marker candidates plus controller/IMU/button feeds into native `move_evidence` buffers, fuses them into `mimir.move_controller_pose.v1`, and Fensalir consumes the resolved pose stream for interactive environments. | `src/Mimir.Runtime/Synchronization/MimirTrackingObservation.cs`, `native/reservoir/src/lib.rs` | | Acoustic path learning | `Mimir.Runtime` | Calibration stages for usable bands, confusion, global delay, group delay, and codebook adaptation. | `src/Mimir.Runtime/Synchronization/MimirAcousticPathLearningConfiguration.cs` | | Acoustic localization | `Mimir.Runtime` + Fensalir | Pairwise TDOA, SRP-PHAT grid, sparse source, and visual-constrained localization profiles plus a small SRP/TDOA scorer. | `src/Mimir.Runtime/Synchronization/MimirAcousticLocalizationConfiguration.cs`, `src/Mimir.Runtime/Synchronization/MimirAcousticLocalizationSolver.cs` | | Benchmark panels | `Mimir.Runtime` | Decoder golf and meatspace acceptance degradation panels with receipt roots and thresholds. | `src/Mimir.Runtime/Synchronization/MimirBenchmarkPanelConfiguration.cs` | @@ -45,10 +45,10 @@ job-interview answer for why it belongs in the body of the machine. but they do not become independent clock authorities. - USB-attached Moves on Starfire and Nightwing publish source-local Muninn evidence streams: glowing-orb marker candidates and controller/IMU/button - state. Odin discovers/projects those streams. Mimir owns buffering, - calibration, association, triangulation, IMU fusion, prediction, and - `mimir.move_controller_pose.v1` publication. Fensalir consumes that resolved - pose stream for interactive environments. + state. Odin discovers/projects those streams. Mimir owns native evidence + buffer admission/layout, calibration, association, triangulation, IMU fusion, + prediction, and `mimir.move_controller_pose.v1` publication. Fensalir + consumes that resolved pose stream for interactive environments. - OBS may receive compatibility program surfaces and stems. It does not own calibration, composition, preview/control, or publication. diff --git a/native/reservoir/include/localcast_reservoir.h b/native/reservoir/include/localcast_reservoir.h index e1f5122..8ae6122 100644 --- a/native/reservoir/include/localcast_reservoir.h +++ b/native/reservoir/include/localcast_reservoir.h @@ -17,7 +17,8 @@ enum LocalcastSampleKind { LOCALCAST_SAMPLE_AUDIO_BLOCK = 5, LOCALCAST_SAMPLE_PHASE_CLAIM = 6, LOCALCAST_SAMPLE_EVENT_CLAIM = 7, - LOCALCAST_SAMPLE_RENDER_PACKET = 8 + LOCALCAST_SAMPLE_RENDER_PACKET = 8, + LOCALCAST_SAMPLE_MOVE_EVIDENCE = 9 }; typedef struct LocalcastReservoir LocalcastReservoir; @@ -79,6 +80,47 @@ typedef struct LocalcastRenderPoint { float confidence; } LocalcastRenderPoint; +enum LocalcastMoveEvidenceKind { + LOCALCAST_MOVE_EVIDENCE_OPTICAL_MARKER = 1, + LOCALCAST_MOVE_EVIDENCE_CONTROLLER_STATE = 2 +}; + +typedef struct LocalcastMoveEvidenceSample { + uint64_t witness_id_hash; + uint64_t controller_id_hash; + uint64_t source_timestamp_ns; + uint64_t arrival_ns; + uint64_t sequence; + uint32_t evidence_kind; + uint32_t flags; + float image_x; + float image_y; + float radius_px; + float confidence; + float accel_x; + float accel_y; + float accel_z; + float gyro_x; + float gyro_y; + float gyro_z; + float trigger; + uint32_t buttons_mask; + uint32_t reserved; + float battery01; + uint32_t reserved1; + uint32_t reserved2; +} LocalcastMoveEvidenceSample; + +typedef struct LocalcastMoveEvidenceBufferDescriptor { + uint64_t sample_buffer_handle; + uint32_t sample_count; + uint32_t sample_stride_bytes; + uint64_t source_time_min_ns; + uint64_t source_time_max_ns; + uint64_t calibration_hash; + uint64_t tracking_space_hash; +} LocalcastMoveEvidenceBufferDescriptor; + typedef struct LocalcastRuntimeStatus { uint64_t edge_ns; uint64_t window_start_ns; @@ -92,6 +134,7 @@ typedef struct LocalcastRuntimeStatus { size_t phase_claim_count; size_t event_claim_count; size_t render_packet_count; + size_t move_evidence_count; } LocalcastRuntimeStatus; uint64_t localcast_hash_source_id(const uint8_t *bytes, size_t byte_len); @@ -144,6 +187,7 @@ bool localcast_runtime_push_audio_block(LocalcastRuntime *runtime, LocalcastSamp bool localcast_runtime_push_phase_claim(LocalcastRuntime *runtime, LocalcastSampleHandle sample); bool localcast_runtime_push_event_claim(LocalcastRuntime *runtime, LocalcastSampleHandle sample); bool localcast_runtime_push_render_packet(LocalcastRuntime *runtime, LocalcastSampleHandle sample); +bool localcast_runtime_push_move_evidence(LocalcastRuntime *runtime, LocalcastSampleHandle sample); bool localcast_runtime_status(const LocalcastRuntime *runtime, LocalcastRuntimeStatus *out_status); size_t localcast_runtime_len(const LocalcastRuntime *runtime); size_t localcast_runtime_view_len(const LocalcastRuntime *runtime, uint32_t sample_kind); @@ -210,6 +254,15 @@ bool localcast_producer_push_render_packet( LocalcastSampleHandle *out_sample ); +bool localcast_producer_push_move_evidence_buffer( + LocalcastProducer *producer, + LocalcastRuntime *runtime, + uint64_t timestamp_ns, + uint64_t arrival_ns, + const LocalcastMoveEvidenceBufferDescriptor *descriptor, + LocalcastSampleHandle *out_sample +); + #ifdef __cplusplus } #endif diff --git a/native/reservoir/src/lib.rs b/native/reservoir/src/lib.rs index afd5cdb..71e42cc 100644 --- a/native/reservoir/src/lib.rs +++ b/native/reservoir/src/lib.rs @@ -17,6 +17,7 @@ pub enum SampleKind { PhaseClaim, EventClaim, RenderPacket, + MoveEvidence, } impl SampleKind { @@ -31,6 +32,7 @@ impl SampleKind { 6 => Some(Self::PhaseClaim), 7 => Some(Self::EventClaim), 8 => Some(Self::RenderPacket), + 9 => Some(Self::MoveEvidence), _ => None, } } @@ -46,6 +48,7 @@ impl SampleKind { Self::PhaseClaim => 6, Self::EventClaim => 7, Self::RenderPacket => 8, + Self::MoveEvidence => 9, } } } @@ -229,6 +232,53 @@ pub struct LocalcastRenderPoint { pub confidence: f32, } +#[repr(C)] +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum LocalcastMoveEvidenceKind { + OpticalMarker = 1, + ControllerState = 2, +} + +#[repr(C)] +#[derive(Debug, Clone, Copy, PartialEq)] +pub struct LocalcastMoveEvidenceSample { + pub witness_id_hash: u64, + pub controller_id_hash: u64, + pub source_timestamp_ns: u64, + pub arrival_ns: u64, + pub sequence: u64, + pub evidence_kind: u32, + pub flags: u32, + pub image_x: f32, + pub image_y: f32, + pub radius_px: f32, + pub confidence: f32, + pub accel_x: f32, + pub accel_y: f32, + pub accel_z: f32, + pub gyro_x: f32, + pub gyro_y: f32, + pub gyro_z: f32, + pub trigger: f32, + pub buttons_mask: u32, + pub reserved: u32, + pub battery01: f32, + pub reserved1: u32, + pub reserved2: u32, +} + +#[repr(C)] +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct LocalcastMoveEvidenceBufferDescriptor { + pub sample_buffer_handle: u64, + pub sample_count: u32, + pub sample_stride_bytes: u32, + pub source_time_min_ns: u64, + pub source_time_max_ns: u64, + pub calibration_hash: u64, + pub tracking_space_hash: u64, +} + pub struct LocalcastReservoir { inner: RollingReservoir, } @@ -258,6 +308,7 @@ pub struct LocalcastRuntimeStatus { pub phase_claim_count: usize, pub event_claim_count: usize, pub render_packet_count: usize, + pub move_evidence_count: usize, } impl Default for LocalcastRuntimeStatus { @@ -275,6 +326,7 @@ impl Default for LocalcastRuntimeStatus { phase_claim_count: 0, event_claim_count: 0, render_packet_count: 0, + move_evidence_count: 0, } } } @@ -308,6 +360,7 @@ impl LocalcastRuntime { phase_claim_count: self.view_len(SampleKind::PhaseClaim), event_claim_count: self.view_len(SampleKind::EventClaim), render_packet_count: self.view_len(SampleKind::RenderPacket), + move_evidence_count: self.view_len(SampleKind::MoveEvidence), } } } @@ -623,6 +676,14 @@ pub extern "C" fn localcast_runtime_push_render_packet( localcast_runtime_push_typed(ptr, SampleKind::RenderPacket, sample) } +#[unsafe(no_mangle)] +pub extern "C" fn localcast_runtime_push_move_evidence( + ptr: *mut LocalcastRuntime, + sample: LocalcastSampleHandle, +) -> bool { + localcast_runtime_push_typed(ptr, SampleKind::MoveEvidence, sample) +} + #[unsafe(no_mangle)] pub extern "C" fn localcast_runtime_status( ptr: *const LocalcastRuntime, @@ -860,6 +921,46 @@ pub extern "C" fn localcast_producer_push_render_packet( ) } +#[unsafe(no_mangle)] +pub extern "C" fn localcast_producer_push_move_evidence_buffer( + producer_ptr: *mut LocalcastProducer, + runtime_ptr: *mut LocalcastRuntime, + timestamp_ns: u64, + arrival_ns: u64, + descriptor: *const LocalcastMoveEvidenceBufferDescriptor, + out_sample: *mut LocalcastSampleHandle, +) -> bool { + let Some(producer) = (unsafe { producer_ptr.as_mut() }) else { + return false; + }; + if producer.kind != SampleKind::MoveEvidence { + return false; + } + let Some(descriptor_ref) = (unsafe { descriptor.as_ref() }) else { + return false; + }; + if descriptor_ref.sample_buffer_handle == 0 { + return false; + } + if descriptor_ref.sample_count == 0 { + return false; + } + if descriptor_ref.sample_stride_bytes as usize + != std::mem::size_of::() + { + return false; + } + let payload_handle = descriptor as u64; + localcast_producer_push( + producer_ptr, + runtime_ptr, + timestamp_ns, + arrival_ns, + payload_handle, + out_sample, + ) +} + fn localcast_runtime_push_typed( ptr: *mut LocalcastRuntime, kind: SampleKind, @@ -943,6 +1044,51 @@ mod tests { } } + fn move_evidence_descriptor( + sample_buffer_handle: u64, + ) -> LocalcastMoveEvidenceBufferDescriptor { + LocalcastMoveEvidenceBufferDescriptor { + sample_buffer_handle, + sample_count: 2, + sample_stride_bytes: std::mem::size_of::() as u32, + source_time_min_ns: 1_000, + source_time_max_ns: 1_120, + calibration_hash: 0xCA11_BA7E, + tracking_space_hash: 0x57A6E, + } + } + + fn move_marker_sample( + witness_id_hash: u64, + controller_id_hash: u64, + ) -> LocalcastMoveEvidenceSample { + LocalcastMoveEvidenceSample { + witness_id_hash, + controller_id_hash, + source_timestamp_ns: 1_000, + arrival_ns: 1_010, + sequence: 7, + evidence_kind: LocalcastMoveEvidenceKind::OpticalMarker as u32, + flags: 0, + image_x: 123.5, + image_y: 87.25, + radius_px: 14.0, + confidence: 0.92, + accel_x: 0.0, + accel_y: 0.0, + accel_z: 0.0, + gyro_x: 0.0, + gyro_y: 0.0, + gyro_z: 0.0, + trigger: 0.0, + buttons_mask: 0, + reserved: 0, + battery01: f32::NAN, + reserved1: 0, + reserved2: 0, + } + } + fn render_point(stable_key_hash: u64) -> LocalcastRenderPoint { LocalcastRenderPoint { stable_key_hash, @@ -1585,6 +1731,125 @@ mod tests { localcast_runtime_destroy(runtime); } + #[test] + fn move_evidence_layout_is_fixed_for_compute_upload() { + let sample = move_marker_sample(11, 22); + + assert_eq!(std::mem::size_of::(), 112); + assert_eq!( + std::mem::size_of::(), + 48 + ); + assert_eq!(11, sample.witness_id_hash); + assert_eq!(22, sample.controller_id_hash); + assert_eq!( + LocalcastMoveEvidenceKind::OpticalMarker as u32, + sample.evidence_kind + ); + assert_eq!(123.5, sample.image_x); + assert_eq!(14.0, sample.radius_px); + } + + #[test] + fn move_evidence_descriptor_enters_native_runtime_window() { + let runtime = localcast_runtime_create(DEFAULT_RESERVOIR_NS); + let producer = localcast_producer_create(SampleKind::MoveEvidence.id(), 101, 3); + let descriptor = move_evidence_descriptor(9_000); + let mut sample = handle(0, 0, 0, 0, 0); + + assert!(localcast_producer_push_move_evidence_buffer( + producer, + runtime, + 1_200, + 1_220, + &descriptor, + &mut sample, + )); + + let mut status = LocalcastRuntimeStatus::default(); + assert!(localcast_runtime_status(runtime, &mut status)); + assert_eq!(1, status.total_sample_count); + assert_eq!(1, status.move_evidence_count); + assert_eq!(0, status.camera_feature_count); + assert_eq!( + (&descriptor as *const LocalcastMoveEvidenceBufferDescriptor) as u64, + sample.payload_handle + ); + assert_eq!( + 1, + localcast_runtime_view_len(runtime, SampleKind::MoveEvidence.id()) + ); + + localcast_producer_destroy(producer); + localcast_runtime_destroy(runtime); + } + + #[test] + fn move_evidence_descriptor_push_rejects_wrong_kind_and_bad_buffers() { + let runtime = localcast_runtime_create(DEFAULT_RESERVOIR_NS); + let render_producer = localcast_producer_create(SampleKind::RenderPacket.id(), 88, 12); + let move_producer = localcast_producer_create(SampleKind::MoveEvidence.id(), 89, 0); + let good = move_evidence_descriptor(9_000); + let zero_handle = move_evidence_descriptor(0); + let zero_count = LocalcastMoveEvidenceBufferDescriptor { + sample_count: 0, + ..good + }; + let bad_stride = LocalcastMoveEvidenceBufferDescriptor { + sample_stride_bytes: 4, + ..good + }; + + assert!(!localcast_producer_push_move_evidence_buffer( + render_producer, + runtime, + 2_000, + 2_010, + &good, + std::ptr::null_mut(), + )); + assert!(!localcast_producer_push_move_evidence_buffer( + move_producer, + runtime, + 2_000, + 2_010, + std::ptr::null(), + std::ptr::null_mut(), + )); + assert!(!localcast_producer_push_move_evidence_buffer( + move_producer, + runtime, + 2_000, + 2_010, + &zero_handle, + std::ptr::null_mut(), + )); + assert!(!localcast_producer_push_move_evidence_buffer( + move_producer, + runtime, + 2_000, + 2_010, + &zero_count, + std::ptr::null_mut(), + )); + assert!(!localcast_producer_push_move_evidence_buffer( + move_producer, + runtime, + 2_000, + 2_010, + &bad_stride, + std::ptr::null_mut(), + )); + assert_eq!( + 0, + localcast_runtime_view_len(runtime, SampleKind::MoveEvidence.id()) + ); + + localcast_producer_destroy(render_producer); + localcast_producer_destroy(move_producer); + localcast_runtime_destroy(runtime); + } + #[test] fn runtime_spine_rejects_nulls() { let sample = handle(1, 2, 3, 4, 5); diff --git a/notes/current-system-map.md b/notes/current-system-map.md index 6513587..cc39295 100644 --- a/notes/current-system-map.md +++ b/notes/current-system-map.md @@ -104,8 +104,10 @@ USB. Muninn represents what each local body can honestly observe: `muninn.move_marker_candidate.v1` for glowing orb candidates from cameras and `muninn.move_controller_state.v1` for controller/IMU/button/battery state from USB-attached Moves. Mimir drinks those feeds into `MimirStreamKind.Tracking` -buffers and publishes resolved `mimir.move_controller_pose.v1` documents for -Fensalir interaction once calibration and fusion have earned a pose. +buffers, admits them into the native `move_evidence` reservoir view as +compute-upload buffers, and publishes resolved `mimir.move_controller_pose.v1` +documents for Fensalir interaction once calibration and fusion have earned a +pose. Authority: @@ -120,9 +122,10 @@ Authority: hardware behavior before Muninn is available on that host. - Odin discovers and projects the stream/schema surface for operator and agent access. -- Mimir owns subscription, rolling retention, clock alignment, calibration, - controller association, triangulation, IMU fusion, prediction, and resolved - wand pose publication. OBS and dashboard summaries do not own this state. +- Mimir owns subscription, rolling retention, native Move evidence buffer + layout, clock alignment, calibration, controller association, triangulation, + IMU fusion, prediction, and resolved wand pose publication. OBS and + dashboard summaries do not own this state. - Fensalir consumes Mimir's resolved Move controller poses as interactive input for environments; it does not synthesize controller truth from raw Muninn evidence. diff --git a/src/Mimir.Runtime/Synchronization/MimirReservoirConfiguration.cs b/src/Mimir.Runtime/Synchronization/MimirReservoirConfiguration.cs index 7a6a8ee..bc1fb7f 100644 --- a/src/Mimir.Runtime/Synchronization/MimirReservoirConfiguration.cs +++ b/src/Mimir.Runtime/Synchronization/MimirReservoirConfiguration.cs @@ -32,6 +32,7 @@ public static class MimirReservoirConfigurations new("audio_block", "PCM/native audio block", "Mimir.Runtime/native audio", RetainsHistory: false, "Timing and field evidence."), new("phase_claim", "audio phase/group-delay evidence", "Mimir.Runtime", RetainsHistory: false, "Calibration/normalization input."), new("event_claim", "timeline anchor", "Mimir.Runtime", RetainsHistory: false, "Bioacoustic/passive/calibration events."), + new("move_evidence", "PS Move marker/controller evidence buffer", "Mimir native reservoir", RetainsHistory: false, "Compute-upload buffer of Muninn optical and controller witnesses before Mimir pose fusion."), new("surface_claim", "spatial surface observation", "Fensalir", RetainsHistory: false, "Fusion candidate."), new("render_packet", "Mimir program packet", "Mimir/Fensalir/Faust", RetainsHistory: false, "Output-ready program view.") ]; diff --git a/state/evidence.jsonl b/state/evidence.jsonl index 5734bb7..a7bf42a 100644 --- a/state/evidence.jsonl +++ b/state/evidence.jsonl @@ -130,3 +130,4 @@ {"ts": "2026-06-10T20:49:06+01:00", "type": "decision", "status": "ok", "note": "Corrected Move optical candidate ownership: the Rust/HLSL tracker moved out of Mimir and into Odin's Muninn organ at E:\\Projects\\Odin\\crates\\muninn-move-tracker. Mimir consumes muninn.move_marker_candidate.v1 streams and no longer owns raw optical extraction."} {"ts": "2026-06-11T07:57:05+01:00", "type": "decision", "status": "ok", "note": "PS Move structured light ownership moved to Odin/Muninn: Mimir requests muninn.move_light_command.v1 over CultNet/CultMesh; Muninn serve consumes host-addressed pending commands, writes local HID report 0x06, and updates command state. Nightwing direct HID launcher path is opt-in smoke/bootstrap only."} {"ts": "2026-06-11T16:59:31+01:00", "type": "decision", "status": "ok", "note": "Move sensor fusion authority moved to Mimir: Muninn publishes source-local marker candidates and controller/IMU/button state, Odin discovers/projects schemas, Mimir owns calibration, association, triangulation, IMU fusion, prediction, confidence, latency, and mimir.move_controller_pose.v1 for Fensalir interaction."} +{"ts": "2026-06-11T18:47:43+01:00", "type": "implementation", "status": "ok", "note": "Added native reservoir Move evidence admission: LocalcastMoveEvidenceSample and LocalcastMoveEvidenceBufferDescriptor define a fixed compute-upload ABI, SampleKind move_evidence enters the shared rolling window, malformed buffers are rejected before live admission, and Mimir docs/maps now route Muninn marker/controller witnesses through native buffers before Mimir pose fusion and Fensalir consumption."} diff --git a/state/map.yaml b/state/map.yaml index bd2c12b..ef45a67 100644 --- a/state/map.yaml +++ b/state/map.yaml @@ -89,6 +89,9 @@ accepted_design: `mimir.move_tracking_observation.v1`; Starfire Moves are local tracking streams, Nightwing Moves are emitted by Muninn on Nightwing, Odin owns discovery/schema projection, and Mimir owns buffering and fusion. + - `native/reservoir` now exposes a `move_evidence` typed view and + `LocalcastMoveEvidenceBufferDescriptor` for compute-upload buffers carrying + Muninn optical marker and controller-state witnesses before Mimir fusion. - `mimir.move_controller_pose.v1` is Mimir's resolved wand pose contract for Fensalir-facing interaction. Mimir owns calibration, association, triangulation, IMU fusion, prediction, confidence, and latency accounting.