Current System Map

Source-level code ownership is mapped in docs/code-algorithm-map. The indexed problem-domain map is docs/perfect-machine-domain-index; the current architecture/optimization/sample-code study lives under research/perfect-machine-study-2026-05-23/.

Mimir is the public Face and product name for this repo. A few lower native ABI names still use localcast until a deliberate rename cut exists.

The live target is the native rolling field machine described in docs/native-rebuild-plan, docs/perfect-machine, and config/perfect-machine.example.json.

Perfect Machine Target

flowchart TD
    A["direct camera drivers"] --> R["Mimir.Runtime rolling buffers"]
    B["mic/loopback drivers"] --> R
    C["Leap timing/IR driver"] --> R
    D["network feed producers"] --> R
    R --> N["native reservoir handles"]
    N --> E["Fensalir GPU fusion + UI"]
    N --> F["Faust/native DSP"]
    E --> G["Spout2/program video"]
    F --> H["program stems + spatial bed"]
    G --> I["OBS"]
    H --> I

Ownership:

Mimir owns configuration, calibration truth, launch, status, persistence, and runtime contracts.
Mimir is a CultMesh app: typed state surfaces for codebooks, schedules, calibration receipts, path response, and offload work must be mesh-syncable so phones, microcontrollers, Raven, and Starfire can participate without becoming independent clock authorities.
Mimir.Runtime owns app-level stream buffers and synchronization.
Native capture workers own device reads.
Fensalir owns dense visual fusion, material/brush/splat reconciliation, D3D12 interop, runtime UI, and Spout2 publication.
Faust/native DSP owns hot audio alignment, suppression, separation, spatialization, and stem generation.
OBS owns broadcast controls.

Invariant: the live window is bounded, in memory, and has one timing authority. No private history outlives the rolling buffer.

Viable Stream App

docs/viable-stream-app defines the near-term app target. Fensalir hosts the running Mimir app, keeps the default five-second runtime in memory, exposes debug/settings/output controls, and emits synchronized OBS program video plus separately controllable audio stems.

Mimir.Runtime currently provides:

MimirSynchronizationHub;
MimirRollingStreamBuffer;
stream descriptors/settings;
IMimirStreamSource;
MimirNativeIngestStreamSource;
MimirProcessStreamSource;
MimirFrameEventProcessStreamSource;
MimirVideoFrameDescriptor;
MimirAudioSynchronizationAnalyzer;
IMimirVideoCaptureDriver;
MimirVideoCaptureDriverSource.

Process-backed sources are bridge/network edges. Local cameras should feed native descriptors with device timestamps and optional native/GPU handles. The frame-events/json-lines adapter is a diagnostic witness only: native probes can emit per-frame JSON metadata so Fensalir sees real sensor cadence in the rolling buffers while the direct ABI driver is being cut. It does not carry pixels and does not own the final six-camera hot path. Multi-camera probes are one process with declared accepted source ids, not one process per camera.

OBS Bridge Utility

flowchart TD
    A["config/localcast.json"] --> B["sender-start.ps1"]
    C["FFmpeg on sender"] --> D["Windows desktop capture"]
    C --> E["DirectShow audio capture"]
    D --> F["NVENC encode"]
    E --> G["audio encode"]
    F --> H["SRT video endpoint"]
    G --> I["SRT audio endpoint(s)"]
    H --> J["OBS Media Source"]
    I --> K["OBS Media Source per audio source"]

The bridge is useful because it is inspectable and already speaks OBS. It does not become the synchronized program authority.

Audio Field

The six-microphone path is separate from the bridge. Scarlett speaker loopback is the current timing authority when active calibration is playing, but Scarlett production capture belongs on ASIO rather than WASAPI shared mode. MimirBioacousticTimeline owns the active runtime watermark described in docs/bioacoustic-timeline-watermark: a low-gain birdsong-like word language with 128 self-identifying word positions, left-speaker and right-speaker variants, four formant-rich syllables per word, rhythm variation, and direct word identity. Any correctly decoded word identifies the event index inside the current operating horizon. Mimir queues the left vocabulary to the left speaker and the right vocabulary to the right speaker through Fensalir audio, then decodes timing as bioacoustic evidence. MimirAudioSynchronizationSettings.Mode chooses whether active calibration is allowed: chirp-only emits the active bioacoustic witness, passive stays silent and uses program-audio phase correlation, and hybrid uses passive evidence by default while emitting active pilot chunks only when passive confidence is weak. Active decoding does not inherit the passive two-second analysis floor: passive still needs a longer program-audio window, while the bioacoustic witness can decode once at least one self-identifying word is present. The bioacoustic detector now uses bounded motif proposals, matched motif scoring, direct word anchors, source clock fitting, and fractional waveform refinement. Pairwise sync compares matched anchors first, then only accepts independent clock-fit offsets inside the live latency horizon so period aliases do not become absurd reports. The old chirp-bin detector remains as a calibration/reference artifact: it carries the full dechirped bin-energy surface and aggregates decodes into a MimirChirpBinCalibrationModel with measured usable bands, expected-symbol versus observed-bin confusion observations, timing residuals, delay hypotheses, phase summaries, and an adaptive codebook plan. The analyzer keeps raw profiles even when no timing report is accepted, so physical mic failures can still guide bioacoustic motif weighting. Reports/states expose delayUs next to fractional sample delay. Mimir.BufferSmoke --bioacoustic-self-test proves direct word anchors. --standalone-bioacoustic-self-test --sample-rate 48000 --delay-samples 1269.5 proves a receiver with only codebook/schedule state can recover delayed source time to below printed microsecond precision. --chirp-only-sync-self-test --sample-rate 48000 recovers a 317.375-sample synthetic delay with printed 0.000 us error using evidence=bioacoustic. --bioacoustic-train is now the receipt-backed tuning harness: it runs multiple indexed cepstral decoder hypotheses across mel-cepstral warp/blur degradations and writes typed CultCache results plus pre-warp, post-warp, and reconstructed-from-detections WAV artifacts. The latest local receipt under artifacts/bioacoustic-training/bioacoustic-20260524-103438/ shows identity survives many degradations, but timing still fails under warped domains, so the next receiver cut is a global delay/clock/path hypothesis over detected words. Reports now carry fractional delay and per-band matched energy. The older MimirChirpletSymbolCodebook / MimirChirpletStreamDecoder path remains a diagnostic reference for constrained chirplet-transform work. The active receiver is the bioacoustic motif timeline; the analyzer only emits active timing reports from matched canonical anchors. The hub also owns cached reports plus smoothed per-source sync state with delay-slope/SRO in ppm. MimirRuntime runs analysis online as a bounded rotating service and can print live telemetry with MIMIR_SYNC_TELEMETRY_SECONDS. UI and telemetry are passive readers of cached sync state; they must not invoke the analyzer. Current app testing proves loopback wakeup, live mic buffers, and confident online sync states, but the next proof is stable canonical anchors through real mic streams. Camera mics are spatial/context witnesses; Focusrite devices are dialogue anchors. Fractional delay and the hot resampler belong in Faust/native DSP.

The WASAPI cadence probe is now a format/state diagnostic: it can request shared or exclusive sample rates, bit depths, channel counts, and float/PCM formats, then report the selected or closest format. The Focusrite driver stack is now installed and registers Focusrite USB ASIO. The Scarlett Solo 4th Gen is attached to Starfire and exposes 4 ASIO inputs / 2 outputs: Input 1, Input 2, Loopback 1, and Loopback 2. native/probes/asio_audio_cadence can instantiate that driver, verify 44.1-192 kHz support, and capture nonzero 4-channel Int32LSB callbacks at 192 kHz with 192-frame preferred buffers. The runtime analyzer accepts Float32, Int16, Int24, and Int32 PCM windows so ASIO/native capture can preserve the interface format. Raven also has a loopback-capable Scarlett ASIO path at 192 kHz for co-streamer/game timing evidence; Starfire still owns the heavy soundfield and sensor-fusion work. The ASIO probe can now play raw mono Float32 timeline audio through the Focusrite outputs while capturing every ASIO input as raw interleaved Float32. Mimir.BufferSmoke --analyze-asio-f32 feeds those captured channels into the same runtime analyzer. --calibrate-chirp-bin-asio-f32 computes and persists the response/confusion/delay model per output/mic path, and --analyze-asio-f32 --calibration ... loads it into the chirp-bin reference decoder. native/asio_capture and MimirAsioStreamSource are now the runtime Scarlett path: Focusrite ASIO callbacks feed sample-bearing 192 kHz Float32 blocks into Mimir.Runtime in process, without the diagnostic JSON/stdout bridge. The minimal config/mimir-runtime.asio.example.json proof ingested more than 12,000 sample-bearing 192 kHz blocks across asio-ch0 through asio-ch3 in two seconds and retained 2,048 blocks per channel, all from the same ASIO callback stream. A real 192 kHz chirp-bin Scarlett artifact decoded Loopback 1 -> Loopback 2 at exactly 0.000 us with 12 matched anchors and 0.999 confidence. The same analyzer now prints calibration profiles for decoded sources. Physical input 1 still fails pairwise timing in the stored artifact, but it leaves a concrete response profile: 14 frames, 12 anchors, 0.865 clock confidence, and strongest bins around 4525, 4075, and 7225 Hz. Acoustic robustness is now the open problem, not clean loopback timing, standalone decoder shape, or basic response evidence. Mimir.BufferSmoke --calibrate-contestant-asio-f32 now owns the active packet-song physical calibration receipt. It runs the canary packet decoder against interleaved ASIO Float32 captures, performs a fast per-channel global delay hypothesis search, then tight scheduled packet scoring with sub-sample refinement. The latest persisted model at calibration/bioacoustic/scarlett-canary-packet-192k-rerun.json ran at 10.7x realtime across four 192 kHz channels and records per-channel polarity, schedule offset, payload reliability, gain, response-normalization bands, and pairwise propagation delay. Current physical precision is loopback 2.524 us MAE, co-streamer shotgun 58.785 us MAE, and cardioid 90.558 us MAE; do not claim physical microsecond sync until those mic MAEs collapse by another order of magnitude. The first actuator proof now exists: faust/mimir_alignment_actuator.dsp owns six channels of bounded fractional delay/gain controls for Faust/native DSP, and Mimir.BufferSmoke --bioacoustic-actuator-self-test --sample-rate 48000 --delay-samples 317.375 proves the control loop shape by estimating a bioacoustic delay, applying fractional correction, and remeasuring residual below printed microsecond precision.

Visual Fusion

Visual fusion belongs in Fensalir over current reservoir claims. Native capture workers provide frames; Fensalir owns feature extraction, matching, material fitting, render budgeting, and publication.

Known Risks

Windows device names vary by driver and localization.
Some FFmpeg builds omit SRT or NVENC.
Separate bridge endpoints can drift.
OBS SRT reconnection behavior can be fussy.
Direct driver work must prove sustained cadence before it becomes timing authority.
PS3 Eye audio endpoints are enumeration/runtime-fragile. A later replug made both mic buffers emit 480-frame WASAPI blocks again while both PS3 Eye camera buffers were live.

12 KiB Raw Blame History