workspace: name: Mimir status: active focus: direct-driver-fensalir-stream-app implementation_name: Mimir public surface / native runtime contracts premise: > Mimir is the local realtime field machine: cameras, microphones, speakers, network feeds, and timing signals become one coherent OBS-facing volumetric program surface. The live path is a C# Fensalir-hosted app with in-memory rolling buffers and native capture/runtime boundaries. objective: primary: > Build one synchronized spatial stream machine from six cameras, six microphones, two speakers, Leap timing, Fensalir GPU rendering, Faust DSP, and OBS output. current: > Make Leap dense geometry visible and calibratable first: live Leap packed stereo IR feeds Fensalir D3D12 disparity, Mimir declares a derived point-cloud Mesh socket, then Fensalir should generate/render points in the editor and feed global residuals into online calibration. constraints: - Sender is Windows. - Receiver uses OBS. - Video encode must use NVENC when the bridge path is active. - Audio sources must remain separately controllable in OBS. - V1 bridge is LAN-only. - Persistent state must clarify rehydrate, not become transcript tax. - Live ingest belongs to Mimir.Runtime, native drivers, Fensalir, and Faust/native DSP. - Edge JSON is schema/diagnostic only; live process data is typed or in memory. - Networked feeds are producers into the live runtime, not independent clock authorities. - Mimir explicitly trades latency and memory for throughput and field quality: the default five-second buffer is compute budget for stream alignment, GPU/audio analysis, and volumetric sound-field extraction. invariants: - OBS composition authority stays in OBS. - FFmpeg owns capture, encoding, and transport only for bridge utilities. - One stable bridge port equals one OBS source endpoint. - One rolling live window owns the timing edge and expiry. - The rolling window is not accidental latency. It must preserve enough time to line up feeds and crunch data before OBS-facing output is emitted. - Typed views may index the rolling buffer, but no per-kind ring owns retention. - Fensalir owns production UI, GPU fusion, D3D12 interop, Spout2 output, and the EVE shared D3D12 program-output texture. - Mimir publishes physical observations, calibration constraints, and surface intent. Fensalir owns field claims, selected lowerings, evidence reservoirs, temporal guide lanes, and program presentation. - Fast stereo depth belongs to Fensalir's D3D12 compute/resource authority. `research/d3d12-stereo-depth-provenance-2026-05-29.md` records libSGM as the permissive north-star reference for a future SGM-shaped stereo disparity lane over synchronized rectified texture pairs and calibration state. CUDA, TensorRT, and monocular depth demos are provenance only, not live dependencies or geometry authority. - The first stereo-depth contract is code-visible: `MimirStereoDepthConfigurations` names the libSGM-provenance D3D12 SGM profile, and `MimirFensalirFieldLowering.BuildStereoDepthCandidateFrame` references caller-declared shader-readable left/right Texture2D resources, then lowers rectified stereo depth into a compute-writable R16F disparity SurfacePage, R8 confidence Texture2D, a Height FieldEvidence claim, and an `AquariumFieldStereoDepthLowering` sidecar tying profile, calibration, camera pair, inputs, disparity output, SGM constants, and depth range together. `BuildLeapPackedStereoDepthCandidateFrame` is the live Leap cut: it recognizes a live `LeapStereoIr` rolling video window, reuses the declared packed R8G8 texture as both left and right resource identity, names the left/right observations as packed R/G lanes, and emits the same depth lowering from the runtime frame. The current libSGM-provenance D3D12 SGM profile publishes min disparity 0, 128 disparities, four aggregation paths, census radius 2, P1 8, and P2 96. Fensalir now has a first packed-Leap D3D12 SAD/block-match kernel that writes R16F disparity through the typed SurfacePage socket; full SGM and calibrated metric depth remain future work. - The first Leap point-cloud root is code-visible: `MimirPointCloudConfigurations.LeapDisparityPointCloudRoot` records the stereo-disparity projection profile and provenance, and `MimirFensalirFieldLowering.BuildLeapPackedStereoPointCloudCandidateFrame` declares a GPU-resident `FieldMesh` PointList resource derived from the Leap disparity SurfacePage. `MimirRuntime` merges that Mesh claim with the live Leap stereo-depth claim. `Mimir.BufferSmoke --leap-point-cloud-root-smoke` proves one `SurfacePage` packet plus one `Mesh` packet and zero deferred requests. Fensalir branch `codex/leap-packed-depth` now owns the matching first native surface-claim lane: generated PointList Mesh buffers are filled from the disparity SurfacePage on D3D12 for diagnostics, while the same compute pass writes approximated normals, curvature, SDF, PBR, and radiosity data into Fensalir's splat reservoir buffers. The point-list draw path is diagnostic-only; live presentation goes through Fensalir's splat/reservoir resolver. The live proof `artifacts/runtime/mimir-leap-surface-claims-clean.png` showed LeapUVC frames uploaded through a Fensalir-owned D3D12 texture, packed stereo depth dispatch-ready/dispatched, 19,200 generated surface claims, point-list rendering at `rendered 0`, and visible reservoir-resolved Leap splats. Calibrated projection constants, full SGM, reservoir-history startup repair, and global residual/calibration ownership remain open. - Fensalir packet payload handles are not authority. Live payload references must resolve through declared typed GPU/native resources with shape, residency, shader access, validity, version, and handle metadata before shader lowering can consume them. - EVE fullscreen output must use native decode/composite over Fensalir-owned program pixels; WebKit/DOM layout is forbidden as pixel authority. - Faust/native DSP owns hot audio alignment, separation, spatialization, and stems. - Local secrets remain untracked. - Mimir is a CultMesh app. Codebooks, schedules, calibration receipts, path response, and compute-offload work need typed mesh-syncable state surfaces so phones, microcontrollers, Raven, and Starfire can cooperate without becoming independent timing authorities. accepted_design: - `Mimir.slnx` owns the app surface. - `src/Mimir.App` launches Fensalir as host/window/D3D12 library. - `src/Mimir.Runtime` owns `MimirSynchronizationHub`, `MimirRollingStreamBuffer`, stream descriptors, `IMimirStreamSource`, direct native ingest, process bridge ingest, and video capture driver seams. - `MimirVideoFrameDescriptor` carries width, height, pixel format, stride, device timestamp, and optional native/GPU handle. - `MimirEveProgramOutputConfigurations.NativeSharedTexture` records the EVE-facing program-output contract: Fensalir publishes `Global\MimirFensalirProgramTexture` when `FENSALIR_PROGRAM_OUTPUT_D3D12=1`; EVE receives decoded native pixels and owns only final composite/input uplink. - `src/Mimir.EveRelay` is the current EVE dashboard transport slice: it opens the named D3D12 program-output texture, feeds low-latency H.264 (`h264_nvenc` by default), frames Annex-B access units over `/stream`, and reports `/health`. EveCanvas uses `AVSampleBufferDisplayLayer` for native decode/composite. The local launch script opens an SSH reverse tunnel to Eve when Windows firewall policy blocks direct inbound TCP, and EveCanvas tries localhost before the LAN host. - `src/Mimir.Broadcast` is the first self-hosted public edge sender: Starfire uses FFmpeg/NVENC to push one final H.264/AAC RTMP stream through an SSH local forward to Yggdrasil. Yggdrasil owns nginx RTMP ingest and HLS segment serving only; it does not transcode or compose. The public viewer route is `https://gamecult.org/livestream`, and it reads the existing Yggdrasil-routed StreamPixels URL `https://streampixels.gamecult.org/mimir/live/hls/mimir.m3u8`. The deployed Yggdrasil origin already validates, listens on localhost RTMP `127.0.0.1:1935`, and produced HLS segments from an eight-second Starfire synthetic NVENC push on 2026-05-30. Dedicated Mimir-site and `live.mimir.gamecult.org` viewer/media hostnames are not needed for v1. - `MimirAudioBlockDescriptor` carries sample rate, channels, sample format, frame count, timestamp, and optional native handle. - `MimirFrameEventProcessStreamSource` is a temporary diagnostic adapter for JSON-line video frame and audio block metadata from native probes into runtime rolling buffers; it does not carry pixels/audio payloads or own the final local-device hot path. - `MimirBioacousticTimeline` is the active runtime watermark. It renders a low-gain birdsong-like word language with 128 self-identifying word positions, left-speaker and right-speaker variants, four formant-rich syllables per word, rhythm variation, and direct word identity instead of a de Bruijn grammar. The active receiver is a song-contour anchor machine: one heard call should expose syllable timing, contour bends, formants, payload ornaments, speaker tint, and log-mel evidence as a cluster of time/frequency anchors. `MimirRuntime` emits separate panned vocabularies per speaker. `MimirAudioSynchronizationAnalyzer` reports active timing as `evidence=bioacoustic`; the old chirp-bin path remains a calibration/reference artifact with ASIO tooling. - The diagnostic audio path can optionally carry base64 float sample payloads for sync smoke tests. `MimirChirpletTimeline` owns the older structured chirplet reference fingerprint: an order-3 de Bruijn sequence over 32 time/frequency constellation symbols, plus PCM segment rendering, matched timing atoms, symbol/event decoding, and per-band response kernels. Symbol identity is carried by start band, glide shape, duration, and following inter-chirp gap. Any three consecutive correctly detected symbols identify a timeline event within the current operating horizon. `MimirChirpletSymbolCodebook` owns the 32 symbol definitions so every symbol has a unique chirp shape, with rhythm as additional evidence. `MimirChirpletStreamDecoder` is the first constrained chirplet-transform receiver: it owns a bounded PCM window, emits transform frames with multiple phase-invariant symbol candidates and refined per-candidate sample offsets, decodes code-valid triplet anchors through gap/clock coherence, and fits a per-source sample clock from those anchors. Its synthetic smoke path is `dotnet run --project .\src\Mimir.BufferSmoke\Mimir.BufferSmoke.csproj -- --chirplet-self-test`. It currently decodes all 15 emitted chirps in a two-second window into 13 anchors for events 0-12 with a 47999.999990 Hz clock fit and 0.000014 sample MAE. `MimirAudioSynchronizationAnalyzer` now treats `loopback-scarlett-speakers` as timing authority and derives delay only from matched decoded timeline anchors. Positive delay means the candidate mic is late relative to loopback. Reports include rounded delay, fractional delay, confidence, and per-band matched energy. `MimirSynchronizationHub` caches the latest per-source reports and `MimirAudioSynchronizationStateTracker` owns the smoothed per-source belief: latest delay, smoothed delay, confidence, per-band response evidence, and delay-slope/SRO estimate in ppm. `MimirRuntime` now runs this analysis online as a bounded rotating service on a fixed cadence, while UI and telemetry read cached reports/states instead of invoking analysis. `MIMIR_SYNC_TELEMETRY_SECONDS` prints live buffer/report/state telemetry. `MimirAudioSynchronizationSettings.Mode` now owns the active evidence policy: `chirp-only` emits the calibration witness, `passive` stays silent and uses program-audio phase correlation, and `hybrid` emits active pilot chunks only while passive confidence is weak. `MimirPassiveAudioSynchronizationEstimator` is the first passive path: it uses PHAT-weighted cross-spectrum correlation between loopback and mic windows before any watermark is needed. Negative passive lags are contradictory evidence and carry zero confidence; single-window passive confidence is capped below certainty until repeated state earns it. `MimirBioacousticTimeline` is now the active calibration witness for both chirp-only and hybrid: 128 self-identifying word positions, left/right speaker variants, four formant-rich syllables per word, rhythm variation, and direct word identity so any correctly decoded word identifies one canonical timeline location. The analyzer refines final active delay with constrained local waveform correlation around the decoded anchor delay. Each detected motif carries per-band response evidence, and the next calibration cut should use those profiles to tune contours, formants, usable bands, gain, and rhythm weighting. `Mimir.BufferSmoke --bioacoustic-cepstral-smoke` now proves the non-brute-force receiver direction: degraded mel-cepstral calls are decoded by an augmented MFCC/projection-hash word index before any expensive confirmation. `Mimir.BufferSmoke --bioacoustic-train` now runs a receipt-backed hypothesis panel over indexed cepstral decoder shapes and writes typed CultCache results plus pre-warp/post-warp/reconstructed WAV artifacts under `artifacts/bioacoustic-training/`. The first receipt `bioacoustic-20260524-103438` showed compact indexing fastest on clean audio and exposed timing error under domain warp. The corrected receipt `bioacoustic-20260524-150729` adds global clock hypothesis scoring over a longer window; compact indexing wins clean throughput, baseline indexing remains steadier under degradation, highband indexing has warped-path value, and anchor density needs its own metric. `MimirChirpBinTimeline` remains the controlled chirp-bin calibration/reference artifact with ASIO tooling and `MimirChirpBinCalibrationModel`; keep those captures as response evidence, not the active runtime sound. Reports and sync states expose delay in microseconds as well as fractional samples. Hybrid emits the bioacoustic witness as low-gain half-second bursts while passive confidence is weak, never as catch-up backlog. The analyzer no longer applies the passive two-second window floor to active fallback; `dotnet run --project .\src\Mimir.BufferSmoke\Mimir.BufferSmoke.csproj -- --chirp-only-sync-self-test --sample-rate 48000` recovers a 317.375-sample synthetic delay with printed 0.000 us error using `evidence=bioacoustic`, while `--hybrid-sync-self-test --sample-rate 48000` correctly accepts strong passive evidence at 2.772 us error. The synthetic standalone receiver proof `--standalone-bioacoustic-self-test --sample-rate 48000 --delay-samples 1269.5` recovers delayed source time below printed microsecond precision using only the codebook/schedule. `--bioacoustic-actuator-self-test --sample-rate 48000 --delay-samples 317.375` now proves the first measurement-to-sample-movement loop: estimate bioacoustic delay, apply fractional correction, and remeasure residual below printed microsecond precision. `faust/mimir_alignment_actuator.dsp` is the first Faust-owned actuator surface for six channels of bounded fractional delay/gain. `MimirAlignmentActuatorBank` now converts runtime sync states into explicit nonnegative Faust holdback commands with stable source slots, a separate reference holdback, and overflow reporting, then queues those commands into `AquariumAudioDocument` as engine audio control frames. Mimir also queues `faust/mimir_alignment_actuator.dsp` as an `AquariumStreamingDspProgram` so Fensalir can compile/host the persistent Faust graph that consumes those controls. The same runtime cadence queues sample-bearing `AquariumStreamingAudioBlock` lanes from the latest actuator source buffers, with channel indices derived from the bank-assigned Faust `sourceN` control path. Fensalir now publishes processed DSP output lanes as `AquariumAudioStemFrame` records on an engine-owned audio stem bus, and Mimir declares deterministic aligned-source stem names with the actuator program. Mimir now consumes that bus through `MimirObsStemPublicationState`, validating ready/missing/unconfigured lanes against the alignment actuator stem-bus profile without copying audio. `MimirObsStemSharedMemoryPublisher` writes validated stems into the `Local\MimirObsStemBus` Windows shared-memory ABI, and `native/obs_stem_source` is the first OBS source plugin bundle. It registers `Mimir Audio Stem`, which reads one named stem from that map and submits it as libobs audio, and `Mimir Program Texture`, which opens Fensalir's named shared D3D12 program-output texture and draws it through an OBS-readable shared D3D11 bridge texture. Fensalir also publishes `Global\MimirFensalirProgramFence` as a program-output publication fence; the OBS source opens that fence and observes completion before copying. The video path is GPU-to-GPU and does not require Spout2 or CPU readback; the remaining copy is the explicit D3D12 to libobs-D3D11 boundary. Fensalir and the OBS source can optionally agree on `FENSALIR_PROGRAM_OUTPUT_RING_COUNT>1`; Fensalir publishes `Global\MimirFensalirProgramTexture.0`, `.1`, and so on, and OBS chooses the latest completed slot from the program-output fence value. OBS can publish `Global\MimirObsProgramConsumerFence`; when Fensalir is launched with `FENSALIR_PROGRAM_OUTPUT_CONSUMER_FENCE_NAME`, it skips publication instead of reusing a ring slot OBS has not acknowledged. The rendering migration's program-output receipt phase is structurally complete: OBS consumes final Mimir/Fensalir program surfaces and stems through the native plugin, raw feeds remain debug inputs, and the ledger records the live texture, texture ring, producer-fence, and consumer-fence proofs. `MimirSynchronizedBufferPlanner` is the first aligned-buffer readout primitive: it chooses one canonical presentation time inside the rolling window and returns per-stream slices for local cameras, network display feeds, and audio blocks. Timing corrections can target a single source or a shared `clockDomainId`; the Raven path uses audio evidence routed into Scarlett to earn a `raven-sync` correction for Raven display frames. Network arrival timestamps remain metadata only. `MimirFfmpegRawVideoStreamSource` is the network-pixel adapter for Raven ingest: FFmpeg owns SRT receive/decode, while Mimir owns exact raw-video frame reads, payload geometry, video descriptors, and rolling-buffer samples. `config/mimir-runtime.raven-eve.example.json` listens for `raven-display` on port 5200. EveCanvas now owns Eve camera/mic capture directly and sends typed frame-events to `Mimir.EveSensorReceiver` on WebSocket ports 8793 (`eve-camera`) and 8794 (`eve-mic`). Raven can send desktop capture with `scripts/start-raven-screen-capture-sender.ps1`. `MimirPresentationControlState` owns program-control intent for Fensalir: video feed visibility/solo/opacity/layer order, audio mute/solo/gain, and global LUT preset selection. The `Mimir Program` panel filters video production surface intents and applies audio mix gain before/inside the Faust streaming DSP path. LUT presets are typed postprocess state; exposure and bloom map into current Fensalir `GraphicsSettings`, while actual LUT texture sampling remains the named renderer hook. `Mimir.EveDashboard` is the first native Eve operator dashboard server: `/eve/dashboard` sends scene/control snapshots and accepts select, move, scale, rotate, visibility, and reset commands. EveCanvas renders the scene graph and source panels natively in UIKit with multitouch transform editing. This is currently fixture dashboard state for the transport/control proof; live binding to `MimirPresentationControlState` and `MimirSceneEditorState` is the next authority cut. `MimirSceneEditorState` owns the separate Mimir-window editor graph: editor camera, scene nodes, selected node, visibility/locks, 2D-plane transforms, reset affordances, and grab/rotate/resize gizmo mode. Rolling video buffers derive sensor-feed panel nodes, while SDF text and model import controls create editor graph nodes. `MimirRuntime` renders derived outlines/handles through the Fensalir frame and exposes the `Mimir Editor` hierarchy/transform panel. This is not OBS program output. World SDF glyph rendering, ASSIMP-style mesh decoding/upload, and pixel-accurate gizmo hit-testing remain Fensalir renderer cuts. `scripts/build-obs-stem-plugin.ps1` stages the upstream OBS plugin template SDK under `artifacts/obs-sdk/` and builds the plugin DLL locally. The synthetic codebook proof is `dotnet run --project .\src\Mimir.BufferSmoke\Mimir.BufferSmoke.csproj -- --bioacoustic-self-test`. `Mimir.BufferSmoke --render-chirp-bin-f32` plus `native/probes/asio_audio_cadence --play-f32-mono --record-f32-interleaved` now provides a real Scarlett ASIO artifact proof for active mode. At 192 kHz, Loopback 1 -> Loopback 2 decoded at 0.000 us with 12 anchors and 0.999 confidence. `Mimir.BufferSmoke --calibrate-chirp-bin-asio-f32` now renders or ingests a calibration capture and persists the response/confusion/delay model under `calibration/chirp-bin/`; `--analyze-asio-f32 --calibration` loads it back into the chirp-bin reference decoder. Physical inputs still fail accepted pairwise timing against loopback in the stored artifact, but decoded mic channels now leave calibration models; physical input 1 produced two reliable symbols and a different strongest measured band set. `native/asio_capture` and `MimirAsioStreamSource` are now the runtime Scarlett path: Focusrite ASIO callbacks feed sample-bearing 192 kHz Float32 blocks into `Mimir.Runtime` in process. `config/mimir-runtime.asio.example.json` proves continuous ASIO runtime ingest: a two-second BufferSmoke run at 192 kHz ingested more than 12,000 sample-bearing blocks across `asio-ch0` through `asio-ch3` and retained 2,048 blocks per channel, all from the Focusrite ASIO callback stream. `Mimir.BufferSmoke --calibrate-contestant-asio-f32` now persists the active packet-song physical response model under `calibration/bioacoustic/`, including per-channel delay search, polarity, payload reliability, response-normalization bands, gain, confidence, and pairwise propagation delay. The latest canary-packet Scarlett receipt runs at 10.7x realtime across four 192 kHz channels: loopback is 37/37 with 2.524 us MAE, co-streamer shotgun is 37/37 payload with 58.785 us MAE, and the cardioid is 26/37 with 90.558 us MAE. This is a serious calibration surface, not yet true physical microsecond sync. A fresh 192 kHz room receipt at `calibration/bioacoustic/scarlett-canary-packet-192k-contour-fresh.json` keeps loopback at 2.524 us MAE, improves the co-streamer shotgun to 37/37 payload with 34.083 us MAE, and improves the cardioid to 31/37 payload with 92.996 us MAE. The next precision cut is intra-call contour anchors, not more sequence machinery. `calibration/bioacoustic/scarlett-canary-packet-anchor-rich-192k.json` is the first anchor-rich receipt: timing chips, formant pivots, harmonic-envelope notches, payload ornaments, renderer template caching, and per-event intra-call anchor measurements. It clears 10x realtime and improves loopback to about 2.23 us MAE, but physical mics regress/hold at tens of microseconds, so anchor survivability and path weighting are now the problem. Path-level loopback truth now compares candidate mic anchors directly against loopback anchors, then refines delay with event-local waveform correlation against the captured loopback packet. On the stored anchor-rich capture, Release calibration runs at 21.1x realtime and improves physical path precision to 7.576 us for the cardioid and 6.578 us for the shotgun. A fresh room capture keeps the shotgun near 5.916 us but worsens the cardioid to 18.930 us. The one-microsecond target remains open; next cut is phase/group-delay correction or a better direct-path anchor family. A naive recursive waveform phase-lock pass was tested and reverted because it could improve one cardioid receipt while worsening the shotgun and locking a fresh cardioid run onto a later reflection lobe. Recursion is only allowed back as a constrained direct-path/group-delay model, not as unconstrained waveform resampling with prettier decimals. The sonar/DSP research note `research/sonar-dsp-recursive-refinement-2026-05-25.md` points the next decoder cut at complex contour matched filters, phase-slope/group-delay fitting, acquisition/tracking loop bandwidths, and sparse multipath residuals. Its source ledger and local mirrors live under `research/sonar-dsp-recursive-refinement-2026-05-25/`. `MimirComplexContourMatchedFilterBank` and `MimirDirectPathTracker` are the first implementation slice of that shape: complex anchor responses keep multiple lobes, the current path prediction gates direct-path ownership, and later coherent clusters are reported as reflection taps. Synthetic 192 kHz reflection smoke currently lands about 5.249 us from expected delay; stored Scarlett shotgun receipts land within about 5.860 us and 7.247 us of seeded path fits, while cardioid receipts land within about 15.817 us and 8.963 us. Low confidence is expected until phase/group-delay surfaces are learned. The tracker now exposes per-band delay/phase residual observations and a `MimirDirectPathChannelModel` correction surface for multi-window calibration; one-window self-correction is not live authority. The tracker now consumes explicit channel models as phase/group-delay correction and downweights bands outside the learned usable surface. `Mimir.BufferSmoke --learn-complex-contour-channel-model` persists `calibration/bioacoustic/complex-contour-channel-model.json` from the replay panel, and `--evaluate-complex-contour-channel-model` persists `calibration/bioacoustic/complex-contour-channel-model-evaluation.json`. Current evaluation improves 3/4 absolute path-seed errors and lowers mean direct-cluster MAE; the fresh cardioid case reaches about 0.202 us from seed, while the stored cardioid case still worsens to about 17.476 us. The model is explicit calibration evidence, not default runtime authority. The complex contour receiver is now wired into runtime as an explicit live lane: when `enableComplexContourRuntime` is true, Mimir emits the configured `bioacousticWitnessProfileId` through `MimirBioacousticContestantRenderer`, loads `complexContourChannelModelPath`, extracts Float32 windows from the rolling buffers, and publishes `evidence=complex-contour` reports. The synthetic runtime-shaped proof `--complex-contour-runtime-self-test` at 192 kHz recovers a 693.5-sample delay with about 0.219 us error. A fresh live Scarlett proof is preserved at `calibration/bioacoustic/complex-contour-live-20260525-063229.md`: Focusrite ASIO playback/capture measured Loopback 1 -> Loopback 2 at -0.014 us with corrected confidence 0.971. The apparent `asio-ch0` near-seed/cardioid estimate is invalidated: signal inspection shows flat low-level noise with zero-lag loopback correlation 0.000032, so the tracker fit noise under seeded timing pressure. `asio-ch1` remains only a weak, non-authoritative input witness. The next cut is signal-presence gating, independent path truth, and better frequency/direct-path discrimination. `calibration/bioacoustic/complex-contour-live-20260528-002313.md` keeps the loopback proof healthy at -0.062 us with 0.956 confidence but finds zero contour hits on both physical inputs, so the current physical mic proof is blocked on input signal/gain/routing/source placement before receiver logic changes. After the mics were armed, `complex-contour-live-20260528-080604.md` proved both physical inputs were alive but was routed to headphones by mistake. The monitor-routed `complex-contour-live-20260528-080727.md` is the first credible fresh physical contour proof: loopback is -0.014 us at 0.971 confidence, `asio-ch0` lands -0.410 us from its stored path seed at 0.524 confidence, and `asio-ch1` lands -2.467 us at 0.363 confidence. Physical mapping: `asio-ch0` is the shotgun, about 1.5 m from the right monitor and 3.0 m from the left; `asio-ch1` is the cardioid, about 0.5 m from the left monitor and 1.5 m from the right. Repeat captures are needed before channel-model updates or actuator authority. `complex-contour-monitor-split-20260528-081813.md` added ASIO `--play-output-channel` and captured output 0/output 1 separately. Loopback channels prove the ASIO output buffers are distinct, but physical mic delays differ by only about 20.885 us for the shotgun and 24.789 us for the cardioid across outputs. This does not match the declared monitor geometry; verify downstream monitor routing/isolation before learning left/right acoustic paths. `calibration/bioacoustic/complex-contour-replay-panel.json` persists the current four-case replay receipt. Acoustic robustness remains open. The old arbitrary-chirplet matcher is a diagnostic/reference artifact only, not the runtime hot path. - `IMimirVideoCaptureDriver` and `MimirVideoCaptureDriverSource` are the live seam for Leap, Media Foundation, DirectShow, libusb, LeapC, or shared texture capture. - `native/reservoir` owns the lower shared-edge rolling-buffer invariant for Fensalir/Faust integration. - `MimirFensalirFieldLowering` now emits `AquariumFieldResourceDeclaration` rows for live payload views. Observation claims reference `mimir:resource:*` keys, and Fensalir validation/planning can reject or defer packets whose resource declaration is missing, CPU-only, duplicated, or backend-incompatible. Audio spectrum/waveform surface intents now use the source observation resource key and plan as `TubeField` packets when the resource is compatible. `Mimir.BufferSmoke --fensalir-field-evidence-smoke` proves one declared resource, three claims, one planned resource-backed `TubeField` packet, two deferred non-backend claims, and no validation errors. `--fensalir-field-dsl-resource-smoke` proves Fensalir's field evidence DSL can bind a declared resource directly and produce one planned `TubeField` packet with no deferred requests. `MimirFensalirFieldLowering` also owns the current camera observation bridge: latest video buffers lower into FieldEvidence camera claims, GPU/native video handles declare shared Texture2D resources, and metadata-only cadence frames do not create camera surface intents or fake payload requests. The old direct `AquariumGpuSensorFrame` builder is no longer the active proof path; Fensalir can now resolve imported shared Texture2D resources by native D3D12 handle and accepts Mimir video format names such as Bgra8, Gray8, Rg8, Nv12, and LeapStereoIr. The preferred live camera path is now Fensalir-owned texture leasing: Mimir receives `AquariumRuntimeServices`, asks the engine field resource broker for a keyed Texture2D lease, writes decoded frames into the returned shared D3D12 texture, signals the returned producer fence, and commits that fence value before the resource key reaches shader sampling. `MimirVideoCaptureDriverSource` forwards that lease client to direct camera drivers that implement `IMimirFensalirTextureLeaseReceiver`. Camera backends must use the closest-to-device path available and report unavoidable copies; process or managed convenience layers are diagnostics, not production camera hot-path owners. Raw single-plane system-memory frames use the explicit fallback upload lane: `MimirVideoCaptureDriverSource` uploads the bytes into the Fensalir lease, drops the managed payload from the live sample, and increments `UnavoidableCopyCount`. NV12 CPU upload is rejected until Fensalir has a real planar upload path; device/GPU NV12 producers should write the leased texture directly. `MimirKsVideoCaptureDriver` plus `native/camera_capture/mimir_camera_capture.dll` is the first production-shaped KS/UVC direct camera path: it opens a Kernel Streaming pin in process, queues uncompressed UVC frames, and feeds the same driver source and Fensalir lease upload path. MJPG/H264 remain future device/GPU decode producer work. `MimirPs3EyeVideoCaptureDriver` plus `native/camera_capture/mimir_ps3eye_capture.dll` owns the equivalent raw WinUSB/libusb PS3 Eye path and emits Bayer8 frames through the same upload and copy-count machinery. Fresh direct-driver smokes uploaded real frames for LeapUVC 640x240 YUY2, PS3 Eye 0/1 320x240 Bayer8, Kiyo Pro 1920x1080 YUY2, and regular Kiyo 640x480 YUY2. Regular Kiyo 1280x720 YUY2 did not open; compressed Kiyo Pro MJPG/H264 now have a first GPU-resident producer: `MimirMediaFoundationGpuVideoCaptureDriver` uses `native/camera_capture/mimir_mf_gpu_capture.dll` to run Media Foundation SourceReader with a D3D11 device manager, decode MJPG/H264 on the GPU, copy the decoded GPU surface into a shared BGRA D3D11 texture, and publish a `shared-d3d11-texture` handle with zero live CPU payload bytes. The MF GPU producer now also creates a shared D3D11 fence, signals it after the texture copy, and carries the fence handle/value through `MimirVideoFrameDescriptor`, `MimirBridgePayloadView`, and the Fensalir field resource declaration so D3D12 can wait before sampling. Kiyo Pro 1920x1080 MJPG->RGB32 and H264->RGB32 smokes passed with nonzero shared texture and producer fence handles. Direct shared NV12 texture creation failed in D3D11, so NV12 remains future plane-aware interop or GPU bridge-copy work. `Mimir.BufferSmoke --fensalir-camera-observation-smoke` verifies the split, and `--fensalir-texture-lease-smoke` verifies the engine-owned lease path. Camera image claims stay debug-view-only for now; production visual fusion/feature extraction lowerings are deferred until the renderer and audio work are fixed. Fensalir now owns the first in-process D3D12 field resource registry: declared shared structured/curve buffers import/alias GPU-resident resources by handle, while Fensalir-owned resources allocate GPU slots only when Fensalir is the producer. Mimir-declared rendering buffers are therefore Fensalir GPU resources in the same runtime, not cross-process payload truth and not CPU payloads to copy back and forth. The TubeField DSL can now express a 2D rolling float buffer as Catmull-Rom XY tubes with modulo column addressing, amplitude power/normalization, radius, ramp texture path, and emission scale. Fensalir now has the first D3D12 TubeField compute/render lowering: compute binds the declared rolling sample buffer as read-only GPU memory, expands endpoint-clamped Catmull-Rom tube envelopes into fixed-budget vertex/index UAVs, honors declared float sample stride with byte-address loads, records one draw batch per lowering, then the TubeField pixel shader binds the matching source buffer and resamples material per pixel by column+spline coordinate with the cheap 5-tap approximation. Texture2D local assets are now first-class Fensalir field resources: the DSL can declare a local PNG ramp resource, TubeField `ramp=` aliases bind automatically by resource key, and D3D12 uploads/binds the texture SRV with a linear-clamp sampler. Missing ramps fall back to a Fensalir-owned 0-1 linear texture. The first material is opaque field geometry with squared normalized emission, fake Fresnel center-bright/edge-dark shaping, and geometry-budget overflow reporting. TubeField compute now emits GPU-resident D3D12 indirect draw argument packets, and render executes those packets after binding each batch's source/ramp material state. SurfacePage resources now have the first Fensalir resolver: engine users can declare GPU-resident 2D pages with dimensions, format, version, and optional local source, and Fensalir allocates/binds them as shader-readable page textures for future Height/SDF/material lowerings. VolumeTexture resources can now be declared and resolved as GPU-resident shader-readable 3D textures for future density/extinction/SDF3D lowerings. Mesh resources now package friendly vertex/index ownership: `Mesh.Vertices` and `Mesh.Indices` describe the GPU buffers, plus topology, index format, bounds, submesh count, and version. The Fensalir DSL can declare those mesh packages, and the D3D12 registry allocates/reuses the backing vertex/index buffers under the mesh resource key. The DSL can also create generic resource-backed claims, so mesh/page/volume resources can reach backend packet planning. Mesh draw/material lowerings remain future engine work. Mesh layout authority is split: imported/user-facing meshes use the standard `PositionNormalUvColor` vertex layout, while generated meshes can be `PipelinePrivate` and let the lowering own the minimal bytes. TubeField now renders through the first explicit pipeline-private generated mesh binding: compute owns the private vertex/index/indirect layout, and render binds that generated mesh before applying TubeField material state. TubeField writes stable field ids, real tube normals, coverage/confidence, and domain-validity guide data into the scene metadata/control/reservoir guide targets consumed by reservoir resolve for spatiotemporal reconstruction. The shared field candidate inlet now keeps four fixed slots per pixel, so producer-specific paths compete inside a resolver-owned budget instead of choosing private winners. Reservoir history update is now a shared Fensalir compute pass over a ping-ponged structured buffer with the same four rows per pixel; it emits the resolved HDR field texture that bloom and presentation consume. The presentation shader is no longer a hidden history/TAA owner. TubeField reservoir input has moved from generated segment packets to rolling-buffer column packets: expansion emits one GPU-resident column record per logical history/source column, tile binning admits those columns, and ReSTIR candidate evaluation samples the Catmull-Rom tube SDF/material directly from the source buffer. Generated mesh draw args are zeroed for the live path and remain only a diagnostic reference. The generated-mesh DrawIndexed indirect command signature is shared ABI now; TubeField only owns the argument buffer it emits. TubeField expansion is now gated by the selected backend packet plan, so lowering metadata cannot bypass validation/planning authority. Validation also rejects TubeSpline lowering metadata if its claim is not Tube-encoded or if its resource does not match the claim payload. Mimir's typed surface-intent lowering now emits TubeSpline lowering metadata for audio spectrum/waveform Tube claims, so the generated mesh path is fed by real Mimir evidence rather than only the DSL smoke. The legacy direct `AquariumSplineFrame` spectrum dashboard is silent: `MimirRuntime` submits an empty spline frame and the old direct spectrum spline builder has been deleted. Live spectrum samples are now normalized into a row-major Float32 matrix, declared as `mimir:resource:spectrum:field-upload`, uploaded into a Fensalir-owned GPU-resident structured buffer, and consumed by the TubeField lowering. `Mimir.BufferSmoke --mimir-spectrum-upload-smoke` verifies the live `MimirRuntime` frame path emits that Float32 upload and planned TubeField packet while direct spline and buffer-field dashboard inputs remain empty. The runtime spectrum path also declares the local blackbody PNG as a GPU-resident Texture2D field resource and binds the TubeField material by `RampResourceKey`. Runtime spectrum columns now represent flattened `(history age, source lane)` pairs. Mimir emits one TubeField claim/lowering per active source. The spectrum resource advertises fixed history and source-lane capacities so frame content versions and source topology changes do not resize the GPU buffer while the rolling trail fills. `MIMIR_SPECTRUM_SOURCE_LANES` sets the lane capacity; surplus sources are truncated and reported in the runtime UI. Lowerings use that capacity as `ColumnStride` through the shared Float32 resource so each active lane renders a true age trail at `z = age * 0.1`. Physical history slots are addressed as a ring with `RollingOffset`, so logical age no longer requires moving every column in the upload matrix before shader sampling. Mimir emits one Float32 resource upload for the newest spectrum ring slot with an explicit element offset instead of uploading the full fixed-capacity backing buffer every frame. Fensalir owns rolling-slot validity and clamps TubeField dispatch so invalid older slots are not sampled after renderer allocation, reset, or partial update. `MIMIR_SPECTRUM_TUBE_SUBDIVISIONS` controls the Mimir-requested Catmull-Rom subdivisions so Fensalir's TubeField truncation report has a direct cost lever. Mimir's active proof path no longer uses the direct `AquariumAcousticFieldFrame` builder: sync states lower into FieldEvidence calibration constraints. Fensalir now selects AudioPath Phase/Confidence claims as DebugOverlay backend packets, so acoustic calibration evidence is planned and inspectable through the FieldEvidence authority. Acoustic source candidates and confidence volumes remain Phase 5 fusion work. The first synthetic acoustic source candidate proof now lowers an SRP/PHAT-style localization result with calibration/source identity and a world-space confidence envelope, and Fensalir plans it as a FieldEvidence DebugOverlay packet. Live room geometry still needs an explicit measured coordinate frame before those candidates become runtime truth. Phase 5's synthetic marker proof now lowers a deterministic multi-camera marker candidate with calibration/marker identity and world-space support, and Fensalir plans it as DebugOverlay evidence while ambiguous raw camera features still defer. - `config/perfect-machine.example.json` declares live owners, sensors, reservoir views, and runtime policy. - `src/Mimir.Runtime/Synchronization` now includes the Perfect Machine module library distilled from the research notes: node profiles, decoder/language profiles, path learning, benchmark panels, actuator strategies, capture and ingest strategies, reservoir strategies, field profiles, compute offload, OBS publication, CultMesh contracts, Fensalir lowering, and exportable manifest generation. - `docs/perfect-machine-module-library.md` indexes those reusable runtime modules and their smoke commands. - `docs/code-algorithm-map.md` is the source-level ownership and algorithm map for the current repo code surface. - `docs/perfect-machine-domain-index.md` indexes Mimir's full problem domain, current evidence taxonomy, hot paths, and implementation cuts. - `research/perfect-machine-study-2026-05-23/` preserves the architecture rumination, optimization ledger, research references, source/hotspot audits, Fensalir/native boundary maps, benchmark plan, calibration-session spec, distributed receiver spec, failure-mode ledger, data dictionary, low-level implementation notes, and sample code sketches for streaming chirp-bin decode, AVX2/GPU scoring, batched native ABI, fractional-delay/SRO actuation, acoustic source localization, Mimir-to-Fensalir lowering, and native capture rings. The corrected deep-study pass ran from `2026-05-23T23:54:42.9041588+01:00` through at least `2026-05-24T01:54:56.9747630+01:00`; use `research/perfect-machine-study-2026-05-23/reading-guide.md` as the re-entry index. - `.voidbot` state is part of repo memory and must be committed separately whenever dirty. active_subgoals: - Verify the live Leap point cloud in the Mimir editor. - Add calibrated projection constants and a global residual/calibration owner that can consume Leap point-cloud errors before PS3 Eyes, Kiyos, and Eve feed wider online calibration. - Add remaining local camera drivers through `IMimirVideoCaptureDriver`. - Use the verified in-process Focusrite ASIO source for local Scarlett mic and loopback ingest; WASAPI remains a diagnostic witness. - Add network audio feed producers that enter the same runtime clock/rebuffer machinery instead of becoming independent authorities. - Bind Fensalir UI to stream health, buffer depth, source timestamps, settings, and output management. - Keep PowerShell/FFmpeg/SRT as bridge utilities until direct program output replaces them. current_status: summary: > The repo owns a C# Mimir app/runtime scaffold, a native reservoir crate, OBS bridge scripts, and persistence surfaces. Script-era packages, tests, helper tools, dependency manifests, and stale docs have been removed from the working tree. Research archives remain as reference-only memory for raw native/GPU implementation shape. A raw PS3 Eye libusb/WinUSB cadence probe measured both PS3 Eyes simultaneously at about 183 fps each at 320x240 and 57-59 fps each at 640x480. A native Kernel Streaming LeapUVC probe can pull frames through usbvideo.sys without Media Foundation, DirectShow, OpenCV, or frame processing; after discarding a two-second warm-up, the useful 640x240 stereo IR mode advertises 115 fps and currently measures about 110.8 fps with eight queued asynchronous KS reads. The half-height 640x120 mode advertises 214 fps and currently measures about 205.9 fps. Plain Kiyo is usable through KS when manual exposure is forced shorter: 1080p MJPG reaches about 30 fps and 720p MJPG reaches about 51.5 fps. Kiyo Pro advertises 60 fps MJPG at 720p/1080p but currently delivers only about 25 fps through the same KS path even with manual exposure tuning. Standard KS low-light compensation off does not restore 60 fps, powerline 60Hz only raises Pro cadence to about 29 fps, and standard extended Video HDR is not exposed on this KS path. The controls-only probe now parses USB VideoControl descriptors and found the Kiyo Pro's two vendor extension units. One extension unit answers writable KS selectors. Moving the Pro directly to a motherboard USB3 port changed the path to `root_hub30`, `bcdUSB=0x0320`, and exposed 720p/1080p YUY2/MJPG/H264/NV12 at 60 fps, but Windows still reports `UsbHighSpeed` rather than `UsbSuperSpeed`, and baseline cadence is still about 25 fps across every tested format. The first simultaneous local camera pull only had three visible devices: LeapUVC, Kiyo Pro, and one PS3 Eye. The KS multi-stream harness plus raw PS3 Eye probe pulled all three at once, but Leap cadence collapsed under shared load: with PS3 Eye at 640x480@60 the set delivered Leap 40.12 fps, Kiyo Pro 24.99 fps, and PS3 Eye 56.06 fps; with PS3 Eye at 320x240@187 it delivered Leap 81.10 fps, Kiyo Pro 25.33 fps, and PS3 Eye 185.74 fps. After port swapping, five camera devices / six optical sensors were visible and pulled at once: LeapUVC's packed stereo IR pair, Kiyo Pro, regular Kiyo, and two PS3 Eyes. With both Eyes at 640x480@60, the set delivered Leap stereo 57.33 fps, Kiyo Pro 24.97 fps, Kiyo 30.30 fps, PS3 Eye 0 59.59 fps, and PS3 Eye 1 57.60 fps. With both Eyes at 320x240@187, it delivered Leap stereo 81.05 fps, Kiyo Pro 24.97 fps, Kiyo 30.28 fps, PS3 Eye 0 187.60 fps, and PS3 Eye 1 187.03 fps. Kiyo Pro still negotiates high-speed USB, and Leap remains topology-sensitive under shared load. A follow-up moved Leap alone onto the blue rear USB3 group/root_hub30#5 while both Kiyos shared root_hub30#7&18eea21b and one visible PS3 Eye used root_hub30#7&124f193f; Leap still delivered only 80.99 fps with the other visible cameras active, so the remaining loss is not explained by same-root-hub contention alone. Splitting Kiyo Pro and one PS3 Eye to Raven left the Mimir host with LeapUVC, regular Kiyo, and one PS3 Eye, but local cadence still stayed low: Leap delivered 81.34 fps with Kiyo+Eye, 82.77 fps with Kiyo only, and only 84.18 fps in a same-session Leap-only baseline. The previous ~110.8 fps Leap result was not reproduced until the Focusrite Scarlett Solo was moved away from the Leap USB neighborhood. After that, Leap-only baseline rose to 95.48 fps, best full-height scenario to 101.67 fps, and the local split set delivered Leap 105.10 fps, regular Kiyo 30.17 fps, and one PS3 Eye 185.81 fps. The split-host layout is viable if the Leap's USB neighborhood stays free of the Focusrite and other heavy devices. Raven's PS3 Eye raw interface now has the same libwdi WinUSB binding as the local Eye and can be opened by the copied cadence probe, but Raven Eye cadence is currently unusable: 320x240@187 delivered 132.24 fps then 78.93 fps, and 640x480@60 delivered 6.32 fps with multi-second gaps. Moving that Eye back to the Mimir host produced the current clean local topology: LeapUVC, regular Kiyo, and two PS3 Eyes run in parallel while Kiyo Pro stays off-host. With both Eyes at 320x240@187, Leap delivered 106.07 fps, Kiyo 30.33 fps, and the Eyes 186.57 / 186.14 fps. With both Eyes at 640x480@60, Leap delivered 108.07 fps, Kiyo 30.34 fps, and the Eyes 57.79 / 58.08 fps. Moving Kiyo Pro back to the Mimir host while regular Kiyo was absent kept Leap healthy: with both Eyes at 320x240@187, Leap delivered 110.80 fps, Kiyo Pro 24.97 fps, and the Eyes 187.46 / 186.89 fps; with both Eyes at 640x480@60, Leap delivered 110.72 fps, Kiyo Pro 24.87 fps, and the Eyes 59.91 / 59.19 fps. Kiyo Pro still negotiates UsbHighSpeed(2), but it does not poison Leap cadence in the current layout. Moving regular Kiyo from front panel to a rear port made all five camera devices / six optical sensors visible locally. With both Eyes at 320x240@187, the full set delivered Leap 110.84 fps, Kiyo Pro 25.08 fps, Kiyo 30.18 fps, PS3 Eye 0 187.59 fps, and PS3 Eye 1 187.02 fps. With both Eyes at 640x480@60, it delivered Leap 110.76 fps, Kiyo Pro 25.14 fps, Kiyo 30.18 fps, PS3 Eye 0 59.66 fps, and PS3 Eye 1 59.95 fps. A mouse in the Leap USB neighborhood does not reproduce the Focusrite-induced collapse. next_action: > Continue the Leap-first visual geometry cut from the verified surface-claim path. Mimir lowers live `LeapStereoIr` packed R/G stereo into a disparity SurfacePage and declares a derived point-cloud Mesh socket. Fensalir now generates Leap surface splats from that disparity, feeds them into the native splat/reservoir resolver, and presents visible Leap-derived points without the diagnostic point-list draw path. Next: replace provisional projection display fit with calibrated Leap intrinsics/extrinsics, repair reservoir-history startup so the temporal pass can stay enabled, then add a global residual/calibration owner that can consume point-cloud alignment error. Do not fan RGB cameras into fake depth: PS3 Eyes are fast motion/feature witnesses, Kiyos and Eve are RGB/context witnesses, and Leap owns the first dense close-range geometry. Keep the Focusrite away from the Leap USB neighborhood. Treat Kiyo Pro as a ~25 fps source until its SuperSpeed negotiation is fixed.