3.2 KiB
3.2 KiB
Implementation Plan
Current Cut Line
Mimir is now a C# app/runtime plus native reservoir project. The live stream machine must be direct-driver and native-buffer first:
src/Mimir.Apphosts Aquarium Engine for windowing, rendering, and the D3D12 bridge.src/Mimir.Runtimeowns stream descriptors, source polling, direct push ingest, one rolling buffer per configured audio/video stream, and the synchronization hub Aquarium can inspect.native/reservoirowns the lower native rolling-buffer invariant for Aquarium/Faust integration.- PowerShell/FFmpeg/SRT remains a bridge utility for LAN OBS feeds. It is not the synchronized program authority.
The old script stack is gone. Do not add a compatibility edge unless it protects a named invariant that the native runtime cannot protect yet.
Implemented
- Mimir public identity, branding, and Face memory.
Mimir.slnxwithsrc/Mimir.Appandsrc/Mimir.Runtime.- Aquarium host bootstrapping from
Mimir.App. MimirSynchronizationHub,MimirRollingStreamBuffer, stream descriptors, andIMimirStreamSource.- Configurable five-second default rolling buffers for local and network audio and video streams.
MimirNativeIngestStreamSourcefor direct push ingest into runtime buffers.MimirProcessStreamSourcefor bridge/network command edges.MimirVideoFrameDescriptorfor dimensions, pixel format, stride, device timestamp, and native/GPU handle metadata.IMimirVideoCaptureDriverandMimirVideoCaptureDriverSourceas the live driver-facing seam for Leap, Media Foundation, DirectShow, libusb, LeapC, or shared texture capture.native/reservoirwith one shared-edge rolling buffer, typed views, C ABI, source-id hashing, producer helpers, and typed audio/render payload descriptors.- Windows bridge scripts for sender discovery/start/stop and simple OBS Media Source ingest.
- Documentation for OBS receiver setup, native rebuild boundaries, the viable stream app, and the Mimir Face.
Temporary
- Audio and video may still traverse separate OBS/SRT endpoints during bridge testing so OBS can preserve independent controls.
- Process-backed stream sources are only acceptable for network bridge feeds or diagnostics. Six-camera local ingest belongs behind direct capture drivers.
- Calibration artifacts may remain on disk as evidence, but live state must be in memory inside Mimir/Aquarium/native runtime surfaces.
Next
- Implement concrete direct capture drivers for Leap stereo IR first, then the other cameras.
- Feed those drivers into
MimirVideoCaptureDriverSourceand prove sustained frame cadence in the rolling buffers. - Add native audio capture workers for local mic, loopback, and network audio feeds.
- Bind Aquarium UI to the synchronization hub so buffer depth, stream cadence, source timestamps, and output settings are visible and adjustable.
- Move GPU feature extraction, fusion, material fitting, render budgeting, and Spout2 publication into Aquarium.
- Move mic alignment, room suppression, voice separation, spatialization, and stem generation into Faust/native DSP.
- Keep the OBS bridge witness ledger as evidence before expanding receiver machinery.