workspace: name: Mimir status: active focus: direct-driver-aquarium-stream-app implementation_name: Mimir public surface / native runtime contracts premise: > Mimir is the local realtime field machine: cameras, microphones, speakers, network feeds, and timing signals become one coherent OBS-facing volumetric program surface. The live path is a C# Aquarium-hosted app with in-memory rolling buffers and native capture/runtime boundaries. objective: primary: > Build one synchronized spatial stream machine from six cameras, six microphones, two speakers, Leap timing, Aquarium GPU rendering, Faust DSP, and OBS output. current: > Implement direct camera/audio capture into Mimir.Runtime buffers, starting with Leap stereo IR as the timing-camera candidate, then expose runtime health/settings/output controls through Aquarium UI. constraints: - Sender is Windows. - Receiver uses OBS. - Video encode must use NVENC when the bridge path is active. - Audio sources must remain separately controllable in OBS. - V1 bridge is LAN-only. - Persistent state must clarify rehydrate, not become transcript tax. - Live ingest belongs to Mimir.Runtime, native drivers, Aquarium, and Faust/native DSP. - Edge JSON is schema/diagnostic only; live process data is typed or in memory. - Networked feeds are producers into the live runtime, not independent clock authorities. invariants: - OBS composition authority stays in OBS. - FFmpeg owns capture, encoding, and transport only for bridge utilities. - One stable bridge port equals one OBS source endpoint. - One rolling live window owns the timing edge and expiry. - Typed views may index the rolling buffer, but no per-kind ring owns retention. - Aquarium owns production UI, GPU fusion, D3D12 interop, and Spout2 output. - Faust/native DSP owns hot audio alignment, separation, spatialization, and stems. - Local secrets remain untracked. accepted_design: - `Mimir.slnx` owns the app surface. - `src/Mimir.App` launches Aquarium Engine as host/window/D3D12 library. - `src/Mimir.Runtime` owns `MimirSynchronizationHub`, `MimirRollingStreamBuffer`, stream descriptors, `IMimirStreamSource`, direct native ingest, process bridge ingest, and video capture driver seams. - `MimirVideoFrameDescriptor` carries width, height, pixel format, stride, device timestamp, and optional native/GPU handle. - `IMimirVideoCaptureDriver` and `MimirVideoCaptureDriverSource` are the live seam for Leap, Media Foundation, DirectShow, libusb, LeapC, or shared texture capture. - `native/reservoir` owns the lower shared-edge rolling-buffer invariant for Aquarium/Faust integration. - `config/perfect-machine.example.json` declares live owners, sensors, reservoir views, and runtime policy. - `.voidbot` state is part of repo memory and must be committed separately whenever dirty. active_subgoals: - Implement the first concrete Leap direct capture driver and measure cadence. - Add remaining local camera drivers through `IMimirVideoCaptureDriver`. - Add native audio capture workers for mic, loopback, and network audio feeds. - Bind Aquarium UI to stream health, buffer depth, source timestamps, settings, and output management. - Keep PowerShell/FFmpeg/SRT as bridge utilities until direct program output replaces them. current_status: summary: > The repo owns a C# Mimir app/runtime scaffold, a native reservoir crate, OBS bridge scripts, and persistence surfaces. Script-era packages, tests, helper tools, dependency manifests, and stale docs have been removed from the working tree. Research archives remain as reference-only memory for raw native/GPU implementation shape. A raw PS3 Eye libusb/WinUSB cadence probe measured both PS3 Eyes simultaneously at about 183 fps each at 320x240 and 57-59 fps each at 640x480. A native Kernel Streaming LeapUVC probe can pull frames through usbvideo.sys without Media Foundation, DirectShow, OpenCV, or frame processing; after discarding a two-second warm-up, the useful 640x240 stereo IR mode advertises 115 fps and currently measures about 110.8 fps with eight queued asynchronous KS reads. The half-height 640x120 mode advertises 214 fps and currently measures about 205.9 fps. Plain Kiyo is usable through KS when manual exposure is forced shorter: 1080p MJPG reaches about 30 fps and 720p MJPG reaches about 51.5 fps. Kiyo Pro advertises 60 fps MJPG at 720p/1080p but currently delivers only about 25 fps through the same KS path even with manual exposure tuning. Standard KS low-light compensation off does not restore 60 fps, powerline 60Hz only raises Pro cadence to about 29 fps, and standard extended Video HDR is not exposed on this KS path. The controls-only probe now parses USB VideoControl descriptors and found the Kiyo Pro's two vendor extension units. One extension unit answers writable KS selectors. Moving the Pro directly to a motherboard USB3 port changed the path to `root_hub30`, `bcdUSB=0x0320`, and exposed 720p/1080p YUY2/MJPG/H264/NV12 at 60 fps, but Windows still reports `UsbHighSpeed` rather than `UsbSuperSpeed`, and baseline cadence is still about 25 fps across every tested format. next_action: > Turn the warmed, queued Leap KS path into an `IMimirVideoCaptureDriver`, preserve direct control hooks for timing-camera state, and decide whether Kiyo Pro reaches 60 fps after confirming a true `UsbSuperSpeed` cable/port path; if it remains locked at 25 fps there, inspect explicit frame-interval negotiation or vendor selectors.