413 lines
28 KiB
YAML
413 lines
28 KiB
YAML
workspace:
|
|
name: Mimir
|
|
status: active
|
|
focus: direct-driver-fensalir-stream-app
|
|
implementation_name: Mimir public surface / native runtime contracts
|
|
premise: >
|
|
Mimir is the local realtime field machine: cameras, microphones, speakers,
|
|
network feeds, and timing signals become one coherent OBS-facing volumetric
|
|
program surface. The live path is a C# Fensalir-hosted app with in-memory
|
|
rolling buffers and native capture/runtime boundaries.
|
|
|
|
objective:
|
|
primary: >
|
|
Build one synchronized spatial stream machine from six cameras, six
|
|
microphones, two speakers, Leap timing, Fensalir GPU rendering, Faust DSP,
|
|
and OBS output.
|
|
current: >
|
|
Promote the active bioacoustic packet-song receiver into runtime state:
|
|
self-identifying song contours should pin multiple time/frequency anchors
|
|
per heard call, learn per-path response, and drive the audio actuator while
|
|
direct camera/audio capture continues feeding Mimir.Runtime buffers.
|
|
|
|
constraints:
|
|
- Sender is Windows.
|
|
- Receiver uses OBS.
|
|
- Video encode must use NVENC when the bridge path is active.
|
|
- Audio sources must remain separately controllable in OBS.
|
|
- V1 bridge is LAN-only.
|
|
- Persistent state must clarify rehydrate, not become transcript tax.
|
|
- Live ingest belongs to Mimir.Runtime, native drivers, Fensalir, and
|
|
Faust/native DSP.
|
|
- Edge JSON is schema/diagnostic only; live process data is typed or in
|
|
memory.
|
|
- Networked feeds are producers into the live runtime, not independent clock
|
|
authorities.
|
|
- Mimir explicitly trades latency and memory for throughput and field quality:
|
|
the default five-second buffer is compute budget for stream alignment,
|
|
GPU/audio analysis, and volumetric sound-field extraction.
|
|
|
|
invariants:
|
|
- OBS composition authority stays in OBS.
|
|
- FFmpeg owns capture, encoding, and transport only for bridge utilities.
|
|
- One stable bridge port equals one OBS source endpoint.
|
|
- One rolling live window owns the timing edge and expiry.
|
|
- The rolling window is not accidental latency. It must preserve enough time
|
|
to line up feeds and crunch data before OBS-facing output is emitted.
|
|
- Typed views may index the rolling buffer, but no per-kind ring owns
|
|
retention.
|
|
- Fensalir owns production UI, GPU fusion, D3D12 interop, and Spout2 output.
|
|
- Faust/native DSP owns hot audio alignment, separation, spatialization, and
|
|
stems.
|
|
- Local secrets remain untracked.
|
|
- Mimir is a CultMesh app. Codebooks, schedules, calibration receipts, path
|
|
response, and compute-offload work need typed mesh-syncable state surfaces
|
|
so phones, microcontrollers, Raven, and Starfire can cooperate without
|
|
becoming independent timing authorities.
|
|
|
|
accepted_design:
|
|
- `Mimir.slnx` owns the app surface.
|
|
- `src/Mimir.App` launches Fensalir as host/window/D3D12 library.
|
|
- `src/Mimir.Runtime` owns `MimirSynchronizationHub`,
|
|
`MimirRollingStreamBuffer`, stream descriptors, `IMimirStreamSource`,
|
|
direct native ingest, process bridge ingest, and video capture driver seams.
|
|
- `MimirVideoFrameDescriptor` carries width, height, pixel format, stride,
|
|
device timestamp, and optional native/GPU handle.
|
|
- `MimirAudioBlockDescriptor` carries sample rate, channels, sample format,
|
|
frame count, timestamp, and optional native handle.
|
|
- `MimirFrameEventProcessStreamSource` is a temporary diagnostic adapter for
|
|
JSON-line video frame and audio block metadata from native probes into
|
|
runtime rolling buffers; it does not carry pixels/audio payloads or own the
|
|
final local-device hot path.
|
|
- `MimirBioacousticTimeline` is the active runtime watermark. It renders a
|
|
low-gain birdsong-like word language with 128 self-identifying word
|
|
positions, left-speaker and right-speaker variants, four formant-rich
|
|
syllables per word, rhythm variation, and direct word identity instead of a
|
|
de Bruijn grammar. The active receiver is a song-contour anchor machine:
|
|
one heard call should expose syllable timing, contour bends, formants,
|
|
payload ornaments, speaker tint, and log-mel evidence as a cluster of
|
|
time/frequency anchors. `MimirRuntime` emits separate panned vocabularies
|
|
per speaker. `MimirAudioSynchronizationAnalyzer` reports active timing as
|
|
`evidence=bioacoustic`; the old chirp-bin path remains a
|
|
calibration/reference artifact with ASIO tooling.
|
|
- The diagnostic audio path can optionally carry base64 float sample payloads
|
|
for sync smoke tests. `MimirChirpletTimeline` owns the older structured
|
|
chirplet reference fingerprint: an order-3 de Bruijn sequence over 32
|
|
time/frequency constellation symbols, plus PCM segment rendering, matched
|
|
timing atoms, symbol/event decoding, and per-band response kernels. Symbol
|
|
identity is carried by start band, glide shape, duration, and following
|
|
inter-chirp gap. Any three consecutive correctly detected symbols identify a
|
|
timeline event within the current operating horizon. `MimirChirpletSymbolCodebook`
|
|
owns the 32 symbol definitions so every symbol has a unique chirp shape, with
|
|
rhythm as additional evidence. `MimirChirpletStreamDecoder` is the first
|
|
constrained chirplet-transform receiver: it owns a bounded PCM window, emits
|
|
transform frames with multiple phase-invariant symbol candidates and refined
|
|
per-candidate sample offsets, decodes code-valid triplet anchors through
|
|
gap/clock coherence, and fits a per-source sample clock from those anchors.
|
|
Its synthetic smoke path is `dotnet run --project
|
|
.\src\Mimir.BufferSmoke\Mimir.BufferSmoke.csproj -- --chirplet-self-test`.
|
|
It currently decodes all 15 emitted chirps in a two-second window into 13
|
|
anchors for events 0-12 with a 47999.999990 Hz clock fit and 0.000014 sample
|
|
MAE. `MimirAudioSynchronizationAnalyzer`
|
|
now treats `loopback-scarlett-speakers` as timing authority and derives
|
|
delay only from matched decoded timeline anchors. Positive delay means
|
|
the candidate mic is late relative to loopback. Reports include rounded
|
|
delay, fractional delay, confidence, and per-band matched energy. `MimirSynchronizationHub`
|
|
caches the latest per-source reports and `MimirAudioSynchronizationStateTracker`
|
|
owns the smoothed per-source belief: latest delay, smoothed delay,
|
|
confidence, per-band response evidence, and delay-slope/SRO estimate in
|
|
ppm. `MimirRuntime` now runs this analysis online as a bounded rotating
|
|
service on a fixed cadence, while UI and telemetry read cached reports/states
|
|
instead of invoking analysis. `MIMIR_SYNC_TELEMETRY_SECONDS` prints live
|
|
buffer/report/state telemetry. `MimirAudioSynchronizationSettings.Mode`
|
|
now owns the active evidence policy: `chirp-only` emits the calibration
|
|
witness, `passive` stays silent and uses program-audio phase correlation,
|
|
and `hybrid` emits active pilot chunks only while passive confidence is
|
|
weak. `MimirPassiveAudioSynchronizationEstimator` is the first passive path:
|
|
it uses PHAT-weighted cross-spectrum correlation between loopback and mic
|
|
windows before any watermark is needed. Negative passive lags are
|
|
contradictory evidence and carry zero confidence; single-window passive
|
|
confidence is capped below certainty until repeated state earns it.
|
|
`MimirBioacousticTimeline` is now the active calibration witness for both
|
|
chirp-only and hybrid: 128 self-identifying word positions, left/right
|
|
speaker variants, four formant-rich syllables per word, rhythm variation,
|
|
and direct word identity so any correctly decoded word identifies one
|
|
canonical timeline location. The analyzer refines final active delay with constrained local
|
|
waveform correlation around the decoded anchor delay. Each detected motif
|
|
carries per-band response evidence, and the next calibration cut should use
|
|
those profiles to tune contours, formants, usable bands, gain, and rhythm
|
|
weighting. `Mimir.BufferSmoke --bioacoustic-cepstral-smoke` now proves the
|
|
non-brute-force receiver direction: degraded mel-cepstral calls are decoded
|
|
by an augmented MFCC/projection-hash word index before any expensive
|
|
confirmation. `Mimir.BufferSmoke --bioacoustic-train` now runs a
|
|
receipt-backed hypothesis panel over indexed cepstral decoder shapes and
|
|
writes typed CultCache results plus pre-warp/post-warp/reconstructed WAV
|
|
artifacts under `artifacts/bioacoustic-training/`. The first receipt
|
|
`bioacoustic-20260524-103438` showed compact indexing fastest on clean
|
|
audio and exposed timing error under domain warp. The corrected receipt
|
|
`bioacoustic-20260524-150729` adds global clock hypothesis scoring over a
|
|
longer window; compact indexing wins clean throughput, baseline indexing
|
|
remains steadier under degradation, highband indexing has warped-path value,
|
|
and anchor density needs its own metric. `MimirChirpBinTimeline` remains the controlled chirp-bin
|
|
calibration/reference artifact with ASIO tooling and
|
|
`MimirChirpBinCalibrationModel`; keep those captures as response evidence,
|
|
not the active runtime sound. Reports and sync states expose delay in
|
|
microseconds as well as fractional samples. Hybrid emits the bioacoustic
|
|
witness as low-gain half-second bursts while passive confidence is weak,
|
|
never as catch-up backlog. The analyzer no longer applies the passive
|
|
two-second window floor to active fallback; `dotnet run --project
|
|
.\src\Mimir.BufferSmoke\Mimir.BufferSmoke.csproj -- --chirp-only-sync-self-test --sample-rate 48000`
|
|
recovers a 317.375-sample synthetic delay with printed 0.000 us error using
|
|
`evidence=bioacoustic`, while `--hybrid-sync-self-test --sample-rate 48000`
|
|
correctly accepts strong passive evidence at 2.772 us error. The synthetic
|
|
standalone receiver proof `--standalone-bioacoustic-self-test --sample-rate
|
|
48000 --delay-samples 1269.5` recovers delayed source time below printed
|
|
microsecond precision using only the codebook/schedule. `--bioacoustic-actuator-self-test
|
|
--sample-rate 48000 --delay-samples 317.375` now proves the first
|
|
measurement-to-sample-movement loop: estimate bioacoustic delay, apply
|
|
fractional correction, and remeasure residual below printed microsecond
|
|
precision. `faust/mimir_alignment_actuator.dsp` is the first Faust-owned
|
|
actuator surface for six channels of bounded fractional delay/gain. The synthetic
|
|
codebook proof is `dotnet run --project
|
|
.\src\Mimir.BufferSmoke\Mimir.BufferSmoke.csproj -- --bioacoustic-self-test`.
|
|
`Mimir.BufferSmoke --render-chirp-bin-f32` plus
|
|
`native/probes/asio_audio_cadence --play-f32-mono --record-f32-interleaved`
|
|
now provides a real Scarlett ASIO artifact proof for active mode. At 192 kHz,
|
|
Loopback 1 -> Loopback 2 decoded at 0.000 us with 12 anchors and 0.999
|
|
confidence. `Mimir.BufferSmoke --calibrate-chirp-bin-asio-f32` now renders
|
|
or ingests a calibration capture and persists the response/confusion/delay
|
|
model under `calibration/chirp-bin/`; `--analyze-asio-f32 --calibration`
|
|
loads it back into the chirp-bin reference decoder. Physical inputs still fail accepted
|
|
pairwise timing against loopback in the stored artifact, but decoded mic
|
|
channels now leave calibration models; physical input 1 produced two reliable
|
|
symbols and a different strongest measured band set.
|
|
`native/asio_capture` and `MimirAsioStreamSource` are now the runtime
|
|
Scarlett path: Focusrite ASIO callbacks feed sample-bearing 192 kHz Float32
|
|
blocks into `Mimir.Runtime` in process. `config/mimir-runtime.asio.example.json`
|
|
proves continuous ASIO runtime ingest: a two-second BufferSmoke run at
|
|
192 kHz ingested more than 12,000 sample-bearing blocks across `asio-ch0`
|
|
through `asio-ch3` and retained 2,048 blocks per channel, all from the
|
|
Focusrite ASIO callback stream.
|
|
`Mimir.BufferSmoke --calibrate-contestant-asio-f32` now persists the active
|
|
packet-song physical response model under `calibration/bioacoustic/`,
|
|
including per-channel delay search, polarity, payload reliability,
|
|
response-normalization bands, gain, confidence, and pairwise propagation
|
|
delay. The latest canary-packet Scarlett receipt runs at 10.7x realtime
|
|
across four 192 kHz channels: loopback is 37/37 with 2.524 us MAE,
|
|
co-streamer shotgun is 37/37 payload with 58.785 us MAE, and the cardioid
|
|
is 26/37 with 90.558 us MAE. This is a serious calibration surface, not yet
|
|
true physical microsecond sync.
|
|
A fresh 192 kHz room receipt at
|
|
`calibration/bioacoustic/scarlett-canary-packet-192k-contour-fresh.json`
|
|
keeps loopback at 2.524 us MAE, improves the co-streamer shotgun to 37/37
|
|
payload with 34.083 us MAE, and improves the cardioid to 31/37 payload
|
|
with 92.996 us MAE. The next precision cut is intra-call contour anchors,
|
|
not more sequence machinery.
|
|
`calibration/bioacoustic/scarlett-canary-packet-anchor-rich-192k.json`
|
|
is the first anchor-rich receipt: timing chips, formant pivots,
|
|
harmonic-envelope notches, payload ornaments, renderer template caching, and
|
|
per-event intra-call anchor measurements. It clears 10x realtime and
|
|
improves loopback to about 2.23 us MAE, but physical mics regress/hold at
|
|
tens of microseconds, so anchor survivability and path weighting are now the
|
|
problem.
|
|
Path-level loopback truth now compares candidate mic anchors directly
|
|
against loopback anchors, then refines delay with event-local waveform
|
|
correlation against the captured loopback packet. On the stored anchor-rich
|
|
capture, Release calibration runs at 21.1x realtime and improves physical
|
|
path precision to 7.576 us for the cardioid and 6.578 us for the shotgun.
|
|
A fresh room capture keeps the shotgun near 5.916 us but worsens the
|
|
cardioid to 18.930 us. The one-microsecond target remains open; next cut is
|
|
phase/group-delay correction or a better direct-path anchor family. A naive
|
|
recursive waveform phase-lock pass was tested and reverted because it could
|
|
improve one cardioid receipt while worsening the shotgun and locking a fresh
|
|
cardioid run onto a later reflection lobe. Recursion is only allowed back as
|
|
a constrained direct-path/group-delay model, not as unconstrained waveform
|
|
resampling with prettier decimals. The sonar/DSP research note
|
|
`research/sonar-dsp-recursive-refinement-2026-05-25.md` points the next
|
|
decoder cut at complex contour matched filters, phase-slope/group-delay
|
|
fitting, acquisition/tracking loop bandwidths, and sparse multipath
|
|
residuals. Its source ledger and local mirrors live under
|
|
`research/sonar-dsp-recursive-refinement-2026-05-25/`.
|
|
`MimirComplexContourMatchedFilterBank` and `MimirDirectPathTracker` are the
|
|
first implementation slice of that shape: complex anchor responses keep
|
|
multiple lobes, the current path prediction gates direct-path ownership, and
|
|
later coherent clusters are reported as reflection taps. Synthetic 192 kHz
|
|
reflection smoke currently lands about 5.249 us from expected delay; stored
|
|
Scarlett shotgun receipts land within about 5.860 us and 7.247 us of seeded
|
|
path fits, while cardioid receipts land within about 15.817 us and 8.963 us.
|
|
Low confidence is expected until phase/group-delay surfaces are learned. The
|
|
tracker now exposes per-band delay/phase residual observations and a
|
|
`MimirDirectPathChannelModel` correction surface for multi-window
|
|
calibration; one-window self-correction is not live authority. The tracker
|
|
now consumes explicit channel models as phase/group-delay correction and
|
|
downweights bands outside the learned usable surface. `Mimir.BufferSmoke
|
|
--learn-complex-contour-channel-model` persists
|
|
`calibration/bioacoustic/complex-contour-channel-model.json` from the
|
|
replay panel, and `--evaluate-complex-contour-channel-model` persists
|
|
`calibration/bioacoustic/complex-contour-channel-model-evaluation.json`.
|
|
Current evaluation improves 3/4 absolute path-seed errors and lowers mean
|
|
direct-cluster MAE; the fresh cardioid case reaches about 0.202 us from
|
|
seed, while the stored cardioid case still worsens to about 17.476 us. The
|
|
model is explicit calibration evidence, not default runtime authority.
|
|
The complex contour receiver is now wired into runtime as an explicit live
|
|
lane: when `enableComplexContourRuntime` is true, Mimir emits the configured
|
|
`bioacousticWitnessProfileId` through `MimirBioacousticContestantRenderer`,
|
|
loads `complexContourChannelModelPath`, extracts Float32 windows from the
|
|
rolling buffers, and publishes `evidence=complex-contour` reports. The
|
|
synthetic runtime-shaped proof `--complex-contour-runtime-self-test` at
|
|
192 kHz recovers a 693.5-sample delay with about 0.219 us error.
|
|
A fresh live Scarlett proof is preserved at
|
|
`calibration/bioacoustic/complex-contour-live-20260525-063229.md`:
|
|
Focusrite ASIO playback/capture measured Loopback 1 -> Loopback 2 at
|
|
-0.014 us with corrected confidence 0.971. The apparent `asio-ch0`
|
|
near-seed/cardioid estimate is invalidated: signal inspection shows flat
|
|
low-level noise with zero-lag loopback correlation 0.000032, so the tracker
|
|
fit noise under seeded timing pressure. `asio-ch1` remains only a weak,
|
|
non-authoritative input witness. The next cut is signal-presence gating,
|
|
independent path truth, and better frequency/direct-path discrimination.
|
|
`calibration/bioacoustic/complex-contour-replay-panel.json` persists the
|
|
current four-case replay receipt.
|
|
Acoustic robustness remains open.
|
|
The old arbitrary-chirplet matcher is a diagnostic/reference artifact only,
|
|
not the runtime hot path.
|
|
- `IMimirVideoCaptureDriver` and `MimirVideoCaptureDriverSource` are the live
|
|
seam for Leap, Media Foundation, DirectShow, libusb, LeapC, or shared
|
|
texture capture.
|
|
- `native/reservoir` owns the lower shared-edge rolling-buffer invariant for
|
|
Fensalir/Faust integration.
|
|
- `config/perfect-machine.example.json` declares live owners, sensors,
|
|
reservoir views, and runtime policy.
|
|
- `src/Mimir.Runtime/Synchronization` now includes the Perfect Machine module
|
|
library distilled from the research notes: node profiles, decoder/language
|
|
profiles, path learning, benchmark panels, actuator strategies, capture and
|
|
ingest strategies, reservoir strategies, field profiles, compute offload,
|
|
OBS publication, CultMesh contracts, Fensalir lowering, and exportable
|
|
manifest generation.
|
|
- `docs/perfect-machine-module-library.md` indexes those reusable runtime
|
|
modules and their smoke commands.
|
|
- `docs/code-algorithm-map.md` is the source-level ownership and algorithm map
|
|
for the current repo code surface.
|
|
- `docs/perfect-machine-domain-index.md` indexes Mimir's full problem domain,
|
|
current evidence taxonomy, hot paths, and implementation cuts.
|
|
- `research/perfect-machine-study-2026-05-23/` preserves the architecture
|
|
rumination, optimization ledger, research references, source/hotspot audits,
|
|
Fensalir/native boundary maps, benchmark plan, calibration-session spec,
|
|
distributed receiver spec, failure-mode ledger, data dictionary, low-level
|
|
implementation notes, and sample code sketches for streaming chirp-bin
|
|
decode, AVX2/GPU scoring, batched native ABI, fractional-delay/SRO
|
|
actuation, acoustic source localization, Mimir-to-Fensalir lowering, and
|
|
native capture rings. The corrected deep-study pass ran from
|
|
`2026-05-23T23:54:42.9041588+01:00` through at least
|
|
`2026-05-24T01:54:56.9747630+01:00`; use
|
|
`research/perfect-machine-study-2026-05-23/reading-guide.md` as the
|
|
re-entry index.
|
|
- `.voidbot` state is part of repo memory and must be committed separately
|
|
whenever dirty.
|
|
|
|
active_subgoals:
|
|
- Implement the first concrete Leap direct capture driver and measure cadence.
|
|
- Add remaining local camera drivers through `IMimirVideoCaptureDriver`.
|
|
- Use the verified in-process Focusrite ASIO source for local Scarlett mic and
|
|
loopback ingest; WASAPI remains a diagnostic witness.
|
|
- Add network audio feed producers that enter the same runtime clock/rebuffer
|
|
machinery instead of becoming independent authorities.
|
|
- Bind Fensalir UI to stream health, buffer depth, source timestamps,
|
|
settings, and output management.
|
|
- Keep PowerShell/FFmpeg/SRT as bridge utilities until direct program output
|
|
replaces them.
|
|
|
|
current_status:
|
|
summary: >
|
|
The repo owns a C# Mimir app/runtime scaffold, a native reservoir crate, OBS
|
|
bridge scripts, and persistence surfaces. Script-era packages, tests,
|
|
helper tools, dependency manifests, and stale docs have been removed from
|
|
the working tree. Research archives remain as reference-only memory for raw
|
|
native/GPU implementation shape. A raw PS3 Eye libusb/WinUSB cadence probe
|
|
measured both PS3 Eyes simultaneously at about 183 fps each at 320x240 and
|
|
57-59 fps each at 640x480. A native Kernel Streaming LeapUVC probe can pull
|
|
frames through usbvideo.sys without Media Foundation, DirectShow, OpenCV, or
|
|
frame processing; after discarding a two-second warm-up, the useful 640x240
|
|
stereo IR mode advertises 115 fps and currently measures about 110.8 fps
|
|
with eight queued asynchronous KS reads. The half-height 640x120 mode
|
|
advertises 214 fps and currently measures about 205.9 fps. Plain Kiyo is
|
|
usable through KS when manual exposure is forced shorter: 1080p MJPG reaches
|
|
about 30 fps and 720p MJPG reaches about 51.5 fps. Kiyo Pro advertises 60
|
|
fps MJPG at 720p/1080p but currently delivers only about 25 fps through the
|
|
same KS path even with manual exposure tuning. Standard KS low-light
|
|
compensation off does not restore 60 fps, powerline 60Hz only raises Pro
|
|
cadence to about 29 fps, and standard extended Video HDR is not exposed on
|
|
this KS path. The controls-only probe now parses USB VideoControl
|
|
descriptors and found the Kiyo Pro's two vendor extension units. One
|
|
extension unit answers writable KS selectors. Moving the Pro directly to a
|
|
motherboard USB3 port changed the path to `root_hub30`, `bcdUSB=0x0320`,
|
|
and exposed 720p/1080p YUY2/MJPG/H264/NV12 at 60 fps, but Windows still
|
|
reports `UsbHighSpeed` rather than `UsbSuperSpeed`, and baseline cadence is
|
|
still about 25 fps across every tested format. The first simultaneous local
|
|
camera pull only had three visible devices: LeapUVC, Kiyo Pro, and one PS3
|
|
Eye. The KS multi-stream harness plus raw PS3 Eye probe pulled all three at
|
|
once, but Leap cadence collapsed under shared load: with PS3 Eye at
|
|
640x480@60 the set delivered Leap 40.12 fps, Kiyo Pro 24.99 fps, and PS3
|
|
Eye 56.06 fps; with PS3 Eye at 320x240@187 it delivered Leap 81.10 fps,
|
|
Kiyo Pro 25.33 fps, and PS3 Eye 185.74 fps. After port swapping, five
|
|
camera devices / six optical sensors were visible and pulled at once:
|
|
LeapUVC's packed stereo IR pair, Kiyo Pro, regular Kiyo, and two PS3 Eyes.
|
|
With both Eyes at 640x480@60, the set delivered Leap stereo 57.33 fps,
|
|
Kiyo Pro 24.97 fps, Kiyo 30.30 fps, PS3 Eye 0 59.59 fps, and PS3 Eye 1
|
|
57.60 fps. With both Eyes at 320x240@187, it delivered Leap stereo 81.05
|
|
fps, Kiyo Pro 24.97 fps, Kiyo 30.28 fps, PS3 Eye 0 187.60 fps, and PS3
|
|
Eye 1 187.03 fps. Kiyo Pro still negotiates high-speed USB, and Leap
|
|
remains topology-sensitive under shared load. A follow-up moved Leap alone
|
|
onto the blue rear USB3 group/root_hub30#5 while both Kiyos shared
|
|
root_hub30#7&18eea21b and one visible PS3 Eye used root_hub30#7&124f193f;
|
|
Leap still delivered only 80.99 fps with the other visible cameras active,
|
|
so the remaining loss is not explained by same-root-hub contention alone.
|
|
Splitting Kiyo Pro and one PS3 Eye to Raven left the Mimir host with
|
|
LeapUVC, regular Kiyo, and one PS3 Eye, but local cadence still stayed low:
|
|
Leap delivered 81.34 fps with Kiyo+Eye, 82.77 fps with Kiyo only, and only
|
|
84.18 fps in a same-session Leap-only baseline. The previous ~110.8 fps
|
|
Leap result was not reproduced until the Focusrite Scarlett Solo was moved
|
|
away from the Leap USB neighborhood. After that, Leap-only baseline rose to
|
|
95.48 fps, best full-height scenario to 101.67 fps, and the local split set
|
|
delivered Leap 105.10 fps, regular Kiyo 30.17 fps, and one PS3 Eye 185.81
|
|
fps. The split-host layout is viable if the Leap's USB neighborhood stays
|
|
free of the Focusrite and other heavy devices. Raven's PS3 Eye raw interface
|
|
now has the same libwdi WinUSB binding as the local Eye and can be opened by
|
|
the copied cadence probe, but Raven Eye cadence is currently unusable:
|
|
320x240@187 delivered 132.24 fps then 78.93 fps, and 640x480@60 delivered
|
|
6.32 fps with multi-second gaps. Moving that Eye back to the Mimir host
|
|
produced the current clean local topology: LeapUVC, regular Kiyo, and two
|
|
PS3 Eyes run in parallel while Kiyo Pro stays off-host. With both Eyes at
|
|
320x240@187, Leap delivered 106.07 fps, Kiyo 30.33 fps, and the Eyes 186.57
|
|
/ 186.14 fps. With both Eyes at 640x480@60, Leap delivered 108.07 fps, Kiyo
|
|
30.34 fps, and the Eyes 57.79 / 58.08 fps. Moving Kiyo Pro back to the
|
|
Mimir host while regular Kiyo was absent kept Leap healthy: with both Eyes
|
|
at 320x240@187, Leap delivered 110.80 fps, Kiyo Pro 24.97 fps, and the Eyes
|
|
187.46 / 186.89 fps; with both Eyes at 640x480@60, Leap delivered 110.72
|
|
fps, Kiyo Pro 24.87 fps, and the Eyes 59.91 / 59.19 fps. Kiyo Pro still
|
|
negotiates UsbHighSpeed(2), but it does not poison Leap cadence in the
|
|
current layout. Moving regular Kiyo from front panel to a rear port made
|
|
all five camera devices / six optical sensors visible locally. With both
|
|
Eyes at 320x240@187, the full set delivered Leap 110.84 fps, Kiyo Pro 25.08
|
|
fps, Kiyo 30.18 fps, PS3 Eye 0 187.59 fps, and PS3 Eye 1 187.02 fps. With
|
|
both Eyes at 640x480@60, it delivered Leap 110.76 fps, Kiyo Pro 25.14 fps,
|
|
Kiyo 30.18 fps, PS3 Eye 0 59.66 fps, and PS3 Eye 1 59.95 fps. A mouse in
|
|
the Leap USB neighborhood does not reproduce the Focusrite-induced collapse.
|
|
next_action: >
|
|
Use the new bioacoustic timeline watermark as the active acoustic sync
|
|
direction. Cut the next implementation in this order: streaming indexed
|
|
log-mel/MFCC motif decoder state with a global clock/path hypothesis fed by
|
|
`--bioacoustic-train` receipts, calibration-session command/replay proof that learns
|
|
per-output/mic motif/formant response, fractional-delay/SRO actuator, then
|
|
native camera payload handles and Fensalir lowering. Keep chirp-bin
|
|
calibration captures as reference evidence, not the runtime target.
|
|
Current app testing proves Fensalir WASAPI output wakes Scarlett speaker
|
|
loopback, live mic buffers stay populated, runtime sync analysis now runs as
|
|
a bounded rotating service without UI/telemetry re-entry, and the bioacoustic
|
|
motif timeline produces deterministic synthetic active sync. The next
|
|
invariant is self-identifying song contours producing stable intra-call
|
|
time/frequency anchors and clock fits through real loopback and microphones,
|
|
so the actuator can use decoded time instead of filtered lag guesses.
|
|
Replace the remaining
|
|
diagnostic JSON-line process bridges with direct native capture drivers that
|
|
push buffer handles into Mimir.Runtime. Mimir.BufferSmoke still proves live buffers and sync reports
|
|
for Scarlett loopback, Focusrite mic, Kiyo mic, and Kiyo Pro mic when
|
|
calibration witnesses are playing. Treat PS3 Eye audio as
|
|
enumeration/runtime-fragile, not unavailable. Keep the Focusrite away from
|
|
the Leap USB neighborhood. The Scarlett Solo 4th Gen is now attached to
|
|
Starfire, and `native/probes/asio_audio_cadence` proves local Focusrite USB
|
|
ASIO access with 4 inputs / 2 outputs including Loopback 1/2, 192-frame
|
|
preferred buffers, 44.1-192 kHz support, and nonzero 4-channel Int32LSB
|
|
input callbacks at 192 kHz. Raven also has a 192 kHz loopback-capable
|
|
Scarlett for co-streamer/game timing evidence, but Starfire owns the heavy
|
|
soundfield and sensor-fusion work. Treat Kiyo Pro as a ~25 fps source until
|
|
its SuperSpeed negotiation is fixed.
|