Add native ASIO runtime source
This commit is contained in:
parent
f0d3cf5815
commit
fad7eb63b7
20 changed files with 925 additions and 134 deletions
13
README.md
13
README.md
|
|
@ -162,8 +162,9 @@ hardware that earns the claim.
|
|||
|
||||
Next audio cuts:
|
||||
|
||||
1. Feed ASIO callbacks directly into `Mimir.Runtime` buffers.
|
||||
2. Use ASIO loopback as the canonical program/timing reference.
|
||||
1. Use ASIO loopback as the canonical program/timing reference.
|
||||
2. Learn per output/mic chirp-bin response models and adapt the codebook to the
|
||||
bands that survive the actual room.
|
||||
3. Align all mic streams with fractional-delay and sample-rate-offset state.
|
||||
4. Move the hot actuator into Faust/native DSP.
|
||||
5. Emit separately controllable OBS stems plus a spatial bed from the same
|
||||
|
|
@ -185,8 +186,11 @@ showed that generic dense chirplet matching is the wrong hot path for our
|
|||
controlled beacon. The current hybrid watermark uses `MimirChirpBinTimeline`: a
|
||||
fixed-slope chirp-bin codebook decoded by dechirp plus a small Goertzel/bin
|
||||
bank, then code-valid de Bruijn triplets become canonical timeline anchors.
|
||||
Reports expose fractional delay and `delayUs`; the synthetic proof currently
|
||||
recovers a `317.375` sample offset with about `0.369 us` error.
|
||||
Reports expose fractional delay and `delayUs`; the synthetic 192 kHz proof now
|
||||
recovers a delayed stream below printed microsecond precision. Physical
|
||||
calibration persists usable bands, expected/observed bin confusion, phase,
|
||||
group-delay, delay hypotheses, and an adaptive codebook plan per output/mic
|
||||
path.
|
||||
|
||||
For the long-form reasoning and source trail, read:
|
||||
|
||||
|
|
@ -288,7 +292,6 @@ Run the old OBS bridge only when you need the compatibility path:
|
|||
|
||||
Audio-side help is especially useful around:
|
||||
|
||||
- ASIO callback capture into `Mimir.Runtime` without stdout/process payloads.
|
||||
- Fractional-delay lines and variable-rate resampling.
|
||||
- Sample-rate-offset estimation over a rolling window.
|
||||
- GCC-PHAT/passive program-audio timing under music/game audio.
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
{
|
||||
"ModelId": "chirp-bin-20260523T192955Z",
|
||||
"CreatedAt": "2026-05-23T19:29:55.6287406+00:00",
|
||||
"ModelId": "chirp-bin-20260523T203522Z",
|
||||
"CreatedAt": "2026-05-23T20:35:22.527425+00:00",
|
||||
"SampleRate": 192000,
|
||||
"ReferenceSourceId": "asio-ch2",
|
||||
"EmissionPlan": {
|
||||
|
|
@ -22,6 +22,7 @@
|
|||
},
|
||||
"Paths": [
|
||||
{
|
||||
"OutputSourceId": "main-speakers",
|
||||
"SourceId": "asio-ch0",
|
||||
"SampleRate": 192000,
|
||||
"Profile": {
|
||||
|
|
@ -319,6 +320,7 @@
|
|||
"DelayHypotheses": []
|
||||
},
|
||||
{
|
||||
"OutputSourceId": "main-speakers",
|
||||
"SourceId": "asio-ch1",
|
||||
"SampleRate": 192000,
|
||||
"Profile": {
|
||||
|
|
@ -618,6 +620,7 @@
|
|||
{
|
||||
"SymbolId": 1,
|
||||
"Reliability": 0.02338947289373043,
|
||||
"PhaseCoherence": 0.8813023361164132,
|
||||
"ExpectedCenterHz": 4075,
|
||||
"ObservedCenterHz": 4468.75,
|
||||
"MeanEnergy": 0.9902022429801735,
|
||||
|
|
@ -629,6 +632,7 @@
|
|||
{
|
||||
"SymbolId": 2,
|
||||
"Reliability": 0.16959717331712612,
|
||||
"PhaseCoherence": 0.7788915227933655,
|
||||
"ExpectedCenterHz": 4525,
|
||||
"ObservedCenterHz": 4475,
|
||||
"MeanEnergy": 0.9196472718940074,
|
||||
|
|
@ -640,6 +644,7 @@
|
|||
{
|
||||
"SymbolId": 3,
|
||||
"Reliability": 1.148185667935492,
|
||||
"PhaseCoherence": 1,
|
||||
"ExpectedCenterHz": 4975,
|
||||
"ObservedCenterHz": 4975,
|
||||
"MeanEnergy": 1.148185667935492,
|
||||
|
|
@ -651,6 +656,7 @@
|
|||
{
|
||||
"SymbolId": 4,
|
||||
"Reliability": 0,
|
||||
"PhaseCoherence": 1,
|
||||
"ExpectedCenterHz": 5425,
|
||||
"ObservedCenterHz": 4975,
|
||||
"MeanEnergy": 0.30966049519505534,
|
||||
|
|
@ -662,6 +668,7 @@
|
|||
{
|
||||
"SymbolId": 6,
|
||||
"Reliability": 0,
|
||||
"PhaseCoherence": 1,
|
||||
"ExpectedCenterHz": 6325,
|
||||
"ObservedCenterHz": 6775,
|
||||
"MeanEnergy": 0.8679642624875925,
|
||||
|
|
@ -673,6 +680,7 @@
|
|||
{
|
||||
"SymbolId": 7,
|
||||
"Reliability": 0,
|
||||
"PhaseCoherence": 0.9999999999999999,
|
||||
"ExpectedCenterHz": 6775,
|
||||
"ObservedCenterHz": 7225,
|
||||
"MeanEnergy": 0.5299773190124039,
|
||||
|
|
@ -684,6 +692,7 @@
|
|||
{
|
||||
"SymbolId": 8,
|
||||
"Reliability": 0.6732785913925556,
|
||||
"PhaseCoherence": 1,
|
||||
"ExpectedCenterHz": 7225,
|
||||
"ObservedCenterHz": 7225,
|
||||
"MeanEnergy": 0.6732785913925556,
|
||||
|
|
@ -984,6 +993,7 @@
|
|||
"DelayHypotheses": []
|
||||
},
|
||||
{
|
||||
"OutputSourceId": "main-speakers",
|
||||
"SourceId": "asio-ch2",
|
||||
"SampleRate": 192000,
|
||||
"Profile": {
|
||||
|
|
@ -1290,6 +1300,7 @@
|
|||
{
|
||||
"SymbolId": 0,
|
||||
"Reliability": 1.9664832701324948,
|
||||
"PhaseCoherence": 0.9999999999999998,
|
||||
"ExpectedCenterHz": 3625,
|
||||
"ObservedCenterHz": 3625,
|
||||
"MeanEnergy": 1.9831606924956895,
|
||||
|
|
@ -1301,6 +1312,7 @@
|
|||
{
|
||||
"SymbolId": 1,
|
||||
"Reliability": 1.9550716359955316,
|
||||
"PhaseCoherence": 0.9999608311103569,
|
||||
"ExpectedCenterHz": 4075,
|
||||
"ObservedCenterHz": 4075,
|
||||
"MeanEnergy": 1.9724028422080755,
|
||||
|
|
@ -1312,6 +1324,7 @@
|
|||
{
|
||||
"SymbolId": 2,
|
||||
"Reliability": 1.98317479150165,
|
||||
"PhaseCoherence": 1,
|
||||
"ExpectedCenterHz": 4525,
|
||||
"ObservedCenterHz": 4525,
|
||||
"MeanEnergy": 1.98317479150165,
|
||||
|
|
@ -1323,6 +1336,7 @@
|
|||
{
|
||||
"SymbolId": 3,
|
||||
"Reliability": 1.9831581977468316,
|
||||
"PhaseCoherence": 1,
|
||||
"ExpectedCenterHz": 4975,
|
||||
"ObservedCenterHz": 4975,
|
||||
"MeanEnergy": 1.9831581977468316,
|
||||
|
|
@ -1334,6 +1348,7 @@
|
|||
{
|
||||
"SymbolId": 4,
|
||||
"Reliability": 1.9831595137173625,
|
||||
"PhaseCoherence": 1,
|
||||
"ExpectedCenterHz": 5425,
|
||||
"ObservedCenterHz": 5425,
|
||||
"MeanEnergy": 1.9831595137173625,
|
||||
|
|
@ -1345,6 +1360,7 @@
|
|||
{
|
||||
"SymbolId": 5,
|
||||
"Reliability": 1.983165098459816,
|
||||
"PhaseCoherence": 1,
|
||||
"ExpectedCenterHz": 5875,
|
||||
"ObservedCenterHz": 5875,
|
||||
"MeanEnergy": 1.983165098459816,
|
||||
|
|
@ -1356,6 +1372,7 @@
|
|||
{
|
||||
"SymbolId": 6,
|
||||
"Reliability": 1.9831633177029786,
|
||||
"PhaseCoherence": 1,
|
||||
"ExpectedCenterHz": 6325,
|
||||
"ObservedCenterHz": 6325,
|
||||
"MeanEnergy": 1.9831633177029786,
|
||||
|
|
@ -1367,6 +1384,7 @@
|
|||
{
|
||||
"SymbolId": 7,
|
||||
"Reliability": 1.9831607953145476,
|
||||
"PhaseCoherence": 0.9999999999999999,
|
||||
"ExpectedCenterHz": 6775,
|
||||
"ObservedCenterHz": 6775,
|
||||
"MeanEnergy": 1.9831607953145476,
|
||||
|
|
@ -1378,6 +1396,7 @@
|
|||
{
|
||||
"SymbolId": 31,
|
||||
"Reliability": 1.9831585964902498,
|
||||
"PhaseCoherence": 1,
|
||||
"ExpectedCenterHz": 17575,
|
||||
"ObservedCenterHz": 17575,
|
||||
"MeanEnergy": 1.9831585964902498,
|
||||
|
|
@ -1733,6 +1752,7 @@
|
|||
]
|
||||
},
|
||||
{
|
||||
"OutputSourceId": "main-speakers",
|
||||
"SourceId": "asio-ch3",
|
||||
"SampleRate": 192000,
|
||||
"Profile": {
|
||||
|
|
@ -2039,6 +2059,7 @@
|
|||
{
|
||||
"SymbolId": 0,
|
||||
"Reliability": 1.9664832701324948,
|
||||
"PhaseCoherence": 0.9999999999999998,
|
||||
"ExpectedCenterHz": 3625,
|
||||
"ObservedCenterHz": 3625,
|
||||
"MeanEnergy": 1.9831606924956895,
|
||||
|
|
@ -2050,6 +2071,7 @@
|
|||
{
|
||||
"SymbolId": 1,
|
||||
"Reliability": 1.9550716359955316,
|
||||
"PhaseCoherence": 0.9999608311103569,
|
||||
"ExpectedCenterHz": 4075,
|
||||
"ObservedCenterHz": 4075,
|
||||
"MeanEnergy": 1.9724028422080755,
|
||||
|
|
@ -2061,6 +2083,7 @@
|
|||
{
|
||||
"SymbolId": 2,
|
||||
"Reliability": 1.98317479150165,
|
||||
"PhaseCoherence": 1,
|
||||
"ExpectedCenterHz": 4525,
|
||||
"ObservedCenterHz": 4525,
|
||||
"MeanEnergy": 1.98317479150165,
|
||||
|
|
@ -2072,6 +2095,7 @@
|
|||
{
|
||||
"SymbolId": 3,
|
||||
"Reliability": 1.9831581977468316,
|
||||
"PhaseCoherence": 1,
|
||||
"ExpectedCenterHz": 4975,
|
||||
"ObservedCenterHz": 4975,
|
||||
"MeanEnergy": 1.9831581977468316,
|
||||
|
|
@ -2083,6 +2107,7 @@
|
|||
{
|
||||
"SymbolId": 4,
|
||||
"Reliability": 1.9831595137173625,
|
||||
"PhaseCoherence": 1,
|
||||
"ExpectedCenterHz": 5425,
|
||||
"ObservedCenterHz": 5425,
|
||||
"MeanEnergy": 1.9831595137173625,
|
||||
|
|
@ -2094,6 +2119,7 @@
|
|||
{
|
||||
"SymbolId": 5,
|
||||
"Reliability": 1.983165098459816,
|
||||
"PhaseCoherence": 1,
|
||||
"ExpectedCenterHz": 5875,
|
||||
"ObservedCenterHz": 5875,
|
||||
"MeanEnergy": 1.983165098459816,
|
||||
|
|
@ -2105,6 +2131,7 @@
|
|||
{
|
||||
"SymbolId": 6,
|
||||
"Reliability": 1.9831633177029786,
|
||||
"PhaseCoherence": 1,
|
||||
"ExpectedCenterHz": 6325,
|
||||
"ObservedCenterHz": 6325,
|
||||
"MeanEnergy": 1.9831633177029786,
|
||||
|
|
@ -2116,6 +2143,7 @@
|
|||
{
|
||||
"SymbolId": 7,
|
||||
"Reliability": 1.9831607953145476,
|
||||
"PhaseCoherence": 0.9999999999999999,
|
||||
"ExpectedCenterHz": 6775,
|
||||
"ObservedCenterHz": 6775,
|
||||
"MeanEnergy": 1.9831607953145476,
|
||||
|
|
@ -2127,6 +2155,7 @@
|
|||
{
|
||||
"SymbolId": 31,
|
||||
"Reliability": 1.9831585964902498,
|
||||
"PhaseCoherence": 1,
|
||||
"ExpectedCenterHz": 17575,
|
||||
"ObservedCenterHz": 17575,
|
||||
"MeanEnergy": 1.9831585964902498,
|
||||
|
|
|
|||
|
|
@ -13,13 +13,9 @@
|
|||
"kind": "Audio",
|
||||
"origin": "LocalDevice",
|
||||
"enabled": true,
|
||||
"adapter": "frame-events",
|
||||
"command": "../native/probes/asio_audio_cadence/build/Release/asio_audio_cadence.exe",
|
||||
"arguments": [
|
||||
"--set-sample-rate",
|
||||
"192000",
|
||||
"--emit-json-blocks"
|
||||
],
|
||||
"adapter": "asio",
|
||||
"command": "../native/asio_capture/build/Release/mimir_asio_capture.dll",
|
||||
"sampleRate": 192000,
|
||||
"acceptSourceIds": [
|
||||
"asio-ch0",
|
||||
"asio-ch1",
|
||||
|
|
|
|||
|
|
@ -13,13 +13,9 @@
|
|||
"kind": "Audio",
|
||||
"origin": "LocalDevice",
|
||||
"enabled": true,
|
||||
"adapter": "frame-events",
|
||||
"command": "../native/probes/asio_audio_cadence/build/Release/asio_audio_cadence.exe",
|
||||
"arguments": [
|
||||
"--set-sample-rate",
|
||||
"192000",
|
||||
"--emit-json-blocks"
|
||||
],
|
||||
"adapter": "asio",
|
||||
"command": "../native/asio_capture/build/Release/mimir_asio_capture.dll",
|
||||
"sampleRate": 192000,
|
||||
"acceptSourceIds": [
|
||||
"asio-ch0",
|
||||
"asio-ch1",
|
||||
|
|
|
|||
|
|
@ -78,11 +78,12 @@ per-band response evidence, so the same stream can calibrate timing and the
|
|||
speaker/room/mic transfer function. `MimirChirpBinCalibrationModel` is the live
|
||||
surface for that evidence. It stores measured usable bands per source,
|
||||
expected-symbol versus observed-bin confusion observations, timing residuals,
|
||||
delay hypotheses, phase summaries, and an adaptive codebook plan. The raw
|
||||
profile is kept even when a timing report is rejected, because a failed sync
|
||||
window can still teach us which bands survived the speaker/room/mic path. The
|
||||
decoder can consume the persisted model to weight reliable symbols, downweight
|
||||
dead bands, apply first-order group-delay correction, and seed global delay
|
||||
delay hypotheses, phase summaries, and an adaptive codebook plan per
|
||||
output/mic path. The raw profile is kept even when a timing report is rejected,
|
||||
because a failed sync window can still teach us which bands survived the
|
||||
speaker/room/mic path. The decoder can consume the persisted model to weight
|
||||
reliable symbols, downweight dead bands, apply phase-coherence weighting,
|
||||
apply first-order group-delay correction, and search joint global delay/bin-shift
|
||||
hypotheses before selecting the coherent anchor path. The model also owns the
|
||||
active emission plan: if the measured path only leaves a smaller alphabet, Mimir
|
||||
emits that reliable symbol set and raises the de Bruijn order so the timeline
|
||||
|
|
@ -130,11 +131,12 @@ buffer, and captures nonzero 4-channel `Int32LSB` input callbacks at 192 kHz.
|
|||
Raven also has a loopback-capable Scarlett ASIO path at 192 kHz for
|
||||
co-streamer/game timing evidence, but Starfire owns the heavy local alignment,
|
||||
soundfield, and sensor-fusion work.
|
||||
The worker now also supports `--emit-json-blocks`, which streams each ASIO
|
||||
callback as sample-bearing `audio-block` events (`asio-ch0` through `asio-ch3`)
|
||||
for `MimirFrameEventProcessStreamSource`. That is a continuous ASIO runtime
|
||||
ingest path through the existing process boundary; the later native ABI cut
|
||||
should remove stdout/base64 overhead, not change ASIO ownership.
|
||||
The runtime hot path is now `native/asio_capture` plus
|
||||
`MimirAsioStreamSource`. The native DLL owns the Focusrite ASIO COM driver and
|
||||
keeps callback buffers in process; the managed source drains one 192 kHz Float32
|
||||
block per ASIO input channel into `Mimir.Runtime` buffers (`asio-ch0` through
|
||||
`asio-ch3`). The older `--emit-json-blocks` worker path remains a diagnostic
|
||||
probe, not the live Scarlett ingest path.
|
||||
|
||||
The probe also has a `--monitor-sweep` mode for measuring the acoustic ceiling.
|
||||
At 192 kHz, ASIO loopback detected emitted 8-40 kHz tones cleanly, but the
|
||||
|
|
@ -146,8 +148,11 @@ The synthetic chirp-bin timing proof now runs at arbitrary sample rates with
|
|||
`--hybrid-sync-self-test --sample-rate N`. On the same physical delay, measured
|
||||
in memory, the active chirp-bin path recovered about 0.369 us error at 48 kHz,
|
||||
0.168 us at 96 kHz, and below printed microsecond precision at 192 kHz. That
|
||||
proves the timing kernel benefits from the Scarlett ASIO rate; the live proof
|
||||
still depends on feeding ASIO callbacks into `Mimir.Runtime`.
|
||||
proves the timing kernel benefits from the Scarlett ASIO rate. The live ingest
|
||||
proof now feeds ASIO callbacks into `Mimir.Runtime` without a process/stdout
|
||||
bridge: a two-second BufferSmoke run at 192 kHz ingested more than 12,000
|
||||
sample-bearing blocks and retained 2,048 blocks per channel across `asio-ch0`
|
||||
through `asio-ch3`.
|
||||
|
||||
The full probe runtime config now enables sample-bearing blocks for every local
|
||||
audio source. `MimirChirpletTimeline` owns the emitted calibration stream and
|
||||
|
|
@ -269,8 +274,9 @@ The older `MimirChirpletTimeline` matcher is a diagnostic/reference artifact,
|
|||
not the active runtime path. `BuildChirpletEnergyTrace` still behaves like a
|
||||
dense sliding matched-filter bank and should not retake authority.
|
||||
|
||||
Next, replace the diagnostic bridge with native audio capture workers that
|
||||
append typed blocks into `Mimir.Runtime`, then expose buffer depth, clock state,
|
||||
delay estimates, and stem routing in Fensalir UI. The analyzer accepts Float32,
|
||||
Int16, Int24, and Int32 PCM windows so the direct driver path can preserve real
|
||||
interface formats before Faust/native DSP owns the hot resampling and alignment.
|
||||
Next, use the in-process ASIO source as the local Scarlett authority, prove
|
||||
stable acoustic anchors against real mics, then expose buffer depth, clock
|
||||
state, delay estimates, and stem routing in Fensalir UI. The analyzer accepts
|
||||
Float32, Int16, Int24, and Int32 PCM windows so direct driver paths can preserve
|
||||
real interface formats before Faust/native DSP owns the hot resampling and
|
||||
alignment.
|
||||
|
|
|
|||
|
|
@ -36,6 +36,10 @@ a named invariant that the native runtime cannot protect yet.
|
|||
from native probes into the same rolling buffers Fensalir inspects. One probe
|
||||
process can accept multiple emitted `sourceId` values so it does not reopen
|
||||
the same camera set once per stream.
|
||||
- `native/asio_capture` plus `MimirAsioStreamSource` provide the first
|
||||
production-shaped Focusrite path: a native in-process ASIO callback source
|
||||
feeds sample-bearing 192 kHz Float32 blocks directly into `Mimir.Runtime`
|
||||
rolling buffers on one interface clock domain.
|
||||
- `src/Mimir.BufferSmoke` loads the runtime config, polls the synchronization
|
||||
hub, and prints the actual rolling buffers. Use `--require-samples` when an
|
||||
empty declared sensor buffer should fail the run. Use `--chirplet-self-test`
|
||||
|
|
@ -106,7 +110,8 @@ a named invariant that the native runtime cannot protect yet.
|
|||
bands, expected-symbol versus observed-bin confusion observations, timing
|
||||
residuals, delay hypotheses, phase summaries, and an adaptive codebook plan.
|
||||
The active decoder can consume that model as learned response weighting,
|
||||
first-order group-delay correction, and global delay-hypothesis seeds. The
|
||||
phase-coherence weighting, first-order group-delay correction, and joint
|
||||
global delay/bin-shift hypotheses. The
|
||||
runtime emitter also consumes the model's emission plan, rendering the smaller
|
||||
reliable symbol alphabet at the higher recommended de Bruijn order when the
|
||||
physical path cannot support all 32 bins.
|
||||
|
|
@ -129,13 +134,14 @@ a named invariant that the native runtime cannot protect yet.
|
|||
loopback timing proof, but failed timing windows now leave usable response
|
||||
evidence instead of silence.
|
||||
- `config/mimir-runtime.asio.example.json` is the minimal continuous Scarlett
|
||||
runtime ingest proof. It starts the ASIO worker at 192 kHz with
|
||||
`--emit-json-blocks` and declares `asio-ch0` through `asio-ch3` as accepted
|
||||
audio sources. A two-second BufferSmoke run ingested 13,324 sample-bearing
|
||||
blocks, 3,331 per channel, proving loopback and mic channels enter
|
||||
`Mimir.Runtime` together in one ASIO clock domain. BufferSmoke does not emit
|
||||
speaker calibration audio, so that proof is ingest-only rather than a sync
|
||||
report.
|
||||
runtime ingest proof. It loads `native/asio_capture` in process at 192 kHz
|
||||
and declares `asio-ch0` through `asio-ch3` as accepted audio sources. A
|
||||
two-second BufferSmoke run ingested more than 12,000 sample-bearing blocks
|
||||
and retained 2,048 blocks per channel, proving loopback and mic channels enter
|
||||
`Mimir.Runtime` together in one ASIO clock domain without stdout/base64
|
||||
transport. BufferSmoke does not
|
||||
emit speaker calibration audio, so that proof is ingest-only rather than a
|
||||
sync report.
|
||||
A standalone 192 kHz synthetic receiver test recovers a 500 ms delayed audio
|
||||
stream to below printed microsecond precision using only the chirp-bin
|
||||
codebook and schedule state.
|
||||
|
|
@ -170,26 +176,22 @@ a named invariant that the native runtime cannot protect yet.
|
|||
other cameras.
|
||||
2. Feed those drivers into `MimirVideoCaptureDriverSource` and prove sustained
|
||||
frame cadence in the rolling buffers.
|
||||
3. Replace the WASAPI frame-event diagnostic bridge with native audio capture
|
||||
workers for local mic, loopback, and network audio feeds. Use the verified
|
||||
Focusrite ASIO path for Scarlett capture; WASAPI remains a diagnostic
|
||||
witness, not the Scarlett hot path.
|
||||
4. Use chirp-bin calibration profiles from real microphones to tune acoustic
|
||||
3. Use chirp-bin calibration profiles from real microphones to tune acoustic
|
||||
bands, gain, and code spacing without weakening the standalone
|
||||
codebook/schedule receiver invariant.
|
||||
5. Add the synchronization actuator: drive a variable-rate resampler and
|
||||
4. Add the synchronization actuator: drive a variable-rate resampler and
|
||||
fractional delay line per non-reference stream from the smoothed
|
||||
`MimirAudioSynchronizationState`. First, prove the constrained chirplet
|
||||
decoder through real loopback and microphone paths so every correctly heard
|
||||
triplet becomes a deterministic timeline anchor before the actuator moves
|
||||
samples.
|
||||
6. Prove the chirp-bin hybrid fallback through real loopback and microphones
|
||||
5. Prove the chirp-bin hybrid fallback through real loopback and microphones
|
||||
with probe durations long enough to keep loopback and mic windows live.
|
||||
7. Bind Fensalir UI to the synchronization hub so buffer depth, stream cadence,
|
||||
6. Bind Fensalir UI to the synchronization hub so buffer depth, stream cadence,
|
||||
source timestamps, and output settings are visible and adjustable.
|
||||
8. Move GPU feature extraction, fusion, material fitting, render budgeting, and
|
||||
7. Move GPU feature extraction, fusion, material fitting, render budgeting, and
|
||||
Spout2 publication into Fensalir.
|
||||
9. Move mic alignment, room suppression, voice separation, spatialization, and
|
||||
8. Move mic alignment, room suppression, voice separation, spatialization, and
|
||||
stem generation into Faust/native DSP.
|
||||
10. Keep the OBS bridge witness ledger as evidence before expanding receiver
|
||||
9. Keep the OBS bridge witness ledger as evidence before expanding receiver
|
||||
machinery.
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@ This directory is for the hot path.
|
|||
|
||||
- `reservoir`: the first native five-second spatiotemporal reservoir core. It
|
||||
owns one shared-edge rolling buffer with typed sample views.
|
||||
Aquarium/Faust integration should build on this invariant instead of copying
|
||||
Fensalir/Faust integration should build on this invariant instead of copying
|
||||
the deadline bridge cache.
|
||||
It exports `rlib`, `cdylib`, and `staticlib` artifacts plus the C header at
|
||||
`reservoir/include/localcast_reservoir.h`.
|
||||
|
|
@ -35,16 +35,16 @@ The reservoir ABI is intentionally small:
|
|||
|
||||
## Runtime ABI
|
||||
|
||||
`LocalcastRuntime` is the live spine for Aquarium/Faust integration:
|
||||
`LocalcastRuntime` is the live spine for Fensalir/Faust integration:
|
||||
|
||||
- it owns one native reservoir;
|
||||
- it exposes typed producer functions for camera frames, camera features, scene
|
||||
rays, surface claims, material claims, audio blocks, phase claims, event
|
||||
claims, and render packets;
|
||||
- it exposes `LocalcastRuntimeStatus` so Aquarium/Faust can inspect the shared
|
||||
- it exposes `LocalcastRuntimeStatus` so Fensalir/Faust can inspect the shared
|
||||
edge, window start, total sample count, and typed-view counts without polling JSON or
|
||||
MessagePack files;
|
||||
- it exposes total and typed-view sample reads so Aquarium/Faust can consume the
|
||||
- it exposes total and typed-view sample reads so Fensalir/Faust can consume the
|
||||
same rolling window producers append to.
|
||||
|
||||
`LocalcastProducer` is the native ingress helper for capture workers:
|
||||
|
|
@ -69,16 +69,16 @@ The reservoir ABI is intentionally small:
|
|||
optional metadata handle;
|
||||
- `localcast_producer_push_render_packet` only accepts `RenderPacket`
|
||||
producers and stores the descriptor pointer as the sample payload handle;
|
||||
- Aquarium owns decoding and GPU upload. The reservoir only indexes when this
|
||||
- Fensalir owns decoding and GPU upload. The reservoir only indexes when this
|
||||
render packet exists in the shared rolling window.
|
||||
|
||||
`LocalcastRenderPoint` is the fixed point-buffer element shape for the initial
|
||||
native/Aquarium render path. It carries a stable-key hash, source timestamp,
|
||||
native/Fensalir render path. It carries a stable-key hash, source timestamp,
|
||||
position, radius, color/alpha, and confidence. Richer material/brush payloads
|
||||
can get their own descriptors later; this point shape exists to kill the old
|
||||
JSON/CultCache render-frame dependency first.
|
||||
|
||||
Payload handles are owned by the caller. Aquarium should treat them as handles
|
||||
Payload handles are owned by the caller. Fensalir should treat them as handles
|
||||
to GPU/native visual memory; Faust/native DSP should treat them as handles to
|
||||
audio buffers. This crate does not interpret those bytes. It is the clocked
|
||||
retention authority, not the renderer or DSP engine in a tiny hat.
|
||||
|
|
|
|||
15
native/asio_capture/CMakeLists.txt
Normal file
15
native/asio_capture/CMakeLists.txt
Normal file
|
|
@ -0,0 +1,15 @@
|
|||
cmake_minimum_required(VERSION 3.20)
|
||||
|
||||
project(mimir_asio_capture LANGUAGES CXX)
|
||||
|
||||
add_library(mimir_asio_capture SHARED
|
||||
mimir_asio_capture.cpp)
|
||||
|
||||
target_compile_features(mimir_asio_capture PRIVATE cxx_std_20)
|
||||
target_compile_definitions(mimir_asio_capture PRIVATE
|
||||
WIN32_LEAN_AND_MEAN
|
||||
NOMINMAX
|
||||
_WIN32_WINNT=0x0A00)
|
||||
target_link_libraries(mimir_asio_capture PRIVATE
|
||||
ole32
|
||||
oleaut32)
|
||||
405
native/asio_capture/mimir_asio_capture.cpp
Normal file
405
native/asio_capture/mimir_asio_capture.cpp
Normal file
|
|
@ -0,0 +1,405 @@
|
|||
#include <windows.h>
|
||||
#include <objbase.h>
|
||||
|
||||
#include <algorithm>
|
||||
#include <atomic>
|
||||
#include <condition_variable>
|
||||
#include <cstdint>
|
||||
#include <cstdio>
|
||||
#include <cstring>
|
||||
#include <deque>
|
||||
#include <mutex>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
using ASIOBool = long;
|
||||
using ASIOError = long;
|
||||
using ASIOSampleRate = double;
|
||||
using ASIOSampleType = long;
|
||||
|
||||
constexpr ASIOError ASE_OK = 0;
|
||||
|
||||
struct ASIOSamples
|
||||
{
|
||||
unsigned long hi;
|
||||
unsigned long lo;
|
||||
};
|
||||
|
||||
struct ASIOTimeStamp
|
||||
{
|
||||
unsigned long hi;
|
||||
unsigned long lo;
|
||||
};
|
||||
|
||||
struct ASIOClockSource
|
||||
{
|
||||
long index;
|
||||
long associatedChannel;
|
||||
long associatedGroup;
|
||||
ASIOBool isCurrentSource;
|
||||
char name[32];
|
||||
};
|
||||
|
||||
struct ASIOChannelInfo
|
||||
{
|
||||
long channel;
|
||||
ASIOBool isInput;
|
||||
ASIOBool isActive;
|
||||
long channelGroup;
|
||||
ASIOSampleType type;
|
||||
char name[32];
|
||||
};
|
||||
|
||||
struct ASIOBufferInfo
|
||||
{
|
||||
ASIOBool isInput;
|
||||
long channelNum;
|
||||
void* buffers[2];
|
||||
};
|
||||
|
||||
struct ASIOTime;
|
||||
|
||||
struct ASIOCallbacks
|
||||
{
|
||||
void (*bufferSwitch)(long doubleBufferIndex, ASIOBool directProcess);
|
||||
void (*sampleRateDidChange)(ASIOSampleRate sRate);
|
||||
long (*asioMessage)(long selector, long value, void* message, double* opt);
|
||||
ASIOTime* (*bufferSwitchTimeInfo)(ASIOTime* params, long doubleBufferIndex, ASIOBool directProcess);
|
||||
};
|
||||
|
||||
struct IASIO : public IUnknown
|
||||
{
|
||||
virtual ASIOBool init(void* sysHandle) = 0;
|
||||
virtual void getDriverName(char* name) = 0;
|
||||
virtual long getDriverVersion() = 0;
|
||||
virtual void getErrorMessage(char* string) = 0;
|
||||
virtual ASIOError start() = 0;
|
||||
virtual ASIOError stop() = 0;
|
||||
virtual ASIOError getChannels(long* numInputChannels, long* numOutputChannels) = 0;
|
||||
virtual ASIOError getLatencies(long* inputLatency, long* outputLatency) = 0;
|
||||
virtual ASIOError getBufferSize(long* minSize, long* maxSize, long* preferredSize, long* granularity) = 0;
|
||||
virtual ASIOError canSampleRate(ASIOSampleRate sampleRate) = 0;
|
||||
virtual ASIOError getSampleRate(ASIOSampleRate* sampleRate) = 0;
|
||||
virtual ASIOError setSampleRate(ASIOSampleRate sampleRate) = 0;
|
||||
virtual ASIOError getClockSources(ASIOClockSource* clocks, long* numSources) = 0;
|
||||
virtual ASIOError setClockSource(long reference) = 0;
|
||||
virtual ASIOError getSamplePosition(ASIOSamples* samplePosition, ASIOTimeStamp* timeStamp) = 0;
|
||||
virtual ASIOError getChannelInfo(ASIOChannelInfo* info) = 0;
|
||||
virtual ASIOError createBuffers(ASIOBufferInfo* bufferInfos, long numChannels, long bufferSize, ASIOCallbacks* callbacks) = 0;
|
||||
virtual ASIOError disposeBuffers() = 0;
|
||||
virtual ASIOError controlPanel() = 0;
|
||||
virtual ASIOError future(long selector, void* opt) = 0;
|
||||
virtual ASIOError outputReady() = 0;
|
||||
};
|
||||
|
||||
struct MimirAsioBlock
|
||||
{
|
||||
int channel = 0;
|
||||
int sampleRate = 0;
|
||||
int frameCount = 0;
|
||||
std::uint64_t sequence = 0;
|
||||
std::int64_t timestampNs = 0;
|
||||
std::vector<float> samples;
|
||||
};
|
||||
|
||||
struct MimirAsioCapture
|
||||
{
|
||||
IASIO* driver = nullptr;
|
||||
ASIOCallbacks callbacks{};
|
||||
std::vector<ASIOBufferInfo> buffers;
|
||||
std::vector<ASIOChannelInfo> channels;
|
||||
std::deque<MimirAsioBlock> queue;
|
||||
std::mutex mutex;
|
||||
long inputs = 0;
|
||||
long outputs = 0;
|
||||
long bufferSize = 0;
|
||||
double sampleRate = 0.0;
|
||||
std::uint64_t frameCursor = 0;
|
||||
std::uint64_t sequence = 0;
|
||||
bool running = false;
|
||||
};
|
||||
|
||||
static MimirAsioCapture* g_capture = nullptr;
|
||||
|
||||
static int bytesPerSample(ASIOSampleType type)
|
||||
{
|
||||
switch (type)
|
||||
{
|
||||
case 16: return 2;
|
||||
case 17: return 3;
|
||||
case 18: return 4;
|
||||
case 19: return 4;
|
||||
case 20: return 8;
|
||||
case 24:
|
||||
case 25:
|
||||
case 26:
|
||||
case 27: return 4;
|
||||
default: return 0;
|
||||
}
|
||||
}
|
||||
|
||||
static double readSample(const unsigned char* sample, ASIOSampleType type)
|
||||
{
|
||||
switch (type)
|
||||
{
|
||||
case 16:
|
||||
return *reinterpret_cast<const short*>(sample) / 32768.0;
|
||||
case 17:
|
||||
{
|
||||
int value = sample[0] | (sample[1] << 8) | (sample[2] << 16);
|
||||
if ((value & 0x00800000) != 0)
|
||||
{
|
||||
value |= static_cast<int>(0xff000000);
|
||||
}
|
||||
return value / 8388608.0;
|
||||
}
|
||||
case 18:
|
||||
case 24:
|
||||
case 25:
|
||||
case 26:
|
||||
case 27:
|
||||
return *reinterpret_cast<const int*>(sample) / 2147483648.0;
|
||||
case 19:
|
||||
return *reinterpret_cast<const float*>(sample);
|
||||
case 20:
|
||||
return *reinterpret_cast<const double*>(sample);
|
||||
default:
|
||||
return 0.0;
|
||||
}
|
||||
}
|
||||
|
||||
static CLSID clsidFromString(const char* raw)
|
||||
{
|
||||
wchar_t wide[64]{};
|
||||
MultiByteToWideChar(CP_UTF8, 0, raw, -1, wide, static_cast<int>(std::size(wide)));
|
||||
CLSID clsid{};
|
||||
CLSIDFromString(wide, &clsid);
|
||||
return clsid;
|
||||
}
|
||||
|
||||
static void pushInputBlocks(MimirAsioCapture& capture, long doubleBufferIndex)
|
||||
{
|
||||
const auto startFrame = capture.frameCursor;
|
||||
const auto timestampNs = static_cast<std::int64_t>(
|
||||
static_cast<double>(startFrame) * 1000000000.0 / capture.sampleRate);
|
||||
std::lock_guard<std::mutex> lock(capture.mutex);
|
||||
for (long channel = 0; channel < capture.inputs; ++channel)
|
||||
{
|
||||
auto& buffer = capture.buffers[static_cast<size_t>(channel)];
|
||||
auto& info = capture.channels[static_cast<size_t>(channel)];
|
||||
auto* raw = static_cast<unsigned char*>(buffer.buffers[doubleBufferIndex]);
|
||||
const auto sampleBytes = bytesPerSample(info.type);
|
||||
if (!raw || sampleBytes <= 0)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
MimirAsioBlock block;
|
||||
block.channel = static_cast<int>(channel);
|
||||
block.sampleRate = static_cast<int>(capture.sampleRate);
|
||||
block.frameCount = static_cast<int>(capture.bufferSize);
|
||||
block.sequence = capture.sequence++;
|
||||
block.timestampNs = timestampNs;
|
||||
block.samples.resize(static_cast<size_t>(capture.bufferSize));
|
||||
for (long frame = 0; frame < capture.bufferSize; ++frame)
|
||||
{
|
||||
block.samples[static_cast<size_t>(frame)] = static_cast<float>(
|
||||
readSample(raw + frame * sampleBytes, info.type));
|
||||
}
|
||||
|
||||
capture.queue.push_back(std::move(block));
|
||||
}
|
||||
|
||||
while (capture.queue.size() > 4096)
|
||||
{
|
||||
capture.queue.pop_front();
|
||||
}
|
||||
|
||||
capture.frameCursor += static_cast<std::uint64_t>(capture.bufferSize);
|
||||
}
|
||||
|
||||
static void bufferSwitch(long doubleBufferIndex, ASIOBool)
|
||||
{
|
||||
if (g_capture && g_capture->running)
|
||||
{
|
||||
pushInputBlocks(*g_capture, doubleBufferIndex);
|
||||
}
|
||||
}
|
||||
|
||||
static void sampleRateDidChange(ASIOSampleRate) {}
|
||||
|
||||
static long asioMessage(long, long, void*, double*)
|
||||
{
|
||||
return 0;
|
||||
}
|
||||
|
||||
static ASIOTime* bufferSwitchTimeInfo(ASIOTime* params, long doubleBufferIndex, ASIOBool directProcess)
|
||||
{
|
||||
bufferSwitch(doubleBufferIndex, directProcess);
|
||||
return params;
|
||||
}
|
||||
|
||||
extern "C"
|
||||
{
|
||||
__declspec(dllexport) MimirAsioCapture* mimir_asio_create(
|
||||
const char* clsidText,
|
||||
double requestedSampleRate,
|
||||
int* sampleRate,
|
||||
int* inputCount,
|
||||
int* preferredBufferSize)
|
||||
{
|
||||
CoInitializeEx(nullptr, COINIT_APARTMENTTHREADED);
|
||||
auto* capture = new MimirAsioCapture();
|
||||
const auto clsid = clsidFromString(clsidText ? clsidText : "{AC4D0455-50D7-4498-B3CD-9A41D130B759}");
|
||||
const auto hr = CoCreateInstance(clsid, nullptr, CLSCTX_INPROC_SERVER, clsid, reinterpret_cast<void**>(&capture->driver));
|
||||
if (FAILED(hr) || !capture->driver)
|
||||
{
|
||||
delete capture;
|
||||
CoUninitialize();
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
HWND hwnd = GetConsoleWindow();
|
||||
if (!hwnd)
|
||||
{
|
||||
hwnd = GetDesktopWindow();
|
||||
}
|
||||
|
||||
if (!capture->driver->init(hwnd))
|
||||
{
|
||||
capture->driver->Release();
|
||||
delete capture;
|
||||
CoUninitialize();
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
capture->driver->getChannels(&capture->inputs, &capture->outputs);
|
||||
long minSize = 0;
|
||||
long maxSize = 0;
|
||||
long granularity = 0;
|
||||
capture->driver->getBufferSize(&minSize, &maxSize, &capture->bufferSize, &granularity);
|
||||
if (requestedSampleRate > 0.0)
|
||||
{
|
||||
capture->driver->setSampleRate(requestedSampleRate);
|
||||
}
|
||||
|
||||
capture->driver->getSampleRate(&capture->sampleRate);
|
||||
capture->buffers.resize(static_cast<size_t>(capture->inputs));
|
||||
capture->channels.resize(static_cast<size_t>(capture->inputs));
|
||||
for (long channel = 0; channel < capture->inputs; ++channel)
|
||||
{
|
||||
capture->buffers[static_cast<size_t>(channel)].isInput = 1;
|
||||
capture->buffers[static_cast<size_t>(channel)].channelNum = channel;
|
||||
capture->channels[static_cast<size_t>(channel)].channel = channel;
|
||||
capture->channels[static_cast<size_t>(channel)].isInput = 1;
|
||||
capture->driver->getChannelInfo(&capture->channels[static_cast<size_t>(channel)]);
|
||||
}
|
||||
|
||||
capture->callbacks.bufferSwitch = bufferSwitch;
|
||||
capture->callbacks.sampleRateDidChange = sampleRateDidChange;
|
||||
capture->callbacks.asioMessage = asioMessage;
|
||||
capture->callbacks.bufferSwitchTimeInfo = bufferSwitchTimeInfo;
|
||||
if (capture->driver->createBuffers(capture->buffers.data(), capture->inputs, capture->bufferSize, &capture->callbacks) != ASE_OK)
|
||||
{
|
||||
capture->driver->Release();
|
||||
delete capture;
|
||||
CoUninitialize();
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
if (sampleRate)
|
||||
{
|
||||
*sampleRate = static_cast<int>(capture->sampleRate);
|
||||
}
|
||||
if (inputCount)
|
||||
{
|
||||
*inputCount = static_cast<int>(capture->inputs);
|
||||
}
|
||||
if (preferredBufferSize)
|
||||
{
|
||||
*preferredBufferSize = static_cast<int>(capture->bufferSize);
|
||||
}
|
||||
|
||||
return capture;
|
||||
}
|
||||
|
||||
__declspec(dllexport) int mimir_asio_start(MimirAsioCapture* capture)
|
||||
{
|
||||
if (!capture || !capture->driver)
|
||||
{
|
||||
return 0;
|
||||
}
|
||||
|
||||
g_capture = capture;
|
||||
capture->running = true;
|
||||
return capture->driver->start() == ASE_OK ? 1 : 0;
|
||||
}
|
||||
|
||||
__declspec(dllexport) int mimir_asio_read(
|
||||
MimirAsioCapture* capture,
|
||||
int* channel,
|
||||
std::int64_t* timestampNs,
|
||||
std::uint64_t* sequence,
|
||||
int* frameCount,
|
||||
float* samples,
|
||||
int maxFrames)
|
||||
{
|
||||
if (!capture || !samples || maxFrames <= 0)
|
||||
{
|
||||
return 0;
|
||||
}
|
||||
|
||||
std::lock_guard<std::mutex> lock(capture->mutex);
|
||||
if (capture->queue.empty())
|
||||
{
|
||||
return 0;
|
||||
}
|
||||
|
||||
auto block = std::move(capture->queue.front());
|
||||
capture->queue.pop_front();
|
||||
const auto count = std::min(block.frameCount, maxFrames);
|
||||
std::memcpy(samples, block.samples.data(), static_cast<size_t>(count) * sizeof(float));
|
||||
if (channel)
|
||||
{
|
||||
*channel = block.channel;
|
||||
}
|
||||
if (timestampNs)
|
||||
{
|
||||
*timestampNs = block.timestampNs;
|
||||
}
|
||||
if (sequence)
|
||||
{
|
||||
*sequence = block.sequence;
|
||||
}
|
||||
if (frameCount)
|
||||
{
|
||||
*frameCount = count;
|
||||
}
|
||||
|
||||
return 1;
|
||||
}
|
||||
|
||||
__declspec(dllexport) void mimir_asio_destroy(MimirAsioCapture* capture)
|
||||
{
|
||||
if (!capture)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
capture->running = false;
|
||||
if (g_capture == capture)
|
||||
{
|
||||
g_capture = nullptr;
|
||||
}
|
||||
|
||||
if (capture->driver)
|
||||
{
|
||||
capture->driver->stop();
|
||||
capture->driver->disposeBuffers();
|
||||
capture->driver->Release();
|
||||
}
|
||||
|
||||
delete capture;
|
||||
CoUninitialize();
|
||||
}
|
||||
}
|
||||
|
|
@ -116,8 +116,9 @@ usable bands, expected-symbol versus observed-bin confusion observations,
|
|||
timing residuals, delay hypotheses, phase summaries, and an adaptive codebook
|
||||
plan. The analyzer keeps raw profiles even when no timing report is accepted,
|
||||
and the active decoder can consume a persisted model for learned response
|
||||
weighting, first-order group-delay correction, and global delay-hypothesis
|
||||
seeding. Runtime chirp-bin emission consumes the same model's emission plan, so
|
||||
weighting, phase-coherence weighting, first-order group-delay correction, and
|
||||
joint global delay/bin-shift hypotheses. Runtime chirp-bin emission consumes the
|
||||
same model's emission plan, so
|
||||
the emitted timeline can use the measured reliable symbol set at a higher
|
||||
de Bruijn order instead of continuing to spend code on dead bins. Reports/states
|
||||
expose `delayUs` next to fractional sample delay.
|
||||
|
|
@ -163,14 +164,15 @@ The ASIO probe can now play raw mono Float32 timeline audio through the
|
|||
Focusrite outputs while capturing every ASIO input as raw interleaved Float32.
|
||||
`Mimir.BufferSmoke --analyze-asio-f32` feeds those captured channels into the
|
||||
same runtime active analyzer. `--calibrate-chirp-bin-asio-f32` computes and
|
||||
persists the response/confusion/delay model, and `--analyze-asio-f32
|
||||
--calibration ...` loads it into the active decoder. The ASIO worker also
|
||||
supports `--emit-json-blocks`, so Focusrite ASIO callbacks can feed
|
||||
sample-bearing `audio-block` events into `Mimir.Runtime` continuously through
|
||||
the existing frame-event adapter while the native ABI cut is pending. The
|
||||
minimal `config/mimir-runtime.asio.example.json` proof ingested 13,324
|
||||
sample-bearing 192 kHz blocks across `asio-ch0` through `asio-ch3` in two
|
||||
seconds, all from the same ASIO callback stream. A real
|
||||
persists the response/confusion/delay model per output/mic path, and
|
||||
`--analyze-asio-f32 --calibration ...` loads it into the active decoder.
|
||||
`native/asio_capture` and `MimirAsioStreamSource` are now the runtime Scarlett
|
||||
path: Focusrite ASIO callbacks feed sample-bearing 192 kHz Float32 blocks into
|
||||
`Mimir.Runtime` in process, without the diagnostic JSON/stdout bridge. The
|
||||
minimal `config/mimir-runtime.asio.example.json` proof ingested more than
|
||||
12,000 sample-bearing 192 kHz blocks across `asio-ch0` through `asio-ch3` in
|
||||
two seconds and retained 2,048 blocks per channel, all from the same ASIO
|
||||
callback stream. A real
|
||||
192 kHz chirp-bin Scarlett artifact decoded
|
||||
`Loopback 1 -> Loopback 2` at exactly `0.000 us` with 12 matched anchors and
|
||||
0.999 confidence. The same analyzer now prints calibration profiles for
|
||||
|
|
|
|||
|
|
@ -92,9 +92,10 @@ Get-Content .\state\evidence.jsonl -Tail 8
|
|||
Each classified chirp carries the full dechirped bin-energy surface; sync
|
||||
reports expose those bands, and `MimirChirpBinCalibrationModel` preserves
|
||||
measured usable bands, response/confusion observations, timing residuals,
|
||||
delay hypotheses, phase summaries, and adaptive codebook plans. The active
|
||||
decoder can load the model for learned weighting, first-order group-delay
|
||||
correction, and global delay-hypothesis seeds. Runtime chirp-bin emission
|
||||
delay hypotheses, phase summaries, and adaptive codebook plans per
|
||||
output/mic path. The active decoder can load the model for learned weighting,
|
||||
phase-coherence weighting, first-order group-delay correction, and joint
|
||||
global delay/bin-shift hypotheses. Runtime chirp-bin emission
|
||||
loads the same model's emission plan, so a physical calibration can shrink the
|
||||
emitted alphabet and raise sequence order instead of continuing to use dead
|
||||
bins.
|
||||
|
|
@ -129,20 +130,21 @@ Get-Content .\state\evidence.jsonl -Tail 8
|
|||
`--calibrate-chirp-bin-asio-f32` now persists a real response/confusion/delay
|
||||
model and `--analyze-asio-f32 --calibration ...` consumes it. In the stored
|
||||
artifact, physical input 1 produced two reliable symbols and a different
|
||||
strongest-bin profile while failing pairwise timing. The ASIO native worker
|
||||
also supports `--emit-json-blocks` so `Mimir.Runtime` can ingest Focusrite ASIO
|
||||
callbacks continuously through the frame-event adapter. The minimal
|
||||
`config/mimir-runtime.asio.example.json` proof ingested 13,324 sample-bearing
|
||||
192 kHz blocks across `asio-ch0` through `asio-ch3` in two seconds. Acoustic robustness is
|
||||
still open, but failed timing now leaves response evidence.
|
||||
strongest-bin profile while failing pairwise timing. `native/asio_capture`
|
||||
plus `MimirAsioStreamSource` now load Focusrite ASIO in process and feed
|
||||
sample-bearing 192 kHz Float32 blocks directly into runtime buffers. The
|
||||
minimal `config/mimir-runtime.asio.example.json` proof ingests more than
|
||||
12,000 blocks across `asio-ch0` through `asio-ch3` in two seconds and retains
|
||||
2,048 blocks per channel. Acoustic robustness is still open, but failed
|
||||
timing now leaves response evidence.
|
||||
- Raven also has a 192 kHz loopback-capable Scarlett for co-streamer/game timing
|
||||
evidence. Do not move the heavy soundfield or sensor-fusion workload there.
|
||||
|
||||
## Immediate Re-entry Instruction
|
||||
|
||||
Use the chirp-bin calibration models from real microphones to tune the active
|
||||
codebook/bands/weighting, then replace the ASIO JSON-block process bridge with
|
||||
a native ABI callback source and turn decoded clock fits into the
|
||||
SRO/fractional-delay actuator. Keep loopback as timing authority. Do not call
|
||||
synchronization analysis from UI/telemetry readouts. Do not restore deleted
|
||||
script infrastructure because a stale doc once missed it.
|
||||
codebook/bands/weighting, then turn decoded clock fits into the
|
||||
SRO/fractional-delay actuator. Keep loopback as timing authority and keep
|
||||
Scarlett capture on the in-process ASIO source. Do not call synchronization
|
||||
analysis from UI/telemetry readouts. Do not restore deleted script
|
||||
infrastructure because a stale doc once missed it.
|
||||
|
|
|
|||
|
|
@ -599,6 +599,7 @@ static int CalibrateChirpBinAsioFloat32(
|
|||
int channels,
|
||||
int referenceChannel)
|
||||
{
|
||||
var outputSourceId = ParseStringOption(args, "--output-source-id", "main-speakers");
|
||||
if (args.Any(arg => string.Equals(arg, "--capture-asio", StringComparison.OrdinalIgnoreCase)))
|
||||
{
|
||||
var probe = ParseStringOption(
|
||||
|
|
@ -636,7 +637,7 @@ static int CalibrateChirpBinAsioFloat32(
|
|||
}
|
||||
|
||||
var referenceSourceId = $"asio-ch{referenceChannel}";
|
||||
var model = MimirChirpBinCalibrationModel.FromDecodes(referenceSourceId, sampleRate, decodes);
|
||||
var model = MimirChirpBinCalibrationModel.FromDecodes(referenceSourceId, sampleRate, decodes, outputSourceId);
|
||||
model.Save(outputPath);
|
||||
Console.WriteLine($"chirp-bin-calibration path={outputPath} input={inputPath} sampleRate={sampleRate} channels={channels} frames={frameCount} reference={referenceSourceId}");
|
||||
foreach (var path in model.Paths)
|
||||
|
|
|
|||
|
|
@ -6,6 +6,7 @@
|
|||
<RootNamespace>Mimir.Runtime</RootNamespace>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<Nullable>enable</Nullable>
|
||||
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
|
||||
<CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>
|
||||
</PropertyGroup>
|
||||
|
||||
|
|
|
|||
225
src/Mimir.Runtime/Synchronization/MimirAsioStreamSource.cs
Normal file
225
src/Mimir.Runtime/Synchronization/MimirAsioStreamSource.cs
Normal file
|
|
@ -0,0 +1,225 @@
|
|||
using System.Runtime.InteropServices;
|
||||
using System.Collections.Concurrent;
|
||||
using System.Runtime.Versioning;
|
||||
|
||||
namespace Mimir.Runtime.Synchronization;
|
||||
|
||||
public sealed record MimirAsioStreamSourceOptions(
|
||||
string NativeLibraryPath,
|
||||
string DriverClsid,
|
||||
int SampleRate,
|
||||
IReadOnlyList<string> SourceIds);
|
||||
|
||||
[SupportedOSPlatform("windows")]
|
||||
public sealed class MimirAsioStreamSource : IMimirStreamSource
|
||||
{
|
||||
private const string DefaultClsid = "{AC4D0455-50D7-4498-B3CD-9A41D130B759}";
|
||||
private readonly ConcurrentQueue<MimirStreamSample> samples = new();
|
||||
private readonly ManualResetEventSlim started = new();
|
||||
private readonly Thread captureThread;
|
||||
private readonly CancellationTokenSource cancellation = new();
|
||||
private readonly string[] sourceIds;
|
||||
private int sampleRate;
|
||||
private string? startError;
|
||||
private bool disposed;
|
||||
|
||||
public MimirAsioStreamSource(
|
||||
MimirStreamDescriptor descriptor,
|
||||
MimirAsioStreamSourceOptions options)
|
||||
{
|
||||
Descriptor = descriptor;
|
||||
Options = options;
|
||||
sourceIds = options.SourceIds.Count > 0
|
||||
? options.SourceIds.ToArray()
|
||||
: Enumerable.Range(0, 8).Select(index => $"asio-ch{index}").ToArray();
|
||||
captureThread = new Thread(CaptureLoop)
|
||||
{
|
||||
IsBackground = true,
|
||||
Name = "Mimir Focusrite ASIO capture",
|
||||
};
|
||||
captureThread.SetApartmentState(ApartmentState.STA);
|
||||
captureThread.Start();
|
||||
if (!started.Wait(TimeSpan.FromSeconds(5)))
|
||||
{
|
||||
throw new InvalidOperationException("Timed out starting Focusrite ASIO capture source");
|
||||
}
|
||||
|
||||
if (startError != null)
|
||||
{
|
||||
throw new InvalidOperationException(startError);
|
||||
}
|
||||
}
|
||||
|
||||
public MimirStreamDescriptor Descriptor { get; }
|
||||
|
||||
public MimirAsioStreamSourceOptions Options { get; }
|
||||
|
||||
public bool ExposesDescriptorBuffer => false;
|
||||
|
||||
public bool TryRead(out MimirStreamSample sample)
|
||||
{
|
||||
sample = default;
|
||||
ObjectDisposedException.ThrowIf(disposed, this);
|
||||
return samples.TryDequeue(out sample);
|
||||
}
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
if (disposed)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
disposed = true;
|
||||
cancellation.Cancel();
|
||||
captureThread.Join(TimeSpan.FromSeconds(2));
|
||||
cancellation.Dispose();
|
||||
started.Dispose();
|
||||
}
|
||||
|
||||
private void CaptureLoop()
|
||||
{
|
||||
nint handle = 0;
|
||||
try
|
||||
{
|
||||
Native.Load(Options.NativeLibraryPath);
|
||||
handle = Native.Create(
|
||||
string.IsNullOrWhiteSpace(Options.DriverClsid) ? DefaultClsid : Options.DriverClsid,
|
||||
Options.SampleRate,
|
||||
out sampleRate,
|
||||
out _,
|
||||
out var maxFrames);
|
||||
if (handle == 0)
|
||||
{
|
||||
startError = "Could not create Focusrite ASIO capture source";
|
||||
started.Set();
|
||||
return;
|
||||
}
|
||||
|
||||
if (!Native.Start(handle))
|
||||
{
|
||||
startError = "Could not start Focusrite ASIO capture source";
|
||||
started.Set();
|
||||
return;
|
||||
}
|
||||
|
||||
started.Set();
|
||||
var sampleBuffer = new float[Math.Max(1, maxFrames)];
|
||||
while (!cancellation.IsCancellationRequested)
|
||||
{
|
||||
if (!Native.Read(
|
||||
handle,
|
||||
out var channel,
|
||||
out var timestampNs,
|
||||
out var sequence,
|
||||
out var frameCount,
|
||||
sampleBuffer,
|
||||
sampleBuffer.Length))
|
||||
{
|
||||
Thread.Sleep(1);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (channel < 0)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
var sourceId = channel < sourceIds.Length ? sourceIds[channel] : $"asio-ch{channel}";
|
||||
var bytes = new byte[frameCount * sizeof(float)];
|
||||
MemoryMarshal.AsBytes(sampleBuffer.AsSpan(0, frameCount)).CopyTo(bytes);
|
||||
samples.Enqueue(new MimirStreamSample(
|
||||
sourceId,
|
||||
MimirStreamKind.Audio,
|
||||
Descriptor.Origin,
|
||||
timestampNs,
|
||||
DateTimeOffset.UtcNow.ToUnixTimeMilliseconds() * 1_000_000L,
|
||||
sequence,
|
||||
0,
|
||||
bytes.Length,
|
||||
bytes,
|
||||
AudioBlock: new MimirAudioBlockDescriptor(
|
||||
sampleRate,
|
||||
1,
|
||||
MimirAudioSampleFormat.Float32,
|
||||
frameCount,
|
||||
timestampNs)));
|
||||
|
||||
while (samples.Count > 8192 && samples.TryDequeue(out _))
|
||||
{
|
||||
}
|
||||
}
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
startError = ex.Message;
|
||||
started.Set();
|
||||
}
|
||||
finally
|
||||
{
|
||||
Native.Destroy(handle);
|
||||
}
|
||||
}
|
||||
|
||||
private static unsafe class Native
|
||||
{
|
||||
private static nint library;
|
||||
private static delegate* unmanaged[Stdcall]<byte*, double, out int, out int, out int, nint> create;
|
||||
private static delegate* unmanaged[Stdcall]<nint, int> start;
|
||||
private static delegate* unmanaged[Stdcall]<nint, out int, out long, out ulong, out int, float*, int, int> read;
|
||||
private static delegate* unmanaged[Stdcall]<nint, void> destroy;
|
||||
|
||||
public static void Load(string path)
|
||||
{
|
||||
if (library != 0)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
if (string.IsNullOrWhiteSpace(path))
|
||||
{
|
||||
path = Path.Combine(AppContext.BaseDirectory, "mimir_asio_capture.dll");
|
||||
}
|
||||
|
||||
library = NativeLibrary.Load(path);
|
||||
create = (delegate* unmanaged[Stdcall]<byte*, double, out int, out int, out int, nint>)NativeLibrary.GetExport(library, "mimir_asio_create");
|
||||
start = (delegate* unmanaged[Stdcall]<nint, int>)NativeLibrary.GetExport(library, "mimir_asio_start");
|
||||
read = (delegate* unmanaged[Stdcall]<nint, out int, out long, out ulong, out int, float*, int, int>)NativeLibrary.GetExport(library, "mimir_asio_read");
|
||||
destroy = (delegate* unmanaged[Stdcall]<nint, void>)NativeLibrary.GetExport(library, "mimir_asio_destroy");
|
||||
}
|
||||
|
||||
public static nint Create(string clsid, int requestedSampleRate, out int sampleRate, out int inputCount, out int preferredBufferSize)
|
||||
{
|
||||
var bytes = System.Text.Encoding.UTF8.GetBytes(clsid + "\0");
|
||||
fixed (byte* clsidPtr = bytes)
|
||||
{
|
||||
return create(clsidPtr, requestedSampleRate, out sampleRate, out inputCount, out preferredBufferSize);
|
||||
}
|
||||
}
|
||||
|
||||
public static bool Start(nint handle) => start(handle) != 0;
|
||||
|
||||
public static bool Read(
|
||||
nint handle,
|
||||
out int channel,
|
||||
out long timestampNs,
|
||||
out ulong sequence,
|
||||
out int frameCount,
|
||||
float[] samples,
|
||||
int maxFrames)
|
||||
{
|
||||
fixed (float* samplePtr = samples)
|
||||
{
|
||||
return read(handle, out channel, out timestampNs, out sequence, out frameCount, samplePtr, maxFrames) != 0;
|
||||
}
|
||||
}
|
||||
|
||||
public static void Destroy(nint handle)
|
||||
{
|
||||
if (handle != 0)
|
||||
{
|
||||
destroy(handle);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -39,11 +39,12 @@ public sealed record MimirChirpBinCalibrationModel(
|
|||
public static MimirChirpBinCalibrationModel FromDecodes(
|
||||
string referenceSourceId,
|
||||
int sampleRate,
|
||||
IReadOnlyDictionary<string, MimirChirpletStreamDecode> decodes)
|
||||
IReadOnlyDictionary<string, MimirChirpletStreamDecode> decodes,
|
||||
string outputSourceId = "main-speakers")
|
||||
{
|
||||
var paths = decodes
|
||||
.OrderBy(pair => pair.Key, StringComparer.Ordinal)
|
||||
.Select(pair => MimirChirpBinPathCalibration.FromDecode(pair.Key, sampleRate, pair.Value))
|
||||
.Select(pair => MimirChirpBinPathCalibration.FromDecode(outputSourceId, pair.Key, sampleRate, pair.Value))
|
||||
.ToArray();
|
||||
var emissionPlan = MimirChirpBinCodebookPlan.ForSharedEmission(paths);
|
||||
paths = paths.Select(path => path with { EmissionPlan = emissionPlan }).ToArray();
|
||||
|
|
@ -58,6 +59,7 @@ public sealed record MimirChirpBinCalibrationModel(
|
|||
}
|
||||
|
||||
public sealed record MimirChirpBinPathCalibration(
|
||||
string OutputSourceId,
|
||||
string SourceId,
|
||||
int SampleRate,
|
||||
MimirChirpBinCalibrationProfile Profile,
|
||||
|
|
@ -79,7 +81,20 @@ public sealed record MimirChirpBinPathCalibration(
|
|||
return symbol?.MeanTimingResidualSamples ?? 0.0;
|
||||
}
|
||||
|
||||
public double PhaseWeight(int symbolId, double observedPhaseRadians)
|
||||
{
|
||||
var symbol = Symbols.FirstOrDefault(item => item.SymbolId == symbolId);
|
||||
if (symbol == null || symbol.ObservationCount <= 0)
|
||||
{
|
||||
return 1.0;
|
||||
}
|
||||
|
||||
var error = Math.Abs(AngularDistance(observedPhaseRadians, symbol.MeanPhaseRadians));
|
||||
return Math.Clamp(1.0 - error / Math.PI, 0.15, 1.0);
|
||||
}
|
||||
|
||||
public static MimirChirpBinPathCalibration FromDecode(
|
||||
string outputSourceId,
|
||||
string sourceId,
|
||||
int sampleRate,
|
||||
MimirChirpletStreamDecode decode)
|
||||
|
|
@ -99,9 +114,11 @@ public sealed record MimirChirpBinPathCalibration(
|
|||
(correct / (double)items.Length) *
|
||||
(1.0 / (1.0 + residualSpread / Math.Max(1.0, sampleRate * 0.00025)));
|
||||
var phase = CircularMean(items.Select(item => item.PhaseRadians));
|
||||
var phaseCoherence = PhaseCoherence(items.Select(item => item.PhaseRadians));
|
||||
return new MimirChirpBinSymbolCalibration(
|
||||
group.Key,
|
||||
reliability,
|
||||
phaseCoherence,
|
||||
items.Average(item => item.ExpectedCenterHz),
|
||||
items.Average(item => item.ObservedCenterHz),
|
||||
items.Average(item => item.ObservedEnergy),
|
||||
|
|
@ -115,6 +132,7 @@ public sealed record MimirChirpBinPathCalibration(
|
|||
var hypotheses = BuildDelayHypotheses(decode);
|
||||
var plan = MimirChirpBinCodebookPlan.FromSymbols(symbols);
|
||||
return new MimirChirpBinPathCalibration(
|
||||
outputSourceId,
|
||||
sourceId,
|
||||
sampleRate,
|
||||
profile,
|
||||
|
|
@ -201,6 +219,27 @@ public sealed record MimirChirpBinPathCalibration(
|
|||
|
||||
return count == 0 ? 0.0 : Math.Atan2(sin / count, cos / count);
|
||||
}
|
||||
|
||||
private static double PhaseCoherence(IEnumerable<double> radians)
|
||||
{
|
||||
var sin = 0.0;
|
||||
var cos = 0.0;
|
||||
var count = 0;
|
||||
foreach (var value in radians)
|
||||
{
|
||||
sin += Math.Sin(value);
|
||||
cos += Math.Cos(value);
|
||||
count++;
|
||||
}
|
||||
|
||||
return count == 0 ? 0.0 : Math.Clamp(Math.Sqrt(sin * sin + cos * cos) / count, 0.0, 1.0);
|
||||
}
|
||||
|
||||
private static double AngularDistance(double a, double b)
|
||||
{
|
||||
var delta = Math.Abs(a - b) % (Math.PI * 2.0);
|
||||
return delta > Math.PI ? Math.PI * 2.0 - delta : delta;
|
||||
}
|
||||
}
|
||||
|
||||
public sealed record MimirChirpBinCodebookPlan(
|
||||
|
|
@ -253,6 +292,7 @@ public sealed record MimirChirpBinCodebookPlan(
|
|||
public sealed record MimirChirpBinSymbolCalibration(
|
||||
int SymbolId,
|
||||
double Reliability,
|
||||
double PhaseCoherence,
|
||||
double ExpectedCenterHz,
|
||||
double ObservedCenterHz,
|
||||
double MeanEnergy,
|
||||
|
|
|
|||
|
|
@ -296,7 +296,8 @@ public sealed class MimirChirpBinTimeline
|
|||
var correctedEnergy = ScoreSymbolEnergy(
|
||||
samples.Slice((int)Math.Round(correctedOffset), chirpSamples),
|
||||
sampleRate,
|
||||
canonicalSymbol) * (calibration?.SymbolWeight(canonicalSymbol) ?? 1.0);
|
||||
canonicalSymbol) *
|
||||
(calibration?.SymbolWeight(canonicalSymbol) ?? 1.0);
|
||||
correctedOffset -= calibration?.GroupDelayCorrectionSamples(canonicalSymbol) ?? 0.0;
|
||||
correctedCandidates.Add(new MimirChirpletSymbolCandidate(
|
||||
canonicalSymbol,
|
||||
|
|
@ -350,7 +351,9 @@ public sealed class MimirChirpBinTimeline
|
|||
.Select((score, symbol) => new MimirChirpletSymbolCandidate(
|
||||
symbol,
|
||||
sampleOffset - (calibration?.GroupDelayCorrectionSamples(symbol) ?? 0.0),
|
||||
score.Energy * (calibration?.SymbolWeight(symbol) ?? 1.0),
|
||||
score.Energy *
|
||||
(calibration?.SymbolWeight(symbol) ?? 1.0) *
|
||||
(calibration?.PhaseWeight(symbol, score.PhaseRadians) ?? 1.0),
|
||||
bandResponses))
|
||||
.OrderByDescending(candidate => candidate.Energy)
|
||||
.Take(MaxSymbolCandidatesPerFrame)
|
||||
|
|
@ -459,35 +462,45 @@ public sealed class MimirChirpBinTimeline
|
|||
|
||||
foreach (var candidatePath in CandidateSymbolPaths(window))
|
||||
{
|
||||
var code = CodeKey(candidatePath.Select(candidate => candidate.SymbolId));
|
||||
if (!plan.CodeToIndex.TryGetValue(code, out var eventIndex))
|
||||
foreach (var globalBinShift in GlobalBinShiftHypotheses(calibration))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
var canonicalSymbols = candidatePath
|
||||
.Select(candidate => ShiftSymbol(candidate.SymbolId, -globalBinShift))
|
||||
.ToArray();
|
||||
var code = CodeKey(canonicalSymbols);
|
||||
if (!plan.CodeToIndex.TryGetValue(code, out var eventIndex))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
var firstEvent = Default.EventForIndex((ulong)eventIndex, calibration?.EmissionPlan);
|
||||
var gapError = 0.0;
|
||||
for (var gap = 0; gap < candidatePath.Count - 1; gap++)
|
||||
{
|
||||
var measuredGap = (candidatePath[gap + 1].SampleOffset - candidatePath[gap].SampleOffset) / sampleRate;
|
||||
gapError += Math.Abs(measuredGap - EventSpacingSeconds);
|
||||
}
|
||||
var firstEvent = Default.EventForIndex((ulong)eventIndex, calibration?.EmissionPlan);
|
||||
var gapError = 0.0;
|
||||
for (var gap = 0; gap < candidatePath.Count - 1; gap++)
|
||||
{
|
||||
var measuredGap = (candidatePath[gap + 1].SampleOffset - candidatePath[gap].SampleOffset) / sampleRate;
|
||||
gapError += Math.Abs(measuredGap - EventSpacingSeconds);
|
||||
}
|
||||
|
||||
var gapConfidence = 1.0 / (1.0 + gapError / 0.003);
|
||||
if (gapConfidence < 0.55)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
var gapConfidence = 1.0 / (1.0 + gapError / 0.003);
|
||||
if (gapConfidence < 0.55)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
var energyConfidence = Math.Clamp(candidatePath.Average(candidate => candidate.Energy), 0.0, 1.0);
|
||||
candidates.Add(new MimirChirpletTimelineAnchor(
|
||||
(ulong)eventIndex,
|
||||
firstEvent.StartSeconds,
|
||||
candidatePath[0].SampleOffset,
|
||||
gapConfidence * 0.60 + energyConfidence * 0.40,
|
||||
candidatePath
|
||||
.Select(candidate => new MimirChirpletSymbolObservation(candidate.SymbolId, candidate.SampleOffset, candidate.Energy))
|
||||
.ToArray()));
|
||||
var energyConfidence = Math.Clamp(candidatePath.Average(candidate => candidate.Energy), 0.0, 1.0);
|
||||
var binShiftPenalty = 1.0 / (1.0 + Math.Abs(globalBinShift) * 0.10);
|
||||
candidates.Add(new MimirChirpletTimelineAnchor(
|
||||
(ulong)eventIndex,
|
||||
firstEvent.StartSeconds,
|
||||
candidatePath[0].SampleOffset,
|
||||
(gapConfidence * 0.60 + energyConfidence * 0.40) * binShiftPenalty,
|
||||
candidatePath
|
||||
.Select((candidate, candidateIndex) => new MimirChirpletSymbolObservation(
|
||||
canonicalSymbols[candidateIndex],
|
||||
candidate.SampleOffset,
|
||||
candidate.Energy))
|
||||
.ToArray()));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -519,6 +532,33 @@ public sealed class MimirChirpBinTimeline
|
|||
return Recurse(0);
|
||||
}
|
||||
|
||||
private static IReadOnlyList<int> GlobalBinShiftHypotheses(MimirChirpBinPathCalibration? calibration)
|
||||
{
|
||||
var shifts = new HashSet<int> { 0 };
|
||||
if (calibration != null)
|
||||
{
|
||||
foreach (var hypothesis in calibration.DelayHypotheses)
|
||||
{
|
||||
shifts.Add(Math.Clamp(hypothesis.BinShift, -MaxTimingBinShift, MaxTimingBinShift));
|
||||
}
|
||||
|
||||
foreach (var group in calibration.Confusion
|
||||
.GroupBy(observation => observation.BinShift)
|
||||
.OrderByDescending(group => group.Count())
|
||||
.Take(4))
|
||||
{
|
||||
shifts.Add(Math.Clamp(group.Key, -MaxTimingBinShift, MaxTimingBinShift));
|
||||
}
|
||||
}
|
||||
|
||||
for (var shift = -MaxTimingBinShift; shift <= MaxTimingBinShift; shift++)
|
||||
{
|
||||
shifts.Add(shift);
|
||||
}
|
||||
|
||||
return shifts.OrderBy(shift => Math.Abs(shift)).ThenBy(shift => shift).ToArray();
|
||||
}
|
||||
|
||||
private static IReadOnlyList<MimirChirpletTimelineAnchor> SelectCoherentAnchorPath(
|
||||
IReadOnlyList<MimirChirpletTimelineAnchor> candidates,
|
||||
int sampleRate,
|
||||
|
|
|
|||
|
|
@ -119,6 +119,24 @@ public sealed class MimirRuntimeConfiguration
|
|||
return new MimirNativeIngestStreamSource(ToDescriptor(stream));
|
||||
}
|
||||
|
||||
if (string.Equals(stream.Adapter, "asio", StringComparison.OrdinalIgnoreCase))
|
||||
{
|
||||
if (!OperatingSystem.IsWindows())
|
||||
{
|
||||
throw new PlatformNotSupportedException("The Mimir ASIO stream source requires Windows.");
|
||||
}
|
||||
|
||||
#pragma warning disable CA1416
|
||||
return new MimirAsioStreamSource(
|
||||
ToDescriptor(stream),
|
||||
new MimirAsioStreamSourceOptions(
|
||||
ResolveCommand(stream.Command, configDirectory),
|
||||
stream.DriverClsid,
|
||||
stream.SampleRate,
|
||||
stream.AcceptSourceIds));
|
||||
#pragma warning restore CA1416
|
||||
}
|
||||
|
||||
if (string.Equals(stream.Adapter, "frame-events", StringComparison.OrdinalIgnoreCase)
|
||||
|| string.Equals(stream.Adapter, "json-lines", StringComparison.OrdinalIgnoreCase))
|
||||
{
|
||||
|
|
@ -243,4 +261,8 @@ public sealed class MimirStreamConfig
|
|||
public string[] AcceptSourceIds { get; set; } = [];
|
||||
|
||||
public int ChunkBytes { get; set; } = 65_536;
|
||||
|
||||
public int SampleRate { get; set; } = 192_000;
|
||||
|
||||
public string DriverClsid { get; set; } = "";
|
||||
}
|
||||
|
|
|
|||
|
|
@ -74,3 +74,6 @@
|
|||
{"ts": "2026-05-23T20:19:36+01:00", "type": "implementation", "status": "ok", "note": "Replaced the profile-only chirp-bin calibration cut with a usable physical calibration model. The model records per-source usable bands, symbol reliability, expected-vs-observed confusion observations, timing residuals, delay hypotheses, phase summaries, and an adaptive codebook plan. MimirChirpBinTimeline can now decode with learned per-symbol weights, group-delay correction, and persisted delay-hypothesis seeds. BufferSmoke can run --calibrate-chirp-bin-asio-f32 to render/capture/analyze/persist the model, and --analyze-asio-f32 can load it via --calibration. The ASIO native worker now supports --emit-json-blocks so Focusrite ASIO callbacks can feed Mimir.Runtime continuously through the existing frame-event adapter while the native ABI cut is still pending."}
|
||||
{"ts": "2026-05-23T20:31:27+01:00", "type": "implementation", "status": "ok", "note": "Completed the physical chirp-bin calibration loop instead of stopping at summaries. The calibration model now owns a shared emission plan from measured reliable symbols, and MimirRuntime renders chirp-bin calibration with that plan so dead bins are removed from the emitted alphabet and sequence order rises as needed. MimirChirpBinTimeline decodes variable-order codewords from the same emission plan, keeps learned response weights/group-delay correction, records bin-shift in confusion/delay hypotheses, and still preserves the default 32-symbol path when no model is loaded. ASIO callback ingestion remains continuous through the ASIO worker's sample-bearing runtime stream path; the next native ABI cut removes process transport overhead, not clock-domain ownership."}
|
||||
{"ts": "2026-05-23T20:33:00+01:00", "type": "verification", "status": "ok", "note": "Verified continuous runtime ASIO ingestion through the ASIO worker stream path. Using config/mimir-runtime.asio.example.json, BufferSmoke ran for 2 seconds against Focusrite USB ASIO at 192 kHz and ingested 13,324 sample-bearing audio blocks across asio-ch0..asio-ch3, 3,331 blocks per channel, each 192 Float32 frames. This proves loopback and mic channels enter Mimir.Runtime buffers continuously in one ASIO clock domain; the run did not sync because BufferSmoke does not emit speaker calibration audio."}
|
||||
{"ts": "2026-05-23T21:41:47+01:00", "type": "implementation", "status": "ok", "note": "Implemented the in-process Focusrite ASIO runtime source. native/asio_capture owns the ASIO COM driver/callback buffers and MimirAsioStreamSource drains 192 kHz Float32 blocks into Mimir.Runtime buffers for asio-ch0..asio-ch3 without stdout/base64 transport. The chirp-bin calibration model is now per output/mic path and the active decoder uses learned symbol weights, phase-coherence weights, group-delay correction, and joint global delay/bin-shift hypotheses."}
|
||||
{"ts": "2026-05-23T21:41:53+01:00", "type": "verification", "status": "ok", "note": "Verified the in-process ASIO runtime ingest and active decoder after the native ASIO cut. dotnet build Mimir.slnx and cmake --build native/asio_capture/build --config Release passed cleanly. BufferSmoke with config/mimir-runtime.asio.example.json ingested 12,196 sample-bearing blocks in two seconds at 192 kHz, with 2,048 buffered 192-frame Float32 blocks on each asio-ch0..asio-ch3 channel. The standalone 192 kHz chirp-bin receiver recovered a 96,000-sample delay with 0.000001 sample error, and calibrated ASIO artifact analysis recovered asio-ch2->asio-ch3 at 0.000 us with 0.812 confidence."}
|
||||
{"ts": "2026-05-23T21:43:33+01:00", "type": "verification", "status": "ok", "note": "Reran the in-process ASIO ingest after native cleanup changes. BufferSmoke with config/mimir-runtime.asio.example.json ingested 12,204 sample-bearing blocks in two seconds at 192 kHz and retained 2,048 192-frame Float32 blocks for each asio-ch0..asio-ch3 buffer. This remains an ingest proof only because BufferSmoke does not emit calibration audio in that run."}
|
||||
|
|
|
|||
|
|
@ -113,9 +113,10 @@ accepted_design:
|
|||
evidence for frequency-response normalization. `MimirChirpBinCalibrationModel`
|
||||
now preserves measured usable bands, expected-symbol versus observed-bin
|
||||
confusion observations, timing residuals, delay hypotheses, phase summaries,
|
||||
and an adaptive codebook plan. The active decoder can consume a persisted
|
||||
model for learned response weighting, first-order group-delay correction,
|
||||
and global delay-hypothesis seeding. Runtime chirp-bin emission consumes the
|
||||
and an adaptive codebook plan per output/mic path. The active decoder can
|
||||
consume a persisted model for learned response weighting, phase-coherence
|
||||
weighting, first-order group-delay correction, and joint global
|
||||
delay/bin-shift hypotheses. Runtime chirp-bin emission consumes the
|
||||
same model's emission plan, so Mimir can render the measured reliable symbol
|
||||
alphabet at a higher de Bruijn order instead of spending code on dead bins.
|
||||
The analyzer keeps calibration profiles
|
||||
|
|
@ -144,10 +145,13 @@ accepted_design:
|
|||
pairwise timing against loopback in the stored artifact, but decoded mic
|
||||
channels now leave calibration models; physical input 1 produced two reliable
|
||||
symbols and a different strongest measured band set.
|
||||
`config/mimir-runtime.asio.example.json` proves continuous ASIO runtime
|
||||
ingest: a two-second BufferSmoke run at 192 kHz ingested 13,324 sample-bearing
|
||||
blocks across `asio-ch0` through `asio-ch3`, all from the Focusrite ASIO
|
||||
callback stream.
|
||||
`native/asio_capture` and `MimirAsioStreamSource` are now the runtime
|
||||
Scarlett path: Focusrite ASIO callbacks feed sample-bearing 192 kHz Float32
|
||||
blocks into `Mimir.Runtime` in process. `config/mimir-runtime.asio.example.json`
|
||||
proves continuous ASIO runtime ingest: a two-second BufferSmoke run at
|
||||
192 kHz ingested more than 12,000 sample-bearing blocks across `asio-ch0`
|
||||
through `asio-ch3` and retained 2,048 blocks per channel, all from the
|
||||
Focusrite ASIO callback stream.
|
||||
Acoustic robustness remains open.
|
||||
The old arbitrary-chirplet matcher is a diagnostic/reference artifact only,
|
||||
not the runtime hot path.
|
||||
|
|
@ -164,9 +168,10 @@ accepted_design:
|
|||
active_subgoals:
|
||||
- Implement the first concrete Leap direct capture driver and measure cadence.
|
||||
- Add remaining local camera drivers through `IMimirVideoCaptureDriver`.
|
||||
- Add native audio capture workers for mic, loopback, and network audio feeds.
|
||||
- Feed verified Focusrite ASIO input callbacks into `Mimir.Runtime`; WASAPI
|
||||
remains a diagnostic witness.
|
||||
- Use the verified in-process Focusrite ASIO source for local Scarlett mic and
|
||||
loopback ingest; WASAPI remains a diagnostic witness.
|
||||
- Add network audio feed producers that enter the same runtime clock/rebuffer
|
||||
machinery instead of becoming independent authorities.
|
||||
- Bind Fensalir UI to stream health, buffer depth, source timestamps,
|
||||
settings, and output management.
|
||||
- Keep PowerShell/FFmpeg/SRT as bridge utilities until direct program output
|
||||
|
|
@ -250,9 +255,7 @@ current_status:
|
|||
the Leap USB neighborhood does not reproduce the Focusrite-induced collapse.
|
||||
next_action: >
|
||||
Use chirp-bin calibration models from real microphones to tune the active
|
||||
codebook/bands/weighting, then replace the ASIO JSON-block process bridge
|
||||
with a native ABI callback source and add the synchronization
|
||||
actuator.
|
||||
codebook/bands/weighting, then add the synchronization actuator.
|
||||
Current app testing proves Fensalir WASAPI output wakes Scarlett speaker
|
||||
loopback, live mic buffers stay populated, runtime sync analysis now runs as
|
||||
a bounded rotating service without UI/telemetry re-entry, and the continuous
|
||||
|
|
@ -260,7 +263,7 @@ current_status:
|
|||
invariant is code-valid triplets producing stable canonical anchors and clock
|
||||
fits, so the actuator can use decoded time instead of filtered lag guesses.
|
||||
Replace
|
||||
the diagnostic JSON-line process bridge with direct native capture drivers that push buffer handles into
|
||||
the remaining diagnostic JSON-line process bridges with direct native capture drivers that push buffer handles into
|
||||
Mimir.Runtime. Mimir.BufferSmoke still proves live buffers and sync reports
|
||||
for Scarlett loopback, Focusrite mic, Kiyo mic, and Kiyo Pro mic when
|
||||
calibration chirplets are playing. Treat PS3 Eye audio as
|
||||
|
|
|
|||
|
|
@ -6,11 +6,11 @@ No active scratch subgoal.
|
|||
|
||||
## Notes
|
||||
|
||||
- Viable stream app contract promoted to docs/map/evidence: Aquarium hosts one
|
||||
- Viable stream app contract promoted to docs/map/evidence: Fensalir hosts one
|
||||
five-second in-memory runtime, local/networked feeds append as producers, and
|
||||
OBS receives synchronized program outputs.
|
||||
- Mimir app scaffold now lives in `Mimir.slnx` with `src/Mimir.App` and
|
||||
`src/Mimir.Runtime`; Aquarium Engine remains an external project reference.
|
||||
`src/Mimir.Runtime`; Fensalir remains an external project reference.
|
||||
- Synchronization ownership started moving into `Mimir.Runtime`: hub, stream
|
||||
descriptors, per-stream rolling buffers, settings, and source adapter seam are
|
||||
in C# now; concrete capture adapters are next.
|
||||
|
|
|
|||
Loading…
Reference in a new issue