Add native ASIO runtime source

This commit is contained in:
Meta Vens 2026-05-23 21:44:36 +01:00
commit fad7eb63b7
20 changed files with 925 additions and 134 deletions

View file

@ -162,8 +162,9 @@ hardware that earns the claim.
Next audio cuts:
1. Feed ASIO callbacks directly into `Mimir.Runtime` buffers.
2. Use ASIO loopback as the canonical program/timing reference.
1. Use ASIO loopback as the canonical program/timing reference.
2. Learn per output/mic chirp-bin response models and adapt the codebook to the
bands that survive the actual room.
3. Align all mic streams with fractional-delay and sample-rate-offset state.
4. Move the hot actuator into Faust/native DSP.
5. Emit separately controllable OBS stems plus a spatial bed from the same
@ -185,8 +186,11 @@ showed that generic dense chirplet matching is the wrong hot path for our
controlled beacon. The current hybrid watermark uses `MimirChirpBinTimeline`: a
fixed-slope chirp-bin codebook decoded by dechirp plus a small Goertzel/bin
bank, then code-valid de Bruijn triplets become canonical timeline anchors.
Reports expose fractional delay and `delayUs`; the synthetic proof currently
recovers a `317.375` sample offset with about `0.369 us` error.
Reports expose fractional delay and `delayUs`; the synthetic 192 kHz proof now
recovers a delayed stream below printed microsecond precision. Physical
calibration persists usable bands, expected/observed bin confusion, phase,
group-delay, delay hypotheses, and an adaptive codebook plan per output/mic
path.
For the long-form reasoning and source trail, read:
@ -288,7 +292,6 @@ Run the old OBS bridge only when you need the compatibility path:
Audio-side help is especially useful around:
- ASIO callback capture into `Mimir.Runtime` without stdout/process payloads.
- Fractional-delay lines and variable-rate resampling.
- Sample-rate-offset estimation over a rolling window.
- GCC-PHAT/passive program-audio timing under music/game audio.

View file

@ -1,6 +1,6 @@
{
"ModelId": "chirp-bin-20260523T192955Z",
"CreatedAt": "2026-05-23T19:29:55.6287406+00:00",
"ModelId": "chirp-bin-20260523T203522Z",
"CreatedAt": "2026-05-23T20:35:22.527425+00:00",
"SampleRate": 192000,
"ReferenceSourceId": "asio-ch2",
"EmissionPlan": {
@ -22,6 +22,7 @@
},
"Paths": [
{
"OutputSourceId": "main-speakers",
"SourceId": "asio-ch0",
"SampleRate": 192000,
"Profile": {
@ -319,6 +320,7 @@
"DelayHypotheses": []
},
{
"OutputSourceId": "main-speakers",
"SourceId": "asio-ch1",
"SampleRate": 192000,
"Profile": {
@ -618,6 +620,7 @@
{
"SymbolId": 1,
"Reliability": 0.02338947289373043,
"PhaseCoherence": 0.8813023361164132,
"ExpectedCenterHz": 4075,
"ObservedCenterHz": 4468.75,
"MeanEnergy": 0.9902022429801735,
@ -629,6 +632,7 @@
{
"SymbolId": 2,
"Reliability": 0.16959717331712612,
"PhaseCoherence": 0.7788915227933655,
"ExpectedCenterHz": 4525,
"ObservedCenterHz": 4475,
"MeanEnergy": 0.9196472718940074,
@ -640,6 +644,7 @@
{
"SymbolId": 3,
"Reliability": 1.148185667935492,
"PhaseCoherence": 1,
"ExpectedCenterHz": 4975,
"ObservedCenterHz": 4975,
"MeanEnergy": 1.148185667935492,
@ -651,6 +656,7 @@
{
"SymbolId": 4,
"Reliability": 0,
"PhaseCoherence": 1,
"ExpectedCenterHz": 5425,
"ObservedCenterHz": 4975,
"MeanEnergy": 0.30966049519505534,
@ -662,6 +668,7 @@
{
"SymbolId": 6,
"Reliability": 0,
"PhaseCoherence": 1,
"ExpectedCenterHz": 6325,
"ObservedCenterHz": 6775,
"MeanEnergy": 0.8679642624875925,
@ -673,6 +680,7 @@
{
"SymbolId": 7,
"Reliability": 0,
"PhaseCoherence": 0.9999999999999999,
"ExpectedCenterHz": 6775,
"ObservedCenterHz": 7225,
"MeanEnergy": 0.5299773190124039,
@ -684,6 +692,7 @@
{
"SymbolId": 8,
"Reliability": 0.6732785913925556,
"PhaseCoherence": 1,
"ExpectedCenterHz": 7225,
"ObservedCenterHz": 7225,
"MeanEnergy": 0.6732785913925556,
@ -984,6 +993,7 @@
"DelayHypotheses": []
},
{
"OutputSourceId": "main-speakers",
"SourceId": "asio-ch2",
"SampleRate": 192000,
"Profile": {
@ -1290,6 +1300,7 @@
{
"SymbolId": 0,
"Reliability": 1.9664832701324948,
"PhaseCoherence": 0.9999999999999998,
"ExpectedCenterHz": 3625,
"ObservedCenterHz": 3625,
"MeanEnergy": 1.9831606924956895,
@ -1301,6 +1312,7 @@
{
"SymbolId": 1,
"Reliability": 1.9550716359955316,
"PhaseCoherence": 0.9999608311103569,
"ExpectedCenterHz": 4075,
"ObservedCenterHz": 4075,
"MeanEnergy": 1.9724028422080755,
@ -1312,6 +1324,7 @@
{
"SymbolId": 2,
"Reliability": 1.98317479150165,
"PhaseCoherence": 1,
"ExpectedCenterHz": 4525,
"ObservedCenterHz": 4525,
"MeanEnergy": 1.98317479150165,
@ -1323,6 +1336,7 @@
{
"SymbolId": 3,
"Reliability": 1.9831581977468316,
"PhaseCoherence": 1,
"ExpectedCenterHz": 4975,
"ObservedCenterHz": 4975,
"MeanEnergy": 1.9831581977468316,
@ -1334,6 +1348,7 @@
{
"SymbolId": 4,
"Reliability": 1.9831595137173625,
"PhaseCoherence": 1,
"ExpectedCenterHz": 5425,
"ObservedCenterHz": 5425,
"MeanEnergy": 1.9831595137173625,
@ -1345,6 +1360,7 @@
{
"SymbolId": 5,
"Reliability": 1.983165098459816,
"PhaseCoherence": 1,
"ExpectedCenterHz": 5875,
"ObservedCenterHz": 5875,
"MeanEnergy": 1.983165098459816,
@ -1356,6 +1372,7 @@
{
"SymbolId": 6,
"Reliability": 1.9831633177029786,
"PhaseCoherence": 1,
"ExpectedCenterHz": 6325,
"ObservedCenterHz": 6325,
"MeanEnergy": 1.9831633177029786,
@ -1367,6 +1384,7 @@
{
"SymbolId": 7,
"Reliability": 1.9831607953145476,
"PhaseCoherence": 0.9999999999999999,
"ExpectedCenterHz": 6775,
"ObservedCenterHz": 6775,
"MeanEnergy": 1.9831607953145476,
@ -1378,6 +1396,7 @@
{
"SymbolId": 31,
"Reliability": 1.9831585964902498,
"PhaseCoherence": 1,
"ExpectedCenterHz": 17575,
"ObservedCenterHz": 17575,
"MeanEnergy": 1.9831585964902498,
@ -1733,6 +1752,7 @@
]
},
{
"OutputSourceId": "main-speakers",
"SourceId": "asio-ch3",
"SampleRate": 192000,
"Profile": {
@ -2039,6 +2059,7 @@
{
"SymbolId": 0,
"Reliability": 1.9664832701324948,
"PhaseCoherence": 0.9999999999999998,
"ExpectedCenterHz": 3625,
"ObservedCenterHz": 3625,
"MeanEnergy": 1.9831606924956895,
@ -2050,6 +2071,7 @@
{
"SymbolId": 1,
"Reliability": 1.9550716359955316,
"PhaseCoherence": 0.9999608311103569,
"ExpectedCenterHz": 4075,
"ObservedCenterHz": 4075,
"MeanEnergy": 1.9724028422080755,
@ -2061,6 +2083,7 @@
{
"SymbolId": 2,
"Reliability": 1.98317479150165,
"PhaseCoherence": 1,
"ExpectedCenterHz": 4525,
"ObservedCenterHz": 4525,
"MeanEnergy": 1.98317479150165,
@ -2072,6 +2095,7 @@
{
"SymbolId": 3,
"Reliability": 1.9831581977468316,
"PhaseCoherence": 1,
"ExpectedCenterHz": 4975,
"ObservedCenterHz": 4975,
"MeanEnergy": 1.9831581977468316,
@ -2083,6 +2107,7 @@
{
"SymbolId": 4,
"Reliability": 1.9831595137173625,
"PhaseCoherence": 1,
"ExpectedCenterHz": 5425,
"ObservedCenterHz": 5425,
"MeanEnergy": 1.9831595137173625,
@ -2094,6 +2119,7 @@
{
"SymbolId": 5,
"Reliability": 1.983165098459816,
"PhaseCoherence": 1,
"ExpectedCenterHz": 5875,
"ObservedCenterHz": 5875,
"MeanEnergy": 1.983165098459816,
@ -2105,6 +2131,7 @@
{
"SymbolId": 6,
"Reliability": 1.9831633177029786,
"PhaseCoherence": 1,
"ExpectedCenterHz": 6325,
"ObservedCenterHz": 6325,
"MeanEnergy": 1.9831633177029786,
@ -2116,6 +2143,7 @@
{
"SymbolId": 7,
"Reliability": 1.9831607953145476,
"PhaseCoherence": 0.9999999999999999,
"ExpectedCenterHz": 6775,
"ObservedCenterHz": 6775,
"MeanEnergy": 1.9831607953145476,
@ -2127,6 +2155,7 @@
{
"SymbolId": 31,
"Reliability": 1.9831585964902498,
"PhaseCoherence": 1,
"ExpectedCenterHz": 17575,
"ObservedCenterHz": 17575,
"MeanEnergy": 1.9831585964902498,

View file

@ -13,13 +13,9 @@
"kind": "Audio",
"origin": "LocalDevice",
"enabled": true,
"adapter": "frame-events",
"command": "../native/probes/asio_audio_cadence/build/Release/asio_audio_cadence.exe",
"arguments": [
"--set-sample-rate",
"192000",
"--emit-json-blocks"
],
"adapter": "asio",
"command": "../native/asio_capture/build/Release/mimir_asio_capture.dll",
"sampleRate": 192000,
"acceptSourceIds": [
"asio-ch0",
"asio-ch1",

View file

@ -13,13 +13,9 @@
"kind": "Audio",
"origin": "LocalDevice",
"enabled": true,
"adapter": "frame-events",
"command": "../native/probes/asio_audio_cadence/build/Release/asio_audio_cadence.exe",
"arguments": [
"--set-sample-rate",
"192000",
"--emit-json-blocks"
],
"adapter": "asio",
"command": "../native/asio_capture/build/Release/mimir_asio_capture.dll",
"sampleRate": 192000,
"acceptSourceIds": [
"asio-ch0",
"asio-ch1",

View file

@ -78,11 +78,12 @@ per-band response evidence, so the same stream can calibrate timing and the
speaker/room/mic transfer function. `MimirChirpBinCalibrationModel` is the live
surface for that evidence. It stores measured usable bands per source,
expected-symbol versus observed-bin confusion observations, timing residuals,
delay hypotheses, phase summaries, and an adaptive codebook plan. The raw
profile is kept even when a timing report is rejected, because a failed sync
window can still teach us which bands survived the speaker/room/mic path. The
decoder can consume the persisted model to weight reliable symbols, downweight
dead bands, apply first-order group-delay correction, and seed global delay
delay hypotheses, phase summaries, and an adaptive codebook plan per
output/mic path. The raw profile is kept even when a timing report is rejected,
because a failed sync window can still teach us which bands survived the
speaker/room/mic path. The decoder can consume the persisted model to weight
reliable symbols, downweight dead bands, apply phase-coherence weighting,
apply first-order group-delay correction, and search joint global delay/bin-shift
hypotheses before selecting the coherent anchor path. The model also owns the
active emission plan: if the measured path only leaves a smaller alphabet, Mimir
emits that reliable symbol set and raises the de Bruijn order so the timeline
@ -130,11 +131,12 @@ buffer, and captures nonzero 4-channel `Int32LSB` input callbacks at 192 kHz.
Raven also has a loopback-capable Scarlett ASIO path at 192 kHz for
co-streamer/game timing evidence, but Starfire owns the heavy local alignment,
soundfield, and sensor-fusion work.
The worker now also supports `--emit-json-blocks`, which streams each ASIO
callback as sample-bearing `audio-block` events (`asio-ch0` through `asio-ch3`)
for `MimirFrameEventProcessStreamSource`. That is a continuous ASIO runtime
ingest path through the existing process boundary; the later native ABI cut
should remove stdout/base64 overhead, not change ASIO ownership.
The runtime hot path is now `native/asio_capture` plus
`MimirAsioStreamSource`. The native DLL owns the Focusrite ASIO COM driver and
keeps callback buffers in process; the managed source drains one 192 kHz Float32
block per ASIO input channel into `Mimir.Runtime` buffers (`asio-ch0` through
`asio-ch3`). The older `--emit-json-blocks` worker path remains a diagnostic
probe, not the live Scarlett ingest path.
The probe also has a `--monitor-sweep` mode for measuring the acoustic ceiling.
At 192 kHz, ASIO loopback detected emitted 8-40 kHz tones cleanly, but the
@ -146,8 +148,11 @@ The synthetic chirp-bin timing proof now runs at arbitrary sample rates with
`--hybrid-sync-self-test --sample-rate N`. On the same physical delay, measured
in memory, the active chirp-bin path recovered about 0.369 us error at 48 kHz,
0.168 us at 96 kHz, and below printed microsecond precision at 192 kHz. That
proves the timing kernel benefits from the Scarlett ASIO rate; the live proof
still depends on feeding ASIO callbacks into `Mimir.Runtime`.
proves the timing kernel benefits from the Scarlett ASIO rate. The live ingest
proof now feeds ASIO callbacks into `Mimir.Runtime` without a process/stdout
bridge: a two-second BufferSmoke run at 192 kHz ingested more than 12,000
sample-bearing blocks and retained 2,048 blocks per channel across `asio-ch0`
through `asio-ch3`.
The full probe runtime config now enables sample-bearing blocks for every local
audio source. `MimirChirpletTimeline` owns the emitted calibration stream and
@ -269,8 +274,9 @@ The older `MimirChirpletTimeline` matcher is a diagnostic/reference artifact,
not the active runtime path. `BuildChirpletEnergyTrace` still behaves like a
dense sliding matched-filter bank and should not retake authority.
Next, replace the diagnostic bridge with native audio capture workers that
append typed blocks into `Mimir.Runtime`, then expose buffer depth, clock state,
delay estimates, and stem routing in Fensalir UI. The analyzer accepts Float32,
Int16, Int24, and Int32 PCM windows so the direct driver path can preserve real
interface formats before Faust/native DSP owns the hot resampling and alignment.
Next, use the in-process ASIO source as the local Scarlett authority, prove
stable acoustic anchors against real mics, then expose buffer depth, clock
state, delay estimates, and stem routing in Fensalir UI. The analyzer accepts
Float32, Int16, Int24, and Int32 PCM windows so direct driver paths can preserve
real interface formats before Faust/native DSP owns the hot resampling and
alignment.

View file

@ -36,6 +36,10 @@ a named invariant that the native runtime cannot protect yet.
from native probes into the same rolling buffers Fensalir inspects. One probe
process can accept multiple emitted `sourceId` values so it does not reopen
the same camera set once per stream.
- `native/asio_capture` plus `MimirAsioStreamSource` provide the first
production-shaped Focusrite path: a native in-process ASIO callback source
feeds sample-bearing 192 kHz Float32 blocks directly into `Mimir.Runtime`
rolling buffers on one interface clock domain.
- `src/Mimir.BufferSmoke` loads the runtime config, polls the synchronization
hub, and prints the actual rolling buffers. Use `--require-samples` when an
empty declared sensor buffer should fail the run. Use `--chirplet-self-test`
@ -106,7 +110,8 @@ a named invariant that the native runtime cannot protect yet.
bands, expected-symbol versus observed-bin confusion observations, timing
residuals, delay hypotheses, phase summaries, and an adaptive codebook plan.
The active decoder can consume that model as learned response weighting,
first-order group-delay correction, and global delay-hypothesis seeds. The
phase-coherence weighting, first-order group-delay correction, and joint
global delay/bin-shift hypotheses. The
runtime emitter also consumes the model's emission plan, rendering the smaller
reliable symbol alphabet at the higher recommended de Bruijn order when the
physical path cannot support all 32 bins.
@ -129,13 +134,14 @@ a named invariant that the native runtime cannot protect yet.
loopback timing proof, but failed timing windows now leave usable response
evidence instead of silence.
- `config/mimir-runtime.asio.example.json` is the minimal continuous Scarlett
runtime ingest proof. It starts the ASIO worker at 192 kHz with
`--emit-json-blocks` and declares `asio-ch0` through `asio-ch3` as accepted
audio sources. A two-second BufferSmoke run ingested 13,324 sample-bearing
blocks, 3,331 per channel, proving loopback and mic channels enter
`Mimir.Runtime` together in one ASIO clock domain. BufferSmoke does not emit
speaker calibration audio, so that proof is ingest-only rather than a sync
report.
runtime ingest proof. It loads `native/asio_capture` in process at 192 kHz
and declares `asio-ch0` through `asio-ch3` as accepted audio sources. A
two-second BufferSmoke run ingested more than 12,000 sample-bearing blocks
and retained 2,048 blocks per channel, proving loopback and mic channels enter
`Mimir.Runtime` together in one ASIO clock domain without stdout/base64
transport. BufferSmoke does not
emit speaker calibration audio, so that proof is ingest-only rather than a
sync report.
A standalone 192 kHz synthetic receiver test recovers a 500 ms delayed audio
stream to below printed microsecond precision using only the chirp-bin
codebook and schedule state.
@ -170,26 +176,22 @@ a named invariant that the native runtime cannot protect yet.
other cameras.
2. Feed those drivers into `MimirVideoCaptureDriverSource` and prove sustained
frame cadence in the rolling buffers.
3. Replace the WASAPI frame-event diagnostic bridge with native audio capture
workers for local mic, loopback, and network audio feeds. Use the verified
Focusrite ASIO path for Scarlett capture; WASAPI remains a diagnostic
witness, not the Scarlett hot path.
4. Use chirp-bin calibration profiles from real microphones to tune acoustic
3. Use chirp-bin calibration profiles from real microphones to tune acoustic
bands, gain, and code spacing without weakening the standalone
codebook/schedule receiver invariant.
5. Add the synchronization actuator: drive a variable-rate resampler and
4. Add the synchronization actuator: drive a variable-rate resampler and
fractional delay line per non-reference stream from the smoothed
`MimirAudioSynchronizationState`. First, prove the constrained chirplet
decoder through real loopback and microphone paths so every correctly heard
triplet becomes a deterministic timeline anchor before the actuator moves
samples.
6. Prove the chirp-bin hybrid fallback through real loopback and microphones
5. Prove the chirp-bin hybrid fallback through real loopback and microphones
with probe durations long enough to keep loopback and mic windows live.
7. Bind Fensalir UI to the synchronization hub so buffer depth, stream cadence,
6. Bind Fensalir UI to the synchronization hub so buffer depth, stream cadence,
source timestamps, and output settings are visible and adjustable.
8. Move GPU feature extraction, fusion, material fitting, render budgeting, and
7. Move GPU feature extraction, fusion, material fitting, render budgeting, and
Spout2 publication into Fensalir.
9. Move mic alignment, room suppression, voice separation, spatialization, and
8. Move mic alignment, room suppression, voice separation, spatialization, and
stem generation into Faust/native DSP.
10. Keep the OBS bridge witness ledger as evidence before expanding receiver
9. Keep the OBS bridge witness ledger as evidence before expanding receiver
machinery.

View file

@ -6,7 +6,7 @@ This directory is for the hot path.
- `reservoir`: the first native five-second spatiotemporal reservoir core. It
owns one shared-edge rolling buffer with typed sample views.
Aquarium/Faust integration should build on this invariant instead of copying
Fensalir/Faust integration should build on this invariant instead of copying
the deadline bridge cache.
It exports `rlib`, `cdylib`, and `staticlib` artifacts plus the C header at
`reservoir/include/localcast_reservoir.h`.
@ -35,16 +35,16 @@ The reservoir ABI is intentionally small:
## Runtime ABI
`LocalcastRuntime` is the live spine for Aquarium/Faust integration:
`LocalcastRuntime` is the live spine for Fensalir/Faust integration:
- it owns one native reservoir;
- it exposes typed producer functions for camera frames, camera features, scene
rays, surface claims, material claims, audio blocks, phase claims, event
claims, and render packets;
- it exposes `LocalcastRuntimeStatus` so Aquarium/Faust can inspect the shared
- it exposes `LocalcastRuntimeStatus` so Fensalir/Faust can inspect the shared
edge, window start, total sample count, and typed-view counts without polling JSON or
MessagePack files;
- it exposes total and typed-view sample reads so Aquarium/Faust can consume the
- it exposes total and typed-view sample reads so Fensalir/Faust can consume the
same rolling window producers append to.
`LocalcastProducer` is the native ingress helper for capture workers:
@ -69,16 +69,16 @@ The reservoir ABI is intentionally small:
optional metadata handle;
- `localcast_producer_push_render_packet` only accepts `RenderPacket`
producers and stores the descriptor pointer as the sample payload handle;
- Aquarium owns decoding and GPU upload. The reservoir only indexes when this
- Fensalir owns decoding and GPU upload. The reservoir only indexes when this
render packet exists in the shared rolling window.
`LocalcastRenderPoint` is the fixed point-buffer element shape for the initial
native/Aquarium render path. It carries a stable-key hash, source timestamp,
native/Fensalir render path. It carries a stable-key hash, source timestamp,
position, radius, color/alpha, and confidence. Richer material/brush payloads
can get their own descriptors later; this point shape exists to kill the old
JSON/CultCache render-frame dependency first.
Payload handles are owned by the caller. Aquarium should treat them as handles
Payload handles are owned by the caller. Fensalir should treat them as handles
to GPU/native visual memory; Faust/native DSP should treat them as handles to
audio buffers. This crate does not interpret those bytes. It is the clocked
retention authority, not the renderer or DSP engine in a tiny hat.

View file

@ -0,0 +1,15 @@
cmake_minimum_required(VERSION 3.20)
project(mimir_asio_capture LANGUAGES CXX)
add_library(mimir_asio_capture SHARED
mimir_asio_capture.cpp)
target_compile_features(mimir_asio_capture PRIVATE cxx_std_20)
target_compile_definitions(mimir_asio_capture PRIVATE
WIN32_LEAN_AND_MEAN
NOMINMAX
_WIN32_WINNT=0x0A00)
target_link_libraries(mimir_asio_capture PRIVATE
ole32
oleaut32)

View file

@ -0,0 +1,405 @@
#include <windows.h>
#include <objbase.h>
#include <algorithm>
#include <atomic>
#include <condition_variable>
#include <cstdint>
#include <cstdio>
#include <cstring>
#include <deque>
#include <mutex>
#include <string>
#include <vector>
using ASIOBool = long;
using ASIOError = long;
using ASIOSampleRate = double;
using ASIOSampleType = long;
constexpr ASIOError ASE_OK = 0;
struct ASIOSamples
{
unsigned long hi;
unsigned long lo;
};
struct ASIOTimeStamp
{
unsigned long hi;
unsigned long lo;
};
struct ASIOClockSource
{
long index;
long associatedChannel;
long associatedGroup;
ASIOBool isCurrentSource;
char name[32];
};
struct ASIOChannelInfo
{
long channel;
ASIOBool isInput;
ASIOBool isActive;
long channelGroup;
ASIOSampleType type;
char name[32];
};
struct ASIOBufferInfo
{
ASIOBool isInput;
long channelNum;
void* buffers[2];
};
struct ASIOTime;
struct ASIOCallbacks
{
void (*bufferSwitch)(long doubleBufferIndex, ASIOBool directProcess);
void (*sampleRateDidChange)(ASIOSampleRate sRate);
long (*asioMessage)(long selector, long value, void* message, double* opt);
ASIOTime* (*bufferSwitchTimeInfo)(ASIOTime* params, long doubleBufferIndex, ASIOBool directProcess);
};
struct IASIO : public IUnknown
{
virtual ASIOBool init(void* sysHandle) = 0;
virtual void getDriverName(char* name) = 0;
virtual long getDriverVersion() = 0;
virtual void getErrorMessage(char* string) = 0;
virtual ASIOError start() = 0;
virtual ASIOError stop() = 0;
virtual ASIOError getChannels(long* numInputChannels, long* numOutputChannels) = 0;
virtual ASIOError getLatencies(long* inputLatency, long* outputLatency) = 0;
virtual ASIOError getBufferSize(long* minSize, long* maxSize, long* preferredSize, long* granularity) = 0;
virtual ASIOError canSampleRate(ASIOSampleRate sampleRate) = 0;
virtual ASIOError getSampleRate(ASIOSampleRate* sampleRate) = 0;
virtual ASIOError setSampleRate(ASIOSampleRate sampleRate) = 0;
virtual ASIOError getClockSources(ASIOClockSource* clocks, long* numSources) = 0;
virtual ASIOError setClockSource(long reference) = 0;
virtual ASIOError getSamplePosition(ASIOSamples* samplePosition, ASIOTimeStamp* timeStamp) = 0;
virtual ASIOError getChannelInfo(ASIOChannelInfo* info) = 0;
virtual ASIOError createBuffers(ASIOBufferInfo* bufferInfos, long numChannels, long bufferSize, ASIOCallbacks* callbacks) = 0;
virtual ASIOError disposeBuffers() = 0;
virtual ASIOError controlPanel() = 0;
virtual ASIOError future(long selector, void* opt) = 0;
virtual ASIOError outputReady() = 0;
};
struct MimirAsioBlock
{
int channel = 0;
int sampleRate = 0;
int frameCount = 0;
std::uint64_t sequence = 0;
std::int64_t timestampNs = 0;
std::vector<float> samples;
};
struct MimirAsioCapture
{
IASIO* driver = nullptr;
ASIOCallbacks callbacks{};
std::vector<ASIOBufferInfo> buffers;
std::vector<ASIOChannelInfo> channels;
std::deque<MimirAsioBlock> queue;
std::mutex mutex;
long inputs = 0;
long outputs = 0;
long bufferSize = 0;
double sampleRate = 0.0;
std::uint64_t frameCursor = 0;
std::uint64_t sequence = 0;
bool running = false;
};
static MimirAsioCapture* g_capture = nullptr;
static int bytesPerSample(ASIOSampleType type)
{
switch (type)
{
case 16: return 2;
case 17: return 3;
case 18: return 4;
case 19: return 4;
case 20: return 8;
case 24:
case 25:
case 26:
case 27: return 4;
default: return 0;
}
}
static double readSample(const unsigned char* sample, ASIOSampleType type)
{
switch (type)
{
case 16:
return *reinterpret_cast<const short*>(sample) / 32768.0;
case 17:
{
int value = sample[0] | (sample[1] << 8) | (sample[2] << 16);
if ((value & 0x00800000) != 0)
{
value |= static_cast<int>(0xff000000);
}
return value / 8388608.0;
}
case 18:
case 24:
case 25:
case 26:
case 27:
return *reinterpret_cast<const int*>(sample) / 2147483648.0;
case 19:
return *reinterpret_cast<const float*>(sample);
case 20:
return *reinterpret_cast<const double*>(sample);
default:
return 0.0;
}
}
static CLSID clsidFromString(const char* raw)
{
wchar_t wide[64]{};
MultiByteToWideChar(CP_UTF8, 0, raw, -1, wide, static_cast<int>(std::size(wide)));
CLSID clsid{};
CLSIDFromString(wide, &clsid);
return clsid;
}
static void pushInputBlocks(MimirAsioCapture& capture, long doubleBufferIndex)
{
const auto startFrame = capture.frameCursor;
const auto timestampNs = static_cast<std::int64_t>(
static_cast<double>(startFrame) * 1000000000.0 / capture.sampleRate);
std::lock_guard<std::mutex> lock(capture.mutex);
for (long channel = 0; channel < capture.inputs; ++channel)
{
auto& buffer = capture.buffers[static_cast<size_t>(channel)];
auto& info = capture.channels[static_cast<size_t>(channel)];
auto* raw = static_cast<unsigned char*>(buffer.buffers[doubleBufferIndex]);
const auto sampleBytes = bytesPerSample(info.type);
if (!raw || sampleBytes <= 0)
{
continue;
}
MimirAsioBlock block;
block.channel = static_cast<int>(channel);
block.sampleRate = static_cast<int>(capture.sampleRate);
block.frameCount = static_cast<int>(capture.bufferSize);
block.sequence = capture.sequence++;
block.timestampNs = timestampNs;
block.samples.resize(static_cast<size_t>(capture.bufferSize));
for (long frame = 0; frame < capture.bufferSize; ++frame)
{
block.samples[static_cast<size_t>(frame)] = static_cast<float>(
readSample(raw + frame * sampleBytes, info.type));
}
capture.queue.push_back(std::move(block));
}
while (capture.queue.size() > 4096)
{
capture.queue.pop_front();
}
capture.frameCursor += static_cast<std::uint64_t>(capture.bufferSize);
}
static void bufferSwitch(long doubleBufferIndex, ASIOBool)
{
if (g_capture && g_capture->running)
{
pushInputBlocks(*g_capture, doubleBufferIndex);
}
}
static void sampleRateDidChange(ASIOSampleRate) {}
static long asioMessage(long, long, void*, double*)
{
return 0;
}
static ASIOTime* bufferSwitchTimeInfo(ASIOTime* params, long doubleBufferIndex, ASIOBool directProcess)
{
bufferSwitch(doubleBufferIndex, directProcess);
return params;
}
extern "C"
{
__declspec(dllexport) MimirAsioCapture* mimir_asio_create(
const char* clsidText,
double requestedSampleRate,
int* sampleRate,
int* inputCount,
int* preferredBufferSize)
{
CoInitializeEx(nullptr, COINIT_APARTMENTTHREADED);
auto* capture = new MimirAsioCapture();
const auto clsid = clsidFromString(clsidText ? clsidText : "{AC4D0455-50D7-4498-B3CD-9A41D130B759}");
const auto hr = CoCreateInstance(clsid, nullptr, CLSCTX_INPROC_SERVER, clsid, reinterpret_cast<void**>(&capture->driver));
if (FAILED(hr) || !capture->driver)
{
delete capture;
CoUninitialize();
return nullptr;
}
HWND hwnd = GetConsoleWindow();
if (!hwnd)
{
hwnd = GetDesktopWindow();
}
if (!capture->driver->init(hwnd))
{
capture->driver->Release();
delete capture;
CoUninitialize();
return nullptr;
}
capture->driver->getChannels(&capture->inputs, &capture->outputs);
long minSize = 0;
long maxSize = 0;
long granularity = 0;
capture->driver->getBufferSize(&minSize, &maxSize, &capture->bufferSize, &granularity);
if (requestedSampleRate > 0.0)
{
capture->driver->setSampleRate(requestedSampleRate);
}
capture->driver->getSampleRate(&capture->sampleRate);
capture->buffers.resize(static_cast<size_t>(capture->inputs));
capture->channels.resize(static_cast<size_t>(capture->inputs));
for (long channel = 0; channel < capture->inputs; ++channel)
{
capture->buffers[static_cast<size_t>(channel)].isInput = 1;
capture->buffers[static_cast<size_t>(channel)].channelNum = channel;
capture->channels[static_cast<size_t>(channel)].channel = channel;
capture->channels[static_cast<size_t>(channel)].isInput = 1;
capture->driver->getChannelInfo(&capture->channels[static_cast<size_t>(channel)]);
}
capture->callbacks.bufferSwitch = bufferSwitch;
capture->callbacks.sampleRateDidChange = sampleRateDidChange;
capture->callbacks.asioMessage = asioMessage;
capture->callbacks.bufferSwitchTimeInfo = bufferSwitchTimeInfo;
if (capture->driver->createBuffers(capture->buffers.data(), capture->inputs, capture->bufferSize, &capture->callbacks) != ASE_OK)
{
capture->driver->Release();
delete capture;
CoUninitialize();
return nullptr;
}
if (sampleRate)
{
*sampleRate = static_cast<int>(capture->sampleRate);
}
if (inputCount)
{
*inputCount = static_cast<int>(capture->inputs);
}
if (preferredBufferSize)
{
*preferredBufferSize = static_cast<int>(capture->bufferSize);
}
return capture;
}
__declspec(dllexport) int mimir_asio_start(MimirAsioCapture* capture)
{
if (!capture || !capture->driver)
{
return 0;
}
g_capture = capture;
capture->running = true;
return capture->driver->start() == ASE_OK ? 1 : 0;
}
__declspec(dllexport) int mimir_asio_read(
MimirAsioCapture* capture,
int* channel,
std::int64_t* timestampNs,
std::uint64_t* sequence,
int* frameCount,
float* samples,
int maxFrames)
{
if (!capture || !samples || maxFrames <= 0)
{
return 0;
}
std::lock_guard<std::mutex> lock(capture->mutex);
if (capture->queue.empty())
{
return 0;
}
auto block = std::move(capture->queue.front());
capture->queue.pop_front();
const auto count = std::min(block.frameCount, maxFrames);
std::memcpy(samples, block.samples.data(), static_cast<size_t>(count) * sizeof(float));
if (channel)
{
*channel = block.channel;
}
if (timestampNs)
{
*timestampNs = block.timestampNs;
}
if (sequence)
{
*sequence = block.sequence;
}
if (frameCount)
{
*frameCount = count;
}
return 1;
}
__declspec(dllexport) void mimir_asio_destroy(MimirAsioCapture* capture)
{
if (!capture)
{
return;
}
capture->running = false;
if (g_capture == capture)
{
g_capture = nullptr;
}
if (capture->driver)
{
capture->driver->stop();
capture->driver->disposeBuffers();
capture->driver->Release();
}
delete capture;
CoUninitialize();
}
}

View file

@ -116,8 +116,9 @@ usable bands, expected-symbol versus observed-bin confusion observations,
timing residuals, delay hypotheses, phase summaries, and an adaptive codebook
plan. The analyzer keeps raw profiles even when no timing report is accepted,
and the active decoder can consume a persisted model for learned response
weighting, first-order group-delay correction, and global delay-hypothesis
seeding. Runtime chirp-bin emission consumes the same model's emission plan, so
weighting, phase-coherence weighting, first-order group-delay correction, and
joint global delay/bin-shift hypotheses. Runtime chirp-bin emission consumes the
same model's emission plan, so
the emitted timeline can use the measured reliable symbol set at a higher
de Bruijn order instead of continuing to spend code on dead bins. Reports/states
expose `delayUs` next to fractional sample delay.
@ -163,14 +164,15 @@ The ASIO probe can now play raw mono Float32 timeline audio through the
Focusrite outputs while capturing every ASIO input as raw interleaved Float32.
`Mimir.BufferSmoke --analyze-asio-f32` feeds those captured channels into the
same runtime active analyzer. `--calibrate-chirp-bin-asio-f32` computes and
persists the response/confusion/delay model, and `--analyze-asio-f32
--calibration ...` loads it into the active decoder. The ASIO worker also
supports `--emit-json-blocks`, so Focusrite ASIO callbacks can feed
sample-bearing `audio-block` events into `Mimir.Runtime` continuously through
the existing frame-event adapter while the native ABI cut is pending. The
minimal `config/mimir-runtime.asio.example.json` proof ingested 13,324
sample-bearing 192 kHz blocks across `asio-ch0` through `asio-ch3` in two
seconds, all from the same ASIO callback stream. A real
persists the response/confusion/delay model per output/mic path, and
`--analyze-asio-f32 --calibration ...` loads it into the active decoder.
`native/asio_capture` and `MimirAsioStreamSource` are now the runtime Scarlett
path: Focusrite ASIO callbacks feed sample-bearing 192 kHz Float32 blocks into
`Mimir.Runtime` in process, without the diagnostic JSON/stdout bridge. The
minimal `config/mimir-runtime.asio.example.json` proof ingested more than
12,000 sample-bearing 192 kHz blocks across `asio-ch0` through `asio-ch3` in
two seconds and retained 2,048 blocks per channel, all from the same ASIO
callback stream. A real
192 kHz chirp-bin Scarlett artifact decoded
`Loopback 1 -> Loopback 2` at exactly `0.000 us` with 12 matched anchors and
0.999 confidence. The same analyzer now prints calibration profiles for

View file

@ -92,9 +92,10 @@ Get-Content .\state\evidence.jsonl -Tail 8
Each classified chirp carries the full dechirped bin-energy surface; sync
reports expose those bands, and `MimirChirpBinCalibrationModel` preserves
measured usable bands, response/confusion observations, timing residuals,
delay hypotheses, phase summaries, and adaptive codebook plans. The active
decoder can load the model for learned weighting, first-order group-delay
correction, and global delay-hypothesis seeds. Runtime chirp-bin emission
delay hypotheses, phase summaries, and adaptive codebook plans per
output/mic path. The active decoder can load the model for learned weighting,
phase-coherence weighting, first-order group-delay correction, and joint
global delay/bin-shift hypotheses. Runtime chirp-bin emission
loads the same model's emission plan, so a physical calibration can shrink the
emitted alphabet and raise sequence order instead of continuing to use dead
bins.
@ -129,20 +130,21 @@ Get-Content .\state\evidence.jsonl -Tail 8
`--calibrate-chirp-bin-asio-f32` now persists a real response/confusion/delay
model and `--analyze-asio-f32 --calibration ...` consumes it. In the stored
artifact, physical input 1 produced two reliable symbols and a different
strongest-bin profile while failing pairwise timing. The ASIO native worker
also supports `--emit-json-blocks` so `Mimir.Runtime` can ingest Focusrite ASIO
callbacks continuously through the frame-event adapter. The minimal
`config/mimir-runtime.asio.example.json` proof ingested 13,324 sample-bearing
192 kHz blocks across `asio-ch0` through `asio-ch3` in two seconds. Acoustic robustness is
still open, but failed timing now leaves response evidence.
strongest-bin profile while failing pairwise timing. `native/asio_capture`
plus `MimirAsioStreamSource` now load Focusrite ASIO in process and feed
sample-bearing 192 kHz Float32 blocks directly into runtime buffers. The
minimal `config/mimir-runtime.asio.example.json` proof ingests more than
12,000 blocks across `asio-ch0` through `asio-ch3` in two seconds and retains
2,048 blocks per channel. Acoustic robustness is still open, but failed
timing now leaves response evidence.
- Raven also has a 192 kHz loopback-capable Scarlett for co-streamer/game timing
evidence. Do not move the heavy soundfield or sensor-fusion workload there.
## Immediate Re-entry Instruction
Use the chirp-bin calibration models from real microphones to tune the active
codebook/bands/weighting, then replace the ASIO JSON-block process bridge with
a native ABI callback source and turn decoded clock fits into the
SRO/fractional-delay actuator. Keep loopback as timing authority. Do not call
synchronization analysis from UI/telemetry readouts. Do not restore deleted
script infrastructure because a stale doc once missed it.
codebook/bands/weighting, then turn decoded clock fits into the
SRO/fractional-delay actuator. Keep loopback as timing authority and keep
Scarlett capture on the in-process ASIO source. Do not call synchronization
analysis from UI/telemetry readouts. Do not restore deleted script
infrastructure because a stale doc once missed it.

View file

@ -599,6 +599,7 @@ static int CalibrateChirpBinAsioFloat32(
int channels,
int referenceChannel)
{
var outputSourceId = ParseStringOption(args, "--output-source-id", "main-speakers");
if (args.Any(arg => string.Equals(arg, "--capture-asio", StringComparison.OrdinalIgnoreCase)))
{
var probe = ParseStringOption(
@ -636,7 +637,7 @@ static int CalibrateChirpBinAsioFloat32(
}
var referenceSourceId = $"asio-ch{referenceChannel}";
var model = MimirChirpBinCalibrationModel.FromDecodes(referenceSourceId, sampleRate, decodes);
var model = MimirChirpBinCalibrationModel.FromDecodes(referenceSourceId, sampleRate, decodes, outputSourceId);
model.Save(outputPath);
Console.WriteLine($"chirp-bin-calibration path={outputPath} input={inputPath} sampleRate={sampleRate} channels={channels} frames={frameCount} reference={referenceSourceId}");
foreach (var path in model.Paths)

View file

@ -6,6 +6,7 @@
<RootNamespace>Mimir.Runtime</RootNamespace>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>
</PropertyGroup>

View file

@ -0,0 +1,225 @@
using System.Runtime.InteropServices;
using System.Collections.Concurrent;
using System.Runtime.Versioning;
namespace Mimir.Runtime.Synchronization;
public sealed record MimirAsioStreamSourceOptions(
string NativeLibraryPath,
string DriverClsid,
int SampleRate,
IReadOnlyList<string> SourceIds);
[SupportedOSPlatform("windows")]
public sealed class MimirAsioStreamSource : IMimirStreamSource
{
private const string DefaultClsid = "{AC4D0455-50D7-4498-B3CD-9A41D130B759}";
private readonly ConcurrentQueue<MimirStreamSample> samples = new();
private readonly ManualResetEventSlim started = new();
private readonly Thread captureThread;
private readonly CancellationTokenSource cancellation = new();
private readonly string[] sourceIds;
private int sampleRate;
private string? startError;
private bool disposed;
public MimirAsioStreamSource(
MimirStreamDescriptor descriptor,
MimirAsioStreamSourceOptions options)
{
Descriptor = descriptor;
Options = options;
sourceIds = options.SourceIds.Count > 0
? options.SourceIds.ToArray()
: Enumerable.Range(0, 8).Select(index => $"asio-ch{index}").ToArray();
captureThread = new Thread(CaptureLoop)
{
IsBackground = true,
Name = "Mimir Focusrite ASIO capture",
};
captureThread.SetApartmentState(ApartmentState.STA);
captureThread.Start();
if (!started.Wait(TimeSpan.FromSeconds(5)))
{
throw new InvalidOperationException("Timed out starting Focusrite ASIO capture source");
}
if (startError != null)
{
throw new InvalidOperationException(startError);
}
}
public MimirStreamDescriptor Descriptor { get; }
public MimirAsioStreamSourceOptions Options { get; }
public bool ExposesDescriptorBuffer => false;
public bool TryRead(out MimirStreamSample sample)
{
sample = default;
ObjectDisposedException.ThrowIf(disposed, this);
return samples.TryDequeue(out sample);
}
public void Dispose()
{
if (disposed)
{
return;
}
disposed = true;
cancellation.Cancel();
captureThread.Join(TimeSpan.FromSeconds(2));
cancellation.Dispose();
started.Dispose();
}
private void CaptureLoop()
{
nint handle = 0;
try
{
Native.Load(Options.NativeLibraryPath);
handle = Native.Create(
string.IsNullOrWhiteSpace(Options.DriverClsid) ? DefaultClsid : Options.DriverClsid,
Options.SampleRate,
out sampleRate,
out _,
out var maxFrames);
if (handle == 0)
{
startError = "Could not create Focusrite ASIO capture source";
started.Set();
return;
}
if (!Native.Start(handle))
{
startError = "Could not start Focusrite ASIO capture source";
started.Set();
return;
}
started.Set();
var sampleBuffer = new float[Math.Max(1, maxFrames)];
while (!cancellation.IsCancellationRequested)
{
if (!Native.Read(
handle,
out var channel,
out var timestampNs,
out var sequence,
out var frameCount,
sampleBuffer,
sampleBuffer.Length))
{
Thread.Sleep(1);
continue;
}
if (channel < 0)
{
continue;
}
var sourceId = channel < sourceIds.Length ? sourceIds[channel] : $"asio-ch{channel}";
var bytes = new byte[frameCount * sizeof(float)];
MemoryMarshal.AsBytes(sampleBuffer.AsSpan(0, frameCount)).CopyTo(bytes);
samples.Enqueue(new MimirStreamSample(
sourceId,
MimirStreamKind.Audio,
Descriptor.Origin,
timestampNs,
DateTimeOffset.UtcNow.ToUnixTimeMilliseconds() * 1_000_000L,
sequence,
0,
bytes.Length,
bytes,
AudioBlock: new MimirAudioBlockDescriptor(
sampleRate,
1,
MimirAudioSampleFormat.Float32,
frameCount,
timestampNs)));
while (samples.Count > 8192 && samples.TryDequeue(out _))
{
}
}
}
catch (Exception ex)
{
startError = ex.Message;
started.Set();
}
finally
{
Native.Destroy(handle);
}
}
private static unsafe class Native
{
private static nint library;
private static delegate* unmanaged[Stdcall]<byte*, double, out int, out int, out int, nint> create;
private static delegate* unmanaged[Stdcall]<nint, int> start;
private static delegate* unmanaged[Stdcall]<nint, out int, out long, out ulong, out int, float*, int, int> read;
private static delegate* unmanaged[Stdcall]<nint, void> destroy;
public static void Load(string path)
{
if (library != 0)
{
return;
}
if (string.IsNullOrWhiteSpace(path))
{
path = Path.Combine(AppContext.BaseDirectory, "mimir_asio_capture.dll");
}
library = NativeLibrary.Load(path);
create = (delegate* unmanaged[Stdcall]<byte*, double, out int, out int, out int, nint>)NativeLibrary.GetExport(library, "mimir_asio_create");
start = (delegate* unmanaged[Stdcall]<nint, int>)NativeLibrary.GetExport(library, "mimir_asio_start");
read = (delegate* unmanaged[Stdcall]<nint, out int, out long, out ulong, out int, float*, int, int>)NativeLibrary.GetExport(library, "mimir_asio_read");
destroy = (delegate* unmanaged[Stdcall]<nint, void>)NativeLibrary.GetExport(library, "mimir_asio_destroy");
}
public static nint Create(string clsid, int requestedSampleRate, out int sampleRate, out int inputCount, out int preferredBufferSize)
{
var bytes = System.Text.Encoding.UTF8.GetBytes(clsid + "\0");
fixed (byte* clsidPtr = bytes)
{
return create(clsidPtr, requestedSampleRate, out sampleRate, out inputCount, out preferredBufferSize);
}
}
public static bool Start(nint handle) => start(handle) != 0;
public static bool Read(
nint handle,
out int channel,
out long timestampNs,
out ulong sequence,
out int frameCount,
float[] samples,
int maxFrames)
{
fixed (float* samplePtr = samples)
{
return read(handle, out channel, out timestampNs, out sequence, out frameCount, samplePtr, maxFrames) != 0;
}
}
public static void Destroy(nint handle)
{
if (handle != 0)
{
destroy(handle);
}
}
}
}

View file

@ -39,11 +39,12 @@ public sealed record MimirChirpBinCalibrationModel(
public static MimirChirpBinCalibrationModel FromDecodes(
string referenceSourceId,
int sampleRate,
IReadOnlyDictionary<string, MimirChirpletStreamDecode> decodes)
IReadOnlyDictionary<string, MimirChirpletStreamDecode> decodes,
string outputSourceId = "main-speakers")
{
var paths = decodes
.OrderBy(pair => pair.Key, StringComparer.Ordinal)
.Select(pair => MimirChirpBinPathCalibration.FromDecode(pair.Key, sampleRate, pair.Value))
.Select(pair => MimirChirpBinPathCalibration.FromDecode(outputSourceId, pair.Key, sampleRate, pair.Value))
.ToArray();
var emissionPlan = MimirChirpBinCodebookPlan.ForSharedEmission(paths);
paths = paths.Select(path => path with { EmissionPlan = emissionPlan }).ToArray();
@ -58,6 +59,7 @@ public sealed record MimirChirpBinCalibrationModel(
}
public sealed record MimirChirpBinPathCalibration(
string OutputSourceId,
string SourceId,
int SampleRate,
MimirChirpBinCalibrationProfile Profile,
@ -79,7 +81,20 @@ public sealed record MimirChirpBinPathCalibration(
return symbol?.MeanTimingResidualSamples ?? 0.0;
}
public double PhaseWeight(int symbolId, double observedPhaseRadians)
{
var symbol = Symbols.FirstOrDefault(item => item.SymbolId == symbolId);
if (symbol == null || symbol.ObservationCount <= 0)
{
return 1.0;
}
var error = Math.Abs(AngularDistance(observedPhaseRadians, symbol.MeanPhaseRadians));
return Math.Clamp(1.0 - error / Math.PI, 0.15, 1.0);
}
public static MimirChirpBinPathCalibration FromDecode(
string outputSourceId,
string sourceId,
int sampleRate,
MimirChirpletStreamDecode decode)
@ -99,9 +114,11 @@ public sealed record MimirChirpBinPathCalibration(
(correct / (double)items.Length) *
(1.0 / (1.0 + residualSpread / Math.Max(1.0, sampleRate * 0.00025)));
var phase = CircularMean(items.Select(item => item.PhaseRadians));
var phaseCoherence = PhaseCoherence(items.Select(item => item.PhaseRadians));
return new MimirChirpBinSymbolCalibration(
group.Key,
reliability,
phaseCoherence,
items.Average(item => item.ExpectedCenterHz),
items.Average(item => item.ObservedCenterHz),
items.Average(item => item.ObservedEnergy),
@ -115,6 +132,7 @@ public sealed record MimirChirpBinPathCalibration(
var hypotheses = BuildDelayHypotheses(decode);
var plan = MimirChirpBinCodebookPlan.FromSymbols(symbols);
return new MimirChirpBinPathCalibration(
outputSourceId,
sourceId,
sampleRate,
profile,
@ -201,6 +219,27 @@ public sealed record MimirChirpBinPathCalibration(
return count == 0 ? 0.0 : Math.Atan2(sin / count, cos / count);
}
private static double PhaseCoherence(IEnumerable<double> radians)
{
var sin = 0.0;
var cos = 0.0;
var count = 0;
foreach (var value in radians)
{
sin += Math.Sin(value);
cos += Math.Cos(value);
count++;
}
return count == 0 ? 0.0 : Math.Clamp(Math.Sqrt(sin * sin + cos * cos) / count, 0.0, 1.0);
}
private static double AngularDistance(double a, double b)
{
var delta = Math.Abs(a - b) % (Math.PI * 2.0);
return delta > Math.PI ? Math.PI * 2.0 - delta : delta;
}
}
public sealed record MimirChirpBinCodebookPlan(
@ -253,6 +292,7 @@ public sealed record MimirChirpBinCodebookPlan(
public sealed record MimirChirpBinSymbolCalibration(
int SymbolId,
double Reliability,
double PhaseCoherence,
double ExpectedCenterHz,
double ObservedCenterHz,
double MeanEnergy,

View file

@ -296,7 +296,8 @@ public sealed class MimirChirpBinTimeline
var correctedEnergy = ScoreSymbolEnergy(
samples.Slice((int)Math.Round(correctedOffset), chirpSamples),
sampleRate,
canonicalSymbol) * (calibration?.SymbolWeight(canonicalSymbol) ?? 1.0);
canonicalSymbol) *
(calibration?.SymbolWeight(canonicalSymbol) ?? 1.0);
correctedOffset -= calibration?.GroupDelayCorrectionSamples(canonicalSymbol) ?? 0.0;
correctedCandidates.Add(new MimirChirpletSymbolCandidate(
canonicalSymbol,
@ -350,7 +351,9 @@ public sealed class MimirChirpBinTimeline
.Select((score, symbol) => new MimirChirpletSymbolCandidate(
symbol,
sampleOffset - (calibration?.GroupDelayCorrectionSamples(symbol) ?? 0.0),
score.Energy * (calibration?.SymbolWeight(symbol) ?? 1.0),
score.Energy *
(calibration?.SymbolWeight(symbol) ?? 1.0) *
(calibration?.PhaseWeight(symbol, score.PhaseRadians) ?? 1.0),
bandResponses))
.OrderByDescending(candidate => candidate.Energy)
.Take(MaxSymbolCandidatesPerFrame)
@ -459,35 +462,45 @@ public sealed class MimirChirpBinTimeline
foreach (var candidatePath in CandidateSymbolPaths(window))
{
var code = CodeKey(candidatePath.Select(candidate => candidate.SymbolId));
if (!plan.CodeToIndex.TryGetValue(code, out var eventIndex))
foreach (var globalBinShift in GlobalBinShiftHypotheses(calibration))
{
continue;
}
var canonicalSymbols = candidatePath
.Select(candidate => ShiftSymbol(candidate.SymbolId, -globalBinShift))
.ToArray();
var code = CodeKey(canonicalSymbols);
if (!plan.CodeToIndex.TryGetValue(code, out var eventIndex))
{
continue;
}
var firstEvent = Default.EventForIndex((ulong)eventIndex, calibration?.EmissionPlan);
var gapError = 0.0;
for (var gap = 0; gap < candidatePath.Count - 1; gap++)
{
var measuredGap = (candidatePath[gap + 1].SampleOffset - candidatePath[gap].SampleOffset) / sampleRate;
gapError += Math.Abs(measuredGap - EventSpacingSeconds);
}
var firstEvent = Default.EventForIndex((ulong)eventIndex, calibration?.EmissionPlan);
var gapError = 0.0;
for (var gap = 0; gap < candidatePath.Count - 1; gap++)
{
var measuredGap = (candidatePath[gap + 1].SampleOffset - candidatePath[gap].SampleOffset) / sampleRate;
gapError += Math.Abs(measuredGap - EventSpacingSeconds);
}
var gapConfidence = 1.0 / (1.0 + gapError / 0.003);
if (gapConfidence < 0.55)
{
continue;
}
var gapConfidence = 1.0 / (1.0 + gapError / 0.003);
if (gapConfidence < 0.55)
{
continue;
}
var energyConfidence = Math.Clamp(candidatePath.Average(candidate => candidate.Energy), 0.0, 1.0);
candidates.Add(new MimirChirpletTimelineAnchor(
(ulong)eventIndex,
firstEvent.StartSeconds,
candidatePath[0].SampleOffset,
gapConfidence * 0.60 + energyConfidence * 0.40,
candidatePath
.Select(candidate => new MimirChirpletSymbolObservation(candidate.SymbolId, candidate.SampleOffset, candidate.Energy))
.ToArray()));
var energyConfidence = Math.Clamp(candidatePath.Average(candidate => candidate.Energy), 0.0, 1.0);
var binShiftPenalty = 1.0 / (1.0 + Math.Abs(globalBinShift) * 0.10);
candidates.Add(new MimirChirpletTimelineAnchor(
(ulong)eventIndex,
firstEvent.StartSeconds,
candidatePath[0].SampleOffset,
(gapConfidence * 0.60 + energyConfidence * 0.40) * binShiftPenalty,
candidatePath
.Select((candidate, candidateIndex) => new MimirChirpletSymbolObservation(
canonicalSymbols[candidateIndex],
candidate.SampleOffset,
candidate.Energy))
.ToArray()));
}
}
}
@ -519,6 +532,33 @@ public sealed class MimirChirpBinTimeline
return Recurse(0);
}
private static IReadOnlyList<int> GlobalBinShiftHypotheses(MimirChirpBinPathCalibration? calibration)
{
var shifts = new HashSet<int> { 0 };
if (calibration != null)
{
foreach (var hypothesis in calibration.DelayHypotheses)
{
shifts.Add(Math.Clamp(hypothesis.BinShift, -MaxTimingBinShift, MaxTimingBinShift));
}
foreach (var group in calibration.Confusion
.GroupBy(observation => observation.BinShift)
.OrderByDescending(group => group.Count())
.Take(4))
{
shifts.Add(Math.Clamp(group.Key, -MaxTimingBinShift, MaxTimingBinShift));
}
}
for (var shift = -MaxTimingBinShift; shift <= MaxTimingBinShift; shift++)
{
shifts.Add(shift);
}
return shifts.OrderBy(shift => Math.Abs(shift)).ThenBy(shift => shift).ToArray();
}
private static IReadOnlyList<MimirChirpletTimelineAnchor> SelectCoherentAnchorPath(
IReadOnlyList<MimirChirpletTimelineAnchor> candidates,
int sampleRate,

View file

@ -119,6 +119,24 @@ public sealed class MimirRuntimeConfiguration
return new MimirNativeIngestStreamSource(ToDescriptor(stream));
}
if (string.Equals(stream.Adapter, "asio", StringComparison.OrdinalIgnoreCase))
{
if (!OperatingSystem.IsWindows())
{
throw new PlatformNotSupportedException("The Mimir ASIO stream source requires Windows.");
}
#pragma warning disable CA1416
return new MimirAsioStreamSource(
ToDescriptor(stream),
new MimirAsioStreamSourceOptions(
ResolveCommand(stream.Command, configDirectory),
stream.DriverClsid,
stream.SampleRate,
stream.AcceptSourceIds));
#pragma warning restore CA1416
}
if (string.Equals(stream.Adapter, "frame-events", StringComparison.OrdinalIgnoreCase)
|| string.Equals(stream.Adapter, "json-lines", StringComparison.OrdinalIgnoreCase))
{
@ -243,4 +261,8 @@ public sealed class MimirStreamConfig
public string[] AcceptSourceIds { get; set; } = [];
public int ChunkBytes { get; set; } = 65_536;
public int SampleRate { get; set; } = 192_000;
public string DriverClsid { get; set; } = "";
}

View file

@ -74,3 +74,6 @@
{"ts": "2026-05-23T20:19:36+01:00", "type": "implementation", "status": "ok", "note": "Replaced the profile-only chirp-bin calibration cut with a usable physical calibration model. The model records per-source usable bands, symbol reliability, expected-vs-observed confusion observations, timing residuals, delay hypotheses, phase summaries, and an adaptive codebook plan. MimirChirpBinTimeline can now decode with learned per-symbol weights, group-delay correction, and persisted delay-hypothesis seeds. BufferSmoke can run --calibrate-chirp-bin-asio-f32 to render/capture/analyze/persist the model, and --analyze-asio-f32 can load it via --calibration. The ASIO native worker now supports --emit-json-blocks so Focusrite ASIO callbacks can feed Mimir.Runtime continuously through the existing frame-event adapter while the native ABI cut is still pending."}
{"ts": "2026-05-23T20:31:27+01:00", "type": "implementation", "status": "ok", "note": "Completed the physical chirp-bin calibration loop instead of stopping at summaries. The calibration model now owns a shared emission plan from measured reliable symbols, and MimirRuntime renders chirp-bin calibration with that plan so dead bins are removed from the emitted alphabet and sequence order rises as needed. MimirChirpBinTimeline decodes variable-order codewords from the same emission plan, keeps learned response weights/group-delay correction, records bin-shift in confusion/delay hypotheses, and still preserves the default 32-symbol path when no model is loaded. ASIO callback ingestion remains continuous through the ASIO worker's sample-bearing runtime stream path; the next native ABI cut removes process transport overhead, not clock-domain ownership."}
{"ts": "2026-05-23T20:33:00+01:00", "type": "verification", "status": "ok", "note": "Verified continuous runtime ASIO ingestion through the ASIO worker stream path. Using config/mimir-runtime.asio.example.json, BufferSmoke ran for 2 seconds against Focusrite USB ASIO at 192 kHz and ingested 13,324 sample-bearing audio blocks across asio-ch0..asio-ch3, 3,331 blocks per channel, each 192 Float32 frames. This proves loopback and mic channels enter Mimir.Runtime buffers continuously in one ASIO clock domain; the run did not sync because BufferSmoke does not emit speaker calibration audio."}
{"ts": "2026-05-23T21:41:47+01:00", "type": "implementation", "status": "ok", "note": "Implemented the in-process Focusrite ASIO runtime source. native/asio_capture owns the ASIO COM driver/callback buffers and MimirAsioStreamSource drains 192 kHz Float32 blocks into Mimir.Runtime buffers for asio-ch0..asio-ch3 without stdout/base64 transport. The chirp-bin calibration model is now per output/mic path and the active decoder uses learned symbol weights, phase-coherence weights, group-delay correction, and joint global delay/bin-shift hypotheses."}
{"ts": "2026-05-23T21:41:53+01:00", "type": "verification", "status": "ok", "note": "Verified the in-process ASIO runtime ingest and active decoder after the native ASIO cut. dotnet build Mimir.slnx and cmake --build native/asio_capture/build --config Release passed cleanly. BufferSmoke with config/mimir-runtime.asio.example.json ingested 12,196 sample-bearing blocks in two seconds at 192 kHz, with 2,048 buffered 192-frame Float32 blocks on each asio-ch0..asio-ch3 channel. The standalone 192 kHz chirp-bin receiver recovered a 96,000-sample delay with 0.000001 sample error, and calibrated ASIO artifact analysis recovered asio-ch2->asio-ch3 at 0.000 us with 0.812 confidence."}
{"ts": "2026-05-23T21:43:33+01:00", "type": "verification", "status": "ok", "note": "Reran the in-process ASIO ingest after native cleanup changes. BufferSmoke with config/mimir-runtime.asio.example.json ingested 12,204 sample-bearing blocks in two seconds at 192 kHz and retained 2,048 192-frame Float32 blocks for each asio-ch0..asio-ch3 buffer. This remains an ingest proof only because BufferSmoke does not emit calibration audio in that run."}

View file

@ -113,9 +113,10 @@ accepted_design:
evidence for frequency-response normalization. `MimirChirpBinCalibrationModel`
now preserves measured usable bands, expected-symbol versus observed-bin
confusion observations, timing residuals, delay hypotheses, phase summaries,
and an adaptive codebook plan. The active decoder can consume a persisted
model for learned response weighting, first-order group-delay correction,
and global delay-hypothesis seeding. Runtime chirp-bin emission consumes the
and an adaptive codebook plan per output/mic path. The active decoder can
consume a persisted model for learned response weighting, phase-coherence
weighting, first-order group-delay correction, and joint global
delay/bin-shift hypotheses. Runtime chirp-bin emission consumes the
same model's emission plan, so Mimir can render the measured reliable symbol
alphabet at a higher de Bruijn order instead of spending code on dead bins.
The analyzer keeps calibration profiles
@ -144,10 +145,13 @@ accepted_design:
pairwise timing against loopback in the stored artifact, but decoded mic
channels now leave calibration models; physical input 1 produced two reliable
symbols and a different strongest measured band set.
`config/mimir-runtime.asio.example.json` proves continuous ASIO runtime
ingest: a two-second BufferSmoke run at 192 kHz ingested 13,324 sample-bearing
blocks across `asio-ch0` through `asio-ch3`, all from the Focusrite ASIO
callback stream.
`native/asio_capture` and `MimirAsioStreamSource` are now the runtime
Scarlett path: Focusrite ASIO callbacks feed sample-bearing 192 kHz Float32
blocks into `Mimir.Runtime` in process. `config/mimir-runtime.asio.example.json`
proves continuous ASIO runtime ingest: a two-second BufferSmoke run at
192 kHz ingested more than 12,000 sample-bearing blocks across `asio-ch0`
through `asio-ch3` and retained 2,048 blocks per channel, all from the
Focusrite ASIO callback stream.
Acoustic robustness remains open.
The old arbitrary-chirplet matcher is a diagnostic/reference artifact only,
not the runtime hot path.
@ -164,9 +168,10 @@ accepted_design:
active_subgoals:
- Implement the first concrete Leap direct capture driver and measure cadence.
- Add remaining local camera drivers through `IMimirVideoCaptureDriver`.
- Add native audio capture workers for mic, loopback, and network audio feeds.
- Feed verified Focusrite ASIO input callbacks into `Mimir.Runtime`; WASAPI
remains a diagnostic witness.
- Use the verified in-process Focusrite ASIO source for local Scarlett mic and
loopback ingest; WASAPI remains a diagnostic witness.
- Add network audio feed producers that enter the same runtime clock/rebuffer
machinery instead of becoming independent authorities.
- Bind Fensalir UI to stream health, buffer depth, source timestamps,
settings, and output management.
- Keep PowerShell/FFmpeg/SRT as bridge utilities until direct program output
@ -250,9 +255,7 @@ current_status:
the Leap USB neighborhood does not reproduce the Focusrite-induced collapse.
next_action: >
Use chirp-bin calibration models from real microphones to tune the active
codebook/bands/weighting, then replace the ASIO JSON-block process bridge
with a native ABI callback source and add the synchronization
actuator.
codebook/bands/weighting, then add the synchronization actuator.
Current app testing proves Fensalir WASAPI output wakes Scarlett speaker
loopback, live mic buffers stay populated, runtime sync analysis now runs as
a bounded rotating service without UI/telemetry re-entry, and the continuous
@ -260,7 +263,7 @@ current_status:
invariant is code-valid triplets producing stable canonical anchors and clock
fits, so the actuator can use decoded time instead of filtered lag guesses.
Replace
the diagnostic JSON-line process bridge with direct native capture drivers that push buffer handles into
the remaining diagnostic JSON-line process bridges with direct native capture drivers that push buffer handles into
Mimir.Runtime. Mimir.BufferSmoke still proves live buffers and sync reports
for Scarlett loopback, Focusrite mic, Kiyo mic, and Kiyo Pro mic when
calibration chirplets are playing. Treat PS3 Eye audio as

View file

@ -6,11 +6,11 @@ No active scratch subgoal.
## Notes
- Viable stream app contract promoted to docs/map/evidence: Aquarium hosts one
- Viable stream app contract promoted to docs/map/evidence: Fensalir hosts one
five-second in-memory runtime, local/networked feeds append as producers, and
OBS receives synchronized program outputs.
- Mimir app scaffold now lives in `Mimir.slnx` with `src/Mimir.App` and
`src/Mimir.Runtime`; Aquarium Engine remains an external project reference.
`src/Mimir.Runtime`; Fensalir remains an external project reference.
- Synchronization ownership started moving into `Mimir.Runtime`: hub, stream
descriptors, per-stream rolling buffers, settings, and source adapter seam are
in C# now; concrete capture adapters are next.