Add chirp-bin calibration profiles

This commit is contained in:
Meta Vens 2026-05-23 20:03:48 +01:00
commit 3a7410d510
10 changed files with 195 additions and 38 deletions

View file

@ -75,7 +75,12 @@ the de Bruijn triplet trellis. A constrained local waveform correlation around
the decoded anchor delay provides the final fractional offset. The decoder keeps
the full bin-energy surface for each classified chirp and aggregates it into
per-band response evidence, so the same stream can calibrate timing and the
speaker/room/mic transfer function. The synthetic invariant is
speaker/room/mic transfer function. `MimirChirpBinCalibrationProfile` is the
first named surface for that evidence: it summarizes frames, anchors, clock
confidence, anchor error, usable bands, and strongest measured bins per source.
The profile is kept even when a timing report is rejected, because a failed
sync window can still teach us which bands survived the speaker/room/mic path.
The synthetic invariant is
`dotnet run --project .\src\Mimir.BufferSmoke\Mimir.BufferSmoke.csproj -- --chirp-bin-self-test`;
it renders the chirp-bin timeline, decodes by dechirp plus Goertzel bins, and
requires code-valid triplet anchors plus a stable source clock. `chirp-only`
@ -168,12 +173,13 @@ produce at least three matched canonical anchors, it has not decoded timing for
that window. Reports carry rounded integer delay, fractional delay, and the
count/confidence of timeline-symbol anchors used to derive the report.
The same timeline also starts the frequency-response path. Each active report
includes per-band dechirped energy for the chirp-bin bank. That is not a
finished room/mic normalizer yet, but it is the live surface that will become
response-curve estimation: loopback carries what was emitted, each mic carries
what survived speaker, air, room, and capsule, and the ratio over the continuous
timeline becomes gain/phase correction evidence.
The same timeline also starts the frequency-response path. Each active chirp-bin
decode emits a calibration profile for the source, whether or not that source
earns a timing report. That is not a finished room/mic normalizer yet, but it is
the live surface that will become response-curve estimation: loopback carries
what was emitted, each mic carries what survived speaker, air, room, and
capsule, and the ratio over the continuous timeline becomes gain/phase
correction evidence.
`MimirAudioSynchronizationStateTracker` turns continuous observations into
state. It confidence-gates reports, smooths fractional delay per source, and
@ -206,7 +212,7 @@ flowchart TD
H --> L["triplet timeline anchors"]
L --> M["per-source clock fit"]
M --> N["delay + SRO"]
G --> I["per-band response estimates"]
G --> I["per-source calibration profile"]
N --> J["fractional delay / resampler actuator"]
I --> K["frequency response normalization"]
```
@ -223,6 +229,12 @@ Continuous chirplet evidence gives both the current delay and the drift/SRO by
watching delay change over time. Per-band energy over the same stream gives the
normalization curve. The important constraint is that all three measurements
must be tied to the same emitted timeline, not three separately invented probes.
The current ASIO artifact proves why this matters: Scarlett loopback channel 2
and loopback channel 3 both decode 12 anchors with 0.996 clock confidence, while
physical input 1 decodes a noisier independent clock and a different strongest
band set but still fails pairwise timing because no canonical events overlap
the loopback path. That is not a blank failure anymore; it is calibration data
for the next codebook/weighting cut.
The symbol layer is intentionally redundant. It does not rely on one fixed
frequency shelf: timing gaps, chirp duration, start band, and glide shape all

View file

@ -102,8 +102,11 @@ a named invariant that the native runtime cannot protect yet.
with constrained local waveform correlation around the decoded active delay.
Each classified chirp carries the full bin-energy response surface, and stream
decodes aggregate that into per-band calibration evidence for frequency
response normalization. Reports/states expose delay in microseconds as well
as fractional samples.
response normalization. `MimirChirpBinCalibrationProfile` preserves that
evidence per source, including frames, anchors, clock confidence, anchor
error, usable band count, and strongest measured bins, even when the source
does not produce a timing report. Reports/states expose delay in microseconds
as well as fractional samples.
Hybrid emits this as low-gain half-second bursts every two seconds only while
passive confidence is weak.
Use `--chirp-bin-self-test` to prove the codebook/decoder and
@ -113,8 +116,12 @@ a named invariant that the native runtime cannot protect yet.
- `Mimir.BufferSmoke --render-chirp-bin-f32` and `--analyze-asio-f32` provide
the current active Scarlett artifact proof. A 192 kHz chirp-bin run decoded
Focusrite `Loopback 1 -> Loopback 2` at `0.000 us` with 12 matched anchors and
0.999 confidence. Physical inputs did not produce matched anchors in that run,
so acoustic robustness remains separate from the clean loopback timing proof.
0.999 confidence. The ASIO analyzer now prints chirp-bin calibration profiles
for every decoded source; physical input 1 still failed pairwise timing in
the stored artifact, but it produced 14 frames, 12 anchors, 0.865 clock
confidence, and a different strongest band set. Acoustic robustness remains
separate from the clean loopback timing proof, but failed timing windows now
leave usable response evidence instead of silence.
A standalone 192 kHz synthetic receiver test recovers a 500 ms delayed audio
stream to below printed microsecond precision using only the chirp-bin
codebook and schedule state.
@ -153,9 +160,9 @@ a named invariant that the native runtime cannot protect yet.
workers for local mic, loopback, and network audio feeds. Use the verified
Focusrite ASIO path for Scarlett capture; WASAPI remains a diagnostic
witness, not the Scarlett hot path.
4. Prove the active chirp-bin path through real microphones, not only Scarlett
loopback; tune acoustic bands/gain/code spacing from measured mic response
without weakening the standalone codebook/schedule receiver invariant.
4. Use chirp-bin calibration profiles from real microphones to tune acoustic
bands, gain, and code spacing without weakening the standalone
codebook/schedule receiver invariant.
5. Add the synchronization actuator: drive a variable-rate resampler and
fractional delay line per non-reference stream from the smoothed
`MimirAudioSynchronizationState`. First, prove the constrained chirplet

View file

@ -111,8 +111,11 @@ scheduled-waveform correlation for standalone source-offset refinement. Pairwise
sync compares matched anchors first, then only accepts independent clock-fit
offsets inside the live latency horizon so period aliases do not become absurd
reports. Each classified chirp carries the full dechirped bin-energy surface,
and decodes aggregate that into per-band response evidence for future
frequency-response normalization. Reports/states expose `delayUs` next to
and decodes aggregate that into a `MimirChirpBinCalibrationProfile` per source
with frames, anchors, clock confidence, anchor error, usable band count, and
strongest bins. The analyzer keeps those profiles even when no timing report is
accepted, so a physical mic path can fail synchronization while still teaching
the next decoder which bands survived. Reports/states expose `delayUs` next to
fractional sample delay.
`Mimir.BufferSmoke
--chirp-only-sync-self-test --sample-rate 192000` and `--hybrid-sync-self-test
@ -157,11 +160,12 @@ Focusrite outputs while capturing every ASIO input as raw interleaved Float32.
`Mimir.BufferSmoke --analyze-asio-f32` feeds those captured channels into the
same runtime active analyzer. A real 192 kHz chirp-bin Scarlett artifact decoded
`Loopback 1 -> Loopback 2` at exactly `0.000 us` with 12 matched anchors and
0.999 confidence. Physical input 2 independently decoded a coherent clock in
that artifact, but its event path did not overlap loopback and its raw
clock-offset difference was a period alias far outside the live latency horizon,
so Mimir correctly rejected it as insufficient anchors. Acoustic robustness is
now the open problem, not clean loopback timing or standalone decoder shape.
0.999 confidence. The same analyzer now prints calibration profiles for
decoded sources. Physical input 1 still fails pairwise timing in the stored
artifact, but it leaves a concrete response profile: 14 frames, 12 anchors,
0.865 clock confidence, and strongest bins around 4525, 4075, and 7225 Hz.
Acoustic robustness is now the open problem, not clean loopback timing,
standalone decoder shape, or basic response evidence.
## Visual Fusion

View file

@ -90,8 +90,8 @@ Get-Content .\state\evidence.jsonl -Tail 8
correlation for final fractional delay. It also supports standalone source
offset recovery from schedule/codebook state, which is the Raven/phone shape.
Each classified chirp carries the full dechirped bin-energy surface; sync
reports expose those bands so the same stream can become frequency-response
calibration evidence.
reports expose those bands, and `MimirChirpBinCalibrationProfile` preserves
per-source calibration evidence even when timing is rejected.
`Mimir.BufferSmoke
--chirp-only-sync-self-test --sample-rate 192000` and
`--hybrid-sync-self-test --sample-rate 192000` both recover a 1269.5-sample
@ -119,16 +119,20 @@ Get-Content .\state\evidence.jsonl -Tail 8
comparison and is now diagnostic only. A real Scarlett chirp-bin artifact run
at 192 kHz decoded `Loopback 1 -> Loopback 2` at `0.000 us` delay with 12
matched anchors and 0.999 confidence in the normal active analyzer. Physical
input 2 independently decoded a coherent clock, but its clock-offset
difference was a period alias outside the live latency horizon and is rejected
as insufficient anchors. Acoustic robustness is still open.
inputs still do not earn accepted timing against loopback, but the active
analyzer now prints calibration profiles for decoded channels. In the stored
artifact, physical input 1 produced 14 frames, 12 anchors, 0.865 clock
confidence, and a different strongest-bin profile while failing pairwise
timing. Acoustic robustness is still open, but failed timing now leaves
response evidence.
- Raven also has a 192 kHz loopback-capable Scarlett for co-streamer/game timing
evidence. Do not move the heavy soundfield or sensor-fusion workload there.
## Immediate Re-entry Instruction
Prove the active chirp-bin path through real microphones, then feed Scarlett
ASIO callbacks into `Mimir.Runtime` and turn decoded clock fits into the
SRO/fractional-delay actuator. Keep loopback as timing authority. Do not call
synchronization analysis from UI/telemetry readouts. Do not restore deleted
script infrastructure because a stale doc once missed it.
Use the chirp-bin calibration profiles from real microphones to tune the active
codebook/bands/weighting, then feed Scarlett ASIO callbacks into `Mimir.Runtime`
and turn decoded clock fits into the SRO/fractional-delay actuator. Keep
loopback as timing authority. Do not call synchronization analysis from
UI/telemetry readouts. Do not restore deleted script infrastructure because a
stale doc once missed it.

View file

@ -133,6 +133,11 @@ foreach (var trace in hub.AudioSynchronizationDecodeTraces.OrderBy(trace => trac
$"sync-decode {trace.ReferenceSourceId}->{trace.SourceId}: status={trace.Status} compared={trace.ComparedSamples} rate={trace.SampleRate} refFrames={trace.ReferenceFrames} refAnchors={trace.ReferenceAnchors} refClock={trace.ReferenceClockConfidence:0.000} refEnergy={trace.ReferenceBestEnergy:0.000} candFrames={trace.CandidateFrames} candAnchors={trace.CandidateAnchors} candClock={trace.CandidateClockConfidence:0.000} candEnergy={trace.CandidateBestEnergy:0.000} matched={trace.MatchedEvents} confidence={trace.Confidence:0.000}");
}
foreach (var profile in hub.AudioChirpBinCalibrationProfiles)
{
Console.WriteLine(DescribeCalibrationProfile("sync-calibration", profile));
}
if (requireSamples && emptyBuffers.Count > 0)
{
Console.Error.WriteLine($"empty buffers: {string.Join(", ", emptyBuffers)}");
@ -177,6 +182,18 @@ static string DescribeBands(IReadOnlyList<MimirChirpletBandResponse> bands)
: string.Join(",", bands.Select(band => $"{band.CenterHz:0}Hz:{band.Energy:0.000}"));
}
static string DescribeCalibrationProfile(string prefix, MimirChirpBinCalibrationProfile profile)
{
var top = profile.Bands.Count == 0
? "none"
: string.Join(",", profile.StrongestBands(8).Select(band =>
$"{band.CenterHz:0}Hz:mean={band.MeanEnergy:0.000}:rel={band.RelativeGain:0.000}:n={band.ObservationCount}"));
var mae = double.IsFinite(profile.MeanAnchorErrorSamples)
? profile.MeanAnchorErrorSamples.ToString("0.000")
: "none";
return $"{prefix} {profile.SourceId}: sampleRate={profile.SampleRate} frames={profile.FrameCount} anchors={profile.AnchorCount} clock={profile.ClockConfidence:0.000} maeSamples={mae} usableBins={profile.UsableBandCount}/{profile.Bands.Count} top={top}";
}
static int RunChirpletSelfTest()
{
var timeline = MimirChirpletTimeline.Default;
@ -525,6 +542,11 @@ static int AnalyzeAsioFloat32(string inputPath, int sampleRate, int channels, in
$"asio-f32-trace {trace.ReferenceSourceId}->{trace.SourceId}: status={trace.Status} compared={trace.ComparedSamples} refFrames={trace.ReferenceFrames} refAnchors={trace.ReferenceAnchors} refClock={trace.ReferenceClockConfidence:0.000} candFrames={trace.CandidateFrames} candAnchors={trace.CandidateAnchors} candClock={trace.CandidateClockConfidence:0.000} matched={trace.MatchedEvents} confidence={trace.Confidence:0.000}");
}
foreach (var profile in analyzer.LastCalibrationProfiles)
{
Console.WriteLine(DescribeCalibrationProfile("asio-f32-calibration", profile));
}
foreach (var report in reports.OrderBy(report => report.SourceId, StringComparer.Ordinal))
{
Console.WriteLine(

View file

@ -11,10 +11,14 @@ public sealed class MimirAudioSynchronizationAnalyzer
private const double MaxPairClockFitDelaySeconds = 10.0;
private const double PassiveReportConfidence = 0.08;
private readonly List<MimirAudioSynchronizationDecodeTrace> lastDecodeTraces = [];
private readonly Dictionary<string, MimirChirpBinCalibrationProfile> lastCalibrationProfiles = new(StringComparer.Ordinal);
private readonly MimirPassiveAudioSynchronizationEstimator passiveEstimator = new();
public IReadOnlyList<MimirAudioSynchronizationDecodeTrace> LastDecodeTraces => lastDecodeTraces;
public IReadOnlyList<MimirChirpBinCalibrationProfile> LastCalibrationProfiles =>
lastCalibrationProfiles.Values.OrderBy(profile => profile.SourceId, StringComparer.Ordinal).ToArray();
public IReadOnlyList<MimirAudioSynchronizationReport> Analyze(
IEnumerable<MimirRollingStreamBuffer> buffers,
string referenceSourceId,
@ -46,6 +50,7 @@ public sealed class MimirAudioSynchronizationAnalyzer
var reports = new List<MimirAudioSynchronizationReport>();
lastDecodeTraces.Clear();
lastCalibrationProfiles.Clear();
foreach (var buffer in audioBuffers)
{
if (ReferenceEquals(buffer, reference))
@ -168,6 +173,12 @@ public sealed class MimirAudioSynchronizationAnalyzer
var candidateDecode = useChirpBinTimeline
? MimirChirpBinTimeline.Default.DecodeStreamWindow(candidateWindow, activeReferenceBlock.SampleRate)
: MimirChirpletTimeline.Default.DecodeStreamWindow(candidateWindow, activeReferenceBlock.SampleRate);
if (useChirpBinTimeline)
{
StoreCalibrationProfile(reference.Descriptor.SourceId, activeReferenceBlock.SampleRate, comparedReferenceDecode);
StoreCalibrationProfile(buffer.Descriptor.SourceId, activeReferenceBlock.SampleRate, candidateDecode);
}
var deterministicFit = EstimateDelayFromDecodedTimeline(comparedReferenceDecode, candidateDecode);
lastDecodeTraces.Add(new MimirAudioSynchronizationDecodeTrace(
reference.Descriptor.SourceId,
@ -217,6 +228,14 @@ public sealed class MimirAudioSynchronizationAnalyzer
return reports;
}
private void StoreCalibrationProfile(
string sourceId,
int sampleRate,
MimirChirpletStreamDecode decode)
{
lastCalibrationProfiles[sourceId] = MimirChirpBinCalibrationProfile.FromDecode(sourceId, sampleRate, decode);
}
private static int MinWindowSamples(int sampleRate, double seconds)
{
return (int)Math.Ceiling(sampleRate * seconds);

View file

@ -0,0 +1,77 @@
namespace Mimir.Runtime.Synchronization;
public sealed record MimirChirpBinCalibrationBand(
double CenterHz,
double MeanEnergy,
double PeakEnergy,
double RelativeGain,
int ObservationCount,
bool Usable);
public sealed record MimirChirpBinCalibrationProfile(
string SourceId,
int SampleRate,
int FrameCount,
int AnchorCount,
double ClockConfidence,
double MeanAnchorErrorSamples,
IReadOnlyList<MimirChirpBinCalibrationBand> Bands)
{
private const double UsableRelativeGain = 0.20;
private const double UsableMeanEnergy = 0.015;
public int UsableBandCount => Bands.Count(band => band.Usable);
public IReadOnlyList<MimirChirpBinCalibrationBand> StrongestBands(int count) =>
Bands
.OrderByDescending(band => band.MeanEnergy)
.ThenBy(band => band.CenterHz)
.Take(Math.Max(0, count))
.ToArray();
public static MimirChirpBinCalibrationProfile FromDecode(
string sourceId,
int sampleRate,
MimirChirpletStreamDecode decode)
{
var grouped = decode.Frames
.SelectMany(frame => frame.BestCandidate.BandResponses ?? [])
.GroupBy(response => response.CenterHz)
.Select(group => new
{
CenterHz = group.Key,
MeanEnergy = group.Average(response => response.Energy),
PeakEnergy = group.Max(response => response.Energy),
ObservationCount = group.Count(),
})
.OrderBy(band => band.CenterHz)
.ToArray();
var strongestMean = grouped.Length == 0
? 0.0
: grouped.Max(band => band.MeanEnergy);
var bands = grouped
.Select(band =>
{
var relativeGain = strongestMean <= 1.0e-12
? 0.0
: band.MeanEnergy / strongestMean;
return new MimirChirpBinCalibrationBand(
band.CenterHz,
band.MeanEnergy,
band.PeakEnergy,
relativeGain,
band.ObservationCount,
relativeGain >= UsableRelativeGain && band.MeanEnergy >= UsableMeanEnergy);
})
.ToArray();
return new MimirChirpBinCalibrationProfile(
sourceId,
sampleRate,
decode.Frames.Count,
decode.Anchors.Count,
decode.ClockFit?.Confidence ?? 0.0,
decode.ClockFit?.MeanAbsoluteErrorSamples ?? double.NaN,
bands);
}
}

View file

@ -36,6 +36,9 @@ public sealed class MimirSynchronizationHub : IDisposable
public IReadOnlyList<MimirAudioSynchronizationDecodeTrace> AudioSynchronizationDecodeTraces =>
audioSynchronization.LastDecodeTraces;
public IReadOnlyList<MimirChirpBinCalibrationProfile> AudioChirpBinCalibrationProfiles =>
audioSynchronization.LastCalibrationProfiles;
public void AddSource(IMimirStreamSource source)
{
ObjectDisposedException.ThrowIf(disposed, this);

View file

@ -70,3 +70,4 @@
{"ts": "2026-05-23T19:22:00+01:00", "type": "implementation", "status": "ok", "note": "Made the active audio sync path use the receiver-cheap chirp-bin decoder for both chirp-only and hybrid modes. The old arbitrary chirplet matcher remains diagnostic/reference only. The chirp-bin decoder now uses bounded energy/onset proposals, dechirp plus fixed Goertzel bins, explicit time/frequency ambiguity candidates, and constrained local waveform correlation for final fractional delay. Synthetic 192 kHz chirp-only and hybrid self-tests both recover a 1269.5-sample delay with printed 0.000 us error; a real Scarlett ASIO chirp-bin artifact decodes Loopback 1 -> Loopback 2 at 0.000 us with 12 anchors and 0.999 confidence, while physical mic channels still need acoustic robustness work."}
{"ts": "2026-05-23T19:40:50+01:00", "type": "implementation", "status": "ok", "note": "Extended active chirp-bin decoding toward the Perfect Machine receiver shape: standalone DecodeStreamWindow now refines source offset against the scheduled waveform so a listener with only codebook/schedule state can place delayed audio on canonical time. Synthetic 192 kHz standalone tests recover 1269.5 samples exactly, 96000 samples / 500 ms to below printed microsecond precision, and 13789.75 samples within 0.021 us. Pairwise Mimir sync now rejects independent clock-fit differences outside a 10s live horizon so coherent period aliases do not become absurd delays. Fensalir project references were updated from removed Aquarium.LocalCast to Aquarium.Fensalir."}
{"ts": "2026-05-23T19:47:34+01:00", "type": "implementation", "status": "ok", "note": "Active chirp-bin decoding now preserves calibration evidence as well as timing: each classified chirp carries the full dechirped 32-bin energy surface, stream decodes aggregate those bins into BandResponses, and sync reports print/expose those per-band response estimates. This makes the chirp stream a timing witness and a frequency-response calibration surface. Verified dotnet build, 192 kHz chirp-only self-test, 500 ms standalone self-test, and Scarlett ASIO loopback artifact with response bands."}
{"ts": "2026-05-23T20:03:21+01:00", "type": "implementation", "status": "ok", "note": "Added chirp-bin calibration profiles as the live meatspace response surface. MimirChirpBinCalibrationProfile summarizes per-source frames, anchors, clock confidence, anchor error, usable bins, and strongest measured bands from active chirp-bin decodes. MimirAudioSynchronizationAnalyzer keeps profiles even when timing reports are rejected, and BufferSmoke now prints them for runtime and ASIO artifact analysis. Verified build, 192 kHz chirp-only self-test, 500 ms standalone receiver self-test, Scarlett loopback sync, and physical input calibration evidence from the stored ASIO artifact."}

View file

@ -110,7 +110,12 @@ accepted_design:
final active delay with constrained local waveform correlation around the
decoded anchor delay. Each classified chirp carries the full dechirped
bin-energy surface, and decodes aggregate that into per-band response
evidence for frequency-response normalization. Reports and sync states expose
evidence for frequency-response normalization. `MimirChirpBinCalibrationProfile`
now preserves that evidence per source with frame count, anchor count, clock
confidence, anchor error, usable band count, and strongest measured bins.
The analyzer keeps calibration profiles even when a source fails pairwise
timing, so physical mic failures can still guide acoustic codebook and
response-weighting work. Reports and sync states expose
delay in microseconds as well as fractional samples. Hybrid emits it as low-gain half-second
bursts every two seconds while passive confidence is weak, never as catch-up
backlog. The analyzer no longer applies the passive two-second window floor
@ -127,9 +132,11 @@ accepted_design:
`native/probes/asio_audio_cadence --play-f32-mono --record-f32-interleaved`
now provides a real Scarlett ASIO artifact proof for active mode. At 192 kHz,
Loopback 1 -> Loopback 2 decoded at 0.000 us with 12 anchors and 0.999
confidence. Physical input 2 independently decoded a coherent clock, but its
clock-offset difference was a period alias outside the live latency horizon
and is rejected as insufficient anchors. Acoustic robustness remains open.
confidence. Physical inputs still fail accepted pairwise timing against
loopback in the stored artifact, but decoded mic channels now leave
calibration profiles; physical input 1 produced 14 frames, 12 anchors, 0.865
clock confidence, and strongest measured bins around 4525, 4075, and 7225 Hz.
Acoustic robustness remains open.
The old arbitrary-chirplet matcher is a diagnostic/reference artifact only,
not the runtime hot path.
- `IMimirVideoCaptureDriver` and `MimirVideoCaptureDriverSource` are the live
@ -230,8 +237,9 @@ current_status:
Kiyo 30.18 fps, PS3 Eye 0 59.66 fps, and PS3 Eye 1 59.95 fps. A mouse in
the Leap USB neighborhood does not reproduce the Focusrite-induced collapse.
next_action: >
Prove the active chirp-bin path through real microphones, then feed Scarlett
ASIO callbacks directly into Mimir.Runtime and add the synchronization
Use chirp-bin calibration profiles from real microphones to tune the active
codebook/bands/weighting, then feed Scarlett ASIO callbacks directly into
Mimir.Runtime and add the synchronization
actuator.
Current app testing proves Fensalir WASAPI output wakes Scarlett speaker
loopback, live mic buffers stay populated, runtime sync analysis now runs as