Start chirplet timeline decoder

This commit is contained in:
Meta Vens 2026-05-22 14:53:26 +01:00
commit ea09212d49
10 changed files with 415 additions and 52 deletions

View file

@ -61,13 +61,24 @@ the following inter-chirp gap all carry code. The point is not ornament. The
timing code is carried by both frequency and rhythm, so it behaves more like a
quiet birdsong texture than a repeated sweep.
`MimirAudioSynchronizationAnalyzer` resamples candidate mic windows into the
loopback sample-rate timeline, uses cheap coarse chirplet energy for acquisition,
then lets the symbol decoder refine delay from matched event indices. The lag
search is wide enough for ordinary remote/network delay, not just local
speaker-to-mic delay. Reports carry rounded integer delay, fractional delay from
parabolic peak interpolation, and the count/confidence of timeline-symbol events
used to refine the report.
The intended decoder is a constrained chirplet transform, not a generic
time-frequency explorer and not an outlier filter around bad guesses. Mimir owns
the emitter, so the receiver projects each mic stream against the known chirplet
dictionary, produces symbol likelihood events, and decodes every valid
consecutive triplet through the de Bruijn map. A decoded triplet is a canonical
timeline anchor: observed sample offset `S` corresponds to emitted event time
`T`. A stream of anchors fits the source clock directly:
```text
observed_sample = source_offset + canonical_seconds * effective_sample_rate
```
Delay and sampling-rate offset are derived by comparing the loopback clock fit
to each mic clock fit over common canonical time. Wide cross-correlation is only
a diagnostic fallback while the transform decoder is being cut over. The state
tracker is not allowed to launder invalid codewords into plausible timing.
Reports carry rounded integer delay, fractional delay, and the count/confidence
of timeline-symbol anchors used to derive the report.
`MimirSynchronizationHub.BuildAlignedAudioFrame` returns a provisional aligned
mono frame: loopback is always channel zero, and other channels enter only when
@ -111,10 +122,12 @@ flowchart TD
D --> F["mic rolling buffers"]
E --> G["matched chirplet traces"]
F --> G
G --> H["delay observations"]
H --> L["smoothed sync state + SRO"]
G --> H["symbol likelihood events"]
H --> L["triplet timeline anchors"]
L --> M["per-source clock fit"]
M --> N["delay + SRO"]
G --> I["per-band response estimates"]
L --> J["fractional delay / resampler actuator"]
N --> J["fractional delay / resampler actuator"]
I --> K["frequency response normalization"]
```

View file

@ -49,11 +49,15 @@ a named invariant that the native runtime cannot protect yet.
detected symbols identify a timeline event inside the current operating
horizon. Symbol identity is carried by start band, glide shape, duration, and
following inter-chirp gap.
- `MimirChirpletStreamDecoder` is the first constrained chirplet-transform
receiver. It owns a bounded PCM window, emits symbol observations, decodes
code-valid triplet anchors onto the canonical timeline, and fits a per-source
sample clock from those anchors.
- `MimirAudioSynchronizationAnalyzer` ports the first live sync measurement:
sample-bearing audio blocks are resampled into the Scarlett loopback timeline
and compared by chirplet-energy delay estimation with fractional peak
interpolation. When timeline symbols are decoded, matched event indices refine
the delay estimate and are exposed in telemetry.
sample-bearing audio blocks are resampled into the Scarlett loopback timeline.
The analyzer now prefers decoded triplet timeline anchors and derives delay
from matched canonical events. The older chirplet-energy lag search remains a
diagnostic fallback while the decoder is being cut over.
- `MimirSynchronizationHub.BuildAlignedAudioFrame` exposes the first
provisional aligned mono frame for loopback-referenced audio channels that
clear the confidence gate.
@ -103,10 +107,9 @@ a named invariant that the native runtime cannot protect yet.
fractional delay line per non-reference stream from the smoothed
`MimirAudioSynchronizationState`. The current aligned frame is integer-delay
only and should not pretend to be the final DSP path.
First, harden the online state filter so the continuous chirplet lock stops
producing jumpy delay estimates. The runtime sync cadence is now bounded and
survives longer real app runs, but the state filter still accepts outliers
too readily.
First, finish the constrained chirplet-transform decoder so every correctly
heard triplet becomes a deterministic timeline anchor and the state tracker no
longer has to launder ambiguous lag guesses.
5. Bind Aquarium UI to the synchronization hub so buffer depth, stream cadence,
source timestamps, and output settings are visible and adjustable.
6. Move GPU feature extraction, fusion, material fitting, render budgeting, and

View file

@ -99,16 +99,21 @@ current operating horizon. Mimir continuously queues that timeline through
Aquarium audio, uses coarse chirplet energy for acquisition, refines delay from
matched event indices when the symbol decoder succeeds, and can build a
provisional aligned mono frame for channels that clear the confidence gate.
Reports now carry fractional delay and per-band matched energy. The hub also
owns cached reports plus smoothed per-source sync state with delay-slope/SRO in
ppm. `MimirRuntime` runs analysis online as a bounded rotating service and can
print live telemetry with `MIMIR_SYNC_TELEMETRY_SECONDS`. UI and telemetry are
passive readers of cached sync state; they must not invoke the analyzer. Current
app testing proves loopback wakeup, live mic buffers, and confident online sync
states, but delay estimates are still jumpy. Camera mics are spatial/context
witnesses; Focusrite devices are dialogue anchors. The current aligned frame is
integer-delay only; fractional delay and the hot resampler still belong in
Faust/native DSP.
Reports now carry fractional delay and per-band matched energy. The first
`MimirChirpletStreamDecoder` exists as the constrained chirplet-transform
receiver: it turns a bounded PCM window into symbol observations, every
code-valid triplet anchor it can prove, and an affine source clock fit. The
analyzer prefers matched canonical anchors before falling back to the older
chirplet-energy lag search. The hub also owns cached reports plus smoothed
per-source sync state with delay-slope/SRO in ppm. `MimirRuntime` runs analysis
online as a bounded rotating service and can print live telemetry with
`MIMIR_SYNC_TELEMETRY_SECONDS`. UI and telemetry are passive readers of cached
sync state; they must not invoke the analyzer. Current app testing proves
loopback wakeup, live mic buffers, and confident online sync states, but the
decoder is not finished until arbitrary valid triplets produce stable anchors in
real mic streams. Camera mics are spatial/context witnesses; Focusrite devices
are dialogue anchors. The current aligned frame is integer-delay only;
fractional delay and the hot resampler still belong in Faust/native DSP.
## Visual Fusion

View file

@ -71,19 +71,22 @@ Get-Content .\state\evidence.jsonl -Tail 8
timeline owns Aquarium PCM segment rendering, coarse acquisition,
symbol/event decoding, and per-band response hooks. Sync reports now include
fractional delay and per-band matched energy. `MimirAudioSynchronizationState`
now tracks smoothed delay and delay-slope/SRO ppm. `MimirRuntime` runs sync
analysis as a bounded rotating service and caches reports/states for UI and
telemetry; readouts must stay passive. Actual Mimir.App testing proves
Aquarium audio wakes Scarlett loopback, keeps mic buffers live, and produces
confident online sync states, but delay estimates are still jumpy. PS3 Eye
audio is
now tracks smoothed delay and delay-slope/SRO ppm. `MimirChirpletStreamDecoder`
is the first constrained chirplet-transform receiver: it emits symbol
observations, code-valid triplet anchors, and a source clock fit from a
bounded PCM window. `MimirRuntime` runs sync analysis as a bounded rotating
service and caches reports/states for UI and telemetry; readouts must stay
passive. Actual Mimir.App testing proves Aquarium audio wakes Scarlett
loopback, keeps mic buffers live, and produces confident online sync states,
but the decoder still needs stronger symbol likelihoods/codebook distance
before arbitrary real triplets can replace the lag fallback. PS3 Eye audio is
enumeration/runtime-fragile: one run saw both mic buffers empty while both
cameras were live, then a replug made both PS3 Eye mic buffers emit
480-frame WASAPI blocks again.
## Immediate Re-entry Instruction
Harden the online sync state filter, then turn those delay reports into a
smoothed SRO/fractional-delay actuator. Keep loopback as timing authority. Do
Finish the constrained chirplet-transform decoder, then turn decoded clock fits
into the SRO/fractional-delay actuator. Keep loopback as timing authority. Do
not call synchronization analysis from UI/telemetry readouts. Do not restore
deleted script infrastructure because a stale doc once missed it.

View file

@ -38,8 +38,6 @@ public sealed class MimirAudioSynchronizationAnalyzer
return [];
}
MimirChirpletTimelinePlacement? referencePlacement = null;
var reports = new List<MimirAudioSynchronizationReport>();
foreach (var buffer in audioBuffers)
{
@ -75,6 +73,31 @@ public sealed class MimirAudioSynchronizationAnalyzer
var referenceWindow = referenceSamples.AsSpan(^compared..);
var candidateWindow = candidateSamples.AsSpan(^compared..);
var referenceDecode = MimirChirpletTimeline.Default.DecodeStreamWindow(referenceWindow, referenceBlock.SampleRate);
var candidateDecode = MimirChirpletTimeline.Default.DecodeStreamWindow(candidateWindow, referenceBlock.SampleRate);
var deterministicFit = EstimateDelayFromDecodedTimeline(referenceDecode, candidateDecode);
if (deterministicFit.MatchedEvents >= 1)
{
var decodedDelaySamples = deterministicFit.DelaySamples;
var decodedBandResponses = MimirChirpletTimeline.Default.EstimateBandResponse(candidateWindow, referenceBlock.SampleRate);
reports.Add(new MimirAudioSynchronizationReport(
reference.Descriptor.SourceId,
buffer.Descriptor.SourceId,
referenceBlock.SampleRate,
(int)Math.Round(decodedDelaySamples),
decodedDelaySamples,
decodedDelaySamples * 1000.0 / referenceBlock.SampleRate,
deterministicFit.Confidence,
decodedBandResponses,
commonEndNs,
compared,
reference.Latest?.Sequence ?? 0,
buffer.Latest?.Sequence ?? 0,
deterministicFit.MatchedEvents,
deterministicFit.Confidence));
continue;
}
var referenceWindowStartSample = referenceSamples.Length - compared;
var referenceSync = MimirChirpletTimeline.Default.BuildTimelineEnergyTrace(referenceWindow, referenceBlock.SampleRate, ChirpletHopSamples);
var candidateSync = MimirChirpletTimeline.Default.BuildTimelineEnergyTrace(candidateWindow, referenceBlock.SampleRate, ChirpletHopSamples);
@ -90,7 +113,7 @@ public sealed class MimirAudioSynchronizationAnalyzer
var timelineConfidence = 0.0;
if (confidence >= 0.03 && double.IsFinite(approximateTimelineSeconds))
{
referencePlacement ??= MimirChirpletTimeline.Default.DecodeReferenceWindow(
var referencePlacement = MimirChirpletTimeline.Default.DecodeReferenceWindow(
referenceSamples,
referenceBlock.SampleRate,
approximateTimelineSeconds);
@ -140,6 +163,61 @@ public sealed class MimirAudioSynchronizationAnalyzer
return reports;
}
private static (double DelaySamples, double Confidence, int MatchedEvents) EstimateDelayFromDecodedTimeline(
MimirChirpletStreamDecode reference,
MimirChirpletStreamDecode candidate)
{
if (reference.Anchors.Count == 0 ||
candidate.Anchors.Count == 0 ||
reference.ClockFit == null ||
candidate.ClockFit == null)
{
return (0.0, 0.0, 0);
}
var candidateByEvent = candidate.Anchors.ToDictionary(anchor => anchor.EventIndex);
var matched = new List<(double Delay, double Weight)>();
foreach (var referenceAnchor in reference.Anchors)
{
if (!candidateByEvent.TryGetValue(referenceAnchor.EventIndex, out var candidateAnchor))
{
continue;
}
matched.Add((
candidateAnchor.SampleOffset - referenceAnchor.SampleOffset,
Math.Sqrt(referenceAnchor.Confidence * candidateAnchor.Confidence)));
}
if (matched.Count >= 1)
{
var totalWeight = matched.Sum(pair => Math.Max(1.0e-6, pair.Weight));
var delay = matched.Sum(pair => pair.Delay * Math.Max(1.0e-6, pair.Weight)) / totalWeight;
var error = matched.Sum(pair => Math.Abs(pair.Delay - delay) * Math.Max(1.0e-6, pair.Weight)) / totalWeight;
var residualConfidence = 1.0 / (1.0 + error / 32.0);
var countConfidence = Math.Clamp(matched.Count / 12.0, 0.0, 1.0);
var energyConfidence = Math.Clamp(totalWeight / matched.Count, 0.0, 1.0);
return (delay, residualConfidence * 0.50 + countConfidence * 0.25 + energyConfidence * 0.25, matched.Count);
}
var referenceFirst = reference.Anchors.Min(anchor => anchor.TimelineSeconds);
var referenceLast = reference.Anchors.Max(anchor => anchor.TimelineSeconds);
var candidateFirst = candidate.Anchors.Min(anchor => anchor.TimelineSeconds);
var candidateLast = candidate.Anchors.Max(anchor => anchor.TimelineSeconds);
var start = Math.Max(referenceFirst, candidateFirst);
var end = Math.Min(referenceLast, candidateLast);
if (end <= start)
{
return (0.0, 0.0, matched.Count);
}
var timelineSeconds = (start + end) * 0.5;
var clockDelay = candidate.ClockFit.SampleForTimelineSeconds(timelineSeconds) -
reference.ClockFit.SampleForTimelineSeconds(timelineSeconds);
var confidence = Math.Sqrt(reference.ClockFit.Confidence * candidate.ClockFit.Confidence) * 0.65;
return (clockDelay, confidence, Math.Min(reference.ClockFit.AnchorCount, candidate.ClockFit.AnchorCount));
}
public MimirAlignedAudioFrame? BuildAlignedFrame(
IEnumerable<MimirRollingStreamBuffer> buffers,
string referenceSourceId,

View file

@ -0,0 +1,29 @@
namespace Mimir.Runtime.Synchronization;
public sealed record MimirChirpletSymbolObservation(
int SymbolId,
double SampleOffset,
double Energy);
public sealed record MimirChirpletTimelineAnchor(
ulong EventIndex,
double TimelineSeconds,
double SampleOffset,
double Confidence,
IReadOnlyList<MimirChirpletSymbolObservation> Symbols);
public sealed record MimirChirpletClockFit(
double SourceOffsetSamples,
double EffectiveSampleRate,
double Confidence,
int AnchorCount,
double MeanAbsoluteErrorSamples)
{
public double SampleForTimelineSeconds(double timelineSeconds) =>
SourceOffsetSamples + timelineSeconds * EffectiveSampleRate;
}
public sealed record MimirChirpletStreamDecode(
IReadOnlyList<MimirChirpletSymbolObservation> Symbols,
IReadOnlyList<MimirChirpletTimelineAnchor> Anchors,
MimirChirpletClockFit? ClockFit);

View file

@ -0,0 +1,79 @@
using System.Runtime.InteropServices;
namespace Mimir.Runtime.Synchronization;
public sealed class MimirChirpletStreamDecoder
{
private readonly MimirChirpletTimeline timeline;
private readonly int sampleRate;
private readonly int maxSamples;
private readonly List<float> samples;
private long originSample;
public MimirChirpletStreamDecoder(
MimirChirpletTimeline? timeline = null,
int sampleRate = MimirChirpletTimeline.SampleRate,
TimeSpan? windowDuration = null)
{
this.timeline = timeline ?? MimirChirpletTimeline.Default;
this.sampleRate = sampleRate;
maxSamples = Math.Max(
sampleRate / 2,
(int)Math.Round((windowDuration ?? TimeSpan.FromSeconds(5)).TotalSeconds * sampleRate));
samples = new List<float>(maxSamples);
}
public long OriginSample => originSample;
public long NextSample => originSample + samples.Count;
public MimirChirpletStreamDecode Decode() =>
WithAbsoluteOffsets(timeline.DecodeStreamWindow(CollectionsMarshal.AsSpan(samples), sampleRate));
public MimirChirpletStreamDecode Append(ReadOnlySpan<float> block)
{
if (block.Length == 0)
{
return Decode();
}
foreach (var sample in block)
{
samples.Add(sample);
}
var overflow = samples.Count - maxSamples;
if (overflow > 0)
{
samples.RemoveRange(0, overflow);
originSample += overflow;
}
return Decode();
}
private MimirChirpletStreamDecode WithAbsoluteOffsets(MimirChirpletStreamDecode decode)
{
if (originSample == 0)
{
return decode;
}
var symbols = decode.Symbols
.Select(symbol => symbol with { SampleOffset = symbol.SampleOffset + originSample })
.ToArray();
var anchors = decode.Anchors
.Select(anchor => anchor with
{
SampleOffset = anchor.SampleOffset + originSample,
Symbols = anchor.Symbols
.Select(symbol => symbol with { SampleOffset = symbol.SampleOffset + originSample })
.ToArray(),
})
.ToArray();
var clock = decode.ClockFit == null
? null
: decode.ClockFit with { SourceOffsetSamples = decode.ClockFit.SourceOffsetSamples + originSample };
return new MimirChirpletStreamDecode(symbols, anchors, clock);
}
}

View file

@ -134,6 +134,24 @@ public sealed class MimirChirpletTimeline
return responses;
}
public MimirChirpletStreamDecode DecodeStreamWindow(ReadOnlySpan<float> samples, int sampleRate)
{
if (samples.Length == 0)
{
return new MimirChirpletStreamDecode([], [], null);
}
var symbols = DetectSymbolEvents(samples, sampleRate, Math.Max(1, sampleRate / 200))
.Select(observation => new MimirChirpletSymbolObservation(
observation.SymbolId,
observation.SampleOffset,
observation.Energy))
.ToArray();
var anchors = PlaceAllTripletAnchors(symbols, sampleRate);
var clock = FitClock(anchors, sampleRate);
return new MimirChirpletStreamDecode(symbols, anchors, clock);
}
public MimirChirpletTimelinePlacement DecodeReferenceWindow(
ReadOnlySpan<float> samples,
int sampleRate,
@ -245,7 +263,11 @@ public sealed class MimirChirpletTimeline
continue;
}
var observation = ClassifySymbolAt(samples, sampleRate, frame * hopSamples, Math.Max(1, hopSamples));
var observation = ClassifySymbolAt(
samples,
sampleRate,
frame * hopSamples,
Math.Max(hopSamples * 3, sampleRate / 100));
if (observation != null)
{
candidates.Add(observation);
@ -287,7 +309,7 @@ public sealed class MimirChirpletTimeline
}
}
return bestSymbol < 0 || bestEnergy < 0.055
return bestSymbol < 0 || bestEnergy < 0.25
? null
: new MimirChirpletEventObservation(0, bestSymbol, 0.0, bestOffset, bestEnergy);
}
@ -363,6 +385,122 @@ public sealed class MimirChirpletTimeline
: FitTimelinePlacement(best, sampleRate);
}
private static IReadOnlyList<MimirChirpletTimelineAnchor> PlaceAllTripletAnchors(
IReadOnlyList<MimirChirpletSymbolObservation> symbols,
int sampleRate)
{
if (symbols.Count < TimelineOrder)
{
return [];
}
var anchors = new List<MimirChirpletTimelineAnchor>();
for (var index = 0; index <= symbols.Count - TimelineOrder; index++)
{
var first = symbols[index];
var second = symbols[index + 1];
var third = symbols[index + 2];
if (!IsConsecutiveByTime(first, second, sampleRate) ||
!IsConsecutiveByTime(second, third, sampleRate))
{
continue;
}
var code = TripleCode(first.SymbolId, second.SymbolId, third.SymbolId);
if (!TripleToIndex.TryGetValue(code, out var eventIndex))
{
continue;
}
var timelineEvent = Default.EventForIndex((ulong)eventIndex);
var predictedSecond = Default.EventForIndex((ulong)eventIndex + 1UL);
var predictedThird = Default.EventForIndex((ulong)eventIndex + 2UL);
var measuredGapA = (second.SampleOffset - first.SampleOffset) / sampleRate;
var measuredGapB = (third.SampleOffset - second.SampleOffset) / sampleRate;
var expectedGapA = predictedSecond.StartSeconds - timelineEvent.StartSeconds;
var expectedGapB = predictedThird.StartSeconds - predictedSecond.StartSeconds;
var gapError = Math.Abs(measuredGapA - expectedGapA) + Math.Abs(measuredGapB - expectedGapB);
var gapConfidence = 1.0 / (1.0 + gapError / 0.006);
if (gapConfidence < 0.45)
{
continue;
}
var energyConfidence = Math.Clamp((first.Energy + second.Energy + third.Energy) / 3.0, 0.0, 1.0);
anchors.Add(new MimirChirpletTimelineAnchor(
(ulong)eventIndex,
timelineEvent.StartSeconds,
first.SampleOffset,
gapConfidence * 0.65 + energyConfidence * 0.35,
[first, second, third]));
}
return anchors
.GroupBy(anchor => anchor.EventIndex)
.Select(group => group.OrderByDescending(anchor => anchor.Confidence).First())
.OrderBy(anchor => anchor.SampleOffset)
.ToArray();
}
private static MimirChirpletClockFit? FitClock(
IReadOnlyList<MimirChirpletTimelineAnchor> anchors,
int sampleRate)
{
if (anchors.Count == 0)
{
return null;
}
if (anchors.Count == 1)
{
var anchor = anchors[0];
return new MimirChirpletClockFit(
anchor.SampleOffset - anchor.TimelineSeconds * sampleRate,
sampleRate,
anchor.Confidence * 0.35,
1,
0.0);
}
var totalWeight = anchors.Sum(anchor => Math.Max(1.0e-6, anchor.Confidence));
var meanTimeline = anchors.Sum(anchor => anchor.TimelineSeconds * Math.Max(1.0e-6, anchor.Confidence)) / totalWeight;
var meanSample = anchors.Sum(anchor => anchor.SampleOffset * Math.Max(1.0e-6, anchor.Confidence)) / totalWeight;
var covariance = 0.0;
var variance = 0.0;
foreach (var anchor in anchors)
{
var weight = Math.Max(1.0e-6, anchor.Confidence);
var dt = anchor.TimelineSeconds - meanTimeline;
covariance += weight * dt * (anchor.SampleOffset - meanSample);
variance += weight * dt * dt;
}
var effectiveSampleRate = variance > 1.0e-12 ? covariance / variance : sampleRate;
if (!double.IsFinite(effectiveSampleRate) ||
effectiveSampleRate < sampleRate * 0.90 ||
effectiveSampleRate > sampleRate * 1.10)
{
effectiveSampleRate = sampleRate;
}
var sourceOffset = meanSample - effectiveSampleRate * meanTimeline;
var meanAbsoluteError = anchors.Sum(anchor =>
{
var weight = Math.Max(1.0e-6, anchor.Confidence);
var predicted = sourceOffset + anchor.TimelineSeconds * effectiveSampleRate;
return Math.Abs(anchor.SampleOffset - predicted) * weight;
}) / totalWeight;
var residualConfidence = 1.0 / (1.0 + meanAbsoluteError / Math.Max(1.0, sampleRate * 0.0015));
var countConfidence = Math.Clamp(anchors.Count / 12.0, 0.0, 1.0);
var anchorConfidence = Math.Clamp(anchors.Average(anchor => anchor.Confidence), 0.0, 1.0);
return new MimirChirpletClockFit(
sourceOffset,
effectiveSampleRate,
residualConfidence * 0.45 + countConfidence * 0.25 + anchorConfidence * 0.30,
anchors.Count,
meanAbsoluteError);
}
private static bool IsConsecutiveByTime(
MimirChirpletEventObservation first,
MimirChirpletEventObservation second,
@ -372,6 +510,15 @@ public sealed class MimirChirpletTimeline
return seconds >= MinEventGapSeconds * 0.55 && seconds <= MaxEventGapSeconds * 1.45;
}
private static bool IsConsecutiveByTime(
MimirChirpletSymbolObservation first,
MimirChirpletSymbolObservation second,
int sampleRate)
{
var seconds = (second.SampleOffset - first.SampleOffset) / sampleRate;
return seconds >= MinEventGapSeconds * 0.55 && seconds <= MaxEventGapSeconds * 1.45;
}
private static MimirChirpletEventObservation PlaceObservation(
MimirChirpletEventObservation observation,
ulong eventIndex)

View file

@ -43,3 +43,4 @@
{"ts":"2026-05-22T14:32:00+01:00","type":"implementation","status":"ok","note":"Reworked MimirChirpletTimeline symbols from fixed frequency slots into time/frequency constellations. The order-3 de Bruijn sequence still gives three-chirp timeline placement, but each symbol now carries identity through start band, glide shape, duration, and following inter-chirp gap. Event start times are precomputed over the de Bruijn period instead of using a fixed stride. Synthetic verification rendered two seconds of canonical timeline audio and decoded interior events 9, 10, and 11 back to a zero-second window start, proving placement does not rely on the opening triplet."}
{"ts":"2026-05-22T14:48:00+01:00","type":"verification","status":"partial","note":"Attempted a 20-second real-world Mimir.App chirplet sync run with all configured local mics. The headless path only emitted one telemetry tick before runtime updates stopped. Switching to the normal app path produced telemetry through t=4.26s with all six audio buffers live and sync states for Focusrite, Kiyo, Kiyo Pro, and both PS3 Eye mics. Timeline-symbol refinement appeared on Focusrite, Kiyo, and one PS3 Eye, but delay estimates were unstable and the app did not continue telemetry to 20 seconds. Replaced the coarse acquisition trace with envelope energy to avoid matched-filter scans in the hot acquisition path; symbol decoding remains the refinement step."}
{"ts":"2026-05-22T15:20:00+01:00","type":"implementation","status":"partial","note":"Removed hidden re-entrant synchronization analysis from the live app path. MimirRuntime now calls a bounded rotating AnalyzeAudioSynchronizationStep every 0.1s, MimirSynchronizationHub caches per-source reports/states, and UI/telemetry read cached state instead of invoking the analyzer or BuildAlignedAudioFrame. FitDelayFromEvents now collapses duplicate candidate event observations by strongest energy, fixing the real app crash 'An item with the same key has already been added. Key: 0'. Dotnet build passed. A visible Mimir.App run with all configured mics no longer crashed at the earlier 4s wall and produced telemetry through runtime t=17.28s in a 22s wall-time harness; it did not reach a full 20 runtime seconds because startup consumed wall time. All non-loopback mics produced cached reports/states, but delay estimates remained unstable and timeline event refinement was intermittent, so the next live invariant is robust outlier rejection/state filtering before any actuator drives resampling."}
{"ts":"2026-05-22T16:05:00+01:00","type":"implementation","status":"partial","note":"Documented the intended decoder as a constrained chirplet transform: known dictionary projection to symbol observations, code-valid de Bruijn triplets to canonical timeline anchors, anchors to affine source clock fits, and delay/SRO from comparing loopback and mic clock fits. Added MimirChirpletStreamDecode records and MimirChirpletStreamDecoder with a bounded PCM window, absolute sample offsets, triplet anchors, and clock fits. MimirAudioSynchronizationAnalyzer now prefers matched decoded timeline anchors before falling back to the older chirplet-energy lag search. Synthetic self-test over two seconds of emitted timeline audio currently decodes 12 symbols and one correct anchor for event 9 at sample 47520 with clock 48000 Hz; this proves the new spine but also proves the symbol likelihood/codebook layer is not finished enough to place every triplet yet."}

View file

@ -71,12 +71,16 @@ accepted_design:
timing atoms, symbol/event decoding, and per-band response kernels. Symbol
identity is carried by start band, glide shape, duration, and following
inter-chirp gap. Any three consecutive correctly detected symbols identify a
timeline event within the current operating horizon. `MimirAudioSynchronizationAnalyzer`
now treats `loopback-scarlett-speakers` as timing authority, uses coarse
chirplet energy for acquisition, and refines delay from matched timeline
event indices when symbol decoding succeeds. Positive delay means the
candidate mic is late relative to loopback. Reports include rounded delay,
fractional delay, confidence, and per-band matched energy. `MimirSynchronizationHub`
timeline event within the current operating horizon. `MimirChirpletStreamDecoder`
is the first constrained chirplet-transform receiver: it owns a bounded PCM
window, emits symbol observations, decodes code-valid triplet anchors onto
the canonical timeline, and fits a per-source sample clock from those
anchors. `MimirAudioSynchronizationAnalyzer` now treats
`loopback-scarlett-speakers` as timing authority, prefers delay from matched
decoded timeline anchors, and keeps the older chirplet-energy lag search as
a diagnostic fallback while the decoder is cut over. Positive delay means
the candidate mic is late relative to loopback. Reports include rounded
delay, fractional delay, confidence, and per-band matched energy. `MimirSynchronizationHub`
caches the latest per-source reports and `MimirAudioSynchronizationStateTracker`
owns the smoothed per-source belief: latest delay, smoothed delay,
confidence, per-band response evidence, and delay-slope/SRO estimate in
@ -183,14 +187,15 @@ current_status:
Kiyo 30.18 fps, PS3 Eye 0 59.66 fps, and PS3 Eye 1 59.95 fps. A mouse in
the Leap USB neighborhood does not reproduce the Focusrite-induced collapse.
next_action: >
Harden the online state filter before adding the synchronization actuator.
Finish the constrained chirplet-transform decoder before adding the
synchronization actuator.
Current app testing proves Aquarium WASAPI output wakes Scarlett speaker
loopback, live mic buffers stay populated, runtime sync analysis now runs as
a bounded rotating service without UI/telemetry re-entry, and the continuous
chirplet timeline can produce confident sync states in the app. Delay
estimates are still jumpy, so the actuator needs outlier rejection and better
SRO smoothing before it drives variable-rate resampling and fractional-delay
correction per non-reference stream. Replace
chirplet timeline can produce confident sync states in the app. The next
invariant is code-valid triplets producing stable canonical anchors and clock
fits, so the actuator can use decoded time instead of filtered lag guesses.
Replace
the diagnostic JSON-line process bridge with direct native capture drivers that push buffer handles into
Mimir.Runtime. Mimir.BufferSmoke still proves loopback-referenced aligned
audio for Scarlett loopback, Focusrite mic, Kiyo mic, and Kiyo Pro mic when