Add chirp-bin watermark decoder

This commit is contained in:
Meta Vens 2026-05-22 19:50:02 +01:00
commit d79bae17bb
8 changed files with 688 additions and 25 deletions

View file

@ -51,12 +51,11 @@ timing evidence Mimir is allowed to emit:
timing witness by estimating delay between the loopback buffer and each mic
buffer.
- `hybrid`: prefer passive program-audio evidence when confidence is high, then
emit a watermark when confidence falls. Today the passive side uses bounded
GCC-PHAT-style phase correlation and the fallback still uses the old active
chirplet pilot in half-second chunks. The intended watermark is the
dechirp/FFT-friendly chirp-bin codebook from the decoder research notes,
shaped and low-gain enough to sit inside the program audio instead of
announcing itself like a lab sweep.
emit a watermark when confidence falls. The passive side uses bounded
GCC-PHAT-style phase correlation. The active fallback now uses
`MimirChirpBinTimeline`: a fixed-slope chirp-bin watermark whose symbols are
decoded by one dechirp and a fixed Goertzel bin bank instead of dense sliding
matched filters.
The mode belongs to the runtime, not the decoder. The decoder should consume
known timing evidence; it should not decide whether Mimir is allowed to make
@ -71,6 +70,13 @@ evidence and carry zero confidence. A single passive window is also capped below
certainty; full confidence belongs to repeated coherent state over time, not
one attractive correlation peak.
The active watermark path is deliberately receiver-cheap. `MimirChirpBinTimeline`
uses a common chirp duration, chirp slope, and Hann window for every symbol.
Symbol identity is the dechirped frequency bin. The synthetic invariant is
`dotnet run --project .\src\Mimir.BufferSmoke\Mimir.BufferSmoke.csproj -- --chirp-bin-self-test`;
it renders the chirp-bin timeline, decodes by dechirp plus Goertzel bins, and
requires code-valid triplet anchors plus a stable source clock.
## Next Cut
The current diagnostic witness is `native/probes/wasapi_audio_cadence`, which
@ -186,13 +192,10 @@ timeline anchors. Current synthetic proof detects all 15 emitted chirps, keeps
device runs still depend on loopback capture staying live; the local Scarlett
loopback has intermittently stopped advancing during short headless sniffs.
This matcher is a proof, not the final hot path. `BuildChirpletEnergyTrace`
still behaves like a dense sliding matched-filter bank. The current research
note at `research/chirplet-sync-decoder/summary.md` points at the better
receiver shape: design the emitted symbol codebook so a mic can dechirp a
candidate event window and classify symbols by FFT or Goertzel bins. The
transmitter is ours; do not preserve arbitrary symbol shapes if they force the
receiver to spend a CPU core proving they exist.
The older `MimirChirpletTimeline` matcher is now the chirp-only lab path, not
the hybrid fallback. `BuildChirpletEnergyTrace` still behaves like a dense
sliding matched-filter bank and should not retake the hot path. The active
hybrid watermark belongs to the chirp-bin machine.
Next, replace the diagnostic bridge with native audio capture workers that
append typed blocks into `Mimir.Runtime`, then expose buffer depth, clock state,

View file

@ -79,6 +79,10 @@ a named invariant that the native runtime cannot protect yet.
path. It estimates loopback-to-mic delay with PHAT-weighted cross-spectrum
correlation so music can act as the default timing witness before any audible
watermark is needed.
- `MimirChirpBinTimeline` is the active hybrid watermark path. It renders a
fixed-slope chirp-bin codebook and decodes symbols with one dechirp plus
fixed Goertzel bins, then feeds the same de Bruijn triplet timeline-anchor
machine. Use `--chirp-bin-self-test` to prove this path without hardware.
- `MimirVideoFrameDescriptor` for dimensions, pixel format, stride, device
timestamp, and native/GPU handle metadata.
- `IMimirVideoCaptureDriver` and `MimirVideoCaptureDriverSource` as the live
@ -118,10 +122,8 @@ a named invariant that the native runtime cannot protect yet.
decoder through real loopback and microphone paths so every correctly heard
triplet becomes a deterministic timeline anchor before the actuator moves
samples.
5. Replace the active chirplet fallback in hybrid mode with shaped watermark
emission when passive confidence drops. The watermark codebook should be
designed for dechirp/FFT or Goertzel classification, not dense sliding
matched filters.
5. Prove the chirp-bin hybrid fallback through real loopback and microphones
with probe durations long enough to keep loopback and mic windows live.
6. Bind Aquarium UI to the synchronization hub so buffer depth, stream cadence,
source timestamps, and output settings are visible and adjustable.
7. Move GPU feature extraction, fusion, material fitting, render budgeting, and

View file

@ -5,6 +5,11 @@ if (args.Any(arg => string.Equals(arg, "--chirplet-self-test", StringComparison.
return RunChirpletSelfTest();
}
if (args.Any(arg => string.Equals(arg, "--chirp-bin-self-test", StringComparison.OrdinalIgnoreCase)))
{
return RunChirpBinSelfTest();
}
if (args.Any(arg => string.Equals(arg, "--passive-sync-self-test", StringComparison.OrdinalIgnoreCase)))
{
return RunPassiveSyncSelfTest();
@ -162,6 +167,41 @@ static int RunChirpletSelfTest()
return 0;
}
static int RunChirpBinSelfTest()
{
var timeline = MimirChirpBinTimeline.Default;
var samples = new List<float>();
for (ulong segment = 0; segment < 6; segment++)
{
samples.AddRange(timeline.RenderSegmentMonoFloat(segment));
}
var decode = timeline.DecodeStreamWindow(samples.ToArray(), MimirChirpBinTimeline.SampleRate);
var meanAbsoluteError = decode.ClockFit?.MeanAbsoluteErrorSamples ?? double.PositiveInfinity;
Console.WriteLine(
$"chirp-bin-self-test frames={decode.Frames.Count} symbols={decode.Symbols.Count} anchors={decode.Anchors.Count} " +
$"clock={(decode.ClockFit is null ? "none" : decode.ClockFit.EffectiveSampleRate.ToString("0.000000"))} " +
$"confidence={(decode.ClockFit?.Confidence ?? 0.0):0.000} mae={meanAbsoluteError:0.000000}");
Console.WriteLine("chirp-bin-expected " + string.Join(",", Enumerable.Range(0, 12).Select(index => timeline.EventForIndex((ulong)index).SymbolId)));
Console.WriteLine("chirp-bin-symbols " + string.Join(",", decode.Symbols.Take(12).Select(symbol => $"{symbol.SymbolId}@{symbol.SampleOffset:0}:{symbol.Energy:0.000}")));
foreach (var anchor in decode.Anchors.Take(16))
{
var expected = timeline.EventForIndex(anchor.EventIndex).StartSeconds * MimirChirpBinTimeline.SampleRate;
Console.WriteLine(
$"chirp-bin-anchor event={anchor.EventIndex} actual={anchor.SampleOffset:0.000} expected={expected:0.000} " +
$"error={anchor.SampleOffset - expected:0.000} confidence={anchor.Confidence:0.000}");
}
if (decode.ClockFit == null || decode.Anchors.Count < 16 || meanAbsoluteError > 2.0)
{
Console.Error.WriteLine("chirp-bin self-test failed: dechirp/bin decoder did not recover stable timeline anchors");
return 1;
}
return 0;
}
static int RunPassiveSyncSelfTest()
{
const int sampleRate = 48_000;

View file

@ -156,14 +156,18 @@ public sealed class MimirRuntime : IAquariumRuntime
}
var segmentCount = audioSyncSettings.Mode == MimirAudioSyncMode.Hybrid ? 1 : CalibrationBatchSegments;
var batch = RenderCalibrationBatchPcm16Base64(calibrationSegmentIndex, segmentCount, out var peak);
var batch = RenderCalibrationBatchPcm16Base64(
calibrationSegmentIndex,
segmentCount,
audioSyncSettings.Mode == MimirAudioSyncMode.Hybrid,
out var peak);
visualRuntime.Audio.EnqueuePcm16Base64(
batch,
MimirChirpletTimeline.SampleRate,
channels: 1,
gain: calibrationGain);
Console.WriteLine(
$"mimir-chirplet-batch firstSegment={calibrationSegmentIndex} segments={segmentCount} seconds={segmentCount * MimirChirpletTimeline.SegmentSeconds:0.00} peak={peak:0.000000} gain={calibrationGain:0.###} base64Bytes={batch.Length}");
$"mimir-chirplet-batch mode={DescribeAudioSyncMode()} firstSegment={calibrationSegmentIndex} segments={segmentCount} seconds={segmentCount * MimirChirpletTimeline.SegmentSeconds:0.00} peak={peak:0.000000} gain={calibrationGain:0.###} base64Bytes={batch.Length}");
calibrationSegmentIndex += (ulong)segmentCount;
}
@ -178,7 +182,11 @@ public sealed class MimirRuntime : IAquariumRuntime
return $"{audioSyncSettings.ReferenceSourceId} {DescribeAudioSyncMode()} timeline {MimirChirpletTimeline.SegmentSeconds:0.00}s segments emitted to {emittedUntilSeconds:0.00}s passiveConfidence={lastPassiveSynchronizationConfidence:0.000}";
}
private static string RenderCalibrationBatchPcm16Base64(ulong firstSegment, int segmentCount, out float peak)
private static string RenderCalibrationBatchPcm16Base64(
ulong firstSegment,
int segmentCount,
bool useChirpBinTimeline,
out float peak)
{
var samplesPerSegment = (int)Math.Round(MimirChirpletTimeline.SegmentSeconds * MimirChirpletTimeline.SampleRate);
var bytes = new byte[samplesPerSegment * Math.Max(1, segmentCount) * sizeof(short)];
@ -186,7 +194,9 @@ public sealed class MimirRuntime : IAquariumRuntime
peak = 0.0f;
for (var segment = 0; segment < segmentCount; segment++)
{
var samples = MimirChirpletTimeline.Default.RenderSegmentMonoFloat(firstSegment + (ulong)segment);
var samples = useChirpBinTimeline
? MimirChirpBinTimeline.Default.RenderSegmentMonoFloat(firstSegment + (ulong)segment)
: MimirChirpletTimeline.Default.RenderSegmentMonoFloat(firstSegment + (ulong)segment);
for (var index = 0; index < samples.Length; index++)
{
peak = Math.Max(peak, Math.Abs(samples[index]));

View file

@ -152,8 +152,12 @@ public sealed class MimirAudioSynchronizationAnalyzer
}
}
var comparedReferenceDecode = MimirChirpletTimeline.Default.DecodeStreamWindow(referenceWindow, activeReferenceBlock.SampleRate);
var candidateDecode = MimirChirpletTimeline.Default.DecodeStreamWindow(candidateWindow, activeReferenceBlock.SampleRate);
var comparedReferenceDecode = mode == MimirAudioSyncMode.Hybrid
? MimirChirpBinTimeline.Default.DecodeStreamWindow(referenceWindow, activeReferenceBlock.SampleRate)
: MimirChirpletTimeline.Default.DecodeStreamWindow(referenceWindow, activeReferenceBlock.SampleRate);
var candidateDecode = mode == MimirAudioSyncMode.Hybrid
? MimirChirpBinTimeline.Default.DecodeStreamWindow(candidateWindow, activeReferenceBlock.SampleRate)
: MimirChirpletTimeline.Default.DecodeStreamWindow(candidateWindow, activeReferenceBlock.SampleRate);
var deterministicFit = EstimateDelayFromDecodedTimeline(comparedReferenceDecode, candidateDecode);
lastDecodeTraces.Add(new MimirAudioSynchronizationDecodeTrace(
reference.Descriptor.SourceId,
@ -177,7 +181,9 @@ public sealed class MimirAudioSynchronizationAnalyzer
}
var decodedDelaySamples = deterministicFit.DelaySamples;
var bandResponses = MimirChirpletTimeline.Default.EstimateBandResponse(candidateWindow, activeReferenceBlock.SampleRate);
var bandResponses = mode == MimirAudioSyncMode.Hybrid
? []
: MimirChirpletTimeline.Default.EstimateBandResponse(candidateWindow, activeReferenceBlock.SampleRate);
reports.Add(new MimirAudioSynchronizationReport(
reference.Descriptor.SourceId,
buffer.Descriptor.SourceId,
@ -193,7 +199,7 @@ public sealed class MimirAudioSynchronizationAnalyzer
buffer.Latest?.Sequence ?? 0,
deterministicFit.MatchedEvents,
deterministicFit.Confidence,
"chirplet"));
mode == MimirAudioSyncMode.Hybrid ? "chirp-bin" : "chirplet"));
}
return reports;

View file

@ -0,0 +1,596 @@
using System.Numerics;
namespace Mimir.Runtime.Synchronization;
public sealed record MimirChirpBinSymbolDefinition(
int SymbolId,
double OffsetHz);
public sealed class MimirChirpBinTimeline
{
public const int SampleRate = 48_000;
public const double SegmentSeconds = 0.5;
public const int SymbolCount = 32;
public const int TimelineOrder = 3;
private const double FirstEventSeconds = 0.08;
private const double EventSpacingSeconds = 0.125;
private const double ChirpDurationSeconds = 0.064;
private const double BaseStartHz = 7_600.0;
private const double BaseEndHz = 14_800.0;
private const double BinSpacingHz = 62.5;
private const double Gain = 0.070;
private const int MaxSymbolCandidatesPerFrame = 6;
private static readonly int[] TimelineSymbols = RotateToDistinctOpening(BuildDeBruijn(SymbolCount, TimelineOrder));
private static readonly Dictionary<int, int> TripleToIndex = BuildTripleIndex(TimelineSymbols);
private static readonly MimirChirpBinSymbolDefinition[] Symbols = BuildSymbols();
public static MimirChirpBinTimeline Default { get; } = new();
public IReadOnlyList<MimirChirpBinSymbolDefinition> Codebook => Symbols;
public float[] RenderSegmentMonoFloat(ulong segmentIndex)
{
var segmentStartSeconds = segmentIndex * SegmentSeconds;
var samples = new float[(int)Math.Round(SegmentSeconds * SampleRate)];
foreach (var timelineEvent in EventsOverlapping(segmentStartSeconds, SegmentSeconds))
{
AddChirp(samples, SampleRate, timelineEvent, segmentStartSeconds);
}
return samples;
}
public MimirChirpletStreamDecode DecodeStreamWindow(ReadOnlySpan<float> samples, int sampleRate)
{
if (samples.Length == 0)
{
return new MimirChirpletStreamDecode([], [], [], null);
}
var frames = DetectFrames(samples, sampleRate);
var symbols = frames
.Select(frame => new MimirChirpletSymbolObservation(
frame.BestCandidate.SymbolId,
frame.SampleOffset,
frame.BestCandidate.Energy))
.ToArray();
var anchors = DecodeAnchors(frames, sampleRate);
var clock = FitClock(anchors, sampleRate);
return new MimirChirpletStreamDecode(frames, symbols, anchors, clock);
}
public MimirChirpletTimelineEvent EventForIndex(ulong eventIndex)
{
var symbolId = SymbolForEvent(eventIndex);
var startSeconds = EventStartSeconds(eventIndex);
var symbol = Symbols[symbolId];
return new MimirChirpletTimelineEvent(
eventIndex,
symbolId,
startSeconds,
new MimirChirpletTone(
startSeconds,
ChirpDurationSeconds,
BaseStartHz + symbol.OffsetHz,
BaseEndHz + symbol.OffsetHz,
Gain));
}
public IReadOnlyList<MimirChirpletTimelineEvent> EventsOverlapping(double startSeconds, double durationSeconds)
{
var endSeconds = startSeconds + durationSeconds;
var firstIndex = Math.Max(0, (long)Math.Floor((startSeconds - FirstEventSeconds - ChirpDurationSeconds) / EventSpacingSeconds) - 2);
var lastIndex = Math.Max(firstIndex, (long)Math.Ceiling((endSeconds - FirstEventSeconds) / EventSpacingSeconds) + 2);
var events = new List<MimirChirpletTimelineEvent>((int)(lastIndex - firstIndex + 1));
for (var eventIndex = firstIndex; eventIndex <= lastIndex; eventIndex++)
{
var timelineEvent = EventForIndex((ulong)eventIndex);
if (timelineEvent.StartSeconds < endSeconds &&
timelineEvent.StartSeconds + ChirpDurationSeconds > startSeconds)
{
events.Add(timelineEvent);
}
}
return events;
}
private static IReadOnlyList<MimirChirpletTransformFrame> DetectFrames(ReadOnlySpan<float> samples, int sampleRate)
{
var chirpSamples = Math.Max(1, (int)Math.Round(ChirpDurationSeconds * sampleRate));
if (samples.Length < chirpSamples)
{
return [];
}
var hopSamples = Math.Max(1, sampleRate / 400);
var energyTrace = BuildEnergyTrace(samples, chirpSamples, hopSamples);
var threshold = AdaptiveThreshold(energyTrace);
var proposals = new List<int>();
for (var index = 1; index < energyTrace.Length - 1; index++)
{
if (energyTrace[index] >= threshold &&
energyTrace[index] >= energyTrace[index - 1] &&
energyTrace[index] >= energyTrace[index + 1])
{
proposals.Add(index * hopSamples);
}
}
var frames = new List<MimirChirpletTransformFrame>();
foreach (var proposal in proposals)
{
var frame = ClassifyAt(samples, sampleRate, proposal, sampleRate / 120);
if (frame != null)
{
frames.Add(frame);
}
}
return SuppressNearbyFrames(frames, sampleRate);
}
private static MimirChirpletTransformFrame? ClassifyAt(
ReadOnlySpan<float> samples,
int sampleRate,
int predictedOffset,
int searchRadiusSamples)
{
var chirpSamples = Math.Max(1, (int)Math.Round(ChirpDurationSeconds * sampleRate));
if (samples.Length < chirpSamples)
{
return null;
}
var start = Math.Max(0, predictedOffset - searchRadiusSamples);
var end = Math.Min(samples.Length - chirpSamples, predictedOffset + searchRadiusSamples);
var bestOffset = predictedOffset;
MimirChirpletSymbolCandidate[] bestCandidates = [];
var bestEnergy = 0.0;
var step = Math.Max(2, searchRadiusSamples / 24);
for (var offset = start; offset <= end; offset += step)
{
var candidates = ScoreBins(samples.Slice(offset, chirpSamples), sampleRate, offset);
var energy = candidates.Length == 0 ? 0.0 : candidates[0].Energy;
if (energy > bestEnergy)
{
bestEnergy = energy;
bestOffset = offset;
bestCandidates = candidates;
}
}
var localStart = Math.Max(start, bestOffset - step);
var localEnd = Math.Min(end, bestOffset + step);
for (var offset = localStart; offset <= localEnd; offset++)
{
var candidates = ScoreBins(samples.Slice(offset, chirpSamples), sampleRate, offset);
var energy = candidates.Length == 0 ? 0.0 : candidates[0].Energy;
if (energy > bestEnergy)
{
bestEnergy = energy;
bestOffset = offset;
bestCandidates = candidates;
}
}
if (bestCandidates.Length == 0 || bestEnergy < 0.11)
{
return null;
}
return new MimirChirpletTransformFrame(bestOffset, bestCandidates);
}
private static MimirChirpletSymbolCandidate[] ScoreBins(
ReadOnlySpan<float> samples,
int sampleRate,
int sampleOffset)
{
var scores = new MimirChirpletSymbolCandidate[Symbols.Length];
var sampleEnergy = 0.0;
var referenceEnergy = 0.0;
for (var index = 0; index < samples.Length; index++)
{
sampleEnergy += samples[index] * samples[index];
var normalized = samples.Length <= 1 ? 1.0 : index / (double)(samples.Length - 1);
var window = 0.5 - 0.5 * Math.Cos(2.0 * Math.PI * normalized);
referenceEnergy += window * window * 0.5;
}
if (sampleEnergy <= 1.0e-12 || referenceEnergy <= 1.0e-12)
{
return [];
}
for (var symbol = 0; symbol < Symbols.Length; symbol++)
{
var score = GoertzelDechirpedBin(samples, sampleRate, Symbols[symbol].OffsetHz);
scores[symbol] = new MimirChirpletSymbolCandidate(
symbol,
sampleOffset,
2.0 * score / Math.Sqrt(sampleEnergy * referenceEnergy));
}
return scores
.OrderByDescending(candidate => candidate.Energy)
.Take(MaxSymbolCandidatesPerFrame)
.ToArray();
}
private static double GoertzelDechirpedBin(ReadOnlySpan<float> samples, int sampleRate, double offsetHz)
{
var real = 0.0;
var imaginary = 0.0;
for (var index = 0; index < samples.Length; index++)
{
var t = index / (double)sampleRate;
var normalized = samples.Length <= 1 ? 1.0 : index / (double)(samples.Length - 1);
var window = 0.5 - 0.5 * Math.Cos(2.0 * Math.PI * normalized);
var basePhase = BasePhase(t);
var binPhase = 2.0 * Math.PI * offsetHz * t;
var phase = -(basePhase + binPhase);
var value = samples[index] * window;
real += value * Math.Cos(phase);
imaginary += value * Math.Sin(phase);
}
return Math.Sqrt(real * real + imaginary * imaginary);
}
private static IReadOnlyList<MimirChirpletTimelineAnchor> DecodeAnchors(
IReadOnlyList<MimirChirpletTransformFrame> frames,
int sampleRate)
{
if (frames.Count < TimelineOrder)
{
return [];
}
var candidates = new List<MimirChirpletTimelineAnchor>();
for (var index = 0; index <= frames.Count - TimelineOrder; index++)
{
var firstFrame = frames[index];
var secondFrame = frames[index + 1];
var thirdFrame = frames[index + 2];
if (!IsConsecutive(firstFrame, secondFrame, sampleRate) ||
!IsConsecutive(secondFrame, thirdFrame, sampleRate))
{
continue;
}
foreach (var first in firstFrame.Candidates)
foreach (var second in secondFrame.Candidates)
foreach (var third in thirdFrame.Candidates)
{
var code = TripleCode(first.SymbolId, second.SymbolId, third.SymbolId);
if (!TripleToIndex.TryGetValue(code, out var eventIndex))
{
continue;
}
var firstEvent = Default.EventForIndex((ulong)eventIndex);
var measuredGapA = (second.SampleOffset - first.SampleOffset) / sampleRate;
var measuredGapB = (third.SampleOffset - second.SampleOffset) / sampleRate;
var gapError = Math.Abs(measuredGapA - EventSpacingSeconds) + Math.Abs(measuredGapB - EventSpacingSeconds);
var gapConfidence = 1.0 / (1.0 + gapError / 0.003);
if (gapConfidence < 0.55)
{
continue;
}
var energyConfidence = Math.Clamp((first.Energy + second.Energy + third.Energy) / 3.0, 0.0, 1.0);
candidates.Add(new MimirChirpletTimelineAnchor(
(ulong)eventIndex,
firstEvent.StartSeconds,
first.SampleOffset,
gapConfidence * 0.60 + energyConfidence * 0.40,
[
new MimirChirpletSymbolObservation(first.SymbolId, first.SampleOffset, first.Energy),
new MimirChirpletSymbolObservation(second.SymbolId, second.SampleOffset, second.Energy),
new MimirChirpletSymbolObservation(third.SymbolId, third.SampleOffset, third.Energy),
]));
}
}
return SelectCoherentAnchorPath(candidates, sampleRate);
}
private static IReadOnlyList<MimirChirpletTimelineAnchor> SelectCoherentAnchorPath(
IReadOnlyList<MimirChirpletTimelineAnchor> candidates,
int sampleRate)
{
if (candidates.Count == 0)
{
return [];
}
List<MimirChirpletTimelineAnchor> bestPath = [];
var bestScore = double.NegativeInfinity;
foreach (var seed in candidates.OrderByDescending(anchor => anchor.Confidence).Take(64))
{
var seedOffset = seed.SampleOffset - seed.TimelineSeconds * sampleRate;
var path = candidates
.Where(anchor => Math.Abs((anchor.SampleOffset - anchor.TimelineSeconds * sampleRate) - seedOffset) <= sampleRate * 0.006)
.GroupBy(anchor => anchor.EventIndex)
.Select(group => group.OrderByDescending(anchor => anchor.Confidence).First())
.OrderBy(anchor => anchor.EventIndex)
.ToList();
var clock = FitClock(path, sampleRate);
if (clock == null)
{
continue;
}
var score = path.Sum(anchor => anchor.Confidence) +
Math.Min(path.Count, 12) * 0.20 -
clock.MeanAbsoluteErrorSamples / Math.Max(1.0, sampleRate * 0.001);
if (score > bestScore)
{
bestScore = score;
bestPath = path;
}
}
var finalClock = FitClock(bestPath, sampleRate);
if (finalClock == null)
{
return [];
}
return bestPath
.Where(anchor => Math.Abs(anchor.SampleOffset - finalClock.SampleForTimelineSeconds(anchor.TimelineSeconds)) <= Math.Max(16.0, sampleRate * 0.0015))
.OrderBy(anchor => anchor.SampleOffset)
.ToArray();
}
private static MimirChirpletClockFit? FitClock(
IReadOnlyList<MimirChirpletTimelineAnchor> anchors,
int sampleRate)
{
if (anchors.Count == 0)
{
return null;
}
if (anchors.Count == 1)
{
var anchor = anchors[0];
return new MimirChirpletClockFit(
anchor.SampleOffset - anchor.TimelineSeconds * sampleRate,
sampleRate,
anchor.Confidence * 0.35,
1,
0.0);
}
var totalWeight = anchors.Sum(anchor => Math.Max(1.0e-6, anchor.Confidence));
var meanTimeline = anchors.Sum(anchor => anchor.TimelineSeconds * Math.Max(1.0e-6, anchor.Confidence)) / totalWeight;
var meanSample = anchors.Sum(anchor => anchor.SampleOffset * Math.Max(1.0e-6, anchor.Confidence)) / totalWeight;
var covariance = 0.0;
var variance = 0.0;
foreach (var anchor in anchors)
{
var weight = Math.Max(1.0e-6, anchor.Confidence);
var dt = anchor.TimelineSeconds - meanTimeline;
covariance += weight * dt * (anchor.SampleOffset - meanSample);
variance += weight * dt * dt;
}
var effectiveSampleRate = variance > 1.0e-12 ? covariance / variance : sampleRate;
if (!double.IsFinite(effectiveSampleRate) ||
effectiveSampleRate < sampleRate * 0.98 ||
effectiveSampleRate > sampleRate * 1.02)
{
effectiveSampleRate = sampleRate;
}
var sourceOffset = meanSample - effectiveSampleRate * meanTimeline;
var meanAbsoluteError = anchors.Sum(anchor =>
{
var weight = Math.Max(1.0e-6, anchor.Confidence);
var predicted = sourceOffset + anchor.TimelineSeconds * effectiveSampleRate;
return Math.Abs(anchor.SampleOffset - predicted) * weight;
}) / totalWeight;
var residualConfidence = 1.0 / (1.0 + meanAbsoluteError / Math.Max(1.0, sampleRate * 0.00075));
var countConfidence = Math.Clamp(anchors.Count / 12.0, 0.0, 1.0);
var anchorConfidence = Math.Clamp(anchors.Average(anchor => anchor.Confidence), 0.0, 1.0);
return new MimirChirpletClockFit(
sourceOffset,
effectiveSampleRate,
residualConfidence * 0.45 + countConfidence * 0.25 + anchorConfidence * 0.30,
anchors.Count,
meanAbsoluteError);
}
private static float[] BuildEnergyTrace(ReadOnlySpan<float> samples, int windowSamples, int hopSamples)
{
var output = new float[1 + (samples.Length - windowSamples) / hopSamples];
var energy = 0.0;
for (var index = 0; index < windowSamples; index++)
{
energy += samples[index] * samples[index];
}
output[0] = (float)Math.Sqrt(energy / windowSamples);
for (var frame = 1; frame < output.Length; frame++)
{
var previousStart = (frame - 1) * hopSamples;
var nextStart = frame * hopSamples;
for (var index = previousStart; index < nextStart; index++)
{
energy -= samples[index] * samples[index];
}
for (var index = previousStart + windowSamples; index < nextStart + windowSamples; index++)
{
energy += samples[index] * samples[index];
}
output[frame] = (float)Math.Sqrt(Math.Max(0.0, energy) / windowSamples);
}
return output;
}
private static double AdaptiveThreshold(float[] energyTrace)
{
if (energyTrace.Length == 0)
{
return double.PositiveInfinity;
}
var mean = energyTrace.Average(value => (double)value);
var variance = energyTrace.Sum(value => (value - mean) * (value - mean)) / energyTrace.Length;
return mean + Math.Sqrt(Math.Max(0.0, variance)) * 0.75;
}
private static IReadOnlyList<MimirChirpletTransformFrame> SuppressNearbyFrames(
IReadOnlyList<MimirChirpletTransformFrame> frames,
int sampleRate)
{
var minimumSpacingSamples = EventSpacingSeconds * sampleRate * 0.65;
var kept = new List<MimirChirpletTransformFrame>();
foreach (var frame in frames.OrderByDescending(frame => frame.BestCandidate.Energy))
{
if (kept.Any(existing => Math.Abs(existing.SampleOffset - frame.SampleOffset) < minimumSpacingSamples))
{
continue;
}
kept.Add(frame);
}
return kept.OrderBy(frame => frame.SampleOffset).ToArray();
}
private static bool IsConsecutive(
MimirChirpletTransformFrame first,
MimirChirpletTransformFrame second,
int sampleRate)
{
var seconds = (second.SampleOffset - first.SampleOffset) / sampleRate;
return seconds >= EventSpacingSeconds * 0.70 && seconds <= EventSpacingSeconds * 1.30;
}
private static void AddChirp(
float[] samples,
int sampleRate,
MimirChirpletTimelineEvent timelineEvent,
double segmentStartSeconds)
{
var symbol = Symbols[timelineEvent.SymbolId];
var startFrame = (int)Math.Round((timelineEvent.StartSeconds - segmentStartSeconds) * sampleRate);
var frameCount = Math.Max(1, (int)Math.Round(ChirpDurationSeconds * sampleRate));
for (var frame = 0; frame < frameCount; frame++)
{
var outputFrame = startFrame + frame;
if (outputFrame < 0 || outputFrame >= samples.Length)
{
continue;
}
var normalized = frameCount <= 1 ? 1.0 : frame / (double)(frameCount - 1);
var t = frame / (double)sampleRate;
var envelope = 0.5 - 0.5 * Math.Cos(2.0 * Math.PI * normalized);
var phase = BasePhase(t) + 2.0 * Math.PI * symbol.OffsetHz * t;
samples[outputFrame] += (float)(Math.Sin(phase) * envelope * Gain);
}
}
private static double BasePhase(double t)
{
var slope = (BaseEndHz - BaseStartHz) / ChirpDurationSeconds;
return 2.0 * Math.PI * (BaseStartHz * t + 0.5 * slope * t * t);
}
private static MimirChirpBinSymbolDefinition[] BuildSymbols()
{
var symbols = new MimirChirpBinSymbolDefinition[SymbolCount];
var firstOffset = -(SymbolCount - 1) * BinSpacingHz * 0.5;
for (var symbol = 0; symbol < symbols.Length; symbol++)
{
symbols[symbol] = new MimirChirpBinSymbolDefinition(symbol, firstOffset + symbol * BinSpacingHz);
}
return symbols;
}
private static int SymbolForEvent(ulong eventIndex) =>
TimelineSymbols[(int)(eventIndex % (ulong)TimelineSymbols.Length)];
private static double EventStartSeconds(ulong eventIndex) =>
FirstEventSeconds + eventIndex * EventSpacingSeconds;
private static int[] BuildDeBruijn(int alphabetSize, int order)
{
var a = new int[alphabetSize * order];
var sequence = new List<int>((int)Math.Pow(alphabetSize, order));
void Db(int t, int p)
{
if (t > order)
{
if (order % p == 0)
{
for (var index = 1; index <= p; index++)
{
sequence.Add(a[index]);
}
}
return;
}
a[t] = a[t - p];
Db(t + 1, p);
for (var j = a[t - p] + 1; j < alphabetSize; j++)
{
a[t] = j;
Db(t + 1, t);
}
}
Db(1, 1);
return sequence.ToArray();
}
private static int[] RotateToDistinctOpening(int[] sequence)
{
for (var index = 0; index < sequence.Length; index++)
{
var first = sequence[index];
var second = sequence[(index + 1) % sequence.Length];
var third = sequence[(index + 2) % sequence.Length];
if (first != second && first != third && second != third)
{
return sequence.Skip(index).Concat(sequence.Take(index)).ToArray();
}
}
return sequence;
}
private static Dictionary<int, int> BuildTripleIndex(IReadOnlyList<int> symbols)
{
var map = new Dictionary<int, int>(symbols.Count);
for (var index = 0; index < symbols.Count; index++)
{
var code = TripleCode(
symbols[index],
symbols[(index + 1) % symbols.Count],
symbols[(index + 2) % symbols.Count]);
if (!map.ContainsKey(code))
{
map[code] = index;
}
}
return map;
}
private static int TripleCode(int first, int second, int third) =>
first * SymbolCount * SymbolCount + second * SymbolCount + third;
}

View file

@ -56,3 +56,4 @@
{"ts": "2026-05-22T19:08:10+01:00", "type": "implementation", "status": "ok", "note": "Renamed the public no-chirp mode to passive and started the real hybrid sync machine. Added MimirPassiveAudioSynchronizationEstimator using PHAT-weighted cross-spectrum delay estimation over loopback/program audio; MimirAudioSynchronizationAnalyzer now reports passive evidence in passive/hybrid modes and falls back to chirplet anchors when hybrid passive confidence is weak. Hybrid runtime emission now queues short active pilot chunks only while passive confidence stays below threshold. dotnet build, passive sync self-test, and chirplet self-test passed."}
{"ts": "2026-05-22T19:08:56+01:00", "type": "verification", "status": "partial", "note": "Passive live smoke with MIMIR_AUDIO_SYNC_MODE=passive and config/mimir-runtime.probes.example.json emitted no calibration audio and produced loopback/program-audio reports: Focusrite delaySamples=6653 confidence=0.805, Kiyo Pro delaySamples=6804 confidence=0.227, USB Camera 0 delaySamples=-882 confidence=1.000 over 88200 compared 44.1k samples. This proves the passive path can lock on real loopback/music, but the negative Eye delay needs timestamp/window-sign inspection before driving an actuator."}
{"ts": "2026-05-22T19:33:21+01:00", "type": "verification", "status": "ok", "note": "Fixed passive sync confidence bug. Negative passive lags now produce status=passive-negative-lag with confidence=0, single-window passive confidence is capped at 0.85, and hub invalidates cached reports/state for negative-lag traces. Passive self-test now proves both positive delay recovery and reversed-input zero confidence. A 14s live passive smoke showed negative Focusrite evidence at confidence 0 without leaving a report/state, while positive passive reports stayed capped at 0.85."}
{"ts": "2026-05-22T19:49:43+01:00", "type": "implementation", "status": "partial", "note": "Built the first chirp-bin active watermark machine. MimirChirpBinTimeline renders a fixed-slope/window 32-bin chirp codebook and decodes with one dechirp plus fixed Goertzel bin scoring, then feeds de Bruijn triplet anchors and clock fitting. Hybrid fallback now renders/decodes chirp-bin evidence instead of the old dense chirplet matcher. Synthetic --chirp-bin-self-test passed with 23 frames, 21 anchors, 48000 Hz clock, and zero MAE; old --chirplet-self-test still passed. A real hybrid app run emitted chirp-bin batches, but the current probe config's 30s capture limit left loopback/mic windows stale before acoustic chirp-bin reports could be proven."}

View file

@ -102,6 +102,11 @@ accepted_design:
windows before any watermark is needed. Negative passive lags are
contradictory evidence and carry zero confidence; single-window passive
confidence is capped below certainty until repeated state earns it.
`MimirChirpBinTimeline` is the active hybrid watermark path: one fixed chirp
slope/window, 32 symbol frequency bins, dechirp plus fixed Goertzel bin
scoring, then the existing de Bruijn triplet anchor and clock-fit machine.
The synthetic proof is `dotnet run --project
.\src\Mimir.BufferSmoke\Mimir.BufferSmoke.csproj -- --chirp-bin-self-test`.
- `IMimirVideoCaptureDriver` and `MimirVideoCaptureDriverSource` are the live
seam for Leap, Media Foundation, DirectShow, libusb, LeapC, or shared
texture capture.