diff --git a/calibration/bioacoustic/complex-contour-channel-model-evaluation.json b/calibration/bioacoustic/complex-contour-channel-model-evaluation.json index 5200402..84c32a4 100644 --- a/calibration/bioacoustic/complex-contour-channel-model-evaluation.json +++ b/calibration/bioacoustic/complex-contour-channel-model-evaluation.json @@ -1,6 +1,6 @@ { "Schema": "mimir.bioacoustic.complex-contour-channel-model-evaluation.v1", - "CreatedUtc": "2026-05-25T00:38:21.3837184+00:00", + "CreatedUtc": "2026-05-25T05:27:12.8192113+00:00", "ReplayReceiptPath": "calibration\\bioacoustic\\complex-contour-replay-panel.json", "ChannelModelPath": "calibration\\bioacoustic\\complex-contour-channel-model.json", "CasesEvaluated": 4, diff --git a/config/mimir-runtime.asio.example.json b/config/mimir-runtime.asio.example.json index 2685f00..f052b64 100644 --- a/config/mimir-runtime.asio.example.json +++ b/config/mimir-runtime.asio.example.json @@ -5,7 +5,10 @@ "referenceSourceId": "asio-ch2", "calibrationGain": 2.0, "watermarkGain": 0.035, - "calibrationModelPath": "../calibration/chirp-bin/scarlett-artifact.json" + "calibrationModelPath": "../calibration/chirp-bin/scarlett-artifact.json", + "complexContourChannelModelPath": "../calibration/bioacoustic/complex-contour-channel-model.json", + "bioacousticWitnessProfileId": "canary-packet-trill", + "enableComplexContourRuntime": true }, "streams": [ { diff --git a/docs/bioacoustic-timeline-watermark.md b/docs/bioacoustic-timeline-watermark.md index 9f7adad..4f44d55 100644 --- a/docs/bioacoustic-timeline-watermark.md +++ b/docs/bioacoustic-timeline-watermark.md @@ -197,3 +197,13 @@ stored cardioid path still worsens from about `15.817 us` to `17.476 us`. That means the learned phase/group-delay surface is real evidence, while the Perfect Machine still needs more captures and a stronger direct-path hypothesis before calibration can be applied automatically. + +The contour machine is no longer artifact-only. `MimirSynchronizationHub` can +now run a live complex-contour lane over rolling Float32 audio buffers when +`enableComplexContourRuntime` is true. In that mode Mimir emits the configured +`bioacousticWitnessProfileId` through the same canary-packet renderer used by +the contour tracker, loads `complexContourChannelModelPath`, reads ASIO-style +rolling buffers directly, and publishes `complex-contour` reports alongside the +existing audio sync reports. `--complex-contour-runtime-self-test` proves the +surface synthetically at 192 kHz: a `693.5` sample delay is recovered at about +`0.219 us` error from runtime-shaped rolling buffers. diff --git a/docs/code-algorithm-map.md b/docs/code-algorithm-map.md index 5c101cc..8ebd9be 100644 --- a/docs/code-algorithm-map.md +++ b/docs/code-algorithm-map.md @@ -882,6 +882,48 @@ Invariant: `MimirDirectPathChannelModel` is applied only when the caller supplies the path model; otherwise the tracker stays measurement-only. +### `MimirComplexContourRuntimeAnalyzer.cs` + +Owns: + +- The live runtime bridge from rolling ASIO-style audio buffers into the complex + contour/direct-path tracker. + +Blocks: + +- `MimirComplexContourRuntimeOptions`: selected witness profile, calibration + schedule start, and reference/candidate search radii. +- `MimirComplexContourRuntimeAnalyzer`: extracts Float32 mono windows from + `MimirRollingStreamBuffer`, maps the current runtime clock into the + configured bioacoustic witness timeline, runs `MimirComplexContourMatchedFilterBank` + on reference and candidate windows, applies any loaded path channel model, and + emits `MimirAudioSynchronizationReport` records with `evidence=complex-contour`. + +Invariant: + +- The live contour lane only runs when explicitly enabled. When enabled, the + runtime emitter and contour analyzer share the same contestant profile instead + of learning one song and playing another. + +### `MimirComplexContourChannelModelDocument.cs` + +Owns: + +- Runtime loading and source-path lookup for the persisted complex contour + channel model. + +Blocks: + +- `MimirComplexContourChannelModelDocument`: schema, provenance, and path list. +- `MimirComplexContourPathChannelModel`: ASIO reference/candidate channel pair, + usable band corrections, reflection taps, reliability, and conversion to + `MimirDirectPathChannelModel`. + +Invariant: + +- Persisted calibration is path-scoped. A model learned for + `asio-ch2->asio-ch1@192000` cannot silently correct another source pair. + ## Stage 9: Chirp-Bin Reference Timeline ### `MimirChirpBinTimeline.cs` diff --git a/notes/current-system-map.md b/notes/current-system-map.md index 282e78a..e1f7118 100644 --- a/notes/current-system-map.md +++ b/notes/current-system-map.md @@ -271,6 +271,17 @@ the fresh cardioid reaching about 0.202 us from the seed, but the stored cardioid still worsens to about 17.476 us. Treat the model as explicit calibration evidence, not default runtime authority, until more captures prove the path surface stable. +The complex contour receiver now has a live runtime lane instead of living only +inside BufferSmoke artifact replay. When `enableComplexContourRuntime` is true, +`MimirRuntime` emits the configured `bioacousticWitnessProfileId` through +`MimirBioacousticContestantRenderer`, `MimirSynchronizationHub` loads +`complexContourChannelModelPath`, and `MimirComplexContourRuntimeAnalyzer` +extracts Float32 windows from rolling audio buffers to publish +`evidence=complex-contour` reports. The synthetic runtime-shaped proof +`--complex-contour-runtime-self-test` at 192 kHz recovers a 693.5-sample delay +with about 0.219 us error from rolling buffers. The next real-world proof is to +run that lane through live Scarlett loopback and mics, not only stored ASIO +artifacts. The first actuator proof now exists: `faust/mimir_alignment_actuator.dsp` owns six channels of bounded fractional delay/gain controls for Faust/native DSP, and `Mimir.BufferSmoke --bioacoustic-actuator-self-test --sample-rate 48000 diff --git a/src/Mimir.BufferSmoke/Program.cs b/src/Mimir.BufferSmoke/Program.cs index 3c44b28..742b821 100644 --- a/src/Mimir.BufferSmoke/Program.cs +++ b/src/Mimir.BufferSmoke/Program.cs @@ -72,6 +72,13 @@ if (args.Any(arg => string.Equals(arg, "--complex-contour-tracker-self-test", St ParseDoubleOption(args, "--reflection-delay-samples", 29.0)); } +if (args.Any(arg => string.Equals(arg, "--complex-contour-runtime-self-test", StringComparison.OrdinalIgnoreCase))) +{ + return RunComplexContourRuntimeSelfTest( + ParseIntOption(args, "--sample-rate", 192_000), + ParseDoubleOption(args, "--delay-samples", 693.5)); +} + if (args.Any(arg => string.Equals(arg, "--perfect-machine-profile-smoke", StringComparison.OrdinalIgnoreCase))) { return RunPerfectMachineProfileSmoke(); @@ -1154,6 +1161,58 @@ static int RunComplexContourTrackerSelfTest(int sampleRate, double delaySamples, : 1; } +static int RunComplexContourRuntimeSelfTest(int sampleRate, double delaySamples) +{ + var renderer = new MimirBioacousticContestantRenderer(MimirBioacousticContestants.CanaryPacketTrill); + const double runtimeSeconds = 1.75; + var reference = renderer.RenderSequenceMonoFloat(runtimeSeconds - 0.5, sampleRate); + var candidate = ApplyFractionalDelay(reference, delaySamples); + var referenceBuffer = BuildSyntheticAudioBuffer("asio-ch2", sampleRate, reference); + var candidateBuffer = BuildSyntheticAudioBuffer("asio-ch1", sampleRate, candidate); + var analyzer = new MimirComplexContourRuntimeAnalyzer( + new MimirComplexContourRuntimeOptions( + ProfileId: MimirBioacousticContestants.CanaryPacketTrill.Id, + ScheduleStartSeconds: 0.5, + SearchRadiusSeconds: 0.020, + ReferenceSearchRadiusSeconds: 0.020)); + var report = analyzer.Analyze([referenceBuffer, candidateBuffer], "asio-ch2", runtimeSeconds).FirstOrDefault(); + if (report == null) + { + Console.WriteLine("complex-contour-runtime-self-test status=no-report"); + return 1; + } + + var errorUs = Math.Abs(report.FractionalDelaySamples - delaySamples) * 1_000_000.0 / sampleRate; + Console.WriteLine( + $"complex-contour-runtime-self-test delaySamples={report.FractionalDelaySamples:0.000000} expected={delaySamples:0.000000} errorUs={errorUs:0.000} confidence={report.Confidence:0.000} directHits={report.TimelineMatchedEvents}"); + return errorUs < 20.0 && report.TimelineMatchedEvents >= 8 ? 0 : 1; +} + +static MimirRollingStreamBuffer BuildSyntheticAudioBuffer(string sourceId, int sampleRate, float[] samples) +{ + var descriptor = new MimirStreamDescriptor(sourceId, MimirStreamKind.Audio, MimirStreamOrigin.LocalDevice); + var buffer = new MimirRollingStreamBuffer(descriptor, TimeSpan.FromSeconds(5)); + var bytes = new byte[samples.Length * sizeof(float)]; + Buffer.BlockCopy(samples, 0, bytes, 0, bytes.Length); + buffer.Append(new MimirStreamSample( + sourceId, + MimirStreamKind.Audio, + MimirStreamOrigin.LocalDevice, + TimestampNs: (long)Math.Round(samples.Length * 1_000_000_000.0 / sampleRate), + ArrivalNs: (long)Math.Round(samples.Length * 1_000_000_000.0 / sampleRate), + Sequence: 1, + PayloadHandle: 0, + ByteLength: bytes.Length, + Data: bytes, + AudioBlock: new MimirAudioBlockDescriptor( + sampleRate, + 1, + MimirAudioSampleFormat.Float32, + samples.Length, + 0))); + return buffer; +} + static int RunPerfectMachineProfileSmoke() { var profiles = MimirPerfectMachineProfiles.All; diff --git a/src/Mimir.Runtime/MimirRuntime.cs b/src/Mimir.Runtime/MimirRuntime.cs index db98c9d..ce4858b 100644 --- a/src/Mimir.Runtime/MimirRuntime.cs +++ b/src/Mimir.Runtime/MimirRuntime.cs @@ -24,6 +24,7 @@ public sealed class MimirRuntime : IAquariumRuntime private readonly float audioSyncUpdateIntervalSeconds; private readonly float calibrationGain; private readonly float watermarkGain; + private readonly MimirBioacousticContestantRenderer? complexContourWitness; private int lastPollCount; private float runtimeSeconds; private float nextAudioSyncSeconds; @@ -61,6 +62,12 @@ public sealed class MimirRuntime : IAquariumRuntime audioSyncUpdateIntervalSeconds = ParseAudioSyncIntervalSeconds(); calibrationGain = settings.Audio.CalibrationGain; watermarkGain = settings.Audio.WatermarkGain; + complexContourWitness = settings.Audio.EnableComplexContourRuntime + ? new MimirBioacousticContestantRenderer( + MimirBioacousticContestants.BuiltIn.FirstOrDefault(profile => + string.Equals(profile.Id, settings.Audio.BioacousticWitnessProfileId, StringComparison.OrdinalIgnoreCase)) + ?? MimirBioacousticContestants.CanaryPacketTrill) + : null; nextAudioSyncSeconds = audioSyncUpdateIntervalSeconds; nextTelemetrySeconds = telemetryIntervalSeconds; foreach (var source in streamSources) @@ -245,7 +252,7 @@ public sealed class MimirRuntime : IAquariumRuntime return $"{audioSyncSettings.ReferenceSourceId} {DescribeAudioSyncMode()} bioacoustic timeline {MimirBioacousticTimeline.SegmentSeconds:0.00}s segments emitted to {emittedUntilSeconds:0.00}s passiveConfidence={lastPassiveSynchronizationConfidence:0.000}"; } - private static string RenderCalibrationBatchPcm16Base64( + private string RenderCalibrationBatchPcm16Base64( ulong firstSegment, int segmentCount, MimirBioacousticSpeaker speaker, @@ -257,10 +264,16 @@ public sealed class MimirRuntime : IAquariumRuntime peak = 0.0f; for (var segment = 0; segment < segmentCount; segment++) { - var samples = MimirBioacousticTimeline.Default.RenderSegmentMonoFloat( - firstSegment + (ulong)segment, - MimirBioacousticTimeline.SampleRate, - speaker); + var samples = complexContourWitness == null + ? MimirBioacousticTimeline.Default.RenderSegmentMonoFloat( + firstSegment + (ulong)segment, + MimirBioacousticTimeline.SampleRate, + speaker) + : complexContourWitness.RenderSegmentMonoFloat( + firstSegment + (ulong)segment, + MimirBioacousticTimeline.SegmentSeconds, + MimirBioacousticTimeline.SampleRate, + speaker); for (var index = 0; index < samples.Length; index++) { peak = Math.Max(peak, Math.Abs(samples[index])); @@ -287,6 +300,10 @@ public sealed class MimirRuntime : IAquariumRuntime var stopwatch = Stopwatch.StartNew(); synchronization.AnalyzeAudioSynchronizationStep(audioSyncSettings.ReferenceSourceId, audioSyncSettings.Mode); + if (synchronization.ComplexContourRuntimeEnabled) + { + synchronization.AnalyzeComplexContourSynchronizationStep(audioSyncSettings.ReferenceSourceId, runtimeSeconds); + } stopwatch.Stop(); lastAudioSyncAnalysisMilliseconds = stopwatch.Elapsed.TotalMilliseconds; lastAudioSynchronizationReports = synchronization.AudioSynchronizationReports; @@ -313,6 +330,12 @@ public sealed class MimirRuntime : IAquariumRuntime $"mimir-sync-report {report.ReferenceSourceId}->{report.SourceId} evidence={report.EvidenceKind} delaySamples={report.FractionalDelaySamples:0.000000} delayUs={report.DelayMicroseconds:0.000} delayMs={report.DelayMilliseconds:0.000} confidence={report.Confidence:0.000} timelineEvents={report.TimelineMatchedEvents} timelineConfidence={report.TimelineConfidence:0.000}"); } + foreach (var report in synchronization.ComplexContourSynchronizationReports.OrderBy(report => report.SourceId, StringComparer.Ordinal)) + { + Console.WriteLine( + $"mimir-complex-contour-report {report.ReferenceSourceId}->{report.SourceId} delaySamples={report.FractionalDelaySamples:0.000000} delayUs={report.DelayMicroseconds:0.000} confidence={report.Confidence:0.000} directHits={report.TimelineMatchedEvents}"); + } + foreach (var state in states) { Console.WriteLine( diff --git a/src/Mimir.Runtime/Synchronization/MimirBioacousticContestant.cs b/src/Mimir.Runtime/Synchronization/MimirBioacousticContestant.cs index 64097c6..cc2d735 100644 --- a/src/Mimir.Runtime/Synchronization/MimirBioacousticContestant.cs +++ b/src/Mimir.Runtime/Synchronization/MimirBioacousticContestant.cs @@ -266,6 +266,32 @@ public sealed class MimirBioacousticContestantRenderer(MimirBioacousticContestan return samples; } + public float[] RenderSegmentMonoFloat(ulong segmentIndex, double segmentSeconds, int sampleRate, MimirBioacousticSpeaker? speaker = null) + { + var segmentStartSeconds = segmentIndex * segmentSeconds; + var samples = new float[Math.Max(1, (int)Math.Round(segmentSeconds * sampleRate))]; + var firstEvent = Math.Max(0, (long)Math.Floor((segmentStartSeconds - MimirBioacousticContestants.FirstEventSeconds - Profile.MotifDurationSeconds) / Profile.EventSpacingSeconds) - 2); + var lastEvent = Math.Max(firstEvent, (long)Math.Ceiling((segmentStartSeconds + segmentSeconds - MimirBioacousticContestants.FirstEventSeconds) / Profile.EventSpacingSeconds) + 2); + for (var eventIndex = firstEvent; eventIndex <= lastEvent; eventIndex++) + { + var eventStart = EventStartSeconds((ulong)eventIndex); + if (eventStart >= segmentStartSeconds + segmentSeconds || + eventStart + Profile.MotifDurationSeconds <= segmentStartSeconds) + { + continue; + } + + if (speaker != null && SpeakerForEvent((ulong)eventIndex) != speaker.Value) + { + continue; + } + + AddEvent(samples, sampleRate, (ulong)eventIndex, PayloadSymbolForEvent((ulong)eventIndex), eventStart, segmentStartSeconds); + } + + return samples; + } + public int ExpectedEventCount(double seconds) => Math.Max(0, (int)Math.Floor((seconds - MimirBioacousticContestants.FirstEventSeconds - Profile.MotifDurationSeconds) / Profile.EventSpacingSeconds) + 1); @@ -371,6 +397,9 @@ public sealed class MimirBioacousticContestantRenderer(MimirBioacousticContestan private static int SymbolForEvent(ulong eventIndex) => (int)((eventIndex * 73UL + eventIndex / 5UL * 19UL) % MimirBioacousticContestants.SymbolCount); + private static MimirBioacousticSpeaker SpeakerForEvent(ulong eventIndex) => + (eventIndex & 1UL) == 0UL ? MimirBioacousticSpeaker.Left : MimirBioacousticSpeaker.Right; + private double RootForSymbol(int symbolId) { var t = symbolId / (double)(MimirBioacousticContestants.SymbolCount - 1); diff --git a/src/Mimir.Runtime/Synchronization/MimirComplexContourChannelModelDocument.cs b/src/Mimir.Runtime/Synchronization/MimirComplexContourChannelModelDocument.cs new file mode 100644 index 0000000..00931a9 --- /dev/null +++ b/src/Mimir.Runtime/Synchronization/MimirComplexContourChannelModelDocument.cs @@ -0,0 +1,86 @@ +using System.Text.Json; + +namespace Mimir.Runtime.Synchronization; + +public sealed record MimirComplexContourChannelModelDocument( + string Schema, + DateTimeOffset CreatedUtc, + string SourceReceiptPath, + IReadOnlyList Paths) +{ + private static readonly JsonSerializerOptions JsonOptions = new() + { + PropertyNameCaseInsensitive = true, + }; + + public static MimirComplexContourChannelModelDocument Load(string path) => + JsonSerializer.Deserialize( + File.ReadAllText(path), + JsonOptions) ?? new MimirComplexContourChannelModelDocument( + "mimir.bioacoustic.complex-contour-channel-model.v1", + DateTimeOffset.MinValue, + "", + []); + + public MimirComplexContourPathChannelModel? PathFor(string referenceSourceId, string sourceId, int sampleRate) + { + var referenceChannel = ParseAsioChannel(referenceSourceId); + var candidateChannel = ParseAsioChannel(sourceId); + if (referenceChannel == null || candidateChannel == null) + { + return null; + } + + return Paths.FirstOrDefault(path => + path.SampleRate == sampleRate && + path.ReferenceChannel == referenceChannel.Value && + path.CandidateChannel == candidateChannel.Value); + } + + private static int? ParseAsioChannel(string sourceId) + { + const string prefix = "asio-ch"; + return sourceId.StartsWith(prefix, StringComparison.OrdinalIgnoreCase) && + int.TryParse(sourceId[prefix.Length..], out var channel) + ? channel + : null; + } +} + +public sealed record MimirComplexContourPathChannelModel( + string PathId, + int SampleRate, + int ReferenceChannel, + int CandidateChannel, + IReadOnlyList CaseIds, + IReadOnlyList Corrections, + IReadOnlyList ReflectionTaps, + int UsableBandCount, + double Reliability, + double DelaySpreadSamples) +{ + public MimirDirectPathChannelModel ToRuntimeModel() => + new(Corrections + .Where(correction => correction.Usable) + .Select(correction => new MimirDirectPathBandCorrection( + correction.CenterHz, + correction.DelayCorrectionSamples, + correction.PhaseCorrectionRadians, + correction.Weight * correction.Reliability)) + .ToArray()); +} + +public sealed record MimirComplexContourBandCorrection( + double CenterHz, + double DelayCorrectionSamples, + double PhaseCorrectionRadians, + double Weight, + int ObservationCount, + double DelayStdDevSamples, + double Reliability, + bool Usable); + +public sealed record MimirComplexContourReflectionCorrection( + double RelativeDelaySamples, + int ObservationCount, + double MeanRelativeDelaySamples); diff --git a/src/Mimir.Runtime/Synchronization/MimirComplexContourRuntimeAnalyzer.cs b/src/Mimir.Runtime/Synchronization/MimirComplexContourRuntimeAnalyzer.cs new file mode 100644 index 0000000..321fdd3 --- /dev/null +++ b/src/Mimir.Runtime/Synchronization/MimirComplexContourRuntimeAnalyzer.cs @@ -0,0 +1,205 @@ +using System.Buffers.Binary; + +namespace Mimir.Runtime.Synchronization; + +public sealed record MimirComplexContourRuntimeOptions( + string ProfileId = "canary-packet-trill", + double ScheduleStartSeconds = 0.5, + double SearchRadiusSeconds = 0.080, + double ReferenceSearchRadiusSeconds = 0.120); + +public sealed class MimirComplexContourRuntimeAnalyzer( + MimirComplexContourRuntimeOptions? options = null, + MimirComplexContourChannelModelDocument? channelModel = null) +{ + private const double MaxWindowSeconds = 1.25; + private readonly MimirComplexContourRuntimeOptions options = options ?? new(); + private readonly MimirBioacousticContestantProfile profile = MimirBioacousticContestants.BuiltIn.FirstOrDefault(profile => + string.Equals(profile.Id, (options ?? new MimirComplexContourRuntimeOptions()).ProfileId, StringComparison.OrdinalIgnoreCase)) + ?? MimirBioacousticContestants.CanaryPacketTrill; + private readonly MimirComplexContourChannelModelDocument? channelModel = channelModel; + private readonly Dictionary banks = []; + + public IReadOnlyList Analyze( + IEnumerable buffers, + string referenceSourceId, + double runtimeSeconds, + IReadOnlyDictionary? predictedStates = null, + IReadOnlySet? candidateSourceIds = null) + { + var audioBuffers = buffers + .Where(buffer => buffer.Descriptor.Kind == MimirStreamKind.Audio) + .ToArray(); + var reference = audioBuffers.FirstOrDefault(buffer => + string.Equals(buffer.Descriptor.SourceId, referenceSourceId, StringComparison.Ordinal)); + if (reference == null || reference.Latest?.AudioBlock == null) + { + return []; + } + + var referenceSamples = ExtractFloatMonoWindow(reference, out var referenceBlock); + if (referenceSamples.Length == 0 || referenceBlock == null) + { + return []; + } + + var reports = new List(); + foreach (var buffer in audioBuffers) + { + if (ReferenceEquals(buffer, reference) || + candidateSourceIds != null && !candidateSourceIds.Contains(buffer.Descriptor.SourceId)) + { + continue; + } + + var candidateSamples = ExtractFloatMonoWindow(buffer, out var candidateBlock); + if (candidateSamples.Length == 0 || candidateBlock == null) + { + continue; + } + + if (candidateBlock.SampleRate != referenceBlock.SampleRate) + { + continue; + } + + var compared = Math.Min(referenceSamples.Length, candidateSamples.Length); + if (compared < referenceBlock.SampleRate * profile.MotifDurationSeconds * 2.0) + { + continue; + } + + var sampleRate = referenceBlock.SampleRate; + var referenceWindow = referenceSamples.AsSpan(^compared..).ToArray(); + var candidateWindow = candidateSamples.AsSpan(^compared..).ToArray(); + var captureDurationSeconds = compared / (double)sampleRate; + var windowStartSeconds = Math.Max( + 0.0, + runtimeSeconds - options.ScheduleStartSeconds - captureDurationSeconds - options.ReferenceSearchRadiusSeconds); + var windowEndSeconds = Math.Max(0.0, runtimeSeconds - options.ScheduleStartSeconds) + options.ReferenceSearchRadiusSeconds; + var firstEvent = Math.Max(0, (long)Math.Floor((windowStartSeconds - MimirBioacousticContestants.FirstEventSeconds - profile.MotifDurationSeconds) / profile.EventSpacingSeconds) - 2); + var lastEvent = Math.Max(firstEvent, (long)Math.Ceiling((windowEndSeconds - MimirBioacousticContestants.FirstEventSeconds) / profile.EventSpacingSeconds) + 2); + var eventIndices = Enumerable.Range((int)firstEvent, (int)(lastEvent - firstEvent + 1)) + .Select(index => (ulong)index) + .ToArray(); + if (eventIndices.Length == 0) + { + continue; + } + + var predictedDelay = predictedStates != null && + predictedStates.TryGetValue(buffer.Descriptor.SourceId, out var state) + ? state.SmoothedDelaySamples + : 0.0; + var sourceOffset = -windowStartSeconds * sampleRate; + var bank = BankFor(sampleRate); + var referenceHits = bank.AnalyzeEvents( + referenceWindow, + eventIndices, + sourceOffset, + Math.Max(8, (int)Math.Round(options.ReferenceSearchRadiusSeconds * sampleRate))); + var candidateHits = bank.AnalyzeEvents( + candidateWindow, + eventIndices, + sourceOffset + predictedDelay, + Math.Max(16, (int)Math.Round(options.SearchRadiusSeconds * sampleRate))); + var model = channelModel? + .PathFor(referenceSourceId, buffer.Descriptor.SourceId, sampleRate)? + .ToRuntimeModel(); + var estimate = new MimirDirectPathTracker( + sampleRate, + new MimirDirectPathTrackerOptions( + PredictionGateSamples: Math.Max(16.0, options.SearchRadiusSeconds * sampleRate), + ChannelModel: model)).Update(referenceHits, candidateHits, predictedDelay); + if (estimate == null) + { + continue; + } + + reports.Add(new MimirAudioSynchronizationReport( + referenceSourceId, + buffer.Descriptor.SourceId, + sampleRate, + (int)Math.Round(estimate.DelaySamples), + estimate.DelaySamples, + estimate.DelaySamples * 1000.0 / sampleRate, + estimate.Confidence, + estimate.BandObservations + .GroupBy(observation => Math.Round(observation.CenterHz / 250.0) * 250.0) + .Select(group => new MimirChirpletBandResponse( + group.Key, + group.Sum(observation => observation.Weight))) + .OrderBy(response => response.CenterHz) + .ToArray(), + Math.Min(reference.Latest?.TimestampNs ?? 0, buffer.Latest?.TimestampNs ?? 0), + compared, + reference.Latest?.Sequence ?? 0, + buffer.Latest?.Sequence ?? 0, + estimate.DirectHitCount, + estimate.Confidence, + "complex-contour")); + } + + return reports; + } + + private MimirComplexContourMatchedFilterBank BankFor(int sampleRate) + { + if (!banks.TryGetValue(sampleRate, out var bank)) + { + bank = new MimirComplexContourMatchedFilterBank(new MimirBioacousticContestantRenderer(profile), sampleRate); + banks[sampleRate] = bank; + } + + return bank; + } + + private static float[] ExtractFloatMonoWindow(MimirRollingStreamBuffer buffer, out MimirAudioBlockDescriptor? latestBlock) + { + latestBlock = buffer.Latest?.AudioBlock; + if (latestBlock == null) + { + return []; + } + + var maxSamples = Math.Max(1, (int)Math.Ceiling(latestBlock.SampleRate * MaxWindowSeconds)); + var samples = new List(maxSamples); + foreach (var sample in buffer.Snapshot() + .Where(sample => sample.AudioBlock is { SampleFormat: MimirAudioSampleFormat.Float32 } && !sample.Data.IsEmpty) + .Reverse()) + { + var block = sample.AudioBlock!; + var mono = ExtractFirstFloatChannel(sample.Data.Span, block.Channels); + for (var index = mono.Length - 1; index >= 0 && samples.Count < maxSamples; index--) + { + samples.Add(mono[index]); + } + + if (samples.Count >= maxSamples) + { + break; + } + } + + samples.Reverse(); + return samples.ToArray(); + } + + private static float[] ExtractFirstFloatChannel(ReadOnlySpan data, int channels) + { + if (channels <= 0) + { + return []; + } + + var frames = data.Length / (sizeof(float) * channels); + var output = new float[frames]; + for (var frame = 0; frame < frames; frame++) + { + var offset = frame * channels * sizeof(float); + output[frame] = BitConverter.Int32BitsToSingle(BinaryPrimitives.ReadInt32LittleEndian(data.Slice(offset, sizeof(float)))); + } + + return output; + } +} diff --git a/src/Mimir.Runtime/Synchronization/MimirRuntimeConfiguration.cs b/src/Mimir.Runtime/Synchronization/MimirRuntimeConfiguration.cs index 550e643..ccdc113 100644 --- a/src/Mimir.Runtime/Synchronization/MimirRuntimeConfiguration.cs +++ b/src/Mimir.Runtime/Synchronization/MimirRuntimeConfiguration.cs @@ -221,6 +221,12 @@ public sealed class MimirAudioSyncConfig public string CalibrationModelPath { get; set; } = ""; + public string ComplexContourChannelModelPath { get; set; } = ""; + + public string BioacousticWitnessProfileId { get; set; } = ""; + + public bool EnableComplexContourRuntime { get; set; } + public MimirAudioSynchronizationSettings ToSettings(MimirAudioSynchronizationSettings fallback) { return new MimirAudioSynchronizationSettings @@ -238,6 +244,13 @@ public sealed class MimirAudioSyncConfig CalibrationModelPath = string.IsNullOrWhiteSpace(CalibrationModelPath) ? fallback.CalibrationModelPath : CalibrationModelPath.Trim(), + ComplexContourChannelModelPath = string.IsNullOrWhiteSpace(ComplexContourChannelModelPath) + ? fallback.ComplexContourChannelModelPath + : ComplexContourChannelModelPath.Trim(), + BioacousticWitnessProfileId = string.IsNullOrWhiteSpace(BioacousticWitnessProfileId) + ? fallback.BioacousticWitnessProfileId + : BioacousticWitnessProfileId.Trim(), + EnableComplexContourRuntime = EnableComplexContourRuntime || fallback.EnableComplexContourRuntime, }; } } diff --git a/src/Mimir.Runtime/Synchronization/MimirSynchronizationHub.cs b/src/Mimir.Runtime/Synchronization/MimirSynchronizationHub.cs index 5936c02..4a16aed 100644 --- a/src/Mimir.Runtime/Synchronization/MimirSynchronizationHub.cs +++ b/src/Mimir.Runtime/Synchronization/MimirSynchronizationHub.cs @@ -4,8 +4,10 @@ public sealed class MimirSynchronizationHub : IDisposable { private readonly List sources = []; private readonly MimirAudioSynchronizationAnalyzer audioSynchronization = new(); + private readonly MimirComplexContourRuntimeAnalyzer? complexContourSynchronization; private readonly MimirAudioSynchronizationStateTracker audioSynchronizationState = new(); private readonly Dictionary audioSynchronizationReports = new(StringComparer.Ordinal); + private readonly Dictionary complexContourReports = new(StringComparer.Ordinal); private readonly MimirChirpBinCalibrationModel? chirpBinCalibrationModel; private ulong ingestedSamples; private int nextAudioSynchronizationCandidate; @@ -15,6 +17,11 @@ public sealed class MimirSynchronizationHub : IDisposable Settings = settings; Buffers = new MimirStreamBufferSet(settings.BufferDuration); chirpBinCalibrationModel = TryLoadCalibrationModel(settings.Audio.CalibrationModelPath); + complexContourSynchronization = settings.Audio.EnableComplexContourRuntime + ? new MimirComplexContourRuntimeAnalyzer( + new MimirComplexContourRuntimeOptions(settings.Audio.BioacousticWitnessProfileId), + TryLoadComplexContourChannelModel(settings.Audio.ComplexContourChannelModelPath)) + : null; foreach (var stream in settings.Streams.Where(stream => stream.Enabled)) { Buffers.EnsureBuffer(stream); @@ -35,6 +42,9 @@ public sealed class MimirSynchronizationHub : IDisposable public IReadOnlyList AudioSynchronizationReports => audioSynchronizationReports.Values.OrderBy(report => report.SourceId, StringComparer.Ordinal).ToArray(); + public IReadOnlyList ComplexContourSynchronizationReports => + complexContourReports.Values.OrderBy(report => report.SourceId, StringComparer.Ordinal).ToArray(); + public IReadOnlyList AudioSynchronizationDecodeTraces => audioSynchronization.LastDecodeTraces; @@ -44,6 +54,8 @@ public sealed class MimirSynchronizationHub : IDisposable public MimirChirpBinCodebookPlan? ChirpBinEmissionPlan => chirpBinCalibrationModel?.EmissionPlan; + public bool ComplexContourRuntimeEnabled => complexContourSynchronization != null; + public void AddSource(IMimirStreamSource source) { ObjectDisposedException.ThrowIf(disposed, this); @@ -131,6 +143,53 @@ public sealed class MimirSynchronizationHub : IDisposable return reports; } + public IReadOnlyList AnalyzeComplexContourSynchronizationStep( + string referenceSourceId, + double runtimeSeconds, + int maxCandidates = 1) + { + ObjectDisposedException.ThrowIf(disposed, this); + if (complexContourSynchronization == null) + { + return []; + } + + var candidates = Buffers.Buffers + .Where(buffer => + buffer.Descriptor.Kind == MimirStreamKind.Audio && + !string.Equals(buffer.Descriptor.SourceId, referenceSourceId, StringComparison.Ordinal) && + buffer.Latest?.AudioBlock is { SampleFormat: MimirAudioSampleFormat.Float32 }) + .OrderBy(buffer => buffer.Descriptor.SourceId, StringComparer.Ordinal) + .Select(buffer => buffer.Descriptor.SourceId) + .ToArray(); + if (candidates.Length == 0) + { + return []; + } + + var selected = new HashSet(StringComparer.Ordinal); + for (var count = 0; count < Math.Min(maxCandidates, candidates.Length); count++) + { + selected.Add(candidates[nextAudioSynchronizationCandidate % candidates.Length]); + nextAudioSynchronizationCandidate++; + } + + var predicted = audioSynchronizationState.States.ToDictionary(state => state.SourceId, StringComparer.Ordinal); + var reports = complexContourSynchronization.Analyze( + Buffers.Buffers, + referenceSourceId, + runtimeSeconds, + predicted, + selected); + StoreComplexContourReports(reports, selected); + if (reports.Count > 0) + { + audioSynchronizationState.Update(reports); + } + + return reports; + } + private bool disposed; private static MimirChirpBinCalibrationModel? TryLoadCalibrationModel(string path) @@ -143,6 +202,16 @@ public sealed class MimirSynchronizationHub : IDisposable return MimirChirpBinCalibrationModel.Load(path); } + private static MimirComplexContourChannelModelDocument? TryLoadComplexContourChannelModel(string path) + { + if (string.IsNullOrWhiteSpace(path) || !File.Exists(path)) + { + return null; + } + + return MimirComplexContourChannelModelDocument.Load(path); + } + private void StoreAudioSynchronizationReports( IReadOnlyList reports, IEnumerable analyzedCandidateIds) @@ -172,6 +241,26 @@ public sealed class MimirSynchronizationHub : IDisposable } } + private void StoreComplexContourReports( + IReadOnlyList reports, + IEnumerable analyzedCandidateIds) + { + var reportIds = new HashSet(reports.Select(report => report.SourceId), StringComparer.Ordinal); + foreach (var sourceId in analyzedCandidateIds) + { + if (!reportIds.Contains(sourceId)) + { + complexContourReports.Remove(sourceId); + } + } + + foreach (var report in reports) + { + complexContourReports[report.SourceId] = report; + audioSynchronizationReports[report.SourceId] = report; + } + } + public void Dispose() { if (disposed) diff --git a/src/Mimir.Runtime/Synchronization/MimirSynchronizationSettings.cs b/src/Mimir.Runtime/Synchronization/MimirSynchronizationSettings.cs index 4065924..8a5e054 100644 --- a/src/Mimir.Runtime/Synchronization/MimirSynchronizationSettings.cs +++ b/src/Mimir.Runtime/Synchronization/MimirSynchronizationSettings.cs @@ -68,6 +68,12 @@ public sealed class MimirAudioSynchronizationSettings public string CalibrationModelPath { get; init; } = ""; + public string ComplexContourChannelModelPath { get; init; } = ""; + + public string BioacousticWitnessProfileId { get; init; } = MimirBioacousticContestants.CurrentBirdcall.Id; + + public bool EnableComplexContourRuntime { get; init; } + public static MimirAudioSynchronizationSettings FromEnvironment() { return new MimirAudioSynchronizationSettings().WithEnvironmentOverrides(); @@ -82,6 +88,9 @@ public sealed class MimirAudioSynchronizationSettings CalibrationGain = ReadCalibrationGain(CalibrationGain), WatermarkGain = ReadWatermarkGain(WatermarkGain), CalibrationModelPath = ReadCalibrationModelPath(CalibrationModelPath), + ComplexContourChannelModelPath = ReadComplexContourChannelModelPath(ComplexContourChannelModelPath), + BioacousticWitnessProfileId = ReadBioacousticWitnessProfileId(BioacousticWitnessProfileId), + EnableComplexContourRuntime = ReadEnableComplexContourRuntime(EnableComplexContourRuntime), }; } @@ -128,6 +137,26 @@ public sealed class MimirAudioSynchronizationSettings var path = Environment.GetEnvironmentVariable("MIMIR_CHIRP_BIN_CALIBRATION"); return string.IsNullOrWhiteSpace(path) ? fallback : path.Trim(); } + + private static string ReadComplexContourChannelModelPath(string fallback) + { + var path = Environment.GetEnvironmentVariable("MIMIR_COMPLEX_CONTOUR_CHANNEL_MODEL"); + return string.IsNullOrWhiteSpace(path) ? fallback : path.Trim(); + } + + private static string ReadBioacousticWitnessProfileId(string fallback) + { + var profile = Environment.GetEnvironmentVariable("MIMIR_BIOACOUSTIC_PROFILE"); + return string.IsNullOrWhiteSpace(profile) ? fallback : profile.Trim(); + } + + private static bool ReadEnableComplexContourRuntime(bool fallback) + { + var raw = Environment.GetEnvironmentVariable("MIMIR_COMPLEX_CONTOUR_RUNTIME"); + return string.IsNullOrWhiteSpace(raw) + ? fallback + : raw is "1" or "true" or "TRUE" or "yes" or "YES"; + } } public enum MimirAudioSyncMode diff --git a/state/evidence.jsonl b/state/evidence.jsonl index 6b33524..59a6cc7 100644 --- a/state/evidence.jsonl +++ b/state/evidence.jsonl @@ -105,3 +105,4 @@ {"ts": "2026-05-25T02:42:00+01:00", "type": "implementation", "status": "partial", "note": "Extended the complex contour tracker with per-band phase/delay residual observations and a MimirDirectPathChannelModel correction surface. BufferSmoke replay panel now prints strongest band residuals and phase MAE: stored/fresh shotgun phase MAE is 0.617/0.852 rad, stored/fresh cardioid phase MAE is 0.687/0.657 rad. A same-window correction experiment improved one case but worsened others, so it was not promoted; channel correction must be learned across time/captures before it can influence live timing authority."} {"ts": "2026-05-25T02:53:00+01:00", "type": "verification", "status": "ok", "note": "Added persisted complex contour replay receipt generation. `dotnet run -c Release --no-build --project src/Mimir.BufferSmoke/Mimir.BufferSmoke.csproj -- --complex-contour-replay-panel --output calibration/bioacoustic/complex-contour-replay-panel.json` writes the four stored/fresh shotgun/cardioid cases with delay error, phase MAE, reflection taps, and strongest band residuals. Build passes and synthetic 192 kHz reflection smoke still reports 5.249 us error."} {"ts": "2026-05-25T03:40:00+01:00", "type": "implementation", "status": "partial", "note": "Built explicit complex contour channel learning and evaluation. `--learn-complex-contour-channel-model` learns path-scoped per-band delay/phase correction and reflection taps from `calibration/bioacoustic/complex-contour-replay-panel.json`, writing `calibration/bioacoustic/complex-contour-channel-model.json`. `MimirDirectPathTracker` now applies supplied channel-model phase/group-delay correction and downweights bands outside the usable learned surface. `--evaluate-complex-contour-channel-model` writes `calibration/bioacoustic/complex-contour-channel-model-evaluation.json`: current model improves 3/4 absolute path-seed errors, mean absolute path error by 2.137 us, and mean cluster MAE by 0.713 samples; fresh cardioid reaches 0.202 us from seed, but stored cardioid worsens from 15.817 us to 17.476 us. Treat the model as explicit calibration evidence, not default runtime authority, until more captures prove stability."} +{"ts": "2026-05-25T04:25:00+01:00", "type": "implementation", "status": "partial", "note": "Moved the complex contour receiver from artifact-only replay into runtime shape. `MimirComplexContourRuntimeAnalyzer` reads Float32 samples from `MimirRollingStreamBuffer`, maps runtime time into the configured bioacoustic witness profile, runs the complex contour matched-filter/direct-path tracker, applies path-scoped `MimirComplexContourChannelModelDocument` corrections, and emits `evidence=complex-contour` synchronization reports. `MimirRuntime` now emits `MimirBioacousticContestantRenderer` audio when `enableComplexContourRuntime` is true, so the live sound and contour decoder share one song profile. Synthetic proof: `--complex-contour-runtime-self-test` at 192 kHz recovers a 693.5-sample delay with 0.219 us error from runtime-shaped rolling buffers. Real Scarlett loopback/mic proof remains the next gate."} diff --git a/state/map.yaml b/state/map.yaml index 14dbc87..0dd4845 100644 --- a/state/map.yaml +++ b/state/map.yaml @@ -239,6 +239,13 @@ accepted_design: direct-cluster MAE; the fresh cardioid case reaches about 0.202 us from seed, while the stored cardioid case still worsens to about 17.476 us. The model is explicit calibration evidence, not default runtime authority. + The complex contour receiver is now wired into runtime as an explicit live + lane: when `enableComplexContourRuntime` is true, Mimir emits the configured + `bioacousticWitnessProfileId` through `MimirBioacousticContestantRenderer`, + loads `complexContourChannelModelPath`, extracts Float32 windows from the + rolling buffers, and publishes `evidence=complex-contour` reports. The + synthetic runtime-shaped proof `--complex-contour-runtime-self-test` at + 192 kHz recovers a 693.5-sample delay with about 0.219 us error. `calibration/bioacoustic/complex-contour-replay-panel.json` persists the current four-case replay receipt. Acoustic robustness remains open.