diff --git a/docs/audio-field.md b/docs/audio-field.md index ba42599..b3e2924 100644 --- a/docs/audio-field.md +++ b/docs/audio-field.md @@ -139,6 +139,15 @@ before selecting the coherent anchor path. If the measured path only leaves a smaller alphabet, Mimir should use that reliable symbol set and raise the sequence order so the timeline remains unique without pretending dead bands still carry code. +The first concrete two-mic cancellation owner is +`MimirCalibratedAudioCompositeBuilder`: after fractional alignment and measured +band correction, it scores calibrated-band coherence across mics, suppresses +incoherent measured-band components, then performs the confidence/noise-weighted +sum. `Mimir.BufferSmoke --dual-mic-coherence-cancellation-smoke` proves the +contract on synthetic shotgun/cardioid witnesses: coherent shared tones survive, +a mic-local `3.52 kHz` interferer is suppressed, and SNR rises from `1.366 dB` +to `16.304 dB`. This is still the C# diagnostic/offline contract; Faust/native +DSP owns the production hot lane. The synthetic invariant is `dotnet run --project .\src\Mimir.BufferSmoke\Mimir.BufferSmoke.csproj -- --bioacoustic-self-test`; it renders the bioacoustic timeline, decodes direct word anchors, and requires a diff --git a/docs/implementation-plan.md b/docs/implementation-plan.md index a11d0db..46b637b 100644 --- a/docs/implementation-plan.md +++ b/docs/implementation-plan.md @@ -810,9 +810,13 @@ a named invariant that the native runtime cannot protect yet. stem generation into Faust/native DSP. The C# diagnostic bridge now exists as `MimirCalibratedAudioCompositeBuilder`: it consumes synchronization states and measured band responses, emits a corrected composite plus scores, - and is proved by `--calibrated-audio-composite-smoke`. Treat that as the - contract and offline scorer for the native/Faust implementation, not as a - parallel live mixer authority. The next matching probe loop also exists as + and is proved by `--calibrated-audio-composite-smoke`. It now also scores + calibrated-band coherence and suppresses incoherent measured-band components + before summing; `--dual-mic-coherence-cancellation-smoke` proves the two-mic + case by cutting a mic-local 3.52 kHz interferer while preserving coherent + shared tones. Treat that as the contract and offline scorer for the + native/Faust implementation, not as a parallel live mixer authority. The + next matching probe loop also exists as `MimirTargetedBioacousticProbePlanner`: residuals and mic pathology notes select gain-bounded shaped probe bands for the bioacoustic renderer. Start with the 192 kHz shotgun/cardioid pair; only widen to every connected mic diff --git a/notes/current-system-map.md b/notes/current-system-map.md index 2083ab3..c549231 100644 --- a/notes/current-system-map.md +++ b/notes/current-system-map.md @@ -302,14 +302,21 @@ artifacts. turning that acoustic evidence into a corrected composite stream. It consumes sample windows plus `MimirAudioSynchronizationState` delay/confidence and per-band response evidence, applies fractional alignment, sparse measured-band -equalization, and confidence/noise-weighted summing, then emits composite -samples plus source flatness/weight reports. This is not the hot room-DSP +equalization, calibrated-band coherence scoring, incoherent-band suppression, +and confidence/noise-weighted summing, then emits composite samples plus source +flatness/weight/coherence reports and per-band coherence/suppression reports. +This is not the hot room-DSP authority and it does not make room nulls disappear; Faust/native DSP still owns the production alignment, suppression, separation, and stem lane. `Mimir.BufferSmoke --calibrated-audio-composite-smoke` proves the contract on synthetic delayed, colored, noisy mic paths: the uncalibrated mix measures -3.171 dB SNR, the calibrated composite measures 15.135 dB SNR, and response flatness improves from 0.548 to 1.000. +`Mimir.BufferSmoke --dual-mic-coherence-cancellation-smoke` proves the +two-witness cancellation cut: shared tones keep about 0.999 coherence, a +mic-local 3.52 kHz interferer scores about 0.001 coherence and 0.948 +suppression, its measured energy falls from 0.039195 to 0.001306, and SNR rises +from 1.366 dB to 16.304 dB. `MimirTargetedBioacousticProbePlanner` is the matching "ask the room a better question" owner. It consumes per-source spectral residuals and optional source pathology notes, then emits gain-bounded shaped probe bands for the diff --git a/src/Mimir.BufferSmoke/Program.cs b/src/Mimir.BufferSmoke/Program.cs index c65381a..f4e7b3e 100644 --- a/src/Mimir.BufferSmoke/Program.cs +++ b/src/Mimir.BufferSmoke/Program.cs @@ -80,6 +80,11 @@ if (args.Any(arg => string.Equals(arg, "--calibrated-audio-composite-smoke", Str return RunCalibratedAudioCompositeSmoke(); } +if (args.Any(arg => string.Equals(arg, "--dual-mic-coherence-cancellation-smoke", StringComparison.OrdinalIgnoreCase))) +{ + return RunDualMicCoherenceCancellationSmoke(); +} + if (args.Any(arg => string.Equals(arg, "--targeted-bioacoustic-probe-smoke", StringComparison.OrdinalIgnoreCase))) { return RunTargetedBioacousticProbeSmoke(); @@ -3411,6 +3416,63 @@ static int RunCalibratedAudioCompositeSmoke() : 1; } +static int RunDualMicCoherenceCancellationSmoke() +{ + const int sampleRate = 48_000; + const int frameCount = sampleRate; + const double poisonHz = 3_520.0; + var targetBands = new[] { 440.0, 880.0, 1_760.0 }; + var measuredBands = new[] { 440.0, 880.0, 1_760.0, poisonHz }; + var reference = RenderCalibrationToneStack(frameCount, sampleRate, targetBands, [1.0, 0.85, 0.70], noiseScale: 0.0, seed: 31); + var micA = reference.ToArray(); + var micB = reference.ToArray(); + AddSine(micA, sampleRate, poisonHz, amplitude: 0.080, phaseRadians: 0.30); + AddSine(micB, sampleRate, 250.0, amplitude: 0.018, phaseRadians: 1.70); + micA = ApplyFractionalDelay(micA, 9.25); + micB = ApplyFractionalDelay(micB, -6.50); + + var uncalibrated = AverageSignals([micA, micB]); + var result = new MimirCalibratedAudioCompositeBuilder(new MimirCalibratedAudioCompositeOptions( + MinimumConfidence: 0.05, + TargetBandEnergy: 1.0, + MinimumBandEnergy: 0.001, + MinimumBandGain: 0.90, + MaximumBandGain: 1.10, + NoiseFloorRms: 1.0e-5, + MinimumCoherence: 0.55, + IncoherentBandSuppression: 0.95)).Build( + [ + NewCompositeSource("shotgun", micA, 9.25, 0.96, measuredBands, [1.0, 1.0, 1.0, 1.0], 0.020), + NewCompositeSource("cardioid", micB, -6.50, 0.96, measuredBands, [1.0, 1.0, 1.0, 1.0], 0.014), + ]); + var uncalibratedSnr = EstimateSnrDb(reference, uncalibrated); + var cancelledSnr = EstimateSnrDb(reference, result.Samples); + var uncalibratedPoison = ToneEnergy(uncalibrated, sampleRate, poisonHz); + var cancelledPoison = ToneEnergy(result.Samples, sampleRate, poisonHz); + var poisonBand = result.BandReports + .OrderBy(report => Math.Abs(report.CenterHz - poisonHz)) + .FirstOrDefault(); + var sharedCoherence = result.BandReports + .Where(report => targetBands.Any(targetHz => Math.Abs(report.CenterHz - targetHz) < 1.0)) + .Select(report => report.Coherence) + .DefaultIfEmpty(0.0) + .Average(); + Console.WriteLine( + $"dual-mic-coherence-cancellation-smoke sources={result.SourceReports.Count} bands={result.BandReports.Count} " + + $"uncalibratedSnrDb={uncalibratedSnr:0.000} cancelledSnrDb={cancelledSnr:0.000} " + + $"poisonBefore={uncalibratedPoison:0.000000} poisonAfter={cancelledPoison:0.000000} " + + $"poisonCoherence={poisonBand?.Coherence:0.000} poisonSuppression={poisonBand?.Suppression:0.000} sharedCoherence={sharedCoherence:0.000}"); + + return result.SourceReports.Count == 2 && + result.BandReports.Count >= measuredBands.Length && + poisonBand is { Coherence: < 0.45, Suppression: > 0.15 } && + sharedCoherence > 0.80 && + cancelledPoison < uncalibratedPoison * 0.70 && + cancelledSnr > uncalibratedSnr + 3.0 + ? 0 + : 1; +} + static int RunTargetedBioacousticProbeSmoke() { var residuals = new[] @@ -4398,6 +4460,15 @@ static float[] RenderCalibrationToneStack( return samples; } +static void AddSine(float[] samples, int sampleRate, double centerHz, double amplitude, double phaseRadians) +{ + var omega = 2.0 * Math.PI * centerHz / sampleRate; + for (var index = 0; index < samples.Length; index++) + { + samples[index] += (float)(Math.Sin(omega * index + phaseRadians) * amplitude); + } +} + static float[] AverageSignals(IReadOnlyList signals) { var frameCount = signals.Min(signal => signal.Length); diff --git a/src/Mimir.Runtime/Synchronization/MimirCalibratedAudioComposite.cs b/src/Mimir.Runtime/Synchronization/MimirCalibratedAudioComposite.cs index 9e1dfd2..b230572 100644 --- a/src/Mimir.Runtime/Synchronization/MimirCalibratedAudioComposite.cs +++ b/src/Mimir.Runtime/Synchronization/MimirCalibratedAudioComposite.cs @@ -6,7 +6,9 @@ public sealed record MimirCalibratedAudioCompositeOptions( double MinimumBandEnergy = 0.01, double MinimumBandGain = 0.20, double MaximumBandGain = 4.00, - double NoiseFloorRms = 1.0e-5); + double NoiseFloorRms = 1.0e-5, + double MinimumCoherence = 0.35, + double IncoherentBandSuppression = 0.72); public sealed record MimirCalibratedAudioCompositeSource( string SourceId, @@ -25,12 +27,20 @@ public sealed record MimirCalibratedAudioCompositeSourceReport( double ResponseFlatnessBefore, double ResponseFlatnessAfter, double Confidence, - double NoiseRms); + double NoiseRms, + double AverageCoherence); + +public sealed record MimirCalibratedAudioCompositeBandReport( + double CenterHz, + double Coherence, + double Suppression, + int SourceCount); public sealed record MimirCalibratedAudioCompositeResult( int SampleRate, float[] Samples, IReadOnlyList SourceReports, + IReadOnlyList BandReports, double CompositeRms, double SourceMeanRms, double ResponseFlatnessBefore, @@ -47,7 +57,7 @@ public sealed class MimirCalibratedAudioCompositeBuilder(MimirCalibratedAudioCom .ToArray(); if (accepted.Length == 0) { - return new MimirCalibratedAudioCompositeResult(0, [], [], 0.0, 0.0, 0.0, 0.0); + return new MimirCalibratedAudioCompositeResult(0, [], [], [], 0.0, 0.0, 0.0, 0.0); } var sampleRate = accepted[0].SampleRate; @@ -59,6 +69,7 @@ public sealed class MimirCalibratedAudioCompositeBuilder(MimirCalibratedAudioCom var frameCount = accepted.Min(source => source.Samples.Length); var output = new float[frameCount]; var reports = new List(accepted.Length); + var prepared = new List(accepted.Length); var totalWeight = 0.0; var sourceRmsTotal = 0.0; var beforeFlatnessTotal = 0.0; @@ -90,25 +101,43 @@ public sealed class MimirCalibratedAudioCompositeBuilder(MimirCalibratedAudioCom var noise = Math.Max(options.NoiseFloorRms, source.NoiseRms); var responseWeight = Math.Clamp(afterFlatness <= 0.0 ? 1.0 : afterFlatness, 0.10, 1.0); var weight = confidence * responseWeight / (noise * noise); - totalWeight += weight; sourceRmsTotal += RootMeanSquare(source.Samples.AsSpan(0, frameCount)); beforeFlatnessTotal += beforeFlatness; afterFlatnessTotal += afterFlatness; - for (var index = 0; index < frameCount; index++) - { - output[index] += (float)(corrected[index] * weight); - } - - reports.Add(new MimirCalibratedAudioCompositeSourceReport( + prepared.Add(new PreparedCompositeSource( source.SourceId, + corrected, + weight, delaySamples, appliedGain, - weight, correctedBands, beforeFlatness, afterFlatness, confidence, - source.NoiseRms)); + source.NoiseRms, + source.BandResponses)); + } + + var bandReports = ApplyCoherenceSuppression(prepared, sampleRate); + foreach (var source in prepared) + { + totalWeight += source.Weight; + for (var index = 0; index < frameCount; index++) + { + output[index] += (float)(source.Samples[index] * source.Weight); + } + + reports.Add(new MimirCalibratedAudioCompositeSourceReport( + source.SourceId, + source.AppliedDelaySamples, + source.AppliedGain, + source.Weight, + source.CorrectedBandCount, + source.ResponseFlatnessBefore, + source.ResponseFlatnessAfter, + source.Confidence, + source.NoiseRms, + AverageSourceCoherence(source, bandReports))); } if (totalWeight > 0.0) @@ -124,12 +153,73 @@ public sealed class MimirCalibratedAudioCompositeBuilder(MimirCalibratedAudioCom sampleRate, output, reports, + bandReports, RootMeanSquare(output), sourceRmsTotal / reportCount, beforeFlatnessTotal / reportCount, afterFlatnessTotal / reportCount); } + private IReadOnlyList ApplyCoherenceSuppression( + IReadOnlyList sources, + int sampleRate) + { + if (sources.Count < 2) + { + return []; + } + + var bandCenters = sources + .SelectMany(source => source.BandResponses.Select(response => response.CenterHz)) + .Where(centerHz => centerHz > 0.0 && centerHz < sampleRate * 0.48) + .DistinctBy(centerHz => Math.Round(12.0 * Math.Log2(centerHz / 110.0))) + .Order() + .ToArray(); + var reports = new List(bandCenters.Length); + foreach (var centerHz in bandCenters) + { + var measurements = sources + .Select(source => new BandMeasurement(source, ToneCoefficient(source.Samples, sampleRate, centerHz))) + .Where(measurement => measurement.Coefficient.Magnitude > 1.0e-9) + .ToArray(); + if (measurements.Length < 2) + { + continue; + } + + var coherence = PairwiseCoherence(measurements.Select(static measurement => measurement.Coefficient).ToArray()); + var suppression = Math.Clamp((options.MinimumCoherence - coherence) / Math.Max(1.0e-6, options.MinimumCoherence), 0.0, 1.0) * + Math.Clamp(options.IncoherentBandSuppression, 0.0, 1.0); + if (suppression > 0.0) + { + foreach (var measurement in measurements) + { + AddBandGainDelta(measurement.Source.Samples, measurement.Source.Samples, sampleRate, centerHz, -suppression); + } + } + + reports.Add(new MimirCalibratedAudioCompositeBandReport( + centerHz, + coherence, + suppression, + measurements.Length)); + } + + return reports; + } + + private static double AverageSourceCoherence( + PreparedCompositeSource source, + IReadOnlyList reports) + { + var centers = source.BandResponses.Select(response => response.CenterHz).ToArray(); + var matched = reports + .Where(report => centers.Any(center => BandsTooClose(center, report.CenterHz))) + .Select(report => report.Coherence) + .ToArray(); + return matched.Length == 0 ? 1.0 : matched.Average(); + } + private float[] ApplySparseBandEqualization( ReadOnlySpan samples, int sampleRate, @@ -186,6 +276,60 @@ public sealed class MimirCalibratedAudioCompositeBuilder(MimirCalibratedAudioCom } } + private static ComplexToneCoefficient ToneCoefficient(ReadOnlySpan samples, int sampleRate, double centerHz) + { + var sine = 0.0; + var cosine = 0.0; + var omega = 2.0 * Math.PI * centerHz / sampleRate; + for (var index = 0; index < samples.Length; index++) + { + var phase = omega * index; + sine += samples[index] * Math.Sin(phase); + cosine += samples[index] * Math.Cos(phase); + } + + var scale = 2.0 / Math.Max(1, samples.Length); + sine *= scale; + cosine *= scale; + return new ComplexToneCoefficient(cosine, sine); + } + + private static double PairwiseCoherence(IReadOnlyList coefficients) + { + var total = 0.0; + var count = 0; + for (var left = 0; left < coefficients.Count; left++) + { + for (var right = left + 1; right < coefficients.Count; right++) + { + var leftCoefficient = coefficients[left]; + var rightCoefficient = coefficients[right]; + var magnitude = leftCoefficient.Magnitude * rightCoefficient.Magnitude; + if (magnitude <= 1.0e-12) + { + continue; + } + + var phaseAgreement = (leftCoefficient.Real * rightCoefficient.Real + leftCoefficient.Imaginary * rightCoefficient.Imaginary) / magnitude; + var energyBalance = Math.Min(leftCoefficient.Magnitude, rightCoefficient.Magnitude) / Math.Max(leftCoefficient.Magnitude, rightCoefficient.Magnitude); + total += Math.Clamp((phaseAgreement + 1.0) * 0.5, 0.0, 1.0) * Math.Sqrt(Math.Clamp(energyBalance, 0.0, 1.0)); + count++; + } + } + + return count == 0 ? 0.0 : total / count; + } + + private static bool BandsTooClose(double leftHz, double rightHz) + { + if (leftHz <= 0.0 || rightHz <= 0.0) + { + return false; + } + + return Math.Abs(Math.Log(leftHz / rightHz)) < 0.06; + } + private double OverallGain(IReadOnlyList responses) { var usable = responses @@ -272,4 +416,24 @@ public sealed class MimirCalibratedAudioCompositeBuilder(MimirCalibratedAudioCom var variance = energies.Sum(value => (value - mean) * (value - mean)) / energies.Length; return 1.0 / (1.0 + Math.Sqrt(variance)); } + + private sealed record PreparedCompositeSource( + string SourceId, + float[] Samples, + double Weight, + double AppliedDelaySamples, + double AppliedGain, + int CorrectedBandCount, + double ResponseFlatnessBefore, + double ResponseFlatnessAfter, + double Confidence, + double NoiseRms, + IReadOnlyList BandResponses); + + private sealed record BandMeasurement(PreparedCompositeSource Source, ComplexToneCoefficient Coefficient); + + private readonly record struct ComplexToneCoefficient(double Real, double Imaginary) + { + public double Magnitude => Math.Sqrt(Real * Real + Imaginary * Imaginary); + } } diff --git a/state/evidence.jsonl b/state/evidence.jsonl index 2515087..c3bcb63 100644 --- a/state/evidence.jsonl +++ b/state/evidence.jsonl @@ -316,3 +316,4 @@ {"ts": "2026-06-01T03:42:48+01:00", "type": "verification", "status": "ok", "note": "Realtek-targeted bioacoustic probe runner proved explicit Starfire Realtek emission through Mimir.WasapiLoopback. It selected Speakers (Realtek(R) Audio), played the targeted probe at gain 0.75, captured Scarlett inputs under artifacts/wasapi/targeted-bioacoustic-realtk-live/20260601-033851, and measured shotgun/cardioid response. The path is real, but 18.6 kHz remained near-zero confidence; planner defaults now cap probe selection at 12 kHz and bias measurable low/mid bands."} {"ts": "2026-06-01T03:54:22+01:00", "type": "verification", "status": "ok", "note": "Ran scheduler-driven Realtek calibration under artifacts/wasapi/realtk-scheduled-calibration-live/20260601-035354. The runner used MimirBioacousticProbeScheduler to choose four targeted probe steps, emitted through Speakers (Realtek(R) Audio) at gain 1.15, captured Scarlett ASIO, and wrote scheduled-response-bands.csv plus balance-plan.md. Local peak-vs-percentile-floor scoring found strong usable response around 910 Hz, 3.6 kHz, and 9.84-9.87 kHz on both shotgun and cardioid; both channels scored high SNR in those asked bands."} {"ts": "2026-06-01T04:01:53+01:00", "type": "verification", "status": "ok", "note": "Bioacoustic scheduler now shares the targeted probe planner's measurable frequency boundary. The scheduler filters residual confidence to schedulable bands before choosing cadence/status and passes the same 120 Hz-12 kHz window to the planner, so unmeasurable 18.5 kHz residuals no longer keep live status pinned to a band the Realtek/speaker/Scarlett path cannot answer. Verified with --bioacoustic-probe-scheduler-smoke and --targeted-bioacoustic-probe-smoke."} +{"ts": "2026-06-01T04:13:53+01:00", "type": "verification", "status": "ok", "note": "Added calibrated-band coherence suppression to MimirCalibratedAudioCompositeBuilder. The composite path now aligns sources, applies measured-band correction, scores phase/energy coherence per calibrated band, suppresses incoherent measured-band components, and reports source/band coherence. Verified with --dual-mic-coherence-cancellation-smoke: shared tones held ~0.999 coherence, a mic-local 3.52 kHz interferer scored ~0.001 coherence with ~0.948 suppression, poison energy fell 0.039195 to 0.001306, and SNR rose 1.366 dB to 16.304 dB."} diff --git a/state/map.yaml b/state/map.yaml index d6d79e0..4bf2716 100644 --- a/state/map.yaml +++ b/state/map.yaml @@ -637,11 +637,18 @@ accepted_design: owner for corrected mic streams: it consumes audio samples plus `MimirAudioSynchronizationState` delay/confidence and measured band responses, applies fractional alignment, sparse measured-band - equalization, and confidence/noise-weighted summing, then emits composite - samples and flatness/weight reports. `Mimir.BufferSmoke + equalization, calibrated-band coherence scoring, incoherent-band + suppression, and confidence/noise-weighted summing, then emits composite + samples plus flatness/weight/coherence reports and per-band + coherence/suppression reports. `Mimir.BufferSmoke --calibrated-audio-composite-smoke` proves the contract on synthetic delayed/noisy/colored mic paths, improving SNR from -3.171 dB uncalibrated to 15.135 dB calibrated and response flatness from 0.548 to 1.000. + `Mimir.BufferSmoke --dual-mic-coherence-cancellation-smoke` proves the + two-witness cancellation cut: shared tones keep about 0.999 coherence, a + mic-local 3.52 kHz interferer scores about 0.001 coherence and 0.948 + suppression, its measured energy falls from 0.039195 to 0.001306, and SNR + rises from 1.366 dB to 16.304 dB. Faust/native DSP still owns the production hot lane; this C# path is a contract, scorer, and offline diagnostic, not a competing live mixer. `MimirTargetedBioacousticProbePlanner` is now the residual-driven probe