From 749aecc5a97b73aa5328bfd380a96002b9f1993b Mon Sep 17 00:00:00 2001 From: Meta Vens Date: Mon, 1 Jun 2026 03:45:37 +0100 Subject: [PATCH] Emit targeted probes through Realtek --- docs/audio-field.md | 16 + docs/implementation-plan.md | 9 + notes/current-system-map.md | 16 + src/Mimir.BufferSmoke/Program.cs | 209 ++++++++- .../MimirTargetedBioacousticProbePlanner.cs | 20 +- .../Mimir.WasapiLoopback.csproj | 2 +- src/Mimir.WasapiLoopback/Program.cs | 428 +++++++++++++++++- state/evidence.jsonl | 1 + state/map.yaml | 15 + 9 files changed, 700 insertions(+), 16 deletions(-) diff --git a/docs/audio-field.md b/docs/audio-field.md index fde6874..5ab8cd0 100644 --- a/docs/audio-field.md +++ b/docs/audio-field.md @@ -100,6 +100,22 @@ is the live hub surface; `MimirRuntime` prints operators can watch confidence climb as probes are spent. The scheduler owns when to spend active sound. It does not own the output device: Starfire Realtek still must be selected explicitly by the eventual live renderer. +The first explicit Realtek renderer proof now lives in the repo tooling: +`Mimir.WasapiLoopback` can enumerate render endpoints, select one by friendly +name substring, and play mono Float32 probe audio through shared-mode WASAPI. +`Mimir.BufferSmoke --targeted-bioacoustic-probe-realtk-live` renders a targeted +probe plan, starts Scarlett ASIO capture, plays the probe through +`Speakers (Realtek(R) Audio)`, and scores the captured shotgun/cardioid bands. +The targeted probe planner now caps default acoustic probe selection at 12 kHz +and mildly biases low/mid measurable bands, because commodity speakers and the +current Scarlett mics have little reason to carry useful calibration evidence +above that range. The 2026-06-01 proof at gain `0.75` selected the Realtek speaker endpoint and +captured `artifacts/wasapi/targeted-bioacoustic-realtk-live/20260601-033851`. +The physical mics heard the burst, but the scored confidence still concentrated +in low/mid evidence: shotgun peak `0.407887`, cardioid peak `0.074494`, 18.6 kHz +confidence near zero, and the best reported band was cardioid 3.6 kHz at +confidence `0.098`. Treat high-band probes as poor questions for this route +until a better speaker/mic/path proves otherwise. The receiver must consume persisted models to weight reliable symbols, downweight dead bands, apply phase-coherence weighting, apply first-order group-delay correction, and search joint global delay/frequency-shift hypotheses diff --git a/docs/implementation-plan.md b/docs/implementation-plan.md index db29fef..2e1fb19 100644 --- a/docs/implementation-plan.md +++ b/docs/implementation-plan.md @@ -142,6 +142,15 @@ a named invariant that the native runtime cannot protect yet. for aggregate confidence, deltas, source probe counts, and weak-band pressure. The scheduler is output-device neutral; a later renderer must explicitly target Starfire Realtek for physical chirp emission in the current routing. +- `Mimir.WasapiLoopback` now has the first explicit Starfire Realtek render + edge: it can list active render endpoints, select `Speakers (Realtek(R) + Audio)` by friendly-name substring, and play mono Float32 probe audio through + WASAPI shared mode. `Mimir.BufferSmoke + --targeted-bioacoustic-probe-realtk-live` couples that renderer with Scarlett + ASIO capture and scores the physical mic response for the scheduled probe. + The targeted probe planner defaults to a 12 kHz ceiling and low/mid-band bias + because the current Realtek/speaker/Scarlett-mic path does not provide useful + high-band calibration evidence. - `src/Mimir.BufferSmoke` loads the runtime config, polls the synchronization hub, and prints the actual rolling buffers. Use `--require-samples` when an empty declared sensor buffer should fail the run. Use `--bioacoustic-self-test` diff --git a/notes/current-system-map.md b/notes/current-system-map.md index edfe8dc..6bdcf9d 100644 --- a/notes/current-system-map.md +++ b/notes/current-system-map.md @@ -334,6 +334,22 @@ climb: simulated probe feedback raises sync confidence from 0.360 to 0.920 and frequency-response confidence from 0.558 to 0.955 over four scheduled probes. The scheduler is not the output-device authority; in the current physical routing the eventual live renderer must explicitly select Starfire Realtek. +That Realtek renderer now exists as a tooling edge. `Mimir.WasapiLoopback` +targets `net10.0-windows`, enumerates active render endpoints, selects by +friendly-name substring, and can play mono Float32 probe audio through the +matched WASAPI render endpoint. `Mimir.BufferSmoke +--targeted-bioacoustic-probe-realtk-live` renders a targeted probe, starts +Scarlett ASIO capture, plays it through the selected Realtek speaker endpoint, +then scores the captured shotgun/cardioid response. The first proof selected +`Speakers (Realtek(R) Audio)` and wrote +`artifacts/wasapi/targeted-bioacoustic-realtk-live/20260601-033851`; at gain +0.75 the shotgun peak was 0.407887 and cardioid peak was 0.074494, but 18.6 kHz +remained near-zero confidence while cardioid 3.6 kHz was the strongest printed +band at 0.098 confidence. Realtek emission is now real; high-band probe choice +is still a bad physical question for this room/speaker path. The default +targeted probe planner now caps selection at 12 kHz and biases measurable +low/mid bands instead of favoring ultrasonic-ish uncertainty that ordinary +speakers and mics cannot answer well. The first live run preserves two artifacts: `artifacts/asio/targeted-bioacoustic-live/20260601-030836` at playback gain 1.0 and `artifacts/asio/targeted-bioacoustic-live/20260601-030857` at gain diff --git a/src/Mimir.BufferSmoke/Program.cs b/src/Mimir.BufferSmoke/Program.cs index 211b26a..fab0f1b 100644 --- a/src/Mimir.BufferSmoke/Program.cs +++ b/src/Mimir.BufferSmoke/Program.cs @@ -101,6 +101,18 @@ if (args.Any(arg => string.Equals(arg, "--targeted-bioacoustic-probe-live", Stri ParseIntOption(args, "--reference-channel", 2)); } +if (args.Any(arg => string.Equals(arg, "--targeted-bioacoustic-probe-realtk-live", StringComparison.OrdinalIgnoreCase))) +{ + return RunTargetedBioacousticProbeRealtekLive( + ParseStringOption(args, "--wasapi", "src/Mimir.WasapiLoopback/bin/Debug/net10.0-windows/Mimir.WasapiLoopback.exe"), + ParseStringOption(args, "--probe", "native/probes/asio_audio_cadence/build/Release/asio_audio_cadence.exe"), + ParseStringOption(args, "--output", "artifacts/wasapi/targeted-bioacoustic-realtk-live"), + ParseStringOption(args, "--device", "Realtek"), + ParseIntOption(args, "--sample-rate", 48_000), + ParseDoubleOption(args, "--gain", 0.45), + ParseIntOption(args, "--channels", 4)); +} + if (args.Any(arg => string.Equals(arg, "--synchronized-buffer-planner-smoke", StringComparison.OrdinalIgnoreCase))) { return RunSynchronizedBufferPlannerSmoke(); @@ -3418,7 +3430,7 @@ static int RunTargetedBioacousticProbeSmoke() var top = plan.Bands.FirstOrDefault(); var hasLowBand = plan.Bands.Any(band => band.CenterHz is > 800.0 and < 1_000.0); var hasMidBand = plan.Bands.Any(band => band.CenterHz is > 3_300.0 and < 3_900.0); - var hasHighBand = plan.Bands.Any(band => band.CenterHz is > 17_000.0 and < 20_000.0); + var hasUpperBand = plan.Bands.Any(band => band.CenterHz is > 8_000.0 and < 12_000.0); Console.WriteLine( $"targeted-bioacoustic-probe-smoke sampleRate={plan.SampleRate} sources={string.Join(",", plan.MeasurementSourceIds)} bands={plan.Bands.Count} " + $"duration={plan.EstimatedDurationSeconds:0.000} top={top?.CenterHz:0}Hz reason={top?.Reason} gain={top?.Gain:0.000} priority={top?.Priority:0.000} " + @@ -3429,9 +3441,10 @@ static int RunTargetedBioacousticProbeSmoke() plan.Bands.Count >= 3 && hasLowBand && hasMidBand && - hasHighBand && - top is { CenterHz: > 17_000.0 and < 20_000.0 } && - top.Reason is MimirBioacousticProbeReason.UnmeasuredBand or MimirBioacousticProbeReason.WeakResponse or MimirBioacousticProbeReason.PhaseUnstable && + hasUpperBand && + top is { CenterHz: > 8_000.0 and < 12_000.0 } && + top.Reason is MimirBioacousticProbeReason.WeakResponse or MimirBioacousticProbeReason.PhaseUnstable && + plan.Bands.All(band => band.CenterHz <= 12_000.0) && plan.Bands.All(band => band.Gain is >= 0.018 and <= 0.085) ? 0 : 1; @@ -3587,6 +3600,97 @@ static int RunTargetedBioacousticProbeLive( return failures == 0 ? 0 : 1; } +static int RunTargetedBioacousticProbeRealtekLive( + string wasapiPath, + string probePath, + string outputDirectory, + string device, + int sampleRate, + double gain, + int channels) +{ + var residuals = ScarlettProfessionalMicProbeResiduals(); + var pathologies = ScarlettProfessionalMicPathologies(); + var plan = new MimirTargetedBioacousticProbePlanner(new MimirTargetedBioacousticProbeOptions(SampleRate: sampleRate)).Plan( + residuals, + ["asio-ch0-shotgun", "asio-ch1-cardioid"], + pathologies); + if (plan.Bands.Count == 0) + { + Console.Error.WriteLine("targeted Realtek bioacoustic live failed: planner emitted no probe bands"); + return 1; + } + + var stamp = DateTimeOffset.Now.ToString("yyyyMMdd-HHmmss", CultureInfo.InvariantCulture); + var runDirectory = Path.Combine(outputDirectory, stamp); + Directory.CreateDirectory(runDirectory); + var renderPath = Path.Combine(runDirectory, "targeted-realtk-probe-f32.raw"); + var capturePath = Path.Combine(runDirectory, "scarlett-realtk-probe-f32.raw"); + var samples = RenderTargetedProbePlan(plan); + WriteFloat32(renderPath, samples); + Console.WriteLine( + $"targeted-bioacoustic-realtk-live-plan dir={runDirectory} device=\"{device}\" sampleRate={sampleRate} gain={gain:0.000} samples={samples.Length} bands={plan.Bands.Count} selected={string.Join(",", plan.Bands.Select(band => $"{band.CenterHz:0}Hz/{band.Reason}/{band.Gain:0.000}"))}"); + + var captureSeconds = samples.Length / (double)sampleRate + 0.65; + var capture = StartAsioCaptureOnly(probePath, capturePath, sampleRate, captureSeconds); + if (capture == null) + { + return 1; + } + + using (capture) + { + Thread.Sleep(180); + var renderExit = RunWasapiRender(wasapiPath, renderPath, device, sampleRate, gain); + capture.WaitForExit(); + if (renderExit != 0) + { + Console.Error.WriteLine($"targeted Realtek bioacoustic live failed: WASAPI renderer exited {renderExit}"); + return renderExit; + } + + if (capture.ExitCode != 0) + { + Console.Error.WriteLine($"targeted Realtek bioacoustic live failed: ASIO capture exited {capture.ExitCode}"); + return capture.ExitCode; + } + } + + if (!File.Exists(capturePath)) + { + Console.Error.WriteLine($"targeted Realtek bioacoustic live failed: capture file missing {capturePath}"); + return 1; + } + + var channelSamples = ReadInterleavedFloat32(capturePath, channels, out var frameCount); + if (frameCount == 0) + { + Console.Error.WriteLine($"targeted Realtek bioacoustic live failed: invalid capture frames={frameCount}"); + return 1; + } + + Console.WriteLine($"targeted-bioacoustic-realtk-live-capture path={capturePath} frames={frameCount} seconds={frameCount / (double)sampleRate:0.000}"); + for (var channel = 0; channel < Math.Min(2, channels); channel++) + { + var label = channel == 0 ? "shotgun" : "cardioid"; + var stats = SignalStats(channelSamples[channel]); + var low = BandLimitedStats(channelSamples[channel], sampleRate, 80.0, 1000.0); + var high = BandLimitedStats(channelSamples[channel], sampleRate, 8000.0, Math.Min(24000.0, sampleRate * 0.45)); + var tiltDb = 20.0 * Math.Log10(Math.Max(1.0e-9, low.Rms) / Math.Max(1.0e-9, high.Rms)); + Console.WriteLine( + $"targeted-bioacoustic-realtk-live-source ch={channel} label={label} rms={stats.Rms:0.000000} peak={stats.Peak:0.000000} lowHighTiltDb={tiltDb:0.000}"); + foreach (var band in plan.Bands) + { + var candidateEnergy = ToneEnergy(channelSamples[channel], sampleRate, band.CenterHz); + var bandConfidence = candidateEnergy / (candidateEnergy + Math.Max(1.0e-9, stats.Rms * 0.20)); + Console.WriteLine( + $"targeted-bioacoustic-realtk-live-band ch={channel} label={label} centerHz={band.CenterHz:0.0} reason={band.Reason} micEnergy={candidateEnergy:0.000000} confidence={bandConfidence:0.000}"); + } + } + + return 0; +} + static int RunSynchronizedBufferPlannerSmoke() { const long t0 = 1_000_000_000; @@ -6145,6 +6249,103 @@ static int RunAsioProbeCapture( return process.ExitCode; } +static Process? StartAsioCaptureOnly( + string probePath, + string capturePath, + int sampleRate, + double seconds) +{ + var startInfo = new ProcessStartInfo(Path.GetFullPath(probePath)) + { + UseShellExecute = false, + RedirectStandardOutput = true, + RedirectStandardError = true, + CreateNoWindow = true, + }; + startInfo.ArgumentList.Add("--set-sample-rate"); + startInfo.ArgumentList.Add(sampleRate.ToString()); + startInfo.ArgumentList.Add("--record-f32-interleaved"); + startInfo.ArgumentList.Add(Path.GetFullPath(capturePath)); + startInfo.ArgumentList.Add("--capture-seconds"); + startInfo.ArgumentList.Add(seconds.ToString("0.###")); + + var process = Process.Start(startInfo); + if (process == null) + { + Console.Error.WriteLine($"ASIO capture failed: could not start {probePath}"); + return null; + } + + process.OutputDataReceived += (_, e) => + { + if (e.Data != null) + { + Console.WriteLine(e.Data); + } + }; + process.ErrorDataReceived += (_, e) => + { + if (e.Data != null) + { + Console.Error.WriteLine(e.Data); + } + }; + process.BeginOutputReadLine(); + process.BeginErrorReadLine(); + return process; +} + +static int RunWasapiRender( + string wasapiPath, + string renderPath, + string device, + int sampleRate, + double gain) +{ + var startInfo = new ProcessStartInfo(Path.GetFullPath(wasapiPath)) + { + UseShellExecute = false, + RedirectStandardOutput = true, + RedirectStandardError = true, + CreateNoWindow = true, + }; + startInfo.ArgumentList.Add("--play-f32-mono"); + startInfo.ArgumentList.Add("--device"); + startInfo.ArgumentList.Add(device); + startInfo.ArgumentList.Add("--input"); + startInfo.ArgumentList.Add(Path.GetFullPath(renderPath)); + startInfo.ArgumentList.Add("--sample-rate"); + startInfo.ArgumentList.Add(sampleRate.ToString()); + startInfo.ArgumentList.Add("--gain"); + startInfo.ArgumentList.Add(gain.ToString("0.#####")); + + using var process = Process.Start(startInfo); + if (process == null) + { + Console.Error.WriteLine($"WASAPI render failed: could not start {wasapiPath}"); + return 1; + } + + process.OutputDataReceived += (_, e) => + { + if (e.Data != null) + { + Console.WriteLine(e.Data); + } + }; + process.ErrorDataReceived += (_, e) => + { + if (e.Data != null) + { + Console.Error.WriteLine(e.Data); + } + }; + process.BeginOutputReadLine(); + process.BeginErrorReadLine(); + process.WaitForExit(); + return process.ExitCode; +} + static float[][] ReadInterleavedFloat32(string inputPath, int channels, out int frameCount) { if (channels <= 0) diff --git a/src/Mimir.Runtime/Synchronization/MimirTargetedBioacousticProbePlanner.cs b/src/Mimir.Runtime/Synchronization/MimirTargetedBioacousticProbePlanner.cs index 9ba149b..f8fbc03 100644 --- a/src/Mimir.Runtime/Synchronization/MimirTargetedBioacousticProbePlanner.cs +++ b/src/Mimir.Runtime/Synchronization/MimirTargetedBioacousticProbePlanner.cs @@ -28,7 +28,7 @@ public sealed record MimirTargetedBioacousticProbeOptions( int SampleRate = 192_000, int MaxProbeBands = 8, double MinFrequencyHz = 120.0, - double MaxFrequencyHz = 42_000.0, + double MaxFrequencyHz = 12_000.0, double TargetConfidence = 0.82, double ResidualDbBudget = 1.5, double WeakResponseThreshold = 0.08, @@ -168,6 +168,12 @@ public sealed class MimirTargetedBioacousticProbePlanner(MimirTargetedBioacousti var phasePressure = Math.Max(0.0, Math.Abs(residual.PhaseResidualRadians) - options.PhaseResidualBudgetRadians) / Math.PI; var delayPressure = Math.Min(1.0, Math.Abs(residual.DelayResidualMicroseconds) / 25.0); var pathologyPressure = 0.0; + var nearSpeechBand = residual.CenterHz is >= 250.0 and <= 5_500.0 + ? 0.22 + : 0.0; + var upperPresenceBand = residual.CenterHz is > 5_500.0 and <= 12_000.0 + ? 0.08 + : 0.0; if (pathologyBySource.TryGetValue(residual.SourceId, out var pathology)) { var highBand = residual.CenterHz >= 3_000.0 @@ -175,10 +181,18 @@ public sealed class MimirTargetedBioacousticProbePlanner(MimirTargetedBioacousti : 0.0; var noisePressure = Math.Clamp(pathology.NoiseFloorRms / 0.050, 0.0, 1.0); var tiltPressure = Math.Clamp(pathology.LowHighTiltDb / 18.0, 0.0, 1.0); - pathologyPressure = highBand * (noisePressure * 0.35 + tiltPressure * 0.75 + Math.Clamp(pathology.HighBandConfidenceBias, 0.0, 1.0)); + var measurableHighBand = Math.Clamp(1.0 - Math.Max(0.0, residual.CenterHz - 10_000.0) / 8_000.0, 0.0, 1.0); + pathologyPressure = highBand * measurableHighBand * (noisePressure * 0.35 + tiltPressure * 0.75 + Math.Clamp(pathology.HighBandConfidenceBias, 0.0, 1.0)); } - return confidenceGap * 1.20 + residualPressure + weakResponse * 0.85 + phasePressure * 0.75 + delayPressure * 0.40 + pathologyPressure; + return confidenceGap * 1.20 + + residualPressure + + weakResponse * 0.85 + + phasePressure * 0.75 + + delayPressure * 0.40 + + pathologyPressure + + nearSpeechBand + + upperPresenceBand; } private MimirBioacousticProbeReason ReasonFor(MimirAudioCalibrationBandResidual residual) diff --git a/src/Mimir.WasapiLoopback/Mimir.WasapiLoopback.csproj b/src/Mimir.WasapiLoopback/Mimir.WasapiLoopback.csproj index 9c1b016..8a52d19 100644 --- a/src/Mimir.WasapiLoopback/Mimir.WasapiLoopback.csproj +++ b/src/Mimir.WasapiLoopback/Mimir.WasapiLoopback.csproj @@ -1,7 +1,7 @@ Exe - net8.0-windows + net10.0-windows enable enable Mimir.WasapiLoopback diff --git a/src/Mimir.WasapiLoopback/Program.cs b/src/Mimir.WasapiLoopback/Program.cs index f9102da..c82522c 100644 --- a/src/Mimir.WasapiLoopback/Program.cs +++ b/src/Mimir.WasapiLoopback/Program.cs @@ -9,20 +9,37 @@ internal static class Program private const int EMultimedia = 1; private const int ECommunications = 2; private const int ClsctxAll = 23; + private const uint DeviceStateActive = 0x00000001; private const uint AudclntSharemodeShared = 0; private const uint AudclntStreamflagsLoopback = 0x00020000; private const int AudclntBufferflagsSilent = 0x2; private static readonly Guid IidAudioClient = new("1CB9AD4C-DBFA-4c32-B178-C2F568A703B2"); private static readonly Guid IidAudioCaptureClient = new("C8ADBD64-E71E-48a0-A4DE-185C395CD317"); + private static readonly Guid IidAudioRenderClient = new("F294ACFC-3146-4483-A7BF-ADDCA7C260E2"); private static readonly Guid PcmSubformat = new("00000001-0000-0010-8000-00aa00389b71"); private static readonly Guid FloatSubformat = new("00000003-0000-0010-8000-00aa00389b71"); + private static readonly PropertyKey DeviceFriendlyNameKey = new(new Guid("A45C254E-DF1C-4EFD-8020-67D146A850E0"), 14); public static int Main(string[] args) { try { var options = Options.Parse(args); - Capture(options); + if (options.ListDevices) + { + ListRenderDevices(); + return 0; + } + + if (options.Mode == RunMode.PlayF32Mono) + { + PlayF32Mono(options); + } + else + { + Capture(options); + } + return 0; } catch (Exception ex) @@ -41,7 +58,7 @@ internal static class Program using var outputLifetime = outputIsStdout ? null : output; IMMDeviceEnumerator enumerator = (IMMDeviceEnumerator)(object)new MMDeviceEnumerator(); - Check(enumerator.GetDefaultAudioEndpoint(ERender, RoleFromName(options.Role), out var device), "GetDefaultAudioEndpoint"); + var device = ResolveRenderDevice(enumerator, options.Role, options.Device); var audioClientId = IidAudioClient; Check(device.Activate(ref audioClientId, ClsctxAll, IntPtr.Zero, out var audioClient), "Activate IAudioClient"); @@ -56,7 +73,8 @@ internal static class Program Check(init, "Initialize loopback"); var captureClientId = IidAudioCaptureClient; - Check(audioClient.GetService(ref captureClientId, out var captureClient), "GetService IAudioCaptureClient"); + Check(audioClient.GetService(ref captureClientId, out var captureService), "GetService IAudioCaptureClient"); + var captureClient = (IAudioCaptureClient)captureService; Check(audioClient.Start(), "Start"); try @@ -74,6 +92,125 @@ internal static class Program } } + private static void PlayF32Mono(Options options) + { + if (string.IsNullOrWhiteSpace(options.InputPath)) + { + throw new ArgumentException("--input is required for --play-f32-mono"); + } + + var inputBytes = File.ReadAllBytes(options.InputPath); + if (inputBytes.Length < sizeof(float)) + { + throw new InvalidOperationException("Input file contains no Float32 samples: " + options.InputPath); + } + + var inputSamples = new float[inputBytes.Length / sizeof(float)]; + Buffer.BlockCopy(inputBytes, 0, inputSamples, 0, inputSamples.Length * sizeof(float)); + + IMMDeviceEnumerator enumerator = (IMMDeviceEnumerator)(object)new MMDeviceEnumerator(); + var device = ResolveRenderDevice(enumerator, options.Role, options.Device); + var audioClientId = IidAudioClient; + Check(device.Activate(ref audioClientId, ClsctxAll, IntPtr.Zero, out var audioClient), "Activate IAudioClient"); + Check(audioClient.GetMixFormat(out var mixFormatPtr), "GetMixFormat"); + + var fmt = Marshal.PtrToStructure(mixFormatPtr); + var sampleFormat = SampleFormat.From(mixFormatPtr); + LogFormat("RenderMixFormat", fmt, sampleFormat); + + var hnsBuffer = Math.Max(100_000L, (long)Math.Round(10_000_000.0 * Math.Min(0.25, Math.Max(0.02, options.RenderBufferSeconds)))); + Check(audioClient.Initialize(AudclntSharemodeShared, 0, hnsBuffer, 0, mixFormatPtr, IntPtr.Zero), "Initialize render"); + Check(audioClient.GetBufferSize(out var bufferFrames), "GetBufferSize"); + + var renderClientId = IidAudioRenderClient; + Check(audioClient.GetService(ref renderClientId, out var renderService), "GetService IAudioRenderClient"); + var renderClient = (IAudioRenderClient)renderService; + Check(audioClient.Start(), "Start"); + + try + { + PumpRender(options, inputSamples, audioClient, renderClient, fmt, sampleFormat, bufferFrames); + } + finally + { + audioClient.Stop(); + Marshal.FreeCoTaskMem(mixFormatPtr); + Marshal.ReleaseComObject(renderClient); + Marshal.ReleaseComObject(audioClient); + Marshal.ReleaseComObject(device); + Marshal.ReleaseComObject(enumerator); + } + } + + private static void PumpRender( + Options options, + float[] inputSamples, + IAudioClient audioClient, + IAudioRenderClient renderClient, + WaveFormatEx fmt, + SampleFormat sampleFormat, + uint bufferFrames) + { + var renderChannels = Math.Max(1, (int)fmt.Channels); + var renderRate = Math.Max(1, (int)fmt.SamplesPerSec); + var bytesPerFrame = fmt.BlockAlign; + var bytesPerSample = Math.Max(1, sampleFormat.ContainerBits / 8); + var sourceFramesPerRenderFrame = options.SampleRate / (double)renderRate; + var sourceFrame = 0.0; + var framesWritten = 0L; + var peak = inputSamples.Select(Math.Abs).DefaultIfEmpty(0f).Max(); + var gain = Math.Clamp(options.Gain, 0.0, 8.0); + var nextMeter = DateTime.UtcNow.AddSeconds(1); + + while (sourceFrame < inputSamples.Length) + { + Check(audioClient.GetCurrentPadding(out var padding), "GetCurrentPadding"); + var available = bufferFrames > padding ? bufferFrames - padding : 0; + if (available == 0) + { + Thread.Sleep(2); + continue; + } + + var frames = (uint)Math.Min(available, Math.Ceiling((inputSamples.Length - sourceFrame) / sourceFramesPerRenderFrame)); + if (frames == 0) + { + break; + } + + Check(renderClient.GetBuffer(frames, out var data), "Render GetBuffer"); + try + { + for (var frame = 0; frame < frames; frame++) + { + var sample = (float)Math.Clamp(ReadInterpolated(inputSamples, sourceFrame) * gain, -1.0, 1.0); + for (var ch = 0; ch < renderChannels; ch++) + { + var target = IntPtr.Add(data, frame * bytesPerFrame + ch * bytesPerSample); + WriteSample(target, sample, sampleFormat); + } + + sourceFrame += sourceFramesPerRenderFrame; + } + } + finally + { + Check(renderClient.ReleaseBuffer(frames, 0), "Render ReleaseBuffer"); + } + + framesWritten += frames; + if (DateTime.UtcNow >= nextMeter) + { + Console.Error.WriteLine("RenderMeter frames={0} source={1:0}/{2} peak={3:0.000000} gain={4:0.000}", framesWritten, sourceFrame, inputSamples.Length, peak, gain); + nextMeter = DateTime.UtcNow.AddSeconds(1); + } + } + + var drainMs = Math.Max(40, (int)Math.Round(options.RenderDrainSeconds * 1000.0)); + Thread.Sleep(drainMs); + Console.Error.WriteLine("RenderComplete frames={0} seconds={1:0.000} inputSamples={2} inputRate={3}", framesWritten, framesWritten / (double)renderRate, inputSamples.Length, options.SampleRate); + } + private static void Pump( Options options, Stream output, @@ -265,6 +402,71 @@ internal static class Program return value; } + private static float ReadInterpolated(IReadOnlyList samples, double frame) + { + if (samples.Count == 0) + { + return 0f; + } + + if (frame <= 0.0) + { + return samples[0]; + } + + var left = (int)Math.Floor(frame); + if (left >= samples.Count - 1) + { + return samples[^1]; + } + + var fraction = frame - left; + return (float)(samples[left] + (samples[left + 1] - samples[left]) * fraction); + } + + private static void WriteSample(IntPtr target, float sample, SampleFormat format) + { + if (format.Kind == SampleKind.IeeeFloat && format.ContainerBits == 32) + { + Marshal.StructureToPtr(sample, target, false); + return; + } + + if (format.Kind != SampleKind.Pcm) + { + return; + } + + sample = Math.Clamp(sample, -1f, 1f); + switch (format.ContainerBits) + { + case 16: + Marshal.WriteInt16(target, (short)Math.Clamp(Math.Round(sample * 32767.0), short.MinValue, short.MaxValue)); + break; + case 24: + WritePacked24(target, (int)Math.Clamp(Math.Round(sample * 8388607.0), -8388608.0, 8388607.0)); + break; + case 32: + var bits = format.ValidBits is > 0 and < 32 ? format.ValidBits : 32; + var max = Math.Pow(2.0, bits - 1) - 1.0; + var value = (int)Math.Clamp(Math.Round(sample * max), -max - 1.0, max); + if (bits < 32) + { + value <<= 32 - bits; + } + + Marshal.WriteInt32(target, value); + break; + } + } + + private static void WritePacked24(IntPtr target, int value) + { + Marshal.WriteByte(target, 0, (byte)(value & 0xff)); + Marshal.WriteByte(target, 1, (byte)((value >> 8) & 0xff)); + Marshal.WriteByte(target, 2, (byte)((value >> 16) & 0xff)); + } + private static void LogFormat(string label, WaveFormatEx fmt, SampleFormat sampleFormat) { Console.Error.WriteLine( @@ -302,20 +504,137 @@ internal static class Program }; } - private sealed record Options(string OutputPath, double Seconds, int SampleRate, int Channels, string Role) + private static void ListRenderDevices() + { + IMMDeviceEnumerator enumerator = (IMMDeviceEnumerator)(object)new MMDeviceEnumerator(); + Check(enumerator.EnumAudioEndpoints(ERender, DeviceStateActive, out var devices), "EnumAudioEndpoints"); + Check(devices.GetCount(out var count), "GetCount"); + for (uint index = 0; index < count; index++) + { + Check(devices.Item(index, out var device), "Item"); + try + { + Console.WriteLine("{0}: {1}", index, DeviceFriendlyName(device)); + } + finally + { + Marshal.ReleaseComObject(device); + } + } + + Marshal.ReleaseComObject(devices); + Marshal.ReleaseComObject(enumerator); + } + + private static IMMDevice ResolveRenderDevice(IMMDeviceEnumerator enumerator, string role, string deviceSubstring) + { + if (string.IsNullOrWhiteSpace(deviceSubstring)) + { + Check(enumerator.GetDefaultAudioEndpoint(ERender, RoleFromName(role), out var defaultDevice), "GetDefaultAudioEndpoint"); + Console.Error.WriteLine("SelectedRenderDevice default role={0} name=\"{1}\"", role, DeviceFriendlyName(defaultDevice)); + return defaultDevice; + } + + Check(enumerator.EnumAudioEndpoints(ERender, DeviceStateActive, out var devices), "EnumAudioEndpoints"); + Check(devices.GetCount(out var count), "GetCount"); + IMMDevice? selected = null; + var selectedName = ""; + try + { + for (uint index = 0; index < count; index++) + { + Check(devices.Item(index, out var device), "Item"); + var name = DeviceFriendlyName(device); + if (name.Contains(deviceSubstring, StringComparison.OrdinalIgnoreCase)) + { + selected = device; + selectedName = name; + break; + } + + Marshal.ReleaseComObject(device); + } + } + finally + { + Marshal.ReleaseComObject(devices); + } + + if (selected == null) + { + throw new InvalidOperationException("No active render endpoint matched --device \"" + deviceSubstring + "\""); + } + + Console.Error.WriteLine("SelectedRenderDevice match=\"{0}\" name=\"{1}\"", deviceSubstring, selectedName); + return selected; + } + + private static string DeviceFriendlyName(IMMDevice device) + { + Check(device.OpenPropertyStore(0, out var store), "OpenPropertyStore"); + try + { + var key = DeviceFriendlyNameKey; + Check(store.GetValue(ref key, out var value), "GetValue FriendlyName"); + try + { + return value.ValueType == 31 && value.Pointer != IntPtr.Zero + ? Marshal.PtrToStringUni(value.Pointer) ?? "" + : ""; + } + finally + { + PropVariantClear(ref value); + } + } + finally + { + Marshal.ReleaseComObject(store); + } + } + + private sealed record Options( + RunMode Mode, + string InputPath, + string OutputPath, + double Seconds, + int SampleRate, + int Channels, + string Role, + string Device, + double Gain, + double RenderBufferSeconds, + double RenderDrainSeconds, + bool ListDevices) { public static Options Parse(string[] args) { + var mode = RunMode.Capture; + var input = ""; var output = "stdout"; var seconds = 0.0; var sampleRate = 48000; var channels = 2; var role = "Console"; + var device = ""; + var gain = 1.0; + var renderBufferSeconds = 0.06; + var renderDrainSeconds = 0.20; + var listDevices = false; for (var i = 0; i < args.Length; i++) { switch (args[i]) { + case "--list-devices": + listDevices = true; + break; + case "--play-f32-mono": + mode = RunMode.PlayF32Mono; + break; + case "--input": + input = ReadValue(args, ref i); + break; case "--output": output = ReadValue(args, ref i); break; @@ -331,9 +650,21 @@ internal static class Program case "--role": role = ReadValue(args, ref i); break; + case "--device": + device = ReadValue(args, ref i); + break; + case "--gain": + gain = double.Parse(ReadValue(args, ref i), System.Globalization.CultureInfo.InvariantCulture); + break; + case "--render-buffer-seconds": + renderBufferSeconds = double.Parse(ReadValue(args, ref i), System.Globalization.CultureInfo.InvariantCulture); + break; + case "--render-drain-seconds": + renderDrainSeconds = double.Parse(ReadValue(args, ref i), System.Globalization.CultureInfo.InvariantCulture); + break; case "--help": case "-h": - Console.Error.WriteLine("Usage: Mimir.WasapiLoopback --output stdout --sample-rate 48000 --channels 2 [--seconds 10] [--role Console]"); + Console.Error.WriteLine("Usage: Mimir.WasapiLoopback [--list-devices] [--device Realtek] [--output stdout --sample-rate 48000 --channels 2 --seconds 10] [--play-f32-mono --input probe.raw --sample-rate 48000 --gain 1]"); Environment.Exit(0); break; default: @@ -341,7 +672,7 @@ internal static class Program } } - return new Options(output, seconds, sampleRate, channels, role); + return new Options(mode, input, output, seconds, sampleRate, channels, role, device, gain, renderBufferSeconds, renderDrainSeconds, listDevices); } private static string ReadValue(string[] args, ref int index) @@ -356,6 +687,12 @@ internal static class Program } } + private enum RunMode + { + Capture, + PlayF32Mono + } + private readonly struct SampleFormat { public SampleFormat(SampleKind kind, int containerBits, int validBits, Guid subFormat) @@ -443,12 +780,24 @@ internal static class Program private interface IMMDeviceEnumerator { [PreserveSig] - int EnumAudioEndpoints(int dataFlow, uint stateMask, out IntPtr devices); + int EnumAudioEndpoints(int dataFlow, uint stateMask, out IMMDeviceCollection devices); [PreserveSig] int GetDefaultAudioEndpoint(int dataFlow, int role, out IMMDevice endpoint); } + [ComImport] + [Guid("0BD7A1BE-7A1A-44DB-8397-CC5392387B5E")] + [InterfaceType(ComInterfaceType.InterfaceIsIUnknown)] + private interface IMMDeviceCollection + { + [PreserveSig] + int GetCount(out uint count); + + [PreserveSig] + int Item(uint index, out IMMDevice device); + } + [ComImport] [Guid("D666063F-1587-4E43-81F1-B948E807363F")] [InterfaceType(ComInterfaceType.InterfaceIsIUnknown)] @@ -456,6 +805,15 @@ internal static class Program { [PreserveSig] int Activate(ref Guid iid, int dwClsCtx, IntPtr pActivationParams, out IAudioClient audioClient); + + [PreserveSig] + int OpenPropertyStore(uint access, out IPropertyStore properties); + + [PreserveSig] + int GetId(out IntPtr id); + + [PreserveSig] + int GetState(out uint state); } [ComImport] @@ -497,7 +855,7 @@ internal static class Program int SetEventHandle(IntPtr eventHandle); [PreserveSig] - int GetService(ref Guid iid, out IAudioCaptureClient captureClient); + int GetService(ref Guid iid, [MarshalAs(UnmanagedType.IUnknown)] out object service); } [ComImport] @@ -514,4 +872,58 @@ internal static class Program [PreserveSig] int GetNextPacketSize(out uint frames); } + + [ComImport] + [Guid("F294ACFC-3146-4483-A7BF-ADDCA7C260E2")] + [InterfaceType(ComInterfaceType.InterfaceIsIUnknown)] + private interface IAudioRenderClient + { + [PreserveSig] + int GetBuffer(uint frames, out IntPtr data); + + [PreserveSig] + int ReleaseBuffer(uint frames, uint flags); + } + + [ComImport] + [Guid("886D8EEB-8CF2-4446-8D02-CDBA1DBDCF99")] + [InterfaceType(ComInterfaceType.InterfaceIsIUnknown)] + private interface IPropertyStore + { + [PreserveSig] + int GetCount(out uint propertyCount); + + [PreserveSig] + int GetAt(uint propertyIndex, out PropertyKey key); + + [PreserveSig] + int GetValue(ref PropertyKey key, out PropVariant value); + + [PreserveSig] + int SetValue(ref PropertyKey key, ref PropVariant value); + + [PreserveSig] + int Commit(); + } + + [StructLayout(LayoutKind.Sequential)] + private readonly struct PropertyKey(Guid formatId, uint propertyId) + { + public readonly Guid FormatId = formatId; + public readonly uint PropertyId = propertyId; + } + + [StructLayout(LayoutKind.Sequential)] + private struct PropVariant + { + public ushort ValueType; + public ushort Reserved1; + public ushort Reserved2; + public ushort Reserved3; + public IntPtr Pointer; + public int Int32Value; + } + + [DllImport("ole32.dll")] + private static extern int PropVariantClear(ref PropVariant variant); } diff --git a/state/evidence.jsonl b/state/evidence.jsonl index 318f4d2..20110b8 100644 --- a/state/evidence.jsonl +++ b/state/evidence.jsonl @@ -313,3 +313,4 @@ {"ts": "2026-06-01T03:11:14+01:00", "type": "verification", "status": "ok", "note": "Louder targeted bioacoustic probes ran at playback gains 4.0 and 8.0. Gain 4 artifact artifacts/asio/targeted-bioacoustic-live/20260601-031030 had shotgun peak 0.020941 and cardioid peak 0.016305; gain 8 artifact artifacts/asio/targeted-bioacoustic-live/20260601-031051 had shotgun peak 0.140715 and cardioid peak 0.027774, so the pass was audible but not clipped. Even at gain 8, 18.6 kHz stayed unusable: shotgun -54.910 dB and cardioid -63.258 dB vs loopback. Shotgun 910 Hz reached -26.549 dB with 0.237 confidence and 3.6 kHz reached -32.637 dB with 0.082 confidence. Current probe loop should bias usable low/mid anchors while treating ultrasonic/high band as failed path evidence."} {"ts": "2026-06-01T03:13:10+01:00", "type": "decision", "status": "ok", "note": "Routing correction: Starfire system default output is now Scarlett for monitoring, while Starfire Realtek is the intended active chirp-emission and loopback-measurement path. The 20260601 targeted ASIO-output probe artifacts remain useful Scarlett-output diagnostics but are not the intended Realtek chirp path. Next active probe runner must explicitly select Realtek output/loopback and must not assume default output or Scarlett ASIO playback."} {"ts": "2026-06-01T03:25:48+01:00", "type": "decision", "status": "ok", "note": "Added MimirBioacousticProbeScheduler as the closed-loop cadence owner above targeted probe planning. Runtime now publishes probe schedule/source confidence telemetry; scheduler owns when to spend active probes, while Realtek-specific physical emission remains the next output-device cut."} +{"ts": "2026-06-01T03:42:48+01:00", "type": "verification", "status": "ok", "note": "Realtek-targeted bioacoustic probe runner proved explicit Starfire Realtek emission through Mimir.WasapiLoopback. It selected Speakers (Realtek(R) Audio), played the targeted probe at gain 0.75, captured Scarlett inputs under artifacts/wasapi/targeted-bioacoustic-realtk-live/20260601-033851, and measured shotgun/cardioid response. The path is real, but 18.6 kHz remained near-zero confidence; planner defaults now cap probe selection at 12 kHz and bias measurable low/mid bands."} diff --git a/state/map.yaml b/state/map.yaml index 64363e1..9f9dd3d 100644 --- a/state/map.yaml +++ b/state/map.yaml @@ -666,6 +666,21 @@ accepted_design: 0.360 to 0.920 and frequency-response confidence from 0.558 to 0.955 over four scheduled probes. The scheduler owns cadence and telemetry only; it does not override the Realtek routing authority for physical emission. + The Realtek physical emission edge now exists in tooling: + `Mimir.WasapiLoopback` targets `net10.0-windows`, can list active render + endpoints, select by friendly-name substring, and play mono Float32 probes + through the selected WASAPI render endpoint. `Mimir.BufferSmoke + --targeted-bioacoustic-probe-realtk-live` couples that renderer with + Scarlett ASIO capture and scores the mic response. The first proof selected + `Speakers (Realtek(R) Audio)` and wrote + `artifacts/wasapi/targeted-bioacoustic-realtk-live/20260601-033851`; at + gain 0.75 the shotgun peak was 0.407887 and cardioid peak was 0.074494, + but 18.6 kHz remained near-zero confidence and the strongest printed band + was cardioid 3.6 kHz at 0.098 confidence. Realtek emission is proven; high + bands remain poor physical probe candidates for this route. The targeted + probe planner now defaults to a 12 kHz ceiling and biases measurable low/mid + bands so the scheduler asks questions the current speakers and mics can + plausibly answer. The first live targeted probe ran through Focusrite ASIO at 192 kHz. Gain 1.0 artifact `artifacts/asio/targeted-bioacoustic-live/20260601-030836` measured shotgun low/high tilt 20.199 dB and cardioid 15.082 dB. Gain 2.0