Record Kiyo with dual mic composite
This commit is contained in:
parent
abc28cc74a
commit
10090ee44c
7 changed files with 423 additions and 1 deletions
|
|
@ -148,6 +148,15 @@ contract on synthetic shotgun/cardioid witnesses: coherent shared tones survive,
|
|||
a mic-local `3.52 kHz` interferer is suppressed, and SNR rises from `1.366 dB`
|
||||
to `16.304 dB`. This is still the C# diagnostic/offline contract; Faust/native
|
||||
DSP owns the production hot lane.
|
||||
The first listenable bridge receipt is
|
||||
`scripts/record-kiyo-dual-mic-composite.ps1`. It records Kiyo video and
|
||||
Scarlett ASIO raw channels together, converts Scarlett channels 0/1 into a
|
||||
computed dual-mic composite WAV with `--asio-dual-mic-composite-wav`, then muxes
|
||||
that WAV with the Kiyo video. The 2026-06-01 receipt at
|
||||
`artifacts/runtime/kiyo-dual-mic-composite/kiyo-dual-mic-composite-latest.mp4`
|
||||
contains 10.000 s of 640x480 30 fps Kiyo video and 9.994 s of mono AAC from the
|
||||
computed composite. Treat it as a bridge proof of audibility/sync, not the final
|
||||
native live DSP path.
|
||||
The synthetic invariant is
|
||||
`dotnet run --project .\src\Mimir.BufferSmoke\Mimir.BufferSmoke.csproj -- --bioacoustic-self-test`;
|
||||
it renders the bioacoustic timeline, decodes direct word anchors, and requires a
|
||||
|
|
|
|||
|
|
@ -814,7 +814,11 @@ a named invariant that the native runtime cannot protect yet.
|
|||
calibrated-band coherence and suppresses incoherent measured-band components
|
||||
before summing; `--dual-mic-coherence-cancellation-smoke` proves the two-mic
|
||||
case by cutting a mic-local 3.52 kHz interferer while preserving coherent
|
||||
shared tones. Treat that as the contract and offline scorer for the
|
||||
shared tones. `scripts/record-kiyo-dual-mic-composite.ps1` is the first
|
||||
listenable bridge: it records Kiyo video plus Scarlett ASIO raw channels,
|
||||
renders the channels 0/1 composite WAV with `--asio-dual-mic-composite-wav`,
|
||||
and muxes the computed mono composite into the Kiyo MP4. Treat that as the
|
||||
contract and offline scorer for the
|
||||
native/Faust implementation, not as a parallel live mixer authority. The
|
||||
next matching probe loop also exists as
|
||||
`MimirTargetedBioacousticProbePlanner`: residuals and mic pathology notes
|
||||
|
|
|
|||
|
|
@ -317,6 +317,15 @@ two-witness cancellation cut: shared tones keep about 0.999 coherence, a
|
|||
mic-local 3.52 kHz interferer scores about 0.001 coherence and 0.948
|
||||
suppression, its measured energy falls from 0.039195 to 0.001306, and SNR rises
|
||||
from 1.366 dB to 16.304 dB.
|
||||
`scripts/record-kiyo-dual-mic-composite.ps1` is the first listenable bridge
|
||||
receipt for that path. It starts Kiyo FFmpeg capture and Scarlett ASIO raw
|
||||
capture together, converts ASIO channels 0/1 to a computed composite WAV through
|
||||
`--asio-dual-mic-composite-wav`, then muxes the composite WAV with Kiyo video.
|
||||
The 2026-06-01 receipt
|
||||
`artifacts/runtime/kiyo-dual-mic-composite/kiyo-dual-mic-composite-latest.mp4`
|
||||
has 10.000 s of 640x480 30 fps Kiyo video and 9.994 s of mono AAC composite
|
||||
audio; the composite report estimated channel-1 delay at about -141.019 samples
|
||||
(-2937.889 us).
|
||||
`MimirTargetedBioacousticProbePlanner` is the matching "ask the room a better
|
||||
question" owner. It consumes per-source spectral residuals and optional source
|
||||
pathology notes, then emits gain-bounded shaped probe bands for the
|
||||
|
|
|
|||
158
scripts/record-kiyo-dual-mic-composite.ps1
Normal file
158
scripts/record-kiyo-dual-mic-composite.ps1
Normal file
|
|
@ -0,0 +1,158 @@
|
|||
param(
|
||||
[int]$DurationSeconds = 15,
|
||||
[string]$OutputPath = "artifacts\runtime\kiyo-dual-mic-composite\kiyo-dual-mic-composite.mp4",
|
||||
[string]$LogRoot = "artifacts\runtime\kiyo-dual-mic-composite",
|
||||
[string]$FfmpegPath = "ffmpeg",
|
||||
[string]$AsioProbePath = "native\probes\asio_audio_cadence\build\Release\asio_audio_cadence.exe",
|
||||
[string]$KiyoVideoDevice = "USB Video Device",
|
||||
[int]$KiyoWidth = 640,
|
||||
[int]$KiyoHeight = 480,
|
||||
[int]$KiyoFrameRate = 30,
|
||||
[int]$SampleRate = 48000,
|
||||
[int]$AsioChannels = 4,
|
||||
[int]$ShotgunChannel = 0,
|
||||
[int]$CardioidChannel = 1,
|
||||
[switch]$DryRun
|
||||
)
|
||||
|
||||
$ErrorActionPreference = "Stop"
|
||||
|
||||
function Quote-ProcessArgument([string]$Value) {
|
||||
if ($Value.Length -eq 0) {
|
||||
return '""'
|
||||
}
|
||||
|
||||
if ($Value -notmatch '[\s"]') {
|
||||
return $Value
|
||||
}
|
||||
|
||||
return '"' + $Value.Replace('"', '\"') + '"'
|
||||
}
|
||||
|
||||
function Start-LoggedProcess([string]$FileName, [string[]]$Arguments, [string]$StdoutPath, [string]$StderrPath) {
|
||||
$startInfo = [System.Diagnostics.ProcessStartInfo]::new()
|
||||
$startInfo.FileName = $FileName
|
||||
$startInfo.UseShellExecute = $false
|
||||
$startInfo.RedirectStandardOutput = $true
|
||||
$startInfo.RedirectStandardError = $true
|
||||
$startInfo.Arguments = ($Arguments | ForEach-Object { Quote-ProcessArgument $_ }) -join " "
|
||||
$process = [System.Diagnostics.Process]::Start($startInfo)
|
||||
if (-not $process) {
|
||||
throw "Could not start $FileName"
|
||||
}
|
||||
|
||||
$stdoutStream = [System.IO.File]::Open($StdoutPath, [System.IO.FileMode]::Create, [System.IO.FileAccess]::Write, [System.IO.FileShare]::Read)
|
||||
$stderrStream = [System.IO.File]::Open($StderrPath, [System.IO.FileMode]::Create, [System.IO.FileAccess]::Write, [System.IO.FileShare]::Read)
|
||||
return [pscustomobject]@{
|
||||
Process = $process
|
||||
StdoutTask = $process.StandardOutput.BaseStream.CopyToAsync($stdoutStream)
|
||||
StderrTask = $process.StandardError.BaseStream.CopyToAsync($stderrStream)
|
||||
StdoutStream = $stdoutStream
|
||||
StderrStream = $stderrStream
|
||||
}
|
||||
}
|
||||
|
||||
function Wait-LoggedProcess($Handle, [int]$TimeoutSeconds, [string]$Name) {
|
||||
if (-not $Handle.Process.WaitForExit($TimeoutSeconds * 1000)) {
|
||||
try { $Handle.Process.Kill($true) } catch {}
|
||||
throw "$Name timed out after ${TimeoutSeconds}s."
|
||||
}
|
||||
|
||||
[void]$Handle.StdoutTask.GetAwaiter().GetResult()
|
||||
[void]$Handle.StderrTask.GetAwaiter().GetResult()
|
||||
$Handle.StdoutStream.Dispose()
|
||||
$Handle.StderrStream.Dispose()
|
||||
if ($Handle.Process.ExitCode -ne 0) {
|
||||
throw "$Name failed with exit code $($Handle.Process.ExitCode)."
|
||||
}
|
||||
}
|
||||
|
||||
$repo = Split-Path -Parent $PSScriptRoot
|
||||
$timestamp = Get-Date -Format "yyyyMMdd-HHmmss"
|
||||
$absoluteLogRoot = if ([System.IO.Path]::IsPathRooted($LogRoot)) { $LogRoot } else { Join-Path $repo $LogRoot }
|
||||
$runDirectory = Join-Path $absoluteLogRoot $timestamp
|
||||
New-Item -ItemType Directory -Force -Path $runDirectory | Out-Null
|
||||
|
||||
$absoluteOutputPath = if ([System.IO.Path]::IsPathRooted($OutputPath)) { $OutputPath } else { Join-Path $repo $OutputPath }
|
||||
$outputDirectory = Split-Path -Parent $absoluteOutputPath
|
||||
New-Item -ItemType Directory -Force -Path $outputDirectory | Out-Null
|
||||
|
||||
$asioRaw = Join-Path $runDirectory "scarlett-f32-interleaved.raw"
|
||||
$kiyoMkv = Join-Path $runDirectory "kiyo-video.mkv"
|
||||
$compositeWav = Join-Path $runDirectory "dual-mic-composite.wav"
|
||||
$absoluteAsioProbe = if ([System.IO.Path]::IsPathRooted($AsioProbePath)) { $AsioProbePath } else { Join-Path $repo $AsioProbePath }
|
||||
|
||||
$asioArgs = @(
|
||||
"--set-sample-rate", $SampleRate.ToString(),
|
||||
"--record-f32-interleaved", $asioRaw,
|
||||
"--capture-seconds", $DurationSeconds.ToString()
|
||||
)
|
||||
$kiyoArgs = @(
|
||||
"-hide_banner",
|
||||
"-y",
|
||||
"-thread_queue_size", "1024",
|
||||
"-rtbufsize", "256M",
|
||||
"-f", "dshow",
|
||||
"-video_size", "${KiyoWidth}x${KiyoHeight}",
|
||||
"-framerate", $KiyoFrameRate.ToString(),
|
||||
"-i", "video=$KiyoVideoDevice",
|
||||
"-t", $DurationSeconds.ToString(),
|
||||
"-r", $KiyoFrameRate.ToString(),
|
||||
"-c:v", "h264_nvenc",
|
||||
"-preset", "p4",
|
||||
"-tune", "ll",
|
||||
"-pix_fmt", "yuv420p",
|
||||
$kiyoMkv
|
||||
)
|
||||
|
||||
Write-Host "Kiyo + computed dual-mic composite recording:"
|
||||
Write-Host " Kiyo: $KiyoVideoDevice ${KiyoWidth}x${KiyoHeight}@$KiyoFrameRate"
|
||||
Write-Host " ASIO: $absoluteAsioProbe sampleRate=$SampleRate channels=$AsioChannels shotgun=$ShotgunChannel cardioid=$CardioidChannel"
|
||||
Write-Host " Run dir: $runDirectory"
|
||||
Write-Host " Output: $absoluteOutputPath"
|
||||
|
||||
if ($DryRun) {
|
||||
Write-Host ($absoluteAsioProbe + " " + ($asioArgs -join " "))
|
||||
Write-Host ($FfmpegPath + " " + ($kiyoArgs -join " "))
|
||||
exit 0
|
||||
}
|
||||
|
||||
$asio = Start-LoggedProcess $absoluteAsioProbe $asioArgs (Join-Path $runDirectory "asio.out.log") (Join-Path $runDirectory "asio.err.log")
|
||||
$kiyo = Start-LoggedProcess $FfmpegPath $kiyoArgs (Join-Path $runDirectory "kiyo.out.log") (Join-Path $runDirectory "kiyo.err.log")
|
||||
|
||||
Wait-LoggedProcess $asio ($DurationSeconds + 15) "ASIO capture"
|
||||
Wait-LoggedProcess $kiyo ($DurationSeconds + 15) "Kiyo capture"
|
||||
|
||||
dotnet run --project (Join-Path $repo "src\Mimir.BufferSmoke\Mimir.BufferSmoke.csproj") -- `
|
||||
--asio-dual-mic-composite-wav `
|
||||
--input $asioRaw `
|
||||
--output $compositeWav `
|
||||
--sample-rate $SampleRate `
|
||||
--channels $AsioChannels `
|
||||
--shotgun-channel $ShotgunChannel `
|
||||
--cardioid-channel $CardioidChannel
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
throw "Dual-mic composite WAV generation failed."
|
||||
}
|
||||
|
||||
$muxArgs = @(
|
||||
"-hide_banner",
|
||||
"-y",
|
||||
"-i", $kiyoMkv,
|
||||
"-i", $compositeWav,
|
||||
"-map", "0:v:0",
|
||||
"-map", "1:a:0",
|
||||
"-c:v", "copy",
|
||||
"-c:a", "aac",
|
||||
"-b:a", "192k",
|
||||
"-ar", "48000",
|
||||
"-movflags", "+faststart",
|
||||
$absoluteOutputPath
|
||||
)
|
||||
$mux = Start-LoggedProcess $FfmpegPath $muxArgs (Join-Path $runDirectory "mux.out.log") (Join-Path $runDirectory "mux.err.log")
|
||||
Wait-LoggedProcess $mux 60 "Mux"
|
||||
|
||||
Write-Host "Recording complete:"
|
||||
Write-Host " $absoluteOutputPath"
|
||||
Write-Host " composite wav: $compositeWav"
|
||||
Write-Host " report: $([System.IO.Path]::ChangeExtension($compositeWav, '.json'))"
|
||||
|
|
@ -85,6 +85,17 @@ if (args.Any(arg => string.Equals(arg, "--dual-mic-coherence-cancellation-smoke"
|
|||
return RunDualMicCoherenceCancellationSmoke();
|
||||
}
|
||||
|
||||
if (args.Any(arg => string.Equals(arg, "--asio-dual-mic-composite-wav", StringComparison.OrdinalIgnoreCase)))
|
||||
{
|
||||
return RunAsioDualMicCompositeWave(
|
||||
ParseStringOption(args, "--input", ""),
|
||||
ParseStringOption(args, "--output", ""),
|
||||
ParseIntOption(args, "--sample-rate", 48_000),
|
||||
ParseIntOption(args, "--channels", 4),
|
||||
ParseIntOption(args, "--shotgun-channel", 0),
|
||||
ParseIntOption(args, "--cardioid-channel", 1));
|
||||
}
|
||||
|
||||
if (args.Any(arg => string.Equals(arg, "--targeted-bioacoustic-probe-smoke", StringComparison.OrdinalIgnoreCase)))
|
||||
{
|
||||
return RunTargetedBioacousticProbeSmoke();
|
||||
|
|
@ -3473,6 +3484,82 @@ static int RunDualMicCoherenceCancellationSmoke()
|
|||
: 1;
|
||||
}
|
||||
|
||||
static int RunAsioDualMicCompositeWave(
|
||||
string inputPath,
|
||||
string outputPath,
|
||||
int sampleRate,
|
||||
int channels,
|
||||
int shotgunChannel,
|
||||
int cardioidChannel)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(inputPath) || !File.Exists(inputPath))
|
||||
{
|
||||
Console.Error.WriteLine("asio-dual-mic-composite-wav failed: --input must name an existing f32 interleaved ASIO capture.");
|
||||
return 1;
|
||||
}
|
||||
|
||||
if (string.IsNullOrWhiteSpace(outputPath))
|
||||
{
|
||||
Console.Error.WriteLine("asio-dual-mic-composite-wav failed: --output is required.");
|
||||
return 1;
|
||||
}
|
||||
|
||||
if (shotgunChannel < 0 || cardioidChannel < 0 || shotgunChannel >= channels || cardioidChannel >= channels)
|
||||
{
|
||||
Console.Error.WriteLine($"asio-dual-mic-composite-wav failed: channel indices must fit 0..{channels - 1}.");
|
||||
return 1;
|
||||
}
|
||||
|
||||
var channelSamples = ReadInterleavedFloat32(inputPath, channels, out var frameCount);
|
||||
if (frameCount == 0)
|
||||
{
|
||||
Console.Error.WriteLine("asio-dual-mic-composite-wav failed: capture has no frames.");
|
||||
return 1;
|
||||
}
|
||||
|
||||
var shotgun = channelSamples[shotgunChannel];
|
||||
var cardioid = channelSamples[cardioidChannel];
|
||||
var delay = EstimateRelativeDelaySamples(shotgun, cardioid, maxLagSamples: Math.Min(sampleRate / 100, 960));
|
||||
var bands = BuildCompositeProbeBands(sampleRate);
|
||||
var shotgunResponses = EstimateBandResponses(shotgun, sampleRate, bands);
|
||||
var cardioidResponses = EstimateBandResponses(cardioid, sampleRate, bands);
|
||||
var result = new MimirCalibratedAudioCompositeBuilder(new MimirCalibratedAudioCompositeOptions(
|
||||
MinimumConfidence: 0.05,
|
||||
TargetBandEnergy: MedianPositiveEnergy(shotgunResponses.Concat(cardioidResponses).Select(static response => response.Energy)),
|
||||
MinimumBandEnergy: 1.0e-8,
|
||||
MinimumBandGain: 0.65,
|
||||
MaximumBandGain: 1.60,
|
||||
NoiseFloorRms: 1.0e-5,
|
||||
MinimumCoherence: 0.42,
|
||||
IncoherentBandSuppression: 0.62)).Build(
|
||||
[
|
||||
NewCompositeSourceFromResponses("asio-ch0-shotgun", shotgun, sampleRate, delaySamples: 0.0, confidence: 0.90, shotgunResponses, NoiseEstimate(shotgun)),
|
||||
NewCompositeSourceFromResponses("asio-ch1-cardioid", cardioid, sampleRate, delaySamples: delay, confidence: 0.90, cardioidResponses, NoiseEstimate(cardioid)),
|
||||
]);
|
||||
var normalized = NormalizePeak(result.Samples, peak: 0.92f);
|
||||
WriteWave(outputPath, normalized, sampleRate);
|
||||
var reportPath = Path.ChangeExtension(outputPath, ".json");
|
||||
var report = new
|
||||
{
|
||||
input = Path.GetFullPath(inputPath),
|
||||
output = Path.GetFullPath(outputPath),
|
||||
sampleRate,
|
||||
frameCount,
|
||||
shotgunChannel,
|
||||
cardioidChannel,
|
||||
estimatedCardioidDelaySamples = delay,
|
||||
estimatedCardioidDelayMicroseconds = delay * 1_000_000.0 / sampleRate,
|
||||
compositeRms = result.CompositeRms,
|
||||
sourceReports = result.SourceReports,
|
||||
bandReports = result.BandReports,
|
||||
};
|
||||
File.WriteAllText(reportPath, JsonSerializer.Serialize(report, new JsonSerializerOptions { WriteIndented = true }));
|
||||
Console.WriteLine(
|
||||
$"asio-dual-mic-composite-wav input={inputPath} output={outputPath} report={reportPath} frames={frameCount} " +
|
||||
$"delaySamples={delay:0.000} delayUs={delay * 1_000_000.0 / sampleRate:0.000} bands={result.BandReports.Count} rms={result.CompositeRms:0.000000}");
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int RunTargetedBioacousticProbeSmoke()
|
||||
{
|
||||
var residuals = new[]
|
||||
|
|
@ -4143,6 +4230,31 @@ static MimirCalibratedAudioCompositeSource NewCompositeSource(
|
|||
return new MimirCalibratedAudioCompositeSource(sourceId, 48_000, samples, state, responses, noiseRms);
|
||||
}
|
||||
|
||||
static MimirCalibratedAudioCompositeSource NewCompositeSourceFromResponses(
|
||||
string sourceId,
|
||||
float[] samples,
|
||||
int sampleRate,
|
||||
double delaySamples,
|
||||
double confidence,
|
||||
IReadOnlyList<MimirChirpletBandResponse> responses,
|
||||
double noiseRms)
|
||||
{
|
||||
var state = new MimirAudioSynchronizationState(
|
||||
"scarlett-asio-direct",
|
||||
sourceId,
|
||||
sampleRate,
|
||||
delaySamples,
|
||||
delaySamples,
|
||||
delaySamples * 1000.0 / sampleRate,
|
||||
0.0,
|
||||
confidence,
|
||||
responses,
|
||||
1_000_000_000L,
|
||||
1,
|
||||
1);
|
||||
return new MimirCalibratedAudioCompositeSource(sourceId, sampleRate, samples, state, responses, noiseRms);
|
||||
}
|
||||
|
||||
static MimirAudioCalibrationBandResidual[] ScarlettProfessionalMicProbeResiduals() =>
|
||||
[
|
||||
new MimirAudioCalibrationBandResidual("asio-ch0-shotgun", 240.0, 0.91, 0.4, ResponseEnergy: 0.62),
|
||||
|
|
@ -4429,6 +4541,126 @@ static double ToneEnergy(ReadOnlySpan<float> samples, int sampleRate, double cen
|
|||
return Math.Sqrt(sine * sine + cosine * cosine) * 2.0 / Math.Max(1, count);
|
||||
}
|
||||
|
||||
static double EstimateRelativeDelaySamples(IReadOnlyList<float> reference, IReadOnlyList<float> candidate, int maxLagSamples)
|
||||
{
|
||||
var frameCount = Math.Min(reference.Count, candidate.Count);
|
||||
if (frameCount == 0)
|
||||
{
|
||||
return 0.0;
|
||||
}
|
||||
|
||||
var bestLag = 0;
|
||||
var bestScore = double.NegativeInfinity;
|
||||
for (var lag = -maxLagSamples; lag <= maxLagSamples; lag++)
|
||||
{
|
||||
var score = 0.0;
|
||||
var refEnergy = 0.0;
|
||||
var candEnergy = 0.0;
|
||||
var start = Math.Max(0, -lag);
|
||||
var end = Math.Min(frameCount, frameCount - lag);
|
||||
for (var index = start; index < end; index++)
|
||||
{
|
||||
var left = reference[index];
|
||||
var right = candidate[index + lag];
|
||||
score += left * right;
|
||||
refEnergy += left * left;
|
||||
candEnergy += right * right;
|
||||
}
|
||||
|
||||
var normalized = score / Math.Sqrt(Math.Max(1.0e-18, refEnergy * candEnergy));
|
||||
if (normalized > bestScore)
|
||||
{
|
||||
bestScore = normalized;
|
||||
bestLag = lag;
|
||||
}
|
||||
}
|
||||
|
||||
if (bestLag <= -maxLagSamples || bestLag >= maxLagSamples)
|
||||
{
|
||||
return bestLag;
|
||||
}
|
||||
|
||||
var leftScore = DelayCorrelation(reference, candidate, bestLag - 1);
|
||||
var centerScore = DelayCorrelation(reference, candidate, bestLag);
|
||||
var rightScore = DelayCorrelation(reference, candidate, bestLag + 1);
|
||||
var denominator = leftScore - 2.0 * centerScore + rightScore;
|
||||
var fractional = Math.Abs(denominator) <= 1.0e-12 ? 0.0 : 0.5 * (leftScore - rightScore) / denominator;
|
||||
return bestLag + Math.Clamp(fractional, -0.5, 0.5);
|
||||
}
|
||||
|
||||
static double DelayCorrelation(IReadOnlyList<float> reference, IReadOnlyList<float> candidate, int lag)
|
||||
{
|
||||
var frameCount = Math.Min(reference.Count, candidate.Count);
|
||||
var score = 0.0;
|
||||
var refEnergy = 0.0;
|
||||
var candEnergy = 0.0;
|
||||
var start = Math.Max(0, -lag);
|
||||
var end = Math.Min(frameCount, frameCount - lag);
|
||||
for (var index = start; index < end; index++)
|
||||
{
|
||||
var left = reference[index];
|
||||
var right = candidate[index + lag];
|
||||
score += left * right;
|
||||
refEnergy += left * left;
|
||||
candEnergy += right * right;
|
||||
}
|
||||
|
||||
return score / Math.Sqrt(Math.Max(1.0e-18, refEnergy * candEnergy));
|
||||
}
|
||||
|
||||
static double[] BuildCompositeProbeBands(int sampleRate)
|
||||
{
|
||||
var nyquistGuard = sampleRate * 0.45;
|
||||
return new[] { 160.0, 250.0, 400.0, 630.0, 1_000.0, 1_600.0, 2_500.0, 3_600.0, 5_600.0, 8_000.0, 10_000.0 }
|
||||
.Where(frequency => frequency < nyquistGuard)
|
||||
.ToArray();
|
||||
}
|
||||
|
||||
static MimirChirpletBandResponse[] EstimateBandResponses(IReadOnlyList<float> samples, int sampleRate, IReadOnlyList<double> bands) =>
|
||||
bands
|
||||
.Select(centerHz =>
|
||||
{
|
||||
var magnitude = ToneEnergy(samples is float[] array ? array : samples.ToArray(), sampleRate, centerHz);
|
||||
return new MimirChirpletBandResponse(centerHz, magnitude * magnitude);
|
||||
})
|
||||
.ToArray();
|
||||
|
||||
static double MedianPositiveEnergy(IEnumerable<double> energies)
|
||||
{
|
||||
var sorted = energies
|
||||
.Where(static energy => energy > 1.0e-10)
|
||||
.Order()
|
||||
.ToArray();
|
||||
return sorted.Length == 0 ? 0.01 : Math.Clamp(sorted[sorted.Length / 2], 0.0001, 1.0);
|
||||
}
|
||||
|
||||
static double NoiseEstimate(IReadOnlyList<float> samples)
|
||||
{
|
||||
var sorted = samples
|
||||
.Select(static sample => Math.Abs(sample))
|
||||
.Order()
|
||||
.ToArray();
|
||||
if (sorted.Length == 0)
|
||||
{
|
||||
return 1.0e-5;
|
||||
}
|
||||
|
||||
var percentile = sorted[Math.Clamp((int)Math.Round((sorted.Length - 1) * 0.20), 0, sorted.Length - 1)];
|
||||
return Math.Clamp(percentile * 1.4826, 1.0e-5, 0.25);
|
||||
}
|
||||
|
||||
static float[] NormalizePeak(IReadOnlyList<float> samples, float peak)
|
||||
{
|
||||
var max = samples.Select(static sample => Math.Abs(sample)).DefaultIfEmpty(0.0f).Max();
|
||||
if (max <= peak || max <= 1.0e-9f)
|
||||
{
|
||||
return samples.ToArray();
|
||||
}
|
||||
|
||||
var scale = peak / max;
|
||||
return samples.Select(sample => sample * scale).ToArray();
|
||||
}
|
||||
|
||||
static float[] RenderCalibrationToneStack(
|
||||
int frameCount,
|
||||
int sampleRate,
|
||||
|
|
|
|||
|
|
@ -317,3 +317,4 @@
|
|||
{"ts": "2026-06-01T03:54:22+01:00", "type": "verification", "status": "ok", "note": "Ran scheduler-driven Realtek calibration under artifacts/wasapi/realtk-scheduled-calibration-live/20260601-035354. The runner used MimirBioacousticProbeScheduler to choose four targeted probe steps, emitted through Speakers (Realtek(R) Audio) at gain 1.15, captured Scarlett ASIO, and wrote scheduled-response-bands.csv plus balance-plan.md. Local peak-vs-percentile-floor scoring found strong usable response around 910 Hz, 3.6 kHz, and 9.84-9.87 kHz on both shotgun and cardioid; both channels scored high SNR in those asked bands."}
|
||||
{"ts": "2026-06-01T04:01:53+01:00", "type": "verification", "status": "ok", "note": "Bioacoustic scheduler now shares the targeted probe planner's measurable frequency boundary. The scheduler filters residual confidence to schedulable bands before choosing cadence/status and passes the same 120 Hz-12 kHz window to the planner, so unmeasurable 18.5 kHz residuals no longer keep live status pinned to a band the Realtek/speaker/Scarlett path cannot answer. Verified with --bioacoustic-probe-scheduler-smoke and --targeted-bioacoustic-probe-smoke."}
|
||||
{"ts": "2026-06-01T04:13:53+01:00", "type": "verification", "status": "ok", "note": "Added calibrated-band coherence suppression to MimirCalibratedAudioCompositeBuilder. The composite path now aligns sources, applies measured-band correction, scores phase/energy coherence per calibrated band, suppresses incoherent measured-band components, and reports source/band coherence. Verified with --dual-mic-coherence-cancellation-smoke: shared tones held ~0.999 coherence, a mic-local 3.52 kHz interferer scored ~0.001 coherence with ~0.948 suppression, poison energy fell 0.039195 to 0.001306, and SNR rose 1.366 dB to 16.304 dB."}
|
||||
{"ts": "2026-06-01T04:24:57+01:00", "type": "verification", "status": "ok", "note": "Recorded a listenable Kiyo plus computed dual-mic composite bridge receipt at artifacts/runtime/kiyo-dual-mic-composite/kiyo-dual-mic-composite-latest.mp4. The script record-kiyo-dual-mic-composite.ps1 started Kiyo FFmpeg capture and Scarlett ASIO raw capture together, generated dual-mic-composite.wav via --asio-dual-mic-composite-wav, then muxed it. ffprobe verified 10.000 s of 640x480 30 fps H.264 video plus 9.994 s of 48 kHz mono AAC. The composite report estimated cardioid delay -141.019 samples (-2937.889 us) relative to shotgun and emitted 11 coherence band reports."}
|
||||
|
|
|
|||
|
|
@ -649,6 +649,15 @@ accepted_design:
|
|||
mic-local 3.52 kHz interferer scores about 0.001 coherence and 0.948
|
||||
suppression, its measured energy falls from 0.039195 to 0.001306, and SNR
|
||||
rises from 1.366 dB to 16.304 dB.
|
||||
`scripts/record-kiyo-dual-mic-composite.ps1` is the first listenable bridge
|
||||
receipt for that path. It starts Kiyo FFmpeg capture and Scarlett ASIO raw
|
||||
capture together, converts ASIO channels 0/1 to a computed composite WAV
|
||||
through `--asio-dual-mic-composite-wav`, then muxes the composite WAV with
|
||||
Kiyo video. The 2026-06-01 receipt
|
||||
`artifacts/runtime/kiyo-dual-mic-composite/kiyo-dual-mic-composite-latest.mp4`
|
||||
has 10.000 s of 640x480 30 fps Kiyo video and 9.994 s of mono AAC composite
|
||||
audio; the composite report estimated channel-1 delay at about -141.019
|
||||
samples (-2937.889 us).
|
||||
Faust/native DSP still owns the production hot lane; this C# path is a
|
||||
contract, scorer, and offline diagnostic, not a competing live mixer.
|
||||
`MimirTargetedBioacousticProbePlanner` is now the residual-driven probe
|
||||
|
|
|
|||
Loading…
Reference in a new issue