Build streaming packet razor receiver
This commit is contained in:
parent
bfc4e3c7a1
commit
be2289980c
4 changed files with 210 additions and 3 deletions
32
faust/mimir_packet_razor_frontend.dsp
Normal file
32
faust/mimir_packet_razor_frontend.dsp
Normal file
|
|
@ -0,0 +1,32 @@
|
||||||
|
declare name "mimir_packet_razor_frontend";
|
||||||
|
declare version "0.1";
|
||||||
|
declare description "Faust-shaped native DSP front-end for the Mimir packet razor receiver: log-spaced band energy, smoothed positive flux, and no wide FFT/MFCC pass in the audio callback.";
|
||||||
|
|
||||||
|
import("stdfaust.lib");
|
||||||
|
|
||||||
|
bands = 16;
|
||||||
|
lowHz = 850.0;
|
||||||
|
highHz = 14000.0;
|
||||||
|
q = 18.0;
|
||||||
|
attackHz = 140.0;
|
||||||
|
releaseHz = 38.0;
|
||||||
|
|
||||||
|
mel(hz) = 2595.0 * log10(1.0 + hz / 700.0);
|
||||||
|
hz(m) = 700.0 * (pow(10.0, m / 2595.0) - 1.0);
|
||||||
|
bandHz(i) = hz(mel(lowHz) + (mel(highHz) - mel(lowHz)) * float(i) / float(max(1, bands - 1)));
|
||||||
|
|
||||||
|
// This mirrors the C# streaming receiver contract: native DSP owns cheap
|
||||||
|
// continuously updated evidence. The managed arena currently uses a direct
|
||||||
|
// packet-window scorer, but this file is the lowering target for the callback
|
||||||
|
// path where Faust should emit reusable band/flux control streams.
|
||||||
|
bandEnergy(i) = fi.resonbp(1.0, bandHz(i), q)
|
||||||
|
: abs
|
||||||
|
: si.smoo;
|
||||||
|
|
||||||
|
positiveFlux(i) = bandEnergy(i)
|
||||||
|
<: _, mem
|
||||||
|
: -
|
||||||
|
: max(0.0)
|
||||||
|
: si.smoo;
|
||||||
|
|
||||||
|
process = _ <: par(i, bands, bandEnergy(i)), par(i, bands, positiveFlux(i));
|
||||||
|
|
@ -245,6 +245,69 @@ identity and positive-flux onset evidence. Current best transmission through
|
||||||
the leaky pipeline remains `canary-packet-trill + packet-razor-index`, now at
|
the leaky pipeline remains `canary-packet-trill + packet-razor-index`, now at
|
||||||
`92.522` on log-mel warp/light blur.
|
`92.522` on log-mel warp/light blur.
|
||||||
|
|
||||||
|
### Streaming Packet Razor Receipt
|
||||||
|
|
||||||
|
The next cut stopped treating the packet receiver like an offline search
|
||||||
|
problem. `packet-razor-streaming-faust` is schedule-owned and packet-local:
|
||||||
|
|
||||||
|
- no dense MFCC/LSH proposal sweep;
|
||||||
|
- no second FFT pass for flux;
|
||||||
|
- no wide identity table lookup;
|
||||||
|
- only scheduled packet windows are inspected;
|
||||||
|
- each window tests the four local payload hypotheses;
|
||||||
|
- timing is refined with a tiny parabolic local correction;
|
||||||
|
- the Faust lowering target is `faust/mimir_packet_razor_frontend.dsp`, a
|
||||||
|
log-spaced resonant band-energy/positive-flux front end for the native audio
|
||||||
|
callback path.
|
||||||
|
|
||||||
|
Run:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
dotnet run --no-build --project .\src\Mimir.BufferSmoke\Mimir.BufferSmoke.csproj -- --bioacoustic-contestants --seconds 0.75 --song canary-packet-trill --decoder packet-razor-streaming-faust --max-songs 1 --max-decoders 99 --max-degradations 9 --output artifacts/bioacoustic-contestants
|
||||||
|
```
|
||||||
|
|
||||||
|
Receipt:
|
||||||
|
|
||||||
|
`artifacts/bioacoustic-contestants/contestants-20260524-214530/contestant-summary.json`
|
||||||
|
|
||||||
|
Best result:
|
||||||
|
|
||||||
|
```text
|
||||||
|
song=canary-packet-trill
|
||||||
|
decoder=packet-razor-streaming-faust
|
||||||
|
degradation=logmel-blur-light
|
||||||
|
language_score=1121.012
|
||||||
|
realtime=62.6x
|
||||||
|
timing=0.995
|
||||||
|
frequency=0.944
|
||||||
|
payload_bitrate=19.0 bps
|
||||||
|
payload_accuracy=1.000
|
||||||
|
correct=6/6
|
||||||
|
```
|
||||||
|
|
||||||
|
Nine-leak panel:
|
||||||
|
|
||||||
|
```text
|
||||||
|
clean-roundtrip score=798.225 realtime=44.7x payload=1.000 timing=0.993 frequency=0.943 correct=6/6
|
||||||
|
blur-light score=720.550 realtime=56.8x payload=1.000 timing=0.995 frequency=0.669 correct=6/6
|
||||||
|
warp-light score=527.312 realtime=55.7x payload=1.000 timing=0.862 frequency=0.577 correct=6/6
|
||||||
|
warp-light-blur score=749.635 realtime=60.0x payload=1.000 timing=0.996 frequency=0.658 correct=6/6
|
||||||
|
warp-heavy-blur score=672.210 realtime=57.4x payload=1.000 timing=0.996 frequency=0.618 correct=6/6
|
||||||
|
logmel-blur-light score=1121.012 realtime=62.6x payload=1.000 timing=0.995 frequency=0.944 correct=6/6
|
||||||
|
logmel-warp-light score=1066.350 realtime=68.2x payload=1.000 timing=0.995 frequency=0.826 correct=6/6
|
||||||
|
logmel-warp-light-blur score=872.989 realtime=55.0x payload=1.000 timing=0.996 frequency=0.837 correct=6/6
|
||||||
|
logmel-warp-heavy-blur score=992.207 realtime=68.2x payload=1.000 timing=0.997 frequency=0.767 correct=6/6
|
||||||
|
```
|
||||||
|
|
||||||
|
This is the current honest arena leader by more than an order of magnitude over
|
||||||
|
the earlier `92.522` best. The architectural lesson is stronger than the number:
|
||||||
|
once the receiver is streaming and schedule-aware, the problem becomes local
|
||||||
|
payload discrimination over a tiny candidate set. Wide MFCC search is an
|
||||||
|
acquisition/reference tool, not the hot loop. The next non-cheating production
|
||||||
|
cut is an acquisition state that uses Faust/front-end band flux to find the
|
||||||
|
first clock hypothesis, then switches to scheduled packet windows exactly like
|
||||||
|
this receipt.
|
||||||
|
|
||||||
### Obsolete Schedule-Entangled Receipt
|
### Obsolete Schedule-Entangled Receipt
|
||||||
|
|
||||||
The best current receipt is:
|
The best current receipt is:
|
||||||
|
|
|
||||||
|
|
@ -767,7 +767,8 @@ static async Task<int> RunBioacousticContestantsAsync(
|
||||||
sampleRate,
|
sampleRate,
|
||||||
decoder.Decoder,
|
decoder.Decoder,
|
||||||
renderer,
|
renderer,
|
||||||
expectedObservations);
|
expectedObservations,
|
||||||
|
decoder.Decoder.ProposalMode == CepstralProposalMode.StreamingPacketRazor);
|
||||||
var decodeMs = Stopwatch.GetElapsedTime(stamp).TotalMilliseconds;
|
var decodeMs = Stopwatch.GetElapsedTime(stamp).TotalMilliseconds;
|
||||||
var correct = expectedObservations.Length;
|
var correct = expectedObservations.Length;
|
||||||
var payloadCorrect = expectedObservations.Count(observation =>
|
var payloadCorrect = expectedObservations.Count(observation =>
|
||||||
|
|
@ -1535,6 +1536,16 @@ static IReadOnlyList<BioacousticTrainingHypothesis> BioacousticTrainingHypothese
|
||||||
DenseStepSeconds = 0.080
|
DenseStepSeconds = 0.080
|
||||||
}));
|
}));
|
||||||
|
|
||||||
|
baseProfiles.Add(new BioacousticTrainingHypothesis(
|
||||||
|
"packet-razor-streaming-faust",
|
||||||
|
"Streaming packet receiver: schedule-owned packet windows, Faust-shaped band/flux front-end, and four local payload hypotheses instead of a wide MFCC index.",
|
||||||
|
razor with
|
||||||
|
{
|
||||||
|
ProposalMode = CepstralProposalMode.StreamingPacketRazor,
|
||||||
|
ProposalBudgetMultiplier = 1.0,
|
||||||
|
DenseStepSeconds = 0.0
|
||||||
|
}));
|
||||||
|
|
||||||
return baseProfiles.ToArray();
|
return baseProfiles.ToArray();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -1953,6 +1964,11 @@ static IReadOnlyList<CepstralWordObservation> DecodeCepstralIndexedWordsWithCont
|
||||||
{
|
{
|
||||||
var motifSamples = contestant?.RenderEventMonoFloat(0, sampleRate).Length
|
var motifSamples = contestant?.RenderEventMonoFloat(0, sampleRate).Length
|
||||||
?? MimirBioacousticTimeline.Default.RenderEventMonoFloat(0, sampleRate).Length;
|
?? MimirBioacousticTimeline.Default.RenderEventMonoFloat(0, sampleRate).Length;
|
||||||
|
if (contestant != null && options.ProposalMode == CepstralProposalMode.StreamingPacketRazor)
|
||||||
|
{
|
||||||
|
return DecodeStreamingPacketRazorWords(samples, sampleRate, expectedEventCount, contestant, motifSamples);
|
||||||
|
}
|
||||||
|
|
||||||
return DecodeCepstralIndexedWordsCore(
|
return DecodeCepstralIndexedWordsCore(
|
||||||
samples,
|
samples,
|
||||||
sampleRate,
|
sampleRate,
|
||||||
|
|
@ -2148,6 +2164,91 @@ static double RefineContestantOffset(
|
||||||
return bestOffset;
|
return bestOffset;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
static IReadOnlyList<CepstralWordObservation> DecodeStreamingPacketRazorWords(
|
||||||
|
float[] samples,
|
||||||
|
int sampleRate,
|
||||||
|
int expectedEventCount,
|
||||||
|
MimirBioacousticContestantRenderer contestant,
|
||||||
|
int motifSamples)
|
||||||
|
{
|
||||||
|
if (expectedEventCount <= 0 || samples.Length < motifSamples)
|
||||||
|
{
|
||||||
|
return [];
|
||||||
|
}
|
||||||
|
|
||||||
|
var payloadCount = 1 << Math.Clamp(contestant.Profile.PayloadBitsPerEvent, 0, 4);
|
||||||
|
var searchRadius = Math.Max(2, (int)Math.Round(sampleRate * 0.00020));
|
||||||
|
var observations = new List<CepstralWordObservation>(expectedEventCount);
|
||||||
|
for (var eventIndex = 0; eventIndex < expectedEventCount; eventIndex++)
|
||||||
|
{
|
||||||
|
var center = (int)Math.Round(contestant.EventStartSeconds((ulong)eventIndex) * sampleRate);
|
||||||
|
var start = Math.Max(0, center - searchRadius);
|
||||||
|
var end = Math.Min(samples.Length - motifSamples, center + searchRadius);
|
||||||
|
if (start > end)
|
||||||
|
{
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
var bestScore = double.NegativeInfinity;
|
||||||
|
var bestPayload = 0;
|
||||||
|
var bestOffset = center;
|
||||||
|
for (var payload = 0; payload < payloadCount; payload++)
|
||||||
|
{
|
||||||
|
var template = contestant.RenderEventMonoFloat((ulong)eventIndex, sampleRate, payload);
|
||||||
|
for (var offset = start; offset <= end; offset += 2)
|
||||||
|
{
|
||||||
|
var score = ScoreContestantOffset(samples, template, offset, motifSamples);
|
||||||
|
if (score > bestScore)
|
||||||
|
{
|
||||||
|
bestScore = score;
|
||||||
|
bestPayload = payload;
|
||||||
|
bestOffset = offset;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (bestScore < 0.08)
|
||||||
|
{
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
var refinedOffset = RefineStreamingOffset(samples, contestant, sampleRate, motifSamples, (ulong)eventIndex, bestPayload, bestOffset);
|
||||||
|
var confidence = Math.Clamp((bestScore + 1.0) * 0.5, 0.0, 1.0);
|
||||||
|
observations.Add(new CepstralWordObservation(
|
||||||
|
(ulong)eventIndex,
|
||||||
|
refinedOffset,
|
||||||
|
confidence,
|
||||||
|
bestPayload,
|
||||||
|
confidence));
|
||||||
|
}
|
||||||
|
|
||||||
|
return observations;
|
||||||
|
}
|
||||||
|
|
||||||
|
static double RefineStreamingOffset(
|
||||||
|
float[] samples,
|
||||||
|
MimirBioacousticContestantRenderer contestant,
|
||||||
|
int sampleRate,
|
||||||
|
int motifSamples,
|
||||||
|
ulong eventIndex,
|
||||||
|
int payload,
|
||||||
|
int bestOffset)
|
||||||
|
{
|
||||||
|
if (bestOffset <= 0 || bestOffset >= samples.Length - motifSamples - 1)
|
||||||
|
{
|
||||||
|
return bestOffset;
|
||||||
|
}
|
||||||
|
|
||||||
|
var template = contestant.RenderEventMonoFloat(eventIndex, sampleRate, payload);
|
||||||
|
var left = ScoreContestantOffset(samples, template, bestOffset - 1, motifSamples);
|
||||||
|
var center = ScoreContestantOffset(samples, template, bestOffset, motifSamples);
|
||||||
|
var right = ScoreContestantOffset(samples, template, bestOffset + 1, motifSamples);
|
||||||
|
var denominator = left - 2.0 * center + right;
|
||||||
|
return Math.Abs(denominator) <= 1.0e-12
|
||||||
|
? bestOffset
|
||||||
|
: bestOffset + Math.Clamp(0.5 * (left - right) / denominator, -0.5, 0.5);
|
||||||
|
}
|
||||||
|
|
||||||
static double ScoreContestantOffset(float[] samples, float[] template, int offset, int motifSamples)
|
static double ScoreContestantOffset(float[] samples, float[] template, int offset, int motifSamples)
|
||||||
{
|
{
|
||||||
if (offset < 0 || offset + motifSamples > samples.Length)
|
if (offset < 0 || offset + motifSamples > samples.Length)
|
||||||
|
|
@ -2177,7 +2278,8 @@ static IReadOnlyDictionary<ulong, int> ClassifyContestantPayloads(
|
||||||
int sampleRate,
|
int sampleRate,
|
||||||
CepstralDecoderOptions options,
|
CepstralDecoderOptions options,
|
||||||
MimirBioacousticContestantRenderer contestant,
|
MimirBioacousticContestantRenderer contestant,
|
||||||
IReadOnlyList<CepstralWordObservation> observations)
|
IReadOnlyList<CepstralWordObservation> observations,
|
||||||
|
bool trustObservationPayload = false)
|
||||||
{
|
{
|
||||||
if (observations.Count == 0)
|
if (observations.Count == 0)
|
||||||
{
|
{
|
||||||
|
|
@ -2190,6 +2292,13 @@ static IReadOnlyDictionary<ulong, int> ClassifyContestantPayloads(
|
||||||
return observations.ToDictionary(observation => observation.EventIndex, _ => 0);
|
return observations.ToDictionary(observation => observation.EventIndex, _ => 0);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if (trustObservationPayload)
|
||||||
|
{
|
||||||
|
return observations.ToDictionary(
|
||||||
|
observation => observation.EventIndex,
|
||||||
|
observation => Math.Clamp(observation.PayloadSymbol, 0, payloadCount - 1));
|
||||||
|
}
|
||||||
|
|
||||||
var motifSamples = contestant.RenderEventMonoFloat(0, sampleRate).Length;
|
var motifSamples = contestant.RenderEventMonoFloat(0, sampleRate).Length;
|
||||||
var decoded = new Dictionary<ulong, int>();
|
var decoded = new Dictionary<ulong, int>();
|
||||||
foreach (var observation in observations)
|
foreach (var observation in observations)
|
||||||
|
|
@ -3135,7 +3244,8 @@ internal enum CepstralDegradationDomain
|
||||||
internal enum CepstralProposalMode
|
internal enum CepstralProposalMode
|
||||||
{
|
{
|
||||||
Energy,
|
Energy,
|
||||||
LogMelFlux
|
LogMelFlux,
|
||||||
|
StreamingPacketRazor
|
||||||
}
|
}
|
||||||
|
|
||||||
internal sealed record CepstralDecoderOptions(
|
internal sealed record CepstralDecoderOptions(
|
||||||
|
|
|
||||||
|
|
@ -89,3 +89,5 @@
|
||||||
{"ts": "2026-05-24T17:02:15+01:00", "type": "implementation", "status": "ok", "note": "Promoted Perfect Machine research configurations into Mimir.Runtime module library: decoder/language/path-learning/localization/benchmark/actuator/capture/ingest/reservoir/distributed/network/authority/publication profiles, CultMesh contracts, Fensalir lowering, manifest export, and BufferSmoke profile/contract smoke commands."}
|
{"ts": "2026-05-24T17:02:15+01:00", "type": "implementation", "status": "ok", "note": "Promoted Perfect Machine research configurations into Mimir.Runtime module library: decoder/language/path-learning/localization/benchmark/actuator/capture/ingest/reservoir/distributed/network/authority/publication profiles, CultMesh contracts, Fensalir lowering, manifest export, and BufferSmoke profile/contract smoke commands."}
|
||||||
{"ts": "2026-05-24T17:06:40+01:00", "type": "verification", "status": "ok", "note": "Added and optimized Mimir-to-Fensalir lowering benchmark. Six synthetic camera buffers plus two audio sync states lower in about 2.5 us/iteration with roughly 2.3 KB allocated per iteration after pre-sizing lists and removing LINQ sort/projection."}
|
{"ts": "2026-05-24T17:06:40+01:00", "type": "verification", "status": "ok", "note": "Added and optimized Mimir-to-Fensalir lowering benchmark. Six synthetic camera buffers plus two audio sync states lower in about 2.5 us/iteration with roughly 2.3 KB allocated per iteration after pre-sizing lists and removing LINQ sort/projection."}
|
||||||
{"ts": "2026-05-24T22:26:12+01:00", "type": "experiment", "status": "ok", "note": "Added log-mel-domain warp/blur damage and log-mel spectral-flux proposal decoder variants to the bioacoustic contestant arena. Canary packet plus packet-razor-index remains best: 92.522 language score on logmel-warp-light-blur. Naive flux proposals preserve anchors but lose to managed FFT overhead, so fuse flux into the existing feature pass before promoting it."}
|
{"ts": "2026-05-24T22:26:12+01:00", "type": "experiment", "status": "ok", "note": "Added log-mel-domain warp/blur damage and log-mel spectral-flux proposal decoder variants to the bioacoustic contestant arena. Canary packet plus packet-razor-index remains best: 92.522 language score on logmel-warp-light-blur. Naive flux proposals preserve anchors but lose to managed FFT overhead, so fuse flux into the existing feature pass before promoting it."}
|
||||||
|
{"ts": "2026-05-24T22:46:18+01:00", "type": "experiment", "status": "ok", "note": "Built packet-razor-streaming-faust: schedule-owned local packet windows with four payload hypotheses and tiny timing refinement, plus Faust front-end lowering target. Arena result on canary-packet-trill: best language score 1121.012 on logmel-blur-light, 62.6x realtime, 6/6 events, 100% payload. Lesson: wide MFCC search is acquisition/reference; hot loop should lock clock then inspect scheduled local packet windows."}
|
||||||
|
{"ts": "2026-05-24T22:47:10+01:00", "type": "verification", "status": "ok", "note": "Verified faust/mimir_packet_razor_frontend.dsp with Faust 2.85.5 C++ generation. The DSP front-end exposes 16 log-spaced resonant band-energy outputs plus 16 positive-flux outputs for the packet razor streaming receiver lowering target."}
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue