Map sonar DSP recursive refinement path

This commit is contained in:
Meta Vens 2026-05-25 00:54:43 +01:00
commit bd5b3a4fc2
4 changed files with 202 additions and 2 deletions

View file

@ -238,7 +238,11 @@ A naive recursive waveform phase-lock pass was also rejected: it improved one
cardioid receipt but worsened the shotgun and locked a fresh cardioid run onto
a later reflection lobe. Recursive refinement remains a good architecture only
after the path model can distinguish direct arrival, phase/group delay, and room
reflection energy.
reflection energy. The follow-up sonar/DSP research note at
`research/sonar-dsp-recursive-refinement-2026-05-25.md` maps that correction:
use complex matched filters, phase-slope/group-delay fitting, acquisition versus
tracking loop bandwidths, and sparse multipath residuals before attempting
another recursive fitter.
The first actuator proof now exists: `faust/mimir_alignment_actuator.dsp` owns
six channels of bounded fractional delay/gain controls for Faust/native DSP,
and `Mimir.BufferSmoke --bioacoustic-actuator-self-test --sample-rate 48000

View file

@ -0,0 +1,191 @@
# Sonar DSP Notes For Recursive Bioacoustic Refinement
## Question
Can Mimir use iterative refinement in the decoder by transforming a sampled song
contour back toward the emitted chirp, resampling, and repeating until phase is
locked?
Short answer: yes, but not as plain waveform correlation. The failed local
experiment rejected the naive objective function, not recursive refinement.
## Sources Checked
- liquid-dsp: open-source C DSP library for SDR-style filters, transforms,
synchronizers, modulators, demodulators, and frame processing.
<https://github.com/jgaeddert/liquid-dsp>
- liquid-dsp release notes: `qdsync` detects frames and estimates/corrects
carrier frequency, carrier phase, and timing offset before callback delivery.
<https://github.com/jgaeddert/liquid-dsp/releases>
- Underwater acoustic modem paper using liquid-dsp: packet starts are found
with chirp matched filters; later flexframe receivers use PN synchronization,
equalization, PLL, AGC, and acquisition/tracking loop bandwidth changes.
<https://signet.dei.unipd.it/wp-content/uploads/2022/02/Oceans2022_ac_modem-2.pdf>
- pyAPRiL passive radar: explicit reference/surveillance processing,
clutter-cancellation families, frequency-domain/batched cross-correlation,
CFAR, and range-Doppler processing.
<https://github.com/pyapril/pyapril>
- GNU Radio correlate access code / stream tags: synchronization emits tags at
precise stream positions so downstream blocks process the same sample truth.
<https://wiki.gnuradio.org/index.php/Correlate_Access_Code_-_Tag_Stream>
<https://wiki.gnuradio.org/index.php/Stream_Tags>
- OpenTDS / time-delay spectrometry: swept source plus tracking band-pass can
isolate direct sound from lower-frequency delayed reflections, then derive
magnitude and phase response.
<https://syarusinsky.github.io/OpenTDS/>
- Signalsmith DSP: high-quality fractional delay/resampling primitives include
Lagrange, polyphase, and Kaiser-sinc interpolation with measured delay error.
<https://signalsmith-audio.co.uk/code/dsp/>
- Open Echo: open-source sonar stack captures raw echo data and emphasizes
transducer/receiver filtering as part of the measurement system.
<https://github.com/Neumi/open_echo>
- ahoi acoustic modem docs: receiver front-end has preamp plus band-pass stage
adapted to the used frequency band before DSP sees samples.
<https://www3.tuhh.de/acps/projects/ahoi/hardware_overview/>
## What The Failed Mimir Pass Got Wrong
The reverted recursive pass treated the best waveform correlation after
fractional resampling as the truth. In a reflective room, that is not a timing
estimator. It is a reflection selector.
Observed failure:
- stored cardioid improved from 7.576 us to 5.141 us;
- stored shotgun worsened from 6.578 us to 10.236 us;
- fresh cardioid locked to a later reflection lobe at 65.666 us instead of the
prior 18.930 us.
This means the correct recursion variable is not "delay that maximizes raw
correlation." It is a channel hypothesis:
```text
direct-path delay
+ sample-rate offset / clock drift
+ frequency-dependent phase/group delay
+ complex path gain per contour/formant band
+ sparse reflection taps with later arrival and lower authority
```
The direct path owns sync. Reflections are channel evidence, not alternate
timelines.
## Better Shape
```text
rolling ASIO window
-> analytic / complex bandbank or short chirp matched-filter bank
-> per-anchor complex response: magnitude, phase, local slope, confidence
-> ambiguity candidates over delay and small frequency/clock offsets
-> direct-path hypothesis tracker
-> sparse multipath residual model
-> recursive refinement:
1. predict direct-path anchor phases from current hypothesis
2. dechirp / derotate each contour into baseband
3. score phase slope and group-delay residuals, not only magnitude
4. update delay/SRO/group-delay with bounded loop bandwidth
5. explain later energy as reflection taps
6. repeat until the direct-path innovation is small
-> output timing anchor, response model, confidence, and residual reflection map
```
This is closer to sonar/acoustic modem practice:
- matched filters acquire the preamble or contour;
- PLL/timing loops refine phase and sample timing;
- equalizers/channel estimators explain multipath;
- CFAR/thresholding prevents noise-floor peaks from becoming detections;
- stream tags carry exact sample-position decisions forward.
## Concrete Mimir Implementation Cut
### 1. Replace Raw Waveform Scoring With Complex Matched Filters
For each shaped contour anchor, build a complex reference template:
```text
template[n] = envelope[n] * exp(j * phase[n])
```
Maintain streaming I/Q accumulators per anchor family:
```text
z = sum window[n] * conj(template[n])
energy = sum abs(window[n])^2
score = abs(z)^2 / energy
phase = atan2(imag(z), real(z))
```
The magnitude says "this contour exists." The phase and phase slope say "the
direct-path hypothesis is early/late."
### 2. Estimate Delay From Phase Slope
Across frequency-separated anchors:
```text
phase(f) ~= phase0 - 2*pi*f*delay + group_delay_error(f)
```
Fit delay from the wrapped phase slope after unwrapping over the usable
anchor set. Weight by learned path response. This is the missing bridge between
"subsample interpolation" and physically meaningful phase lock.
### 3. Track, Do Not Reacquire Every Window
Use two loop bandwidths:
- acquisition: broader ambiguity search, more tolerance, accepts less precise
anchors;
- tracking: narrow predicted window around the existing path, rejects peaks
that require impossible acceleration, clock drift, or propagation jumps.
This is how recursion becomes stable: each pass refines a live channel state
instead of discovering a new room every frame.
### 4. Model Multipath Explicitly
After subtracting the predicted direct-path complex template, keep a small
sparse list of later residual taps:
```text
tap = delay_after_direct, complex_gain_by_band, decay/confidence
```
Those taps are useful for room mapping and frequency-response normalization, but
they cannot vote the direct-path delay backward or forward unless the direct
path is absent.
### 5. Use Real Fractional Delay Tools
Linear interpolation is not good enough for the last mile. The actuator and
decoder should use at least Lagrange or polyphase/Kaiser-sinc fractional delay
when comparing/resampling high-rate anchors. Signalsmith's public DSP library is
a useful reference for delay-line quality gates.
## What To Build Next
1. A `MimirComplexContourMatchedFilter` over the canary packet anchors.
2. A per-mic `MimirAcousticPathTracker` with:
- direct delay;
- SRO ppm;
- phase intercept;
- per-band complex gain;
- sparse reflection taps.
3. A calibration replay command that runs both stored ASIO receipts and reports:
- direct delay MAE;
- phase-slope residual;
- reflection-tap energy;
- usable anchor bands;
- convergence time;
- realtime factor.
4. Only then reintroduce recursive refinement as a bounded tracker update.
## Current Belief
Iterative refinement is absolutely worth pursuing. The correct version is a
complex matched-filter / phase-slope / channel-tracker machine, not a raw
waveform-correlation machine. "Nanosecond" remains a dangerous word in air with
consumer-ish acoustic paths, but the route toward lower microseconds is real:
make phase/group delay first-class and stop letting reflections masquerade as
time.

View file

@ -99,3 +99,4 @@
{"ts": "2026-05-25T00:08:44+01:00", "type": "experiment", "status": "partial", "note": "Built the first anchor-rich canary packet: timing chips, formant pivots, harmonic-envelope notches, payload ornaments, renderer-level template caching, and per-event intra-call anchor measurements in the physical calibration receipt. Fresh 192 kHz ASIO receipt calibration/bioacoustic/scarlett-canary-packet-anchor-rich-192k.json clears 10x realtime and records hundreds of anchors per channel. Loopback improves to about 2.23 us MAE with anchor residuals under 0.3 samples. Physical mics did not improve yet: shotgun 36/37 payload=0.973 at 56.365 us MAE, cardioid 27/37 payload=0.730 at 101.522 us MAE. The new anchors expose path bias; the next cut should shape/weight anchors for acoustic survivability instead of trusting every formant feature equally."}
{"ts": "2026-05-25T00:42:12+01:00", "type": "experiment", "status": "partial", "note": "Added path-level loopback truth to the anchor-rich calibration: matched mic anchors are compared directly against loopback anchors, then event-local waveform correlation against the captured loopback packet chooses the path delay. Release calibration clears the realtime budget at 21.1x on the stored anchor-rich capture. Best stored-result path precision improved to cardioid 7.576 us and shotgun 6.578 us. A fresh room capture at calibration/bioacoustic/scarlett-canary-packet-anchor-rich-latest-192k.json keeps shotgun near 5.916 us but worsens cardioid to 18.930 us. The razor timing-chip emitter mutation was rejected because it reduced physical reliability. Current floor is single-digit microseconds for the shotgun, not yet the one-microsecond target; next required cut is phase/group-delay correction or a direct-path anchor family that survives both mics."}
{"ts": "2026-05-25T01:05:00+01:00", "type": "experiment", "status": "rejected", "note": "Tested a recursive event-local waveform delay fitter: start from the current path hypothesis, resample candidate song windows against loopback, shrink the residual search radius, and weight events by phase-lock consistency. The naive time-domain version improved one stored cardioid path from 7.576 us to 5.141 us, but it also worsened the stored shotgun from 6.578 us to 10.236 us and locked the fresh cardioid onto a later reflection lobe at 65.666 us instead of the prior 18.930 us. Reverted the code and receipts. Lesson: recursive refinement is a valid shape only when constrained by a real phase/group-delay/path model; plain waveform resampling is not sufficient and nanosecond claims are physically dishonest in the current acoustic path."}
{"ts": "2026-05-25T01:25:00+01:00", "type": "research", "status": "ok", "note": "Looked at open realtime DSP/sonar-adjacent systems after the recursive-fit rejection. liquid-dsp, an underwater acoustic modem using liquid-dsp, pyAPRiL passive radar, GNU Radio stream/tag synchronization, OpenTDS, Signalsmith DSP, Open Echo, and ahoi all point the same way: acquisition should use matched filters/ambiguity search; refinement should track complex phase, timing, SRO, and channel response; multipath should be modeled as later residual taps; and exact stream positions should be propagated as tags/state. Added research/sonar-dsp-recursive-refinement-2026-05-25.md. The next implementation cut should be a complex contour matched-filter bank plus direct-path acoustic tracker, not another raw waveform recursion pass."}

View file

@ -212,7 +212,11 @@ accepted_design:
improve one cardioid receipt while worsening the shotgun and locking a fresh
cardioid run onto a later reflection lobe. Recursion is only allowed back as
a constrained direct-path/group-delay model, not as unconstrained waveform
resampling with prettier decimals.
resampling with prettier decimals. The sonar/DSP research note
`research/sonar-dsp-recursive-refinement-2026-05-25.md` points the next
decoder cut at complex contour matched filters, phase-slope/group-delay
fitting, acquisition/tracking loop bandwidths, and sparse multipath
residuals.
Acoustic robustness remains open.
The old arbitrary-chirplet matcher is a diagnostic/reference artifact only,
not the runtime hot path.