web: band-split dB-flux onset detector, synthetic drum bench
Replaces the log1p-compressed whole-spectrum flux with four-band flux in dB, a two-frame max-filtered reference, a per-band median+8 MAD threshold with a per-band minimum rise, and one refractory period. Level-independent. tools/onset-bench.mjs synthesises kick/snare/hat patterns with known onset times (3 patterns x 3 tempos x 3 levels x 3 beds = 81 signals), scores hit rate, false positives and median timing at 30 fps and 512-sample hops. tools/onset-legacy.mjs freezes the old detector for the before numbers. Synthetic only. Nothing was listened to; no real music, no real GPU. Corpus adjustments made after the first baseline run: coincident kick+snare count as one onset; voices end in a 60 ms fade because truncation clicks read as onsets in the near-silent gaps; hats raised relative to the snare tail. Baseline was rerun after each change. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Aqua
parent
f53a0d1c00
commit
908e8dcb3b
7 changed files with 346 additions and 49 deletions
|
|
@ -16,7 +16,7 @@ Any static server works (`python3 -m http.server -d web`). `localhost` counts as
|
|||
|
||||
| Piece | File | What it does |
|
||||
|---|---|---|
|
||||
| Onset detector | `web/src/dsp.js` `OnsetDetector` | Log-compressed spectral flux against a max-filtered previous frame, moving-median adaptive threshold, local-peak pick, refractory period. Causal: confirms an onset one frame late. |
|
||||
| Onset detector | `web/src/dsp.js` `OnsetDetector` | Four-band spectral flux in dB (40-150, 150-600, 600-2500, 2.5-12 kHz) against a per-bin max of the two previous frames (max-filtered across +-1 bin), per-band median + 8 MAD threshold with a per-band minimum rise, local-peak pick, one refractory period over all bands. Level-independent. Causal: confirms an onset one frame late. Synthetic-drum scores: `tools/onset-bench.mjs`. |
|
||||
| Key estimate | `web/src/dsp.js` `KeyEstimator` | Peak-based chroma (parabolic-interpolated spectral peaks folded to 12 pitch classes, 4 s half-life) correlated against the 24 Krumhansl-Kessler key profiles. The key picks the scene tint: tonic walks the circle of fifths for hue, mode sets warmth. |
|
||||
| Analysis state | `web/src/analysis.js` | One object all scenes read: 128 log bands, bass/mid/treble, onset counter, chroma, key, tint. Also `DemoFeed`. |
|
||||
| Perlin noise | `web/src/noise.js` | Classic 3D Perlin and a 2D curl field (divergence-free flow) for the CPU-side paths. |
|
||||
|
|
@ -32,10 +32,13 @@ Any static server works (`python3 -m http.server -d web`). `localhost` counts as
|
|||
## Tests and measurements
|
||||
|
||||
```sh
|
||||
cd web && npm test # node --test test/dsp.test.mjs (node >= 20, nothing to install)
|
||||
cd web && npm test # node --test test/dsp.test.mjs test/onset.test.mjs (node >= 20, nothing to install)
|
||||
cd web && npm run bench:onsets # full synthetic-drum corpus, before/after table, about 30 s
|
||||
```
|
||||
|
||||
Covers: ten synthetic clicks found at two hop sizes (11.6 ms and 16.7 ms) with no false positives; silence, steady noise and a sustained tone give no onsets; a 20 ms doublet merges under the refractory period; the key estimate reads C major, A minor and (transposed) B minor from synthesized chord progressions. Fixtures are deliberately plain: the estimate is known to confuse relative major/minor when a clip leans on the dominant, and one test fixture was rewritten to put its tonic in more chords (the estimator was not tuned to the fixture).
|
||||
Covers (dsp.test): ten synthetic clicks found at two hop sizes (11.6 ms and 16.7 ms) with no false positives; silence, steady noise and a sustained tone give no onsets; a 20 ms doublet merges under the refractory period; the key estimate reads C major, A minor and (transposed) B minor from synthesized chord progressions. Fixtures are deliberately plain: the estimate is known to confuse relative major/minor when a clip leans on the dominant, and one test fixture was rewritten to put its tonic in more chords (the estimator was not tuned to the fixture).
|
||||
|
||||
Covers (onset.test, synthetic drum corpus from `tools/onset-bench.mjs`: kick/snare/hat patterns at 90/120/150 bpm, three levels, bass+noise beds, 30 fps and 512-sample hops): hit rate, false positives and timing floors for the shipped detector, a margin over the frozen legacy detector, level independence, determinism. Synthetic only; the corpus was adjusted twice while building it (coincident kick+snare counted as one onset; voice tails faded so truncation clicks are not onsets), recorded in the commit message.
|
||||
|
||||
Frame time: the HUD shows rolling p50/p95 of requestAnimationFrame intervals against the 16.7 ms budget and CPU submit time. `?bench` runs the three scenes on the synthetic feed and prints JSON (`?w=1920&h=1080` fixes the drawing-buffer size, `&sync` reads a pixel back after each frame so the measured frame time is real cost, not capped by vsync, `&ms=` sets seconds per scene). `web/tools/headless.mjs` drives headless Chrome over CDP to collect it. Results of the last run are in `web/BENCH.md`.
|
||||
|
||||
|
|
|
|||
|
|
@ -6,7 +6,8 @@
|
|||
"license": "MIT",
|
||||
"private": true,
|
||||
"scripts": {
|
||||
"test": "node --test test/dsp.test.mjs",
|
||||
"test": "node --test test/dsp.test.mjs test/onset.test.mjs",
|
||||
"bench:onsets": "node tools/onset-bench.mjs",
|
||||
"serve": "node tools/serve.mjs"
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -14,7 +14,7 @@ export class Analysis {
|
|||
constructor(sampleRate, binCount) {
|
||||
this.sampleRate = sampleRate; this.binCount = binCount;
|
||||
this.lin = new Float32Array(binCount);
|
||||
this.onsets = new OnsetDetector({ maxBin: Math.min(binCount, Math.round(8000 / (sampleRate / 2 / binCount))) });
|
||||
this.onsets = new OnsetDetector({ binHz: sampleRate / 2 / binCount });
|
||||
this.keyEst = new KeyEstimator(sampleRate, binCount);
|
||||
this.bands = new Float32Array(NB);
|
||||
this.row = new Uint8Array(NB); // newest row, as texture bytes
|
||||
|
|
|
|||
|
|
@ -50,71 +50,81 @@ export function dbToLinear(db, out) {
|
|||
}
|
||||
|
||||
/**
|
||||
* Onset detector: log-compressed spectral flux against a max-filtered previous frame
|
||||
* (the max filter ignores vibrato wobble), a moving-median adaptive threshold, local-peak picking
|
||||
* and a refractory period. Causal: an onset on frame n-1 is confirmed when frame n arrives.
|
||||
* Onset detector: band-split spectral flux in decibels. Each band has its own flux (mean positive dB rise
|
||||
* against a max-filtered previous frame, which ignores vibrato wobble) and its own adaptive threshold
|
||||
* (median + k * MAD over the last second), so a hat is judged against the hats and a kick against the
|
||||
* kick, and the level of the track does not matter. Causal: a flux peak on frame n-1 is confirmed when
|
||||
* frame n arrives. Bands are merged behind one refractory period.
|
||||
*
|
||||
* push(mags, t) -> null | { time, strength } (time = the onset frame's own timestamp, seconds)
|
||||
* push(mags, t) -> null | { time, strength, band } (time = the onset frame's own timestamp, seconds)
|
||||
*/
|
||||
export class OnsetDetector {
|
||||
constructor(opts = {}) {
|
||||
this.compress = opts.compress ?? 200; // log(1 + compress * mag)
|
||||
this.window = opts.window ?? 43; // median history, frames
|
||||
this.delta = opts.delta ?? 0.08; // absolute floor above the median, in flux units
|
||||
this.ratio = opts.ratio ?? 1.6; // and the flux must exceed ratio * median
|
||||
this.refractory = opts.refractory ?? 0.06; // seconds
|
||||
this.minBin = opts.minBin ?? 2;
|
||||
this.binHz = opts.binHz ?? 48000 / 2048;
|
||||
this.edgesHz = opts.edgesHz ?? [40, 150, 600, 2500, 12000];
|
||||
this.maxBin = opts.maxBin ?? 0; // 0 = all bins
|
||||
this.bands = opts.bands ?? 6; // log-spaced flux bands between minBin and maxBin
|
||||
this.floorDb = opts.floorDb ?? -110; // magnitudes below this are silence
|
||||
this.history = opts.history ?? 1.0; // seconds of flux per band for the threshold
|
||||
this.k = opts.k ?? 8; // threshold = median + k * MAD (MAD scaled to sigma)
|
||||
this.minRise = opts.minRise ?? [6, 3, 2, 2]; // and at least this many dB mean rise in the band (per band: the few bass bins are noisy)
|
||||
this.refractory = opts.refractory ?? 0.05; // seconds, all bands
|
||||
this.edges = null;
|
||||
this.prev = null;
|
||||
this.hist = []; // recent flux values
|
||||
this.f1 = 0; this.f2 = 0; this.t1 = 0; // flux[n-1], flux[n-2], time[n-1]
|
||||
this.lag = opts.lag ?? 2; // reference = per-bin max over this many previous frames (1 or 2)
|
||||
this.prev = null; this.prev2 = null;
|
||||
this.hist = null; // per band: { t[], f[] }
|
||||
this.f1 = null; this.f2 = null; this.t1 = 0; // per band flux[n-1], flux[n-2], time[n-1]
|
||||
this.last = -1e9;
|
||||
this.flux = 0; // latest raw flux, for display
|
||||
this.flux = 0; // largest band flux this frame, for display
|
||||
}
|
||||
|
||||
push(mags, t) {
|
||||
const n = this.maxBin || mags.length;
|
||||
const n = Math.min(this.maxBin || mags.length, mags.length);
|
||||
if (!this.edges) {
|
||||
this.edges = Array.from({ length: this.bands + 1 }, (_, i) => Math.round(this.minBin * Math.pow(n / this.minBin, i / this.bands)));
|
||||
for (let i = 1; i < this.edges.length; i++) this.edges[i] = Math.max(this.edges[i], this.edges[i - 1] + 1);
|
||||
this.edges = this.edgesHz.map(h => Math.min(n, Math.max(1, Math.round(h / this.binHz))));
|
||||
for (let i = 1; i < this.edges.length; i++) this.edges[i] = Math.min(n, Math.max(this.edges[i], this.edges[i - 1] + 1));
|
||||
const nb = this.edges.length - 1;
|
||||
this.hist = Array.from({ length: nb }, () => ({ t: [], f: [] }));
|
||||
this.f1 = new Float64Array(nb); this.f2 = new Float64Array(nb);
|
||||
}
|
||||
const nb = this.edges.length - 1, lo = Math.pow(10, this.floorDb / 20);
|
||||
const cur = new Float32Array(n);
|
||||
for (let k = 0; k < n; k++) cur[k] = Math.log1p(this.compress * mags[k]);
|
||||
let flux = 0;
|
||||
for (let k = 0; k < n; k++) cur[k] = 20 * Math.log10(Math.max(mags[k], lo));
|
||||
const flux = new Float64Array(nb);
|
||||
if (this.prev) {
|
||||
const p = this.prev;
|
||||
// flux per log-spaced band, each divided by sqrt(its width): a kick (a few bass bins) then counts
|
||||
// as much as a hat (hundreds of treble bins), which one whole-spectrum sum would bury
|
||||
for (let b = 0; b < this.edges.length - 1; b++) {
|
||||
let bf = 0;
|
||||
for (let k = this.edges[b]; k < this.edges[b + 1]; k++) {
|
||||
const ref = Math.max(p[k - 1] ?? 0, p[k], p[k + 1] ?? 0);
|
||||
const p = this.prev, q = this.lag > 1 ? this.prev2 : null;
|
||||
for (let b = 0; b < nb; b++) {
|
||||
const a = this.edges[b], z = this.edges[b + 1];
|
||||
let sum = 0;
|
||||
for (let k = a; k < z; k++) {
|
||||
let ref = Math.max(p[k - 1] ?? this.floorDb, p[k], p[k + 1] ?? this.floorDb);
|
||||
if (q) ref = Math.max(ref, q[k - 1] ?? this.floorDb, q[k], q[k + 1] ?? this.floorDb);
|
||||
const d = cur[k] - ref;
|
||||
if (d > 0) bf += d;
|
||||
if (d > 0) sum += d;
|
||||
}
|
||||
flux += bf / Math.sqrt(this.edges[b + 1] - this.edges[b]);
|
||||
flux[b] = sum / (z - a);
|
||||
}
|
||||
flux /= this.edges.length - 1;
|
||||
}
|
||||
this.prev = cur;
|
||||
this.flux = flux;
|
||||
this.prev2 = this.prev; this.prev = cur;
|
||||
this.flux = Math.max(...flux);
|
||||
|
||||
// candidate is the previous frame: f1 must be a local maximum over f2 and the new flux
|
||||
let out = null;
|
||||
const cand = this.f1;
|
||||
if (this.hist.length >= 8 && cand > this.f2 && cand >= flux) {
|
||||
const sorted = this.hist.slice().sort((a, b) => a - b);
|
||||
const med = sorted[sorted.length >> 1];
|
||||
if (cand > med * this.ratio + this.delta && this.t1 - this.last >= this.refractory) {
|
||||
out = { time: this.t1, strength: Math.min(1, (cand - med) / (cand + 1e-6) * 1.5) };
|
||||
this.last = this.t1;
|
||||
// per band: the previous frame is a candidate when it is a local flux maximum over the threshold
|
||||
let best = null;
|
||||
for (let b = 0; b < nb; b++) {
|
||||
const h = this.hist[b], cand = this.f1[b];
|
||||
if (h.f.length >= 8 && cand > this.f2[b] && cand >= flux[b] && cand >= this.minRise[b]) {
|
||||
const sorted = h.f.slice().sort((x, y) => x - y), med = sorted[sorted.length >> 1];
|
||||
const dev = h.f.map(v => Math.abs(v - med)).sort((x, y) => x - y), mad = dev[dev.length >> 1] * 1.4826;
|
||||
if (cand > med + this.k * mad && (!best || cand > best.flux)) best = { flux: cand, band: b };
|
||||
}
|
||||
h.t.push(t); h.f.push(flux[b]);
|
||||
while (h.t.length > 2 && t - h.t[0] > this.history) { h.t.shift(); h.f.shift(); }
|
||||
}
|
||||
this.hist.push(flux);
|
||||
if (this.hist.length > this.window) this.hist.shift();
|
||||
this.f2 = this.f1; this.f1 = flux; this.t1 = t;
|
||||
let out = null;
|
||||
if (best && this.t1 - this.last >= this.refractory) {
|
||||
out = { time: this.t1, strength: Math.min(1, best.flux / 15), band: best.band };
|
||||
this.last = this.t1;
|
||||
}
|
||||
this.f2.set(this.f1); this.f1.set(flux); this.t1 = t;
|
||||
return out;
|
||||
}
|
||||
}
|
||||
|
|
|
|||
49
web/test/onset.test.mjs
Normal file
49
web/test/onset.test.mjs
Normal file
|
|
@ -0,0 +1,49 @@
|
|||
// node --test test/onset.test.mjs Synthetic drum corpus (tools/onset-bench.mjs): floors for the shipped detector.
|
||||
// Synthetic only: these numbers say nothing about real music and nothing was listened to.
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { run, corpus, synthesize, spectra, score, detect, RATES } from '../tools/onset-bench.mjs';
|
||||
import { OnsetDetector } from '../src/dsp.js';
|
||||
|
||||
const subset = corpus().filter((_, i) => i % 3 === 0); // 27 of 81 signals, every pattern/tempo/level/bed combination class
|
||||
|
||||
test('drum corpus: hit rate, false positives and timing at 30 fps and full rate', () => {
|
||||
const res = run(['legacy', 'current'], subset);
|
||||
for (const v of ['legacy', 'current']) for (const [r, a] of Object.entries(res[v]))
|
||||
console.log(` ${v.padEnd(8)} ${r.padEnd(5)} hit ${(100 * a.hitRate).toFixed(1)}% fp ${a.fp} (${a.fpPerMin.toFixed(1)}/min) median err ${a.medianMs.toFixed(1)} ms`);
|
||||
for (const [r, a] of Object.entries(res.current)) {
|
||||
assert.ok(a.hitRate >= 0.85, `${r} hit rate ${a.hitRate}`);
|
||||
assert.ok(a.fpPerMin <= 2, `${r} false positives/min ${a.fpPerMin}`);
|
||||
assert.ok(a.medianMs <= 40, `${r} median timing error ${a.medianMs} ms`);
|
||||
assert.ok(a.hitRate > res.legacy[r].hitRate + 0.2, `${r} must beat the legacy detector by 20 points`);
|
||||
}
|
||||
});
|
||||
|
||||
test('hit rate does not depend on the level of the track', () => {
|
||||
const res = run(['current'], subset);
|
||||
for (const [r, a] of Object.entries(res.current)) {
|
||||
const rates = Object.values(a.lv).map(p => p.hit / p.n);
|
||||
assert.ok(Math.max(...rates) - Math.min(...rates) < 0.05, `${r} spread across 0/-20/-40 dB: ${rates}`);
|
||||
}
|
||||
});
|
||||
|
||||
test('corpus is deterministic', () => {
|
||||
const c = corpus()[5], a = synthesize(c), b = synthesize(c);
|
||||
assert.deepEqual(a.truth, b.truth);
|
||||
assert.ok(a.x.every((v, i) => v === b.x[i]));
|
||||
});
|
||||
|
||||
test('steady bass drone with a noise bed and no drums gives no onsets', () => {
|
||||
const { x } = synthesize({ pattern: 'rock', bpm: 120, level: 1, bed: 'loud', seed: 9 });
|
||||
const quiet = x.slice(0, 48000 * 1); // the lead-in before the first drum hit
|
||||
for (const hop of Object.values(RATES)) {
|
||||
const found = detect(() => new OnsetDetector({ binHz: 48000 / 2048 }), spectra(quiet, hop)).filter(t => t > 0.6);
|
||||
assert.equal(found.length, 0, `hop ${hop}: ${found}`);
|
||||
}
|
||||
});
|
||||
|
||||
test('a single hit is reported once, within the match window', () => {
|
||||
const { x, truth } = synthesize({ pattern: 'four', bpm: 60, level: 0.1, bed: 'none', seed: 4 });
|
||||
const s = score(truth, detect(() => new OnsetDetector({ binHz: 48000 / 2048 }), spectra(x, 512)));
|
||||
assert.ok(s.fp <= 1, `fp ${s.fp}`);
|
||||
});
|
||||
169
web/tools/onset-bench.mjs
Normal file
169
web/tools/onset-bench.mjs
Normal file
|
|
@ -0,0 +1,169 @@
|
|||
// Onset detector bench: synthesises drum patterns with known onset times, runs a detector over them at the
|
||||
// page's 30 fps feature rate and at full rate (512-sample hop), and scores hit rate, false positives and timing.
|
||||
// Deterministic (fixed LCG seeds), no network, no dependencies. node web/tools/onset-bench.mjs [variant ...]
|
||||
// SYNTHETIC ONLY: nothing here has been listened to, and a score here says nothing about real music.
|
||||
import { magnitudeSpectrum, OnsetDetector } from '../src/dsp.js';
|
||||
import { LegacyOnsetDetector } from './onset-legacy.mjs';
|
||||
|
||||
export const SR = 48000, FFT = 2048;
|
||||
export const RATES = { fps30: Math.round(SR / 30), full: 512 }; // hop in samples
|
||||
const LEAD = 1.0, DURATION = 14, SCORE_FROM = 0.6; // seconds
|
||||
const MATCH_EARLY = 0.02, MATCH_LATE = 0.09; // a frame at time T (window end) matches an onset at t if t-0.02 <= T <= t+0.09
|
||||
|
||||
function rng(seed) { let s = seed >>> 0; return () => ((s = (s * 1664525 + 1013904223) >>> 0) / 4294967296) * 2 - 1; }
|
||||
|
||||
// 16 steps per bar. kick/snare/hat step lists.
|
||||
export const PATTERNS = {
|
||||
rock: { kick: [0, 8, 10], snare: [4, 12], hat: [0, 2, 4, 6, 8, 10, 12, 14] },
|
||||
four: { kick: [0, 4, 8, 12], snare: [4, 12], hat: [2, 6, 10, 14] },
|
||||
busy: { kick: [0, 3, 6, 10, 13], snare: [4, 12], hat: Array.from({ length: 16 }, (_, i) => i) },
|
||||
};
|
||||
export const BEDS = { none: { bass: 0, noise: 0 }, bed: { bass: 0.2, noise: 0.02 }, loud: { bass: 0.4, noise: 0.08 } };
|
||||
export const LEVELS = { '0dB': 1, '-20dB': 0.1, '-40dB': 0.01 };
|
||||
export const TEMPOS = [90, 120, 150];
|
||||
|
||||
// every voice ends with a 60 ms linear fade so a truncated tail cannot click and read as an onset
|
||||
const taper = (i, n) => Math.min(1, (n - i) / (0.06 * SR));
|
||||
function addKick(x, s, r) {
|
||||
let ph = 0; const n = Math.round(0.5 * SR);
|
||||
for (let i = 0; i < n && s + i < x.length; i++) {
|
||||
const t = i / SR; ph += 2 * Math.PI * (45 + 90 * Math.exp(-t / 0.03)) / SR;
|
||||
x[s + i] += taper(i, n) * 0.9 * Math.exp(-t / 0.12) * Math.sin(ph);
|
||||
}
|
||||
}
|
||||
function addSnare(x, s, r) {
|
||||
const n = Math.round(0.4 * SR); let p = 0;
|
||||
for (let i = 0; i < n && s + i < x.length; i++) {
|
||||
const t = i / SR, w = r(), hp = w - p; p = w;
|
||||
x[s + i] += taper(i, n) * (0.3 * Math.exp(-t / 0.06) * Math.sin(2 * Math.PI * 185 * t) + 0.5 * Math.exp(-t / 0.09) * hp * 0.7);
|
||||
}
|
||||
}
|
||||
function addHat(x, s, r, amp) {
|
||||
const n = Math.round(0.15 * SR); let p1 = 0, p2 = 0;
|
||||
for (let i = 0; i < n && s + i < x.length; i++) {
|
||||
const t = i / SR, w = r(), a = w - p1, b = a - p2; p1 = w; p2 = a;
|
||||
x[s + i] += taper(i, n) * amp * Math.exp(-t / 0.025) * b * 0.5;
|
||||
}
|
||||
}
|
||||
|
||||
/** -> { x: Float32Array, truth: [{ time, inst }] } */
|
||||
export function synthesize({ pattern, bpm, level, bed, seed = 1 }) {
|
||||
const x = new Float32Array(DURATION * SR), r = rng(seed), P = PATTERNS[pattern], B = BEDS[bed];
|
||||
const step = 60 / bpm / 4, truth = [];
|
||||
const place = (inst, k, fn, ...a) => { const t = LEAD + k * step; truth.push({ time: t, inst }); fn(x, Math.round(t * SR), r, ...a); };
|
||||
for (let bar = 0, k0 = 0; LEAD + k0 * step < DURATION - 0.4; bar++, k0 += 16) {
|
||||
for (const s of P.kick) if (LEAD + (k0 + s) * step < DURATION - 0.4) place('kick', k0 + s, addKick);
|
||||
for (const s of P.snare) if (LEAD + (k0 + s) * step < DURATION - 0.4) place('snare', k0 + s, addSnare);
|
||||
for (const s of P.hat) if (LEAD + (k0 + s) * step < DURATION - 0.4) place('hat', k0 + s, addHat, s % 4 === 0 ? 0.7 : 0.4);
|
||||
}
|
||||
// beds: a bass drone that glides between notes (no onsets) and pink-ish noise, faded in over 0.3 s
|
||||
const notes = [55, 55, 65.4, 49], rb = rng(seed + 100);
|
||||
let ph = 0, f = 55, pink = 0;
|
||||
for (let i = 0; i < x.length; i++) {
|
||||
const t = i / SR, target = notes[Math.floor(t / 2) % notes.length];
|
||||
f += (target - f) * (1 - Math.exp(-1 / (0.3 * SR)));
|
||||
ph += 2 * Math.PI * f / SR;
|
||||
const fade = Math.min(1, t / 0.3), trem = 0.85 + 0.15 * Math.sin(2 * Math.PI * 0.25 * t);
|
||||
pink = 0.98 * pink + 0.1 * rb();
|
||||
x[i] += fade * (B.bass * trem * (Math.sin(ph) + 0.4 * Math.sin(2 * ph)) + B.noise * pink * 3);
|
||||
}
|
||||
const rf = rng(seed + 200);
|
||||
for (let i = 0; i < x.length; i++) x[i] = x[i] * level + rf() * 1e-5; // dither floor, ~ -100 dBFS
|
||||
return { x, truth };
|
||||
}
|
||||
|
||||
/** Magnitude frames the way AnalyserNode hands them over: window ends at 'time', values round-trip through dB clamped at -120. */
|
||||
export function spectra(x, hop) {
|
||||
const frames = [], mags = new Float32Array(FFT >> 1);
|
||||
for (let off = 0; off + FFT <= x.length; off += hop) {
|
||||
magnitudeSpectrum(x, off, FFT, mags);
|
||||
const m = new Float32Array(mags.length);
|
||||
for (let k = 0; k < m.length; k++) m[k] = Math.pow(10, Math.max(20 * Math.log10(mags[k] + 1e-12), -120) / 20);
|
||||
frames.push({ time: (off + FFT) / SR, mags: m });
|
||||
}
|
||||
return frames;
|
||||
}
|
||||
|
||||
/** Coincident hits (a kick and a snare on one step) are one onset: a detector emits one event per moment.
|
||||
* Greedy one-to-one match in time order; each onset takes the earliest unused detection in its window.
|
||||
* Per-instrument rows count an instrument as hit when the event it sounds in was hit. */
|
||||
export function score(truth, found) {
|
||||
const events = [];
|
||||
for (const t of truth.slice().sort((p, q) => p.time - q.time)) {
|
||||
const e = events[events.length - 1];
|
||||
if (e && t.time - e.time < 0.005) e.insts.push(t.inst); else events.push({ time: t.time, insts: [t.inst] });
|
||||
}
|
||||
const used = new Set(), errs = [], per = {};
|
||||
let n = 0;
|
||||
for (const e of events) {
|
||||
if (e.time < SCORE_FROM) continue;
|
||||
n++;
|
||||
const i = found.findIndex((f, j) => !used.has(j) && f >= e.time - MATCH_EARLY && f <= e.time + MATCH_LATE);
|
||||
if (i >= 0) { used.add(i); errs.push(found[i] - e.time); }
|
||||
for (const inst of e.insts) { const p = per[inst] ??= { n: 0, hit: 0 }; p.n++; if (i >= 0) p.hit++; }
|
||||
}
|
||||
return { n, hit: errs.length, fp: found.filter((f, j) => !used.has(j) && f >= SCORE_FROM).length, errs, per };
|
||||
}
|
||||
|
||||
export function detect(make, frames) {
|
||||
const det = make(), out = [];
|
||||
for (const fr of frames) { const o = det.push(fr.mags, fr.time); if (o) out.push(o.time); }
|
||||
return out;
|
||||
}
|
||||
|
||||
export function corpus() {
|
||||
const cases = [];
|
||||
for (const pattern of Object.keys(PATTERNS)) for (const bpm of TEMPOS) for (const [lv, level] of Object.entries(LEVELS)) for (const bed of Object.keys(BEDS))
|
||||
cases.push({ pattern, bpm, lv, level, bed, seed: cases.length + 1 });
|
||||
return cases;
|
||||
}
|
||||
|
||||
// detector as Analysis builds it: bins up to 8 kHz
|
||||
const maxBin = Math.round(8000 / (SR / 2 / (FFT / 2))); // what the legacy Analysis passed
|
||||
export const VARIANTS = {
|
||||
legacy: () => new LegacyOnsetDetector({ maxBin }),
|
||||
current: () => new OnsetDetector({ binHz: SR / FFT }),
|
||||
};
|
||||
|
||||
const median = a => { if (!a.length) return NaN; const s = a.slice().sort((p, q) => p - q); return s[s.length >> 1]; };
|
||||
|
||||
/** run variants over the corpus -> { [variant]: { [rate]: { n, hit, fp, minutes, medianMs, byInst, byBed, byLevel } } } */
|
||||
export function run(variantNames = Object.keys(VARIANTS), cases = corpus()) {
|
||||
const acc = {};
|
||||
for (const v of variantNames) { acc[v] = {}; for (const r of Object.keys(RATES)) acc[v][r] = { n: 0, hit: 0, fp: 0, seconds: 0, errs: [], inst: {}, bed: {}, lv: {} }; }
|
||||
for (const c of cases) {
|
||||
const { x, truth } = synthesize(c);
|
||||
for (const [rate, hop] of Object.entries(RATES)) {
|
||||
const frames = spectra(x, hop);
|
||||
for (const v of variantNames) {
|
||||
const s = score(truth, detect(VARIANTS[v], frames)), a = acc[v][rate];
|
||||
a.n += s.n; a.hit += s.hit; a.fp += s.fp; a.seconds += DURATION - SCORE_FROM; a.errs.push(...s.errs);
|
||||
for (const [k, p] of Object.entries(s.per)) { const q = a.inst[k] ??= { n: 0, hit: 0 }; q.n += p.n; q.hit += p.hit; }
|
||||
for (const key of ['bed', 'lv']) { const q = a[key][c[key]] ??= { n: 0, hit: 0, fp: 0 }; q.n += s.n; q.hit += s.hit; q.fp += s.fp; }
|
||||
}
|
||||
}
|
||||
}
|
||||
const out = {};
|
||||
for (const v of variantNames) { out[v] = {}; for (const [r, a] of Object.entries(acc[v])) out[v][r] = { ...a, hitRate: a.hit / a.n, fpPerMin: a.fp / (a.seconds / 60), medianMs: median(a.errs) * 1000 }; }
|
||||
return out;
|
||||
}
|
||||
|
||||
const pct = (h, n) => (100 * h / n).toFixed(1).padStart(5) + '%';
|
||||
export function report(res) {
|
||||
const lines = [];
|
||||
for (const [v, rates] of Object.entries(res)) for (const [r, a] of Object.entries(rates)) {
|
||||
const i = Object.entries(a.inst).map(([k, p]) => `${k} ${pct(p.hit, p.n)}`).join(' ');
|
||||
const b = Object.entries(a.bed).map(([k, p]) => `${k} ${pct(p.hit, p.n)}/${p.fp}fp`).join(' ');
|
||||
const l = Object.entries(a.lv).map(([k, p]) => `${k} ${pct(p.hit, p.n)}/${p.fp}fp`).join(' ');
|
||||
lines.push(`${v.padEnd(9)} ${r.padEnd(5)} hit ${pct(a.hit, a.n)} (${a.hit}/${a.n}) fp ${a.fp} (${a.fpPerMin.toFixed(1)}/min) median err ${a.medianMs.toFixed(1)} ms`);
|
||||
lines.push(` by instrument: ${i}`, ` by bed: ${b}`, ` by level: ${l}`);
|
||||
}
|
||||
return lines.join('\n');
|
||||
}
|
||||
|
||||
if (import.meta.url === `file://${process.argv[1]}`) {
|
||||
const names = process.argv.slice(2).length ? process.argv.slice(2) : Object.keys(VARIANTS);
|
||||
const cs = corpus();
|
||||
console.log(`corpus: ${cs.length} signals x ${DURATION} s, ${Object.keys(RATES).map(r => `${r}=hop ${RATES[r]}`).join(', ')}, SR ${SR}, FFT ${FFT}; match window -${MATCH_EARLY * 1000}..+${MATCH_LATE * 1000} ms of window end`);
|
||||
console.log(report(run(names, cs)));
|
||||
}
|
||||
65
web/tools/onset-legacy.mjs
Normal file
65
web/tools/onset-legacy.mjs
Normal file
|
|
@ -0,0 +1,65 @@
|
|||
// Frozen copy of the v2 onset detector as of commit f53a0d1, kept only so tools/onset-bench.mjs can reproduce the 'before' numbers.
|
||||
// Not imported by the page. Delete when the baseline stops being interesting.
|
||||
|
||||
export class LegacyOnsetDetector {
|
||||
constructor(opts = {}) {
|
||||
this.compress = opts.compress ?? 200; // log(1 + compress * mag)
|
||||
this.window = opts.window ?? 43; // median history, frames
|
||||
this.delta = opts.delta ?? 0.08; // absolute floor above the median, in flux units
|
||||
this.ratio = opts.ratio ?? 1.6; // and the flux must exceed ratio * median
|
||||
this.refractory = opts.refractory ?? 0.06; // seconds
|
||||
this.minBin = opts.minBin ?? 2;
|
||||
this.maxBin = opts.maxBin ?? 0; // 0 = all bins
|
||||
this.bands = opts.bands ?? 6; // log-spaced flux bands between minBin and maxBin
|
||||
this.edges = null;
|
||||
this.prev = null;
|
||||
this.hist = []; // recent flux values
|
||||
this.f1 = 0; this.f2 = 0; this.t1 = 0; // flux[n-1], flux[n-2], time[n-1]
|
||||
this.last = -1e9;
|
||||
this.flux = 0; // latest raw flux, for display
|
||||
}
|
||||
|
||||
push(mags, t) {
|
||||
const n = this.maxBin || mags.length;
|
||||
if (!this.edges) {
|
||||
this.edges = Array.from({ length: this.bands + 1 }, (_, i) => Math.round(this.minBin * Math.pow(n / this.minBin, i / this.bands)));
|
||||
for (let i = 1; i < this.edges.length; i++) this.edges[i] = Math.max(this.edges[i], this.edges[i - 1] + 1);
|
||||
}
|
||||
const cur = new Float32Array(n);
|
||||
for (let k = 0; k < n; k++) cur[k] = Math.log1p(this.compress * mags[k]);
|
||||
let flux = 0;
|
||||
if (this.prev) {
|
||||
const p = this.prev;
|
||||
// flux per log-spaced band, each divided by sqrt(its width): a kick (a few bass bins) then counts
|
||||
// as much as a hat (hundreds of treble bins), which one whole-spectrum sum would bury
|
||||
for (let b = 0; b < this.edges.length - 1; b++) {
|
||||
let bf = 0;
|
||||
for (let k = this.edges[b]; k < this.edges[b + 1]; k++) {
|
||||
const ref = Math.max(p[k - 1] ?? 0, p[k], p[k + 1] ?? 0);
|
||||
const d = cur[k] - ref;
|
||||
if (d > 0) bf += d;
|
||||
}
|
||||
flux += bf / Math.sqrt(this.edges[b + 1] - this.edges[b]);
|
||||
}
|
||||
flux /= this.edges.length - 1;
|
||||
}
|
||||
this.prev = cur;
|
||||
this.flux = flux;
|
||||
|
||||
// candidate is the previous frame: f1 must be a local maximum over f2 and the new flux
|
||||
let out = null;
|
||||
const cand = this.f1;
|
||||
if (this.hist.length >= 8 && cand > this.f2 && cand >= flux) {
|
||||
const sorted = this.hist.slice().sort((a, b) => a - b);
|
||||
const med = sorted[sorted.length >> 1];
|
||||
if (cand > med * this.ratio + this.delta && this.t1 - this.last >= this.refractory) {
|
||||
out = { time: this.t1, strength: Math.min(1, (cand - med) / (cand + 1e-6) * 1.5) };
|
||||
this.last = this.t1;
|
||||
}
|
||||
}
|
||||
this.hist.push(flux);
|
||||
if (this.hist.length > this.window) this.hist.shift();
|
||||
this.f2 = this.f1; this.f1 = flux; this.t1 = t;
|
||||
return out;
|
||||
}
|
||||
}
|
||||
Loading…
Reference in a new issue