Perlines/web/README.md

Perlines v2 (browser)

Zero-install audio visualizer: one static folder, WebAudio for analysis, WebGL2 for drawing, no build step, no dependencies, no network calls.

Status, plainly: no human has watched this run. It was written and tested by an agent. The DSP is checked by synthetic-signal tests (below). The renderers were exercised only by headless-Chrome captures and a frame-time script; nobody has listened to music through it, looked at it on a real display, or tried the microphone path. Expect rough looks and tuning mistakes.

Run

node web/tools/serve.mjs 8080      # then open http://localhost:8080/

Any static server works (python3 -m http.server -d web). localhost counts as a secure context, so the microphone prompt works; opened from file:// the mic may be refused. Then: drop an audio file on the page, press Microphone, or press Demo (a silent synthetic 120 bpm spectrum, for looking at scenes without sound). Keys: 1 2 3 scenes, F fullscreen, H HUD.

What is in it

Piece	File	What it does
Onset detector	web/src/dsp.js OnsetDetector	Four-band spectral flux in dB (40-150, 150-600, 600-2500, 2.5-12 kHz) against a per-bin max of the two previous frames (max-filtered across +-1 bin), per-band median + 8 MAD threshold with a per-band minimum rise, local-peak pick, one refractory period over all bands. Level-independent. Causal: confirms an onset one frame late. Synthetic-drum scores: tools/onset-bench.mjs.
Key estimate	web/src/dsp.js KeyEstimator	Peak-based chroma (parabolic-interpolated spectral peaks folded to 12 pitch classes, 4 s half-life) correlated against the 24 Krumhansl-Kessler key profiles. The key picks the scene tint: tonic walks the circle of fifths for hue, mode sets warmth.
Analysis state	web/src/analysis.js	One object all scenes read: 128 log bands, bass/mid/treble, onset counter, chroma, key, tint. Also DemoFeed.
Perlin noise	web/src/noise.js	Classic 3D Perlin and a 2D curl field (divergence-free flow) for the CPU-side paths.
Scenes	web/src/scenes/	Aurora disk, burning water, vector trails (below).
Persistence	web/src/gl.js Feedback	Two render targets; each frame fades the previous one and the scene adds on top.

Scenes

• Aurora disk (aurora.js): spectrum-to-mesh extrusion. A 112 x 224 polar mesh; angle is frequency (mirrored), radius is age, vertex height is the band level from a 128-row spectrum-history texture, modulated by noise. Onsets pulse and spin it.
• Burning water (water.js): a ripple plane lit in flame colours, plus an onset-driven spray. Particles are stateless: onsets write (birth, seed, strength, lane) into a ring buffer, and the vertex shader computes every position from age.
• Vector trails (trails.js): 96 heads, one per spectrum lane, ride a Perlin curl field; each leaves a 72-point polyline. An onset kicks the speed and respawns a spray of heads from the centre.

Tests and measurements

cd web && npm test        # node --test test/dsp.test.mjs test/onset.test.mjs (node >= 20, nothing to install)
cd web && npm run bench:onsets   # full synthetic-drum corpus, before/after table, about 30 s

Covers (dsp.test): ten synthetic clicks found at two hop sizes (11.6 ms and 16.7 ms) with no false positives; silence, steady noise and a sustained tone give no onsets; a 20 ms doublet merges under the refractory period; the key estimate reads C major, A minor and (transposed) B minor from synthesized chord progressions. Fixtures are deliberately plain: the estimate is known to confuse relative major/minor when a clip leans on the dominant, and one test fixture was rewritten to put its tonic in more chords (the estimator was not tuned to the fixture).

Covers (onset.test, synthetic drum corpus from tools/onset-bench.mjs: kick/snare/hat patterns at 90/120/150 bpm, three levels, bass+noise beds, 30 fps and 512-sample hops): hit rate, false positives and timing floors for the shipped detector, a margin over the frozen legacy detector, level independence, determinism. Synthetic only; the corpus was adjusted twice while building it (coincident kick+snare counted as one onset; voice tails faded so truncation clicks are not onsets), recorded in the commit message.

Frame time: the HUD shows rolling p50/p95 of requestAnimationFrame intervals against the 16.7 ms budget and CPU submit time. ?bench runs the three scenes on the synthetic feed and prints JSON (?w=1920&h=1080 fixes the drawing-buffer size, &sync reads a pixel back after each frame so the measured frame time is real cost, not capped by vsync, &ms= sets seconds per scene). web/tools/headless.mjs drives headless Chrome over CDP to collect it. Results of the last run are in web/BENCH.md.

Provenance

The ideas come from the 2013 Unity "Aethervis" project (Perlin-noise paths with spray and trails; spectrum-driven mesh extrusion; the Aurora, BurningWater and vectortrails scenes) and from the author's analysis-vocabulary notes (onsets, chroma, key). No source, shader or asset was copied; this is fresh code under the repository's licence. The analysis is written from published techniques (spectral flux, Krumhansl-Kessler profiles) and does not port Vamp/aubio/QM-DSP code, which is GPL-family.

Not done

Known weakness: on the synthetic demo feed at a simulated 30 fps the detector finds only about half the kicks (60 fps and 11.6 ms hops are fine); it has never been run on real music. Not built: beat/tempo tracking, per-note lanes (the C# client has them), audio-worklet capture (onsets are quantized to the render frame rate), mobile tuning, a quality auto-scaler.