GameCult/AquaSynth
Watch
9
0
Fork
You've already forked AquaSynth
 0
Code Issues Pull requests 4 Projects Releases Packages Wiki Activity Actions

Rebuild vocal tract lowering around primitives

Browse source
This commit is contained in:
Meta Vens 2026-05-29 16:30:59 +01:00
commit 4d7d26030b
12 changed files with 1247 additions and 396 deletions
Download patch file Download diff file Expand all files Collapse all files

20
state/spine.yaml
View file

@ -7,14 +7,12 @@ prime_directive: >
not to justify piling parameters onto the current patch DSL.
current_objective: >
Build the utterance parity curriculum for AquaSynth speech: structured
utterance metadata lowers into a learned utterance embedding, then into a
learned synth-driver automation vector for the vocal tract, envelopes, LFOs,
filters, and spectral lanes. The immediate proof is a fast harness that
renders simple utterances against a compact speech synthesizer reference,
scores synth output versus ground truth, and exposes a useful loss landscape
for backpropagation. External synth digestion is background pressure until it
directly helps this speech harness.
Rebuild vocal acoustics around a compact primitive machine before more
utterance golf: PT-shaped tract scripts adapt into continuous AreaFunction
morphology, WaveguidePath propagation, SourcePort excitation,
ConstrictionContact pressure/release, BranchPort coupling, RadiationLoad
boundary output, ProbeTimeline telemetry, and VocalNetwork ownership. Baby
words remain downstream witnesses after primitive flow is observable.
current_mechanism:
parser: src/AquaSynth.Core/PatchScript.cs
@ -31,6 +29,7 @@ current_mechanism:
dx7_reference_renderer_tests: tests/AquaSynth.Dsl.Tests/Dx7ReferenceParityTests.cs
zyn_reference_source: external/zynaddsubfx
pink_trombone_reference_source: external/pink-trombone-mod
primitive_vocal_timeline: src/AquaSynth.Core/ProbeTimelineReport.cs
zyn_upstream_instrument_bank: external/zynaddsubfx/instruments
zyn_reference_build_script: tools/zyn-reference/build-zyn-reference.ps1
zyn_pad_reference_renderer: tools/zyn-reference/ZynPadReference.cpp
@ -117,7 +116,7 @@ live_invariants:
- Do not hide missing graph structure behind unrelated voice fields.
- The speech lane is the active center: IPA parity, utterance embeddings, synth-driver automation, rendered loss, and compiled Faust control sweeps take priority over new DX7/Zyn digestion.
- Every learned speech arrow must stay inspectable: utterance metadata -> embedding, embedding plus phonetic features -> automation, automation -> rendered audio, rendered audio -> loss/gradient.
- Pink Trombone parity is structural acoustic-graph pressure, not permission to add more formant proxy knobs or revive the old waveguide lane. AquaSynth has a `tract` DSL command with PT-shaped runtime controls, but vocal acoustics now lower through the typed `AcousticPortNetwork` graph; `Voice.Tract` is authoring sugar, not a second audio authority.
- Pink Trombone parity is structural primitive-vocal pressure, not permission to add more formant proxy knobs or revive the old waveguide lane. AquaSynth has a `tract` DSL command with PT-shaped runtime controls, but default vocal acoustics now lower through `VocalNetwork`; `Voice.Tract` is authoring sugar, not a second audio authority.
- Physical voice modeling should generalize around continuous acoustic morphologies, acoustic port networks, source ports, branch/radiation ports, and wave-clock policies. A larynx, syrinx, or alien voice should differ by topology and source-port configuration, not by special-case modules. Faust `physmodels.lib`/`delays.lib` provide useful primitives (`pm.chain`, `pm.l2s`, `pm.waveguideFd*`, `de.fdelay*`, Thiran/allpass and Lagrange delays), but AquaSynth should still generate tract/branch scattering where live area controls and inspectable training parameters matter.
- The physical voice research packet now includes a practical implementation/researcher map (`research/physical-voice-modeling/PEOPLE.md`) and talk/transcript ledger (`research/physical-voice-modeling/talks/TALKS.md`). Birkholz/VocalTractLab pressures multi-level articulatory control and gestures; Story/TubeTalker pressures continuous parametric area functions and length/coupling; Smith pressures bidirectional delay-line scattering and fractional-delay waveguides; Fels/ArtiSynth pressures separation between biomechanics, acoustics, timeline controls, and diagnostics. These are reusable DSL semantics, not modules to clone.
- Physical voice implementation has moved PT pressure into neutral acoustic owners: `AcousticPath`, `AcousticTerminal`, `AcousticConnection`, `AcousticSourcePort`, `AcousticBranch`, `AcousticRadiationPort`, `WaveClockPolicy`, and `AcousticPortNetwork`. Parser commands `path`, `terminal`, `connect`, `source_port`, `branch`, `radiation_port`, `wave_clock`, and `acoustic_network` exist. Existing `tract_shape`, `glottis`, `tract_injection`, `nasal_branch`, and `tract` commands populate acoustic records as authoring aliases; the legacy `Voice.Tract` resonator/waveguide renderers are removed.
@ -130,10 +129,11 @@ live_invariants:
- `docs/acoustic-graph-dsp-audit.md` is the current Jenga audit for graph voice DSP. It says the path graph, typed terminals, KL/PT two-port scatter, source/topology separation, velum side-port area, generated lip geometry ownership, area-dependent segment loss, and utterance separability should stay. The first audit cut replaced the generic connection fallback pressure/admittance blend and PT-shaped three-port branch recognizer with one passive N-port area-scattering law for all non-bypass connection groups; emitted Faust no longer contains `graph_connection_pressure_*` for graph connections. The second audit cut moved graph radiation color/admittance into one named local primitive with emitted `graph_radiation_reference_area_*`, `graph_radiation_differentiation_*`, `graph_radiation_highpass_*`, `graph_radiation_admittance_*`, and `graph_radiation_flow_*` signals instead of inline `30/70` and `area + 1.0` expressions. The third audit cut hoisted graph turbulence/release into named per-source `_closure`, `_release`, `_reservoir`, `_reservoir_drive`, `_pressure_drive`, `_release_pressure`, `_noise_gate`, `_release_gate`, `_sustained`, `_burst_noise`, and `_release_pulse` locals. Reservoir charging now uses directed upstream-minus-downstream pressure for ordinary two-port path nodes and positive local incident pressure as fallback; latest artifacts `artifacts/parity/pink-trombone-utterance-logmel/20260528T115452268` and `artifacts/parity/pink-trombone-logmel/20260528T115719882` still show no metric movement. This points away from source-local reservoir charging and toward missing contact-aware closure state that participates in scattering/flow. Remaining suspect owners are unowned segment loss constant, still-empirical radiation/source constants, nearest-node source attachment, and missing contact-aware closure state.
- Graph source lowering now names per-node incident pressure and node-local source sums inside the recursive graph loop. Generated tract injection source banks now use cell-centered positions and `1 / sections` index scaling, so turbulence/release energy is born inside tube cells rather than directly on graph/radiation nodes. Latest source-position artifact under `artifacts/parity/pink-trombone-utterance-logmel/20260527T172949293`: mama/papa/thrombosis cosine 0.6654/0.7746/0.5522, rmsRatio 3.0010/2.5712/2.4943, articulation 0.1444/0.1177/0.1193. Static closure-release RMS improved to 1.0078 under `artifacts/parity/pink-trombone-logmel/20260527T172843945`, but papa articulation worsened; keep the cell-centered source placement because it is the correct graph-level source geometry, not because metrics liked it.
- Source ports now have an explicit `impedance`/`load` control and graph lowering uses node incident pressure to make glottal, labial/syrinx, reed, click, synthetic, and turbulence sources load-aware. Generated tract area terminals are `Junction` geometry owners again; generated `voices_0_contact_*` terminals are a sparse contact overlay capped at four interior sections, with named graph-owned `graph_contact_*_closure`, `_reservoir`, and `_release` signals so closure pressure storage is no longer either every area sample or only a turbulence-source trick. Eight generated contacts plus inline occlusion overflowed the native debug Faust compile; four sparse contacts pass the opt-in thrombosis graph probe. `patches/advanced/bird-syrinx.aqua` is the first checked-in differentiable syrinx patch: paired labial source ports, bronchial paths, a tracheal path, an area-scattering merge, and beak radiation, all through `AcousticPortNetwork`.
- Generated PT-style `tract` lowering now compacts the Faust graph without changing user-facing morphology resolution: `VocalTract.Sections` and `AcousticPath.AreaFunction.Sections` still preserve the declared tract grid, but generated oral area terminals are capped at 10, generated moving injection source ports at 8, and generated contacts at 4. Source position weighting widens to the compact source spacing so the source bank approximates continuous placement instead of restoring one port per cell. Latest debug evidence `artifacts/parity/pink-trombone-graph-thrombosis-probes/20260529T125048965` drops generated debug DSP size from 1652 to 532 lines, with source graph lines 687->162 and next-state lines 89->27; accepted utterance artifact `artifacts/parity/pink-trombone-utterance-logmel/20260529T125227926` still fails articulation, so this is backend coherence/speed, not parity victory.
- Historical graph compaction pressure: before the primitive cut, generated PT-style `tract` lowering capped oral area terminals at 10, moving injection source ports at 8, and contacts at 4 to keep Faust alive. That path is no longer the default vocal owner; keep the lesson, not the generator.
- Moving tract injection placement now uses a raised-cosine kernel over the compact source bank instead of a hard triangular source-position weight. Keep this direction: smoother continuous heuristics over fewer generated ports, not more discrete source terminals. Artifact `artifacts/parity/pink-trombone-utterance-logmel/20260529T140038717` still fails articulation but improves thrombosis active ratio/silence/speech-band slightly without increasing graph size.
- Generated lip radiation opening must mirror live tract lip opening. The path-end geometry and radiation aperture are one mouth owner, not separate truths. Artifact `artifacts/parity/pink-trombone-utterance-logmel/20260529T141822464` improves papa's plosive failure sharply (RMS 0.8369, motor-band ratio 0.0555), but mama still sounds vowel-poor and thrombosis is still mostly absent; next pressure is vowel body/radiation color and fricative continuity.
- Proprioception correction says baby words are downstream witnesses, not the rebuild owner. The live cut is to collapse generated graph-vocal lowering into a small set of instrumented physical primitives, compare primitive/sample-flow behavior against VTL, ArtiSynth, Story/TubeTalker, Smith/Faust, and PT/SndKit, then return to utterances once the tract body is observable. `docs/acoustic-graph-proprioception.md` owns this cut line.
- First primitive vocal teardown cut is live. `tract`, `glottis`, `tract_injection`, and `nasal_branch` now adapt PT fixtures into `AreaFunction`, `WaveguidePath`, `SourcePort`, `ConstrictionContact`, `BranchPort`, `RadiationLoad`, `ProbeTimeline`, and `VocalNetwork` records instead of emitting generated `voices_0_area_*`, `voices_0_contact_*`, or `voices_0_inj_*` acoustic banks. The obsolete `EnsureTractAcousticNetwork` generator is deleted. Faust lowering for `Voice.VocalNetwork` emits compact primitive path/source/contact/branch/radiation locals using `de.fdelay*` delay families and debug probes under `/debug/...`; `ProbeTimelineReport` writes deterministic primitive CSV samples. Next pressure is passivity/energy timeline samples plus PT/SndKit timeline comparison, not intelligibility tuning.
- TurbulenceJet now includes a release-timed burst-noise component in addition to the deterministic pressure pulse, but a stronger trial barely moved papa; the source is not the main remaining articulation bottleneck. Graph radiation now mixes 30% raw boundary flow with 70% high-passed flow after local admittance weighting, down from the over-bright 15/85 blend. Latest radiation/source artifact under `artifacts/parity/pink-trombone-utterance-logmel/20260527T174236049`: mama/papa/thrombosis cosine 0.6578/0.7692/0.5585, rmsRatio 3.0006/2.6153/2.3488, articulation 0.1437/0.1173/0.1191. Static `closure-release` under `artifacts/parity/pink-trombone-logmel/20260527T174139661` has rmsRatio 0.9900 and speechBandRatio 0.0842. This is still not speaking; next pressure is true tract resonance/passive scattering and pressure storage, not more burst gain.
- Same-path two-port area discontinuities now emit `graph_area_energy_in_*` and `graph_area_energy_out_*` probes alongside the three-port branch `graph_connection_energy_*` probes. Use these to inspect passivity and formant-body failures at the scatter sites rather than judging only final output loudness.
- `FaustExportOptions.DebugProbeUi` wraps graph probe signals in Faust `vbargraph` UI controls under `/debug/...` without changing audio output arity. Native streaming Faust patches now separate writable controls from read-only probe zones and can read/snapshot probe values after `ProcessBlock`; use this for acoustic graph passivity/radiation inspection, leave it off for normal parity renders.