AquaSynth/docs/physical-voice-modeling-research.md

# Physical Voice Modeling Research Pass

Canonical research packet: `research/physical-voice-modeling/`.

That folder contains downloaded/captured sources where possible, a source
manifest with hashes, talks/transcripts where available, a practical
researcher map, and the distilled implementation summary. This document remains
the docs-facing copy of the same architectural direction.

## Objective

Generalize AquaSynth voice modeling beyond Pink Trombone. The target is a
performant, Faust-friendly acoustic modeling layer that can express a larynx,
syrinx, or stranger anatomy from the same primitives:

```text
continuous morphology + acoustic source ports + propagation network
  -> generated Faust
  -> differentiable controls and log-mel/audio parity
```

Pink Trombone remains useful pressure, but its 44 oral cells, 28 nasal cells,
and twice-per-sample step loop are implementation evidence, not the language.

## Research Grounding

The useful physical model is still a wave/acoustic network, not a formant knob
pile.

- Digital waveguides are a standard efficient representation for 1D acoustic
  systems. Faust's `physmodels.lib` is built around this exact class of
  structures: bidirectional chains, waveguides, terminations, tubes, reeds,
  lips, bows, and other port interactions.
- Faust `physmodels.lib` exposes `pm.chain` for bidirectional physical-model
  blocks, `pm.l2s` for length-to-samples conversion using wave speed and sample
  rate, and `pm.waveguideUd`, `pm.waveguideFd`, `pm.waveguideFd2`, and
  `pm.waveguideFd4` for unit and fractional delay waveguides.
- Faust `delays.lib` exposes several delay families relevant to this work:
  `de.fdelay` for simple linear fractional delay, Lagrange fractional delays
  through `de.fdelayltv` and `de.fdelay1..5`, and Thiran/allpass delays through
  `de.fdelay1a..4a`. The allpass family preserves magnitude better but has
  explicit lower-delay stability limits; first-order Thiran bottoms out around
  a half sample.
- Vocal-tract waveguide literature treats half-sample Kelly-Lochbaum models as
  useful for speech research and uses fractional-delay filters to vary segment
  lengths continuously without changing sampling rate.
- Recent differentiable speech work is converging on the same division:
  physically meaningful low-dimensional articulatory controls feed differentiable
  acoustic models or differentiable approximations of them. The important point
  is not whether the model is literally PT; it is whether the controls are
  smooth, bounded, interpretable, and connected to the rendered loss.

Primary sources consulted:

- Faust libraries, `physmodels.lib`: https://faustlibraries.grame.fr/libs/physmodels/
- Local Faust 2.85.5 `physmodels.lib` and `delays.lib` under
  `C:\Program Files\Faust\share\faust\`.
- Mathur, Story, Rodriguez, "Vocal-tract modeling: Fractional elongation of
  segment lengths in a waveguide model with half-sample delays":
  https://doi.org/10.1109/TSA.2005.858550
- Mullen, Howard, Murphy, "Waveguide Physical Modeling of Vocal Tract
  Acoustics":
  https://eprints.whiterose.ac.uk/3713/
- Frontiers DDSP review section on differentiable source-filter and
  differentiable Pink Trombone-style articulatory estimation:
  https://www.frontiersin.org/journals/signal-processing/articles/10.3389/frsip.2023.1284100/full
- Peter Birkholz / VocalTractLab sources and talk pages:
  https://www.vocaltractlab.de/ ,
  https://github.com/TUD-STKS/VocalTractLabBackend-dev ,
  https://lpp.cnrs.fr/evenement/srpp-de-peter-birkholz/ ,
  https://brahms.ircam.fr/en/media/xfb9e0a_peter-birkholz-how-physical-models-of-th
- Brad Story / TubeTalker and resonance tutorial:
  https://ncvs.org/vocal-tract-resonances-in-vowel-production/ ,
  https://www.youtube.com/watch?v=q23bAG-b6OA ,
  https://bpb-us-e2.wpmucdn.com/sites.arizona.edu/dist/f/80/files/2023/10/Story-2011_0-1.pdf
- Julius Smith CIRMMT physical-modeling talk:
  https://www.cirmmt.org/en/events/distinguished-lectures/Smith
- Sidney Fels / John Lloyd ArtiSynth talk:
  https://www.microsoft.com/en-us/research/video/developing-physically-based-dynamic-vocal-tract-models-using-artisynth/

The expanded packet now includes `PEOPLE.md` and `talks/TALKS.md`. Two public
YouTube transcripts were downloaded: Julius Smith's physical-modeling lecture
and Brad Story's NCVS vocal-tract resonance tutorial. Birkholz and Fels/Lloyd
pages were preserved with transcript-unavailable notes because their public
pages did not expose transcript text.

## Authority Map

Owner: `AcousticMorphology`

- Owns continuous geometry: length, cross-sectional area/diameter curves,
  branch curves, radiation openings, and source-port positions.
- Does not own emitted grid size, PT section count, or Faust delay mechanics.

Owner: `AcousticPortNetwork`

- Owns topology: ports, directed wave variables, tube segments, scattering
  junctions, source couplings, branch junctions, and terminations.
- A larynx and a syrinx differ here mostly by source topology:
  - larynx: one primary glottal source feeding one tract;
  - syrinx: two independently controllable labial/bronchial source ports that
    can couple into a shared tracheal/oral path.
- Alien voices are additional source ports, branch graphs, nonlinear junctions,
  and radiation ports, not new special-case voice modules.

Owner: `WaveClock`

- Owns conversion from physical length to delay samples:
  `delaySamples = lengthMeters * sampleRate / propagationSpeed`.
- Chooses a lowering strategy per segment:
  - unit-delay compiled grid for cheap static geometry;
  - first-order Thiran/allpass for near-half-sample stable fractional delays;
  - Lagrange/linear delay for FIR-like interpolation where magnitude error is
    acceptable;
  - smoothed crossfaded variable delay when length changes rapidly.
- `substeps` is no longer an owner. It is a historical clue.

Owner: `DifferentiableControls`

- Owns the trainable/control surface:
  - normalized positions along named paths;
  - positive radii/areas via bounded or softplus-like transforms;
  - source pressure/tension/open quotient/noise as bounded continuous controls;
  - coupling/opening coefficients constrained into passive ranges;
  - slew rates and gesture envelopes as smooth controls.
- Does not expose raw emitted section indices as the primary interface.

## Faust Lowering Choices

### Keep Custom Generated Scattering For Articulatory Tracts

Faust `pm.chain` is attractive for generic bidirectional blocks, but AquaSynth
still needs generated named equations for tract-like acoustic networks because:

- the number of junctions and branch topology are compile-time graph shape;
- live controls must update areas, reflection coefficients, and source weights;
- generated code needs inspectable parameter names for training/playground use;
- nasal, lateral, syrinx, and alien branch points need multi-port scattering
  rather than a fixed string/tube model.

The old `wg_loop ~ si.bus(...)` tract backend was useful evidence, then became
a second vocal audio authority. It has been cut. The live state owner is the
typed acoustic graph (`graph_loop ~ si.bus(...)`), which should keep learning
from Faust's physical-modeling library without reviving a PT-shaped backend.

### Use Faust Library Delay Blocks Where They Match The Segment

For future non-uniform segments, generate delay elements from the library:

- `de.fdelay(max, delay)` for simple fractional delay and quick parity probes.
- `de.fdelay1a(max, delay)` or related Thiran/allpass delays for magnitude-flat
  near-half-sample interpolation when the delay is within the documented stable
  range.
- `de.fdelayltv(order, max, delay)` or `de.fdelay1..5` when smooth variable
  segment length matters more than allpass phase behavior.
- `pm.waveguideFd4(max, delay)` for larger bidirectional tube segments that can
  live as chain blocks.

Do not blindly place arbitrary fractional delay inside every scattering edge.
For very short sections below a stable allpass delay bound, either:

- choose a coarser emitted scattering grid and let the area curve resample onto
  it; or
- use a half-sample KL structure where the delay model is explicitly designed
  for that regime.

The graph lowering now follows that rule. Segment delay is derived from
physical length without forcing a one-sample minimum; the wave-clock strategy
owns the legal lower bound. Generated graph tracts use `HalfSampleGrid`, which
matches the PT/Kelly-Lochbaum half-sample regime for roughly 44 oral samples
over a 17 cm tract. A canonical reflection-sign retest was rejected because
Aqua's current right/left port ordering already encodes the opposite local
orientation; changing the sign made `thrombosis` unstable/loud instead of more
speech-like.

### Separate Fast Training From Final Rendering

Aqua may eventually support at least two lowerings from the same acoustic
morphology, but only if their authority is explicit:

1. `graph-waveguide`: realtime Faust graph network for listening, gameplay,
   and high-fidelity parity.
2. `training-proxy`: differentiable transfer-function or modal/SOS proxy for
   fast optimization, initialized from the same geometry and periodically
   checked against the graph render.

That proxy is not allowed to be a fallback voice. It is acceptable only as a
named training artifact that cannot answer normal audio rendering requests.

## DSL Direction

Replace PT-shaped source words with physical voice words.

Preferred concepts:

- `morphology`: named continuous acoustic body.
- `path`: oral, nasal, bronchial, lateral, resonator, alien named tube/path.
- `branch`: topology edge between paths, with opening/coupling control.
- `source_port`: acoustic excitation port; can be glottal, labial, reed,
  turbulent jet, click cavity, or synthetic/alien.
- `radiation_port`: lip, nostril, beak, membrane aperture, side vent.
- `gesture`: smooth control over areas, openings, source parameters, and
  branch couplings.
- `wave_clock`: propagation speed, delay strategy, and emitted-grid policy.

Avoid concepts as primary public abstractions:

- `substeps`;
- `44 sections`;
- `glottis` as the only source kind;
- `nose` as the only branch kind;
- integer cell index controls as the trainable interface.

## Next Implementation Cut

First implementation cut is now underway in `src/AquaSynth.Core/Model.cs` and
`src/AquaSynth.Core/PatchScript.cs`. The new neutral records are the ownership
surface:

- `AcousticPath`: name, length, area curve, propagation speed, loss profile.
- `AcousticSourcePort`: path/position, source model, pressure/tension/noise
  controls.
- `AcousticBranch`: from path/position to path/position, opening/coupling,
  branch kind.
- `AcousticRadiationPort`: path/position, opening/reflection/radiation filter.
- `WaveClockPolicy`: unit grid, half-sample grid, fractional delay order, max
  delay, and smoothing mode.
- `AcousticPortNetwork`: names one primary path and the source, branch,
  radiation, and wave-clock records that make it renderable.

Parser support exists for `path`, `source_port`, `branch`, `radiation_port`,
`wave_clock`, and `acoustic_network`. Existing PT-pressure commands now also
feed the acoustic record lists: `tract_shape` demotes to an `AcousticPath`,
`glottis`/`tract_injection` demote to `AcousticSourcePort`, `nasal_branch`
demotes to an `AcousticPath` plus `AcousticBranch`, and each `tract` voice
gets an `AcousticPortNetwork` over generated oral/source/branch/radiation
records.

The second cut made those records audible, and the teardown cut removed the
temporary response proxy. An `acoustic` / `acoustic_voice` command can reference
an `acoustic_network`; Faust lowering now compiles the typed terminals into a
bidirectional segment graph. Source-port pressure/tension/opening and
branch/radiation opening-style controls bind through normal Aqua parameters, so
the knobs used by training and playground surfaces are attached to acoustic
records instead of side-channel state.

A syrinx can now be authored as two labial `source_port` records plus
bronchial/tract `branch` topology without a species mode. `AcousticPortNetwork`
now owns the vocal renderer directly; `Voice.Tract` is a larynx-shaped
authoring convenience over the same network rather than a separate audio
authority.

The topology decision is now a path graph with typed terminals. `AcousticPath`
owns tube geometry. `AcousticTerminal` owns a named position on a path and its
role (`junction`, `source`, `radiation`, boundary, or diagnostic probe).
`AcousticConnection` owns the set of terminals that scatter together and the
connection law. `source_port`, `radiation_port`, and `branch` remain ergonomic
commands, but they populate typed graph records rather than becoming the graph
law themselves.

The first graph compiler cut now exists. It splits paths at terminal positions,
creates bidirectional segment state in Faust, scatters connected terminals by
area-weighted pressure continuity, injects declared source ports at source
terminals, and radiates declared radiation ports. Faust validation passes for a
three-path trachea/oral/nasal graph. The next cut added wave-clock lowering and
same-node aggregation: segment delay now follows the network's `wave_clock`
strategy (`unit`, half-sample, linear fractional, Lagrange, or Thiran/allpass),
and co-located terminals on one path become a shared graph node so source,
junction, boundary, and radiation roles can inhabit the same anatomical point.
The remaining limits are now parity and law depth rather than graph ownership:
connection laws beyond area/pressure/bypass need sharper physical semantics,
runtime morphology changes still require care around recompilation boundaries,
and the old PT-shaped `Voice.Tract` renderer remains as a baseline. A tract can
now opt into graph lowering with `propagation=graph`, which reuses the
PT-shaped authoring commands while routing audio through the typed acoustic
network.