AquaSynth/docs/acoustic-graph-proprioception.md
2026-05-29 21:53:13 +01:00

318 lines
15 KiB
Markdown

# Acoustic Graph Proprioception Pass
Date: 2026-05-29
This pass corrects the direction of the rebuild. The failure is not that the
graph lacks one more utterance target. The failure is that AquaSynth has been
letting a large discrete PT-shaped graph survive as the working body while
adding compensators around it. That is Jenga.
The live objective is to collapse the graph into a small set of physical
modeling primitives, instrument those primitives directly, and compare their
sample/flow behavior against serious reference implementations: especially
VocalTractLab, ArtiSynth, Story/TubeTalker, Smith-style waveguides, and only
then PT/SndKit as a compact stress fixture.
Baby words remain end-to-end witnesses. They are not the rebuild owner.
## Objective
Replace the monstrous discrete graph with as few faithful physical primitives
as possible:
- continuous tract geometry and area functions;
- source primitives coupled to tract load;
- branch/coupling primitives;
- radiation/load primitives;
- delay-line propagation and scattering primitives;
- contact/constriction primitives;
- graph instrumentation that exposes wave/sample flow at those owners.
The target is not a better audio golf loop. The target is an acoustic machine
whose internal behavior can be inspected and compared to reference simulators.
## Research Authority Map
### VocalTractLab / Birkholz
Owner lesson: separate anatomy, gestures, vocal-fold/source models, acoustic
simulation, and copy-synthesis controls.
Pressure on Aqua: `tract` should not lower into many little terminal records
that become the de facto anatomy. It should lower into continuous morphology
and explicit primitive controls that a gesture layer can drive and diagnostics
can observe.
### Story / TubeTalker
Owner lesson: continuous area functions and tract length are the compact bridge
between anatomy and acoustics. Formants are derived evidence.
Pressure on Aqua: emitted sections, sample delays, and source/contact sample
points are lowering choices. They must not become the authoring truth.
### Julius Smith / Digital Waveguides / Faust Primitives
Owner lesson: bidirectional delay lines, scattering junctions, loss filters,
fractional delay, and passive boundaries are the cheap realtime substrate.
Pressure on Aqua: generated Faust should use compact waveguide and delay
primitives where they preserve the owner contract. If custom emitted scattering
is needed for differentiable controls, it should still look like a small
physical primitive, not hundreds of section-local expressions.
### ArtiSynth / Fels and Lloyd
Owner lesson: body geometry, mechanical contacts, airflow/acoustics, timeline
controls, and diagnostics are separate organs in an integrated simulation.
Pressure on Aqua: instrumentation is not optional. We need probes that observe
geometry, contact state, pressure/flow, wave variables, source load, radiation,
and timeline transitions at the layer where the primitive claims authority.
### Pink Trombone / SndKit
Owner lesson: PT is a compact Kelly-Lochbaum stress case with oral tube, nasal
branch, source injection, obstruction history, and radiation.
Pressure on Aqua: use PT to compare behavior and catch regressions, but do not
let PT's cell grid or utterance sketches become the architecture.
## Current Mechanism
`tract` authoring is now a PT-shaped fixture adapter. It preserves the
high-resolution `VocalTract` authoring surface, but it no longer lowers into
generated `voices_0_area_*`, `voices_0_contact_*`, or `voices_0_inj_*` acoustic
banks.
The vocal DSP owner is the primitive network:
- `AreaFunction` owns continuous tract/nasal geometry, physical length, live
deformation controls, and the resampling policy for emitted Faust.
- `WaveguidePath` owns propagation speed, delay strategy, loss, and semantic
tube identity.
- `SourcePort` owns pressure/flow excitation and tract-load coupling.
- `ConstrictionContact` owns opening, resistance, stored pressure, and released
flow.
- `BranchPort` owns side-port admittance and exchanged flow between paths.
- `RadiationLoad` owns aperture, impedance/reflection, boundary flow, flow, and
radiated output.
- `ProbeTimeline` owns deterministic primitive telemetry for comparison before
utterance metrics.
- `VocalNetwork` ties those records together as the speech-path authority.
Explicit `AcousticPortNetwork` scripts still exist for lower-level acoustic
graph experiments, but PT-style `tract`, `glottis`, `tract_injection`, and
`nasal_branch` no longer own vocal DSP by generating a banked graph. They adapt
fixtures into primitive records.
## Invariants
- `VocalNetwork` owns PT-shaped vocal acoustics. Explicit `AcousticPortNetwork`
scripts remain a lower-level lab surface, not the default tract lowering.
- Continuous morphology owns tract shape and length. Generated section/terminal
count is backend approximation, not anatomy.
- Faust output must be compact and inspectable enough to compile, probe, and
compare.
- Primitives must state what physical quantity they own: area, pressure, flow,
wave variable, delay, coupling, contact, source load, or radiation.
- Instrumentation must observe primitive internals, not only final audio or
final metrics.
- PT utterances are witnesses after the primitive behavior is legible, not the
steering authority.
## Jenga Findings
### The Discrete Graph Is Still Too Much Body
The graph was reduced enough for Faust to compile, but its shape is still
section-bank driven: area terminals, source terminals, contact terminals, and
node-local expressions. This keeps pushing us toward local fixes because every
generated component looks like a place to add one more rule.
### We Need Primitive Flow Probes, Not Just Audio Reports
Current probes can show selected source/contact/radiation values, but they do
not yet provide a full primitive timeline: incoming/outgoing wave variables,
segment delay states, scattering inputs/outputs, source load pressure/flow,
contact opening/pressure, branch admittance, and radiation impedance over time.
Without that, comparing to VTL/ArtiSynth/PT is mostly listening plus aggregate
spectral evidence. That is not enough.
### Motion And Contact Are In The Wrong Shape
`tract_motion` parses gesture intent, but graph area/delay lowering does not
own motion/passivity as a primitive. Contact release exists, but it is still a
scalar reservoir pulse instead of a contact/constriction flow primitive that
can be inspected as pressure, opening, resistance, and released flow.
### Faust Compile Pressure Is Architectural Evidence
When Faust refuses to compile or becomes slow enough that three toy utterances
are painful, that is not tooling bad luck. It means the lowering is asking
Faust to carry too many discrete local expressions. The correct response is to
collapse into primitives, not to tune around the generated graph.
## Primitive Collapse Plan
### 1. Define The Minimal Primitive Set
Create or document the target primitives before adding another acoustic law:
- `AreaFunction`: continuous length/area geometry with live deformation.
- `WaveguidePath`: compact bidirectional propagation over that area function.
- `ScatterJunction`: two-port and N-port passive scattering over admittance.
- `SourcePort`: pressure/flow source with load coupling.
- `ConstrictionContact`: opening, resistance, stored pressure, and released
flow.
- `BranchPort`: side-port admittance and coupling.
- `RadiationLoad`: boundary reflection, aperture, impedance, and radiated flow.
- `ProbeTimeline`: dev/test-only sampling of the primitive state above.
- `ControlSurface`: the public normalized control for a primitive field. The
primitive owns the physical law; the surface owns the runtime knob and maps
`0..1` into the primitive field's physical range.
- `ControlSpline`: the gesture timeline over a `ControlSurface`. It owns
future-time motion, including linear, hold, and Bezier segments. Primitives do
not hide local slew.
If an existing record cannot be mapped onto one of these owners, it should be
deleted, demoted to authoring sugar, or kept only as a compatibility shim that
delegates to a primitive.
### 2. Compare Primitive Behavior Against References
Reference comparison should look at internal behavior, not only waveform
similarity:
- VTL/Birkholz: gesture-to-tract/source separation and copy-synthesis control
surfaces.
- ArtiSynth: body/contact/acoustic/timeline diagnostic separation.
- Story/TubeTalker: continuous area-function parameterization and formant
consequences.
- Smith/Faust: delay-line/scatter/loss/radiation primitive economics and
passivity.
- PT/SndKit: compact oral/nasal KL behavior, obstruction history, and radiation
under known fixtures.
### 3. Instrument Sample Flow Through The Graph
Add a dev/test probe path that can write per-block or per-window timelines:
- path area and delay samples;
- incoming/outgoing wave variables at selected scatter sites;
- source load pressure, source flow, and injected wave contribution;
- contact opening/resistance/reservoir/released flow;
- branch admittance and exchanged flow;
- radiation boundary reflection, flow, and emitted output;
- derived energy/passivity checks.
This is the layer that lets us say "the tract behaves like the reference" or
"this primitive is lying" before touching the final utterance.
### 4. Collapse Generated PT Lowering
Generated PT-style tract lowering is now an authoring adapter over the
primitive set. It may preserve PT-shaped controls for fixtures, but it cannot
decide the physical model by emitting banks of terminals where a continuous
primitive can own the same behavior.
## Cut Line
Cut any new feature whose main evidence is that it improves `mama`, `papa`, or
`thrombosis` without making the primitive flow more legible.
Cut any generated component that cannot explain which primitive owner it
serves.
Cut any probe that observes final audio while leaving source/load/contact/path
flow hidden.
## Next Work
The first primitive cut is in place: parser/model records, primitive Faust
lowering, tract fixture adaptation, and deterministic `ProbeTimeline` CSV
support. The second cut adds explicit path wave/passivity probes and a
PT/SndKit comparison report:
- `ProbeTimelineReport` now reports path `incoming_wave`, `outgoing_wave`,
`energy_in`, `energy_out`, and `passivity_ratio`, plus contact resistance.
- Primitive Faust debug probes expose the same path wave and passivity fields
under `/debug/...`.
- `PrimitiveReferenceReport.ComparePinkTrombone` compares primitive geometry,
delay, branch admittance, contact, and radiation fields against local
PT/SndKit fixture controls without adding a native reference dependency to CI.
- `PinkTromboneReferenceRenderer.RenderTimeline` exposes PT internal oral/nasal
path, branch, contact, and radiation timeline fields from its own arrays, and
`PrimitiveReferenceReport.ComparePinkTromboneTimeline` role-normalizes those
fields against Aqua primitive timelines.
- `PinkTrombonePrimitiveTimelineComparisonWritesArtifacts` writes per-fixture
Aqua timelines, PT internal timelines, comparison CSVs, and a summary under
`artifacts/parity/primitive-vocal-timeline`.
- The first comparison-driven correction demoted hidden source unit gain into
explicit `SourcePort.FlowScale`, stopped aperture from rewriting
`RadiationLoad.Reflection`, and fixed `ProbeTimelineReport` radiation flow to
observe path wave flow rather than `aperture / impedance` alone.
The next implementation pass should harden the primitives further:
- harden articulator/area spline ownership now that velopharynx motion lowers
through `ControlSpline` instead of primitive-local smoothing;
- add optional VTL/ArtiSynth adapters only as artifact comparators, not default
CI dependencies;
- keep `mama`, `papa`, and `thrombosis` as downstream witnesses until primitive
flow evidence improves.
## Gesture Surface Cut
The separate gesture/timeline owner is now the live direction:
- Primitive vocal lowering auto-publishes normalized `ControlSurface` records
for path speed/loss, area deformation fields, source pressure/tension/opening,
contact opening/resistance/storage, branch opening/coupling, and radiation
aperture/reflection/impedance.
- `control_surface` can declare an explicit surface, and `control_spline`
or `gesture surface=...` can attach linear, hold, or Bezier point timelines.
- Faust lowering emits normalized surface controls and spline point controls
under `/splines/<name>/<point>/...`, then maps the effective normalized value
back into the primitive's physical field.
- Explicit old `@/param` field bindings remain compatibility base controls;
spline deltas apply on top of that base instead of bypassing it.
- `ControlSurfaceCatalog.FromPatch(patch)` is the public gesture discovery API.
Hosts can query surfaces by normalized path/owner/field/range, create a
`ControlSplineTimeline`, edit future points, and call `ControlValuesAt(t)` to
stream ordinary surface controls without recompiling. This is the runtime path
for clients that want to author their own gestures.
- `AquaSynthCompiledPatch.ControlSurfaces` carries the same catalog through the
native Faust compile path, and `AquaSynthCompiledPatch.ControlValuesAt(...)`
filters timeline output to the controls actually exported by that compiled
DSP. The semantic catalog remains richer than optimized Faust UI zones.
- `tract_motion` now adapts velopharynx opening into a branch `ControlSpline`.
A generated contact-opening spline was tested and cut because PT evidence
showed it invented an open-then-close transient; contact closure must come
from articulator/area control, not a compensating contact-port story.
- `phoneme_gesture` is the first IPA/anatomy sugar layer. It accepts known IPA
symbols or explicit descriptors such as
`descriptor=voiceless_labial-velar_fricative`, identifies the primary organs
from place/manner/voicing labels, and emits ordinary `ControlSpline`s onto
source, area, branch, contact, and radiation surfaces. It is a deterministic
seed, not truth: local reference audio/articulation datasets golf the DSL
defaults and the patch's voice code.
Example:
```aqua
phoneme_gesture name=ma ipa=m start=0 dur=.16 intensity=.8
phoneme_gesture name=hw ipa=ʍ descriptor=voiceless_labial-velar_fricative start=.2 dur=.12
phoneme_gesture name=vowel ipa=a start=.34 dur=.20
```
Patches may still use normal FM, AM, modulators, envelopes, filters, and
post-voice processing around the anatomical control stream. Those tools shape
expression; they do not replace the phoneme-to-surface ownership map.
Latest focused evidence: `PatchScriptTests` pass 92/92, and
`PinkTrombonePrimitiveTimelineComparisonWritesArtifacts` writes artifact
`artifacts/parity/primitive-vocal-timeline/20260529T185955965`, where
`closure-release` returns to `0.155925` mean absolute primitive timeline error
after removing the bogus contact spline.