Document acoustic graph proprioception cut
This commit is contained in:
parent
f148fb575c
commit
677c543c91
7 changed files with 184 additions and 4 deletions
|
|
@ -204,8 +204,8 @@ What it simulates: time-varying tract cross-sectional area derived from rest
|
|||
diameter plus tongue, constriction, and lip controls.
|
||||
|
||||
Current state: useful but simplistic. Diameter is squared into area, which is
|
||||
the correct broad convention. Generated lip opening now owns path-end geometry
|
||||
instead of also owning radiation opening.
|
||||
the correct broad convention. Generated lip opening now owns both path-end
|
||||
geometry and radiation aperture.
|
||||
|
||||
Smells:
|
||||
|
||||
|
|
@ -441,6 +441,8 @@ Cut or replace these before piling on more tuning:
|
|||
1. Unowned segment contact-loss constant `0.02`.
|
||||
2. Turbulence/release scalar constants without a named pressure-flow model.
|
||||
3. Nearest-node source attachment as the only interior source placement model.
|
||||
4. Any new vocal feature added before the baby-word vowel-body witness in
|
||||
`docs/acoustic-graph-proprioception.md`.
|
||||
|
||||
Cut in the 2026-05-28 N-port pass:
|
||||
|
||||
|
|
|
|||
163
docs/acoustic-graph-proprioception.md
Normal file
163
docs/acoustic-graph-proprioception.md
Normal file
|
|
@ -0,0 +1,163 @@
|
|||
# Acoustic Graph Proprioception Pass
|
||||
|
||||
Date: 2026-05-29
|
||||
|
||||
The failure is not lack of research. The failure is that research vocabulary
|
||||
outpaced owner contracts. The graph can name sources, contacts, radiation,
|
||||
area, losses, branches, and wave clocks, but the baby-word witness still says
|
||||
the machine does not own an open vowel, a nasal vowel transition, or a shaped
|
||||
plosive as audible speech.
|
||||
|
||||
## Objective
|
||||
|
||||
Make AquaSynth's graph voice able to say first words before adding more vocal
|
||||
machinery. `mama` and `papa` are not toy cases; they are the smallest useful
|
||||
truth test for vowel body, lip closure/opening, nasal/oral coupling, source
|
||||
continuity, and radiation color.
|
||||
|
||||
## Current Mechanism
|
||||
|
||||
`tract` authoring builds a high-resolution morphology surface:
|
||||
|
||||
- `VocalTract.Sections` owns section/index semantics.
|
||||
- `AcousticPath.AreaControl` owns live area expressions from tongue,
|
||||
constriction, lip, and velum controls.
|
||||
- `tract_motion` parses motor-intent slew and obstruction parameters, but graph
|
||||
area and delay lowering do not yet own those slew laws.
|
||||
|
||||
Generated tract lowering then compacts that morphology into a Faust-sized graph:
|
||||
|
||||
- up to ten oral area terminals,
|
||||
- up to eight moving injection source ports,
|
||||
- up to four contact terminals,
|
||||
- one lip radiation port and optional nasal branch/radiation.
|
||||
|
||||
`FaustExport` lowers the typed graph into bidirectional segment state:
|
||||
|
||||
- segment delay and area reflection own propagation,
|
||||
- sources inject pressure/flow into nodes,
|
||||
- contact terminals store and release scalar pressure,
|
||||
- radiation ports read boundary flow and emit output,
|
||||
- probes observe selected source/contact/radiation internals.
|
||||
|
||||
The latest artifact is
|
||||
`artifacts/parity/pink-trombone-utterance-logmel/20260529T141822464`.
|
||||
It improved the `papa` drum-hit failure after lip radiation started following
|
||||
live lip opening, but `mama` still reads as closed-mouth humming and
|
||||
`thrombosis` remains mostly absent.
|
||||
|
||||
## Invariants
|
||||
|
||||
- Typed `AcousticPortNetwork` is the only vocal audio authority. Do not restore
|
||||
the old proxy tract renderer.
|
||||
- Declared morphology resolution and compiled Faust graph density are separate
|
||||
truths. More tract sections must not imply more Faust terminals.
|
||||
- The mouth has one owner. Lip opening owns path-end geometry and radiation
|
||||
aperture together.
|
||||
- Source placement over a compact graph must be continuous enough for moving
|
||||
articulation. Do not restore one source terminal per tract cell by default.
|
||||
- Contact state must describe a pressure/flow event in the tract, not an output
|
||||
click or plosive gain rescue.
|
||||
- Listening verdicts override proxy metrics. A high cosine that cannot say
|
||||
`a` is evidence against the witness, not evidence for the voice.
|
||||
|
||||
## Jenga Findings
|
||||
|
||||
### Research Names Outpaced Owner Tests
|
||||
|
||||
The code has acquired real terms from the literature: source impedance, contact
|
||||
pressure, radiation load, fractional delay, area scattering. Those names are
|
||||
useful only when each has a falsifiable owner test. Right now the graph can
|
||||
pass structural checks while failing the smallest speech task.
|
||||
|
||||
### The Model Has Too Many Local Truths
|
||||
|
||||
Source, contact, area, loss, radiation, and output balance each implement part
|
||||
of a pressure/flow story. They are not all wrong, but they can each be locally
|
||||
reasonable while the whole graph says "buzz", "hum", or silence. The lip
|
||||
radiation bug was the clean example: path geometry knew the mouth opened, while
|
||||
radiation aperture did not.
|
||||
|
||||
### The Vowel Body Has No Gate
|
||||
|
||||
The current parity lane can render utterance WAVs and report log-mel,
|
||||
articulation, band ratios, and silence mismatch. It does not yet have a hard
|
||||
vowel-body gate that says: before plosives, before fricatives, before
|
||||
`thrombosis`, the compact graph must sustain an open `a` and a nasal/open
|
||||
`ma` transition with audible oral vowel body.
|
||||
|
||||
That missing gate let complexity accumulate around the wrong failure surface.
|
||||
|
||||
### Discrete Components Are Carrying Continuous Work
|
||||
|
||||
The compacting pass fixed the worst Faust compile pressure, but the graph still
|
||||
uses sparse banks of area terminals, source ports, and contact terminals as a
|
||||
compiled approximation of continuous articulation. That may be acceptable as a
|
||||
backend lowering choice, but the source/contact laws must become continuous
|
||||
heuristics over that compact bank. Adding density is the expensive answer and
|
||||
already proved toxic.
|
||||
|
||||
### Contact Release Is Not Yet Speech Mechanics
|
||||
|
||||
Graph contact release now reaches lip radiation. That was a real ownership cut.
|
||||
It is still a scalar closure/reservoir/release pulse, with directional weights
|
||||
and decay constants. A baby's `p` is a constrained-flow history through a
|
||||
changing tract aperture, not a drum trigger. The latest `papa` improvement came
|
||||
from fixing mouth ownership, not from adding release force.
|
||||
|
||||
### Motion Is Authored Outside The Acoustic Owner
|
||||
|
||||
`tract_motion` exists, and fixture curves smooth controls, but graph area and
|
||||
delay semantics do not own motor smoothing/passivity. Abrupt morphology changes
|
||||
can still enter fractional delay and area reflection directly. That leaves
|
||||
gesture realism outside the organ that emits sound.
|
||||
|
||||
### The Probes Do Not Yet Watch The Missing Vowel
|
||||
|
||||
Source/contact/radiation probes proved the graph is alive upstream. They do not
|
||||
yet report a vowel-body witness: formant/body energy, open-mouth radiation
|
||||
state, nasal/oral balance, and per-phone windows for `mama`/`papa`. We are
|
||||
observing the machine's organs, but not the baby-word invariant at the layer
|
||||
where the user hears the failure.
|
||||
|
||||
### Verification Health Is Part Of The Machine
|
||||
|
||||
The native Faust render path is slow and has needed process cleanup and larger
|
||||
timeouts. That pressure is not incidental. A graph that takes minutes to render
|
||||
three toy words and previously overflowed debug compilation is already telling
|
||||
us the implementation surface is too discrete and too costly.
|
||||
|
||||
## Cut Line
|
||||
|
||||
Stop adding acoustic features until the graph passes a vowel-body proof.
|
||||
|
||||
Before the next physics addition, add a focused witness for:
|
||||
|
||||
1. sustained open `a`,
|
||||
2. `ma` as nasal closure into open vowel,
|
||||
3. `pa` as lip closure/release into open vowel,
|
||||
4. the same windows in the accepted utterance artifacts.
|
||||
|
||||
The witness should write listening WAVs plus a short report of active vowel
|
||||
body, speech-band energy, mouth/radiation opening, and nasal/oral balance. It
|
||||
should fail loudly when the result is closed-mouth humming, a drum hit, or
|
||||
silence.
|
||||
|
||||
Candidate simplifications after that witness exists:
|
||||
|
||||
- demote contact release constants into a simpler aperture-shaped constrained
|
||||
flow primitive, or cut the scalar reservoir if it cannot prove speech value;
|
||||
- move `tract_motion` slew into graph-owned area/delay expressions;
|
||||
- replace nearest-node source attachment pressure with a continuous
|
||||
interpolated injection primitive over adjacent segment nodes;
|
||||
- use Faust physical-modeling/delay primitives where they collapse generated
|
||||
graph bulk without hiding owner contracts;
|
||||
- keep compact generated terminal counts unless a failed witness proves a
|
||||
specific missing sample point.
|
||||
|
||||
## Next Work
|
||||
|
||||
The next implementation pass should be a baby-word gate, not another graph
|
||||
feature. Once sustained `a`, `ma`, and `pa` are visibly and audibly wrong or
|
||||
right, the graph will have a clean target for cuts. Without that gate, every
|
||||
new coefficient is just another little brick in the stack.
|
||||
|
|
@ -224,3 +224,4 @@
|
|||
{"date":"2026-05-29","area":"acoustic-graph","summary":"Generated tract lowering now compacts the Faust graph: declared morphology sections remain intact, but generated oral area terminals are capped at 10, generated injection source ports at 8, and generated contacts at 4. The thrombosis debug DSP dropped from 1652 to 532 lines, source graph lines from 687 to 162, and next-state lines from 89 to 27; opt-in native debug compile/render and accepted utterance diagnostics pass faster. Metrics confirm this is a backend coherence cut, not articulation parity: thrombosis RMS improved to 0.1715 but active ratio/silence mismatch remain bad at 0.1598/0.7811.","evidence":"docs/acoustic-graph-dsp-audit.md; artifacts/parity/pink-trombone-graph-thrombosis-probes/20260529T125048965; artifacts/parity/pink-trombone-utterance-logmel/20260529T125227926"}
|
||||
{"date":"2026-05-29","area":"acoustic-graph","summary":"Moving tract injection source placement now uses a raised-cosine kernel over the compact source bank instead of a hard triangular weight. This keeps graph size fixed while making source ownership more continuous and DSP-friendly. The latest accepted utterance diagnostic still fails articulation, but thrombosis moves in the right direction: active ratio 0.1598->0.1826, silence mismatch 0.7811->0.7643, and speech-band ratio 0.2537->0.2704.","evidence":"docs/acoustic-graph-dsp-audit.md; artifacts/parity/pink-trombone-graph-thrombosis-probes/20260529T135910192; artifacts/parity/pink-trombone-utterance-logmel/20260529T140038717"}
|
||||
{"date":"2026-05-29","area":"acoustic-graph","summary":"Generated lip radiation now mirrors live tract lip opening into the radiation port opening. This fixes split ownership where path-end geometry followed /pink/lip/opening but radiation aperture stayed at its default. Latest accepted utterance artifact improves papa in the exact listening-failure direction: RMS ratio 0.3242->0.8369, cosine 0.7725->0.7996, and motor-band ratio 0.5238->0.0555, reducing the drum-hit plosive signature. Mama and thrombosis still fail, so remaining work is vowel body/radiation color and fricative continuity, not more plosive gain.","evidence":"docs/acoustic-graph-dsp-audit.md; artifacts/parity/pink-trombone-utterance-logmel/20260529T141822464"}
|
||||
{"date":"2026-05-29","area":"acoustic-graph","summary":"Proprioception pass confirmed the Jenga smell: research vocabulary and named graph components outpaced baby-word owner tests. The live cut line is now to stop adding graph-vocal physics until a focused vowel-body witness proves sustained open `a`, `ma`, and `pa` with listening-aligned reports. The likely missing owners are vowel body/radiation color, graph-owned tract motion, continuous source/contact heuristics over the compact graph, and probes that observe the word-level failure instead of only organ internals.","evidence":"docs/acoustic-graph-proprioception.md; docs/acoustic-graph-dsp-audit.md; artifacts/parity/pink-trombone-utterance-logmel/20260529T141822464"}
|
||||
|
|
|
|||
|
|
@ -20,6 +20,7 @@ surfaces:
|
|||
utterance_parity_research: docs/utterance-parity-research.md
|
||||
pink_trombone_parity: docs/pink-trombone-parity.md
|
||||
pink_trombone_dsl_parity_plan: docs/pink-trombone-dsl-parity-plan.md
|
||||
acoustic_graph_proprioception: docs/acoustic-graph-proprioception.md
|
||||
bird_syrinx_golf_research: research/bird-syrinx-golf/README.md
|
||||
syrinx_modeling_research: research/syrinx-modeling/README.md
|
||||
wet_voice_witness_fixture: tests/AquaSynth.Dsl.Tests/Fixtures/Speech/WetVoice01
|
||||
|
|
@ -53,6 +54,7 @@ modules:
|
|||
tract_shape_model: src/AquaSynth.Core/Model.cs
|
||||
acoustic_graph_lowering: src/AquaSynth.Core/FaustExport.cs
|
||||
acoustic_graph_missing_pieces_audit: docs/acoustic-graph-missing-pieces-audit.md
|
||||
acoustic_graph_proprioception: docs/acoustic-graph-proprioception.md
|
||||
pink_trombone_parity_fixtures: src/AquaSynth.Core/PinkTromboneParityFixtures.cs
|
||||
pink_trombone_utterance_fixtures: src/AquaSynth.Core/PinkTromboneUtteranceFixtures.cs
|
||||
bird_syrinx_patch: patches/advanced/bird-syrinx.aqua
|
||||
|
|
|
|||
|
|
@ -78,6 +78,7 @@
|
|||
"docs/ipa-vocal-tract-roadmap.md",
|
||||
"docs/utterance-parity-research.md",
|
||||
"docs/pink-trombone-dsl-parity-plan.md",
|
||||
"docs/acoustic-graph-proprioception.md",
|
||||
"state/spine.yaml"
|
||||
],
|
||||
"reference_strategy": {
|
||||
|
|
@ -90,7 +91,7 @@
|
|||
"Surge XT for modern hybrid synth modulation",
|
||||
"Cardinal/VCV-style patches for modular cable/rate semantics"
|
||||
],
|
||||
"rule": "For speech, add schema fields and learned-control lanes only when the parity harness has a loss surface that can exercise them. For reference synths, add DSL abstractions only when rebuilds prove the missing invariant. Parsing alone is inventory, not evidence of expressivity. Pink Trombone should be decomposed into reusable low-level Aqua primitives before audio golf, not copied as a bespoke clone or inflated into PT-shaped modules."
|
||||
"rule": "For speech, add schema fields and learned-control lanes only when the parity harness has a loss surface that can exercise them. For reference synths, add DSL abstractions only when rebuilds prove the missing invariant. Parsing alone is inventory, not evidence of expressivity. Pink Trombone should be decomposed into reusable low-level Aqua primitives before audio golf, not copied as a bespoke clone or inflated into PT-shaped modules. After the 2026-05-29 proprioception pass, do not add more graph-vocal physics until a baby-word vowel-body witness proves sustained open `a`, `ma`, and `pa`."
|
||||
},
|
||||
"ipa_vocal_tract_path": {
|
||||
"status": "This is now the active project center. Roadmap and research notes align the curriculum around structured utterance metadata -> learned utterance embedding -> learned synth automation -> vocal tract/envelope/LFO/filter/spectral controls -> rendered audio loss. The first Core contracts still defend the boundary: PhoneticIntent is the Weksa/language-owned handoff, ArticulatoryPlan is the AquaSynth realization layer, VocalTractMorphology declares anatomy capabilities, ArticulatoryConstraintReport is the host-visible veto surface, and VocalTractPlanResult is the tiny end-to-end proof artifact for host branching. IPA is an authoring input only: tokenization and feature mapping lower into language phonology profiles, timed articulatory gestures, morphology-specific tract/excitation curves, and Faust DSP. The staged support plan starts with tokenization and vowels, then core pulmonic consonants, expanded human IPA, airstream/phonation, and Weksa/alien morphologies. Optional eSpeak NG tests provide the first generated speech workout when ESPEAK_NG or espeak-ng is installed; eSpeak is intelligibility/timing/log-mel pressure, not anatomy truth. AquaSynth owns the learned utterance embedding encoder and synth driver: speech text embeddings plus prosody/emphasis hints plus Ghostlight/Epiphany-shaped character state vectors compress into utterance embeddings, and phonetic features plus that embedding feed a packed-buffer native C# backprop/Adam mapper that predicts tract, AM/FM, LFO, filter, and mel spectral-envelope controls. SpeechBackpropagationPipeline chains the two models: synth-driver target loss returns gradients with respect to its input, slices the utterance-embedding gradient, and updates the utterance encoder from downstream speech loss. The generated ground-truth training fixture renders eSpeak syllables, extracts log-mel evidence, and trains that chained pipeline toward eSpeak-seeded mel/control targets. CultMesh speech training now has payload manifests, peer/lease admission receipts, worker assignment, result collection, and central checkpoint application; workers do not own model updates. Real remote artifact chunk transfer and process supervision remain CultMesh infrastructure surfaces, not AquaSynth workaround territory.",
|
||||
|
|
|
|||
|
|
@ -2,6 +2,14 @@
|
|||
|
||||
Current slice:
|
||||
|
||||
- Proprioception pass after user called Jenga on the graph voice. Diagnosis is
|
||||
now durable in `docs/acoustic-graph-proprioception.md`: research vocabulary
|
||||
and named graph components outpaced baby-word owner tests. Do not add more
|
||||
graph-vocal physics until a focused vowel-body witness proves sustained open
|
||||
`a`, `ma`, and `pa` with listening-aligned reports. Next implementation pass
|
||||
should add that witness, then simplify contact/motion/source placement from
|
||||
the evidence instead of adding another coefficient.
|
||||
|
||||
- Implemented the first graph-owned closure reservoir cut. Contact terminals no
|
||||
longer contribute through the generic node source sum. Ordinary two-port path
|
||||
scatter now emits per-contact closure, reservoir drive, reservoir, release,
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
project: AquaSynth
|
||||
spine_version: 1
|
||||
updated: 2026-05-29T12:33:04+01:00
|
||||
updated: 2026-05-29T15:36:32+01:00
|
||||
|
||||
prime_directive: >
|
||||
Build coherent synth machinery. References exist to reveal missing structure,
|
||||
|
|
@ -48,6 +48,7 @@ current_mechanism:
|
|||
physical_voice_modeling_research: docs/physical-voice-modeling-research.md
|
||||
acoustic_graph_dsp_audit: docs/acoustic-graph-dsp-audit.md
|
||||
acoustic_graph_missing_pieces_audit: docs/acoustic-graph-missing-pieces-audit.md
|
||||
acoustic_graph_proprioception: docs/acoustic-graph-proprioception.md
|
||||
physical_voice_modeling_packet: research/physical-voice-modeling
|
||||
parameter_contracts: src/AquaSynth.Core/Model.cs
|
||||
|
||||
|
|
@ -132,6 +133,7 @@ live_invariants:
|
|||
- Generated PT-style `tract` lowering now compacts the Faust graph without changing user-facing morphology resolution: `VocalTract.Sections` and `AcousticPath.AreaFunction.Sections` still preserve the declared tract grid, but generated oral area terminals are capped at 10, generated moving injection source ports at 8, and generated contacts at 4. Source position weighting widens to the compact source spacing so the source bank approximates continuous placement instead of restoring one port per cell. Latest debug evidence `artifacts/parity/pink-trombone-graph-thrombosis-probes/20260529T125048965` drops generated debug DSP size from 1652 to 532 lines, with source graph lines 687->162 and next-state lines 89->27; accepted utterance artifact `artifacts/parity/pink-trombone-utterance-logmel/20260529T125227926` still fails articulation, so this is backend coherence/speed, not parity victory.
|
||||
- Moving tract injection placement now uses a raised-cosine kernel over the compact source bank instead of a hard triangular source-position weight. Keep this direction: smoother continuous heuristics over fewer generated ports, not more discrete source terminals. Artifact `artifacts/parity/pink-trombone-utterance-logmel/20260529T140038717` still fails articulation but improves thrombosis active ratio/silence/speech-band slightly without increasing graph size.
|
||||
- Generated lip radiation opening must mirror live tract lip opening. The path-end geometry and radiation aperture are one mouth owner, not separate truths. Artifact `artifacts/parity/pink-trombone-utterance-logmel/20260529T141822464` improves papa's plosive failure sharply (RMS 0.8369, motor-band ratio 0.0555), but mama still sounds vowel-poor and thrombosis is still mostly absent; next pressure is vowel body/radiation color and fricative continuity.
|
||||
- Proprioception pass says the current speech graph failed by letting research vocabulary outpace baby-word owner tests. Stop adding vocal features until a focused vowel-body witness proves sustained open `a`, `ma`, and `pa` windows against listening-aligned reports. `docs/acoustic-graph-proprioception.md` owns this cut line.
|
||||
- TurbulenceJet now includes a release-timed burst-noise component in addition to the deterministic pressure pulse, but a stronger trial barely moved papa; the source is not the main remaining articulation bottleneck. Graph radiation now mixes 30% raw boundary flow with 70% high-passed flow after local admittance weighting, down from the over-bright 15/85 blend. Latest radiation/source artifact under `artifacts/parity/pink-trombone-utterance-logmel/20260527T174236049`: mama/papa/thrombosis cosine 0.6578/0.7692/0.5585, rmsRatio 3.0006/2.6153/2.3488, articulation 0.1437/0.1173/0.1191. Static `closure-release` under `artifacts/parity/pink-trombone-logmel/20260527T174139661` has rmsRatio 0.9900 and speechBandRatio 0.0842. This is still not speaking; next pressure is true tract resonance/passive scattering and pressure storage, not more burst gain.
|
||||
- Same-path two-port area discontinuities now emit `graph_area_energy_in_*` and `graph_area_energy_out_*` probes alongside the three-port branch `graph_connection_energy_*` probes. Use these to inspect passivity and formant-body failures at the scatter sites rather than judging only final output loudness.
|
||||
- `FaustExportOptions.DebugProbeUi` wraps graph probe signals in Faust `vbargraph` UI controls under `/debug/...` without changing audio output arity. Native streaming Faust patches now separate writable controls from read-only probe zones and can read/snapshot probe values after `ProcessBlock`; use this for acoustic graph passivity/radiation inspection, leave it off for normal parity renders.
|
||||
|
|
@ -943,4 +945,5 @@ cut_line:
|
|||
- "Cut voice-field sprawl when a target needs topology."
|
||||
- "Cut any design that requires recompiling a patch to move an exposed musical parameter."
|
||||
- "Split a lower-level graph DSL if Faust/Cardinal-style references make the voice DSL lie."
|
||||
- "Cut any new graph-vocal feature that lands before the baby-word vowel-body witness can prove open `a`, `ma`, and `pa`."
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue