diff --git a/README.md b/README.md
index 2a44f9f..a0d7d51 100644
--- a/README.md
+++ b/README.md
@@ -122,14 +122,25 @@ See [`docs/faust-toolchain-boundary.md`](docs/faust-toolchain-boundary.md).
## Status
-The current slice covers the modular graph surface needed by the reference
-scripts, SFXR atoms, script scoring, audio comparison, presets, Faust emission,
-native Faust compilation/rendering, and installed Faust validation. Migration
-coverage is tracked in
+The current center of gravity is the speech-learning lane: structured utterance
+metadata -> utterance embedding -> synth automation -> vocal-tract/rendered-loss
+parity. The first proof path uses tiny eSpeak NG fixtures, log-mel evidence,
+packed C# backprop, and compiled Faust speech controls so many candidate weights
+can be tested against generated speech references without recompiling topology.
+
+The modular graph surface, SFXR atoms, DX7/Zyn reference work, script scoring,
+audio comparison, presets, Faust emission, native Faust compilation/rendering,
+and installed Faust validation remain active foundations. The broader
+reference-synth digestion project is paused unless it directly improves the
+speech harness or render-loss machinery. Migration coverage is tracked in
[`docs/migration-checklist.md`](docs/migration-checklist.md), because leaving
important things behind in the old repo would be a very efficient way to become
our own haunted house.
Reference-driven DSL growth is tracked in
-[`docs/reference-synth-roadmap.md`](docs/reference-synth-roadmap.md). Agent
+[`docs/reference-synth-roadmap.md`](docs/reference-synth-roadmap.md). The vocal
+tract curriculum is tracked in
+[`docs/ipa-vocal-tract-roadmap.md`](docs/ipa-vocal-tract-roadmap.md), with
+prior-work notes in
+[`docs/utterance-parity-research.md`](docs/utterance-parity-research.md). Agent
handoff state lives in [`state/spine.yaml`](state/spine.yaml).
diff --git a/docs/audio-doctrine.md b/docs/audio-doctrine.md
index ab0d10f..9c82f22 100644
--- a/docs/audio-doctrine.md
+++ b/docs/audio-doctrine.md
@@ -144,6 +144,12 @@ owns text, phonology, allophones, and phonetic intent. AquaSynth owns the
articulatory realization: gesture planning, morphology constraints, tract-area
targets, excitation events, and Faust lowering.
+The current speech-learning lane adds a trainable surface above that boundary:
+structured utterance metadata lowers into an utterance embedding, then into a
+synth-driver automation vector for tract controls, envelopes, LFOs, filters, and
+spectral lanes. Those learned models are controllers. They do not get to erase
+the tract boundary or make morphology optional.
+
The boundary must stay split into three inspectable records:
- `PhoneticIntent`: language-owned phones, features, timing, and prosody.
@@ -157,6 +163,11 @@ Anatomy has veto power. If a beaked morphology cannot form a bilabial closure,
the planner must report the missing capability instead of silently substituting
a fake sound and calling it expressive.
+The first parity reference is eSpeak NG because it is compact, deterministic,
+and cheap to render. It is timing and intelligibility pressure, not anatomy
+authority. Use it to populate loss surfaces and train controller weights; do not
+mistake matching its formant shortcuts for building a vocal tract.
+
## Metrics
Analysis exists to catch regressions and support search, not to crown winners.
@@ -170,6 +181,7 @@ Useful metrics:
- envelope distance for shape and timing
- log-mel spectrogram distance for perceptual-ish spectral shape
+- speech-parity loss over tiny utterance fixtures before broader IPA claims
- peak/RMS ratios for loudness sanity
- zero-crossing and centroid ratios for noise/brightness drift
- script readability and terseness scores for DSL golf work
diff --git a/docs/ipa-vocal-tract-roadmap.md b/docs/ipa-vocal-tract-roadmap.md
index 026593d..e8df69c 100644
--- a/docs/ipa-vocal-tract-roadmap.md
+++ b/docs/ipa-vocal-tract-roadmap.md
@@ -1,5 +1,34 @@
# IPA Vocal Tract Roadmap
+## Current Objective
+
+AquaSynth is currently focused on a speech-learning parity harness. The active
+machine is:
+
+```text
+structured utterance metadata
+ -> learned utterance embedding
+ -> learned synth automation
+ -> vocal tract simulation plus envelopes, LFOs, filters, and spectral controls
+ -> rendered audio
+ -> loss against a simple speech synthesizer reference
+```
+
+Every arrow in that chain is a trainable artifact. The immediate job is to make
+small utterances cheap to render, score, and backpropagate through, so hundreds
+of controller weights can be tested against generated ground truth. The older
+project of eating DX7, ZynAddSubFX, and other synthesizers remains useful
+reference pressure, but it is not the current center of gravity.
+
+The curriculum is deliberate:
+
+1. master basic IPA-level speech parity against a simple reference;
+2. grow from vowels into CV/CVC contrasts and core pulmonic consonants;
+3. only then add intonation, prosody, emphasis, emotional context, and
+ personality;
+4. finally let Weksa and nonhuman morphologies stress the machine without
+ letting IPA pretend it owns alien anatomy.
+
AquaSynth can get close to a Pink Trombone-class tract model, then move past it
by treating IPA as an authoring input rather than the synth itself. The machine
we want is not a giant table from symbol to sound. It is a pipeline:
@@ -110,6 +139,8 @@ Useful starting sources:
- eSpeak NG phoneme-table implementation reference:
+- Utterance parity research notes:
+
## Architecture
@@ -449,6 +480,23 @@ Metrics:
- intelligibility/listening notes for complete syllables;
- script/IPA terseness only after audio is credible.
+## Speech Parity Harness
+
+The first harness must stay humble and fast:
+
+- render a tiny utterance set from eSpeak NG or another compact reference;
+- extract normalized log-mel, timing, and envelope evidence;
+- train `SpeechBackpropagationPipeline` through the utterance encoder and synth
+ driver;
+- drive a compiled Faust candidate by changing `/speech/output/N` controls, not
+ by recompiling the patch for every weight probe;
+- write reports under ignored `artifacts/parity/...` so metrics and listening
+ receipts stay visible.
+
+Ground truth means "reference pressure," not "the truth about throats." eSpeak
+can teach early intelligibility, timing, and rough spectral targets. It cannot
+decide AquaSynth's anatomy, coarticulation, or Weksa morphology.
+
## Optional eSpeak NG Workout
The first IPA-adjacent parity lane uses eSpeak NG as a development reference.
diff --git a/docs/reference-synth-roadmap.md b/docs/reference-synth-roadmap.md
index 2e8de63..607bc73 100644
--- a/docs/reference-synth-roadmap.md
+++ b/docs/reference-synth-roadmap.md
@@ -1,5 +1,19 @@
# Reference Synth Roadmap
+## Current Focus
+
+The active AquaSynth work has moved to the utterance parity curriculum:
+structured utterance metadata lowers through a learned utterance embedding and a
+learned synth-driver automation layer into vocal-tract controls, envelopes,
+LFOs, filters, and spectral lanes. The immediate proof is a fast parity harness
+against a simple speech synthesizer reference, so many controller weights can be
+tested as rendered synth output versus generated ground truth.
+
+DX7, ZynAddSubFX, Faust examples, Surge, and Cardinal remain reference pressure.
+They are on the backburner while the speech lane earns IPA parity. Do not start
+new synth-digestion work unless it directly improves the vocal-tract harness,
+the Faust render path, or the loss/automation surfaces.
+
AquaSynth needs external synthesis references for the same reason the
SFXR path was useful: a real target prevents the DSL from becoming a pile of
locally plausible knobs. The goal is not to clone every synth's internals. The
@@ -53,10 +67,14 @@ missing graph feature instead of smuggling it through unrelated parameters.
### New Lane: IPA Vocal Tract / Weksa
-This is not merely another reference synth. It is a new authoring and rendering
-lane: IPA text lowers into phonetic features, timed articulatory gestures,
-morphology-specific tract/excitation curves, and Faust DSP. The roadmap lives in
-[`docs/ipa-vocal-tract-roadmap.md`](ipa-vocal-tract-roadmap.md).
+This is the current main lane, not merely another reference synth. Structured
+utterance metadata lowers into a learned utterance embedding, then into learned
+synth automation for the vocal tract and associated envelopes, LFOs, filters,
+and spectral controls. IPA text is the first curriculum surface for phonetic
+coverage. The roadmap lives in
+[`docs/ipa-vocal-tract-roadmap.md`](ipa-vocal-tract-roadmap.md), and the
+research pass lives in
+[`docs/utterance-parity-research.md`](utterance-parity-research.md).
Pressure on AquaSynth DSL:
@@ -69,13 +87,14 @@ Pressure on AquaSynth DSL:
First deliverable:
-- Stage 0/1 IPA tokenization and vowel feature fixtures.
-- Human baseline vowel tract-area planner.
-- A small render proof that can say a few vowels before the supported IPA table
- expands.
-- Optional eSpeak NG workout fixtures that render tiny text/IPA-adjacent speech
- references when `ESPEAK_NG` or an `espeak-ng` binary is available. These are
- intelligibility/timing references, not tract-anatomy truth.
+- A parity harness that renders tiny eSpeak NG utterances, extracts log-mel and
+ timing evidence, and trains the chained utterance encoder plus synth driver.
+- A compiled Faust speech-loss surface that can vary `/speech/output/N` controls
+ across many candidates without recompiling topology.
+- Stage 0/1 IPA tokenization and vowel feature fixtures tied to the harness.
+- Human baseline vowel tract-area planner and audible vowel proof.
+- CV/CVC fixtures only after the vowel/control harness produces useful loss
+ evidence. These are intelligibility/timing references, not tract-anatomy truth.
### Tier 1: DX7/Dexed
diff --git a/docs/utterance-parity-research.md b/docs/utterance-parity-research.md
new file mode 100644
index 0000000..d3e4e37
--- /dev/null
+++ b/docs/utterance-parity-research.md
@@ -0,0 +1,166 @@
+# Utterance Parity Research Notes
+
+Research pass: 2026-05-20.
+
+## Current Machine
+
+AquaSynth is growing a speech curriculum, not a theatrical pile of IPA
+decorations. The active pipeline is:
+
+```text
+structured utterance metadata
+ -> learned utterance embedding
+ -> learned synth automation
+ -> vocal tract, envelopes, LFOs, filters, and spectral controls
+ -> rendered audio
+ -> loss against a speech reference
+```
+
+Each arrow is a machine-learning artifact. The first useful job is not
+universal speech. It is a tight parity harness where small utterances can be
+rendered against a simple reference synthesizer, scored quickly, and used to
+probe hundreds of controller weights. The loss landscape comes first. Poetry
+can wait outside and smoke.
+
+## Prior Work Signals
+
+### Physical And Articulatory Synthesis
+
+Kelly-Lochbaum-style tube models remain the practical starting point for
+realtime tract acoustics. Pink Trombone and related ports prove that a simplified
+tract, glottis, noise source, and tract geometry can be interactive enough to
+use as a control target. Recent Pink Trombone optimization work treats that
+model as a realtime Kelly-Lochbaum articulatory synthesizer and fits physical
+parameters, which is close to AquaSynth's black-box rendered-loss pressure.
+
+VocalTractLab is the heavyweight reference for articulatory synthesis: a 3D
+tract model, glottal source, aeroacoustic simulation, and articulatory control.
+Its papers are a warning label: speech quality depends on control trajectories
+and coarticulation, not only on static phoneme shapes.
+
+Story's area-function model is especially useful for AquaSynth because it gives
+a middle representation between symbolic phonetic intent and tract acoustics.
+Vowels are shaped by tract modes; consonants can be expressed as constriction
+location, area, and range imposed on that substrate. That matches the local
+doctrine: area curves are the synthesis authority, formants are diagnostics.
+
+### Gesture Planning
+
+Articulatory phonology and task dynamics are the right control metaphor. Phones
+should lower into overlapping gestures, not a row of isolated presets. Haskins
+TADA is important because it treats an utterance as a gestural score feeding a
+task-dynamic model. AquaSynth's utterance schema should eventually resemble a
+score of targets, timings, coupling, emphasis, and speaking style; it should not
+be a giant enum of phonemes with cute flags.
+
+### Classical Formant References
+
+Klatt-style and eSpeak-style formant synthesis are useful early references
+because they are compact, deterministic, intelligible, and easy to render in
+bulk. They are not anatomy truth. If AquaSynth learns only to imitate eSpeak's
+formant shortcuts, it will build a loyal little mimic and call it a throat.
+
+The correct use is narrow: use eSpeak NG for IPA/text coverage, timing pressure,
+log-mel targets, and fast generated ground truth while the physical tract earns
+basic vowels and consonants. Then move the curriculum toward tract-shaped
+targets and listening fixtures.
+
+### Differentiable DSP And Neural Controllers
+
+DDSP is the strongest precedent for this shape of work: combine neural networks
+with known signal models so the model learns control rather than reinventing
+acoustics from raw samples. Magenta's DDSP work shows the general bargain:
+smaller models, stronger inductive bias, and trainable synthesis controls.
+
+More recent DDSP articulatory vocoder work synthesizes speech from articulatory
+measurements, F0, and loudness. That is not AquaSynth's exact setup, because
+AquaSynth starts from metadata and generated speech references rather than EMA
+capture. The useful lesson is architectural: split source controls from tract or
+filter controls, keep features interpretable, and train against rendered audio.
+
+Neural source-filter models are also relevant. They show that neural pieces can
+drive physically meaningful source/filter structure instead of replacing the
+whole backend with a black box. AquaSynth should use that as permission to learn
+automation and embeddings, not permission to throw away tract ownership.
+
+### IPA And Feature References
+
+PanPhon maps IPA segments to articulatory feature vectors and PHOIBLE provides
+language inventories plus distinctive features. Both are reference material for
+metadata and sanity checks. Neither should become the permanent internal truth.
+
+The reason is simple: Weksa and alien morphologies need "human IPA-like intent"
+without letting human phonetics own nonhuman anatomy. IPA features can seed
+intent; morphology owns whether the requested articulation is possible.
+
+## Implications For AquaSynth
+
+Keep the curriculum brutally small:
+
+1. Render tiny eSpeak fixtures and extract normalized log-mel evidence.
+2. Train the utterance encoder and synth-driver chain from that rendered loss.
+3. Add a compiled Faust candidate path that can vary controller outputs without
+ recompiling topology.
+4. Use vowels first, then CV/CVC contrasts, then broader IPA.
+5. Only move to intonation, prosody, emotional context, and personality after
+ the harness can show phoneme-level loss improvements without handwaving.
+
+The universal utterance schema should stay boring at first:
+
+- segment inventory and source spans;
+- duration, stress, tone, and boundary hints;
+- speaker/morphology identity;
+- pitch, loudness, speaking-rate, and breath controls;
+- emphasis and emotional/context vectors as named side inputs;
+- diagnostics when a field is not yet consumed.
+
+Do not encode every future expressive dimension before the parity harness can
+make `/a pa ta ka sa ma/` less embarrassing. Grand schemas are where bugs go to
+earn tenure.
+
+## Source Ledger
+
+- Pink Trombone interactive vocal tract:
+
+- Pink Trombone optimization / Kelly-Lochbaum parameter fitting:
+
+- Sndkit tract notes, Pink Trombone/Kelly-Lochbaum lineage:
+
+- VocalTractLab project:
+
+- VocalTractLab features:
+
+- VocalTractLab coarticulation model paper:
+
+- Story 2005 area-function model:
+
+- Stop consonants and Story area-function model discussion:
+
+- Haskins TADA:
+
+- eSpeak NG repository and formant-synthesis reference:
+
+- PanPhon package:
+
+- PanPhon paper:
+
+- PHOIBLE:
+
+- PHOIBLE FAQ:
+
+- Magenta DDSP:
+
+- DDSP paper:
+
+- DDSP review for music and speech synthesis:
+
+- Introduction to differentiable synthesizer programming:
+
+- DDSP articulatory vocoder:
+
+- Neural source-filter waveform model:
+
+- LF/voice source model comparison:
+
+- LF model fitting for statistical parametric speech synthesis:
+
diff --git a/state/evidence.jsonl b/state/evidence.jsonl
index 07443c7..c7c8717 100644
--- a/state/evidence.jsonl
+++ b/state/evidence.jsonl
@@ -119,3 +119,5 @@
{"date":"2026-05-19","topic":"cultcache-sound-claim-cards","lesson":"Serious AquaSynth sound claims should be distributed as typed data beside the patch database. Patch and speech claim cards must name the reference/source, intent, tier, proof artifact, latest listening receipt surface, metrics, and known lies; stock sketches stay in the database but should not masquerade as parity without receipts.","evidence":"Added CultCachePatchDocument types, patches/aquasynth-patch-cultcache.json, and tests that load the typed document, check patch database count against library.yaml, and require every ReferenceRebuildCatalog entry to have a claim card."}
{"date":"2026-05-19","topic":"cultcache-cc-artifact-boundary","lesson":"AquaSynth must not impersonate CultCache persistence. The sound-claim fixture should be a real .cc CultCache store produced and consumed through CultLib; AquaSynth owns typed sound-claim meaning, while CultLib owns serialization, schema cataloging, record keys, and MessagePack backing-store mechanics.","evidence":"Deleted the local CultCacheBinaryCodec, replaced the JSON cache artifact with patches/aquasynth-patch-cultcache.cc, added a GameCult.Caching.MessagePack project reference, and updated CultCachePatchDocumentTests to load the fixture through CultCachePatchDocumentStore.ReadAsync."}
{"date":"2026-05-19","topic":"distributed-speech-training-cache-contract","lesson":"The speech-learning machine should distribute expensive render evidence instead of renderer authority. AquaSynth emits typed speech render requests, Faust workers write typed score results, and the trainer consumes synced gradients to update the utterance encoder and synth driver; CultCache/CultMesh carry the documents between machines.","evidence":"Added SpeechDistributedTraining.cs with SpeechRenderRequest, SpeechRenderResult, SpeechTrainingCheckpoint, CultCache store helpers, and a coordinator proof. SpeechDistributedTrainingTests round-trip request/result/checkpoint .cc stores and prove synced score gradients reduce local pipeline loss."}
+{"date":"2026-05-20","topic":"utterance-parity-current-objective","lesson":"AquaSynth's active objective is now the utterance parity curriculum: structured utterance metadata lowers into a learned utterance embedding, then into learned synth automation for the vocal tract and associated envelopes/LFOs/filters/spectral lanes, then into rendered speech loss. Broad DX7/Zyn synth digestion is backburner work unless it directly improves this harness.","evidence":"Updated README, docs/audio-doctrine.md, docs/reference-synth-roadmap.md, docs/ipa-vocal-tract-roadmap.md, state/map.yaml, state/spine.yaml, and state/memory.json to make the speech parity harness the current center."}
+{"date":"2026-05-20","topic":"utterance-parity-prior-work","lesson":"Prior work supports AquaSynth's architecture when kept in its lane: Kelly-Lochbaum/Pink Trombone and Story area functions support tract/area authority; TADA/articulatory phonology supports overlapping gestures; eSpeak/Klatt-style formant synthesis is useful fast ground truth but not anatomy; DDSP/neural source-filter work supports learned controllers over interpretable DSP rather than black-box throat replacement.","evidence":"Added docs/utterance-parity-research.md with source ledger covering Pink Trombone, VocalTractLab, Story, TADA, eSpeak NG, PanPhon, PHOIBLE, DDSP, DDSP articulatory vocoders, neural source-filter models, and LF voice-source references."}
diff --git a/state/map.yaml b/state/map.yaml
index 846c8c5..81fda63 100644
--- a/state/map.yaml
+++ b/state/map.yaml
@@ -17,6 +17,7 @@ surfaces:
tests: tests/AquaSynth.Dsl.Tests
reference_roadmap: docs/reference-synth-roadmap.md
ipa_vocal_tract_roadmap: docs/ipa-vocal-tract-roadmap.md
+ utterance_parity_research: docs/utterance-parity-research.md
faust_toolchain_boundary: docs/faust-toolchain-boundary.md
dx7_public_domain_fixtures: tests/AquaSynth.Dsl.Tests/Fixtures/Dx7/PublicDomain
zyn_reference_source: external/zynaddsubfx
@@ -49,6 +50,7 @@ modules:
zyn_reference_reader: src/AquaSynth.Core/ZynInstrument.cs
zyn_coverage_survey: src/AquaSynth.Core/ZynInstrument.cs
ipa_vocal_tract_roadmap: docs/ipa-vocal-tract-roadmap.md
+ utterance_parity_research: docs/utterance-parity-research.md
package_boundary_tests: tests/AquaSynth.Dsl.Tests/PackageBoundaryTests.cs
vocal_tract_model_tests: tests/AquaSynth.Dsl.Tests/VocalTractModelTests.cs
utterance_embedding_learning_tests: tests/AquaSynth.Dsl.Tests/UtteranceEmbeddingLearningTests.cs
@@ -86,6 +88,7 @@ boundaries:
- optional eSpeak NG generated ground-truth fixture for supervised utterance-embedding and synth-driver gradient descent
- Faust-rendered speech loss surface in AquaSynth.Faust, using rendered audio analysis and finite-difference controller-output gradients
- compiled native Faust speech-loss surface with exported /speech/output/N controls for batch candidate rendering without recompiling DSP
+ - utterance parity curriculum research and docs for structured metadata -> utterance embedding -> synth automation -> rendered speech loss
- typed CultCache sound-claim document distributed as a real .cc MessagePack-backed CultCache store via CultLib, bundling the patch database with fixture-backed patch claim cards and speech-lane claim cards
dev_only:
- public-domain DX7 SysEx fixtures
diff --git a/state/memory.json b/state/memory.json
index 6af70f3..a017e14 100644
--- a/state/memory.json
+++ b/state/memory.json
@@ -51,6 +51,7 @@
"generated DSP artifact manifests for engine hosting",
"live patch compilation, native Faust factory lifetime, and offline sample-buffer render proof for engine hosts",
"phonetic intent, articulatory plan, morphology capability, and constraint-report contracts for vocal tract synthesis",
+ "utterance parity curriculum research and docs",
"patch script parser",
"SFXR atoms and parameter mapping",
"analysis/comparison metrics",
@@ -71,24 +72,26 @@
"docs/dx7-calibration-plan.md",
"docs/migration-checklist.md",
"docs/reference-synth-roadmap.md",
+ "docs/ipa-vocal-tract-roadmap.md",
+ "docs/utterance-parity-research.md",
"state/spine.yaml"
],
"reference_strategy": {
- "purpose": "Use external synths and DSP corpora as parity pressure for AquaSynth's patch DSL and Faust emitter: the DSL must reproduce behavior in terse, readable AquaSynth syntax, not clone foreign internals.",
+ "purpose": "Current priority is the utterance parity curriculum: structured utterance metadata -> utterance embedding -> synth automation -> rendered vocal-tract output -> loss. External synths and DSP corpora remain parity pressure for AquaSynth's patch DSL and Faust emitter, but broad synth digestion waits unless it directly improves the speech harness.",
"priority_order": [
+ "IPA/Weksa vocal tract synthesis for phonetic gesture planning, utterance embeddings, synth automation, human and alien tract morphologies, and conlang speech rendering",
"DX7/Dexed SysEx for FM operator topology",
"ZynAddSubFX instruments for additive, subtractive, PAD, formant, and layered patches",
"Faust examples/libraries for general DSP graph topology",
"Surge XT for modern hybrid synth modulation",
- "Cardinal/VCV-style patches for modular cable/rate semantics",
- "IPA/Weksa vocal tract synthesis for phonetic gesture planning, human and alien tract morphologies, and conlang speech rendering"
+ "Cardinal/VCV-style patches for modular cable/rate semantics"
],
- "rule": "Add DSL abstractions only when reference rebuilds prove the missing invariant. Do not consume references that cannot be backed by parity tests; parsing alone is inventory, not evidence of expressivity."
+ "rule": "For speech, add schema fields and learned-control lanes only when the parity harness has a loss surface that can exercise them. For reference synths, add DSL abstractions only when rebuilds prove the missing invariant. Parsing alone is inventory, not evidence of expressivity."
},
"ipa_vocal_tract_path": {
- "status": "Roadmap drafted for an IPA-to-vocal-tract lane, and the first Core contracts now defend the boundary: PhoneticIntent is the Weksa/language-owned handoff, ArticulatoryPlan is the AquaSynth realization layer, VocalTractMorphology declares anatomy capabilities, ArticulatoryConstraintReport is the host-visible veto surface, and VocalTractPlanResult is the tiny end-to-end proof artifact for host branching. IPA is an authoring input only: tokenization and feature mapping lower into language phonology profiles, timed articulatory gestures, morphology-specific tract/excitation curves, and Faust DSP. The staged support plan starts with tokenization and vowels, then core pulmonic consonants, expanded human IPA, airstream/phonation, and Weksa/alien morphologies. Optional eSpeak NG tests now provide a first external speech workout when ESPEAK_NG or espeak-ng is installed; eSpeak is intelligibility/timing pressure, not anatomy truth. AquaSynth now owns the learned utterance embedding encoder and synth driver: speech text embeddings plus prosody/emphasis hints plus Ghostlight/Epiphany-shaped character state vectors compress into utterance embeddings, and phonetic features plus that embedding feed a packed-buffer native C# backprop/Adam mapper that predicts tract, AM/FM, LFO, filter, and mel spectral-envelope controls. SpeechBackpropagationPipeline now chains the two models: synth-driver target loss returns gradients with respect to its input, slices the utterance-embedding gradient, and updates the utterance encoder from downstream speech loss. The generated ground-truth training fixture renders eSpeak syllables, extracts log-mel evidence, and trains that chained pipeline toward eSpeak-seeded mel/control targets.",
+ "status": "This is now the active project center. Roadmap and research notes align the curriculum around structured utterance metadata -> learned utterance embedding -> learned synth automation -> vocal tract/envelope/LFO/filter/spectral controls -> rendered audio loss. The first Core contracts still defend the boundary: PhoneticIntent is the Weksa/language-owned handoff, ArticulatoryPlan is the AquaSynth realization layer, VocalTractMorphology declares anatomy capabilities, ArticulatoryConstraintReport is the host-visible veto surface, and VocalTractPlanResult is the tiny end-to-end proof artifact for host branching. IPA is an authoring input only: tokenization and feature mapping lower into language phonology profiles, timed articulatory gestures, morphology-specific tract/excitation curves, and Faust DSP. The staged support plan starts with tokenization and vowels, then core pulmonic consonants, expanded human IPA, airstream/phonation, and Weksa/alien morphologies. Optional eSpeak NG tests provide the first generated speech workout when ESPEAK_NG or espeak-ng is installed; eSpeak is intelligibility/timing/log-mel pressure, not anatomy truth. AquaSynth owns the learned utterance embedding encoder and synth driver: speech text embeddings plus prosody/emphasis hints plus Ghostlight/Epiphany-shaped character state vectors compress into utterance embeddings, and phonetic features plus that embedding feed a packed-buffer native C# backprop/Adam mapper that predicts tract, AM/FM, LFO, filter, and mel spectral-envelope controls. SpeechBackpropagationPipeline chains the two models: synth-driver target loss returns gradients with respect to its input, slices the utterance-embedding gradient, and updates the utterance encoder from downstream speech loss. The generated ground-truth training fixture renders eSpeak syllables, extracts log-mel evidence, and trains that chained pipeline toward eSpeak-seeded mel/control targets.",
"lesson": "Do not build a complete IPA parser before the tract can say basic syllables. The durable machine is tokenizer -> feature mapper -> phonology profile -> gesture planner -> morphology -> tract plan -> Faust renderer, with dependency-injected seams and mocks at each boundary. Weksa should use IPA-like phonetic input without letting human IPA semantics own alien anatomy. Anatomy has veto power: if a morphology lacks bilabial closure or another required capability, AquaSynth reports that failure instead of smearing over it.",
- "next": "Implement Stage 0/1 fixtures: IPA token spans, vowel feature bundles, human baseline vowel tract-area targets, and a tiny audible vowel render proof before expanding consonants."
+ "next": "Tighten the parity harness before expanding expressivity: render tiny utterance fixtures, sweep controller outputs through compiled Faust controls, score log-mel/timing/envelope loss, backpropagate into the utterance encoder plus synth driver, then connect Stage 0/1 IPA token spans, vowel feature bundles, and human baseline tract-area targets to that loop."
},
"dx7_reference_path": {
"status": "DX7 SysEx parser extracts voice edit buffers, packed voices, packed 32-voice banks, six operators, envelopes, algorithm, feedback, oscillator sync, LFO, transpose, voice name, checksum validation, ReferencePatch features, algorithm topology from the reverse-engineered OPS algorithm ROM table, envelope-to-ADSR approximations, measured carrier output amplitude, isolated two-op modulation route scale, summed/cascaded route scale, calibrated feedback amount table, ratio detune lowering, voice transpose lowering, fixed-frequency operator lowering, ROM COM output compensation cut, operator key/rate scaling at the effective played note, topology-aware route lowering, staged operator envelopes with per-segment curves, raw and block-interpolated DX7 envelope tracing, applied-envelope curved lowering with optional sustain floor, two AquaSynth DSL rebuild attempts, first-class OperatorGraph support with graph-level vibrato, public-domain SysEx fixture coverage, dexed-py reference rendering, project-authored Dexed patch rendering, AquaSynth candidate rendering through Faust-generated C#, one thresholded rendered parity test for analog1.syx voice 13, a hard PRC SYNTH1 parity probe with listening WAV artifacts and log-mel <= 0.145, project-authored algorithm-8 summed-stack/cascade/summed-pair parity, rendered community parity for Mooger, Piano Bass, RES SYNTH1, and ANLGSYN 1 with a sustained high-band gate, a MELLOWSOLO pressure artifact test, and renderable delayed operator feedback.",
diff --git a/state/spine.yaml b/state/spine.yaml
index 0c46097..502bb80 100644
--- a/state/spine.yaml
+++ b/state/spine.yaml
@@ -1,15 +1,20 @@
project: AquaSynth
spine_version: 1
-updated: 2026-05-15T14:15:00+01:00
+updated: 2026-05-20T22:00:00+01:00
prime_directive: >
Build coherent synth machinery. References exist to reveal missing structure,
not to justify piling parameters onto the current patch DSL.
current_objective: >
- Turn AquaSynth into a reference-driven patch authoring and Faust
- generation library. Use external synths and DSP corpora as pressure tests so
- AquaSynth's DSL grows only when a real target proves the abstraction belongs.
+ Build the utterance parity curriculum for AquaSynth speech: structured
+ utterance metadata lowers into a learned utterance embedding, then into a
+ learned synth-driver automation vector for the vocal tract, envelopes, LFOs,
+ filters, and spectral lanes. The immediate proof is a fast harness that
+ renders simple utterances against a compact speech synthesizer reference,
+ scores synth output versus ground truth, and exposes a useful loss landscape
+ for backpropagation. External synth digestion is background pressure until it
+ directly helps this speech harness.
current_mechanism:
parser: src/AquaSynth.Core/PatchScript.cs
@@ -36,6 +41,7 @@ current_mechanism:
dx7_calibration_plan: docs/dx7-calibration-plan.md
zyn_pressure_survey: docs/zynaddsubfx-pressure-survey.md
roadmap: docs/reference-synth-roadmap.md
+ utterance_parity_research: docs/utterance-parity-research.md
parameter_contracts: src/AquaSynth.Core/Model.cs
live_invariants:
@@ -101,8 +107,23 @@ live_invariants:
- Audio metrics are evidence, not final judgment.
- Do not add a generic graph escape hatch until specific references force it.
- Do not hide missing graph structure behind unrelated voice fields.
+ - The speech lane is the active center: IPA parity, utterance embeddings, synth-driver automation, rendered loss, and compiled Faust control sweeps take priority over new DX7/Zyn digestion.
+ - Every learned speech arrow must stay inspectable: utterance metadata -> embedding, embedding plus phonetic features -> automation, automation -> rendered audio, rendered audio -> loss/gradient.
+ - eSpeak NG is an early parity reference for timing, IPA coverage, and log-mel pressure, not an anatomy oracle or final target.
+ - Do not expand into prosody, emotional context, personality, or broad Weksa expressivity until the tiny IPA utterance harness can show useful loss movement.
reference_targets:
+ ipa_vocal_tract_weksa:
+ priority: 0
+ purpose: "Current speech-learning lane: utterance metadata to embedding to synth automation to tract/rendered loss."
+ sources:
+ - "docs/ipa-vocal-tract-roadmap.md"
+ - "docs/utterance-parity-research.md"
+ status: "Boundary contracts, packed neural networks, utterance encoder, synth-driver mapper, chained backprop, optional eSpeak rendered fixtures, Faust finite-difference loss surface, compiled Faust speech-loss controls, distributed render request/result/checkpoint contracts, and sound-claim cards exist. The active need is a fast parity harness that can test many controller weights as rendered speech output against simple ground truth."
+ first_slice:
+ - "Keep the utterance schema small and testable: segments, spans, duration/stress/boundary hints, speaker/morphology identity, pitch/loudness/rate/breath controls, and named context vectors."
+ - "Use eSpeak NG fixtures for generated ground truth while remembering that formant parity is not anatomy parity."
+ - "Connect Stage 0/1 IPA and vowel tract-area fixtures to the compiled Faust loss surface before expanding consonants."
dx7_dexed:
priority: 1
purpose: "Structured FM/operator graph pressure test."
@@ -146,33 +167,24 @@ reference_targets:
next_agent_start_here:
recommended_first_task: >
- Continue the ZynAddSubFX phase. The first parser/classifier slice exists in
- `src/AquaSynth.Core/ZynInstrument.cs` with project-authored fixtures
- under `tests/AquaSynth.Dsl.Tests/Fixtures/ZynAddSubFX/ProjectAuthored`.
- Named `layer` declarations own source identity, additive `harmonics` banks
- attach to those layers, PAD `spectrum` banks attach to those layers as
- PADsynth-style FFT wavetable sources, and staged `env=rl` envelopes now
- work on normal voices/layer defaults. The pinned Zyn source now builds
- through `tools/zyn-reference/build-zyn-reference.ps1` using an ignored
- patched artifact copy and builds `ZynPadReference`, which emits PAD table
- samples or note audio from Zyn. The first optional project-authored PAD
- comparison writes WAV/report artifacts under `artifacts/parity/zyn-pad-reference`.
- User listening called the reference wineglass-pure and AquaSynth angry bees.
- The current de-bee/pitch pass cuts unearned noise/drive/soft-clip,
- interpolates spectral wavetable reads, uses 131072-frame PAD tables, splits
- table `root` from playback `freq`, removes gratuitous spread/vibrato,
- collapses same-note PAD voices into one coherent table, trims gain, and now
- reports log-mel 0.35669, envelope 0.347572, RMS ratio 1.001704, centroid
- ratio 1.004072, score 0.6211. A PAD fixture survey now writes Zyn note/table
- WAVs for `pad-pure`, `pad-texture`, and `pad-harmonic`; `pad-pure` and
- `pad-texture` are identical in the oracle, while `pad-harmonic` is real
- new pressure. Next pressure is source-table/profile and envelope semantics,
- not more decorative layers.
+ Continue the utterance parity harness. Start from
+ `docs/ipa-vocal-tract-roadmap.md`,
+ `docs/utterance-parity-research.md`,
+ `src/AquaSynth.Core/SpeechBackpropagationPipeline.cs`,
+ `src/AquaSynth.Core/UtteranceEmbeddingLearning.cs`,
+ `src/AquaSynth.Core/VocalTractLearning.cs`,
+ `src/AquaSynth.Faust/FaustRenderedSpeechLossSurface.cs`, and the optional
+ eSpeak tests. The goal is fast rendered speech parity for tiny IPA-adjacent
+ utterances: generate ground truth, sweep controller outputs through compiled
+ Faust controls, score log-mel/timing/envelope loss, and backpropagate into
+ the utterance encoder plus synth driver. Do not resume broad Zyn/DX7 work
+ unless it directly improves this harness.
must_read:
- AGENTS.md
- docs/audio-doctrine.md
- - docs/dx7-calibration-plan.md
- docs/reference-synth-roadmap.md
+ - docs/ipa-vocal-tract-roadmap.md
+ - docs/utterance-parity-research.md
- state/spine.yaml
- state/memory.json
- state/evidence.jsonl