diff --git a/docs/ipa-vocal-tract-roadmap.md b/docs/ipa-vocal-tract-roadmap.md index e4406b9..c7ec624 100644 --- a/docs/ipa-vocal-tract-roadmap.md +++ b/docs/ipa-vocal-tract-roadmap.md @@ -244,12 +244,16 @@ AquaSynth should treat it as the training input contract: - 64-float projected Ghostlight/Epiphany-shaped character-state vector; - 64-float AquaSynth-owned learned utterance embedding output. -Training on serious speech fixtures is blocked until Weksa packets use that -fixed contract or a deliberately versioned successor. Toy vectors are allowed -for plumbing tests only. Do not feed IPA strings into the semantic text -embedding channel and pretend that it means speech shape; English semantics and -PanPhon sequence evidence are separate inputs. AquaSynth owns the learned -compression from PanPhon sequence to 256D phonetic realization embedding. +The first Weksa v0.1 training artifact now exists at +`E:\Projects\weksa\examples\speech-training\tiny-panphon-v0.1\batch.json`. +It covers the crawl-stage IPA set `a`, `pa`, `ta`, `ka`, `sa`, and `ma` with +fixed vector widths, zeroed semantic text embeddings, inline PanPhon-style +sequence evidence, deterministic prosody hints, and an Epiphany-compatible +neutral character-state vector. Toy vectors are allowed for plumbing tests only. +Do not feed IPA strings into the semantic text embedding channel and pretend +that it means speech shape; English semantics and PanPhon sequence evidence are +separate inputs. AquaSynth owns the learned compression from PanPhon sequence to +256D phonetic realization embedding. ### 4. Morphology Model diff --git a/state/evidence.jsonl b/state/evidence.jsonl index dba8037..49832c6 100644 --- a/state/evidence.jsonl +++ b/state/evidence.jsonl @@ -124,3 +124,4 @@ {"date":"2026-05-20","topic":"speech-training-time-witness","lesson":"Speech training receipts need time and confidence as evidence, not only loss. Each curriculum step should record when the stage decided, observed latency, intended latency budget, confidence, and artifact/model identifiers; a good-sounding result from an unbounded clockless path is not trustworthy.","evidence":"Added SpeechTimingReceipt to SpeechRenderRequest, SpeechRenderResult, and SpeechTrainingCheckpoint; seeded timing/confidence receipts in the local distributed-training proof; updated audio doctrine, IPA roadmap, utterance parity research notes, spine, and memory."} {"date":"2026-05-20","topic":"weksa-utterance-embedding-contract","lesson":"Do not start serious AquaSynth speech gradient descent from toy Weksa vectors, and do not treat semantic text embeddings as IPA embeddings. The coordinated Weksa handoff contract is `weksa.utterance_embedding_handoff.v0.1`: 1024 `bge-m3:latest` semantic speech-text floats, variable-length PanPhon sequence evidence, 256 AquaSynth-owned phonetic-realization floats produced by `aquasynth.panphon_sequence_encoder.v0.1`, 32 prosody/emphasis floats, 64 projected character-state floats, and a 64-float AquaSynth-owned utterance embedding output.","evidence":"Updated E:/Projects/weksa docs/schema/state to split speech text embedding from PanPhon sequence evidence and marked the existing Nibu toy fixture as non-training; mirrored the training block in AquaSynth IPA roadmap, utterance parity research notes, spine, and memory."} {"date":"2026-05-20","topic":"panphon-sequence-encoder-organ","lesson":"The 256D phonetic realization surface is now an AquaSynth-owned learned organ, not fixed phone slots. Weksa supplies variable-length PanPhon feature sequences; AquaSynth encodes them through a trainable phonetic sequence encoder before the utterance embedding encoder consumes the 256D result.","evidence":"Added PhoneticSequenceEncoding.cs with PanPhon feature frames, deterministic sequence sketching, PhoneticSequenceNeuralEncoder, training records, and tests proving the encoder learns compact PanPhon realization embeddings."} +{"date":"2026-05-20","topic":"weksa-v01-training-artifact","lesson":"The Weksa blocker has moved from schema definition to artifact consumption. Weksa now emits a machine-readable `weksa.utterance_embedding_handoff.v0.1` tiny PanPhon seed batch for `a`, `pa`, `ta`, `ka`, `sa`, and `ma`; AquaSynth's next serious training pass should load that artifact rather than local toy vectors.","evidence":"Added Weksa emitter `E:/Projects/weksa/tools/emit_tiny_panphon_handoffs.py` and generated `E:/Projects/weksa/examples/speech-training/tiny-panphon-v0.1/batch.json` with fixed 1024/32/64 input widths, inline PanPhon-style frames, and pending AquaSynth-owned 256/64 learned outputs."} diff --git a/state/memory.json b/state/memory.json index 3da01cb..fd29018 100644 --- a/state/memory.json +++ b/state/memory.json @@ -92,7 +92,7 @@ "ipa_vocal_tract_path": { "status": "This is now the active project center. Roadmap and research notes align the curriculum around structured utterance metadata -> learned utterance embedding -> learned synth automation -> vocal tract/envelope/LFO/filter/spectral controls -> rendered audio loss. The first Core contracts still defend the boundary: PhoneticIntent is the Weksa/language-owned handoff, ArticulatoryPlan is the AquaSynth realization layer, VocalTractMorphology declares anatomy capabilities, ArticulatoryConstraintReport is the host-visible veto surface, and VocalTractPlanResult is the tiny end-to-end proof artifact for host branching. IPA is an authoring input only: tokenization and feature mapping lower into language phonology profiles, timed articulatory gestures, morphology-specific tract/excitation curves, and Faust DSP. The staged support plan starts with tokenization and vowels, then core pulmonic consonants, expanded human IPA, airstream/phonation, and Weksa/alien morphologies. Optional eSpeak NG tests provide the first generated speech workout when ESPEAK_NG or espeak-ng is installed; eSpeak is intelligibility/timing/log-mel pressure, not anatomy truth. AquaSynth owns the learned utterance embedding encoder and synth driver: speech text embeddings plus prosody/emphasis hints plus Ghostlight/Epiphany-shaped character state vectors compress into utterance embeddings, and phonetic features plus that embedding feed a packed-buffer native C# backprop/Adam mapper that predicts tract, AM/FM, LFO, filter, and mel spectral-envelope controls. SpeechBackpropagationPipeline chains the two models: synth-driver target loss returns gradients with respect to its input, slices the utterance-embedding gradient, and updates the utterance encoder from downstream speech loss. The generated ground-truth training fixture renders eSpeak syllables, extracts log-mel evidence, and trains that chained pipeline toward eSpeak-seeded mel/control targets.", "lesson": "Do not build a complete IPA parser before the tract can say basic syllables. The durable machine is tokenizer -> feature mapper -> phonology profile -> gesture planner -> morphology -> tract plan -> Faust renderer, with dependency-injected seams and mocks at each boundary. Weksa should use IPA-like phonetic input without letting human IPA semantics own alien anatomy. Anatomy has veto power: if a morphology lacks bilabial closure or another required capability, AquaSynth reports that failure instead of smearing over it.", - "next": "Do not start serious speech gradient descent from toy embedding packets. Mirror Weksa `weksa.utterance_embedding_handoff.v0.1` first: 1024 bge-m3 semantic speech-text floats, variable-length PanPhon sequence evidence, 256 AquaSynth-owned phonetic-realization embedding floats from `aquasynth.panphon_sequence_encoder.v0.1`, 32 prosody/emphasis floats, 64 projected character-state floats, and 64 AquaSynth utterance-embedding output floats. Keep semantic text embeddings and phonetic realization separate. Then tighten the parity harness: render tiny utterance fixtures, sweep controller outputs through compiled Faust controls, score log-mel/timing/envelope loss, record per-stage decision time/latency/budget/confidence receipts, backpropagate into the utterance encoder plus synth driver, and connect Stage 0/1 IPA token spans, vowel feature bundles, and human baseline tract-area targets to that loop." + "next": "Start the next speech gradient descent pass from Weksa's emitted v0.1 seed batch at `E:\\Projects\\weksa\\examples\\speech-training\\tiny-panphon-v0.1\\batch.json`: 1024 bge-m3 semantic speech-text floats, variable-length PanPhon sequence evidence, 256 AquaSynth-owned phonetic-realization embedding floats from `aquasynth.panphon_sequence_encoder.v0.1`, 32 prosody/emphasis floats, 64 projected character-state floats, and 64 AquaSynth utterance-embedding output floats. Keep semantic text embeddings and phonetic realization separate. Next tighten the parity harness: load that batch into AquaSynth training records, render tiny utterance fixtures, sweep controller outputs through compiled Faust controls, score log-mel/timing/envelope loss, record per-stage decision time/latency/budget/confidence receipts, backpropagate into the utterance encoder plus synth driver, and connect Stage 0/1 IPA token spans, vowel feature bundles, and human baseline tract-area targets to that loop." }, "dx7_reference_path": { "status": "DX7 SysEx parser extracts voice edit buffers, packed voices, packed 32-voice banks, six operators, envelopes, algorithm, feedback, oscillator sync, LFO, transpose, voice name, checksum validation, ReferencePatch features, algorithm topology from the reverse-engineered OPS algorithm ROM table, envelope-to-ADSR approximations, measured carrier output amplitude, isolated two-op modulation route scale, summed/cascaded route scale, calibrated feedback amount table, ratio detune lowering, voice transpose lowering, fixed-frequency operator lowering, ROM COM output compensation cut, operator key/rate scaling at the effective played note, topology-aware route lowering, staged operator envelopes with per-segment curves, raw and block-interpolated DX7 envelope tracing, applied-envelope curved lowering with optional sustain floor, two AquaSynth DSL rebuild attempts, first-class OperatorGraph support with graph-level vibrato, public-domain SysEx fixture coverage, dexed-py reference rendering, project-authored Dexed patch rendering, AquaSynth candidate rendering through Faust-generated C#, one thresholded rendered parity test for analog1.syx voice 13, a hard PRC SYNTH1 parity probe with listening WAV artifacts and log-mel <= 0.145, project-authored algorithm-8 summed-stack/cascade/summed-pair parity, rendered community parity for Mooger, Piano Bass, RES SYNTH1, and ANLGSYN 1 with a sustained high-band gate, a MELLOWSOLO pressure artifact test, and renderable delayed operator feedback.", diff --git a/state/spine.yaml b/state/spine.yaml index 7ea695c..501238c 100644 --- a/state/spine.yaml +++ b/state/spine.yaml @@ -112,7 +112,7 @@ live_invariants: - eSpeak NG is an early parity reference for timing, IPA coverage, and log-mel pressure, not an anatomy oracle or final target. - Do not expand into prosody, emotional context, personality, or broad Weksa expressivity until the tiny IPA utterance harness can show useful loss movement. - Speech training receipts must keep time as a witness: every request, render result, and checkpoint should record stage decision time, latency spent, latency budget, confidence, and artifact/model identifiers. - - Serious speech gradient descent is blocked until Weksa packets use `weksa.utterance_embedding_handoff.v0.1` or a deliberate successor: 1024 semantic speech-text floats, variable-length PanPhon sequence evidence, 256 AquaSynth-owned phonetic realization embedding floats, 32 prosody/emphasis floats, 64 projected character-state floats, and a 64-float AquaSynth utterance embedding output. + - Serious speech gradient descent may now start from Weksa's first emitted v0.1 seed batch at `E:\Projects\weksa\examples\speech-training\tiny-panphon-v0.1\batch.json`: 1024 semantic speech-text floats, variable-length PanPhon sequence evidence, 256 AquaSynth-owned phonetic realization embedding floats, 32 prosody/emphasis floats, 64 projected character-state floats, and a 64-float AquaSynth utterance embedding output. - Do not use semantic text embeddings as a proxy for IPA or alien phonetic strings. Phonetic/IPA realization starts as PanPhon sequence evidence and is compressed by AquaSynth's phonetic sequence encoder. reference_targets: @@ -122,10 +122,10 @@ reference_targets: sources: - "docs/ipa-vocal-tract-roadmap.md" - "docs/utterance-parity-research.md" - status: "Boundary contracts, packed neural networks, utterance encoder, synth-driver mapper, chained backprop, optional eSpeak rendered fixtures, Faust finite-difference loss surface, compiled Faust speech-loss controls, distributed render request/result/checkpoint contracts, timing/confidence receipts, and sound-claim cards exist. The active need is a fast parity harness that can test many controller weights as rendered speech output against simple ground truth." + status: "Boundary contracts, packed neural networks, utterance encoder, phonetic sequence encoder, synth-driver mapper, chained backprop, optional eSpeak rendered fixtures, Faust finite-difference loss surface, compiled Faust speech-loss controls, distributed render request/result/checkpoint contracts, timing/confidence receipts, sound-claim cards, and a Weksa-emitted v0.1 tiny PanPhon batch exist. The active need is a loader/training pass that consumes Weksa's batch artifact and tests many controller weights as rendered speech output against simple ground truth." first_slice: - "Keep the utterance schema small and testable: segments, spans, duration/stress/boundary hints, speaker/morphology identity, pitch/loudness/rate/breath controls, and named context vectors." - - "Mirror Weksa v0.1 in AquaSynth constants/tests before any serious gradient descent run." + - "Load Weksa's tiny PanPhon v0.1 batch artifact into AquaSynth training records before the next serious gradient descent run." - "Use eSpeak NG fixtures for generated ground truth while remembering that formant parity is not anatomy parity." - "Connect Stage 0/1 IPA and vowel tract-area fixtures to the compiled Faust loss surface before expanding consonants." dx7_dexed: