Force IPA trial agents to learn from failed rounds
This commit is contained in:
Meta Vens
parent
40ac243212
commit
06875c1eef
6 changed files with 49 additions and 7 deletions
|
|
@ -740,6 +740,20 @@ specific, comparable, falsifiable, and reusable by the next round. A pretty
|
|||
report with vague receipts is a failed science artifact even if it obeys the
|
||||
file-count contract.
|
||||
|
||||
The next round must learn from the previous one. A family that was flat,
|
||||
regressed, or failed compliance is falsified until the hypothesis worker states
|
||||
what changed: retired owner, surviving lesson, new perturbation, or source
|
||||
owner fork. Reusing a family name without that mutation is evidence churn, not
|
||||
learning.
|
||||
|
||||
Hypothesis families must also stay aligned with phonetic class. Stop-release
|
||||
families belong on stop lanes and `mix-p`; nasal-branch families belong on
|
||||
nasal/approximant lanes and `mix-m`; frication families belong on fricatives
|
||||
and `mix-s`; vowel/source-shape families belong on vowels and vowel-like mixed
|
||||
lanes. Cross-class transfer tests are allowed only when the matrix names the
|
||||
class-specific controls changed for that target. Round-robin family assignment
|
||||
is failed science even when all 25 files exist.
|
||||
|
||||
When the loop is run with source edits enabled, hypothesis agents may golf DSL
|
||||
syntax, parser/model records, Faust lowering, worker evidence surfaces, and
|
||||
tests. That is not permission to smear metrics. A source edit must name the
|
||||
|
|
|
|||
|
|
@ -246,3 +246,4 @@
|
|||
{"date":"2026-05-31","area":"ipa-trial-primitive-timeline-facts","lesson":"IPA trial evidence must carry primitive phase witnesses inside the CultCache record, not only as CSV artifacts or prose claims. `PrimitiveTimelineFactExtractor` turns `ProbeTimelineReport` samples into contact release, source flow, branch admittance, radiation output, path energy, and passivity facts; `IpaTrialWorker` indexes and reports those facts through search/show/dump so hypothesis agents can cite release/radiation/passivity evidence directly.","evidence":"src/AquaSynth.Core/PrimitiveTimelineFactExtractor.cs; src/AquaSynth.Core/IpaTrialResults.cs; tools/IpaTrialWorker/Program.cs; tests/AquaSynth.Dsl.Tests/IpaTrialOrchestratorTests.cs; artifacts/parity/ipa-timeline-fact-smoke/20260531T233217641"}
|
||||
{"date":"2026-05-31","area":"ipa-trial-source-golf-scene-voices","lesson":"IPA hypothesis agents need authority to golf the DSL/Faust machine when repeated evidence points at a missing owner, and every parity patch should model the reference as a scene with named voices. Source-edit-enabled rounds may now change parser/model/lowering/Faust/tests/docs with a source-change plan and pre-score build; candidate reports must separate primary articulation, tract noise, release/transient, background-room, and recording-condition voices. General synth primitives are allowed for scene/background modeling, but helper voices cannot impersonate articulation without lower-layer evidence.","evidence":"tools/run-ipa-trial-loop.ps1; docs/ipa-vocal-tract-roadmap.md; docs/audio-doctrine.md"}
|
||||
{"date":"2026-05-31","area":"ipa-trial-source-golf-smoke","lesson":"The first loose source-edit-enabled IPA round completed without tracked source edits and did not improve the aggregate metrics, which is valid negative evidence rather than a failure of authority. It exposed two harness contracts to tighten: evaluators must judge `hypotheses.md` as the report of record instead of the external agent's final `hypothesis-agent.md` chat summary, and scene/noise families must implement at least one actual helper/background `.aqua` `voice` declaration instead of satisfying scene modeling with prose.","evidence":"artifacts/parity/ipa-trial-loops/20260531T001831690; tools/run-ipa-trial-loop.ps1; docs/ipa-vocal-tract-roadmap.md; state/scratch.md"}
|
||||
{"date":"2026-05-31","area":"ipa-trial-learning-loop","lesson":"The loose source-golf round obeyed the file-count ritual but did not learn: it rotated families across unrelated phonetic targets, assigned stop-release ideas to vowel/approximant lanes, gave `p` an oral-pressure tweak instead of the stop-release owner, and produced aggregate regressions without converting them into the next hypothesis. The trial loop now passes previous evaluation paths into the hypothesizer, requires `Previous Round Learning`, requires a `Target-Class Lane Plan`, forbids round-robin family assignment, and lets the evaluator fail class alignment when families do not match phonetic owner classes or explicit transfer-test mutations.","evidence":"artifacts/parity/ipa-trial-loops/20260531T001831690/round-001/hypotheses.md; artifacts/parity/ipa-trial-loops/20260531T001831690/round-001/evaluation.md; tools/run-ipa-trial-loop.ps1; docs/ipa-vocal-tract-roadmap.md"}
|
||||
|
|
|
|||
|
|
@ -92,7 +92,7 @@
|
|||
"Surge XT for modern hybrid synth modulation",
|
||||
"Cardinal/VCV-style patches for modular cable/rate semantics"
|
||||
],
|
||||
"rule": "For speech, add schema fields and learned-control lanes only when the parity harness has a loss surface that can exercise them. For reference synths, add DSL abstractions only when rebuilds prove the missing invariant. Parsing alone is inventory, not evidence of expressivity. Pink Trombone should be decomposed into reusable low-level Aqua primitives before audio golf, not copied as a bespoke clone or inflated into PT-shaped modules. After the 2026-05-29 proprioception correction, collapse graph-vocal lowering into a small set of instrumented physical primitives before more utterance golf. Gesture motion belongs in ControlSurface/ControlSpline timeline ownership, not primitive-local smoothing; cut any generated contact/port spline that invents motion not supported by primitive reference timelines. Public clients author gestures through ControlSurfaceCatalog plus ControlSplineTimeline and stream surface control dictionaries; higher-level gesture templates and phoneme_gesture are only spline emitters. IPA descriptors seed which anatomical surfaces move, while local datasets golf numeric spline defaults and patch voice code. Training reports must separate gesture_score, clean_vocal_score, and full_parity_score so full-patch recording-condition helpers cannot impersonate articulation. Frozen IPA gesture rounds own candidate scripts, primitive timelines, gesture metrics, manifests, evidence receipts, metric summaries, candidate clusters, and science briefs; `IpaTrialResult` CultCache records own rendered trial outcomes plus extracted primitive timeline facts for contact release, source flow, branch admittance, radiation output, path energy, and passivity. External Codex IPA exploration must query the `.cc` store through the worker's semantic `search`/`show` organ, backed by Qdrant/Ollama vector retrieval when available; Qdrant is a derived index over CultCache and local artifacts, not a second truth. Search reports expose class focus/match and use phonetic class-affinity plus timeline-fact text so agents do not treat generic weak-articulation language as cross-class evidence. Source-edit-enabled IPA rounds may golf DSL syntax, parser/model records, Faust lowering, worker evidence surfaces, and tests when repeated evidence identifies a missing owner; every source edit needs an owner map, invariant, expected metric movement, verification, and rollback trigger. `hypotheses.md` is the hypothesis worker report of record; `hypothesis-agent.md` is only the external agent's final chat summary. IPA parity candidates should model scene voices explicitly: primary articulation, tract noise, release/transient, background-room, and recording-condition. At least one scene/noise or recording-condition family in a source-golf round must use real auxiliary `.aqua` `voice` declarations when claiming helper/background scene modeling. General synth primitives are allowed for helper scene voices, but they cannot count as articulation without gesture, primitive timeline, and clean-vocal movement. Crude full-store dumps are audit fallback, not agent memory. Statistical science workers and audio parity stages consume those bundles without rewriting gesture evidence. If gesture scores rise but rendered audio is silent, fix the primitive source/tract owner before adding full-patch dressing."
|
||||
"rule": "For speech, add schema fields and learned-control lanes only when the parity harness has a loss surface that can exercise them. For reference synths, add DSL abstractions only when rebuilds prove the missing invariant. Parsing alone is inventory, not evidence of expressivity. Pink Trombone should be decomposed into reusable low-level Aqua primitives before audio golf, not copied as a bespoke clone or inflated into PT-shaped modules. After the 2026-05-29 proprioception correction, collapse graph-vocal lowering into a small set of instrumented physical primitives before more utterance golf. Gesture motion belongs in ControlSurface/ControlSpline timeline ownership, not primitive-local smoothing; cut any generated contact/port spline that invents motion not supported by primitive reference timelines. Public clients author gestures through ControlSurfaceCatalog plus ControlSplineTimeline and stream surface control dictionaries; higher-level gesture templates and phoneme_gesture are only spline emitters. IPA descriptors seed which anatomical surfaces move, while local datasets golf numeric spline defaults and patch voice code. Training reports must separate gesture_score, clean_vocal_score, and full_parity_score so full-patch recording-condition helpers cannot impersonate articulation. Frozen IPA gesture rounds own candidate scripts, primitive timelines, gesture metrics, manifests, evidence receipts, metric summaries, candidate clusters, and science briefs; `IpaTrialResult` CultCache records own rendered trial outcomes plus extracted primitive timeline facts for contact release, source flow, branch admittance, radiation output, path energy, and passivity. External Codex IPA exploration must query the `.cc` store through the worker's semantic `search`/`show` organ, backed by Qdrant/Ollama vector retrieval when available; Qdrant is a derived index over CultCache and local artifacts, not a second truth. Search reports expose class focus/match and use phonetic class-affinity plus timeline-fact text so agents do not treat generic weak-articulation language as cross-class evidence. Source-edit-enabled IPA rounds may golf DSL syntax, parser/model records, Faust lowering, worker evidence surfaces, and tests when repeated evidence identifies a missing owner; every source edit needs an owner map, invariant, expected metric movement, verification, and rollback trigger. `hypotheses.md` is the hypothesis worker report of record; `hypothesis-agent.md` is only the external agent's final chat summary. Each new round must learn from the previous evaluation: flat/regressed/failed families are falsified until the report states what changed, and family/target assignment must be phonetic-class aligned instead of round-robin. Stop-release families belong on stop and `mix-p` lanes, nasal-branch on nasal/approximant and `mix-m`, frication on fricative and `mix-s`, and vowel/source-shape on vowel-like lanes unless an explicit transfer-test mutation names class-specific controls. IPA parity candidates should model scene voices explicitly: primary articulation, tract noise, release/transient, background-room, and recording-condition. At least one scene/noise or recording-condition family in a source-golf round must use real auxiliary `.aqua` `voice` declarations when claiming helper/background scene modeling. General synth primitives are allowed for helper scene voices, but they cannot count as articulation without gesture, primitive timeline, and clean-vocal movement. Crude full-store dumps are audit fallback, not agent memory. Statistical science workers and audio parity stages consume those bundles without rewriting gesture evidence. If gesture scores rise but rendered audio is silent, fix the primitive source/tract owner before adding full-patch dressing."
|
||||
},
|
||||
"ipa_vocal_tract_path": {
|
||||
"status": "This is now the active project center. Roadmap and research notes align the curriculum around structured utterance metadata -> learned utterance embedding -> learned synth automation -> vocal tract/envelope/LFO/filter/spectral controls -> rendered audio loss. The first Core contracts still defend the boundary: PhoneticIntent is the Weksa/language-owned handoff, ArticulatoryPlan is the AquaSynth realization layer, VocalTractMorphology declares anatomy capabilities, ArticulatoryConstraintReport is the host-visible veto surface, and VocalTractPlanResult is the tiny end-to-end proof artifact for host branching. IPA is an authoring input only: tokenization and feature mapping lower into language phonology profiles, timed articulatory gestures, morphology-specific tract/excitation curves, and Faust DSP. The staged support plan starts with tokenization and vowels, then core pulmonic consonants, expanded human IPA, airstream/phonation, and Weksa/alien morphologies. Optional eSpeak NG tests provide the first generated speech workout when ESPEAK_NG or espeak-ng is installed; eSpeak is intelligibility/timing/log-mel pressure, not anatomy truth. AquaSynth owns the learned utterance embedding encoder and synth driver: speech text embeddings plus prosody/emphasis hints plus Ghostlight/Epiphany-shaped character state vectors compress into utterance embeddings, and phonetic features plus that embedding feed a packed-buffer native C# backprop/Adam mapper that predicts tract, AM/FM, LFO, filter, and mel spectral-envelope controls. SpeechBackpropagationPipeline chains the two models: synth-driver target loss returns gradients with respect to its input, slices the utterance-embedding gradient, and updates the utterance encoder from downstream speech loss. The generated ground-truth training fixture renders eSpeak syllables, extracts log-mel evidence, and trains that chained pipeline toward eSpeak-seeded mel/control targets. CultMesh speech training now has payload manifests, peer/lease admission receipts, worker assignment, result collection, and central checkpoint application; workers do not own model updates. Real remote artifact chunk transfer and process supervision remain CultMesh infrastructure surfaces, not AquaSynth workaround territory. IpaGestureExperiment writes frozen descriptor/variant rounds with candidate scripts, primitive timelines, gesture metrics, manifests, and evidence JSONL, then analyzes them into metric summaries, score-band clusters, and a science brief. IpaTrialOrchestrator now writes local five-seed IPA render trials, accepts externally authored candidate scripts through RunCandidateScriptsAsync, writes WAV/report artifacts, evaluator reports, extracted primitive timeline facts, and `ipa-trial-results.cc` CultCache records. `tools/IpaTrialWorker` is the public worker tool for seed/score/search/show/index over the store; search now auto-upserts Qdrant/Ollama evidence chunks including timeline-fact text and returns vector-backed reports, while `tools/run-ipa-trial-loop.ps1` dispatches external Codex hypothesis/evaluator workers around five five-phoneme target sets. The latest seed run improved voiced vowels/nasals after primitive `SourcePort` lowering gained an oscillatory carrier; plosive closure/release remains the next weak owner.",
|
||||
|
|
@ -122,8 +122,8 @@
|
|||
},
|
||||
"last_verified": {
|
||||
"command": "powershell -NoProfile -ExecutionPolicy Bypass -File tools\\run-ipa-trial-loop.ps1 -Rounds 0 -SkipLocalTrialRun; dotnet build tests\\AquaSynth.Dsl.Tests\\AquaSynth.Dsl.Tests.csproj --no-restore --disable-build-servers -p:UseSharedCompilation=false -p:BuildInParallel=false -v minimal",
|
||||
"result": "IPA trial loop prompts now allow source-edit-enabled DSL/Faust/worker golf with a source-change plan and pre-score build, treat `hypotheses.md` as the report of record, and require at least one real auxiliary `.aqua` helper voice when a scene/noise family claims background or recording-condition modeling.",
|
||||
"commit": "pending ipa source golf scene voice contract"
|
||||
"result": "IPA trial loop prompts now require previous-round learning, target-class lane plans, phonetic class-aligned family assignment, source-edit owner maps, `hypotheses.md` as the report of record, and at least one real auxiliary `.aqua` helper voice when scene/noise families claim background or recording-condition modeling.",
|
||||
"commit": "pending ipa learning-loop correction"
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -2,6 +2,18 @@
|
|||
|
||||
Current slice:
|
||||
|
||||
- User correctly called out that the loose agents learned nothing. The actual
|
||||
failure was not just weak metrics; the candidate assignment was incoherent.
|
||||
Families were rotated across targets instead of aligned to phonetic owners:
|
||||
stop-release candidates landed on vowel/approximant lanes, while `p` got an
|
||||
oral-pressure tweak despite the evidence naming stop-release ownership as
|
||||
the weak seam. `tools/run-ipa-trial-loop.ps1` now passes the previous
|
||||
evaluation path into the hypothesizer, requires `Previous Round Learning`,
|
||||
requires a `Target-Class Lane Plan`, forbids round-robin family assignment,
|
||||
and instructs the evaluator to fail class alignment when family/target
|
||||
assignments do not match phonetic owner classes or explicit transfer-test
|
||||
mutations.
|
||||
|
||||
- Loose one-round IPA source-golf run completed under
|
||||
`artifacts/parity/ipa-trial-loops/20260531T001831690` with 25 candidates,
|
||||
source edits enabled, and focused pre-score build passing. The external agent
|
||||
|
|
|
|||
|
|
@ -146,7 +146,7 @@ live_invariants:
|
|||
- IPA training scoring contract is documented. Reports must keep `gesture_score`, `clean_vocal_score`, and `full_parity_score` separate. Gesture score measures descriptor-to-surface coverage, motion direction, spline contour/timing, primitive timeline consequences, and optional external articulation landmarks. Full parity patches may use FM/AM/modulators/envelopes/filters, auxiliary animated voices, and recording-condition helpers, but those helpers cannot count as articulation evidence unless gesture and clean-vocal scores move coherently.
|
||||
- `IpaGestureExperiment.WriteRound` is the first frozen batch harness for IPA hypothesis work. It writes `manifest.yaml`, candidate `.aqua` scripts, primitive `ProbeTimelineReport` CSVs, gesture-layer `metrics.csv`, and per-candidate `evidence.jsonl` under a caller-provided artifact root. `IpaGestureExperiment.AnalyzeRound` writes `analysis/metric-summary.csv`, `analysis/candidate-clusters.csv`, and `analysis/science-brief.md` for the science-worker handoff. This harness owns frozen descriptor/spline/primitive evidence only; clean vocal identity, full spectrogram parity, optimizer checkpoints, and worker orchestration stay downstream. Focused verification: `PatchScriptTests` pass 93/93.
|
||||
- `IpaTrialOrchestrator.RunAsync` is the first local render/scoring worker for IPA trials. It generates candidate patches through `IpaGestureExperiment`, renders them through Faust, compares against local PT fixture references with `AudioAnalyzer`, writes WAV/report artifacts, and stores typed `IpaTrialResult` records in `ipa-trial-results.cc`. The five-seed trial run at `tests/AquaSynth.Dsl.Tests/bin/Debug/net10.0/artifacts/parity/ipa-trials/20260529T214127955/five-seed-trials` showed voiced vowels/nasals became audible only after primitive `SourcePort` lowering gained a true oscillatory voiced carrier; sibilants are pressure, plosives remain weak. Verification: opt-in IPA trial test passed, and focused `PatchScriptTests|IpaTrialResultsRoundTripThroughCultCache` pass 94/94.
|
||||
- External IPA exploration now has a worker/tool boundary. `tools/IpaTrialWorker` owns `seed`, `score`, `search`, `show`, and audit-only `dump` commands over the typed `.cc` trial-result store; `score` renders external Codex-authored `.aqua` candidates through `IpaTrialOrchestrator.RunCandidateScriptsAsync`, while `search` is the semantic retrieval organ for hypothesis agents. `IpaTrialResult` records now embed extracted primitive timeline facts from `ProbeTimelineReport` for contact closure/reservoir/release, source load/flow, branch admittance/exchanged flow, radiation boundary/output, path area/delay/energy, and passivity; `search`, `show`, `dump`, and summary CSV output surface release/radiation/passivity witnesses directly from the `.cc` record. `search` follows VoidBot's Qdrant shape: `.cc` records plus timeline facts and referenced local artifacts become evidence chunks, local Ollama `qwen3-embedding:0.6b` produces 1024-dim vectors, Qdrant collection `aquasynth_ipa_trial_results` stores stable SHA-256 UUID points with trial/candidate payload metadata, and the report surfaces vector scores, excerpts, tags, metric facts, timeline facts, contrast candidates, `class_focus`, per-hit `class_match`, and `show` commands. Qdrant is a derived index; CultCache remains truth, and lexical/metric search is fallback when the vector stack is unavailable. Retrieval tuning added phonetic class-affinity scoring and candidate-specific mixed-lane ownership so `mix-p` can support stop seams without every mixed candidate matching stop evidence. `tools/run-ipa-trial-loop.ps1` orchestrates external `codex exec` hypothesis/evaluator workers around that store. Source-edit-enabled rounds now allow agents to golf DSL syntax, parser/model records, Faust lowering, worker evidence surfaces, and tests when evidence points at a repeated owner failure; agents must write `source-change-plan.md`, and the loop runs a focused build before scoring. `hypotheses.md` is the hypothesis worker report of record; `hypothesis-agent.md` is only the external agent's final chat summary. Candidate reports must include a `Scene Voice Model` naming primary-articulatory, tract-noise, release/transient, background-room, and recording-condition voices, with general AquaSynth synth primitives allowed for helper scene voices but not accepted as articulation evidence unless lower-layer witnesses move. At least one scene/noise or recording-condition family in a source-golf round must implement real auxiliary `.aqua` `voice` declarations instead of satisfying scene modeling with prose. The active trial shape is five target sets of five phoneme lanes each: vowels, nasals/approximants, fricatives, stops, and mixed generalization. Candidate names are exactly `<targetId>__<family>__<hypothesis-name>.aqua`, with lowercase kebab-case family ids. Three dry external Codex smoke passes refined hypothesizer/evaluator prompts into exact 25-lane output, retrieval receipts, claim audits, evidence-quality ledgers/verdicts, and fail-closed evaluator compliance. A later five-pass prompt optimization showed the best science comes from fixed receipt packets plus constrained reasoning; production prompts now require contrast pairs, `show` metric values, primitive timeline facts/excerpts or explicit `missing`, class-consistent reference matrices, scene voice models, `X owns Y so Z remains true` owner sentences, at least five micro-sweep perturbations before a seam is actionable, and a novelty gate rejecting loudness/dressing-only families as loss-landscape novelty. Verification: focused builds/tests pass, old-store show compatibility passes, and a fresh one-candidate smoke under `artifacts/parity/ipa-timeline-fact-smoke/20260531T233217641` proves release/radiation/passivity facts persist and retrieve through Qdrant/Ollama.
|
||||
- External IPA exploration now has a worker/tool boundary. `tools/IpaTrialWorker` owns `seed`, `score`, `search`, `show`, and audit-only `dump` commands over the typed `.cc` trial-result store; `score` renders external Codex-authored `.aqua` candidates through `IpaTrialOrchestrator.RunCandidateScriptsAsync`, while `search` is the semantic retrieval organ for hypothesis agents. `IpaTrialResult` records now embed extracted primitive timeline facts from `ProbeTimelineReport` for contact closure/reservoir/release, source load/flow, branch admittance/exchanged flow, radiation boundary/output, path area/delay/energy, and passivity; `search`, `show`, `dump`, and summary CSV output surface release/radiation/passivity witnesses directly from the `.cc` record. `search` follows VoidBot's Qdrant shape: `.cc` records plus timeline facts and referenced local artifacts become evidence chunks, local Ollama `qwen3-embedding:0.6b` produces 1024-dim vectors, Qdrant collection `aquasynth_ipa_trial_results` stores stable SHA-256 UUID points with trial/candidate payload metadata, and the report surfaces vector scores, excerpts, tags, metric facts, timeline facts, contrast candidates, `class_focus`, per-hit `class_match`, and `show` commands. Qdrant is a derived index; CultCache remains truth, and lexical/metric search is fallback when the vector stack is unavailable. Retrieval tuning added phonetic class-affinity scoring and candidate-specific mixed-lane ownership so `mix-p` can support stop seams without every mixed candidate matching stop evidence. `tools/run-ipa-trial-loop.ps1` orchestrates external `codex exec` hypothesis/evaluator workers around that store. Source-edit-enabled rounds now allow agents to golf DSL syntax, parser/model records, Faust lowering, worker evidence surfaces, and tests when evidence points at a repeated owner failure; agents must write `source-change-plan.md`, and the loop runs a focused build before scoring. `hypotheses.md` is the hypothesis worker report of record; `hypothesis-agent.md` is only the external agent's final chat summary. The first loose source-edit round produced no source edits and no aggregate metric win, then exposed a deeper learning-loop fault: family ideas were rotated across unrelated targets instead of aligned to phonetic owners. Production prompts now pass the previous evaluation path into the hypothesizer, require `Previous Round Learning`, require `Target-Class Lane Plan`, forbid round-robin family assignment, and let the evaluator fail class alignment when family/target assignments do not match phonetic owner classes or explicit transfer-test mutations. Candidate reports must include a `Scene Voice Model` naming primary-articulatory, tract-noise, release/transient, background-room, and recording-condition voices, with general AquaSynth synth primitives allowed for helper scene voices but not accepted as articulation evidence unless lower-layer witnesses move. At least one scene/noise or recording-condition family in a source-golf round must implement real auxiliary `.aqua` `voice` declarations instead of satisfying scene modeling with prose. The active trial shape is five target sets of five phoneme lanes each: vowels, nasals/approximants, fricatives, stops, and mixed generalization. Candidate names are exactly `<targetId>__<family>__<hypothesis-name>.aqua`, with lowercase kebab-case family ids. Three dry external Codex smoke passes refined hypothesizer/evaluator prompts into exact 25-lane output, retrieval receipts, claim audits, evidence-quality ledgers/verdicts, and fail-closed evaluator compliance. A later five-pass prompt optimization showed the best science comes from fixed receipt packets plus constrained reasoning; production prompts now require contrast pairs, `show` metric values, primitive timeline facts/excerpts or explicit `missing`, class-consistent reference matrices, scene voice models, `X owns Y so Z remains true` owner sentences, at least five micro-sweep perturbations before a seam is actionable, and a novelty gate rejecting loudness/dressing-only families as loss-landscape novelty. Verification: focused builds/tests pass, old-store show compatibility passes, and a fresh one-candidate smoke under `artifacts/parity/ipa-timeline-fact-smoke/20260531T233217641` proves release/radiation/passivity facts persist and retrieve through Qdrant/Ollama.
|
||||
- TurbulenceJet now includes a release-timed burst-noise component in addition to the deterministic pressure pulse, but a stronger trial barely moved papa; the source is not the main remaining articulation bottleneck. Graph radiation now mixes 30% raw boundary flow with 70% high-passed flow after local admittance weighting, down from the over-bright 15/85 blend. Latest radiation/source artifact under `artifacts/parity/pink-trombone-utterance-logmel/20260527T174236049`: mama/papa/thrombosis cosine 0.6578/0.7692/0.5585, rmsRatio 3.0006/2.6153/2.3488, articulation 0.1437/0.1173/0.1191. Static `closure-release` under `artifacts/parity/pink-trombone-logmel/20260527T174139661` has rmsRatio 0.9900 and speechBandRatio 0.0842. This is still not speaking; next pressure is true tract resonance/passive scattering and pressure storage, not more burst gain.
|
||||
- Same-path two-port area discontinuities now emit `graph_area_energy_in_*` and `graph_area_energy_out_*` probes alongside the three-port branch `graph_connection_energy_*` probes. Use these to inspect passivity and formant-body failures at the scatter sites rather than judging only final output loudness.
|
||||
- `FaustExportOptions.DebugProbeUi` wraps graph probe signals in Faust `vbargraph` UI controls under `/debug/...` without changing audio output arity. Native streaming Faust patches now separate writable controls from read-only probe zones and can read/snapshot probe values after `ProcessBlock`; use this for acoustic graph passivity/radiation inspection, leave it off for normal parity renders.
|
||||
|
|
|
|||
|
|
@ -271,6 +271,8 @@ for ($round = 1; $round -le $Rounds; $round++) {
|
|||
|
||||
$hypothesisOutput = Join-Path $roundDir "hypothesis-agent.md"
|
||||
$hypothesisLog = Join-Path $logRoot "$roundId-hypothesis-agent.log"
|
||||
$previousRoundId = "round-{0:000}" -f ($round - 1)
|
||||
$previousEvaluation = if ($round -gt 1) { Join-Path (Join-Path $loopRoot $previousRoundId) "evaluation.md" } else { "none" }
|
||||
$hypothesisPrompt = @"
|
||||
You are an AquaSynth IPA loss-landscape hypothesis and patch-writing worker.
|
||||
|
||||
|
|
@ -280,6 +282,9 @@ $preEvidence
|
|||
The backing store is:
|
||||
$storePath
|
||||
|
||||
Previous round evaluation:
|
||||
$previousEvaluation
|
||||
|
||||
Use the worker as your search organ. Do not ask for crude full dumps unless retrieval fails:
|
||||
`$env:DOTNET_CLI_HOME="$roundDir/dotnet-home"; `$env:DOTNET_SKIP_FIRST_TIME_EXPERIENCE="1"; `$env:DOTNET_CLI_TELEMETRY_OPTOUT="1"; dotnet run --no-build --no-restore --project tools/IpaTrialWorker/IpaTrialWorker.csproj -- search --store "$storePath" --query "<your evidence question>" --limit 20 --output "$roundDir/search-<topic>.md" --skip-index true
|
||||
`$env:DOTNET_CLI_HOME="$roundDir/dotnet-home"; `$env:DOTNET_SKIP_FIRST_TIME_EXPERIENCE="1"; `$env:DOTNET_CLI_TELEMETRY_OPTOUT="1"; dotnet run --no-build --no-restore --project tools/IpaTrialWorker/IpaTrialWorker.csproj -- show --store "$storePath" --trial-id "<trial id from search>" --output "$roundDir/detail-<candidate>.md"
|
||||
|
|
@ -315,6 +320,7 @@ Before writing candidates:
|
|||
5. Do not run additional retrieval unless one of the six required receipts is empty or unrelated; spend the rest of the turn reasoning over the evidence.
|
||||
6. For every seam you name, cite at least one contrast pair: a weak record and a stronger related record, or two variants that isolate one axis. If the receipts do not contain a contrast pair, mark the seam `not actionable`.
|
||||
7. Extract primitive timeline facts when available: closure duration, burst/release window, voicing onset, constriction window, branch admittance, radiation flow, or passivity/energy. If timelines are absent, mark timeline evidence `missing` instead of implying it.
|
||||
8. Read the previous round evaluation when it exists. A failed, flat, or regressed family is falsified until you state exactly what changed. Do not reuse family names or rotate old family ideas across new targets unless the report includes a `Previous Round Learning` section with the prior metric movement, accepted lesson, rejected lesson, and mutation that makes the new test different.
|
||||
|
||||
Task:
|
||||
1. Read the current loss landscape across gesture, log-mel, RMS, articulation, and primitive timeline evidence.
|
||||
|
|
@ -359,6 +365,9 @@ Candidate design contract:
|
|||
- The family segment in filenames is the canonical hypothesis-family id used by the evaluator.
|
||||
- The same family name should appear in filenames across multiple target sets when you are testing generalization.
|
||||
- A family is transferable only if it appears in at least three target sets; otherwise label it exploratory in the report.
|
||||
- A family must be phonetic-class aligned. Stop-release families belong on stop and `mix-p` lanes; nasal-branch families belong on nasal/approximant and `mix-m` lanes; frication families belong on fricative and `mix-s` lanes; vowel-shape/source families belong on vowel and vowel-like mixed lanes. Cross-class use is allowed only when the family explicitly tests transfer and the candidate changes a class-appropriate control subset for that target.
|
||||
- Do not distribute families round-robin across target ids. A candidate matrix that assigns stop-release to vowels, nasal-branch to stops, or scene-noise to arbitrary phonemes without a class-specific control plan is failed science even when the file count is correct.
|
||||
- Each family must include a target-class lane plan: which target classes it applies to, why those classes are lawful for the owner, and which class-specific controls change. If a target is outside the family class, the matrix must say `transfer-test` and name the class-specific mutation.
|
||||
- Keep `.aqua` scripts parseable and self-contained. Prefer modifying values and gesture contours already visible in seed candidates before inventing new syntax.
|
||||
- Do not use source edits as a way to rescue malformed candidate patches; use source edits to make a coherent syntax/lowering/Faust owner possible.
|
||||
- Only edit source if two or more retrieved trials indicate the same owner-layer failure pattern, or if at least two scene-voice candidates need the same missing DSL/lowering primitive; otherwise `Source Edit Decision: none`.
|
||||
|
|
@ -371,18 +380,20 @@ Candidate design contract:
|
|||
- Include a class-consistent reference matrix that maps each target id to the exact reference trial or fixture id used for comparison; mixed-set evidence cannot silently stand in for single-phoneme evidence.
|
||||
|
||||
Required `$roundDir/hypotheses.md` shape:
|
||||
- `Previous Round Learning`: if a previous evaluation exists, summarize which family claims were falsified/flat/improved, what lesson survives, and exactly how this round mutates the next test. If no previous round exists, write `first round`.
|
||||
- `PreEvidence Digest`: five facts with exact metric keys and values from $preEvidence.
|
||||
- `Retrieval Receipts`: search/show files used.
|
||||
- `Loss Landscape Read`: strongest and weakest evidence by family.
|
||||
- `Reference Matrix`: one row per target id with reference trial/fixture id, candidate baseline if known, and whether the evidence is class-consistent or mixed/weak.
|
||||
- `Hypothesis Families`: 3-5 families, each with owner layer, transfer status, evidence_refs, cited trial_ids, at least one concrete metric key/value from `show`, a contrast pair, an owner sentence of the form `X owns Y so Z remains true`, five planned micro-sweep perturbations, and predicted metric movement using `gesture:+/-/flat`, `logmel:+/-/flat`, `articulation:+/-/flat`, `rms:+/-/flat`, `timeline:+/-/flat/risk`.
|
||||
- `Target-Class Lane Plan`: one row per family naming allowed target classes, forbidden target classes, lawful transfer lanes, and class-specific controls that change.
|
||||
- `Scene Voice Model`: one row per family with primary voice, helper/background voices, AquaSynth primitives used, controls exposed, expected non-articulation witnesses, and dressing risk.
|
||||
- `Claim Audit`: one row per family: claim -> evidence file(s) -> contrast pair -> metric key/value(s) -> primitive timeline excerpt or `missing` -> predicted movement.
|
||||
- `Evidence Quality Ledger`: one row per family with specificity, comparability, falsifiability, reuse value for the next round, and the weakest missing evidence. Use `high`, `medium`, or `low` for each quality field.
|
||||
- `Novelty Gate`: one row per family saying which new owner/control axis/timing contour/primitive relationship is being explored; write `dressing-only` if the family mainly changes loudness/FM/AM/noise/envelopes.
|
||||
- `Candidate Matrix`: exactly 25 rows with target id, filename, family, expected metric movement, and risk.
|
||||
- `Source Edit Decision`: `none` unless source edits are allowed and justified by evidence; if source was edited, link `$roundDir/source-change-plan.md`, list files changed, tests/builds run or pending, and rollback trigger.
|
||||
- `Acceptance Checklist`: 25 files, one per target id, filename schema, required report sections, >=3 search receipts, >=3 show receipts, and at least one family with actual auxiliary `voice` helper/background declarations in its candidate scripts.
|
||||
- `Acceptance Checklist`: 25 files, one per target id, filename schema, required report sections, >=3 search receipts, >=3 show receipts, target-class lane plan present, no round-robin family assignment, and at least one family with actual auxiliary `voice` helper/background declarations in its candidate scripts.
|
||||
|
||||
Authority boundary:
|
||||
- Source edits allowed: $AllowCodexSourceEdits
|
||||
|
|
@ -514,11 +525,14 @@ Verify compliance from disk as well as the hypothesis report:
|
|||
- derive canonical family ids only from the filename family segment; ignore prose family names when they disagree.
|
||||
- validate that each actionable seam has a contrast pair, a primitive timeline excerpt or explicit `missing`, an owner sentence, and at least five planned micro-sweep perturbations.
|
||||
- validate that `$roundDir/hypotheses.md` includes a class-consistent reference matrix, scene voice model, and novelty gate. `hypothesis-agent.md` is only the external agent's final chat summary and must not be treated as the full report.
|
||||
- validate that `$roundDir/hypotheses.md` includes `Previous Round Learning` and `Target-Class Lane Plan`.
|
||||
- validate target-class alignment from filenames and family plan: stop-release families should not be assigned to vowel lanes, nasal-branch families should not be assigned to stop lanes, frication families should not be assigned to vowel lanes, and scene-noise families should not be arbitrary catchalls unless each off-class lane is marked `transfer-test` with a class-specific mutation.
|
||||
- fail `class alignment` when families are distributed round-robin across target ids or when candidate files reuse the same family script shape without target-class-specific controls.
|
||||
- validate that every family identifies primary-articulatory, tract-noise, release/transient, background-room, and recording-condition roles, even if some are explicitly silent.
|
||||
- validate that at least one family actually adds ordinary helper/background `voice` declarations in its `.aqua` scripts for scene noise or recording-condition modeling.
|
||||
- validate that helper/background voices are evaluated separately from articulation evidence.
|
||||
- validate source edit compliance if source edits were allowed: coherent owner map, edited file list, invariant, expected metric movement, verification command, and rollback trigger.
|
||||
- If any required section, receipt class, per-target validation, filename validation, family extraction, reference matrix, scene voice model, contrast pair, owner sentence, or novelty gate is incomplete, set `Round Compliance: failed`.
|
||||
- If any required section, receipt class, per-target validation, filename validation, family extraction, reference matrix, previous-round learning, target-class lane plan, class alignment, scene voice model, contrast pair, owner sentence, or novelty gate is incomplete, set `Round Compliance: failed`.
|
||||
|
||||
Task:
|
||||
1. Evaluate whether the worker's proposed hypotheses follow from the measured evidence.
|
||||
|
|
@ -530,7 +544,7 @@ $roundDir/evaluation.md
|
|||
Required `$roundDir/evaluation.md` shape:
|
||||
- `Retrieval Receipts`: exactly five receipts: three search outputs plus one weak show output and one promising show output. Cite no other evidence files in this section. Only cite claims backed by these file paths; write `unknown` when evidence is absent.
|
||||
- `Commands Run`: exact search/show commands and output paths used to support claims.
|
||||
- `Round Compliance`: passed/failed, candidate count, target coverage, naming validity, family extraction, reference matrix, contrast-pair coverage, owner-sentence coverage, micro-sweep coverage, novelty gate, auxiliary scene voice presence, and whether `hypotheses.md` had the required matrix.
|
||||
- `Round Compliance`: passed/failed, candidate count, target coverage, naming validity, family extraction, reference matrix, previous-round learning, target-class lane plan, class alignment, contrast-pair coverage, owner-sentence coverage, micro-sweep coverage, novelty gate, auxiliary scene voice presence, and whether `hypotheses.md` had the required matrix.
|
||||
- `Family Verdicts`: one row per canonical filename family with improved/flat/regressed/unknown metrics. Treat |delta| < 0.01 as flat; if deltas are not present in cited show artifacts, write `unknown` and reason=`no delta in receipts`.
|
||||
- `Evidence Quality Verdicts`: one row per family with specificity, comparability, falsifiability, reuse value, primitive timeline support, and weakest missing evidence. Penalize vague receipts, search-summary-only claims, missing contrast pairs, missing reference ids, and missing timeline excerpts even when metrics improve.
|
||||
- `Seam Audit`: one row per named seam with owner sentence, contrast pair, timeline excerpt, planned perturbation count, and verdict `actionable` or `not actionable`; fewer than five perturbations means `not actionable`.
|
||||
|
|
@ -538,6 +552,7 @@ Required `$roundDir/evaluation.md` shape:
|
|||
- `Source Edit Audit`: if source edits were allowed, summarize source files changed, invariant claimed, verification result, and whether the edit should survive; otherwise `not allowed`.
|
||||
- `Novelty Audit`: one row per family saying whether it moves a new owner/control axis/timing contour/primitive relationship/scene-voice model or is mainly loudness/dressing.
|
||||
- `Generalization Read`: whether effects transferred across target sets or overfit one phoneme.
|
||||
- `Learning Audit`: whether failed/flat previous families were mutated, retired, or blindly repeated.
|
||||
- `Dressing Audit`: articulation evidence requires movement in gesture/articulation/primitive timeline metrics; FM/AM/noise/envelope evidence without those is dressing.
|
||||
- `Next Target`: exactly one markdown bullet, final line of the file, and no continuation text: `<file-or-module> | <invariant> | <expected metric movement>`.
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue