Allow IPA agents to golf source and scene voices
This commit is contained in:
parent
f16ea641af
commit
b1e9f056cf
7 changed files with 149 additions and 20 deletions
|
|
@ -150,6 +150,31 @@ function Assert-CandidateBatch {
|
|||
}
|
||||
}
|
||||
|
||||
function Invoke-SourceEditVerification {
|
||||
param(
|
||||
[string]$LogPath,
|
||||
[bool]$AllowSourceEdits
|
||||
)
|
||||
|
||||
if (-not $AllowSourceEdits) {
|
||||
return
|
||||
}
|
||||
|
||||
Invoke-LoggedProcess `
|
||||
-FilePath "dotnet" `
|
||||
-ArgumentList @(
|
||||
"build",
|
||||
"tests\AquaSynth.Dsl.Tests\AquaSynth.Dsl.Tests.csproj",
|
||||
"--no-restore",
|
||||
"--disable-build-servers",
|
||||
"-p:UseSharedCompilation=false",
|
||||
"-p:BuildInParallel=false",
|
||||
"-v",
|
||||
"minimal"
|
||||
) `
|
||||
-LogPath $LogPath
|
||||
}
|
||||
|
||||
$repoRoot = Resolve-RepoRoot
|
||||
$codex = Get-Command $CodexCommand -ErrorAction Stop
|
||||
$loopId = New-Timestamp
|
||||
|
|
@ -234,7 +259,7 @@ for ($round = 1; $round -le $Rounds; $round++) {
|
|||
"--store",
|
||||
$storePath,
|
||||
"--query",
|
||||
"weak promising generalization vowel nasal fricative stop articulation owner source tract radiation gesture primitive dressing",
|
||||
"weak promising generalization vowel nasal fricative stop articulation owner source tract radiation gesture primitive dressing DSL lowering Faust machinery scene voices background noise",
|
||||
"--limit",
|
||||
"40",
|
||||
"--output",
|
||||
|
|
@ -267,9 +292,15 @@ Retrieval failure policy:
|
|||
|
||||
Useful repo surfaces:
|
||||
- docs/ipa-vocal-tract-roadmap.md
|
||||
- docs/audio-doctrine.md
|
||||
- src/AquaSynth.Core/IpaGestureExperiment.cs
|
||||
- src/AquaSynth.Core/Model.cs
|
||||
- src/AquaSynth.Core/PatchScript.cs
|
||||
- src/AquaSynth.Core/ControlSplineTimeline.cs
|
||||
- src/AquaSynth.Core/ProbeTimelineReport.cs
|
||||
- src/AquaSynth.Core/FaustExport.cs
|
||||
- src/AquaSynth.Faust/IpaTrialOrchestrator.cs
|
||||
- tools/IpaTrialWorker/Program.cs
|
||||
- artifacts under $loopRoot
|
||||
|
||||
Before writing candidates:
|
||||
|
|
@ -277,7 +308,7 @@ Before writing candidates:
|
|||
1. Run exactly three targeted semantic searches and save the outputs under ${roundDir}:
|
||||
- one for weakest stop/plosive closure evidence;
|
||||
- one for best vowel/nasal/fricative transferable successes;
|
||||
- one for dressing/FM/AM/noise versus articulation-owner failures.
|
||||
- one for DSL lowering/Faust machinery, scene voices, background-noise modeling, and dressing/FM/AM/noise versus articulation-owner failures.
|
||||
2. Open exactly three detailed trial records with `show`: one weak stop/plosive, one transferable success, and one dressing-vs-articulation case when available.
|
||||
3. In $roundDir/hypotheses.md, include a `Retrieval Receipts` section listing the exact search/show files you used.
|
||||
4. If fewer than three search receipts and three show receipts exist on disk, stop and create the missing retrieval artifacts before writing `.aqua` files.
|
||||
|
|
@ -301,22 +332,40 @@ $patchDir
|
|||
6. Write a concise hypothesis report to:
|
||||
$roundDir/hypotheses.md
|
||||
|
||||
Source-code golf contract:
|
||||
- Source edits allowed: $AllowCodexSourceEdits
|
||||
- When source edits are allowed, you may edit `src/`, `tests/`, `docs/`, and `tools/` to test DSL syntax, DSL lowering, Faust emission, worker scoring, or scene-voice evidence surfaces.
|
||||
- Do not edit generated artifacts, existing `.cc` stores, existing trial results, or previous round reports.
|
||||
- Source edits must move one named owner, not add compensators. Use the sentence `X owns Y so Z remains true`.
|
||||
- Valid source hypotheses include: new patch DSL syntax, parser/model records, Faust lowering changes, ProbeTimeline fields, IpaTrialWorker evidence/reporting, or tests that prove the new syntax/lowering path.
|
||||
- Invalid source hypotheses include: broad rewrites, changing metric thresholds to flatter candidates, deleting hard evidence, mutating references, or hardcoding a candidate name.
|
||||
- If you edit source, write `$roundDir/source-change-plan.md` with: owner map, files edited, invariant, expected metric movement, tests/build commands to run, and rollback trigger.
|
||||
- If you do not edit source, write `Source Edit Decision: none` and explain which evidence is still missing.
|
||||
|
||||
Scene voice model:
|
||||
- Every candidate patch represents a scene, not a single naked waveform. The report must name the scene voices and say how each is synthesized.
|
||||
- Required scene voice roles: `primary-articulatory`, `tract-noise`, `release/transient`, `background-room`, and `recording-condition`.
|
||||
- Use ordinary AquaSynth primitives for helper voices where useful: `voice` layers, noise, FM, AM/tremolo, envelopes, filters, formants, additive/spectral voices, macros, and modulation. These helper voices are allowed in full parity.
|
||||
- Helper/background voices must not count as articulation evidence unless gesture, primitive timeline, and clean vocal metrics move coherently.
|
||||
- For each family, say which voices are active, which controls they expose, and which metric/timeline witness would prove the helper is not merely dressing.
|
||||
|
||||
Candidate design contract:
|
||||
- Produce exactly 25 files: exactly one candidate for each target lane. No extras.
|
||||
- Use 3-5 named hypothesis families across the 25 files, for example source-carrier, constriction-place, stop-release, nasal-branch, and dressing-control.
|
||||
- Use 3-5 named hypothesis families across the 25 files, for example source-carrier, constriction-place, stop-release, nasal-branch, scene-noise-bed, and lowering-release-owner.
|
||||
- Filename schema is mandatory: <targetId>__<family>__<hypothesis-name>.aqua.
|
||||
- Family ids must be lowercase kebab-case and reused verbatim in filenames and `Hypothesis Families`.
|
||||
- The family segment in filenames is the canonical hypothesis-family id used by the evaluator.
|
||||
- The same family name should appear in filenames across multiple target sets when you are testing generalization.
|
||||
- A family is transferable only if it appears in at least three target sets; otherwise label it exploratory in the report.
|
||||
- Keep `.aqua` scripts parseable and self-contained. Prefer modifying values and gesture contours already visible in seed candidates before inventing new syntax.
|
||||
- Do not use source edits as a way to rescue malformed candidate patches.
|
||||
- Only edit source if two or more retrieved trials indicate the same owner-layer failure pattern; otherwise `Source Edit Decision: none`.
|
||||
- Do not use source edits as a way to rescue malformed candidate patches; use source edits to make a coherent syntax/lowering/Faust owner possible.
|
||||
- Only edit source if two or more retrieved trials indicate the same owner-layer failure pattern, or if at least two scene-voice candidates need the same missing DSL/lowering primitive; otherwise `Source Edit Decision: none`.
|
||||
- If Source edits allowed is False, do not edit any file outside $patchDir and $roundDir/hypotheses.md.
|
||||
- Before finishing, list $patchDir and verify exact count, target coverage, and filename schema in `Acceptance Checklist`.
|
||||
- A seam claim must use an owner sentence: `X owns Y so Z remains true`.
|
||||
- A seam is actionable only if the next candidate family proposes at least five micro-sweep perturbations across that seam, spread across target lanes when possible.
|
||||
- Novelty gate: a family must move a new owner, control axis, timing contour, or primitive relationship. Raising loudness, burst gain, or dressing alone is not novel.
|
||||
- Scene novelty can be valid only when it changes a named voice role, routing/lowering primitive, modulation relationship, or background-noise synthesis model and predicts a non-articulation witness such as noise-floor, spectral tilt, RMS stability, or room/recording match.
|
||||
- Include a class-consistent reference matrix that maps each target id to the exact reference trial or fixture id used for comparison; mixed-set evidence cannot silently stand in for single-phoneme evidence.
|
||||
|
||||
Required `$roundDir/hypotheses.md` shape:
|
||||
|
|
@ -325,11 +374,12 @@ Required `$roundDir/hypotheses.md` shape:
|
|||
- `Loss Landscape Read`: strongest and weakest evidence by family.
|
||||
- `Reference Matrix`: one row per target id with reference trial/fixture id, candidate baseline if known, and whether the evidence is class-consistent or mixed/weak.
|
||||
- `Hypothesis Families`: 3-5 families, each with owner layer, transfer status, evidence_refs, cited trial_ids, at least one concrete metric key/value from `show`, a contrast pair, an owner sentence of the form `X owns Y so Z remains true`, five planned micro-sweep perturbations, and predicted metric movement using `gesture:+/-/flat`, `logmel:+/-/flat`, `articulation:+/-/flat`, `rms:+/-/flat`, `timeline:+/-/flat/risk`.
|
||||
- `Scene Voice Model`: one row per family with primary voice, helper/background voices, AquaSynth primitives used, controls exposed, expected non-articulation witnesses, and dressing risk.
|
||||
- `Claim Audit`: one row per family: claim -> evidence file(s) -> contrast pair -> metric key/value(s) -> primitive timeline excerpt or `missing` -> predicted movement.
|
||||
- `Evidence Quality Ledger`: one row per family with specificity, comparability, falsifiability, reuse value for the next round, and the weakest missing evidence. Use `high`, `medium`, or `low` for each quality field.
|
||||
- `Novelty Gate`: one row per family saying which new owner/control axis/timing contour/primitive relationship is being explored; write `dressing-only` if the family mainly changes loudness/FM/AM/noise/envelopes.
|
||||
- `Candidate Matrix`: exactly 25 rows with target id, filename, family, expected metric movement, and risk.
|
||||
- `Source Edit Decision`: `none` unless source edits are allowed and justified by evidence.
|
||||
- `Source Edit Decision`: `none` unless source edits are allowed and justified by evidence; if source was edited, link `$roundDir/source-change-plan.md`, list files changed, tests/builds run or pending, and rollback trigger.
|
||||
- `Acceptance Checklist`: 25 files, one per target id, filename schema, required report sections, >=3 search receipts, >=3 show receipts.
|
||||
|
||||
Authority boundary:
|
||||
|
|
@ -337,7 +387,7 @@ Authority boundary:
|
|||
- Do not rewrite existing trial artifacts or the CultCache store directly.
|
||||
- The worker process owns measurement and .cc writes; you own candidate invention and optional source refinements.
|
||||
- Keep gesture_score, clean vocal/audio evidence, and full parity dressing separate.
|
||||
- Patch candidates may use FM, AM, modulators, envelopes, and extra animated voices, but call out when dressing is compensating for failed articulation.
|
||||
- Patch candidates should make full use of AquaSynth general synth primitives when modeling the scene: FM, AM, modulators, envelopes, filters, formants, noise voices, additive/spectral layers, and extra animated voices. Call out when these are background/recording-condition voices rather than articulation.
|
||||
- Prefer fixing primitive owners over adding audio dressing when the evidence says air is missing.
|
||||
|
||||
Return a short final summary naming the files you wrote.
|
||||
|
|
@ -353,6 +403,11 @@ Return a short final summary naming the files you wrote.
|
|||
|
||||
Assert-CandidateBatch -PatchDirectory $patchDir
|
||||
|
||||
$sourceVerifyLog = Join-Path $logRoot "$roundId-source-edit-verification.log"
|
||||
Invoke-SourceEditVerification `
|
||||
-LogPath $sourceVerifyLog `
|
||||
-AllowSourceEdits:$AllowCodexSourceEdits
|
||||
|
||||
$scoreLog = Join-Path $logRoot "$roundId-score-candidates.log"
|
||||
Invoke-LoggedProcess `
|
||||
-FilePath "dotnet" `
|
||||
|
|
@ -404,7 +459,7 @@ Return a short final summary naming the files you wrote.
|
|||
"--store",
|
||||
$storePath,
|
||||
"--query",
|
||||
"latest round weak promising transfer generalization vowel nasal fricative stop articulation owner source tract radiation gesture primitive dressing",
|
||||
"latest round weak promising transfer generalization vowel nasal fricative stop articulation owner source tract radiation gesture primitive dressing DSL lowering Faust machinery scene voices background noise",
|
||||
"--limit",
|
||||
"60",
|
||||
"--output",
|
||||
|
|
@ -429,12 +484,13 @@ Round artifacts are under:
|
|||
$roundDir
|
||||
|
||||
Before judging:
|
||||
1. Run at least three targeted semantic searches into ${roundDir}:
|
||||
1. Run exactly three targeted semantic searches into ${roundDir}:
|
||||
- current round candidates by hypothesizer id or round id;
|
||||
- weak regressions / failed articulation;
|
||||
- promising transfer / generalization.
|
||||
- promising transfer / generalization, including DSL lowering/Faust/scene-voice source-edit evidence when source edits were allowed.
|
||||
2. Use `show` on at least one promising and one weak candidate from the current round.
|
||||
3. If the hypothesis worker skipped the required 25-lane matrix or retrieval receipts, mark orchestration/prompt compliance as failed even if some audio metrics improved.
|
||||
4. If source edits were allowed, inspect `$roundDir/source-change-plan.md` when present and verify the source edit is a coherent owner move, not a metric hack.
|
||||
|
||||
Use the worker as your search organ:
|
||||
`$env:DOTNET_CLI_HOME="$roundDir/dotnet-home"; `$env:DOTNET_SKIP_FIRST_TIME_EXPERIENCE="1"; `$env:DOTNET_CLI_TELEMETRY_OPTOUT="1"; dotnet run --no-build --no-restore --project tools/IpaTrialWorker/IpaTrialWorker.csproj -- search --store "$storePath" --query "$roundId external-codex-$roundId current round candidates" --limit 30 --output "$roundDir/eval-search-current-round.md" --skip-index true
|
||||
|
|
@ -455,13 +511,16 @@ Verify compliance from disk as well as the hypothesis report:
|
|||
- validate family ids are lowercase kebab-case.
|
||||
- derive canonical family ids only from the filename family segment; ignore prose family names when they disagree.
|
||||
- validate that each actionable seam has a contrast pair, a primitive timeline excerpt or explicit `missing`, an owner sentence, and at least five planned micro-sweep perturbations.
|
||||
- validate that the hypothesis report includes a class-consistent reference matrix and a novelty gate.
|
||||
- If any required section, receipt class, per-target validation, filename validation, family extraction, reference matrix, contrast pair, owner sentence, or novelty gate is incomplete, set `Round Compliance: failed`.
|
||||
- validate that the hypothesis report includes a class-consistent reference matrix, scene voice model, and novelty gate.
|
||||
- validate that every family identifies primary-articulatory, tract-noise, release/transient, background-room, and recording-condition roles, even if some are explicitly silent.
|
||||
- validate that helper/background voices are evaluated separately from articulation evidence.
|
||||
- validate source edit compliance if source edits were allowed: coherent owner map, edited file list, invariant, expected metric movement, verification command, and rollback trigger.
|
||||
- If any required section, receipt class, per-target validation, filename validation, family extraction, reference matrix, scene voice model, contrast pair, owner sentence, or novelty gate is incomplete, set `Round Compliance: failed`.
|
||||
|
||||
Task:
|
||||
1. Evaluate whether the worker's proposed hypotheses follow from the measured evidence.
|
||||
2. Call out any hypothesis that is likely full-patch dressing impersonating articulation.
|
||||
3. Recommend exactly one next implementation/refinement target, with file/module owner and why.
|
||||
3. Recommend exactly one next implementation/refinement target, with file/module owner and why. It may be DSL syntax, Faust lowering, scene-voice modeling, or worker evidence if the receipts support it.
|
||||
4. Write your evaluator report to:
|
||||
$roundDir/evaluation.md
|
||||
|
||||
|
|
@ -472,7 +531,9 @@ Required `$roundDir/evaluation.md` shape:
|
|||
- `Family Verdicts`: one row per canonical filename family with improved/flat/regressed/unknown metrics. Treat |delta| < 0.01 as flat; if deltas are not present in cited show artifacts, write `unknown` and reason=`no delta in receipts`.
|
||||
- `Evidence Quality Verdicts`: one row per family with specificity, comparability, falsifiability, reuse value, primitive timeline support, and weakest missing evidence. Penalize vague receipts, search-summary-only claims, missing contrast pairs, missing reference ids, and missing timeline excerpts even when metrics improve.
|
||||
- `Seam Audit`: one row per named seam with owner sentence, contrast pair, timeline excerpt, planned perturbation count, and verdict `actionable` or `not actionable`; fewer than five perturbations means `not actionable`.
|
||||
- `Novelty Audit`: one row per family saying whether it moves a new owner/control axis/timing contour/primitive relationship or is mainly loudness/dressing.
|
||||
- `Scene Voice Audit`: one row per family naming scene voices, helper/background synthesis primitives, non-articulation witness, and whether the helper voices are honest context or dressing.
|
||||
- `Source Edit Audit`: if source edits were allowed, summarize source files changed, invariant claimed, verification result, and whether the edit should survive; otherwise `not allowed`.
|
||||
- `Novelty Audit`: one row per family saying whether it moves a new owner/control axis/timing contour/primitive relationship/scene-voice model or is mainly loudness/dressing.
|
||||
- `Generalization Read`: whether effects transferred across target sets or overfit one phoneme.
|
||||
- `Dressing Audit`: articulation evidence requires movement in gesture/articulation/primitive timeline metrics; FM/AM/noise/envelope evidence without those is dressing.
|
||||
- `Next Target`: exactly one markdown bullet, final line of the file, and no continuation text: `<file-or-module> | <invariant> | <expected metric movement>`.
|
||||
|
|
@ -480,7 +541,7 @@ Required `$roundDir/evaluation.md` shape:
|
|||
Authority boundary:
|
||||
- Do not edit source files.
|
||||
- Do not rewrite trial artifacts.
|
||||
- Be explicit about which layer owns the failure: gesture DSL, primitive source, tract/radiation lowering, audio scoring, or orchestration.
|
||||
- Be explicit about which layer owns the failure: gesture DSL, scene voice model, primitive source, tract/radiation lowering, Faust machinery, audio scoring, or orchestration.
|
||||
- Treat the CultCache .cc store as the shared trial memory that future hypothesis workers will query through the worker search/show commands.
|
||||
|
||||
Return a short final summary naming the report you wrote.
|
||||
|
|
|
|||
Loading…
Reference in a new issue