Record song snippet phase evidence

This commit is contained in:
Meta Vens 2026-05-31 02:55:00 +01:00
commit 8f2372d9d9
5 changed files with 67 additions and 32 deletions

View file

@ -103,6 +103,13 @@ then bind the parameter into voice fields such as `freq=@/seq/pitch` or
`gain=@/seq/kick`. Native `scale`, `note`, and pattern sugar should be added
only when the DSL owns the lowering and the emitted Faust remains inspectable.
Early song-snippet experiments show useful pressure for named scene instruments:
`alien-lead-chatter`, `dust-hat-grid`, `codec-bed-hum`, and `air-wash-hiss`.
Those names are not DSP owners yet. They are proposed authoring roles that
should lower into explicit `param`, `curve`, and `voice` records. The invariant
for future sugar is: role syntax may own brevity and defaults, but the lowered
graph still owns pitch lanes, gates, filters, FM/AM, noise, and mix behavior.
## Faust Boundary
Faust describes pure DSP: inputs to outputs. Architecture files and host code

View file

@ -248,3 +248,4 @@
{"date":"2026-05-31","area":"ipa-trial-source-golf-smoke","lesson":"The first loose source-edit-enabled IPA round completed without tracked source edits and did not improve the aggregate metrics, which is valid negative evidence rather than a failure of authority. It exposed two harness contracts to tighten: evaluators must judge `hypotheses.md` as the report of record instead of the external agent's final `hypothesis-agent.md` chat summary, and scene/noise families must implement at least one actual helper/background `.aqua` `voice` declaration instead of satisfying scene modeling with prose.","evidence":"artifacts/parity/ipa-trial-loops/20260531T001831690; tools/run-ipa-trial-loop.ps1; docs/ipa-vocal-tract-roadmap.md; state/scratch.md"}
{"date":"2026-05-31","area":"ipa-trial-learning-loop","lesson":"The loose source-golf round obeyed the file-count ritual but did not learn: it rotated families across unrelated phonetic targets, assigned stop-release ideas to vowel/approximant lanes, gave `p` an oral-pressure tweak instead of the stop-release owner, and produced aggregate regressions without converting them into the next hypothesis. The trial loop now passes previous evaluation paths into the hypothesizer, requires `Previous Round Learning`, requires a `Target-Class Lane Plan`, forbids round-robin family assignment, and lets the evaluator fail class alignment when families do not match phonetic owner classes or explicit transfer-test mutations.","evidence":"artifacts/parity/ipa-trial-loops/20260531T001831690/round-001/hypotheses.md; artifacts/parity/ipa-trial-loops/20260531T001831690/round-001/evaluation.md; tools/run-ipa-trial-loop.ps1; docs/ipa-vocal-tract-roadmap.md"}
{"date":"2026-05-31","area":"song-snippet-trial-swarm","lesson":"Song-scene parity needs musical control evidence in addition to spectrogram loss. The new song challenge lane freezes a seeded local clip, derives tempo from onset-envelope autocorrelation, derives a register/root/scale hint from dominant-frequency autocorrelation plus spectral fallback, and prompts agents to sequence explicit AquaSynth `param` + `curve interp=hold loop=true` controls instead of inventing unimplemented Strudel syntax. Register-bounded scale frequencies are a temporary explicit contract until native scale/note sugar has a real DSL/lowering owner. Song scoring must not require vocal-network primitive timelines; non-vocal scene patches now score with an empty timeline witness instead of failing before render.","evidence":"tools/IpaTrialWorker/Program.cs; tools/run-song-snippet-trial-swarm.ps1; docs/audio-doctrine.md; artifacts/parity/song-snippet-smoke/challenge.md; artifacts/parity/song-snippet-swarms/20260531T012627197/challenge.json; artifacts/parity/song-snippet-score-smoke/smoke-001/evaluator-report.md"}
{"date":"2026-05-31","area":"song-snippet-experiment-phases","lesson":"Three song-snippet phases across different segments improved best log-mel cosine from `0.308233` to `0.380805`, still pressure rather than close parity. Winning candidates converged on bright FM/formant `alien-lead` plus clocked high-band `dust-hat` noise and a continuous `codec-bed`/air layer; syrinx-heavy proposals scored worse. The durable DSL pressure is named scene-instrument sugar (`alien-lead-chatter`, `dust-hat-grid`, `codec-bed-hum`, `air-wash-hiss`) that lowers into explicit `param`, `curve interp=hold loop=true`, and `voice` records. Role sugar may own brevity/defaults; lowered controls remain DSP authority.","evidence":"artifacts/parity/song-snippet-swarms/20260531T014033022/experiment-report.md; artifacts/parity/song-snippet-swarms/20260531T014033022/song-trial-results.cc; docs/audio-doctrine.md; tools/run-song-snippet-trial-swarm.ps1"}

View file

@ -93,7 +93,7 @@
"Surge XT for modern hybrid synth modulation",
"Cardinal/VCV-style patches for modular cable/rate semantics"
],
"rule": "For speech, add schema fields and learned-control lanes only when the parity harness has a loss surface that can exercise them. For reference synths, add DSL abstractions only when rebuilds prove the missing invariant. Parsing alone is inventory, not evidence of expressivity. Pink Trombone should be decomposed into reusable low-level Aqua primitives before audio golf, not copied as a bespoke clone or inflated into PT-shaped modules. After the 2026-05-29 proprioception correction, collapse graph-vocal lowering into a small set of instrumented physical primitives before more utterance golf. Gesture motion belongs in ControlSurface/ControlSpline timeline ownership, not primitive-local smoothing; cut any generated contact/port spline that invents motion not supported by primitive reference timelines. Public clients author gestures through ControlSurfaceCatalog plus ControlSplineTimeline and stream surface control dictionaries; higher-level gesture templates and phoneme_gesture are only spline emitters. IPA descriptors seed which anatomical surfaces move, while local datasets golf numeric spline defaults and patch voice code. Training reports must separate gesture_score, clean_vocal_score, and full_parity_score so full-patch recording-condition helpers cannot impersonate articulation. Frozen IPA gesture rounds own candidate scripts, primitive timelines, gesture metrics, manifests, evidence receipts, metric summaries, candidate clusters, and science briefs; `IpaTrialResult` CultCache records own rendered trial outcomes plus extracted primitive timeline facts for contact release, source flow, branch admittance, radiation output, path energy, and passivity. External Codex IPA exploration must query the `.cc` store through the worker's semantic `search`/`show` organ, backed by Qdrant/Ollama vector retrieval when available; Qdrant is a derived index over CultCache and local artifacts, not a second truth. Search reports expose class focus/match and use phonetic class-affinity plus timeline-fact text so agents do not treat generic weak-articulation language as cross-class evidence. Source-edit-enabled IPA rounds may golf DSL syntax, parser/model records, Faust lowering, worker evidence surfaces, and tests when repeated evidence identifies a missing owner; every source edit needs an owner map, invariant, expected metric movement, verification, and rollback trigger. `hypotheses.md` is the hypothesis worker report of record; `hypothesis-agent.md` is only the external agent's final chat summary. Each new round must learn from the previous evaluation: flat/regressed/failed families are falsified until the report states what changed, and family/target assignment must be phonetic-class aligned instead of round-robin. Stop-release families belong on stop and `mix-p` lanes, nasal-branch on nasal/approximant and `mix-m`, frication on fricative and `mix-s`, and vowel/source-shape on vowel-like lanes unless an explicit transfer-test mutation names class-specific controls. IPA parity candidates should model scene voices explicitly: primary articulation, tract noise, release/transient, background-room, and recording-condition. At least one scene/noise or recording-condition family in a source-golf round must use real auxiliary `.aqua` `voice` declarations when claiming helper/background scene modeling. Song-snippet parity is scene-audio pressure: derive tempo with onset autocorrelation, derive register/root/scale hints from the clip, and sequence explicit `param` plus `curve interp=hold loop=true` controls until native Strudel-like scale/note sugar has a real owner. General synth primitives are allowed for helper scene voices, but they cannot count as articulation without gesture, primitive timeline, and clean-vocal movement. Crude full-store dumps are audit fallback, not agent memory. Statistical science workers and audio parity stages consume those bundles without rewriting gesture evidence. If gesture scores rise but rendered audio is silent, fix the primitive source/tract owner before adding full-patch dressing."
"rule": "For speech, add schema fields and learned-control lanes only when the parity harness has a loss surface that can exercise them. For reference synths, add DSL abstractions only when rebuilds prove the missing invariant. Parsing alone is inventory, not evidence of expressivity. Pink Trombone should be decomposed into reusable low-level Aqua primitives before audio golf, not copied as a bespoke clone or inflated into PT-shaped modules. After the 2026-05-29 proprioception correction, collapse graph-vocal lowering into a small set of instrumented physical primitives before more utterance golf. Gesture motion belongs in ControlSurface/ControlSpline timeline ownership, not primitive-local smoothing; cut any generated contact/port spline that invents motion not supported by primitive reference timelines. Public clients author gestures through ControlSurfaceCatalog plus ControlSplineTimeline and stream surface control dictionaries; higher-level gesture templates and phoneme_gesture are only spline emitters. IPA descriptors seed which anatomical surfaces move, while local datasets golf numeric spline defaults and patch voice code. Training reports must separate gesture_score, clean_vocal_score, and full_parity_score so full-patch recording-condition helpers cannot impersonate articulation. Frozen IPA gesture rounds own candidate scripts, primitive timelines, gesture metrics, manifests, evidence receipts, metric summaries, candidate clusters, and science briefs; `IpaTrialResult` CultCache records own rendered trial outcomes plus extracted primitive timeline facts for contact release, source flow, branch admittance, radiation output, path energy, and passivity. External Codex IPA exploration must query the `.cc` store through the worker's semantic `search`/`show` organ, backed by Qdrant/Ollama vector retrieval when available; Qdrant is a derived index over CultCache and local artifacts, not a second truth. Search reports expose class focus/match and use phonetic class-affinity plus timeline-fact text so agents do not treat generic weak-articulation language as cross-class evidence. Source-edit-enabled IPA rounds may golf DSL syntax, parser/model records, Faust lowering, worker evidence surfaces, and tests when repeated evidence identifies a missing owner; every source edit needs an owner map, invariant, expected metric movement, verification, and rollback trigger. `hypotheses.md` is the hypothesis worker report of record; `hypothesis-agent.md` is only the external agent's final chat summary. Each new round must learn from the previous evaluation: flat/regressed/failed families are falsified until the report states what changed, and family/target assignment must be phonetic-class aligned instead of round-robin. Stop-release families belong on stop and `mix-p` lanes, nasal-branch on nasal/approximant and `mix-m`, frication on fricative and `mix-s`, and vowel/source-shape on vowel-like lanes unless an explicit transfer-test mutation names class-specific controls. IPA parity candidates should model scene voices explicitly: primary articulation, tract noise, release/transient, background-room, and recording-condition. At least one scene/noise or recording-condition family in a source-golf round must use real auxiliary `.aqua` `voice` declarations when claiming helper/background scene modeling. Song-snippet parity is scene-audio pressure: derive tempo with onset autocorrelation, derive register/root/scale hints from the clip, and sequence explicit `param` plus `curve interp=hold loop=true` controls until native Strudel-like scale/note sugar has a real owner. Three song phases showed reusable named roles (`alien-lead-chatter`, `dust-hat-grid`, `codec-bed-hum`, `air-wash-hiss`) can guide agents; role sugar may own brevity/defaults but must lower into explicit params, curves, and voices. General synth primitives are allowed for helper scene voices, but they cannot count as articulation without gesture, primitive timeline, and clean-vocal movement. Crude full-store dumps are audit fallback, not agent memory. Statistical science workers and audio parity stages consume those bundles without rewriting gesture evidence. If gesture scores rise but rendered audio is silent, fix the primitive source/tract owner before adding full-patch dressing."
},
"ipa_vocal_tract_path": {
"status": "This is now the active project center. Roadmap and research notes align the curriculum around structured utterance metadata -> learned utterance embedding -> learned synth automation -> vocal tract/envelope/LFO/filter/spectral controls -> rendered audio loss. The first Core contracts still defend the boundary: PhoneticIntent is the Weksa/language-owned handoff, ArticulatoryPlan is the AquaSynth realization layer, VocalTractMorphology declares anatomy capabilities, ArticulatoryConstraintReport is the host-visible veto surface, and VocalTractPlanResult is the tiny end-to-end proof artifact for host branching. IPA is an authoring input only: tokenization and feature mapping lower into language phonology profiles, timed articulatory gestures, morphology-specific tract/excitation curves, and Faust DSP. The staged support plan starts with tokenization and vowels, then core pulmonic consonants, expanded human IPA, airstream/phonation, and Weksa/alien morphologies. Optional eSpeak NG tests provide the first generated speech workout when ESPEAK_NG or espeak-ng is installed; eSpeak is intelligibility/timing/log-mel pressure, not anatomy truth. AquaSynth owns the learned utterance embedding encoder and synth driver: speech text embeddings plus prosody/emphasis hints plus Ghostlight/Epiphany-shaped character state vectors compress into utterance embeddings, and phonetic features plus that embedding feed a packed-buffer native C# backprop/Adam mapper that predicts tract, AM/FM, LFO, filter, and mel spectral-envelope controls. SpeechBackpropagationPipeline chains the two models: synth-driver target loss returns gradients with respect to its input, slices the utterance-embedding gradient, and updates the utterance encoder from downstream speech loss. The generated ground-truth training fixture renders eSpeak syllables, extracts log-mel evidence, and trains that chained pipeline toward eSpeak-seeded mel/control targets. CultMesh speech training now has payload manifests, peer/lease admission receipts, worker assignment, result collection, and central checkpoint application; workers do not own model updates. Real remote artifact chunk transfer and process supervision remain CultMesh infrastructure surfaces, not AquaSynth workaround territory. IpaGestureExperiment writes frozen descriptor/variant rounds with candidate scripts, primitive timelines, gesture metrics, manifests, and evidence JSONL, then analyzes them into metric summaries, score-band clusters, and a science brief. IpaTrialOrchestrator now writes local five-seed IPA render trials, accepts externally authored candidate scripts through RunCandidateScriptsAsync, writes WAV/report artifacts, evaluator reports, extracted primitive timeline facts, and `ipa-trial-results.cc` CultCache records. `tools/IpaTrialWorker` is the public worker tool for seed/score/search/show/index over the store; search now auto-upserts Qdrant/Ollama evidence chunks including timeline-fact text and returns vector-backed reports, while `tools/run-ipa-trial-loop.ps1` dispatches external Codex hypothesis/evaluator workers around five five-phoneme target sets. The latest seed run improved voiced vowels/nasals after primitive `SourcePort` lowering gained an oscillatory carrier; plosive closure/release remains the next weak owner.",
@ -123,8 +123,8 @@
},
"last_verified": {
"command": "dotnet build tools\\IpaTrialWorker\\IpaTrialWorker.csproj --no-restore --disable-build-servers -p:UseSharedCompilation=false -p:BuildInParallel=false -v minimal; dotnet run --project tools\\IpaTrialWorker\\IpaTrialWorker.csproj --no-build -- song-prepare --source \"D:\\Music\\Reinier\\Heyoka\\Gate Code\\Heyoka - Alien Gibberish.mp3\" --artifact-root artifacts\\parity\\song-snippet-smoke --duration-seconds 10 --seed 123 --output artifacts\\parity\\song-snippet-smoke\\challenge.json; powershell -NoProfile -ExecutionPolicy Bypass -File tools\\run-song-snippet-trial-swarm.ps1 -Passes 0 -Seed 123 -DurationSeconds 10; dotnet run --project tools\\IpaTrialWorker\\IpaTrialWorker.csproj --no-build -- song-score --patch-root artifacts\\parity\\song-snippet-score-smoke\\candidates --challenge artifacts\\parity\\song-snippet-smoke\\challenge.json --artifact-root artifacts\\parity\\song-snippet-score-smoke --batch-id smoke-001 --store artifacts\\parity\\song-snippet-score-smoke\\song-trial-results.cc --hypothesizer smoke-song-worker",
"result": "Song-snippet worker and swarm prep now build, decode a seeded local clip, emit tempo/register/scale challenge facts, prepare a loop skeleton without launching agents, and score an ordinary non-vocal scene patch into song trial-result artifacts.",
"commit": "pending song-snippet trial swarm"
"result": "Three-phase song-snippet experiment across different seeded segments completed under artifacts/parity/song-snippet-swarms/20260531T014033022. Best log-mel cosine improved to 0.380805 through formant/noise scene roles, still pressure rather than close parity.",
"commit": "pending song-snippet phase evidence"
}
}

View file

@ -33,6 +33,18 @@ Current slice:
primitive timelines are now optional witnesses for song scenes instead of a
required anatomy gate.
- Ran a three-phase, three-agent song-snippet experiment across different seeded
segments under `artifacts/parity/song-snippet-swarms/20260531T014033022`.
Segment starts were `143.0295s`, `169.7126s`, and `196.3956s`; tempo hints
varied (`200`, `63.82979`, `150` BPM) while register stayed B4-ish with
`dominant_hz=2004.545` and scale frequencies `493.883..987.767`. Best
log-mel cosine climbed from `0.308233` in pass 1 to `0.356515` in pass 2 and
`0.380805` in pass 3. Still not close; the winning evidence is that agents
converged on a reusable scene model: bright FM/formant `alien-lead`, clocked
high-band `dust-hat`, continuous `codec-bed`, and `air-wash-hiss`. Syrinx
narrative lost to boring formant/noise machinery. Report:
`artifacts/parity/song-snippet-swarms/20260531T014033022/experiment-report.md`.
- User correctly called out that the loose agents learned nothing. The actual
failure was not just weak metrics; the candidate assignment was incoherent.
Families were rotated across targets instead of aligned to phonetic owners:

View file

@ -4,6 +4,7 @@ param(
[string]$Source = "D:\Music\Reinier\Heyoka\Gate Code\Heyoka - Alien Gibberish.mp3",
[float]$DurationSeconds = 10,
[int]$Seed = 0,
[switch]$NewSegmentPerPass,
[string]$CodexCommand = "codex",
[string]$CodexModel = "gpt-5.3-codex"
)
@ -138,10 +139,8 @@ $loopRoot = Join-Path $repoRoot "artifacts/parity/song-snippet-swarms/$loopId"
$logRoot = Join-Path $loopRoot "logs"
$workerProject = Join-Path $repoRoot "tools/IpaTrialWorker/IpaTrialWorker.csproj"
$storePath = Join-Path $loopRoot "song-trial-results.cc"
$challengePath = Join-Path $loopRoot "challenge.json"
$challengeRoot = Join-Path $loopRoot "challenge"
$seedValue = if ($Seed -eq 0) { Get-Random -Minimum 1 -Maximum ([int]::MaxValue) } else { $Seed }
New-Item -ItemType Directory -Force -Path $loopRoot, $logRoot, $challengeRoot | Out-Null
New-Item -ItemType Directory -Force -Path $loopRoot, $logRoot | Out-Null
Set-Content -LiteralPath (Join-Path $loopRoot "loop-index.md") -Value @"
# Song Snippet Trial Swarm $loopId
@ -153,40 +152,45 @@ Set-Content -LiteralPath (Join-Path $loopRoot "loop-index.md") -Value @"
- agents_per_pass: $AgentsPerPass
- seed: $seedValue
- duration_seconds: $DurationSeconds
- challenge: $challengePath
- new_segment_per_pass: $NewSegmentPerPass
- store: $storePath
- worker: $workerProject
"@
Invoke-LoggedProcess `
-FilePath "dotnet" `
-ArgumentList @(
"run",
"--project",
$workerProject,
"--",
"song-prepare",
"--source",
$Source,
"--artifact-root",
$challengeRoot,
"--duration-seconds",
([string]$DurationSeconds),
"--seed",
([string]$seedValue),
"--output",
$challengePath
) `
-LogPath (Join-Path $logRoot "challenge-prepare.log")
$challengeMarkdown = Join-Path $challengeRoot "challenge.md"
for ($pass = 1; $pass -le $Passes; $pass++) {
$passId = "pass-{0:000}" -f $pass
$passDir = Join-Path $loopRoot $passId
$patchRoot = Join-Path $passDir "proposed-patches"
New-Item -ItemType Directory -Force -Path $passDir, $patchRoot | Out-Null
$challengeRoot = Join-Path $passDir "challenge"
$challengePath = Join-Path $challengeRoot "challenge.json"
$phaseSeed = if ($NewSegmentPerPass) { $seedValue + (($pass - 1) * 7919) } else { $seedValue }
New-Item -ItemType Directory -Force -Path $passDir, $patchRoot, $challengeRoot | Out-Null
Invoke-LoggedProcess `
-FilePath "dotnet" `
-ArgumentList @(
"run",
"--project",
$workerProject,
"--",
"song-prepare",
"--source",
$Source,
"--artifact-root",
$challengeRoot,
"--duration-seconds",
([string]$DurationSeconds),
"--seed",
([string]$phaseSeed),
"--challenge-id",
"song-snippet-$loopId-$passId",
"--output",
$challengePath
) `
-LogPath (Join-Path $logRoot "$passId-challenge-prepare.log")
$challengeMarkdown = Join-Path $challengeRoot "challenge.md"
$preEvidence = Join-Path $passDir "semantic-search-before.md"
if (Test-Path -LiteralPath $storePath) {
@ -233,6 +237,8 @@ Challenge:
- shared prior evidence: $preEvidence
- patch output directory: $agentPatchDir
- candidate filename prefix: $agentId
- phase id: $passId
- phase seed: $phaseSeed
Goal:
Write exactly one parseable AquaSynth `.aqua` patch that tries to reproduce the frozen 10 second reference clip. This is scene-audio parity, not IPA articulation. You may use ordinary voices, FM, AM/tremolo, filters, formants, noise layers, acoustic vocal/syrinx primitives, curves, and helper voices.
@ -256,9 +262,18 @@ Register and scale:
- bind it: `freq=@/seq/pitch`
- Do not invent native `scale`, `note`, or Strudel pattern syntax in the patch unless you also edit the DSL/lowering source and document the new syntax. For candidate patches, explicit scale frequencies are the contract.
Instrument and DSL convention invention:
- Invent at least one reusable instrument role for this segment, such as `alien-lead`, `dust-hat`, `codec-bed`, `rubber-bass`, `vowel-drone`, or another precise name.
- Write $agentDir/instrument-conventions.md with:
- the proposed instrument role names;
- the control surfaces each role would want;
- any Strudel-like sugar that would make the patch shorter;
- the exact lowering into today's AquaSynth syntax using explicit `voice`, `param`, and `curve` lines.
- The `.aqua` candidate must use today's implemented syntax only. Proposed sugar is evidence for the next DSL cut, not magic accepted by the parser.
Evidence contract:
- Read $challengeMarkdown and $preEvidence.
- Write $agentDir/hypotheses.md with: reference features, tempo/rhythm plan, register/scale plan, scene voices, synthesis owners, expected metric movement, known risks.
- Write $agentDir/hypotheses.md with: reference features, tempo/rhythm plan, register/scale plan, scene voices, synthesis owners, invented instrument roles, expected metric movement, known risks.
- Write exactly one `.aqua` file under $agentPatchDir named `<agent-id>__<family>__<hypothesis>.aqua`.
- Include at least three scene voices or layers in the patch, such as primary alien-voice/body, rhythmic transient/noise, and background/recording color.
- Make the patch duration/gates cover the 10 second target.