Harden IPA trial prompt evidence quality
This commit is contained in:
Meta Vens
parent
62f3fc808a
commit
7e13c72691
6 changed files with 71 additions and 17 deletions
|
|
@ -704,6 +704,18 @@ specific, comparable, falsifiable, and reusable by the next round. A pretty
|
|||
report with vague receipts is a failed science artifact even if it obeys the
|
||||
file-count contract.
|
||||
|
||||
Five prompt-optimization smokes on 2026-05-29 sharpened that contract. The best
|
||||
agent output came from a fixed receipt packet and a constrained reasoning pass,
|
||||
not from letting an external agent wander the store while thinking. Production
|
||||
workers therefore have to earn each seam claim with a contrast pair, exact
|
||||
metric values from `show`, primitive-timeline excerpts when available, a
|
||||
class-consistent reference matrix, and an owner sentence of the form
|
||||
`X owns Y so Z remains true`. A seam is not actionable until the next family
|
||||
proposes at least five micro-sweep perturbations across it. A new candidate
|
||||
family must move a new owner, control axis, timing contour, or primitive
|
||||
relationship; loudness, burst gain, and recording-condition dressing alone are
|
||||
parity tools, not loss-landscape novelty.
|
||||
|
||||
## CultMesh Render/Scoring Work
|
||||
|
||||
CultMesh is the transport, admission, and worker-orchestration layer for speech
|
||||
|
|
|
|||
|
|
@ -240,3 +240,4 @@
|
|||
{"date":"2026-05-29","area":"ipa-trial-loop","lesson":"External IPA exploration needs a semantic retrieval tool over the CultCache .cc trial store, not crude full-memory dumps. Hypothesis agents should use search/show to retrieve ranked metric and artifact context, then write batches across the five five-phoneme target sets; the worker owns rendering, scoring, and .cc upserts."}
|
||||
{"date":"2026-05-29","area":"ipa-trial-loop-prompts","lesson":"Prompt smoke found that agent prompts need hard acceptance contracts, not just instructions: exact candidate count, canonical filename schema, normalized family ids, retrieval receipts, claim-to-metric audit rows, and fail-closed evaluator compliance prevent report-shaped fog from passing as loss-landscape science."}
|
||||
{"date":"2026-05-29","area":"ipa-trial-evidence-quality","lesson":"The IPA exploration loop should optimize evidence quality, not just prompt compliance. Hypothesizers must publish specificity/comparability/falsifiability/reuse ledgers per family, and evaluators must penalize vague receipts even when candidate metrics look better."}
|
||||
{"date":"2026-05-29","area":"ipa-trial-prompt-optimization","lesson":"Five prompt optimization passes showed that live external-agent retrieval is too slow and noisy for science unless tightly bounded. The sharpest hypothesizer smoke used a fixed six-receipt packet and a reasoning-only boundary, then scored evidence by contrast pairs, concrete show metrics, primitive timeline excerpts, class-consistent references, owner sentences, micro-sweep plans, and novelty gates. Production prompts now require those witnesses before treating a DSL seam as actionable."}
|
||||
|
|
|
|||
|
|
@ -121,9 +121,9 @@
|
|||
"binding_rule": "Parameter binding uses field-site references such as `lpf=@/macro/brightness` and `/opgraphs/0/operators/2/level`; the field owns meaning and the parameter owns the runtime value. Avoid target-routing semantics unless a new graph authority is deliberately mapped."
|
||||
},
|
||||
"last_verified": {
|
||||
"command": "dotnet test tests\\AquaSynth.Dsl.Tests\\AquaSynth.Dsl.Tests.csproj --no-restore --disable-build-servers -p:UseSharedCompilation=false -p:BuildInParallel=false --filter \"FullyQualifiedName~PatchScriptTests\" -v minimal",
|
||||
"result": "IPA trial render orchestration added. Opt-in five-seed trial run passed with Faust installed and wrote artifacts/parity/ipa-trials/20260529T214127955/five-seed-trials. Focused PatchScriptTests plus IpaTrialResultsRoundTripThroughCultCache pass 94/94. Earlier full unfiltered runs hit the five-minute wall after leaving orphan dotnet workers; focused suites are the current reliable verification path.",
|
||||
"commit": "pending ipa trial orchestration"
|
||||
"command": "powershell -NoProfile -ExecutionPolicy Bypass -File tools\\run-ipa-trial-loop.ps1 -Rounds 0 -SkipLocalTrialRun; node -e \"JSON.parse(require('fs').readFileSync('state/memory.json','utf8')); console.log('memory-json-ok')\"; git diff --check",
|
||||
"result": "Five IPA prompt-optimization passes hardened the external trial-loop prompts around evidence quality: contrast pairs, show-backed metrics, primitive timeline excerpts or explicit missing, class-consistent references, owner sentences, five micro-sweep perturbations, and novelty gates. Loop skeleton, memory JSON parse, and whitespace diff check pass.",
|
||||
"commit": "pending ipa prompt evidence quality"
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -2,6 +2,23 @@
|
|||
|
||||
Current slice:
|
||||
|
||||
- Ran five IPA prompt-optimization passes against the external trial-loop
|
||||
agent prompts. Pass 1 produced useful seam science but treated fallback file
|
||||
reads as evidence after `dotnet run` failed. Pass 2 proved live search/show
|
||||
receipts could work but showed retrieval wandering eats the turn. Pass 3
|
||||
showed capped live retrieval was still too slow for prompt iteration through
|
||||
external `codex exec`. Pass 4 exposed an incoherent "do not run commands"
|
||||
boundary because the agent could neither read receipts nor write a report.
|
||||
Pass 5 succeeded with a fixed six-receipt packet and a reasoning-only
|
||||
boundary, producing sharper seam evidence: stop release ownership, voicing
|
||||
loudness leak, fricative place collapse, mixed-set contamination, and
|
||||
single-candidate optimizer ownership. Production prompts now require contrast
|
||||
pairs, `show` metric values, primitive timeline excerpts or explicit
|
||||
`missing`, class-consistent reference matrices, `X owns Y so Z remains true`
|
||||
owner sentences, at least five micro-sweep perturbations before a seam is
|
||||
actionable, and a novelty gate that rejects loudness/dressing-only families
|
||||
as loss-landscape novelty.
|
||||
|
||||
- Added the external IPA trial loop as a real worker boundary instead of an
|
||||
in-chat report shuffle. `tools/IpaTrialWorker` now exposes `seed`, `score`,
|
||||
`search`, `show`, and audit-only `dump` commands over a shared
|
||||
|
|
|
|||
|
|
@ -145,7 +145,7 @@ live_invariants:
|
|||
- IPA training scoring contract is documented. Reports must keep `gesture_score`, `clean_vocal_score`, and `full_parity_score` separate. Gesture score measures descriptor-to-surface coverage, motion direction, spline contour/timing, primitive timeline consequences, and optional external articulation landmarks. Full parity patches may use FM/AM/modulators/envelopes/filters, auxiliary animated voices, and recording-condition helpers, but those helpers cannot count as articulation evidence unless gesture and clean-vocal scores move coherently.
|
||||
- `IpaGestureExperiment.WriteRound` is the first frozen batch harness for IPA hypothesis work. It writes `manifest.yaml`, candidate `.aqua` scripts, primitive `ProbeTimelineReport` CSVs, gesture-layer `metrics.csv`, and per-candidate `evidence.jsonl` under a caller-provided artifact root. `IpaGestureExperiment.AnalyzeRound` writes `analysis/metric-summary.csv`, `analysis/candidate-clusters.csv`, and `analysis/science-brief.md` for the science-worker handoff. This harness owns frozen descriptor/spline/primitive evidence only; clean vocal identity, full spectrogram parity, optimizer checkpoints, and worker orchestration stay downstream. Focused verification: `PatchScriptTests` pass 93/93.
|
||||
- `IpaTrialOrchestrator.RunAsync` is the first local render/scoring worker for IPA trials. It generates candidate patches through `IpaGestureExperiment`, renders them through Faust, compares against local PT fixture references with `AudioAnalyzer`, writes WAV/report artifacts, and stores typed `IpaTrialResult` records in `ipa-trial-results.cc`. The five-seed trial run at `tests/AquaSynth.Dsl.Tests/bin/Debug/net10.0/artifacts/parity/ipa-trials/20260529T214127955/five-seed-trials` showed voiced vowels/nasals became audible only after primitive `SourcePort` lowering gained a true oscillatory voiced carrier; sibilants are pressure, plosives remain weak. Verification: opt-in IPA trial test passed, and focused `PatchScriptTests|IpaTrialResultsRoundTripThroughCultCache` pass 94/94.
|
||||
- External IPA exploration now has a worker/tool boundary. `tools/IpaTrialWorker` owns `seed`, `score`, `search`, `show`, and audit-only `dump` commands over the typed `.cc` trial-result store; `score` renders external Codex-authored `.aqua` candidates through `IpaTrialOrchestrator.RunCandidateScriptsAsync`, while `search` is the semantic retrieval organ for hypothesis agents. `tools/run-ipa-trial-loop.ps1` orchestrates external `codex exec` hypothesis/evaluator workers around that store. The active trial shape is five target sets of five phoneme lanes each: vowels, nasals/approximants, fricatives, stops, and mixed generalization. Candidate names are exactly `<targetId>__<family>__<hypothesis-name>.aqua`, with lowercase kebab-case family ids. Three dry external Codex smoke passes refined hypothesizer/evaluator prompts into exact 25-lane output, retrieval receipts, claim audits, evidence-quality ledgers/verdicts, and fail-closed evaluator compliance. Verification: `IpaTrialWorker` builds, `search` smoke reads the seed store, and the loop skeleton runs with `-Rounds 0 -SkipLocalTrialRun`.
|
||||
- External IPA exploration now has a worker/tool boundary. `tools/IpaTrialWorker` owns `seed`, `score`, `search`, `show`, and audit-only `dump` commands over the typed `.cc` trial-result store; `score` renders external Codex-authored `.aqua` candidates through `IpaTrialOrchestrator.RunCandidateScriptsAsync`, while `search` is the semantic retrieval organ for hypothesis agents. `tools/run-ipa-trial-loop.ps1` orchestrates external `codex exec` hypothesis/evaluator workers around that store. The active trial shape is five target sets of five phoneme lanes each: vowels, nasals/approximants, fricatives, stops, and mixed generalization. Candidate names are exactly `<targetId>__<family>__<hypothesis-name>.aqua`, with lowercase kebab-case family ids. Three dry external Codex smoke passes refined hypothesizer/evaluator prompts into exact 25-lane output, retrieval receipts, claim audits, evidence-quality ledgers/verdicts, and fail-closed evaluator compliance. A later five-pass prompt optimization showed the best science comes from fixed receipt packets plus constrained reasoning; production prompts now require contrast pairs, `show` metric values, primitive timeline excerpts or explicit `missing`, class-consistent reference matrices, `X owns Y so Z remains true` owner sentences, at least five micro-sweep perturbations before a seam is actionable, and a novelty gate rejecting loudness/dressing-only families as loss-landscape novelty. Verification: `IpaTrialWorker` builds, `search` smoke reads the seed store, and the loop skeleton runs with `-Rounds 0 -SkipLocalTrialRun`.
|
||||
- TurbulenceJet now includes a release-timed burst-noise component in addition to the deterministic pressure pulse, but a stronger trial barely moved papa; the source is not the main remaining articulation bottleneck. Graph radiation now mixes 30% raw boundary flow with 70% high-passed flow after local admittance weighting, down from the over-bright 15/85 blend. Latest radiation/source artifact under `artifacts/parity/pink-trombone-utterance-logmel/20260527T174236049`: mama/papa/thrombosis cosine 0.6578/0.7692/0.5585, rmsRatio 3.0006/2.6153/2.3488, articulation 0.1437/0.1173/0.1191. Static `closure-release` under `artifacts/parity/pink-trombone-logmel/20260527T174139661` has rmsRatio 0.9900 and speechBandRatio 0.0842. This is still not speaking; next pressure is true tract resonance/passive scattering and pressure storage, not more burst gain.
|
||||
- Same-path two-port area discontinuities now emit `graph_area_energy_in_*` and `graph_area_energy_out_*` probes alongside the three-port branch `graph_connection_energy_*` probes. Use these to inspect passivity and formant-body failures at the scatter sites rather than judging only final output loudness.
|
||||
- `FaustExportOptions.DebugProbeUi` wraps graph probe signals in Faust `vbargraph` UI controls under `/debug/...` without changing audio output arity. Native streaming Faust patches now separate writable controls from read-only probe zones and can read/snapshot probe values after `ProcessBlock`; use this for acoustic graph passivity/radiation inspection, leave it off for normal parity renders.
|
||||
|
|
|
|||
|
|
@ -238,8 +238,14 @@ The backing store is:
|
|||
$storePath
|
||||
|
||||
Use the worker as your search organ. Do not ask for crude full dumps unless retrieval fails:
|
||||
dotnet run --project tools/IpaTrialWorker/IpaTrialWorker.csproj -- search --store "$storePath" --query "<your evidence question>" --limit 20 --output "$roundDir/search-<topic>.md"
|
||||
dotnet run --project tools/IpaTrialWorker/IpaTrialWorker.csproj -- show --store "$storePath" --trial-id "<trial id from search>" --output "$roundDir/detail-<candidate>.md"
|
||||
`$env:DOTNET_CLI_HOME="$roundDir/dotnet-home"; `$env:DOTNET_SKIP_FIRST_TIME_EXPERIENCE="1"; `$env:DOTNET_CLI_TELEMETRY_OPTOUT="1"; dotnet run --no-build --no-restore --project tools/IpaTrialWorker/IpaTrialWorker.csproj -- search --store "$storePath" --query "<your evidence question>" --limit 20 --output "$roundDir/search-<topic>.md"
|
||||
`$env:DOTNET_CLI_HOME="$roundDir/dotnet-home"; `$env:DOTNET_SKIP_FIRST_TIME_EXPERIENCE="1"; `$env:DOTNET_CLI_TELEMETRY_OPTOUT="1"; dotnet run --no-build --no-restore --project tools/IpaTrialWorker/IpaTrialWorker.csproj -- show --store "$storePath" --trial-id "<trial id from search>" --output "$roundDir/detail-<candidate>.md"
|
||||
|
||||
Retrieval failure policy:
|
||||
- A failed search/show command is itself evidence about the worker environment, but it is low-quality evidence for speech hypotheses.
|
||||
- Fallback artifact reads may appear in `Weakest Missing Evidence`, but they do not satisfy the search/show receipt counts.
|
||||
- Do not write patches until live `search` and `show` receipts exist.
|
||||
- Search-summary-only claims are low-quality until corroborated by a `show` receipt with concrete metric values or artifact paths.
|
||||
|
||||
Useful repo surfaces:
|
||||
- docs/ipa-vocal-tract-roadmap.md
|
||||
|
|
@ -250,13 +256,16 @@ Useful repo surfaces:
|
|||
|
||||
Before writing candidates:
|
||||
0. Open $preEvidence and include a `PreEvidence Digest` section in $roundDir/hypotheses.md with five metric-bearing facts from it.
|
||||
1. Run at least three targeted semantic searches and save the outputs under ${roundDir}:
|
||||
1. Run exactly three targeted semantic searches and save the outputs under ${roundDir}:
|
||||
- one for weakest stop/plosive closure evidence;
|
||||
- one for best vowel/nasal/fricative transferable successes;
|
||||
- one for dressing/FM/AM/noise versus articulation-owner failures.
|
||||
2. Open at least three detailed trial records with `show`: one weak stop/plosive, one transferable success, and one dressing-vs-articulation case when available.
|
||||
2. Open exactly three detailed trial records with `show`: one weak stop/plosive, one transferable success, and one dressing-vs-articulation case when available.
|
||||
3. In $roundDir/hypotheses.md, include a `Retrieval Receipts` section listing the exact search/show files you used.
|
||||
4. If fewer than three search receipts and three show receipts exist on disk, stop and create the missing retrieval artifacts before writing `.aqua` files.
|
||||
5. Do not run additional retrieval unless one of the six required receipts is empty or unrelated; spend the rest of the turn reasoning over the evidence.
|
||||
6. For every seam you name, cite at least one contrast pair: a weak record and a stronger related record, or two variants that isolate one axis. If the receipts do not contain a contrast pair, mark the seam `not actionable`.
|
||||
7. Extract primitive timeline facts when available: closure duration, burst/release window, voicing onset, constriction window, branch admittance, radiation flow, or passivity/energy. If timelines are absent, mark timeline evidence `missing` instead of implying it.
|
||||
|
||||
Task:
|
||||
1. Read the current loss landscape across gesture, log-mel, RMS, articulation, and primitive timeline evidence.
|
||||
|
|
@ -287,14 +296,20 @@ Candidate design contract:
|
|||
- Only edit source if two or more retrieved trials indicate the same owner-layer failure pattern; otherwise `Source Edit Decision: none`.
|
||||
- If Source edits allowed is False, do not edit any file outside $patchDir and $roundDir/hypotheses.md.
|
||||
- Before finishing, list $patchDir and verify exact count, target coverage, and filename schema in `Acceptance Checklist`.
|
||||
- A seam claim must use an owner sentence: `X owns Y so Z remains true`.
|
||||
- A seam is actionable only if the next candidate family proposes at least five micro-sweep perturbations across that seam, spread across target lanes when possible.
|
||||
- Novelty gate: a family must move a new owner, control axis, timing contour, or primitive relationship. Raising loudness, burst gain, or dressing alone is not novel.
|
||||
- Include a class-consistent reference matrix that maps each target id to the exact reference trial or fixture id used for comparison; mixed-set evidence cannot silently stand in for single-phoneme evidence.
|
||||
|
||||
Required `$roundDir/hypotheses.md` shape:
|
||||
- `PreEvidence Digest`: five facts with exact metric keys and values from $preEvidence.
|
||||
- `Retrieval Receipts`: search/show files used.
|
||||
- `Loss Landscape Read`: strongest and weakest evidence by family.
|
||||
- `Hypothesis Families`: 3-5 families, each with owner layer, transfer status, evidence_refs, cited trial_ids, at least one concrete metric key/value from `show`, and predicted metric movement using `gesture:+/-/flat`, `logmel:+/-/flat`, `articulation:+/-/flat`, `rms:+/-/flat`, `timeline:+/-/flat/risk`.
|
||||
- `Claim Audit`: one row per family: claim -> evidence file(s) -> metric key/value(s) -> predicted movement.
|
||||
- `Reference Matrix`: one row per target id with reference trial/fixture id, candidate baseline if known, and whether the evidence is class-consistent or mixed/weak.
|
||||
- `Hypothesis Families`: 3-5 families, each with owner layer, transfer status, evidence_refs, cited trial_ids, at least one concrete metric key/value from `show`, a contrast pair, an owner sentence of the form `X owns Y so Z remains true`, five planned micro-sweep perturbations, and predicted metric movement using `gesture:+/-/flat`, `logmel:+/-/flat`, `articulation:+/-/flat`, `rms:+/-/flat`, `timeline:+/-/flat/risk`.
|
||||
- `Claim Audit`: one row per family: claim -> evidence file(s) -> contrast pair -> metric key/value(s) -> primitive timeline excerpt or `missing` -> predicted movement.
|
||||
- `Evidence Quality Ledger`: one row per family with specificity, comparability, falsifiability, reuse value for the next round, and the weakest missing evidence. Use `high`, `medium`, or `low` for each quality field.
|
||||
- `Novelty Gate`: one row per family saying which new owner/control axis/timing contour/primitive relationship is being explored; write `dressing-only` if the family mainly changes loudness/FM/AM/noise/envelopes.
|
||||
- `Candidate Matrix`: exactly 25 rows with target id, filename, family, expected metric movement, and risk.
|
||||
- `Source Edit Decision`: `none` unless source edits are allowed and justified by evidence.
|
||||
- `Acceptance Checklist`: 25 files, one per target id, filename schema, required report sections, >=3 search receipts, >=3 show receipts.
|
||||
|
|
@ -386,10 +401,15 @@ Before judging:
|
|||
3. If the hypothesis worker skipped the required 25-lane matrix or retrieval receipts, mark orchestration/prompt compliance as failed even if some audio metrics improved.
|
||||
|
||||
Use the worker as your search organ:
|
||||
dotnet run --project tools/IpaTrialWorker/IpaTrialWorker.csproj -- search --store "$storePath" --query "$roundId external-codex-$roundId current round candidates" --limit 30 --output "$roundDir/eval-search-current-round.md"
|
||||
dotnet run --project tools/IpaTrialWorker/IpaTrialWorker.csproj -- search --store "$storePath" --query "weak regression failed articulation stop plosive source tract radiation" --limit 30 --output "$roundDir/eval-search-weak.md"
|
||||
dotnet run --project tools/IpaTrialWorker/IpaTrialWorker.csproj -- search --store "$storePath" --query "promising transfer generalization vowel nasal fricative articulation" --limit 30 --output "$roundDir/eval-search-promising.md"
|
||||
dotnet run --project tools/IpaTrialWorker/IpaTrialWorker.csproj -- show --store "$storePath" --trial-id "<trial id from current round search>" --output "$roundDir/eval-detail-<candidate>.md"
|
||||
`$env:DOTNET_CLI_HOME="$roundDir/dotnet-home"; `$env:DOTNET_SKIP_FIRST_TIME_EXPERIENCE="1"; `$env:DOTNET_CLI_TELEMETRY_OPTOUT="1"; dotnet run --no-build --no-restore --project tools/IpaTrialWorker/IpaTrialWorker.csproj -- search --store "$storePath" --query "$roundId external-codex-$roundId current round candidates" --limit 30 --output "$roundDir/eval-search-current-round.md"
|
||||
`$env:DOTNET_CLI_HOME="$roundDir/dotnet-home"; `$env:DOTNET_SKIP_FIRST_TIME_EXPERIENCE="1"; `$env:DOTNET_CLI_TELEMETRY_OPTOUT="1"; dotnet run --no-build --no-restore --project tools/IpaTrialWorker/IpaTrialWorker.csproj -- search --store "$storePath" --query "weak regression failed articulation stop plosive source tract radiation" --limit 30 --output "$roundDir/eval-search-weak.md"
|
||||
`$env:DOTNET_CLI_HOME="$roundDir/dotnet-home"; `$env:DOTNET_SKIP_FIRST_TIME_EXPERIENCE="1"; `$env:DOTNET_CLI_TELEMETRY_OPTOUT="1"; dotnet run --no-build --no-restore --project tools/IpaTrialWorker/IpaTrialWorker.csproj -- search --store "$storePath" --query "promising transfer generalization vowel nasal fricative articulation" --limit 30 --output "$roundDir/eval-search-promising.md"
|
||||
`$env:DOTNET_CLI_HOME="$roundDir/dotnet-home"; `$env:DOTNET_SKIP_FIRST_TIME_EXPERIENCE="1"; `$env:DOTNET_CLI_TELEMETRY_OPTOUT="1"; dotnet run --no-build --no-restore --project tools/IpaTrialWorker/IpaTrialWorker.csproj -- show --store "$storePath" --trial-id "<trial id from current round search>" --output "$roundDir/eval-detail-<candidate>.md"
|
||||
|
||||
Retrieval failure policy:
|
||||
- Failed search/show commands do not count as evidence receipts for family verdicts.
|
||||
- If live worker retrieval fails, set `Round Compliance: failed` and focus the report on the retrieval blocker instead of inventing metric conclusions.
|
||||
- Search-summary-only claims are low-quality until corroborated by `show` receipts.
|
||||
|
||||
Verify compliance from disk as well as the hypothesis report:
|
||||
- count .aqua files under $patchDir;
|
||||
|
|
@ -398,7 +418,9 @@ Verify compliance from disk as well as the hypothesis report:
|
|||
- validate filename schema <targetId>__<family>__<hypothesis-name>.aqua.
|
||||
- validate family ids are lowercase kebab-case.
|
||||
- derive canonical family ids only from the filename family segment; ignore prose family names when they disagree.
|
||||
- If any required section, receipt class, per-target validation, filename validation, or family extraction is incomplete, set `Round Compliance: failed`.
|
||||
- validate that each actionable seam has a contrast pair, a primitive timeline excerpt or explicit `missing`, an owner sentence, and at least five planned micro-sweep perturbations.
|
||||
- validate that the hypothesis report includes a class-consistent reference matrix and a novelty gate.
|
||||
- If any required section, receipt class, per-target validation, filename validation, family extraction, reference matrix, contrast pair, owner sentence, or novelty gate is incomplete, set `Round Compliance: failed`.
|
||||
|
||||
Task:
|
||||
1. Evaluate whether the worker's proposed hypotheses follow from the measured evidence.
|
||||
|
|
@ -410,9 +432,11 @@ $roundDir/evaluation.md
|
|||
Required `$roundDir/evaluation.md` shape:
|
||||
- `Retrieval Receipts`: exactly five receipts: three search outputs plus one weak show output and one promising show output. Cite no other evidence files in this section. Only cite claims backed by these file paths; write `unknown` when evidence is absent.
|
||||
- `Commands Run`: exact search/show commands and output paths used to support claims.
|
||||
- `Round Compliance`: passed/failed, candidate count, target coverage, naming validity, family extraction, and whether the report had the required matrix.
|
||||
- `Round Compliance`: passed/failed, candidate count, target coverage, naming validity, family extraction, reference matrix, contrast-pair coverage, owner-sentence coverage, micro-sweep coverage, novelty gate, and whether the report had the required matrix.
|
||||
- `Family Verdicts`: one row per canonical filename family with improved/flat/regressed/unknown metrics. Treat |delta| < 0.01 as flat; if deltas are not present in cited show artifacts, write `unknown` and reason=`no delta in receipts`.
|
||||
- `Evidence Quality Verdicts`: one row per family with specificity, comparability, falsifiability, reuse value, and weakest missing evidence. Penalize vague receipts even when metrics improve.
|
||||
- `Evidence Quality Verdicts`: one row per family with specificity, comparability, falsifiability, reuse value, primitive timeline support, and weakest missing evidence. Penalize vague receipts, search-summary-only claims, missing contrast pairs, missing reference ids, and missing timeline excerpts even when metrics improve.
|
||||
- `Seam Audit`: one row per named seam with owner sentence, contrast pair, timeline excerpt, planned perturbation count, and verdict `actionable` or `not actionable`; fewer than five perturbations means `not actionable`.
|
||||
- `Novelty Audit`: one row per family saying whether it moves a new owner/control axis/timing contour/primitive relationship or is mainly loudness/dressing.
|
||||
- `Generalization Read`: whether effects transferred across target sets or overfit one phoneme.
|
||||
- `Dressing Audit`: articulation evidence requires movement in gesture/articulation/primitive timeline metrics; FM/AM/noise/envelope evidence without those is dressing.
|
||||
- `Next Target`: exactly one markdown bullet, final line of the file, and no continuation text: `<file-or-module> | <invariant> | <expected metric movement>`.
|
||||
|
|
|
|||
Loading…
Reference in a new issue