diff --git a/docs/audio-doctrine.md b/docs/audio-doctrine.md index 4c4832c..e69b8ac 100644 --- a/docs/audio-doctrine.md +++ b/docs/audio-doctrine.md @@ -341,6 +341,13 @@ voice for scene noise. Full-duration raw `wave=noise` beds with broad filters are failed evidence: they sound like broken subtractive synthesis because that is exactly what they are. +Song continuity is a scored production invariant. A one-second phrase followed +by low-motion air, pink noise, room tone, or pad wash is mode collapse, not +arrangement. The evaluator should publish motion coverage, active coverage, +first-second energy share, tail energy share, and mode-collapse risk; distillation +should treat high collapse risk as negative pressure even when log-mel similarity +looks superficially acceptable. + Song-scene instrument roles are scored as curriculum evidence. Voice-like leads should prefer syrinx/acoustic source ownership with pressure/opening motion and radiation shaping; percussion should prefer subtractive pitched body diff --git a/state/evidence.jsonl b/state/evidence.jsonl index 8079c32..6f65ed1 100644 --- a/state/evidence.jsonl +++ b/state/evidence.jsonl @@ -261,3 +261,4 @@ {"date":"2026-05-31","area":"cultcache-storage","summary":"Full-song song-trial stores can exceed 2 GB; single-file MessagePack snapshot flushing is the wrong owner for hot trial writes because it rewrites every cold record and can OOM. CultLib now has DirectoryMessagePackBackingStore: .cc is a small schema manifest, records live under .records/*.msgpack, and AquaSynth trial-result stores open with UseDirectoryStore=true.","verification":"dotnet test E:\\Projects\\CultLib\\tests\\GameCult.Caching.Tests\\GameCult.Caching.Tests.csproj --no-restore --filter BackingStoreTests: 14 passed; dotnet build E:\\Projects\\AquaSynth\\tools\\IpaTrialWorker\\IpaTrialWorker.csproj --no-restore: passed; dotnet test E:\\Projects\\AquaSynth\\tests\\AquaSynth.Dsl.Tests\\AquaSynth.Dsl.Tests.csproj --no-restore --filter CultCache: 7 passed"} {"date":"2026-05-31","area":"music-production-knowledge-cultcache","summary":"Fresh music curriculum stores should be distilled from official rendered evidence, not copied from hot trial stores. `music-distill` now writes paged `MusicProductionKnowledgeDocument` records: one master quality standard, reusable scene-role doctrine for syrinx voice, subtractive drums, additive/PAD beds, and shaped texture, quarantined render-failure pressure, ranked rendered candidate patterns, and abstraction-ledger syntax-sugar pressure. Explorer commands `music-search` and `music-show` make the new store inspectable by fresh agents before patch authoring.","verification":"dotnet build E:\\Projects\\AquaSynth\\tools\\IpaTrialWorker\\IpaTrialWorker.csproj --no-restore: passed; dotnet test E:\\Projects\\AquaSynth\\tests\\AquaSynth.Dsl.Tests\\AquaSynth.Dsl.Tests.csproj --no-restore --filter CultCache: 7 passed; music-distill over artifacts\\parity\\song-snippet-swarms\\20260531T121228337 wrote 37 records from 28 official rows; music-search syrinx/drums and music-show quality-standard succeeded","evidence":"src/AquaSynth.Core/IpaTrialResults.cs; tools/IpaTrialWorker/Program.cs; artifacts/parity/music-knowledge/20260531T121228337/music-production-knowledge.cc; artifacts/parity/music-knowledge/20260531T121228337/distillation-report.md; artifacts/parity/music-knowledge/20260531T121228337/show-quality-standard.md"} {"date":"2026-05-31","area":"song-corpus-three-pass-curriculum","summary":"A three-pass 10-second corpus curriculum run across D:\\Music\\Reinier completed with 25 official assignments per pass. Pass 1 produced 25 official candidates, 23 rendered, 2 render failures, best cosine 0.299361, mean cosine 0.162788, mean score 0.198594, and 34 promoted music knowledge records. Pass 2 produced 25 official candidates, 20 rendered, 5 render failures, best cosine 0.351068, mean cosine 0.137059, mean score 0.170592, and 56 promoted music knowledge records. Pass 3 produced 25 official candidates, 15 rendered, 10 render failures, best cosine 0.304571, mean cosine 0.115347, mean score 0.128224, and 58 promoted music knowledge records. The learned signal is mixed: knowledge injection increased structured artifacts and reusable abstraction ledgers, but later-pass render failure and score degradation must be treated as quality pressure, not progress.","verification":"pass-001/pass-002/pass-003 each have 25 challenge.json files, 25 official candidates, 25 official summary.csv files, top-scoring-candidates.m3u, distillation-report.md, semantic-search-after.md, vector-index-after.md, music-production-knowledge.cc, music-production-knowledge-report.md, and music-production-quality-standard.md. Pass 3 was manually finished after external Codex jobs failed by promoting existing attempt-05 patches and running official scoring/promotion with IpaTrialWorker.exe.","evidence":"artifacts/parity/song-snippet-swarms/20260531T171802519/pass-001/music-production-knowledge.cc; artifacts/parity/song-snippet-swarms/20260531T171802519/pass-002/music-production-knowledge.cc; artifacts/parity/song-snippet-swarms/20260531T171802519/pass-003/music-production-knowledge.cc; artifacts/parity/song-snippet-swarms/20260531T171802519/pass-003/top-scoring-candidates.m3u; artifacts/parity/song-snippet-swarms/20260531T171802519/pass-003/distillation-report.md"} +{"date":"2026-05-31","area":"song-mode-collapse-evaluator","summary":"The corpus run exposed a scoring blind spot: agents could write a short opening phrase and coast for the remaining clip on low-motion texture/noise while still receiving weak-but-ranked evidence. `song-score` now emits song-continuity metrics for active coverage, motion coverage, first-second energy share, tail energy share, and mode-collapse risk; high collapse risk changes verdict to `weak-mode-collapse`, lowers distillation ranking, and is carried into music knowledge failure modes. Swarm prompts now require later-section motifs/events and explicitly reject one-second riff plus nine seconds of texture as submitted song evidence.","verification":"dotnet build E:\\Projects\\AquaSynth\\tools\\IpaTrialWorker\\IpaTrialWorker.csproj --no-restore: passed; dotnet test E:\\Projects\\AquaSynth\\tests\\AquaSynth.Dsl.Tests\\AquaSynth.Dsl.Tests.csproj --no-restore --filter CultCache: 7 passed; rescoring pass-003 agent_02__retrievedmesh__target_04_a05 produced verdict weak-mode-collapse, candidate_first_second_energy_share 0.883627, candidate_motion_coverage 0.573295, mode_collapse_risk 1","evidence":"tools/IpaTrialWorker/Program.cs; tools/run-song-snippet-trial-swarm.ps1; docs/audio-doctrine.md; artifacts/parity/song-continuity-smoke/20260531Tmodecollapse-v2/continuity-smoke/summary.csv"} diff --git a/state/scratch.md b/state/scratch.md index 8e72952..3260f41 100644 --- a/state/scratch.md +++ b/state/scratch.md @@ -2,6 +2,19 @@ Current slice: +- User identified the three-pass corpus run's audible failure correctly: mode + collapse into a short opening phrase followed by low-motion texture/noise. + `song-score` now owns song continuity metrics: + `candidate_active_coverage`, `candidate_motion_coverage`, + `candidate_first_second_energy_share`, `candidate_tail_energy_share`, and + `mode_collapse_risk`. The old top pass-3 candidate + `agent_02__retrievedmesh__target_04_a05` now scores as + `weak-mode-collapse` with first-second energy share `0.883627` and collapse + risk `1`, proving the evaluator sees the cheap trick. Distillation ranking + demotes collapse risk, candidate knowledge records carry the failure mode, + and the swarm prompt now requires later-section motifs/events instead of + hiding behind noise or pads. + - Completed a three-pass corpus song curriculum run under `artifacts/parity/song-snippet-swarms/20260531T171802519`: each pass prepared 25 ten-second targets from `D:\Music\Reinier` for five agents with five diff --git a/tools/IpaTrialWorker/Program.cs b/tools/IpaTrialWorker/Program.cs index ff03f73..cae3783 100644 --- a/tools/IpaTrialWorker/Program.cs +++ b/tools/IpaTrialWorker/Program.cs @@ -287,11 +287,13 @@ static async Task SongScoreAsync(Dictionary options) analyzer); artifacts.AddRange(candidateAnalysisArtifacts.Select(ArtifactFromAnalysis)); var comparison = analyzer.Compare(referenceSamples, matched); + var continuity = AnalyzeSongContinuity(comparison); metrics.AddRange(SongComparisonMetrics(comparison)); - verdict = SongVerdict(comparison, instrumentProfile); - evaluation = SongEvaluationSentence(challenge, comparison, verdict, instrumentProfile); + metrics.AddRange(SongContinuityMetrics(continuity)); + verdict = SongVerdict(comparison, instrumentProfile, continuity); + evaluation = SongEvaluationSentence(challenge, comparison, verdict, instrumentProfile, continuity); var comparisonPath = Path.Combine(audioDir, "comparison.txt"); - await File.WriteAllTextAsync(comparisonPath, SongComparisonReport(challenge, candidate, comparison, verdict, instrumentProfile), Encoding.UTF8); + await File.WriteAllTextAsync(comparisonPath, SongComparisonReport(challenge, candidate, comparison, verdict, instrumentProfile, continuity), Encoding.UTF8); artifacts.Add(Artifact("comparison-report", comparisonPath)); } } @@ -1685,6 +1687,7 @@ static IEnumerable BuildMusicKnowledgeDocument .OrderByDescending(row => MusicSignalScore(row)) .ToArray(); var failed = rows.Where(row => row.Verdict.Equals("render-failed", StringComparison.OrdinalIgnoreCase)).ToArray(); + var collapsed = rows.Where(row => row.ModeCollapseRisk >= .45f).ToArray(); yield return new MusicProductionKnowledgeDocument( "music-production-quality-standard-v1", @@ -1695,6 +1698,7 @@ static IEnumerable BuildMusicKnowledgeDocument "Fresh agents should retrieve role owners, transfer rules, failure modes, and AquaSynth patterns before writing patches. New runs must add compact, evidence-backed documents, not whole spectrogram dumps or undigested trial chatter.", [ "Admit a candidate as reusable knowledge only when it renders, has explicit instrument-role ownership, and states what production job it owns.", + "Reject one-phrase-plus-texture arrangements as failure pressure; continuity across the clip is part of the production contract.", "Prefer compact role abstractions over one-off target mimicry; a reusable owner sentence is more valuable than a small metric bump.", "Keep failed renders as failure-mode pressure only; do not let syntax wreckage dominate retrieval.", "Store full artifacts on disk and cite them; CultCache curriculum records should carry bounded summaries, metrics, and transfer rules." @@ -1707,14 +1711,16 @@ static IEnumerable BuildMusicKnowledgeDocument ], [ "Single-file stores or raw spectrogram payloads are not curriculum; distill to paged CultCache records.", - "Metric winners with unclear role ownership should be pressure, not doctrine." + "Metric winners with unclear role ownership should be pressure, not doctrine.", + "High cosine with high mode-collapse risk is a trap: it copies a moment, not a song." ], rows.Select(row => row.TrialId).Take(24).ToArray(), rendered.Select(row => row.CandidateId).Distinct(StringComparer.OrdinalIgnoreCase).Take(24).ToArray(), [ new SpeechScoreMetric("input_trial_count", rows.Length, 0), new SpeechScoreMetric("rendered_trial_count", rendered.Length, 0), - new SpeechScoreMetric("render_failed_count", failed.Length, 0) + new SpeechScoreMetric("render_failed_count", failed.Length, 0), + new SpeechScoreMetric("mode_collapsed_count", collapsed.Length, 0) ], [artifactRoot], created); @@ -1740,7 +1746,7 @@ static IEnumerable BuildMusicKnowledgeDocument $"music-candidate-{SafeName(row.CandidateId)}-{StableUuid(row.TrialId)[..8]}", "candidate-pattern", row.CandidateId, - row.LogMelCosine >= .12f || row.AudioScore >= .22f ? "strong-pressure" : "weak-pressure", + row.ModeCollapseRisk >= .45f ? "failure-pressure" : row.LogMelCosine >= .12f || row.AudioScore >= .22f ? "strong-pressure" : "weak-pressure", $"Candidate `{row.CandidateId}` owns a reusable rendered song-pattern witness for `{row.ReferenceId}`.", CandidateKnowledgeSummary(row, profile, analysisExcerpt), CandidateTransferRules(row, profile), @@ -1848,6 +1854,7 @@ static IEnumerable MusicFailureDocuments( string created, string artifactRoot) { + var collapsed = all.Where(row => row.ModeCollapseRisk >= .45f).ToArray(); yield return new MusicProductionKnowledgeDocument( "music-failure-rendered-syntax-pressure", "failure-mode", @@ -1870,6 +1877,40 @@ static IEnumerable MusicFailureDocuments( ], [artifactRoot], created); + + if (collapsed.Length == 0) + { + yield break; + } + + yield return new MusicProductionKnowledgeDocument( + "music-failure-mode-collapse-pressure", + "failure-mode", + "one-phrase-plus-texture candidates are not song exemplars", + "failure-pressure", + "The mode-collapse failure record owns evidence that a rendered candidate can satisfy static audio metrics while failing song continuity.", + $"This run produced {collapsed.Length} official candidates with mode-collapse risk >= 0.45. These candidates transfer only as negative pressure unless their abstractions explicitly add later motifs, section changes, or eventful motion.", + [ + "Do not promote candidates that spend most musical energy in the first second and coast on low-motion texture.", + "Require motif mutations or distinct musical events after seconds 2, 4, 6, and 8 for ten-second curriculum clips.", + "Use motion coverage and first-second energy share beside cosine; similarity to a moment is not similarity to the song." + ], + ["mode_collapse_risk", "candidate_motion_coverage", "candidate_first_second_energy_share", "motif mutation", "section event"], + [ + "front-loaded musical energy", + "low-motion pink-noise or pad tail", + "high cosine that copies only the opening phrase" + ], + collapsed.Select(row => row.TrialId).Take(20).ToArray(), + collapsed.Select(row => row.CandidateId).Distinct(StringComparer.OrdinalIgnoreCase).Take(20).ToArray(), + [ + new SpeechScoreMetric("mode_collapsed_count", collapsed.Length, 0), + new SpeechScoreMetric("input_trial_count", all.Count, 0), + new SpeechScoreMetric("max_mode_collapse_risk", collapsed.Select(row => row.ModeCollapseRisk).DefaultIfEmpty(0).Max(), 0), + new SpeechScoreMetric("mean_first_second_energy_share", collapsed.Select(row => row.CandidateFirstSecondEnergyShare).DefaultIfEmpty(0).Average(), 0) + ], + ExistingPaths(collapsed.Select(row => row.SummaryPath).Concat([artifactRoot]).Take(24)), + created); } static IEnumerable MusicAbstractionLedgerDocuments(string artifactRoot, string created) @@ -2060,7 +2101,7 @@ static IEnumerable ReadSongSummaryRows(string path) return column >= 0 && column < parts.Length ? parts[column] : ""; } - float Float(string name) => float.TryParse(Value(name), NumberStyles.Float, CultureInfo.InvariantCulture, out var value) ? value : 0; + float Float(string name, float fallback = 0) => float.TryParse(Value(name), NumberStyles.Float, CultureInfo.InvariantCulture, out var value) ? value : fallback; yield return new SongSummaryRow( Value("trial_id"), Value("candidate_id"), @@ -2079,6 +2120,15 @@ static IEnumerable ReadSongSummaryRows(string path) Float("instrument_voice_syrinx"), Float("instrument_drum_subtractive"), Float("instrument_pad_additive"), + Float("candidate_active_coverage", 1), + Float("active_coverage_ratio", 1), + Float("candidate_motion_coverage", .75f), + Float("motion_coverage_ratio", 1), + Float("candidate_first_second_energy_share"), + Float("first_second_energy_excess"), + Float("candidate_tail_energy_share", .2f), + Float("tail_energy_ratio", 1), + Float("mode_collapse_risk"), path); } } @@ -2088,6 +2138,8 @@ static float MusicSignalScore(SongSummaryRow row) => + row.AudioScore + row.MusicalInstrumentScore * .35f - row.ChipDistressRisk * .5f + - row.ModeCollapseRisk * .9f + - MathF.Max(0, .55f - row.CandidateMotionCoverage) * .35f - (row.Verdict.Equals("render-failed", StringComparison.OrdinalIgnoreCase) ? 100f : 0f); static bool IsOfficialMusicArtifactPath(string path) @@ -2118,13 +2170,22 @@ static SpeechScoreMetric[] RowMetrics(SongSummaryRow row) => new("articulation_score", row.ArticulationScore, 1), new("musical_instrument_score", row.MusicalInstrumentScore, 1), new("chip_distress_risk", row.ChipDistressRisk, 1), + new("candidate_active_coverage", row.CandidateActiveCoverage, 1), + new("active_coverage_ratio", row.ActiveCoverageRatio, 1), + new("candidate_motion_coverage", row.CandidateMotionCoverage, 1), + new("motion_coverage_ratio", row.MotionCoverageRatio, 1), + new("candidate_first_second_energy_share", row.CandidateFirstSecondEnergyShare, 1), + new("first_second_energy_excess", row.FirstSecondEnergyExcess, 1), + new("candidate_tail_energy_share", row.CandidateTailEnergyShare, 1), + new("tail_energy_ratio", row.TailEnergyRatio, 1), + new("mode_collapse_risk", row.ModeCollapseRisk, 1), new("rms_ratio", row.RmsRatio, 0), new("centroid_ratio", row.CentroidRatio, 0), new("zero_crossing_ratio", row.ZeroCrossingRatio, 0) ]; static string CandidateKnowledgeSummary(SongSummaryRow row, SongInstrumentProfile profile, string analysisExcerpt) => - $"Rendered candidate `{row.CandidateId}` against `{row.ReferenceId}` with cosine {row.LogMelCosine:0.######}, score {row.AudioScore:0.######}, instrument score {row.MusicalInstrumentScore:0.###}, chip risk {row.ChipDistressRisk:0.###}. Instrument profile: {profile.Summary}. Candidate analysis excerpt: {TrimForReport(analysisExcerpt, 900)}"; + $"Rendered candidate `{row.CandidateId}` against `{row.ReferenceId}` with cosine {row.LogMelCosine:0.######}, score {row.AudioScore:0.######}, instrument score {row.MusicalInstrumentScore:0.###}, chip risk {row.ChipDistressRisk:0.###}, motion coverage {row.CandidateMotionCoverage:0.###}, first-second energy share {row.CandidateFirstSecondEnergyShare:0.###}, collapse risk {row.ModeCollapseRisk:0.###}. Instrument profile: {profile.Summary}. Candidate analysis excerpt: {TrimForReport(analysisExcerpt, 900)}"; static string[] CandidateTransferRules(SongSummaryRow row, SongInstrumentProfile profile) { @@ -2134,6 +2195,7 @@ static string[] CandidateTransferRules(SongSummaryRow row, SongInstrumentProfile if (profile.HasAdditivePad) rules.Add("Transfer the bed role as additive/PAD harmonic material with slow control motion."); if (profile.HasTexture) rules.Add("Transfer texture as a shaped role with band limits, gates, and motion."); if (row.RmsRatio < .35f) rules.Add("Candidate is quiet against the reference; normalize loudness after role selection."); + if (row.ModeCollapseRisk >= .45f) rules.Add("Do not transfer the arrangement as-is: it concentrates too much musical action near the opening and must be rewritten with distributed sections."); if (rules.Count == 0) rules.Add("Use as weak pressure only; demand a clearer role owner before promoting this pattern."); return rules.ToArray(); } @@ -2153,6 +2215,9 @@ static string[] CandidateFailureModes(SongSummaryRow row) if (row.RmsRatio < .35f) modes.Add("under-loud candidate can hide arrangement ideas behind weak level"); if (row.ZeroCrossingRatio > 8f) modes.Add("excess zero-crossing ratio suggests noisy or too-bright material"); if (row.ChipDistressRisk > 0) modes.Add("chip-distress risk must be justified as deliberate style before transfer"); + if (row.ModeCollapseRisk >= .45f) modes.Add("mode collapse: short opening phrase followed by low-motion texture or noise"); + if (row.CandidateFirstSecondEnergyShare >= .35f) modes.Add("front-loaded energy; require later motifs or section changes before promotion"); + if (row.CandidateMotionCoverage > 0 && row.CandidateMotionCoverage < .45f) modes.Add("low musical motion coverage across the ten-second target"); return modes.Count == 0 ? ["still pressure, not mastered parity"] : modes.ToArray(); } @@ -2470,6 +2535,11 @@ static IEnumerable TransferRulesFrom(IReadOnlyList selec yield return "Normalize loudness after role selection; high cosine can still hide bad RMS balance."; } + if (selected.Any(result => MetricValue(result, "mode_collapse_risk") >= .45f)) + { + yield return "Reject one-phrase-plus-texture arrangements as reusable song knowledge; distribute motifs, rhythmic events, and harmonic motion across the full clip."; + } + yield return "Use `texture` for background/recording noise and reserve raw `wave=noise` for short gated transients with narrow filters."; yield return "When the scene wants a singing or creature voice, start with a syrinx/acoustic voice role and modulate pressure/opening/radiation before falling back to ordinary formant oscillators."; yield return "When the scene wants a bed or pad, start with additive/PAD banks (`layer`, `harmonics`, `spectrum`) instead of stacked simple waves."; @@ -2504,6 +2574,12 @@ static IEnumerable FailurePatternsFrom(IReadOnlyList all { yield return "Low instrument-role coverage is a scoring smell: require at least one owned role such as syrinx voice, subtractive drums, additive/PAD bed, or shaped texture."; } + + var collapsed = allResults.Count(result => MetricValue(result, "mode_collapse_risk") >= .45f); + if (collapsed > 0) + { + yield return $"{collapsed} trials show mode-collapse risk: a short opening phrase followed by low-motion texture/noise. Keep them as negative pressure unless later sections carry motifs or eventful motion."; + } } static float ScoreSort(IpaTrialResult result) @@ -2511,7 +2587,9 @@ static float ScoreSort(IpaTrialResult result) var cosine = MetricValue(result, "log_mel_cosine"); var instrument = MetricValue(result, "musical_instrument_score"); var chipRisk = MetricValue(result, "chip_distress_risk"); - return cosine + instrument * .10f - chipRisk * .15f; + var collapseRisk = MetricValue(result, "mode_collapse_risk"); + var motionCoverage = MetricValue(result, "candidate_motion_coverage"); + return cosine + instrument * .10f + motionCoverage * .08f - chipRisk * .15f - collapseRisk * .25f; } static string Metric(IpaTrialResult result, string name) => @@ -3361,6 +3439,22 @@ static IReadOnlyList SongComparisonMetrics(AudioComparison co new("speech_band_ratio", comparison.Articulation.SpeechBandRatio, .05f) ]; +static IReadOnlyList SongContinuityMetrics(SongContinuityProfile profile) => +[ + new("candidate_active_coverage", profile.CandidateActiveCoverage, .08f), + new("target_active_coverage", profile.TargetActiveCoverage, 0), + new("active_coverage_ratio", profile.ActiveCoverageRatio, .08f), + new("candidate_motion_coverage", profile.CandidateMotionCoverage, .12f), + new("target_motion_coverage", profile.TargetMotionCoverage, 0), + new("motion_coverage_ratio", profile.MotionCoverageRatio, .12f), + new("candidate_first_second_energy_share", profile.CandidateFirstSecondEnergyShare, .10f), + new("target_first_second_energy_share", profile.TargetFirstSecondEnergyShare, 0), + new("first_second_energy_excess", profile.FirstSecondEnergyExcess, .10f), + new("candidate_tail_energy_share", profile.CandidateTailEnergyShare, .05f), + new("tail_energy_ratio", profile.TailEnergyRatio, .05f), + new("mode_collapse_risk", profile.ModeCollapseRisk, .15f) +]; + static IReadOnlyList SongInstrumentMetrics(SongInstrumentProfile profile) => [ new("instrument_voice_syrinx", profile.HasSyrinxVoice ? 1 : 0, .05f), @@ -3436,8 +3530,13 @@ static int CountOccurrences(string haystack, string needle) return count; } -static string SongVerdict(AudioComparison comparison, SongInstrumentProfile profile) +static string SongVerdict(AudioComparison comparison, SongInstrumentProfile profile, SongContinuityProfile continuity) { + if (continuity.ModeCollapseRisk >= .60f) + { + return "weak-mode-collapse"; + } + if (profile.ChipDistressRisk >= .75f && comparison.LogMelCosineSimilarity < .60f) { return "weak"; @@ -3456,6 +3555,179 @@ static string SongVerdict(AudioComparison comparison, SongInstrumentProfile prof return "weak"; } +static SongContinuityProfile AnalyzeSongContinuity(AudioComparison comparison) +{ + var durationSeconds = Math.Max(1e-3f, comparison.Candidate.Features.DurationSeconds); + var targetActive = ActiveCoverage(comparison.Reference.RmsEnvelope); + var candidateActive = ActiveCoverage(comparison.Candidate.RmsEnvelope); + var targetMotion = MotionCoverage(comparison.Reference.RmsEnvelope, comparison.Reference.LogMelSpectrogram); + var candidateMotion = MotionCoverage(comparison.Candidate.RmsEnvelope, comparison.Candidate.LogMelSpectrogram); + var targetFirst = EnergyShare(comparison.Reference.RmsEnvelope, comparison.Reference.Features.DurationSeconds, 0, 1); + var candidateFirst = EnergyShare(comparison.Candidate.RmsEnvelope, durationSeconds, 0, 1); + var targetTail = EnergyShare(comparison.Reference.RmsEnvelope, comparison.Reference.Features.DurationSeconds, MathF.Min(1, comparison.Reference.Features.DurationSeconds * .20f), comparison.Reference.Features.DurationSeconds); + var candidateTail = EnergyShare(comparison.Candidate.RmsEnvelope, durationSeconds, MathF.Min(1, durationSeconds * .20f), durationSeconds); + var activeRatio = SafeMetricRatio(candidateActive, targetActive); + var motionRatio = SafeMetricRatio(candidateMotion, targetMotion); + var firstExcess = MathF.Max(0, candidateFirst - MathF.Max(.20f, targetFirst * 1.6f)); + var tailRatio = SafeMetricRatio(candidateTail, targetTail); + var collapseRisk = Math.Clamp( + firstExcess * 1.6f + + MathF.Max(0, .72f - candidateMotion) * .85f + + MathF.Max(0, .65f - motionRatio) * .55f + + MathF.Max(0, .55f - candidateActive) * .40f + + MathF.Max(0, .50f - tailRatio) * .25f, + 0, + 1); + return new SongContinuityProfile( + targetActive, + candidateActive, + activeRatio, + targetMotion, + candidateMotion, + motionRatio, + targetFirst, + candidateFirst, + firstExcess, + candidateTail, + tailRatio, + collapseRisk); +} + +static float ActiveCoverage(IReadOnlyList envelope) +{ + if (envelope.Count == 0) + { + return 0; + } + + var peak = envelope.Max(); + var threshold = Math.Max(peak * .08f, 1e-5f); + return envelope.Count(value => value >= threshold) / (float)envelope.Count; +} + +static float MotionCoverage(IReadOnlyList envelope, Spectrogram spectrogram) +{ + var envelopeFlux = FrameFlux(envelope); + var spectralFlux = SpectralFluxSeries(spectrogram); + var frames = Math.Max(envelopeFlux.Length, spectralFlux.Length); + if (frames == 0) + { + return 0; + } + + var envelopeThreshold = AdaptiveFluxThreshold(envelopeFlux); + var spectralThreshold = AdaptiveFluxThreshold(spectralFlux); + var moving = 0; + for (var index = 0; index < frames; index++) + { + var env = ResampledAt(envelopeFlux, index, frames); + var spec = ResampledAt(spectralFlux, index, frames); + if (env >= envelopeThreshold || spec >= spectralThreshold) + { + moving++; + } + } + + return moving / (float)frames; +} + +static float[] FrameFlux(IReadOnlyList values) +{ + if (values.Count < 2) + { + return [0]; + } + + var flux = new float[values.Count - 1]; + for (var index = 1; index < values.Count; index++) + { + flux[index - 1] = MathF.Abs(values[index] - values[index - 1]); + } + + return flux; +} + +static float[] SpectralFluxSeries(Spectrogram spectrogram) +{ + if (spectrogram.Frames < 2) + { + return [0]; + } + + var flux = new float[spectrogram.Frames - 1]; + for (var frame = 1; frame < spectrogram.Frames; frame++) + { + var sum = 0f; + for (var band = 0; band < spectrogram.Bands; band++) + { + sum += MathF.Abs(spectrogram.At(frame, band) - spectrogram.At(frame - 1, band)); + } + + flux[frame - 1] = sum / Math.Max(1, spectrogram.Bands); + } + + return flux; +} + +static float AdaptiveFluxThreshold(IReadOnlyList values) +{ + if (values.Count == 0) + { + return float.PositiveInfinity; + } + + var sorted = values.Order().ToArray(); + var median = sorted[sorted.Length / 2]; + var high = sorted[Math.Clamp((int)MathF.Round((sorted.Length - 1) * .75f), 0, sorted.Length - 1)]; + return Math.Max(1e-6f, median + (high - median) * .35f); +} + +static float EnergyShare(IReadOnlyList envelope, float durationSeconds, float startSeconds, float endSeconds) +{ + if (envelope.Count == 0 || durationSeconds <= 0) + { + return 0; + } + + var total = envelope.Sum(value => value * value); + if (total <= float.Epsilon) + { + return 0; + } + + var start = Math.Clamp((int)MathF.Floor(startSeconds / durationSeconds * envelope.Count), 0, envelope.Count - 1); + var end = Math.Clamp((int)MathF.Ceiling(endSeconds / durationSeconds * envelope.Count), start + 1, envelope.Count); + var segment = 0f; + for (var index = start; index < end; index++) + { + segment += envelope[index] * envelope[index]; + } + + return segment / total; +} + +static float SafeMetricRatio(float candidate, float reference) => + reference <= float.Epsilon ? (candidate <= float.Epsilon ? 1 : 10) : candidate / reference; + +static float ResampledAt(IReadOnlyList values, int index, int outputLength) +{ + if (values.Count == 0) + { + return 0; + } + + if (values.Count == 1 || outputLength <= 1) + { + return values[0]; + } + + var position = index / (float)(outputLength - 1) * (values.Count - 1); + var left = Math.Clamp((int)MathF.Floor(position), 0, values.Count - 1); + var right = Math.Clamp(left + 1, 0, values.Count - 1); + var t = position - left; + return values[left] * (1 - t) + values[right] * t; +} + static float SongConfidence(IReadOnlyList metrics) { var cosine = metrics.FirstOrDefault(metric => metric.Name == "log_mel_cosine")?.Value ?? 0; @@ -3463,8 +3735,8 @@ static float SongConfidence(IReadOnlyList metrics) return Math.Clamp((cosine + score) * .5f, 0, 1); } -static string SongEvaluationSentence(SongChallenge challenge, AudioComparison comparison, string verdict, SongInstrumentProfile profile) => - $"Song snippet `{challenge.ChallengeId}` is `{verdict}`: logMelCosine={comparison.LogMelCosineSimilarity:0.0000}, logMelDistance={comparison.LogMelDistance:0.0000}, score={comparison.Score:0.0000}, rmsRatio={comparison.RmsRatio:0.0000}, centroidRatio={comparison.CentroidRatio:0.0000}. Instrument profile: {profile.Summary}."; +static string SongEvaluationSentence(SongChallenge challenge, AudioComparison comparison, string verdict, SongInstrumentProfile profile, SongContinuityProfile continuity) => + $"Song snippet `{challenge.ChallengeId}` is `{verdict}`: logMelCosine={comparison.LogMelCosineSimilarity:0.0000}, logMelDistance={comparison.LogMelDistance:0.0000}, score={comparison.Score:0.0000}, rmsRatio={comparison.RmsRatio:0.0000}, centroidRatio={comparison.CentroidRatio:0.0000}, activeCoverage={continuity.CandidateActiveCoverage:0.0000}, motionCoverage={continuity.CandidateMotionCoverage:0.0000}, firstSecondEnergyShare={continuity.CandidateFirstSecondEnergyShare:0.0000}, modeCollapseRisk={continuity.ModeCollapseRisk:0.0000}. Instrument profile: {profile.Summary}."; static string SongAnalysisReport(SongChallenge challenge, AudioAnalysis analysis) { @@ -3571,7 +3843,7 @@ static string SongChallengeReport(SongChallenge challenge) => whitened_spectral_autocorr_csv: `{challenge.Artifacts?.WhitenedSpectralAutocorrCsv ?? ""}` """; -static string SongComparisonReport(SongChallenge challenge, IpaTrialScriptCandidate candidate, AudioComparison comparison, string verdict, SongInstrumentProfile profile) => +static string SongComparisonReport(SongChallenge challenge, IpaTrialScriptCandidate candidate, AudioComparison comparison, string verdict, SongInstrumentProfile profile, SongContinuityProfile continuity) => $""" candidate={candidate.CandidateId} challenge={challenge.ChallengeId} @@ -3585,6 +3857,15 @@ static string SongComparisonReport(SongChallenge challenge, IpaTrialScriptCandid zeroCrossingRatio={comparison.ZeroCrossingRatio:0.######} articulation={comparison.Articulation.ArticulationScore:0.######} speechBandRatio={comparison.Articulation.SpeechBandRatio:0.######} + candidateActiveCoverage={continuity.CandidateActiveCoverage:0.######} + activeCoverageRatio={continuity.ActiveCoverageRatio:0.######} + candidateMotionCoverage={continuity.CandidateMotionCoverage:0.######} + motionCoverageRatio={continuity.MotionCoverageRatio:0.######} + candidateFirstSecondEnergyShare={continuity.CandidateFirstSecondEnergyShare:0.######} + firstSecondEnergyExcess={continuity.FirstSecondEnergyExcess:0.######} + candidateTailEnergyShare={continuity.CandidateTailEnergyShare:0.######} + tailEnergyRatio={continuity.TailEnergyRatio:0.######} + modeCollapseRisk={continuity.ModeCollapseRisk:0.######} instrumentVoiceSyrinx={(profile.HasSyrinxVoice ? 1 : 0)} instrumentDrumSubtractive={(profile.HasSubtractiveDrums ? 1 : 0)} instrumentPadAdditive={(profile.HasAdditivePad ? 1 : 0)} @@ -3596,7 +3877,7 @@ static string SongComparisonReport(SongChallenge challenge, IpaTrialScriptCandid static string SongSummaryCsv(IReadOnlyList results) { var builder = new StringBuilder(); - builder.AppendLine("trial_id,candidate_id,reference_id,verdict,log_mel_cosine,log_mel_distance,audio_score,envelope_distance,rms_ratio,centroid_ratio,zero_crossing_ratio,articulation_score,musical_instrument_score,chip_distress_risk,instrument_voice_syrinx,instrument_drum_subtractive,instrument_pad_additive"); + builder.AppendLine("trial_id,candidate_id,reference_id,verdict,log_mel_cosine,log_mel_distance,audio_score,envelope_distance,rms_ratio,centroid_ratio,zero_crossing_ratio,articulation_score,musical_instrument_score,chip_distress_risk,instrument_voice_syrinx,instrument_drum_subtractive,instrument_pad_additive,candidate_active_coverage,active_coverage_ratio,candidate_motion_coverage,motion_coverage_ratio,candidate_first_second_energy_share,first_second_energy_excess,candidate_tail_energy_share,tail_energy_ratio,mode_collapse_risk"); foreach (var result in results) { builder.Append(Escape(result.TrialId)).Append(','); @@ -3615,7 +3896,16 @@ static string SongSummaryCsv(IReadOnlyList results) builder.Append(Metric(result, "chip_distress_risk")).Append(','); builder.Append(Metric(result, "instrument_voice_syrinx")).Append(','); builder.Append(Metric(result, "instrument_drum_subtractive")).Append(','); - builder.AppendLine(Metric(result, "instrument_pad_additive")); + builder.Append(Metric(result, "instrument_pad_additive")).Append(','); + builder.Append(Metric(result, "candidate_active_coverage")).Append(','); + builder.Append(Metric(result, "active_coverage_ratio")).Append(','); + builder.Append(Metric(result, "candidate_motion_coverage")).Append(','); + builder.Append(Metric(result, "motion_coverage_ratio")).Append(','); + builder.Append(Metric(result, "candidate_first_second_energy_share")).Append(','); + builder.Append(Metric(result, "first_second_energy_excess")).Append(','); + builder.Append(Metric(result, "candidate_tail_energy_share")).Append(','); + builder.Append(Metric(result, "tail_energy_ratio")).Append(','); + builder.AppendLine(Metric(result, "mode_collapse_risk")); } return builder.ToString(); @@ -3628,11 +3918,11 @@ static string SongEvaluatorReport(SongChallenge challenge, IReadOnlyList MetricValue(result, "log_mel_cosine"))) { - builder.AppendLine(CultureInfo.InvariantCulture, $"| {result.CandidateId} | {result.Verdict} | {MetricValue(result, "log_mel_cosine"):0.######} | {MetricValue(result, "audio_score"):0.######} | {MetricValue(result, "musical_instrument_score"):0.######} | {MetricValue(result, "chip_distress_risk"):0.######} | {MetricValue(result, "envelope_distance"):0.######} | {MetricValue(result, "rms_ratio"):0.######} | {MetricValue(result, "centroid_ratio"):0.######} |"); + builder.AppendLine(CultureInfo.InvariantCulture, $"| {result.CandidateId} | {result.Verdict} | {MetricValue(result, "log_mel_cosine"):0.######} | {MetricValue(result, "audio_score"):0.######} | {MetricValue(result, "musical_instrument_score"):0.######} | {MetricValue(result, "chip_distress_risk"):0.######} | {MetricValue(result, "candidate_motion_coverage"):0.######} | {MetricValue(result, "candidate_first_second_energy_share"):0.######} | {MetricValue(result, "mode_collapse_risk"):0.######} | {MetricValue(result, "rms_ratio"):0.######} | {MetricValue(result, "centroid_ratio"):0.######} |"); } return builder.ToString(); @@ -3828,6 +4118,20 @@ sealed record SongInstrumentProfile( float ChipDistressRisk, string Summary); +sealed record SongContinuityProfile( + float TargetActiveCoverage, + float CandidateActiveCoverage, + float ActiveCoverageRatio, + float TargetMotionCoverage, + float CandidateMotionCoverage, + float MotionCoverageRatio, + float TargetFirstSecondEnergyShare, + float CandidateFirstSecondEnergyShare, + float FirstSecondEnergyExcess, + float CandidateTailEnergyShare, + float TailEnergyRatio, + float ModeCollapseRisk); + sealed record SongSummaryRow( string TrialId, string CandidateId, @@ -3846,6 +4150,15 @@ sealed record SongSummaryRow( float InstrumentVoiceSyrinx, float InstrumentDrumSubtractive, float InstrumentPadAdditive, + float CandidateActiveCoverage, + float ActiveCoverageRatio, + float CandidateMotionCoverage, + float MotionCoverageRatio, + float CandidateFirstSecondEnergyShare, + float FirstSecondEnergyExcess, + float CandidateTailEnergyShare, + float TailEnergyRatio, + float ModeCollapseRisk, string SummaryPath); sealed record MusicKnowledgeHit( diff --git a/tools/run-song-snippet-trial-swarm.ps1 b/tools/run-song-snippet-trial-swarm.ps1 index c46e40d..fae5479 100644 --- a/tools/run-song-snippet-trial-swarm.ps1 +++ b/tools/run-song-snippet-trial-swarm.ps1 @@ -172,6 +172,9 @@ function Write-PassPlaylist { $candidateId = $row.candidate_id $candidateWav = Join-Path $batchDirectory (Join-Path "song-candidates" (Join-Path $candidateId "audio/candidate.wav")) $referenceWav = Join-Path (Split-Path -Parent $batchDirectory) "reference.wav" + $motionCoverage = if ([string]::IsNullOrWhiteSpace($row.candidate_motion_coverage)) { 0 } else { [double]$row.candidate_motion_coverage } + $firstSecondEnergyShare = if ([string]::IsNullOrWhiteSpace($row.candidate_first_second_energy_share)) { 1 } else { [double]$row.candidate_first_second_energy_share } + $modeCollapseRisk = if ([string]::IsNullOrWhiteSpace($row.mode_collapse_risk)) { 1 } else { [double]$row.mode_collapse_risk } $rows += [pscustomobject]@{ CandidateId = $candidateId TrialId = $row.trial_id @@ -181,6 +184,9 @@ function Write-PassPlaylist { AudioScore = [double]$row.audio_score RmsRatio = [double]$row.rms_ratio CentroidRatio = [double]$row.centroid_ratio + MotionCoverage = $motionCoverage + FirstSecondEnergyShare = $firstSecondEnergyShare + ModeCollapseRisk = $modeCollapseRisk CandidateWav = $candidateWav ReferenceWav = $referenceWav SummaryPath = $summaryFile.FullName @@ -189,7 +195,7 @@ function Write-PassPlaylist { } $playable = @($rows | Where-Object { Test-Path -LiteralPath $_.CandidateWav }) - $ranked = @($playable | Sort-Object -Property @{ Expression = "LogMelCosine"; Descending = $true }, @{ Expression = "AudioScore"; Descending = $true } | Select-Object -First $TopCount) + $ranked = @($playable | Sort-Object -Property @{ Expression = "ModeCollapseRisk"; Descending = $false }, @{ Expression = "MotionCoverage"; Descending = $true }, @{ Expression = "LogMelCosine"; Descending = $true }, @{ Expression = "AudioScore"; Descending = $true } | Select-Object -First $TopCount) $skipped = @($rows | Where-Object { -not (Test-Path -LiteralPath $_.CandidateWav) }) $playlistPath = Join-Path $PassDirectory "top-scoring-candidates.m3u" $reportPath = Join-Path $PassDirectory "top-scoring-candidates.md" @@ -203,11 +209,11 @@ function Write-PassPlaylist { $rank = 1 foreach ($row in $ranked) { - $title = "#{0} {1} cosine={2:0.######} score={3:0.######} verdict={4}" -f $rank, $row.CandidateId, $row.LogMelCosine, $row.AudioScore, $row.Verdict + $title = "#{0} {1} collapse={2:0.######} motion={3:0.######} cosine={4:0.######} score={5:0.######} verdict={6}" -f $rank, $row.CandidateId, $row.ModeCollapseRisk, $row.MotionCoverage, $row.LogMelCosine, $row.AudioScore, $row.Verdict $playlist.Add("#EXTINF:-1,$title") $playlist.Add($row.CandidateWav) - $report.Add(('- {0}. `{1}` / verdict `{2}` / cosine `{3:0.######}` / score `{4:0.######}` / rms `{5:0.######}` / centroid `{6:0.######}`' -f $rank, $row.CandidateId, $row.Verdict, $row.LogMelCosine, $row.AudioScore, $row.RmsRatio, $row.CentroidRatio)) + $report.Add(('- {0}. `{1}` / verdict `{2}` / collapse `{3:0.######}` / motion `{4:0.######}` / first-second `{5:0.######}` / cosine `{6:0.######}` / score `{7:0.######}` / rms `{8:0.######}` / centroid `{9:0.######}`' -f $rank, $row.CandidateId, $row.Verdict, $row.ModeCollapseRisk, $row.MotionCoverage, $row.FirstSecondEnergyShare, $row.LogMelCosine, $row.AudioScore, $row.RmsRatio, $row.CentroidRatio)) $report.Add((' - candidate: `{0}`' -f $row.CandidateWav)) $report.Add((' - reference: `{0}`' -f $row.ReferenceWav)) $report.Add((' - summary: `{0}`' -f $row.SummaryPath)) @@ -217,8 +223,8 @@ function Write-PassPlaylist { if ($skipped.Count -gt 0) { $report.Add("") $report.Add("## Skipped Non-Playable Candidates") - foreach ($row in $skipped | Sort-Object -Property @{ Expression = "LogMelCosine"; Descending = $true }, @{ Expression = "AudioScore"; Descending = $true }) { - $report.Add(('- `{0}` / verdict `{1}` / cosine `{2:0.######}` / score `{3:0.######}` / expected candidate: `{4}`' -f $row.CandidateId, $row.Verdict, $row.LogMelCosine, $row.AudioScore, $row.CandidateWav)) + foreach ($row in $skipped | Sort-Object -Property @{ Expression = "ModeCollapseRisk"; Descending = $false }, @{ Expression = "MotionCoverage"; Descending = $true }, @{ Expression = "LogMelCosine"; Descending = $true }, @{ Expression = "AudioScore"; Descending = $true }) { + $report.Add(('- `{0}` / verdict `{1}` / collapse `{2:0.######}` / motion `{3:0.######}` / cosine `{4:0.######}` / score `{5:0.######}` / expected candidate: `{6}`' -f $row.CandidateId, $row.Verdict, $row.ModeCollapseRisk, $row.MotionCoverage, $row.LogMelCosine, $row.AudioScore, $row.CandidateWav)) } } @@ -421,8 +427,10 @@ Self-iteration loop: - Each attempt must write a candidate under `$iterationRoot/attempt-NN//patch`, run the scoring worker against that one target, inspect the rendered candidate evidence, and revise the next attempt. - Scoring command pattern: `dotnet run --project "$workerProject" -- song-score --patch-root "" --challenge "" --artifact-root "" --batch-id "--attempt-NN" --store "/iteration-results.cc" --hypothesizer "$agentId-iteration"` -- After each score, read the attempt's `evaluator-report.md`, `summary.csv`, `audio/comparison.txt`, `audio/candidate-analysis.md`, `audio/candidate-logmel-band-stats.csv`, `audio/candidate-rms-envelope-autocorr.csv`, and `audio/candidate-whitened-spectral-autocorr.csv`. +- After each score, read the attempt's `evaluator-report.md` and `summary.csv`. If the attempt rendered, also read `audio/comparison.txt`, `audio/candidate-analysis.md`, `audio/candidate-logmel-band-stats.csv`, `audio/candidate-rms-envelope-autocorr.csv`, and `audio/candidate-whitened-spectral-autocorr.csv`. If the attempt is `render-failed`, do not try to read missing candidate audio analysis files; treat the parse/lowering/render failure as negative syntax evidence and fix that before changing the composition. - Use those candidate-side facts the same way you use target facts: compare spectral bands, envelope autocorrelation, whitened spectral autocorrelation, centroid/RMS, instrument-role metrics, and chip-distress risk. Move timings, filters, gains, sources, roles, and patterns between attempts. +- The evaluator records song-continuity metrics: `candidate_motion_coverage`, `motion_coverage_ratio`, `candidate_first_second_energy_share`, `first_second_energy_excess`, `candidate_tail_energy_share`, and `mode_collapse_risk`. A patch that plays a short phrase in the first second and then coasts on low-motion texture/noise is failed song evidence even if its RMS or log-mel score looks tolerable. +- If an attempt has `mode_collapse_risk >= .45`, the next attempt must add later-section motifs or eventful motion after seconds 2, 4, 6, and 8. Do not merely raise the noise bed or pad gain; that is hiding, not composing. - Keep intermediate candidates inside `$iterationRoot`; publish only one final `.aqua` per target to the official target patch output directory. Reusable abstraction mining: @@ -489,6 +497,7 @@ Evidence contract: - Write $agentDir/hypotheses.md with: per-target reference features, shared cross-target invariants, cited analysis artifacts, tempo/rhythm plan, register/scale plan, scene voices, synthesis owners, invented instrument roles, expected metric movement on both targets, known risks. - Write exactly one `.aqua` file under each target patch output directory named `____.aqua`. - Include at least three scene voices or layers in the patch: a primary voice/body, a rhythmic drum/transient/noise role, and a pad/bed/recording-color role. Prefer syrinx for the primary voice when it is voice-like, subtractive for the drum/transient, and additive/PAD or shaped texture for the bed. +- The ten-second clip must have full-form continuity: at least four distinct musical events or motif mutations after the first second, with audible activity distributed across the beginning, middle, and ending. A one-second riff followed by nine seconds of gently textured pink noise is mode collapse and should not be submitted. - Background/recording-color layers must use `texture` or an equivalently gated and band-shaped explicit voice; a static full-duration white-noise bed is failed evidence. - Make each patch duration/gates cover its matching target duration from the challenge report.