Make music knowledge store cumulative

This commit is contained in:
Meta Vens 2026-06-01 00:30:48 +01:00
commit 554a74e2d4
5 changed files with 250 additions and 56 deletions

View file

@ -0,0 +1,103 @@
# Music Intelligence Source Map For AquaSynth
kind: source-map
tier: current-source-doctrine
topic: external music intelligence signals for song practice targets
## Owner
This source map owns how AquaSynth should use external genre, mood, tempo, key,
and audio-feature services while training song agents. External metadata is
context and contrast, not authority over the local reference audio.
## Current Landscape
- Every Noise at Once is reachable as a genre/listening-map snapshot, but it is
no longer a maintained live discovery organ. Use it for genre vocabulary,
neighborhood clues, and artist/genre relation hypotheses, not current-release
truth.
- Echo Nest is gone. Its public API was shut down years ago after Spotify
acquired it; do not design new infrastructure around Echo Nest endpoints.
- Spotify audio features and audio analysis are deprecated/restricted for new
Web API use cases. Spotify also warns that Spotify content may not be used to
train or ingest into ML/AI models. Do not feed Spotify content or proprietary
feature payloads into CultCache training memory.
- AcousticBrainz stopped collecting new data in 2022, but its public dataset
and API remain useful for historical MusicBrainz-linked acoustic descriptors
when a recording MBID is available.
- Essentia is the strongest local/open path: it provides feature extractors and
models for genre/style, mood, instrumentation, tonality, pitch, tempo, source
separation, and tagging. Prefer local Essentia-style extraction over remote
black-box labels when we have the audio files.
- Paid or hosted APIs such as Soundcharts, Tunebat, Beatlyze, SoundStat,
MetaMagican, Kapiko, and RapidAPI-hosted track-analysis services can provide
BPM/key/mood/genre/energy hints. Treat them as optional adapters with license,
privacy, quota, and provenance checks.
## Transfer Rules
- Local audio analysis owns primary truth: waveform, log-mel bands, RMS,
autocorrelation, key/register estimates, sections, loudness, and rendered
comparison metrics.
- External metadata owns vocabulary and priors: likely genre family, mood tags,
related styles, energy/danceability labels, and production reference words.
- If an external service disagrees with the local audio, preserve both and let
the producer brief state the conflict.
- Never promote a genre label unless it changes a concrete AquaSynth decision:
tempo grid, drum pattern, bass role, harmonic language, section plan, timbre,
mix priority, or failure check.
- Every external feature should carry provider, timestamp, source track id or
file hash, confidence if available, and licensing/usage note.
## AquaSynth Integration Pattern
Recommended per-target `intelligence.md` shape:
```markdown
# Target Music Intelligence
- local_file_hash:
- source_title:
- local_features: bpm, key/root, register, active duty, spectral centroid,
flux, section-energy hints
- external_labels:
- provider:
query:
result:
confidence:
license_or_policy_note:
- production_translation:
- genre/mood clue:
- AquaSynth consequence:
- rejected assumption:
```
Recommended retrieval query additions:
```text
genre style mood energy danceability key tempo section instrumentation
EveryNoise AcousticBrainz Essentia local audio analysis production reference
```
## Failure Modes
- Treating Spotify/Echo-Nest-derived features as available public training
infrastructure in 2026.
- Copying a genre label into the prompt without translating it into composition
or production choices.
- Letting remote labels override local audio artifacts.
- Forgetting provenance and later being unable to explain why the agent believed
a track was halftime, dark, danceable, acoustic, aggressive, or vocal-heavy.
- Using black-box API output as curriculum doctrine instead of bounded,
provider-labeled evidence.
## Source Anchors
- Every Noise at Once: https://everynoise.com/
- Spotify Web API changes, 2024-11-27:
https://developer.spotify.com/blog/2024-11-27-changes-to-the-web-api
- Spotify Get Track's Audio Features reference:
https://developer.spotify.com/documentation/web-api/reference/get-audio-features
- AcousticBrainz: https://acousticbrainz.org/
- Essentia models: https://essentia.upf.edu/models.html

View file

@ -266,3 +266,4 @@
{"timestamp":"2026-05-31T23:15:00+01:00","topic":"producer-apprenticeship-curriculum-gate","claim":"Song curriculum admission must be owned by producer evidence and audible structure, not syntax attendance. `song-score` now records studio document coverage, producer musicianship, composition section score, template-loop risk, noise-percussion risk, and Aqua gap count; verdicts can fail as `weak-producer-missing` or `weak-slop-template`. `music-distill` promotes candidate and role doctrine only from rendered rows that clear the producer gate, while slop/noise/template rows become failure pressure.","evidence":"tools/IpaTrialWorker/Program.cs; tools/run-song-snippet-trial-swarm.ps1; docs/audio-doctrine.md; state/scratch.md"}
{"timestamp":"2026-06-01T00:12:00+01:00","topic":"qdrant-music-knowledge-and-curator-pass","claim":"Music-production knowledge injection is now Qdrant-backed instead of lexical summary matching. `music-index` embeds full knowledge packets in `aquasynth_music_production_knowledge`; `music-search` defaults to Qdrant/Ollama retrieval and emits full owner, transfer rules, patterns, failure modes, and retrieved chunks. The song swarm now runs a dedicated knowledge-curator agent after each musician phase, and `music-distill` ingests `knowledge-curation.md` as curator-distillation doctrine pressure before indexing the next pass store.","evidence":"tools/IpaTrialWorker/Program.cs; tools/run-song-snippet-trial-swarm.ps1; docs/audio-doctrine.md; artifacts/parity/music-qdrant-smoke/20260601T000755/music-search.md"}
{"timestamp":"2026-06-01T00:22:00+01:00","topic":"music-production-manual-ingestion","claim":"Fresh music knowledge stores now include repo-native manuals before agent-derived pressure. `docs/music-production-knowledge/aqua-dsp-usage-manual.md` teaches AquaSynth composition/DSP lowering, and `docs/music-production-knowledge/open-music-theory-production-distill.md` distills open music theory and digital production sources into actionable AquaSynth patterns. `music-distill` ingests these as `manual` documents, and Qdrant search retrieves them as full context packets.","evidence":"docs/music-production-knowledge/aqua-dsp-usage-manual.md; docs/music-production-knowledge/open-music-theory-production-distill.md; tools/IpaTrialWorker/Program.cs; artifacts/parity/manual-knowledge-smoke/20260601T001932/music-search.md"}
{"timestamp":"2026-06-01T00:31:00+01:00","topic":"cumulative-music-knowledge-and-external-signal-map","claim":"The song swarm now owns one cumulative loop-level music-production `.cc`: it seeds manuals once, indexes them before pass one, then appends pass-learned worker/curator evidence with `music-distill --learned-only true --include-manuals false`. Added a current music-intelligence source map: Every Noise is static genre-vocabulary context, Echo Nest is gone, Spotify audio features are restricted/deprecated and not a training substrate, AcousticBrainz is historical, and local Essentia-style extraction should own primary audio signal.","evidence":"tools/run-song-snippet-trial-swarm.ps1; tools/IpaTrialWorker/Program.cs; docs/music-production-knowledge/music-intelligence-source-map.md; artifacts/parity/song-snippet-swarms/20260531T232719334/music-production-knowledge-seed.md; artifacts/parity/manual-knowledge-smoke/20260601T002939/report.md"}

View file

@ -1983,3 +1983,18 @@ Next likely slice:
produced a 16-document knowledge store with 2 manuals, indexed it into
Qdrant, and `music-search` retrieved the AquaSynth DSP manual with
`retrieval: qdrant-ollama`.
2026-06-01 cumulative music knowledge store correction:
- User caught that `music-distill` should not reseed theory/manual documents
after every pass. The swarm now creates one loop-level
`music-production-knowledge.cc`, seeds it once with manuals, indexes that
base store, and keeps using that store for all pass retrieval.
- After each pass, `music-distill` runs with `--input-store` equal to the
loop-level store, `--learned-only true`, and `--include-manuals false`, so it
appends/upserts worker/curator/candidate/failure evidence without rewriting
the manual spine.
- Added `music-intelligence-source-map.md` as a third manual: Every Noise is
genre-vocabulary context, Echo Nest is dead, Spotify audio features are not a
usable training substrate, AcousticBrainz is historical, and local
Essentia-style extraction is the preferred signal path.

View file

@ -422,19 +422,33 @@ static async Task MusicDistillAsync(Dictionary<string, string> options)
var artifactRoot = Required(options, "artifact-root");
var outputStore = Required(options, "output-store");
var output = Required(options, "output");
var inputStore = Value(options, "input-store", "");
var includeManuals = !Value(options, "include-manuals", "true").Equals("false", StringComparison.OrdinalIgnoreCase);
var manualsOnly = BoolValue(options, "manuals-only");
var learnedOnly = BoolValue(options, "learned-only");
var maxCandidates = int.TryParse(Value(options, "max-candidates", "16"), NumberStyles.Integer, CultureInfo.InvariantCulture, out var parsed)
? Math.Clamp(parsed, 1, 100)
: 16;
var documents = BuildMusicKnowledgeDocuments(artifactRoot, maxCandidates).ToArray();
var newDocuments = BuildMusicKnowledgeDocuments(artifactRoot, maxCandidates, includeManuals, manualsOnly, learnedOnly).ToArray();
var existingDocuments = !string.IsNullOrWhiteSpace(inputStore) && File.Exists(inputStore)
? await IpaTrialResultCultCacheStore.ReadMusicProductionKnowledgeAsync(inputStore)
: [];
var documents = existingDocuments
.Concat(newDocuments)
.GroupBy(document => document.KnowledgeId, StringComparer.OrdinalIgnoreCase)
.Select(group => group.Last())
.ToArray();
Directory.CreateDirectory(Path.GetDirectoryName(Path.GetFullPath(outputStore))!);
if (File.Exists(outputStore))
var sameStore = !string.IsNullOrWhiteSpace(inputStore) &&
Path.GetFullPath(inputStore).Equals(Path.GetFullPath(outputStore), StringComparison.OrdinalIgnoreCase);
if (!sameStore && File.Exists(outputStore))
{
File.Delete(outputStore);
}
var recordDirectory = DirectoryMessagePackBackingStore.DefaultRecordDirectoryPath(outputStore);
if (Directory.Exists(recordDirectory))
if (!sameStore && Directory.Exists(recordDirectory))
{
Directory.Delete(recordDirectory, recursive: true);
}
@ -1847,7 +1861,12 @@ static string DetailReport(string store, IpaTrialResult result)
return builder.ToString();
}
static IEnumerable<MusicProductionKnowledgeDocument> BuildMusicKnowledgeDocuments(string artifactRoot, int maxCandidates)
static IEnumerable<MusicProductionKnowledgeDocument> BuildMusicKnowledgeDocuments(
string artifactRoot,
int maxCandidates,
bool includeManuals = true,
bool manualsOnly = false,
bool learnedOnly = false)
{
if (!Directory.Exists(artifactRoot))
{
@ -1855,6 +1874,16 @@ static IEnumerable<MusicProductionKnowledgeDocument> BuildMusicKnowledgeDocument
}
var created = DateTimeOffset.UtcNow.ToString("O", CultureInfo.InvariantCulture);
if (manualsOnly)
{
foreach (var manualDocument in MusicManualDocuments(created))
{
yield return manualDocument;
}
yield break;
}
var rows = Directory.EnumerateFiles(artifactRoot, "summary.csv", SearchOption.AllDirectories)
.Where(IsOfficialMusicArtifactPath)
.SelectMany(ReadSongSummaryRows)
@ -1871,52 +1900,55 @@ static IEnumerable<MusicProductionKnowledgeDocument> BuildMusicKnowledgeDocument
var slopRejected = rows.Where(row => row.TemplateLoopRisk >= .55f || row.NoisePercussionRisk >= .55f).ToArray();
var missingProducerEvidence = rows.Where(row => row.RequiredStudioDocsPresent < .5f || row.ProducerMusicianshipScore < .45f).ToArray();
yield return new MusicProductionKnowledgeDocument(
"music-production-quality-standard-v1",
"quality-standard",
"future song curriculum admission standard",
"master-standard",
"The music curriculum store owns distilled production knowledge; raw trials and giant feature dumps are only evidence, not curriculum.",
"Fresh agents should retrieve role owners, transfer rules, failure modes, and AquaSynth patterns before writing patches. New runs must add compact, evidence-backed documents, not whole spectrogram dumps or undigested trial chatter.",
[
"Admit a candidate as reusable knowledge only when it renders, has explicit instrument-role ownership, includes producer/listening/gap evidence, and states what production job it owns.",
"Reject one-phrase-plus-texture arrangements as failure pressure; continuity across the clip is part of the production contract.",
"Require an explicit composition map: meter, progression or tonal center, instrument lanes, section events, composition-scale automation, and mix movement.",
"Prefer compact role abstractions over one-off target mimicry; a reusable owner sentence is more valuable than a small metric bump.",
"Reject stock loop skeletons and raw-noise percussion as failure pressure unless the listening journal proves target-specific role behavior.",
"Keep failed renders as failure-mode pressure only; do not let syntax wreckage dominate retrieval.",
"Store full artifacts on disk and cite them; CultCache curriculum records should carry bounded summaries, metrics, and transfer rules."
],
[
"voice-like lead -> syrinx/acoustic source ports with pressure, opening, and radiation motion",
"drums/transients -> subtractive body plus filtered noise skin plus pattern gate",
"pads/beds -> additive/PAD layer, harmonics, and spectrum banks",
"recording color -> texture role with band limits and gates, not full-duration raw noise",
"composition form -> `meter`, `sequence`, `chords`, `scale`, `pattern`, and `mix` sugar lowering to visible `param`/`curve` owners"
],
[
"Single-file stores or raw spectrogram payloads are not curriculum; distill to paged CultCache records.",
"Metric winners with unclear role ownership should be pressure, not doctrine.",
"High cosine with high mode-collapse risk is a trap: it copies a moment, not a song.",
"Instrument timbre without phrase, progression, lane entrances, and mix movement is sound design, not composition."
],
rows.Select(row => row.TrialId).Take(24).ToArray(),
admissible.Select(row => row.CandidateId).Distinct(StringComparer.OrdinalIgnoreCase).Take(24).ToArray(),
[
new SpeechScoreMetric("input_trial_count", rows.Length, 0),
new SpeechScoreMetric("rendered_trial_count", rendered.Length, 0),
new SpeechScoreMetric("admissible_curriculum_candidate_count", admissible.Length, 0),
new SpeechScoreMetric("render_failed_count", failed.Length, 0),
new SpeechScoreMetric("mode_collapsed_count", collapsed.Length, 0),
new SpeechScoreMetric("slop_template_rejected_count", slopRejected.Length, 0),
new SpeechScoreMetric("missing_producer_evidence_count", missingProducerEvidence.Length, 0)
],
[artifactRoot],
created);
foreach (var role in MusicRoleDocuments(admissible, created))
if (!learnedOnly)
{
yield return role;
yield return new MusicProductionKnowledgeDocument(
"music-production-quality-standard-v1",
"quality-standard",
"future song curriculum admission standard",
"master-standard",
"The music curriculum store owns distilled production knowledge; raw trials and giant feature dumps are only evidence, not curriculum.",
"Fresh agents should retrieve role owners, transfer rules, failure modes, and AquaSynth patterns before writing patches. New runs must add compact, evidence-backed documents, not whole spectrogram dumps or undigested trial chatter.",
[
"Admit a candidate as reusable knowledge only when it renders, has explicit instrument-role ownership, includes producer/listening/gap evidence, and states what production job it owns.",
"Reject one-phrase-plus-texture arrangements as failure pressure; continuity across the clip is part of the production contract.",
"Require an explicit composition map: meter, progression or tonal center, instrument lanes, section events, composition-scale automation, and mix movement.",
"Prefer compact role abstractions over one-off target mimicry; a reusable owner sentence is more valuable than a small metric bump.",
"Reject stock loop skeletons and raw-noise percussion as failure pressure unless the listening journal proves target-specific role behavior.",
"Keep failed renders as failure-mode pressure only; do not let syntax wreckage dominate retrieval.",
"Store full artifacts on disk and cite them; CultCache curriculum records should carry bounded summaries, metrics, and transfer rules."
],
[
"voice-like lead -> syrinx/acoustic source ports with pressure, opening, and radiation motion",
"drums/transients -> subtractive body plus filtered noise skin plus pattern gate",
"pads/beds -> additive/PAD layer, harmonics, and spectrum banks",
"recording color -> texture role with band limits and gates, not full-duration raw noise",
"composition form -> `meter`, `sequence`, `chords`, `scale`, `pattern`, and `mix` sugar lowering to visible `param`/`curve` owners"
],
[
"Single-file stores or raw spectrogram payloads are not curriculum; distill to paged CultCache records.",
"Metric winners with unclear role ownership should be pressure, not doctrine.",
"High cosine with high mode-collapse risk is a trap: it copies a moment, not a song.",
"Instrument timbre without phrase, progression, lane entrances, and mix movement is sound design, not composition."
],
rows.Select(row => row.TrialId).Take(24).ToArray(),
admissible.Select(row => row.CandidateId).Distinct(StringComparer.OrdinalIgnoreCase).Take(24).ToArray(),
[
new SpeechScoreMetric("input_trial_count", rows.Length, 0),
new SpeechScoreMetric("rendered_trial_count", rendered.Length, 0),
new SpeechScoreMetric("admissible_curriculum_candidate_count", admissible.Length, 0),
new SpeechScoreMetric("render_failed_count", failed.Length, 0),
new SpeechScoreMetric("mode_collapsed_count", collapsed.Length, 0),
new SpeechScoreMetric("slop_template_rejected_count", slopRejected.Length, 0),
new SpeechScoreMetric("missing_producer_evidence_count", missingProducerEvidence.Length, 0)
],
[artifactRoot],
created);
foreach (var role in MusicRoleDocuments(admissible, created))
{
yield return role;
}
}
foreach (var document in MusicFailureDocuments(failed, rows, created, artifactRoot))
@ -1958,9 +1990,12 @@ static IEnumerable<MusicProductionKnowledgeDocument> BuildMusicKnowledgeDocument
yield return curatorDocument;
}
foreach (var manualDocument in MusicManualDocuments(created))
if (includeManuals)
{
yield return manualDocument;
foreach (var manualDocument in MusicManualDocuments(created))
{
yield return manualDocument;
}
}
}
@ -4715,7 +4750,7 @@ static void PrintHelp()
song-score --patch-root <dir> --challenge <challenge.json> --artifact-root <dir> [--batch-id round-001] [--store <trial-results.cc>] [--hypothesizer id]
dump --store <trial-results.cc> --output <report.md>
distill --store <trial-results.cc> --output-store <distilled.cc> --output <report.md> [--min-cosine .35] [--max-results 40]
music-distill --artifact-root <song-swarm-dir> --output-store <music-knowledge.cc> --output <report.md> [--max-candidates 16]
music-distill --artifact-root <song-swarm-dir> --output-store <music-knowledge.cc> --output <report.md> [--input-store <music-knowledge.cc>] [--max-candidates 16] [--manuals-only true] [--learned-only true]
music-index --store <music-knowledge.cc> [--output <report.md>] [--force true]
music-search --store <music-knowledge.cc> --query <text> --output <report.md> [--limit 12] [--allow-lexical-fallback true]
music-show --store <music-knowledge.cc> --knowledge-id <id-or-topic> --output <detail.md>

View file

@ -267,7 +267,42 @@ Set-Content -LiteralPath (Join-Path $loopRoot "loop-index.md") -Value @"
"@
$latestDistilledStore = ""
$latestMusicKnowledgeStore = ""
$musicKnowledgeStore = Join-Path $loopRoot "music-production-knowledge.cc"
Invoke-LoggedProcess `
-FilePath "dotnet" `
-ArgumentList @(
"run",
"--project",
$workerProject,
"--",
"music-distill",
"--artifact-root",
$loopRoot,
"--output-store",
$musicKnowledgeStore,
"--output",
(Join-Path $loopRoot "music-production-knowledge-seed.md"),
"--manuals-only",
"true"
) `
-LogPath (Join-Path $logRoot "music-knowledge-seed.log")
Invoke-LoggedProcess `
-FilePath "dotnet" `
-ArgumentList @(
"run",
"--project",
$workerProject,
"--",
"music-index",
"--store",
$musicKnowledgeStore,
"--output",
(Join-Path $loopRoot "music-vector-index-seed.md"),
"--timeout-seconds",
"300"
) `
-LogPath (Join-Path $logRoot "music-knowledge-seed-index.log")
$latestMusicKnowledgeStore = $musicKnowledgeStore
for ($pass = 1; $pass -le $Passes; $pass++) {
$passId = "pass-{0:000}" -f $pass
$passDir = Join-Path $loopRoot $passId
@ -690,7 +725,6 @@ Return a short final note naming knowledge-curation.md.
-LogPath (Join-Path $logRoot "$passId-distill.log")
$latestDistilledStore = $distilledStore
$musicKnowledgeStore = Join-Path $passDir "music-production-knowledge.cc"
Invoke-LoggedProcess `
-FilePath "dotnet" `
-ArgumentList @(
@ -700,13 +734,19 @@ Return a short final note naming knowledge-curation.md.
"--",
"music-distill",
"--artifact-root",
$loopRoot,
$passDir,
"--input-store",
$musicKnowledgeStore,
"--output-store",
$musicKnowledgeStore,
"--output",
(Join-Path $passDir "music-production-knowledge-report.md"),
"--max-candidates",
"40"
"40",
"--include-manuals",
"false",
"--learned-only",
"true"
) `
-LogPath (Join-Path $logRoot "$passId-music-distill.log")