GameCult/Mimir
Watch
9
0
Fork
You've already forked Mimir
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Route spectrum dashboard through spline tube surfaces

Browse source
This commit is contained in:
Meta Vens 2026-05-26 10:50:37 +01:00
commit da5030cd09
4 changed files with 84 additions and 165 deletions
Download patch file Download diff file Expand all files Collapse all files

26
docs/implementation-plan.md
View file

@ -106,13 +106,17 @@ a named invariant that the native runtime cannot protect yet.
service and can emit live sync telemetry with
`MIMIR_SYNC_TELEMETRY_SECONDS`. UI and telemetry read cached reports/states;
they do not run synchronization analysis.
- `MimirRuntime` publishes live ASIO spectrum history to Fensalir as one
`AquariumBufferFieldFrame` texture-spline reservoir surface. Direct
`AquariumSplineFrame` submission is disabled for this dashboard. The current
tuning defaults to 32,768 reservoir samples with 8,192 updates per frame and
exposes `MIMIR_SPECTRUM_RESERVOIR_SPLATS`,
`MIMIR_SPECTRUM_RESERVOIR_UPDATES`, and
`MIMIR_SPECTRUM_RESERVOIR_CANDIDATES` for budget cuts.
- `MimirRuntime` publishes live ASIO spectrum history to Fensalir as direct
`AquariumSplineFrame` tube surfaces. The spline shader owns visible coverage:
it evaluates capsule/tube distance, applies native blue-noise jitter, and
writes scene color plus temporal reservoir guide MRTs. The old
`AquariumBufferFieldFrame` / `ReservoirSplats` dashboard path is no longer
allowed to decide this surface. Current budget knobs are
`MIMIR_SPECTRUM_SPLINE_WINDOW_STRIDE`,
`MIMIR_SPECTRUM_SPLINE_BAND_STRIDE`, and
`MIMIR_SPECTRUM_SPLINE_SUBDIVISIONS`; defaults intentionally undersample the
current frame so the temporal path amortizes the surface instead of drawing
every equivalent segment every frame.
- `MimirRuntime` queues bioacoustic timeline PCM through Fensalir audio when the
active timing witness is allowed. `MimirAudioSynchronizationSettings.Mode`
selects `chirp-only`, `passive`, or `hybrid`; passive disables active
@ -245,10 +249,10 @@ a named invariant that the native runtime cannot protect yet.
with probe durations long enough to keep loopback and mic windows live.
6. Bind Fensalir UI to the synchronization hub so buffer depth, stream cadence,
source timestamps, and output settings are visible and adjustable.
7. Promote the spectrum reservoir producer from hash-jittered texture-spline
samples to native blue-noise compute sampling once Fensalir exposes the
blue-noise tile to the reservoir compute root. Keep direct spline triangles
off for this dashboard unless explicitly debugging the spline shader.
7. Tighten the spectrum spline tube temporal path until it behaves like a
production debug surface: shader-owned tube SDF coverage, native blue-noise
sample jitter, and temporal accumulation from sparse frame budgets. Do not
route this dashboard back through fractal point splats.
8. Move GPU feature extraction, fusion, material fitting, render budgeting, and
Spout2 publication into Fensalir.
9. Move mic alignment, room suppression, voice separation, spatialization, and

26
notes/current-system-map.md
View file

@ -309,18 +309,22 @@ Visual fusion belongs in Fensalir over current reservoir claims. Native capture
workers provide frames; Fensalir owns feature extraction, matching, material
fitting, render budgeting, and publication.
The live debug spectrum view is a Fensalir buffer-field reservoir surface, not
direct spline triangles. Mimir submits one `AquariumBufferFieldFrame` texture
spline program for the rolling ASIO spectrum: frequency lives on X, amplitude
on Y, history on Z, and channels stack along Y. Direct `AquariumSplineFrame`
submission is disabled for this surface so there is one visual authority.
The live debug spectrum view is a Fensalir spline-tube SDF surface. Mimir
submits rolling ASIO spectrum history as `AquariumSplineFrame` trails:
frequency lives on X, amplitude on Y, history on Z, and channels stack along
Y. `AquariumBufferFieldFrame` / fractal `ReservoirSplats` no longer own this
dashboard, because point splats were the wrong visible primitive for a surface
whose truth is a tube SDF.
The current reservoir cut defaults to 32,768 spectrum samples, 8,192 sample
updates per frame, one candidate per update, and `WorldCenterRadius=(0,0,0,1)`
so sample radii stay in spectrum-world units instead of inflating into giant
screen plates. Fensalir still uses hash/jittered compute sampling for this
producer; native blue-noise sampling is available in scene shaders but is not
yet bound into the reservoir compute root.
Fensalir's spline shader owns visible coverage. Geometry is only the
conservative segment envelope; the pixel shader evaluates capsule/tube
distance, applies native blue-noise sample jitter, writes color/metadata/control,
and emits `reservoirGuide` for the scene temporal path. Mimir intentionally
submits a sparse rotating surface budget with
`MIMIR_SPECTRUM_SPLINE_WINDOW_STRIDE`, `MIMIR_SPECTRUM_SPLINE_BAND_STRIDE`, and
`MIMIR_SPECTRUM_SPLINE_SUBDIVISIONS` so the temporal reservoir/TAA path
amortizes the surface instead of paying the full polygon-equivalent cost every
frame.
## Known Risks

196
src/Mimir.Runtime/MimirRuntime.cs
View file

@ -31,9 +31,9 @@ public sealed class MimirRuntime : IAquariumRuntime
private const float SpectrumCameraFitPadding = 1.12f;
private const float SpectrumFrustumMinimumNear = 0.01f;
private const float SpectrumSplineTubePadding = 0.18f;
private const int DefaultSpectrumReservoirSplatCount = 32768;
private const int DefaultSpectrumReservoirUpdatesPerFrame = 8192;
private const int DefaultSpectrumReservoirCandidates = 1;
private const int DefaultSpectrumSplineWindowStride = 4;
private const int DefaultSpectrumSplineBandStride = 2;
private const int DefaultSpectrumSplineSubdivisions = 1;
private readonly MimirSynchronizationHub synchronization;
private readonly MimirAudioSpectrumAnalyzer spectrumAnalyzer;
private readonly IReadOnlyList<MimirStreamSourceFactory> sourceFactories;
@ -60,6 +60,7 @@ public sealed class MimirRuntime : IAquariumRuntime
private double lastPassiveSynchronizationConfidence;
private float spectrumCameraDistanceMultiplier = SpectrumCameraDefaultDistanceMultiplier;
private float spectrumCameraAngleDegrees = SpectrumCameraDefaultAngleDegrees;
private ulong spectrumSurfaceFrameIndex;
private AquariumCameraFrustum lastSpectrumFrustum = AquariumCameraFrustum.Default;
private Vector3 lastSpectrumCameraPosition;
private Vector3 lastSpectrumCameraTarget;
@ -137,7 +138,7 @@ public sealed class MimirRuntime : IAquariumRuntime
private AquariumFrame CreateFrame()
{
var channelCount = Math.Max(1, lastAudioSpectra.Count);
var bufferFieldFrame = BuildSpectrumBufferFieldFrame();
var splineFrame = BuildSpectrumSplineFrame(spectrumSurfaceFrameIndex++);
var windowCount = Math.Max(1, spectrumHistory.Count);
var (splineMin, splineMax) = SpectrumSplineAabb(channelCount, windowCount);
var cameraTarget = (splineMin + splineMax) * 0.5f;
@ -162,8 +163,8 @@ public sealed class MimirRuntime : IAquariumRuntime
{
TraceHeightFieldSurface = false,
UseStarfieldBackground = false,
BufferFieldFrame = bufferFieldFrame,
SplineFrame = AquariumSplineFrame.Empty,
BufferFieldFrame = AquariumBufferFieldFrame.Empty,
SplineFrame = splineFrame,
});
}
@ -686,25 +687,25 @@ public sealed class MimirRuntime : IAquariumRuntime
: 96;
}
private static int ParseSpectrumReservoirSplatCount()
private static int ParseSpectrumSplineWindowStride()
{
return int.TryParse(Environment.GetEnvironmentVariable("MIMIR_SPECTRUM_RESERVOIR_SPLATS"), out var value)
? Math.Clamp(value, 1024, 524288)
: DefaultSpectrumReservoirSplatCount;
return int.TryParse(Environment.GetEnvironmentVariable("MIMIR_SPECTRUM_SPLINE_WINDOW_STRIDE"), out var value)
? Math.Clamp(value, 1, 16)
: DefaultSpectrumSplineWindowStride;
}
private static int ParseSpectrumReservoirUpdatesPerFrame()
private static int ParseSpectrumSplineBandStride()
{
return int.TryParse(Environment.GetEnvironmentVariable("MIMIR_SPECTRUM_RESERVOIR_UPDATES"), out var value)
? Math.Clamp(value, 1, 524288)
: DefaultSpectrumReservoirUpdatesPerFrame;
return int.TryParse(Environment.GetEnvironmentVariable("MIMIR_SPECTRUM_SPLINE_BAND_STRIDE"), out var value)
? Math.Clamp(value, 1, 16)
: DefaultSpectrumSplineBandStride;
}
private static int ParseSpectrumReservoirCandidates()
private static int ParseSpectrumSplineSubdivisions()
{
return int.TryParse(Environment.GetEnvironmentVariable("MIMIR_SPECTRUM_RESERVOIR_CANDIDATES"), out var value)
return int.TryParse(Environment.GetEnvironmentVariable("MIMIR_SPECTRUM_SPLINE_SUBDIVISIONS"), out var value)
? Math.Clamp(value, 1, 8)
: DefaultSpectrumReservoirCandidates;
: DefaultSpectrumSplineSubdivisions;
}
private static bool IsTruthy(string? value) =>
@ -858,7 +859,7 @@ public sealed class MimirRuntime : IAquariumRuntime
return (min, max);
}
private AquariumSplineFrame BuildSpectrumSplineFrame()
private AquariumSplineFrame BuildSpectrumSplineFrame(ulong frameIndex)
{
if (!sceneReady || spectrumHistory.Count == 0)
{
@ -876,12 +877,22 @@ public sealed class MimirRuntime : IAquariumRuntime
return AquariumSplineFrame.Empty;
}
var splines = new List<AquariumSpline3D>(windows.Length * sourceIds.Length + sourceIds.Length * 4);
var windowStride = ParseSpectrumSplineWindowStride();
var bandStride = ParseSpectrumSplineBandStride();
var subdivisions = ParseSpectrumSplineSubdivisions();
var windowPhase = (int)(frameIndex % (ulong)windowStride);
var approximateWindowCount = Math.Max(1, (windows.Length + windowStride - 1) / windowStride);
var splines = new List<AquariumSpline3D>(approximateWindowCount * sourceIds.Length + sourceIds.Length * 4);
AddSpectrumGridSplines(splines, sourceIds.Length, windows.Length);
for (var windowIndex = 0; windowIndex < windows.Length; windowIndex++)
{
var ageFromNewest = windows.Length - 1 - windowIndex;
if (windowStride > 1 && ageFromNewest != 0 && ageFromNewest % windowStride != windowPhase)
{
continue;
}
var ageT = windows.Length <= 1 ? 0.0f : ageFromNewest / (float)(windows.Length - 1);
var alpha = Math.Clamp(0.10f + MathF.Pow(1.0f - ageT, 1.35f) * 0.90f, 0.10f, 1.0f);
var z = ageFromNewest * SpectrumWindowDepthSeparation;
@ -893,155 +904,54 @@ public sealed class MimirRuntime : IAquariumRuntime
continue;
}
var points = BuildSpectrumWindowSpline(spectrum, channelIndex * SpectrumChannelSeparation, z, alpha);
var bandPhase = bandStride <= 1
? 0
: (int)((frameIndex + (ulong)channelIndex + (ulong)ageFromNewest) % (ulong)bandStride);
var points = BuildSpectrumWindowSpline(
spectrum,
channelIndex * SpectrumChannelSeparation,
z,
alpha,
bandStride,
bandPhase);
splines.Add(new AquariumSpline3D(
$"spectrum-{spectrum.SourceId}-{windowIndex}",
points,
new AquariumSplineStyle(0.018f, 1.35f, 1.0f, 0.78f, 0.22f),
CatmullRomSubdivisions: 3));
CatmullRomSubdivisions: subdivisions));
}
}
return new AquariumSplineFrame { Splines = splines };
}
private AquariumBufferFieldFrame BuildSpectrumBufferFieldFrame()
{
if (!sceneReady || spectrumHistory.Count == 0)
{
return AquariumBufferFieldFrame.Empty;
}
var windows = spectrumHistory.ToArray();
var latest = windows[^1];
var sourceIds = latest
.Select(spectrum => spectrum.SourceId)
.OrderBy(sourceId => sourceId, StringComparer.Ordinal)
.ToArray();
if (sourceIds.Length == 0)
{
return AquariumBufferFieldFrame.Empty;
}
var bandCount = latest.FirstOrDefault(spectrum => spectrum.BandDecibels.Count > 0)?.BandDecibels.Count ?? 0;
if (bandCount <= 1)
{
return AquariumBufferFieldFrame.Empty;
}
var rowCount = SpectrumHistoryWindowCount * sourceIds.Length;
var samples = new float[bandCount * rowCount];
for (var windowIndex = 0; windowIndex < windows.Length; windowIndex++)
{
var ageFromNewest = windows.Length - 1 - windowIndex;
var spectraBySource = windows[windowIndex].ToDictionary(spectrum => spectrum.SourceId, StringComparer.Ordinal);
for (var channelIndex = 0; channelIndex < sourceIds.Length; channelIndex++)
{
if (!spectraBySource.TryGetValue(sourceIds[channelIndex], out var spectrum) || spectrum.BandDecibels.Count != bandCount)
{
continue;
}
var row = channelIndex * SpectrumHistoryWindowCount + ageFromNewest;
WriteSpectrumRow(samples, row * bandCount, spectrum);
}
}
var splatCount = ParseSpectrumReservoirSplatCount();
var updatesPerFrame = Math.Clamp(ParseSpectrumReservoirUpdatesPerFrame(), 1, splatCount);
var bandStep = SpectrumWidth / Math.Max(1, bandCount - 1);
var reservoir = new AquariumFractalReservoirField
{
SplatCount = splatCount,
Depth = 1,
Seed = 0xB11FF13Du,
CandidatesPerReservoirUpdate = ParseSpectrumReservoirCandidates(),
SplatUpdatesPerFrame = updatesPerFrame,
ReservoirUpdatesPerPass = updatesPerFrame,
WorldCenterRadius = new Vector4(0.0f, 0.0f, 0.0f, 1.0f),
};
var program = new AquariumTextureSplineFieldProgram(
"asioSpectrumWaterfall",
"spectrumHistory",
AquariumTextureAxis.X,
FirstColumn: 0,
ColumnCount: rowCount,
ColumnStride: 1,
RollingWindowModulo: rowCount,
Subdivisions: 1,
Origin: new Vector3(-SpectrumWidth * 0.5f, 0.0f, 0.0f),
AxisStep: new Vector3(bandStep, 0.0f, 0.0f),
ColumnStep: new Vector3(0.0f, 0.0f, SpectrumWindowDepthSeparation),
ColumnGroupSize: SpectrumHistoryWindowCount,
ColumnGroupStep: new Vector3(0.0f, SpectrumChannelSeparation, 0.0f),
AmplitudeScale: SpectrumAmplitudeHeight,
Appearance: new AquariumSplineTubeAppearance(
new Vector4(1.0f, 0.68f, 0.12f, 2.5f),
Radius: 0.012f,
Alpha: 0.82f,
ZeroThreshold: 0.68f,
Feather: 0.08f,
TangentWeight: 1.0f,
CurvatureWeight: 0.68f,
NormalWeight: 0.34f,
DerivativeWeight: 0.88f),
ProbePolicy: new AquariumSplineTubeProbePolicy(
MaxProbeCount: updatesPerFrame,
BaseDensity: 1.0f,
MinimumVisualContribution: 0.004f,
Seed: 0xB11FF13Du),
SurfaceGraph: []);
return AquariumBufferFieldFrame.Compose(builder => builder
.Texture(
"spectrumHistory",
bandCount,
rowCount,
1,
0,
AquariumRollingModuloMode.Rows,
samples)
.TextureSplineField(program)
.Reservoir(reservoir)
.Lowering(new AquariumFieldLoweringPolicy(
AquariumFieldLoweringMode.ReservoirSplats,
MaxDirectSplines: 1,
MaxDirectControlPoints: 2,
MaxReservoirSplats: splatCount,
LodBias: 1.0f)));
}
private static void WriteSpectrumRow(float[] samples, int offset, MimirAudioSpectrumSnapshot spectrum)
{
var bands = spectrum.BandDecibels;
var floor = Math.Min(spectrum.NoiseFloorDb, bands.Min());
var ceiling = Math.Max(-24.0, bands.Max());
var span = Math.Max(18.0, ceiling - floor);
for (var index = 0; index < bands.Count; index++)
{
samples[offset + index] = (float)Math.Clamp((bands[index] - floor) / span, 0.0, 1.0);
}
}
private static IReadOnlyList<AquariumSplineVertex> BuildSpectrumWindowSpline(
MimirAudioSpectrumSnapshot spectrum,
float channelBaseY,
float z,
float alpha)
float alpha,
int bandStride,
int bandPhase)
{
var bands = spectrum.BandDecibels;
var floor = Math.Min(spectrum.NoiseFloorDb, bands.Min());
var ceiling = Math.Max(-24.0, bands.Max());
var span = Math.Max(18.0, ceiling - floor);
var points = new AquariumSplineVertex[bands.Count];
var points = new List<AquariumSplineVertex>((bands.Count + bandStride - 1) / bandStride + 2);
for (var index = 0; index < bands.Count; index++)
{
var isEdge = index == 0 || index == bands.Count - 1;
if (!isEdge && bandStride > 1 && index % bandStride != bandPhase)
{
continue;
}
var x = -SpectrumWidth * 0.5f + SpectrumWidth * index / Math.Max(1, bands.Count - 1);
var normalized = (float)Math.Clamp((bands[index] - floor) / span, 0.0, 1.0);
var y = channelBaseY + normalized * SpectrumAmplitudeHeight;
points[index] = new AquariumSplineVertex(
points.Add(new AquariumSplineVertex(
new Vector3(x, y, z),
SpectrumColor(normalized, alpha));
SpectrumColor(normalized, alpha)));
}
return points;

1
state/evidence.jsonl
View file

@ -127,3 +127,4 @@
{"ts": "2026-05-25T23:02:30+01:00", "type": "implementation", "status": "ok", "note": "Mimir local spectrum dashboard now forces Fensalir texture-spline field lowering to ReservoirSplats instead of auto direct tubes. Auto mode chose direct tubes for 160 columns, so the screen was not the final sampled reservoir. Spectrum rows now map age-from-newest explicitly: newest/front at z=0, older windows recede +Z. Live run shows 131,072 visible reservoir splats and four 192 kHz ASIO lanes."}
{"status":"ok","note":"Spectrum dashboard authority cut: Mimir no longer submits AquariumBufferFieldFrame/reservoir splats for the live audio spectrum view. The surface is direct AquariumSplineFrame tube trails only; config/mimir-spectrum-field.aquafield was deleted so the old ReservoirSplats path cannot reappear through configuration.","type":"correction","ts":"2026-05-26T10:08:18+01:00"}
{"status":"ok","note":"Corrected the previous overcut: the spectrum dashboard is back on a single AquariumBufferFieldFrame reservoir authority with direct spline triangles disabled. Defaults are 32768 reservoir samples, 8192 updates/frame, one candidate/update, and WorldCenterRadius=(0,0,0,1) to prevent sample radii from inflating into giant discs. Fensalir transparent-field splats no longer enforce the 0.035*fieldRadius minimum billboard radius. Runtime proof logs 32768 visible splats at about 3-6 ms CPU frame time and about 0.2 ms command recording. Compute sampling is still hash-jittered rather than true blue-noise texture driven.","type":"correction","ts":"2026-05-26T10:31:00+01:00"}
{"ts": "2026-05-26T10:48:57+01:00", "type": "correction", "status": "ok", "note": "Spectrum dashboard authority is direct spline-tube SDF again: Mimir submits AquariumSplineFrame trails and leaves AquariumBufferFieldFrame empty, so fractal ReservoirSplats cannot own the view. The live run artifacts/runtime/mimir-spectrum-sdf-temporal-sparse.out.log showed four 192 kHz ASIO lanes, no D3D12 fractal reservoir lines, and about 4-7 ms D3D12 CPU frames with record near 3 ms using sparse rotating spline budgets."}