diff --git a/docs/implementation-plan.md b/docs/implementation-plan.md index 2dafe69..d1ed06a 100644 --- a/docs/implementation-plan.md +++ b/docs/implementation-plan.md @@ -201,6 +201,15 @@ a named invariant that the native runtime cannot protect yet. The display feed remains derived evidence, not an independent timing owner. `Mimir.BufferSmoke --synchronized-buffer-planner-smoke` proves the aligned local-camera, Raven-display, loopback, and mic buffer shape. +- `MimirPresentationControlState` owns operator intent for the Fensalir program + surface: video feed visibility/solo/opacity/layer order, audio mute/solo/gain, + and global LUT preset selection. The `Mimir Program` UI panel is deliberately + compact and fadeable. Video choices filter production surface intents before + Fensalir composition; audio choices modify Faust gain controls and the samples + sent through Fensalir streaming DSP. LUT presets are typed postprocess state + with preset paths and strength; until the renderer grows LUT texture sampling, + preset exposure and bloom are mapped into existing `GraphicsSettings`. + `Mimir.BufferSmoke --presentation-control-smoke` proves the state owner. - `MimirRuntime` no longer submits the legacy direct `AquariumSplineFrame` spectrum dashboard. Live spectrum visualization authority is the `AquariumFieldEvidenceFrame`: Mimir declares the rolling resource, emits a diff --git a/docs/program-control-surface.md b/docs/program-control-surface.md new file mode 100644 index 0000000..f124bfc --- /dev/null +++ b/docs/program-control-surface.md @@ -0,0 +1,42 @@ +# Program Control Surface + +Mimir now exposes a compact Fensalir-hosted program panel for final presentation +intent. + +Authority map: + +- Owner: `MimirPresentationControlState` owns operator intent for program video, + audio mix, and global color preset. +- Inputs: runtime rolling buffers, aligned audio lanes, LUT preset selection, + feed visibility, feed opacity, stem mute/solo, and stem gain. +- Outputs: filtered video surface intents, Faust/native DSP gain controls, + sample gain applied before Fensalir streaming DSP, and a postprocess control + snapshot. +- Derived state: feed lists are derived from active rolling buffers. LUT preset + exposure and bloom are currently mapped into `GraphicsSettings`. +- Forbidden writers: OBS, bridge endpoints, debug readouts, and network arrival + timestamps must not choose program composition. +- Shared paths: local cameras, Raven display capture, and future network video + feeds all enter rolling buffers first, then the presentation state decides + whether they contribute to production surface intent. +- Deletion line: direct compositor shortcuts outside Fensalir FieldEvidence and + audio DSP are diagnostic only. + +The current UI lives in the `Mimir Program` panel: + +- `Video`: select feed, visible, solo, opacity, layer ordering. +- `Audio`: select stem/source, mute, solo, gain. +- `Color`: LUT preset and LUT strength. + +The shader-side LUT path is explicit state, not finished renderer authority yet. +`MimirLookupTablePreset` carries `LutPath`, strength, contrast, saturation, +temperature, and tint so Fensalir can consume the same contract when the global +post shader grows real LUT sampling. Until that renderer cut lands, Mimir maps +the preset exposure and bloom values into existing Fensalir `GraphicsSettings`. + +Smoke proof: + +```powershell +dotnet run --project .\src\Mimir.BufferSmoke\Mimir.BufferSmoke.csproj -- --presentation-control-smoke +``` + diff --git a/notes/current-system-map.md b/notes/current-system-map.md index ee3345c..6cbac34 100644 --- a/notes/current-system-map.md +++ b/notes/current-system-map.md @@ -374,6 +374,14 @@ shape is explicit: Raven controls both display pixels and an audio timing signal routed into Scarlett, so Scarlett-decoded Raven audio evidence can earn a `raven-sync` correction that is applied to the `raven-display` buffer. Network arrival timestamps remain metadata, not timing authority. +`MimirPresentationControlState` now owns the Fensalir program-control intent: +video feed visibility/solo/opacity/layer order, audio mute/solo/gain, and +global LUT preset selection. The `Mimir Program` panel exposes those controls +without giving OBS or bridge endpoints composition authority. Video controls +filter production surface intents before Fensalir composition; audio controls +modify Faust gain controls and sample gain before Fensalir streaming DSP. LUT +presets are typed postprocess state, with exposure/bloom currently mapped into +`GraphicsSettings` and LUT texture sampling left as the next renderer hook. ## Visual Fusion diff --git a/src/Mimir.BufferSmoke/Program.cs b/src/Mimir.BufferSmoke/Program.cs index aca7245..193361a 100644 --- a/src/Mimir.BufferSmoke/Program.cs +++ b/src/Mimir.BufferSmoke/Program.cs @@ -79,6 +79,11 @@ if (args.Any(arg => string.Equals(arg, "--synchronized-buffer-planner-smoke", St return RunSynchronizedBufferPlannerSmoke(); } +if (args.Any(arg => string.Equals(arg, "--presentation-control-smoke", StringComparison.OrdinalIgnoreCase))) +{ + return RunPresentationControlSmoke(); +} + if (args.Any(arg => string.Equals(arg, "--audio-stem-publication-smoke", StringComparison.OrdinalIgnoreCase))) { return RunAudioStemPublicationSmoke(); @@ -1935,6 +1940,41 @@ static int RunSynchronizedBufferPlannerSmoke() : 1; } +static int RunPresentationControlSmoke() +{ + var controls = new MimirPresentationControlState(); + var camera = NewVideoBuffer("camera-main", MimirStreamOrigin.LocalDevice, 1_000_000_000, 1280, 720, MimirVideoPixelFormat.Bgra8, 1); + var display = NewVideoBuffer("raven-display", MimirStreamOrigin.Network, 1_000_000_000, 1920, 1080, MimirVideoPixelFormat.Bgra8, 2, "raven-sync"); + var mic = NewAudioBlockBuffer("asio-ch0", 1_000_000_000, 48_000, 480, 3); + var loopback = NewAudioBlockBuffer("asio-ch2", 1_000_000_000, 48_000, 480, 4); + controls.SyncFromBuffers([camera, display, mic, loopback]); + + controls.SelectedVideoIndex = 1; + controls.SelectedVideo!.Solo = true; + controls.SelectedVideo.Opacity = 0.42f; + controls.SelectedAudioIndex = 0; + controls.SelectedAudio!.Muted = true; + controls.SelectLutPreset(2); + controls.LutStrength = 0.65f; + controls.RefreshPostprocess(); + + Console.WriteLine( + $"presentation-control-smoke video={controls.VideoFeeds.Count} audio={controls.AudioFeeds.Count} " + + $"cameraIncluded={controls.IncludesVideo("camera-main")} displayIncluded={controls.IncludesVideo("raven-display")} displayOpacity={controls.VideoOpacity("raven-display"):0.00} " + + $"micGain={controls.AudioGain("asio-ch0"):0.00} lut={controls.Postprocess.PresetId} strength={controls.Postprocess.LutStrength:0.00}"); + + return controls.VideoFeeds.Count == 2 && + controls.AudioFeeds.Count == 2 && + !controls.IncludesVideo("camera-main") && + controls.IncludesVideo("raven-display") && + Math.Abs(controls.VideoOpacity("raven-display") - 0.42f) < 0.001f && + controls.AudioGain("asio-ch0") == 0.0f && + string.Equals(controls.Postprocess.PresetId, "soft-film", StringComparison.Ordinal) && + Math.Abs(controls.Postprocess.LutStrength - 0.65f) < 0.001f + ? 0 + : 1; +} + static int RunAudioStemPublicationSmoke() { var publication = new MimirObsStemPublicationState(MimirObsPublicationConfigurations.AlignmentActuatorStemBus); diff --git a/src/Mimir.Runtime/MimirRuntime.cs b/src/Mimir.Runtime/MimirRuntime.cs index 8aa4361..134b46b 100644 --- a/src/Mimir.Runtime/MimirRuntime.cs +++ b/src/Mimir.Runtime/MimirRuntime.cs @@ -40,6 +40,7 @@ public sealed class MimirRuntime : IAquariumRuntime, IAquariumRuntimeServicesRec private readonly MimirFensalirFieldLowering fieldLowering; private readonly MimirAudioSpectrumAnalyzer spectrumAnalyzer; private readonly MimirAlignmentActuatorBank audioActuatorBank = new(); + private readonly MimirPresentationControlState presentationControls = new(); private readonly MimirObsStemPublicationState obsStemPublication = new(MimirObsPublicationConfigurations.AlignmentActuatorStemBus); private readonly MimirObsStemSharedMemoryPublisher? obsStemPublisher; private readonly IReadOnlyList sourceFactories; @@ -141,7 +142,9 @@ public sealed class MimirRuntime : IAquariumRuntime, IAquariumRuntimeServicesRec RenderDebugMode: 0, SceneExposure: 1.0f, BloomIntensity: 0.075f, - BloomVeilIntensity: 0.0f); + BloomVeilIntensity: 0.0f, + FieldReservoirMode: GraphicsSettings.FieldReservoirModeNativeDomain, + FieldReservoirScale: 0.5f); } obsStemPublisher = CreateObsStemPublisher(); @@ -151,6 +154,7 @@ public sealed class MimirRuntime : IAquariumRuntime, IAquariumRuntimeServicesRec string.Equals(profile.Id, settings.Audio.BioacousticWitnessProfileId, StringComparison.OrdinalIgnoreCase)) ?? MimirBioacousticContestants.CanaryPacketTrill) : null; + presentationControls.SyncFromBuffers(synchronization.Buffers.Buffers); nextAudioSyncSeconds = audioSyncUpdateIntervalSeconds; nextTelemetrySeconds = telemetryIntervalSeconds; ui = CreateUi(); @@ -274,7 +278,12 @@ public sealed class MimirRuntime : IAquariumRuntime, IAquariumRuntimeServicesRec continue; } - intents.Add(BuildSurfaceIntent(window, observation)); + if (!presentationControls.IncludesVideo(window.StreamId)) + { + continue; + } + + intents.Add(BuildSurfaceIntent(window, observation, presentationControls.VideoOpacity(window.StreamId))); } var frame = fieldLowering.BuildFieldEvidenceFrame(windows, observations, constraints, intents); @@ -299,7 +308,7 @@ public sealed class MimirRuntime : IAquariumRuntime, IAquariumRuntimeServicesRec } } - private static MimirSurfaceIntent BuildSurfaceIntent(MimirRollingStreamWindow window, MimirObservation observation) + private static MimirSurfaceIntent BuildSurfaceIntent(MimirRollingStreamWindow window, MimirObservation observation, float opacity = 1.0f) { if (window.SourceKind == MimirStreamKind.Audio) { @@ -314,7 +323,15 @@ public sealed class MimirRuntime : IAquariumRuntime, IAquariumRuntimeServicesRec Purpose: MimirSurfaceIntentPurpose.Debug); } - return MimirFensalirBridgeMapper.MapDefaultSurfaceIntent(window, observation); + return new MimirSurfaceIntent( + IntentKey: $"{window.WindowId}:surface", + SourceObservationKeys: [observation.ObservationKey], + Domain: MimirSurfaceDomain.CameraImage, + Axes: new MimirSurfaceAxes("image-x", "image-y", "time-age", "stream"), + SupportPolicy: new MimirSurfaceSupportPolicy("program-composite", 0.0, window.Duration.TotalSeconds), + MaterialGraph: new MimirSurfaceMaterialIntent("program-video", "source-evidence", Math.Clamp(opacity, 0.0f, 1.0f)), + UpdateBudget: new MimirSurfaceUpdateBudget(0.0, 1, Math.Max(0, window.Payload.ByteLength)), + Purpose: MimirSurfaceIntentPurpose.Production); } private AquariumFieldEvidenceFrame AddSpectrumFieldEvidence(AquariumFieldEvidenceFrame frame) @@ -690,6 +707,7 @@ public sealed class MimirRuntime : IAquariumRuntime, IAquariumRuntimeServicesRec public void RegisterStreamSource(IMimirStreamSource source) { synchronization.AddSource(source); + presentationControls.SyncFromBuffers(synchronization.Buffers.Buffers); } public void Start() @@ -707,6 +725,8 @@ public sealed class MimirRuntime : IAquariumRuntime, IAquariumRuntimeServicesRec runtimeSeconds += Math.Max(deltaSeconds, 0.0f); lastPollCount = synchronization.PollSources(); + presentationControls.SyncFromBuffers(synchronization.Buffers.Buffers); + ApplyPresentationPostprocess(); UpdateObsStemPublication(); QueueCalibrationTimeline(); UpdateAudioSpectra(); @@ -747,7 +767,70 @@ public sealed class MimirRuntime : IAquariumRuntime, IAquariumRuntimeServicesRec private AquariumUiDocument CreateUi() { return new AquariumUiDocument() - .Panel("Mimir Sync", 18.0f, 82.0f, 390.0f, panel => + .Panel("Mimir Program", 18.0f, 82.0f, 360.0f, fadeWhenMouseDistant: true, panel => + { + panel.Section("Video"); + panel.Options("Feed", () => presentationControls.SelectedVideoIndex, value => presentationControls.SelectedVideoIndex = value, VideoFeedOptions(), "Selects the video feed edited by the controls below."); + panel.Toggle("Visible", () => presentationControls.SelectedVideo?.Enabled ?? false, value => + { + if (presentationControls.SelectedVideo is { } feed) + { + feed.Enabled = value; + } + }); + panel.Toggle("Solo", () => presentationControls.SelectedVideo?.Solo ?? false, value => + { + if (presentationControls.SelectedVideo is { } feed) + { + feed.Solo = value; + } + }); + panel.Slider("Opacity", () => presentationControls.SelectedVideo?.Opacity ?? 0.0f, value => + { + if (presentationControls.SelectedVideo is { } feed) + { + feed.Opacity = Math.Clamp(value, 0.0f, 1.0f); + } + }, 0.0f, 1.0f, "0.00"); + panel.Button("Layer Up", () => presentationControls.MoveSelectedVideo(-1)); + panel.Button("Layer Down", () => presentationControls.MoveSelectedVideo(1)); + panel.Readout("Composite", DescribeProgramVideo); + + panel.Section("Audio"); + panel.Options("Stem", () => presentationControls.SelectedAudioIndex, value => presentationControls.SelectedAudioIndex = value, AudioFeedOptions(), "Selects the audio lane edited by the controls below."); + panel.Toggle("Mute", () => presentationControls.SelectedAudio?.Muted ?? false, value => + { + if (presentationControls.SelectedAudio is { } feed) + { + feed.Muted = value; + } + }); + panel.Toggle("Solo Stem", () => presentationControls.SelectedAudio?.Solo ?? false, value => + { + if (presentationControls.SelectedAudio is { } feed) + { + feed.Solo = value; + } + }); + panel.Slider("Gain", () => presentationControls.SelectedAudio?.Gain ?? 0.0f, value => + { + if (presentationControls.SelectedAudio is { } feed) + { + feed.Gain = Math.Clamp(value, 0.0f, 2.0f); + } + }, 0.0f, 2.0f, "0.00"); + panel.Readout("Mix", DescribeProgramAudio); + + panel.Section("Color"); + panel.Options("LUT", () => presentationControls.SelectedLutPresetIndex, presentationControls.SelectLutPreset, LutPresetOptions(), "Global color lookup table preset for the Fensalir post pass."); + panel.Slider("LUT Strength", () => presentationControls.LutStrength, value => + { + presentationControls.LutStrength = Math.Clamp(value, 0.0f, 1.0f); + presentationControls.RefreshPostprocess(); + }, 0.0f, 1.0f, "0.00"); + panel.Readout("Post", DescribePostprocess); + }) + .Panel("Mimir Sync", 392.0f, 82.0f, 390.0f, fadeWhenMouseDistant: true, panel => { panel.Section("Rolling Buffers"); panel.Readout("Window", () => $"{synchronization.Settings.BufferDuration.TotalSeconds:0.###}s"); @@ -817,6 +900,18 @@ public sealed class MimirRuntime : IAquariumRuntime, IAquariumRuntimeServicesRec return app.Plan; } + private void ApplyPresentationPostprocess() + { + presentationControls.RefreshPostprocess(); + var post = presentationControls.Postprocess; + GraphicsSettings = GraphicsSettings with + { + SceneExposure = Math.Clamp(post.Exposure, GraphicsSettings.MinSceneExposure, GraphicsSettings.MaxSceneExposure), + BloomIntensity = Math.Clamp(post.BloomIntensity, GraphicsSettings.MinBloomIntensity, GraphicsSettings.MaxBloomIntensity), + BloomVeilIntensity = Math.Clamp(post.BloomVeilIntensity, GraphicsSettings.MinBloomVeilIntensity, GraphicsSettings.MaxBloomVeilIntensity), + }; + } + private void QueueCalibrationTimeline() { if (!ShouldEmitCalibrationTimeline()) @@ -942,6 +1037,61 @@ public sealed class MimirRuntime : IAquariumRuntime, IAquariumRuntimeServicesRec return string.Join(" | ", synchronization.Buffers.Buffers.Select(DescribeBuffer)); } + private IReadOnlyList VideoFeedOptions() + { + var feeds = presentationControls.VideoFeeds; + return feeds.Count == 0 + ? [new AquariumUiOption(0, "No video")] + : feeds.Select((feed, index) => new AquariumUiOption(index, $"{feed.Layer + 1}. {feed.DisplayName}")).ToArray(); + } + + private IReadOnlyList AudioFeedOptions() + { + var feeds = presentationControls.AudioFeeds; + return feeds.Count == 0 + ? [new AquariumUiOption(0, "No audio")] + : feeds.Select((feed, index) => new AquariumUiOption(index, feed.DisplayName)).ToArray(); + } + + private IReadOnlyList LutPresetOptions() => + presentationControls.LutPresets + .Select((preset, index) => new AquariumUiOption(index, preset.DisplayName)) + .ToArray(); + + private string DescribeProgramVideo() + { + var feeds = presentationControls.VideoFeeds; + if (feeds.Count == 0) + { + return "no video buffers"; + } + + var active = feeds + .Where(feed => presentationControls.IncludesVideo(feed.SourceId)) + .Select(feed => $"{feed.Layer + 1}:{feed.SourceId}@{feed.Opacity:0.00}"); + return string.Join(" | ", active.DefaultIfEmpty("black")); + } + + private string DescribeProgramAudio() + { + var feeds = presentationControls.AudioFeeds; + if (feeds.Count == 0) + { + return "no audio buffers"; + } + + var active = feeds + .Select(feed => $"{feed.SourceId}:{presentationControls.AudioGain(feed.SourceId):0.00}") + .ToArray(); + return string.Join(" | ", active); + } + + private string DescribePostprocess() + { + var post = presentationControls.Postprocess; + return $"{post.PresetName} strength={post.LutStrength:0.00} exp={post.Exposure:0.000} contrast={post.Contrast:0.00} sat={post.Saturation:0.00} lut={post.LutPath}"; + } + private void UpdateAudioSynchronization() { if (runtimeSeconds < nextAudioSyncSeconds) @@ -984,7 +1134,7 @@ public sealed class MimirRuntime : IAquariumRuntime, IAquariumRuntimeServicesRec command.TargetDelaySamples, command.ResampleRatio, command.Confidence, - command.FaustControls)) + ApplyPresentationGain(command))) .ToArray(), frame.TruncatedSourceCount, ++audioActuatorFrameSequence)); @@ -1053,7 +1203,7 @@ public sealed class MimirRuntime : IAquariumRuntime, IAquariumRuntimeServicesRec continue; } - var samples = ExtractMonoAudioBlock(latest, block); + var samples = ApplyGain(ExtractMonoAudioBlock(latest, block), presentationControls.AudioGain(commands[index].SourceId)); if (samples.Length == 0) { continue; @@ -1091,6 +1241,18 @@ public sealed class MimirRuntime : IAquariumRuntime, IAquariumRuntimeServicesRec sequence)); } + private IReadOnlyDictionary ApplyPresentationGain(MimirActuatorCommand command) + { + var gain = presentationControls.AudioGain(command.SourceId); + var controls = new Dictionary(command.FaustControls, StringComparer.Ordinal); + foreach (var key in controls.Keys.Where(key => key.EndsWith("/gain", StringComparison.Ordinal)).ToArray()) + { + controls[key] = Math.Clamp(controls[key] * gain, 0.0f, 2.0f); + } + + return controls; + } + private void UpdateObsStemPublication() { var consumed = false; @@ -1298,6 +1460,23 @@ public sealed class MimirRuntime : IAquariumRuntime, IAquariumRuntimeServicesRec return output; } + private static float[] ApplyGain(float[] samples, float gain) + { + var safeGain = Math.Clamp(gain, 0.0f, 2.0f); + if (samples.Length == 0 || Math.Abs(safeGain - 1.0f) < 0.0001f) + { + return samples; + } + + var output = new float[samples.Length]; + for (var index = 0; index < samples.Length; index++) + { + output[index] = Math.Clamp(samples[index] * safeGain, -1.0f, 1.0f); + } + + return output; + } + private static int FaustSourceIndex(MimirActuatorCommand command) { foreach (var key in command.FaustControls.Keys) diff --git a/src/Mimir.Runtime/Synchronization/MimirPresentationControlState.cs b/src/Mimir.Runtime/Synchronization/MimirPresentationControlState.cs new file mode 100644 index 0000000..68d737d --- /dev/null +++ b/src/Mimir.Runtime/Synchronization/MimirPresentationControlState.cs @@ -0,0 +1,292 @@ +namespace Mimir.Runtime.Synchronization; + +public sealed record MimirLookupTablePreset( + string Id, + string DisplayName, + string LutPath, + float Exposure, + float Contrast, + float Saturation, + float Temperature, + float Tint, + float BloomIntensity, + float BloomVeilIntensity) +{ + public static IReadOnlyList BuiltIn { get; } = + [ + new( + "neutral", + "Neutral", + "luts/neutral.cube", + Exposure: 0.16f, + Contrast: 1.00f, + Saturation: 1.00f, + Temperature: 0.00f, + Tint: 0.00f, + BloomIntensity: 0.072f, + BloomVeilIntensity: 0.014f), + new( + "clean-rec709", + "Clean Rec.709", + "luts/clean-rec709.cube", + Exposure: 0.18f, + Contrast: 1.04f, + Saturation: 0.98f, + Temperature: 0.00f, + Tint: 0.00f, + BloomIntensity: 0.045f, + BloomVeilIntensity: 0.006f), + new( + "soft-film", + "Soft Film", + "luts/soft-film.cube", + Exposure: 0.15f, + Contrast: 0.94f, + Saturation: 1.08f, + Temperature: 0.08f, + Tint: 0.02f, + BloomIntensity: 0.095f, + BloomVeilIntensity: 0.022f), + new( + "cool-monitor", + "Cool Monitor", + "luts/cool-monitor.cube", + Exposure: 0.17f, + Contrast: 1.02f, + Saturation: 0.92f, + Temperature: -0.10f, + Tint: 0.01f, + BloomIntensity: 0.060f, + BloomVeilIntensity: 0.010f), + new( + "warm-room", + "Warm Room", + "luts/warm-room.cube", + Exposure: 0.14f, + Contrast: 0.98f, + Saturation: 1.06f, + Temperature: 0.12f, + Tint: -0.01f, + BloomIntensity: 0.085f, + BloomVeilIntensity: 0.018f), + ]; +} + +public sealed class MimirVideoPresentationControl +{ + public MimirVideoPresentationControl(string sourceId, string displayName, int layer) + { + SourceId = sourceId; + DisplayName = string.IsNullOrWhiteSpace(displayName) ? sourceId : displayName; + Layer = Math.Max(0, layer); + } + + public string SourceId { get; } + + public string DisplayName { get; private set; } + + public bool Enabled { get; set; } = true; + + public bool Solo { get; set; } + + public float Opacity { get; set; } = 1.0f; + + public int Layer { get; set; } + + public void RefreshLabel(string displayName) + { + if (!string.IsNullOrWhiteSpace(displayName)) + { + DisplayName = displayName; + } + } +} + +public sealed class MimirAudioPresentationControl +{ + public MimirAudioPresentationControl(string sourceId, string displayName) + { + SourceId = sourceId; + DisplayName = string.IsNullOrWhiteSpace(displayName) ? sourceId : displayName; + } + + public string SourceId { get; } + + public string DisplayName { get; private set; } + + public bool Muted { get; set; } + + public bool Solo { get; set; } + + public float Gain { get; set; } = 1.0f; + + public void RefreshLabel(string displayName) + { + if (!string.IsNullOrWhiteSpace(displayName)) + { + DisplayName = displayName; + } + } +} + +public sealed record MimirPostprocessControlSnapshot( + string PresetId, + string PresetName, + string LutPath, + float LutStrength, + float Exposure, + float Contrast, + float Saturation, + float Temperature, + float Tint, + float BloomIntensity, + float BloomVeilIntensity); + +public sealed class MimirPresentationControlState +{ + private readonly Dictionary video = new(StringComparer.Ordinal); + private readonly Dictionary audio = new(StringComparer.Ordinal); + + public IReadOnlyList LutPresets => MimirLookupTablePreset.BuiltIn; + + public int SelectedVideoIndex { get; set; } + + public int SelectedAudioIndex { get; set; } + + public int SelectedLutPresetIndex { get; private set; } + + public float LutStrength { get; set; } = 1.0f; + + public MimirPostprocessControlSnapshot Postprocess { get; private set; } = + SnapshotFor(MimirLookupTablePreset.BuiltIn[0], 1.0f); + + public IReadOnlyList VideoFeeds => + video.Values.OrderBy(feed => feed.Layer).ThenBy(feed => feed.SourceId, StringComparer.Ordinal).ToArray(); + + public IReadOnlyList AudioFeeds => + audio.Values.OrderBy(feed => feed.SourceId, StringComparer.Ordinal).ToArray(); + + public MimirVideoPresentationControl? SelectedVideo => + VideoFeeds.Count == 0 ? null : VideoFeeds[Math.Clamp(SelectedVideoIndex, 0, VideoFeeds.Count - 1)]; + + public MimirAudioPresentationControl? SelectedAudio => + AudioFeeds.Count == 0 ? null : AudioFeeds[Math.Clamp(SelectedAudioIndex, 0, AudioFeeds.Count - 1)]; + + public void SyncFromBuffers(IEnumerable buffers) + { + var videoIndex = video.Count; + foreach (var buffer in buffers) + { + if (buffer.Descriptor.Kind == MimirStreamKind.Video) + { + if (!video.TryGetValue(buffer.Descriptor.SourceId, out var control)) + { + control = new MimirVideoPresentationControl(buffer.Descriptor.SourceId, buffer.Descriptor.Label, videoIndex++); + video.Add(buffer.Descriptor.SourceId, control); + } + + control.RefreshLabel(buffer.Descriptor.Label); + continue; + } + + if (buffer.Descriptor.Kind == MimirStreamKind.Audio) + { + if (!audio.TryGetValue(buffer.Descriptor.SourceId, out var control)) + { + control = new MimirAudioPresentationControl(buffer.Descriptor.SourceId, buffer.Descriptor.Label); + audio.Add(buffer.Descriptor.SourceId, control); + } + + control.RefreshLabel(buffer.Descriptor.Label); + } + } + + SelectedVideoIndex = ClampIndex(SelectedVideoIndex, VideoFeeds.Count); + SelectedAudioIndex = ClampIndex(SelectedAudioIndex, AudioFeeds.Count); + } + + public bool IncludesVideo(string sourceId) + { + if (!video.TryGetValue(sourceId, out var control)) + { + return true; + } + + var hasSolo = video.Values.Any(feed => feed.Solo); + return hasSolo ? control.Solo : control.Enabled; + } + + public float VideoOpacity(string sourceId) => + video.TryGetValue(sourceId, out var control) + ? Math.Clamp(control.Opacity, 0.0f, 1.0f) + : 1.0f; + + public float AudioGain(string sourceId) + { + if (!audio.TryGetValue(sourceId, out var control)) + { + return 1.0f; + } + + var hasSolo = audio.Values.Any(feed => feed.Solo); + if (control.Muted || (hasSolo && !control.Solo)) + { + return 0.0f; + } + + return Math.Clamp(control.Gain, 0.0f, 2.0f); + } + + public void MoveSelectedVideo(int delta) + { + var selected = SelectedVideo; + if (selected == null) + { + return; + } + + selected.Layer = Math.Clamp(selected.Layer + delta, 0, Math.Max(0, video.Count - 1)); + NormalizeVideoLayers(); + SelectedVideoIndex = VideoFeeds.ToList().FindIndex(feed => string.Equals(feed.SourceId, selected.SourceId, StringComparison.Ordinal)); + } + + public void SelectLutPreset(int index) + { + SelectedLutPresetIndex = ClampIndex(index, LutPresets.Count); + Postprocess = SnapshotFor(LutPresets[SelectedLutPresetIndex], LutStrength); + } + + public void RefreshPostprocess() + { + Postprocess = SnapshotFor(LutPresets[SelectedLutPresetIndex], LutStrength); + } + + private void NormalizeVideoLayers() + { + var ordered = video.Values + .OrderBy(feed => feed.Layer) + .ThenBy(feed => feed.SourceId, StringComparer.Ordinal) + .ToArray(); + for (var index = 0; index < ordered.Length; index++) + { + ordered[index].Layer = index; + } + } + + private static int ClampIndex(int index, int count) => + count <= 0 ? 0 : Math.Clamp(index, 0, count - 1); + + private static MimirPostprocessControlSnapshot SnapshotFor(MimirLookupTablePreset preset, float strength) => + new( + preset.Id, + preset.DisplayName, + preset.LutPath, + Math.Clamp(strength, 0.0f, 1.0f), + preset.Exposure, + preset.Contrast, + preset.Saturation, + preset.Temperature, + preset.Tint, + preset.BloomIntensity, + preset.BloomVeilIntensity); +} diff --git a/state/evidence.jsonl b/state/evidence.jsonl index 91748ee..737fd5f 100644 --- a/state/evidence.jsonl +++ b/state/evidence.jsonl @@ -228,3 +228,4 @@ {"timestamp":"2026-05-29T01:46:00+01:00","type":"decision","status":"ok","note":"Fensalir TubeField spectrum rendering now uses rolling-buffer column packets as reservoir candidate authority rather than generated segment packets. Expansion emits one GPU-resident column packet per logical history/source column, ReSTIR evaluates Catmull-Rom SDF/material samples directly from the source buffer, and generated mesh draw args are zeroed for the live path. Verified through Mimir.App headless synthetic spectrum capture artifacts/runtime/mimir-column-packet-final-headless.png."} {"ts": "2026-05-29T22:23:04+01:00", "type": "decision", "status": "ok", "note": "Raven display/audio sync uses shared clock-domain correction: Raven display frames can be tagged raven-sync, while Raven audio routed into Scarlett earns the timing offset applied by MimirSynchronizedBufferPlanner; network arrival timestamps remain metadata."} {"ts": "2026-05-29T23:50:56+01:00", "type": "decision", "status": "ok", "note": "Fast stereo depth is mapped as a future Fensalir-owned D3D12 compute lane, with libSGM as permissive SGM provenance rather than a CUDA dependency. External TensorRT/monocular depth projects remain reference evidence only; live depth must consume Mimir-declared texture resources, calibration, and fences, then emit GPU disparity/depth/confidence resources and FieldEvidence claims."} +{"ts": "2026-05-29T23:52:11+01:00", "type": "decision", "status": "ok", "note": "Mimir Program panel now owns Fensalir presentation intent: video visibility/solo/opacity/layer filters production surface intents, audio mute/solo/gain feeds Faust controls and streaming samples, and LUT presets are typed postprocess state with exposure/bloom mapped to current GraphicsSettings until renderer LUT sampling lands."} diff --git a/state/map.yaml b/state/map.yaml index 3e7ff76..19fbb83 100644 --- a/state/map.yaml +++ b/state/map.yaml @@ -230,6 +230,13 @@ accepted_design: shared `clockDomainId`; the Raven path uses audio evidence routed into Scarlett to earn a `raven-sync` correction for Raven display frames. Network arrival timestamps remain metadata only. + `MimirPresentationControlState` owns program-control intent for Fensalir: + video feed visibility/solo/opacity/layer order, audio mute/solo/gain, and + global LUT preset selection. The `Mimir Program` panel filters video + production surface intents and applies audio mix gain before/inside the + Faust streaming DSP path. LUT presets are typed postprocess state; exposure + and bloom map into current Fensalir `GraphicsSettings`, while actual LUT + texture sampling remains the named renderer hook. `scripts/build-obs-stem-plugin.ps1` stages the upstream OBS plugin template SDK under `artifacts/obs-sdk/` and builds the plugin DLL locally. The synthetic