Polish compositor editor affordances

This commit is contained in:
Meta Vens 2026-06-01 14:26:44 +01:00
commit 09202f1a7c
8 changed files with 217 additions and 15 deletions

View file

@ -306,6 +306,10 @@ a named invariant that the native runtime cannot protect yet.
resizing through direct preview handles, sliders, or the scale gizmo. Explicit
transform/source controls are inspector-only and only appear while a program
source is selected; the compositor canvas handles are the primary edit path.
Empty-canvas clicks clear source selection, hover/active handles and snap
guides are visible through the Fensalir CultUI preview state, keyboard arrows
nudge selected layers, PageUp/PageDown reorder layers, and the inspector owns
the selected source's contain/cover/stretch fit mode.
See [[scene-editor-control-surface|Mimir Scene Editor Control Surface]].
`Mimir.BufferSmoke --scene-editor-smoke` proves the graph/control/gizmo owner
path. World SDF text glyph rendering, ASSIMP-style mesh import/upload, and

View file

@ -551,7 +551,10 @@ layers inside the program frame, snaps near edges/center, and preserves source
aspect while resizing through direct preview handles, sliders, or the scale
gizmo. Explicit transform/source controls are inspector-only and gated to a
selected program source; the center canvas handles are the primary compositor
edit path. World SDF glyph rendering, ASSIMP-style
edit path. Empty-canvas clicks clear source selection; hover/active handles and
snap guides come from the Fensalir CultUI preview state; arrows nudge the
selected layer, PageUp/PageDown reorder layers, and the inspector owns the
selected source contain/cover/stretch fit mode. World SDF glyph rendering, ASSIMP-style
mesh decoding/upload, and pixel-accurate gizmo hit-testing are explicit
Fensalir renderer cuts, mapped in
[[docs/scene-editor-control-surface|Mimir Scene Editor Control Surface]].

View file

@ -54,6 +54,9 @@ Get-Content .\state\evidence.jsonl -Tail 8
while resizing through direct preview handles, sliders, or the scale gizmo.
Explicit source/transform controls are inspector-only and only visible when a
program source is selected.
Hover/active handle state, snap guides, empty-canvas deselect, keyboard
nudging, PageUp/PageDown layer ordering, and selected-source
contain/cover/stretch fit mode are wired through `MimirSceneEditorState`.
Camera/SDF/model editor affordances have no live UI authority, and editor
camera/SDF/spline output no longer contributes to the runtime frame. Smoke
with `--scene-editor-smoke`.

View file

@ -4249,6 +4249,19 @@ static int RunSceneEditorSmoke()
var previewHandleAspectLocked = handlePlacement.HasValue &&
Math.Abs(handlePlacement.Value.Width - handlePlacement.Value.Height) < 0.001f &&
handlePlacement.Value.Width > draggedPlacement?.Width;
editor.HandlePreviewInteraction(new AquariumUiPreviewInteraction(displayNode.Id, "se", "hover", 0.75f, 0.75f, 0.0f, 0.0f));
var hoverTracked = editor.PreviewState.HoverItemId == displayNode.Id && editor.PreviewState.HoverHandle == "se";
editor.SetSelectedProgramX(0.5f);
editor.HandlePreviewInteraction(new AquariumUiPreviewInteraction(displayNode.Id, "move", "drag", 0.5f, 0.75f, 0.0f, 0.0f));
var guideVisible = editor.PreviewGuides.Any(guide => guide.Axis == "x" && Math.Abs(guide.Position - 0.5f) < 0.001f);
editor.SetSelectedFitModeIndex(1);
var fitModeSet = editor.SelectedFitModeIndex == 1 && editor.DescribeSelection().Contains("Cover", StringComparison.Ordinal);
editor.NudgeSelectedProgramLayer(1.0f, 0.0f, fine: false);
var nudgedPlacement = editor.PlacementForSource("raven-display");
var nudgeHandled = nudgedPlacement.HasValue && Math.Abs(nudgedPlacement.Value.CenterX - 0.51f) < 0.001f;
editor.HandlePreviewInteraction(new AquariumUiPreviewInteraction("", "canvas", "begin", 0.02f, 0.02f, 0.0f, 0.0f));
var emptyClickDeselected = !editor.HasSelectedProgramSource;
editor.SelectNode(displayNode.Id);
editor.CenterSelectedProgramLayer();
editor.MoveSelectedLayer(-1);
@ -4267,7 +4280,7 @@ static int RunSceneEditorSmoke()
Console.WriteLine(
$"scene-editor-smoke nodes={editor.Nodes.Count} feeds={editor.Nodes.Count(node => node.Kind == MimirSceneEditorNodeKind.SensorFeedPanel)} " +
$"preview={previewItems.Count} selected={editor.SelectedNode?.DisplayName} hasDisplay={hasDisplay} cameraPreview={hasCameraPreview} " +
$"placement={placement?.CenterX:0.000},{placement?.CenterY:0.000} programCentered={programCentered} aspectLocked={aspectLocked} snapped={snappedToBounds} previewMove={previewMoveHandled} previewHandle={previewHandleAspectLocked} cameraSource={editor.HasSelectedProgramSource}");
$"placement={placement?.CenterX:0.000},{placement?.CenterY:0.000} programCentered={programCentered} aspectLocked={aspectLocked} snapped={snappedToBounds} previewMove={previewMoveHandled} previewHandle={previewHandleAspectLocked} hover={hoverTracked} guide={guideVisible} fit={fitModeSet} nudge={nudgeHandled} empty={emptyClickDeselected} cameraSource={editor.HasSelectedProgramSource}");
return editor.Nodes.Count == 3 &&
hasDisplay &&
@ -4278,6 +4291,11 @@ static int RunSceneEditorSmoke()
snappedToBounds &&
previewMoveHandled &&
previewHandleAspectLocked &&
hoverTracked &&
guideVisible &&
fitModeSet &&
nudgeHandled &&
emptyClickDeselected &&
cameraIsNotProgramSource
? 0
: 1;

View file

@ -893,7 +893,15 @@ public sealed class MimirRuntime : IAquariumRuntime, IAquariumRuntimeServicesRec
shell.Vertical("mimir.editor-center", center =>
{
center.Preview("scene.view-preview", "", () => sceneEditor.PreviewItems, weight: 2.2f, blitOutputBuffer: true, handleInteraction: sceneEditor.HandlePreviewInteraction);
center.Preview(
"scene.view-preview",
"",
() => sceneEditor.PreviewItems,
weight: 2.2f,
blitOutputBuffer: true,
handleInteraction: sceneEditor.HandlePreviewInteraction,
readState: () => sceneEditor.PreviewState,
readGuides: () => sceneEditor.PreviewGuides);
center.Pane("mimir.audio-mixer", "Audio Mixer", mixer =>
{
mixer.Horizontal("audio.channel-bank", bank =>
@ -933,6 +941,7 @@ public sealed class MimirRuntime : IAquariumRuntime, IAquariumRuntimeServicesRec
source.Slider("program.w", "W", () => sceneEditor.SelectedProgramWidth, sceneEditor.SetSelectedProgramWidth, 0.01f, 1.0f, "0.000");
source.Slider("program.h", "H", () => sceneEditor.SelectedProgramHeight, sceneEditor.SetSelectedProgramHeight, 0.01f, 1.0f, "0.000");
source.Slider("program.rotation", "Rotation", () => sceneEditor.SelectedNode?.Transform.RotationRadians ?? 0.0f, sceneEditor.SetSelectedRotation, -MathF.PI, MathF.PI, "0.00");
source.Options("program.fit-mode", "Fit", () => sceneEditor.SelectedFitModeIndex, sceneEditor.SetSelectedFitModeIndex, sceneEditor.FitModeOptions);
source.Row("program.fit", fit =>
{
fit.Button("program.center", "Center", sceneEditor.CenterSelectedProgramLayer);

View file

@ -15,6 +15,17 @@ public sealed class MimirSceneEditorState
private const float ProgramSnapThreshold = 0.015f;
private readonly List<MimirSceneEditorNode> nodes = [];
private readonly HashSet<string> configuredProgramSources = new(StringComparer.Ordinal);
private readonly AquariumUiOption[] fitModeOptions =
[
new(0, "Contain"),
new(1, "Cover"),
new(2, "Stretch"),
];
private IReadOnlyList<AquariumUiPreviewGuide> previewGuides = [];
private string hoverItemId = "";
private string hoverHandle = "";
private string activeItemId = "";
private string activeHandle = "";
private bool selectedConfiguredSource;
private Vector2? previousMousePosition;
@ -45,6 +56,12 @@ public sealed class MimirSceneEditorState
public IReadOnlyList<MimirSceneEditorNode> VisibleNodes =>
nodes.Where(node => node.Visible).ToArray();
public AquariumUiPreviewState PreviewState => new(hoverItemId, hoverHandle, activeItemId, activeHandle);
public IReadOnlyList<AquariumUiPreviewGuide> PreviewGuides => previewGuides;
public IReadOnlyList<AquariumUiOption> FitModeOptions => fitModeOptions;
public IReadOnlyList<AquariumUiPreviewItem> PreviewItems =>
nodes
.Where(static node => node.Visible && node.Kind == MimirSceneEditorNodeKind.SensorFeedPanel)
@ -52,9 +69,15 @@ public sealed class MimirSceneEditorState
.Select(node =>
{
var placement = PlacementForNode(node);
var fit = node.FitMode switch
{
MimirProgramLayerFitMode.Cover => "cover",
MimirProgramLayerFitMode.Stretch => "stretch",
_ => "contain",
};
return new AquariumUiPreviewItem(
node.Id,
node.DisplayName,
$"{node.DisplayName} lock {fit}",
placement.CenterX - placement.Width * 0.5f,
placement.CenterY - placement.Height * 0.5f,
placement.Width,
@ -168,11 +191,51 @@ public sealed class MimirSceneEditorState
if (input.IsKeyPressed(KeyCode.LeftArrow))
{
SelectPrevious();
if (input.IsKeyDown(KeyCode.Control))
{
MoveSelectedLayer(-1);
}
else if (HasSelectedProgramSource)
{
NudgeSelectedProgramLayer(-1.0f, 0.0f, input.IsKeyDown(KeyCode.Shift));
}
else
{
SelectPrevious();
}
}
else if (input.IsKeyPressed(KeyCode.RightArrow))
{
SelectNext();
if (input.IsKeyDown(KeyCode.Control))
{
MoveSelectedLayer(1);
}
else if (HasSelectedProgramSource)
{
NudgeSelectedProgramLayer(1.0f, 0.0f, input.IsKeyDown(KeyCode.Shift));
}
else
{
SelectNext();
}
}
if (input.IsKeyPressed(KeyCode.UpArrow) && HasSelectedProgramSource)
{
NudgeSelectedProgramLayer(0.0f, -1.0f, input.IsKeyDown(KeyCode.Shift));
}
else if (input.IsKeyPressed(KeyCode.DownArrow) && HasSelectedProgramSource)
{
NudgeSelectedProgramLayer(0.0f, 1.0f, input.IsKeyDown(KeyCode.Shift));
}
if (input.IsKeyPressed(KeyCode.PageUp))
{
MoveSelectedLayer(-1);
}
else if (input.IsKeyPressed(KeyCode.PageDown))
{
MoveSelectedLayer(1);
}
if (input.IsKeyPressed(KeyCode.Home))
@ -229,6 +292,23 @@ public sealed class MimirSceneEditorState
public void SelectPrevious() => SelectRelative(-1);
public void NudgeSelectedProgramLayer(float xSteps, float ySteps, bool fine)
{
if (SelectedNode is not { Kind: MimirSceneEditorNodeKind.SensorFeedPanel } selected)
{
return;
}
var step = fine ? 0.0025f : 0.01f;
var placement = PlacementForNode(selected);
SetNodeProgramPlacement(selected, placement with
{
CenterX = placement.CenterX + xSteps * step,
CenterY = placement.CenterY + ySteps * step,
}, snap: false);
previewGuides = GuidesForPlacement(PlacementForNode(selected));
}
public void SetSelectedVisible(bool visible)
{
if (SelectedNode is { } selected)
@ -372,6 +452,18 @@ public sealed class MimirSceneEditorState
}
}
public int SelectedFitModeIndex => SelectedNode is { Kind: MimirSceneEditorNodeKind.SensorFeedPanel } selected
? (int)selected.FitMode
: 0;
public void SetSelectedFitModeIndex(int value)
{
if (SelectedNode is { Kind: MimirSceneEditorNodeKind.SensorFeedPanel } selected)
{
selected.FitMode = (MimirProgramLayerFitMode)Math.Clamp(value, 0, 2);
}
}
public void MoveSelectedLayer(int delta)
{
if (SelectedNode is not { Kind: MimirSceneEditorNodeKind.SensorFeedPanel } selected)
@ -406,6 +498,31 @@ public sealed class MimirSceneEditorState
public void HandlePreviewInteraction(AquariumUiPreviewInteraction interaction)
{
hoverItemId = interaction.Phase == "hover" ? interaction.ItemId : hoverItemId;
hoverHandle = interaction.Phase == "hover" ? interaction.Handle : hoverHandle;
if (interaction.Phase == "end")
{
activeItemId = "";
activeHandle = "";
previewGuides = [];
return;
}
if (interaction.Phase == "begin")
{
activeItemId = interaction.ItemId;
activeHandle = interaction.Handle;
}
if (interaction.Handle == "canvas" && interaction.Phase == "begin")
{
SelectedNodeId = "editor-camera";
activeItemId = "";
activeHandle = "";
previewGuides = [];
return;
}
if (nodes.FirstOrDefault(node => string.Equals(node.Id, interaction.ItemId, StringComparison.Ordinal)) is not { Kind: MimirSceneEditorNodeKind.SensorFeedPanel } node ||
node.Locked)
{
@ -426,10 +543,12 @@ public sealed class MimirSceneEditorState
CenterX = placement.CenterX + interaction.DeltaX,
CenterY = placement.CenterY + interaction.DeltaY,
});
previewGuides = GuidesForPlacement(PlacementForNode(node));
return;
}
SetNodeProgramPlacement(node, ResizeProgramPlacementFromHandle(node, placement, interaction.Handle, interaction.DeltaX, interaction.DeltaY));
previewGuides = GuidesForPlacement(PlacementForNode(node));
}
public bool IncludesVideoSource(string sourceId) =>
@ -488,7 +607,7 @@ public sealed class MimirSceneEditorState
}
var placement = PlacementForNode(selected);
return $"{selected.Layer + 1}. {selected.DisplayName} source={selected.SourceId} frame={placement.CenterX:0.000},{placement.CenterY:0.000} {placement.Width:0.000}x{placement.Height:0.000}";
return $"{selected.Layer + 1}. {selected.DisplayName} source={selected.SourceId} {selected.FitMode} frame={placement.CenterX:0.000},{placement.CenterY:0.000} {placement.Width:0.000}x{placement.Height:0.000}";
}
public AquariumSplineFrame BuildEditorSplineFrame()
@ -782,14 +901,14 @@ public sealed class MimirSceneEditorState
Layer: node.Layer);
}
private static void SetNodeProgramPlacement(MimirSceneEditorNode node, MimirCompositorPlacement placement)
private static void SetNodeProgramPlacement(MimirSceneEditorNode node, MimirCompositorPlacement placement, bool snap = true)
{
if (node.Locked)
{
return;
}
var normalized = NormalizeProgramPlacement(placement);
var normalized = NormalizeProgramPlacement(placement, snap);
node.Transform = node.Transform with
{
Position = new Vector3((normalized.CenterX - 0.5f) * ProgramWorldWidth, (0.5f - normalized.CenterY) * ProgramWorldHeight, node.Transform.Position.Z),
@ -798,12 +917,24 @@ public sealed class MimirSceneEditorState
};
}
private static MimirCompositorPlacement NormalizeProgramPlacement(MimirCompositorPlacement placement)
private static MimirCompositorPlacement NormalizeProgramPlacement(MimirCompositorPlacement placement, bool snap)
{
var width = SnapExtent(Math.Clamp(placement.Width, MinProgramExtent, 1.0f));
var height = SnapExtent(Math.Clamp(placement.Height, MinProgramExtent, 1.0f));
var centerX = SnapCenter(Math.Clamp(placement.CenterX, width * 0.5f, 1.0f - width * 0.5f), width);
var centerY = SnapCenter(Math.Clamp(placement.CenterY, height * 0.5f, 1.0f - height * 0.5f), height);
var width = Math.Clamp(placement.Width, MinProgramExtent, 1.0f);
var height = Math.Clamp(placement.Height, MinProgramExtent, 1.0f);
if (snap)
{
width = SnapExtent(width);
height = SnapExtent(height);
}
var centerX = Math.Clamp(placement.CenterX, width * 0.5f, 1.0f - width * 0.5f);
var centerY = Math.Clamp(placement.CenterY, height * 0.5f, 1.0f - height * 0.5f);
if (snap)
{
centerX = SnapCenter(centerX, width);
centerY = SnapCenter(centerY, height);
}
return placement with
{
CenterX = centerX,
@ -813,6 +944,26 @@ public sealed class MimirSceneEditorState
};
}
private static IReadOnlyList<AquariumUiPreviewGuide> GuidesForPlacement(MimirCompositorPlacement placement)
{
var guides = new List<AquariumUiPreviewGuide>(6);
AddGuide(guides, "x", placement.CenterX, 0.5f, "cool");
AddGuide(guides, "y", placement.CenterY, 0.5f, "cool");
AddGuide(guides, "x", placement.CenterX - placement.Width * 0.5f, 0.0f, "warm");
AddGuide(guides, "x", placement.CenterX + placement.Width * 0.5f, 1.0f, "warm");
AddGuide(guides, "y", placement.CenterY - placement.Height * 0.5f, 0.0f, "warm");
AddGuide(guides, "y", placement.CenterY + placement.Height * 0.5f, 1.0f, "warm");
return guides;
}
private static void AddGuide(List<AquariumUiPreviewGuide> guides, string axis, float value, float target, string tone)
{
if (MathF.Abs(value - target) <= ProgramSnapThreshold + 0.0001f)
{
guides.Add(new AquariumUiPreviewGuide(axis, target, tone));
}
}
private static float SnapExtent(float value) =>
MathF.Abs(1.0f - value) <= ProgramSnapThreshold ? 1.0f : value;
@ -867,6 +1018,8 @@ public sealed class MimirSceneEditorNode(
public float ProgramAspectRatio { get; init; } = Math.Clamp(defaultTransform.Scale.X / Math.Max(defaultTransform.Scale.Y, 0.001f), 0.05f, 32.0f);
public MimirProgramLayerFitMode FitMode { get; set; } = MimirProgramLayerFitMode.Contain;
public string Text { get; set; } = "";
public string ModelPath { get; init; } = "";
@ -891,3 +1044,10 @@ public enum MimirSceneEditorGizmoMode
Rotate,
Scale,
}
public enum MimirProgramLayerFitMode
{
Contain = 0,
Cover = 1,
Stretch = 2,
}

View file

@ -330,3 +330,4 @@
{"ts": "2026-06-01T13:22:54+01:00", "type": "implementation", "status": "ok", "note": "Added MimirSceneEditorState program-placement affordances: one placement commit path clamps layers inside the program frame, snaps centers/edges near boundaries, and preserves the source aspect ratio for width/height slider edits plus scale-gizmo resizing. --scene-editor-smoke now asserts aspect locking and boundary snapping."}
{"ts": "2026-06-01T13:32:22+01:00", "type": "implementation", "status": "ok", "note": "Direct compositor handles are now wired: Fensalir CultUI preview hit-tests selected item corner handles/body drags and emits normalized AquariumUiPreviewInteraction gestures, while MimirSceneEditorState handles those gestures through the same snapping/aspect-preserving placement owner. Sliders remain fallback/precision controls."}
{"ts": "2026-06-01T13:37:34+01:00", "type": "implementation", "status": "ok", "note": "Gated explicit Mimir inspector source/transform controls behind HasSelectedProgramSource. Direct compositor preview handles remain the primary edit path; inspector sliders/buttons are selected-source precision/fallback controls only. Scene-editor smoke now proves the editor camera is not a program source."}
{"ts": "2026-06-01T14:26:12+01:00", "type": "implementation", "status": "ok", "note": "Picked the compositor UX low-hanging fruit: hover/active preview state, snap guides, empty-canvas deselect, selected-source contain/cover/stretch fit mode, arrow-key nudging that bypasses snap for precision, and PageUp/PageDown layer ordering. Scene-editor smoke now asserts hover, guide, fit, nudge, empty-deselect, direct move/resize handles, aspect lock, snap, and non-source inspector gating."}

View file

@ -65,7 +65,11 @@ invariants:
body/corner-handle gestures wired to the same placement owner; sliders are
fallback/precision controls, not the primary compositor surface. Explicit
source/transform controls are inspector-only and visible only while a
program source is selected.
program source is selected. Empty-canvas clicks clear source selection;
hover/active handle state and snap guides are supplied through Fensalir's
CultUI preview state; keyboard arrows nudge selected layers without
snapping, PageUp/PageDown reorder layers, and the inspector owns the
selected source contain/cover/stretch fit mode.
- `MimirProgramRecorder` owns the first app-level recording control. The Video
Sources rail exposes `Record` / `Stop`, status text, latest output path, and
process-tree shutdown. Its current actuator delegates to