GameCult/Mimir
Watch
9
0
Fork
You've already forked Mimir
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Emit stereo depth lowering metadata

Browse source
This commit is contained in:
Meta Vens 2026-05-30 00:40:53 +01:00
commit 2273a0c9ab
8 changed files with 108 additions and 24 deletions
Download patch file Download diff file Expand all files Collapse all files

18
docs/implementation-plan.md
View file

@ -43,10 +43,13 @@ monocular model demos are provenance only. The live owner is Fensalir D3D12
compute over Mimir-declared texture resources and calibration state.
The first contract slice now exists in `MimirStereoDepthConfigurations` and
`MimirFensalirFieldLowering.BuildStereoDepthCandidateFrame`: a calibrated
stereo pair profile emits a GPU-resident compute-writable R16F disparity
`SurfacePage`, an R8 confidence texture, and a Height FieldEvidence claim
planned to the `SurfacePage` backend. This is the socket for the kernel, not
the kernel.
stereo pair profile references caller-declared shader-readable left/right input
textures, emits a GPU-resident compute-writable R16F disparity `SurfacePage`,
an R8 confidence texture, a Height FieldEvidence claim planned to the
`SurfacePage` backend, and an `AquariumFieldStereoDepthLowering` sidecar tying
profile, calibration, camera pair, inputs, disparity output, disparity
settings, and depth range together. This is the socket for the kernel, not the
kernel.
Those claims are not inherently pixel-sized. Pixel-level resolve consumes the
reservoir, but claim support is chosen from the represented field. Smooth
@ -317,9 +320,10 @@ a named invariant that the native runtime cannot protect yet.
and `--fensalir-texture-lease-smoke` verifies the engine-owned lease path.
`Mimir.BufferSmoke --stereo-depth-contract-smoke` verifies the next visual
socket: libSGM-provenance D3D12 SGM is represented as a non-dependency
profile, and synthetic rectified Leap stereo depth lowers to one planned
`SurfacePage` packet with no CUDA, CPU disparity image, or monocular metric
authority.
profile, and synthetic rectified Leap stereo depth lowers through
caller-declared left/right input textures, one stereo-depth lowering sidecar,
and one planned `SurfacePage` packet with no CUDA, CPU disparity image, or
monocular metric authority.
- Mimir's active proof path no longer uses the direct
`AquariumAcousticFieldFrame` builder either. Sync states lower into
FieldEvidence calibration constraints through `MapCalibrationConstraints`,

9
docs/perfect-machine-domain-index.md
View file

@ -178,10 +178,11 @@ Likely shape:
then lower into FieldEvidence claims.
- Current contract proof:
`Mimir.BufferSmoke --stereo-depth-contract-smoke` validates the profile and
FieldEvidence socket. The synthetic Leap stereo pair produces an R16F
compute-writable disparity `SurfacePage`, an R8 confidence `Texture2D`, one
Height claim, and one planned `SurfacePage` packet. It is not yet a live
compute kernel.
FieldEvidence socket. The synthetic Leap stereo pair uses caller-declared
shader-readable left/right texture declarations, an R16F compute-writable disparity
`SurfacePage`, an R8 confidence `Texture2D`, one Height claim, one
`AquariumFieldStereoDepthLowering` sidecar, and one planned `SurfacePage`
packet. It is not yet a live compute kernel.
- Native reservoir preserves rolling evidence.
- Fusion emits surface/material/render claims.
- Program video publishes via Spout2.

10
notes/current-system-map.md
View file

@ -409,9 +409,13 @@ research references only.
The first live contract exists: `MimirStereoDepthConfigurations` names the
libSGM-provenance D3D12 SGM profile, and
`MimirFensalirFieldLowering.BuildStereoDepthCandidateFrame` lowers a synthetic
rectified stereo result into a GPU-resident compute-writable disparity
SurfacePage, confidence texture, and Height FieldEvidence claim planned as a
`SurfacePage` packet. The kernel itself is still future work.
rectified stereo result by referencing caller-declared shader-readable
left/right input textures, then emitting a GPU-resident compute-writable
disparity SurfacePage, confidence texture, Height FieldEvidence claim, and
`AquariumFieldStereoDepthLowering` sidecar. The lowering sidecar is the dispatch contract: it binds the
libSGM-provenance profile, calibration id, camera pair, inputs, disparity
output, disparity levels, aggregation paths, and depth range. The kernel itself
is still future work.
Fensalir must not be treated as a traditional rendering pipeline. Mimir does
not ask it to "draw a thing" and hope post-processing makes the result true.

6
notes/fresh-workspace-handoff.md
View file

@ -94,8 +94,10 @@ Get-Content .\state\evidence.jsonl -Tail 8
The first code contract is in `MimirStereoDepthConfigurations` and
`MimirFensalirFieldLowering.BuildStereoDepthCandidateFrame`; run
`dotnet run --project .\src\Mimir.BufferSmoke\Mimir.BufferSmoke.csproj -- --stereo-depth-contract-smoke`
to verify the synthetic compute-writable R16F disparity `SurfacePage`
FieldEvidence socket. This proves the lane shape, not the SGM kernel.
to verify caller-declared synthetic rectified left/right input textures,
compute-writable R16F disparity `SurfacePage`, confidence texture, Height
claim, and `AquariumFieldStereoDepthLowering` dispatch metadata. This proves
the lane shape, not the SGM kernel.
- Fensalir also owns the EVE-facing dashboard pixels through
`Global\MimirFensalirProgramTexture`. `src/Mimir.EveRelay` opens that shared
texture, encodes H.264 Annex-B with NVENC by default, and serves EveCanvas

47
src/Mimir.BufferSmoke/Program.cs
View file

@ -2306,6 +2306,38 @@ static AquariumFieldEvidenceFrame BuildSyntheticStereoDepthFrame(MimirFensalirFi
MaxDepthMeters: 4.0,
Confidence: 0.93,
ObservedTimeNs: 1_000_000_000L)
],
[
new AquariumFieldResourceDeclaration(
ResourceKey: "mimir:resource:texture2d:leap-left-ir",
Kind: AquariumFieldResourceKind.Texture2D,
Residency: AquariumFieldResourceResidency.SharedGpu,
Access: AquariumFieldShaderAccess.ShaderResource,
Format: "R8_UNorm",
Width: 640,
Height: 240,
DepthOrCount: 1,
StrideBytes: 1,
ValidFromNs: 1_000_000_000L,
ValidUntilNs: 1_000_000_000L,
Version: 1_000_000_000UL,
NativeHandle: new IntPtr(0x5001),
NativeHandleKind: "synthetic-shared-d3d12-texture"),
new AquariumFieldResourceDeclaration(
ResourceKey: "mimir:resource:texture2d:leap-right-ir",
Kind: AquariumFieldResourceKind.Texture2D,
Residency: AquariumFieldResourceResidency.SharedGpu,
Access: AquariumFieldShaderAccess.ShaderResource,
Format: "R8_UNorm",
Width: 640,
Height: 240,
DepthOrCount: 1,
StrideBytes: 1,
ValidFromNs: 1_000_000_000L,
ValidUntilNs: 1_000_000_000L,
Version: 1_000_000_000UL,
NativeHandle: new IntPtr(0x5002),
NativeHandleKind: "synthetic-shared-d3d12-texture")
]);
}
@ -2316,6 +2348,7 @@ static int RunStereoDepthContractSmoke()
var validation = AquariumFieldEvidenceValidator.Validate(frame);
var plan = AquariumFieldLoweringPlanner.Plan(frame);
var claim = frame.Claims.FirstOrDefault();
var stereoLowering = frame.StereoDepthLowerings.FirstOrDefault();
var disparity = frame.Resources.FirstOrDefault(resource => resource.ResourceKey == "mimir:resource:stereo-depth:leap-disparity-r16f");
var confidence = frame.Resources.FirstOrDefault(resource => resource.ResourceKey == "mimir:resource:stereo-depth:leap-confidence-r8");
@ -2324,7 +2357,7 @@ static int RunStereoDepthContractSmoke()
Console.WriteLine(
$"stereo-depth-field resources={frame.Resources.Count} claims={frame.Claims.Count} planned={plan.Packets.Count} deferred={plan.DeferredRequests.Count} errors={validation.HasErrors}");
Console.WriteLine(
$"stereo-depth-output disparity={disparity.ResourceKey} kind={disparity.Kind} access={disparity.Access} format={disparity.Format} confidence={confidence.ResourceKey} confidenceFormat={confidence.Format} backend={(plan.Packets.Count > 0 ? plan.Packets[0].Backend : AquariumFieldBackendKind.Unknown)}");
$"stereo-depth-output disparity={disparity.ResourceKey} kind={disparity.Kind} access={disparity.Access} format={disparity.Format} confidence={confidence.ResourceKey} confidenceFormat={confidence.Format} lowerings={frame.StereoDepthLowerings.Count} left={stereoLowering.LeftResourceKey} right={stereoLowering.RightResourceKey} backend={(plan.Packets.Count > 0 ? plan.Packets[0].Backend : AquariumFieldBackendKind.Unknown)}");
return !validation.HasErrors &&
!profile.LiveDependency &&
@ -2333,7 +2366,12 @@ static int RunStereoDepthContractSmoke()
profile.RequiresRectifiedInputs &&
profile.RequiresCalibration &&
profile.NegativeChecks.Any(check => check.Contains("no CUDA", StringComparison.OrdinalIgnoreCase)) &&
frame.Resources.Count == 2 &&
frame.Resources.Count == 4 &&
frame.StereoDepthLowerings.Count == 1 &&
stereoLowering.ProfileKey == profile.Id &&
stereoLowering.LeftResourceKey == "mimir:resource:texture2d:leap-left-ir" &&
stereoLowering.RightResourceKey == "mimir:resource:texture2d:leap-right-ir" &&
stereoLowering.DisparityResourceKey == disparity.ResourceKey &&
disparity.Kind == AquariumFieldResourceKind.SurfacePage &&
disparity.Residency == AquariumFieldResourceResidency.GpuResident &&
disparity.Access == AquariumFieldShaderAccess.UnorderedAccess &&
@ -2517,7 +2555,7 @@ static int RunPerfectMachineProfileSmoke()
Console.WriteLine(
$"perfect-machine-visual-marker claims={visualMarkerFrame.Claims.Count} planned={visualMarkerPlan.Packets.Count} deferred={visualMarkerPlan.DeferredRequests.Count}");
Console.WriteLine(
$"perfect-machine-stereo-depth profiles={stereoDepthProfiles.Count} resources={stereoDepthFrame.Resources.Count} claims={stereoDepthFrame.Claims.Count} planned={stereoDepthPlan.Packets.Count} deferred={stereoDepthPlan.DeferredRequests.Count}");
$"perfect-machine-stereo-depth profiles={stereoDepthProfiles.Count} resources={stereoDepthFrame.Resources.Count} claims={stereoDepthFrame.Claims.Count} lowerings={stereoDepthFrame.StereoDepthLowerings.Count} planned={stereoDepthPlan.Packets.Count} deferred={stereoDepthPlan.DeferredRequests.Count}");
Console.WriteLine(
$"perfect-machine-authority appliesTo=raven-scarlett-witness decision={authority.Decision} rule={authority.RuleId}");
Console.WriteLine(
@ -2563,8 +2601,9 @@ static int RunPerfectMachineProfileSmoke()
visualMarkerPlan.DeferredRequests.Count == 0 &&
visualMarkerFrame.Claims[0].ClaimKey.Contains("synthetic-camera-rig-calibration", StringComparison.Ordinal) &&
visualMarkerFrame.Claims[0].ClaimKey.Contains("synthetic-marker-a", StringComparison.Ordinal) &&
stereoDepthFrame.Resources.Count == 2 &&
stereoDepthFrame.Resources.Count == 4 &&
stereoDepthFrame.Claims.Count == 1 &&
stereoDepthFrame.StereoDepthLowerings.Count == 1 &&
stereoDepthPlan.Packets.Count == 1 &&
stereoDepthPlan.Packets[0].Backend == AquariumFieldBackendKind.SurfacePage &&
stereoDepthPlan.DeferredRequests.Count == 0 &&

31
src/Mimir.Runtime/Synchronization/MimirFensalirFieldLowering.cs
View file

@ -181,15 +181,22 @@ public sealed class MimirFensalirFieldLowering(MimirFensalirLoweringOptions? opt
}
public AquariumFieldEvidenceFrame BuildStereoDepthCandidateFrame(
IEnumerable<MimirStereoDepthFieldCandidate> depthCandidates)
IEnumerable<MimirStereoDepthFieldCandidate> depthCandidates,
IEnumerable<AquariumFieldResourceDeclaration>? inputResources = null)
{
var domains = new List<AquariumFieldDomain>();
var claims = new List<AquariumFieldClaim>();
var candidates = new List<AquariumFieldCandidate>();
var resources = new List<AquariumFieldResourceDeclaration>();
var stereoDepthLowerings = new List<AquariumFieldStereoDepthLowering>();
var seenDomains = new HashSet<string>(StringComparer.Ordinal);
var seenResources = new HashSet<string>(StringComparer.Ordinal);
foreach (var inputResource in inputResources ?? [])
{
AddResource(resources, seenResources, inputResource);
}
foreach (var depthCandidate in depthCandidates)
{
if (string.IsNullOrWhiteSpace(depthCandidate.CandidateKey) ||
@ -206,6 +213,7 @@ public sealed class MimirFensalirFieldLowering(MimirFensalirLoweringOptions? opt
var claim = ClaimForStereoDepthCandidate(depthCandidate, domain.DomainKey);
claims.Add(claim);
candidates.Add(CandidateForClaim(claim, AquariumFieldGuide.Valid(claim.Confidence)));
stereoDepthLowerings.Add(StereoDepthLoweringForCandidate(depthCandidate, claim.ClaimKey));
}
return new AquariumFieldEvidenceFrame
@ -214,6 +222,7 @@ public sealed class MimirFensalirFieldLowering(MimirFensalirLoweringOptions? opt
Claims = claims,
Candidates = candidates,
Resources = resources,
StereoDepthLowerings = stereoDepthLowerings,
AccumulationWindowSeconds = (float)options.AccumulationWindowSeconds,
PresentationDelaySeconds = (float)options.PresentationDelaySeconds
};
@ -744,6 +753,26 @@ public sealed class MimirFensalirFieldLowering(MimirFensalirLoweringOptions? opt
NativeHandleKind: "fensalir-stereo-depth-confidence");
}
private static AquariumFieldStereoDepthLowering StereoDepthLoweringForCandidate(
MimirStereoDepthFieldCandidate depthCandidate,
string claimKey) =>
new(
LoweringKey: $"stereo-depth:{depthCandidate.CalibrationId}:{depthCandidate.CameraPairId}:{depthCandidate.CandidateKey}",
ClaimKey: claimKey,
ProfileKey: depthCandidate.ProfileId,
CalibrationKey: depthCandidate.CalibrationId,
CameraPairKey: depthCandidate.CameraPairId,
LeftResourceKey: depthCandidate.LeftResourceKey,
RightResourceKey: depthCandidate.RightResourceKey,
DisparityResourceKey: depthCandidate.DisparityResourceKey,
ConfidenceResourceKey: depthCandidate.ConfidenceResourceKey,
Width: Math.Max(1, depthCandidate.Width),
Height: Math.Max(1, depthCandidate.Height),
DisparityLevels: Math.Max(1, depthCandidate.DisparityLevels),
AggregationPathCount: Math.Max(1, depthCandidate.AggregationPathCount),
MinDepthMeters: (float)depthCandidate.MinDepthMeters,
MaxDepthMeters: (float)depthCandidate.MaxDepthMeters);
private AquariumFieldSupport SupportForSurfaceIntent(MimirSurfaceIntent intent)
{
var age = Math.Max(0.0f, (float)intent.SupportPolicy.MaximumAgeSeconds);

2
state/evidence.jsonl
View file

@ -232,3 +232,5 @@
{"ts": "2026-05-30T00:21:36+01:00", "type": "implementation", "status": "ok", "note": "Added the first stereo-depth contract slice: MimirStereoDepthConfigurations names a non-dependency libSGM-provenance D3D12 SGM profile, compute/visual/module catalogs include stereo depth, and MimirFensalirFieldLowering.BuildStereoDepthCandidateFrame lowers synthetic rectified stereo depth into GPU-resident R16F disparity SurfacePage plus R8 confidence Texture2D and a Height FieldEvidence claim. Verified with --stereo-depth-contract-smoke and --perfect-machine-profile-smoke; actual HLSL/D3D12 SGM kernel remains future work."}
{"ts": "2026-05-30T00:29:32+01:00", "type": "implementation", "status": "ok", "note": "Stereo-depth FieldEvidence socket now declares its R16F disparity SurfacePage as compute-writable UnorderedAccess, matching Fensalir's new UAV-capable SurfacePage resolver. Verified Mimir build after shutting down stale build servers plus --stereo-depth-contract-smoke and --perfect-machine-profile-smoke; Fensalir focused test verifies UAV SurfacePage allocation/descriptors/reuse. The SGM HLSL kernels still remain future work."}
{"ts": "2026-05-30T00:30:23+01:00", "type": "decision", "status": "ok", "note": "MimirSceneEditorState now owns the Mimir-window editor graph: editor camera, source/text/model nodes, visibility, transform reset, grab/rotate/resize gizmo intent, and derived Fensalir spline/handle presentation. Smoke: --scene-editor-smoke. World SDF glyph rendering, ASSIMP mesh import/upload, and pixel-accurate gizmo hit-testing remain Fensalir renderer cuts."}
{"ts":"2026-05-30T00:38:19+01:00","type":"implementation","status":"ok","note":"Stereo-depth FieldEvidence now emits dispatch metadata, not just an output page. MimirFensalirFieldLowering declares synthetic rectified left/right Texture2D inputs, the UAV R16F disparity SurfacePage, the confidence texture, the Height claim, and an AquariumFieldStereoDepthLowering sidecar carrying profile, calibration, camera pair, inputs, output, disparity settings, and depth range. Verified with Mimir build, --stereo-depth-contract-smoke, --perfect-machine-profile-smoke, and Fensalir StereoDepth contract tests. The SGM HLSL kernel remains future work."}
{"ts":"2026-05-30T00:39:40+01:00","type":"decision","status":"ok","note":"Stereo-depth lowerer does not fabricate shared input handles. BuildStereoDepthCandidateFrame accepts caller-declared input resources, then emits the UAV disparity SurfacePage, confidence texture, Height claim, and AquariumFieldStereoDepthLowering sidecar that references those real texture resource keys. The BufferSmoke proof supplies synthetic input declarations only for contract validation."}

9
state/map.yaml
View file

@ -60,9 +60,12 @@ invariants:
- The first stereo-depth contract is code-visible:
`MimirStereoDepthConfigurations` names the libSGM-provenance D3D12 SGM
profile, and `MimirFensalirFieldLowering.BuildStereoDepthCandidateFrame`
lowers synthetic rectified stereo depth into a compute-writable R16F
disparity SurfacePage, R8 confidence Texture2D, and Height FieldEvidence
claim. The actual SGM HLSL/D3D12 kernel is still future work.
references caller-declared shader-readable left/right Texture2D resources,
then lowers synthetic rectified stereo depth into a compute-writable R16F
disparity SurfacePage, R8 confidence Texture2D, a Height FieldEvidence claim, and an
`AquariumFieldStereoDepthLowering` sidecar tying profile, calibration,
camera pair, inputs, disparity output, and depth range together. The actual
SGM HLSL/D3D12 kernel is still future work.
- Fensalir packet payload handles are not authority. Live payload references
must resolve through declared typed GPU/native resources with shape,
residency, shader access, validity, version, and handle metadata before