diff --git a/docs/implementation-plan.md b/docs/implementation-plan.md index 0f0a426..409dcc3 100644 --- a/docs/implementation-plan.md +++ b/docs/implementation-plan.md @@ -43,10 +43,13 @@ monocular model demos are provenance only. The live owner is Fensalir D3D12 compute over Mimir-declared texture resources and calibration state. The first contract slice now exists in `MimirStereoDepthConfigurations` and `MimirFensalirFieldLowering.BuildStereoDepthCandidateFrame`: a calibrated -stereo pair profile emits a GPU-resident compute-writable R16F disparity -`SurfacePage`, an R8 confidence texture, and a Height FieldEvidence claim -planned to the `SurfacePage` backend. This is the socket for the kernel, not -the kernel. +stereo pair profile references caller-declared shader-readable left/right input +textures, emits a GPU-resident compute-writable R16F disparity `SurfacePage`, +an R8 confidence texture, a Height FieldEvidence claim planned to the +`SurfacePage` backend, and an `AquariumFieldStereoDepthLowering` sidecar tying +profile, calibration, camera pair, inputs, disparity output, disparity +settings, and depth range together. This is the socket for the kernel, not the +kernel. Those claims are not inherently pixel-sized. Pixel-level resolve consumes the reservoir, but claim support is chosen from the represented field. Smooth @@ -317,9 +320,10 @@ a named invariant that the native runtime cannot protect yet. and `--fensalir-texture-lease-smoke` verifies the engine-owned lease path. `Mimir.BufferSmoke --stereo-depth-contract-smoke` verifies the next visual socket: libSGM-provenance D3D12 SGM is represented as a non-dependency - profile, and synthetic rectified Leap stereo depth lowers to one planned - `SurfacePage` packet with no CUDA, CPU disparity image, or monocular metric - authority. + profile, and synthetic rectified Leap stereo depth lowers through + caller-declared left/right input textures, one stereo-depth lowering sidecar, + and one planned `SurfacePage` packet with no CUDA, CPU disparity image, or + monocular metric authority. - Mimir's active proof path no longer uses the direct `AquariumAcousticFieldFrame` builder either. Sync states lower into FieldEvidence calibration constraints through `MapCalibrationConstraints`, diff --git a/docs/perfect-machine-domain-index.md b/docs/perfect-machine-domain-index.md index 8c40e6b..5b65d2d 100644 --- a/docs/perfect-machine-domain-index.md +++ b/docs/perfect-machine-domain-index.md @@ -178,10 +178,11 @@ Likely shape: then lower into FieldEvidence claims. - Current contract proof: `Mimir.BufferSmoke --stereo-depth-contract-smoke` validates the profile and - FieldEvidence socket. The synthetic Leap stereo pair produces an R16F - compute-writable disparity `SurfacePage`, an R8 confidence `Texture2D`, one - Height claim, and one planned `SurfacePage` packet. It is not yet a live - compute kernel. + FieldEvidence socket. The synthetic Leap stereo pair uses caller-declared + shader-readable left/right texture declarations, an R16F compute-writable disparity + `SurfacePage`, an R8 confidence `Texture2D`, one Height claim, one + `AquariumFieldStereoDepthLowering` sidecar, and one planned `SurfacePage` + packet. It is not yet a live compute kernel. - Native reservoir preserves rolling evidence. - Fusion emits surface/material/render claims. - Program video publishes via Spout2. diff --git a/notes/current-system-map.md b/notes/current-system-map.md index c740b9f..c03c048 100644 --- a/notes/current-system-map.md +++ b/notes/current-system-map.md @@ -409,9 +409,13 @@ research references only. The first live contract exists: `MimirStereoDepthConfigurations` names the libSGM-provenance D3D12 SGM profile, and `MimirFensalirFieldLowering.BuildStereoDepthCandidateFrame` lowers a synthetic -rectified stereo result into a GPU-resident compute-writable disparity -SurfacePage, confidence texture, and Height FieldEvidence claim planned as a -`SurfacePage` packet. The kernel itself is still future work. +rectified stereo result by referencing caller-declared shader-readable +left/right input textures, then emitting a GPU-resident compute-writable +disparity SurfacePage, confidence texture, Height FieldEvidence claim, and +`AquariumFieldStereoDepthLowering` sidecar. The lowering sidecar is the dispatch contract: it binds the +libSGM-provenance profile, calibration id, camera pair, inputs, disparity +output, disparity levels, aggregation paths, and depth range. The kernel itself +is still future work. Fensalir must not be treated as a traditional rendering pipeline. Mimir does not ask it to "draw a thing" and hope post-processing makes the result true. diff --git a/notes/fresh-workspace-handoff.md b/notes/fresh-workspace-handoff.md index 1b6ca41..0fea3d5 100644 --- a/notes/fresh-workspace-handoff.md +++ b/notes/fresh-workspace-handoff.md @@ -94,8 +94,10 @@ Get-Content .\state\evidence.jsonl -Tail 8 The first code contract is in `MimirStereoDepthConfigurations` and `MimirFensalirFieldLowering.BuildStereoDepthCandidateFrame`; run `dotnet run --project .\src\Mimir.BufferSmoke\Mimir.BufferSmoke.csproj -- --stereo-depth-contract-smoke` - to verify the synthetic compute-writable R16F disparity `SurfacePage` - FieldEvidence socket. This proves the lane shape, not the SGM kernel. + to verify caller-declared synthetic rectified left/right input textures, + compute-writable R16F disparity `SurfacePage`, confidence texture, Height + claim, and `AquariumFieldStereoDepthLowering` dispatch metadata. This proves + the lane shape, not the SGM kernel. - Fensalir also owns the EVE-facing dashboard pixels through `Global\MimirFensalirProgramTexture`. `src/Mimir.EveRelay` opens that shared texture, encodes H.264 Annex-B with NVENC by default, and serves EveCanvas diff --git a/src/Mimir.BufferSmoke/Program.cs b/src/Mimir.BufferSmoke/Program.cs index f0753f7..c2dd4c1 100644 --- a/src/Mimir.BufferSmoke/Program.cs +++ b/src/Mimir.BufferSmoke/Program.cs @@ -2306,6 +2306,38 @@ static AquariumFieldEvidenceFrame BuildSyntheticStereoDepthFrame(MimirFensalirFi MaxDepthMeters: 4.0, Confidence: 0.93, ObservedTimeNs: 1_000_000_000L) + ], + [ + new AquariumFieldResourceDeclaration( + ResourceKey: "mimir:resource:texture2d:leap-left-ir", + Kind: AquariumFieldResourceKind.Texture2D, + Residency: AquariumFieldResourceResidency.SharedGpu, + Access: AquariumFieldShaderAccess.ShaderResource, + Format: "R8_UNorm", + Width: 640, + Height: 240, + DepthOrCount: 1, + StrideBytes: 1, + ValidFromNs: 1_000_000_000L, + ValidUntilNs: 1_000_000_000L, + Version: 1_000_000_000UL, + NativeHandle: new IntPtr(0x5001), + NativeHandleKind: "synthetic-shared-d3d12-texture"), + new AquariumFieldResourceDeclaration( + ResourceKey: "mimir:resource:texture2d:leap-right-ir", + Kind: AquariumFieldResourceKind.Texture2D, + Residency: AquariumFieldResourceResidency.SharedGpu, + Access: AquariumFieldShaderAccess.ShaderResource, + Format: "R8_UNorm", + Width: 640, + Height: 240, + DepthOrCount: 1, + StrideBytes: 1, + ValidFromNs: 1_000_000_000L, + ValidUntilNs: 1_000_000_000L, + Version: 1_000_000_000UL, + NativeHandle: new IntPtr(0x5002), + NativeHandleKind: "synthetic-shared-d3d12-texture") ]); } @@ -2316,6 +2348,7 @@ static int RunStereoDepthContractSmoke() var validation = AquariumFieldEvidenceValidator.Validate(frame); var plan = AquariumFieldLoweringPlanner.Plan(frame); var claim = frame.Claims.FirstOrDefault(); + var stereoLowering = frame.StereoDepthLowerings.FirstOrDefault(); var disparity = frame.Resources.FirstOrDefault(resource => resource.ResourceKey == "mimir:resource:stereo-depth:leap-disparity-r16f"); var confidence = frame.Resources.FirstOrDefault(resource => resource.ResourceKey == "mimir:resource:stereo-depth:leap-confidence-r8"); @@ -2324,7 +2357,7 @@ static int RunStereoDepthContractSmoke() Console.WriteLine( $"stereo-depth-field resources={frame.Resources.Count} claims={frame.Claims.Count} planned={plan.Packets.Count} deferred={plan.DeferredRequests.Count} errors={validation.HasErrors}"); Console.WriteLine( - $"stereo-depth-output disparity={disparity.ResourceKey} kind={disparity.Kind} access={disparity.Access} format={disparity.Format} confidence={confidence.ResourceKey} confidenceFormat={confidence.Format} backend={(plan.Packets.Count > 0 ? plan.Packets[0].Backend : AquariumFieldBackendKind.Unknown)}"); + $"stereo-depth-output disparity={disparity.ResourceKey} kind={disparity.Kind} access={disparity.Access} format={disparity.Format} confidence={confidence.ResourceKey} confidenceFormat={confidence.Format} lowerings={frame.StereoDepthLowerings.Count} left={stereoLowering.LeftResourceKey} right={stereoLowering.RightResourceKey} backend={(plan.Packets.Count > 0 ? plan.Packets[0].Backend : AquariumFieldBackendKind.Unknown)}"); return !validation.HasErrors && !profile.LiveDependency && @@ -2333,7 +2366,12 @@ static int RunStereoDepthContractSmoke() profile.RequiresRectifiedInputs && profile.RequiresCalibration && profile.NegativeChecks.Any(check => check.Contains("no CUDA", StringComparison.OrdinalIgnoreCase)) && - frame.Resources.Count == 2 && + frame.Resources.Count == 4 && + frame.StereoDepthLowerings.Count == 1 && + stereoLowering.ProfileKey == profile.Id && + stereoLowering.LeftResourceKey == "mimir:resource:texture2d:leap-left-ir" && + stereoLowering.RightResourceKey == "mimir:resource:texture2d:leap-right-ir" && + stereoLowering.DisparityResourceKey == disparity.ResourceKey && disparity.Kind == AquariumFieldResourceKind.SurfacePage && disparity.Residency == AquariumFieldResourceResidency.GpuResident && disparity.Access == AquariumFieldShaderAccess.UnorderedAccess && @@ -2517,7 +2555,7 @@ static int RunPerfectMachineProfileSmoke() Console.WriteLine( $"perfect-machine-visual-marker claims={visualMarkerFrame.Claims.Count} planned={visualMarkerPlan.Packets.Count} deferred={visualMarkerPlan.DeferredRequests.Count}"); Console.WriteLine( - $"perfect-machine-stereo-depth profiles={stereoDepthProfiles.Count} resources={stereoDepthFrame.Resources.Count} claims={stereoDepthFrame.Claims.Count} planned={stereoDepthPlan.Packets.Count} deferred={stereoDepthPlan.DeferredRequests.Count}"); + $"perfect-machine-stereo-depth profiles={stereoDepthProfiles.Count} resources={stereoDepthFrame.Resources.Count} claims={stereoDepthFrame.Claims.Count} lowerings={stereoDepthFrame.StereoDepthLowerings.Count} planned={stereoDepthPlan.Packets.Count} deferred={stereoDepthPlan.DeferredRequests.Count}"); Console.WriteLine( $"perfect-machine-authority appliesTo=raven-scarlett-witness decision={authority.Decision} rule={authority.RuleId}"); Console.WriteLine( @@ -2563,8 +2601,9 @@ static int RunPerfectMachineProfileSmoke() visualMarkerPlan.DeferredRequests.Count == 0 && visualMarkerFrame.Claims[0].ClaimKey.Contains("synthetic-camera-rig-calibration", StringComparison.Ordinal) && visualMarkerFrame.Claims[0].ClaimKey.Contains("synthetic-marker-a", StringComparison.Ordinal) && - stereoDepthFrame.Resources.Count == 2 && + stereoDepthFrame.Resources.Count == 4 && stereoDepthFrame.Claims.Count == 1 && + stereoDepthFrame.StereoDepthLowerings.Count == 1 && stereoDepthPlan.Packets.Count == 1 && stereoDepthPlan.Packets[0].Backend == AquariumFieldBackendKind.SurfacePage && stereoDepthPlan.DeferredRequests.Count == 0 && diff --git a/src/Mimir.Runtime/Synchronization/MimirFensalirFieldLowering.cs b/src/Mimir.Runtime/Synchronization/MimirFensalirFieldLowering.cs index edb0e85..0cb1399 100644 --- a/src/Mimir.Runtime/Synchronization/MimirFensalirFieldLowering.cs +++ b/src/Mimir.Runtime/Synchronization/MimirFensalirFieldLowering.cs @@ -181,15 +181,22 @@ public sealed class MimirFensalirFieldLowering(MimirFensalirLoweringOptions? opt } public AquariumFieldEvidenceFrame BuildStereoDepthCandidateFrame( - IEnumerable depthCandidates) + IEnumerable depthCandidates, + IEnumerable? inputResources = null) { var domains = new List(); var claims = new List(); var candidates = new List(); var resources = new List(); + var stereoDepthLowerings = new List(); var seenDomains = new HashSet(StringComparer.Ordinal); var seenResources = new HashSet(StringComparer.Ordinal); + foreach (var inputResource in inputResources ?? []) + { + AddResource(resources, seenResources, inputResource); + } + foreach (var depthCandidate in depthCandidates) { if (string.IsNullOrWhiteSpace(depthCandidate.CandidateKey) || @@ -206,6 +213,7 @@ public sealed class MimirFensalirFieldLowering(MimirFensalirLoweringOptions? opt var claim = ClaimForStereoDepthCandidate(depthCandidate, domain.DomainKey); claims.Add(claim); candidates.Add(CandidateForClaim(claim, AquariumFieldGuide.Valid(claim.Confidence))); + stereoDepthLowerings.Add(StereoDepthLoweringForCandidate(depthCandidate, claim.ClaimKey)); } return new AquariumFieldEvidenceFrame @@ -214,6 +222,7 @@ public sealed class MimirFensalirFieldLowering(MimirFensalirLoweringOptions? opt Claims = claims, Candidates = candidates, Resources = resources, + StereoDepthLowerings = stereoDepthLowerings, AccumulationWindowSeconds = (float)options.AccumulationWindowSeconds, PresentationDelaySeconds = (float)options.PresentationDelaySeconds }; @@ -744,6 +753,26 @@ public sealed class MimirFensalirFieldLowering(MimirFensalirLoweringOptions? opt NativeHandleKind: "fensalir-stereo-depth-confidence"); } + private static AquariumFieldStereoDepthLowering StereoDepthLoweringForCandidate( + MimirStereoDepthFieldCandidate depthCandidate, + string claimKey) => + new( + LoweringKey: $"stereo-depth:{depthCandidate.CalibrationId}:{depthCandidate.CameraPairId}:{depthCandidate.CandidateKey}", + ClaimKey: claimKey, + ProfileKey: depthCandidate.ProfileId, + CalibrationKey: depthCandidate.CalibrationId, + CameraPairKey: depthCandidate.CameraPairId, + LeftResourceKey: depthCandidate.LeftResourceKey, + RightResourceKey: depthCandidate.RightResourceKey, + DisparityResourceKey: depthCandidate.DisparityResourceKey, + ConfidenceResourceKey: depthCandidate.ConfidenceResourceKey, + Width: Math.Max(1, depthCandidate.Width), + Height: Math.Max(1, depthCandidate.Height), + DisparityLevels: Math.Max(1, depthCandidate.DisparityLevels), + AggregationPathCount: Math.Max(1, depthCandidate.AggregationPathCount), + MinDepthMeters: (float)depthCandidate.MinDepthMeters, + MaxDepthMeters: (float)depthCandidate.MaxDepthMeters); + private AquariumFieldSupport SupportForSurfaceIntent(MimirSurfaceIntent intent) { var age = Math.Max(0.0f, (float)intent.SupportPolicy.MaximumAgeSeconds); diff --git a/state/evidence.jsonl b/state/evidence.jsonl index 8595c46..25a0c29 100644 --- a/state/evidence.jsonl +++ b/state/evidence.jsonl @@ -232,3 +232,5 @@ {"ts": "2026-05-30T00:21:36+01:00", "type": "implementation", "status": "ok", "note": "Added the first stereo-depth contract slice: MimirStereoDepthConfigurations names a non-dependency libSGM-provenance D3D12 SGM profile, compute/visual/module catalogs include stereo depth, and MimirFensalirFieldLowering.BuildStereoDepthCandidateFrame lowers synthetic rectified stereo depth into GPU-resident R16F disparity SurfacePage plus R8 confidence Texture2D and a Height FieldEvidence claim. Verified with --stereo-depth-contract-smoke and --perfect-machine-profile-smoke; actual HLSL/D3D12 SGM kernel remains future work."} {"ts": "2026-05-30T00:29:32+01:00", "type": "implementation", "status": "ok", "note": "Stereo-depth FieldEvidence socket now declares its R16F disparity SurfacePage as compute-writable UnorderedAccess, matching Fensalir's new UAV-capable SurfacePage resolver. Verified Mimir build after shutting down stale build servers plus --stereo-depth-contract-smoke and --perfect-machine-profile-smoke; Fensalir focused test verifies UAV SurfacePage allocation/descriptors/reuse. The SGM HLSL kernels still remain future work."} {"ts": "2026-05-30T00:30:23+01:00", "type": "decision", "status": "ok", "note": "MimirSceneEditorState now owns the Mimir-window editor graph: editor camera, source/text/model nodes, visibility, transform reset, grab/rotate/resize gizmo intent, and derived Fensalir spline/handle presentation. Smoke: --scene-editor-smoke. World SDF glyph rendering, ASSIMP mesh import/upload, and pixel-accurate gizmo hit-testing remain Fensalir renderer cuts."} +{"ts":"2026-05-30T00:38:19+01:00","type":"implementation","status":"ok","note":"Stereo-depth FieldEvidence now emits dispatch metadata, not just an output page. MimirFensalirFieldLowering declares synthetic rectified left/right Texture2D inputs, the UAV R16F disparity SurfacePage, the confidence texture, the Height claim, and an AquariumFieldStereoDepthLowering sidecar carrying profile, calibration, camera pair, inputs, output, disparity settings, and depth range. Verified with Mimir build, --stereo-depth-contract-smoke, --perfect-machine-profile-smoke, and Fensalir StereoDepth contract tests. The SGM HLSL kernel remains future work."} +{"ts":"2026-05-30T00:39:40+01:00","type":"decision","status":"ok","note":"Stereo-depth lowerer does not fabricate shared input handles. BuildStereoDepthCandidateFrame accepts caller-declared input resources, then emits the UAV disparity SurfacePage, confidence texture, Height claim, and AquariumFieldStereoDepthLowering sidecar that references those real texture resource keys. The BufferSmoke proof supplies synthetic input declarations only for contract validation."} diff --git a/state/map.yaml b/state/map.yaml index d36965d..26c136c 100644 --- a/state/map.yaml +++ b/state/map.yaml @@ -60,9 +60,12 @@ invariants: - The first stereo-depth contract is code-visible: `MimirStereoDepthConfigurations` names the libSGM-provenance D3D12 SGM profile, and `MimirFensalirFieldLowering.BuildStereoDepthCandidateFrame` - lowers synthetic rectified stereo depth into a compute-writable R16F - disparity SurfacePage, R8 confidence Texture2D, and Height FieldEvidence - claim. The actual SGM HLSL/D3D12 kernel is still future work. + references caller-declared shader-readable left/right Texture2D resources, + then lowers synthetic rectified stereo depth into a compute-writable R16F + disparity SurfacePage, R8 confidence Texture2D, a Height FieldEvidence claim, and an + `AquariumFieldStereoDepthLowering` sidecar tying profile, calibration, + camera pair, inputs, disparity output, and depth range together. The actual + SGM HLSL/D3D12 kernel is still future work. - Fensalir packet payload handles are not authority. Live payload references must resolve through declared typed GPU/native resources with shape, residency, shader access, validity, version, and handle metadata before