From ddad34f69bbc710d2a04b656a6d02e9472f513ea Mon Sep 17 00:00:00 2001 From: Meta Vens Date: Wed, 27 May 2026 12:44:01 +0100 Subject: [PATCH] Bridge camera observations through field evidence --- docs/fensalir-rendering-rebuild-migration.md | 38 +++--- docs/implementation-plan.md | 22 ++-- notes/current-system-map.md | 15 ++- src/Mimir.BufferSmoke/Program.cs | 117 ++++++++++++++++-- src/Mimir.Runtime/MimirRuntime.cs | 5 + .../MimirFensalirFieldLowering.cs | 81 +++--------- state/evidence.jsonl | 1 + state/map.yaml | 10 +- 8 files changed, 190 insertions(+), 99 deletions(-) diff --git a/docs/fensalir-rendering-rebuild-migration.md b/docs/fensalir-rendering-rebuild-migration.md index 401dac9..e26820a 100644 --- a/docs/fensalir-rendering-rebuild-migration.md +++ b/docs/fensalir-rendering-rebuild-migration.md @@ -24,16 +24,8 @@ drivers / ASIO / network feeds -> OBS ``` -But the current debug-spectrum path still exposes the pressure leak: - -```text -ASIO spectrum history --> AquariumSplineFrame --> direct Fensalir spline tube debug surface -``` - -That path is allowed as a deterministic diagnostic. It is not yet the full -Perfect Machine. The production shape is: +The old direct debug-spectrum pressure leak has been cut. Current spectrum +debug output is already in the production-shaped bridge: ```text rolling buffer slice + source identity + calibration + surface intent @@ -93,8 +85,9 @@ Derived state: - spectrum lines, debug views, screenshots, OBS bridge endpoints, FFmpeg/SRT helpers, and Eve dashboard feeds are views or diagnostics. -- The old direct spline-spectrum dashboard is silent. FieldEvidence TubeField - owns the spectrum surface path; screenshots and logs remain diagnostics. +- The old direct spline-spectrum dashboard and old direct GPU sensor frame + bridge are silent. FieldEvidence owns spectrum and camera observation + publication; screenshots and logs remain diagnostics. Forbidden writers: @@ -200,6 +193,11 @@ Current proof surface: - `Mimir.BufferSmoke --fensalir-field-dsl-resource-smoke` uses Fensalir's field evidence DSL to bind the declared resource directly and produce one planned `TubeField` packet with no deferred requests. +- `Mimir.BufferSmoke --fensalir-camera-observation-smoke` maps one + GPU-backed video frame and one metadata-only cadence witness into field + evidence. Only the shared D3D12 video handle becomes a declared `Texture2D` + resource and camera surface intent; the metadata-only frame remains + observation evidence with no render payload. Fensalir now owns the first D3D12 resource resolver cut below this contract: shared structured/curve buffers import/alias GPU-resident resources by handle, @@ -250,6 +248,13 @@ can distinguish source-lane claims. Mimir exposes `MIMIR_SPECTRUM_SOURCE_LANES` as the source-lane capacity lever and `MIMIR_SPECTRUM_TUBE_SUBDIVISIONS` as the geometry cost lever for responding to Fensalir's requested/dispatched/truncated/invalid TubeField budget report. +Camera descriptors now follow the same evidence/resource boundary: +`MimirFensalirFieldLowering.BuildCameraObservationFrame` lowers latest video +buffers into observation claims and declares GPU-backed frames as shared +`Texture2D` resources. Metadata-only/process-cadence frames do not create +surface intents, so a missing payload cannot become a fake `latest-window` +render request. Camera image claims currently defer because Fensalir has not +selected a visual-fusion/render lowering for camera textures yet. ### 3. Calibration Constraint @@ -374,9 +379,12 @@ Lower native camera descriptors into observation surfaces: Acceptance: -- at least one local camera stream appears as Fensalir sensor observations; -- missing frames are absence, not zero-valued evidence; -- Fensalir owns any feature extraction result. +- implemented bridge proof: GPU-backed video descriptors become declared + shared `Texture2D` resources plus camera observation/surface claims; +- metadata-only cadence descriptors remain observation evidence and do not + become render payloads; +- no selected camera-image lowering exists yet, so Fensalir defers those claims + until visual fusion/feature extraction owns them. ### Phase 4: Audio Field Observation Bridge diff --git a/docs/implementation-plan.md b/docs/implementation-plan.md index 7903af6..935b8c8 100644 --- a/docs/implementation-plan.md +++ b/docs/implementation-plan.md @@ -174,6 +174,14 @@ a named invariant that the native runtime cannot protect yet. `Mimir.BufferSmoke --fensalir-field-dsl-resource-smoke` proves Fensalir's DSL evidence compiler can bind a declared resource and produce one planned `TubeField` packet with no deferred requests. +- `MimirFensalirFieldLowering.BuildCameraObservationFrame` is the current + camera bridge proof. It lowers latest video buffers into FieldEvidence + observation claims, declares GPU/native video payloads as shared `Texture2D` + resources, and creates camera surface intent only when a resource exists. + Metadata-only cadence frames remain observations with empty payload handles; + they do not become fake render requests. The old direct + `AquariumGpuSensorFrame` builder has been removed from Mimir's proof path. + `Mimir.BufferSmoke --fensalir-camera-observation-smoke` verifies the split. - Fensalir now owns the first in-process D3D12 field resource resolver cut: shared structured/curve buffer resources import/alias GPU-resident handles, and Fensalir-owned resources allocate GPU slots only when Fensalir is the @@ -300,8 +308,8 @@ a named invariant that the native runtime cannot protect yet. proof, `--perfect-machine-manifest` to export the module manifest for tooling/UI/remote witness use, and `--perfect-machine-lowering-benchmark` to measure the Mimir-to-Fensalir lowering path. The current six-camera/two-audio - synthetic lowering benchmark runs at roughly 2.5 us per iteration with about - 2.3 KB allocated per iteration. + synthetic lowering benchmark now measures FieldEvidence camera/resource + lowering rather than the retired direct GPU sensor DTO path. - `MimirVideoFrameDescriptor` for dimensions, pixel format, stride, device timestamp, and native/GPU handle metadata. - `IMimirVideoCaptureDriver` and `MimirVideoCaptureDriverSource` as the live @@ -359,11 +367,11 @@ a named invariant that the native runtime cannot protect yet. volume textures, and mesh packages, and the evidence DSL can plan generic resource-backed claims against those resources. The next engine cuts are selected render lowerings that consume those resources as mesh geometry, - height/SDF/material pages, and density/extinction/SDF3D volumes. The generic - imported mesh ABI is no longer the blocker; TubeField now occupies the first - generated-mesh lowering lane. The next renderer decisions are how to expose - additional generated mesh producers and which page/volume/material lowering - should follow. + camera/feature textures, height/SDF/material pages, and + density/extinction/SDF3D volumes. The generic imported mesh ABI is no longer + the blocker; TubeField now occupies the first generated-mesh lowering lane. + The next renderer decisions are how to expose additional generated mesh + producers and which camera/page/volume/material lowering should follow. 9. Move mic alignment, room suppression, voice separation, spatialization, and stem generation into Faust/native DSP. 10. Keep the OBS bridge witness ledger as evidence before expanding receiver diff --git a/notes/current-system-map.md b/notes/current-system-map.md index b84a3a8..0a8994c 100644 --- a/notes/current-system-map.md +++ b/notes/current-system-map.md @@ -326,11 +326,19 @@ shader lowering. Mimir-declared rendering buffers are Fensalir GPU resources in the same runtime; native/shared handle metadata is an import edge, not a separate authority regime. Rendering-relevant buffers move to GPU residency as early as possible and stay there; CPU readback/tessellation is diagnostic only. +Camera descriptors now enter this bridge too: latest video buffers lower into +FieldEvidence camera observations, GPU/native handles declare shared +`Texture2D` resources, and metadata-only cadence frames do not create camera +surface intents or fake payload requests. The old direct `AquariumGpuSensorFrame` +bridge proof has been removed from Mimir's active proof path. Camera image +claims currently defer until Fensalir owns a selected visual-fusion lowering for +camera textures. The current smokes prove planning, not visible packet rendering: `--fensalir-field-evidence-smoke` now produces one planned resource-backed `TubeField` packet from Mimir's spectrum intent, and `--fensalir-field-dsl-resource-smoke` proves the Fensalir DSL can bind the same -kind of declared resource directly. Fensalir now has the first D3D12 resolver +kind of declared resource directly. `--fensalir-camera-observation-smoke` +proves the camera observation/resource split. Fensalir now has the first D3D12 resolver cut for structured/curve buffers: it imports shared GPU buffers by handle and allocates engine-owned GPU slots only for Fensalir-produced resources. The DSL can describe a 2D rolling float buffer as Catmull-Rom XY tubes with modulo @@ -341,8 +349,9 @@ textures allocate shader-readable GPU textures, and mesh packages own `Mesh.Vertices`/`Mesh.Indices` GPU buffer shape under one resource key. The DSL can plan generic resource-backed claims over those declarations. The next blocker is no longer resolver ownership; it is selected render lowerings that -consume mesh/page/volume resources as geometry, height/SDF/material pages, or -density/extinction/SDF3D domains. Mesh layout authority is split by source: +consume camera/feature textures, mesh/page/volume resources as geometry, +height/SDF/material pages, or density/extinction/SDF3D domains. Mesh layout +authority is split by source: imported/user meshes use the standard `PositionNormalUvColor` layout, and generated meshes can be `PipelinePrivate` so each lowering owns the bytes it emits and consumes. TubeField is the first concrete generated-mesh consumer: diff --git a/src/Mimir.BufferSmoke/Program.cs b/src/Mimir.BufferSmoke/Program.cs index b433309..4e7e5e5 100644 --- a/src/Mimir.BufferSmoke/Program.cs +++ b/src/Mimir.BufferSmoke/Program.cs @@ -103,6 +103,11 @@ if (args.Any(arg => string.Equals(arg, "--fensalir-field-dsl-resource-smoke", St return RunFensalirFieldDslResourceSmoke(); } +if (args.Any(arg => string.Equals(arg, "--fensalir-camera-observation-smoke", StringComparison.OrdinalIgnoreCase))) +{ + return RunFensalirCameraObservationSmoke(); +} + if (args.Any(arg => string.Equals(arg, "--mimir-spectrum-upload-smoke", StringComparison.OrdinalIgnoreCase))) { return RunMimirSpectrumUploadSmoke(); @@ -627,6 +632,92 @@ tubespline id=spectrum-trail resource=spectrum domain=mimir:domain:spectrum conf : 1; } +static int RunFensalirCameraObservationSmoke() +{ + var gpuBuffer = new MimirRollingStreamBuffer( + new MimirStreamDescriptor("kiyo-pro-rgb", MimirStreamKind.Video, MimirStreamOrigin.LocalDevice), + TimeSpan.FromSeconds(5)); + gpuBuffer.Append(new MimirStreamSample( + "kiyo-pro-rgb", + MimirStreamKind.Video, + MimirStreamOrigin.LocalDevice, + TimestampNs: 1_000_000_000L, + ArrivalNs: 1_000_000_010L, + Sequence: 7, + PayloadHandle: 0, + ByteLength: 1920 * 1080 * 4, + Data: default, + VideoFrame: new MimirVideoFrameDescriptor( + 1920, + 1080, + MimirVideoPixelFormat.Bgra8, + 1920 * 4, + 1_000_000_000L, + NativeHandle: 0xCAFEul, + NativeHandleKind: "shared-d3d12-texture"))); + + var metadataOnlyBuffer = new MimirRollingStreamBuffer( + new MimirStreamDescriptor("json-cadence-witness", MimirStreamKind.Video, MimirStreamOrigin.LocalDevice), + TimeSpan.FromSeconds(5)); + metadataOnlyBuffer.Append(new MimirStreamSample( + "json-cadence-witness", + MimirStreamKind.Video, + MimirStreamOrigin.LocalDevice, + TimestampNs: 1_000_000_050L, + ArrivalNs: 1_000_000_060L, + Sequence: 8, + PayloadHandle: 0, + ByteLength: 0, + Data: default, + VideoFrame: new MimirVideoFrameDescriptor( + 640, + 480, + MimirVideoPixelFormat.Gray8, + 640, + 1_000_000_050L))); + + MimirRollingStreamBuffer[] buffers = [gpuBuffer, metadataOnlyBuffer]; + var lowerer = new MimirFensalirFieldLowering(); + var frame = lowerer.BuildCameraObservationFrame(buffers); + var validation = AquariumFieldEvidenceValidator.Validate(frame); + var plan = AquariumFieldLoweringPlanner.Plan(frame); + var requests = AquariumFieldEvidenceNormalizer.BuildLoweringRequests(frame); + var cameraResourceKey = "mimir:resource:shared-d3d12-texture:cafe"; + + Console.WriteLine( + $"fensalir-camera-observation-smoke domains={frame.Domains.Count} resources={frame.Resources.Count} claims={frame.Claims.Count} requests={requests.Count} planned={plan.Packets.Count} deferred={plan.DeferredRequests.Count} errors={validation.HasErrors}"); + + if (validation.HasErrors) + { + foreach (var issue in validation.Issues) + { + Console.Error.WriteLine($"camera-observation-issue {issue.Severity} {issue.Key}: {issue.Message}"); + } + } + + return !validation.HasErrors && + frame.Resources.Count == 1 && + frame.Resources[0].ResourceKey == cameraResourceKey && + frame.Resources[0].Kind == AquariumFieldResourceKind.Texture2D && + frame.Resources[0].Residency == AquariumFieldResourceResidency.SharedGpu && + frame.Resources[0].Access == AquariumFieldShaderAccess.ShaderResource && + frame.Resources[0].Format == nameof(MimirVideoPixelFormat.Bgra8) && + frame.Resources[0].Width == 1920 && + frame.Resources[0].Height == 1080 && + frame.Resources[0].NativeHandle == new IntPtr(0xCAFE) && + frame.Claims.Count == 3 && + frame.Claims.Any(claim => claim.ClaimKey.StartsWith("observation:Video:LocalDevice:json-cadence-witness", StringComparison.Ordinal) && + string.IsNullOrWhiteSpace(claim.PayloadHandle)) && + frame.Claims.Any(claim => claim.ClaimKey.StartsWith("intent:Video:LocalDevice:kiyo-pro-rgb:surface", StringComparison.Ordinal) && + claim.PayloadHandle == cameraResourceKey) && + frame.Claims.All(claim => !string.Equals(claim.PayloadHandle, "latest-window", StringComparison.Ordinal)) && + requests.Count == 3 && + plan.Packets.Count == 0 && + plan.DeferredRequests.Count == 3 + ? 0 + : 1; +} + static int RunMimirSpectrumUploadSmoke() { var previousSyntheticPreview = Environment.GetEnvironmentVariable("MIMIR_SYNTHETIC_SPECTRUM_PREVIEW"); @@ -1604,6 +1695,7 @@ static int RunPerfectMachineProfileSmoke() ArrivalNs: 1_000_000_010L, Sequence: 1, PayloadHandle: 0, + ByteLength: 1920 * 1080 * 4, VideoFrame: new MimirVideoFrameDescriptor( 1920, 1080, @@ -1613,7 +1705,8 @@ static int RunPerfectMachineProfileSmoke() NativeHandle: 42, NativeHandleKind: "shared-d3d12-texture"))); var lowerer = new MimirFensalirFieldLowering(); - var gpuFrame = lowerer.BuildGpuSensorFrame([videoBuffer]); + var cameraFieldFrame = lowerer.BuildCameraObservationFrame([videoBuffer]); + var cameraPlan = AquariumFieldLoweringPlanner.Plan(cameraFieldFrame); var acousticFrame = lowerer.BuildAcousticFieldFrame([ new MimirAudioSynchronizationState( "loopback-scarlett-speakers", @@ -1665,7 +1758,7 @@ static int RunPerfectMachineProfileSmoke() Console.WriteLine( $"perfect-machine-clock anchors={clock?.AnchorCount ?? 0} coverage={clock?.AnchorCoverage ?? 0:0.000} offset={clock?.SourceOffsetSamples ?? double.NaN:0.000} confidence={clock?.Confidence ?? 0:0.000}"); Console.WriteLine( - $"perfect-machine-fensalir gpuCameras={gpuFrame.Cameras.Count} gpuTextures={gpuFrame.ExternalTextures.Count} acousticConstraints={acousticFrame.Constraints.Count} timingConfidence={acousticFrame.TimingConfidence:0.000}"); + $"perfect-machine-fensalir cameraResources={cameraFieldFrame.Resources.Count} cameraClaims={cameraFieldFrame.Claims.Count} cameraPlanned={cameraPlan.Packets.Count} cameraDeferred={cameraPlan.DeferredRequests.Count} acousticConstraints={acousticFrame.Constraints.Count} timingConfidence={acousticFrame.TimingConfidence:0.000}"); Console.WriteLine( $"perfect-machine-localization candidates={localizationGrid.Count} best=({localization?.PositionMeters.X ?? float.NaN:0.000},{localization?.PositionMeters.Y ?? float.NaN:0.000},{localization?.PositionMeters.Z ?? float.NaN:0.000}) score={localization?.Score ?? 0.0:0.000}"); Console.WriteLine( @@ -1695,7 +1788,10 @@ static int RunPerfectMachineProfileSmoke() codebook.Motifs.Length == MimirBioacousticTimeline.SymbolCount && decoder.Configuration.TemplateAugmentations.Length > 0 && actuator.FaustControls.Length >= 2 && - gpuFrame.HasInput && + cameraFieldFrame.Resources.Count == 1 && + cameraFieldFrame.Claims.Count == 2 && + cameraPlan.Packets.Count == 0 && + cameraPlan.DeferredRequests.Count == 2 && acousticFrame.HasInput && localization is { Score: > 0.90 } && authority.Decision == MimirAuthorityDecision.TrustedEvidence && @@ -1913,6 +2009,7 @@ static int RunPerfectMachineLoweringBenchmark(int iterations) ArrivalNs: 1_000_000_050L, Sequence: 1, PayloadHandle: 0, + ByteLength: Math.Max(1, profile.PreferredWidth) * Math.Max(1, profile.PreferredHeight) * 4, VideoFrame: new MimirVideoFrameDescriptor( Math.Max(1, profile.PreferredWidth), Math.Max(1, profile.PreferredHeight), @@ -1954,18 +2051,20 @@ static int RunPerfectMachineLoweringBenchmark(int iterations) 1) }; var lowerer = new MimirFensalirFieldLowering(); - _ = lowerer.BuildGpuSensorFrame(buffers); + _ = lowerer.BuildCameraObservationFrame(buffers); _ = lowerer.BuildAcousticFieldFrame(states); var allocatedBefore = GC.GetAllocatedBytesForCurrentThread(); var stamp = Stopwatch.GetTimestamp(); - var cameras = 0; + var cameraResources = 0; + var cameraClaims = 0; var constraints = 0; for (var iteration = 0; iteration < iterations; iteration++) { - var gpuFrame = lowerer.BuildGpuSensorFrame(buffers); + var cameraFrame = lowerer.BuildCameraObservationFrame(buffers); var acousticFrame = lowerer.BuildAcousticFieldFrame(states); - cameras += gpuFrame.Cameras.Count; + cameraResources += cameraFrame.Resources.Count; + cameraClaims += cameraFrame.Claims.Count; constraints += acousticFrame.Constraints.Count; } @@ -1974,8 +2073,8 @@ static int RunPerfectMachineLoweringBenchmark(int iterations) var perIterationUs = elapsed.TotalMilliseconds * 1000.0 / iterations; var allocatedPerIteration = allocatedBytes / (double)iterations; Console.WriteLine( - $"perfect-machine-lowering-benchmark iterations={iterations} camerasPer={cameras / (double)iterations:0.0} constraintsPer={constraints / (double)iterations:0.0} elapsedMs={elapsed.TotalMilliseconds:0.000} perIterationUs={perIterationUs:0.000} allocatedPerIterationBytes={allocatedPerIteration:0.0}"); - return perIterationUs < 250.0 && allocatedPerIteration < 10_000.0 + $"perfect-machine-lowering-benchmark iterations={iterations} cameraResourcesPer={cameraResources / (double)iterations:0.0} cameraClaimsPer={cameraClaims / (double)iterations:0.0} constraintsPer={constraints / (double)iterations:0.0} elapsedMs={elapsed.TotalMilliseconds:0.000} perIterationUs={perIterationUs:0.000} allocatedPerIterationBytes={allocatedPerIteration:0.0}"); + return perIterationUs < 250.0 && allocatedPerIteration < 64_000.0 ? 0 : 1; } diff --git a/src/Mimir.Runtime/MimirRuntime.cs b/src/Mimir.Runtime/MimirRuntime.cs index 5ac8247..af29eee 100644 --- a/src/Mimir.Runtime/MimirRuntime.cs +++ b/src/Mimir.Runtime/MimirRuntime.cs @@ -204,6 +204,11 @@ public sealed class MimirRuntime : IAquariumRuntime continue; } + if (!window.Payload.HasResource) + { + continue; + } + intents.Add(BuildSurfaceIntent(window, observation)); } diff --git a/src/Mimir.Runtime/Synchronization/MimirFensalirFieldLowering.cs b/src/Mimir.Runtime/Synchronization/MimirFensalirFieldLowering.cs index 563542c..9e2a3fe 100644 --- a/src/Mimir.Runtime/Synchronization/MimirFensalirFieldLowering.cs +++ b/src/Mimir.Runtime/Synchronization/MimirFensalirFieldLowering.cs @@ -87,53 +87,31 @@ public sealed class MimirFensalirFieldLowering(MimirFensalirLoweringOptions? opt }; } - public AquariumGpuSensorFrame BuildGpuSensorFrame(IEnumerable buffers) + public AquariumFieldEvidenceFrame BuildCameraObservationFrame(IEnumerable buffers) { - var capacity = buffers.TryGetNonEnumeratedCount(out var count) ? count : 0; - var textures = new List(capacity); - var cameras = new List(capacity); - foreach (var buffer in buffers) + var videoBuffers = buffers + .Where(static buffer => buffer.Descriptor.Kind == MimirStreamKind.Video) + .ToArray(); + var windows = new List(videoBuffers.Length); + var observations = new List(videoBuffers.Length); + MimirFensalirBridgeMapper.MapWindows(videoBuffers, windows); + MimirFensalirBridgeMapper.MapLatestObservations(videoBuffers, observations); + + var observationsByWindow = observations.ToDictionary(static observation => observation.WindowId, StringComparer.Ordinal); + var intents = new List(windows.Count); + foreach (var window in windows) { - if (buffer.Descriptor.Kind != MimirStreamKind.Video || buffer.Latest?.VideoFrame is not { } frame) + if (window.Status != MimirBridgeWindowStatus.Live || + !window.Payload.HasResource || + !observationsByWindow.TryGetValue(window.WindowId, out var observation)) { continue; } - var firstTexture = textures.Count; - if (frame.NativeHandle != 0) - { - textures.Add(new AquariumExternalGpuTexture( - default, - new IntPtr(unchecked((long)frame.NativeHandle)), - frame.Width, - frame.Height, - ToFensalirPixelFormat(frame.PixelFormat), - frame.DeviceTimestampNs, - SharedHandleName: frame.NativeHandleKind)); - } - - cameras.Add(new AquariumGpuSensorCamera( - buffer.Descriptor.SourceId, - ToFensalirSensorKind(buffer.Descriptor.SourceId, frame.PixelFormat), - Matrix4x4.Identity, - Matrix4x4.Identity, - Vector4.Zero, - Vector4.Zero, - Vector4.Zero, - frame.Width, - frame.Height, - firstTexture, - textures.Count - firstTexture, - frame.DeviceTimestampNs)); + intents.Add(MimirFensalirBridgeMapper.MapDefaultSurfaceIntent(window, observation)); } - return new AquariumGpuSensorFrame - { - Cameras = cameras, - ExternalTextures = textures, - AccumulationWindowSeconds = (float)options.AccumulationWindowSeconds, - PresentationDelaySeconds = (float)options.PresentationDelaySeconds - }; + return BuildFieldEvidenceFrame(windows, observations, [], intents); } public AquariumAcousticFieldFrame BuildAcousticFieldFrame(IEnumerable states) @@ -185,31 +163,6 @@ public sealed class MimirFensalirFieldLowering(MimirFensalirLoweringOptions? opt }; } - private static AquariumGpuSensorKind ToFensalirSensorKind(string sourceId, MimirVideoPixelFormat pixelFormat) - { - if (pixelFormat == MimirVideoPixelFormat.LeapStereoIr || sourceId.Contains("leap", StringComparison.OrdinalIgnoreCase)) - { - return AquariumGpuSensorKind.LeapPackedMap; - } - - if (sourceId.Contains("eye", StringComparison.OrdinalIgnoreCase)) - { - return AquariumGpuSensorKind.HighRateTracker; - } - - return AquariumGpuSensorKind.RgbCamera; - } - - private static AquariumGpuSensorPixelFormat ToFensalirPixelFormat(MimirVideoPixelFormat pixelFormat) => - pixelFormat switch - { - MimirVideoPixelFormat.Gray8 or MimirVideoPixelFormat.R8 or MimirVideoPixelFormat.Bayer8 => AquariumGpuSensorPixelFormat.R8Unorm, - MimirVideoPixelFormat.Rg8 => AquariumGpuSensorPixelFormat.Rg8Unorm, - MimirVideoPixelFormat.Bgra8 => AquariumGpuSensorPixelFormat.Bgra8Unorm, - MimirVideoPixelFormat.LeapStereoIr => AquariumGpuSensorPixelFormat.LeapPackedMap, - _ => AquariumGpuSensorPixelFormat.Unknown - }; - private static void AddDomain( ICollection domains, ISet seenDomains, diff --git a/state/evidence.jsonl b/state/evidence.jsonl index 898d1a8..ac217d2 100644 --- a/state/evidence.jsonl +++ b/state/evidence.jsonl @@ -170,3 +170,4 @@ {"ts": "2026-05-27T12:00:41+01:00", "type": "implementation", "status": "ok", "note": "Mimir now emits partial spectrum resource uploads for the active ring slots it samples. The declared GPU buffer remains fixed at 48 bands x 8 lanes x 40 history slots, but the smoke now verifies three offset uploads totaling 1152 floats for the three active ages instead of a full 15360-float backing-store upload. Headless Fensalir rendered the offset-copy path with empty stderr."} {"ts": "2026-05-27T12:21:38+01:00", "type": "implementation", "status": "ok", "note": "Mimir now sends only the newest spectrum ring slot as an offset Float32 resource upload. Fensalir owns rolling structured-buffer slot validity and clamps TubeField dispatch to valid contiguous logical columns, so invalid older slots cannot reach shader sampling after renderer allocation or partial update. Smoke reports uploadFloats=384 and headless render succeeded with empty stderr."} {"ts": "2026-05-27T12:32:42+01:00", "type": "implementation", "status": "ok", "note": "Updated docs/fensalir-rendering-rebuild-migration.md so the rendering migration map matches the live spectrum TubeField path: newest-slot Float32 uploads, fixed source-lane capacity, RollingOffset ring addressing, Fensalir-owned valid-slot gating, stable claim-derived TubeField metadata ids, and the current budget levers."} +{"ts": "2026-05-27T12:43:45+01:00", "type": "implementation", "status": "ok", "note": "Camera descriptors now lower through FieldEvidence instead of the retired direct GPU sensor frame bridge. GPU/native video handles declare shared Texture2D resources and camera claims; metadata-only cadence frames remain observations with no render payload, so missing pixels cannot become fake latest-window requests. --fensalir-camera-observation-smoke verifies the split and camera image claims defer until Fensalir owns a selected visual-fusion lowering."} diff --git a/state/map.yaml b/state/map.yaml index 51af716..53998c9 100644 --- a/state/map.yaml +++ b/state/map.yaml @@ -299,7 +299,15 @@ accepted_design: non-backend claims, and no validation errors. `--fensalir-field-dsl-resource-smoke` proves Fensalir's field evidence DSL can bind a declared resource directly and produce one planned `TubeField` - packet with no deferred requests. Fensalir now owns the first in-process + packet with no deferred requests. `MimirFensalirFieldLowering` also owns + the current camera observation bridge: latest video buffers lower into + FieldEvidence camera claims, GPU/native video handles declare shared + Texture2D resources, and metadata-only cadence frames do not create camera + surface intents or fake payload requests. The old direct + `AquariumGpuSensorFrame` builder is no longer the active proof path; + `Mimir.BufferSmoke --fensalir-camera-observation-smoke` verifies the split. + Camera image claims currently defer until Fensalir has a selected visual + fusion/feature extraction lowering for camera textures. Fensalir now owns the first in-process D3D12 field resource registry: declared shared structured/curve buffers import/alias GPU-resident resources by handle, while Fensalir-owned resources allocate GPU slots only when Fensalir is the producer.