diff --git a/config/mimir-runtime.asio.example.json b/config/mimir-runtime.asio.example.json index 98110e5..0704c62 100644 --- a/config/mimir-runtime.asio.example.json +++ b/config/mimir-runtime.asio.example.json @@ -31,6 +31,23 @@ "asio-ch2": "Focusrite Loopback 1", "asio-ch3": "Focusrite Loopback 2" } + }, + { + "sourceId": "raven-display", + "displayName": "Raven network display", + "clockDomainId": "raven-sync", + "kind": "Video", + "origin": "Network", + "enabled": false, + "adapter": "frame-events", + "command": "../native/probes/raven_display_capture_receiver/build/Release/raven_display_capture_receiver.exe", + "arguments": [ + "--emit-json-frames", + "--source-id", + "raven-display", + "--clock-domain", + "raven-sync" + ] } ] } diff --git a/docs/code-algorithm-map.md b/docs/code-algorithm-map.md index 8ebd9be..9ed7367 100644 --- a/docs/code-algorithm-map.md +++ b/docs/code-algorithm-map.md @@ -51,6 +51,9 @@ reflection taps. - The five-second default buffer is compute budget, not accidental delay. - Loopback/reference audio is timing authority; network feeds are producers, not clock authorities. +- `MimirSynchronizedBufferPlanner` owns aligned readout from retained buffers: + it chooses one canonical presentation time and applies source or + `clockDomainId` corrections before Fensalir/Faust consume slices. - UI and telemetry read cached sync state; they do not run analysis. - Native capture owns hardware calls. Runtime owns sample identity and buffer placement. diff --git a/docs/implementation-plan.md b/docs/implementation-plan.md index 68468c1..c049626 100644 --- a/docs/implementation-plan.md +++ b/docs/implementation-plan.md @@ -183,6 +183,16 @@ a named invariant that the native runtime cannot protect yet. instead of reusing a ring slot OBS has not acknowledged. `scripts/build-obs-stem-plugin.ps1` stages the upstream OBS plugin template SDK under `artifacts/obs-sdk/` and builds the plugin DLL locally. +- `MimirSynchronizedBufferPlanner` now builds one aligned presentation frame + from the live rolling buffers. It chooses a canonical presentation time inside + the shared retained window, applies source or clock-domain timing corrections, + and returns per-stream slices for cameras, network display feeds, and audio + blocks. This is the low-level Raven display/audio shape: Raven display frames + can be tagged with `clockDomainId: raven-sync`, while a Raven audio timing + signal routed into Scarlett earns the correction for that shared clock domain. + The display feed remains derived evidence, not an independent timing owner. + `Mimir.BufferSmoke --synchronized-buffer-planner-smoke` proves the aligned + local-camera, Raven-display, loopback, and mic buffer shape. - `MimirRuntime` no longer submits the legacy direct `AquariumSplineFrame` spectrum dashboard. Live spectrum visualization authority is the `AquariumFieldEvidenceFrame`: Mimir declares the rolling resource, emits a diff --git a/docs/network-display-sync.md b/docs/network-display-sync.md new file mode 100644 index 0000000..b4f1b1a --- /dev/null +++ b/docs/network-display-sync.md @@ -0,0 +1,45 @@ +# Network Display Synchronization + +Raven can be a low-level synchronized producer when it controls both its display +output and an audio timing signal routed into the Scarlett. + +Authority map: + +- Owner: `Mimir.Runtime` owns aligned rolling-buffer frames. +- Inputs: camera buffers, Raven display frames, Scarlett ASIO audio buffers, and + explicit timing corrections learned from audio evidence. +- Outputs: `MimirSynchronizedBufferFrame` slices at one canonical presentation + time for Fensalir/Faust/OBS publication. +- Derived state: Raven display timing is derived from the Raven audio sync + signal. The display feed is not a timing authority by itself. +- Forbidden writers: OBS, SRT bridge endpoints, frame-event probes, and network + receive timestamps must not decide cross-stream alignment. +- Shared path: local cameras, network display frames, mic channels, loopback + channels, and Raven audio sync channels all enter `MimirRollingStreamBuffer` + first, then the synchronized planner chooses slices. +- Deletion line: any compositor path that directly consumes Raven pixels outside + the runtime buffer/alignment path is diagnostic only. + +Practical shape: + +1. Raven renders a visible timing marker or encoded frame identity on the + captured display. +2. Raven emits the matching audio timing signal through its audio output. +3. That audio is physically or digitally routed into a Scarlett input beside the + mic channels. +4. Mimir decodes the Scarlett audio evidence, learns the Raven clock offset, and + applies that clock-domain correction to the `raven-display` video buffer. +5. Fensalir receives aligned buffer slices and still owns composition. + +The disabled example in `config/mimir-runtime.asio.example.json` tags the +network display stream with `clockDomainId: raven-sync`. The display receiver is +expected to emit frame-event metadata with Raven source timestamps and frame +identity. The Scarlett channel carrying Raven's sync audio is the evidence path +that earns the `raven-sync` timing correction. + +Smoke proof: + +```powershell +dotnet run --project .\src\Mimir.BufferSmoke\Mimir.BufferSmoke.csproj -- --synchronized-buffer-planner-smoke +``` + diff --git a/notes/current-system-map.md b/notes/current-system-map.md index 265b871..355f304 100644 --- a/notes/current-system-map.md +++ b/notes/current-system-map.md @@ -366,6 +366,14 @@ complete: OBS consumes final Mimir/Fensalir surfaces and stems through the native plugin, raw feeds remain debug inputs, and the ledger records the live program texture, texture ring, dedicated producer fence, and consumer-fence proofs. +`MimirSynchronizedBufferPlanner` is now the low-level aligned-buffer primitive: +it picks one canonical presentation time inside the retained rolling window and +returns per-stream slices for cameras, network display feeds, and audio. Timing +corrections may target either one source or a shared `clockDomainId`. The Raven +shape is explicit: Raven controls both display pixels and an audio timing signal +routed into Scarlett, so Scarlett-decoded Raven audio evidence can earn a +`raven-sync` correction that is applied to the `raven-display` buffer. Network +arrival timestamps remain metadata, not timing authority. ## Visual Fusion diff --git a/src/Mimir.BufferSmoke/Program.cs b/src/Mimir.BufferSmoke/Program.cs index 10d5498..aca7245 100644 --- a/src/Mimir.BufferSmoke/Program.cs +++ b/src/Mimir.BufferSmoke/Program.cs @@ -74,6 +74,11 @@ if (args.Any(arg => string.Equals(arg, "--audio-actuator-bank-smoke", StringComp return RunAudioActuatorBankSmoke(); } +if (args.Any(arg => string.Equals(arg, "--synchronized-buffer-planner-smoke", StringComparison.OrdinalIgnoreCase))) +{ + return RunSynchronizedBufferPlannerSmoke(); +} + if (args.Any(arg => string.Equals(arg, "--audio-stem-publication-smoke", StringComparison.OrdinalIgnoreCase))) { return RunAudioStemPublicationSmoke(); @@ -1893,6 +1898,43 @@ static int RunAudioActuatorBankSmoke() : 1; } +static int RunSynchronizedBufferPlannerSmoke() +{ + const long t0 = 1_000_000_000; + const int sampleRate = 48_000; + var duration = TimeSpan.FromSeconds(5); + var camera = NewVideoBuffer("local-camera-0", MimirStreamOrigin.LocalDevice, t0, 640, 480, MimirVideoPixelFormat.Bgra8, 1); + var display = NewVideoBuffer("raven-display", MimirStreamOrigin.Network, t0 + 10_000_000, 1920, 1080, MimirVideoPixelFormat.Bgra8, 7, "raven-sync"); + var reference = NewAudioBlockBuffer("loopback-scarlett-speakers", t0, sampleRate, 480, 2); + var delayedMic = NewAudioBlockBuffer("asio-ch0", t0 + 480 * 1_000_000_000L / sampleRate, sampleRate, 480, 3); + var state = NewSyncState("asio-ch0", 1920.0, 0.92); + + var frame = new MimirSynchronizedBufferPlanner().BuildFrame( + [camera, display, reference, delayedMic], + TimeSpan.Zero, + [ + .. MimirSynchronizedBufferPlanner.CorrectionsFromAudioStates([state]), + new MimirSourceTimingCorrection("raven-audio-sync", "raven-sync", -10_000_000, 0.88, "scarlett-raven-audio") + ]); + var cameraSlice = frame.Slices.First(slice => string.Equals(slice.SourceId, "local-camera-0", StringComparison.Ordinal)); + var displaySlice = frame.Slices.First(slice => string.Equals(slice.SourceId, "raven-display", StringComparison.Ordinal)); + var micSlice = frame.Slices.First(slice => string.Equals(slice.SourceId, "asio-ch0", StringComparison.Ordinal)); + Console.WriteLine( + $"synchronized-buffer-planner-smoke presentationNs={frame.PresentationTimeNs} slices={frame.Slices.Count} complete={frame.IsComplete} " + + $"camera={cameraSlice.Status} display={displaySlice.Status} displayOrigin={displaySlice.Origin} displayOffsetNs={displaySlice.TimingOffsetNs} micOffsetNs={micSlice.TimingOffsetNs} micStatus={micSlice.Status}"); + + return frame.Slices.Count == 4 && + frame.IsComplete && + displaySlice.Origin == MimirStreamOrigin.Network && + displaySlice.Kind == MimirStreamKind.Video && + displaySlice.TimingOffsetNs == -10_000_000 && + micSlice.TimingOffsetNs == -10_000_000 && + micSlice.Status == MimirSynchronizedSliceStatus.Ready && + cameraSlice.Status != MimirSynchronizedSliceStatus.Missing + ? 0 + : 1; +} + static int RunAudioStemPublicationSmoke() { var publication = new MimirObsStemPublicationState(MimirObsPublicationConfigurations.AlignmentActuatorStemBus); @@ -1981,6 +2023,70 @@ static MimirAudioSynchronizationState NewSyncState(string sourceId, double delay 10, 10); +static MimirRollingStreamBuffer NewVideoBuffer( + string sourceId, + MimirStreamOrigin origin, + long timestampNs, + int width, + int height, + MimirVideoPixelFormat pixelFormat, + ulong sequence, + string clockDomainId = "") +{ + var descriptor = new MimirStreamDescriptor(sourceId, MimirStreamKind.Video, origin, ClockDomainId: clockDomainId); + var buffer = new MimirRollingStreamBuffer(descriptor, TimeSpan.FromSeconds(5)); + var stride = pixelFormat == MimirVideoPixelFormat.Bgra8 + ? width * 4 + : width; + buffer.Append(new MimirStreamSample( + sourceId, + MimirStreamKind.Video, + origin, + timestampNs, + timestampNs, + sequence, + PayloadHandle: sequence, + ByteLength: stride * height, + VideoFrame: new MimirVideoFrameDescriptor( + width, + height, + pixelFormat, + stride, + timestampNs, + NativeHandle: sequence, + NativeHandleKind: "test-shared-texture"))); + return buffer; +} + +static MimirRollingStreamBuffer NewAudioBlockBuffer( + string sourceId, + long timestampNs, + int sampleRate, + int frameCount, + ulong sequence) +{ + var descriptor = new MimirStreamDescriptor(sourceId, MimirStreamKind.Audio, MimirStreamOrigin.LocalDevice); + var buffer = new MimirRollingStreamBuffer(descriptor, TimeSpan.FromSeconds(5)); + var bytes = new byte[frameCount * sizeof(float)]; + buffer.Append(new MimirStreamSample( + sourceId, + MimirStreamKind.Audio, + MimirStreamOrigin.LocalDevice, + timestampNs, + timestampNs, + sequence, + PayloadHandle: 0, + ByteLength: bytes.Length, + Data: bytes, + AudioBlock: new MimirAudioBlockDescriptor( + sampleRate, + 1, + MimirAudioSampleFormat.Float32, + frameCount, + timestampNs))); + return buffer; +} + static int RunComplexContourTrackerSelfTest(int sampleRate, double delaySamples, double reflectionDelaySamples) { var renderer = new MimirBioacousticContestantRenderer(MimirBioacousticContestants.CanaryPacketTrill); diff --git a/src/Mimir.Runtime/Synchronization/MimirRuntimeConfiguration.cs b/src/Mimir.Runtime/Synchronization/MimirRuntimeConfiguration.cs index 49440ed..cda45f4 100644 --- a/src/Mimir.Runtime/Synchronization/MimirRuntimeConfiguration.cs +++ b/src/Mimir.Runtime/Synchronization/MimirRuntimeConfiguration.cs @@ -102,7 +102,8 @@ public sealed class MimirRuntimeConfiguration ParseKind(stream.Kind), ParseOrigin(stream.Origin), stream.Enabled, - stream.DisplayName); + stream.DisplayName, + stream.ClockDomainId); } private static IEnumerable ToDescriptors(MimirStreamConfig stream) @@ -120,7 +121,8 @@ public sealed class MimirRuntimeConfiguration ParseKind(stream.Kind), ParseOrigin(stream.Origin), stream.Enabled, - stream.DisplayNameForSource(sourceId)); + stream.DisplayNameForSource(sourceId), + stream.ClockDomainId); } } @@ -360,6 +362,8 @@ public sealed class MimirStreamConfig public string DisplayName { get; set; } = ""; + public string ClockDomainId { get; set; } = ""; + public string Kind { get; set; } = "Video"; public string Origin { get; set; } = "LocalDevice"; diff --git a/src/Mimir.Runtime/Synchronization/MimirStreamDescriptor.cs b/src/Mimir.Runtime/Synchronization/MimirStreamDescriptor.cs index bd7527e..8e14a90 100644 --- a/src/Mimir.Runtime/Synchronization/MimirStreamDescriptor.cs +++ b/src/Mimir.Runtime/Synchronization/MimirStreamDescriptor.cs @@ -5,11 +5,16 @@ public sealed record MimirStreamDescriptor( MimirStreamKind Kind, MimirStreamOrigin Origin, bool Enabled = true, - string DisplayName = "") + string DisplayName = "", + string ClockDomainId = "") { public string BufferKey => $"{Kind}:{Origin}:{SourceId}"; public string Label => string.IsNullOrWhiteSpace(DisplayName) ? SourceId : DisplayName; + + public string EffectiveClockDomainId => string.IsNullOrWhiteSpace(ClockDomainId) + ? SourceId + : ClockDomainId; } diff --git a/src/Mimir.Runtime/Synchronization/MimirSynchronizationHub.cs b/src/Mimir.Runtime/Synchronization/MimirSynchronizationHub.cs index 876caf0..5542d68 100644 --- a/src/Mimir.Runtime/Synchronization/MimirSynchronizationHub.cs +++ b/src/Mimir.Runtime/Synchronization/MimirSynchronizationHub.cs @@ -57,6 +57,16 @@ public sealed class MimirSynchronizationHub : IDisposable public bool ComplexContourRuntimeEnabled => complexContourSynchronization != null; + public MimirSynchronizedBufferFrame BuildSynchronizedBufferFrame(TimeSpan? presentationDelay = null) + { + ObjectDisposedException.ThrowIf(disposed, this); + var delay = presentationDelay ?? TimeSpan.FromTicks(Settings.BufferDuration.Ticks / 2); + return new MimirSynchronizedBufferPlanner().BuildFrame( + Buffers.Buffers, + delay, + MimirSynchronizedBufferPlanner.CorrectionsFromAudioStates(audioSynchronizationState.States)); + } + public void AddSource(IMimirStreamSource source) { ObjectDisposedException.ThrowIf(disposed, this); diff --git a/src/Mimir.Runtime/Synchronization/MimirSynchronizedBufferPlanner.cs b/src/Mimir.Runtime/Synchronization/MimirSynchronizedBufferPlanner.cs new file mode 100644 index 0000000..299ffbe --- /dev/null +++ b/src/Mimir.Runtime/Synchronization/MimirSynchronizedBufferPlanner.cs @@ -0,0 +1,267 @@ +namespace Mimir.Runtime.Synchronization; + +public enum MimirSynchronizedSliceStatus +{ + Missing, + Ready, + HoldingPreviousSample, + FutureSampleOnly, +} + +public readonly record struct MimirSourceTimingCorrection( + string SourceId, + string ClockDomainId, + long OffsetNs, + double Confidence, + string EvidenceKind) +{ + public long ToCanonicalNs(long sourceTimestampNs) => checked(sourceTimestampNs + OffsetNs); + + public bool AppliesTo(MimirStreamDescriptor descriptor) => + string.Equals(SourceId, descriptor.SourceId, StringComparison.Ordinal) || + (!string.IsNullOrWhiteSpace(ClockDomainId) && + string.Equals(ClockDomainId, descriptor.EffectiveClockDomainId, StringComparison.Ordinal)); +} + +public readonly record struct MimirSynchronizedStreamSlice( + string SourceId, + MimirStreamKind Kind, + MimirStreamOrigin Origin, + MimirSynchronizedSliceStatus Status, + MimirStreamSample? Sample, + long SourceTimestampNs, + long CanonicalStartNs, + long CanonicalEndNs, + long PresentationTimeNs, + long TimingOffsetNs, + long DistanceFromPresentationNs, + double TimingConfidence, + string TimingEvidenceKind); + +public sealed record MimirSynchronizedBufferFrame( + long PresentationTimeNs, + long WindowStartNs, + long WindowEndNs, + TimeSpan PresentationDelay, + IReadOnlyList Slices) +{ + public bool IsComplete => Slices.Count > 0 && Slices.All(static slice => slice.Status != MimirSynchronizedSliceStatus.Missing); + + public IEnumerable VideoSlices => + Slices.Where(static slice => slice.Kind == MimirStreamKind.Video); + + public IEnumerable AudioSlices => + Slices.Where(static slice => slice.Kind == MimirStreamKind.Audio); +} + +public sealed class MimirSynchronizedBufferPlanner +{ + public static IReadOnlyList CorrectionsFromAudioStates( + IEnumerable states) + { + var corrections = new List(); + var references = new HashSet(StringComparer.Ordinal); + foreach (var state in states) + { + if (state.SampleRate <= 0) + { + continue; + } + + if (references.Add(state.ReferenceSourceId)) + { + corrections.Add(new MimirSourceTimingCorrection( + state.ReferenceSourceId, + "", + 0, + 1.0, + "reference")); + } + + var offsetNs = checked((long)Math.Round(-state.SmoothedDelaySamples * 1_000_000_000.0 / state.SampleRate)); + corrections.Add(new MimirSourceTimingCorrection( + state.SourceId, + "", + offsetNs, + Math.Clamp(state.Confidence, 0.0, 1.0), + "audio-sync")); + } + + return corrections; + } + + public MimirSynchronizedBufferFrame BuildFrame( + IEnumerable buffers, + TimeSpan presentationDelay, + IEnumerable? timingCorrections = null) + { + var activeBuffers = buffers + .Where(static buffer => buffer.Latest.HasValue) + .OrderBy(static buffer => buffer.Descriptor.SourceId, StringComparer.Ordinal) + .ToArray(); + if (activeBuffers.Length == 0) + { + return new MimirSynchronizedBufferFrame(0, 0, 0, presentationDelay, []); + } + + var corrections = BuildCorrectionMap(timingCorrections); + var windowStartNs = activeBuffers.Max(buffer => Correct(corrections, buffer.Descriptor, buffer.WindowStartNs)); + var windowEndNs = activeBuffers.Min(buffer => + { + var latest = buffer.Latest!.Value; + return SampleCanonicalEndNs(latest, CorrectionFor(corrections, buffer.Descriptor)); + }); + var delayNs = checked((long)Math.Round(Math.Max(0.0, presentationDelay.TotalSeconds) * 1_000_000_000.0)); + var presentationTimeNs = Math.Clamp(windowEndNs - delayNs, windowStartNs, windowEndNs); + + var slices = new List(activeBuffers.Length); + foreach (var buffer in activeBuffers) + { + slices.Add(SelectSlice(buffer, presentationTimeNs, CorrectionFor(corrections, buffer.Descriptor))); + } + + return new MimirSynchronizedBufferFrame( + presentationTimeNs, + windowStartNs, + windowEndNs, + presentationDelay, + slices); + } + + private static Dictionary BuildCorrectionMap( + IEnumerable? timingCorrections) + { + var map = new Dictionary(StringComparer.Ordinal); + if (timingCorrections == null) + { + return map; + } + + foreach (var correction in timingCorrections) + { + if (!string.IsNullOrWhiteSpace(correction.SourceId)) + { + map[$"source:{correction.SourceId}"] = correction; + } + + if (!string.IsNullOrWhiteSpace(correction.ClockDomainId)) + { + map[$"clock:{correction.ClockDomainId}"] = correction; + } + } + + return map; + } + + private static MimirSourceTimingCorrection CorrectionFor( + IReadOnlyDictionary corrections, + MimirStreamDescriptor descriptor) + { + if (corrections.TryGetValue($"source:{descriptor.SourceId}", out var sourceCorrection)) + { + return sourceCorrection; + } + + if (!string.IsNullOrWhiteSpace(descriptor.EffectiveClockDomainId) && + corrections.TryGetValue($"clock:{descriptor.EffectiveClockDomainId}", out var clockCorrection)) + { + return clockCorrection; + } + + return new MimirSourceTimingCorrection(descriptor.SourceId, descriptor.EffectiveClockDomainId, 0, 0.0, ""); + } + + private static long Correct( + IReadOnlyDictionary corrections, + MimirStreamDescriptor descriptor, + long timestampNs) => + CorrectionFor(corrections, descriptor).ToCanonicalNs(timestampNs); + + private static MimirSynchronizedStreamSlice SelectSlice( + MimirRollingStreamBuffer buffer, + long presentationTimeNs, + MimirSourceTimingCorrection correction) + { + var samples = buffer.Snapshot(); + if (samples.Count == 0) + { + return new MimirSynchronizedStreamSlice( + buffer.Descriptor.SourceId, + buffer.Descriptor.Kind, + buffer.Descriptor.Origin, + MimirSynchronizedSliceStatus.Missing, + null, + 0, + 0, + 0, + presentationTimeNs, + correction.OffsetNs, + long.MaxValue, + correction.Confidence, + correction.EvidenceKind); + } + + MimirStreamSample? best = null; + MimirSynchronizedSliceStatus status = MimirSynchronizedSliceStatus.Missing; + long bestDistance = long.MaxValue; + foreach (var sample in samples) + { + var startNs = correction.ToCanonicalNs(sample.TimestampNs); + var endNs = SampleCanonicalEndNs(sample, correction); + if (presentationTimeNs >= startNs && presentationTimeNs < endNs) + { + best = sample; + status = MimirSynchronizedSliceStatus.Ready; + bestDistance = 0; + break; + } + + var distance = presentationTimeNs < startNs + ? startNs - presentationTimeNs + : presentationTimeNs - endNs; + if (distance < bestDistance) + { + best = sample; + bestDistance = distance; + status = presentationTimeNs >= endNs + ? MimirSynchronizedSliceStatus.HoldingPreviousSample + : MimirSynchronizedSliceStatus.FutureSampleOnly; + } + } + + var selected = best!.Value; + var selectedStartNs = correction.ToCanonicalNs(selected.TimestampNs); + var selectedEndNs = SampleCanonicalEndNs(selected, correction); + return new MimirSynchronizedStreamSlice( + buffer.Descriptor.SourceId, + buffer.Descriptor.Kind, + buffer.Descriptor.Origin, + status, + selected, + selected.TimestampNs, + selectedStartNs, + selectedEndNs, + presentationTimeNs, + correction.OffsetNs, + bestDistance, + correction.Confidence, + correction.EvidenceKind); + } + + private static long SampleCanonicalEndNs(MimirStreamSample sample, MimirSourceTimingCorrection correction) + { + var startNs = correction.ToCanonicalNs(sample.TimestampNs); + var durationNs = SampleDurationNs(sample); + return checked(startNs + Math.Max(1, durationNs)); + } + + private static long SampleDurationNs(MimirStreamSample sample) + { + if (sample.AudioBlock is { SampleRate: > 0, FrameCount: > 0 } block) + { + return checked((long)Math.Ceiling(block.FrameCount * 1_000_000_000.0 / block.SampleRate)); + } + + return 1; + } +} diff --git a/state/evidence.jsonl b/state/evidence.jsonl index a6ecbb8..8c92471 100644 --- a/state/evidence.jsonl +++ b/state/evidence.jsonl @@ -226,3 +226,4 @@ {"ts": "2026-05-28T21:45:00+01:00", "type": "correction", "status": "ok", "note": "Noisy reservoir smoke traced to TubeField reuse/export, not Mimir packet lowering. Fensalir now re-evaluates shifted temporal/spatial TubeField samples at the receiving pixel and exports the best four deterministic current candidates into shared reservoir rows. Verified with Fensalir/Mimir builds, --mimir-spectrum-upload-smoke, and headless capture artifacts/runtime/reservoir-deterministic-top4-spectrum.png. Remaining dashboard busyness is many age-column tubes plus generated proxy segment structure, not fixed by smoke timing."} {"ts": "2026-05-28T22:05:00+01:00", "type": "decision", "status": "ok", "note": "For many-hundreds-spline rendering, Fensalir stays on cheap segmented TubeField proxies rather than spline-wide SDF evaluation. The current cut fixes segment continuity by carrying consistent start/end radius pairs through generated proxy vertices and storing true start/end radii in compute segment packets. Verified with Fensalir/Mimir builds, --mimir-spectrum-upload-smoke, and headless capture artifacts/runtime/tubefield-cheap-continuity-spectrum.png."} {"timestamp":"2026-05-29T01:46:00+01:00","type":"decision","status":"ok","note":"Fensalir TubeField spectrum rendering now uses rolling-buffer column packets as reservoir candidate authority rather than generated segment packets. Expansion emits one GPU-resident column packet per logical history/source column, ReSTIR evaluates Catmull-Rom SDF/material samples directly from the source buffer, and generated mesh draw args are zeroed for the live path. Verified through Mimir.App headless synthetic spectrum capture artifacts/runtime/mimir-column-packet-final-headless.png."} +{"ts": "2026-05-29T22:23:04+01:00", "type": "decision", "status": "ok", "note": "Raven display/audio sync uses shared clock-domain correction: Raven display frames can be tagged raven-sync, while Raven audio routed into Scarlett earns the timing offset applied by MimirSynchronizedBufferPlanner; network arrival timestamps remain metadata."} diff --git a/state/map.yaml b/state/map.yaml index 932df0f..872f9db 100644 --- a/state/map.yaml +++ b/state/map.yaml @@ -217,6 +217,13 @@ accepted_design: through the native plugin, raw feeds remain debug inputs, and the ledger records the live texture, texture ring, producer-fence, and consumer-fence proofs. + `MimirSynchronizedBufferPlanner` is the first aligned-buffer readout + primitive: it chooses one canonical presentation time inside the rolling + window and returns per-stream slices for local cameras, network display + feeds, and audio blocks. Timing corrections can target a single source or a + shared `clockDomainId`; the Raven path uses audio evidence routed into + Scarlett to earn a `raven-sync` correction for Raven display frames. + Network arrival timestamps remain metadata only. `scripts/build-obs-stem-plugin.ps1` stages the upstream OBS plugin template SDK under `artifacts/obs-sdk/` and builds the plugin DLL locally. The synthetic