Mimir/docs/network-display-sync.md

Network Display Synchronization

Raven can be a low-level synchronized producer when it controls both its display output and an audio timing signal routed into the Scarlett.

Authority map:

• Owner: Mimir.Runtime owns aligned rolling-buffer frames.
• Inputs: camera buffers, Raven display frames, Scarlett ASIO audio buffers, and explicit timing corrections learned from audio evidence.
• Outputs: MimirSynchronizedBufferFrame slices at one canonical presentation time for Fensalir/Faust/OBS publication.
• Derived state: Raven display timing is derived from the Raven audio sync signal. The display feed is not a timing authority by itself.
• Forbidden writers: OBS, SRT bridge endpoints, frame-event probes, and network receive timestamps must not decide cross-stream alignment. CultMesh reliable UDP packet arrival is transport evidence only, not clock authority.
• Shared path: local cameras, network display frames, mic channels, loopback channels, and Raven audio sync channels all enter MimirRollingStreamBuffer first, then the synchronized planner chooses slices.
• Deletion line: any compositor path that directly consumes Raven pixels outside the runtime buffer/alignment path is diagnostic only.

Practical shape:

1. Raven renders a visible timing marker or encoded frame identity on the captured display.
2. Raven emits the matching audio timing signal through its audio output.
3. That audio is physically or digitally routed into a Scarlett input beside the mic channels.
4. Mimir decodes the Scarlett audio evidence, learns the Raven clock offset, and applies that clock-domain correction to the raven-display video buffer.
5. Fensalir receives aligned buffer slices and still owns composition.

config/mimir-runtime.raven-eve.example.json is the active bridge setup shape for the network producer pass:

• raven-display listens for Raven screen capture on SRT port 5200, decodes through FFmpeg, and emits BGRA video samples into the runtime buffer path.
• eve-camera listens for EveCanvas camera frame-events on WebSocket port 8793.
• eve-mic listens for EveCanvas microphone frame-events on WebSocket port 8794.

The target Verse shape replaces those SRT/WebSocket bridge edges with CultMesh reliable-UDP lanes: typed mimir.cultmesh_stream_frame.v1 envelopes for per-frame media metadata and refs, plus bounded media body shards for payloads too large to inline. Raven display frames remain media observations entering the same rolling buffer; Raven sync still comes from audio evidence and clock-domain correction, not packet arrival.

The older disabled frame-events example in config/mimir-runtime.asio.example.json remains diagnostic only. It can witness source timestamps and frame identity, but it does not carry pixels. The Scarlett channel carrying Raven's sync audio is still the evidence path that earns the raven-sync timing correction.

Smoke proof:

dotnet run --project .\src\Mimir.BufferSmoke\Mimir.BufferSmoke.csproj -- --synchronized-buffer-planner-smoke
dotnet run --project .\src\Mimir.BufferSmoke\Mimir.BufferSmoke.csproj -- --ffmpeg-rawvideo-source-smoke