OBS Spout Streaming

Objective

Publish the live visual fusion state as a named GPU texture that OBS can receive through the Spout2 plugin.

This is the deadline stream surface. It is not a preview window and it is not a desktop-capture workaround. LocalCastBridge emits a typed render-frame packet; the Spout sink turns that packet into a GPU texture named for OBS.

Current Mechanism

flowchart TD
    A["Sensor fusion producer"] --> B["CultCache render-frame doc"]
    B --> C["scripts/stream_spout.py"]
    C --> D["ScreenBrushPacket lowering"]
    D --> E["OpenGL texture/FBO"]
    E --> F["Spout sender texture"]
    F --> G["OBS Spout2 Capture"]

Aquarium Engine remains the intended authority for dense brush/splat rendering. The useful boundary is the typed localcast.visual.render_frame CultCache document: Aquarium can replace the OpenGL renderer behind the same packet contract without changing capture, fusion, timing, or OBS ingestion.

The deadline sink borrows the Zyphos brush rule rather than pretending points are enough: each render point lowers into a compact anisotropic screen brush with center, radii, rotation, and color. That shape mirrors Aquarium's bounded brush/splat direction while keeping the stream path simple enough to verify under pressure.

Run

Install calibration/runtime dependencies once:

.\.venv\Scripts\python.exe -m pip install -r .\requirements-calibration.txt

Start the Spout sender:

.\.venv\Scripts\python.exe .\scripts\stream_spout.py `
  --source cultcache `
  --frame-cache .\calibration\runs\visual-state.msgpack `
  --demo-if-missing `
  --sender-name "LocalCastBridge Point Cloud" `
  --width 1920 `
  --height 1080 `
  --fps 60 `
  --status .\calibration\runs\stream-spout-status.json `
  --status-cache .\calibration\runs\visual-stream-status.msgpack `
  --audio-cache .\calibration\runs\audio-state.msgpack `
  --audio-events-cache .\calibration\runs\audio-events.msgpack `
  --remote-video-url "srt://0.0.0.0:5100?mode=listener&latency=120000&timeout=5000000" `
  --remote-video-latency-ms 250

Start the current live fusion producer:

.\.venv\Scripts\python.exe .\scripts\live_sensor_fusion.py `
  --cache .\calibration\runs\visual-state.msgpack `
  --fps 30 `
  --points 256

For OBS, add a Spout2 Capture source and select LocalCastBridge Point Cloud. On this workstation, OBS websocket was enabled locally and the Off World Live win-spout plugin was installed under C:\ProgramData\obs-studio\plugins\win-spout.

Status

The sender writes a heartbeat JSON file:

Get-Content .\calibration\runs\stream-spout-status.json

Fields:

sender_name: the Spout sender OBS should see.
frames_sent: count of successful sendTexture calls.
point_count: number of packet points in the last status update.
frame_path: packet source path.
last_error: null during healthy output.

The typed status document is written to calibration/runs/visual-stream-status.msgpack. The JSON file is only the blunt terminal heartbeat.

The sync heartbeat is written to calibration/runs/av-sync-status.json. It includes the audio frame delta and the remote SRT video artifact:

remote_video.source_name: OBS/Aquarium-facing source name.
remote_video.url: the SRT listener URL for the neighbor feed.
remote_video.expected_latency_ns: configured presentation delay, not just the SRT socket latency.
remote_video.delta_ns: difference between the render presentation clock and the expected remote video presentation time.
remote_video.synchronized: whether the delta is inside tolerance.

Invariants

OBS receives a named Spout texture.
The Spout sink owns presentation only; it does not own camera calibration, fusion, or scene truth.
CultCache MessagePack documents are the live local state authority. CultNet document replication is the process/network boundary.
JSON render-frame polling is compatibility scaffolding only.
Latency is explicit in the packet timestamps. The renderer may buffer, but it may not silently erase the visual/audio alignment fields.
The neighbor SRT video feed is a timed artifact. OBS must not be left to eyeball-sync it against the Spout texture and AmbiX bed.

Aquarium Cut

The next renderer cut is to move the RenderFramePacket consumer into Aquarium Engine:

RenderFramePacket
-> CultRenderFrame / CultNet document_put
-> Aquarium typed runtime state
-> compact anisotropic brush/splat buffers
-> D3D render target
-> Spout2 sender texture
-> OBS Spout2 Capture

That is the coherent route to millions of splats. The current OpenGL sender exists so the stream has a live OBS texture today while preserving the boundary Aquarium needs tomorrow.

4.7 KiB Raw Blame History