Mimir/docs/obs-stem-plugin.md

OBS Native Plugin

The production OBS path is a Mimir/Fensalir-owned program texture plus a Mimir/Fensalir-owned stem bus, exposed through a narrow native OBS plugin.

Authority

• Fensalir owns the completed program pixels.
• Mimir/Fensalir own timing, alignment, DSP, and stem identity.
• OBS owns broadcast mixing and scene composition.
• The plugin imports already-produced program pixels and reads already-processed stem samples. It does not synchronize raw sources, estimate clocks, render the field, or decide which payload is true.

Current Contract

Video

Fensalir publishes the completed backbuffer into a named shared D3D12 B8G8R8A8_UNorm texture when these environment variables are set:

FENSALIR_PROGRAM_OUTPUT_D3D12=1
FENSALIR_PROGRAM_OUTPUT_NAME=Global\MimirFensalirProgramTexture

The native OBS plugin in native/obs_stem_source exposes a Mimir Program Texture video source. Set:

Shared D3D12 texture name: Global\MimirFensalirProgramTexture
Shared D3D12 fence name: Global\MimirFensalirProgramFence
Consumer D3D12 fence name: Global\MimirObsProgramConsumerFence
Texture ring slots: 1

OBS/libobs is D3D11 internally on Windows, so this source opens the named D3D12 texture, creates an OBS-readable legacy shared D3D11 texture, opens that same bridge texture from D3D12, and performs a GPU-to-GPU copy during OBS render before drawing the source. This avoids CPU readback and does not require Spout2. It is not pure zero-copy yet; the remaining copy is the D3D12-to-libobs-D3D11 boundary. Fensalir also publishes a named D3D12 program-output fence; the OBS source opens it and observes publication completion before copying. This prevents blind read-before-producer-completion behavior without coupling OBS to Fensalir's private frame fence.

For OBS-only stress testing, Fensalir and the OBS source can agree on a bounded texture ring:

FENSALIR_PROGRAM_OUTPUT_RING_COUNT=3
FENSALIR_PROGRAM_OUTPUT_CONSUMER_FENCE_NAME=Global\MimirObsProgramConsumerFence
Texture ring slots: 3
Consumer D3D12 fence name: Global\MimirObsProgramConsumerFence

With a ring count above one, Fensalir publishes textures named Global\MimirFensalirProgramTexture.0, .1, and so on. The OBS source uses the program-output fence value to copy the latest completed slot and signals the consumer fence after its GPU copy. When Fensalir is launched with the matching consumer-fence environment variable, it will not reuse a ring slot until OBS has acknowledged a copy at or beyond that slot's published fence value. Leave the ring count at 1 for EVE until the EVE relay speaks the same ring contract.

For a visible interop witness while the real Mimir scene is sparse, launch Mimir with:

MIMIR_OBS_PROOF_VISUAL=1

That enables a small bright SDF/starfield proof visual. It is a debug witness, not production scene policy.

Audio

MimirObsStemSharedMemoryPublisher writes the latest validated MimirObsStemPublicationSnapshot into a Windows shared-memory map named Local\MimirObsStemBus by default. The native OBS plugin in native/obs_stem_source exposes a Mimir Audio Stem input source. Add one source per stem and set:

Shared memory map: Local\MimirObsStemBus
Stem id: aligned_source_0

Use aligned_source_0 through aligned_source_5 for the first actuator-stage source lanes.

Build

.\scripts\build-obs-stem-plugin.ps1

The script stages obsproject/obs-plugintemplate under artifacts/obs-sdk/, lets the upstream template fetch/build the matching OBS/libobs development surface, then builds native/obs_stem_source against that SDK. The staged SDK and native build products are local artifacts, not repository state.

The managed shared-memory publisher is verified by:

dotnet run --no-build --project .\src\Mimir.BufferSmoke\Mimir.BufferSmoke.csproj -- --obs-stem-shared-memory-smoke

The local native plugin build currently succeeds and emits:

native\obs_stem_source\build\Release\mimir_obs_stem_source.dll

That DLL currently registers both source types:

Mimir Program Texture
Mimir Audio Stem
Muninn Stream

Muninn Streams

The Muninn Stream source is a thin OBS wrapper over Muninn's typed CultCache/CultMesh stream catalog. The operator-facing UI is intentionally just the stream dropdown. Store paths, command endpoints, and media return endpoints are source defaults and migration state, not user-facing stream selection.

The source listens for Raven's typed OBS catalog projection on local UDP port 17874. Raven Muninn sends the catalog over a small CultNet RUDP discovery lane using connection id 0x6d750003; OBS uses that live catalog for the dropdown when present.

If live discovery is absent, the source falls back to Muninn's local .cc store:

C:\Meta\Odin\state\muninn.telemetry.cc

The source looks for record key obs with type:

muninn.obs_stream_catalog

Muninn owns that catalog as typed CultCache state. OBS owns only the selected scene source and media rendering. When a selected stream uses Muninn's RUDP media profile, the source publishes the same typed muninn.capture_stream_command that an operator would publish manually. Raven Muninn serve still owns whether capture starts and which local FFmpeg/WASAPI children exist.

If both live discovery and the local catalog projection are absent or stale, the source keeps a LAN-default Raven A/V stream option instead of showing an empty source. That is a resilience fallback for the OBS machine, not a second stream authority. Stale catalog rudp:// URLs that do not carry the current low-latency LAN profile are sanitized back to the source-derived URL before the bridge starts.

The current Raven-room default route is LAN-first: command requests go to 192.168.1.84:17873, and Raven sends media back to Starfire at 192.168.1.66:5204. Older saved source settings that still point at the deprecated WireGuard defaults 10.77.0.4:17873 and 10.77.0.2 are migrated on load. Custom operator-entered endpoints are left alone.

The source polls the catalog while active, but reconnect is owned by the RUDP receiver bridge rather than catalog labels. The bridge tracks the last packet and the last forwarded video/audio payload. If startup produces no media, or a previously live stream goes stale, OBS refreshes its local FFmpeg child readers and republishes the typed activation command on a short bounded cadence. A Raven daemon restart or media child exit should therefore recover without reopening source properties.

For rudp:// stream URLs, the plugin binds the advertised media port, accepts the CultNet RUDP handshake, acknowledges packets, reassembles RUDP fragments, decodes typed muninn.media_video_access_unit.v1 and muninn.media_audio_packet.v1 records, and lowers elementary H.264 plus ADTS AAC into local loopback UDP sockets for OBS FFmpeg child sources. The bridge does not impose a global packet-sequence FIFO over the media lane: CultNet RUDP media is reliable but unordered, so frame ids, chunk ids, and the advertised assembly_deadline_ms own video reconstruction. Incomplete video frames expire inside that budget and emit muninn.media_receiver_feedback.v1 keyframe pressure instead of stalling newer media behind an obsolete sequence gap. RUDP fragment ids are quarantined briefly after completion/expiry so late duplicates cannot seed a corrupt assembly when the sender's 16-bit fragment id space wraps under sustained LAN pressure. The local OBS FFmpeg UDP readers use large loopback buffers; those buffers absorb receiver bursts without becoming a second transport authority.

Install

For the standard OBS plugin search path:

.\scripts\install-obs-mimir-plugin.ps1

By default this writes:

C:\ProgramData\obs-studio\plugins\mimir_obs_stem_source\bin\64bit\mimir_obs_stem_source.dll

For a portable OBS tree or direct OBS installation root:

.\scripts\install-obs-mimir-plugin.ps1 -ObsRoot D:\Tools\obs-studio

The script does not create scene items or alter OBS profiles.