749 lines
52 KiB
YAML
749 lines
52 KiB
YAML
workspace:
|
|
name: Mimir
|
|
status: active
|
|
focus: direct-driver-fensalir-stream-app
|
|
implementation_name: Mimir public surface / native runtime contracts
|
|
premise: >
|
|
Mimir is the local realtime field machine: cameras, microphones, speakers,
|
|
network feeds, and timing signals become one coherent Mimir-owned volumetric
|
|
program surface. The live path is a C# Fensalir-hosted app with in-memory
|
|
rolling buffers and native capture/runtime boundaries.
|
|
|
|
objective:
|
|
primary: >
|
|
Build one synchronized spatial stream machine from six cameras, six
|
|
microphones, two speakers, Leap timing, Fensalir GPU rendering, Faust DSP,
|
|
Eve operator surfaces, and Yggdrasil site publication.
|
|
current: >
|
|
Promote the active bioacoustic packet-song receiver into runtime state:
|
|
self-identifying song contours should pin multiple time/frequency anchors
|
|
per heard call, learn per-path response, and drive the audio actuator while
|
|
direct camera/audio capture continues feeding Mimir.Runtime buffers.
|
|
|
|
constraints:
|
|
- Sender is Windows.
|
|
- OBS is a temporary compatibility sink, not a composition or broadcast authority.
|
|
- Video encode must use NVENC when the bridge path is active.
|
|
- Audio sources must remain separately controllable through Mimir/Eve and publishable as stems.
|
|
- V1 bridge is LAN-only.
|
|
- Persistent state must clarify rehydrate, not become transcript tax.
|
|
- Live ingest belongs to Mimir.Runtime, native drivers, Fensalir, and
|
|
Faust/native DSP.
|
|
- Edge JSON is schema/diagnostic only; live process data is typed or in
|
|
memory.
|
|
- Networked feeds are producers into the live runtime, not independent clock
|
|
authorities.
|
|
- USB-attached Moves on Starfire and Nightwing are tracking witnesses. Muninn
|
|
publishes source-local marker candidates plus controller/IMU/button state.
|
|
Odin may discover/project their stream schemas, but it does not own pose
|
|
capture or Mimir fusion.
|
|
- CultMesh/CultNet is the default Raven-to-Starfire media transit. OBS may
|
|
consume a Starfire-local UDP MPEG-TS egress as a compatibility sink, but
|
|
Mimir owns program selection, composition, preview/control, and publication.
|
|
- CultLib is moving every runtime's typed CultNet/CultMesh document transport
|
|
to `cultnet.transport.rudp.v0`. Health, provider advertisements, command
|
|
boundaries, transport profiles, and program/media document lanes should use
|
|
RUDP by default. Product/debug render surfaces are lowerings,
|
|
compatibility probes, or migration debt unless a specific runtime has not
|
|
earned its RUDP path yet.
|
|
- Daemon health and freshness are separate in authority from media transport,
|
|
not separate because they need TCP. The old Mimir Eve dashboard broker is
|
|
archived and no longer publishes Idunn health, deck routes, or WebSocket
|
|
commands. Idunn owns continuity once Odin has accepted a daemon; Mimir-owned
|
|
surfaces may only return as typed CultMesh/Odin state publishers.
|
|
- Mimir explicitly trades latency and memory for throughput and field quality:
|
|
the default five-second buffer is compute budget for stream alignment,
|
|
GPU/audio analysis, and volumetric sound-field extraction.
|
|
|
|
invariants:
|
|
- Mimir owns composition, preview/control state, source subscription policy,
|
|
and program publication intent.
|
|
- FFmpeg owns capture, encoding, and transport only for bridge utilities.
|
|
- One stable bridge port equals one OBS source endpoint.
|
|
- One rolling live window owns the timing edge and expiry.
|
|
- The rolling window is not accidental latency. It must preserve enough time
|
|
to line up feeds and crunch data before Mimir program output is emitted.
|
|
- Typed views may index the rolling buffer, but no per-kind ring owns
|
|
retention.
|
|
- Fensalir owns GPU lowering for the Mimir program scene graph, D3D12 interop,
|
|
and local program texture output.
|
|
- Faust/native DSP owns hot audio alignment, separation, spatialization, and
|
|
stems.
|
|
- Eve GUI/TUI surfaces lower Mimir's operator controls, previews, and stats on
|
|
any device without owning scene truth.
|
|
- A Yggdrasil-facing publisher daemon consumes Mimir program output and
|
|
publishes it to the site without owning composition.
|
|
- Local secrets remain untracked.
|
|
- Mimir is a CultMesh app. Codebooks, schedules, calibration receipts, path
|
|
response, and compute-offload work need typed mesh-syncable state surfaces
|
|
so phones, microcontrollers, Raven, and Starfire can cooperate without
|
|
becoming independent timing authorities.
|
|
- Move tracking observations are stream samples, not program controls or
|
|
dashboard truth. Starfire-local and Nightwing-remote Move samples must enter
|
|
Mimir.Runtime as typed tracking buffers before Fensalir or Eve acts on them.
|
|
- Structured PS Move light pulses are Muninn output commands. Mimir requests
|
|
`muninn.move_light_command.v1` over CultNet/CultMesh from the Muninn daemon
|
|
on the host with the USB-attached Move; direct HID writes in Mimir scripts
|
|
are temporary smoke/bootstrap edges only.
|
|
|
|
accepted_design:
|
|
- Receiver-driven video recovery now reaches the actual NVENC frame owner.
|
|
Odin's `native/muninn-video-encoder` is a long-lived libavdevice desktop
|
|
capture and `h264_nvenc` process; a new decode-chain invalidation edge makes
|
|
Muninn write `IDR` to its stdin without restarting video, audio, or CultNet.
|
|
A live 2560x1440 D3D11 proof with a deliberately 600-frame GOP produced the
|
|
requested second IDR at frame 22, and its Annex-B stream decoded cleanly.
|
|
The normal quarter-second scheduled IDR remains independent fallback.
|
|
The repeatable bundle includes the OBS experimental FFmpeg dependency
|
|
snapshot. Raven installation and real impairment acceptance remain pending
|
|
because Raven's documented WireGuard SSH route timed out on 2026-07-16.
|
|
- Muninn input now bounds disconnected unacknowledged button history at 256
|
|
edges. Overflow rotates the controller epoch and rebases Sleipnir from the
|
|
next complete latest-state frame, rather than dropping an arbitrary press
|
|
or retaining history forever. Sleipnir rejects sparse future edge gaps over
|
|
256. Full Muninn (174) and Sleipnir (26) suites pass.
|
|
- The 2026-07-16 loss-recovery cut is now live on both sides of the Muninn
|
|
audio boundary. Odin emits typed `muninn.media_audio_parity_shard.v1`
|
|
records using fixed 4+2 GF(256) parity for constant-size 10 ms PCM packets.
|
|
Mimir's production OBS receiver retains a bounded recent data window,
|
|
reconstructs up to two erased packets per block before the 40 ms playout
|
|
deadline, and then hands recovered packets to the same reorder queue that
|
|
owns normal playout. The FEC cache is not a second timeline. Native tests
|
|
exhaust every one- and two-erasure permutation, and the production plugin
|
|
DLL builds with all three native suites passing. The OBS input ceiling is
|
|
now 500 ms and the default remains 100 ms; old 1.2-2.0 second test/config
|
|
assumptions are not realtime authority.
|
|
- Odin now carries a deterministic bidirectional UDP impairment proxy with
|
|
seeded clean, iid loss, burst loss, delay/jitter, reorder, duplicate, and
|
|
stall profiles. This is the acceptance substrate for real CultNet endpoints,
|
|
not a simulated transport replacement. The remaining critical sender gap
|
|
is an encoder owner that can actuate a receiver-requested next IDR without
|
|
restarting the video session; stock FFmpeg CLI capture cannot own that
|
|
command dynamically.
|
|
- The 2026-07-16 production receiver audit cut two false contracts. Native
|
|
Mimir/OBS early video repair feedback no longer marks live frames late, so
|
|
Odin can actually return cached missing chunks; only assembly expiry emits
|
|
`late_frame_ids`. A C++-generated feedback fixture now decodes in Odin's
|
|
Rust tests. Muninn's typed audio body is 10 ms `pcm-f32le-interleaved`, so
|
|
the OBS bridge now opens its local decoder as `f32le` stereo 48 kHz instead
|
|
of AAC, uses a 40 ms reorder budget instead of 2000 ms, and emits bounded
|
|
silence packets for short unrecoverable gaps to preserve cadence. Opus
|
|
FEC/PLC remains required before audio reaches Moonlight-level loss quality.
|
|
- Odin's 2026-07-16 Moonlight-derived reliability cut now runs Muninn and
|
|
Sleipnir against the experimental CultLib runtime snapshot at `8965f3c0`.
|
|
HID latest state and ordered button edges have separate epoch/sequence and
|
|
application-ack semantics; wall timestamps no longer order input. Muninn
|
|
media uses CultNet's unreliable `realtime` lane rather than the experimental
|
|
reliable-without-expiry `media` lane, defaults LAN usefulness to 100 ms,
|
|
bounds producer/pending queues, expires repair material, refuses repair for
|
|
receiver-late frames, and reserves keyframe requests for decode-chain
|
|
invalidation. Odin records the live authority and remaining gaps in
|
|
`docs/realtime-delivery-authority.md`.
|
|
- The Moonlight comparison at
|
|
`docs/research/moonlight-muninn-sleipnir-study-2026-07-16.md` establishes
|
|
that realtime delivery policy belongs to signal semantics, not generic RUDP
|
|
reliability. Muninn media already owns access-unit boundaries, deadlines,
|
|
video parity, dependency/keyframe metadata, receiver damage feedback, and a
|
|
bounded repair cache; these foundations stay. The next transport cut must
|
|
close feedback into sender pacing/bitrate/parity and derive retry/repair
|
|
lifetime from video/audio consumption deadlines instead of the current
|
|
broad two-second default. Sleipnir/Muninn input must explicitly separate
|
|
replaceable latest state from ordered non-replaceable edges, coalesce before
|
|
enqueue, acknowledge applied edge/state sequences, and use a proved clock
|
|
model before reporting cross-host wall-clock subtraction as one-way latency.
|
|
- Nightwing Muninn `d0eded6` now owns PSMoveAPI observation admission in each
|
|
private Eye worker. It rejects stale, sub-2 px, out-of-frame, and
|
|
discontinuous positions before the parent aggregates evidence; typed
|
|
tracker health exposes rejection counts. PSMoveAPI age is no longer emitted
|
|
as optical confidence: admitted candidates carry uncalibrated score `0.5`.
|
|
- `MimirMoveSphereCalibrationFitter` owns the bounded two-Eye optical fit
|
|
after spatial collection. Eye 0 defines the tracking-space gauge; a robust
|
|
training consensus estimates both focal lengths and Eye 1 rigid pose, while
|
|
a deterministic untouched 20 percent partition owns promotion evidence.
|
|
Promotion requires low held-out median/P95 reprojection error and at least
|
|
65 percent held-out consensus. Physical orb radius is explicit and never
|
|
guessed. PSMoveAPI blob radius and position age are not calibrated sphere
|
|
radius or optical confidence.
|
|
- `Mimir.slnx` owns the app surface.
|
|
- `src/Mimir.App` launches Fensalir as host/window/D3D12 library.
|
|
- `src/Mimir.Runtime` owns `MimirSynchronizationHub`,
|
|
`MimirRollingStreamBuffer`, stream descriptors, `IMimirStreamSource`,
|
|
direct native ingest, process bridge ingest, and video capture driver seams.
|
|
- `MimirVideoFrameDescriptor` carries width, height, pixel format, stride,
|
|
device timestamp, and optional native/GPU handle.
|
|
- `MimirAudioBlockDescriptor` carries sample rate, channels, sample format,
|
|
frame count, timestamp, and optional native handle.
|
|
- `MimirTrackingObservation` carries typed PS Move evidence samples as
|
|
`mimir.move_tracking_observation.v1`; Starfire Moves are local tracking
|
|
streams, Nightwing Moves are emitted by Muninn on Nightwing, Odin owns
|
|
discovery/schema projection, and Mimir owns buffering and fusion.
|
|
- `native/reservoir` now exposes a `move_evidence` typed view and
|
|
`LocalcastMoveEvidenceBufferDescriptor` for compute-upload buffers carrying
|
|
Muninn optical marker and controller-state witnesses before Mimir fusion.
|
|
- `MimirNativeReservoirRuntime` loads the native reservoir DLL and pins Move
|
|
evidence sample/descriptor batches so admitted native handles remain valid.
|
|
- `MimirMuninnMoveEvidenceAdapter` mirrors Muninn's
|
|
`muninn.move_marker_candidate.v1` and `muninn.move_controller_state.v1`
|
|
document shapes and maps them into native Move evidence samples without
|
|
claiming association or pose authority.
|
|
- `MimirMuninnMoveEvidenceAdapter` also consumes
|
|
`muninn.move_identity.v1` records into a deduped Move roster projection.
|
|
The roster can show a controller as USB-visible on one Muninn host and
|
|
Bluetooth-waiting/connected on Nightwing at the same time, so blind USB
|
|
rotation remains inspectable without promoting identity-only records into
|
|
motion evidence.
|
|
- Realtime Muninn Move witnesses enter over CultMesh streaming frames. CultCache
|
|
remains persistence/receipt/debug state, not the hot tracking feed.
|
|
- `mimir.move_controller_pose.v1` is Mimir's resolved wand pose contract for
|
|
Fensalir-facing interaction. Mimir owns calibration, association,
|
|
triangulation, IMU fusion, prediction, confidence, and latency accounting.
|
|
- `MimirMoveFusion` is the current Move pose owner. It consumes native Muninn
|
|
Move evidence samples, refuses to publish poses without calibrated optical
|
|
witnesses, triangulates multi-camera orb position, carries controller
|
|
buttons/trigger/gyro into `mimir.move_controller_pose.v1`, and marks
|
|
orientation as `orientation:imu-unresolved` until IMU fusion/prediction
|
|
earns a full 6DoF controller contract.
|
|
- `MimirMovePoseStream` publishes resolved Mimir-owned Move poses as
|
|
`mimir.move_controller_pose_stream_frame.v1` CultMesh shared-memory byte
|
|
frames for Fensalir and other realtime consumers; it carries poses and does
|
|
not own calibration or fusion decisions.
|
|
- `MimirMoveProofSurface` publishes the observer-only
|
|
`mimir.move_proof_surface.v1` receipt and Fensalir spline probe for the
|
|
named proof spine. It combines the Muninn evidence admission receipt, the
|
|
Mimir-owned pose frame, and a Fensalir presentation/probe timestamp without
|
|
becoming a pose owner. Single-ray fallback is surfaced as non-final; only
|
|
triangulated calibrated optical marker evidence can earn a full-pose
|
|
verdict.
|
|
- `MimirMoveProofPipeline` is the single shared commit primitive for one
|
|
Move proof frame. It reads a Muninn CultMesh evidence frame, admits native
|
|
`move_evidence`, runs Mimir fusion, emits the Mimir pose frame, creates the
|
|
proof surface, and returns the Fensalir spline probe. Direct user actions,
|
|
programmatic smokes, and future live subscribers should use this same path
|
|
so the proof chain does not split into separately repaired truths.
|
|
- `MimirRuntime` now owns the actual Fensalir presentation attachment for
|
|
Move proof surfaces. `PublishMoveProofSurface` accepts the latest
|
|
Mimir-owned proof document, and `CreateFrame` composes its observer-only
|
|
`move-proof-*` probe splines into `AquariumFrame.Scene.SplineFrame` beside
|
|
the audio spectrum preview. Fensalir remains a renderer, not a proof
|
|
authority.
|
|
- `MimirMoveProofRuntimeDriver` is the runtime bridge from a Muninn CultMesh
|
|
shared-memory evidence ring into `MimirRuntime.PublishMoveProofSurface`.
|
|
It derives the Mimir evidence, Mimir pose, and Fensalir frame ids from the
|
|
actual Muninn frame suffix, calls `MimirMoveProofPipeline`, and suppresses
|
|
duplicate frame admission so the live loop cannot re-admit one frame and
|
|
impersonate progress.
|
|
- `MimirMoveProofRuntimeConfiguration` is the typed selection and validation
|
|
surface for that runtime bridge. An enabled proof source names the Muninn
|
|
evidence stream, native reservoir path, Mimir evidence/pose frame prefixes,
|
|
Fensalir presented-frame prefix, fusion authority, consumer contract, and
|
|
at least two calibrated camera witnesses. It may create a
|
|
`MimirMoveProofRuntimeDriver` only when the supplied CultMesh ring stream id
|
|
matches the configured evidence stream.
|
|
- `MimirRuntime` owns configured Move proof activation. On scene-ready it asks
|
|
an `IMimirMoveProofEvidenceRingProvider` for the evidence ring, opens the
|
|
configured native reservoir, registers the runtime driver, retains the
|
|
driver resources, and exposes activation status. The default provider fails
|
|
explicitly because the current C# CultMesh ring is still in-process only;
|
|
`Mimir.BufferSmoke --move-proof-runtime-activation-smoke` injects an
|
|
in-process provider to prove the activation path without claiming live
|
|
Nightwing/Starfire field evidence.
|
|
- `mimir.move_proof_runtime_activation.v1` is the typed readiness/proof-spine
|
|
surface for configured Move proof sources. It reports the configured stream,
|
|
provider kind, active/driver state, native reservoir path, calibrated camera
|
|
ids, latest same-stream proof ids, latest verdict, and unavailable-provider
|
|
diagnostic. `Mimir.BufferSmoke --move-proof-runtime-activation-surface-smoke`
|
|
proves both an active injected-ring document and the inactive default
|
|
runtime document without claiming live field evidence.
|
|
- `mimir.move_proof_evidence_frame_snapshot.v1` is the one-copy fallback for
|
|
captured Muninn evidence frames while C# CultMesh has no cross-process ring
|
|
opener. A configured `EvidenceSnapshotPath` is loaded by
|
|
`MimirConfiguredMoveProofEvidenceRingProvider`, metadata-checked against the
|
|
encoded payload, copied into an owned in-process ring, and then consumed by
|
|
the same runtime driver/pipeline. `Mimir.BufferSmoke
|
|
--move-proof-runtime-snapshot-smoke` proves the named proof spine through
|
|
that file boundary. It is field capture/replay support, not the final live
|
|
shared-memory proof.
|
|
- Odin/Muninn can write the matching producer-side latest snapshot with
|
|
`--move-evidence-snapshot <path>`. The snapshot is emitted by the same
|
|
Muninn Move evidence publisher after the frame payload is accepted into the
|
|
stream ring, and Odin's
|
|
`move_evidence_snapshot_writes_mimir_compatible_frame_artifact` unit decodes
|
|
both the `mimir.move_proof_evidence_frame_snapshot.v1` tuple and embedded
|
|
Muninn payload. This supports Nightwing field capture/replay through Mimir's
|
|
snapshot provider while the final live ring/page transport remains pending.
|
|
- Nightwing field capture/replay has proven the one-copy snapshot mechanics,
|
|
but the first `FullPose` replay was physically false-positive evidence.
|
|
Muninn `d4f229a` bundles latest controller state with aggregated marker
|
|
candidates from both Nightwing PS3 Eyes into one evidence frame, with the
|
|
snapshot payload encoded as MessagePack bytes. The preserved artifact
|
|
`artifacts/runtime/nightwing-move-proof-20260712-1041-aggregate.mpack`
|
|
replays through `Mimir.BufferSmoke --move-proof-runtime-snapshot-file-smoke`
|
|
as `muninn:nightwing:move-evidence:1 -> mimir:starfire:move-evidence:1 ->
|
|
mimir:starfire:move-pose:1 -> fensalir:starfire:presented-frame:1`, but
|
|
the operator confirmed the Move markers were dark during that capture. Treat
|
|
its 99 marker candidates as bright image blobs/noise and use the artifact
|
|
only as evidence that Muninn/Mimir/Fensalir replay plumbing works and needs
|
|
marker-lit validation. A valid field pose proof still requires lighting the
|
|
PS Move orb, capturing real optical marker candidates from calibrated camera
|
|
witnesses, and preserving the explicit same-sequence proof spine. The final
|
|
live cross-process CultMesh ring/page proof remains pending.
|
|
- `MimirMoveProofDevSurface` is a dev-only bootstrap gated by
|
|
`MIMIR_MOVE_PROOF_DEV_SURFACE`. It publishes a synthetic calibrated proof
|
|
surface through the same MimirRuntime attachment so headless Mimir.App can
|
|
capture and pixel-check a Fensalir-presented proof probe without making
|
|
Fensalir a proof owner. It is renderer evidence, not field evidence.
|
|
- `MimirMoveCalibrationProtocol` defines the runnable Starfire/Nightwing Move
|
|
calibration preflight as typed state: required Muninn Move evidence streams,
|
|
optional Quest headset/controller pose witnesses over USB,
|
|
stillness/sweep/validation phases, and the calibration outputs Mimir must
|
|
derive before orientation can stop being `orientation:imu-unresolved`.
|
|
- Muninn owns Quest USB access as `muninn.quest_access.v1`. It proves ADB
|
|
authorization/device identity for the optional external VR reference and
|
|
advertises Quest input, pose, and warped-video input stream ids. Mimir
|
|
consumes those surfaces for calibration; it does not publish Quest access.
|
|
- Starfire should have Muninn for source-local Move HID control and local
|
|
sensor publication, but some Starfire audio/video paths still enter Mimir
|
|
through local Mimir producers or diagnostic process bridges while native
|
|
drivers mature. Same-host Move evidence and pose frames are CultMesh
|
|
shared-memory byte rings. Current remote media body transport is not
|
|
same-machine zero-copy into Fensalir; Raven media uses CultNet/CultMesh
|
|
documents over the mesh and lowers to local UDP only for OBS compatibility.
|
|
- The old `scripts/start-nightwing-move-tracking.ps1` Nightwing Eye/Move
|
|
witness path is archived. Starfire no longer owns a receiver/recorder
|
|
socket lane for this evidence; Nightwing/Mimir witness publication must use
|
|
typed CultMesh stream frames or Odin-discovered CultMesh documents.
|
|
- Odin's Muninn organ owns Move optical candidate extraction in
|
|
`E:\Projects\Odin\crates\muninn-move-tracker` and publishes/feeds
|
|
`muninn.move_marker_candidate.v1` records. Mimir consumes those candidate
|
|
streams; it does not own raw optical extraction. Mimir explicitly owns
|
|
calibration, multi-camera association, triangulation, IMU fusion,
|
|
prediction, and final 6DoF wand pose.
|
|
- Muninn's Move evidence stream frame now uses canonical
|
|
`muninn.move_marker_candidate.v1` records for optical candidates instead of
|
|
a hardcoded empty marker slice. A daemon unit proves a bright Y8 frame
|
|
extracted by `muninn-move-tracker` serializes as non-empty Mimir-compatible
|
|
evidence. The daemon now has a source-local Y8 extraction/publish seam and
|
|
a first `serve` camera producer: `--move-marker-camera <camera-id>=<device-path>`
|
|
polls a Unix V4L2 YUYV frame, converts it to compact Y8, and feeds that
|
|
same seam. Nightwing now runs one public Muninn provider whose two private
|
|
per-eye PSMoveAPI subprocesses exchange hue state, observations, and health
|
|
with the parent through bounded MessagePack pipes. Children own no store,
|
|
provider advertisement, command route, or light output. Typed
|
|
`muninn.move_tracker_health.v1` records expose camera backend, calibration,
|
|
update/observation counts, RGB range, and expected-hue pixel evidence.
|
|
Live field proof on 2026-07-13 established Eye 0 at exposure 0.3 and Eye 1
|
|
at 0.1; both calibrated four Moves and produced observations. A Mimir
|
|
visibility window now sees all four IDs on both eyes. Muninn's dedicated
|
|
evidence aggregator owns frame sequencing and transport independently of
|
|
the 15-second CultCache telemetry loop; controller state is bundled but
|
|
cannot create an optical frame. Its deadline clock is capped by the
|
|
configured camera cadence. Field proof on 2026-07-13 kept one Muninn parent
|
|
PID alive, received 622 unique frames in 15 seconds, carried 2,647 camera
|
|
observations and 1,384 same-ID cross-camera correspondences, and saw every
|
|
stable Move ID from both eyes. Producer health reported 5,302 frames, 5,301
|
|
remote handoffs, 771 sends during the subscription, and zero local-ring
|
|
admissions. Cadence and full overlap are earned; calibrated camera matrices
|
|
still require intrinsics and spatially diverse correspondence capture.
|
|
- `MimirMoveStereoCalibrationAssessment` owns the pre-fit observability gate
|
|
as `mimir.move_stereo_calibration_assessment.v1`. It consumes a typed
|
|
visibility receipt, counts only same-frame/same-Move stereo pairs, measures
|
|
per-Eye image-grid and radius coverage, and refuses promotion when
|
|
intrinsics, measured orb radius, synchronized sample count, or spatial
|
|
diversity are missing. The corrected 2026-07-13 live receipt contained 130
|
|
honest same-frame pairs at zero frame skew, but Eye 0 occupied only 4/12
|
|
grid cells and Eye 1 only 2/12; no camera matrices were promoted.
|
|
- `MimirMuninnMoveEvidenceAdapter.TryAdmitLatestCultMeshFrame` now exposes a
|
|
typed admission receipt with decoded frame id, producer, publish/read
|
|
timestamps, sample counts by kind, source/arrival ranges, native reservoir
|
|
handle, and reservoir edge/window so the named proof spine can be traced.
|
|
- `MimirFrameEventProcessStreamSource` is a temporary diagnostic adapter for
|
|
JSON-line video frame and audio block metadata from native probes into
|
|
runtime rolling buffers; it does not carry pixels/audio payloads or own the
|
|
final local-device hot path.
|
|
- `MimirBioacousticTimeline` is the active runtime watermark. It renders a
|
|
low-gain birdsong-like word language with 128 self-identifying word
|
|
positions, left-speaker and right-speaker variants, four formant-rich
|
|
syllables per word, rhythm variation, and direct word identity instead of a
|
|
de Bruijn grammar. The active receiver is a song-contour anchor machine:
|
|
one heard call should expose syllable timing, contour bends, formants,
|
|
payload ornaments, speaker tint, and log-mel evidence as a cluster of
|
|
time/frequency anchors. `MimirRuntime` emits separate panned vocabularies
|
|
per speaker. `MimirAudioSynchronizationAnalyzer` reports active timing as
|
|
`evidence=bioacoustic`; the old chirp-bin path remains a
|
|
calibration/reference artifact with ASIO tooling.
|
|
- The diagnostic audio path can optionally carry base64 float sample payloads
|
|
for sync smoke tests. `MimirChirpletTimeline` owns the older structured
|
|
chirplet reference fingerprint: an order-3 de Bruijn sequence over 32
|
|
time/frequency constellation symbols, plus PCM segment rendering, matched
|
|
timing atoms, symbol/event decoding, and per-band response kernels. Symbol
|
|
identity is carried by start band, glide shape, duration, and following
|
|
inter-chirp gap. Any three consecutive correctly detected symbols identify a
|
|
timeline event within the current operating horizon. `MimirChirpletSymbolCodebook`
|
|
owns the 32 symbol definitions so every symbol has a unique chirp shape, with
|
|
rhythm as additional evidence. `MimirChirpletStreamDecoder` is the first
|
|
constrained chirplet-transform receiver: it owns a bounded PCM window, emits
|
|
transform frames with multiple phase-invariant symbol candidates and refined
|
|
per-candidate sample offsets, decodes code-valid triplet anchors through
|
|
gap/clock coherence, and fits a per-source sample clock from those anchors.
|
|
Its synthetic smoke path is `dotnet run --project
|
|
.\src\Mimir.BufferSmoke\Mimir.BufferSmoke.csproj -- --chirplet-self-test`.
|
|
It currently decodes all 15 emitted chirps in a two-second window into 13
|
|
anchors for events 0-12 with a 47999.999990 Hz clock fit and 0.000014 sample
|
|
MAE. `MimirAudioSynchronizationAnalyzer`
|
|
now treats `loopback-scarlett-speakers` as timing authority and derives
|
|
delay only from matched decoded timeline anchors. Positive delay means
|
|
the candidate mic is late relative to loopback. Reports include rounded
|
|
delay, fractional delay, confidence, and per-band matched energy. `MimirSynchronizationHub`
|
|
caches the latest per-source reports and `MimirAudioSynchronizationStateTracker`
|
|
owns the smoothed per-source belief: latest delay, smoothed delay,
|
|
confidence, per-band response evidence, and delay-slope/SRO estimate in
|
|
ppm. `MimirRuntime` now runs this analysis online as a bounded rotating
|
|
service on a fixed cadence, while UI and telemetry read cached reports/states
|
|
instead of invoking analysis. `MIMIR_SYNC_TELEMETRY_SECONDS` prints live
|
|
buffer/report/state telemetry. `MimirAudioSynchronizationSettings.Mode`
|
|
now owns the active evidence policy: `chirp-only` emits the calibration
|
|
witness, `passive` stays silent and uses program-audio phase correlation,
|
|
and `hybrid` emits active pilot chunks only while passive confidence is
|
|
weak. `MimirPassiveAudioSynchronizationEstimator` is the first passive path:
|
|
it uses PHAT-weighted cross-spectrum correlation between loopback and mic
|
|
windows before any watermark is needed. Negative passive lags are
|
|
contradictory evidence and carry zero confidence; single-window passive
|
|
confidence is capped below certainty until repeated state earns it.
|
|
`MimirBioacousticTimeline` is now the active calibration witness for both
|
|
chirp-only and hybrid: 128 self-identifying word positions, left/right
|
|
speaker variants, four formant-rich syllables per word, rhythm variation,
|
|
and direct word identity so any correctly decoded word identifies one
|
|
canonical timeline location. The analyzer refines final active delay with constrained local
|
|
waveform correlation around the decoded anchor delay. Each detected motif
|
|
carries per-band response evidence, and the next calibration cut should use
|
|
those profiles to tune contours, formants, usable bands, gain, and rhythm
|
|
weighting. `Mimir.BufferSmoke --bioacoustic-cepstral-smoke` now proves the
|
|
non-brute-force receiver direction: degraded mel-cepstral calls are decoded
|
|
by an augmented MFCC/projection-hash word index before any expensive
|
|
confirmation. `Mimir.BufferSmoke --bioacoustic-train` now runs a
|
|
receipt-backed hypothesis panel over indexed cepstral decoder shapes and
|
|
writes typed CultCache results plus pre-warp/post-warp/reconstructed WAV
|
|
artifacts under `artifacts/bioacoustic-training/`. The first receipt
|
|
`bioacoustic-20260524-103438` showed compact indexing fastest on clean
|
|
audio and exposed timing error under domain warp. The corrected receipt
|
|
`bioacoustic-20260524-150729` adds global clock hypothesis scoring over a
|
|
longer window; compact indexing wins clean throughput, baseline indexing
|
|
remains steadier under degradation, highband indexing has warped-path value,
|
|
and anchor density needs its own metric. `MimirChirpBinTimeline` remains the controlled chirp-bin
|
|
calibration/reference artifact with ASIO tooling and
|
|
`MimirChirpBinCalibrationModel`; keep those captures as response evidence,
|
|
not the active runtime sound. Reports and sync states expose delay in
|
|
microseconds as well as fractional samples. Hybrid emits the bioacoustic
|
|
witness as low-gain half-second bursts while passive confidence is weak,
|
|
never as catch-up backlog. The analyzer no longer applies the passive
|
|
two-second window floor to active fallback; `dotnet run --project
|
|
.\src\Mimir.BufferSmoke\Mimir.BufferSmoke.csproj -- --chirp-only-sync-self-test --sample-rate 48000`
|
|
recovers a 317.375-sample synthetic delay with printed 0.000 us error using
|
|
`evidence=bioacoustic`, while `--hybrid-sync-self-test --sample-rate 48000`
|
|
correctly accepts strong passive evidence at 2.772 us error. The synthetic
|
|
standalone receiver proof `--standalone-bioacoustic-self-test --sample-rate
|
|
48000 --delay-samples 1269.5` recovers delayed source time below printed
|
|
microsecond precision using only the codebook/schedule. `--bioacoustic-actuator-self-test
|
|
--sample-rate 48000 --delay-samples 317.375` now proves the first
|
|
measurement-to-sample-movement loop: estimate bioacoustic delay, apply
|
|
fractional correction, and remeasure residual below printed microsecond
|
|
precision. `faust/mimir_alignment_actuator.dsp` is the first Faust-owned
|
|
actuator surface for six channels of bounded fractional delay/gain. The synthetic
|
|
codebook proof is `dotnet run --project
|
|
.\src\Mimir.BufferSmoke\Mimir.BufferSmoke.csproj -- --bioacoustic-self-test`.
|
|
`Mimir.BufferSmoke --render-chirp-bin-f32` plus
|
|
`native/probes/asio_audio_cadence --play-f32-mono --record-f32-interleaved`
|
|
now provides a real Scarlett ASIO artifact proof for active mode. At 192 kHz,
|
|
Loopback 1 -> Loopback 2 decoded at 0.000 us with 12 anchors and 0.999
|
|
confidence. `Mimir.BufferSmoke --calibrate-chirp-bin-asio-f32` now renders
|
|
or ingests a calibration capture and persists the response/confusion/delay
|
|
model under `calibration/chirp-bin/`; `--analyze-asio-f32 --calibration`
|
|
loads it back into the chirp-bin reference decoder. Physical inputs still fail accepted
|
|
pairwise timing against loopback in the stored artifact, but decoded mic
|
|
channels now leave calibration models; physical input 1 produced two reliable
|
|
symbols and a different strongest measured band set.
|
|
`native/asio_capture` and `MimirAsioStreamSource` are now the runtime
|
|
Scarlett path: Focusrite ASIO callbacks feed sample-bearing 192 kHz Float32
|
|
blocks into `Mimir.Runtime` in process. `config/mimir-runtime.asio.example.json`
|
|
proves continuous ASIO runtime ingest: a two-second BufferSmoke run at
|
|
192 kHz ingested more than 12,000 sample-bearing blocks across `asio-ch0`
|
|
through `asio-ch3` and retained 2,048 blocks per channel, all from the
|
|
Focusrite ASIO callback stream.
|
|
`Mimir.BufferSmoke --calibrate-contestant-asio-f32` now persists the active
|
|
packet-song physical response model under `calibration/bioacoustic/`,
|
|
including per-channel delay search, polarity, payload reliability,
|
|
response-normalization bands, gain, confidence, and pairwise propagation
|
|
delay. The latest canary-packet Scarlett receipt runs at 10.7x realtime
|
|
across four 192 kHz channels: loopback is 37/37 with 2.524 us MAE,
|
|
co-streamer shotgun is 37/37 payload with 58.785 us MAE, and the cardioid
|
|
is 26/37 with 90.558 us MAE. This is a serious calibration surface, not yet
|
|
true physical microsecond sync.
|
|
A fresh 192 kHz room receipt at
|
|
`calibration/bioacoustic/scarlett-canary-packet-192k-contour-fresh.json`
|
|
keeps loopback at 2.524 us MAE, improves the co-streamer shotgun to 37/37
|
|
payload with 34.083 us MAE, and improves the cardioid to 31/37 payload
|
|
with 92.996 us MAE. The next precision cut is intra-call contour anchors,
|
|
not more sequence machinery.
|
|
`calibration/bioacoustic/scarlett-canary-packet-anchor-rich-192k.json`
|
|
is the first anchor-rich receipt: timing chips, formant pivots,
|
|
harmonic-envelope notches, payload ornaments, renderer template caching, and
|
|
per-event intra-call anchor measurements. It clears 10x realtime and
|
|
improves loopback to about 2.23 us MAE, but physical mics regress/hold at
|
|
tens of microseconds, so anchor survivability and path weighting are now the
|
|
problem.
|
|
Path-level loopback truth now compares candidate mic anchors directly
|
|
against loopback anchors, then refines delay with event-local waveform
|
|
correlation against the captured loopback packet. On the stored anchor-rich
|
|
capture, Release calibration runs at 21.1x realtime and improves physical
|
|
path precision to 7.576 us for the cardioid and 6.578 us for the shotgun.
|
|
A fresh room capture keeps the shotgun near 5.916 us but worsens the
|
|
cardioid to 18.930 us. The one-microsecond target remains open; next cut is
|
|
phase/group-delay correction or a better direct-path anchor family. A naive
|
|
recursive waveform phase-lock pass was tested and reverted because it could
|
|
improve one cardioid receipt while worsening the shotgun and locking a fresh
|
|
cardioid run onto a later reflection lobe. Recursion is only allowed back as
|
|
a constrained direct-path/group-delay model, not as unconstrained waveform
|
|
resampling with prettier decimals. The sonar/DSP research note
|
|
`research/sonar-dsp-recursive-refinement-2026-05-25.md` points the next
|
|
decoder cut at complex contour matched filters, phase-slope/group-delay
|
|
fitting, acquisition/tracking loop bandwidths, and sparse multipath
|
|
residuals. Its source ledger and local mirrors live under
|
|
`research/sonar-dsp-recursive-refinement-2026-05-25/`.
|
|
`MimirComplexContourMatchedFilterBank` and `MimirDirectPathTracker` are the
|
|
first implementation slice of that shape: complex anchor responses keep
|
|
multiple lobes, the current path prediction gates direct-path ownership, and
|
|
later coherent clusters are reported as reflection taps. Synthetic 192 kHz
|
|
reflection smoke currently lands about 5.249 us from expected delay; stored
|
|
Scarlett shotgun receipts land within about 5.860 us and 7.247 us of seeded
|
|
path fits, while cardioid receipts land within about 15.817 us and 8.963 us.
|
|
Low confidence is expected until phase/group-delay surfaces are learned. The
|
|
tracker now exposes per-band delay/phase residual observations and a
|
|
`MimirDirectPathChannelModel` correction surface for multi-window
|
|
calibration; one-window self-correction is not live authority. The tracker
|
|
now consumes explicit channel models as phase/group-delay correction and
|
|
downweights bands outside the learned usable surface. `Mimir.BufferSmoke
|
|
--learn-complex-contour-channel-model` persists
|
|
`calibration/bioacoustic/complex-contour-channel-model.json` from the
|
|
replay panel, and `--evaluate-complex-contour-channel-model` persists
|
|
`calibration/bioacoustic/complex-contour-channel-model-evaluation.json`.
|
|
Current evaluation improves 3/4 absolute path-seed errors and lowers mean
|
|
direct-cluster MAE; the fresh cardioid case reaches about 0.202 us from
|
|
seed, while the stored cardioid case still worsens to about 17.476 us. The
|
|
model is explicit calibration evidence, not default runtime authority.
|
|
The complex contour receiver is now wired into runtime as an explicit live
|
|
lane: when `enableComplexContourRuntime` is true, Mimir emits the configured
|
|
`bioacousticWitnessProfileId` through `MimirBioacousticContestantRenderer`,
|
|
loads `complexContourChannelModelPath`, extracts Float32 windows from the
|
|
rolling buffers, and publishes `evidence=complex-contour` reports. The
|
|
synthetic runtime-shaped proof `--complex-contour-runtime-self-test` at
|
|
192 kHz recovers a 693.5-sample delay with about 0.219 us error.
|
|
A fresh live Scarlett proof is preserved at
|
|
`calibration/bioacoustic/complex-contour-live-20260525-063229.md`:
|
|
Focusrite ASIO playback/capture measured Loopback 1 -> Loopback 2 at
|
|
-0.014 us with corrected confidence 0.971. The apparent `asio-ch0`
|
|
near-seed/cardioid estimate is invalidated: signal inspection shows flat
|
|
low-level noise with zero-lag loopback correlation 0.000032, so the tracker
|
|
fit noise under seeded timing pressure. `asio-ch1` remains only a weak,
|
|
non-authoritative input witness. The next cut is signal-presence gating,
|
|
independent path truth, and better frequency/direct-path discrimination.
|
|
`calibration/bioacoustic/complex-contour-replay-panel.json` persists the
|
|
current four-case replay receipt.
|
|
Acoustic robustness remains open.
|
|
The old arbitrary-chirplet matcher is a diagnostic/reference artifact only,
|
|
not the runtime hot path.
|
|
- `IMimirVideoCaptureDriver` and `MimirVideoCaptureDriverSource` are the live
|
|
seam for Leap, Media Foundation, DirectShow, libusb, LeapC, or shared
|
|
texture capture.
|
|
- `native/reservoir` owns the lower shared-edge rolling-buffer invariant for
|
|
Fensalir/Faust integration.
|
|
- `config/perfect-machine.example.json` declares live owners, sensors,
|
|
reservoir views, and runtime policy.
|
|
- `src/Mimir.Runtime/Synchronization` now includes the Perfect Machine module
|
|
library distilled from the research notes: node profiles, decoder/language
|
|
profiles, path learning, benchmark panels, actuator strategies, capture and
|
|
ingest strategies, reservoir strategies, field profiles, compute offload,
|
|
program publication, CultMesh contracts, Fensalir lowering, and exportable
|
|
manifest generation.
|
|
- `docs/perfect-machine-module-library.md` indexes those reusable runtime
|
|
modules and their smoke commands.
|
|
- `docs/code-algorithm-map.md` is the source-level ownership and algorithm map
|
|
for the current repo code surface.
|
|
- `docs/perfect-machine-domain-index.md` indexes Mimir's full problem domain,
|
|
current evidence taxonomy, hot paths, and implementation cuts.
|
|
- `docs/mimir-program-composition.md` owns the current program-composition
|
|
authority map: Muninn observes, Mimir composes/controls, Eve lowers operator
|
|
surfaces, Yggdrasil publishes, and OBS is compatibility-only.
|
|
- `src/Mimir.Runtime/Synchronization` now includes typed program contracts:
|
|
`mimir.program_scene.v1`, `mimir.program_output.v1`, and
|
|
`mimir.eve_operator_surface.v1`.
|
|
- `state/mimir-program-composition.cc` is the current imported Mimir program
|
|
state, generated from OBS scene JSON by `Mimir.BufferSmoke
|
|
--import-obs-program-scene`. It preserves the current Starfire monitor,
|
|
Discord crop/chroma key, Raven monitor, camera, browser, and Mimir texture
|
|
layers as Mimir-owned typed state.
|
|
- `src/Mimir.CultMeshMedia` owns the first explicit CultMesh media/body
|
|
bridge. Raven publishes muxed H.264/AAC MPEG-TS chunks as rolling
|
|
`mimir.cultmesh_media_frame` documents over CultNet reliable UDP,
|
|
Yggdrasil relay routing comes from
|
|
`cultmesh://asgard.yggdrasil.mimir/media/raven-primary-av` through
|
|
Odin/CultMesh discovery, and Starfire lowers the subscribed stream to local
|
|
UDP for the OBS source `Raven Monitor + Realtek`. The bridge now uses
|
|
explicit CultLib RUDP socket/session helpers for relay, sender, and
|
|
receiver. The old `CultMesh.StartNodeAsync`/`CultMesh.ConnectClient`
|
|
node/client path and raw sender/receiver host-port config are no longer
|
|
transport authorities for this media lane.
|
|
- `src/Mimir.EveDashboard` is archived as a fail-closed tombstone. It no
|
|
longer opens a TCP listener, serves `/eve/deck`, accepts WebSocket commands,
|
|
or publishes Idunn health. Its replacement must publish provider
|
|
advertisement, retained state, command boundary, and transport profile as
|
|
typed CultMesh/Odin documents; product/debug render surfaces remain client
|
|
lowerings.
|
|
- `src/Mimir.EveBrowserReference` serves a static Eve/browser lowering and
|
|
can publish its own `idunn.daemon_health` record over
|
|
`cultnet.transport.rudp.v0`. It is a renderer/reference health witness, not
|
|
an authority over Mimir program state or Idunn keepalive decisions.
|
|
- `research/perfect-machine-study-2026-05-23/` preserves the architecture
|
|
rumination, optimization ledger, research references, source/hotspot audits,
|
|
Fensalir/native boundary maps, benchmark plan, calibration-session spec,
|
|
distributed receiver spec, failure-mode ledger, data dictionary, low-level
|
|
implementation notes, and sample code sketches for streaming chirp-bin
|
|
decode, AVX2/GPU scoring, batched native ABI, fractional-delay/SRO
|
|
actuation, acoustic source localization, Mimir-to-Fensalir lowering, and
|
|
native capture rings. The corrected deep-study pass ran from
|
|
`2026-05-23T23:54:42.9041588+01:00` through at least
|
|
`2026-05-24T01:54:56.9747630+01:00`; use
|
|
`research/perfect-machine-study-2026-05-23/reading-guide.md` as the
|
|
re-entry index.
|
|
- `.voidbot` state is part of repo memory and must be committed separately
|
|
whenever dirty.
|
|
|
|
active_subgoals:
|
|
- Implement the first concrete Leap direct capture driver and measure cadence.
|
|
- Add remaining local camera drivers through `IMimirVideoCaptureDriver`.
|
|
- Use the verified in-process Focusrite ASIO source for local Scarlett mic and
|
|
loopback ingest; WASAPI remains a diagnostic witness.
|
|
- Add network audio feed producers that enter the same runtime clock/rebuffer
|
|
machinery instead of becoming independent authorities.
|
|
- Bind Fensalir UI to stream health, buffer depth, source timestamps,
|
|
settings, and output management.
|
|
- Keep PowerShell/FFmpeg/SRT as bridge utilities until direct program output
|
|
replaces them.
|
|
- Launch Raven's CultMesh sender directly on Raven because SSH to Raven over
|
|
WireGuard timed out during the initial relay deployment pass.
|
|
- Keep Mimir Eve dashboard and browser reference aligned with Odin/Idunn
|
|
doctrine: Odin accepts Verse/service visibility, Idunn owns keepalive, and
|
|
Mimir surfaces publish typed CultMesh/Odin state rather than private
|
|
watchdog loops, product status probes, or socket brokers.
|
|
- Rebuild `src/Mimir.EveDashboard` only after daemon truth, provider catalogs,
|
|
command boundaries, and typed program document paths use explicit CultMesh/
|
|
Odin documents. `Mimir.EveSensorReceiver`, `Mimir.VerseRecorder`, the
|
|
Nightwing typed witness publisher, and `start-nightwing-move-tracking.ps1`
|
|
are archived so the old witness subscription lane cannot retake transport
|
|
authority. `src/Mimir.CultMeshMedia` has completed this cut for the
|
|
Raven/Yggdrasil/Starfire media-frame lane.
|
|
|
|
current_status:
|
|
summary: >
|
|
The repo owns a C# Mimir app/runtime scaffold, a native reservoir crate, OBS
|
|
bridge scripts, and persistence surfaces. Script-era packages, tests,
|
|
helper tools, dependency manifests, and stale docs have been removed from
|
|
the working tree. Research archives remain as reference-only memory for raw
|
|
native/GPU implementation shape. A raw PS3 Eye libusb/WinUSB cadence probe
|
|
measured both PS3 Eyes simultaneously at about 183 fps each at 320x240 and
|
|
57-59 fps each at 640x480. A native Kernel Streaming LeapUVC probe can pull
|
|
frames through usbvideo.sys without Media Foundation, DirectShow, OpenCV, or
|
|
frame processing; after discarding a two-second warm-up, the useful 640x240
|
|
stereo IR mode advertises 115 fps and currently measures about 110.8 fps
|
|
with eight queued asynchronous KS reads. The half-height 640x120 mode
|
|
advertises 214 fps and currently measures about 205.9 fps. Plain Kiyo is
|
|
usable through KS when manual exposure is forced shorter: 1080p MJPG reaches
|
|
about 30 fps and 720p MJPG reaches about 51.5 fps. Kiyo Pro advertises 60
|
|
fps MJPG at 720p/1080p but currently delivers only about 25 fps through the
|
|
same KS path even with manual exposure tuning. Standard KS low-light
|
|
compensation off does not restore 60 fps, powerline 60Hz only raises Pro
|
|
cadence to about 29 fps, and standard extended Video HDR is not exposed on
|
|
this KS path. The controls-only probe now parses USB VideoControl
|
|
descriptors and found the Kiyo Pro's two vendor extension units. One
|
|
extension unit answers writable KS selectors. Moving the Pro directly to a
|
|
motherboard USB3 port changed the path to `root_hub30`, `bcdUSB=0x0320`,
|
|
and exposed 720p/1080p YUY2/MJPG/H264/NV12 at 60 fps, but Windows still
|
|
reports `UsbHighSpeed` rather than `UsbSuperSpeed`, and baseline cadence is
|
|
still about 25 fps across every tested format. The first simultaneous local
|
|
camera pull only had three visible devices: LeapUVC, Kiyo Pro, and one PS3
|
|
Eye. The KS multi-stream harness plus raw PS3 Eye probe pulled all three at
|
|
once, but Leap cadence collapsed under shared load: with PS3 Eye at
|
|
640x480@60 the set delivered Leap 40.12 fps, Kiyo Pro 24.99 fps, and PS3
|
|
Eye 56.06 fps; with PS3 Eye at 320x240@187 it delivered Leap 81.10 fps,
|
|
Kiyo Pro 25.33 fps, and PS3 Eye 185.74 fps. After port swapping, five
|
|
camera devices / six optical sensors were visible and pulled at once:
|
|
LeapUVC's packed stereo IR pair, Kiyo Pro, regular Kiyo, and two PS3 Eyes.
|
|
With both Eyes at 640x480@60, the set delivered Leap stereo 57.33 fps,
|
|
Kiyo Pro 24.97 fps, Kiyo 30.30 fps, PS3 Eye 0 59.59 fps, and PS3 Eye 1
|
|
57.60 fps. With both Eyes at 320x240@187, it delivered Leap stereo 81.05
|
|
fps, Kiyo Pro 24.97 fps, Kiyo 30.28 fps, PS3 Eye 0 187.60 fps, and PS3
|
|
Eye 1 187.03 fps. Kiyo Pro still negotiates high-speed USB, and Leap
|
|
remains topology-sensitive under shared load. A follow-up moved Leap alone
|
|
onto the blue rear USB3 group/root_hub30#5 while both Kiyos shared
|
|
root_hub30#7&18eea21b and one visible PS3 Eye used root_hub30#7&124f193f;
|
|
Leap still delivered only 80.99 fps with the other visible cameras active,
|
|
so the remaining loss is not explained by same-root-hub contention alone.
|
|
Splitting Kiyo Pro and one PS3 Eye to Raven left the Mimir host with
|
|
LeapUVC, regular Kiyo, and one PS3 Eye, but local cadence still stayed low:
|
|
Leap delivered 81.34 fps with Kiyo+Eye, 82.77 fps with Kiyo only, and only
|
|
84.18 fps in a same-session Leap-only baseline. The previous ~110.8 fps
|
|
Leap result was not reproduced until the Focusrite Scarlett Solo was moved
|
|
away from the Leap USB neighborhood. After that, Leap-only baseline rose to
|
|
95.48 fps, best full-height scenario to 101.67 fps, and the local split set
|
|
delivered Leap 105.10 fps, regular Kiyo 30.17 fps, and one PS3 Eye 185.81
|
|
fps. The split-host layout is viable if the Leap's USB neighborhood stays
|
|
free of the Focusrite and other heavy devices. Raven's PS3 Eye raw interface
|
|
now has the same libwdi WinUSB binding as the local Eye and can be opened by
|
|
the copied cadence probe, but Raven Eye cadence is currently unusable:
|
|
320x240@187 delivered 132.24 fps then 78.93 fps, and 640x480@60 delivered
|
|
6.32 fps with multi-second gaps. Moving that Eye back to the Mimir host
|
|
produced the current clean local topology: LeapUVC, regular Kiyo, and two
|
|
PS3 Eyes run in parallel while Kiyo Pro stays off-host. With both Eyes at
|
|
320x240@187, Leap delivered 106.07 fps, Kiyo 30.33 fps, and the Eyes 186.57
|
|
/ 186.14 fps. With both Eyes at 640x480@60, Leap delivered 108.07 fps, Kiyo
|
|
30.34 fps, and the Eyes 57.79 / 58.08 fps. Moving Kiyo Pro back to the
|
|
Mimir host while regular Kiyo was absent kept Leap healthy: with both Eyes
|
|
at 320x240@187, Leap delivered 110.80 fps, Kiyo Pro 24.97 fps, and the Eyes
|
|
187.46 / 186.89 fps; with both Eyes at 640x480@60, Leap delivered 110.72
|
|
fps, Kiyo Pro 24.87 fps, and the Eyes 59.91 / 59.19 fps. Kiyo Pro still
|
|
negotiates UsbHighSpeed(2), but it does not poison Leap cadence in the
|
|
current layout. Moving regular Kiyo from front panel to a rear port made
|
|
all five camera devices / six optical sensors visible locally. With both
|
|
Eyes at 320x240@187, the full set delivered Leap 110.84 fps, Kiyo Pro 25.08
|
|
fps, Kiyo 30.18 fps, PS3 Eye 0 187.59 fps, and PS3 Eye 1 187.02 fps. With
|
|
both Eyes at 640x480@60, it delivered Leap 110.76 fps, Kiyo Pro 25.14 fps,
|
|
Kiyo 30.18 fps, PS3 Eye 0 59.66 fps, and PS3 Eye 1 59.95 fps. A mouse in
|
|
the Leap USB neighborhood does not reproduce the Focusrite-induced collapse.
|
|
next_action: >
|
|
Use the stable single-provider Nightwing Move stream to collect a spatially
|
|
diverse stereo calibration session. Calibrate each Eye's intrinsics, fit
|
|
their relative extrinsics from timestamped same-ID correspondences, use
|
|
apparent orb radius only as a weighted distance cue after radius calibration,
|
|
and publish typed camera calibration plus residual evidence. Do not claim
|
|
precise camera matrices from overlap or radius alone. Then feed calibrated
|
|
witnesses through MimirMoveFusion and the named Fensalir proof spine.
|
|
Current app testing proves Fensalir WASAPI output wakes Scarlett speaker
|
|
loopback, live mic buffers stay populated, runtime sync analysis now runs as
|
|
a bounded rotating service without UI/telemetry re-entry, and the bioacoustic
|
|
motif timeline produces deterministic synthetic active sync. The next
|
|
invariant is self-identifying song contours producing stable intra-call
|
|
time/frequency anchors and clock fits through real loopback and microphones,
|
|
so the actuator can use decoded time instead of filtered lag guesses.
|
|
Replace the remaining
|
|
diagnostic JSON-line process bridges with direct native capture drivers that
|
|
push buffer handles into Mimir.Runtime. Mimir.BufferSmoke still proves live buffers and sync reports
|
|
for Scarlett loopback, Focusrite mic, Kiyo mic, and Kiyo Pro mic when
|
|
calibration witnesses are playing. Treat PS3 Eye audio as
|
|
enumeration/runtime-fragile, not unavailable. Keep the Focusrite away from
|
|
the Leap USB neighborhood. The Scarlett Solo 4th Gen is now attached to
|
|
Starfire, and `native/probes/asio_audio_cadence` proves local Focusrite USB
|
|
ASIO access with 4 inputs / 2 outputs including Loopback 1/2, 192-frame
|
|
preferred buffers, 44.1-192 kHz support, and nonzero 4-channel Int32LSB
|
|
input callbacks at 192 kHz. Raven also has a 192 kHz loopback-capable
|
|
Scarlett for co-streamer/game timing evidence, but Starfire owns the heavy
|
|
soundfield and sensor-fusion work. Treat Kiyo Pro as a ~25 fps source until
|
|
its SuperSpeed negotiation is fixed.
|