59 lines
21 KiB
JSON
59 lines
21 KiB
JSON
{"ts":"2026-05-18T00:00:00+00:00","type":"decision","status":"ok","note":"Use FFmpeg SRT endpoints plus OBS Media Source for v1; OBS plugin or fork is not justified until standard ingest fails a concrete invariant."}
|
|
|
|
{"ts": "2026-05-18T09:55:45+00:00", "type": "verification", "status": "ok", "note": "State helper and sender dry-run produce valid repo status and SRT URLs with explicit ports for the example config."}
|
|
{"ts": "2026-05-18T10:24:49+00:00", "type": "deployment", "status": "ok", "note": "Neighbor sender 192.168.1.84 has LocalCastBridge under C:\\Meta\\LocalCastBridge, FFmpeg 8.1.1 with SRT/h264_nvenc, and desktop start/stop launchers targeting receiver 192.168.1.66 ports 5100/5101."}
|
|
{"ts": "2026-05-18T10:27:41+00:00", "type": "deployment", "status": "ok", "note": "Receiver OBS Untitled scene now has Neighbor PC - Video and Neighbor PC - Focusrite ffmpeg_source Media Sources listening on SRT ports 5100 and 5101; original scene JSON was backed up before edit."}
|
|
{"ts": "2026-05-18T10:52:05+00:00", "type": "deployment", "status": "ok", "note": "Installed Voicemeeter and SoundVolumeView on sender, added system-loopback using DirectShow device Voicemeeter Out B3 on SRT port 5102, and added matching OBS source Neighbor PC - System Audio."}
|
|
{"ts": "2026-05-18T11:38:25+00:00", "type": "fix", "status": "ok", "note": "Transmission failure traced to three issues: stale OBS SRT source state, sender live launcher splitting/closing FFmpeg due Start-Process/stdin handling, and SSH-launched gdigrab seeing a 1024x768 remote desktop with capture error 5. Patched sender-start with quoted argument string, -nostdin, repo-root logs, and set sender video config to 1024x768; OBS restarted and listening on UDP 5100-5102."}
|
|
{"ts": "2026-05-18T11:39:19+00:00", "type": "decision", "status": "ok", "note": "Ambisonic AquaSynth path should use one clocked multichannel capture feeding a declared ambisonic bus; four heterogeneous mics must not be treated as raw first-order A-format without geometry and calibration."}
|
|
{"ts": "2026-05-18T11:43:08+00:00", "type": "fix", "status": "ok", "note": "Madman's interactive launcher could not resolve winget aliases for SoundVolumeView, so desktop CMD files now use absolute winget-link paths. Sender video config restored to 1920x1080 because 1024x768 was only the SSH session display and captured a slice."}
|
|
{"ts": "2026-05-18T11:45:18+00:00", "type": "fix", "status": "ok", "note": "Replaced path-heavy desktop CMD launchers with PowerShell wrappers start-localcast-desktop.ps1 and stop-localcast-desktop.ps1 after cmd.exe parsed Voicemeeter/SoundVolumeView paths incorrectly on Madman's interactive desktop."}
|
|
{"ts": "2026-05-18T11:50:22+00:00", "type": "decision", "status": "ok", "note": "Live adaptive sync research supports a two-loop design: slow SRO estimation/variable-rate resampling plus faster delay/TDOA tracking; prove with a two-mic harness before feeding Ambisonic AquaSynth."}
|
|
{"ts": "2026-05-18T11:52:12+00:00", "type": "decision", "status": "ok", "note": "AquaSynth Ambisonic output must include a playback profile: speaker layout decoder plus per-speaker delay/gain/response correction and latency ledger; playback calibration is separate from capture sync."}
|
|
{"ts": "2026-05-18T11:55:34+00:00", "type": "decision", "status": "ok", "note": "Speaker feedback should be mined as an online system-identification signal: estimate a speaker-to-mic transfer matrix with confidence/convergence gates, separate from clock sync, localization, and Ambisonic decoding."}
|
|
{"ts": "2026-05-18T12:01:09+00:00", "type": "decision", "status": "ok", "note": "Visual tracking subsystem assumes PlayStation Eye / PS3 Eye cameras for high-rate low-res tracking; PS2 EyeToy is only a naming confusion/rejected path."}
|
|
{"ts": "2026-05-18T12:16:09+00:00", "type": "decision", "status": "ok", "note": "Visual rig is two PS3 Eyes for high-rate low-res tracking, two Razer Kiyo-class RGB/mic cameras (one Pro), and one Leap Motion/LeapUVC stereo IR sensor for near-field fusion; all publish into one world model."}
|
|
{"ts": "2026-05-18T12:21:18+00:00", "type": "constraint", "status": "ok", "note": "LeapUVC driver mode makes the Leap Motion appear as a generic UVC device and prevents addressing it through Ultraleap software; do not require LeapUVC raw frames and Ultraleap tracking simultaneously unless the actual machine proves both can coexist."}
|
|
{"ts": "2026-05-18T12:25:08+00:00", "type": "decision", "status": "ok", "note": "Fusion/rendering approach should follow EpiphanyAquarium domain-cache discipline: world-space semantic claims are truth, volumetric brushstroke/splat packets are cached render lowerings, and ambisonics reads listener/source claims rather than rendered geometry."}
|
|
{"ts": "2026-05-18T12:32:40+00:00", "type": "decision", "status": "ok", "note": "GPU fusion/splatting pass: use pinned host frame rings, async CUDA uploads, GPU feature/marker/stereo extraction, GPU-shaped observation and brush packet buffers, and CUDA/Vulkan/D3D splat rendering with minimal readback; CPU owns orchestration and semantics."}
|
|
{"ts": "2026-05-18T12:46:08+00:00", "type": "verification", "status": "partial", "note": "Calibration harness enumerated the local rig: LeapUVC raw IR video, two Razer Kiyo-class RGB cameras, Scarlett/Kiyo/PS3 Eye microphones, and Scarlett speakers are accessible; both PS3 Eye video interfaces are attached but report Windows problem code 28 (missing video-side driver)."}
|
|
{"ts": "2026-05-18T13:14:03+00:00", "type": "decision", "status": "ok", "note": "PS3 Eye video driver path should prefer the opentrack open driver via Zadig/libusb on interface MI_00; CL Eye remains historical reference only, not a repo dependency or redistribution target."}
|
|
{"ts": "2026-05-18T13:24:02+00:00", "type": "verification", "status": "partial", "note": "Footage status: LeapUVC and two Kiyo-class cameras produce snapshots; PS3 Eye video still needs opentrack/Zadig driver. LeapUVC MSMF accepted 320x240x120 and reported about 115 fps, but rough OpenCV measured throughput was about 52 fps in a short probe; Kiyo FPS mode probes timed out."}
|
|
{"ts": "2026-05-18T13:48:09+00:00", "type": "verification", "status": "partial", "note": "PS3 Eye MI_00 interfaces now bind as libusb-win32 and opentrack enumerates valid modes up to 320x240@205Hz/640x480@83Hz, but frame reads fail with payload error data[1]=78; next cuts are libusbK or WinUSB fallback and one-camera/direct-port USB tests."}
|
|
{"ts": "2026-05-18T13:51:32+00:00", "type": "verification", "status": "partial", "note": "USB topology after PS3 Eye driver install: Eyes, Kiyos, and Leap share an external hub/root path; one PS3 Eye composite/audio side and two descriptor-failed devices show Unknown while opentrack mode enumeration works but frame streaming fails. Treat hub topology/bandwidth/scheduling as live suspect."}
|
|
{"ts": "2026-05-18T13:56:00+00:00", "type": "verification", "status": "partial", "note": "With one PS3 Eye active after hub replug and switching to the other Eye, opentrack again enumerates high-rate modes but frame reads fail with payload error data[1]=78/bad header. This reduces suspicion of a single bad Eye and points to libusb0/backend or hub streaming."}
|
|
{"ts": "2026-05-18T14:02:51+00:00", "type": "verification", "status": "partial", "note": "Both PS3 Eyes direct to motherboard and live MI_00 instances bound as libusbK: opentrack mode-test now reports all listed modes GOOD including 320x240@30, but frame-test still fails with payload error data[1]=78/79 and bad headers. Stale Unknown PS3 Eye instances from previous hub path remain in PnP."}
|
|
{"ts": "2026-05-18T14:05:40+00:00", "type": "verification", "status": "failed", "note": "WinUSB on the two live PS3 Eye MI_00 instances still enumerates high-rate modes but frame streaming fails: one device open hit Access is denied, the other repeated payload error data[1]=78/79 until killed. libusb0, libusbK, and WinUSB have all failed frame reads via opentrack on this machine."}
|
|
{"ts": "2026-05-18T14:22:36+00:00", "type": "verification", "status": "partial", "note": "Audio-field sidecar added and offline FOA encode verified; current PortAudio enumeration shows no matching six-channel WASAPI input device, so live Ambisonic capture needs a shared six-input interface/profile or a separately proven adaptive sync path before using independent USB microphones."}
|
|
{"ts": "2026-05-18T14:25:36+00:00", "type": "verification", "status": "ok", "note": "PS3 Eye video path found: AllanCat PS3 Eye Universal Driver / PS3EyeDirectShow exposes both Eyes as DirectShow indices 3 and 4 after registering the source filter for two cameras; simultaneous default 640x480 capture produced about 30 fps from both. High-FPS DirectShow mode negotiation remains unproven."}
|
|
{"ts": "2026-05-18T14:48:48+00:00", "type": "verification", "status": "ok", "note": "PS3 Eye frame-rate squeeze: OpenCV DirectShow indices are reliable only at default 640x480@30; FFmpeg DirectShow capture by device name negotiates high-rate modes. Both PS3 Eyes simultaneously delivered 320x240@60 with 300 frames per Eye over five seconds. Dual 640x480@60 works but settles around 54-56 fps per Eye. Requests above 60 fps through the installed PS3EyeDirectShow filter do not increase delivered frame rate."}
|
|
{"ts": "2026-05-18T14:50:58+00:00", "type": "verification", "status": "partial", "note": "Audio profile updated for actual distributed rig: local Focusrite shielded cardioid and neighbor Focusrite shotgun are dialogue anchors; Kiyo/PS Eye mics are spatial/context sources. Local PortAudio/WASAPI probe: Scarlett input accepts 48 kHz and rejects 96 kHz; Scarlett output accepts 44.1 kHz and rejects 48/96 kHz, so ASIO/native Focusrite access must be checked before high-rate calibration."}
|
|
{"ts": "2026-05-18T14:55:13+00:00", "type": "decision", "status": "ok", "note": "Audio field live output may use bounded buffering: latency is acceptable while delay/SRO estimates settle, but cache depth must converge toward real-time. Core audio code now has testable seams for buffers, latency policy, injectable capture/alignment/encoder/sink ports, and pipeline orchestration."}
|
|
{"ts": "2026-05-18T14:57:32+00:00", "type": "decision", "status": "ok", "note": "Sensor-fusion sidecar starts with semantic observation authority: calibrated cameras plus timestamped 2D marker observations triangulate into cached 3D point claims, then lower to PLY/brush/splat packets. Capture adapters and render lowerings stay outside the pure fusion core so unit tests can inject synthetic cameras, frames, and observations."}
|
|
{"ts": "2026-05-18T15:07:24+00:00", "type": "decision", "status": "ok", "note": "OBS visualization boundary: LocalCastBridge should emit typed render-frame packets from the sensor-fusion cache; Aquarium Engine should consume them, render points/brushes/splats to a GPU render target, and publish that texture as Spout sender 'LocalCastBridge Point Cloud'. Render packets carry visual presentation and audio-alignment timestamps so latency can converge with the ambisonic output timeline."}
|
|
{"ts": "2026-05-18T15:12:32+00:00", "type": "verification", "status": "partial", "note": "Deadline audio path now supports distributed run manifests, local concurrent mic capture, sweep-arrival analysis, aligned six-channel WAV assembly with missing channels zero-filled, and FOA encoding. Live smoke captured local Kiyo-left, both PS Eye mics, and local Focusrite at 48 kHz; Kiyo-right profile query is currently missing and neighbor Focusrite remains remote/manual import."}
|
|
{"ts": "2026-05-18T15:19:47+00:00", "type": "decision", "status": "ok", "note": "Deadline OBS Spout path is now a live GPU sink: RenderFramePacket JSON feeds scripts/stream_spout.py, which renders to an OpenGL FBO and publishes Spout sender LocalCastBridge Point Cloud with heartbeat status. Aquarium remains the intended dense splat renderer behind the same packet boundary."}
|
|
{"ts": "2026-05-18T15:21:20+00:00", "type": "decision", "status": "ok", "note": "Calibration delay analysis now uses local chirplet refinement: coarse matched-filter sweep detection initializes a small fractional-delay/chirp-rate atom search, following chirplet parameter-estimation guidance. Plugged-in left/right sweeps produced sub-sample chirplet delays for Kiyo-left, both PS Eye mics, and local Focusrite; Kiyo-right remains missing and neighbor Focusrite still needs import."}
|
|
{"ts": "2026-05-18T15:22:33+00:00", "type": "decision", "status": "ok", "note": "Runtime mic sync must treat known speaker output as chirplet telemetry every block/frame, updating delay, SRO/rate-scale, phase, and confidence fields dynamically. Static sweep offsets are only initial state; low-confidence runtime chirplet observations should freeze updates rather than corrupt alignment."}
|
|
{"ts": "2026-05-18T15:23:38+00:00", "type": "decision", "status": "ok", "note": "Active sync probes are now an explicit optimizer responsibility: when passive chirplet telemetry leaves any source below confidence threshold, the system may emit bounded extra chirplets in masked windows, maximizing confidence gain per audible intrusion while respecting spacing and level limits."}
|
|
{"ts": "2026-05-18T15:35:24+00:00", "type": "verification", "status": "partial", "note": "Passive sync test using Scarlett output loopback as ground-truth content: 20s music run with normalized 4s/1s window correlation produced local Focusrite median delay 20115.5 samples/419.07 ms with 17 sample/0.35 ms MAD; PS Eye left 23848 samples/496.83 ms with 17 sample/0.35 ms MAD; Kiyo-left and PS Eye-right were noisier at about 3.9-4.2 ms MAD. Soundcard WASAPI loopback reported discontinuities, so loopback clock quality remains suspect."}
|
|
{"ts": "2026-05-18T15:38:07+00:00", "type": "verification", "status": "ok", "note": "OBS puppet pass installed win-spout, enabled local obs-websocket without auth, created Spout2 Capture source LocalCastBridge Point Cloud, and verified a nonblack OBS screenshot from the live Spout sender. Zyphos sample/research shifted the deadline sink from point sprites toward compact anisotropic screen brush packets."}
|
|
{"ts": "2026-05-18T15:44:14+00:00", "type": "verification", "status": "ok", "note": "Remote Focusrite shotgun capture path verified: SSH to 192.168.1.84 runs remote FFmpeg against Analogue 1 + 2 (Focusrite USB Audio), SFTP pulls a mono 48 kHz float WAV into a distributed calibration run, and the manifest records mic_focusrite_neighbor as field channel 5."}
|
|
{"ts": "2026-05-18T15:59:44+00:00", "type": "verification", "status": "partial", "note": "Simultaneous audio-field calibration run audio-full-sync-20260518-165751 captured local Kiyo-left, both PS Eye mics, local Focusrite, and neighbor Focusrite shotgun. Chirplet analysis aligned five sources; Kiyo-right remains missing. Response-compensated six-channel field and FOA AmbiX WAV were produced under calibration/runs/audio-full-sync-20260518-165751."}
|
|
{"ts": "2026-05-18T16:06:28+00:00", "type": "verification", "status": "ok", "note": "Visual pipeline now uses typed CultCache MessagePack state: live_sensor_fusion.py writes localcast.visual.render_frame to visual-state.msgpack, stream_spout.py consumes that document and publishes Spout. CultNet document_put is documented as the API boundary; JSON render-frame files are compatibility only. Live OBS heartbeat shows 256 fused points from visual-state.msgpack."}
|
|
{"ts": "2026-05-18T16:08:52+00:00", "type": "decision", "status": "ok", "note": "Aquarium/Faust spatial audio boundary is a typed CultCache document localcast.audio.spatial_frame carrying live 48 kHz AmbiX ACN/SN3D W,Y,Z,X float32 blocks with start_sample and audio_time_ns; Aquarium owns native Faust DSP while LocalCastBridge owns capture, sync, calibration, and AmbiX publication."}
|
|
{"ts": "2026-05-18T16:31:23+00:00", "type": "verification", "status": "partial", "note": "Dense chirplet texture run 20260518T163026.517259Z-audio-probe-train emitted 120 usable low-level multi-band speaker probes over 8s. Band-aware analysis produced per-mic phase-frequency correction vectors and smoothed sub-sample phase deltas, but coarse delays still include large device/driver latency terms; geometry requires a joint solve separating latency/drift from acoustic path."}
|
|
{"ts": "2026-05-18T16:40:56+00:00", "type": "verification", "status": "partial", "note": "Room suppression first cut added: suppress-room treats Focusrite dialogue anchors as desired direct sources and camera mics as room/transient witnesses, attenuating witness-heavy broadband transients before FOA. On audio-full-sync-20260518-165751 it wrote field-cleaned.wav and field-cleaned-foa-ambix.wav, reducing aligned RMS from 0.00162 to 0.00126 and FOA RMS from 0.000408 to 0.000308 while firing on 101 transient blocks."}
|
|
{"ts": "2026-05-19T00:42:22+00:00", "type": "decision", "status": "ok", "note": "Live fusion horizon is a five-second spatiotemporal reservoir: frame history expires from the newest shared sample, LOD evidence is clipped to that window, and RenderFramePacket declares the source window. Live audio field now owns local mic/phase capture with closed-loop Scarlett probes, bounded probe artifacts, and 44.1 kHz probe playback resampling."}
|
|
{"ts": "2026-05-19T00:45:41+00:00", "type": "decision", "status": "ok", "note": "Multi-LOD scene cache now carries first-pass material evidence per cell: weighted albedo plus roughness/metallic hints derived from source evidence. Aquarium owns final BRDF refinement, but it now receives relightable material state from the five-second reservoir."}
|
|
{"ts": "2026-05-19T00:57:57+00:00", "type": "verification", "status": "partial", "note": "Direct synchronous visual capture cannot currently satisfy the five-second live reservoir: RGB-only 8s probe took ~21.8s and Leap-only 8s probe took ~13.5s with MSMF/libusb errors. Deadline run now uses rgb/leap fallback-only mode plus Spout reservoir clamping, producing synchronized OBS output at audio_delta_ns 0 with ~1.5k points. Next cut is nonblocking camera ingress."}
|
|
{"ts": "2026-05-19T01:05:40+00:00", "type": "verification", "status": "ok", "note": "Nonblocking camera ingress cut: LatestFramePump moves OpenCV capture reads off the live fusion hot loop. Restarted live visual producer with lower per-frame density (--points 64, rgb/leap step 16); Spout heartbeat reported audio_delta_ns 0 and ~1.6k visual points under the five-second reservoir."}
|
|
{"ts": "2026-05-19T01:09:34+00:00", "type": "verification", "status": "ok", "note": "Active probe maintenance now filters to live captured source ids in stream_live_audio_field; missing placeholders such as co-streamer-focusrite remain stable output channels but no longer consume chirp budget. Live restart showed probes targeting ps-eye-0 with playback error empty."}
|
|
{"ts": "2026-05-19T06:31:33+00:00", "type": "decision", "status": "ok", "note": "Perfect Machine target is now explicit: LocalCastBridge owns config/calibration/status, Aquarium owns dense GPU fusion/material/brush rendering and Spout publication, Faust owns hot audio DSP, and one five-second native reservoir is the live authority. Python remains tooling and compatibility, not the stream hot path."}
|
|
{"ts": "2026-05-19T06:35:13+00:00", "type": "implementation", "status": "ok", "note": "Started native Perfect Machine runtime with native/reservoir Rust crate. It owns shared-edge five-second sample retention for typed rings and passes cargo tests; Aquarium/Faust bindings should build on this invariant instead of copying deadline bridge caches."}
|
|
{"ts": "2026-05-19T07:39:41+00:00", "type": "decision", "status": "ok", "note": "Native rebuild cut line recorded: live path must abandon Python producers, OpenGL Spout sink, JSON live stores, and independent typed rings; next foundation is one native rolling reservoir with typed indexes/views feeding Aquarium, Faust, OBS, and typed CultNet docs."}
|
|
{"ts": "2026-05-19T07:52:44+00:00", "type": "implementation", "status": "ok", "note": "Native reservoir first cut complete: native/reservoir now uses one time-ordered rolling buffer with typed views, total/view length ABI, latest-by-kind/source lookup, and tests proving single-edge expiry plus stale late-sample exclusion. Python/OpenGL bridge remains next quarantine target."}
|
|
{"ts": "2026-05-19T07:57:59+00:00", "type": "implementation", "status": "ok", "note": "OpenGL Spout sink quarantined: localcast.sensor_fusion no longer exports spout_output surfaces; the old SpoutGL renderer now lives under localcast.diagnostics.spout_output and scripts/stream_spout imports it as diagnostic/migration code only."}
|
|
{"ts": "2026-05-19T08:01:04+00:00", "type": "implementation", "status": "ok", "note": "Python visual producer quarantined by name: scripts/live_sensor_fusion.py is now scripts/diagnostic_live_sensor_fusion.py, tests/docs import the diagnostic path, and future native/Aquarium work should delete or split its pure pieces rather than extend it."}
|
|
{"ts": "2026-05-19T08:03:24+00:00", "type": "implementation", "status": "ok", "note": "Diagnostic Spout launcher renamed: scripts/stream_spout.py is now scripts/diagnostic_stream_spout.py, keeping the OpenGL bridge visibly outside the production path while Aquarium Spout publication remains the target."}
|
|
{"ts": "2026-05-19T08:07:19+00:00", "type": "implementation", "status": "ok", "note": "Native reservoir ABI now exposes indexed reads over the total rolling buffer and typed views via localcast_reservoir_sample_at and localcast_reservoir_view_sample_at, allowing Aquarium/Faust consumers to walk the live window without owning private histories."}
|