Add common-space calibration and surface tracking
This commit is contained in:
parent
9cf6ea7527
commit
905fc60dbd
10 changed files with 610 additions and 16 deletions
|
|
@ -22,6 +22,8 @@
|
|||
- `audio_field/` contains testable core modules for source buffering, bounded latency convergence, injectable capture/alignment/encoder/sink ports, and pipeline orchestration.
|
||||
- `docs/audio-field.md` maps the audio pipeline and its invariants.
|
||||
- Sensor-fusion render bridge:
|
||||
- `localcast.sensor_fusion.calibration_space` solves fixed-board ChArUco observations into camera poses in one common world frame.
|
||||
- `localcast.sensor_fusion.surface_features` matches cross-view features and triangulates calibrated surface tracks.
|
||||
- `localcast.sensor_fusion.render_bridge` emits render-frame packets with visual/audio timing metadata and Spout sender identity.
|
||||
- `localcast.sensor_fusion.cultcache_docs` stores live visual state as typed CultCache MessagePack documents.
|
||||
- `localcast.sensor_fusion.spout_output` renders render-frame packets into a GPU texture and publishes it as a named Spout sender for OBS.
|
||||
|
|
@ -56,6 +58,7 @@
|
|||
9. Create local `config/audio-field.json`, confirm local Kiyo/PS Eye/Focusrite device matches, and confirm the neighbor Focusrite shotgun capture/transport path.
|
||||
10. Replace placeholder mic/speaker geometry with measured world coordinates, then build the delay/SRO alignment stage that feeds the bounded field cache and emits aligned six-channel blocks before FOA encoding.
|
||||
11. Move the render-frame consumer into Aquarium Engine so dense brush/splat rendering replaces the deadline OpenGL point sink behind the same OBS Spout boundary.
|
||||
12. Replace synthetic live-fusion observations with PS3 Eye detector observations.
|
||||
13. Keep the neighbor loopback leg running through the scheduled interactive WASAPI capture path and use it as the timing witness once actual neighbor app audio is present.
|
||||
14. Let the co-streamer surface delay drive the shared presentation buffer horizon for audio stems, AmbiX, and remote video.
|
||||
12. Capture fixed-board ChArUco observations for every camera, solve `config/sensor-fusion.json`, and verify cross-view feature triangulation with `scripts/triangulate_surface_features.py`.
|
||||
13. Replace synthetic live-fusion observations with PS3 Eye detector observations and calibrated surface-feature tracks.
|
||||
14. Keep the neighbor loopback leg running through the scheduled interactive WASAPI capture path and use it as the timing witness once actual neighbor app audio is present.
|
||||
15. Let the co-streamer surface delay drive the shared presentation buffer horizon for audio stems, AmbiX, and remote video.
|
||||
|
|
|
|||
Loading…
Reference in a new issue