138 lines
5.4 KiB
Markdown
138 lines
5.4 KiB
Markdown
# Eve CultMesh Streaming UI Framework
|
|
|
|
Eve is the shared UI and sensor edge for the GameCult mesh. It is not only the
|
|
current jailbroken iPad app. The iPad client is the first native proof of a
|
|
larger framework: apps publish structured surfaces through CultNet, Eve clients
|
|
render those surfaces, and local devices publish timestamped sensor packets
|
|
back into the mesh.
|
|
|
|
## Authority Map
|
|
|
|
- Owner: each app owns its provider state and command side effects.
|
|
- Browser renderer: owns canonical layout behavior, accessibility semantics,
|
|
visual reference, and cross-client comparison tests.
|
|
- Native clients: own platform rendering, local input capture, media decode, and
|
|
sensor sampling for their device.
|
|
- CultNet API: owns transport of surface snapshots, command messages, and sensor
|
|
packets.
|
|
- CultMesh documents: own typed state shape, provenance, versioning, and replay.
|
|
|
|
Eve does not become the business-logic owner for every dashboard. It is the
|
|
composition and device edge. Providers decide truth; Eve makes that truth
|
|
visible and touchable.
|
|
|
|
## Surface Model
|
|
|
|
Every app-facing UI surface should be expressible as a retained compositing
|
|
document. The old flat `nodes` list is a compatibility/data projection; the
|
|
distributed UI layer is `surface.root`, a CultUI-shaped tree:
|
|
|
|
- `surface`: provider id, title, schema, version, updated timestamp.
|
|
- `surface.root`: panels, rails, stacks, grids, controls, cards, trees, media
|
|
views, avatars, text blocks, metric bars, graphs, and inspector panes.
|
|
- `nodes`: semantic state/selection/debug projection for clients that need a
|
|
graph-shaped view or older compatibility.
|
|
- `selection`: current focus and provider-owned state path.
|
|
- `commands`: allowed user intents such as select, open provider, move, scale,
|
|
rotate, toggle visibility, invoke action, edit value, and apply preset.
|
|
- `assets`: image URLs, local cache keys, font/material hints, and media stream
|
|
references.
|
|
- `detail`: structured text, state paths, diagnostics, and provenance.
|
|
|
|
The browser is the reference renderer for this model. Native renderers should
|
|
match the document semantics, then choose native controls where that gives the
|
|
device better latency, touch, media, or sensor access.
|
|
|
|
## Sensor Model
|
|
|
|
Eve clients publish timestamped sensor streams:
|
|
|
|
- camera frames or frame events;
|
|
- microphone blocks;
|
|
- pointer, touch, Pencil, keyboard, and gamepad input;
|
|
- accelerometer, gyroscope, magnetometer, and device pose;
|
|
- display timing, viewport, scale, and dropped-frame diagnostics.
|
|
|
|
Each packet needs:
|
|
|
|
- source id;
|
|
- device id;
|
|
- monotonic device timestamp;
|
|
- wall-clock observation timestamp when available;
|
|
- sequence number;
|
|
- format metadata;
|
|
- payload reference or inline payload;
|
|
- clock-domain hint.
|
|
|
|
Mimir and other consumers can then align signals without pretending the client
|
|
is a timing authority for the whole system.
|
|
|
|
## Client Targets
|
|
|
|
Browser:
|
|
|
|
- canonical Eve runtime;
|
|
- layout and behavior oracle;
|
|
- fast iteration surface for new provider types;
|
|
- visual regression target for iOS and Android.
|
|
|
|
iOS:
|
|
|
|
- native fullscreen renderer;
|
|
- multitouch/Pencil control surface;
|
|
- camera, mic, and motion sensor publisher;
|
|
- low-latency media display and dashboard controller.
|
|
|
|
Android:
|
|
|
|
- same provider/sensor contracts as iOS;
|
|
- camera, mic, motion, and display timing publisher;
|
|
- native rendering where browser embedding would be weaker.
|
|
|
|
Flutter:
|
|
|
|
- likely shared native-client implementation path for Android/iOS parity;
|
|
- useful for matching controls, layout primitives, animation, and local media
|
|
surfaces across native devices;
|
|
- not the browser ground truth. Flutter Web may be a compatibility target, but
|
|
the canonical behavior still belongs to the browser Eve runtime and recorded
|
|
CultMesh surface fixtures.
|
|
|
|
Fensalir Direct2D:
|
|
|
|
- desktop/native client surface inside the Fensalir runtime;
|
|
- lowers Eve/CultMesh surface documents into the existing DirectWrite/Direct2D
|
|
overlay path;
|
|
- uses the same provider ownership and command contract as browser, iOS, and
|
|
Android clients;
|
|
- keeps renderer mechanics in Fensalir while app-specific dashboard meaning
|
|
remains with the provider repo.
|
|
|
|
## Current Proof
|
|
|
|
The current iOS app consumes Mimir's `/eve/deck` WebSocket broker. That broker is
|
|
already shaped like the future API: provider manifests, retained dashboard
|
|
state, provider switching, commands, and typed node metadata. The VoidBot
|
|
provider proves that Eve can render an app-specific native cockpit from a
|
|
structured mesh snapshot: CTB rail, avatar images, selected Face/status pane,
|
|
state tree, and detail panel.
|
|
|
|
The Android proof under `android/` builds directly against the installed Android
|
|
SDK and runs on Periwinkle. It is deliberately small: display Eve's role, poll
|
|
the Mimir broker health endpoint, and show timestamped motion/touch samples.
|
|
It is a device-edge proof while the browser reference and Flutter/native shared
|
|
client are still being cut.
|
|
|
|
## Next Cut
|
|
|
|
1. Extract the shared surface schema from the iOS app and Mimir broker into a
|
|
versioned CultMesh contract.
|
|
2. Build the browser Eve runtime as the canonical renderer and comparison
|
|
target.
|
|
3. Add a replay harness that feeds recorded surface/sensor documents to browser
|
|
and native clients.
|
|
4. Replace JSON/base64 sensor payloads with binary packets while preserving the
|
|
typed envelope.
|
|
5. Add Android client scaffolding against the same provider and sensor API.
|
|
6. Add the Fensalir Direct2D lowering for the shared surface contract so engine
|
|
clients can show the same dashboards without embedding a browser.
|