README and contract: true of the CPU process tree, no retired runtime named
The front door drops its claim about the Raven instance and describes the encoder's explicit chat header and the daemon's ownership of the worker's process group. docs/verse-service-contract.md, which the front door names as an entry point, is rewritten for the CPU tree: witnesses, command boundary and worker boundary, with the request fields left to the README. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Meta Vens
parent
6651662b2c
commit
f8931270e2
2 changed files with 54 additions and 145 deletions
20
README.md
20
README.md
|
|
@ -13,7 +13,7 @@ supervises Vili.
|
|||
- **Problem:** Personas and games need generated motion, but cold per-request model loads cost minutes. One daemon owns job identity, worker lifecycle, the thread budget and artifact references, so callers touch no model state.
|
||||
- **Entry points:** `scripts/vili-daemon.mjs` (the daemon; `npm start`, `npm run health`, `npm test`), `scripts/kimodo-resident-worker.py` (the worker it spawns), `scripts/kimodo-text-encode.py` (the one-shot encoder child), `POST /motion/generate` (see Motion Generation), `docs/verse-service-contract.md` and `docs/motion-pipeline-target.md`.
|
||||
- **Live:** the process tree below, run by hand: `POST /motion/generate`, `GET /health`, `GET /operator-state`, and the CultCache service store. It produces `motion.npz` and optional BVH on the SOMA skeleton; there is no FBX export.
|
||||
- **Planned:** admission to Idunn on Yggdrasil with a locked virtual environment, then FBX export and a text-prompt-to-Unity-clip path with no hand steps (`docs/motion-pipeline-target.md`). The Raven instance still runs a retired branch until the admission cut tears it down.
|
||||
- **Planned:** admission to Idunn on Yggdrasil with a locked virtual environment, then FBX export and a text-prompt-to-Unity-clip path with no hand steps (`docs/motion-pipeline-target.md`).
|
||||
|
||||
## Authority
|
||||
|
||||
|
|
@ -34,7 +34,7 @@ One Linux CPU process tree:
|
|||
```text
|
||||
vili-daemon.mjs job intake, worker lifecycle, thread cap, nice 10
|
||||
kimodo-resident-worker.py resident Kimodo model, one generation at a time
|
||||
kimodo-text-encode.py one child per request with cache misses, assembly B on CPU
|
||||
kimodo-text-encode.py one child per request with cache misses, on CPU
|
||||
```
|
||||
|
||||
Setup: `git submodule update --init`; in `vendor/CultLib`, `npm ci --ignore-scripts` and
|
||||
|
|
@ -59,7 +59,7 @@ The daemon starts the worker on the first job, on a port the worker binds itself
|
|||
and reports in its ready line, and respawns it after an exit. The worker runs at
|
||||
nice 10 with `torch` limited to `--threads` (also `OMP_NUM_THREADS` and
|
||||
`MKL_NUM_THREADS`), so a shared host keeps its other services. Its log is
|
||||
`<state>/logs/worker.log`. SIGTERM stops the whole tree.
|
||||
`<state>/logs/worker.log`. The daemon owns the whole worker process group: it is killed when the worker exits for any reason, on SIGTERM or SIGINT, and at start-up if a crashed daemon left one (`<state>/worker.pid`). A service unit should also own the tree (systemd `KillMode=control-group`).
|
||||
|
||||
`POST /motion/generate` refuses any POST that carries an `Origin` or is not
|
||||
`application/json` with `403 cross-site`; no route answers a preflight.
|
||||
|
|
@ -143,16 +143,18 @@ worker answers Kimodo's encoder call from a CultCache store of float32 vectors,
|
|||
`<state>/embeddings/text-embeddings.cc` (`gamecult.vili.text_embedding.v0`, through
|
||||
the cultcache-py in `vendor/CultLib`). A vector's key is
|
||||
`sha256(encoderIdentity + newline + text)`. The identity names the three Hugging
|
||||
Face snapshot revisions, the encoder assembly, the device, dtype, the torch,
|
||||
Face snapshot revisions, the encoder assembly and prompt form, the device, dtype, the torch,
|
||||
transformers, peft and accelerate versions, the Kimodo commit and the sha256 of
|
||||
`kimodo-text-encode.py`, so a changed input misses instead of serving a stale
|
||||
vector. Thread count is provenance, recorded in the worker identity, not identity.
|
||||
|
||||
The encoder is assembly B: the Llama 3 base with the MNTP adapter merged, then the
|
||||
supervised adapter, in bfloat16 on CPU. Kimodo's own loader never applies the MNTP
|
||||
weights under the pinned transformers; the A/B comparison is in
|
||||
`docs/research/vili-on-yggdrasil-cpu-2026-10.md`, which also holds the measured
|
||||
timings. Kimodo normalizes the prompt before encoding (`A person shrugs sadly.`),
|
||||
The encoder is the Llama 3 base with the MNTP adapter merged, then the supervised
|
||||
adapter, in bfloat16 on CPU, and `kimodo-text-encode.py` wraps every prompt in
|
||||
Llama 3 Instruct's chat header itself (`text_encoding.chat_header`, pooled after
|
||||
LLM2Vec's query marker), as Kimodo's LLM2Vec does when loaded from the repo ids. The identity names that prompt form.
|
||||
A fresh child vector matches Kimodo's own encoder from the repo ids
|
||||
(`test_text_encoding.ReferenceVector`, run on Yggdrasil). The measured timings are
|
||||
in `docs/research/vili-on-yggdrasil-cpu-2026-10.md`. Kimodo normalizes the prompt before encoding (`A person shrugs sadly.`),
|
||||
and the cache keys on that text. A request's misses go to one transient child that
|
||||
loads the encoder, encodes, writes its answer and exits, so the OS reclaims its
|
||||
RAM; hits never touch it. Each job carries the encoder record (hits, misses,
|
||||
|
|
|
|||
|
|
@ -2,43 +2,29 @@
|
|||
|
||||
## Objective
|
||||
|
||||
Vili turns Persona performance intent into animation work. The near-term body is
|
||||
Kimodo on Raven; the durable contract is a service surface that can accept a
|
||||
gesture/action prompt, run or queue motion synthesis, and publish inspectable
|
||||
state for Odin and Idunn.
|
||||
|
||||
Vili is an animation service, not a per-request CLI wrapper. Its Kimodo backend
|
||||
is a resident worker container that owns model lifetime for generation requests.
|
||||
Vili turns Persona performance intent into animation work. It is an animation
|
||||
service, not a per-request CLI wrapper: one daemon accepts a gesture or action
|
||||
prompt, owns a resident Kimodo worker on CPU, and publishes inspectable state
|
||||
for Odin and Idunn. The request fields and the runtime flags are in `README.md`
|
||||
(Motion Generation, Local Runtime); this page holds the contract around them.
|
||||
|
||||
## CultMesh Verses
|
||||
|
||||
- `vili.service`: service identity, provider advertisement, health, and runtime.
|
||||
- `vili.service`: service identity, health, and runtime.
|
||||
- `vili.animation`: animation jobs, prompts, backend choices, and artifact refs.
|
||||
- `vili.operator`: daemon status, backend readiness, and Eve/CultUI surface.
|
||||
- `vili.operator`: daemon status and worker readiness, as typed state. Operator
|
||||
surfaces wait for Thing 2.
|
||||
|
||||
## Witnesses
|
||||
|
||||
- `.vili/vili.service.cc`: Vili-owned CultCache store for provider
|
||||
advertisement, operator state, Eve surface, command boundary, and transport
|
||||
profile records.
|
||||
- `.vili/provider-advertisement.json`: compatibility export derived from the
|
||||
provider advertisement record.
|
||||
- `.vili/operator-state.json`: compatibility export derived from daemon health
|
||||
and backend readiness.
|
||||
- `.vili/eve-operator-surface.json`: compatibility export of the current
|
||||
operator surface.
|
||||
- `.vili/command-boundary.json`: compatibility export of Vili command
|
||||
authority.
|
||||
- `.vili/transport-profile.json`: compatibility export of Vili transport
|
||||
state.
|
||||
- `.vili/vili.service.cc`: the daemon-owned CultCache store for operator state,
|
||||
the command boundary and the transport profile.
|
||||
- `.vili/jobs/{jobId}.json`: accepted animation job records.
|
||||
- `.vili/artifacts/{jobId}/`: generated motion outputs when generation succeeds.
|
||||
- `vili-kimodo-worker`: resident Docker container exposing the loaded Kimodo
|
||||
model to Vili inside Raven's `Ubuntu-24.04` WSL runtime.
|
||||
|
||||
JSON is a compatibility witness. Vili's service/operator/command/transport
|
||||
truth is the daemon-owned `.cc` store; animation job schemas may still be
|
||||
promoted from JSON once that request/receipt shape stabilizes.
|
||||
- `.vili/embeddings/text-embeddings.cc`: the worker's text-embedding cache.
|
||||
- `.vili/worker.pid`: the process group of the running worker, so a restarted
|
||||
daemon can find and kill a worker its crashed predecessor left behind.
|
||||
- `.vili/logs/worker.log`: the worker's stdout and stderr.
|
||||
|
||||
## Command Boundary
|
||||
|
||||
|
|
@ -47,112 +33,33 @@ supported Kimodo generation controls to the resident worker. Vili owns job
|
|||
identity, request persistence, worker lifecycle, timeout handling, and the
|
||||
artifact stem. Kimodo owns synthesis and model-specific export.
|
||||
|
||||
```json
|
||||
{
|
||||
"prompt": "a person gives a small friendly wave",
|
||||
"duration": 1,
|
||||
"diffusionSteps": 20,
|
||||
"seed": 123,
|
||||
"bvh": true
|
||||
}
|
||||
```
|
||||
|
||||
Kimodo metadata can be sent directly when callers need multi-segment or
|
||||
multi-sample generation:
|
||||
|
||||
```json
|
||||
{
|
||||
"meta": {
|
||||
"texts": [
|
||||
"a person gives a small friendly wave",
|
||||
"a person takes one careful step"
|
||||
],
|
||||
"durations": [0.5, 0.5],
|
||||
"num_samples": 1,
|
||||
"diffusion_steps": 20
|
||||
},
|
||||
"model": "Kimodo-SOMA-RP-v1.1",
|
||||
"numTransitionFrames": 3,
|
||||
"cfgType": "regular",
|
||||
"cfgWeight": [1.5],
|
||||
"seed": 6101,
|
||||
"bvh": true,
|
||||
"bvhStandardTpose": true,
|
||||
"saveExampleDir": true
|
||||
}
|
||||
```
|
||||
|
||||
Request fields accepted by Vili:
|
||||
|
||||
- `prompt` / `utterance` plus `duration` / `durationSeconds`
|
||||
- `texts` plus `durations`, or a full `meta` object
|
||||
- `inputFolder` / `input_folder` for a container-visible Kimodo input folder
|
||||
containing `meta.json` and optional `constraints.json`
|
||||
- `model`; the daemon keeps one resident model loaded, so a model change
|
||||
restarts `vili-kimodo-worker`
|
||||
- `diffusionSteps` / `diffusion_steps`
|
||||
- `numSamples` / `num_samples`
|
||||
- `numTransitionFrames` / `num_transition_frames`
|
||||
- `constraints` as an object/list, or `constraints`, `constraintsPath`, or
|
||||
`constraints_path` as a container-visible path
|
||||
- `seed`
|
||||
- `cfg`, `cfgType` / `cfg_type`, `cfgWeight` / `cfg_weight`
|
||||
- `bvh`, `bvhStandardTpose` / `bvh_standard_tpose`
|
||||
- `noPostprocess` / `no_postprocess`
|
||||
- `saveExampleDir` / `save_example_dir`
|
||||
- `timeoutMs`, `workerTimeoutMs`
|
||||
|
||||
Vili does not accept arbitrary `output` paths from callers. Every generation
|
||||
writes beneath `.vili/artifacts/{jobId}/`, with the worker receiving
|
||||
`/outputs/{jobId}/motion`. The job record is the authority for artifact refs.
|
||||
Model-specific exports are preserved: SOMA writes NPZ and optional BVH, SMPLX
|
||||
also writes AMASS NPZ, and G1 also writes CSV.
|
||||
writes beneath `.vili/artifacts/{jobId}/`, and the job record is the authority
|
||||
for artifact refs. SOMA writes NPZ and optional BVH, SMPLX also writes AMASS NPZ,
|
||||
and G1 also writes CSV.
|
||||
|
||||
The daemon may reject or defer requests when:
|
||||
A POST that carries an `Origin` header or is not `application/json` is refused
|
||||
`403 cross-site` before its body is read.
|
||||
|
||||
- Kimodo image is unavailable.
|
||||
- Docker/GPU backend is unhealthy.
|
||||
- Hugging Face token is absent and the selected backend requires gated model
|
||||
access.
|
||||
- The resident Kimodo worker cannot start or does not become healthy before the
|
||||
request timeout.
|
||||
- The request omits concrete prompt, metadata, batch text, or input-folder
|
||||
intent.
|
||||
- Batch `texts` are provided without matching `durations`.
|
||||
The daemon refuses or fails a request when:
|
||||
|
||||
## Credential State
|
||||
- the models root, Kimodo commit or Python environment id is not configured
|
||||
(`503 worker-unconfigured`, before anything spawns);
|
||||
- the worker cannot start or does not become healthy in time (a `502` job);
|
||||
- the request omits concrete prompt, metadata, batch text, or input-folder
|
||||
intent;
|
||||
- batch `texts` are provided without matching `durations`.
|
||||
|
||||
Raven has a Hugging Face token installed in the `Ubuntu-24.04` WSL cache, and
|
||||
Vili mounts it into Kimodo containers read-only. Verified on `2026-06-12`: a
|
||||
Kimodo container launched by Raven could read gated metadata for
|
||||
`meta-llama/Meta-Llama-3-8B-Instruct`.
|
||||
## Worker Boundary
|
||||
|
||||
## Kimodo Worker Boundary
|
||||
The daemon spawns the resident worker as a child on the first job, in its own
|
||||
process group, at nice 10, with a closed environment and the thread budget
|
||||
`--threads`. The worker binds its own port and reports it in a ready line. The
|
||||
daemon owns the group: it kills the whole group when the worker exits for any
|
||||
reason (so no encoder child survives), on SIGTERM or SIGINT, and at start-up for
|
||||
a group left by a crashed daemon. Under a service manager, the unit should also
|
||||
own the tree from outside (systemd `KillMode=control-group`).
|
||||
|
||||
The resident worker is started by Vili as Docker container `vili-kimodo-worker`.
|
||||
Because Raven's Docker Engine lives inside WSL, Vili also starts a minimal WSL
|
||||
keepalive process recorded at `/tmp/vili-wsl-keepalive.pid`; it owns no service
|
||||
truth and exists only to keep the Docker substrate alive while the resident
|
||||
worker holds the model.
|
||||
|
||||
It mounts:
|
||||
|
||||
- `/root/.cache/huggingface` for model/token access.
|
||||
- `E:\Projects\Vili\scripts\kimodo-resident-worker.py` as the worker program.
|
||||
- `E:\Projects\Vili\.vili\artifacts` as `/outputs` for generated artifacts.
|
||||
|
||||
Inside the worker, Kimodo owns synthesis. Vili owns worker lifecycle,
|
||||
request/response persistence, timeout handling, and artifact references. Direct
|
||||
user requests, programmatic requests, and smoke-started warmup all use the same
|
||||
resident worker health and generation path.
|
||||
|
||||
Observed on `2026-06-12`: first worker model load took about 100 seconds. Once
|
||||
resident, five 1 second / 20 step service-path generations averaged about
|
||||
54.4 seconds per clip, or about `0.0184x` realtime.
|
||||
|
||||
## First Runtime Invariant
|
||||
|
||||
Health checks must be cheap. They may verify process, port, backend metadata,
|
||||
and cached readiness, but they must not launch a Kimodo generation job or start
|
||||
model load. Smoke checks may start the resident worker because their purpose is
|
||||
backend readiness, not cheap liveness.
|
||||
Inside the worker, Kimodo owns synthesis and a one-shot encoder child owns
|
||||
text encoding on a cache miss. Health checks must be cheap: `/health` reads the
|
||||
daemon's own worker state and never starts a model load or a generation.
|
||||
|
|
|
|||
Loading…
Reference in a new issue