Compare commits

...
Sign in to create a new pull request.
Author SHA1 Message Date
Meta Vens
3adb616497 Docs and launch scripts for the smoke deletion and the listen addresses
package.json loses the smoke script; the README and the service contract say no GET changes anything and name the listen addresses and the missing CORS; the launch and install scripts default -HostName to 127.0.0.1,10.77.0.4 instead of 0.0.0.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 03:09:22 +02:00
Meta Vens
f71aeedad1 Pin that a client hang-up mid-job leaves the slot held
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 03:09:06 +02:00
Meta Vens
9e79b00ce1 Drop access-control-allow-origin: * from every response
No browser caller exists, so no page needs to read the daemon. The wildcard let any page read /gpu-permission and /operator-state.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 03:09:05 +02:00
Meta Vens
3fa19d7590 Delete GET /smoke, make GET read-only, listen on loopback and WireGuard only
GET /smoke ran docker run --gpus all and started the resident worker outside the gate, reachable by any page or LAN host while the household's switch was off. Its only callers were scripts/smoke-vili.mjs and the Eve Smoke button, so the route, the script, the button and the advertised endpoints go. refreshBackendStatus lost its includeKimodoHelp GPU branch and every service docker start: a GET now only asks. The worker starts only inside a job that holds the slot.

--host takes the addresses to listen on, one server each (Raven: 127.0.0.1,10.77.0.4); a wildcard refuses to start. Callers are the tray and health (loopback) and WireGuard peers (Starfire, services).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 03:09:05 +02:00
Meta Vens
a3052ddf63 Contract: the POST routes refuse cross-site requests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 02:30:07 +02:00
Meta Vens
e37a2ad3ec Pin the daemon's slot hold at the HTTP edge with a stubbed slow worker
A real daemon on stub wsl.exe and nvidia-smi.exe runs one job held open by the test; requests during
it are 503 source slot, the slot frees at the job's end, and the next job is admitted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 02:29:49 +02:00
Meta Vens
cb36beb09a Refuse cross-site POSTs to the state-changing routes and grant no CORS preflight
POST /gpu-permission and POST /motion/generate now require Content-Type application/json and
refuse a request whose Origin is present and not the daemon's own loopback origin, 403 cross-site,
before the body is read. The blanket OPTIONS handler that approved any preflight is deleted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 02:29:43 +02:00
Meta Vens
5a60255ac2 Make the slot's release idempotent so a stale second release cannot free another job's slot
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 02:26:04 +02:00
Meta Vens
455cedb3c3 Mutation triage: drop a redundant null guard in parseFreeMiB
String(null) and String(undefined) already fail the integer test, so the
?? "" guard decided nothing; the Stryker mutant of it survived.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 01:49:01 +02:00
Meta Vens
375c2faf4e Add the household's tray toggle and its logon task
vili-gpu-tray.ps1 is one NotifyIcon in the interactive session: it lowers
GET /gpu-permission to an icon and tooltip and POSTs an absolute value on click.
It holds no state, reads no process list and writes no file.
install-vili-tray-task.ps1 registers only GameCult\ViliTray at logon.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 01:45:44 +02:00
Meta Vens
736a1a7ab8 Build the GPU permission: the household's yes or no, owned by the daemon
gpu-tenancy.mjs judges permission, then the one slot, then one host free-VRAM
read per admission. The permission is a gamecult.vili.gpu_permission.v1
document in vili.service.cc, set only from loopback at POST /gpu-permission;
absent means not allowed. The daemon holds one CultCache instance. Tests pin
the gate with fake ports and the daemon edge over HTTP, including the slot
wiring through acquire() (finding s2 slot-wiring-unpinned).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 01:45:08 +02:00
Meta Vens
ed07c5a4af Delete the GPU detection gate: probe child, supervisor, thresholds and their tests
The household's use is no longer detected. gpu-probe.mjs, the PowerShell probe
child, the hysteresis/3D/full-screen/own-VRAM rules, three env variables and the
README paragraph go; the gate keeps only the single slot and the placement
refusal until the permission lands in the next commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 01:42:33 +02:00
Meta Vens
b489d2885d Map: GPU gate becomes a household tray toggle (r2 rationale and probes)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 01:40:54 +02:00
Meta Vens
a2b9983183 Mutation triage: drop two redundant guards in the probe, pin baseline, double release and env messages
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 00:29:13 +02:00
Meta Vens
bacc3faaa0 Tests: probe respawn tests keep the watchdog out of the way; watchdog test timing
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 00:20:29 +02:00
Meta Vens
4ac85a65a5 Tests: fix a broken string literal in the probe watchdog test
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 00:19:58 +02:00
Meta Vens
5050de58b5 Gate: one slot judged where the GPU is taken, own VRAM discounted, truthful placement body, env refusal
acquire() judges at the slot: household signals first (admission), then a held slot (slot);
nothing queues, so a request can no longer wait on the worker lock past a changed gate. While
Vili's job holds the slot, and until free VRAM recovers to its pre-job level (capped), low free
VRAM is not the household's; full-screen and 3D always count. A sample with no resolved 3D
counters is a probe fault and the gate fails closed. The placement 503 carries no gate status.
A malformed VILI_GPU_* override refuses to start, naming the variable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 00:19:08 +02:00
Meta Vens
9ec048844b Probe: own module, 3D counter path as a tested constant, engines count, hung-child watchdog
The probe's sample plumbing leaves the gate module (the gate never kills; the probe may kill
its own hung child). The counter path is one exported constant passed to the script as
-Util3dPath, the script reports how many 3D engine counters resolved, and a child that is
alive but silent for 3 intervals is killed and respawned.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 00:19:07 +02:00
Meta Vens
62485dab15 Probe: read nvidia-smi's exit code after the pipeline drains
Select-Object -First 1 closes the pipe early and leaves LASTEXITCODE -1, so
every iteration threw and the probe wrote no samples (found on Raven).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 23:32:09 +02:00
Meta Vens
d921731a11 Tests: kill the mutants that were real (threshold edges, stale since, probe argv and respawn timing)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 23:27:32 +02:00
Meta Vens
d9d396748d Tests: the clear hold counts from the first clear sample
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 23:24:20 +02:00
Meta Vens
2b5b900eca Tests: free the gate through a real clear hold before asserting transitions
The hold is measured between consecutive samples; a 180 s gap is a stale
probe, not a hold.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 23:22:08 +02:00
Meta Vens
a9a449225c Docs: GPU tenancy paragraph, placement wording, 503 gpu-busy contract
Drops the claim that the child's 2 GiB check refuses under contention
(closes gpu-busy-blind's README overstatement).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 23:13:50 +02:00
Meta Vens
880f1c2dcf Worker: error body carries the typed code (closes gpu-busy-untyped-http)
worker_error_body is the worker's one error wire shape; GpuBusy.code
reaches the daemon as a field. GpuBusy is documented as placement only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 23:13:50 +02:00
Meta Vens
38491f8917 Daemon: admit /motion/generate through the tenancy gate, typed 503 gpu-busy
Refuse a new job with 503 + retry-after 60 while the GPU is in use (source
admission); a placement refusal from the encoder child also answers 503
(source placement). Publish the gate as operator_state.gpu. A running job
is never touched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 23:13:49 +02:00
Meta Vens
54e680f284 GPU tenancy gate: hysteresis gate, probe child and tests
One owner for "may heavy GPU work start": numbers-only samples (full-screen
state, sustained 3D use, host free VRAM) in, { inUse, since, checkedAt,
probeOk } out; fail closed at start and when the probe goes stale.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 23:13:49 +02:00
18 changed files with 1139 additions and 184 deletions

View file

@ -10,8 +10,8 @@ first backend is Kimodo on Raven, running through Docker Engine inside
- **Value:** a warm, supervised path from a text prompt to generated body motion. Vili runs Kimodo as a resident GPU worker on Raven and returns motion artifacts (`motion.npz`, optional BVH) for a job.
- **Problem:** Personas and games need generated motion, but cold per-request container runs took about 195 seconds per 1 second clip, versus about 54 warm. One daemon owns job identity, worker lifecycle and artifact references so callers do not touch Docker, WSL or model state.
- **Entry points:** `scripts/vili-daemon.mjs` (the daemon; `npm start`, `npm run health`, `npm run smoke`, `npm run provider`), `scripts/kimodo-resident-worker.py` (the resident worker), `POST /motion/generate` (see Motion Generation), `docs/verse-service-contract.md` and `docs/motion-pipeline-target.md`.
- **Live:** the resident Kimodo worker on Raven, `POST /motion/generate`, `GET /smoke`, and the CultCache service store. It runs under a Raven scheduled task, not under Idunn. It has `motion.npz` and optional BVH output on the SOMA skeleton; there is no FBX export.
- **Entry points:** `scripts/vili-daemon.mjs` (the daemon; `npm start`, `npm run health`, `npm run provider`), `scripts/kimodo-resident-worker.py` (the resident worker), `POST /motion/generate` (see Motion Generation), `docs/verse-service-contract.md` and `docs/motion-pipeline-target.md`.
- **Live:** the resident Kimodo worker on Raven, `POST /motion/generate`, and the CultCache service store. It runs under a Raven scheduled task, not under Idunn. It has `motion.npz` and optional BVH output on the SOMA skeleton; there is no FBX export.
- **Planned (target, 2026-10-20):** FBX animation export on the SOMA skeleton and a text-prompt-to-Unity-clip path with no hand steps, with each artifact recording prompt, constraints, seed and model. Retargeting stays with the consuming engine; see `docs/motion-pipeline-target.md`.
## Authority
@ -41,6 +41,11 @@ Default compatibility endpoint:
http://10.77.0.4:8824
```
The daemon listens only on the addresses `--host` names, comma-separated, one server each: on
Raven `127.0.0.1` (the tray and `--health`) and `10.77.0.4` (the WireGuard peers: Starfire and
the services that call `POST /motion/generate`). A wildcard such as `0.0.0.0` refuses to start.
No response carries a CORS header, so a web page cannot read the daemon either; no browser
caller exists.
Vili is not an Idunn target today. Its lifecycle is the `GameCult\Vili`
scheduled task on Raven, and `GET /health` is an operator lowering. Admission
to Idunn is motion-pipeline:question:vili-under-idunn.
@ -63,7 +68,6 @@ Important routes:
- `GET /eve/operator`
- `GET /eve/deck/providers`
- `WS /eve/deck/vili.animation`
- `GET /smoke`
- `POST /motion/generate`
## Motion Generation
@ -145,8 +149,8 @@ Vili keeps Kimodo warm through a resident worker container named
the loaded model. Because Raven's Docker Engine runs inside WSL, Vili also
keeps a tiny WSL sleep-loop alive while the worker is resident so the Docker
substrate does not disappear underneath the model. `GET /health` observes
worker state without launching model load. `GET /smoke` verifies the backend
and starts the resident worker when it is absent.
worker state without launching model load and without changing anything. Only a
`POST /motion/generate` that holds the GPU slot starts the worker.
Kimodo's 8B text encoder (LLM2Vec, Llama 3) never lives in the worker. The
worker answers Kimodo's encoder call from a CultCache store of float32 vectors,
@ -159,12 +163,25 @@ Kimodo normalizes the prompt before encoding (`A person shrugs sadly.`), and the
cache keys on that text. A request's misses go to one transient child,
`scripts/kimodo-text-encode.py`: it loads the encoder in bf16 on the GPU, with the
layers that do not fit in free VRAM held in RAM, encodes, writes its answer and
exits, so the OS reclaims its RAM and VRAM. It refuses with a typed `gpu-busy`
below 2 GiB of free VRAM and has no CPU fallback; hits never touch it. Each
exits, so the OS reclaims its RAM and VRAM. It sizes its own placement from the free VRAM torch
can see and refuses a placement below 2 GiB with a typed `gpu-busy` (source `placement`);
it has no CPU fallback, and hits never touch it. Each
`/motion/generate` job carries the encoder record (hits, misses, seconds, vector
hashes) and the worker identity. `GET /health` shows the identity and the stored
count.
GPU permission. Raven's GPU is the household's first. A tray icon in the household's session
(`scripts/vili-gpu-tray.ps1`, logon task `GameCult\ViliTray`, installed by
`scripts/install-vili-tray-task.ps1`) shows whether Vili may use the GPU and flips it with one click.
The daemon owns the yes or no in `vili.service.cc` (`gamecult.vili.gpu_permission.v1`): off on a
fresh install, set only from loopback (`POST /gpu-permission { allowed }`, read at
`GET /gpu-permission`). A `POST /motion/generate` is refused with HTTP 503, `retry-after: 60` and
`{ ok: false, code: "gpu-busy", source }` where `source` is `household` (the permission is off),
`slot` (another job holds the one GPU slot) or `vram` (host `nvidia-smi.exe` free VRAM is below
`VILI_GPU_MIN_FREE_MIB`, default 10240, or unreadable); nothing queues, and the consumer owns the
retry. A job already running finishes; the gate never stops work. `operator_state.gpu` is
`{ allowed, changedAt, jobRunning, lastRefusal }`. `npm test` runs the gate's tests.
Measured on Raven on `2026-10-22`, for a 2 second clip at 20 diffusion steps:
the worker is healthy 29 seconds after `docker rm` (was 179) and rests at 2.3 GiB
resident (was 10.4 GiB, with 5.8 GB swapped). A cache miss takes 59 to 103

View file

@ -7,8 +7,33 @@ cuts not yet mapped. The target's rationale is `docs/motion-pipeline-target.md`.
## Body facts
### Raven tray-toggle probes, 2026-10-22 23:30 to 23:36 UTC
Run read-only from Starfire over `ssh raven` (session 0, elevated), one
PowerShell 5.1 script copied to `C:\Users\Public` and deleted in the same
command. Nothing on Raven was changed: no task edits, no restarts, no GPU jobs.
They inform cut spec `motion-pipeline:cut_spec:cut-raven-gpu-tenancy-gate.r2`,
which replaces detection with the household's toggle
(`motion-pipeline:ruling:raven-gpu-tray-toggle`).
| Probe | Result |
|---|---|
| Who the household is on Raven | One account. `query user`: `madman's lullaby`, console, session 1, Active since 2026-10-18. The `\GameCult\Vili` task runs as that same user (`LogonType Interactive`, `RunLevel Limited`), so a tray process at that user's logon is in the same session and the same user as the daemon, with no admin rights. |
| Daemon listen address | `node.exe ... vili-daemon.mjs --host 0.0.0.0 --port 8824 --state-root E:\Projects\Vili\.vili`, pid 33776, session 1, at detached head `a2b9983`. Listens on `0.0.0.0:8824`, so the WireGuard peers can reach every route; a write route must check the caller itself. `GET http://127.0.0.1:8824/operator-state` from Raven answers in 3 ms. |
| Live task action versus the installer | The live `\GameCult\Vili` action passes `-IdunnRudpHealth 10.77.0.2:17870 -IdunnDaemon vili -IdunnHealthContract ... -IdunnHealthIntervalSeconds 15` to `start-vili-daemon.ps1`; `scripts/install-vili-task.ps1` does not. Rerunning that installer would rewrite the live task, so the tray gets its own installer. |
| Tray substrate | `powershell.exe` 5.1.26100, apartment `STA` by default; `System.Windows.Forms` and `System.Drawing` load (`v4.0.30319`, .NET Framework release 534296). Execution policy is Undefined at every scope (the task launcher passes `-ExecutionPolicy Bypass`). `scripts/run-hidden-powershell.vbs` already starts a hidden PowerShell with no console flash. No new dependency is needed. |
| Tray overflow | `HKCU:\Control Panel\NotifyIconSettings` exists with 47 entries: Windows 11 records each notify icon there and puts new ones in the overflow until the user promotes it. A fresh tray icon will not be on the taskbar until the household drags it there once (or turns it on under Settings, Personalization, Taskbar, Other system tray icons). |
| Startup folder | `C:\Users\Madman's Lullaby\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup`. Not used: from SSH (session 0) nothing in it can be started in session 1, while `Start-ScheduledTask` on an Interactive task can. |
| Scheduled tasks under `\GameCult\` | `Backup-Yggdrasil-Pull`, `ComfyUI`, `Vili`, all Ready. ComfyUI is a second possible heavy GPU consumer on Raven; the toggle governs Vili only. |
| Host free VRAM | `C:\WINDOWS\system32\nvidia-smi.exe --query-gpu=memory.free --format=csv,noheader,nounits`: 159 ms from a cold PowerShell, exit 0, free 5001 MiB at 23:3x UTC (the household or another consumer held about 11 GiB). No `nvidia-smi.exe` process was left running. |
| Vili's store | `E:\Projects\Vili\.vili\vili.service.cc`, 11810 bytes. Newest job file `vili-1792710792764.json`, written 23:20:12 UTC. The live gate (`a2b9983`) read `inUse true, probeOk true`. |
| cultcache-ts on Raven | `E:\Projects\CultLib` at `b9497d35`, package `0.14.1`; its build carries `pullAllBackingStores`. `SingleFileMessagePackBackingStore.push` reads the file, replaces one `(type, key)` entry and writes all, inside a write queue that belongs to one store instance. |
### Raven GPU tenancy probes, 2026-10-22 20:15 to 20:23 UTC
These informed r1's detection gate, which r2 deletes. The session, principal and
`nvidia-smi` rows still hold.
Run read-only from Starfire over `ssh -o HostName=192.168.178.165 raven`
(alias in `~/.ssh/config`; LAN address per `gamecult-ops/inventory.md`), as
`powershell -NoProfile -NonInteractive -EncodedCommand`. Nothing on Raven was
@ -21,56 +46,83 @@ category (compositor, WSL family, or "other").
| Session of the Vili daemon | `node.exe ... vili-daemon.mjs --host 0.0.0.0 --port 8824` runs in **session 1**, the console session of `madman's lullaby` (`query user`: console, ID 1, Active). It is the interactive desktop session, not session 0. |
| Vili task principal | `\GameCult\Vili`: `LogonType Interactive`, `RunLevel Limited`, boot trigger (`MSFT_TaskBootTrigger`), last run 2026-10-22 21:18:55 local, result 0. Children the daemon spawns inherit session 1. |
| SSH context | Windows `sshd` runs commands in **session 0**, elevated (`IsInRole(Administrator)` true). `gamecult-ops/runbooks/idunn-host-raven.md` already records that sshd cannot start anything in the user's session without a scheduled-task hop. |
| `SHQueryUserNotificationState` from session 0 | `hr=0`, state `2` (`QUNS_BUSY`) on every call, with the GPU idle (3D engine total 0.01 to 0.02 %). Session 0 has no shell, so this value says nothing about the user's desktop. The call must run in session 1, which is where the daemon and its children are. It was **not** observed from session 1: doing that without a task edit is impossible from SSH. |
| `nvidia-smi.exe` | Present at `C:\WINDOWS\system32\nvidia-smi.exe`. `--query-gpu=memory.total,memory.used,memory.free,utilization.gpu`: `16380, 1592, 14516, 2` MiB/%. 63 ms cold, about 49 ms per call (10 calls 491 ms). Per-process `--query-compute-apps` returns `[N/A]` memory under WDDM. |
| `\GPU Engine(*engtype_3D)\Utilization Percentage` | Readable from session 0 as admin; 115 instances. `Get-Counter -SampleInterval 1 -MaxSamples 1` takes 1.08 s wall, 16 ms CPU warm, 328 ms CPU on first use in a process. Summed with `Measure-Object` it needs no instance names. Not tried as a Limited token in session 1. |
| `\GPU Adapter Memory(*)\Dedicated Usage` | Readable; 1.0 s wall; 1597 MiB on the RTX 4060 Ti, 0 on the Parsec virtual adapter. |
| `\GPU Process Memory(*)\Dedicated Usage` | Readable; 1.0 s wall. Two instances above 50 MiB: the desktop compositor (1643 MiB, session 1) and one session-0 process outside the WSL family (1405 MiB). Per-process figures overlap: they sum above the adapter total. |
| WSL attribution | The Kimodo worker container was resident (`/health` ok, uptime 3454 s, 6 generations) while `vmmemWSL` (and `vmwp`) had **no** `GPU Process Memory` instance. Host counters do not attribute WSL CUDA memory to a nameable host process. WSL `nvidia-smi` reported the same `1592 MiB used, 14516 MiB free` as the host. |
| PowerShell cost | `powershell.exe` 5.1 cold start 266 ms CPU, `Add-Type` for the shell32 call 78 ms CPU, first counter read 328 ms CPU: about 0.7 s CPU per cold spawn. A warm loop iteration (QUNS + `nvidia-smi` + one 3D counter read) is about 1.1 s wall and 16 ms of its own CPU. |
| `SHQueryUserNotificationState` from session 0 | `hr=0`, state `2` (`QUNS_BUSY`) on every call, with the GPU idle. Session 0 has no shell, so this value says nothing about the user's desktop. |
| `nvidia-smi.exe` | Present at `C:\WINDOWS\system32\nvidia-smi.exe`. `--query-gpu=memory.total,memory.used,memory.free,utilization.gpu`: `16380, 1592, 14516, 2` MiB/%. 63 ms cold, about 49 ms per call. Per-process `--query-compute-apps` returns `[N/A]` memory under WDDM. |
| `\GPU Engine(*engtype_3D)\Utilization Percentage` | Readable from session 0 as admin; 115 instances; `Get-Counter` 1.08 s wall. |
| `\GPU Process Memory(*)\Dedicated Usage` | Per-process figures overlap and sum above the adapter total. |
| WSL attribution | Host counters do not attribute WSL CUDA memory to a nameable host process. Host `nvidia-smi.exe` does see a WSL container's allocation: cut report h1 measured free falling 14292 to 1866 MiB under a 12 GiB holder. |
| Adapters | `NVIDIA GeForce RTX 4060 Ti` driver 32.0.16.1088, 16 GiB; `Parsec Virtual Display Adapter`. |
What follows from these, and what does not:
### What the detection gate taught (r1, h1, h2, s1, s2)
- The daemon's own session sees the user's desktop, so the tenancy probe runs as
a child of the daemon. No fork about where it runs: the scheduled task already
puts it in the right session.
- Free VRAM must come from the host (`nvidia-smi.exe`), not from torch in WSL:
Soul's s1 probe on cut `vili-encoder-placement` showed `mem_get_info` inside a
container blind to another container's 12 GiB. Whether host `nvidia-smi` sees
a WSL container's allocation is **unverified** (the resident worker's share
could not be separated from the compositor's); the gate cut checks it first.
- The 3D-engine signal cannot be used to tell Vili's own work from the
household's by process, and need not be: Kimodo runs on CUDA (compute
engines), games on the 3D engine. Summing only `engtype_3D` leaves Vili out.
- A long-lived probe child costs about 1 ms of CPU per second at a 15 s period;
a fresh PowerShell per sample would cost about 45 ms per second. Neither
loads the GPU.
- Host free VRAM cannot tell Vili's encoder child from a household load: the
child sizes itself to the VRAM that was free (s1 `self-tenancy-refusal`).
Discounting Vili's own VRAM during a job blinds the gate to a household load
that starts mid-job (s2 `hold-blinds-household-vram`).
- Live, the probe child went silent during 2 of 5 of Vili's own encoder jobs and
the gate stayed stale after the job (s2 `own-load-starves-probe`; Raven had
770 MB of host RAM available at one stall). Every repair added a watchdog, a
drain or a baseline, and each brought its own finding.
- After a job ends, host free VRAM is back at 14.6 to 15.2 GiB at the next
2-second sample (s2), so one `nvidia-smi` read before a job, with no job
holding the slot, sees the card as the household left it.
## Model page
| Kind | Named by | Over time | Decided by |
|---|---|---|---|
| GPU tenancy state (`gpu` in `gamecult.vili.operator_state.v1`) | the daemon, one per host | rewritten every 5 s with the operator state; holds `inUse`, `since`, `checkedAt`, `probeOk` and nothing else | the daemon's tenancy gate, from host samples; never persisted as truth, rebuilt from samples at start |
| GPU permission (`gamecult.vili.gpu_permission.v1` in `.vili/vili.service.cc`, key `raven`) | the daemon, one per host | written only when the household changes it, before the change is acknowledged; read once at daemon start; absent means not allowed | the household, through the tray, through the daemon's loopback-only `POST /gpu-permission`; the daemon is its one writer |
| GPU status (`gpu` in `gamecult.vili.operator_state.v1`) | the daemon | rewritten every 5 s with the operator state; `{ allowed, changedAt, jobRunning, lastRefusal }` | derived from the permission, the slot and the last refusal; display only, never read back |
| Motion job (`.vili/jobs/<jobId>.json`) | `vili-<epoch ms>` | written once when a job ends | the daemon; a job refused at admission writes none |
| Text embedding (`.vili/embeddings/text-embeddings.cc`) | `sha256(identity, text)` | append on miss | the worker's cache |
## Rationale
- **Refuse, do not queue (recommended, pending the operator).** `/motion/generate`
is synchronous today. Holding a request open for as long as a game runs, or
adding a persisted job queue, are bigger machines than the one consumer needs:
RPS clip generation is a script an agent runs, and it can retry a typed
`gpu-busy`. A queue belongs with the authoring UX's takes, when that target
exists (`motion-pipeline:follow_up:vili-authoring-ux-target-2`).
- **Fail closed.** If the probe child dies or stops answering, the gate reports
in use. A wrongly deferred clip costs minutes; a wrongly admitted encoder child
pages a game's VRAM and thrashes Raven.
- **The household decides; nothing detects.** The operator replaced detection
with a tray toggle (`motion-pipeline:ruling:raven-gpu-tray-toggle`). The
detection gate's findings were the cost of guessing intent from counters; a
yes from the person at the desk needs no guess, no hysteresis and no probe.
- **The daemon is the one owner of the permission; the tray is a lowering.**
The tray reads and sets the state only through the daemon's HTTP route, so
admission and the stored value cannot disagree and the tray holds no state of
its own. The alternative, the tray writing a CultCache document the daemon
reads, would give the document two writers on two clocks and make the daemon
poll a file to learn its own admission rule.
- **Off on a fresh install, and off when the document is missing.** The safe
default for the household: Vili does nothing on Raven's GPU until someone at
the desk says yes. A stored yes survives restarts and reboots.
- **Loopback only for the write.** The daemon listens on `0.0.0.0`; the
household's switch must not be flippable from a WireGuard peer. A request
from a non-loopback address is refused 403. Agents read the state through
`GET /gpu-permission` or `/operator-state` and never set it.
- **One cache instance for the daemon.** The 5-second surface writer built a
new store per call. Two store instances writing one file each read, modify and
write in their own queue, so a permission write could be lost under a
concurrent surface write. One module-level cache serializes every write to
`vili.service.cc`.
- **One slot, one fresh VRAM read, fail closed.** h2's slot stays: one job at a
time, the rest refused at once with source `slot`. The slot is taken
synchronously before the VRAM read, so two requests cannot both pass it. A
failed or unparseable read refuses with source `vram`, so a job never starts
onto a full card on a guess. The permission is checked again after the read.
- **Refuse, do not queue** (`motion-pipeline:ruling:gpu-defer-refuse-typed`).
`/motion/generate` is synchronous; the consumer retries a typed `gpu-busy`
after `retry-after`. A queue belongs with the authoring UX's takes
(`motion-pipeline:follow_up:vili-authoring-ux-target-2`).
- **A logon task, not the Startup folder.** `Start-ScheduledTask` on an
Interactive task puts the tray in the household's session from SSH at deploy
time; a Startup folder entry would wait for the next logon. The task pattern
matches `\GameCult\Vili`.
- **The privacy rule is moot.** "Never record program names" protected the
household from a probe that watched its programs. With no probe, nothing in
Vili observes the household's programs; the only household fact Vili holds is
the yes or no the household chose.
- **The child's own VRAM check is placement, not tenancy.** `plan_placement`
still sizes accelerate's `max_memory` from torch's view and can still refuse;
it answers "can the encoder be placed", not "may heavy work start".
still sizes accelerate's `max_memory` from torch's view and can still refuse
(source `placement`); it answers "can the encoder be placed", not "may heavy
work start".
## Cuts not yet mapped
- RPS beats through Vili (`motion-pipeline:follow_up:rps-clips-from-vili`), after
the gate and `vili-fbx-export`.
the gate and `vili-fbx-export`. They wait on the household's yes on Raven.

View file

@ -45,7 +45,22 @@ promoted from JSON once that request/receipt shape stabilizes.
`POST /motion/generate` accepts an explicit animation request and forwards the
supported Kimodo generation controls to the resident worker. Vili owns job
identity, request persistence, worker lifecycle, timeout handling, and the
artifact stem. Kimodo owns synthesis and model-specific export.
artifact stem. Kimodo owns synthesis and model-specific export. Vili also owns GPU
admission from the household's permission: the daemon holds one GPU slot, and `POST /motion/generate` is judged
when it reaches the slot. It answers HTTP 503 with `retry-after: 60` and one of four bodies, by
`source`: `{ "ok": false, "code": "gpu-busy", "source": "household", "gpu": { allowed: false,
changedAt } }` while the household's permission is off; `{ "ok": false, "code": "gpu-busy",
"source": "slot" }` while another job holds the slot (nothing queues); `{ "ok": false, "code":
"gpu-busy", "source": "vram", freeMiB, minFreeMiB }` when host free VRAM is below the minimum or
unreadable (`freeMiB` null); and `{ "ok": false, "code": "gpu-busy", "source": "placement",
"jobId" }` when the encoder child refused its own placement, where the job file was written and no
household status is claimed. `GET /gpu-permission` answers `{ allowed, changedAt, jobRunning,
lastRefusal }` to anyone; `POST /gpu-permission { "allowed": boolean }` is the household's switch and
answers 403 `{ "ok": false, "code": "household-only" }` to any caller that is not on the host's
loopback. Both POST routes require `Content-Type: application/json` and refuse a request whose
`Origin` header is present and not the daemon's own loopback origin with 403 `{ "ok": false, "code":
"cross-site" }`, so a web page cannot drive them; the daemon answers no CORS preflight. The `code` field is the only carrier; a consumer retries after `retry-after`, never sets
the permission, and never matches error text. A job already running finishes.
```json
{
@ -143,7 +158,7 @@ It mounts:
Inside the worker, Kimodo owns synthesis. Vili owns worker lifecycle,
request/response persistence, timeout handling, and artifact references. Direct
user requests, programmatic requests, and smoke-started warmup all use the same
user requests and programmatic requests use the same
resident worker health and generation path.
Observed on `2026-06-12`: first worker model load took about 100 seconds. Once
@ -154,5 +169,5 @@ resident, five 1 second / 20 step service-path generations averaged about
Health checks must be cheap. They may verify process, port, backend metadata,
and cached readiness, but they must not launch a Kimodo generation job or start
model load. Smoke checks may start the resident worker because their purpose is
backend readiness, not cheap liveness.
model load. They change nothing: no GET route starts the worker, Docker or the GPU. Only a
`POST /motion/generate` that holds the GPU slot starts the worker.

View file

@ -7,7 +7,8 @@
"scripts": {
"start": "node scripts/vili-daemon.mjs",
"health": "node scripts/vili-daemon.mjs --health",
"smoke": "node scripts/smoke-vili.mjs",
"test": "node --test scripts/test-gpu-tenancy.mjs",
"test:http": "node --test scripts/test-gpu-gate-http.mjs",
"provider": "node scripts/vili-daemon.mjs --print-provider-advertisement"
},
"dependencies": {

103
scripts/gpu-tenancy.mjs Normal file
View file

@ -0,0 +1,103 @@
// GPU tenancy gate: the one owner of "may heavy GPU work start now". The household decides, through
// the permission it sets at the tray; the daemon owns that yes or no, the one GPU slot, and one host
// free-VRAM read per admission. The gate refuses new work; it never stops work.
//
// acquire() judges in a fixed order and nothing waits: the permission, then the slot, then host free
// VRAM. The slot is taken before the VRAM read so concurrent requests are refused at once.
export const RETRY_AFTER_SECONDS = 60;
export function createGpuTenancy({ now, permission, savePermission, readFreeMiB, minFreeMiB = 10240 }) {
let allowed = permission?.allowed === true;
let changedAt = permission?.changedAt ?? null;
let held = false;
let lastRefusal = null;
const status = () => ({ allowed, changedAt, jobRunning: held, lastRefusal });
function refuse(source, extra = {}) {
lastRefusal = { at: new Date(now()).toISOString(), source };
return { refusal: { ok: false, code: "gpu-busy", source, ...extra } };
}
const household = () => refuse("household", { gpu: { allowed: false, changedAt } });
async function setAllowed(next) {
if (typeof next !== "boolean") throw new TypeError("allowed must be a boolean");
if (next === allowed) return status();
const value = { allowed: next, changedAt: new Date(now()).toISOString() };
await savePermission(value);
allowed = value.allowed;
changedAt = value.changedAt;
return status();
}
async function acquire() {
if (!allowed) return household();
if (held) return refuse("slot");
held = true;
let freeMiB;
try {
freeMiB = await readFreeMiB();
} catch {
freeMiB = null;
}
if (!allowed) {
held = false;
return household();
}
if (!Number.isFinite(freeMiB) || freeMiB < minFreeMiB) {
held = false;
return refuse("vram", { freeMiB: Number.isFinite(freeMiB) ? freeMiB : null, minFreeMiB });
}
let released = false;
return {
release() {
if (released) return;
released = true;
held = false;
},
};
}
return { status, setAllowed, acquire };
}
// Takes the slot, runs the job to its end and returns its result, or returns the typed refusal
// without calling the job. The gate never holds, pauses or cancels a job.
export async function admitThen(gate, job) {
const slot = await gate.acquire();
if (slot.refusal) return slot.refusal;
try {
return await job();
} finally {
slot.release();
}
}
// nvidia-smi's first line is the free MiB as a base-10 integer; anything else is null.
export function parseFreeMiB(stdout) {
const line = String(stdout).split("\n")[0].trim();
return /^\d+$/.test(line) ? Number(line) : null;
}
// The encoder child's placement refusal reaches the daemon as a typed code on the worker's error body;
// it is never read out of error text. Placement is the child's own view of free VRAM, not the
// household gate, so the body carries no gate status. Returns the typed 503 body, or null for any
// other outcome.
export function placementRefusal(workerBody, jobId) {
if (workerBody?.code !== "gpu-busy") return null;
return { ok: false, code: "gpu-busy", source: "placement", jobId };
}
// Operator override. A malformed value would silently defeat a comparison (NaN is never below
// anything), so it refuses to start instead, naming the variable and never its value.
export function gpuTenancyOptionsFromEnv(env) {
const options = {};
const raw = env.VILI_GPU_MIN_FREE_MIB;
if (raw !== undefined) {
const value = raw.trim() === "" ? Number.NaN : Number(raw);
if (!Number.isFinite(value) || value < 0) throw new Error("VILI_GPU_MIN_FREE_MIB must be a finite non-negative number");
options.minFreeMiB = value;
}
return options;
}

View file

@ -1,7 +1,7 @@
param(
[string]$TaskName = "GameCult\Vili",
[int]$Port = 8824,
[string]$HostName = "0.0.0.0",
[string]$HostName = "127.0.0.1,10.77.0.4",
[string]$StateRoot = "E:\Projects\Vili\.vili"
)

View file

@ -0,0 +1,17 @@
param(
[string]$TaskName = "GameCult\ViliTray",
[int]$Port = 8824
)
$ErrorActionPreference = "Stop"
$ProgressPreference = "SilentlyContinue"
$HiddenLauncher = Join-Path $PSScriptRoot "run-hidden-powershell.vbs"
$TrayScript = Join-Path $PSScriptRoot "vili-gpu-tray.ps1"
$UserId = [System.Security.Principal.WindowsIdentity]::GetCurrent().Name
$Action = New-ScheduledTaskAction -Execute "wscript.exe" -Argument "//B //nologo `"$HiddenLauncher`" -NoProfile -NonInteractive -WindowStyle Hidden -ExecutionPolicy Bypass -STA -File `"$TrayScript`" -Port $Port"
$Trigger = New-ScheduledTaskTrigger -AtLogOn -User $UserId
$Principal = New-ScheduledTaskPrincipal -UserId $UserId -LogonType Interactive -RunLevel Limited
$Settings = New-ScheduledTaskSettingsSet -AllowStartIfOnBatteries -DontStopIfGoingOnBatteries -ExecutionTimeLimit ([TimeSpan]::Zero) -MultipleInstances IgnoreNew -Hidden
Register-ScheduledTask -TaskName $TaskName -Action $Action -Trigger $Trigger -Principal $Principal -Settings $Settings -Force | Out-Null
Write-Host "Installed scheduled task $TaskName"

View file

@ -4,7 +4,6 @@ import json
import os
import shutil
import time
import traceback
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from threading import Lock
from types import SimpleNamespace
@ -394,11 +393,7 @@ class Handler(BaseHTTPRequestHandler):
payload = json.loads(self.rfile.read(length).decode("utf-8") or "{}")
write_json(self, 200, generate(payload))
except Exception as error:
write_json(self, 500, {
"ok": False,
"error": str(error),
"traceback": traceback.format_exc(),
})
write_json(self, 500, text_encoding.worker_error_body(error))
def log_message(self, fmt, *values):
print(f"{self.address_string()} - {fmt % values}", flush=True)

View file

@ -1,6 +1,6 @@
param(
[int]$Port = 8824,
[string]$HostName = "0.0.0.0",
[string]$HostName = "127.0.0.1,10.77.0.4",
[string]$StateRoot = "E:\Projects\Vili\.vili"
)

View file

@ -1,6 +1,6 @@
param(
[int]$Port = 8824,
[string]$HostName = "0.0.0.0",
[string]$HostName = "127.0.0.1,10.77.0.4",
[string]$StateRoot = "E:\Projects\Vili\.vili"
)

View file

@ -1,36 +0,0 @@
#!/usr/bin/env node
const baseUrl = process.env.VILI_URL || process.argv[2] || "http://10.77.0.4:8824";
async function get(path) {
const response = await fetch(`${baseUrl}${path}`);
const text = await response.text();
let body;
try {
body = JSON.parse(text);
} catch {
body = text;
}
if (!response.ok) {
throw new Error(`${path} returned ${response.status}: ${text.slice(0, 500)}`);
}
return body;
}
const health = await get("/health");
const providers = await get("/eve/deck/providers");
const smoke = await get("/smoke");
console.log(JSON.stringify({
ok: true,
baseUrl,
health: {
service: health.service,
providerId: health.providerId,
backend: health.state?.backend,
},
providers,
smoke: {
ok: smoke.ok,
backend: smoke.backendStatus,
},
}, null, 2));

View file

@ -1,6 +1,6 @@
param(
[int]$Port = 8824,
[string]$HostName = "0.0.0.0",
[string]$HostName = "127.0.0.1,10.77.0.4",
[string]$StateRoot = "E:\Projects\Vili\.vili"
)

View file

@ -0,0 +1,417 @@
// Boots the real daemon and observes the gate where it is decided: the HTTP edge. Needs the
// daemon's own dependencies (ws, cultcache-ts), so it runs on a machine that has them. Every boot
// sets VILI_GPU_MIN_FREE_MIB=999999 so an allowed request ends in a VRAM refusal and never starts
// the worker: no test here may start a GPU job.
import assert from "node:assert/strict";
import { spawn, spawnSync } from "node:child_process";
import { existsSync, readFileSync } from "node:fs";
import { copyFile, mkdtemp, readdir, rm, writeFile } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import test from "node:test";
import { fileURLToPath } from "node:url";
const daemonPath = path.join(path.dirname(fileURLToPath(import.meta.url)), "vili-daemon.mjs");
let nextPort = 18000 + Math.floor(Math.random() * 1000);
async function bootDaemon(stateRoot, { host = "127.0.0.1", env = { VILI_GPU_MIN_FREE_MIB: "999999" } } = {}) {
const port = nextPort++;
const child = spawn(process.execPath, [daemonPath, "--host", host, "--port", String(port), "--state-root", stateRoot], {
env: { ...process.env, ...env },
stdio: ["ignore", "pipe", "pipe"],
windowsHide: true,
});
let output = "";
child.stdout.on("data", (chunk) => { output += chunk; });
child.stderr.on("data", (chunk) => { output += chunk; });
const deadline = Date.now() + 30000;
while (!output.includes("Vili listening")) {
if (child.exitCode !== null || Date.now() > deadline) throw new Error(`daemon did not start: ${output.slice(-500)}`);
await new Promise((resolve) => setTimeout(resolve, 100));
}
const stop = () => new Promise((resolve) => {
if (child.exitCode !== null) resolve();
else { child.once("exit", resolve); child.kill(); }
});
return { child, port, stop };
}
const url = (d, route, host = "127.0.0.1") => `http://${host}:${d.port}${route}`;
const generate = (d, body) => fetch(url(d, "/motion/generate"), {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify(body),
});
const setPermission = (d, body, host = "127.0.0.1") => fetch(url(d, "/gpu-permission", host), {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify(body),
});
const readPermission = async (d) => (await fetch(url(d, "/gpu-permission"))).json();
test("http: refusal at the daemon edge", async () => {
const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-"));
const d = await bootDaemon(stateRoot);
try {
assert.deepEqual(await readPermission(d), { allowed: false, changedAt: null, jobRunning: false, lastRefusal: null });
assert.equal((await generate(d, {})).status, 400, "request-shape errors come ahead of the gate");
const refused = await generate(d, { prompt: "a person waves hello, never seen before" });
assert.equal(refused.status, 503);
assert.equal(refused.headers.get("retry-after"), "60");
assert.deepEqual(await refused.json(), { ok: false, code: "gpu-busy", source: "household", gpu: { allowed: false, changedAt: null } });
assert.deepEqual(await readdir(path.join(stateRoot, "jobs")), [], "no job file");
assert.equal((await readdir(stateRoot)).includes("artifacts"), false, "no artifact directory");
const state = await (await fetch(url(d, "/operator-state"))).json();
assert.equal(state.gpu.lastRefusal.source, "household");
assert.equal(state.gpu.allowed, false);
} finally {
await d.stop();
await rm(stateRoot, { recursive: true, force: true });
}
});
test("http: the permission persists across a daemon restart", async () => {
const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-persist-"));
let d = await bootDaemon(stateRoot);
try {
const on = await setPermission(d, { allowed: true });
assert.equal(on.status, 200);
const onBody = await on.json();
assert.equal(onBody.ok, true);
assert.equal(onBody.gpu.allowed, true);
const before = await readPermission(d);
assert.equal(before.allowed, true);
assert.ok(before.changedAt);
assert.equal((await setPermission(d, { allowed: "yes" })).status, 400);
assert.equal((await setPermission(d, {})).status, 400);
assert.deepEqual(await readPermission(d), before, "a refused write changes nothing");
await d.stop();
d = await bootDaemon(stateRoot);
const after = await readPermission(d);
assert.equal(after.allowed, true);
assert.equal(after.changedAt, before.changedAt);
await setPermission(d, { allowed: false });
await d.stop();
d = await bootDaemon(stateRoot);
assert.equal((await readPermission(d)).allowed, false, "off persists too");
} finally {
await d.stop();
await rm(stateRoot, { recursive: true, force: true });
}
});
test("http: an allowed request reaches the slot and the VRAM read", async () => {
const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-slot-"));
const d = await bootDaemon(stateRoot);
try {
await setPermission(d, { allowed: true });
const refused = await generate(d, { prompt: "a person waves hello, never seen before" });
assert.equal(refused.status, 503);
const body = await refused.json();
assert.equal(body.source, "vram", "admitted through acquire(), then refused for VRAM");
assert.equal(body.minFreeMiB, 999999);
assert.deepEqual(await readdir(path.join(stateRoot, "jobs")), [], "no job file");
assert.equal((await readdir(stateRoot)).includes("artifacts"), false);
const state = await readPermission(d);
assert.equal(state.lastRefusal.source, "vram");
assert.equal(state.jobRunning, false);
} finally {
await d.stop();
await rm(stateRoot, { recursive: true, force: true });
}
});
test("http: the switch is the household's", async () => {
const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-lan-"));
const addresses = Object.values(os.networkInterfaces()).flat().filter((i) => i.family === "IPv4" && !i.internal).map((i) => i.address);
const lan = addresses[0];
assert.ok(lan, "the host has a non-loopback IPv4 address");
const d = await bootDaemon(stateRoot, { host: `127.0.0.1,${lan}` });
try {
const before = await readPermission(d);
const refused = await setPermission(d, { allowed: true }, lan);
assert.equal(refused.status, 403);
assert.deepEqual(await refused.json(), { ok: false, code: "household-only" });
assert.deepEqual(await readPermission(d), before, "GET still reads the previous value");
assert.equal((await setPermission(d, { allowed: true })).status, 200, "loopback still sets it");
const other = addresses.find((address) => address !== lan);
if (other) {
await assert.rejects(fetch(url(d, "/gpu-permission", other)), "an address --host does not name is not served");
}
} finally {
await d.stop();
await rm(stateRoot, { recursive: true, force: true });
}
});
test("http: a web page cannot flip the household's switch or start a job", async () => {
const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-csrf-"));
const d = await bootDaemon(stateRoot);
const post = (route, headers, body) => fetch(url(d, route), { method: "POST", headers, body });
const prompt = JSON.stringify({ prompt: "a person waves hello, never seen before" });
try {
const before = await readPermission(d);
for (const route of ["/gpu-permission", "/motion/generate"]) {
const payload = route === "/gpu-permission" ? JSON.stringify({ allowed: true }) : prompt;
const simple = await post(route, { "content-type": "text/plain" }, payload);
assert.equal(simple.status, 403, `${route}: a simple text/plain POST`);
assert.deepEqual(await simple.json(), { ok: false, code: "cross-site" });
assert.equal((await post(route, {}, payload)).status, 403, `${route}: no content-type`);
const hostile = await post(route, { "content-type": "application/json", origin: "http://evil.example-origin" }, payload);
assert.equal(hostile.status, 403, `${route}: a hostile Origin`);
const body = JSON.stringify(await hostile.json());
assert.equal(body.includes("evil.example-origin"), false, "the refusal never echoes the Origin");
assert.equal(
(await post(route, { "content-type": "application/json", origin: `http://127.0.0.1:${d.port}.evil.example` }, payload)).status,
403,
`${route}: an Origin that only starts like ours`,
);
}
assert.deepEqual(await readPermission(d), before, "no refused request changed anything");
assert.deepEqual(await readdir(path.join(stateRoot, "jobs")), [], "no job file");
const preflight = await fetch(url(d, "/gpu-permission"), {
method: "OPTIONS",
headers: { origin: "http://evil.example-origin", "access-control-request-method": "POST", "access-control-request-headers": "content-type" },
});
assert.equal(preflight.ok, false, "the daemon approves no CORS preflight");
assert.equal(preflight.headers.get("access-control-allow-methods"), null);
const own = await post("/gpu-permission", { "content-type": "application/json; charset=utf-8", origin: `http://127.0.0.1:${d.port}` }, JSON.stringify({ allowed: true }));
assert.equal(own.status, 200, "the daemon's own origin still works");
assert.equal((await setPermission(d, { allowed: false })).status, 200, "a caller with no Origin (the tray) still works");
} finally {
await d.stop();
await rm(stateRoot, { recursive: true, force: true });
}
});
// The daemon reaches the GPU only through the processes wsl.exe and nvidia-smi.exe. This test puts
// one small compiled stub on PATH under both names (Windows, where the daemon lives), so the real
// daemon runs a real job against a worker that takes as long as the test says and touches no GPU:
// nvidia-smi.exe answers 20000 free MiB, the worker health call answers ok, and the "/generate"
// call waits until the test creates a release file.
const stubSource = `
using System; using System.IO; using System.Reflection; using System.Threading;
class Stub {
static int Main(string[] args) {
string exe = Path.GetFileNameWithoutExtension(Assembly.GetEntryAssembly().Location).ToLowerInvariant();
string dir = Environment.GetEnvironmentVariable("VILI_STUB_DIR");
if (exe == "nvidia-smi") { Console.WriteLine("20000"); return 0; }
string script = args.Length > 0 ? args[args.Length - 1] : "";
if (script.Contains("docker run")) File.AppendAllText(Path.Combine(dir, "dockerrun"), "x");
if (script.Contains("service docker start")) File.AppendAllText(Path.Combine(dir, "dockerstart"), "x");
if (script.Contains("/health")) { Console.WriteLine("{\\"ok\\":true,\\"model\\":\\"stub\\"}"); return 0; }
if (script.Contains("/generate")) {
File.AppendAllText(Path.Combine(dir, "started"), "x");
while (!File.Exists(Path.Combine(dir, "release"))) Thread.Sleep(25);
Console.WriteLine("{\\"ok\\":true}");
}
return 0;
}
}`;
async function stubbedDaemon(stubDir, stateRoot) {
const source = path.join(stubDir, "stub.cs");
await writeFile(source, stubSource);
const build = path.join(stubDir, "build.ps1");
await writeFile(build, [
"param($Source, $Out)",
"Add-Type -TypeDefinition (Get-Content -Raw $Source) -OutputAssembly $Out -OutputType ConsoleApplication",
].join("\r\n"));
const built = spawnSync("powershell.exe", [
"-NoProfile", "-NonInteractive", "-ExecutionPolicy", "Bypass", "-File", build, source, path.join(stubDir, "wsl.exe"),
], { encoding: "utf8", windowsHide: true });
assert.equal(built.status, 0, `the stub did not compile: ${built.stderr || built.stdout}`);
await copyFile(path.join(stubDir, "wsl.exe"), path.join(stubDir, "nvidia-smi.exe"));
const pathKey = Object.keys(process.env).find((key) => key.toLowerCase() === "path") ?? "PATH";
return bootDaemon(stateRoot, {
env: {
[pathKey]: `${stubDir}${path.delimiter}${process.env[pathKey]}`,
VILI_STUB_DIR: stubDir,
VILI_GPU_MIN_FREE_MIB: "1",
},
});
}
test("http: the daemon holds the slot for the whole job", async () => {
const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-hold-"));
const stubDir = await mkdtemp(path.join(os.tmpdir(), "vili-gate-stub-"));
const d = await stubbedDaemon(stubDir, stateRoot);
const prompt = { prompt: "a person waves hello, never seen before" };
const within = (promise, ms, what) => Promise.race([
promise,
new Promise((_, reject) => setTimeout(() => reject(new Error(`${what} did not answer within ${ms} ms`)), ms)),
]);
try {
await setPermission(d, { allowed: true });
const first = generate(d, prompt);
const deadline = Date.now() + 20000;
while (!(await readPermission(d)).jobRunning) {
assert.ok(Date.now() < deadline, "the first job never took the slot");
await new Promise((resolve) => setTimeout(resolve, 50));
}
while (!existsSync(path.join(stubDir, "started"))) {
assert.ok(Date.now() < deadline, "the first job never reached the worker");
await new Promise((resolve) => setTimeout(resolve, 50));
}
for (let i = 0; i < 3; i += 1) {
const second = await within(generate(d, prompt), 5000, "a request during the job");
assert.equal(second.status, 503, "a request during the job is refused at once");
assert.deepEqual(await second.json(), { ok: false, code: "gpu-busy", source: "slot" });
}
assert.equal(readFileSync(path.join(stubDir, "started"), "utf8"), "x", "exactly one job reached the worker");
assert.equal((await readPermission(d)).jobRunning, true, "the slot is still held");
await writeFile(path.join(stubDir, "release"), "");
const done = await within(first, 20000, "the first job");
assert.equal(done.status, 200);
assert.equal((await readPermission(d)).jobRunning, false, "the slot frees when the job ends");
const next = await within(generate(d, prompt), 20000, "the next job");
assert.equal(next.status, 200, "the next job is admitted after the slot frees");
assert.equal(readFileSync(path.join(stubDir, "started"), "utf8"), "xx");
} finally {
await d.stop();
await rm(stateRoot, { recursive: true, force: true });
await rm(stubDir, { recursive: true, force: true });
}
});
test("http: a malformed override refuses to start and names the variable, not its value", async () => {
const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-env-"));
const child = spawn(process.execPath, [daemonPath, "--host", "127.0.0.1", "--port", String(nextPort++), "--state-root", stateRoot], {
env: { ...process.env, VILI_GPU_MIN_FREE_MIB: "10240MiB-secret" },
stdio: ["ignore", "pipe", "pipe"],
windowsHide: true,
});
let output = "";
child.stdout.on("data", (chunk) => { output += chunk; });
child.stderr.on("data", (chunk) => { output += chunk; });
try {
const code = await new Promise((resolve) => child.on("exit", resolve));
assert.notEqual(code, 0);
assert.match(output, /VILI_GPU_MIN_FREE_MIB must be a finite non-negative number/);
assert.equal(output.includes("10240MiB-secret"), false);
assert.equal(output.includes("Vili listening"), false);
} finally {
child.kill();
await rm(stateRoot, { recursive: true, force: true });
}
});
test("http: the daemon refuses to listen on a wildcard address", async () => {
const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-wild-"));
try {
for (const host of ["0.0.0.0", "127.0.0.1,0.0.0.0", "::"]) {
const run = spawnSync(process.execPath, [daemonPath, "--host", host, "--port", String(nextPort++), "--state-root", stateRoot], {
encoding: "utf8",
windowsHide: true,
timeout: 20000,
});
assert.equal(run.status, 2, `--host ${host} exits 2`);
assert.equal(`${run.stdout}${run.stderr}`.includes("Vili listening"), false);
}
} finally {
await rm(stateRoot, { recursive: true, force: true });
}
});
test("http: no response grants a web page access", async () => {
const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-cors-"));
const d = await bootDaemon(stateRoot);
try {
const hostile = { origin: "http://evil.example-origin" };
const responses = [
await fetch(url(d, "/gpu-permission"), { headers: hostile }),
await fetch(url(d, "/operator-state"), { headers: hostile }),
await fetch(url(d, "/nowhere"), { headers: hostile }),
await fetch(url(d, "/gpu-permission"), { method: "POST", headers: { ...hostile, "content-type": "application/json" }, body: "{}" }),
];
assert.deepEqual(responses.map((r) => r.status), [200, 200, 404, 403]);
for (const response of responses) {
const cors = [...response.headers.keys()].filter((name) => name.startsWith("access-control-"));
assert.deepEqual(cors, [], "no access-control header on any response");
}
} finally {
await d.stop();
await rm(stateRoot, { recursive: true, force: true });
}
});
test("http: with the permission off nothing a web page or a LAN caller sends reaches the GPU", async () => {
const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-off-"));
const stubDir = await mkdtemp(path.join(os.tmpdir(), "vili-gate-stub-"));
const d = await stubbedDaemon(stubDir, stateRoot);
const page = { origin: "http://evil.example-origin", "sec-fetch-site": "cross-site", "sec-fetch-mode": "no-cors", "sec-fetch-dest": "image" };
try {
const before = await readPermission(d);
assert.equal(before.allowed, false);
for (const route of ["/smoke", "/health", "/", "/operator-state", "/eve/operator", "/provider-advertisement", "/eve/deck/providers", "/gpu-permission"]) {
const response = await fetch(url(d, route), { headers: page });
assert.equal(response.status, route === "/smoke" ? 404 : 200, `GET ${route}`);
}
const refused = await generate(d, { prompt: "a person waves hello, never seen before" });
assert.equal(refused.status, 503);
assert.equal((await refused.json()).source, "household");
assert.equal((await setPermission(d, { allowed: true }, "127.0.0.1")).status, 200);
assert.equal((await setPermission(d, { allowed: false }, "127.0.0.1")).status, 200);
assert.deepEqual((await readPermission(d)).allowed, false);
assert.equal(existsSync(path.join(stubDir, "dockerrun")), false, "nothing ran docker run (the worker or any GPU container)");
assert.equal(existsSync(path.join(stubDir, "dockerstart")), false, "no GET started Docker");
assert.equal(existsSync(path.join(stubDir, "started")), false, "nothing reached the worker");
assert.deepEqual(await readdir(path.join(stateRoot, "jobs")), [], "no job file");
} finally {
await d.stop();
await rm(stateRoot, { recursive: true, force: true });
await rm(stubDir, { recursive: true, force: true });
}
});
test("http: a client that hangs up mid-job does not free the slot", async () => {
const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-hangup-"));
const stubDir = await mkdtemp(path.join(os.tmpdir(), "vili-gate-stub-"));
const d = await stubbedDaemon(stubDir, stateRoot);
const prompt = { prompt: "a person waves hello, never seen before" };
const hangUp = new AbortController();
try {
await setPermission(d, { allowed: true });
const first = fetch(url(d, "/motion/generate"), {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify(prompt),
signal: hangUp.signal,
}).catch((error) => error);
const deadline = Date.now() + 20000;
while (!existsSync(path.join(stubDir, "started"))) {
assert.ok(Date.now() < deadline, "the job never reached the worker");
await new Promise((resolve) => setTimeout(resolve, 50));
}
hangUp.abort();
assert.equal((await first).name, "AbortError", "the client hung up");
await new Promise((resolve) => setTimeout(resolve, 500));
assert.equal((await readPermission(d)).jobRunning, true, "the slot is still held after the hang-up");
const second = await generate(d, prompt);
assert.equal(second.status, 503);
assert.equal((await second.json()).source, "slot");
assert.equal(readFileSync(path.join(stubDir, "started"), "utf8"), "x", "no second job reached the worker");
await writeFile(path.join(stubDir, "release"), "");
const freed = Date.now() + 20000;
while ((await readPermission(d)).jobRunning) {
assert.ok(Date.now() < freed, "the slot never freed when the job ended");
await new Promise((resolve) => setTimeout(resolve, 50));
}
} finally {
await d.stop();
await rm(stateRoot, { recursive: true, force: true });
await rm(stubDir, { recursive: true, force: true });
}
});

View file

@ -0,0 +1,211 @@
import assert from "node:assert/strict";
import test from "node:test";
import { admitThen, createGpuTenancy, gpuTenancyOptionsFromEnv, parseFreeMiB, placementRefusal } from "./gpu-tenancy.mjs";
function deferred() {
let resolve;
const promise = new Promise((r) => { resolve = r; });
return { promise, resolve };
}
// Fake ports: a clock, a store spy and a free-VRAM source. No GPU, no daemon, no cache.
function rig({ permission, free = 12000, minFreeMiB } = {}) {
const r = { t: Date.parse("2026-10-23T10:00:00Z"), saved: [], reads: 0, free, failSave: false };
r.gate = createGpuTenancy({
now: () => r.t,
permission,
minFreeMiB,
savePermission: async (value) => {
if (r.failSave) throw new Error("disk full");
r.saved.push(value);
},
readFreeMiB: async () => {
r.reads += 1;
if (typeof r.free === "function") return r.free();
return r.free;
},
});
return r;
}
const ON = { allowed: true, changedAt: "2026-10-23T09:00:00.000Z" };
test("gate: toggle off refuses and creates no job", async () => {
const r = rig();
let calls = 0;
const refused = await admitThen(r.gate, async () => { calls += 1; });
assert.deepEqual(refused, { ok: false, code: "gpu-busy", source: "household", gpu: { allowed: false, changedAt: null } });
assert.equal(calls, 0);
assert.equal(r.reads, 0, "the VRAM is never read for a household refusal");
assert.deepEqual(r.gate.status(), {
allowed: false,
changedAt: null,
jobRunning: false,
lastRefusal: { at: "2026-10-23T10:00:00.000Z", source: "household" },
});
});
test("gate: toggle on admits", async () => {
const r = rig({ permission: ON });
let calls = 0;
const result = await admitThen(r.gate, async () => {
calls += 1;
assert.equal(r.gate.status().jobRunning, true);
return { ok: true, value: "finished" };
});
assert.deepEqual(result, { ok: true, value: "finished" });
assert.equal(calls, 1);
assert.equal(r.gate.status().jobRunning, false);
assert.equal(r.gate.status().lastRefusal, null);
});
test("gate: toggle off mid-job lets the job finish and refuses the next", async () => {
const r = rig({ permission: ON });
const end = deferred();
const running = admitThen(r.gate, async () => { await end.promise; return { ok: true, value: "finished" }; });
while (!r.gate.status().jobRunning) await new Promise((resolve) => setImmediate(resolve));
const off = await r.gate.setAllowed(false);
assert.equal(off.allowed, false);
assert.equal(off.jobRunning, true, "the job is still running");
let calls = 0;
const next = await admitThen(r.gate, async () => { calls += 1; });
assert.equal(next.source, "household");
assert.equal(calls, 0);
end.resolve();
assert.deepEqual(await running, { ok: true, value: "finished" });
assert.equal(r.gate.status().jobRunning, false);
});
test("gate: the concurrent slot behaves as h2's did", async () => {
const vram = deferred();
const r = rig({ permission: ON, free: () => vram.promise });
const end = deferred();
let calls = 0;
const job = async () => { calls += 1; await end.promise; return "done"; };
const all = [admitThen(r.gate, job), admitThen(r.gate, job), admitThen(r.gate, job)];
// the two losers are decided before the VRAM read resolves
const losers = await Promise.all([all[1], all[2]]);
assert.deepEqual(losers, [{ ok: false, code: "gpu-busy", source: "slot" }, { ok: false, code: "gpu-busy", source: "slot" }]);
assert.equal(r.reads, 1, "only the slot holder reads VRAM");
vram.resolve(12000);
end.resolve();
assert.equal(await all[0], "done");
assert.equal(calls, 1);
assert.equal(r.gate.status().jobRunning, false);
await assert.rejects(admitThen(r.gate, async () => { throw new Error("boom"); }), /boom/);
assert.equal(r.gate.status().jobRunning, false, "a throwing job frees the slot");
r.free = 100;
assert.equal((await admitThen(r.gate, async () => "x")).source, "vram");
const readsBefore = r.reads;
r.free = 12000;
assert.equal(await admitThen(r.gate, async () => "again"), "again", "a vram refusal frees the slot");
assert.equal(r.reads, readsBefore + 1, "the next admission reads VRAM again");
});
test("gate: VRAM read fails closed", async () => {
const cases = [[9000, 9000], [10239, 10239], [null, null], [Number.NaN, null], [() => { throw new Error("smi"); }, null]];
for (const [free, expected] of cases) {
const r = rig({ permission: ON, free });
let calls = 0;
const refused = await admitThen(r.gate, async () => { calls += 1; });
assert.deepEqual(refused, { ok: false, code: "gpu-busy", source: "vram", freeMiB: expected, minFreeMiB: 10240 });
assert.equal(calls, 0);
assert.equal(r.gate.status().jobRunning, false);
assert.equal(r.gate.status().lastRefusal.source, "vram");
}
const edge = rig({ permission: ON, free: 10240 });
assert.equal(await admitThen(edge.gate, async () => "in"), "in", "exactly the minimum admits");
const custom = rig({ permission: ON, free: 5000, minFreeMiB: 4096 });
assert.equal(await admitThen(custom.gate, async () => "in"), "in", "the minimum is the configured one");
});
test("gate: a toggle during the VRAM read wins", async () => {
const vram = deferred();
const r = rig({ permission: ON, free: () => vram.promise });
let calls = 0;
const pending = admitThen(r.gate, async () => { calls += 1; });
await r.gate.setAllowed(false);
vram.resolve(12000);
const refused = await pending;
assert.equal(refused.source, "household");
assert.equal(calls, 0);
assert.equal(r.gate.status().jobRunning, false);
});
test("gate: setAllowed saves before it answers", async () => {
const r = rig();
r.failSave = true;
await assert.rejects(r.gate.setAllowed(true), /disk full/);
assert.equal(r.gate.status().allowed, false, "a failed save leaves the state unchanged");
assert.equal(r.gate.status().changedAt, null);
r.failSave = false;
await assert.rejects(r.gate.setAllowed("yes"), TypeError);
await assert.rejects(r.gate.setAllowed(undefined), TypeError);
assert.deepEqual(r.saved, [], "a non-boolean saves nothing");
r.t += 1000;
const on = await r.gate.setAllowed(true);
assert.deepEqual(r.saved, [{ allowed: true, changedAt: "2026-10-23T10:00:01.000Z" }]);
assert.equal(on.allowed, true);
assert.equal(on.changedAt, "2026-10-23T10:00:01.000Z");
r.t += 5000;
const same = await r.gate.setAllowed(true);
assert.equal(r.saved.length, 1, "a same-value call saves nothing");
assert.equal(same.changedAt, "2026-10-23T10:00:01.000Z", "and keeps changedAt");
});
test("gate: a second release is a no-op and cannot free another job's slot", async () => {
const r = rig({ permission: ON });
const first = await r.gate.acquire();
first.release();
const second = await r.gate.acquire();
assert.equal(r.gate.status().jobRunning, true);
first.release();
assert.equal(r.gate.status().jobRunning, true, "a stale release leaves the new holder's slot held");
assert.equal((await r.gate.acquire()).refusal.source, "slot");
second.release();
second.release();
assert.equal(r.gate.status().jobRunning, false);
assert.equal(typeof (await r.gate.acquire()).release, "function", "the slot is free after the real release");
});
test("gate: a stored permission is the starting state", () => {
assert.deepEqual(rig({ permission: ON }).gate.status(), { allowed: true, changedAt: ON.changedAt, jobRunning: false, lastRefusal: null });
assert.equal(rig({ permission: { allowed: "yes", changedAt: "x" } }).gate.status().allowed, false, "only a true boolean allows");
});
test("parseFreeMiB reads the first line as a base-10 integer", () => {
assert.equal(parseFreeMiB("5001\r\n"), 5001);
assert.equal(parseFreeMiB("0"), 0);
assert.equal(parseFreeMiB(" 12 \n7\n"), 12);
for (const bad of ["", "[N/A]", "abc", "-5", "1e3", "12.5", "12 MiB", "\n5", null, undefined]) {
assert.equal(parseFreeMiB(bad), null, JSON.stringify(bad));
}
});
test("placement refusal is keyed on the typed code, never on text, and carries no gate status", () => {
assert.deepEqual(placementRefusal({ ok: false, code: "gpu-busy", error: "x" }, "vili-1"), {
ok: false, code: "gpu-busy", source: "placement", jobId: "vili-1",
});
assert.equal(placementRefusal({ ok: false, error: "gpu-busy: 0.50 GiB of VRAM free" }, "vili-1"), null);
assert.equal(placementRefusal({ ok: false, code: null }, "vili-1"), null);
assert.equal(placementRefusal(null, "vili-1"), null);
});
test("env override: a valid value is read, absent is left to the default", () => {
assert.deepEqual(gpuTenancyOptionsFromEnv({}), {});
assert.deepEqual(gpuTenancyOptionsFromEnv({ VILI_GPU_MIN_FREE_MIB: "8192.5" }), { minFreeMiB: 8192.5 });
assert.deepEqual(gpuTenancyOptionsFromEnv({ VILI_GPU_MIN_FREE_MIB: "0" }), { minFreeMiB: 0 });
});
test("env override: a malformed value refuses to start and names the variable, not the value", () => {
for (const bad of ["10240MiB", "", " ", "NaN", "Infinity", "-5"]) {
assert.throws(
() => gpuTenancyOptionsFromEnv({ VILI_GPU_MIN_FREE_MIB: bad }),
(error) => error.message === "VILI_GPU_MIN_FREE_MIB must be a finite non-negative number",
JSON.stringify(bad),
);
}
});

View file

@ -180,6 +180,12 @@ class ChildBoundary(unittest.TestCase):
""")
self.assertEqual(raised.exception.free_vram, 1234)
def test_worker_error_body_carries_the_code(self):
busy = te.worker_error_body(te.GpuBusy(GIB))
self.assertIs(busy["ok"], False)
self.assertEqual(busy["code"], "gpu-busy")
self.assertIsNone(te.worker_error_body(RuntimeError("x"))["code"])
def test_other_failures_name_the_exit_code_and_never_the_text(self):
with self.assertRaises(RuntimeError) as raised:
self.run_stub("import sys; sys.exit(1)", texts=(SECRET,))

View file

@ -12,6 +12,7 @@ import subprocess
import sys
import tempfile
import time
import traceback
from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path
@ -33,7 +34,11 @@ LLAMA_BASE = "meta-llama/Meta-Llama-3-8B-Instruct" # the base model the mntp ad
class GpuBusy(RuntimeError):
"""The card has too little free VRAM to host the encoder; there is no CPU fallback."""
"""Placement refusal: too little free VRAM to host the encoder, and no CPU fallback.
Whether heavy work may start at all is the daemon's tenancy gate, which sees the host; this is only
the child sizing its own placement from what torch can see.
"""
code = "gpu-busy"
@ -49,6 +54,11 @@ def plan_placement(free_vram: int, mem_available: int) -> dict:
return {0: free_vram - GPU_RESERVE, "cpu": max(mem_available - CPU_RESERVE, 0)}
def worker_error_body(error: BaseException) -> dict:
"""The worker's one error wire shape: a typed code (GpuBusy.code, else None) beside the text."""
return {"ok": False, "error": str(error), "code": getattr(error, "code", None), "traceback": traceback.format_exc()}
def read_snapshot_revision(hf_home: Path, repo_id: str) -> str:
"""The commit the mounted Hugging Face cache resolves a repo's main ref to."""
ref = Path(hf_home) / "hub" / ("models--" + repo_id.replace("/", "--")) / "refs" / "main"

View file

@ -6,6 +6,7 @@ import { fileURLToPath } from "node:url";
import { createRequire } from "node:module";
import { execFile } from "node:child_process";
import { WebSocketServer } from "ws";
import { RETRY_AFTER_SECONDS, admitThen, createGpuTenancy, gpuTenancyOptionsFromEnv, parseFreeMiB, placementRefusal } from "./gpu-tenancy.mjs";
const __dirname = path.dirname(fileURLToPath(import.meta.url));
const require = createRequire(import.meta.url);
@ -59,6 +60,20 @@ const operatorStateDefinition = defineDocumentType({
{ slot: 1, memberName: "providerId", typeName: "string" },
{ slot: 2, memberName: "updatedAt", typeName: "string" },
{ slot: 3, memberName: "backend", typeName: "object" },
{ slot: 4, memberName: "gpu", typeName: "object" },
],
});
const gpuPermissionDefinition = defineDocumentType({
type: "gamecult.vili.gpu_permission",
schemaName: "gamecult.vili.gpu_permission",
schemaId: "gamecult.vili.gpu_permission.v1",
schemaVersion: "gamecult.vili.gpu_permission.v1",
name: (value) => value?.host || "raven",
schema: objectSchema,
members: [
{ slot: 0, memberName: "host", typeName: "string", isName: true },
{ slot: 1, memberName: "allowed", typeName: "boolean" },
{ slot: 2, memberName: "changedAt", typeName: "string" },
],
});
const eveSurfaceDefinition = defineDocumentType({
@ -108,6 +123,7 @@ const transportProfileDefinition = defineDocumentType({
const viliDocumentRegistry = defineDocumentRegistry(
providerAdvertisementDefinition,
operatorStateDefinition,
gpuPermissionDefinition,
eveSurfaceDefinition,
commandBoundaryDefinition,
transportProfileDefinition,
@ -115,6 +131,8 @@ const viliDocumentRegistry = defineDocumentRegistry(
function parseArgs(argv) {
const args = {
// Comma-separated addresses to listen on, one server each. Never 0.0.0.0: the daemon answers
// this machine and the WireGuard address, not whatever network the machine joins next.
host: process.env.VILI_DAEMON_HOST || "127.0.0.1",
port: Number(process.env.VILI_DAEMON_PORT || 8824),
stateRoot: process.env.VILI_STATE_ROOT || path.join(projectRoot, ".vili"),
@ -134,13 +152,31 @@ function parseArgs(argv) {
}
const args = parseArgs(process.argv.slice(2));
const hosts = args.host.split(",").map((host) => host.trim()).filter(Boolean);
const startedAt = new Date().toISOString();
const gpuOptions = gpuTenancyOptionsFromEnv(process.env);
// The daemon's one cache and store instance: every write to vili.service.cc goes through one queue.
const viliCache = CultCache.builder()
.withRegistry(viliDocumentRegistry)
.withGenericStore(new SingleFileMessagePackBackingStore(path.join(args.stateRoot, viliStoreFileName)))
.build();
let storedPermission;
if (!args.health && !args.printProviderAdvertisement) {
await viliCache.pullAllBackingStores();
storedPermission = viliCache.get(gpuPermissionDefinition, "raven");
}
const gpuTenancy = createGpuTenancy({
now: Date.now,
...gpuOptions,
permission: storedPermission,
savePermission: (value) => viliCache.put(gpuPermissionDefinition, "raven", { host: "raven", ...value }),
readFreeMiB: hostFreeMiB,
});
let backendStatus = {
checkedAt: null,
dockerImage: "unknown",
nvidiaRuntime: "unknown",
huggingFaceToken: "unknown",
kimodoHelp: "unknown",
kimodoWorker: "unknown",
kimodoWorkerDetail: null,
lastError: null,
@ -171,6 +207,11 @@ function wsl(script, options = {}) {
return run("wsl.exe", ["-d", "Ubuntu-24.04", "-u", "root", "--", "bash", "-lc", script], options);
}
async function hostFreeMiB() {
const r = await run("nvidia-smi.exe", ["--query-gpu=memory.free", "--format=csv,noheader,nounits"], { timeout: 5000 });
return r.ok ? parseFreeMiB(r.stdout) : null;
}
function shQuote(value) {
return `'${String(value).replaceAll("'", "'\\''")}'`;
}
@ -193,12 +234,12 @@ async function wslJson(script, options = {}) {
}
}
function json(response, statusCode, body) {
function json(response, statusCode, body, headers = {}) {
const payload = JSON.stringify(body, null, 2);
response.writeHead(statusCode, {
...headers,
"content-type": "application/json; charset=utf-8",
"cache-control": "no-store",
"access-control-allow-origin": "*",
});
response.end(payload);
}
@ -207,7 +248,6 @@ function text(response, statusCode, body) {
response.writeHead(statusCode, {
"content-type": "text/plain; charset=utf-8",
"cache-control": "no-store",
"access-control-allow-origin": "*",
});
response.end(body);
}
@ -226,7 +266,6 @@ function providerAdvertisement() {
endpoints: {
http: ravenBaseUrl,
health: `${ravenBaseUrl}/health`,
smoke: `${ravenBaseUrl}/smoke`,
providerAdvertisement: `${ravenBaseUrl}/provider-advertisement`,
operatorState: `${ravenBaseUrl}/operator-state`,
operatorSurface: `${ravenBaseUrl}/eve/operator`,
@ -252,6 +291,7 @@ function operatorState() {
startedAt,
updatedAt: new Date().toISOString(),
backend: backendStatus,
gpu: gpuTenancy.status(),
authority: {
owns: [
"animation job intake",
@ -273,8 +313,8 @@ function operatorState() {
operatorState: "/operator-state",
operatorSurface: "/eve/operator",
deckProviders: "/eve/deck/providers",
smoke: "/smoke",
generate: "/motion/generate",
gpuPermission: "/gpu-permission",
},
};
}
@ -298,10 +338,10 @@ function commandBoundary() {
"motion artifact references",
],
},
smoke: {
method: "GET",
path: "/smoke",
owns: ["backend readiness proof", "resident worker warmup"],
gpuPermission: {
method: "POST",
path: "/gpu-permission",
owns: ["the household GPU permission for this host, from loopback only"],
},
liveness: {
method: "GET",
@ -353,10 +393,8 @@ function eveSurface() {
{ type: "stat", label: "Verse", value: "raven.local" },
{ type: "stat", label: "Docker image", value: state.backend.dockerImage },
{ type: "stat", label: "NVIDIA runtime", value: state.backend.nvidiaRuntime },
{ type: "stat", label: "Kimodo", value: state.backend.kimodoHelp },
{ type: "stat", label: "Kimodo worker", value: state.backend.kimodoWorker },
{ type: "stat", label: "HF token", value: state.backend.huggingFaceToken },
{ type: "command", label: "Smoke", method: "GET", href: `${ravenBaseUrl}/smoke` },
{ type: "command", label: "Generate motion", method: "POST", href: `${ravenBaseUrl}/motion/generate` },
],
},
@ -377,59 +415,31 @@ async function persistSurfaces() {
}
async function persistCultCacheSurfaces() {
const cache = CultCache.builder()
.withRegistry(viliDocumentRegistry)
.withGenericStore(new SingleFileMessagePackBackingStore(path.join(args.stateRoot, viliStoreFileName)))
.build();
await cache.put(providerAdvertisementDefinition, providerId, providerAdvertisement());
await cache.put(operatorStateDefinition, "vili", operatorState());
await cache.put(eveSurfaceDefinition, providerId, eveSurface());
await cache.put(commandBoundaryDefinition, "vili", commandBoundary());
await cache.put(transportProfileDefinition, "vili", transportProfile());
await viliCache.put(providerAdvertisementDefinition, providerId, providerAdvertisement());
await viliCache.put(operatorStateDefinition, "vili", operatorState());
await viliCache.put(eveSurfaceDefinition, providerId, eveSurface());
await viliCache.put(commandBoundaryDefinition, "vili", commandBoundary());
await viliCache.put(transportProfileDefinition, "vili", transportProfile());
}
async function refreshBackendStatus({ includeKimodoHelp = false } = {}) {
const docker = await wsl([
"set -e",
"service docker start >/dev/null 2>&1 || true",
"docker images gamecult/kimodo:latest --format '{{.Repository}}:{{.Tag}}' | head -n 1",
].join("; "), { timeout: 20000 });
let nvidia = await wsl([
"set -e",
"service docker start >/dev/null 2>&1 || true",
"docker info --format '{{json .Runtimes}}' | grep -q nvidia && echo configured || echo missing",
].join("; "), { timeout: 20000 });
// A read: it only asks. Starting docker and the worker belongs to a job that holds the GPU slot.
async function refreshBackendStatus() {
const docker = await wsl("docker images gamecult/kimodo:latest --format '{{.Repository}}:{{.Tag}}' | head -n 1", { timeout: 20000 });
const nvidia = await wsl("docker info --format '{{json .Runtimes}}' | grep -q nvidia && echo configured || echo missing", { timeout: 20000 });
const token = await wsl("test -f /root/.cache/huggingface/token && echo present || echo missing", { timeout: 10000 });
const worker = await kimodoWorkerHealth(20000);
let help = { ok: true, stdout: backendStatus.kimodoHelp === "ok" ? "ok" : "not checked", stderr: "" };
if (includeKimodoHelp) {
nvidia = await wsl([
"set -e",
"service docker start >/dev/null 2>&1 || true",
"docker run --rm --runtime=nvidia --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi --query-gpu=name --format=csv,noheader | head -n 1",
].join("; "), { timeout: 45000 });
help = await wsl([
"set -e",
"service docker start >/dev/null 2>&1 || true",
dockerHuggingFaceEnv,
`docker run --rm --runtime=nvidia --gpus all ${dockerHuggingFaceArgs} gamecult/kimodo:latest kimodo_gen --help | sed -n '1,6p'`,
].join("; "), { timeout: 60000 });
}
backendStatus = {
checkedAt: new Date().toISOString(),
dockerImage: docker.ok && docker.stdout ? docker.stdout : "missing",
nvidiaRuntime: nvidia.ok && nvidia.stdout ? nvidia.stdout : "unavailable",
huggingFaceToken: token.ok ? token.stdout : "unknown",
kimodoHelp: includeKimodoHelp ? (help.ok ? "ok" : "failed") : backendStatus.kimodoHelp,
kimodoWorker: worker.ok && worker.body?.ok ? "resident" : "not running",
kimodoWorkerDetail: worker.body || null,
lastError: [docker, nvidia, token, help].find((result) => !result.ok)?.stderr || null,
lastError: [docker, nvidia, token].find((result) => !result.ok)?.stderr || null,
};
await persistSurfaces();
return { docker, nvidia, token, help, backendStatus };
return backendStatus;
}
async function kimodoWorkerHealth(timeout = 10000) {
@ -502,7 +512,43 @@ async function readRequestBody(request) {
return raw ? JSON.parse(raw) : {};
}
const loopbackAddresses = new Set(["127.0.0.1", "::1", "::ffff:127.0.0.1"]);
// A state-changing POST is a service's or the tray's call, never a web page's. A browser sends a
// cross-site page's POST as a no-preflight simple request (text/plain) or with an Origin that is not
// ours, so both are refused before the body is read. The daemon grants no CORS preflight at all.
function refuseCrossSite(request, response) {
const origin = request.headers.origin;
const ownOrigins = [`http://127.0.0.1:${args.port}`, `http://localhost:${args.port}`, `http://[::1]:${args.port}`];
const type = String(request.headers["content-type"] || "").split(";")[0].trim().toLowerCase();
if (type === "application/json" && (origin === undefined || ownOrigins.includes(origin))) return false;
json(response, 403, { ok: false, code: "cross-site" });
return true;
}
// The household's switch: only a caller on this machine (the tray, in the household's session) sets it.
async function setGpuPermission(request, response) {
if (refuseCrossSite(request, response)) return;
if (!loopbackAddresses.has(request.socket.remoteAddress)) {
json(response, 403, { ok: false, code: "household-only" });
return;
}
let body;
try {
body = await readRequestBody(request);
} catch (error) {
json(response, 400, { ok: false, error: `Invalid JSON: ${error.message}` });
return;
}
if (typeof body?.allowed !== "boolean") {
json(response, 400, { ok: false, error: "allowed must be a boolean." });
return;
}
json(response, 200, { ok: true, gpu: await gpuTenancy.setAllowed(body.allowed) });
}
async function generateMotion(request, response) {
if (refuseCrossSite(request, response)) return;
let body;
try {
body = await readRequestBody(request);
@ -526,6 +572,13 @@ async function generateMotion(request, response) {
return;
}
const admitted = await admitThen(gpuTenancy, () => runMotionJob(body, prompt, response));
if (admitted?.code === "gpu-busy") {
json(response, 503, admitted, { "retry-after": String(RETRY_AFTER_SECONDS) });
}
}
async function runMotionJob(body, prompt, response) {
const jobId = `vili-${Date.now()}`;
const artifactDir = path.join(args.stateRoot, "artifacts", jobId);
await mkdir(artifactDir, { recursive: true });
@ -650,23 +703,15 @@ async function generateMotion(request, response) {
await mkdir(path.join(args.stateRoot, "jobs"), { recursive: true });
await writeFile(path.join(args.stateRoot, "jobs", `${jobId}.json`), JSON.stringify(job, null, 2));
});
json(response, result.ok ? 200 : 502, job);
const placement = result.ok ? null : placementRefusal(result.body, jobId);
if (placement) json(response, 503, placement, { "retry-after": String(RETRY_AFTER_SECONDS) });
else json(response, result.ok ? 200 : 502, job);
}
async function handleRequest(request, response) {
const url = new URL(request.url, `http://${request.headers.host || `${args.host}:${args.port}`}`);
if (request.method === "OPTIONS") {
response.writeHead(204, {
"access-control-allow-origin": "*",
"access-control-allow-methods": "GET,POST,OPTIONS",
"access-control-allow-headers": "content-type",
});
response.end();
return;
}
const url = new URL(request.url, `http://${request.headers.host || `${hosts[0]}:${args.port}`}`);
if (url.pathname === "/" || url.pathname === "/health") {
await refreshBackendStatus({ includeKimodoHelp: false });
await refreshBackendStatus();
json(response, 200, { ok: true, service: "vili", providerId, state: operatorState() });
} else if (url.pathname === "/provider-advertisement") {
json(response, 200, providerAdvertisement());
@ -683,12 +728,10 @@ async function handleRequest(request, response) {
endpoint: `${ravenDeckUrl}/${providerId}`,
}],
});
} else if (url.pathname === "/smoke") {
const result = await refreshBackendStatus({ includeKimodoHelp: true });
const worker = await ensureKimodoWorker();
await refreshBackendStatus({ includeKimodoHelp: false });
const ok = result.backendStatus.kimodoHelp === "ok" && worker.ok && worker.body?.ok;
json(response, ok ? 200 : 502, { ok, ...result, worker: worker.body || worker });
} else if (url.pathname === "/gpu-permission" && request.method === "GET") {
json(response, 200, gpuTenancy.status());
} else if (url.pathname === "/gpu-permission" && request.method === "POST") {
await setGpuPermission(request, response);
} else if (url.pathname === "/motion/generate" && request.method === "POST") {
await generateMotion(request, response);
} else {
@ -697,8 +740,7 @@ async function handleRequest(request, response) {
}
async function healthCheck() {
const host = args.host === "0.0.0.0" ? "127.0.0.1" : args.host;
const response = await fetch(`http://${host}:${args.port}/health`);
const response = await fetch(`http://${hosts[0]}:${args.port}/health`);
if (!response.ok) throw new Error(`health returned ${response.status}`);
const body = await response.json();
if (!body.ok) throw new Error("health returned ok=false");
@ -715,15 +757,13 @@ if (args.health) {
process.exit(1);
});
} else {
if (hosts.length === 0 || hosts.includes("0.0.0.0") || hosts.includes("::")) {
console.error("--host must name the addresses to listen on, never a wildcard.");
process.exit(2);
}
await mkdir(path.join(args.stateRoot, "jobs"), { recursive: true });
await persistSurfaces();
const server = http.createServer((request, response) => {
handleRequest(request, response).catch((error) => {
json(response, 500, { ok: false, error: error.message, stack: process.env.NODE_ENV === "development" ? error.stack : undefined });
});
});
const wss = new WebSocketServer({ noServer: true });
wss.on("connection", (socket) => {
const sendSurface = () => {
@ -734,18 +774,27 @@ if (args.health) {
socket.on("close", () => clearInterval(timer));
});
server.on("upgrade", (request, socket, head) => {
const url = new URL(request.url, `http://${request.headers.host || `${args.host}:${args.port}`}`);
if (url.pathname === `/eve/deck/${providerId}` || url.pathname === "/eve/deck") {
wss.handleUpgrade(request, socket, head, (ws) => wss.emit("connection", ws, request));
} else {
socket.destroy();
}
});
server.listen(args.port, args.host, () => {
console.log(`Vili listening on http://${args.host}:${args.port}`);
const servers = hosts.map((host) => {
const server = http.createServer((request, response) => {
handleRequest(request, response).catch((error) => {
json(response, 500, { ok: false, error: error.message, stack: process.env.NODE_ENV === "development" ? error.stack : undefined });
});
});
server.on("upgrade", (request, socket, head) => {
const url = new URL(request.url, `http://${request.headers.host || `${hosts[0]}:${args.port}`}`);
if (url.pathname === `/eve/deck/${providerId}` || url.pathname === "/eve/deck") {
wss.handleUpgrade(request, socket, head, (ws) => wss.emit("connection", ws, request));
} else {
socket.destroy();
}
});
return new Promise((resolve, reject) => {
server.once("error", reject);
server.listen(args.port, host, resolve);
});
});
await Promise.all(servers);
console.log(`Vili listening on ${hosts.map((host) => `http://${host}:${args.port}`).join(" ")}`);
setInterval(() => persistSurfaces().catch((error) => console.error(error)), 5000);
}

98
scripts/vili-gpu-tray.ps1 Normal file
View file

@ -0,0 +1,98 @@
# The household's switch for Raven's GPU: one tray icon in the interactive session. It shows whether
# Vili may use the GPU and flips it with one click. It holds no state of its own: the icon and tooltip
# are a lowering of GET /gpu-permission, and the click is one POST with an absolute value.
param([int]$Port = 8824, [int]$PollMs = 5000)
Add-Type -AssemblyName System.Windows.Forms
Add-Type -AssemblyName System.Drawing
$mutex = New-Object System.Threading.Mutex($false, 'Local\GameCult.ViliGpuTray')
if (-not $mutex.WaitOne(0)) { exit 0 }
$base = "http://127.0.0.1:$Port/gpu-permission"
$script:state = 'down'
$script:jobRunning = $false
$clock = [Diagnostics.Stopwatch]::StartNew()
$script:lastToggleMs = -10000
function New-TrayIcon([string]$kind) {
$bmp = New-Object System.Drawing.Bitmap 16, 16
$g = [System.Drawing.Graphics]::FromImage($bmp)
$g.SmoothingMode = 'AntiAlias'
if ($kind -eq 'on') {
$g.FillEllipse([System.Drawing.Brushes]::LimeGreen, 1, 1, 13, 13)
} elseif ($kind -eq 'off') {
$g.FillEllipse([System.Drawing.Brushes]::Crimson, 1, 1, 13, 13)
$g.FillRectangle([System.Drawing.Brushes]::White, 4, 7, 8, 2)
} else {
$pen = New-Object System.Drawing.Pen ([System.Drawing.Color]::Gray), 2
$g.DrawEllipse($pen, 2, 2, 11, 11)
$pen.Dispose()
}
$g.Dispose()
$icon = [System.Drawing.Icon]::FromHandle($bmp.GetHicon())
return $icon
}
$icons = @{ on = (New-TrayIcon 'on'); off = (New-TrayIcon 'off'); down = (New-TrayIcon 'down') }
$tray = New-Object System.Windows.Forms.NotifyIcon
$menu = New-Object System.Windows.Forms.ContextMenuStrip
$item = $menu.Items.Add('Let Vili use the GPU')
$tray.ContextMenuStrip = $menu
function Show-State {
$tray.Icon = $icons[$script:state]
$item.Checked = ($script:state -eq 'on')
$item.Enabled = ($script:state -ne 'down')
if ($script:state -eq 'on' -and $script:jobRunning) { $tray.Text = 'Vili may use the GPU (making a clip). Click to stop.' }
elseif ($script:state -eq 'on') { $tray.Text = 'Vili may use the GPU. Click to stop.' }
elseif ($script:state -eq 'off') { $tray.Text = 'Vili may not use the GPU. Click to allow.' }
else { $tray.Text = 'Vili is not running, so it is not using the GPU.' }
}
function Update-State {
try {
$r = Invoke-RestMethod -Uri $base -Method Get -TimeoutSec 2
if ($r.allowed -eq $true) { $script:state = 'on' } elseif ($r.allowed -eq $false) { $script:state = 'off' } else { $script:state = 'down' }
$script:jobRunning = ($r.jobRunning -eq $true)
} catch {
$script:state = 'down'
$script:jobRunning = $false
}
Show-State
}
function Switch-Permission {
try {
if ($clock.ElapsedMilliseconds - $script:lastToggleMs -lt 1000) { return }
$script:lastToggleMs = $clock.ElapsedMilliseconds
if ($script:state -eq 'down') {
$tray.ShowBalloonTip(3000, 'Vili', 'Vili is not running. Nothing to change.', 'Info')
return
}
$body = ConvertTo-Json -Compress @{ allowed = ($script:state -ne 'on') }
try {
[void](Invoke-RestMethod -Uri $base -Method Post -Body $body -ContentType 'application/json' -TimeoutSec 2)
} catch {
$tray.ShowBalloonTip(3000, 'Vili', 'Could not reach Vili. Nothing changed.', 'Warning')
}
Update-State
} catch {
}
}
$tray.Add_MouseUp({
param($sender, $e)
try { if ($e.Button -eq [System.Windows.Forms.MouseButtons]::Left) { Switch-Permission } } catch {}
})
$item.Add_Click({ try { Switch-Permission } catch {} })
$timer = New-Object System.Windows.Forms.Timer
$timer.Interval = $PollMs
$timer.Add_Tick({ try { Update-State } catch {} })
Update-State
$tray.Visible = $true
$timer.Start()
[System.Windows.Forms.Application]::Run()