diff --git a/README.md b/README.md index 801ff26..6dfe397 100644 --- a/README.md +++ b/README.md @@ -10,8 +10,8 @@ first backend is Kimodo on Raven, running through Docker Engine inside - **Value:** a warm, supervised path from a text prompt to generated body motion. Vili runs Kimodo as a resident GPU worker on Raven and returns motion artifacts (`motion.npz`, optional BVH) for a job. - **Problem:** Personas and games need generated motion, but cold per-request container runs took about 195 seconds per 1 second clip, versus about 54 warm. One daemon owns job identity, worker lifecycle and artifact references so callers do not touch Docker, WSL or model state. -- **Entry points:** `scripts/vili-daemon.mjs` (the daemon; `npm start`, `npm run health`, `npm run smoke`, `npm run provider`), `scripts/kimodo-resident-worker.py` (the resident worker), `POST /motion/generate` (see Motion Generation), `docs/verse-service-contract.md` and `docs/motion-pipeline-target.md`. -- **Live:** the resident Kimodo worker on Raven, `POST /motion/generate`, `GET /smoke`, and the CultCache service store. It runs under a Raven scheduled task, not under Idunn. It has `motion.npz` and optional BVH output on the SOMA skeleton; there is no FBX export. +- **Entry points:** `scripts/vili-daemon.mjs` (the daemon; `npm start`, `npm run health`, `npm run provider`), `scripts/kimodo-resident-worker.py` (the resident worker), `POST /motion/generate` (see Motion Generation), `docs/verse-service-contract.md` and `docs/motion-pipeline-target.md`. +- **Live:** the resident Kimodo worker on Raven, `POST /motion/generate`, and the CultCache service store. It runs under a Raven scheduled task, not under Idunn. It has `motion.npz` and optional BVH output on the SOMA skeleton; there is no FBX export. - **Planned (target, 2026-10-20):** FBX animation export on the SOMA skeleton and a text-prompt-to-Unity-clip path with no hand steps, with each artifact recording prompt, constraints, seed and model. Retargeting stays with the consuming engine; see `docs/motion-pipeline-target.md`. ## Authority @@ -41,6 +41,11 @@ Default compatibility endpoint: http://10.77.0.4:8824 ``` +The daemon listens only on the addresses `--host` names, comma-separated, one server each: on +Raven `127.0.0.1` (the tray and `--health`) and `10.77.0.4` (the WireGuard peers: Starfire and +the services that call `POST /motion/generate`). A wildcard such as `0.0.0.0` refuses to start. +No response carries a CORS header, so a web page cannot read the daemon either; no browser +caller exists. Vili is not an Idunn target today. Its lifecycle is the `GameCult\Vili` scheduled task on Raven, and `GET /health` is an operator lowering. Admission to Idunn is motion-pipeline:question:vili-under-idunn. @@ -63,7 +68,6 @@ Important routes: - `GET /eve/operator` - `GET /eve/deck/providers` - `WS /eve/deck/vili.animation` -- `GET /smoke` - `POST /motion/generate` ## Motion Generation @@ -145,8 +149,8 @@ Vili keeps Kimodo warm through a resident worker container named the loaded model. Because Raven's Docker Engine runs inside WSL, Vili also keeps a tiny WSL sleep-loop alive while the worker is resident so the Docker substrate does not disappear underneath the model. `GET /health` observes -worker state without launching model load. `GET /smoke` verifies the backend -and starts the resident worker when it is absent. +worker state without launching model load and without changing anything. Only a +`POST /motion/generate` that holds the GPU slot starts the worker. Kimodo's 8B text encoder (LLM2Vec, Llama 3) never lives in the worker. The worker answers Kimodo's encoder call from a CultCache store of float32 vectors, @@ -159,12 +163,25 @@ Kimodo normalizes the prompt before encoding (`A person shrugs sadly.`), and the cache keys on that text. A request's misses go to one transient child, `scripts/kimodo-text-encode.py`: it loads the encoder in bf16 on the GPU, with the layers that do not fit in free VRAM held in RAM, encodes, writes its answer and -exits, so the OS reclaims its RAM and VRAM. It refuses with a typed `gpu-busy` -below 2 GiB of free VRAM and has no CPU fallback; hits never touch it. Each +exits, so the OS reclaims its RAM and VRAM. It sizes its own placement from the free VRAM torch +can see and refuses a placement below 2 GiB with a typed `gpu-busy` (source `placement`); +it has no CPU fallback, and hits never touch it. Each `/motion/generate` job carries the encoder record (hits, misses, seconds, vector hashes) and the worker identity. `GET /health` shows the identity and the stored count. +GPU permission. Raven's GPU is the household's first. A tray icon in the household's session +(`scripts/vili-gpu-tray.ps1`, logon task `GameCult\ViliTray`, installed by +`scripts/install-vili-tray-task.ps1`) shows whether Vili may use the GPU and flips it with one click. +The daemon owns the yes or no in `vili.service.cc` (`gamecult.vili.gpu_permission.v1`): off on a +fresh install, set only from loopback (`POST /gpu-permission { allowed }`, read at +`GET /gpu-permission`). A `POST /motion/generate` is refused with HTTP 503, `retry-after: 60` and +`{ ok: false, code: "gpu-busy", source }` where `source` is `household` (the permission is off), +`slot` (another job holds the one GPU slot) or `vram` (host `nvidia-smi.exe` free VRAM is below +`VILI_GPU_MIN_FREE_MIB`, default 10240, or unreadable); nothing queues, and the consumer owns the +retry. A job already running finishes; the gate never stops work. `operator_state.gpu` is +`{ allowed, changedAt, jobRunning, lastRefusal }`. `npm test` runs the gate's tests. + Measured on Raven on `2026-10-22`, for a 2 second clip at 20 diffusion steps: the worker is healthy 29 seconds after `docker rm` (was 179) and rests at 2.3 GiB resident (was 10.4 GiB, with 5.8 GB swapped). A cache miss takes 59 to 103 diff --git a/docs/motion-pipeline-map.md b/docs/motion-pipeline-map.md index f79792b..c7ec160 100644 --- a/docs/motion-pipeline-map.md +++ b/docs/motion-pipeline-map.md @@ -7,8 +7,33 @@ cuts not yet mapped. The target's rationale is `docs/motion-pipeline-target.md`. ## Body facts +### Raven tray-toggle probes, 2026-10-22 23:30 to 23:36 UTC + +Run read-only from Starfire over `ssh raven` (session 0, elevated), one +PowerShell 5.1 script copied to `C:\Users\Public` and deleted in the same +command. Nothing on Raven was changed: no task edits, no restarts, no GPU jobs. +They inform cut spec `motion-pipeline:cut_spec:cut-raven-gpu-tenancy-gate.r2`, +which replaces detection with the household's toggle +(`motion-pipeline:ruling:raven-gpu-tray-toggle`). + +| Probe | Result | +|---|---| +| Who the household is on Raven | One account. `query user`: `madman's lullaby`, console, session 1, Active since 2026-10-18. The `\GameCult\Vili` task runs as that same user (`LogonType Interactive`, `RunLevel Limited`), so a tray process at that user's logon is in the same session and the same user as the daemon, with no admin rights. | +| Daemon listen address | `node.exe ... vili-daemon.mjs --host 0.0.0.0 --port 8824 --state-root E:\Projects\Vili\.vili`, pid 33776, session 1, at detached head `a2b9983`. Listens on `0.0.0.0:8824`, so the WireGuard peers can reach every route; a write route must check the caller itself. `GET http://127.0.0.1:8824/operator-state` from Raven answers in 3 ms. | +| Live task action versus the installer | The live `\GameCult\Vili` action passes `-IdunnRudpHealth 10.77.0.2:17870 -IdunnDaemon vili -IdunnHealthContract ... -IdunnHealthIntervalSeconds 15` to `start-vili-daemon.ps1`; `scripts/install-vili-task.ps1` does not. Rerunning that installer would rewrite the live task, so the tray gets its own installer. | +| Tray substrate | `powershell.exe` 5.1.26100, apartment `STA` by default; `System.Windows.Forms` and `System.Drawing` load (`v4.0.30319`, .NET Framework release 534296). Execution policy is Undefined at every scope (the task launcher passes `-ExecutionPolicy Bypass`). `scripts/run-hidden-powershell.vbs` already starts a hidden PowerShell with no console flash. No new dependency is needed. | +| Tray overflow | `HKCU:\Control Panel\NotifyIconSettings` exists with 47 entries: Windows 11 records each notify icon there and puts new ones in the overflow until the user promotes it. A fresh tray icon will not be on the taskbar until the household drags it there once (or turns it on under Settings, Personalization, Taskbar, Other system tray icons). | +| Startup folder | `C:\Users\Madman's Lullaby\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup`. Not used: from SSH (session 0) nothing in it can be started in session 1, while `Start-ScheduledTask` on an Interactive task can. | +| Scheduled tasks under `\GameCult\` | `Backup-Yggdrasil-Pull`, `ComfyUI`, `Vili`, all Ready. ComfyUI is a second possible heavy GPU consumer on Raven; the toggle governs Vili only. | +| Host free VRAM | `C:\WINDOWS\system32\nvidia-smi.exe --query-gpu=memory.free --format=csv,noheader,nounits`: 159 ms from a cold PowerShell, exit 0, free 5001 MiB at 23:3x UTC (the household or another consumer held about 11 GiB). No `nvidia-smi.exe` process was left running. | +| Vili's store | `E:\Projects\Vili\.vili\vili.service.cc`, 11810 bytes. Newest job file `vili-1792710792764.json`, written 23:20:12 UTC. The live gate (`a2b9983`) read `inUse true, probeOk true`. | +| cultcache-ts on Raven | `E:\Projects\CultLib` at `b9497d35`, package `0.14.1`; its build carries `pullAllBackingStores`. `SingleFileMessagePackBackingStore.push` reads the file, replaces one `(type, key)` entry and writes all, inside a write queue that belongs to one store instance. | + ### Raven GPU tenancy probes, 2026-10-22 20:15 to 20:23 UTC +These informed r1's detection gate, which r2 deletes. The session, principal and +`nvidia-smi` rows still hold. + Run read-only from Starfire over `ssh -o HostName=192.168.178.165 raven` (alias in `~/.ssh/config`; LAN address per `gamecult-ops/inventory.md`), as `powershell -NoProfile -NonInteractive -EncodedCommand`. Nothing on Raven was @@ -21,56 +46,83 @@ category (compositor, WSL family, or "other"). | Session of the Vili daemon | `node.exe ... vili-daemon.mjs --host 0.0.0.0 --port 8824` runs in **session 1**, the console session of `madman's lullaby` (`query user`: console, ID 1, Active). It is the interactive desktop session, not session 0. | | Vili task principal | `\GameCult\Vili`: `LogonType Interactive`, `RunLevel Limited`, boot trigger (`MSFT_TaskBootTrigger`), last run 2026-10-22 21:18:55 local, result 0. Children the daemon spawns inherit session 1. | | SSH context | Windows `sshd` runs commands in **session 0**, elevated (`IsInRole(Administrator)` true). `gamecult-ops/runbooks/idunn-host-raven.md` already records that sshd cannot start anything in the user's session without a scheduled-task hop. | -| `SHQueryUserNotificationState` from session 0 | `hr=0`, state `2` (`QUNS_BUSY`) on every call, with the GPU idle (3D engine total 0.01 to 0.02 %). Session 0 has no shell, so this value says nothing about the user's desktop. The call must run in session 1, which is where the daemon and its children are. It was **not** observed from session 1: doing that without a task edit is impossible from SSH. | -| `nvidia-smi.exe` | Present at `C:\WINDOWS\system32\nvidia-smi.exe`. `--query-gpu=memory.total,memory.used,memory.free,utilization.gpu`: `16380, 1592, 14516, 2` MiB/%. 63 ms cold, about 49 ms per call (10 calls 491 ms). Per-process `--query-compute-apps` returns `[N/A]` memory under WDDM. | -| `\GPU Engine(*engtype_3D)\Utilization Percentage` | Readable from session 0 as admin; 115 instances. `Get-Counter -SampleInterval 1 -MaxSamples 1` takes 1.08 s wall, 16 ms CPU warm, 328 ms CPU on first use in a process. Summed with `Measure-Object` it needs no instance names. Not tried as a Limited token in session 1. | -| `\GPU Adapter Memory(*)\Dedicated Usage` | Readable; 1.0 s wall; 1597 MiB on the RTX 4060 Ti, 0 on the Parsec virtual adapter. | -| `\GPU Process Memory(*)\Dedicated Usage` | Readable; 1.0 s wall. Two instances above 50 MiB: the desktop compositor (1643 MiB, session 1) and one session-0 process outside the WSL family (1405 MiB). Per-process figures overlap: they sum above the adapter total. | -| WSL attribution | The Kimodo worker container was resident (`/health` ok, uptime 3454 s, 6 generations) while `vmmemWSL` (and `vmwp`) had **no** `GPU Process Memory` instance. Host counters do not attribute WSL CUDA memory to a nameable host process. WSL `nvidia-smi` reported the same `1592 MiB used, 14516 MiB free` as the host. | -| PowerShell cost | `powershell.exe` 5.1 cold start 266 ms CPU, `Add-Type` for the shell32 call 78 ms CPU, first counter read 328 ms CPU: about 0.7 s CPU per cold spawn. A warm loop iteration (QUNS + `nvidia-smi` + one 3D counter read) is about 1.1 s wall and 16 ms of its own CPU. | +| `SHQueryUserNotificationState` from session 0 | `hr=0`, state `2` (`QUNS_BUSY`) on every call, with the GPU idle. Session 0 has no shell, so this value says nothing about the user's desktop. | +| `nvidia-smi.exe` | Present at `C:\WINDOWS\system32\nvidia-smi.exe`. `--query-gpu=memory.total,memory.used,memory.free,utilization.gpu`: `16380, 1592, 14516, 2` MiB/%. 63 ms cold, about 49 ms per call. Per-process `--query-compute-apps` returns `[N/A]` memory under WDDM. | +| `\GPU Engine(*engtype_3D)\Utilization Percentage` | Readable from session 0 as admin; 115 instances; `Get-Counter` 1.08 s wall. | +| `\GPU Process Memory(*)\Dedicated Usage` | Per-process figures overlap and sum above the adapter total. | +| WSL attribution | Host counters do not attribute WSL CUDA memory to a nameable host process. Host `nvidia-smi.exe` does see a WSL container's allocation: cut report h1 measured free falling 14292 to 1866 MiB under a 12 GiB holder. | | Adapters | `NVIDIA GeForce RTX 4060 Ti` driver 32.0.16.1088, 16 GiB; `Parsec Virtual Display Adapter`. | -What follows from these, and what does not: +### What the detection gate taught (r1, h1, h2, s1, s2) -- The daemon's own session sees the user's desktop, so the tenancy probe runs as - a child of the daemon. No fork about where it runs: the scheduled task already - puts it in the right session. -- Free VRAM must come from the host (`nvidia-smi.exe`), not from torch in WSL: - Soul's s1 probe on cut `vili-encoder-placement` showed `mem_get_info` inside a - container blind to another container's 12 GiB. Whether host `nvidia-smi` sees - a WSL container's allocation is **unverified** (the resident worker's share - could not be separated from the compositor's); the gate cut checks it first. -- The 3D-engine signal cannot be used to tell Vili's own work from the - household's by process, and need not be: Kimodo runs on CUDA (compute - engines), games on the 3D engine. Summing only `engtype_3D` leaves Vili out. -- A long-lived probe child costs about 1 ms of CPU per second at a 15 s period; - a fresh PowerShell per sample would cost about 45 ms per second. Neither - loads the GPU. +- Host free VRAM cannot tell Vili's encoder child from a household load: the + child sizes itself to the VRAM that was free (s1 `self-tenancy-refusal`). + Discounting Vili's own VRAM during a job blinds the gate to a household load + that starts mid-job (s2 `hold-blinds-household-vram`). +- Live, the probe child went silent during 2 of 5 of Vili's own encoder jobs and + the gate stayed stale after the job (s2 `own-load-starves-probe`; Raven had + 770 MB of host RAM available at one stall). Every repair added a watchdog, a + drain or a baseline, and each brought its own finding. +- After a job ends, host free VRAM is back at 14.6 to 15.2 GiB at the next + 2-second sample (s2), so one `nvidia-smi` read before a job, with no job + holding the slot, sees the card as the household left it. ## Model page | Kind | Named by | Over time | Decided by | |---|---|---|---| -| GPU tenancy state (`gpu` in `gamecult.vili.operator_state.v1`) | the daemon, one per host | rewritten every 5 s with the operator state; holds `inUse`, `since`, `checkedAt`, `probeOk` and nothing else | the daemon's tenancy gate, from host samples; never persisted as truth, rebuilt from samples at start | +| GPU permission (`gamecult.vili.gpu_permission.v1` in `.vili/vili.service.cc`, key `raven`) | the daemon, one per host | written only when the household changes it, before the change is acknowledged; read once at daemon start; absent means not allowed | the household, through the tray, through the daemon's loopback-only `POST /gpu-permission`; the daemon is its one writer | +| GPU status (`gpu` in `gamecult.vili.operator_state.v1`) | the daemon | rewritten every 5 s with the operator state; `{ allowed, changedAt, jobRunning, lastRefusal }` | derived from the permission, the slot and the last refusal; display only, never read back | | Motion job (`.vili/jobs/.json`) | `vili-` | written once when a job ends | the daemon; a job refused at admission writes none | | Text embedding (`.vili/embeddings/text-embeddings.cc`) | `sha256(identity, text)` | append on miss | the worker's cache | ## Rationale -- **Refuse, do not queue (recommended, pending the operator).** `/motion/generate` - is synchronous today. Holding a request open for as long as a game runs, or - adding a persisted job queue, are bigger machines than the one consumer needs: - RPS clip generation is a script an agent runs, and it can retry a typed - `gpu-busy`. A queue belongs with the authoring UX's takes, when that target - exists (`motion-pipeline:follow_up:vili-authoring-ux-target-2`). -- **Fail closed.** If the probe child dies or stops answering, the gate reports - in use. A wrongly deferred clip costs minutes; a wrongly admitted encoder child - pages a game's VRAM and thrashes Raven. +- **The household decides; nothing detects.** The operator replaced detection + with a tray toggle (`motion-pipeline:ruling:raven-gpu-tray-toggle`). The + detection gate's findings were the cost of guessing intent from counters; a + yes from the person at the desk needs no guess, no hysteresis and no probe. +- **The daemon is the one owner of the permission; the tray is a lowering.** + The tray reads and sets the state only through the daemon's HTTP route, so + admission and the stored value cannot disagree and the tray holds no state of + its own. The alternative, the tray writing a CultCache document the daemon + reads, would give the document two writers on two clocks and make the daemon + poll a file to learn its own admission rule. +- **Off on a fresh install, and off when the document is missing.** The safe + default for the household: Vili does nothing on Raven's GPU until someone at + the desk says yes. A stored yes survives restarts and reboots. +- **Loopback only for the write.** The daemon listens on `0.0.0.0`; the + household's switch must not be flippable from a WireGuard peer. A request + from a non-loopback address is refused 403. Agents read the state through + `GET /gpu-permission` or `/operator-state` and never set it. +- **One cache instance for the daemon.** The 5-second surface writer built a + new store per call. Two store instances writing one file each read, modify and + write in their own queue, so a permission write could be lost under a + concurrent surface write. One module-level cache serializes every write to + `vili.service.cc`. +- **One slot, one fresh VRAM read, fail closed.** h2's slot stays: one job at a + time, the rest refused at once with source `slot`. The slot is taken + synchronously before the VRAM read, so two requests cannot both pass it. A + failed or unparseable read refuses with source `vram`, so a job never starts + onto a full card on a guess. The permission is checked again after the read. +- **Refuse, do not queue** (`motion-pipeline:ruling:gpu-defer-refuse-typed`). + `/motion/generate` is synchronous; the consumer retries a typed `gpu-busy` + after `retry-after`. A queue belongs with the authoring UX's takes + (`motion-pipeline:follow_up:vili-authoring-ux-target-2`). +- **A logon task, not the Startup folder.** `Start-ScheduledTask` on an + Interactive task puts the tray in the household's session from SSH at deploy + time; a Startup folder entry would wait for the next logon. The task pattern + matches `\GameCult\Vili`. +- **The privacy rule is moot.** "Never record program names" protected the + household from a probe that watched its programs. With no probe, nothing in + Vili observes the household's programs; the only household fact Vili holds is + the yes or no the household chose. - **The child's own VRAM check is placement, not tenancy.** `plan_placement` - still sizes accelerate's `max_memory` from torch's view and can still refuse; - it answers "can the encoder be placed", not "may heavy work start". + still sizes accelerate's `max_memory` from torch's view and can still refuse + (source `placement`); it answers "can the encoder be placed", not "may heavy + work start". ## Cuts not yet mapped - RPS beats through Vili (`motion-pipeline:follow_up:rps-clips-from-vili`), after - the gate and `vili-fbx-export`. + the gate and `vili-fbx-export`. They wait on the household's yes on Raven. diff --git a/docs/verse-service-contract.md b/docs/verse-service-contract.md index 25bb563..abb4ca7 100644 --- a/docs/verse-service-contract.md +++ b/docs/verse-service-contract.md @@ -45,7 +45,22 @@ promoted from JSON once that request/receipt shape stabilizes. `POST /motion/generate` accepts an explicit animation request and forwards the supported Kimodo generation controls to the resident worker. Vili owns job identity, request persistence, worker lifecycle, timeout handling, and the -artifact stem. Kimodo owns synthesis and model-specific export. +artifact stem. Kimodo owns synthesis and model-specific export. Vili also owns GPU +admission from the household's permission: the daemon holds one GPU slot, and `POST /motion/generate` is judged +when it reaches the slot. It answers HTTP 503 with `retry-after: 60` and one of four bodies, by +`source`: `{ "ok": false, "code": "gpu-busy", "source": "household", "gpu": { allowed: false, +changedAt } }` while the household's permission is off; `{ "ok": false, "code": "gpu-busy", +"source": "slot" }` while another job holds the slot (nothing queues); `{ "ok": false, "code": +"gpu-busy", "source": "vram", freeMiB, minFreeMiB }` when host free VRAM is below the minimum or +unreadable (`freeMiB` null); and `{ "ok": false, "code": "gpu-busy", "source": "placement", +"jobId" }` when the encoder child refused its own placement, where the job file was written and no +household status is claimed. `GET /gpu-permission` answers `{ allowed, changedAt, jobRunning, +lastRefusal }` to anyone; `POST /gpu-permission { "allowed": boolean }` is the household's switch and +answers 403 `{ "ok": false, "code": "household-only" }` to any caller that is not on the host's +loopback. Both POST routes require `Content-Type: application/json` and refuse a request whose +`Origin` header is present and not the daemon's own loopback origin with 403 `{ "ok": false, "code": +"cross-site" }`, so a web page cannot drive them; the daemon answers no CORS preflight. The `code` field is the only carrier; a consumer retries after `retry-after`, never sets +the permission, and never matches error text. A job already running finishes. ```json { @@ -143,7 +158,7 @@ It mounts: Inside the worker, Kimodo owns synthesis. Vili owns worker lifecycle, request/response persistence, timeout handling, and artifact references. Direct -user requests, programmatic requests, and smoke-started warmup all use the same +user requests and programmatic requests use the same resident worker health and generation path. Observed on `2026-06-12`: first worker model load took about 100 seconds. Once @@ -154,5 +169,5 @@ resident, five 1 second / 20 step service-path generations averaged about Health checks must be cheap. They may verify process, port, backend metadata, and cached readiness, but they must not launch a Kimodo generation job or start -model load. Smoke checks may start the resident worker because their purpose is -backend readiness, not cheap liveness. +model load. They change nothing: no GET route starts the worker, Docker or the GPU. Only a +`POST /motion/generate` that holds the GPU slot starts the worker. diff --git a/package.json b/package.json index 25f6a94..5f67561 100644 --- a/package.json +++ b/package.json @@ -7,7 +7,8 @@ "scripts": { "start": "node scripts/vili-daemon.mjs", "health": "node scripts/vili-daemon.mjs --health", - "smoke": "node scripts/smoke-vili.mjs", + "test": "node --test scripts/test-gpu-tenancy.mjs", + "test:http": "node --test scripts/test-gpu-gate-http.mjs", "provider": "node scripts/vili-daemon.mjs --print-provider-advertisement" }, "dependencies": { diff --git a/scripts/gpu-tenancy.mjs b/scripts/gpu-tenancy.mjs new file mode 100644 index 0000000..7eabccd --- /dev/null +++ b/scripts/gpu-tenancy.mjs @@ -0,0 +1,103 @@ +// GPU tenancy gate: the one owner of "may heavy GPU work start now". The household decides, through +// the permission it sets at the tray; the daemon owns that yes or no, the one GPU slot, and one host +// free-VRAM read per admission. The gate refuses new work; it never stops work. +// +// acquire() judges in a fixed order and nothing waits: the permission, then the slot, then host free +// VRAM. The slot is taken before the VRAM read so concurrent requests are refused at once. + +export const RETRY_AFTER_SECONDS = 60; + +export function createGpuTenancy({ now, permission, savePermission, readFreeMiB, minFreeMiB = 10240 }) { + let allowed = permission?.allowed === true; + let changedAt = permission?.changedAt ?? null; + let held = false; + let lastRefusal = null; + + const status = () => ({ allowed, changedAt, jobRunning: held, lastRefusal }); + + function refuse(source, extra = {}) { + lastRefusal = { at: new Date(now()).toISOString(), source }; + return { refusal: { ok: false, code: "gpu-busy", source, ...extra } }; + } + const household = () => refuse("household", { gpu: { allowed: false, changedAt } }); + + async function setAllowed(next) { + if (typeof next !== "boolean") throw new TypeError("allowed must be a boolean"); + if (next === allowed) return status(); + const value = { allowed: next, changedAt: new Date(now()).toISOString() }; + await savePermission(value); + allowed = value.allowed; + changedAt = value.changedAt; + return status(); + } + + async function acquire() { + if (!allowed) return household(); + if (held) return refuse("slot"); + held = true; + let freeMiB; + try { + freeMiB = await readFreeMiB(); + } catch { + freeMiB = null; + } + if (!allowed) { + held = false; + return household(); + } + if (!Number.isFinite(freeMiB) || freeMiB < minFreeMiB) { + held = false; + return refuse("vram", { freeMiB: Number.isFinite(freeMiB) ? freeMiB : null, minFreeMiB }); + } + let released = false; + return { + release() { + if (released) return; + released = true; + held = false; + }, + }; + } + + return { status, setAllowed, acquire }; +} + +// Takes the slot, runs the job to its end and returns its result, or returns the typed refusal +// without calling the job. The gate never holds, pauses or cancels a job. +export async function admitThen(gate, job) { + const slot = await gate.acquire(); + if (slot.refusal) return slot.refusal; + try { + return await job(); + } finally { + slot.release(); + } +} + +// nvidia-smi's first line is the free MiB as a base-10 integer; anything else is null. +export function parseFreeMiB(stdout) { + const line = String(stdout).split("\n")[0].trim(); + return /^\d+$/.test(line) ? Number(line) : null; +} + +// The encoder child's placement refusal reaches the daemon as a typed code on the worker's error body; +// it is never read out of error text. Placement is the child's own view of free VRAM, not the +// household gate, so the body carries no gate status. Returns the typed 503 body, or null for any +// other outcome. +export function placementRefusal(workerBody, jobId) { + if (workerBody?.code !== "gpu-busy") return null; + return { ok: false, code: "gpu-busy", source: "placement", jobId }; +} + +// Operator override. A malformed value would silently defeat a comparison (NaN is never below +// anything), so it refuses to start instead, naming the variable and never its value. +export function gpuTenancyOptionsFromEnv(env) { + const options = {}; + const raw = env.VILI_GPU_MIN_FREE_MIB; + if (raw !== undefined) { + const value = raw.trim() === "" ? Number.NaN : Number(raw); + if (!Number.isFinite(value) || value < 0) throw new Error("VILI_GPU_MIN_FREE_MIB must be a finite non-negative number"); + options.minFreeMiB = value; + } + return options; +} diff --git a/scripts/install-vili-task.ps1 b/scripts/install-vili-task.ps1 index f00b14a..10cd66a 100644 --- a/scripts/install-vili-task.ps1 +++ b/scripts/install-vili-task.ps1 @@ -1,7 +1,7 @@ param( [string]$TaskName = "GameCult\Vili", [int]$Port = 8824, - [string]$HostName = "0.0.0.0", + [string]$HostName = "127.0.0.1,10.77.0.4", [string]$StateRoot = "E:\Projects\Vili\.vili" ) diff --git a/scripts/install-vili-tray-task.ps1 b/scripts/install-vili-tray-task.ps1 new file mode 100644 index 0000000..0dfb881 --- /dev/null +++ b/scripts/install-vili-tray-task.ps1 @@ -0,0 +1,17 @@ +param( + [string]$TaskName = "GameCult\ViliTray", + [int]$Port = 8824 +) + +$ErrorActionPreference = "Stop" +$ProgressPreference = "SilentlyContinue" +$HiddenLauncher = Join-Path $PSScriptRoot "run-hidden-powershell.vbs" +$TrayScript = Join-Path $PSScriptRoot "vili-gpu-tray.ps1" +$UserId = [System.Security.Principal.WindowsIdentity]::GetCurrent().Name +$Action = New-ScheduledTaskAction -Execute "wscript.exe" -Argument "//B //nologo `"$HiddenLauncher`" -NoProfile -NonInteractive -WindowStyle Hidden -ExecutionPolicy Bypass -STA -File `"$TrayScript`" -Port $Port" +$Trigger = New-ScheduledTaskTrigger -AtLogOn -User $UserId +$Principal = New-ScheduledTaskPrincipal -UserId $UserId -LogonType Interactive -RunLevel Limited +$Settings = New-ScheduledTaskSettingsSet -AllowStartIfOnBatteries -DontStopIfGoingOnBatteries -ExecutionTimeLimit ([TimeSpan]::Zero) -MultipleInstances IgnoreNew -Hidden + +Register-ScheduledTask -TaskName $TaskName -Action $Action -Trigger $Trigger -Principal $Principal -Settings $Settings -Force | Out-Null +Write-Host "Installed scheduled task $TaskName" diff --git a/scripts/kimodo-resident-worker.py b/scripts/kimodo-resident-worker.py index 846ace9..d699a2c 100644 --- a/scripts/kimodo-resident-worker.py +++ b/scripts/kimodo-resident-worker.py @@ -4,7 +4,6 @@ import json import os import shutil import time -import traceback from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer from threading import Lock from types import SimpleNamespace @@ -394,11 +393,7 @@ class Handler(BaseHTTPRequestHandler): payload = json.loads(self.rfile.read(length).decode("utf-8") or "{}") write_json(self, 200, generate(payload)) except Exception as error: - write_json(self, 500, { - "ok": False, - "error": str(error), - "traceback": traceback.format_exc(), - }) + write_json(self, 500, text_encoding.worker_error_body(error)) def log_message(self, fmt, *values): print(f"{self.address_string()} - {fmt % values}", flush=True) diff --git a/scripts/restart-vili-daemon.ps1 b/scripts/restart-vili-daemon.ps1 index e935d12..44706ff 100644 --- a/scripts/restart-vili-daemon.ps1 +++ b/scripts/restart-vili-daemon.ps1 @@ -1,6 +1,6 @@ param( [int]$Port = 8824, - [string]$HostName = "0.0.0.0", + [string]$HostName = "127.0.0.1,10.77.0.4", [string]$StateRoot = "E:\Projects\Vili\.vili" ) diff --git a/scripts/run-vili-daemon.ps1 b/scripts/run-vili-daemon.ps1 index 0e10cba..dcf1333 100644 --- a/scripts/run-vili-daemon.ps1 +++ b/scripts/run-vili-daemon.ps1 @@ -1,6 +1,6 @@ param( [int]$Port = 8824, - [string]$HostName = "0.0.0.0", + [string]$HostName = "127.0.0.1,10.77.0.4", [string]$StateRoot = "E:\Projects\Vili\.vili" ) diff --git a/scripts/smoke-vili.mjs b/scripts/smoke-vili.mjs deleted file mode 100644 index 30cc401..0000000 --- a/scripts/smoke-vili.mjs +++ /dev/null @@ -1,36 +0,0 @@ -#!/usr/bin/env node -const baseUrl = process.env.VILI_URL || process.argv[2] || "http://10.77.0.4:8824"; - -async function get(path) { - const response = await fetch(`${baseUrl}${path}`); - const text = await response.text(); - let body; - try { - body = JSON.parse(text); - } catch { - body = text; - } - if (!response.ok) { - throw new Error(`${path} returned ${response.status}: ${text.slice(0, 500)}`); - } - return body; -} - -const health = await get("/health"); -const providers = await get("/eve/deck/providers"); -const smoke = await get("/smoke"); - -console.log(JSON.stringify({ - ok: true, - baseUrl, - health: { - service: health.service, - providerId: health.providerId, - backend: health.state?.backend, - }, - providers, - smoke: { - ok: smoke.ok, - backend: smoke.backendStatus, - }, -}, null, 2)); diff --git a/scripts/start-vili-daemon.ps1 b/scripts/start-vili-daemon.ps1 index 3d49de7..d167917 100644 --- a/scripts/start-vili-daemon.ps1 +++ b/scripts/start-vili-daemon.ps1 @@ -1,6 +1,6 @@ param( [int]$Port = 8824, - [string]$HostName = "0.0.0.0", + [string]$HostName = "127.0.0.1,10.77.0.4", [string]$StateRoot = "E:\Projects\Vili\.vili" ) diff --git a/scripts/test-gpu-gate-http.mjs b/scripts/test-gpu-gate-http.mjs new file mode 100644 index 0000000..7454b5e --- /dev/null +++ b/scripts/test-gpu-gate-http.mjs @@ -0,0 +1,417 @@ +// Boots the real daemon and observes the gate where it is decided: the HTTP edge. Needs the +// daemon's own dependencies (ws, cultcache-ts), so it runs on a machine that has them. Every boot +// sets VILI_GPU_MIN_FREE_MIB=999999 so an allowed request ends in a VRAM refusal and never starts +// the worker: no test here may start a GPU job. +import assert from "node:assert/strict"; +import { spawn, spawnSync } from "node:child_process"; +import { existsSync, readFileSync } from "node:fs"; +import { copyFile, mkdtemp, readdir, rm, writeFile } from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import test from "node:test"; +import { fileURLToPath } from "node:url"; + +const daemonPath = path.join(path.dirname(fileURLToPath(import.meta.url)), "vili-daemon.mjs"); +let nextPort = 18000 + Math.floor(Math.random() * 1000); + +async function bootDaemon(stateRoot, { host = "127.0.0.1", env = { VILI_GPU_MIN_FREE_MIB: "999999" } } = {}) { + const port = nextPort++; + const child = spawn(process.execPath, [daemonPath, "--host", host, "--port", String(port), "--state-root", stateRoot], { + env: { ...process.env, ...env }, + stdio: ["ignore", "pipe", "pipe"], + windowsHide: true, + }); + let output = ""; + child.stdout.on("data", (chunk) => { output += chunk; }); + child.stderr.on("data", (chunk) => { output += chunk; }); + const deadline = Date.now() + 30000; + while (!output.includes("Vili listening")) { + if (child.exitCode !== null || Date.now() > deadline) throw new Error(`daemon did not start: ${output.slice(-500)}`); + await new Promise((resolve) => setTimeout(resolve, 100)); + } + const stop = () => new Promise((resolve) => { + if (child.exitCode !== null) resolve(); + else { child.once("exit", resolve); child.kill(); } + }); + return { child, port, stop }; +} + +const url = (d, route, host = "127.0.0.1") => `http://${host}:${d.port}${route}`; +const generate = (d, body) => fetch(url(d, "/motion/generate"), { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify(body), +}); +const setPermission = (d, body, host = "127.0.0.1") => fetch(url(d, "/gpu-permission", host), { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify(body), +}); +const readPermission = async (d) => (await fetch(url(d, "/gpu-permission"))).json(); + +test("http: refusal at the daemon edge", async () => { + const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-")); + const d = await bootDaemon(stateRoot); + try { + assert.deepEqual(await readPermission(d), { allowed: false, changedAt: null, jobRunning: false, lastRefusal: null }); + assert.equal((await generate(d, {})).status, 400, "request-shape errors come ahead of the gate"); + + const refused = await generate(d, { prompt: "a person waves hello, never seen before" }); + assert.equal(refused.status, 503); + assert.equal(refused.headers.get("retry-after"), "60"); + assert.deepEqual(await refused.json(), { ok: false, code: "gpu-busy", source: "household", gpu: { allowed: false, changedAt: null } }); + + assert.deepEqual(await readdir(path.join(stateRoot, "jobs")), [], "no job file"); + assert.equal((await readdir(stateRoot)).includes("artifacts"), false, "no artifact directory"); + + const state = await (await fetch(url(d, "/operator-state"))).json(); + assert.equal(state.gpu.lastRefusal.source, "household"); + assert.equal(state.gpu.allowed, false); + } finally { + await d.stop(); + await rm(stateRoot, { recursive: true, force: true }); + } +}); + +test("http: the permission persists across a daemon restart", async () => { + const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-persist-")); + let d = await bootDaemon(stateRoot); + try { + const on = await setPermission(d, { allowed: true }); + assert.equal(on.status, 200); + const onBody = await on.json(); + assert.equal(onBody.ok, true); + assert.equal(onBody.gpu.allowed, true); + const before = await readPermission(d); + assert.equal(before.allowed, true); + assert.ok(before.changedAt); + + assert.equal((await setPermission(d, { allowed: "yes" })).status, 400); + assert.equal((await setPermission(d, {})).status, 400); + assert.deepEqual(await readPermission(d), before, "a refused write changes nothing"); + + await d.stop(); + d = await bootDaemon(stateRoot); + const after = await readPermission(d); + assert.equal(after.allowed, true); + assert.equal(after.changedAt, before.changedAt); + + await setPermission(d, { allowed: false }); + await d.stop(); + d = await bootDaemon(stateRoot); + assert.equal((await readPermission(d)).allowed, false, "off persists too"); + } finally { + await d.stop(); + await rm(stateRoot, { recursive: true, force: true }); + } +}); + +test("http: an allowed request reaches the slot and the VRAM read", async () => { + const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-slot-")); + const d = await bootDaemon(stateRoot); + try { + await setPermission(d, { allowed: true }); + const refused = await generate(d, { prompt: "a person waves hello, never seen before" }); + assert.equal(refused.status, 503); + const body = await refused.json(); + assert.equal(body.source, "vram", "admitted through acquire(), then refused for VRAM"); + assert.equal(body.minFreeMiB, 999999); + assert.deepEqual(await readdir(path.join(stateRoot, "jobs")), [], "no job file"); + assert.equal((await readdir(stateRoot)).includes("artifacts"), false); + const state = await readPermission(d); + assert.equal(state.lastRefusal.source, "vram"); + assert.equal(state.jobRunning, false); + } finally { + await d.stop(); + await rm(stateRoot, { recursive: true, force: true }); + } +}); + +test("http: the switch is the household's", async () => { + const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-lan-")); + const addresses = Object.values(os.networkInterfaces()).flat().filter((i) => i.family === "IPv4" && !i.internal).map((i) => i.address); + const lan = addresses[0]; + assert.ok(lan, "the host has a non-loopback IPv4 address"); + const d = await bootDaemon(stateRoot, { host: `127.0.0.1,${lan}` }); + try { + const before = await readPermission(d); + const refused = await setPermission(d, { allowed: true }, lan); + assert.equal(refused.status, 403); + assert.deepEqual(await refused.json(), { ok: false, code: "household-only" }); + assert.deepEqual(await readPermission(d), before, "GET still reads the previous value"); + assert.equal((await setPermission(d, { allowed: true })).status, 200, "loopback still sets it"); + const other = addresses.find((address) => address !== lan); + if (other) { + await assert.rejects(fetch(url(d, "/gpu-permission", other)), "an address --host does not name is not served"); + } + } finally { + await d.stop(); + await rm(stateRoot, { recursive: true, force: true }); + } +}); + +test("http: a web page cannot flip the household's switch or start a job", async () => { + const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-csrf-")); + const d = await bootDaemon(stateRoot); + const post = (route, headers, body) => fetch(url(d, route), { method: "POST", headers, body }); + const prompt = JSON.stringify({ prompt: "a person waves hello, never seen before" }); + try { + const before = await readPermission(d); + for (const route of ["/gpu-permission", "/motion/generate"]) { + const payload = route === "/gpu-permission" ? JSON.stringify({ allowed: true }) : prompt; + const simple = await post(route, { "content-type": "text/plain" }, payload); + assert.equal(simple.status, 403, `${route}: a simple text/plain POST`); + assert.deepEqual(await simple.json(), { ok: false, code: "cross-site" }); + assert.equal((await post(route, {}, payload)).status, 403, `${route}: no content-type`); + const hostile = await post(route, { "content-type": "application/json", origin: "http://evil.example-origin" }, payload); + assert.equal(hostile.status, 403, `${route}: a hostile Origin`); + const body = JSON.stringify(await hostile.json()); + assert.equal(body.includes("evil.example-origin"), false, "the refusal never echoes the Origin"); + assert.equal( + (await post(route, { "content-type": "application/json", origin: `http://127.0.0.1:${d.port}.evil.example` }, payload)).status, + 403, + `${route}: an Origin that only starts like ours`, + ); + } + assert.deepEqual(await readPermission(d), before, "no refused request changed anything"); + assert.deepEqual(await readdir(path.join(stateRoot, "jobs")), [], "no job file"); + + const preflight = await fetch(url(d, "/gpu-permission"), { + method: "OPTIONS", + headers: { origin: "http://evil.example-origin", "access-control-request-method": "POST", "access-control-request-headers": "content-type" }, + }); + assert.equal(preflight.ok, false, "the daemon approves no CORS preflight"); + assert.equal(preflight.headers.get("access-control-allow-methods"), null); + + const own = await post("/gpu-permission", { "content-type": "application/json; charset=utf-8", origin: `http://127.0.0.1:${d.port}` }, JSON.stringify({ allowed: true })); + assert.equal(own.status, 200, "the daemon's own origin still works"); + assert.equal((await setPermission(d, { allowed: false })).status, 200, "a caller with no Origin (the tray) still works"); + } finally { + await d.stop(); + await rm(stateRoot, { recursive: true, force: true }); + } +}); + +// The daemon reaches the GPU only through the processes wsl.exe and nvidia-smi.exe. This test puts +// one small compiled stub on PATH under both names (Windows, where the daemon lives), so the real +// daemon runs a real job against a worker that takes as long as the test says and touches no GPU: +// nvidia-smi.exe answers 20000 free MiB, the worker health call answers ok, and the "/generate" +// call waits until the test creates a release file. +const stubSource = ` +using System; using System.IO; using System.Reflection; using System.Threading; +class Stub { + static int Main(string[] args) { + string exe = Path.GetFileNameWithoutExtension(Assembly.GetEntryAssembly().Location).ToLowerInvariant(); + string dir = Environment.GetEnvironmentVariable("VILI_STUB_DIR"); + if (exe == "nvidia-smi") { Console.WriteLine("20000"); return 0; } + string script = args.Length > 0 ? args[args.Length - 1] : ""; + if (script.Contains("docker run")) File.AppendAllText(Path.Combine(dir, "dockerrun"), "x"); + if (script.Contains("service docker start")) File.AppendAllText(Path.Combine(dir, "dockerstart"), "x"); + if (script.Contains("/health")) { Console.WriteLine("{\\"ok\\":true,\\"model\\":\\"stub\\"}"); return 0; } + if (script.Contains("/generate")) { + File.AppendAllText(Path.Combine(dir, "started"), "x"); + while (!File.Exists(Path.Combine(dir, "release"))) Thread.Sleep(25); + Console.WriteLine("{\\"ok\\":true}"); + } + return 0; + } +}`; + +async function stubbedDaemon(stubDir, stateRoot) { + const source = path.join(stubDir, "stub.cs"); + await writeFile(source, stubSource); + const build = path.join(stubDir, "build.ps1"); + await writeFile(build, [ + "param($Source, $Out)", + "Add-Type -TypeDefinition (Get-Content -Raw $Source) -OutputAssembly $Out -OutputType ConsoleApplication", + ].join("\r\n")); + const built = spawnSync("powershell.exe", [ + "-NoProfile", "-NonInteractive", "-ExecutionPolicy", "Bypass", "-File", build, source, path.join(stubDir, "wsl.exe"), + ], { encoding: "utf8", windowsHide: true }); + assert.equal(built.status, 0, `the stub did not compile: ${built.stderr || built.stdout}`); + await copyFile(path.join(stubDir, "wsl.exe"), path.join(stubDir, "nvidia-smi.exe")); + const pathKey = Object.keys(process.env).find((key) => key.toLowerCase() === "path") ?? "PATH"; + return bootDaemon(stateRoot, { + env: { + [pathKey]: `${stubDir}${path.delimiter}${process.env[pathKey]}`, + VILI_STUB_DIR: stubDir, + VILI_GPU_MIN_FREE_MIB: "1", + }, + }); +} + +test("http: the daemon holds the slot for the whole job", async () => { + const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-hold-")); + const stubDir = await mkdtemp(path.join(os.tmpdir(), "vili-gate-stub-")); + const d = await stubbedDaemon(stubDir, stateRoot); + const prompt = { prompt: "a person waves hello, never seen before" }; + const within = (promise, ms, what) => Promise.race([ + promise, + new Promise((_, reject) => setTimeout(() => reject(new Error(`${what} did not answer within ${ms} ms`)), ms)), + ]); + try { + await setPermission(d, { allowed: true }); + const first = generate(d, prompt); + const deadline = Date.now() + 20000; + while (!(await readPermission(d)).jobRunning) { + assert.ok(Date.now() < deadline, "the first job never took the slot"); + await new Promise((resolve) => setTimeout(resolve, 50)); + } + while (!existsSync(path.join(stubDir, "started"))) { + assert.ok(Date.now() < deadline, "the first job never reached the worker"); + await new Promise((resolve) => setTimeout(resolve, 50)); + } + + for (let i = 0; i < 3; i += 1) { + const second = await within(generate(d, prompt), 5000, "a request during the job"); + assert.equal(second.status, 503, "a request during the job is refused at once"); + assert.deepEqual(await second.json(), { ok: false, code: "gpu-busy", source: "slot" }); + } + assert.equal(readFileSync(path.join(stubDir, "started"), "utf8"), "x", "exactly one job reached the worker"); + assert.equal((await readPermission(d)).jobRunning, true, "the slot is still held"); + + await writeFile(path.join(stubDir, "release"), ""); + const done = await within(first, 20000, "the first job"); + assert.equal(done.status, 200); + assert.equal((await readPermission(d)).jobRunning, false, "the slot frees when the job ends"); + const next = await within(generate(d, prompt), 20000, "the next job"); + assert.equal(next.status, 200, "the next job is admitted after the slot frees"); + assert.equal(readFileSync(path.join(stubDir, "started"), "utf8"), "xx"); + } finally { + await d.stop(); + await rm(stateRoot, { recursive: true, force: true }); + await rm(stubDir, { recursive: true, force: true }); + } +}); + +test("http: a malformed override refuses to start and names the variable, not its value", async () => { + const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-env-")); + const child = spawn(process.execPath, [daemonPath, "--host", "127.0.0.1", "--port", String(nextPort++), "--state-root", stateRoot], { + env: { ...process.env, VILI_GPU_MIN_FREE_MIB: "10240MiB-secret" }, + stdio: ["ignore", "pipe", "pipe"], + windowsHide: true, + }); + let output = ""; + child.stdout.on("data", (chunk) => { output += chunk; }); + child.stderr.on("data", (chunk) => { output += chunk; }); + try { + const code = await new Promise((resolve) => child.on("exit", resolve)); + assert.notEqual(code, 0); + assert.match(output, /VILI_GPU_MIN_FREE_MIB must be a finite non-negative number/); + assert.equal(output.includes("10240MiB-secret"), false); + assert.equal(output.includes("Vili listening"), false); + } finally { + child.kill(); + await rm(stateRoot, { recursive: true, force: true }); + } +}); + +test("http: the daemon refuses to listen on a wildcard address", async () => { + const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-wild-")); + try { + for (const host of ["0.0.0.0", "127.0.0.1,0.0.0.0", "::"]) { + const run = spawnSync(process.execPath, [daemonPath, "--host", host, "--port", String(nextPort++), "--state-root", stateRoot], { + encoding: "utf8", + windowsHide: true, + timeout: 20000, + }); + assert.equal(run.status, 2, `--host ${host} exits 2`); + assert.equal(`${run.stdout}${run.stderr}`.includes("Vili listening"), false); + } + } finally { + await rm(stateRoot, { recursive: true, force: true }); + } +}); + +test("http: no response grants a web page access", async () => { + const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-cors-")); + const d = await bootDaemon(stateRoot); + try { + const hostile = { origin: "http://evil.example-origin" }; + const responses = [ + await fetch(url(d, "/gpu-permission"), { headers: hostile }), + await fetch(url(d, "/operator-state"), { headers: hostile }), + await fetch(url(d, "/nowhere"), { headers: hostile }), + await fetch(url(d, "/gpu-permission"), { method: "POST", headers: { ...hostile, "content-type": "application/json" }, body: "{}" }), + ]; + assert.deepEqual(responses.map((r) => r.status), [200, 200, 404, 403]); + for (const response of responses) { + const cors = [...response.headers.keys()].filter((name) => name.startsWith("access-control-")); + assert.deepEqual(cors, [], "no access-control header on any response"); + } + } finally { + await d.stop(); + await rm(stateRoot, { recursive: true, force: true }); + } +}); + +test("http: with the permission off nothing a web page or a LAN caller sends reaches the GPU", async () => { + const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-off-")); + const stubDir = await mkdtemp(path.join(os.tmpdir(), "vili-gate-stub-")); + const d = await stubbedDaemon(stubDir, stateRoot); + const page = { origin: "http://evil.example-origin", "sec-fetch-site": "cross-site", "sec-fetch-mode": "no-cors", "sec-fetch-dest": "image" }; + try { + const before = await readPermission(d); + assert.equal(before.allowed, false); + for (const route of ["/smoke", "/health", "/", "/operator-state", "/eve/operator", "/provider-advertisement", "/eve/deck/providers", "/gpu-permission"]) { + const response = await fetch(url(d, route), { headers: page }); + assert.equal(response.status, route === "/smoke" ? 404 : 200, `GET ${route}`); + } + const refused = await generate(d, { prompt: "a person waves hello, never seen before" }); + assert.equal(refused.status, 503); + assert.equal((await refused.json()).source, "household"); + assert.equal((await setPermission(d, { allowed: true }, "127.0.0.1")).status, 200); + assert.equal((await setPermission(d, { allowed: false }, "127.0.0.1")).status, 200); + assert.deepEqual((await readPermission(d)).allowed, false); + assert.equal(existsSync(path.join(stubDir, "dockerrun")), false, "nothing ran docker run (the worker or any GPU container)"); + assert.equal(existsSync(path.join(stubDir, "dockerstart")), false, "no GET started Docker"); + assert.equal(existsSync(path.join(stubDir, "started")), false, "nothing reached the worker"); + assert.deepEqual(await readdir(path.join(stateRoot, "jobs")), [], "no job file"); + } finally { + await d.stop(); + await rm(stateRoot, { recursive: true, force: true }); + await rm(stubDir, { recursive: true, force: true }); + } +}); + +test("http: a client that hangs up mid-job does not free the slot", async () => { + const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-hangup-")); + const stubDir = await mkdtemp(path.join(os.tmpdir(), "vili-gate-stub-")); + const d = await stubbedDaemon(stubDir, stateRoot); + const prompt = { prompt: "a person waves hello, never seen before" }; + const hangUp = new AbortController(); + try { + await setPermission(d, { allowed: true }); + const first = fetch(url(d, "/motion/generate"), { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify(prompt), + signal: hangUp.signal, + }).catch((error) => error); + const deadline = Date.now() + 20000; + while (!existsSync(path.join(stubDir, "started"))) { + assert.ok(Date.now() < deadline, "the job never reached the worker"); + await new Promise((resolve) => setTimeout(resolve, 50)); + } + hangUp.abort(); + assert.equal((await first).name, "AbortError", "the client hung up"); + await new Promise((resolve) => setTimeout(resolve, 500)); + + assert.equal((await readPermission(d)).jobRunning, true, "the slot is still held after the hang-up"); + const second = await generate(d, prompt); + assert.equal(second.status, 503); + assert.equal((await second.json()).source, "slot"); + assert.equal(readFileSync(path.join(stubDir, "started"), "utf8"), "x", "no second job reached the worker"); + + await writeFile(path.join(stubDir, "release"), ""); + const freed = Date.now() + 20000; + while ((await readPermission(d)).jobRunning) { + assert.ok(Date.now() < freed, "the slot never freed when the job ended"); + await new Promise((resolve) => setTimeout(resolve, 50)); + } + } finally { + await d.stop(); + await rm(stateRoot, { recursive: true, force: true }); + await rm(stubDir, { recursive: true, force: true }); + } +}); diff --git a/scripts/test-gpu-tenancy.mjs b/scripts/test-gpu-tenancy.mjs new file mode 100644 index 0000000..f2ba3e7 --- /dev/null +++ b/scripts/test-gpu-tenancy.mjs @@ -0,0 +1,211 @@ +import assert from "node:assert/strict"; +import test from "node:test"; +import { admitThen, createGpuTenancy, gpuTenancyOptionsFromEnv, parseFreeMiB, placementRefusal } from "./gpu-tenancy.mjs"; + +function deferred() { + let resolve; + const promise = new Promise((r) => { resolve = r; }); + return { promise, resolve }; +} + +// Fake ports: a clock, a store spy and a free-VRAM source. No GPU, no daemon, no cache. +function rig({ permission, free = 12000, minFreeMiB } = {}) { + const r = { t: Date.parse("2026-10-23T10:00:00Z"), saved: [], reads: 0, free, failSave: false }; + r.gate = createGpuTenancy({ + now: () => r.t, + permission, + minFreeMiB, + savePermission: async (value) => { + if (r.failSave) throw new Error("disk full"); + r.saved.push(value); + }, + readFreeMiB: async () => { + r.reads += 1; + if (typeof r.free === "function") return r.free(); + return r.free; + }, + }); + return r; +} +const ON = { allowed: true, changedAt: "2026-10-23T09:00:00.000Z" }; + +test("gate: toggle off refuses and creates no job", async () => { + const r = rig(); + let calls = 0; + const refused = await admitThen(r.gate, async () => { calls += 1; }); + assert.deepEqual(refused, { ok: false, code: "gpu-busy", source: "household", gpu: { allowed: false, changedAt: null } }); + assert.equal(calls, 0); + assert.equal(r.reads, 0, "the VRAM is never read for a household refusal"); + assert.deepEqual(r.gate.status(), { + allowed: false, + changedAt: null, + jobRunning: false, + lastRefusal: { at: "2026-10-23T10:00:00.000Z", source: "household" }, + }); +}); + +test("gate: toggle on admits", async () => { + const r = rig({ permission: ON }); + let calls = 0; + const result = await admitThen(r.gate, async () => { + calls += 1; + assert.equal(r.gate.status().jobRunning, true); + return { ok: true, value: "finished" }; + }); + assert.deepEqual(result, { ok: true, value: "finished" }); + assert.equal(calls, 1); + assert.equal(r.gate.status().jobRunning, false); + assert.equal(r.gate.status().lastRefusal, null); +}); + +test("gate: toggle off mid-job lets the job finish and refuses the next", async () => { + const r = rig({ permission: ON }); + const end = deferred(); + const running = admitThen(r.gate, async () => { await end.promise; return { ok: true, value: "finished" }; }); + while (!r.gate.status().jobRunning) await new Promise((resolve) => setImmediate(resolve)); + const off = await r.gate.setAllowed(false); + assert.equal(off.allowed, false); + assert.equal(off.jobRunning, true, "the job is still running"); + let calls = 0; + const next = await admitThen(r.gate, async () => { calls += 1; }); + assert.equal(next.source, "household"); + assert.equal(calls, 0); + end.resolve(); + assert.deepEqual(await running, { ok: true, value: "finished" }); + assert.equal(r.gate.status().jobRunning, false); +}); + +test("gate: the concurrent slot behaves as h2's did", async () => { + const vram = deferred(); + const r = rig({ permission: ON, free: () => vram.promise }); + const end = deferred(); + let calls = 0; + const job = async () => { calls += 1; await end.promise; return "done"; }; + const all = [admitThen(r.gate, job), admitThen(r.gate, job), admitThen(r.gate, job)]; + // the two losers are decided before the VRAM read resolves + const losers = await Promise.all([all[1], all[2]]); + assert.deepEqual(losers, [{ ok: false, code: "gpu-busy", source: "slot" }, { ok: false, code: "gpu-busy", source: "slot" }]); + assert.equal(r.reads, 1, "only the slot holder reads VRAM"); + vram.resolve(12000); + end.resolve(); + assert.equal(await all[0], "done"); + assert.equal(calls, 1); + assert.equal(r.gate.status().jobRunning, false); + + await assert.rejects(admitThen(r.gate, async () => { throw new Error("boom"); }), /boom/); + assert.equal(r.gate.status().jobRunning, false, "a throwing job frees the slot"); + r.free = 100; + assert.equal((await admitThen(r.gate, async () => "x")).source, "vram"); + const readsBefore = r.reads; + r.free = 12000; + assert.equal(await admitThen(r.gate, async () => "again"), "again", "a vram refusal frees the slot"); + assert.equal(r.reads, readsBefore + 1, "the next admission reads VRAM again"); +}); + +test("gate: VRAM read fails closed", async () => { + const cases = [[9000, 9000], [10239, 10239], [null, null], [Number.NaN, null], [() => { throw new Error("smi"); }, null]]; + for (const [free, expected] of cases) { + const r = rig({ permission: ON, free }); + let calls = 0; + const refused = await admitThen(r.gate, async () => { calls += 1; }); + assert.deepEqual(refused, { ok: false, code: "gpu-busy", source: "vram", freeMiB: expected, minFreeMiB: 10240 }); + assert.equal(calls, 0); + assert.equal(r.gate.status().jobRunning, false); + assert.equal(r.gate.status().lastRefusal.source, "vram"); + } + const edge = rig({ permission: ON, free: 10240 }); + assert.equal(await admitThen(edge.gate, async () => "in"), "in", "exactly the minimum admits"); + const custom = rig({ permission: ON, free: 5000, minFreeMiB: 4096 }); + assert.equal(await admitThen(custom.gate, async () => "in"), "in", "the minimum is the configured one"); +}); + +test("gate: a toggle during the VRAM read wins", async () => { + const vram = deferred(); + const r = rig({ permission: ON, free: () => vram.promise }); + let calls = 0; + const pending = admitThen(r.gate, async () => { calls += 1; }); + await r.gate.setAllowed(false); + vram.resolve(12000); + const refused = await pending; + assert.equal(refused.source, "household"); + assert.equal(calls, 0); + assert.equal(r.gate.status().jobRunning, false); +}); + +test("gate: setAllowed saves before it answers", async () => { + const r = rig(); + r.failSave = true; + await assert.rejects(r.gate.setAllowed(true), /disk full/); + assert.equal(r.gate.status().allowed, false, "a failed save leaves the state unchanged"); + assert.equal(r.gate.status().changedAt, null); + r.failSave = false; + + await assert.rejects(r.gate.setAllowed("yes"), TypeError); + await assert.rejects(r.gate.setAllowed(undefined), TypeError); + assert.deepEqual(r.saved, [], "a non-boolean saves nothing"); + + r.t += 1000; + const on = await r.gate.setAllowed(true); + assert.deepEqual(r.saved, [{ allowed: true, changedAt: "2026-10-23T10:00:01.000Z" }]); + assert.equal(on.allowed, true); + assert.equal(on.changedAt, "2026-10-23T10:00:01.000Z"); + + r.t += 5000; + const same = await r.gate.setAllowed(true); + assert.equal(r.saved.length, 1, "a same-value call saves nothing"); + assert.equal(same.changedAt, "2026-10-23T10:00:01.000Z", "and keeps changedAt"); +}); + +test("gate: a second release is a no-op and cannot free another job's slot", async () => { + const r = rig({ permission: ON }); + const first = await r.gate.acquire(); + first.release(); + const second = await r.gate.acquire(); + assert.equal(r.gate.status().jobRunning, true); + first.release(); + assert.equal(r.gate.status().jobRunning, true, "a stale release leaves the new holder's slot held"); + assert.equal((await r.gate.acquire()).refusal.source, "slot"); + second.release(); + second.release(); + assert.equal(r.gate.status().jobRunning, false); + assert.equal(typeof (await r.gate.acquire()).release, "function", "the slot is free after the real release"); +}); + +test("gate: a stored permission is the starting state", () => { + assert.deepEqual(rig({ permission: ON }).gate.status(), { allowed: true, changedAt: ON.changedAt, jobRunning: false, lastRefusal: null }); + assert.equal(rig({ permission: { allowed: "yes", changedAt: "x" } }).gate.status().allowed, false, "only a true boolean allows"); +}); + +test("parseFreeMiB reads the first line as a base-10 integer", () => { + assert.equal(parseFreeMiB("5001\r\n"), 5001); + assert.equal(parseFreeMiB("0"), 0); + assert.equal(parseFreeMiB(" 12 \n7\n"), 12); + for (const bad of ["", "[N/A]", "abc", "-5", "1e3", "12.5", "12 MiB", "\n5", null, undefined]) { + assert.equal(parseFreeMiB(bad), null, JSON.stringify(bad)); + } +}); + +test("placement refusal is keyed on the typed code, never on text, and carries no gate status", () => { + assert.deepEqual(placementRefusal({ ok: false, code: "gpu-busy", error: "x" }, "vili-1"), { + ok: false, code: "gpu-busy", source: "placement", jobId: "vili-1", + }); + assert.equal(placementRefusal({ ok: false, error: "gpu-busy: 0.50 GiB of VRAM free" }, "vili-1"), null); + assert.equal(placementRefusal({ ok: false, code: null }, "vili-1"), null); + assert.equal(placementRefusal(null, "vili-1"), null); +}); + +test("env override: a valid value is read, absent is left to the default", () => { + assert.deepEqual(gpuTenancyOptionsFromEnv({}), {}); + assert.deepEqual(gpuTenancyOptionsFromEnv({ VILI_GPU_MIN_FREE_MIB: "8192.5" }), { minFreeMiB: 8192.5 }); + assert.deepEqual(gpuTenancyOptionsFromEnv({ VILI_GPU_MIN_FREE_MIB: "0" }), { minFreeMiB: 0 }); +}); + +test("env override: a malformed value refuses to start and names the variable, not the value", () => { + for (const bad of ["10240MiB", "", " ", "NaN", "Infinity", "-5"]) { + assert.throws( + () => gpuTenancyOptionsFromEnv({ VILI_GPU_MIN_FREE_MIB: bad }), + (error) => error.message === "VILI_GPU_MIN_FREE_MIB must be a finite non-negative number", + JSON.stringify(bad), + ); + } +}); diff --git a/scripts/test_text_encoding.py b/scripts/test_text_encoding.py index 65aa1e3..9619ae2 100644 --- a/scripts/test_text_encoding.py +++ b/scripts/test_text_encoding.py @@ -180,6 +180,12 @@ class ChildBoundary(unittest.TestCase): """) self.assertEqual(raised.exception.free_vram, 1234) + def test_worker_error_body_carries_the_code(self): + busy = te.worker_error_body(te.GpuBusy(GIB)) + self.assertIs(busy["ok"], False) + self.assertEqual(busy["code"], "gpu-busy") + self.assertIsNone(te.worker_error_body(RuntimeError("x"))["code"]) + def test_other_failures_name_the_exit_code_and_never_the_text(self): with self.assertRaises(RuntimeError) as raised: self.run_stub("import sys; sys.exit(1)", texts=(SECRET,)) diff --git a/scripts/text_encoding.py b/scripts/text_encoding.py index 668e529..572cdad 100644 --- a/scripts/text_encoding.py +++ b/scripts/text_encoding.py @@ -12,6 +12,7 @@ import subprocess import sys import tempfile import time +import traceback from dataclasses import dataclass from datetime import datetime, timezone from pathlib import Path @@ -33,7 +34,11 @@ LLAMA_BASE = "meta-llama/Meta-Llama-3-8B-Instruct" # the base model the mntp ad class GpuBusy(RuntimeError): - """The card has too little free VRAM to host the encoder; there is no CPU fallback.""" + """Placement refusal: too little free VRAM to host the encoder, and no CPU fallback. + + Whether heavy work may start at all is the daemon's tenancy gate, which sees the host; this is only + the child sizing its own placement from what torch can see. + """ code = "gpu-busy" @@ -49,6 +54,11 @@ def plan_placement(free_vram: int, mem_available: int) -> dict: return {0: free_vram - GPU_RESERVE, "cpu": max(mem_available - CPU_RESERVE, 0)} +def worker_error_body(error: BaseException) -> dict: + """The worker's one error wire shape: a typed code (GpuBusy.code, else None) beside the text.""" + return {"ok": False, "error": str(error), "code": getattr(error, "code", None), "traceback": traceback.format_exc()} + + def read_snapshot_revision(hf_home: Path, repo_id: str) -> str: """The commit the mounted Hugging Face cache resolves a repo's main ref to.""" ref = Path(hf_home) / "hub" / ("models--" + repo_id.replace("/", "--")) / "refs" / "main" diff --git a/scripts/vili-daemon.mjs b/scripts/vili-daemon.mjs index e75e0ac..a8fc87d 100644 --- a/scripts/vili-daemon.mjs +++ b/scripts/vili-daemon.mjs @@ -6,6 +6,7 @@ import { fileURLToPath } from "node:url"; import { createRequire } from "node:module"; import { execFile } from "node:child_process"; import { WebSocketServer } from "ws"; +import { RETRY_AFTER_SECONDS, admitThen, createGpuTenancy, gpuTenancyOptionsFromEnv, parseFreeMiB, placementRefusal } from "./gpu-tenancy.mjs"; const __dirname = path.dirname(fileURLToPath(import.meta.url)); const require = createRequire(import.meta.url); @@ -59,6 +60,20 @@ const operatorStateDefinition = defineDocumentType({ { slot: 1, memberName: "providerId", typeName: "string" }, { slot: 2, memberName: "updatedAt", typeName: "string" }, { slot: 3, memberName: "backend", typeName: "object" }, + { slot: 4, memberName: "gpu", typeName: "object" }, + ], +}); +const gpuPermissionDefinition = defineDocumentType({ + type: "gamecult.vili.gpu_permission", + schemaName: "gamecult.vili.gpu_permission", + schemaId: "gamecult.vili.gpu_permission.v1", + schemaVersion: "gamecult.vili.gpu_permission.v1", + name: (value) => value?.host || "raven", + schema: objectSchema, + members: [ + { slot: 0, memberName: "host", typeName: "string", isName: true }, + { slot: 1, memberName: "allowed", typeName: "boolean" }, + { slot: 2, memberName: "changedAt", typeName: "string" }, ], }); const eveSurfaceDefinition = defineDocumentType({ @@ -108,6 +123,7 @@ const transportProfileDefinition = defineDocumentType({ const viliDocumentRegistry = defineDocumentRegistry( providerAdvertisementDefinition, operatorStateDefinition, + gpuPermissionDefinition, eveSurfaceDefinition, commandBoundaryDefinition, transportProfileDefinition, @@ -115,6 +131,8 @@ const viliDocumentRegistry = defineDocumentRegistry( function parseArgs(argv) { const args = { + // Comma-separated addresses to listen on, one server each. Never 0.0.0.0: the daemon answers + // this machine and the WireGuard address, not whatever network the machine joins next. host: process.env.VILI_DAEMON_HOST || "127.0.0.1", port: Number(process.env.VILI_DAEMON_PORT || 8824), stateRoot: process.env.VILI_STATE_ROOT || path.join(projectRoot, ".vili"), @@ -134,13 +152,31 @@ function parseArgs(argv) { } const args = parseArgs(process.argv.slice(2)); +const hosts = args.host.split(",").map((host) => host.trim()).filter(Boolean); const startedAt = new Date().toISOString(); +const gpuOptions = gpuTenancyOptionsFromEnv(process.env); +// The daemon's one cache and store instance: every write to vili.service.cc goes through one queue. +const viliCache = CultCache.builder() + .withRegistry(viliDocumentRegistry) + .withGenericStore(new SingleFileMessagePackBackingStore(path.join(args.stateRoot, viliStoreFileName))) + .build(); +let storedPermission; +if (!args.health && !args.printProviderAdvertisement) { + await viliCache.pullAllBackingStores(); + storedPermission = viliCache.get(gpuPermissionDefinition, "raven"); +} +const gpuTenancy = createGpuTenancy({ + now: Date.now, + ...gpuOptions, + permission: storedPermission, + savePermission: (value) => viliCache.put(gpuPermissionDefinition, "raven", { host: "raven", ...value }), + readFreeMiB: hostFreeMiB, +}); let backendStatus = { checkedAt: null, dockerImage: "unknown", nvidiaRuntime: "unknown", huggingFaceToken: "unknown", - kimodoHelp: "unknown", kimodoWorker: "unknown", kimodoWorkerDetail: null, lastError: null, @@ -171,6 +207,11 @@ function wsl(script, options = {}) { return run("wsl.exe", ["-d", "Ubuntu-24.04", "-u", "root", "--", "bash", "-lc", script], options); } +async function hostFreeMiB() { + const r = await run("nvidia-smi.exe", ["--query-gpu=memory.free", "--format=csv,noheader,nounits"], { timeout: 5000 }); + return r.ok ? parseFreeMiB(r.stdout) : null; +} + function shQuote(value) { return `'${String(value).replaceAll("'", "'\\''")}'`; } @@ -193,12 +234,12 @@ async function wslJson(script, options = {}) { } } -function json(response, statusCode, body) { +function json(response, statusCode, body, headers = {}) { const payload = JSON.stringify(body, null, 2); response.writeHead(statusCode, { + ...headers, "content-type": "application/json; charset=utf-8", "cache-control": "no-store", - "access-control-allow-origin": "*", }); response.end(payload); } @@ -207,7 +248,6 @@ function text(response, statusCode, body) { response.writeHead(statusCode, { "content-type": "text/plain; charset=utf-8", "cache-control": "no-store", - "access-control-allow-origin": "*", }); response.end(body); } @@ -226,7 +266,6 @@ function providerAdvertisement() { endpoints: { http: ravenBaseUrl, health: `${ravenBaseUrl}/health`, - smoke: `${ravenBaseUrl}/smoke`, providerAdvertisement: `${ravenBaseUrl}/provider-advertisement`, operatorState: `${ravenBaseUrl}/operator-state`, operatorSurface: `${ravenBaseUrl}/eve/operator`, @@ -252,6 +291,7 @@ function operatorState() { startedAt, updatedAt: new Date().toISOString(), backend: backendStatus, + gpu: gpuTenancy.status(), authority: { owns: [ "animation job intake", @@ -273,8 +313,8 @@ function operatorState() { operatorState: "/operator-state", operatorSurface: "/eve/operator", deckProviders: "/eve/deck/providers", - smoke: "/smoke", generate: "/motion/generate", + gpuPermission: "/gpu-permission", }, }; } @@ -298,10 +338,10 @@ function commandBoundary() { "motion artifact references", ], }, - smoke: { - method: "GET", - path: "/smoke", - owns: ["backend readiness proof", "resident worker warmup"], + gpuPermission: { + method: "POST", + path: "/gpu-permission", + owns: ["the household GPU permission for this host, from loopback only"], }, liveness: { method: "GET", @@ -353,10 +393,8 @@ function eveSurface() { { type: "stat", label: "Verse", value: "raven.local" }, { type: "stat", label: "Docker image", value: state.backend.dockerImage }, { type: "stat", label: "NVIDIA runtime", value: state.backend.nvidiaRuntime }, - { type: "stat", label: "Kimodo", value: state.backend.kimodoHelp }, { type: "stat", label: "Kimodo worker", value: state.backend.kimodoWorker }, { type: "stat", label: "HF token", value: state.backend.huggingFaceToken }, - { type: "command", label: "Smoke", method: "GET", href: `${ravenBaseUrl}/smoke` }, { type: "command", label: "Generate motion", method: "POST", href: `${ravenBaseUrl}/motion/generate` }, ], }, @@ -377,59 +415,31 @@ async function persistSurfaces() { } async function persistCultCacheSurfaces() { - const cache = CultCache.builder() - .withRegistry(viliDocumentRegistry) - .withGenericStore(new SingleFileMessagePackBackingStore(path.join(args.stateRoot, viliStoreFileName))) - .build(); - await cache.put(providerAdvertisementDefinition, providerId, providerAdvertisement()); - await cache.put(operatorStateDefinition, "vili", operatorState()); - await cache.put(eveSurfaceDefinition, providerId, eveSurface()); - await cache.put(commandBoundaryDefinition, "vili", commandBoundary()); - await cache.put(transportProfileDefinition, "vili", transportProfile()); + await viliCache.put(providerAdvertisementDefinition, providerId, providerAdvertisement()); + await viliCache.put(operatorStateDefinition, "vili", operatorState()); + await viliCache.put(eveSurfaceDefinition, providerId, eveSurface()); + await viliCache.put(commandBoundaryDefinition, "vili", commandBoundary()); + await viliCache.put(transportProfileDefinition, "vili", transportProfile()); } -async function refreshBackendStatus({ includeKimodoHelp = false } = {}) { - const docker = await wsl([ - "set -e", - "service docker start >/dev/null 2>&1 || true", - "docker images gamecult/kimodo:latest --format '{{.Repository}}:{{.Tag}}' | head -n 1", - ].join("; "), { timeout: 20000 }); - - let nvidia = await wsl([ - "set -e", - "service docker start >/dev/null 2>&1 || true", - "docker info --format '{{json .Runtimes}}' | grep -q nvidia && echo configured || echo missing", - ].join("; "), { timeout: 20000 }); - +// A read: it only asks. Starting docker and the worker belongs to a job that holds the GPU slot. +async function refreshBackendStatus() { + const docker = await wsl("docker images gamecult/kimodo:latest --format '{{.Repository}}:{{.Tag}}' | head -n 1", { timeout: 20000 }); + const nvidia = await wsl("docker info --format '{{json .Runtimes}}' | grep -q nvidia && echo configured || echo missing", { timeout: 20000 }); const token = await wsl("test -f /root/.cache/huggingface/token && echo present || echo missing", { timeout: 10000 }); const worker = await kimodoWorkerHealth(20000); - let help = { ok: true, stdout: backendStatus.kimodoHelp === "ok" ? "ok" : "not checked", stderr: "" }; - if (includeKimodoHelp) { - nvidia = await wsl([ - "set -e", - "service docker start >/dev/null 2>&1 || true", - "docker run --rm --runtime=nvidia --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi --query-gpu=name --format=csv,noheader | head -n 1", - ].join("; "), { timeout: 45000 }); - help = await wsl([ - "set -e", - "service docker start >/dev/null 2>&1 || true", - dockerHuggingFaceEnv, - `docker run --rm --runtime=nvidia --gpus all ${dockerHuggingFaceArgs} gamecult/kimodo:latest kimodo_gen --help | sed -n '1,6p'`, - ].join("; "), { timeout: 60000 }); - } backendStatus = { checkedAt: new Date().toISOString(), dockerImage: docker.ok && docker.stdout ? docker.stdout : "missing", nvidiaRuntime: nvidia.ok && nvidia.stdout ? nvidia.stdout : "unavailable", huggingFaceToken: token.ok ? token.stdout : "unknown", - kimodoHelp: includeKimodoHelp ? (help.ok ? "ok" : "failed") : backendStatus.kimodoHelp, kimodoWorker: worker.ok && worker.body?.ok ? "resident" : "not running", kimodoWorkerDetail: worker.body || null, - lastError: [docker, nvidia, token, help].find((result) => !result.ok)?.stderr || null, + lastError: [docker, nvidia, token].find((result) => !result.ok)?.stderr || null, }; await persistSurfaces(); - return { docker, nvidia, token, help, backendStatus }; + return backendStatus; } async function kimodoWorkerHealth(timeout = 10000) { @@ -502,7 +512,43 @@ async function readRequestBody(request) { return raw ? JSON.parse(raw) : {}; } +const loopbackAddresses = new Set(["127.0.0.1", "::1", "::ffff:127.0.0.1"]); + +// A state-changing POST is a service's or the tray's call, never a web page's. A browser sends a +// cross-site page's POST as a no-preflight simple request (text/plain) or with an Origin that is not +// ours, so both are refused before the body is read. The daemon grants no CORS preflight at all. +function refuseCrossSite(request, response) { + const origin = request.headers.origin; + const ownOrigins = [`http://127.0.0.1:${args.port}`, `http://localhost:${args.port}`, `http://[::1]:${args.port}`]; + const type = String(request.headers["content-type"] || "").split(";")[0].trim().toLowerCase(); + if (type === "application/json" && (origin === undefined || ownOrigins.includes(origin))) return false; + json(response, 403, { ok: false, code: "cross-site" }); + return true; +} + +// The household's switch: only a caller on this machine (the tray, in the household's session) sets it. +async function setGpuPermission(request, response) { + if (refuseCrossSite(request, response)) return; + if (!loopbackAddresses.has(request.socket.remoteAddress)) { + json(response, 403, { ok: false, code: "household-only" }); + return; + } + let body; + try { + body = await readRequestBody(request); + } catch (error) { + json(response, 400, { ok: false, error: `Invalid JSON: ${error.message}` }); + return; + } + if (typeof body?.allowed !== "boolean") { + json(response, 400, { ok: false, error: "allowed must be a boolean." }); + return; + } + json(response, 200, { ok: true, gpu: await gpuTenancy.setAllowed(body.allowed) }); +} + async function generateMotion(request, response) { + if (refuseCrossSite(request, response)) return; let body; try { body = await readRequestBody(request); @@ -526,6 +572,13 @@ async function generateMotion(request, response) { return; } + const admitted = await admitThen(gpuTenancy, () => runMotionJob(body, prompt, response)); + if (admitted?.code === "gpu-busy") { + json(response, 503, admitted, { "retry-after": String(RETRY_AFTER_SECONDS) }); + } +} + +async function runMotionJob(body, prompt, response) { const jobId = `vili-${Date.now()}`; const artifactDir = path.join(args.stateRoot, "artifacts", jobId); await mkdir(artifactDir, { recursive: true }); @@ -650,23 +703,15 @@ async function generateMotion(request, response) { await mkdir(path.join(args.stateRoot, "jobs"), { recursive: true }); await writeFile(path.join(args.stateRoot, "jobs", `${jobId}.json`), JSON.stringify(job, null, 2)); }); - json(response, result.ok ? 200 : 502, job); + const placement = result.ok ? null : placementRefusal(result.body, jobId); + if (placement) json(response, 503, placement, { "retry-after": String(RETRY_AFTER_SECONDS) }); + else json(response, result.ok ? 200 : 502, job); } async function handleRequest(request, response) { - const url = new URL(request.url, `http://${request.headers.host || `${args.host}:${args.port}`}`); - if (request.method === "OPTIONS") { - response.writeHead(204, { - "access-control-allow-origin": "*", - "access-control-allow-methods": "GET,POST,OPTIONS", - "access-control-allow-headers": "content-type", - }); - response.end(); - return; - } - + const url = new URL(request.url, `http://${request.headers.host || `${hosts[0]}:${args.port}`}`); if (url.pathname === "/" || url.pathname === "/health") { - await refreshBackendStatus({ includeKimodoHelp: false }); + await refreshBackendStatus(); json(response, 200, { ok: true, service: "vili", providerId, state: operatorState() }); } else if (url.pathname === "/provider-advertisement") { json(response, 200, providerAdvertisement()); @@ -683,12 +728,10 @@ async function handleRequest(request, response) { endpoint: `${ravenDeckUrl}/${providerId}`, }], }); - } else if (url.pathname === "/smoke") { - const result = await refreshBackendStatus({ includeKimodoHelp: true }); - const worker = await ensureKimodoWorker(); - await refreshBackendStatus({ includeKimodoHelp: false }); - const ok = result.backendStatus.kimodoHelp === "ok" && worker.ok && worker.body?.ok; - json(response, ok ? 200 : 502, { ok, ...result, worker: worker.body || worker }); + } else if (url.pathname === "/gpu-permission" && request.method === "GET") { + json(response, 200, gpuTenancy.status()); + } else if (url.pathname === "/gpu-permission" && request.method === "POST") { + await setGpuPermission(request, response); } else if (url.pathname === "/motion/generate" && request.method === "POST") { await generateMotion(request, response); } else { @@ -697,8 +740,7 @@ async function handleRequest(request, response) { } async function healthCheck() { - const host = args.host === "0.0.0.0" ? "127.0.0.1" : args.host; - const response = await fetch(`http://${host}:${args.port}/health`); + const response = await fetch(`http://${hosts[0]}:${args.port}/health`); if (!response.ok) throw new Error(`health returned ${response.status}`); const body = await response.json(); if (!body.ok) throw new Error("health returned ok=false"); @@ -715,15 +757,13 @@ if (args.health) { process.exit(1); }); } else { + if (hosts.length === 0 || hosts.includes("0.0.0.0") || hosts.includes("::")) { + console.error("--host must name the addresses to listen on, never a wildcard."); + process.exit(2); + } await mkdir(path.join(args.stateRoot, "jobs"), { recursive: true }); await persistSurfaces(); - const server = http.createServer((request, response) => { - handleRequest(request, response).catch((error) => { - json(response, 500, { ok: false, error: error.message, stack: process.env.NODE_ENV === "development" ? error.stack : undefined }); - }); - }); - const wss = new WebSocketServer({ noServer: true }); wss.on("connection", (socket) => { const sendSurface = () => { @@ -734,18 +774,27 @@ if (args.health) { socket.on("close", () => clearInterval(timer)); }); - server.on("upgrade", (request, socket, head) => { - const url = new URL(request.url, `http://${request.headers.host || `${args.host}:${args.port}`}`); - if (url.pathname === `/eve/deck/${providerId}` || url.pathname === "/eve/deck") { - wss.handleUpgrade(request, socket, head, (ws) => wss.emit("connection", ws, request)); - } else { - socket.destroy(); - } - }); - - server.listen(args.port, args.host, () => { - console.log(`Vili listening on http://${args.host}:${args.port}`); + const servers = hosts.map((host) => { + const server = http.createServer((request, response) => { + handleRequest(request, response).catch((error) => { + json(response, 500, { ok: false, error: error.message, stack: process.env.NODE_ENV === "development" ? error.stack : undefined }); + }); + }); + server.on("upgrade", (request, socket, head) => { + const url = new URL(request.url, `http://${request.headers.host || `${hosts[0]}:${args.port}`}`); + if (url.pathname === `/eve/deck/${providerId}` || url.pathname === "/eve/deck") { + wss.handleUpgrade(request, socket, head, (ws) => wss.emit("connection", ws, request)); + } else { + socket.destroy(); + } + }); + return new Promise((resolve, reject) => { + server.once("error", reject); + server.listen(args.port, host, resolve); + }); }); + await Promise.all(servers); + console.log(`Vili listening on ${hosts.map((host) => `http://${host}:${args.port}`).join(" ")}`); setInterval(() => persistSurfaces().catch((error) => console.error(error)), 5000); } diff --git a/scripts/vili-gpu-tray.ps1 b/scripts/vili-gpu-tray.ps1 new file mode 100644 index 0000000..b5e9e86 --- /dev/null +++ b/scripts/vili-gpu-tray.ps1 @@ -0,0 +1,98 @@ +# The household's switch for Raven's GPU: one tray icon in the interactive session. It shows whether +# Vili may use the GPU and flips it with one click. It holds no state of its own: the icon and tooltip +# are a lowering of GET /gpu-permission, and the click is one POST with an absolute value. +param([int]$Port = 8824, [int]$PollMs = 5000) + +Add-Type -AssemblyName System.Windows.Forms +Add-Type -AssemblyName System.Drawing + +$mutex = New-Object System.Threading.Mutex($false, 'Local\GameCult.ViliGpuTray') +if (-not $mutex.WaitOne(0)) { exit 0 } + +$base = "http://127.0.0.1:$Port/gpu-permission" +$script:state = 'down' +$script:jobRunning = $false +$clock = [Diagnostics.Stopwatch]::StartNew() +$script:lastToggleMs = -10000 + +function New-TrayIcon([string]$kind) { + $bmp = New-Object System.Drawing.Bitmap 16, 16 + $g = [System.Drawing.Graphics]::FromImage($bmp) + $g.SmoothingMode = 'AntiAlias' + if ($kind -eq 'on') { + $g.FillEllipse([System.Drawing.Brushes]::LimeGreen, 1, 1, 13, 13) + } elseif ($kind -eq 'off') { + $g.FillEllipse([System.Drawing.Brushes]::Crimson, 1, 1, 13, 13) + $g.FillRectangle([System.Drawing.Brushes]::White, 4, 7, 8, 2) + } else { + $pen = New-Object System.Drawing.Pen ([System.Drawing.Color]::Gray), 2 + $g.DrawEllipse($pen, 2, 2, 11, 11) + $pen.Dispose() + } + $g.Dispose() + $icon = [System.Drawing.Icon]::FromHandle($bmp.GetHicon()) + return $icon +} + +$icons = @{ on = (New-TrayIcon 'on'); off = (New-TrayIcon 'off'); down = (New-TrayIcon 'down') } + +$tray = New-Object System.Windows.Forms.NotifyIcon +$menu = New-Object System.Windows.Forms.ContextMenuStrip +$item = $menu.Items.Add('Let Vili use the GPU') +$tray.ContextMenuStrip = $menu + +function Show-State { + $tray.Icon = $icons[$script:state] + $item.Checked = ($script:state -eq 'on') + $item.Enabled = ($script:state -ne 'down') + if ($script:state -eq 'on' -and $script:jobRunning) { $tray.Text = 'Vili may use the GPU (making a clip). Click to stop.' } + elseif ($script:state -eq 'on') { $tray.Text = 'Vili may use the GPU. Click to stop.' } + elseif ($script:state -eq 'off') { $tray.Text = 'Vili may not use the GPU. Click to allow.' } + else { $tray.Text = 'Vili is not running, so it is not using the GPU.' } +} + +function Update-State { + try { + $r = Invoke-RestMethod -Uri $base -Method Get -TimeoutSec 2 + if ($r.allowed -eq $true) { $script:state = 'on' } elseif ($r.allowed -eq $false) { $script:state = 'off' } else { $script:state = 'down' } + $script:jobRunning = ($r.jobRunning -eq $true) + } catch { + $script:state = 'down' + $script:jobRunning = $false + } + Show-State +} + +function Switch-Permission { + try { + if ($clock.ElapsedMilliseconds - $script:lastToggleMs -lt 1000) { return } + $script:lastToggleMs = $clock.ElapsedMilliseconds + if ($script:state -eq 'down') { + $tray.ShowBalloonTip(3000, 'Vili', 'Vili is not running. Nothing to change.', 'Info') + return + } + $body = ConvertTo-Json -Compress @{ allowed = ($script:state -ne 'on') } + try { + [void](Invoke-RestMethod -Uri $base -Method Post -Body $body -ContentType 'application/json' -TimeoutSec 2) + } catch { + $tray.ShowBalloonTip(3000, 'Vili', 'Could not reach Vili. Nothing changed.', 'Warning') + } + Update-State + } catch { + } +} + +$tray.Add_MouseUp({ + param($sender, $e) + try { if ($e.Button -eq [System.Windows.Forms.MouseButtons]::Left) { Switch-Permission } } catch {} +}) +$item.Add_Click({ try { Switch-Permission } catch {} }) + +$timer = New-Object System.Windows.Forms.Timer +$timer.Interval = $PollMs +$timer.Add_Tick({ try { Update-State } catch {} }) + +Update-State +$tray.Visible = $true +$timer.Start() +[System.Windows.Forms.Application]::Run()