Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
3adb616497 | ||
|
|
f71aeedad1 | ||
|
|
9e79b00ce1 | ||
|
|
3fa19d7590 | ||
|
|
a3052ddf63 | ||
|
|
e37a2ad3ec | ||
|
|
cb36beb09a | ||
|
|
5a60255ac2 | ||
|
|
455cedb3c3 | ||
|
|
375c2faf4e | ||
|
|
736a1a7ab8 | ||
|
|
ed07c5a4af | ||
|
|
b489d2885d | ||
|
|
a2b9983183 | ||
|
|
bacc3faaa0 | ||
|
|
4ac85a65a5 | ||
|
|
5050de58b5 | ||
|
|
9ec048844b | ||
|
|
62485dab15 | ||
|
|
d921731a11 | ||
|
|
d9d396748d | ||
|
|
2b5b900eca | ||
|
|
a9a449225c | ||
|
|
880f1c2dcf | ||
|
|
38491f8917 | ||
|
|
54e680f284 |
18 changed files with 1139 additions and 184 deletions
31
README.md
31
README.md
|
|
@ -10,8 +10,8 @@ first backend is Kimodo on Raven, running through Docker Engine inside
|
|||
|
||||
- **Value:** a warm, supervised path from a text prompt to generated body motion. Vili runs Kimodo as a resident GPU worker on Raven and returns motion artifacts (`motion.npz`, optional BVH) for a job.
|
||||
- **Problem:** Personas and games need generated motion, but cold per-request container runs took about 195 seconds per 1 second clip, versus about 54 warm. One daemon owns job identity, worker lifecycle and artifact references so callers do not touch Docker, WSL or model state.
|
||||
- **Entry points:** `scripts/vili-daemon.mjs` (the daemon; `npm start`, `npm run health`, `npm run smoke`, `npm run provider`), `scripts/kimodo-resident-worker.py` (the resident worker), `POST /motion/generate` (see Motion Generation), `docs/verse-service-contract.md` and `docs/motion-pipeline-target.md`.
|
||||
- **Live:** the resident Kimodo worker on Raven, `POST /motion/generate`, `GET /smoke`, and the CultCache service store. It runs under a Raven scheduled task, not under Idunn. It has `motion.npz` and optional BVH output on the SOMA skeleton; there is no FBX export.
|
||||
- **Entry points:** `scripts/vili-daemon.mjs` (the daemon; `npm start`, `npm run health`, `npm run provider`), `scripts/kimodo-resident-worker.py` (the resident worker), `POST /motion/generate` (see Motion Generation), `docs/verse-service-contract.md` and `docs/motion-pipeline-target.md`.
|
||||
- **Live:** the resident Kimodo worker on Raven, `POST /motion/generate`, and the CultCache service store. It runs under a Raven scheduled task, not under Idunn. It has `motion.npz` and optional BVH output on the SOMA skeleton; there is no FBX export.
|
||||
- **Planned (target, 2026-10-20):** FBX animation export on the SOMA skeleton and a text-prompt-to-Unity-clip path with no hand steps, with each artifact recording prompt, constraints, seed and model. Retargeting stays with the consuming engine; see `docs/motion-pipeline-target.md`.
|
||||
|
||||
## Authority
|
||||
|
|
@ -41,6 +41,11 @@ Default compatibility endpoint:
|
|||
http://10.77.0.4:8824
|
||||
```
|
||||
|
||||
The daemon listens only on the addresses `--host` names, comma-separated, one server each: on
|
||||
Raven `127.0.0.1` (the tray and `--health`) and `10.77.0.4` (the WireGuard peers: Starfire and
|
||||
the services that call `POST /motion/generate`). A wildcard such as `0.0.0.0` refuses to start.
|
||||
No response carries a CORS header, so a web page cannot read the daemon either; no browser
|
||||
caller exists.
|
||||
Vili is not an Idunn target today. Its lifecycle is the `GameCult\Vili`
|
||||
scheduled task on Raven, and `GET /health` is an operator lowering. Admission
|
||||
to Idunn is motion-pipeline:question:vili-under-idunn.
|
||||
|
|
@ -63,7 +68,6 @@ Important routes:
|
|||
- `GET /eve/operator`
|
||||
- `GET /eve/deck/providers`
|
||||
- `WS /eve/deck/vili.animation`
|
||||
- `GET /smoke`
|
||||
- `POST /motion/generate`
|
||||
|
||||
## Motion Generation
|
||||
|
|
@ -145,8 +149,8 @@ Vili keeps Kimodo warm through a resident worker container named
|
|||
the loaded model. Because Raven's Docker Engine runs inside WSL, Vili also
|
||||
keeps a tiny WSL sleep-loop alive while the worker is resident so the Docker
|
||||
substrate does not disappear underneath the model. `GET /health` observes
|
||||
worker state without launching model load. `GET /smoke` verifies the backend
|
||||
and starts the resident worker when it is absent.
|
||||
worker state without launching model load and without changing anything. Only a
|
||||
`POST /motion/generate` that holds the GPU slot starts the worker.
|
||||
|
||||
Kimodo's 8B text encoder (LLM2Vec, Llama 3) never lives in the worker. The
|
||||
worker answers Kimodo's encoder call from a CultCache store of float32 vectors,
|
||||
|
|
@ -159,12 +163,25 @@ Kimodo normalizes the prompt before encoding (`A person shrugs sadly.`), and the
|
|||
cache keys on that text. A request's misses go to one transient child,
|
||||
`scripts/kimodo-text-encode.py`: it loads the encoder in bf16 on the GPU, with the
|
||||
layers that do not fit in free VRAM held in RAM, encodes, writes its answer and
|
||||
exits, so the OS reclaims its RAM and VRAM. It refuses with a typed `gpu-busy`
|
||||
below 2 GiB of free VRAM and has no CPU fallback; hits never touch it. Each
|
||||
exits, so the OS reclaims its RAM and VRAM. It sizes its own placement from the free VRAM torch
|
||||
can see and refuses a placement below 2 GiB with a typed `gpu-busy` (source `placement`);
|
||||
it has no CPU fallback, and hits never touch it. Each
|
||||
`/motion/generate` job carries the encoder record (hits, misses, seconds, vector
|
||||
hashes) and the worker identity. `GET /health` shows the identity and the stored
|
||||
count.
|
||||
|
||||
GPU permission. Raven's GPU is the household's first. A tray icon in the household's session
|
||||
(`scripts/vili-gpu-tray.ps1`, logon task `GameCult\ViliTray`, installed by
|
||||
`scripts/install-vili-tray-task.ps1`) shows whether Vili may use the GPU and flips it with one click.
|
||||
The daemon owns the yes or no in `vili.service.cc` (`gamecult.vili.gpu_permission.v1`): off on a
|
||||
fresh install, set only from loopback (`POST /gpu-permission { allowed }`, read at
|
||||
`GET /gpu-permission`). A `POST /motion/generate` is refused with HTTP 503, `retry-after: 60` and
|
||||
`{ ok: false, code: "gpu-busy", source }` where `source` is `household` (the permission is off),
|
||||
`slot` (another job holds the one GPU slot) or `vram` (host `nvidia-smi.exe` free VRAM is below
|
||||
`VILI_GPU_MIN_FREE_MIB`, default 10240, or unreadable); nothing queues, and the consumer owns the
|
||||
retry. A job already running finishes; the gate never stops work. `operator_state.gpu` is
|
||||
`{ allowed, changedAt, jobRunning, lastRefusal }`. `npm test` runs the gate's tests.
|
||||
|
||||
Measured on Raven on `2026-10-22`, for a 2 second clip at 20 diffusion steps:
|
||||
the worker is healthy 29 seconds after `docker rm` (was 179) and rests at 2.3 GiB
|
||||
resident (was 10.4 GiB, with 5.8 GB swapped). A cache miss takes 59 to 103
|
||||
|
|
|
|||
|
|
@ -7,8 +7,33 @@ cuts not yet mapped. The target's rationale is `docs/motion-pipeline-target.md`.
|
|||
|
||||
## Body facts
|
||||
|
||||
### Raven tray-toggle probes, 2026-10-22 23:30 to 23:36 UTC
|
||||
|
||||
Run read-only from Starfire over `ssh raven` (session 0, elevated), one
|
||||
PowerShell 5.1 script copied to `C:\Users\Public` and deleted in the same
|
||||
command. Nothing on Raven was changed: no task edits, no restarts, no GPU jobs.
|
||||
They inform cut spec `motion-pipeline:cut_spec:cut-raven-gpu-tenancy-gate.r2`,
|
||||
which replaces detection with the household's toggle
|
||||
(`motion-pipeline:ruling:raven-gpu-tray-toggle`).
|
||||
|
||||
| Probe | Result |
|
||||
|---|---|
|
||||
| Who the household is on Raven | One account. `query user`: `madman's lullaby`, console, session 1, Active since 2026-10-18. The `\GameCult\Vili` task runs as that same user (`LogonType Interactive`, `RunLevel Limited`), so a tray process at that user's logon is in the same session and the same user as the daemon, with no admin rights. |
|
||||
| Daemon listen address | `node.exe ... vili-daemon.mjs --host 0.0.0.0 --port 8824 --state-root E:\Projects\Vili\.vili`, pid 33776, session 1, at detached head `a2b9983`. Listens on `0.0.0.0:8824`, so the WireGuard peers can reach every route; a write route must check the caller itself. `GET http://127.0.0.1:8824/operator-state` from Raven answers in 3 ms. |
|
||||
| Live task action versus the installer | The live `\GameCult\Vili` action passes `-IdunnRudpHealth 10.77.0.2:17870 -IdunnDaemon vili -IdunnHealthContract ... -IdunnHealthIntervalSeconds 15` to `start-vili-daemon.ps1`; `scripts/install-vili-task.ps1` does not. Rerunning that installer would rewrite the live task, so the tray gets its own installer. |
|
||||
| Tray substrate | `powershell.exe` 5.1.26100, apartment `STA` by default; `System.Windows.Forms` and `System.Drawing` load (`v4.0.30319`, .NET Framework release 534296). Execution policy is Undefined at every scope (the task launcher passes `-ExecutionPolicy Bypass`). `scripts/run-hidden-powershell.vbs` already starts a hidden PowerShell with no console flash. No new dependency is needed. |
|
||||
| Tray overflow | `HKCU:\Control Panel\NotifyIconSettings` exists with 47 entries: Windows 11 records each notify icon there and puts new ones in the overflow until the user promotes it. A fresh tray icon will not be on the taskbar until the household drags it there once (or turns it on under Settings, Personalization, Taskbar, Other system tray icons). |
|
||||
| Startup folder | `C:\Users\Madman's Lullaby\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup`. Not used: from SSH (session 0) nothing in it can be started in session 1, while `Start-ScheduledTask` on an Interactive task can. |
|
||||
| Scheduled tasks under `\GameCult\` | `Backup-Yggdrasil-Pull`, `ComfyUI`, `Vili`, all Ready. ComfyUI is a second possible heavy GPU consumer on Raven; the toggle governs Vili only. |
|
||||
| Host free VRAM | `C:\WINDOWS\system32\nvidia-smi.exe --query-gpu=memory.free --format=csv,noheader,nounits`: 159 ms from a cold PowerShell, exit 0, free 5001 MiB at 23:3x UTC (the household or another consumer held about 11 GiB). No `nvidia-smi.exe` process was left running. |
|
||||
| Vili's store | `E:\Projects\Vili\.vili\vili.service.cc`, 11810 bytes. Newest job file `vili-1792710792764.json`, written 23:20:12 UTC. The live gate (`a2b9983`) read `inUse true, probeOk true`. |
|
||||
| cultcache-ts on Raven | `E:\Projects\CultLib` at `b9497d35`, package `0.14.1`; its build carries `pullAllBackingStores`. `SingleFileMessagePackBackingStore.push` reads the file, replaces one `(type, key)` entry and writes all, inside a write queue that belongs to one store instance. |
|
||||
|
||||
### Raven GPU tenancy probes, 2026-10-22 20:15 to 20:23 UTC
|
||||
|
||||
These informed r1's detection gate, which r2 deletes. The session, principal and
|
||||
`nvidia-smi` rows still hold.
|
||||
|
||||
Run read-only from Starfire over `ssh -o HostName=192.168.178.165 raven`
|
||||
(alias in `~/.ssh/config`; LAN address per `gamecult-ops/inventory.md`), as
|
||||
`powershell -NoProfile -NonInteractive -EncodedCommand`. Nothing on Raven was
|
||||
|
|
@ -21,56 +46,83 @@ category (compositor, WSL family, or "other").
|
|||
| Session of the Vili daemon | `node.exe ... vili-daemon.mjs --host 0.0.0.0 --port 8824` runs in **session 1**, the console session of `madman's lullaby` (`query user`: console, ID 1, Active). It is the interactive desktop session, not session 0. |
|
||||
| Vili task principal | `\GameCult\Vili`: `LogonType Interactive`, `RunLevel Limited`, boot trigger (`MSFT_TaskBootTrigger`), last run 2026-10-22 21:18:55 local, result 0. Children the daemon spawns inherit session 1. |
|
||||
| SSH context | Windows `sshd` runs commands in **session 0**, elevated (`IsInRole(Administrator)` true). `gamecult-ops/runbooks/idunn-host-raven.md` already records that sshd cannot start anything in the user's session without a scheduled-task hop. |
|
||||
| `SHQueryUserNotificationState` from session 0 | `hr=0`, state `2` (`QUNS_BUSY`) on every call, with the GPU idle (3D engine total 0.01 to 0.02 %). Session 0 has no shell, so this value says nothing about the user's desktop. The call must run in session 1, which is where the daemon and its children are. It was **not** observed from session 1: doing that without a task edit is impossible from SSH. |
|
||||
| `nvidia-smi.exe` | Present at `C:\WINDOWS\system32\nvidia-smi.exe`. `--query-gpu=memory.total,memory.used,memory.free,utilization.gpu`: `16380, 1592, 14516, 2` MiB/%. 63 ms cold, about 49 ms per call (10 calls 491 ms). Per-process `--query-compute-apps` returns `[N/A]` memory under WDDM. |
|
||||
| `\GPU Engine(*engtype_3D)\Utilization Percentage` | Readable from session 0 as admin; 115 instances. `Get-Counter -SampleInterval 1 -MaxSamples 1` takes 1.08 s wall, 16 ms CPU warm, 328 ms CPU on first use in a process. Summed with `Measure-Object` it needs no instance names. Not tried as a Limited token in session 1. |
|
||||
| `\GPU Adapter Memory(*)\Dedicated Usage` | Readable; 1.0 s wall; 1597 MiB on the RTX 4060 Ti, 0 on the Parsec virtual adapter. |
|
||||
| `\GPU Process Memory(*)\Dedicated Usage` | Readable; 1.0 s wall. Two instances above 50 MiB: the desktop compositor (1643 MiB, session 1) and one session-0 process outside the WSL family (1405 MiB). Per-process figures overlap: they sum above the adapter total. |
|
||||
| WSL attribution | The Kimodo worker container was resident (`/health` ok, uptime 3454 s, 6 generations) while `vmmemWSL` (and `vmwp`) had **no** `GPU Process Memory` instance. Host counters do not attribute WSL CUDA memory to a nameable host process. WSL `nvidia-smi` reported the same `1592 MiB used, 14516 MiB free` as the host. |
|
||||
| PowerShell cost | `powershell.exe` 5.1 cold start 266 ms CPU, `Add-Type` for the shell32 call 78 ms CPU, first counter read 328 ms CPU: about 0.7 s CPU per cold spawn. A warm loop iteration (QUNS + `nvidia-smi` + one 3D counter read) is about 1.1 s wall and 16 ms of its own CPU. |
|
||||
| `SHQueryUserNotificationState` from session 0 | `hr=0`, state `2` (`QUNS_BUSY`) on every call, with the GPU idle. Session 0 has no shell, so this value says nothing about the user's desktop. |
|
||||
| `nvidia-smi.exe` | Present at `C:\WINDOWS\system32\nvidia-smi.exe`. `--query-gpu=memory.total,memory.used,memory.free,utilization.gpu`: `16380, 1592, 14516, 2` MiB/%. 63 ms cold, about 49 ms per call. Per-process `--query-compute-apps` returns `[N/A]` memory under WDDM. |
|
||||
| `\GPU Engine(*engtype_3D)\Utilization Percentage` | Readable from session 0 as admin; 115 instances; `Get-Counter` 1.08 s wall. |
|
||||
| `\GPU Process Memory(*)\Dedicated Usage` | Per-process figures overlap and sum above the adapter total. |
|
||||
| WSL attribution | Host counters do not attribute WSL CUDA memory to a nameable host process. Host `nvidia-smi.exe` does see a WSL container's allocation: cut report h1 measured free falling 14292 to 1866 MiB under a 12 GiB holder. |
|
||||
| Adapters | `NVIDIA GeForce RTX 4060 Ti` driver 32.0.16.1088, 16 GiB; `Parsec Virtual Display Adapter`. |
|
||||
|
||||
What follows from these, and what does not:
|
||||
### What the detection gate taught (r1, h1, h2, s1, s2)
|
||||
|
||||
- The daemon's own session sees the user's desktop, so the tenancy probe runs as
|
||||
a child of the daemon. No fork about where it runs: the scheduled task already
|
||||
puts it in the right session.
|
||||
- Free VRAM must come from the host (`nvidia-smi.exe`), not from torch in WSL:
|
||||
Soul's s1 probe on cut `vili-encoder-placement` showed `mem_get_info` inside a
|
||||
container blind to another container's 12 GiB. Whether host `nvidia-smi` sees
|
||||
a WSL container's allocation is **unverified** (the resident worker's share
|
||||
could not be separated from the compositor's); the gate cut checks it first.
|
||||
- The 3D-engine signal cannot be used to tell Vili's own work from the
|
||||
household's by process, and need not be: Kimodo runs on CUDA (compute
|
||||
engines), games on the 3D engine. Summing only `engtype_3D` leaves Vili out.
|
||||
- A long-lived probe child costs about 1 ms of CPU per second at a 15 s period;
|
||||
a fresh PowerShell per sample would cost about 45 ms per second. Neither
|
||||
loads the GPU.
|
||||
- Host free VRAM cannot tell Vili's encoder child from a household load: the
|
||||
child sizes itself to the VRAM that was free (s1 `self-tenancy-refusal`).
|
||||
Discounting Vili's own VRAM during a job blinds the gate to a household load
|
||||
that starts mid-job (s2 `hold-blinds-household-vram`).
|
||||
- Live, the probe child went silent during 2 of 5 of Vili's own encoder jobs and
|
||||
the gate stayed stale after the job (s2 `own-load-starves-probe`; Raven had
|
||||
770 MB of host RAM available at one stall). Every repair added a watchdog, a
|
||||
drain or a baseline, and each brought its own finding.
|
||||
- After a job ends, host free VRAM is back at 14.6 to 15.2 GiB at the next
|
||||
2-second sample (s2), so one `nvidia-smi` read before a job, with no job
|
||||
holding the slot, sees the card as the household left it.
|
||||
|
||||
## Model page
|
||||
|
||||
| Kind | Named by | Over time | Decided by |
|
||||
|---|---|---|---|
|
||||
| GPU tenancy state (`gpu` in `gamecult.vili.operator_state.v1`) | the daemon, one per host | rewritten every 5 s with the operator state; holds `inUse`, `since`, `checkedAt`, `probeOk` and nothing else | the daemon's tenancy gate, from host samples; never persisted as truth, rebuilt from samples at start |
|
||||
| GPU permission (`gamecult.vili.gpu_permission.v1` in `.vili/vili.service.cc`, key `raven`) | the daemon, one per host | written only when the household changes it, before the change is acknowledged; read once at daemon start; absent means not allowed | the household, through the tray, through the daemon's loopback-only `POST /gpu-permission`; the daemon is its one writer |
|
||||
| GPU status (`gpu` in `gamecult.vili.operator_state.v1`) | the daemon | rewritten every 5 s with the operator state; `{ allowed, changedAt, jobRunning, lastRefusal }` | derived from the permission, the slot and the last refusal; display only, never read back |
|
||||
| Motion job (`.vili/jobs/<jobId>.json`) | `vili-<epoch ms>` | written once when a job ends | the daemon; a job refused at admission writes none |
|
||||
| Text embedding (`.vili/embeddings/text-embeddings.cc`) | `sha256(identity, text)` | append on miss | the worker's cache |
|
||||
|
||||
## Rationale
|
||||
|
||||
- **Refuse, do not queue (recommended, pending the operator).** `/motion/generate`
|
||||
is synchronous today. Holding a request open for as long as a game runs, or
|
||||
adding a persisted job queue, are bigger machines than the one consumer needs:
|
||||
RPS clip generation is a script an agent runs, and it can retry a typed
|
||||
`gpu-busy`. A queue belongs with the authoring UX's takes, when that target
|
||||
exists (`motion-pipeline:follow_up:vili-authoring-ux-target-2`).
|
||||
- **Fail closed.** If the probe child dies or stops answering, the gate reports
|
||||
in use. A wrongly deferred clip costs minutes; a wrongly admitted encoder child
|
||||
pages a game's VRAM and thrashes Raven.
|
||||
- **The household decides; nothing detects.** The operator replaced detection
|
||||
with a tray toggle (`motion-pipeline:ruling:raven-gpu-tray-toggle`). The
|
||||
detection gate's findings were the cost of guessing intent from counters; a
|
||||
yes from the person at the desk needs no guess, no hysteresis and no probe.
|
||||
- **The daemon is the one owner of the permission; the tray is a lowering.**
|
||||
The tray reads and sets the state only through the daemon's HTTP route, so
|
||||
admission and the stored value cannot disagree and the tray holds no state of
|
||||
its own. The alternative, the tray writing a CultCache document the daemon
|
||||
reads, would give the document two writers on two clocks and make the daemon
|
||||
poll a file to learn its own admission rule.
|
||||
- **Off on a fresh install, and off when the document is missing.** The safe
|
||||
default for the household: Vili does nothing on Raven's GPU until someone at
|
||||
the desk says yes. A stored yes survives restarts and reboots.
|
||||
- **Loopback only for the write.** The daemon listens on `0.0.0.0`; the
|
||||
household's switch must not be flippable from a WireGuard peer. A request
|
||||
from a non-loopback address is refused 403. Agents read the state through
|
||||
`GET /gpu-permission` or `/operator-state` and never set it.
|
||||
- **One cache instance for the daemon.** The 5-second surface writer built a
|
||||
new store per call. Two store instances writing one file each read, modify and
|
||||
write in their own queue, so a permission write could be lost under a
|
||||
concurrent surface write. One module-level cache serializes every write to
|
||||
`vili.service.cc`.
|
||||
- **One slot, one fresh VRAM read, fail closed.** h2's slot stays: one job at a
|
||||
time, the rest refused at once with source `slot`. The slot is taken
|
||||
synchronously before the VRAM read, so two requests cannot both pass it. A
|
||||
failed or unparseable read refuses with source `vram`, so a job never starts
|
||||
onto a full card on a guess. The permission is checked again after the read.
|
||||
- **Refuse, do not queue** (`motion-pipeline:ruling:gpu-defer-refuse-typed`).
|
||||
`/motion/generate` is synchronous; the consumer retries a typed `gpu-busy`
|
||||
after `retry-after`. A queue belongs with the authoring UX's takes
|
||||
(`motion-pipeline:follow_up:vili-authoring-ux-target-2`).
|
||||
- **A logon task, not the Startup folder.** `Start-ScheduledTask` on an
|
||||
Interactive task puts the tray in the household's session from SSH at deploy
|
||||
time; a Startup folder entry would wait for the next logon. The task pattern
|
||||
matches `\GameCult\Vili`.
|
||||
- **The privacy rule is moot.** "Never record program names" protected the
|
||||
household from a probe that watched its programs. With no probe, nothing in
|
||||
Vili observes the household's programs; the only household fact Vili holds is
|
||||
the yes or no the household chose.
|
||||
- **The child's own VRAM check is placement, not tenancy.** `plan_placement`
|
||||
still sizes accelerate's `max_memory` from torch's view and can still refuse;
|
||||
it answers "can the encoder be placed", not "may heavy work start".
|
||||
still sizes accelerate's `max_memory` from torch's view and can still refuse
|
||||
(source `placement`); it answers "can the encoder be placed", not "may heavy
|
||||
work start".
|
||||
|
||||
## Cuts not yet mapped
|
||||
|
||||
- RPS beats through Vili (`motion-pipeline:follow_up:rps-clips-from-vili`), after
|
||||
the gate and `vili-fbx-export`.
|
||||
the gate and `vili-fbx-export`. They wait on the household's yes on Raven.
|
||||
|
|
|
|||
|
|
@ -45,7 +45,22 @@ promoted from JSON once that request/receipt shape stabilizes.
|
|||
`POST /motion/generate` accepts an explicit animation request and forwards the
|
||||
supported Kimodo generation controls to the resident worker. Vili owns job
|
||||
identity, request persistence, worker lifecycle, timeout handling, and the
|
||||
artifact stem. Kimodo owns synthesis and model-specific export.
|
||||
artifact stem. Kimodo owns synthesis and model-specific export. Vili also owns GPU
|
||||
admission from the household's permission: the daemon holds one GPU slot, and `POST /motion/generate` is judged
|
||||
when it reaches the slot. It answers HTTP 503 with `retry-after: 60` and one of four bodies, by
|
||||
`source`: `{ "ok": false, "code": "gpu-busy", "source": "household", "gpu": { allowed: false,
|
||||
changedAt } }` while the household's permission is off; `{ "ok": false, "code": "gpu-busy",
|
||||
"source": "slot" }` while another job holds the slot (nothing queues); `{ "ok": false, "code":
|
||||
"gpu-busy", "source": "vram", freeMiB, minFreeMiB }` when host free VRAM is below the minimum or
|
||||
unreadable (`freeMiB` null); and `{ "ok": false, "code": "gpu-busy", "source": "placement",
|
||||
"jobId" }` when the encoder child refused its own placement, where the job file was written and no
|
||||
household status is claimed. `GET /gpu-permission` answers `{ allowed, changedAt, jobRunning,
|
||||
lastRefusal }` to anyone; `POST /gpu-permission { "allowed": boolean }` is the household's switch and
|
||||
answers 403 `{ "ok": false, "code": "household-only" }` to any caller that is not on the host's
|
||||
loopback. Both POST routes require `Content-Type: application/json` and refuse a request whose
|
||||
`Origin` header is present and not the daemon's own loopback origin with 403 `{ "ok": false, "code":
|
||||
"cross-site" }`, so a web page cannot drive them; the daemon answers no CORS preflight. The `code` field is the only carrier; a consumer retries after `retry-after`, never sets
|
||||
the permission, and never matches error text. A job already running finishes.
|
||||
|
||||
```json
|
||||
{
|
||||
|
|
@ -143,7 +158,7 @@ It mounts:
|
|||
|
||||
Inside the worker, Kimodo owns synthesis. Vili owns worker lifecycle,
|
||||
request/response persistence, timeout handling, and artifact references. Direct
|
||||
user requests, programmatic requests, and smoke-started warmup all use the same
|
||||
user requests and programmatic requests use the same
|
||||
resident worker health and generation path.
|
||||
|
||||
Observed on `2026-06-12`: first worker model load took about 100 seconds. Once
|
||||
|
|
@ -154,5 +169,5 @@ resident, five 1 second / 20 step service-path generations averaged about
|
|||
|
||||
Health checks must be cheap. They may verify process, port, backend metadata,
|
||||
and cached readiness, but they must not launch a Kimodo generation job or start
|
||||
model load. Smoke checks may start the resident worker because their purpose is
|
||||
backend readiness, not cheap liveness.
|
||||
model load. They change nothing: no GET route starts the worker, Docker or the GPU. Only a
|
||||
`POST /motion/generate` that holds the GPU slot starts the worker.
|
||||
|
|
|
|||
|
|
@ -7,7 +7,8 @@
|
|||
"scripts": {
|
||||
"start": "node scripts/vili-daemon.mjs",
|
||||
"health": "node scripts/vili-daemon.mjs --health",
|
||||
"smoke": "node scripts/smoke-vili.mjs",
|
||||
"test": "node --test scripts/test-gpu-tenancy.mjs",
|
||||
"test:http": "node --test scripts/test-gpu-gate-http.mjs",
|
||||
"provider": "node scripts/vili-daemon.mjs --print-provider-advertisement"
|
||||
},
|
||||
"dependencies": {
|
||||
|
|
|
|||
103
scripts/gpu-tenancy.mjs
Normal file
103
scripts/gpu-tenancy.mjs
Normal file
|
|
@ -0,0 +1,103 @@
|
|||
// GPU tenancy gate: the one owner of "may heavy GPU work start now". The household decides, through
|
||||
// the permission it sets at the tray; the daemon owns that yes or no, the one GPU slot, and one host
|
||||
// free-VRAM read per admission. The gate refuses new work; it never stops work.
|
||||
//
|
||||
// acquire() judges in a fixed order and nothing waits: the permission, then the slot, then host free
|
||||
// VRAM. The slot is taken before the VRAM read so concurrent requests are refused at once.
|
||||
|
||||
export const RETRY_AFTER_SECONDS = 60;
|
||||
|
||||
export function createGpuTenancy({ now, permission, savePermission, readFreeMiB, minFreeMiB = 10240 }) {
|
||||
let allowed = permission?.allowed === true;
|
||||
let changedAt = permission?.changedAt ?? null;
|
||||
let held = false;
|
||||
let lastRefusal = null;
|
||||
|
||||
const status = () => ({ allowed, changedAt, jobRunning: held, lastRefusal });
|
||||
|
||||
function refuse(source, extra = {}) {
|
||||
lastRefusal = { at: new Date(now()).toISOString(), source };
|
||||
return { refusal: { ok: false, code: "gpu-busy", source, ...extra } };
|
||||
}
|
||||
const household = () => refuse("household", { gpu: { allowed: false, changedAt } });
|
||||
|
||||
async function setAllowed(next) {
|
||||
if (typeof next !== "boolean") throw new TypeError("allowed must be a boolean");
|
||||
if (next === allowed) return status();
|
||||
const value = { allowed: next, changedAt: new Date(now()).toISOString() };
|
||||
await savePermission(value);
|
||||
allowed = value.allowed;
|
||||
changedAt = value.changedAt;
|
||||
return status();
|
||||
}
|
||||
|
||||
async function acquire() {
|
||||
if (!allowed) return household();
|
||||
if (held) return refuse("slot");
|
||||
held = true;
|
||||
let freeMiB;
|
||||
try {
|
||||
freeMiB = await readFreeMiB();
|
||||
} catch {
|
||||
freeMiB = null;
|
||||
}
|
||||
if (!allowed) {
|
||||
held = false;
|
||||
return household();
|
||||
}
|
||||
if (!Number.isFinite(freeMiB) || freeMiB < minFreeMiB) {
|
||||
held = false;
|
||||
return refuse("vram", { freeMiB: Number.isFinite(freeMiB) ? freeMiB : null, minFreeMiB });
|
||||
}
|
||||
let released = false;
|
||||
return {
|
||||
release() {
|
||||
if (released) return;
|
||||
released = true;
|
||||
held = false;
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
return { status, setAllowed, acquire };
|
||||
}
|
||||
|
||||
// Takes the slot, runs the job to its end and returns its result, or returns the typed refusal
|
||||
// without calling the job. The gate never holds, pauses or cancels a job.
|
||||
export async function admitThen(gate, job) {
|
||||
const slot = await gate.acquire();
|
||||
if (slot.refusal) return slot.refusal;
|
||||
try {
|
||||
return await job();
|
||||
} finally {
|
||||
slot.release();
|
||||
}
|
||||
}
|
||||
|
||||
// nvidia-smi's first line is the free MiB as a base-10 integer; anything else is null.
|
||||
export function parseFreeMiB(stdout) {
|
||||
const line = String(stdout).split("\n")[0].trim();
|
||||
return /^\d+$/.test(line) ? Number(line) : null;
|
||||
}
|
||||
|
||||
// The encoder child's placement refusal reaches the daemon as a typed code on the worker's error body;
|
||||
// it is never read out of error text. Placement is the child's own view of free VRAM, not the
|
||||
// household gate, so the body carries no gate status. Returns the typed 503 body, or null for any
|
||||
// other outcome.
|
||||
export function placementRefusal(workerBody, jobId) {
|
||||
if (workerBody?.code !== "gpu-busy") return null;
|
||||
return { ok: false, code: "gpu-busy", source: "placement", jobId };
|
||||
}
|
||||
|
||||
// Operator override. A malformed value would silently defeat a comparison (NaN is never below
|
||||
// anything), so it refuses to start instead, naming the variable and never its value.
|
||||
export function gpuTenancyOptionsFromEnv(env) {
|
||||
const options = {};
|
||||
const raw = env.VILI_GPU_MIN_FREE_MIB;
|
||||
if (raw !== undefined) {
|
||||
const value = raw.trim() === "" ? Number.NaN : Number(raw);
|
||||
if (!Number.isFinite(value) || value < 0) throw new Error("VILI_GPU_MIN_FREE_MIB must be a finite non-negative number");
|
||||
options.minFreeMiB = value;
|
||||
}
|
||||
return options;
|
||||
}
|
||||
|
|
@ -1,7 +1,7 @@
|
|||
param(
|
||||
[string]$TaskName = "GameCult\Vili",
|
||||
[int]$Port = 8824,
|
||||
[string]$HostName = "0.0.0.0",
|
||||
[string]$HostName = "127.0.0.1,10.77.0.4",
|
||||
[string]$StateRoot = "E:\Projects\Vili\.vili"
|
||||
)
|
||||
|
||||
|
|
|
|||
17
scripts/install-vili-tray-task.ps1
Normal file
17
scripts/install-vili-tray-task.ps1
Normal file
|
|
@ -0,0 +1,17 @@
|
|||
param(
|
||||
[string]$TaskName = "GameCult\ViliTray",
|
||||
[int]$Port = 8824
|
||||
)
|
||||
|
||||
$ErrorActionPreference = "Stop"
|
||||
$ProgressPreference = "SilentlyContinue"
|
||||
$HiddenLauncher = Join-Path $PSScriptRoot "run-hidden-powershell.vbs"
|
||||
$TrayScript = Join-Path $PSScriptRoot "vili-gpu-tray.ps1"
|
||||
$UserId = [System.Security.Principal.WindowsIdentity]::GetCurrent().Name
|
||||
$Action = New-ScheduledTaskAction -Execute "wscript.exe" -Argument "//B //nologo `"$HiddenLauncher`" -NoProfile -NonInteractive -WindowStyle Hidden -ExecutionPolicy Bypass -STA -File `"$TrayScript`" -Port $Port"
|
||||
$Trigger = New-ScheduledTaskTrigger -AtLogOn -User $UserId
|
||||
$Principal = New-ScheduledTaskPrincipal -UserId $UserId -LogonType Interactive -RunLevel Limited
|
||||
$Settings = New-ScheduledTaskSettingsSet -AllowStartIfOnBatteries -DontStopIfGoingOnBatteries -ExecutionTimeLimit ([TimeSpan]::Zero) -MultipleInstances IgnoreNew -Hidden
|
||||
|
||||
Register-ScheduledTask -TaskName $TaskName -Action $Action -Trigger $Trigger -Principal $Principal -Settings $Settings -Force | Out-Null
|
||||
Write-Host "Installed scheduled task $TaskName"
|
||||
|
|
@ -4,7 +4,6 @@ import json
|
|||
import os
|
||||
import shutil
|
||||
import time
|
||||
import traceback
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from threading import Lock
|
||||
from types import SimpleNamespace
|
||||
|
|
@ -394,11 +393,7 @@ class Handler(BaseHTTPRequestHandler):
|
|||
payload = json.loads(self.rfile.read(length).decode("utf-8") or "{}")
|
||||
write_json(self, 200, generate(payload))
|
||||
except Exception as error:
|
||||
write_json(self, 500, {
|
||||
"ok": False,
|
||||
"error": str(error),
|
||||
"traceback": traceback.format_exc(),
|
||||
})
|
||||
write_json(self, 500, text_encoding.worker_error_body(error))
|
||||
|
||||
def log_message(self, fmt, *values):
|
||||
print(f"{self.address_string()} - {fmt % values}", flush=True)
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
param(
|
||||
[int]$Port = 8824,
|
||||
[string]$HostName = "0.0.0.0",
|
||||
[string]$HostName = "127.0.0.1,10.77.0.4",
|
||||
[string]$StateRoot = "E:\Projects\Vili\.vili"
|
||||
)
|
||||
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
param(
|
||||
[int]$Port = 8824,
|
||||
[string]$HostName = "0.0.0.0",
|
||||
[string]$HostName = "127.0.0.1,10.77.0.4",
|
||||
[string]$StateRoot = "E:\Projects\Vili\.vili"
|
||||
)
|
||||
|
||||
|
|
|
|||
|
|
@ -1,36 +0,0 @@
|
|||
#!/usr/bin/env node
|
||||
const baseUrl = process.env.VILI_URL || process.argv[2] || "http://10.77.0.4:8824";
|
||||
|
||||
async function get(path) {
|
||||
const response = await fetch(`${baseUrl}${path}`);
|
||||
const text = await response.text();
|
||||
let body;
|
||||
try {
|
||||
body = JSON.parse(text);
|
||||
} catch {
|
||||
body = text;
|
||||
}
|
||||
if (!response.ok) {
|
||||
throw new Error(`${path} returned ${response.status}: ${text.slice(0, 500)}`);
|
||||
}
|
||||
return body;
|
||||
}
|
||||
|
||||
const health = await get("/health");
|
||||
const providers = await get("/eve/deck/providers");
|
||||
const smoke = await get("/smoke");
|
||||
|
||||
console.log(JSON.stringify({
|
||||
ok: true,
|
||||
baseUrl,
|
||||
health: {
|
||||
service: health.service,
|
||||
providerId: health.providerId,
|
||||
backend: health.state?.backend,
|
||||
},
|
||||
providers,
|
||||
smoke: {
|
||||
ok: smoke.ok,
|
||||
backend: smoke.backendStatus,
|
||||
},
|
||||
}, null, 2));
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
param(
|
||||
[int]$Port = 8824,
|
||||
[string]$HostName = "0.0.0.0",
|
||||
[string]$HostName = "127.0.0.1,10.77.0.4",
|
||||
[string]$StateRoot = "E:\Projects\Vili\.vili"
|
||||
)
|
||||
|
||||
|
|
|
|||
417
scripts/test-gpu-gate-http.mjs
Normal file
417
scripts/test-gpu-gate-http.mjs
Normal file
|
|
@ -0,0 +1,417 @@
|
|||
// Boots the real daemon and observes the gate where it is decided: the HTTP edge. Needs the
|
||||
// daemon's own dependencies (ws, cultcache-ts), so it runs on a machine that has them. Every boot
|
||||
// sets VILI_GPU_MIN_FREE_MIB=999999 so an allowed request ends in a VRAM refusal and never starts
|
||||
// the worker: no test here may start a GPU job.
|
||||
import assert from "node:assert/strict";
|
||||
import { spawn, spawnSync } from "node:child_process";
|
||||
import { existsSync, readFileSync } from "node:fs";
|
||||
import { copyFile, mkdtemp, readdir, rm, writeFile } from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import test from "node:test";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
const daemonPath = path.join(path.dirname(fileURLToPath(import.meta.url)), "vili-daemon.mjs");
|
||||
let nextPort = 18000 + Math.floor(Math.random() * 1000);
|
||||
|
||||
async function bootDaemon(stateRoot, { host = "127.0.0.1", env = { VILI_GPU_MIN_FREE_MIB: "999999" } } = {}) {
|
||||
const port = nextPort++;
|
||||
const child = spawn(process.execPath, [daemonPath, "--host", host, "--port", String(port), "--state-root", stateRoot], {
|
||||
env: { ...process.env, ...env },
|
||||
stdio: ["ignore", "pipe", "pipe"],
|
||||
windowsHide: true,
|
||||
});
|
||||
let output = "";
|
||||
child.stdout.on("data", (chunk) => { output += chunk; });
|
||||
child.stderr.on("data", (chunk) => { output += chunk; });
|
||||
const deadline = Date.now() + 30000;
|
||||
while (!output.includes("Vili listening")) {
|
||||
if (child.exitCode !== null || Date.now() > deadline) throw new Error(`daemon did not start: ${output.slice(-500)}`);
|
||||
await new Promise((resolve) => setTimeout(resolve, 100));
|
||||
}
|
||||
const stop = () => new Promise((resolve) => {
|
||||
if (child.exitCode !== null) resolve();
|
||||
else { child.once("exit", resolve); child.kill(); }
|
||||
});
|
||||
return { child, port, stop };
|
||||
}
|
||||
|
||||
const url = (d, route, host = "127.0.0.1") => `http://${host}:${d.port}${route}`;
|
||||
const generate = (d, body) => fetch(url(d, "/motion/generate"), {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
const setPermission = (d, body, host = "127.0.0.1") => fetch(url(d, "/gpu-permission", host), {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
const readPermission = async (d) => (await fetch(url(d, "/gpu-permission"))).json();
|
||||
|
||||
test("http: refusal at the daemon edge", async () => {
|
||||
const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-"));
|
||||
const d = await bootDaemon(stateRoot);
|
||||
try {
|
||||
assert.deepEqual(await readPermission(d), { allowed: false, changedAt: null, jobRunning: false, lastRefusal: null });
|
||||
assert.equal((await generate(d, {})).status, 400, "request-shape errors come ahead of the gate");
|
||||
|
||||
const refused = await generate(d, { prompt: "a person waves hello, never seen before" });
|
||||
assert.equal(refused.status, 503);
|
||||
assert.equal(refused.headers.get("retry-after"), "60");
|
||||
assert.deepEqual(await refused.json(), { ok: false, code: "gpu-busy", source: "household", gpu: { allowed: false, changedAt: null } });
|
||||
|
||||
assert.deepEqual(await readdir(path.join(stateRoot, "jobs")), [], "no job file");
|
||||
assert.equal((await readdir(stateRoot)).includes("artifacts"), false, "no artifact directory");
|
||||
|
||||
const state = await (await fetch(url(d, "/operator-state"))).json();
|
||||
assert.equal(state.gpu.lastRefusal.source, "household");
|
||||
assert.equal(state.gpu.allowed, false);
|
||||
} finally {
|
||||
await d.stop();
|
||||
await rm(stateRoot, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("http: the permission persists across a daemon restart", async () => {
|
||||
const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-persist-"));
|
||||
let d = await bootDaemon(stateRoot);
|
||||
try {
|
||||
const on = await setPermission(d, { allowed: true });
|
||||
assert.equal(on.status, 200);
|
||||
const onBody = await on.json();
|
||||
assert.equal(onBody.ok, true);
|
||||
assert.equal(onBody.gpu.allowed, true);
|
||||
const before = await readPermission(d);
|
||||
assert.equal(before.allowed, true);
|
||||
assert.ok(before.changedAt);
|
||||
|
||||
assert.equal((await setPermission(d, { allowed: "yes" })).status, 400);
|
||||
assert.equal((await setPermission(d, {})).status, 400);
|
||||
assert.deepEqual(await readPermission(d), before, "a refused write changes nothing");
|
||||
|
||||
await d.stop();
|
||||
d = await bootDaemon(stateRoot);
|
||||
const after = await readPermission(d);
|
||||
assert.equal(after.allowed, true);
|
||||
assert.equal(after.changedAt, before.changedAt);
|
||||
|
||||
await setPermission(d, { allowed: false });
|
||||
await d.stop();
|
||||
d = await bootDaemon(stateRoot);
|
||||
assert.equal((await readPermission(d)).allowed, false, "off persists too");
|
||||
} finally {
|
||||
await d.stop();
|
||||
await rm(stateRoot, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("http: an allowed request reaches the slot and the VRAM read", async () => {
|
||||
const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-slot-"));
|
||||
const d = await bootDaemon(stateRoot);
|
||||
try {
|
||||
await setPermission(d, { allowed: true });
|
||||
const refused = await generate(d, { prompt: "a person waves hello, never seen before" });
|
||||
assert.equal(refused.status, 503);
|
||||
const body = await refused.json();
|
||||
assert.equal(body.source, "vram", "admitted through acquire(), then refused for VRAM");
|
||||
assert.equal(body.minFreeMiB, 999999);
|
||||
assert.deepEqual(await readdir(path.join(stateRoot, "jobs")), [], "no job file");
|
||||
assert.equal((await readdir(stateRoot)).includes("artifacts"), false);
|
||||
const state = await readPermission(d);
|
||||
assert.equal(state.lastRefusal.source, "vram");
|
||||
assert.equal(state.jobRunning, false);
|
||||
} finally {
|
||||
await d.stop();
|
||||
await rm(stateRoot, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("http: the switch is the household's", async () => {
|
||||
const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-lan-"));
|
||||
const addresses = Object.values(os.networkInterfaces()).flat().filter((i) => i.family === "IPv4" && !i.internal).map((i) => i.address);
|
||||
const lan = addresses[0];
|
||||
assert.ok(lan, "the host has a non-loopback IPv4 address");
|
||||
const d = await bootDaemon(stateRoot, { host: `127.0.0.1,${lan}` });
|
||||
try {
|
||||
const before = await readPermission(d);
|
||||
const refused = await setPermission(d, { allowed: true }, lan);
|
||||
assert.equal(refused.status, 403);
|
||||
assert.deepEqual(await refused.json(), { ok: false, code: "household-only" });
|
||||
assert.deepEqual(await readPermission(d), before, "GET still reads the previous value");
|
||||
assert.equal((await setPermission(d, { allowed: true })).status, 200, "loopback still sets it");
|
||||
const other = addresses.find((address) => address !== lan);
|
||||
if (other) {
|
||||
await assert.rejects(fetch(url(d, "/gpu-permission", other)), "an address --host does not name is not served");
|
||||
}
|
||||
} finally {
|
||||
await d.stop();
|
||||
await rm(stateRoot, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("http: a web page cannot flip the household's switch or start a job", async () => {
|
||||
const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-csrf-"));
|
||||
const d = await bootDaemon(stateRoot);
|
||||
const post = (route, headers, body) => fetch(url(d, route), { method: "POST", headers, body });
|
||||
const prompt = JSON.stringify({ prompt: "a person waves hello, never seen before" });
|
||||
try {
|
||||
const before = await readPermission(d);
|
||||
for (const route of ["/gpu-permission", "/motion/generate"]) {
|
||||
const payload = route === "/gpu-permission" ? JSON.stringify({ allowed: true }) : prompt;
|
||||
const simple = await post(route, { "content-type": "text/plain" }, payload);
|
||||
assert.equal(simple.status, 403, `${route}: a simple text/plain POST`);
|
||||
assert.deepEqual(await simple.json(), { ok: false, code: "cross-site" });
|
||||
assert.equal((await post(route, {}, payload)).status, 403, `${route}: no content-type`);
|
||||
const hostile = await post(route, { "content-type": "application/json", origin: "http://evil.example-origin" }, payload);
|
||||
assert.equal(hostile.status, 403, `${route}: a hostile Origin`);
|
||||
const body = JSON.stringify(await hostile.json());
|
||||
assert.equal(body.includes("evil.example-origin"), false, "the refusal never echoes the Origin");
|
||||
assert.equal(
|
||||
(await post(route, { "content-type": "application/json", origin: `http://127.0.0.1:${d.port}.evil.example` }, payload)).status,
|
||||
403,
|
||||
`${route}: an Origin that only starts like ours`,
|
||||
);
|
||||
}
|
||||
assert.deepEqual(await readPermission(d), before, "no refused request changed anything");
|
||||
assert.deepEqual(await readdir(path.join(stateRoot, "jobs")), [], "no job file");
|
||||
|
||||
const preflight = await fetch(url(d, "/gpu-permission"), {
|
||||
method: "OPTIONS",
|
||||
headers: { origin: "http://evil.example-origin", "access-control-request-method": "POST", "access-control-request-headers": "content-type" },
|
||||
});
|
||||
assert.equal(preflight.ok, false, "the daemon approves no CORS preflight");
|
||||
assert.equal(preflight.headers.get("access-control-allow-methods"), null);
|
||||
|
||||
const own = await post("/gpu-permission", { "content-type": "application/json; charset=utf-8", origin: `http://127.0.0.1:${d.port}` }, JSON.stringify({ allowed: true }));
|
||||
assert.equal(own.status, 200, "the daemon's own origin still works");
|
||||
assert.equal((await setPermission(d, { allowed: false })).status, 200, "a caller with no Origin (the tray) still works");
|
||||
} finally {
|
||||
await d.stop();
|
||||
await rm(stateRoot, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
// The daemon reaches the GPU only through the processes wsl.exe and nvidia-smi.exe. This test puts
|
||||
// one small compiled stub on PATH under both names (Windows, where the daemon lives), so the real
|
||||
// daemon runs a real job against a worker that takes as long as the test says and touches no GPU:
|
||||
// nvidia-smi.exe answers 20000 free MiB, the worker health call answers ok, and the "/generate"
|
||||
// call waits until the test creates a release file.
|
||||
const stubSource = `
|
||||
using System; using System.IO; using System.Reflection; using System.Threading;
|
||||
class Stub {
|
||||
static int Main(string[] args) {
|
||||
string exe = Path.GetFileNameWithoutExtension(Assembly.GetEntryAssembly().Location).ToLowerInvariant();
|
||||
string dir = Environment.GetEnvironmentVariable("VILI_STUB_DIR");
|
||||
if (exe == "nvidia-smi") { Console.WriteLine("20000"); return 0; }
|
||||
string script = args.Length > 0 ? args[args.Length - 1] : "";
|
||||
if (script.Contains("docker run")) File.AppendAllText(Path.Combine(dir, "dockerrun"), "x");
|
||||
if (script.Contains("service docker start")) File.AppendAllText(Path.Combine(dir, "dockerstart"), "x");
|
||||
if (script.Contains("/health")) { Console.WriteLine("{\\"ok\\":true,\\"model\\":\\"stub\\"}"); return 0; }
|
||||
if (script.Contains("/generate")) {
|
||||
File.AppendAllText(Path.Combine(dir, "started"), "x");
|
||||
while (!File.Exists(Path.Combine(dir, "release"))) Thread.Sleep(25);
|
||||
Console.WriteLine("{\\"ok\\":true}");
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
}`;
|
||||
|
||||
async function stubbedDaemon(stubDir, stateRoot) {
|
||||
const source = path.join(stubDir, "stub.cs");
|
||||
await writeFile(source, stubSource);
|
||||
const build = path.join(stubDir, "build.ps1");
|
||||
await writeFile(build, [
|
||||
"param($Source, $Out)",
|
||||
"Add-Type -TypeDefinition (Get-Content -Raw $Source) -OutputAssembly $Out -OutputType ConsoleApplication",
|
||||
].join("\r\n"));
|
||||
const built = spawnSync("powershell.exe", [
|
||||
"-NoProfile", "-NonInteractive", "-ExecutionPolicy", "Bypass", "-File", build, source, path.join(stubDir, "wsl.exe"),
|
||||
], { encoding: "utf8", windowsHide: true });
|
||||
assert.equal(built.status, 0, `the stub did not compile: ${built.stderr || built.stdout}`);
|
||||
await copyFile(path.join(stubDir, "wsl.exe"), path.join(stubDir, "nvidia-smi.exe"));
|
||||
const pathKey = Object.keys(process.env).find((key) => key.toLowerCase() === "path") ?? "PATH";
|
||||
return bootDaemon(stateRoot, {
|
||||
env: {
|
||||
[pathKey]: `${stubDir}${path.delimiter}${process.env[pathKey]}`,
|
||||
VILI_STUB_DIR: stubDir,
|
||||
VILI_GPU_MIN_FREE_MIB: "1",
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
test("http: the daemon holds the slot for the whole job", async () => {
|
||||
const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-hold-"));
|
||||
const stubDir = await mkdtemp(path.join(os.tmpdir(), "vili-gate-stub-"));
|
||||
const d = await stubbedDaemon(stubDir, stateRoot);
|
||||
const prompt = { prompt: "a person waves hello, never seen before" };
|
||||
const within = (promise, ms, what) => Promise.race([
|
||||
promise,
|
||||
new Promise((_, reject) => setTimeout(() => reject(new Error(`${what} did not answer within ${ms} ms`)), ms)),
|
||||
]);
|
||||
try {
|
||||
await setPermission(d, { allowed: true });
|
||||
const first = generate(d, prompt);
|
||||
const deadline = Date.now() + 20000;
|
||||
while (!(await readPermission(d)).jobRunning) {
|
||||
assert.ok(Date.now() < deadline, "the first job never took the slot");
|
||||
await new Promise((resolve) => setTimeout(resolve, 50));
|
||||
}
|
||||
while (!existsSync(path.join(stubDir, "started"))) {
|
||||
assert.ok(Date.now() < deadline, "the first job never reached the worker");
|
||||
await new Promise((resolve) => setTimeout(resolve, 50));
|
||||
}
|
||||
|
||||
for (let i = 0; i < 3; i += 1) {
|
||||
const second = await within(generate(d, prompt), 5000, "a request during the job");
|
||||
assert.equal(second.status, 503, "a request during the job is refused at once");
|
||||
assert.deepEqual(await second.json(), { ok: false, code: "gpu-busy", source: "slot" });
|
||||
}
|
||||
assert.equal(readFileSync(path.join(stubDir, "started"), "utf8"), "x", "exactly one job reached the worker");
|
||||
assert.equal((await readPermission(d)).jobRunning, true, "the slot is still held");
|
||||
|
||||
await writeFile(path.join(stubDir, "release"), "");
|
||||
const done = await within(first, 20000, "the first job");
|
||||
assert.equal(done.status, 200);
|
||||
assert.equal((await readPermission(d)).jobRunning, false, "the slot frees when the job ends");
|
||||
const next = await within(generate(d, prompt), 20000, "the next job");
|
||||
assert.equal(next.status, 200, "the next job is admitted after the slot frees");
|
||||
assert.equal(readFileSync(path.join(stubDir, "started"), "utf8"), "xx");
|
||||
} finally {
|
||||
await d.stop();
|
||||
await rm(stateRoot, { recursive: true, force: true });
|
||||
await rm(stubDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("http: a malformed override refuses to start and names the variable, not its value", async () => {
|
||||
const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-env-"));
|
||||
const child = spawn(process.execPath, [daemonPath, "--host", "127.0.0.1", "--port", String(nextPort++), "--state-root", stateRoot], {
|
||||
env: { ...process.env, VILI_GPU_MIN_FREE_MIB: "10240MiB-secret" },
|
||||
stdio: ["ignore", "pipe", "pipe"],
|
||||
windowsHide: true,
|
||||
});
|
||||
let output = "";
|
||||
child.stdout.on("data", (chunk) => { output += chunk; });
|
||||
child.stderr.on("data", (chunk) => { output += chunk; });
|
||||
try {
|
||||
const code = await new Promise((resolve) => child.on("exit", resolve));
|
||||
assert.notEqual(code, 0);
|
||||
assert.match(output, /VILI_GPU_MIN_FREE_MIB must be a finite non-negative number/);
|
||||
assert.equal(output.includes("10240MiB-secret"), false);
|
||||
assert.equal(output.includes("Vili listening"), false);
|
||||
} finally {
|
||||
child.kill();
|
||||
await rm(stateRoot, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("http: the daemon refuses to listen on a wildcard address", async () => {
|
||||
const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-wild-"));
|
||||
try {
|
||||
for (const host of ["0.0.0.0", "127.0.0.1,0.0.0.0", "::"]) {
|
||||
const run = spawnSync(process.execPath, [daemonPath, "--host", host, "--port", String(nextPort++), "--state-root", stateRoot], {
|
||||
encoding: "utf8",
|
||||
windowsHide: true,
|
||||
timeout: 20000,
|
||||
});
|
||||
assert.equal(run.status, 2, `--host ${host} exits 2`);
|
||||
assert.equal(`${run.stdout}${run.stderr}`.includes("Vili listening"), false);
|
||||
}
|
||||
} finally {
|
||||
await rm(stateRoot, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("http: no response grants a web page access", async () => {
|
||||
const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-cors-"));
|
||||
const d = await bootDaemon(stateRoot);
|
||||
try {
|
||||
const hostile = { origin: "http://evil.example-origin" };
|
||||
const responses = [
|
||||
await fetch(url(d, "/gpu-permission"), { headers: hostile }),
|
||||
await fetch(url(d, "/operator-state"), { headers: hostile }),
|
||||
await fetch(url(d, "/nowhere"), { headers: hostile }),
|
||||
await fetch(url(d, "/gpu-permission"), { method: "POST", headers: { ...hostile, "content-type": "application/json" }, body: "{}" }),
|
||||
];
|
||||
assert.deepEqual(responses.map((r) => r.status), [200, 200, 404, 403]);
|
||||
for (const response of responses) {
|
||||
const cors = [...response.headers.keys()].filter((name) => name.startsWith("access-control-"));
|
||||
assert.deepEqual(cors, [], "no access-control header on any response");
|
||||
}
|
||||
} finally {
|
||||
await d.stop();
|
||||
await rm(stateRoot, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("http: with the permission off nothing a web page or a LAN caller sends reaches the GPU", async () => {
|
||||
const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-off-"));
|
||||
const stubDir = await mkdtemp(path.join(os.tmpdir(), "vili-gate-stub-"));
|
||||
const d = await stubbedDaemon(stubDir, stateRoot);
|
||||
const page = { origin: "http://evil.example-origin", "sec-fetch-site": "cross-site", "sec-fetch-mode": "no-cors", "sec-fetch-dest": "image" };
|
||||
try {
|
||||
const before = await readPermission(d);
|
||||
assert.equal(before.allowed, false);
|
||||
for (const route of ["/smoke", "/health", "/", "/operator-state", "/eve/operator", "/provider-advertisement", "/eve/deck/providers", "/gpu-permission"]) {
|
||||
const response = await fetch(url(d, route), { headers: page });
|
||||
assert.equal(response.status, route === "/smoke" ? 404 : 200, `GET ${route}`);
|
||||
}
|
||||
const refused = await generate(d, { prompt: "a person waves hello, never seen before" });
|
||||
assert.equal(refused.status, 503);
|
||||
assert.equal((await refused.json()).source, "household");
|
||||
assert.equal((await setPermission(d, { allowed: true }, "127.0.0.1")).status, 200);
|
||||
assert.equal((await setPermission(d, { allowed: false }, "127.0.0.1")).status, 200);
|
||||
assert.deepEqual((await readPermission(d)).allowed, false);
|
||||
assert.equal(existsSync(path.join(stubDir, "dockerrun")), false, "nothing ran docker run (the worker or any GPU container)");
|
||||
assert.equal(existsSync(path.join(stubDir, "dockerstart")), false, "no GET started Docker");
|
||||
assert.equal(existsSync(path.join(stubDir, "started")), false, "nothing reached the worker");
|
||||
assert.deepEqual(await readdir(path.join(stateRoot, "jobs")), [], "no job file");
|
||||
} finally {
|
||||
await d.stop();
|
||||
await rm(stateRoot, { recursive: true, force: true });
|
||||
await rm(stubDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("http: a client that hangs up mid-job does not free the slot", async () => {
|
||||
const stateRoot = await mkdtemp(path.join(os.tmpdir(), "vili-gate-hangup-"));
|
||||
const stubDir = await mkdtemp(path.join(os.tmpdir(), "vili-gate-stub-"));
|
||||
const d = await stubbedDaemon(stubDir, stateRoot);
|
||||
const prompt = { prompt: "a person waves hello, never seen before" };
|
||||
const hangUp = new AbortController();
|
||||
try {
|
||||
await setPermission(d, { allowed: true });
|
||||
const first = fetch(url(d, "/motion/generate"), {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify(prompt),
|
||||
signal: hangUp.signal,
|
||||
}).catch((error) => error);
|
||||
const deadline = Date.now() + 20000;
|
||||
while (!existsSync(path.join(stubDir, "started"))) {
|
||||
assert.ok(Date.now() < deadline, "the job never reached the worker");
|
||||
await new Promise((resolve) => setTimeout(resolve, 50));
|
||||
}
|
||||
hangUp.abort();
|
||||
assert.equal((await first).name, "AbortError", "the client hung up");
|
||||
await new Promise((resolve) => setTimeout(resolve, 500));
|
||||
|
||||
assert.equal((await readPermission(d)).jobRunning, true, "the slot is still held after the hang-up");
|
||||
const second = await generate(d, prompt);
|
||||
assert.equal(second.status, 503);
|
||||
assert.equal((await second.json()).source, "slot");
|
||||
assert.equal(readFileSync(path.join(stubDir, "started"), "utf8"), "x", "no second job reached the worker");
|
||||
|
||||
await writeFile(path.join(stubDir, "release"), "");
|
||||
const freed = Date.now() + 20000;
|
||||
while ((await readPermission(d)).jobRunning) {
|
||||
assert.ok(Date.now() < freed, "the slot never freed when the job ended");
|
||||
await new Promise((resolve) => setTimeout(resolve, 50));
|
||||
}
|
||||
} finally {
|
||||
await d.stop();
|
||||
await rm(stateRoot, { recursive: true, force: true });
|
||||
await rm(stubDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
211
scripts/test-gpu-tenancy.mjs
Normal file
211
scripts/test-gpu-tenancy.mjs
Normal file
|
|
@ -0,0 +1,211 @@
|
|||
import assert from "node:assert/strict";
|
||||
import test from "node:test";
|
||||
import { admitThen, createGpuTenancy, gpuTenancyOptionsFromEnv, parseFreeMiB, placementRefusal } from "./gpu-tenancy.mjs";
|
||||
|
||||
function deferred() {
|
||||
let resolve;
|
||||
const promise = new Promise((r) => { resolve = r; });
|
||||
return { promise, resolve };
|
||||
}
|
||||
|
||||
// Fake ports: a clock, a store spy and a free-VRAM source. No GPU, no daemon, no cache.
|
||||
function rig({ permission, free = 12000, minFreeMiB } = {}) {
|
||||
const r = { t: Date.parse("2026-10-23T10:00:00Z"), saved: [], reads: 0, free, failSave: false };
|
||||
r.gate = createGpuTenancy({
|
||||
now: () => r.t,
|
||||
permission,
|
||||
minFreeMiB,
|
||||
savePermission: async (value) => {
|
||||
if (r.failSave) throw new Error("disk full");
|
||||
r.saved.push(value);
|
||||
},
|
||||
readFreeMiB: async () => {
|
||||
r.reads += 1;
|
||||
if (typeof r.free === "function") return r.free();
|
||||
return r.free;
|
||||
},
|
||||
});
|
||||
return r;
|
||||
}
|
||||
const ON = { allowed: true, changedAt: "2026-10-23T09:00:00.000Z" };
|
||||
|
||||
test("gate: toggle off refuses and creates no job", async () => {
|
||||
const r = rig();
|
||||
let calls = 0;
|
||||
const refused = await admitThen(r.gate, async () => { calls += 1; });
|
||||
assert.deepEqual(refused, { ok: false, code: "gpu-busy", source: "household", gpu: { allowed: false, changedAt: null } });
|
||||
assert.equal(calls, 0);
|
||||
assert.equal(r.reads, 0, "the VRAM is never read for a household refusal");
|
||||
assert.deepEqual(r.gate.status(), {
|
||||
allowed: false,
|
||||
changedAt: null,
|
||||
jobRunning: false,
|
||||
lastRefusal: { at: "2026-10-23T10:00:00.000Z", source: "household" },
|
||||
});
|
||||
});
|
||||
|
||||
test("gate: toggle on admits", async () => {
|
||||
const r = rig({ permission: ON });
|
||||
let calls = 0;
|
||||
const result = await admitThen(r.gate, async () => {
|
||||
calls += 1;
|
||||
assert.equal(r.gate.status().jobRunning, true);
|
||||
return { ok: true, value: "finished" };
|
||||
});
|
||||
assert.deepEqual(result, { ok: true, value: "finished" });
|
||||
assert.equal(calls, 1);
|
||||
assert.equal(r.gate.status().jobRunning, false);
|
||||
assert.equal(r.gate.status().lastRefusal, null);
|
||||
});
|
||||
|
||||
test("gate: toggle off mid-job lets the job finish and refuses the next", async () => {
|
||||
const r = rig({ permission: ON });
|
||||
const end = deferred();
|
||||
const running = admitThen(r.gate, async () => { await end.promise; return { ok: true, value: "finished" }; });
|
||||
while (!r.gate.status().jobRunning) await new Promise((resolve) => setImmediate(resolve));
|
||||
const off = await r.gate.setAllowed(false);
|
||||
assert.equal(off.allowed, false);
|
||||
assert.equal(off.jobRunning, true, "the job is still running");
|
||||
let calls = 0;
|
||||
const next = await admitThen(r.gate, async () => { calls += 1; });
|
||||
assert.equal(next.source, "household");
|
||||
assert.equal(calls, 0);
|
||||
end.resolve();
|
||||
assert.deepEqual(await running, { ok: true, value: "finished" });
|
||||
assert.equal(r.gate.status().jobRunning, false);
|
||||
});
|
||||
|
||||
test("gate: the concurrent slot behaves as h2's did", async () => {
|
||||
const vram = deferred();
|
||||
const r = rig({ permission: ON, free: () => vram.promise });
|
||||
const end = deferred();
|
||||
let calls = 0;
|
||||
const job = async () => { calls += 1; await end.promise; return "done"; };
|
||||
const all = [admitThen(r.gate, job), admitThen(r.gate, job), admitThen(r.gate, job)];
|
||||
// the two losers are decided before the VRAM read resolves
|
||||
const losers = await Promise.all([all[1], all[2]]);
|
||||
assert.deepEqual(losers, [{ ok: false, code: "gpu-busy", source: "slot" }, { ok: false, code: "gpu-busy", source: "slot" }]);
|
||||
assert.equal(r.reads, 1, "only the slot holder reads VRAM");
|
||||
vram.resolve(12000);
|
||||
end.resolve();
|
||||
assert.equal(await all[0], "done");
|
||||
assert.equal(calls, 1);
|
||||
assert.equal(r.gate.status().jobRunning, false);
|
||||
|
||||
await assert.rejects(admitThen(r.gate, async () => { throw new Error("boom"); }), /boom/);
|
||||
assert.equal(r.gate.status().jobRunning, false, "a throwing job frees the slot");
|
||||
r.free = 100;
|
||||
assert.equal((await admitThen(r.gate, async () => "x")).source, "vram");
|
||||
const readsBefore = r.reads;
|
||||
r.free = 12000;
|
||||
assert.equal(await admitThen(r.gate, async () => "again"), "again", "a vram refusal frees the slot");
|
||||
assert.equal(r.reads, readsBefore + 1, "the next admission reads VRAM again");
|
||||
});
|
||||
|
||||
test("gate: VRAM read fails closed", async () => {
|
||||
const cases = [[9000, 9000], [10239, 10239], [null, null], [Number.NaN, null], [() => { throw new Error("smi"); }, null]];
|
||||
for (const [free, expected] of cases) {
|
||||
const r = rig({ permission: ON, free });
|
||||
let calls = 0;
|
||||
const refused = await admitThen(r.gate, async () => { calls += 1; });
|
||||
assert.deepEqual(refused, { ok: false, code: "gpu-busy", source: "vram", freeMiB: expected, minFreeMiB: 10240 });
|
||||
assert.equal(calls, 0);
|
||||
assert.equal(r.gate.status().jobRunning, false);
|
||||
assert.equal(r.gate.status().lastRefusal.source, "vram");
|
||||
}
|
||||
const edge = rig({ permission: ON, free: 10240 });
|
||||
assert.equal(await admitThen(edge.gate, async () => "in"), "in", "exactly the minimum admits");
|
||||
const custom = rig({ permission: ON, free: 5000, minFreeMiB: 4096 });
|
||||
assert.equal(await admitThen(custom.gate, async () => "in"), "in", "the minimum is the configured one");
|
||||
});
|
||||
|
||||
test("gate: a toggle during the VRAM read wins", async () => {
|
||||
const vram = deferred();
|
||||
const r = rig({ permission: ON, free: () => vram.promise });
|
||||
let calls = 0;
|
||||
const pending = admitThen(r.gate, async () => { calls += 1; });
|
||||
await r.gate.setAllowed(false);
|
||||
vram.resolve(12000);
|
||||
const refused = await pending;
|
||||
assert.equal(refused.source, "household");
|
||||
assert.equal(calls, 0);
|
||||
assert.equal(r.gate.status().jobRunning, false);
|
||||
});
|
||||
|
||||
test("gate: setAllowed saves before it answers", async () => {
|
||||
const r = rig();
|
||||
r.failSave = true;
|
||||
await assert.rejects(r.gate.setAllowed(true), /disk full/);
|
||||
assert.equal(r.gate.status().allowed, false, "a failed save leaves the state unchanged");
|
||||
assert.equal(r.gate.status().changedAt, null);
|
||||
r.failSave = false;
|
||||
|
||||
await assert.rejects(r.gate.setAllowed("yes"), TypeError);
|
||||
await assert.rejects(r.gate.setAllowed(undefined), TypeError);
|
||||
assert.deepEqual(r.saved, [], "a non-boolean saves nothing");
|
||||
|
||||
r.t += 1000;
|
||||
const on = await r.gate.setAllowed(true);
|
||||
assert.deepEqual(r.saved, [{ allowed: true, changedAt: "2026-10-23T10:00:01.000Z" }]);
|
||||
assert.equal(on.allowed, true);
|
||||
assert.equal(on.changedAt, "2026-10-23T10:00:01.000Z");
|
||||
|
||||
r.t += 5000;
|
||||
const same = await r.gate.setAllowed(true);
|
||||
assert.equal(r.saved.length, 1, "a same-value call saves nothing");
|
||||
assert.equal(same.changedAt, "2026-10-23T10:00:01.000Z", "and keeps changedAt");
|
||||
});
|
||||
|
||||
test("gate: a second release is a no-op and cannot free another job's slot", async () => {
|
||||
const r = rig({ permission: ON });
|
||||
const first = await r.gate.acquire();
|
||||
first.release();
|
||||
const second = await r.gate.acquire();
|
||||
assert.equal(r.gate.status().jobRunning, true);
|
||||
first.release();
|
||||
assert.equal(r.gate.status().jobRunning, true, "a stale release leaves the new holder's slot held");
|
||||
assert.equal((await r.gate.acquire()).refusal.source, "slot");
|
||||
second.release();
|
||||
second.release();
|
||||
assert.equal(r.gate.status().jobRunning, false);
|
||||
assert.equal(typeof (await r.gate.acquire()).release, "function", "the slot is free after the real release");
|
||||
});
|
||||
|
||||
test("gate: a stored permission is the starting state", () => {
|
||||
assert.deepEqual(rig({ permission: ON }).gate.status(), { allowed: true, changedAt: ON.changedAt, jobRunning: false, lastRefusal: null });
|
||||
assert.equal(rig({ permission: { allowed: "yes", changedAt: "x" } }).gate.status().allowed, false, "only a true boolean allows");
|
||||
});
|
||||
|
||||
test("parseFreeMiB reads the first line as a base-10 integer", () => {
|
||||
assert.equal(parseFreeMiB("5001\r\n"), 5001);
|
||||
assert.equal(parseFreeMiB("0"), 0);
|
||||
assert.equal(parseFreeMiB(" 12 \n7\n"), 12);
|
||||
for (const bad of ["", "[N/A]", "abc", "-5", "1e3", "12.5", "12 MiB", "\n5", null, undefined]) {
|
||||
assert.equal(parseFreeMiB(bad), null, JSON.stringify(bad));
|
||||
}
|
||||
});
|
||||
|
||||
test("placement refusal is keyed on the typed code, never on text, and carries no gate status", () => {
|
||||
assert.deepEqual(placementRefusal({ ok: false, code: "gpu-busy", error: "x" }, "vili-1"), {
|
||||
ok: false, code: "gpu-busy", source: "placement", jobId: "vili-1",
|
||||
});
|
||||
assert.equal(placementRefusal({ ok: false, error: "gpu-busy: 0.50 GiB of VRAM free" }, "vili-1"), null);
|
||||
assert.equal(placementRefusal({ ok: false, code: null }, "vili-1"), null);
|
||||
assert.equal(placementRefusal(null, "vili-1"), null);
|
||||
});
|
||||
|
||||
test("env override: a valid value is read, absent is left to the default", () => {
|
||||
assert.deepEqual(gpuTenancyOptionsFromEnv({}), {});
|
||||
assert.deepEqual(gpuTenancyOptionsFromEnv({ VILI_GPU_MIN_FREE_MIB: "8192.5" }), { minFreeMiB: 8192.5 });
|
||||
assert.deepEqual(gpuTenancyOptionsFromEnv({ VILI_GPU_MIN_FREE_MIB: "0" }), { minFreeMiB: 0 });
|
||||
});
|
||||
|
||||
test("env override: a malformed value refuses to start and names the variable, not the value", () => {
|
||||
for (const bad of ["10240MiB", "", " ", "NaN", "Infinity", "-5"]) {
|
||||
assert.throws(
|
||||
() => gpuTenancyOptionsFromEnv({ VILI_GPU_MIN_FREE_MIB: bad }),
|
||||
(error) => error.message === "VILI_GPU_MIN_FREE_MIB must be a finite non-negative number",
|
||||
JSON.stringify(bad),
|
||||
);
|
||||
}
|
||||
});
|
||||
|
|
@ -180,6 +180,12 @@ class ChildBoundary(unittest.TestCase):
|
|||
""")
|
||||
self.assertEqual(raised.exception.free_vram, 1234)
|
||||
|
||||
def test_worker_error_body_carries_the_code(self):
|
||||
busy = te.worker_error_body(te.GpuBusy(GIB))
|
||||
self.assertIs(busy["ok"], False)
|
||||
self.assertEqual(busy["code"], "gpu-busy")
|
||||
self.assertIsNone(te.worker_error_body(RuntimeError("x"))["code"])
|
||||
|
||||
def test_other_failures_name_the_exit_code_and_never_the_text(self):
|
||||
with self.assertRaises(RuntimeError) as raised:
|
||||
self.run_stub("import sys; sys.exit(1)", texts=(SECRET,))
|
||||
|
|
|
|||
|
|
@ -12,6 +12,7 @@ import subprocess
|
|||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
import traceback
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
|
@ -33,7 +34,11 @@ LLAMA_BASE = "meta-llama/Meta-Llama-3-8B-Instruct" # the base model the mntp ad
|
|||
|
||||
|
||||
class GpuBusy(RuntimeError):
|
||||
"""The card has too little free VRAM to host the encoder; there is no CPU fallback."""
|
||||
"""Placement refusal: too little free VRAM to host the encoder, and no CPU fallback.
|
||||
|
||||
Whether heavy work may start at all is the daemon's tenancy gate, which sees the host; this is only
|
||||
the child sizing its own placement from what torch can see.
|
||||
"""
|
||||
|
||||
code = "gpu-busy"
|
||||
|
||||
|
|
@ -49,6 +54,11 @@ def plan_placement(free_vram: int, mem_available: int) -> dict:
|
|||
return {0: free_vram - GPU_RESERVE, "cpu": max(mem_available - CPU_RESERVE, 0)}
|
||||
|
||||
|
||||
def worker_error_body(error: BaseException) -> dict:
|
||||
"""The worker's one error wire shape: a typed code (GpuBusy.code, else None) beside the text."""
|
||||
return {"ok": False, "error": str(error), "code": getattr(error, "code", None), "traceback": traceback.format_exc()}
|
||||
|
||||
|
||||
def read_snapshot_revision(hf_home: Path, repo_id: str) -> str:
|
||||
"""The commit the mounted Hugging Face cache resolves a repo's main ref to."""
|
||||
ref = Path(hf_home) / "hub" / ("models--" + repo_id.replace("/", "--")) / "refs" / "main"
|
||||
|
|
|
|||
|
|
@ -6,6 +6,7 @@ import { fileURLToPath } from "node:url";
|
|||
import { createRequire } from "node:module";
|
||||
import { execFile } from "node:child_process";
|
||||
import { WebSocketServer } from "ws";
|
||||
import { RETRY_AFTER_SECONDS, admitThen, createGpuTenancy, gpuTenancyOptionsFromEnv, parseFreeMiB, placementRefusal } from "./gpu-tenancy.mjs";
|
||||
|
||||
const __dirname = path.dirname(fileURLToPath(import.meta.url));
|
||||
const require = createRequire(import.meta.url);
|
||||
|
|
@ -59,6 +60,20 @@ const operatorStateDefinition = defineDocumentType({
|
|||
{ slot: 1, memberName: "providerId", typeName: "string" },
|
||||
{ slot: 2, memberName: "updatedAt", typeName: "string" },
|
||||
{ slot: 3, memberName: "backend", typeName: "object" },
|
||||
{ slot: 4, memberName: "gpu", typeName: "object" },
|
||||
],
|
||||
});
|
||||
const gpuPermissionDefinition = defineDocumentType({
|
||||
type: "gamecult.vili.gpu_permission",
|
||||
schemaName: "gamecult.vili.gpu_permission",
|
||||
schemaId: "gamecult.vili.gpu_permission.v1",
|
||||
schemaVersion: "gamecult.vili.gpu_permission.v1",
|
||||
name: (value) => value?.host || "raven",
|
||||
schema: objectSchema,
|
||||
members: [
|
||||
{ slot: 0, memberName: "host", typeName: "string", isName: true },
|
||||
{ slot: 1, memberName: "allowed", typeName: "boolean" },
|
||||
{ slot: 2, memberName: "changedAt", typeName: "string" },
|
||||
],
|
||||
});
|
||||
const eveSurfaceDefinition = defineDocumentType({
|
||||
|
|
@ -108,6 +123,7 @@ const transportProfileDefinition = defineDocumentType({
|
|||
const viliDocumentRegistry = defineDocumentRegistry(
|
||||
providerAdvertisementDefinition,
|
||||
operatorStateDefinition,
|
||||
gpuPermissionDefinition,
|
||||
eveSurfaceDefinition,
|
||||
commandBoundaryDefinition,
|
||||
transportProfileDefinition,
|
||||
|
|
@ -115,6 +131,8 @@ const viliDocumentRegistry = defineDocumentRegistry(
|
|||
|
||||
function parseArgs(argv) {
|
||||
const args = {
|
||||
// Comma-separated addresses to listen on, one server each. Never 0.0.0.0: the daemon answers
|
||||
// this machine and the WireGuard address, not whatever network the machine joins next.
|
||||
host: process.env.VILI_DAEMON_HOST || "127.0.0.1",
|
||||
port: Number(process.env.VILI_DAEMON_PORT || 8824),
|
||||
stateRoot: process.env.VILI_STATE_ROOT || path.join(projectRoot, ".vili"),
|
||||
|
|
@ -134,13 +152,31 @@ function parseArgs(argv) {
|
|||
}
|
||||
|
||||
const args = parseArgs(process.argv.slice(2));
|
||||
const hosts = args.host.split(",").map((host) => host.trim()).filter(Boolean);
|
||||
const startedAt = new Date().toISOString();
|
||||
const gpuOptions = gpuTenancyOptionsFromEnv(process.env);
|
||||
// The daemon's one cache and store instance: every write to vili.service.cc goes through one queue.
|
||||
const viliCache = CultCache.builder()
|
||||
.withRegistry(viliDocumentRegistry)
|
||||
.withGenericStore(new SingleFileMessagePackBackingStore(path.join(args.stateRoot, viliStoreFileName)))
|
||||
.build();
|
||||
let storedPermission;
|
||||
if (!args.health && !args.printProviderAdvertisement) {
|
||||
await viliCache.pullAllBackingStores();
|
||||
storedPermission = viliCache.get(gpuPermissionDefinition, "raven");
|
||||
}
|
||||
const gpuTenancy = createGpuTenancy({
|
||||
now: Date.now,
|
||||
...gpuOptions,
|
||||
permission: storedPermission,
|
||||
savePermission: (value) => viliCache.put(gpuPermissionDefinition, "raven", { host: "raven", ...value }),
|
||||
readFreeMiB: hostFreeMiB,
|
||||
});
|
||||
let backendStatus = {
|
||||
checkedAt: null,
|
||||
dockerImage: "unknown",
|
||||
nvidiaRuntime: "unknown",
|
||||
huggingFaceToken: "unknown",
|
||||
kimodoHelp: "unknown",
|
||||
kimodoWorker: "unknown",
|
||||
kimodoWorkerDetail: null,
|
||||
lastError: null,
|
||||
|
|
@ -171,6 +207,11 @@ function wsl(script, options = {}) {
|
|||
return run("wsl.exe", ["-d", "Ubuntu-24.04", "-u", "root", "--", "bash", "-lc", script], options);
|
||||
}
|
||||
|
||||
async function hostFreeMiB() {
|
||||
const r = await run("nvidia-smi.exe", ["--query-gpu=memory.free", "--format=csv,noheader,nounits"], { timeout: 5000 });
|
||||
return r.ok ? parseFreeMiB(r.stdout) : null;
|
||||
}
|
||||
|
||||
function shQuote(value) {
|
||||
return `'${String(value).replaceAll("'", "'\\''")}'`;
|
||||
}
|
||||
|
|
@ -193,12 +234,12 @@ async function wslJson(script, options = {}) {
|
|||
}
|
||||
}
|
||||
|
||||
function json(response, statusCode, body) {
|
||||
function json(response, statusCode, body, headers = {}) {
|
||||
const payload = JSON.stringify(body, null, 2);
|
||||
response.writeHead(statusCode, {
|
||||
...headers,
|
||||
"content-type": "application/json; charset=utf-8",
|
||||
"cache-control": "no-store",
|
||||
"access-control-allow-origin": "*",
|
||||
});
|
||||
response.end(payload);
|
||||
}
|
||||
|
|
@ -207,7 +248,6 @@ function text(response, statusCode, body) {
|
|||
response.writeHead(statusCode, {
|
||||
"content-type": "text/plain; charset=utf-8",
|
||||
"cache-control": "no-store",
|
||||
"access-control-allow-origin": "*",
|
||||
});
|
||||
response.end(body);
|
||||
}
|
||||
|
|
@ -226,7 +266,6 @@ function providerAdvertisement() {
|
|||
endpoints: {
|
||||
http: ravenBaseUrl,
|
||||
health: `${ravenBaseUrl}/health`,
|
||||
smoke: `${ravenBaseUrl}/smoke`,
|
||||
providerAdvertisement: `${ravenBaseUrl}/provider-advertisement`,
|
||||
operatorState: `${ravenBaseUrl}/operator-state`,
|
||||
operatorSurface: `${ravenBaseUrl}/eve/operator`,
|
||||
|
|
@ -252,6 +291,7 @@ function operatorState() {
|
|||
startedAt,
|
||||
updatedAt: new Date().toISOString(),
|
||||
backend: backendStatus,
|
||||
gpu: gpuTenancy.status(),
|
||||
authority: {
|
||||
owns: [
|
||||
"animation job intake",
|
||||
|
|
@ -273,8 +313,8 @@ function operatorState() {
|
|||
operatorState: "/operator-state",
|
||||
operatorSurface: "/eve/operator",
|
||||
deckProviders: "/eve/deck/providers",
|
||||
smoke: "/smoke",
|
||||
generate: "/motion/generate",
|
||||
gpuPermission: "/gpu-permission",
|
||||
},
|
||||
};
|
||||
}
|
||||
|
|
@ -298,10 +338,10 @@ function commandBoundary() {
|
|||
"motion artifact references",
|
||||
],
|
||||
},
|
||||
smoke: {
|
||||
method: "GET",
|
||||
path: "/smoke",
|
||||
owns: ["backend readiness proof", "resident worker warmup"],
|
||||
gpuPermission: {
|
||||
method: "POST",
|
||||
path: "/gpu-permission",
|
||||
owns: ["the household GPU permission for this host, from loopback only"],
|
||||
},
|
||||
liveness: {
|
||||
method: "GET",
|
||||
|
|
@ -353,10 +393,8 @@ function eveSurface() {
|
|||
{ type: "stat", label: "Verse", value: "raven.local" },
|
||||
{ type: "stat", label: "Docker image", value: state.backend.dockerImage },
|
||||
{ type: "stat", label: "NVIDIA runtime", value: state.backend.nvidiaRuntime },
|
||||
{ type: "stat", label: "Kimodo", value: state.backend.kimodoHelp },
|
||||
{ type: "stat", label: "Kimodo worker", value: state.backend.kimodoWorker },
|
||||
{ type: "stat", label: "HF token", value: state.backend.huggingFaceToken },
|
||||
{ type: "command", label: "Smoke", method: "GET", href: `${ravenBaseUrl}/smoke` },
|
||||
{ type: "command", label: "Generate motion", method: "POST", href: `${ravenBaseUrl}/motion/generate` },
|
||||
],
|
||||
},
|
||||
|
|
@ -377,59 +415,31 @@ async function persistSurfaces() {
|
|||
}
|
||||
|
||||
async function persistCultCacheSurfaces() {
|
||||
const cache = CultCache.builder()
|
||||
.withRegistry(viliDocumentRegistry)
|
||||
.withGenericStore(new SingleFileMessagePackBackingStore(path.join(args.stateRoot, viliStoreFileName)))
|
||||
.build();
|
||||
await cache.put(providerAdvertisementDefinition, providerId, providerAdvertisement());
|
||||
await cache.put(operatorStateDefinition, "vili", operatorState());
|
||||
await cache.put(eveSurfaceDefinition, providerId, eveSurface());
|
||||
await cache.put(commandBoundaryDefinition, "vili", commandBoundary());
|
||||
await cache.put(transportProfileDefinition, "vili", transportProfile());
|
||||
await viliCache.put(providerAdvertisementDefinition, providerId, providerAdvertisement());
|
||||
await viliCache.put(operatorStateDefinition, "vili", operatorState());
|
||||
await viliCache.put(eveSurfaceDefinition, providerId, eveSurface());
|
||||
await viliCache.put(commandBoundaryDefinition, "vili", commandBoundary());
|
||||
await viliCache.put(transportProfileDefinition, "vili", transportProfile());
|
||||
}
|
||||
|
||||
async function refreshBackendStatus({ includeKimodoHelp = false } = {}) {
|
||||
const docker = await wsl([
|
||||
"set -e",
|
||||
"service docker start >/dev/null 2>&1 || true",
|
||||
"docker images gamecult/kimodo:latest --format '{{.Repository}}:{{.Tag}}' | head -n 1",
|
||||
].join("; "), { timeout: 20000 });
|
||||
|
||||
let nvidia = await wsl([
|
||||
"set -e",
|
||||
"service docker start >/dev/null 2>&1 || true",
|
||||
"docker info --format '{{json .Runtimes}}' | grep -q nvidia && echo configured || echo missing",
|
||||
].join("; "), { timeout: 20000 });
|
||||
|
||||
// A read: it only asks. Starting docker and the worker belongs to a job that holds the GPU slot.
|
||||
async function refreshBackendStatus() {
|
||||
const docker = await wsl("docker images gamecult/kimodo:latest --format '{{.Repository}}:{{.Tag}}' | head -n 1", { timeout: 20000 });
|
||||
const nvidia = await wsl("docker info --format '{{json .Runtimes}}' | grep -q nvidia && echo configured || echo missing", { timeout: 20000 });
|
||||
const token = await wsl("test -f /root/.cache/huggingface/token && echo present || echo missing", { timeout: 10000 });
|
||||
const worker = await kimodoWorkerHealth(20000);
|
||||
let help = { ok: true, stdout: backendStatus.kimodoHelp === "ok" ? "ok" : "not checked", stderr: "" };
|
||||
if (includeKimodoHelp) {
|
||||
nvidia = await wsl([
|
||||
"set -e",
|
||||
"service docker start >/dev/null 2>&1 || true",
|
||||
"docker run --rm --runtime=nvidia --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi --query-gpu=name --format=csv,noheader | head -n 1",
|
||||
].join("; "), { timeout: 45000 });
|
||||
help = await wsl([
|
||||
"set -e",
|
||||
"service docker start >/dev/null 2>&1 || true",
|
||||
dockerHuggingFaceEnv,
|
||||
`docker run --rm --runtime=nvidia --gpus all ${dockerHuggingFaceArgs} gamecult/kimodo:latest kimodo_gen --help | sed -n '1,6p'`,
|
||||
].join("; "), { timeout: 60000 });
|
||||
}
|
||||
|
||||
backendStatus = {
|
||||
checkedAt: new Date().toISOString(),
|
||||
dockerImage: docker.ok && docker.stdout ? docker.stdout : "missing",
|
||||
nvidiaRuntime: nvidia.ok && nvidia.stdout ? nvidia.stdout : "unavailable",
|
||||
huggingFaceToken: token.ok ? token.stdout : "unknown",
|
||||
kimodoHelp: includeKimodoHelp ? (help.ok ? "ok" : "failed") : backendStatus.kimodoHelp,
|
||||
kimodoWorker: worker.ok && worker.body?.ok ? "resident" : "not running",
|
||||
kimodoWorkerDetail: worker.body || null,
|
||||
lastError: [docker, nvidia, token, help].find((result) => !result.ok)?.stderr || null,
|
||||
lastError: [docker, nvidia, token].find((result) => !result.ok)?.stderr || null,
|
||||
};
|
||||
await persistSurfaces();
|
||||
return { docker, nvidia, token, help, backendStatus };
|
||||
return backendStatus;
|
||||
}
|
||||
|
||||
async function kimodoWorkerHealth(timeout = 10000) {
|
||||
|
|
@ -502,7 +512,43 @@ async function readRequestBody(request) {
|
|||
return raw ? JSON.parse(raw) : {};
|
||||
}
|
||||
|
||||
const loopbackAddresses = new Set(["127.0.0.1", "::1", "::ffff:127.0.0.1"]);
|
||||
|
||||
// A state-changing POST is a service's or the tray's call, never a web page's. A browser sends a
|
||||
// cross-site page's POST as a no-preflight simple request (text/plain) or with an Origin that is not
|
||||
// ours, so both are refused before the body is read. The daemon grants no CORS preflight at all.
|
||||
function refuseCrossSite(request, response) {
|
||||
const origin = request.headers.origin;
|
||||
const ownOrigins = [`http://127.0.0.1:${args.port}`, `http://localhost:${args.port}`, `http://[::1]:${args.port}`];
|
||||
const type = String(request.headers["content-type"] || "").split(";")[0].trim().toLowerCase();
|
||||
if (type === "application/json" && (origin === undefined || ownOrigins.includes(origin))) return false;
|
||||
json(response, 403, { ok: false, code: "cross-site" });
|
||||
return true;
|
||||
}
|
||||
|
||||
// The household's switch: only a caller on this machine (the tray, in the household's session) sets it.
|
||||
async function setGpuPermission(request, response) {
|
||||
if (refuseCrossSite(request, response)) return;
|
||||
if (!loopbackAddresses.has(request.socket.remoteAddress)) {
|
||||
json(response, 403, { ok: false, code: "household-only" });
|
||||
return;
|
||||
}
|
||||
let body;
|
||||
try {
|
||||
body = await readRequestBody(request);
|
||||
} catch (error) {
|
||||
json(response, 400, { ok: false, error: `Invalid JSON: ${error.message}` });
|
||||
return;
|
||||
}
|
||||
if (typeof body?.allowed !== "boolean") {
|
||||
json(response, 400, { ok: false, error: "allowed must be a boolean." });
|
||||
return;
|
||||
}
|
||||
json(response, 200, { ok: true, gpu: await gpuTenancy.setAllowed(body.allowed) });
|
||||
}
|
||||
|
||||
async function generateMotion(request, response) {
|
||||
if (refuseCrossSite(request, response)) return;
|
||||
let body;
|
||||
try {
|
||||
body = await readRequestBody(request);
|
||||
|
|
@ -526,6 +572,13 @@ async function generateMotion(request, response) {
|
|||
return;
|
||||
}
|
||||
|
||||
const admitted = await admitThen(gpuTenancy, () => runMotionJob(body, prompt, response));
|
||||
if (admitted?.code === "gpu-busy") {
|
||||
json(response, 503, admitted, { "retry-after": String(RETRY_AFTER_SECONDS) });
|
||||
}
|
||||
}
|
||||
|
||||
async function runMotionJob(body, prompt, response) {
|
||||
const jobId = `vili-${Date.now()}`;
|
||||
const artifactDir = path.join(args.stateRoot, "artifacts", jobId);
|
||||
await mkdir(artifactDir, { recursive: true });
|
||||
|
|
@ -650,23 +703,15 @@ async function generateMotion(request, response) {
|
|||
await mkdir(path.join(args.stateRoot, "jobs"), { recursive: true });
|
||||
await writeFile(path.join(args.stateRoot, "jobs", `${jobId}.json`), JSON.stringify(job, null, 2));
|
||||
});
|
||||
json(response, result.ok ? 200 : 502, job);
|
||||
const placement = result.ok ? null : placementRefusal(result.body, jobId);
|
||||
if (placement) json(response, 503, placement, { "retry-after": String(RETRY_AFTER_SECONDS) });
|
||||
else json(response, result.ok ? 200 : 502, job);
|
||||
}
|
||||
|
||||
async function handleRequest(request, response) {
|
||||
const url = new URL(request.url, `http://${request.headers.host || `${args.host}:${args.port}`}`);
|
||||
if (request.method === "OPTIONS") {
|
||||
response.writeHead(204, {
|
||||
"access-control-allow-origin": "*",
|
||||
"access-control-allow-methods": "GET,POST,OPTIONS",
|
||||
"access-control-allow-headers": "content-type",
|
||||
});
|
||||
response.end();
|
||||
return;
|
||||
}
|
||||
|
||||
const url = new URL(request.url, `http://${request.headers.host || `${hosts[0]}:${args.port}`}`);
|
||||
if (url.pathname === "/" || url.pathname === "/health") {
|
||||
await refreshBackendStatus({ includeKimodoHelp: false });
|
||||
await refreshBackendStatus();
|
||||
json(response, 200, { ok: true, service: "vili", providerId, state: operatorState() });
|
||||
} else if (url.pathname === "/provider-advertisement") {
|
||||
json(response, 200, providerAdvertisement());
|
||||
|
|
@ -683,12 +728,10 @@ async function handleRequest(request, response) {
|
|||
endpoint: `${ravenDeckUrl}/${providerId}`,
|
||||
}],
|
||||
});
|
||||
} else if (url.pathname === "/smoke") {
|
||||
const result = await refreshBackendStatus({ includeKimodoHelp: true });
|
||||
const worker = await ensureKimodoWorker();
|
||||
await refreshBackendStatus({ includeKimodoHelp: false });
|
||||
const ok = result.backendStatus.kimodoHelp === "ok" && worker.ok && worker.body?.ok;
|
||||
json(response, ok ? 200 : 502, { ok, ...result, worker: worker.body || worker });
|
||||
} else if (url.pathname === "/gpu-permission" && request.method === "GET") {
|
||||
json(response, 200, gpuTenancy.status());
|
||||
} else if (url.pathname === "/gpu-permission" && request.method === "POST") {
|
||||
await setGpuPermission(request, response);
|
||||
} else if (url.pathname === "/motion/generate" && request.method === "POST") {
|
||||
await generateMotion(request, response);
|
||||
} else {
|
||||
|
|
@ -697,8 +740,7 @@ async function handleRequest(request, response) {
|
|||
}
|
||||
|
||||
async function healthCheck() {
|
||||
const host = args.host === "0.0.0.0" ? "127.0.0.1" : args.host;
|
||||
const response = await fetch(`http://${host}:${args.port}/health`);
|
||||
const response = await fetch(`http://${hosts[0]}:${args.port}/health`);
|
||||
if (!response.ok) throw new Error(`health returned ${response.status}`);
|
||||
const body = await response.json();
|
||||
if (!body.ok) throw new Error("health returned ok=false");
|
||||
|
|
@ -715,15 +757,13 @@ if (args.health) {
|
|||
process.exit(1);
|
||||
});
|
||||
} else {
|
||||
if (hosts.length === 0 || hosts.includes("0.0.0.0") || hosts.includes("::")) {
|
||||
console.error("--host must name the addresses to listen on, never a wildcard.");
|
||||
process.exit(2);
|
||||
}
|
||||
await mkdir(path.join(args.stateRoot, "jobs"), { recursive: true });
|
||||
await persistSurfaces();
|
||||
|
||||
const server = http.createServer((request, response) => {
|
||||
handleRequest(request, response).catch((error) => {
|
||||
json(response, 500, { ok: false, error: error.message, stack: process.env.NODE_ENV === "development" ? error.stack : undefined });
|
||||
});
|
||||
});
|
||||
|
||||
const wss = new WebSocketServer({ noServer: true });
|
||||
wss.on("connection", (socket) => {
|
||||
const sendSurface = () => {
|
||||
|
|
@ -734,18 +774,27 @@ if (args.health) {
|
|||
socket.on("close", () => clearInterval(timer));
|
||||
});
|
||||
|
||||
server.on("upgrade", (request, socket, head) => {
|
||||
const url = new URL(request.url, `http://${request.headers.host || `${args.host}:${args.port}`}`);
|
||||
if (url.pathname === `/eve/deck/${providerId}` || url.pathname === "/eve/deck") {
|
||||
wss.handleUpgrade(request, socket, head, (ws) => wss.emit("connection", ws, request));
|
||||
} else {
|
||||
socket.destroy();
|
||||
}
|
||||
});
|
||||
|
||||
server.listen(args.port, args.host, () => {
|
||||
console.log(`Vili listening on http://${args.host}:${args.port}`);
|
||||
const servers = hosts.map((host) => {
|
||||
const server = http.createServer((request, response) => {
|
||||
handleRequest(request, response).catch((error) => {
|
||||
json(response, 500, { ok: false, error: error.message, stack: process.env.NODE_ENV === "development" ? error.stack : undefined });
|
||||
});
|
||||
});
|
||||
server.on("upgrade", (request, socket, head) => {
|
||||
const url = new URL(request.url, `http://${request.headers.host || `${hosts[0]}:${args.port}`}`);
|
||||
if (url.pathname === `/eve/deck/${providerId}` || url.pathname === "/eve/deck") {
|
||||
wss.handleUpgrade(request, socket, head, (ws) => wss.emit("connection", ws, request));
|
||||
} else {
|
||||
socket.destroy();
|
||||
}
|
||||
});
|
||||
return new Promise((resolve, reject) => {
|
||||
server.once("error", reject);
|
||||
server.listen(args.port, host, resolve);
|
||||
});
|
||||
});
|
||||
await Promise.all(servers);
|
||||
console.log(`Vili listening on ${hosts.map((host) => `http://${host}:${args.port}`).join(" ")}`);
|
||||
|
||||
setInterval(() => persistSurfaces().catch((error) => console.error(error)), 5000);
|
||||
}
|
||||
|
|
|
|||
98
scripts/vili-gpu-tray.ps1
Normal file
98
scripts/vili-gpu-tray.ps1
Normal file
|
|
@ -0,0 +1,98 @@
|
|||
# The household's switch for Raven's GPU: one tray icon in the interactive session. It shows whether
|
||||
# Vili may use the GPU and flips it with one click. It holds no state of its own: the icon and tooltip
|
||||
# are a lowering of GET /gpu-permission, and the click is one POST with an absolute value.
|
||||
param([int]$Port = 8824, [int]$PollMs = 5000)
|
||||
|
||||
Add-Type -AssemblyName System.Windows.Forms
|
||||
Add-Type -AssemblyName System.Drawing
|
||||
|
||||
$mutex = New-Object System.Threading.Mutex($false, 'Local\GameCult.ViliGpuTray')
|
||||
if (-not $mutex.WaitOne(0)) { exit 0 }
|
||||
|
||||
$base = "http://127.0.0.1:$Port/gpu-permission"
|
||||
$script:state = 'down'
|
||||
$script:jobRunning = $false
|
||||
$clock = [Diagnostics.Stopwatch]::StartNew()
|
||||
$script:lastToggleMs = -10000
|
||||
|
||||
function New-TrayIcon([string]$kind) {
|
||||
$bmp = New-Object System.Drawing.Bitmap 16, 16
|
||||
$g = [System.Drawing.Graphics]::FromImage($bmp)
|
||||
$g.SmoothingMode = 'AntiAlias'
|
||||
if ($kind -eq 'on') {
|
||||
$g.FillEllipse([System.Drawing.Brushes]::LimeGreen, 1, 1, 13, 13)
|
||||
} elseif ($kind -eq 'off') {
|
||||
$g.FillEllipse([System.Drawing.Brushes]::Crimson, 1, 1, 13, 13)
|
||||
$g.FillRectangle([System.Drawing.Brushes]::White, 4, 7, 8, 2)
|
||||
} else {
|
||||
$pen = New-Object System.Drawing.Pen ([System.Drawing.Color]::Gray), 2
|
||||
$g.DrawEllipse($pen, 2, 2, 11, 11)
|
||||
$pen.Dispose()
|
||||
}
|
||||
$g.Dispose()
|
||||
$icon = [System.Drawing.Icon]::FromHandle($bmp.GetHicon())
|
||||
return $icon
|
||||
}
|
||||
|
||||
$icons = @{ on = (New-TrayIcon 'on'); off = (New-TrayIcon 'off'); down = (New-TrayIcon 'down') }
|
||||
|
||||
$tray = New-Object System.Windows.Forms.NotifyIcon
|
||||
$menu = New-Object System.Windows.Forms.ContextMenuStrip
|
||||
$item = $menu.Items.Add('Let Vili use the GPU')
|
||||
$tray.ContextMenuStrip = $menu
|
||||
|
||||
function Show-State {
|
||||
$tray.Icon = $icons[$script:state]
|
||||
$item.Checked = ($script:state -eq 'on')
|
||||
$item.Enabled = ($script:state -ne 'down')
|
||||
if ($script:state -eq 'on' -and $script:jobRunning) { $tray.Text = 'Vili may use the GPU (making a clip). Click to stop.' }
|
||||
elseif ($script:state -eq 'on') { $tray.Text = 'Vili may use the GPU. Click to stop.' }
|
||||
elseif ($script:state -eq 'off') { $tray.Text = 'Vili may not use the GPU. Click to allow.' }
|
||||
else { $tray.Text = 'Vili is not running, so it is not using the GPU.' }
|
||||
}
|
||||
|
||||
function Update-State {
|
||||
try {
|
||||
$r = Invoke-RestMethod -Uri $base -Method Get -TimeoutSec 2
|
||||
if ($r.allowed -eq $true) { $script:state = 'on' } elseif ($r.allowed -eq $false) { $script:state = 'off' } else { $script:state = 'down' }
|
||||
$script:jobRunning = ($r.jobRunning -eq $true)
|
||||
} catch {
|
||||
$script:state = 'down'
|
||||
$script:jobRunning = $false
|
||||
}
|
||||
Show-State
|
||||
}
|
||||
|
||||
function Switch-Permission {
|
||||
try {
|
||||
if ($clock.ElapsedMilliseconds - $script:lastToggleMs -lt 1000) { return }
|
||||
$script:lastToggleMs = $clock.ElapsedMilliseconds
|
||||
if ($script:state -eq 'down') {
|
||||
$tray.ShowBalloonTip(3000, 'Vili', 'Vili is not running. Nothing to change.', 'Info')
|
||||
return
|
||||
}
|
||||
$body = ConvertTo-Json -Compress @{ allowed = ($script:state -ne 'on') }
|
||||
try {
|
||||
[void](Invoke-RestMethod -Uri $base -Method Post -Body $body -ContentType 'application/json' -TimeoutSec 2)
|
||||
} catch {
|
||||
$tray.ShowBalloonTip(3000, 'Vili', 'Could not reach Vili. Nothing changed.', 'Warning')
|
||||
}
|
||||
Update-State
|
||||
} catch {
|
||||
}
|
||||
}
|
||||
|
||||
$tray.Add_MouseUp({
|
||||
param($sender, $e)
|
||||
try { if ($e.Button -eq [System.Windows.Forms.MouseButtons]::Left) { Switch-Permission } } catch {}
|
||||
})
|
||||
$item.Add_Click({ try { Switch-Permission } catch {} })
|
||||
|
||||
$timer = New-Object System.Windows.Forms.Timer
|
||||
$timer.Interval = $PollMs
|
||||
$timer.Add_Tick({ try { Update-State } catch {} })
|
||||
|
||||
Update-State
|
||||
$tray.Visible = $true
|
||||
$timer.Start()
|
||||
[System.Windows.Forms.Application]::Run()
|
||||
Loading…
Reference in a new issue