Commit graph Vili/scripts
Author SHA1 Message Date
Meta Vens
f69e24eeff Test: the spawn is gated at the spawn, not by the request's earlier checks
The fake Idunn signer can lose the lease once a job's meta.json exists, which
pins the guards in startWorker (no logs or embeddings directory, no worker).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 03:06:18 +02:00
Meta Vens
a1579efbbc One owner by one bind, one gate at every write: the abstract-socket lock and guard()
The hand-run lock is a bind in the abstract socket namespace named by the state
root: atomic, exclusive, gone with its owner, so the probe, the unlink and the
takeover race are deleted. Every write to the state root and every spawn calls
guard(), which asks held() at that moment; Idunn's held() reads the lease each
time and reports a loss through the handler the daemon registered, so a request
admitted before the loss stops at its next write and a respawn cannot outlive it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 03:04:31 +02:00
Meta Vens
17e84c65da Tests first: crashed-lock race, in-flight fence, SIGTERM stop, foreign reclaim, port range
Red at 4fd3f60 for stale-lock-takeover-race, fence-admits-in-flight-request and
bind-port-65536-accepted; green guards for reclaim-root-unpinned and the SIGTERM
property. The daemon's stop() no longer leaves a daemon holding the run open.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 03:01:17 +02:00
Meta Vens
4fd3f60c88 Test: a second daemon that fails to refuse is killed, not left holding the test run open
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 01:33:22 +02:00
Meta Vens
6a80b3b4b8 Test: the child refuses a vector of the wrong width
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 00:46:41 +02:00
Meta Vens
235321ed86 Entry: derive the daemon's environment in one exported function, and test its argument contract
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 00:45:01 +02:00
Meta Vens
956fdf8ff8 Daemon serves through a runtime port: Idunn write lease or hand-run socket lock
serve(runtime) listens first, writes the state root only once the runtime
holds it, answers Idunn's route challenge ahead of the cross-site refusal,
and fences on lease loss. --health is deleted; the pid file and /proc
reclaim move behind the hand-run runtime. Adds scripts/idunn-entry.mjs,
the kimodo and SOMA-X gitlinks, encode_texts for the child, and tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 00:23:07 +02:00
Meta Vens
70f60b9b8a Daemon tests: the pid file's life, strays that ignore SIGTERM, a worker with no encoder child, a live worker left alone
Closes the survivors StrykerJS reported over the new supervisor code (61 mutants, 41 killed, 20 survived): the pid
file is removed when its worker exits and not when it spawned nothing, the restart kill is SIGKILL (a stubborn stray
and a SIGTERM-proof encoder child), the group match is the pid file's, not any marked process. killGroup loses its
unused return value.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 17:54:01 +02:00
Meta Vens
6651662b2c Encoder: pool after LLM2Vec's query marker, live_identity reads its own versions, tests reach every field
The header-on vector now matches Kimodo's encoder loaded from the repo ids (cosine 1.0; the bare header gave
0.9974) because the wrapped text carries LLM2Vec's instruction/query separator, which keeps the header out of
the pooling. live_identity reads the installed versions itself, so the worker passes no version it could fake,
and the tests pin installed_versions against substituted modules, the prompt form and device in the identity,
and a text that differs past any prefix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 17:40:08 +02:00
Meta Vens
2bf4a1dba7 Daemon: own the worker's whole tree, across worker exits and daemon crashes
The worker's process group is killed when the worker exits for any reason, so no encoder child survives it.
A pid file in the state root names the group; the next daemon start kills a group left by a crashed
predecessor if one of its members carries this state root's embeddings directory, before serving. The
retired operator route's 404 test returns (built from parts, so the grep stays empty) and the test script
gets a per-test timeout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 17:20:01 +02:00
Meta Vens
4fe1fc963c Encoder: apply the Llama-3 chat header explicitly, and pin the identity at the worker's call site
The child wraps every text with text_encoding.chat_header and tokenizes it itself, so the prompt form no
longer rides on the model's _name_or_path. The identity names the assembly truthfully (header form
included, as a hash of the template) and re-keys the cache. The worker builds its identity through
live_identity and installed_versions, which the tests reach; the identity test now uses live-length
revisions, so a truncated identity or key fails. An env-gated test compares a fresh child vector with
Kimodo's own encoder loaded from the repo ids.

Supersedes encoder-assembly-b (motion-pipeline:ruling:encoder-chat-header-on).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 17:20:01 +02:00
Meta Vens
e673ec1d4a Test: the retired provider-advertisement route answers 404
Keeps the Eve path out of scripts/ so the negative grep over scripts/ is empty.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 15:28:55 +02:00
Meta Vens
b3ef8f3090 Test: drop the lastError check that no mutant could fail
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 15:22:45 +02:00
Meta Vens
c3b048a2ab Daemon: drop the unread worker.model; tests for grace, host default, relative models root, early death
Mutation triage (StrykerJS, 81 survivors on the first full run) showed worker.model was
written and never read. The new tests pin the survivors that were behaviour: the loopback
default, --host, stray flags, the models-root resolution, the SIGTERM grace, a worker that
dies while the daemon waits for its health, the POST route, and lastError after a
requested stop.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 15:07:15 +02:00
Meta Vens
3843a772f8 Tests: worker failure paths, env twins, group kill, job record, store refresh
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:16:50 +02:00
Meta Vens
94e08d1fdc Test: write the snapshot revision file with a real newline escape
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 12:32:53 +02:00
Meta Vens
6487c7a063 Test: the worker applies the thread cap before it loads the model
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 12:32:04 +02:00
Meta Vens
ffa2c7bf58 Encoder B loads from snapshot directories, as the reference vector did
Measured on Yggdrasil: loading by repo id leaves the model's _name_or_path
equal to the Llama repo id, so prepare_for_tokenization wraps the text in
the chat header and the vector matches assembly A (cosine 1.0, 0.909 to the
reference B). Loading from the snapshot paths reproduces the reference.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 12:22:25 +02:00
Meta Vens
ef0edc4073 Daemon: supervise the worker child, drop WSL, Docker, Eve and CORS
vili-daemon.mjs spawns the Python worker itself (no shell, port 0 from the
ready line, nice 10, one child at a time, respawn after exit, group kill on
SIGTERM) with a closed environment. WSL/Docker, the Eve surfaces and deck
WebSocket, the provider advertisement, /smoke, CORS, JSON sidecars and the
pid file are deleted. POSTs with an Origin or a non-JSON body are refused
403 cross-site. cultcache-ts comes from the vendor/CultLib submodule at
b549d731; ws is dropped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 12:14:10 +02:00
Meta Vens
c3684b733b Encoder: assembly B on CPU with a complete identity and a thread cap
The encoder child is CPU only (GPU placement, GpuBusy and plan_placement
are deleted) and assembles LLM2Vec B: Llama base, MNTP merged, then the
supervised adapter. The cache identity names assembly, device, dtype,
torch/transformers/peft/accelerate versions, Kimodo commit and the child's
own sha256 beside the three revisions. VILI_THREADS (1..16) caps torch in
the worker and the child. The worker binds port 0 and reports it in its
ready line; errors name a class and place, never the request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 12:14:09 +02:00
Meta Vens
0348b3a283 Delete the Raven launchers and the /smoke client
Raven scheduled-task installer, launchers, restart shims and the smoke client
(GET /smoke on 10.77.0.4) leave main under the Loud Rebuild Contract
(motion-pipeline:ruling:vili-moves-to-yggdrasil-now).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 12:06:38 +02:00
Meta Vens
02ca1cd6d8 Vili README and tests: describe the embedding cache and the transient encoder
Replaces the 2026-06-12 timing paragraph with the cache, the child, and the numbers
measured on Raven on 2026-10-22. Adds the tests hand mutation showed missing: the
exact 2 GiB boundary, a child answer with too few vectors, and the 4096 dimension.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 21:40:42 +02:00
Meta Vens
f6acfc1c85 Vili worker: Kimodo's text encoder is the cache, not a resident 8B model
The worker builds CachedTextEncoder and passes it to load_model, so Kimodo's own
encoder selection (and the CPU encoder behind TEXT_ENCODER_DEVICE) is never built.
generate() returns the per-request encoder record (identity, hits, misses, seconds,
vector hashes) and the worker identity; /health adds the identity and stored count.
startKimodoWorker drops TEXT_ENCODER_DEVICE and mounts the scripts directory, the
cultcache-py source and .vili/embeddings, with the Kimodo commit and image id read
once per worker start. The encode clock is perf_counter: time.time went backwards
under WSL during a probe.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 21:18:17 +02:00
Meta Vens
76ab8308eb Vili: text-embedding cache and one-shot GPU encoder child
text_encoding.py holds the CultCache embedding store (gamecult.vili.text_embedding.v0
through cultcache-py), the encoder identity and key, plan_placement with its typed
gpu-busy refusal, and CachedTextEncoder, which sends a request's misses to one child
call. kimodo-text-encode.py is that child: it assembles Kimodo's LLM2Vec encoder in
bf16 with its own loader (operator ruling encoder-own-loader; peft's offload path
raises TypeError in Kimodo's loader), encodes each text alone, writes a MessagePack
answer and exits. GPU reserve 3 GiB: four cold runs on Raven's 16 GB card produced
identical vectors with no CUDA error. The worker and daemon are not wired yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 21:10:01 +02:00
Meta Vens
3778830df4 Vili: delete the unsigned Idunn RUDP health publisher
It published idunn.daemon_health to 10.77.0.2:17870 (the retired Windows Idunn)
every 15 s and failed each time; Idunn on Yggdrasil binds no RUDP health ingress.
Deletes idunn-rudp.cjs, the publisher and its status, the --idunn-* args in the
daemon and the three launcher scripts, operator_state slot 4, and the
cultnet-ts and direct @msgpack/msgpack dependencies. transportProfile now states
local-cultcache-only. All commits build (node --check passes); not yet run on Raven.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 05:12:00 +02:00
Meta Vens
8773c153c8 Remove unused readFile import and dead backend-status.json initializer
REC-120:

- Removed the unused `readFile` import (`readFile(` had zero call
  sites in the file).
- Removed the never-read `backend-status.json` initializer block; the
  `backendStatus` in-memory object it snapshotted is still very much
  alive (read at lines 287, 378, 489, 504, 509, 515, 768) and untouched
  here.
- Also removed the `existsSync` import, which the initializer block
  was its only caller for; left in place it would have become a second
  dead import introduced by this same cut.

Kill attempts that failed: none.
- `rg -n "readFile\(" scripts/vili-daemon.mjs` -> no matches (repeated
  the census command against current HEAD).
- `rg -n -F --glob '!scripts/vili-daemon.mjs' 'backend-status.json' .`
  -> no matches in this repo; cross-repo `rg -n 'backend-status\.json'`
  over F:/Projects only turns up Odin's
  docs/transport-shortcut-inventory.md, which documents the shortcut
  rather than reading the file.

Checks run: `node --check scripts/vili-daemon.mjs` -> syntax OK. No
package.json/test suite exists in this repo to run beyond that.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-13 14:18:49 +02:00
Meta Vens
8627297a6e Write Vili service state to CultCache 2026-06-16 10:39:54 +01:00
Meta Vens
50c424380c Publish Vili health over Idunn RUDP 2026-06-16 10:22:31 +01:00
Meta Vens
7cb320d11f Keep Raven Vili task background-only 2026-06-16 02:25:13 +01:00
Meta Vens
16d74cb674 Expose Kimodo generation options 2026-06-13 00:55:34 +01:00
Meta Vens
3560548146 Make Kimodo generation resident 2026-06-12 18:04:43 +01:00
Meta Vens
c22913c041 Run Vili scheduled task hidden 2026-06-12 17:22:20 +01:00
Meta Vens
b7297acbcd Use Kimodo input folders for generation prompts 2026-06-12 15:07:01 +01:00
Meta Vens
2f6a5bd2ce Decode generation prompts inside WSL 2026-06-12 15:00:08 +01:00
Meta Vens
974656a13b Allow Kimodo containers to warm Hugging Face cache 2026-06-12 14:32:34 +01:00
Meta Vens
e8dc7d08d7 Use Kimodo output stem for generation jobs 2026-06-12 14:24:58 +01:00
Meta Vens
ea9c7f8845 Pass Hugging Face token into Kimodo containers 2026-06-12 13:57:14 +01:00
Meta Vens
01f6bb9c6e Run Vili task with Raven user WSL access 2026-06-12 12:49:19 +01:00
Meta Vens
d59a76d8ea Run Vili scheduled task as SYSTEM 2026-06-12 12:47:54 +01:00
Meta Vens
de2ad53e48 Run Vili through a Raven scheduled task 2026-06-12 12:46:44 +01:00
Meta Vens
ca6a172cdd Initial Vili animation daemon 2026-06-12 12:42:33 +01:00