Tool-calling cut §5: summary/wait verify, snapshot mutation runner #1
2026-10-01
Tool-calling map: record the follow-ups from the tools' Soul passes
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mutation-table: delete the dead cur branch of the cleanup trap
The --rm mutant container is gone by the time the trap runs, so `docker rm -f $cur` never had a container to remove. The stopgap's watchdog kills every job-labelled container before it signals the driver, so nothing needs tracking here. Also restores a line continuation the control-run edit had collapsed. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
ygg-verify: a RUN_ON_HOST job the watchdog killed exits 124, in its work directory, and its process group is signalled
A RUN_ON_HOST job whose TIMEOUT fired still exited 0: kill_job only killed labelled containers and the status came from the host command (Soul F2). The host command now runs in its own process group (set -m), in the job's work directory (it ran in the ssh home), with stdin from /dev/null. kill_job sends the group TERM, then KILL 20 s later, and the verdict is forced to 124 when the watchdog killed the job, whatever the command exited with. The TIMEOUT/IDLE notice now prints before the kill so the 20 s grace cannot swallow it. Proved on Yggdrasil with the sleep-past-TIMEOUT probe (exit 124, cwd = the work dir, no sleep left) and with a command that traps TERM and exits 0 (still 124); probes recorded in tools/fixtures/mutation-probe/README.txt, with the expected rows for the fresh and jsr reproducers. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
testparse: jest names come only from "●" lines under a FAIL header; real jest fixtures
Jest without a TTY prints "FAIL jt/add.test.js" with one space; the parser matched two, so a real jest load failure read ERROR, not UNVIABLE. The old `^ +● ` denylist also named process.exit's "● process.exit called with "1"" as a killing test (CAUGHT). Names now need a FAIL file header above them; a crash with no named failing test is ERROR. The "no test counts recognised" line names jest. The synthetic jest logs and mock are replaced by real jest 29.7.0 output captured on Yggdrasil from tools/fixtures/mutation-probe/jsr (green, wrong, load, exit runs in logs/jest-real.log). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
mutation-table: a control run of the unmutated snapshot gates the fan-out (SNAPSHOT RED)
A test that cannot run twice (create_dir("target/fresh-marker").unwrap())
passes the baseline and then fails in every mutant's container, so every
mutant read CAUGHT and the job exited GREEN (Soul F1). The baseline's test
loop is now one function, `unmutated`, run again from the committed snapshot
in a --control container before any mutant; red or empty there aborts with
SNAPSHOT RED, exit 2, no rows counted. Mock case E3 (oncetest.sh) fails if
the control is removed. Reproducer crate: tools/fixtures/mutation-probe/fresh.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Fixtures: Soul's fresh (rerun-hostile test) and jsr (jest/vitest mutants) reproducers
Copied from the soul3 scratch repos so the kills that found F1, F3 and F4 are committed evidence; no behavior change. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Fixtures and docs: real nonce log from the soul2 reproducer, INDEX describes the snapshot isolation
mutation-table-nonce.log is now a real run of the snapshot runner (untested-equiv SURVIVED, exit 1). The probe README names the soul2 case. ygg-verify.sh no longer says a reused log path is guarded by a run id (fresh paths are the rule). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
mutation-table: Docker owns the per-mutant snapshot; classify jest and vitest file-level load failures
The git-status restore could not see ignored files, so a test keeping state under target/ polluted the next mutant (an uncovered mutant read CAUGHT). The baselines now run once in one container whose filesystem holds the revision; that container is `docker commit`ted to a throwaway image and each mutant runs in a fresh --rm container from it. restore(), dirtied() and tree_state() are deleted. ygg-verify.sh gains RUN_ON_HOST=1 (the driver owns container topology; the watchdog kills by job label). testparse.sh: LOADFAIL for file-level FAIL lines (no " > "), jest "●" names and jest Tests: counts. Mock docker, jest and vitest-variant fixtures, soul2 reproducer. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Merge parser and wait fixes into the tool-calling branch
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mutation-table: name the mutated file from the workdir; briefs: wait calls take a 280000 ms Bash timeout
The pollution note listed nothing when the workdir was below the repo root (ls-files ran from the root); the mock now runs from a subdirectory so that regresses visibly. Every brief/INDEX line showing ygg-verify wait now says to pass a Bash timeout of at least 280000 ms. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Fixtures: Soul's real-toolchain mutation probes (rust data.txt pollution, vitest import break)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
mutation-table: BUILD baseline, field-based outcomes, whole-tree restore, file-level load failures, live rows, CRLF, nonce tag
A: BUILD runs on the unmutated tree; red or missing aborts as BASELINE RED. B: outcome is its own field, tallied from fields; labels may contain pipes. C: after each mutant the whole tree is reset and run-created untracked entries removed; an unrestorable tree aborts; dirtying is noted in the row. D: a vitest file-level FAIL with no test name is UNVIABLE, not CAUGHT. E: one row per mutant as it finishes; whole-job TIMEOUT is explicit. F: LF before-text matches CRLF files. Rows carry the per-run YGG_NONCE tag. Local mock fixtures and test-tools cases fail on each regression. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
ygg-verify: per-run nonce authenticates carried lines, run id guards wait on a reused path
Carried summary lines must start YGG-SUMMARY/<nonce>:, the nonce exported into the job; others are counted and ignored. Carried lines no longer drop failure sections. Red summaries say NAMES MISSING when counts exceed names. wait trusts a .done only when its run id matches .run; a missing summary never yields a bare exit 1. mutation-table.sh must print the nonce prefix (separate branch). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
testparse: zero-test dotnet/pytest runs, -q pytest counts, dotnet and pytest names with spaces
Soul fixtures (real dotnet, pytest, vitest, cargo logs) committed as evidence. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Briefs and skill: summary/wait verify, call budgets, range dump, Forgejo contributions
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Tools: ygg-verify summary and wait modes, mutation-table runner, shared test parser
SUMMARY=<log> keeps the full log and prints a bounded verdict summary; wait <log> blocks at most 270 s and returns it once. mutation-table.sh runs hand mutants in one verify job and returns a kill table. testparse.sh is the one parser for both. Fixtures: real cargo and mutation logs from Yggdrasil (tiny-crate), plus hand-written dotnet, vitest, pytest, exit-0-but-failed and no-verdict logs. Every commit here builds; verified live on Yggdrasil. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>