GameCult/Eureka
Watch
9
0
Fork
You've already forked Eureka
 0
Code Issues Pull requests 3 Projects Releases Packages Wiki Activity Actions

Tool-calling cut §5: summary/wait verify, snapshot mutation runner #1

Merged
eureka-admin merged 17 commits from hands/tool-calling-s5 into main 2026-10-01 16:39:49 +00:00
Conversation 0 Commits 17 Files changed 82 +1935 -35
eureka-admin commented 2026-10-01 16:39:48 +00:00
Owner
Copy link

Finishes the tool-calling map §5 text changes and adds the shared tools: ygg-verify.sh summary and wait modes, and tools/mutation-table.sh, which runs each hand mutant in a fresh container from a committed baseline snapshot (operator ruling: option B).

Four Soul passes; the last (at 838b859) cleared merge. Follow-ups are recorded in references/tool-calling-cut.md §11.

🤖 Generated with Claude Code

Finishes the tool-calling map §5 text changes and adds the shared tools: `ygg-verify.sh` summary and wait modes, and `tools/mutation-table.sh`, which runs each hand mutant in a fresh container from a committed baseline snapshot (operator ruling: option B). Four Soul passes; the last (at `838b859`) cleared merge. Follow-ups are recorded in `references/tool-calling-cut.md` §11. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
SUMMARY=<log> keeps the full log and prints a bounded verdict summary;
wait <log> blocks at most 270 s and returns it once. mutation-table.sh runs
hand mutants in one verify job and returns a kill table. testparse.sh is the
one parser for both. Fixtures: real cargo and mutation logs from Yggdrasil
(tiny-crate), plus hand-written dotnet, vitest, pytest, exit-0-but-failed and
no-verdict logs. Every commit here builds; verified live on Yggdrasil.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Soul fixtures (real dotnet, pytest, vitest, cargo logs) committed as evidence.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Carried summary lines must start YGG-SUMMARY/<nonce>:, the nonce exported into the
job; others are counted and ignored. Carried lines no longer drop failure sections.
Red summaries say NAMES MISSING when counts exceed names. wait trusts a .done only
when its run id matches .run; a missing summary never yields a bare exit 1.
mutation-table.sh must print the nonce prefix (separate branch).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A: BUILD runs on the unmutated tree; red or missing aborts as BASELINE RED.
B: outcome is its own field, tallied from fields; labels may contain pipes.
C: after each mutant the whole tree is reset and run-created untracked
entries removed; an unrestorable tree aborts; dirtying is noted in the row.
D: a vitest file-level FAIL with no test name is UNVIABLE, not CAUGHT.
E: one row per mutant as it finishes; whole-job TIMEOUT is explicit.
F: LF before-text matches CRLF files.
Rows carry the per-run YGG_NONCE tag. Local mock fixtures and test-tools
cases fail on each regression.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The pollution note listed nothing when the workdir was below the repo root
(ls-files ran from the root); the mock now runs from a subdirectory so that
regresses visibly. Every brief/INDEX line showing ygg-verify wait now says to
pass a Bash timeout of at least 280000 ms.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The git-status restore could not see ignored files, so a test keeping state under
target/ polluted the next mutant (an uncovered mutant read CAUGHT). The baselines now
run once in one container whose filesystem holds the revision; that container is
`docker commit`ted to a throwaway image and each mutant runs in a fresh --rm container
from it. restore(), dirtied() and tree_state() are deleted. ygg-verify.sh gains
RUN_ON_HOST=1 (the driver owns container topology; the watchdog kills by job label).
testparse.sh: LOADFAIL for file-level FAIL lines (no " > "), jest "●" names and jest
Tests: counts. Mock docker, jest and vitest-variant fixtures, soul2 reproducer.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
mutation-table-nonce.log is now a real run of the snapshot runner (untested-equiv
SURVIVED, exit 1). The probe README names the soul2 case. ygg-verify.sh no longer says
a reused log path is guarded by a run id (fresh paths are the rule).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Copied from the soul3 scratch repos so the kills that found F1, F3 and F4 are
committed evidence; no behavior change.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A test that cannot run twice (create_dir("target/fresh-marker").unwrap())
passes the baseline and then fails in every mutant's container, so every
mutant read CAUGHT and the job exited GREEN (Soul F1). The baseline's test
loop is now one function, `unmutated`, run again from the committed snapshot
in a --control container before any mutant; red or empty there aborts with
SNAPSHOT RED, exit 2, no rows counted. Mock case E3 (oncetest.sh) fails if
the control is removed. Reproducer crate: tools/fixtures/mutation-probe/fresh.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Jest without a TTY prints "FAIL jt/add.test.js" with one space; the parser
matched two, so a real jest load failure read ERROR, not UNVIABLE. The old
`^ +● ` denylist also named process.exit's "●  process.exit called with "1""
as a killing test (CAUGHT). Names now need a FAIL file header above them; a
crash with no named failing test is ERROR. The "no test counts recognised"
line names jest. The synthetic jest logs and mock are replaced by real jest
29.7.0 output captured on Yggdrasil from tools/fixtures/mutation-probe/jsr
(green, wrong, load, exit runs in logs/jest-real.log).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A RUN_ON_HOST job whose TIMEOUT fired still exited 0: kill_job only killed
labelled containers and the status came from the host command (Soul F2). The
host command now runs in its own process group (set -m), in the job's work
directory (it ran in the ssh home), with stdin from /dev/null. kill_job sends
the group TERM, then KILL 20 s later, and the verdict is forced to 124 when the
watchdog killed the job, whatever the command exited with. The TIMEOUT/IDLE
notice now prints before the kill so the 20 s grace cannot swallow it.
Proved on Yggdrasil with the sleep-past-TIMEOUT probe (exit 124, cwd = the work
dir, no sleep left) and with a command that traps TERM and exits 0 (still 124);
probes recorded in tools/fixtures/mutation-probe/README.txt, with the expected
rows for the fresh and jsr reproducers.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The --rm mutant container is gone by the time the trap runs, so
`docker rm -f $cur` never had a container to remove. The stopgap's watchdog
kills every job-labelled container before it signals the driver, so nothing
needs tracking here. Also restores a line continuation the control-run edit
had collapsed.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
GameCult/Eureka!1
No description provided.