Eureka/tools/INDEX.md

11 KiB

Shared tools

One line per script. Briefs name the exact script they want; agents do not browse. A line whose consuming brief no longer names the script is dead and is deleted. Contributions come by branch and Forgejo pull request (this repo lives on Yggdrasil's Forgejo), never a push to main. A script that verdicts depend on gets a Soul pass before merge.

script purpose consumer named by
tools/stopgap/ygg-verify.sh Pushes an exact revision to a mirror on Yggdrasil and runs one command in a capped, niced container (RUN_ON_HOST=1: run a host-side command in the same slot, clone and watchdog instead; the watchdog kills every container carrying the job label). SUMMARY=<log> keeps the full log and prints a bounded summary (verdict, counts, ZERO TESTS RAN, failing tests); wait <log> blocks at most 270 s and returns the summary once; call it with a Bash tool timeout of at least 280000 ms. On the Yggdrasil seat itself (hostname is yggdrasil or yggdrasil-*, no YGG_HOST; YGG_VERIFY_LOCAL=1 forces, =0 forces ssh) it runs local: same code, local sudo -n docker, state under $YGG_VERIFY_ROOT (default $HOME/eureka-verify). Dies with the stopgap when Idunn's verify transaction lands. Hands, Soul, Imagination probes references/briefs.md (Hands long jobs and verification, Soul long jobs), SKILL.md (Yggdrasil)
tools/wait-on.sh One general waiter for anything that is not a Yggdrasil job: pid <pid> [<log>], log <log> <done-regex> (with WAIT_PID=<pid> a dead process with no match is exit 98; an invalid ERE is exit 3), file <path>. The pid form prints GONE (no completion evidence): prefer log whenever the job writes a done line. Same contract as ygg-verify.sh wait: blocks at most 270 s, exit 0 done with a tail of at most 60 log lines, RUNNING <n>s exit 99, exit 98 gone with no completion evidence; call it with a Bash tool timeout of at least 280000 ms. Replaces hand-rolled poll loops and idle true/echo/sleep calls. Hands, Soul, Imagination probes references/briefs.md (Hands long jobs, Soul long jobs, detached scripts)
tools/mutation-table.sh One verify job that runs a list of hand mutants (file, before, after, test) against a revision and returns a kill table: CAUGHT with the killing test, SURVIVED, UNVIABLE, TIMEOUT, NOAPPLY, ERROR. Refuses a red or zero-test baseline and a red BUILD; rows print live. Isolation is Docker's: the baselines run once in a container that holds the revision, that container is committed to a throwaway image, and every mutant runs in a fresh --rm container from it, so nothing a run writes (tracked, untracked or ignored, such as target/) reaches the next mutant; the image is removed when the job ends, killed or not. The shared registry and sccache volumes and the network are not isolated. A file-level load failure (vitest FAIL <file> [ <file> ], jest FAIL <file> plus Test suite failed to run) reads UNVIABLE. Replaces per-agent runmut/mutrun/strykrun scripts. Soul, Hands (scoped run) references/briefs.md (Hands verification, Soul reruns)
tools/cargo-mutants.sh SUMMARY=<log> tools/cargo-mutants.sh <repo> <rev> <base> [workdir]: cargo-mutants on a revision range as ONE verify job, the one owner of running it. Plan container lists the range's mutants (diff taken at the workspace root); stage 1 runs each file's mutants against that file's tests only (filter derived from the path, FILTERS overrides), deciding only CAUGHT and UNVIABLE; stage 2 confirms every survivor, timeout, unfiltered file and zero-test file on the full suite, sharded; merge counts each name once and prints MUTANTS totals, FILTER lines and the final MISSED/TIMEOUT names. Containers = min(SHARDS, slots, units, 1 + free slots), leased with flock -n from Yggdrasil's slot pool (none while IO pressure full avg60 > 20). NAMES=<file> reruns chosen survivors (no stage 1), TEST_ARGS skips flaky tests, JOBS, MIN_TEST_TIMEOUT, CONFIRM_TIMEOUT_MULT, TIMEOUT. Exit 0 on a green baseline whatever survives; a red baseline or a failed container is non-zero with no totals. Run once per pass. Hands, Soul references/briefs.md (Hands verification, Soul reruns), SKILL.md (mutation bullet)
tools/dotnet-self tools/dotnet-self <dotnet args>: runs one dotnet command for a Self whose HOME is read-only, with HOME, DOTNET_CLI_HOME and NUGET_PACKAGES set for that process only (default ${XDG_CACHE_HOME:-${TMPDIR:-/tmp}}/dotnet-self-home-<uid>, 0700, DOTNET_SELF_HOME overrides) and telemetry, first-run, MSBuild node reuse, shared compilation and the build server off. Env defined in tools/dotnet-self-env.sh, which is sourced only inside a subshell: sourcing it in your shell exports HOME and moves git and gh credentials. Selves and their workers running dotnet SKILL.md (Git and tooling rules)
tools/testparse.sh Sourced library: failing test names and pass/fail/skip counts from cargo, dotnet, vitest, jest and pytest logs, and test files that failed to load (LOADFAIL). Shared by the summary and the kill table. tools/stopgap/ygg-verify.sh, tools/mutation-table.sh not named by a brief; it lives and dies with them
tools/test-tools.sh Local fixture check of the parsers and the summary over tools/fixtures/logs (real cargo and mutation logs, plus hand-written dotnet, vitest, jest, pytest, exit-0-but-failed and no-verdict logs), and of the mutation runners' host drivers and container halves (tools/mutation-table.sh, tools/cargo-mutants.sh) under a mock docker, a fake cargo mutants and real slot locks. Run it before merging a change to the scripts above. whoever changes those scripts SKILL.md (Shared tools)
tools/context-pack.py py -3 tools/context-pack.py <spec file or -> <worktree> [--since <sha>] [--only <k>|r<k>[,...]] [--max-lines <n>]. Carries a cut_spec's file_changes and reads anchors (read rows r<k>; a read absent at base is MISSING) from the spec's base to the worktree by git diff hunks (the working tree included) and prints one header, a status table (SAME, MOVED, TOUCHED, RENAMED, NEW, GONE, OUT_OF_RANGE, UNRESOLVED, MISSING, plus SYMBOL_MISSING when an anchor's symbol is absent from its window) and windows of at most 100 lines numbered in worktree lines, then one capped caller list per symbol (git grep, hits outside the windows), within --max-lines (800). Exit 3 when an anchor cannot be carried, 4 when the spec is unreadable. The spec is the document huginn view --raw <cut_spec id> prints, saved to a file or piped to -; context-pack never fetches it, and an Unavailable from huginn is the caller's one retry. --since <sha> re-prints only the windows edited since that commit. Standard library plus git; writes nothing. Hands, Soul, Imagination references/briefs.md (Imagination map, Hands, Soul), SKILL.md (step 1)
tools/bloom-sheet/bloom_sheet.py bloom-sheet notes.md -o sheet.png [--visitor a.svg]...: a markdown document as one neon A4 page. The first top-level heading is the title, every other heading a card sized by its text; a plant grows through what's left, and each SVG visitor gets a disc it grows around and is drawn with the plant's pen, so it must be flat strokes or fills that touch; large fills become a wash and a heavy outline. --theme NAME swaps the page file for themes/NAME.tex (calligraphic: ink on parchment); byte-equal rebuilds need SOURCE_DATE_EPOCH=1 FORCE_SOURCE_DATE=1, and a PDF's /ID differs every run regardless. PDF or PNG, about 2.5 minutes. grow.py owns the plant, page.tex the look, test_visitors.py what a visitor may be, test_theme.py what a theme must be. Minds and Self, for info renders the bloom-sheet skill; bloom-sheet on Yggdrasil's PATH

From a Self seat on Yggdrasil

A Mind's Self run reads these scripts from a read-only, noexec tmpfs mount of the skill, and holds no ygg ssh alias. So:

  • tools/<script>.sh as a command fails Permission denied whatever its git mode; noexec ignores the exec bit. Run a script as bash tools/<script>.sh; the scripts that call each other do so through bash for the same reason.

  • tools/stopgap/ygg-verify.sh, tools/cargo-mutants.sh and tools/mutation-table.sh run on the Yggdrasil seat in local mode: the seat is the host ygg names, so ygg-verify.sh skips ssh and uses local sudo -n docker (check sudo -n docker ps first), git push to a local mirror and local files under $YGG_VERIFY_ROOT (default $HOME/eureka-verify; a Self's HOME is a small tmpfs, so set YGG_VERIFY_ROOT to a workspace path for large checkouts). Detection: hostname is yggdrasil, yggdrasil-* or yggdrasil.* (case-blind) and YGG_HOST is unset; YGG_VERIFY_LOCAL=1 forces local and =0 forces ssh. A seat without that sudo still stops at Could not resolve hostname ygg; do not widen it to make them run.

  • cargo-mutants need not be on PATH for those scripts: the rust image installs it. For a Self, install it once into the workspace, which is writable and executable, and run it natively on the range:

    cargo install --locked cargo-mutants --root "$PWD/tools-bin"   # 27.1.0 built clean
    PATH="$PWD/tools-bin/bin:$PATH"
    git -C <repo> diff <base> <rev> > "$PWD/range.diff"      # absolute: the next line changes directory
    cd <repo>/<workspace-dir> && cargo mutants --in-diff "$OLDPWD/range.diff" --list   # the mutants
    cargo mutants --in-diff "$OLDPWD/range.diff" -j1 -- <test filter>                  # run them
    

    The diff is taken at the workspace root and --in-diff paths match relative to it (see tools/cargo-mutants.sh header). A native run is a plain cargo mutants: no per-file filter stage, no confirm stage, no slots, no CPU or memory cap (set CARGO_BUILD_JOBS, and -j1 unless --in-place; point CARGO_TARGET_DIR and TMPDIR at a disk with room, since cargo-mutants copies the tree to TMPDIR and a small run took 2.3 GB), and its survivors are not confirmed on the full suite. Run survivors again unfiltered before reporting them MISSED.

    Learned on CultLib's Rust runtime (Libby, 38 mutants: 32 caught, 5 unviable, 1 missed, 7 min):

    • Add --in-place when tests read files by the repo's original path (CultLib's vector tests): the copy-mode baseline fails (15 failures). Replace -j1 in the run command above with --in-place: cargo-mutants rejects the two together, so in-place runs are serial.
    • Offline resolution needs the lockfile's exact crate versions in the cargo registry (one checkout wanted cc 1.11.0, which was missing); fetch it first or the baseline fails before any mutant runs.
    • Launch as the harness's background run. A nohup ... & from the shell died silently.