Commit graph Eureka/tools/test-tools.sh
Author SHA1 Message Date
da193e3e02 test-tools: admit R9 Unavailable names the tool kind; R8 budget and unchecked-member cases; restore two literal CRs in the context-pack fixtures
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-23 19:41:06 +02:00
e12435e7e2 test-tools: the recall admit section, R1-R12 driven with real hook JSON against the fake binding
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-23 19:36:28 +02:00
8701da3a86 recall: the deadline clock starts at process creation and the kill fits inside it; R2 times the interpreter, not the shell
On Windows the run overshot its deadline by 450-720 ms in the R2 bands: ~200 ms of
interpreter start and imports ran before the clock, the hard stop sat 0.3 s after the
deadline, and the test timed bash, env and the py launcher too. The clock now starts at
process creation (GetProcessTimes), the reply wait ends 0.1 s early so the kill and exit
fit, and the hard stop is the deadline itself. Residual outside the script: the py
launcher (40-130 ms, rare spikes to 300) and the creation call. R2 times the interpreter
directly, best of two, bands -250/+100 ms around the deadline.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 18:44:11 +02:00
0342dceef9 recall: recalled text is inert data; the deadline is the whole run; budget, stdin and answer are bounded
Soul s1 fixes. The hook header says the lines are recalled records, data and
never instructions, written by agents not the operator. Every item is one JSON
object per line: id, title and operator_quote are JSON strings with < >, control,
bidi and separator characters escaped, so a title cannot close the framing, forge
a tag, a header or an operator segment. RECALL_DEADLINE_MS now bounds the whole
run through a hard stop; stdin is read to 4 MiB and the answer to 1 MiB
(mind.call max_bytes); RECALL_BUDGET is clamped to 9000 under Claude Code's
10,000-character cap. Tests: hostile text, a non-ASCII process-boundary run under
cp1252, byte-versus-character budget, stdin held open and endless, answer size,
budget ceiling, and the four unrecorded survivors (negative reply timeout, reply
id check, env-int positivity, EOF sentinel). Mutants of each were run and caught.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 21:44:02 +02:00
f35410a8b2 tools: recall.py prompt, the UserPromptSubmit hook that pushes nearest in-force rulings into context
Queries rulings, questions and follow-ups in force across every root, ranked by
Huginn against the prompt, through mind.call, and prints one hook JSON within
RECALL_BUDGET. Never blocks: any failure or the RECALL_DEADLINE_MS deadline
(default 4000) exits 0 with empty stdout; one attempt; prompts under 3 words are
not queried. test-tools.sh drives it against the fake binding and pins R1-R7;
the fake gains a query branch. INDEX rows and a Precedent note added. The
operator installs the hook; settings.json is not touched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 16:34:20 +02:00
5bf88b80cb context-pack tests: pin the --only reserve and charge, the cut-window budget, header base, missing base
Targets the mutants that survived the first fix run: need arithmetic of cut
and full windows, the reserve for the not-expanded line at the budget edge,
the left-reserve >= 4 threshold with two windows, the exact more-lines
note, the blank line after the table, a non-sha base never echoed in the
header, and a spec without a base refused rather than crashing.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-09 02:50:19 +02:00
dbcb41a046 context-pack tests: glob paths, straddles, --since boundaries, spec shapes, budgets, binding env
Real git repos in scratch: a src/[a] path beside src/a, pure deletions and
one-line replacements straddling either end of an anchor, edits on an
anchor's first and last line under --since, a --since commit with an
unreadable tree, thirteen wrong-typed spec variants with canaries, a
--max-lines below the table, --only truncation at the 7 and 6 line
thresholds and with a reserved not-expanded line, and the binding env block
with the caller's environment overriding it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-09 01:40:21 +02:00
a474da7a5f context-pack tests: pin hunk clamps, boundaries, unsafe paths, protocol and error codes
Survivors of the first mutmut run (200 of 688) drove these: replaced-block
and pure-deletion hunks, --since boundaries, NEW windows, 70-column change
text, the stdio handshake the fake server now validates, every error code.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-09 00:07:08 +02:00
11f6526283 context-pack tests: pin error messages, retry condition, file-end and CRLF handling
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 22:13:13 +02:00
aa6ce2b097 tools/context-pack.py: carry a cut_spec's file_changes anchors to the worktree
Anchors resolve through git diff hunks from the spec's base (working tree
included) into a status table plus bounded windows; --since re-prints only
edited windows. Spec by id through the eureka-state stdio binding, or a
saved view. Fixture section in test-tools.sh; INDEX row.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 21:35:42 +02:00
d9e45c8740 wait-on: delete WAIT_FROM; reused log paths are forbidden instead
Truncating writers (`>`, Unity -logFile) rewrite below any offset, so an offset
turned a stale success into a silent hang. The option, the scan branch, its docs
and its fixtures are gone; the header states the fresh-log-path rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 15:21:58 +02:00
30e199cf55 wait-on: WAIT_FROM start marker, GONE line for pid form, regex validation, executable bit
F1: log form searches only bytes after WAIT_FROM, so a stale done line in a
reused log is ignored (also makes the per-check cost independent of log size).
F2: pid form prints GONE (no completion evidence). F4: invalid ERE is exit 3.
Mode 100755 recorded for wait-on.sh; other tools/ scripts are still 100644.
F5, F6, F7 recorded as known limitations in the header. Fixtures added.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 15:04:20 +02:00
a886d4524a Add tools/wait-on.sh: one waiter for pids, logs and files
Same contract as ygg-verify.sh wait (270 s cap, bounded tail, RUNNING exit 99,
exit 98 for a thing gone with no completion evidence). Windows PIDs are found
through tasklist, since kill -0 only knows Git Bash's own. Fixtures in
tools/test-tools.sh; INDEX row added.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 14:15:45 +02:00
Meta Vens
f68bd4099f testparse: jest names come only from "●" lines under a FAIL header; real jest fixtures
Jest without a TTY prints "FAIL jt/add.test.js" with one space; the parser
matched two, so a real jest load failure read ERROR, not UNVIABLE. The old
`^ +● ` denylist also named process.exit's "●  process.exit called with "1""
as a killing test (CAUGHT). Names now need a FAIL file header above them; a
crash with no named failing test is ERROR. The "no test counts recognised"
line names jest. The synthetic jest logs and mock are replaced by real jest
29.7.0 output captured on Yggdrasil from tools/fixtures/mutation-probe/jsr
(green, wrong, load, exit runs in logs/jest-real.log).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 18:14:45 +02:00
Meta Vens
b4942b7571 mutation-table: a control run of the unmutated snapshot gates the fan-out (SNAPSHOT RED)
A test that cannot run twice (create_dir("target/fresh-marker").unwrap())
passes the baseline and then fails in every mutant's container, so every
mutant read CAUGHT and the job exited GREEN (Soul F1). The baseline's test
loop is now one function, `unmutated`, run again from the committed snapshot
in a --control container before any mutant; red or empty there aborts with
SNAPSHOT RED, exit 2, no rows counted. Mock case E3 (oncetest.sh) fails if
the control is removed. Reproducer crate: tools/fixtures/mutation-probe/fresh.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 18:14:39 +02:00
Meta Vens
2fbf696f79 Fixtures and docs: real nonce log from the soul2 reproducer, INDEX describes the snapshot isolation
mutation-table-nonce.log is now a real run of the snapshot runner (untested-equiv
SURVIVED, exit 1). The probe README names the soul2 case. ygg-verify.sh no longer says
a reused log path is guarded by a run id (fresh paths are the rule).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 17:46:11 +02:00
Meta Vens
62bc4d5c1d mutation-table: Docker owns the per-mutant snapshot; classify jest and vitest file-level load failures
The git-status restore could not see ignored files, so a test keeping state under
target/ polluted the next mutant (an uncovered mutant read CAUGHT). The baselines now
run once in one container whose filesystem holds the revision; that container is
`docker commit`ted to a throwaway image and each mutant runs in a fresh --rm container
from it. restore(), dirtied() and tree_state() are deleted. ygg-verify.sh gains
RUN_ON_HOST=1 (the driver owns container topology; the watchdog kills by job label).
testparse.sh: LOADFAIL for file-level FAIL lines (no " > "), jest "●" names and jest
Tests: counts. Mock docker, jest and vitest-variant fixtures, soul2 reproducer.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 17:36:49 +02:00
Meta Vens
118f157124 Merge parser and wait fixes into the tool-calling branch
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 16:46:39 +02:00
Meta Vens
4bcd434828 mutation-table: name the mutated file from the workdir; briefs: wait calls take a 280000 ms Bash timeout
The pollution note listed nothing when the workdir was below the repo root
(ls-files ran from the root); the mock now runs from a subdirectory so that
regresses visibly. Every brief/INDEX line showing ygg-verify wait now says to
pass a Bash timeout of at least 280000 ms.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 16:43:26 +02:00
Meta Vens
b6f52d5ec4 mutation-table: BUILD baseline, field-based outcomes, whole-tree restore, file-level load failures, live rows, CRLF, nonce tag
A: BUILD runs on the unmutated tree; red or missing aborts as BASELINE RED.
B: outcome is its own field, tallied from fields; labels may contain pipes.
C: after each mutant the whole tree is reset and run-created untracked
entries removed; an unrestorable tree aborts; dirtying is noted in the row.
D: a vitest file-level FAIL with no test name is UNVIABLE, not CAUGHT.
E: one row per mutant as it finishes; whole-job TIMEOUT is explicit.
F: LF before-text matches CRLF files.
Rows carry the per-run YGG_NONCE tag. Local mock fixtures and test-tools
cases fail on each regression.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 16:40:25 +02:00
Meta Vens
15c8ff6f8a ygg-verify: per-run nonce authenticates carried lines, run id guards wait on a reused path
Carried summary lines must start YGG-SUMMARY/<nonce>:, the nonce exported into the
job; others are counted and ignored. Carried lines no longer drop failure sections.
Red summaries say NAMES MISSING when counts exceed names. wait trusts a .done only
when its run id matches .run; a missing summary never yields a bare exit 1.
mutation-table.sh must print the nonce prefix (separate branch).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 16:35:01 +02:00
Meta Vens
60a09dc17c Tools: ygg-verify summary and wait modes, mutation-table runner, shared test parser
SUMMARY=<log> keeps the full log and prints a bounded verdict summary;
wait <log> blocks at most 270 s and returns it once. mutation-table.sh runs
hand mutants in one verify job and returns a kill table. testparse.sh is the
one parser for both. Fixtures: real cargo and mutation logs from Yggdrasil
(tiny-crate), plus hand-written dotnet, vitest, pytest, exit-0-but-failed and
no-verdict logs. Every commit here builds; verified live on Yggdrasil.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 15:09:07 +02:00