ygg-verify.sh: local mode for the Yggdrasil seat, executable modes #65

Merged
eureka merged 1 commit from yv-local-mode into main 2026-11-11 16:30:03 +00:00
7 changed files with 123 additions and 19 deletions

View file

@ -2,6 +2,14 @@
Record each change to the skill together with the evidence that motivated it.
## 2026-11-11: ygg-verify.sh runs on the Yggdrasil seat
Hands, follow-up `eureka-substrate:follow_up:chat-mind-20261111T150417226Z-t35`.
Evidence. A Self seat on Yggdrasil could not launch `tools/stopgap/ygg-verify.sh` (git mode 644, and the skill mount is `noexec`) and, through `bash`, stopped at `ssh: Could not resolve hostname ygg`, although the seat is the host `ygg` names and `sudo -n docker` works there.
What changed. One code path with a transport switch: local mode (`hostname` `yggdrasil`/`yggdrasil-*`/`yggdrasil.*` with `YGG_HOST` unset, or `YGG_VERIFY_LOCAL=1`; `=0` forces ssh) runs the same remote half through local `bash -s`, with `cp` and a local `git push` in place of `scp` and the ssh push, under `$YGG_VERIFY_ROOT` (default `$HOME/eureka-verify`). The remote half takes the root as an optional 14th argument, empty over ssh, so the ssh path is unchanged. The SUMMARY re-exec and the two runners' call of `ygg-verify.sh` go through `bash`, which a `noexec` mount needs. `ygg-verify.sh`, `cargo-mutants.sh`, `mutation-table.sh`, `context-pack.py` and `test-tools.sh` are committed 100755. `tools/test-tools.sh` covers the local branch and, under a mock ssh, the remote branch and the detection rule; it also now passes the cargo-mutants cases that failed with `Permission denied` on Linux.
## 2026-11-07: Eureka tools stop reaching the mind
Hands, cut `eureka-substrate:cut_spec:cut-skill-tools-off-mcp.r2`, base `54b3cfd` (the spec named `6485496`; main moved by `dotnet-self-env.sh` only).

View file

@ -8,7 +8,7 @@ gets a Soul pass before merge.
| script | purpose | consumer | named by |
|---|---|---|---|
| `tools/stopgap/ygg-verify.sh` | Pushes an exact revision to a mirror on Yggdrasil and runs one command in a capped, niced container (`RUN_ON_HOST=1`: run a host-side command in the same slot, clone and watchdog instead; the watchdog kills every container carrying the job label). `SUMMARY=<log>` keeps the full log and prints a bounded summary (verdict, counts, ZERO TESTS RAN, failing tests); `wait <log>` blocks at most 270 s and returns the summary once; call it with a Bash tool `timeout` of at least 280000 ms. Dies with the stopgap when Idunn's verify transaction lands. | Hands, Soul, Imagination probes | `references/briefs.md` (Hands long jobs and verification, Soul long jobs), `SKILL.md` (Yggdrasil) |
| `tools/stopgap/ygg-verify.sh` | Pushes an exact revision to a mirror on Yggdrasil and runs one command in a capped, niced container (`RUN_ON_HOST=1`: run a host-side command in the same slot, clone and watchdog instead; the watchdog kills every container carrying the job label). `SUMMARY=<log>` keeps the full log and prints a bounded summary (verdict, counts, ZERO TESTS RAN, failing tests); `wait <log>` blocks at most 270 s and returns the summary once; call it with a Bash tool `timeout` of at least 280000 ms. On the Yggdrasil seat itself (`hostname` is `yggdrasil` or `yggdrasil-*`, no `YGG_HOST`; `YGG_VERIFY_LOCAL=1` forces, `=0` forces ssh) it runs local: same code, local `sudo -n docker`, state under `$YGG_VERIFY_ROOT` (default `$HOME/eureka-verify`). Dies with the stopgap when Idunn's verify transaction lands. | Hands, Soul, Imagination probes | `references/briefs.md` (Hands long jobs and verification, Soul long jobs), `SKILL.md` (Yggdrasil) |
| `tools/wait-on.sh` | One general waiter for anything that is not a Yggdrasil job: `pid <pid> [<log>]`, `log <log> <done-regex>` (with `WAIT_PID=<pid>` a dead process with no match is exit 98; an invalid ERE is exit 3), `file <path>`. The `pid` form prints `GONE (no completion evidence)`: prefer `log` whenever the job writes a done line. Same contract as `ygg-verify.sh wait`: blocks at most 270 s, exit 0 done with a tail of at most 60 log lines, `RUNNING <n>s` exit 99, exit 98 gone with no completion evidence; call it with a Bash tool `timeout` of at least 280000 ms. Replaces hand-rolled poll loops and idle `true`/`echo`/`sleep` calls. | Hands, Soul, Imagination probes | `references/briefs.md` (Hands long jobs, Soul long jobs, detached scripts) |
| `tools/mutation-table.sh` | One verify job that runs a list of hand mutants (file, before, after, test) against a revision and returns a kill table: CAUGHT with the killing test, SURVIVED, UNVIABLE, TIMEOUT, NOAPPLY, ERROR. Refuses a red or zero-test baseline and a red BUILD; rows print live. Isolation is Docker's: the baselines run once in a container that holds the revision, that container is committed to a throwaway image, and every mutant runs in a fresh `--rm` container from it, so nothing a run writes (tracked, untracked or ignored, such as `target/`) reaches the next mutant; the image is removed when the job ends, killed or not. The shared registry and sccache volumes and the network are not isolated. A file-level load failure (vitest `FAIL <file> [ <file> ]`, jest `FAIL <file>` plus `Test suite failed to run`) reads UNVIABLE. Replaces per-agent runmut/mutrun/strykrun scripts. | Soul, Hands (scoped run) | `references/briefs.md` (Hands verification, Soul reruns) |
| `tools/cargo-mutants.sh` | `SUMMARY=<log> tools/cargo-mutants.sh <repo> <rev> <base> [workdir]`: cargo-mutants on a revision range as ONE verify job, the one owner of running it. Plan container lists the range's mutants (diff taken at the workspace root); stage 1 runs each file's mutants against that file's tests only (filter derived from the path, `FILTERS` overrides), deciding only CAUGHT and UNVIABLE; stage 2 confirms every survivor, timeout, unfiltered file and zero-test file on the full suite, sharded; merge counts each name once and prints `MUTANTS` totals, `FILTER` lines and the final `MISSED`/`TIMEOUT` names. Containers = min(`SHARDS`, slots, units, 1 + free slots), leased with `flock -n` from Yggdrasil's slot pool (none while IO pressure full avg60 > 20). `NAMES=<file>` reruns chosen survivors (no stage 1), `TEST_ARGS` skips flaky tests, `JOBS`, `MIN_TEST_TIMEOUT`, `CONFIRM_TIMEOUT_MULT`, `TIMEOUT`. Exit 0 on a green baseline whatever survives; a red baseline or a failed container is non-zero with no totals. Run once per pass. | Hands, Soul | `references/briefs.md` (Hands verification, Soul reruns), `SKILL.md` (mutation bullet) |
@ -21,16 +21,20 @@ gets a Soul pass before merge.
## From a Self seat on Yggdrasil
A Mind's Self run reads these scripts from a read-only, `noexec` tmpfs mount of
the skill, and holds no Docker socket and no `ygg` ssh alias. So:
the skill, and holds no `ygg` ssh alias. So:
- `tools/<script>.sh` as a command fails `Permission denied` whatever its git
mode; `noexec` ignores the exec bit. Run a script as `bash tools/<script>.sh`;
of the shared scripts, `wait-on.sh` is the one that is useful this way.
the scripts that call each other do so through `bash` for the same reason.
- `tools/stopgap/ygg-verify.sh`, `tools/cargo-mutants.sh` and
`tools/mutation-table.sh` drive a remote `sudo docker` over `ssh ygg`
(`ygg-verify.sh` from line 280: the `ssh` calls, `sudo docker` at 321 and 410). Through `bash` they still stop at `Could not resolve
hostname ygg`. They are for the operator's workstation, or a seat that holds
Docker. Do not widen a seat to make them run.
`tools/mutation-table.sh` run on the Yggdrasil seat in local mode: the seat is
the host `ygg` names, so `ygg-verify.sh` skips ssh and uses local `sudo -n docker`
(check `sudo -n docker ps` first), `git push` to a local mirror and local files under
`$YGG_VERIFY_ROOT` (default `$HOME/eureka-verify`; a Self's HOME is a small tmpfs, so
set `YGG_VERIFY_ROOT` to a workspace path for large checkouts). Detection: `hostname`
is `yggdrasil`, `yggdrasil-*` or `yggdrasil.*` (case-blind) and `YGG_HOST` is unset;
`YGG_VERIFY_LOCAL=1` forces local and `=0` forces ssh. A seat without that sudo
still stops at `Could not resolve hostname ygg`; do not widen it to make them run.
- `cargo-mutants` need not be on PATH for those scripts: the `rust` image
installs it. For a Self, install it once into the workspace, which is
writable and executable, and run it natively on the range:

2
tools/cargo-mutants.sh Normal file → Executable file
View file

@ -113,7 +113,7 @@ if [ "${1:-}" != --host ] && [ "${1:-}" != --plan ] && [ "${1:-}" != --item ]; t
# The driver runs on the host: it unpacks the bundle into its own directory (mounted read-only into every container) and removes it when done.
cmd=$(printf 'd=$(mktemp -d) && echo %s | base64 -d | tar -xz -C "$d" && WORKDIR=%q BASE=%q FILTERS=%q TEST_ARGS=%q SHARDS=%q JOBS=%q MIN_TEST_TIMEOUT=%q CONFIRM_TIMEOUT_MULT=%q CM_NAMES=%q CM_DIR="$d" bash "$d/cargo-mutants.sh" --host' \
"$bundle" "$workdir" "$sha_base" "${FILTERS:-}" "${TEST_ARGS:-}" "$SHARDS" "$JOBS" "$MIN_TEST_TIMEOUT" "$CONFIRM_TIMEOUT_MULT" "${NAMES:+1}")
RUN_ON_HOST=1 TIMEOUT=$TIMEOUT "$here/stopgap/ygg-verify.sh" "$repo" "$sha_rev" rust "$cmd"
RUN_ON_HOST=1 TIMEOUT=$TIMEOUT bash "$here/stopgap/ygg-verify.sh" "$repo" "$sha_rev" rust "$cmd"
exit $?
fi

0
tools/context-pack.py Normal file → Executable file
View file

2
tools/mutation-table.sh Normal file → Executable file
View file

@ -101,7 +101,7 @@ if [ "${1:-}" != --host ] && [ "${1:-}" != --baseline ] && [ "${1:-}" != --mutan
# The driver runs on the host: it unpacks the bundle into its own directory (mounted read-only into every container) and removes it when done.
cmd=$(printf 'd=$(mktemp -d) && echo %s | base64 -d | tar -xz -C "$d" && WORKDIR=%q BUILD=%q TEST=%q MUT_TIMEOUT=%q JOB_TIMEOUT=%q MT_BUNDLE="$d" bash "$d/mutation-table.sh" --host' \
"$bundle" "$workdir" "${BUILD:-}" "${TEST:-}" "${MUT_TIMEOUT:-600}" "${TIMEOUT:-3600}")
RUN_ON_HOST=1 TIMEOUT=${TIMEOUT:-3600} "$here/stopgap/ygg-verify.sh" "$repo" "$rev" "$image" "$cmd"
RUN_ON_HOST=1 TIMEOUT=${TIMEOUT:-3600} bash "$here/stopgap/ygg-verify.sh" "$repo" "$rev" "$image" "$cmd"
exit $?
fi

54
tools/stopgap/ygg-verify.sh Normal file → Executable file
View file

@ -95,6 +95,15 @@
# DOCKER_ARGS='--network host'). Word-split by the remote shell, so quote a
# flag's own value inside the string if it needs one.
#
# Local mode (the Yggdrasil seat itself, which has no `ygg` alias): the same script, the same
# remote half and the same flags, with the transport switched from ssh to local bash, cp and
# git. Chosen by YGG_VERIFY_LOCAL=1 (force local) or =0 (force ssh); otherwise automatically
# when `hostname` is yggdrasil, yggdrasil-<anything> or yggdrasil.<anything> (case-blind)
# and YGG_HOST is unset. State root: $YGG_VERIFY_ROOT, default $HOME/eureka-verify (on a
# Self seat HOME is a small tmpfs: point it at a workspace path for big checkouts). Needs
# `sudo -n docker` and `flock`, as the remote half does. A Self seat's skill mount is noexec:
# launch it as `bash tools/stopgap/ygg-verify.sh ...`.
#
# Summary and wait modes (agents use these; they save the polling calls):
# SUMMARY=<log-path> ygg-verify.sh <repo> <rev> <image> '<command>'
# Runs the same job but keeps the whole output in <log-path> and prints only
@ -270,7 +279,7 @@ if [ -n "${SUMMARY:-}" ] && [ -z "${YGG_SUMMARY_INNER:-}" ]; then
exit "$1"
}
trap '[ -z "$child" ] || kill "$child" 2>/dev/null; ygg_finish 143 "launcher killed by a signal; the remote job may still be running"' TERM INT HUP
YGG_SUMMARY_INNER=1 "$0" "$@" >"$slog" 2>&1 &
YGG_SUMMARY_INNER=1 bash "$0" "$@" >"$slog" 2>&1 &
child=$!
sst=0; wait "$child" || sst=$?
ygg_finish "$sst"
@ -282,6 +291,26 @@ timeout_s=${TIMEOUT:-3600}; idle_s=${IDLE:-900}
docker_args=${DOCKER_ARGS:-}
sshopts=(-o BatchMode=yes -o ServerAliveInterval=30 -o ServerAliveCountMax=4)
# Transport: ssh to Yggdrasil, or local when this host is Yggdrasil (see "Local mode").
case "${YGG_VERIFY_LOCAL:-auto}" in
1) local_mode=1 ;;
0) local_mode=0 ;;
auto|'') local_mode=0
if [ -z "${YGG_HOST:-}" ]; then
case "$(hostname 2>/dev/null | tr '[:upper:]' '[:lower:]')" in yggdrasil|yggdrasil-*|yggdrasil.*) local_mode=1 ;; esac
fi ;;
*) echo "ygg-verify: YGG_VERIFY_LOCAL must be 0, 1 or unset" >&2; exit 2 ;;
esac
if [ "$local_mode" = 1 ]; then
vroot=${YGG_VERIFY_ROOT:-$HOME/eureka-verify}; vroot_arg=$vroot
case "$vroot" in /*) ;; *) echo "ygg-verify: YGG_VERIFY_ROOT must be an absolute path" >&2; exit 2 ;; esac
qroot=$(printf '%q' "$vroot")
else
vroot='~/eureka-verify'; vroot_arg=''; qroot=$vroot
fi
# xsh <shell command>: run it on the Yggdrasil side (ssh, or local bash).
xsh() { if [ "$local_mode" = 1 ]; then bash -c "$1"; else ssh "${sshopts[@]}" "$host" "$1"; fi; }
sha=$(git -C "$repo" rev-parse --verify "$rev^{commit}")
name=$(basename "$(git -C "$repo" rev-parse --show-toplevel)")
case "$image" in
@ -289,21 +318,23 @@ case "$image" in
dotnet) image=mcr.microsoft.com/dotnet/sdk:10.0 ;;
esac
ssh "${sshopts[@]}" "$host" "mkdir -p ~/eureka-verify/repos ~/eureka-verify/work ~/eureka-verify/images && \
{ test -d ~/eureka-verify/repos/$name.git || git init -q --bare ~/eureka-verify/repos/$name.git; }"
xsh "mkdir -p $qroot/repos $qroot/work $qroot/images && \
{ test -d $qroot/repos/$name.git || git init -q --bare $qroot/repos/$name.git; }"
# ssh:// rather than scp-style host:path: Git LFS's pre-push hook rejects the
# scp form as a remote name and stalls (Aetheria, 2026-09-30). The verify mirror
# never needs LFS objects pushed; a job that needs one smudges it itself.
GIT_LFS_SKIP_PUSH=1 git -C "$repo" push -q "ssh://$host/~/eureka-verify/repos/$name.git" "$sha:refs/verify/$sha" --force
if [ "$local_mode" = 1 ]; then pushurl=$vroot/repos/$name.git; else pushurl="ssh://$host/~/eureka-verify/repos/$name.git"; fi
GIT_LFS_SKIP_PUSH=1 git -C "$repo" push -q "$pushurl" "$sha:refs/verify/$sha" --force
case "$image" in
eureka-verify-*:*) df=${image#eureka-verify-}; df=${df%%:*}
scp -q "$here/$df.Dockerfile" "$host:eureka-verify/images/$df.Dockerfile" ;;
if [ "$local_mode" = 1 ]; then cp "$here/$df.Dockerfile" "$vroot/images/$df.Dockerfile"
else scp -q "$here/$df.Dockerfile" "$host:eureka-verify/images/$df.Dockerfile"; fi ;;
esac
# ssh joins its arguments into one string that the remote shell splits again,
# so every argument is quoted here. Without that, `a && b` in the command runs
# b on the host.
remote_args=$(printf '%q ' "$name" "$sha" "$image" "$cpus" "$mem" "$slots" "${KEEP:-0}" "$cmd" "$timeout_s" "$docker_args" "$idle_s" "${YGG_NONCE:-}" "${RUN_ON_HOST:-0}")
remote_args=$(printf '%q ' "$name" "$sha" "$image" "$cpus" "$mem" "$slots" "${KEEP:-0}" "$cmd" "$timeout_s" "$docker_args" "$idle_s" "${YGG_NONCE:-}" "${RUN_ON_HOST:-0}" "$vroot_arg")
# The heredoc below travels with whatever line endings this file has on disk.
# On Windows that is CRLF, and the remote bash then fails on `set -euo pipefail `
# BEFORE `set -e` is in force, so the script limps on and can exit 0 — a job that
@ -311,10 +342,15 @@ remote_args=$(printf '%q ' "$name" "$sha" "$image" "$cpus" "$mem" "$slots" "${KE
# far side before bash ever sees them. The sentinel below is the second guard: if
# the remote script dies early, no verdict line is printed and the local side
# refuses to report a status it did not receive.
(ssh "${sshopts[@]}" "$host" "tr -d ' ' | bash -s -- $remote_args" <<'REMOTE'
# run_half: the heredoc below on stdin, as `bash -s` on Yggdrasil (ssh) or here (local mode).
run_half() {
if [ "$local_mode" = 1 ]; then eval "bash -s -- $remote_args"
else ssh "${sshopts[@]}" "$host" "tr -d ' ' | bash -s -- $remote_args"; fi
}
(run_half <<'REMOTE'
set -euo pipefail
name=$1 sha=$2 image=$3 cpus=$4 mem=$5 slots=$6 keep=$7 cmd=$8 timeout_s=$9 docker_args=${10} idle_s=${11} nonce=${12:-} host_mode=${13:-0}
root=~/eureka-verify
name=$1 sha=$2 image=$3 cpus=$4 mem=$5 slots=$6 keep=$7 cmd=$8 timeout_s=$9 docker_args=${10} idle_s=${11} nonce=${12:-} host_mode=${13:-0} root=${14:-}
[ -n "$root" ] || root=~/eureka-verify
case "$image" in
eureka-verify-*:*)
df=${image#eureka-verify-}; df=${df%%:*}

58
tools/test-tools.sh Normal file → Executable file
View file

@ -7,7 +7,8 @@
# log, split by "=== MARK <name>" lines); the rest are hand-written.
# The mutation runners tools/mutation-table.sh and tools/cargo-mutants.sh (launcher, host driver
# and container halves) run under a mock docker and, for cargo-mutants, a fake cargo and flock
# (fixtures/mutation-mock/fakebin), with real slot locks.
# (fixtures/mutation-mock/fakebin), with real slot locks. ygg-verify.sh's local branch and (through a mock ssh)
# its remote branch run end to end over the same mock docker.
# Run: bash tools/test-tools.sh (no network, no builds)
set -uo pipefail
here=$(cd "$(dirname "$0")" && pwd)
@ -852,4 +853,59 @@ r=$(cpo "$(wp "$T/cp-spec30.json")" --max-lines 202); cchk "reserve line at the
[ "$(grep -c '^== \[' "$T/cp.out")" = 9 ] && [ "$(wc -l <"$T/cp.out" | tr -d ' ')" -le 202 ] && echo "ok context-pack a window that would crowd out the not-expanded line is skipped" || { echo "FAIL context-pack reserve at 202: $(wc -l <"$T/cp.out")"; fail=1; }
# standard library plus git only
if grep -qE '^import (requests|yaml|tree_sitter)|^from (requests|yaml|tree_sitter)|ctags|tree-sitter|"rg"' "$CP"; then echo "FAIL context-pack: a dependency beyond git"; fail=1; else echo "ok context-pack: standard library and git only"; fi
# ---- ygg-verify.sh transport switch: the local branch (this host is Yggdrasil) and, under a mock
# ssh that runs the remote command string in a shell with a fake remote HOME, the remote branch.
# Docker is the mutation-mock fake; its container is a directory and does not mount the work
# directory, so these jobs prove the transport, slots, clone, verdict and summary, not a toolchain.
YL=$T/yl; mkdir -p "$YL/bin" "$YL/rhome"
cp "$here/fixtures/mutation-mock/fakebin/"{docker,sudo,flock} "$YL/bin/"
printf '#!/usr/bin/env bash\necho "ssh $*" >>"$YL_SSHLOG"\nwhile [ "$1" = -o ]; do shift 2; done\nshift\nHOME=$YL_RHOME exec bash -c "$*"\n' >"$YL/bin/ssh"
printf '#!/usr/bin/env bash\necho "scp $*" >>"$YL_SSHLOG"\nwhile [ "$1" = -q ]; do shift; done\nsrc=$1; dst=${2#*:}; cp "$src" "$YL_RHOME/$dst"\n' >"$YL/bin/scp"
printf '#!/usr/bin/env bash\necho "${YL_HOSTNAME:-starfire}"\n' >"$YL/bin/hostname"
chmod +x "$YL/bin/"*
YLR=$YL/repo; mkdir -p "$YLR"; git -C "$YLR" init -q; echo marker >"$YLR/m.txt"
git -C "$YLR" add m.txt; git -C "$YLR" -c user.name=t -c user.email=t@t commit -qm m
ylrun() { # ylrun <extra env assignments...> -- <args>: ygg-verify under the mocks; sets yl_out yl_st
local envs=(); while [ "$1" != -- ]; do envs+=("$1"); shift; done; shift
: >"$YL/ssh.log"
yl_out=$(env PATH="$YL/bin:$PATH" FAKE_DOCKER="$YL/fd" YL_SSHLOG="$YL/ssh.log" YL_RHOME="$YL/rhome" YGG_VERIFY_ROOT="$YL/root" \
TIMEOUT=60 "${envs[@]}" bash "$V" "$@" 2>&1); yl_st=$?
}
ylchk() { # ylchk <label> <expected status> <must-contain> <ssh used: 0|1>
local used=0; [ -s "$YL/ssh.log" ] && used=1
if [ "$yl_st" != "$2" ]; then echo "FAIL $1: exit $yl_st, wanted $2: $yl_out"; fail=1
elif ! printf '%s\n' "$yl_out" | grep -qF -- "$3"; then echo "FAIL $1: missing: $3 in: $yl_out"; fail=1
elif [ "$used" != "$4" ]; then echo "FAIL $1: ssh used=$used, wanted $4: $(cat "$YL/ssh.log")"; fail=1
else echo "ok $1"; fi
}
ylrun YGG_VERIFY_LOCAL=1 -- "$YLR" HEAD base 'echo localjob-ran; exit 3'
ylchk "ygg-verify local: the job's status and output come back, no ssh" 3 "localjob-ran" 0
[ -d "$YL/root/repos/repo.git" ] && [ -z "$(ls "$YL/root/work" | grep -v '\.killed$')" ] && echo "ok ygg-verify local: mirror under YGG_VERIFY_ROOT, work directory removed" || { echo "FAIL ygg-verify local: mirror or work dir state: $(ls -R "$YL/root" | head)"; fail=1; }
ylrun YGG_VERIFY_LOCAL=1 RUN_ON_HOST=1 -- "$YLR" HEAD base 'cat m.txt; echo "root=$YGG_ROOT slots=$YGG_SLOTS"; test -e "$YGG_WORK/m.txt"'
ylchk "ygg-verify local: RUN_ON_HOST runs in the checked-out revision with its environment" 0 "root=$YL/root slots=5" 0
ylrun YGG_VERIFY_LOCAL=1 SUMMARY="$YL/s.log" -- "$YLR" HEAD base 'echo "test result: ok. 4 passed; 0 failed; 0 ignored"'
ylchk "ygg-verify local: SUMMARY prints the bounded summary" 0 "FRAMEWORK cargo passed=4 failed=0" 0
out=$(WAIT_MAX=20 SETTLE=0 bash "$V" wait "$YL/s.log"); st=$?
[ "$st" = 0 ] && printf '%s\n' "$out" | grep -qF "FRAMEWORK cargo passed=4" && echo "ok ygg-verify local: wait returns the finished job" || { echo "FAIL ygg-verify local wait: $st $out"; fail=1; }
# all slots held by other jobs: the job waits, then takes the slot that frees (real flock on the slot files)
( exec 8>"$YL/root/slot-1.lock"; flock -n 8; sleep 7 ) & sb=$!
sleep 1
ylrun YGG_VERIFY_LOCAL=1 SLOTS=1 -- "$YLR" HEAD base 'echo slotted'
wait "$sb"
ylchk "ygg-verify local: waits for a busy slot, then runs" 0 "all 1 slots busy; waiting" 0
# detection: hostname yggdrasil-candidate is local without the env; another host, or YGG_VERIFY_LOCAL=0, goes over ssh
ylrun YL_HOSTNAME=yggdrasil-candidate -- "$YLR" HEAD base 'echo auto-local'
ylchk "ygg-verify detection: hostname yggdrasil-candidate runs locally" 0 "auto-local" 0
ylrun YL_HOSTNAME=Yggdrasil -- "$YLR" HEAD base 'echo auto-local2'
ylchk "ygg-verify detection: hostname Yggdrasil runs locally" 0 "auto-local2" 0
ylrun YL_HOSTNAME=yggdrasil-candidate YGG_VERIFY_LOCAL=0 -- "$YLR" HEAD base 'echo forced-remote; exit 5'
ylchk "ygg-verify detection: YGG_VERIFY_LOCAL=0 forces ssh (mock ssh: remote command string in a fake HOME)" 5 "forced-remote" 1
ylrun YL_HOSTNAME=starfire -- "$YLR" HEAD base 'echo workstation'
ylchk "ygg-verify detection: another hostname uses ssh" 0 "workstation" 1
[ -d "$YL/rhome/eureka-verify/repos/repo.git" ] && echo "ok ygg-verify remote (mock ssh): mirror pushed to the remote ~/eureka-verify" || { echo "FAIL ygg-verify remote mirror missing: $(cat "$YL/ssh.log")"; fail=1; }
ylrun YL_HOSTNAME=yggdrasil-candidate YGG_HOST=ygg -- "$YLR" HEAD base 'echo explicit-host'
ylchk "ygg-verify detection: an explicit YGG_HOST keeps the ssh path" 0 "explicit-host" 1
ylrun YGG_VERIFY_LOCAL=maybe -- "$YLR" HEAD base 'true'
ylchk "ygg-verify detection: a bad YGG_VERIFY_LOCAL value is refused" 2 "must be 0, 1 or unset" 0
exit $fail