diff --git a/SKILL.md b/SKILL.md index 30cf124..6a83e86 100644 --- a/SKILL.md +++ b/SKILL.md @@ -324,6 +324,15 @@ cut.** In the CultCache migration, Cuts 6 and 6b each needed four Soul passes, a the later passes still found real defects. Stop when a pass finds nothing that blocks, or only findings the operator chooses to record. +**Quota is spent through scope, not through the number of passes.** On +2026-09-23 the operator flagged that 60% of the weekly quota had gone in one +day with little of it on the StreamPixels critical path. After that, Self ran +one full Soul gate per cut. A fix batch went only to findings on the critical +path, and the rest were recorded as follow-ups. Every later Soul pass covers +only the fix batch's diff, as the second-pass clause in `briefs.md` says, and a +small delta gets a narrow Sonnet pass. This does not cap the loop on a +foundation cut. It keeps each pass the right size. + When the same kind of finding recurs, the brief was missing context. Fix the brief template, not only the instance. That is how "operator rulings each need their own failing mutation" entered the Hands brief. diff --git a/references/briefs.md b/references/briefs.md index 003404a..0a87155 100644 --- a/references/briefs.md +++ b/references/briefs.md @@ -115,6 +115,23 @@ reading what it prints: that surface is the one a test suite cannot check. A test's inputs come from the production path the rule is about. A helper that re-spells what production prints, parses or derives makes the test agree with a copy, and the rule then breaks in production with the suite green. +A green run is not evidence until it shows a non-zero count for the tests you +meant to run. Paste the count for each named file. On StreamPixels, all three +of these produced a green that had run nothing, or the wrong thing: +- A `node --test` glob that matches no files exits 0. +- A package script's `--dir .` made vitest discover zero tests (`cc71ce2`). +- `pnpm --filter X test -- ` ran the whole suite instead of the named + files (`4624b53`). +Set environment variables only on the step that needs them. Exporting +`DATABASE_URL` for the whole job silently moves `app.test.ts` onto the +Postgres driver (`e019de0`). +"Pre-existing" is not a diagnosis. A failure that is not yours still gets a +one-line cause: what it depends on, and whether it has ever passed on a clean +checkout. Several agents waved a StreamPixels test through as pre-existing. It +read a git-ignored, licensed asset pack and had never passed on a clean +checkout (`27053f9`). Reproducing a failure at the base commit proves nothing +when both runs share the cause, for example a Windows file lock on the same +machine. A hand mutation hits the rule at the layer it protects: the production call site, observed where the consequence lands (disk, the kernel, the request sent). Mutating a helper or an in-memory mirror proves the helper, not the @@ -142,6 +159,13 @@ one worker's head, and the brief says how: - **Carry the cut inline.** Quote the cut's own section in the brief instead of pointing at a long map; a worker that reads 900 lines to find 60 has spent its budget before it starts. +- **One fix batch per file.** A batch that packs many findings, a + subtraction, two hosts and a late addition does not fit in one head. On + StreamPixels on 2026-09-23, one batch of about ten items ran to about 800k + tokens over 2.6 hours, and another passed 550k and produced sloppy code. + After the findings were split per file, batches stayed at about 200-400k. A + late item gets its own brief. Do not add it to a batch that is already + planned. - **Keep expensive scaffolding alive across cuts** (a pinned dependency worktree, a warm build) rather than creating and removing it per pass. - **Hand back rather than push through.** A worker that finds itself far past diff --git a/references/changelog.md b/references/changelog.md index 5cd74ef..4cb0b94 100644 --- a/references/changelog.md +++ b/references/changelog.md @@ -1006,3 +1006,36 @@ constructed the shape that would break it. evidence; the property is what is in question. - **A guard that filters by kind is not duplicate authority** merely because a downstream owner also filters. Ask what each one protects. + +## 2026-09-24: quota, oversized fix batches, and greens that ran nothing + +Evidence from the StreamPixels transport campaign (Cuts E and F, merged in +StreamPixels `2d4215c`): + +- **Quota.** Partway through 2026-09-23, the operator flagged that 60% of the + weekly quota had gone in one day, with little of it on the StreamPixels path, + and redirected all work there ("We need that TO EAT"). SKILL.md section 5 + now says that quota is spent through scope: one full Soul gate per cut, fix + batches only for findings on the critical path, and narrow Sonnet passes for + small deltas. +- **Oversized fix batches.** One brief packed about ten items, a subtraction, + two hosts and a late addition. It ran to about 800k tokens over 2.6 hours. + Another batch passed 550k and produced sloppy code. After batches were split + per file, they stayed at about 200-400k. This is the 2026-09-18 context + budget rule again. `briefs.md` now says to use one fix batch per file. +- **Greens that ran nothing.** A zero-match `node --test` glob exited 0. + vitest discovered zero tests under a script's `--dir .` (`cc71ce2`). + `pnpm --filter X test -- ` did not filter (`4624b53`). A job-wide + `DATABASE_URL` would have switched unrelated suites onto Postgres + (`e019de0`). `briefs.md` now requires a non-zero count for each named file. +- **"Pre-existing" was used as a verdict.** Several agents waved a failing test + through as pre-existing. It had never passed on a clean checkout, because it + read a git-ignored, licensed asset pack (`27053f9`). A transient Windows file + lock was also called pre-existing, because the base commit was tested under + the same lock. `briefs.md` now asks for a one-line cause for every failure, + including ones that are not yours. +- **An agent waited forever on a hung job.** The fix went into the owner: + `ygg-verify.sh` got a watchdog (`9359e9c`), and then the watchdog's `sleep` + was stopped from holding a slot lock through an inherited file descriptor + (`dbe594e`). The commits and the script header record the details. +