Record quota sizing, per-file fix batches, and greens that ran nothing

From the StreamPixels transport campaign, 2026-09-23 to 2026-09-24:
- SKILL.md section 5: quota is spent through scope. One Soul gate per cut,
  fix batches only for critical-path findings, and narrow passes for
  small deltas.
- briefs.md: one fix batch per file (800k and 550k runs); require a
  non-zero test count for each named file; a failure called
  "pre-existing" still needs a one-line cause.
- changelog: the evidence, with StreamPixels commit refs.

Staged by path. The unrelated uncommitted Ghostlight hunks in briefs.md
and changelog.md are left in the working tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Meta Vens 2026-09-24 16:09:38 +02:00
commit 82afae1afc
3 changed files with 66 additions and 0 deletions

View file

@ -324,6 +324,15 @@ cut.** In the CultCache migration, Cuts 6 and 6b each needed four Soul passes, a
the later passes still found real defects. Stop when a pass finds nothing that
blocks, or only findings the operator chooses to record.
**Quota is spent through scope, not through the number of passes.** On
2026-09-23 the operator flagged that 60% of the weekly quota had gone in one
day with little of it on the StreamPixels critical path. After that, Self ran
one full Soul gate per cut. A fix batch went only to findings on the critical
path, and the rest were recorded as follow-ups. Every later Soul pass covers
only the fix batch's diff, as the second-pass clause in `briefs.md` says, and a
small delta gets a narrow Sonnet pass. This does not cap the loop on a
foundation cut. It keeps each pass the right size.
When the same kind of finding recurs, the brief was missing context. Fix the
brief template, not only the instance. That is how "operator rulings each need
their own failing mutation" entered the Hands brief.

View file

@ -115,6 +115,23 @@ reading what it prints: that surface is the one a test suite cannot check.
A test's inputs come from the production path the rule is about. A helper
that re-spells what production prints, parses or derives makes the test agree
with a copy, and the rule then breaks in production with the suite green.
A green run is not evidence until it shows a non-zero count for the tests you
meant to run. Paste the count for each named file. On StreamPixels, all three
of these produced a green that had run nothing, or the wrong thing:
- A `node --test` glob that matches no files exits 0.
- A package script's `--dir .` made vitest discover zero tests (`cc71ce2`).
- `pnpm --filter X test -- <files>` ran the whole suite instead of the named
files (`4624b53`).
Set environment variables only on the step that needs them. Exporting
`DATABASE_URL` for the whole job silently moves `app.test.ts` onto the
Postgres driver (`e019de0`).
"Pre-existing" is not a diagnosis. A failure that is not yours still gets a
one-line cause: what it depends on, and whether it has ever passed on a clean
checkout. Several agents waved a StreamPixels test through as pre-existing. It
read a git-ignored, licensed asset pack and had never passed on a clean
checkout (`27053f9`). Reproducing a failure at the base commit proves nothing
when both runs share the cause, for example a Windows file lock on the same
machine.
A hand mutation hits the rule at the layer it protects: the production call
site, observed where the consequence lands (disk, the kernel, the request
sent). Mutating a helper or an in-memory mirror proves the helper, not the
@ -142,6 +159,13 @@ one worker's head, and the brief says how:
- **Carry the cut inline.** Quote the cut's own section in the brief instead of
pointing at a long map; a worker that reads 900 lines to find 60 has spent
its budget before it starts.
- **One fix batch per file.** A batch that packs many findings, a
subtraction, two hosts and a late addition does not fit in one head. On
StreamPixels on 2026-09-23, one batch of about ten items ran to about 800k
tokens over 2.6 hours, and another passed 550k and produced sloppy code.
After the findings were split per file, batches stayed at about 200-400k. A
late item gets its own brief. Do not add it to a batch that is already
planned.
- **Keep expensive scaffolding alive across cuts** (a pinned dependency
worktree, a warm build) rather than creating and removing it per pass.
- **Hand back rather than push through.** A worker that finds itself far past

View file

@ -1006,3 +1006,36 @@ constructed the shape that would break it.
evidence; the property is what is in question.
- **A guard that filters by kind is not duplicate authority** merely because a
downstream owner also filters. Ask what each one protects.
## 2026-09-24: quota, oversized fix batches, and greens that ran nothing
Evidence from the StreamPixels transport campaign (Cuts E and F, merged in
StreamPixels `2d4215c`):
- **Quota.** Partway through 2026-09-23, the operator flagged that 60% of the
weekly quota had gone in one day, with little of it on the StreamPixels path,
and redirected all work there ("We need that TO EAT"). SKILL.md section 5
now says that quota is spent through scope: one full Soul gate per cut, fix
batches only for findings on the critical path, and narrow Sonnet passes for
small deltas.
- **Oversized fix batches.** One brief packed about ten items, a subtraction,
two hosts and a late addition. It ran to about 800k tokens over 2.6 hours.
Another batch passed 550k and produced sloppy code. After batches were split
per file, they stayed at about 200-400k. This is the 2026-09-18 context
budget rule again. `briefs.md` now says to use one fix batch per file.
- **Greens that ran nothing.** A zero-match `node --test` glob exited 0.
vitest discovered zero tests under a script's `--dir .` (`cc71ce2`).
`pnpm --filter X test -- <files>` did not filter (`4624b53`). A job-wide
`DATABASE_URL` would have switched unrelated suites onto Postgres
(`e019de0`). `briefs.md` now requires a non-zero count for each named file.
- **"Pre-existing" was used as a verdict.** Several agents waved a failing test
through as pre-existing. It had never passed on a clean checkout, because it
read a git-ignored, licensed asset pack (`27053f9`). A transient Windows file
lock was also called pre-existing, because the base commit was tested under
the same lock. `briefs.md` now asks for a one-line cause for every failure,
including ones that are not yours.
- **An agent waited forever on a hung job.** The fix went into the owner:
`ygg-verify.sh` got a watchdog (`9359e9c`), and then the watchdog's `sleep`
was stopped from holding a slot lock through an inherited file descriptor
(`dbe594e`). The commits and the script header record the details.