Record quota sizing, per-file fix batches, and greens that ran nothing
From the StreamPixels transport campaign, 2026-09-23 to 2026-09-24: - SKILL.md section 5: quota is spent through scope. One Soul gate per cut, fix batches only for critical-path findings, and narrow passes for small deltas. - briefs.md: one fix batch per file (800k and 550k runs); require a non-zero test count for each named file; a failure called "pre-existing" still needs a one-line cause. - changelog: the evidence, with StreamPixels commit refs. Staged by path. The unrelated uncommitted Ghostlight hunks in briefs.md and changelog.md are left in the working tree. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
dbe594e395
commit
82afae1afc
3 changed files with 66 additions and 0 deletions
9
SKILL.md
9
SKILL.md
|
|
@ -324,6 +324,15 @@ cut.** In the CultCache migration, Cuts 6 and 6b each needed four Soul passes, a
|
|||
the later passes still found real defects. Stop when a pass finds nothing that
|
||||
blocks, or only findings the operator chooses to record.
|
||||
|
||||
**Quota is spent through scope, not through the number of passes.** On
|
||||
2026-09-23 the operator flagged that 60% of the weekly quota had gone in one
|
||||
day with little of it on the StreamPixels critical path. After that, Self ran
|
||||
one full Soul gate per cut. A fix batch went only to findings on the critical
|
||||
path, and the rest were recorded as follow-ups. Every later Soul pass covers
|
||||
only the fix batch's diff, as the second-pass clause in `briefs.md` says, and a
|
||||
small delta gets a narrow Sonnet pass. This does not cap the loop on a
|
||||
foundation cut. It keeps each pass the right size.
|
||||
|
||||
When the same kind of finding recurs, the brief was missing context. Fix the
|
||||
brief template, not only the instance. That is how "operator rulings each need
|
||||
their own failing mutation" entered the Hands brief.
|
||||
|
|
|
|||
|
|
@ -115,6 +115,23 @@ reading what it prints: that surface is the one a test suite cannot check.
|
|||
A test's inputs come from the production path the rule is about. A helper
|
||||
that re-spells what production prints, parses or derives makes the test agree
|
||||
with a copy, and the rule then breaks in production with the suite green.
|
||||
A green run is not evidence until it shows a non-zero count for the tests you
|
||||
meant to run. Paste the count for each named file. On StreamPixels, all three
|
||||
of these produced a green that had run nothing, or the wrong thing:
|
||||
- A `node --test` glob that matches no files exits 0.
|
||||
- A package script's `--dir .` made vitest discover zero tests (`cc71ce2`).
|
||||
- `pnpm --filter X test -- <files>` ran the whole suite instead of the named
|
||||
files (`4624b53`).
|
||||
Set environment variables only on the step that needs them. Exporting
|
||||
`DATABASE_URL` for the whole job silently moves `app.test.ts` onto the
|
||||
Postgres driver (`e019de0`).
|
||||
"Pre-existing" is not a diagnosis. A failure that is not yours still gets a
|
||||
one-line cause: what it depends on, and whether it has ever passed on a clean
|
||||
checkout. Several agents waved a StreamPixels test through as pre-existing. It
|
||||
read a git-ignored, licensed asset pack and had never passed on a clean
|
||||
checkout (`27053f9`). Reproducing a failure at the base commit proves nothing
|
||||
when both runs share the cause, for example a Windows file lock on the same
|
||||
machine.
|
||||
A hand mutation hits the rule at the layer it protects: the production call
|
||||
site, observed where the consequence lands (disk, the kernel, the request
|
||||
sent). Mutating a helper or an in-memory mirror proves the helper, not the
|
||||
|
|
@ -142,6 +159,13 @@ one worker's head, and the brief says how:
|
|||
- **Carry the cut inline.** Quote the cut's own section in the brief instead of
|
||||
pointing at a long map; a worker that reads 900 lines to find 60 has spent
|
||||
its budget before it starts.
|
||||
- **One fix batch per file.** A batch that packs many findings, a
|
||||
subtraction, two hosts and a late addition does not fit in one head. On
|
||||
StreamPixels on 2026-09-23, one batch of about ten items ran to about 800k
|
||||
tokens over 2.6 hours, and another passed 550k and produced sloppy code.
|
||||
After the findings were split per file, batches stayed at about 200-400k. A
|
||||
late item gets its own brief. Do not add it to a batch that is already
|
||||
planned.
|
||||
- **Keep expensive scaffolding alive across cuts** (a pinned dependency
|
||||
worktree, a warm build) rather than creating and removing it per pass.
|
||||
- **Hand back rather than push through.** A worker that finds itself far past
|
||||
|
|
|
|||
|
|
@ -1006,3 +1006,36 @@ constructed the shape that would break it.
|
|||
evidence; the property is what is in question.
|
||||
- **A guard that filters by kind is not duplicate authority** merely because a
|
||||
downstream owner also filters. Ask what each one protects.
|
||||
|
||||
## 2026-09-24: quota, oversized fix batches, and greens that ran nothing
|
||||
|
||||
Evidence from the StreamPixels transport campaign (Cuts E and F, merged in
|
||||
StreamPixels `2d4215c`):
|
||||
|
||||
- **Quota.** Partway through 2026-09-23, the operator flagged that 60% of the
|
||||
weekly quota had gone in one day, with little of it on the StreamPixels path,
|
||||
and redirected all work there ("We need that TO EAT"). SKILL.md section 5
|
||||
now says that quota is spent through scope: one full Soul gate per cut, fix
|
||||
batches only for findings on the critical path, and narrow Sonnet passes for
|
||||
small deltas.
|
||||
- **Oversized fix batches.** One brief packed about ten items, a subtraction,
|
||||
two hosts and a late addition. It ran to about 800k tokens over 2.6 hours.
|
||||
Another batch passed 550k and produced sloppy code. After batches were split
|
||||
per file, they stayed at about 200-400k. This is the 2026-09-18 context
|
||||
budget rule again. `briefs.md` now says to use one fix batch per file.
|
||||
- **Greens that ran nothing.** A zero-match `node --test` glob exited 0.
|
||||
vitest discovered zero tests under a script's `--dir .` (`cc71ce2`).
|
||||
`pnpm --filter X test -- <files>` did not filter (`4624b53`). A job-wide
|
||||
`DATABASE_URL` would have switched unrelated suites onto Postgres
|
||||
(`e019de0`). `briefs.md` now requires a non-zero count for each named file.
|
||||
- **"Pre-existing" was used as a verdict.** Several agents waved a failing test
|
||||
through as pre-existing. It had never passed on a clean checkout, because it
|
||||
read a git-ignored, licensed asset pack (`27053f9`). A transient Windows file
|
||||
lock was also called pre-existing, because the base commit was tested under
|
||||
the same lock. `briefs.md` now asks for a one-line cause for every failure,
|
||||
including ones that are not yours.
|
||||
- **An agent waited forever on a hung job.** The fix went into the owner:
|
||||
`ygg-verify.sh` got a watchdog (`9359e9c`), and then the watchdog's `sleep`
|
||||
was stopped from holding a slot lock through an inherited file descriptor
|
||||
(`dbe594e`). The commits and the script header record the details.
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue