Tell agents how to wait; loop random-draw tests; probe the why
Three lessons from Ghostlight L1. 1. Three agents ended their turns while their own detached cargo job was still running, even though the briefs already said to wait for long jobs. The stall stopped only once the briefs named the mechanism: block inside a tool call. 2. A test built on a random draw passed by luck, failing 3 of 40 looped runs. 3. An operator's question about why a mechanism works exposed a defect that had been present all along. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
0a01979330
commit
6bbf6bd206
1 changed files with 29 additions and 0 deletions
|
|
@ -2,6 +2,35 @@
|
|||
|
||||
Record each change to the skill together with the evidence that motivated it.
|
||||
|
||||
## 2026-09-16: agents must be told how to wait, not only to wait
|
||||
|
||||
Evidence: Ghostlight L1, the stock lenses
|
||||
(`docs/architecture/ghostlight-stock-lenses-cut.md` and its postmortem).
|
||||
|
||||
- **"Wait for long jobs" was not enough.** Three agents in one campaign ended
|
||||
their turns while a build they had started was still running: a Soul pass
|
||||
on Cut 1 and a Hands pass on Cut 2, twice. Each launched cargo detached,
|
||||
then ended the turn "to wait". A detached process is not a tracked
|
||||
child of the agent, so its completion never woke it; the work stopped until
|
||||
the root agent resumed it by hand. The rule to wait was already in the
|
||||
briefs. What worked was naming the mechanism: block inside a tool call,
|
||||
with cargo in the foreground under a long tool timeout, or `Wait-Process` on
|
||||
the pid, and never start a second cargo run while one holds the target
|
||||
lock. After that sentence entered the briefs, no agent stalled again.
|
||||
- **A green test can be a coin flip.** A Cut 3 test that rewrote a session's
|
||||
lens to a fixed value collided with a randomly drawn lens about one run in
|
||||
eight; Hands' single green run was luck. It failed 3 of 40 looped runs and
|
||||
passed 120 of 120 after a deterministic fix. For any test built on a random
|
||||
draw or a random identity, looping it is part of verification.
|
||||
- **An operator question can hide the design fault.** Self told the operator
|
||||
a lens "has to be recorded" in the command identity. The planner then found
|
||||
that keying identity on mutable weights strands rows, and the operator's
|
||||
follow-up ("are you relying on some deterministic code outputting the same
|
||||
id rather than just checking the state?") exposed a pre-existing defect: the
|
||||
elaborator's repair loop had never worked across sweeps. When the operator
|
||||
asks why a mechanism works, answer from the Body and probe it; it may be
|
||||
the most valuable question of the campaign.
|
||||
|
||||
## 2026-09-16: a mutation suite needs a no-op control and byte-exact I/O
|
||||
|
||||
Evidence: the Eureka pipeline-state campaign's third Soul pass on Cut 6, and the
|
||||
|
|
|
|||
Loading…
Reference in a new issue