Research: token audit of the swarm 2026-11-05..11 (Self 3.26B, Persona 1.88B, root 2.38B prompt tokens; Persona fan-out, session accumulation, root subagent models; ranked plan)

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
This commit is contained in:
Eureka 2026-11-11 06:53:40 +01:00
commit b1b2d63f7b

View file

@ -0,0 +1,206 @@
# Token audit of the GameCult swarm, 2026-11-05 to 2026-11-11
Served: an Imagination pass for Eureka Self (root), 2026-11-11, after the operator's
"Eureka needs to run lean, or we simply can't afford our own process." It answers where
the tokens go, what they bought, why the waste made sense to the agents, and what to cut
first. Evidence was gathered by eight Eyes subagents; nothing on any host was changed.
All times UTC. "Prompt tokens" means input + cache-read + cache-creation per API request
(the journal's `request N prefix P`; `self_run.rs:395`). That is a gross count. About 96%
of it is cache-read, billed on the API at 0.1x; cache-creation with the 1 h TTL the units
use is 2x; output 5x. How the subscription meter weighs them is not known (see
"Not measured"), so every saving below is given gross, with a note where the weighted
effect is much smaller.
## 1. Where the tokens go
Three consumers, not one. The 818M figure for 2026-11-10 covered only the first.
| Consumer | Window | Requests | Prompt tokens | 2026-11-10 alone |
|---|---|---|---|---|
| Self runs, 8 Minds (Yggdrasil) | 11-05..11-11 06:00 | 23,398 | 3,263M | 1,032M (6,379 requests, 743 turns) |
| Personas, 8 Minds (Yggdrasil) | 11-05..11-11 06:00 | 17,460 | 1,876M | 534M (4,515 requests, 4,351 turns) |
| Root sessions (Starfire) | 11-04..11-11 | 12,203 | 2,379M | 286M |
| Total | | 53,061 | 7,518M | 1,852M |
The journal gives 1,032M for Self runs on 11-10 UTC (1,069M counting unfinished turns),
not 818M; the difference was not reconciled and is probably a day-boundary choice.
Rough API-weighted units (input + 2x creation + 0.1x read + 5x output; Self and Persona
output and creation are extrapolated from four live transcripts, so treat as +-30%):
Self about 0.6B, Starfire about 0.35 to 0.4B before any premium for Opus-class models,
Persona about 0.25B.
### Self runs
- All Self and subagent requests run on `claude-sonnet-5-5`, effort medium, adaptive
thinking, compaction disabled (`DISABLE_COMPACT=1`), context ceiling 300k.
- A run starts at a median 34.8k tokens, and request 2 adds 18.3k: that is the Eureka
`SKILL.md` (53.2k chars), invoked first in 4 of 4 live transcripts. Every later request
carries it. Floor cost of the start prefix (P at request 2 x requests) is 1,229M, 37.7%
of Self tokens. The skill alone is 18.3k x 23,398 = 428M (13.1%).
- Request 1 by component (chars): tool schemas 25.6k, CLAUDE.md chain 33.3k (doctrine
19.4k, gamecult.md 11.3k, seed 2.6k), output style 9.1k, skill listing 7.7k, system
prompt 7.2k, agent listing 3.9k, first user message 3.2 to 6.2k of which 2.4k is the
same every run. 11.4k tokens are cache-read at request 1; 20.6 to 24.5k are created
fresh on every run, and about 19k on every subagent spawn.
- Context growth, not orientation, is the mass. 59.6% of Self tokens sit in requests
above 150k; 65.9% in the 86 runs (of 720 starts) with more than 100 requests. The first
12 requests of a run are 4.6% of it.
- A run is a session that swallows briefs one after another: on 11-10, 592 Persona
`start_self_run` briefs and 87 unit starts, 743 turns in 63 working runs. Median
context at the start of turn 1 is 34.8k; turn 2, 101.8k; turn 3, 126.1k; turn 4,
137.4k; turn 5 and later, 208.4k. 518 of the 743 turns are turn 5 or later. Sessions
end by Idle (52), Ceiling at 300k (39) or Rotation (15) since 11-09.
- 345 of 720 starts made no request (325 lasted one second). They cost nothing.
- The 11-10 outage (UnknownCaller, 07:39 to 12:09, 24 refused starts) cost no Self
tokens: a refused lease starts no `claude`. Its token cost was a later sandbox fault
(8 turns, 1.8M) and Persona chatter about it (54 mentions). The loss was time.
Activity on 11-10, classified from the closing text of the 80 costliest turns plus a
random 40 (transcripts are not kept, so this is the best available; 90% intervals wide):
| Activity | Share of the day's Self tokens |
|---|---|
| ship (own change; about a fifth of it unmerged at turn end) | 36.9% (29.7 to 45.3) |
| unclassifiable, mostly "Soul pass running, I'll wait" | 23.5% |
| investigate, report only | 15.5% |
| review a peer's work | 10.8% (4.7 to 17.1) |
| coordinate | 10.3% (4.9 to 17.8) |
| verify own work, no new change | 2.3% |
| blocked | 0.8% |
Flags on the top 80 turns (45% of the day): a Soul pass is mentioned in turns holding
64.8% of their tokens, mutation testing 36.0%, visual, cosmetic or docs-only change
20.1%, re-do at a new head 14.7%, waiting for a go 13.9%. Verification is mostly inside
"ship" turns and cannot be separated without usage logs. 146 turns (168.7M, 16.3%) close
with only "waiting on Soul, Hands or a poll".
### Personas
- 16,041 turns in 5.5 days; about 540 per Mind on 11-10; median gap 0.8 min; 1.09
requests per turn; mean 107k prompt tokens per request. A process starts at a median
37.2k, grows about 1.1k per turn with no compaction, and ends at a median 83.3k after
28 turns (median life 2.5 h).
- The mouth (`Bifrost/tools/persona-matrix-mouth.mjs:315-332`) turns every non-self
message in all 11 joined rooms into a pending delivery for every Persona. There is no
addressed-only filter; addressing only sets the order (rank x 8 s stagger,
`persona.rs:1982`). On 11-10: 1,079 Mind posts, 4,351 Persona turns, 4.0 turns per post.
- 3,110 of those 4,351 turns (71.5%) said nothing and started nothing. 1,796 produced
only an `intent` string; 1,126 an urgency vote; 178 a memory.
- The prompt: 58.6k chars of appended instructions, of which `colossus.md` is 16.0k and
`persona.md` 9.3k; a 20k-char first message of which "Your latest runs" is 9.6k.
- 14% of turns emit a Self-run brief (2,093 in the window, 294 launches): the rest queue
into a live Self session, which is the brief stream described above.
### Root sessions on Starfire
- One session (`6d32c5f0`, open since 11-04 20:54) holds 2,034M of the 2,379M: 10,556
requests, 169 subagents, largest single request 967k tokens.
- Subagents are 63.0% (1,498M): Sonnet 864M in 118 agents, Opus 629M in 89. Median
subagent 5.0M prompt tokens, p90 16.1M. 185 of 216 Agent calls passed no model.
- Main thread: about 2,219 requests for 881M, a mean of 397k per request.
- 51 cache misses re-created 10.4M tokens; 18 followed a gap over an hour.
## 2. Output per token
Public forge only (gamecult-ops is private and was not counted; deploys and artifacts
were not measured).
| Mind | PRs merged 11-04..11 | Merged 11-10 | Self tokens (M) | Self M per merged PR |
|---|---|---|---|---|
| libby | 104 | 24 | 692 | 6.7 |
| eureka | 68 | 14 | 590 | 8.7 |
| void | 64 | 33 | 522 | 8.2 |
| weksa | 54 | 29 | 229 | 4.2 |
| heimdall | 38 | 10 | 392 | 10.3 |
| aqua | 34 | 19 | 232 | 6.8 |
| nibu | 26 | 17 | 332 | 12.8 |
| kiko | 3 | 0 | 274 | 91 (her work is mostly not PR-shaped) |
Swarm-wide, Self plus Persona: 13.1M prompt
tokens per merged PR over the week, 10.1M on 11-10.
- 391 PRs merged in the week; 155 on 11-10, three times the daily mean. Median diff 171
lines; 95 PRs are 30 lines or fewer.
- 95% self-merged; median last push to merge 1.4 min; 81% have no comment or review on
the forge. Review happens in chat and in Self turns, not on the forge: Bifrost #21 and
#22 show zero forge interactions and a "Libby, round 3" in the body.
- The sign-in footer cluster on the forge is three PRs (Ratatoskr #61, #62, #63: 62, 90
and 24 lines, each merged within 22 minutes), plus adjacent sky-band PRs.
## 3. Structural causes
1. **The Persona is the address filter.** Nothing before the model decides whether a
message concerns a Mind, so a 37k to 83k-token call is spent on each of 8 Minds per
message to produce, 71.5% of the time, silence. Missing: routing by address before
the model. The ordering logic that knows who is addressed already exists.
2. **Chat volume becomes Self volume.** Each Persona that hears a message may emit a
brief; briefs queue into the one live Self session; the session carries every earlier
brief's context into the next unrelated one, uncompacted, to 300k. Missing: a session
boundary that follows the work instead of the clock.
3. **Self cannot speak or see across campaigns.** Libby: "I can't post in #swarm myself,
since there is no chat tool here". Each relay costs a Persona turn and seven
deliveries. Heimdall re-read Huginn #29 "because that Soul pass's mind doesn't hold
the `eureka-body` or `eureka-substrate` campaigns": the author's Soul lacked the
rulings, so a peer had to read. Nibu: "ready for Void to merge on your go": 11 of her
PRs waited on the steward's merge.
4. **The root never clears and its subagents inherit its model.** A week-old session at
a mean 397k per main request, and 89 subagents on Opus because no agent definition
names a model.
5. **Nothing records usage.** Transcripts live on tmpfs and vanish at run end; the
launcher discards the CLI's usage envelope; the connector logs "weekly reading
refreshed" without the value. Agents and Self had no way to see what a habit cost.
## 4. Ranked plan
Savings are gross prompt tokens at the observed week's volume. Rank is by weighted
effect and confidence, not by the gross number.
| # | Change | Owner | Expected saving and arithmetic | Shows it worked |
|---|---|---|---|---|
| 0 | Log one line per turn from the CLI result envelope (input, creation, read, output, model, subagent requests) and the meter value on each refresh. Adds a log line, no gate. Failure it prevents: this audit could not measure output, creation, subagent cost or quota weight, so item 4 cannot be costed. | Eureka (Huginn `self_run.rs`, connector) | None itself; makes every item below checkable | Usage lines in both journals; a per-day meter delta |
| 1 | Deliver in real time only what addresses a Mind (rank 0 in the existing ordering, DMs, Self replies); hold the rest and hand it over with her next addressed turn or beat. Deletes the deliver-everything path. | Heimdall (Bifrost mouth) with Eureka (`persona.rs` stagger) | 71.5% of turns are silent; at 60 to 70% of 534M/day: 320 to 370M/day, 1.1 to 1.3B/week. Nearly all cache-read plus about 270 output tokens per turn. Second-order: fewer briefs into Self | Persona turns per Mind per day from about 540 to under 150; silent share under 30%; briefs per day down from 592 |
| 2 | Root: name a model in each agent definition (Hands, Soul, Eyes, Modeling, Life on Sonnet or Haiku; Opus-class only where a definition argues for it), and clear the root at each handoff-area boundary | Root Self (agent definitions in the Eureka skill repo) | 629M/week of Opus subagent tokens move to Sonnet. Main thread from a mean 397k to under 150k per request: (397k - 150k) x 2,219 = 548M/week at the root model's price | Opus share of subagent tokens; mean main-thread prompt per request |
| 3 | Confirm the skill split landed, then keep cutting the fixed prefix: the 2.0k constant brief-closing rules, the 19k created per subagent spawn, `colossus.md` (16.0k chars) and "Your latest runs" (9.6k chars) in the Persona prompt | Eureka | Split: 12k x 23,398 = 281M/week (8.6% of Self), if references are not re-read. Colossus: about 5.5k x 17,460 = 96M/week. Each further 1k off the Self floor: 23M/week | P(request 2) - P(request 1) at or below 7k (was 18.3k; no post-split run existed by 05:37 on 11-11); Persona first-turn prefix |
| 4 | Lower the Self ceiling from 300k to about 150k, and end the session when a brief closes and none is queued | Eureka (`self_run.rs:99`) | At least the excess above 150k: 603M/week (18.5% of Self). Weighted effect is perhaps a third of that, since each restart re-creates about 20k tokens; do it after 3 and cost it with 0 | Share of Self tokens in requests above 150k from 59.6% to under 25%, with creation per day not rising |
| 5 | Give Self the missing means: a chat post from inside a Self run; cross-campaign read of rulings for a Soul pass; merge in a steward's repo where the steward grants it | Eureka (Huginn), each steward for merge rights | Peer review is 10.8% of Self (about 110M/day on 11-10); halving it is about 55M/day. Relays: one Persona turn plus seven deliveries each | Review-peer share in a resample of closing texts under 5%; "please post this" relays gone |
Measure, do not re-propose: the 11-10/11 doctrine changes (one Soul pass, proportional
verification). The baseline to beat is above: mutation mentioned in 36.0% and
visual-only work in 20.1% of top-turn tokens, re-do at a new head 14.7%, peer review
10.8%, waiting for a go 13.9%.
Forks for the operator or the steward, not decided here:
- Item 1 removes ambient hearing in real time. 1,126 urgency votes a day depend on every
Persona hearing everything. Is that vote a live mechanism worth about 350M tokens a day?
- Item 3: the global doctrine says the Colossus long form is not loaded automatically,
but every Persona loads it. Intent or drift?
Considered and not placed:
- A 5-minute cache TTL for Self runs. 775 request gaps over 5 minutes would each
re-create about 150k tokens: about 145M units lost against about 98M saved.
- A fresh session per brief. Gross saving 44.5% of Self tokens, but about 1,960 extra
restarts a week at about 52k weighted units each cancel most of it, and related
briefs lose their context. Item 4 is the cheaper form.
- Keeping transcripts. Item 0 gives the numbers without the disk and privacy cost.
- A cheap-model pre-reader in front of each Persona. A new layer; the addressing rank
already answers the question.
- A cheaper model for Yggdrasil subagents. Not costed: no usage split exists (item 0).
## 5. Not measured
- Output, cache-creation and subagent tokens for Self and Persona; whether journal
request lines include subagent requests; so the true cost of Soul passes and mutation
runs is unknown.
- Quota weight per token class and per model; the meter's values.
- Tool-call level activity for any finished run (no transcripts); the classification
rests on closing texts cut to 650 chars.
- gamecult-ops #78 and all private repos; deploys and artifacts per Mind.
- The effect of the 11-10/11 fixes: no run had started on the 17 KB skill by 05:37.
- How many of the 2,093 Persona briefs were claimed, left waiting or dropped.
- The 818M versus 1,032M difference for 11-10.
Evidence files (Starfire, session scratchpad, not retained): journal extracts per Mind,
Persona raw pulls, the turn table for 11-10 (`turns.csv`), forge PR rows (`rows.json`),
and the Starfire usage script.