Eureka/demo
Repository files (latest commit first)
Filename Latest commit message Latest commit date
2026-11-08 05:48:29 +00:00
..
fixture demo: tabsplit fixture with a seeded rule-3 bug, setup script, Soul brief 2026-11-08 03:19:09 +00:00
EXPECTED.md demo: tabsplit fixture with a seeded rule-3 bug, setup script, Soul brief 2026-11-08 03:19:09 +00:00
README.md demo README: forge clone URL, step 1 survives a pre-demo clone, charters not auto-installed 2026-11-08 05:48:29 +00:00
setup.sh demo: tabsplit fixture with a seeded rule-3 bug, setup script, Soul brief 2026-11-08 03:19:09 +00:00
soul-brief.md demo: tabsplit fixture with a seeded rule-3 bug, setup script, Soul brief 2026-11-08 03:19:09 +00:00

Eureka in ten minutes

The rule the skill is built on: the agent that writes the code does not certify it. This walk shows it on a toy repo. An author reports "done, 6 tests green". A second agent that did not write the code, told to attack it, proves the report false.

What you need: Claude Code logged in, git, python3 (standard library only). No Huginn, no mind, no Yggdrasil, no account on our forge.

What this is not: the full pipeline. The real loop (Imagination maps, Hands cuts, Soul falsifies, typed state in Huginn) needs the huginn CLI and a daemon; see the repo README. This demo runs one organ, Soul, on its own charter (agents/soul.md), with its verdict returned as text instead of admitted to a mind.

1. Install the skill

git clone https://eureka.gamecult.org/forge/GameCult/Eureka ~/.claude/skills/eureka

Claude Code finds the skill from SKILL.md. If demo/ is missing from the clone (it predates the demo), run git -C ~/.claude/skills/eureka pull.

The faculty charters in agents/ are not auto-installed as subagent types. Only demo/setup.sh copies one, agents/soul.md, into the fixture's own .claude/agents/, so nothing global is touched. To use Imagination, Hands or the others in your own projects, copy their charters into ~/.claude/agents/ yourself.

2. Make the fixture

~/.claude/skills/eureka/demo/setup.sh ~/tabsplit-demo
cd ~/tabsplit-demo
fixture ready in /home/you/tabsplit-demo

You now have a git repo with one commit, the author's: tabsplit.py splits a bill between payers, SPEC.md holds its four rules, HANDS_REPORT.md is the author's report. Read SPEC.md and the report. The report says rule 3 ("no two shares differ by more than a cent") is done, and that the status is done.

3. See the author's evidence

python3 -m unittest
Ran 6 tests in 0.000s

OK

Green. This is where an agent that grades itself stops.

4. Soul falsifies

Start Claude Code in the fixture (claude), and paste the contents of soul-brief.md after the words "Use the soul subagent." The brief gives Soul the report and the spec, asks for a verdict, and forbids editing. Do not open EXPECTED.md yet.

Captured output, abridged only where marked (fresh clone, fresh directory, run 2026-11-08, Sonnet 5.5; your wording will differ, the finding should not):

Verdict: the work is not done. Rule 3 is Broken, and the report's "Status: done" claim is false.

| Rule 3, fair (max − min ≤ 1) | Broken | split_cents(10,4) returns [2,2,2,4], a spread of 2. split_cents(7,5) returns [1,1,1,1,3], also a spread of 2. | | Report: "last payer absorbs the rounding, so the split is fair" | Broken | This is the Rule 3 failure. Dumping the whole remainder on one payer is the opposite of fair. | | Report: no test covers Rule 3 | Unproven | The report cites no test for Rule 3 and none exists. The suite is green because nothing checks fairness. |

Finding 1: Rule 3 violated (severity high). tabsplit.py, round(total/n) followed by shares[-1] += total - share*n. split_cents(10, 4) gives [2, 2, 2, 4]. For 100 payers and a 49-cent total, round(0.49) is 0, so 99 payers pay nothing and the last pays 49.

Finding 2: no test for fairness (severity medium). The only multi-payer cases are 90/3 (divides evenly) and 100/3 (last share is 34, within one cent). The suite cannot fail on Rule 3.

(Rules 1, 2, 4 and the "6 tests pass" claim: Holds, each with its probe output. Omitted here.)

Soul's rule-by-rule table also checks that the three rules the author did test really hold. It is not hunting for something to say.

5. Do not take Soul's word either

A verdict is a claim. Check it with no model in the loop:

python3 -c "
from tabsplit import split_cents
bad = [(t, n) for n in range(1, 13) for t in range(0, 200)
       if max(split_cents(t, n)) - min(split_cents(t, n)) > 1]
print(len(bad), 'of', 12 * 200, 'cases break rule 3; first:', bad[:3])"
1034 of 2400 cases break rule 3; first: [(2, 4), (6, 4), (10, 4)]

Now open EXPECTED.md for the seeded bug and why the author's tests could not see it.

6. Fix, with a test that fails first

The author's tests passed because they pinned the author's spelling. The fix adds a test for the rule itself. Ask Claude to fix it, or do it by hand: replace the body with divmod, and add a test over n in 1..12 and totals 0..199 that checks the sum and the one-cent spread.

q, r = divmod(total, n)
return [q + 1] * r + [q] * (n - r)

Against the old code the new test fails:

AssertionError: 2 not less than or equal to 1 : (2, 4)
FAILED (failures=1)

Against the fix:

Ran 7 tests in 0.002s

OK

The failing-first run matters: a test that never failed against the broken code has not shown it can. For anything that counts, run step 4 again in a fresh session. The pass that finds nothing is the one that earns "done".

Where this leaves you

You ran one faculty. The skill's loop around it (a mapper that writes the spec Hands cuts against, Hands working one cut at a time, rulings from an operator, typed state in Huginn) is in SKILL.md.

Files: fixture/ is the repo copied by setup.sh; soul-brief.md is the prompt; EXPECTED.md is the answer key.