From 5d2bd3e93c724cfe1df54b117f3be49a37df2b36 Mon Sep 17 00:00:00 2001 From: Meta Vens Date: Tue, 22 Sep 2026 07:32:57 +0200 Subject: [PATCH] Fix H1: stop silently discarding an edit made while a mutation run is in flight Restore-Targets never checked what a target's bytes actually were before restoring over them, so a build step or stray edit landing mid-run or between entries was overwritten without a trace (Soul's finding on the Huginn memory-organ Cut 10 fourth fix batch, 2026-09-22; that same batch had lost a Hands edit this way). Restore-Targets and the entry loop's pre-write step now hash the target first: a match against the M0 original or this entry's own mutant restores as before, anything else is saved to .eureka-mutation-overwritten, warned about loudly, and fails the run with a non-zero exit and no further entries. Demonstrated in the scratchpad with a throwaway repo whose test command appends a line to the target mid-run: pre-fix the run went green and the target came back pristine; post-fix it failed loudly and kept the appended line. Recorded in references/changelog.md. Co-Authored-By: Claude Sonnet 5 --- references/changelog.md | 24 ++++++++++++++++++++ tools/eureka-mutations.ps1 | 45 ++++++++++++++++++++++++++++++++++---- 2 files changed, 65 insertions(+), 4 deletions(-) diff --git a/references/changelog.md b/references/changelog.md index addc724..565756f 100644 --- a/references/changelog.md +++ b/references/changelog.md @@ -2,6 +2,30 @@ Record each change to the skill together with the evidence that motivated it. +## 2026-09-22: the harness silently discarded an edit made while a run was in flight (H1) + +Evidence: Soul's pass on the Huginn memory-organ Cut 10 fourth fix batch +(2026-09-22) found that `eureka-mutations.ps1` never checked what a target's +bytes actually were before restoring over them or before the next entry wrote +its mutant. That same campaign's fourth fix batch had lost edits Hands made to +a target this way; the harness gave no warning and no non-zero exit at the +time, so the loss surfaced only later. + +`Restore-Targets` (`:163-182`) now hashes the target's current bytes first: a +match against the M0 original or, when supplied, the exact mutant the caller +just wrote, is restored as before; anything else is saved beside the target as +`.eureka-mutation-overwritten`, printed as a loud `EDIT LOST` warning, +and the run stops (non-zero exit, no further entries) via the new +`Assert-TargetUnedited`. The same check runs again before each entry writes +its own mutant, so an edit landing between two entries is caught before it is +silently buried under the next one. + +Demonstrated in the scratchpad with a throwaway repo whose test command +appends a line to the target mid-run: against the pre-fix script the run +reported green and the target came back pristine, the appended line gone; +against the fixed script the run failed loudly with `EDIT LOST`, exit code 1, +and the target still carried the appended line afterward. + ## 2026-09-22: a cleanup by image name killed the whole machine's Node Evidence: during the QUIC seventh fix batch's Windows stress round, Hands diff --git a/tools/eureka-mutations.ps1 b/tools/eureka-mutations.ps1 index a9395bf..c11bce5 100644 --- a/tools/eureka-mutations.ps1 +++ b/tools/eureka-mutations.ps1 @@ -153,18 +153,45 @@ function Measure-Sites([string] $text, [string] $anchor) { $count } +# H1: a target's bytes are never restored, and never overwritten with the +# next mutant, without first being proven to still be one of two things: the +# M0 original, or (when supplied) the exact mutant this entry itself wrote. +# Anything else means a build step, a stray edit, or another tool changed the +# target while a run was in flight or between entries -- the case Soul found +# the harness silently discarding (H1: `eureka-mutations.ps1:163-175, 393, +# 415`). Such an edit is saved beside the target, a loud warning names it, and +# the run stops: continuing would either bury the edit under a restore or, for +# the next entry's write, bury it under a different mutant, either way losing +# it exactly as before. +function Assert-TargetUnedited([string] $file, [string] $mutantHash) { + $path = $targets[$file] + if (-not (Test-Path -LiteralPath $path)) { return } + $current = Get-Hash $path + if ($current -eq $hashes[$file]) { return } + if ($mutantHash -and $current -eq $mutantHash) { return } + $overwritten = "$path.eureka-mutation-overwritten" + [System.IO.File]::WriteAllBytes($overwritten, [System.IO.File]::ReadAllBytes($path)) + $message = "EDIT LOST: $path (SHA-256 $current) is neither the M0 original ($($hashes[$file])) nor this entry's own mutant. Something edited it outside the harness while a run was in flight or between entries. Its bytes are kept in $overwritten. Stopping the run rather than silently discarding them." + Write-Host $message + throw $message +} + # Writes the original bytes back to each named target and proves it by hash, # then removes the target's sidecar. A target already hashing to its original # is not rewritten, so a write that never opened the file (a locked target) # is not reported as a failed restore. Every target is attempted even when an # earlier one fails; a target that cannot be restored is printed with RESTORE # FAILED and its original SHA-256, its sidecar is kept, and the first failure -# is rethrown once the rest have been attempted. -function Restore-Targets([string[]] $files) { +# is rethrown once the rest have been attempted. `$mutantHashes[$file]`, when +# supplied, is the SHA-256 of the mutant this call's caller itself wrote to +# `$file`, the one departure from the M0 original `Assert-TargetUnedited` lets +# through before restoring over it. +function Restore-Targets([string[]] $files, [hashtable] $mutantHashes = @{}) { $failure = $null foreach ($file in $files) { $path = $targets[$file] try { + Assert-TargetUnedited $file $mutantHashes[$file] if ((Get-Hash $path) -ne $hashes[$file]) { [System.IO.File]::WriteAllBytes($path, $bytes[$file]) } @@ -407,6 +434,11 @@ foreach ($mutation in $suite) { } $files = @($texts.Keys) $run = $null + # H1: before this entry writes its mutant, every target it touches must + # still be the M0 original -- never a prior entry's leftover mutant, and + # never an edit that landed between entries. Anything else stops the run + # rather than burying an unexplained edit under the next mutant. + foreach ($file in $files) { Assert-TargetUnedited $file $null } # Every sidecar goes down before any target is written, and the writes sit # inside the `try`, so a write that throws on the second target still # restores the first. @@ -417,8 +449,13 @@ foreach ($mutation in $suite) { } finally { # Restore is the original bytes, whatever the entry or the build left - # behind, verified by hash. - Restore-Targets $files + # behind, verified by hash -- but only once each target is proven to + # still be either the M0 original or exactly the mutant just written, + # so an edit made by the test command itself (H1) is caught here + # rather than silently overwritten by the restore. + $mutantHashes = @{} + foreach ($file in $files) { $mutantHashes[$file] = Get-BytesHash (Encode-Text $texts[$file]) } + Restore-Targets $files $mutantHashes } $named = $mutation.Test.Split(':')[-1] $verdict = if ($run.TimedOut) {