From b305debd49f0be133463f4910ac17dd02ee008d2 Mon Sep 17 00:00:00 2001 From: Meta Vens Date: Tue, 22 Sep 2026 15:59:06 +0200 Subject: [PATCH] Cut 11 map: mutation testing moves to Stryker, and the survivors it found Co-Authored-By: Claude Opus 5 --- docs/fire-control-cut.md | 97 +++++++++++++++++++ .../Aetheria.Shared.Tests/stryker-config.json | 12 +++ 2 files changed, 109 insertions(+) create mode 100644 tests/Aetheria.Shared.Tests/stryker-config.json diff --git a/docs/fire-control-cut.md b/docs/fire-control-cut.md index f1e4a3577..5e5021d59 100644 --- a/docs/fire-control-cut.md +++ b/docs/fire-control-cut.md @@ -1697,3 +1697,100 @@ configuration, not the range it claims), plus the harness-audit items: the tree-clean verdict restores before it measures, a red result cannot distinguish a killed mutant from a compile failure, and the no-op control covers only one file per harness. + +--- + +## Cut 11: mutation testing moves to Stryker, and the survivors it found + +Date: 2026-09-22. Operator, on the hand-written anchor harnesses: "What is the +point of those mutation anchors? Tests should cover how the code behaves, not how +it is shaped." And on Stryker: "sounds good; we don't need to run with every +mutation active, right? > to >= on float thresholds is indeed petty." + +### Why + +The Eureka principle (2026-09-15) is sound and stays: every rule needs a test that +fails when the rule breaks. The mechanism that accreted on it over the following +week -- literal-text anchors, perl substitution, per-cut bash and Python harnesses +-- coupled mutation testing to code shape. It stranded anchors on every refactor +(`72c0109c`, Cut 9, Cut 10), and at Cut 10 an anchor kept matching exactly once +while silently re-targeting onto `Inspect`, mutating code the tests no longer ran. +Soul's harness audit also found the tree-clean verdict restoring before it +measured, and a compile failure counting as a kill. + +A Stryker spike against `FireControl.cs` alone: 276 mutants, 7m18s, 208 killed, +67 survived. It killed every mutation the hand-written harnesses defended, never +edits a source file, and has nothing to re-anchor. + +### 11.1 Retire the ServerShared harnesses + +Delete the 25 hand-rolled harnesses whose targets are all under +`Assets/Scripts/ServerShared` -- every `mutation_tests_fire_control_*`, every +`mutation_tests_stats_power_*`, `mutation_tests_shield_reserve.py`, +`mutation_tests_condition_ratio_cut8.py` and `mutation_tests.py`. Stryker's scope +covers all of their targets; the behavioural tests they measured stay. + +**Kept, recorded, not yet answered:** `mutation_tests_addressables_cut.py` and the +three `mutation_tests_shield_panel_cut*.py`. They target Unity Editor probes +(`EngineAssetCheck.cs`, `ShieldPanelCut*Verify.cs`) that Unity compiles outside +`Aetheria.Shared`, so Stryker cannot reach them. They carry the same anchor +fragility; the right answer for GPU and editor code is probably that the batchmode +probes are themselves the behavioural tests, but that is a separate decision. + +### 11.2 Scope, pinned + +`tests/Aetheria.Shared.Tests/stryker-config.json` scopes mutation to +`Assets/Scripts/ServerShared/**` and excludes plugins. Per cut, run against the +cut's own diff: + + dotnet tool restore + cd tests/Aetheria.Shared.Tests + CULTLIB_ROOT= dotnet stryker --since: + +**Float-threshold boundary flips are known-equivalent.** Stryker files `>`→`>=` +under the same Equality mutator as `==`→`!=`, and the latter catches real bugs, so +the mutator stays on. A survivor that only moves a float comparison across its +boundary is triaged as equivalent and never chased. For the same reason the raw +mutation score is not a gate; the survivor list is the signal. + +### 11.3 Degenerate fixtures + +Most of the 67 survivors trace to three constants shared across the fire-control +fixtures, each of which makes a family of mutants arithmetically identical to the +original -- the same defect class as the zero-velocity target and the durability +coincidence, found here as a pattern instead of one at a time: + +- **Shooter at `float3.zero`.** `target - source` equals `target + source`, so four + separate position mutations survive (`FireControl.cs` HitProbability, Inspect, + Fire, Apply). +- **Fire time zero.** `now - FireTime` equals `now + FireTime`, so the elapsed-time + mutation survives -- which means `EvasionCountsUntilCommitAndNotAfter` still does + not defend the rule Cut 5 gave it a moving target to defend. +- **Spread zero everywhere.** `PSpread` returns 1 on every fixture, so it is + effectively untested, and `*` versus `/` against it is indistinguishable. + +Move shooters off the origin, fire at nonzero times, and give some fixtures real +spread. Fix the fixture, never weaken the assertion. + +### 11.4 Behavioural tests the survivors point at + +- **Splash handedness.** Flipping the sign in `var right = float2(forward.y, + -forward.x)` passes every test. That is the operator's "damage on the wrong side + of the ship," and Self said on 2026-09-19 that Cut 6b would add a test pinning + that a port blast damages port cells; it never landed and nothing checked. Pin it. +- **The shield loses energy when it absorbs.** Deleting `Shield.TakeHit` from the + discrete path survives, because tests assert the hull was untouched and never + that the shield paid for it. +- **Splash's shield-absorb branch** has no coverage at all. +- **Zones roll differently.** `CombatSeed *` → `/` makes the seed zone-independent + and survives; `TheDieIsUniform` checks uniformity within a zone only. Tighten it + enough that the `MixSeed` weakenings that survive it (`>>`→`<<`, `^=`→`|=`) die. +- **A shot whose target left the zone resolves as a miss** (Soul C4). Today only + the uncommitted branch rewrites the outcome; a committed hit on a target that + died mid-flight is published as a hit. Fix the code and pin it. + +### Verification + +Stryker over `FireControl.cs` before and after: every non-equivalent survivor +above killed, the remainder triaged by name. 231 tests plus the new ones; Unity +batchmode clean. diff --git a/tests/Aetheria.Shared.Tests/stryker-config.json b/tests/Aetheria.Shared.Tests/stryker-config.json new file mode 100644 index 000000000..f1708b314 --- /dev/null +++ b/tests/Aetheria.Shared.Tests/stryker-config.json @@ -0,0 +1,12 @@ +{ + "stryker-config": { + "project": "Aetheria.Shared.csproj", + "mutate": [ + "**/Assets/Scripts/ServerShared/**/*.cs", + "!**/Assets/Plugins/**" + ], + "mutation-level": "Standard", + "reporters": [ "progress", "html", "json" ], + "thresholds": { "high": 80, "low": 60, "break": 0 } + } +}