From 13a11ed64a434f99114f07470226cecc3516606d Mon Sep 17 00:00:00 2001 From: Meta Vens Date: Fri, 25 Sep 2026 19:30:29 +0200 Subject: [PATCH] Correct F2's vstest survivor triage in mutation-testing.md The three TryGetOtherEndpoint survivors were triaged as a vstest coverage false-survivor, mirroring the confirmed mtp/static-readonly caveat above. That triage was wrong: the actual mutant is `return false` -> `return true` inside each per-axis guard, which is equivalent (it makes the method report an edge going nowhere, so the caller compares a sample against itself and skips it exactly as it would with a correct `false`). Rewrote the paragraph to say so, and removed the unsupported claim that this project has a confirmed vstest false-survivor case. Also recorded, per this cut's verification requirements, that `--since` needs a full commit SHA and that `safe.directory` must be set in a container. Co-Authored-By: Claude Sonnet 5 --- docs/mutation-testing.md | 45 ++++++++++++++++++++++++++++++---------- 1 file changed, 34 insertions(+), 11 deletions(-) diff --git a/docs/mutation-testing.md b/docs/mutation-testing.md index 189173a8..50d135db 100644 --- a/docs/mutation-testing.md +++ b/docs/mutation-testing.md @@ -35,6 +35,18 @@ project: dotnet stryker -t mtp --since: ``` +Use a full commit SHA for `--since`, not an abbreviation: `git rev-parse +` first, then pass the full result. Stryker resolves `--since` +through the same git plumbing a short SHA can resolve ambiguously (or fail to +resolve at all) depending on the checkout's object database, and a +misresolved base silently changes what "since" means instead of erroring. + +In a container, also set `git config --global --add safe.directory '*'` +before running Stryker (or any other command that shells out to `git` +against this checkout) - git refuses to operate on a repository owned by a +different user by default, which is the common case for a bind-mounted +checkout in a container. + Reports land in `StrykerOutput/` next to the test project (git-ignored). Open the `reports/mutation-report.html` file for a browsable view, or read `reports/mutation-report.json` for a scriptable one. @@ -75,17 +87,28 @@ inside a `static readonly` field initializer as unproven until confirmed with a direct `dotnet test` run against the same mutation; do not triage it as a missing test without that check. -The default `vstest` runner has its own analog. GameCult.Geometry.Tests's -first `--since` pass reported three `Boolean mutation -> true` survivors in -`CultGeometrySurfaceNets.TryGetOtherEndpoint` (the per-axis `if (x + 1 >= -sizeX) return false;` guards, mutated to `if (true) return false;`), despite -X/Y/Z-axis open-boundary tests that exercise exactly those guards. Hand- -mutating all three and running `dotnet test tests/GameCult.Geometry.Tests ---filter FullyQualifiedName~SurfaceNetsTests` directly failed 11 of 24 tests, -confirming the mutants are well-covered and Stryker's coverage-based test -selection mis-attributed (or dropped) their covering tests. Treat a survivor -that looks like it must be covered as unproven until confirmed the same way, -regardless of which runner is in play. +A survivor that looks like it must be covered is unproven until confirmed +with a direct `dotnet test` run against the same hand-applied mutation - but +confirm what the mutant actually is before drawing a tooling conclusion from +it. An earlier pass here reported three survivors in +`CultGeometrySurfaceNets.TryGetOtherEndpoint`'s per-axis `if (x + 1 >= +sizeX) return false;` guards and initially triaged them as a `vstest` +coverage false-survivor (the same shape as the `mtp`/`static readonly` +caveat above). That triage was wrong: the actual mutant Stryker reports +there is `return false` -> `return true` inside the guard, which is +equivalent, not a coverage miss. Returning `true` from +`TryGetOtherEndpoint` still leaves the out `other` coordinate equal to the +current coordinate (see the method: `other` is cloned from `coord` and only +one component is incremented when the method returns `true`), so the caller +compares a sample against itself, the inside/outside flags always match, +and the crossing is skipped exactly as it would have been had the method +correctly returned `false`. No test can distinguish the two branches because +there is nothing to distinguish - it is a legitimate equivalent mutant, not +evidence of a `vstest` false survivor. There is no confirmed `vstest` +coverage-false-survivor case in this project's history; if one shows up, it +still needs the same hand-mutation-plus-direct-`dotnet-test` confirmation the +`mtp`/`static readonly` caveat above requires before being triaged as a tool +bug. ## Follow-ups