Mutation testing: scope GameCult.Caching.Tests; record the narrow pass, CultElementIds survivors and equivalents
Some checks failed
CultMesh portability and performance / ubuntu-latest reactive semantics and leases (pull_request) Has been cancelled

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Libby 2026-11-13 08:43:50 +00:00
commit d38b57284e
2 changed files with 90 additions and 7 deletions

View file

@ -3,7 +3,7 @@
CultLib measures its C# test suites with Stryker.NET, pinned as a repo-local
tool in `dotnet-tools.json`. Coverage is added project by project; a project
with no `stryker-config.json` next to its test project has not been scoped
yet (see "Follow-ups" below for what's outstanding).
yet (see "Scope and follow-ups" below for what's outstanding).
## Running it
@ -126,14 +126,17 @@ project as precedent for the `vstest` caveat itself - that caveat is
`mtp`-runner-specific and documented above for `static readonly` field
initializers, and GameCult.Geometry has never actually hit it.
## Follow-ups
## Scope and follow-ups
CultMath.Tests and GameCult.Geometry.Tests are the only projects currently
scoped. GameCult.Geometry's `stryker-config.json` has no `test-runner`
entry: the project is NUnit 4 with `NUnit3TestAdapter` and
CultMath.Tests, GameCult.Geometry.Tests and GameCult.Caching.Tests are the
projects currently scoped. GameCult.Geometry's `stryker-config.json` has no
`test-runner` entry: the project is NUnit 4 with `NUnit3TestAdapter` and
`Microsoft.NET.Test.Sdk`, so Stryker's default `vstest` runner applies
directly (no `mtp` workaround needed, unlike CultMath.Tests's xunit.v3
suite).
suite). GameCult.Caching.Tests is the same shape (NUnit, default `vstest`)
and its `stryker-config.json` is driven the same way, by the repo-local
`dotnet-stryker` tool: `dotnet tool restore` once, then `dotnet tool run
dotnet-stryker` from `tests/GameCult.Caching.Tests`.
GameCult.Geometry has only ever been run `--since:`-scoped to the cut that
added surface nets; a full-project baseline (running mutation against all of
@ -151,7 +154,76 @@ file, anything under `Fixtures/`) loses `--since` line-level scoping for that
run; a future cut whose diff is C#-only, against a base commit that already
has `stryker-config.json`, should scope correctly.
`GameCult.Caching`, `GameCult.Mesh` (+ `.Quic`, `.Quic.Native`),
### GameCult.Caching.Tests
`tests/GameCult.Caching.Tests/stryker-config.json` mutates all of
`src/GameCult.Caching` (the project is `GameCult.Caching.csproj`; Standard
mutation level, `break` 0, the same thresholds as Geometry). Caching was
scoped before Networking because its suite is deterministic and quick (1415
tests, about 40 s) over about 7.8k lines of source, where GameCult.Networking
is about 20k lines and its tests lean on socket timing, so a mutant timing out
there says little about the test. A full Caching run is about 2,200 mutants at
roughly 1.6 s each, about an hour; run it detached and read the JSON report,
never as one attached session.
The first measurement was a narrow pass over the three files that carry the
store-replace and element-id machinery, from `tests/GameCult.Caching.Tests`:
```bash
dotnet tool run dotnet-stryker \
-m '**/src/GameCult.Caching/StoreReplace.cs' \
-m '**/src/GameCult.Caching/CultStoreLinks.cs' \
-m '**/src/GameCult.Caching/CultElementIds.cs'
```
It tested 243 mutants and scored 84.91% in about 8.5 minutes. Two Stryker
runs overlapped during that pass, and it showed: the same tests that kill a
mutant by hand reported it `Survived` or `Timeout` (for example the `Holds`
guards at `CultElementIds.cs` 311 and 316), which inflated that file's survivor
list to 114. Never run two Stryker runs against one checkout at once, and check
`ps` before starting one. Re-run on `CultElementIds.cs` alone, after the tests
below, it tested 228 mutants: 208 `Killed`, 8 `Timeout`, 12 `Survived`, a score
of 94.74%, and every one of the 12 is an equivalent mutant.
Real gaps that run exposed, each now pinned by `ElementIdWalkTests` (and each
failing when its mutant is applied by hand): the order of the id path (the
codec refuses at the first type MessagePack cannot allow, so order is what an
error names first; `Append` to `Prepend` at 163, 167 and 180); the
union-cycle guard in `ConcreteTypes` (248; the mutant is a stack overflow, so
the red is a crashed test host); a list of values reporting that it holds an
id (`Holds`, 322); `[Key]` read through an override (`KeyOf`, 583); and the
`IsWrittenWhole` and `IsKeyed` predicates (629 to 631 and 636: formatter,
`[MessagePackObject]` and `KeyAsPropertyName` inherited from a base, and the
`[DataContract]` without `[MessagePackObject]` and `||` arms).
Equivalent survivors in `CultElementIds.cs`, each hand-checked against the full
suite; none can be told apart by any test:
- 142, 251, 636 (second arm), 506 (`IsDefined(UnionAttribute, false)` to
`true`): `UnionAttribute` is `Inherited = false`, so its `inherit` argument
changes nothing. Likewise 630's `DataContractAttribute` `inherit` argument
(`DataContractAttribute` is `Inherited = false`).
- 264, 542: `group.First()` to `FirstOrDefault()` over `GroupBy` groups, which
are never empty.
- 491: `new StringBuilder(bytes.Length * 2)` to `/ 2` changes only the initial
capacity.
- 321: `!elements` to `elements` in `Holds`; `elements` implies the inner
type classifies as an owner, so `elements && Leaf` is never true and the
early return is a pure shortcut (the loop below returns the same `false`).
- 578: removing the `Nullable.GetUnderlyingType` unwrap in `WalksAsOwner`;
`Classify` unwraps nullables itself, so the unwrap is dead. A cut
candidate, not a test gap.
- 645, 657: `Append(type)` to `Prepend(type)` in `ElementTypeOf` and
`MapTypes`; `type` matches only when it is itself `IEnumerable<>`,
`IDictionary<,>` or `IReadOnlyDictionary<,>`, and none of those has a second
matching interface, so the position of `type` never decides the answer.
The Windows-only P/Invoke group (the native-library and `RuntimeInformation`
paths, such as `StoreReplace.cs` line 45) can only be exercised on Windows. It
is a target for a Windows-leg mutation run, not a Linux coverage gap, and its
`NoCoverage` mutants on Linux are not findings.
`GameCult.Mesh` (+ `.Quic`, `.Quic.Native`),
`GameCult.Networking` (+ `.WebSockets`), and the non-.NET runtimes
(`cultcache-rs/py/ts`, `cultmesh-*`, `cultnet-*`) each still need their own
`stryker-config.json` (or ecosystem-equivalent tool: `cargo-mutants` for

View file

@ -0,0 +1,11 @@
{
"stryker-config": {
"project": "GameCult.Caching.csproj",
"mutate": [
"**/src/GameCult.Caching/**/*.cs"
],
"mutation-level": "Standard",
"reporters": [ "progress", "html", "json" ],
"thresholds": { "high": 80, "low": 60, "break": 0 }
}
}