Stryker scope extends to GameCult.Caching.Tests; CultElementIds survivors killed or recorded (mutation-testing-scope) (#163)

docs/mutation-testing.md

@@ -3,7 +3,7 @@
 CultLib measures its C# test suites with Stryker.NET, pinned as a repo-local
 tool in `dotnet-tools.json`. Coverage is added project by project; a project
 with no `stryker-config.json` next to its test project has not been scoped
-yet (see "Follow-ups" below for what's outstanding).
+yet (see "Scope and follow-ups" below for what's outstanding).
 
 ## Running it
 
@@ -126,14 +126,17 @@ project as precedent for the `vstest` caveat itself - that caveat is
 `mtp`-runner-specific and documented above for `static readonly` field
 initializers, and GameCult.Geometry has never actually hit it.
 
-## Follow-ups
+## Scope and follow-ups
 
-CultMath.Tests and GameCult.Geometry.Tests are the only projects currently
-scoped. GameCult.Geometry's `stryker-config.json` has no `test-runner`
-entry: the project is NUnit 4 with `NUnit3TestAdapter` and
+CultMath.Tests, GameCult.Geometry.Tests and GameCult.Caching.Tests are the
+projects currently scoped. GameCult.Geometry's `stryker-config.json` has no
+`test-runner` entry: the project is NUnit 4 with `NUnit3TestAdapter` and
 `Microsoft.NET.Test.Sdk`, so Stryker's default `vstest` runner applies
 directly (no `mtp` workaround needed, unlike CultMath.Tests's xunit.v3
-suite).
+suite). GameCult.Caching.Tests is the same shape (NUnit, default `vstest`)
+and its `stryker-config.json` is driven the same way, by the repo-local
+`dotnet-stryker` tool: `dotnet tool restore` once, then `dotnet tool run
+dotnet-stryker` from `tests/GameCult.Caching.Tests`.
 
 GameCult.Geometry has only ever been run `--since:`-scoped to the cut that
 added surface nets; a full-project baseline (running mutation against all of
@@ -151,7 +154,81 @@ file, anything under `Fixtures/`) loses `--since` line-level scoping for that
 run; a future cut whose diff is C#-only, against a base commit that already
 has `stryker-config.json`, should scope correctly.
 
-`GameCult.Caching`, `GameCult.Mesh` (+ `.Quic`, `.Quic.Native`),
+### GameCult.Caching.Tests
+
+`tests/GameCult.Caching.Tests/stryker-config.json` mutates all of
+`src/GameCult.Caching` (the project is `GameCult.Caching.csproj`; Standard
+mutation level, `break` 0, the same thresholds as Geometry). Caching was
+scoped before Networking because its suite is deterministic and quick (1415
+tests, about 40 s) over about 7.8k lines of source, where GameCult.Networking
+is about 20k lines and its tests lean on socket timing, so a mutant timing out
+there says little about the test. A full Caching run is about 2,900 candidate
+mutants (3,602 created, minus 730 that fail to compile) at about 2 s each,
+about 1.5 hours; run it detached and read the JSON report,
+never as one attached session.
+
+The first measurement was a narrow pass over the three files that carry the
+store-replace and element-id machinery, from `tests/GameCult.Caching.Tests`:
+
+```bash
+dotnet tool run dotnet-stryker \
+  -m '**/src/GameCult.Caching/StoreReplace.cs' \
+  -m '**/src/GameCult.Caching/CultStoreLinks.cs' \
+  -m '**/src/GameCult.Caching/CultElementIds.cs'
+```
+
+A clean single run of that command tested 243 mutants and scored 86.79% in
+9 minutes 55 seconds. The first measurement of it overlapped two Stryker
+runs, so its numbers (and the 114 survivors it showed for `CultElementIds.cs`,
+where tests that kill a mutant by hand reported `Survived` or `Timeout`) are
+not measurements and are not quoted here. Never run two Stryker runs against
+one checkout at once, and check `ps` before starting one. A clean single run
+on `CultElementIds.cs` alone, after the tests below, tested 228 mutants: 208
+`Killed`, 8 `Timeout`, 12 `Survived`, a score of 94.74%. The 12th survivor
+was 321, which the throwing-enumerator case in `ListsOfValuesHoldNoIds` now
+kills; the other 11 are equivalent mutants. With that case a clean single run
+on `CultElementIds.cs` is 209 `Killed`, 8 `Timeout`, 11 `Survived`, 95.18%.
+
+Real gaps that run exposed, each now pinned by `ElementIdWalkTests` (and each
+failing when its mutant is applied by hand): the order of the id path (the
+codec refuses at the first type MessagePack cannot allow, so order is what an
+error names first; `Append` to `Prepend` at 163, 167 and 180); the
+union-cycle guard in `ConcreteTypes` (248; the mutant is a stack overflow, so
+the red is a crashed test host: a depth guard would need production code in
+`ConcreteTypes`, since the recursion is private and a stack overflow cannot be
+caught by the test, so the regression test is left to crash loudly); a list of values reporting that it holds an
+id (`Holds`, 322); `[Key]` read through an override (`KeyOf`, 583); and the
+`IsWrittenWhole` and `IsKeyed` predicates (629 to 631 and 636: formatter,
+`[MessagePackObject]` and `KeyAsPropertyName` inherited from a base, and the
+`[DataContract]` without `[MessagePackObject]` and `||` arms).
+
+Equivalent survivors in `CultElementIds.cs` (11), each hand-checked against the full
+suite; none can be told apart by any test:
+
+- 142, 251, 636 (second arm), 506 (`IsDefined(UnionAttribute, false)` to
+  `true`): `UnionAttribute` is `Inherited = false`, so its `inherit` argument
+  changes nothing. Likewise 630's `DataContractAttribute` `inherit` argument
+  (`DataContractAttribute` is `Inherited = false`).
+- 264, 542: `group.First()` to `FirstOrDefault()` over `GroupBy` groups, which
+  are never empty.
+- 491: `new StringBuilder(bytes.Length * 2)` to `/ 2` changes only the initial
+  capacity.
+- 578: removing the `Nullable.GetUnderlyingType` unwrap in `WalksAsOwner`;
+  `Classify` unwraps nullables itself, so the unwrap is dead. A cut
+  candidate, not a test gap.
+- 645, 657: `Append(type)` to `Prepend(type)` in `ElementTypeOf` and
+  `MapTypes`; `type` matches only when it is itself `IEnumerable<>`,
+  `IDictionary<,>` or `IReadOnlyDictionary<,>`, and none of those has a second
+  matching interface, so the position of `type` never decides the answer.
+
+The Windows-only P/Invoke group is exercised only on Windows: its mutants are
+`NoCoverage` on Linux and are a target for a Windows-leg mutation run, not a
+Linux gap or a finding. `StoreReplace.cs` line 45 is a different case: its
+negated `IsOSPlatform(Windows)` mutant is `Survived` on Linux, because the
+negated branch calls `ReplaceWindows`, whose `DllNotFound` fallback gives the
+same result.
+
+`GameCult.Mesh` (+ `.Quic`, `.Quic.Native`),
 `GameCult.Networking` (+ `.WebSockets`), and the non-.NET runtimes
 (`cultcache-rs/py/ts`, `cultmesh-*`, `cultnet-*`) each still need their own
 `stryker-config.json` (or ecosystem-equivalent tool: `cargo-mutants` for

tests/GameCult.Caching.Tests/ElementIdWalkTests.cs

@@ -0,0 +1,151 @@
+#nullable enable
+using System;
+using System.Collections.Generic;
+using System.Linq;
+using System.Runtime.Serialization;
+using GameCult.Caching;
+using MessagePack;
+using NUnit.Framework;
+
+namespace GameCult.Caching.Tests
+{
+    // The id walk's own contracts, one mutant-sized fact at a time: the order of the id path, a union that names itself, a list of
+    // values never counting as an id, and the attribute predicates that decide whether a type is walked or written whole.
+    public class ElementIdWalkTests
+    {
+        // The codec asks MessagePack about the id path in order and refuses at the first type it cannot allow, so the order is what
+        // an error names first: the document, then the collection types from the outermost in, then the element and its arms.
+        [Test]
+        public void TheIdPathListsCollectionsFromTheOutermostIn()
+        {
+            Assert.That(CultElementIds.Inspect(typeof(MsMapMap)).IdPath, Is.EqualTo(new[]
+            {
+                typeof(MsMapMap), typeof(Dictionary<string, Dictionary<string, List<ElementIdEngineTests.EIded>>>),
+                typeof(Dictionary<string, List<ElementIdEngineTests.EIded>>), typeof(List<ElementIdEngineTests.EIded>), typeof(ElementIdEngineTests.EIded)
+            }), "maps nested in maps");
+            Assert.That(CultElementIds.Inspect(typeof(MsListDeep)).IdPath, Is.EqualTo(new[]
+            {
+                typeof(MsListDeep), typeof(List<List<List<ElementIdEngineTests.EIded>>>), typeof(List<List<ElementIdEngineTests.EIded>>),
+                typeof(List<ElementIdEngineTests.EIded>), typeof(ElementIdEngineTests.EIded)
+            }), "lists nested in lists");
+        }
+
+        // ConcreteTypes stops at a type it has already expanded, so a union that lists its own base is finished, not a stack overflow.
+        [Test]
+        public void AUnionThatNamesItselfIsWalkedOnce()
+        {
+            var survey = CultElementIds.Inspect(typeof(MsSelfHolder));
+            Assert.That(survey.Problems, Is.Empty);
+            Assert.That(survey.IdPath, Is.EquivalentTo(new[] { typeof(MsSelfHolder), typeof(List<MsSelfBase>), typeof(MsSelfBase), typeof(MsSelfArm) }));
+        }
+
+        // Value lists hold no ids however they are nested, and a root that is itself a value list holds none either.
+        [Test]
+        public void ListsOfValuesHoldNoIds()
+        {
+            var values = new MsValues { Numbers = { 1 }, Tags = { "a" }, Grid = { new List<int> { 1 } }, Map = { ["k"] = new List<int> { 2 } } };
+            Assert.That(CultElementIds.Holds(typeof(MsValues), values), Is.False);
+            Assert.That(CultElementIds.Holds(typeof(List<int>), new List<int> { 1, 2 }), Is.False);
+            Assert.That(CultElementIds.Holds(typeof(List<List<int>>), new List<List<int>> { new() { 1 } }), Is.False);
+            Assert.That(CultElementIds.Holds(typeof(List<string>), new List<string> { "a" }), Is.False);
+            Assert.That(CultElementIds.Holds(typeof(MsBoom), new MsBoom()), Is.False, "a sequence of values is not enumerated");
+        }
+
+        // A nullable struct is walked as the struct it wraps; a nullable value is not walked at all.
+        [Test]
+        public void ANullableKeyedStructWalksAsItsOwner()
+        {
+            Assert.That(CultElementIds.WalksAsOwner(typeof(MsKeyedStruct)), Is.True);
+            Assert.That(CultElementIds.WalksAsOwner(typeof(MsKeyedStruct?)), Is.True);
+            Assert.That(CultElementIds.WalksAsOwner(typeof(int?)), Is.False);
+            Assert.That(CultElementIds.WalksAsOwner(typeof(int)), Is.False);
+        }
+
+        // [Key] is read through an override, so an element whose id property overrides a keyed base member is still an id with a key.
+        [Test]
+        public void AKeyOnTheBaseOfAnOverrideCountsForTheOverride()
+        {
+            Assert.That(CultElementIds.Inspect(typeof(MsOverrideHolder)).Problems, Is.Empty);
+            Assert.That(CultElementIds.Inspect(typeof(MsOverrideHolder)).IdPath, Does.Contain(typeof(MsOverrideEl)));
+        }
+
+        // A type is walked by key when it is [MessagePackObject] or [Union], read through its bases for the former; it is one whole
+        // value when it declares its own formatter, is [DataContract] without being [MessagePackObject], or is keyed by name.
+        [Test]
+        public void KeyedTypesWalkAndWholeTypesDoNot()
+        {
+            Assert.That(CultElementIds.WalksAsOwner(typeof(MsMpBase)), Is.True, "[MessagePackObject]");
+            Assert.That(CultElementIds.WalksAsOwner(typeof(MsUBase)), Is.True, "[Union]");
+            Assert.That(CultElementIds.WalksAsOwner(typeof(MsMpDerived)), Is.True, "[MessagePackObject] inherited");
+            Assert.That(CultElementIds.WalksAsOwner(typeof(MsPlain)), Is.False, "[Key] members without [MessagePackObject] or [Union]");
+            Assert.That(CultElementIds.WalksAsOwner(typeof(MsUArm)), Is.True, "an arm that is itself [MessagePackObject]");
+
+            Assert.That(CultElementIds.WalksAsOwner(typeof(MsDc)), Is.False, "[DataContract] alone");
+            Assert.That(CultElementIds.WalksAsOwner(typeof(MsDcMp)), Is.True, "[DataContract] beside [MessagePackObject]");
+            Assert.That(CultElementIds.WalksAsOwner(typeof(MsDcOverMpBase)), Is.True, "[DataContract] over an inherited [MessagePackObject]");
+
+            Assert.That(CultElementIds.WalksAsOwner(typeof(MsPropName)), Is.False, "KeyAsPropertyName");
+            Assert.That(CultElementIds.WalksAsOwner(typeof(MsPropNameDerived)), Is.False, "KeyAsPropertyName inherited");
+            Assert.That(CultElementIds.WalksAsOwner(typeof(MsStringKey)), Is.False, "a string [Key]");
+
+            Assert.That(CultElementIds.WalksAsOwner(typeof(MsFormattedBase)), Is.False, "[MessagePackFormatter]");
+            Assert.That(CultElementIds.WalksAsOwner(typeof(MsFormattedDerived)), Is.False, "[MessagePackFormatter] inherited");
+        }
+
+        [MessagePackObject] public class MsMapMap { [Key(0)] public Dictionary<string, Dictionary<string, List<ElementIdEngineTests.EIded>>> M { get; set; } = new(); }
+        [MessagePackObject] public class MsListDeep { [Key(0)] public List<List<List<ElementIdEngineTests.EIded>>> G { get; set; } = new(); }
+
+        [Union(0, typeof(MsSelfBase)), Union(1, typeof(MsSelfArm))] public abstract class MsSelfBase { }
+        [MessagePackObject] public class MsSelfArm : MsSelfBase { [Key(0)] [CultElementId] public string Id { get; set; } = ""; }
+        [MessagePackObject] public class MsSelfHolder { [Key(0)] public List<MsSelfBase> Items { get; set; } = new(); }
+
+        [MessagePackObject]
+        public class MsValues
+        {
+            [Key(0)] public List<int> Numbers { get; set; } = new();
+            [Key(1)] public List<string> Tags { get; set; } = new();
+            [Key(2)] public List<List<int>> Grid { get; set; } = new();
+            [Key(3)] public Dictionary<string, List<int>> Map { get; set; } = new();
+        }
+
+        public sealed class MsBoom : IEnumerable<int>
+        {
+            public IEnumerator<int> GetEnumerator() => throw new InvalidOperationException("enumerated");
+            System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator() => GetEnumerator();
+        }
+
+        [MessagePackObject] public struct MsKeyedStruct { [Key(0)] public int X; }
+
+        [MessagePackObject]
+        public class MsOverrideBase
+        {
+            [Key(0)] [CultElementId] public virtual string Id { get; set; } = "";
+        }
+
+        [MessagePackObject]
+        public class MsOverrideEl : MsOverrideBase
+        {
+            public override string Id { get; set; } = "";
+        }
+
+        [MessagePackObject] public class MsOverrideHolder { [Key(0)] public List<MsOverrideEl> Items { get; set; } = new(); }
+
+        [MessagePackObject] public class MsMpBase { [Key(0)] public int A { get; set; } }
+        public class MsMpDerived : MsMpBase { }
+        public class MsPlain { [Key(0)] public int A { get; set; } }
+        [Union(0, typeof(MsUArm))] public abstract class MsUBase { }
+        [MessagePackObject] public class MsUArm : MsUBase { [Key(0)] public int A { get; set; } }
+
+        [DataContract] public class MsDc { [DataMember] public int A { get; set; } }
+        [DataContract] [MessagePackObject] public class MsDcMp { [Key(0)] public int A { get; set; } }
+        [DataContract] public class MsDcOverMpBase : MsMpBase { }
+
+        [MessagePackObject(keyAsPropertyName: true)] public class MsPropName { public int A { get; set; } }
+        public class MsPropNameDerived : MsPropName { }
+        [MessagePackObject] public class MsStringKey { [Key("a")] public int A { get; set; } }
+
+        public sealed class MsNoFormatter { }
+        [MessagePackFormatter(typeof(MsNoFormatter))] [MessagePackObject] public class MsFormattedBase { [Key(0)] public int A { get; set; } }
+        [MessagePackObject] public class MsFormattedDerived : MsFormattedBase { [Key(1)] public int B { get; set; } }
+    }
+}

tests/GameCult.Caching.Tests/stryker-config.json

@@ -0,0 +1,11 @@
+{
+  "stryker-config": {
+    "project": "GameCult.Caching.csproj",
+    "mutate": [
+      "**/src/GameCult.Caching/**/*.cs"
+    ],
+    "mutation-level": "Standard",
+    "reporters": [ "progress", "html", "json" ],
+    "thresholds": { "high": 80, "low": 60, "break": 0 }
+  }
+}