Soul found commit 2 didn't pass (docs/cultnet-selection-cut.md, 2026-09-22):
empty/blank schemas-keys still reached the evaluator, the shard snapshot ran
its own loop instead of the evaluator, Mesh's v0 wire silently dropped v1-only
selection terms, the Mesh one-pass read could return the wrong schema's
record at a shared key, two decode rules for a foreign schema id disagreed,
dead identity-comparison code from the _projectRecord removal was still live,
and a disposed-socket race could abort a mutation run instead of recording
NO VERDICT. This is Self's fix batch for all of it.
- CultNetSelection.Validate now refuses a blank/whitespace entry inside
schemas or keys, not just an empty array. CultNetSelectionEvaluator no
longer treats [] as "no filter" (the door already refuses it before
evaluation). CultNetV0SelectionLowering is the one place that lowers an
empty or all-blank v0 list to null; CreateRawSnapshotResponse and
CreateShardSnapshotResponse both route through it, and CultMesh's own
Clean() now delegates to it instead of duplicating the rule.
- CreateShardSnapshotResponse no longer carries a private loop or its own
schema/key matching: it builds a CultNetSelection and answers through
CultNetDocumentRegistry.SelectPage (the same evaluator path as the
non-shard snapshot), with the shard's own membership check as a row
filter. Binding-alias expansion now runs on both paths identically.
- ResolveDefaultSelection decides "no schema filter" from the cleaned
recordKeys, not the raw array length. OverlaySelection stops deciding what
a missing override means: any explicit schemaIds/recordKeys override
replaces the default selection wholesale; no override keeps the default
untouched.
- CultMeshSnapshotRequestOptions.Selection is now checked by
EnsureV0Compatible before every send: fields, cites, cited, limit, cursor,
descending, and any non-default projection are refused typed
(CultNetSelectionInvalidException), never silently dropped from the v0
wire message. Mesh sending v1 is FU-Mesh-v1, not this cut.
- CultMeshHotBodySubscription.Selection returns a defensive copy on every
read so its arrays can't be mutated out from under the stored selection.
- ReadDocumentFromSnapshotResponse now searches candidates at one recordKey
in one order - exact schema, then alias, then a foreign schema id decoded
from the payload - instead of one OR'd condition where a case-insensitive
or structurally-lucky candidate could win first. The foreign-id decode
reads the payload's embedded schema version with the registry's shared
CultNetDocumentRegistry.TryReadSchemaVersion and alias-matches it against
the caller's own descriptor directly, rather than resolving to "whichever
registered type owns this schema string" - that resolution is ambiguous
when two document types alias the same schema id (MeshNoteDocument and
MeshNoteAliasDocument in the existing fixtures), which is what broke on
the first pass; TryResolveDescriptorByPayloadSchema stays for its existing
caller (ResolveDescriptorForRawDocument) with a note on when not to use it.
DecodeSnapshotDocuments and the Mesh copy of the payload-schema decode
(RawSnapshotPayloadMatchesSchema/TryReadSchemaVersion) are deleted in
favour of the shared one.
- CultNetDatabaseSubscriptionServer: the dead RecordKey/SchemaId identity
comparisons in Reconcile and ApplyProjectedChange, and the duplicate-key
throw in CreateProjectedSnapshot, are deleted - they can no longer fire
now that a projection's dictionary key is always the record's own key.
A new test flips authorizeRecord per key while the whole request stays
authorized (so DemandActive never toggles) and proves both halves of the
remaining diff: the removal of a now-unauthorized key and the add loop
picking up a newly-authorized one.
- Every NetworkingTests.cs test that starts an IsBackground poll thread now
joins it before its server disposes (about 20 tests; one already did).
- scripts/mutate-dotnet.ps1: a dotnet-test-host crash writing to stderr no
longer throws the whole run away under PowerShell 5.1's
$ErrorActionPreference = 'Stop' (2>&1 on a native command wraps stderr in
a terminating NativeCommandError); Run-Tests catches it and the entry loop
records NO VERDICT instead of aborting every remaining entry.
- Two probe tests: CultNetDatabase_ShardAndNonShardSnapshot_Agree{On
UnfilteredRequest,OnBindingSchemaIdOverride} prove the shard and non-shard
snapshot paths agree, including shard-membership exclusion of a document
outside the shard's key prefix and a binding schema-id override matching
on both paths.
- Mutation entries updated/added to match: the stale NET-D4-FourthMatcher
entry (its anchor was the private loop this batch deleted, not refactored)
is replaced by NET-D4-ShardMembership-Revert; the Mesh entries file's
header miscount ("two of three" loosening entries when there are only
two) is corrected and both previously-surviving loosening mutants
(MESH-DEFAULT-Loosening, MESH-KEY-Loosening) now have killing fixtures;
scripts/mutate-cultnet-selection-fixbatch2.entries.ps1 is new, covering
Reconcile's remaining two mutants.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
221 lines
11 KiB
PowerShell
221 lines
11 KiB
PowerShell
# Generic mutation runner for dotnet/NUnit suites - the CultNet selection campaign's counterpart to
|
|
# tools/eureka-mutations.ps1, whose Invoke-Cargo parses `cargo test` output and appends `-- --exact
|
|
# <test>`, and is therefore not usable against `dotnet test`. This is not scripts/mutate-cultmesh.mjs
|
|
# either - that file is the QUIC campaign's, live and dirty in this tree, and per
|
|
# docs/cultnet-selection-cut.md section 10 this campaign adds its entries in its own file rather than
|
|
# editing the QUIC campaign's targets.
|
|
#
|
|
# One copy serves every dotnet target in this campaign (and, once proven out, any future one): pass
|
|
# -Target, -TestProject and -Entries. Same discipline as the cargo harness: every entry is a byte-exact
|
|
# splice against one anchor that must match exactly once, restored from the original bytes (never from
|
|
# git) with a SHA-256 check per target, a sidecar (<target>.mutation-original) protects a hard kill
|
|
# mid-mutation, and M0 proves the read/write path round-trips and the unmutated suite is green before
|
|
# any entry runs.
|
|
#
|
|
# powershell -File scripts/mutate-dotnet.ps1 `
|
|
# -Target src\GameCult.Caching\CultCache.cs `
|
|
# -TestProject tests\GameCult.Caching.Tests\GameCult.Caching.Tests.csproj `
|
|
# -Entries scripts\mutate-cultnet-selection-caching.entries.ps1 `
|
|
# -ControlFilter 'FullyQualifiedName~CultDocumentSelectionSurfaceTests'
|
|
#
|
|
# -Target one or more files entries mutate, relative to the repo root. An entry with no `File`
|
|
# uses the sole target; an entry with more than one target names each edit's `File`.
|
|
# -TestProject the .csproj `dotnet test` runs. One project per invocation; entries needing another
|
|
# project are a separate run (as networking and caching already are).
|
|
# -Entries a .ps1 that returns an array of `@{ Id; Rule; Mutant; Killer; Old; New; File? }` (or
|
|
# `Edits = @(@{ File; Old; New }, ...)` for a multi-file entry). `Killer` is the
|
|
# `dotnet test --filter` value for the one test that must fail under this mutant.
|
|
# -ControlFilter the `dotnet test --filter` M0 runs to prove the suite is green unmutated. Defaults to
|
|
# running the whole project.
|
|
# -Repo repo root -Target/-TestProject/-Entries are relative to. Defaults to this script's
|
|
# parent's parent (scripts/..).
|
|
#
|
|
# Exit code: 0 when every entry's mutant was KILLED, non-zero otherwise (SURVIVED or NO VERDICT).
|
|
|
|
param(
|
|
[Parameter(Mandatory = $true)] [string[]] $Target,
|
|
[Parameter(Mandatory = $true)] [string] $TestProject,
|
|
[Parameter(Mandatory = $true)] [string] $Entries,
|
|
[string] $ControlFilter,
|
|
[string] $Repo
|
|
)
|
|
|
|
$ErrorActionPreference = 'Stop'
|
|
|
|
$repo = if ($Repo) { (Resolve-Path -LiteralPath $Repo).Path } else { (Resolve-Path (Join-Path $PSScriptRoot '..')).Path }
|
|
$testProjectPath = Join-Path $repo $TestProject
|
|
$utf8 = [System.Text.UTF8Encoding]::new($false)
|
|
|
|
function Read-Bytes([string]$path) { [System.IO.File]::ReadAllBytes($path) }
|
|
function Get-BytesHash([byte[]]$bytes) {
|
|
([System.Security.Cryptography.SHA256]::Create().ComputeHash($bytes) | ForEach-Object { $_.ToString('x2') }) -join ''
|
|
}
|
|
function Write-Text([string]$path, [string]$text) {
|
|
[System.IO.File]::WriteAllBytes($path, [byte[]]($utf8.GetPreamble() + $utf8.GetBytes($text)))
|
|
}
|
|
function Get-SidecarPath([string]$path) { "$path.mutation-original" }
|
|
|
|
function Run-Tests([string]$filter) {
|
|
$args = @('test', $testProjectPath, '-c', 'Debug', '--nologo')
|
|
if ($filter) { $args += @('--filter', $filter) }
|
|
# A test-host crash (e.g. S2-1's disposed-socket race on a background poll thread) can write to
|
|
# stderr; under PowerShell 5.1, `2>&1` on a native command wraps each stderr line in a
|
|
# NativeCommandError, and with the script's own $ErrorActionPreference = 'Stop' that becomes a
|
|
# terminating error that would otherwise throw the whole run away. Run this one call under
|
|
# 'Continue' and catch anything that still throws, so a host crash is captured as output and
|
|
# classified as NO VERDICT by the caller instead of aborting every remaining entry.
|
|
$previousPreference = $ErrorActionPreference
|
|
$ErrorActionPreference = 'Continue'
|
|
$crashed = $false
|
|
try {
|
|
$output = & dotnet @args 2>&1 | Out-String
|
|
}
|
|
catch {
|
|
$output = "$_"
|
|
$crashed = $true
|
|
}
|
|
finally {
|
|
$ErrorActionPreference = $previousPreference
|
|
}
|
|
Write-Host $output
|
|
$passedMatch = [regex]::Match($output, 'Passed:\s+(\d+)')
|
|
$failedMatch = [regex]::Match($output, 'Failed:\s+(\d+)')
|
|
$compiled = -not ($output -match 'error CS\d+')
|
|
$passedLine = $output -match 'Passed!'
|
|
$failedLine = $output -match 'Failed!'
|
|
[pscustomobject]@{
|
|
Compiled = $compiled
|
|
Crashed = $crashed
|
|
Green = (-not $crashed) -and $compiled -and $passedLine -and -not $failedLine -and $passedMatch.Success -and [int]$passedMatch.Groups[1].Value -ge 1
|
|
Passed = if ($passedMatch.Success) { [int]$passedMatch.Groups[1].Value } else { 0 }
|
|
Failed = if ($failedMatch.Success) { [int]$failedMatch.Groups[1].Value } else { 0 }
|
|
}
|
|
}
|
|
|
|
$targets = @{}
|
|
foreach ($file in $Target) {
|
|
$path = Join-Path $repo $file
|
|
if (-not (Test-Path -LiteralPath $path)) { throw "Target $file does not exist." }
|
|
$targets[$file] = $path
|
|
}
|
|
|
|
# --- Repair a run that died mid-mutation, for every target -------------------
|
|
|
|
foreach ($file in $Target) {
|
|
$sidecar = Get-SidecarPath $targets[$file]
|
|
if (Test-Path -LiteralPath $sidecar) {
|
|
$original = Read-Bytes $sidecar
|
|
[System.IO.File]::WriteAllBytes($targets[$file], $original)
|
|
Remove-Item -LiteralPath $sidecar -Force
|
|
Write-Host "Repaired: a previous run left $sidecar; $($targets[$file]) restored from it and the sidecar removed."
|
|
}
|
|
}
|
|
|
|
# --- M0: no-op control, every target -----------------------------------------
|
|
|
|
Write-Host '--- M0: no-op control'
|
|
$originalBytes = @{}
|
|
$originalHash = @{}
|
|
foreach ($file in $Target) {
|
|
$path = $targets[$file]
|
|
$bytes = Read-Bytes $path
|
|
$hash = Get-BytesHash $bytes
|
|
$text = $utf8.GetString($bytes)
|
|
$reencoded = [byte[]]($utf8.GetPreamble() + $utf8.GetBytes($text))
|
|
if ((Get-BytesHash $reencoded) -ne $hash) {
|
|
throw "harness broken: $file does not round-trip through the UTF-8 read/write path byte for byte. Nothing was written and no entry ran."
|
|
}
|
|
$sidecar = Get-SidecarPath $path
|
|
[System.IO.File]::WriteAllBytes($sidecar, $bytes)
|
|
try {
|
|
Write-Text $path $text
|
|
if ((Get-BytesHash (Read-Bytes $path)) -ne $hash) {
|
|
throw "harness broken: writing $file back through the harness path changed its bytes."
|
|
}
|
|
}
|
|
finally {
|
|
[System.IO.File]::WriteAllBytes($path, $bytes)
|
|
if ((Get-BytesHash (Read-Bytes $path)) -ne $hash) { throw "RESTORE FAILED after M0 for $file; original SHA-256 $hash is kept in $sidecar." }
|
|
Remove-Item -LiteralPath $sidecar -Force
|
|
}
|
|
$originalBytes[$file] = $bytes
|
|
$originalHash[$file] = $hash
|
|
Write-Host "M0: $file decoded, re-encoded and rewritten, bytes unchanged (SHA-256 $hash)"
|
|
}
|
|
$control = Run-Tests $ControlFilter
|
|
if (-not $control.Green) {
|
|
throw "harness broken: the suite is not green unmutated (compiled=$($control.Compiled) passed=$($control.Passed) failed=$($control.Failed)). No entry ran."
|
|
}
|
|
Write-Host "M0: green, $($control.Passed) passed"
|
|
|
|
# --- Load entries --------------------------------------------------------------
|
|
|
|
$entriesPath = Join-Path $repo $Entries
|
|
$mutations = & $entriesPath
|
|
if (-not $mutations -or @($mutations).Count -lt 1) { throw "$Entries`: no mutations." }
|
|
|
|
# --- Entries -----------------------------------------------------------------
|
|
|
|
$exitCode = 0
|
|
foreach ($mutation in $mutations) {
|
|
if (-not $mutation.Id) { throw 'An entry has no Id.' }
|
|
if (-not $mutation.Killer) { throw "$($mutation.Id): no Killer." }
|
|
$edits = if ($mutation.ContainsKey('Edits')) { @($mutation.Edits) } else { @(@{ File = $mutation.File; Old = $mutation.Old; New = $mutation.New }) }
|
|
if ($edits.Count -lt 1) { throw "$($mutation.Id): no edits." }
|
|
|
|
Write-Host "--- $($mutation.Id) ($($mutation.Mutant)): $($mutation.Rule)"
|
|
$texts = @{}
|
|
$files = @($edits | ForEach-Object { if ($_.File) { $_.File } elseif ($Target.Count -eq 1) { $Target[0] } else { throw "$($mutation.Id): an edit names no File and there is more than one target." } } | Select-Object -Unique)
|
|
foreach ($file in $files) { $texts[$file] = $utf8.GetString($originalBytes[$file]) }
|
|
|
|
foreach ($edit in $edits) {
|
|
$file = if ($edit.File) { $edit.File } else { $Target[0] }
|
|
$text = $texts[$file]
|
|
$eol = if ($text.Contains("`r`n")) { "`r`n" } else { "`n" }
|
|
$old = ($edit.Old -replace "`r`n", "`n")
|
|
$normalized = $text -replace "`r`n", "`n"
|
|
$count = 0
|
|
$at = $normalized.IndexOf($old, [System.StringComparison]::Ordinal)
|
|
while ($at -ge 0) { $count++; $at = $normalized.IndexOf($old, $at + 1, [System.StringComparison]::Ordinal) }
|
|
if ($count -ne 1) { throw "$($mutation.Id): anchor matches $count times in $file, expected exactly 1." }
|
|
$newValue = if ($null -ne $edit.New) { $edit.New } else { '' }
|
|
$mutatedNormalized = $normalized.Replace($old, ($newValue -replace "`r`n", "`n"))
|
|
$texts[$file] = $mutatedNormalized -replace "`n", $eol
|
|
}
|
|
|
|
foreach ($file in $files) { [System.IO.File]::WriteAllBytes((Get-SidecarPath $targets[$file]), $originalBytes[$file]) }
|
|
$verdict = 'NO VERDICT'
|
|
try {
|
|
foreach ($file in $files) { Write-Text $targets[$file] $texts[$file] }
|
|
$run = Run-Tests $mutation.Killer
|
|
if ($run.Crashed) { $verdict = 'NO VERDICT (test host crashed)' }
|
|
elseif (-not $run.Compiled) { $verdict = 'NO VERDICT (did not compile)' }
|
|
elseif ($run.Failed -ge 1) { $verdict = 'KILLED' }
|
|
else { $verdict = 'SURVIVED' }
|
|
}
|
|
catch {
|
|
# Not yet reached by Run-Tests itself (it no longer throws on a host crash), but a defensive
|
|
# backstop: any other exception during this entry's run is recorded and the campaign continues
|
|
# rather than aborting every remaining entry, never labeled "unreachable".
|
|
Write-Host "NO VERDICT: $($mutation.Id) - test run threw: $_"
|
|
$verdict = 'NO VERDICT (harness exception)'
|
|
}
|
|
finally {
|
|
foreach ($file in $files) {
|
|
[System.IO.File]::WriteAllBytes($targets[$file], $originalBytes[$file])
|
|
if ((Get-BytesHash (Read-Bytes $targets[$file])) -ne $originalHash[$file]) {
|
|
Write-Host "RESTORE FAILED: $($targets[$file]) not restored; original SHA-256 $($originalHash[$file]) kept in $(Get-SidecarPath $targets[$file])."
|
|
throw "restore failed for $($mutation.Id)"
|
|
}
|
|
Remove-Item -LiteralPath (Get-SidecarPath $targets[$file]) -Force
|
|
}
|
|
}
|
|
Write-Host "$($mutation.Id): $verdict"
|
|
if ($verdict -ne 'KILLED') { $exitCode = 1 }
|
|
}
|
|
|
|
$final = Run-Tests $ControlFilter
|
|
if (-not $final.Green) { throw "post-run sanity check: the suite is not green after restore (passed=$($final.Passed) failed=$($final.Failed))." }
|
|
Write-Host "Final: green, $($final.Passed) passed"
|
|
|
|
exit $exitCode
|