CultLib/scripts/mutate-dotnet.ps1
Meta Vens c01bc0a129 CultNet selection cut1 fix batch: R-L (contract, Python, mutate-dotnet.ps1)
- cultnet.selection.schema.json: schemas/keys gain minItems: 1 (present
  and empty is refused at the door, never "every schema/key" - omit the
  field instead); limit documents that it is clamped 1..=200 by the
  answering server rather than refused outside that range.
- cultnet_py/interop_peer.py: handle_server_message refused only an
  exact ".v1" suffix, so a v2+ selection message fell through to a
  silent empty-list answer instead of a loud refusal. is_unsupported_cultnet_version
  matches any cultnet.*.vN with N > 0 instead. test_cultnet.py's R-2
  test now covers v2 and v10, not only v1.
- mutate-dotnet.ps1: an entry's File (from -Entries, often written with
  "/") is looked up against the $targets dictionary keyed by -Target's
  own spelling; a "\\" vs "/" mismatch didn't throw at the lookup -
  $targets[$file] silently returned $null - so the crash surfaced one
  call later as a null-argument error inside a .NET path API instead of
  as "unknown target". Both -Target and every File read from an entry
  now normalise through Normalize-TargetPath before indexing $targets.

Discrepancy: the ruling names "both R-2 sites" for the Python fix; only
one site with this exact narrow-match bug exists in packages/cultnet-py
and packages/cultmesh-py (cultmesh_py/server.py's handle_message already
refuses every unrecognised schemaVersion, v1+ included, through its own
default branch). Fixed the one that exists; did not invent a second.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-22 09:39:53 +02:00

230 lines
12 KiB
PowerShell

# Generic mutation runner for dotnet/NUnit suites - the CultNet selection campaign's counterpart to
# tools/eureka-mutations.ps1, whose Invoke-Cargo parses `cargo test` output and appends `-- --exact
# <test>`, and is therefore not usable against `dotnet test`. This is not scripts/mutate-cultmesh.mjs
# either - that file is the QUIC campaign's, live and dirty in this tree, and per
# docs/cultnet-selection-cut.md section 10 this campaign adds its entries in its own file rather than
# editing the QUIC campaign's targets.
#
# One copy serves every dotnet target in this campaign (and, once proven out, any future one): pass
# -Target, -TestProject and -Entries. Same discipline as the cargo harness: every entry is a byte-exact
# splice against one anchor that must match exactly once, restored from the original bytes (never from
# git) with a SHA-256 check per target, a sidecar (<target>.mutation-original) protects a hard kill
# mid-mutation, and M0 proves the read/write path round-trips and the unmutated suite is green before
# any entry runs.
#
# powershell -File scripts/mutate-dotnet.ps1 `
# -Target src\GameCult.Caching\CultCache.cs `
# -TestProject tests\GameCult.Caching.Tests\GameCult.Caching.Tests.csproj `
# -Entries scripts\mutate-cultnet-selection-caching.entries.ps1 `
# -ControlFilter 'FullyQualifiedName~CultDocumentSelectionSurfaceTests'
#
# -Target one or more files entries mutate, relative to the repo root. An entry with no `File`
# uses the sole target; an entry with more than one target names each edit's `File`.
# -TestProject the .csproj `dotnet test` runs. One project per invocation; entries needing another
# project are a separate run (as networking and caching already are).
# -Entries a .ps1 that returns an array of `@{ Id; Rule; Mutant; Killer; Old; New; File? }` (or
# `Edits = @(@{ File; Old; New }, ...)` for a multi-file entry). `Killer` is the
# `dotnet test --filter` value for the one test that must fail under this mutant.
# -ControlFilter the `dotnet test --filter` M0 runs to prove the suite is green unmutated. Defaults to
# running the whole project.
# -Repo repo root -Target/-TestProject/-Entries are relative to. Defaults to this script's
# parent's parent (scripts/..).
#
# Exit code: 0 when every entry's mutant was KILLED, non-zero otherwise (SURVIVED or NO VERDICT).
param(
[Parameter(Mandatory = $true)] [string[]] $Target,
[Parameter(Mandatory = $true)] [string] $TestProject,
[Parameter(Mandatory = $true)] [string] $Entries,
[string] $ControlFilter,
[string] $Repo
)
$ErrorActionPreference = 'Stop'
# R-L: an entry's `File` (from -Entries, often written with forward slashes) is looked up against the
# $targets dictionary keyed by -Target's own spelling. A bare string-key mismatch ("a\b.cs" vs
# "a/b.cs") doesn't throw at the lookup - $targets[$file] just silently returns $null - so the crash
# happens one call later, deep inside a .NET path/string API, as a null-argument error that reads
# nothing like "unknown target". Normalizing both sides to '/' up front turns that into either a
# clean match or an honest "Target ... does not exist" at line 99, never a null-argument surprise.
function Normalize-TargetPath([string]$path) { $path -replace '\\', '/' }
$Target = @($Target | ForEach-Object { Normalize-TargetPath $_ })
$repo = if ($Repo) { (Resolve-Path -LiteralPath $Repo).Path } else { (Resolve-Path (Join-Path $PSScriptRoot '..')).Path }
$testProjectPath = Join-Path $repo $TestProject
$utf8 = [System.Text.UTF8Encoding]::new($false)
function Read-Bytes([string]$path) { [System.IO.File]::ReadAllBytes($path) }
function Get-BytesHash([byte[]]$bytes) {
([System.Security.Cryptography.SHA256]::Create().ComputeHash($bytes) | ForEach-Object { $_.ToString('x2') }) -join ''
}
function Write-Text([string]$path, [string]$text) {
[System.IO.File]::WriteAllBytes($path, [byte[]]($utf8.GetPreamble() + $utf8.GetBytes($text)))
}
function Get-SidecarPath([string]$path) { "$path.mutation-original" }
function Run-Tests([string]$filter) {
$args = @('test', $testProjectPath, '-c', 'Debug', '--nologo')
if ($filter) { $args += @('--filter', $filter) }
# A test-host crash (e.g. S2-1's disposed-socket race on a background poll thread) can write to
# stderr; under PowerShell 5.1, `2>&1` on a native command wraps each stderr line in a
# NativeCommandError, and with the script's own $ErrorActionPreference = 'Stop' that becomes a
# terminating error that would otherwise throw the whole run away. Run this one call under
# 'Continue' and catch anything that still throws, so a host crash is captured as output and
# classified as NO VERDICT by the caller instead of aborting every remaining entry.
$previousPreference = $ErrorActionPreference
$ErrorActionPreference = 'Continue'
$crashed = $false
try {
$output = & dotnet @args 2>&1 | Out-String
}
catch {
$output = "$_"
$crashed = $true
}
finally {
$ErrorActionPreference = $previousPreference
}
Write-Host $output
$passedMatch = [regex]::Match($output, 'Passed:\s+(\d+)')
$failedMatch = [regex]::Match($output, 'Failed:\s+(\d+)')
$compiled = -not ($output -match 'error CS\d+')
$passedLine = $output -match 'Passed!'
$failedLine = $output -match 'Failed!'
[pscustomobject]@{
Compiled = $compiled
Crashed = $crashed
Green = (-not $crashed) -and $compiled -and $passedLine -and -not $failedLine -and $passedMatch.Success -and [int]$passedMatch.Groups[1].Value -ge 1
Passed = if ($passedMatch.Success) { [int]$passedMatch.Groups[1].Value } else { 0 }
Failed = if ($failedMatch.Success) { [int]$failedMatch.Groups[1].Value } else { 0 }
}
}
$targets = @{}
foreach ($file in $Target) {
$path = Join-Path $repo $file
if (-not (Test-Path -LiteralPath $path)) { throw "Target $file does not exist." }
$targets[$file] = $path
}
# --- Repair a run that died mid-mutation, for every target -------------------
foreach ($file in $Target) {
$sidecar = Get-SidecarPath $targets[$file]
if (Test-Path -LiteralPath $sidecar) {
$original = Read-Bytes $sidecar
[System.IO.File]::WriteAllBytes($targets[$file], $original)
Remove-Item -LiteralPath $sidecar -Force
Write-Host "Repaired: a previous run left $sidecar; $($targets[$file]) restored from it and the sidecar removed."
}
}
# --- M0: no-op control, every target -----------------------------------------
Write-Host '--- M0: no-op control'
$originalBytes = @{}
$originalHash = @{}
foreach ($file in $Target) {
$path = $targets[$file]
$bytes = Read-Bytes $path
$hash = Get-BytesHash $bytes
$text = $utf8.GetString($bytes)
$reencoded = [byte[]]($utf8.GetPreamble() + $utf8.GetBytes($text))
if ((Get-BytesHash $reencoded) -ne $hash) {
throw "harness broken: $file does not round-trip through the UTF-8 read/write path byte for byte. Nothing was written and no entry ran."
}
$sidecar = Get-SidecarPath $path
[System.IO.File]::WriteAllBytes($sidecar, $bytes)
try {
Write-Text $path $text
if ((Get-BytesHash (Read-Bytes $path)) -ne $hash) {
throw "harness broken: writing $file back through the harness path changed its bytes."
}
}
finally {
[System.IO.File]::WriteAllBytes($path, $bytes)
if ((Get-BytesHash (Read-Bytes $path)) -ne $hash) { throw "RESTORE FAILED after M0 for $file; original SHA-256 $hash is kept in $sidecar." }
Remove-Item -LiteralPath $sidecar -Force
}
$originalBytes[$file] = $bytes
$originalHash[$file] = $hash
Write-Host "M0: $file decoded, re-encoded and rewritten, bytes unchanged (SHA-256 $hash)"
}
$control = Run-Tests $ControlFilter
if (-not $control.Green) {
throw "harness broken: the suite is not green unmutated (compiled=$($control.Compiled) passed=$($control.Passed) failed=$($control.Failed)). No entry ran."
}
Write-Host "M0: green, $($control.Passed) passed"
# --- Load entries --------------------------------------------------------------
$entriesPath = Join-Path $repo $Entries
$mutations = & $entriesPath
if (-not $mutations -or @($mutations).Count -lt 1) { throw "$Entries`: no mutations." }
# --- Entries -----------------------------------------------------------------
$exitCode = 0
foreach ($mutation in $mutations) {
if (-not $mutation.Id) { throw 'An entry has no Id.' }
if (-not $mutation.Killer) { throw "$($mutation.Id): no Killer." }
$edits = if ($mutation.ContainsKey('Edits')) { @($mutation.Edits) } else { @(@{ File = $mutation.File; Old = $mutation.Old; New = $mutation.New }) }
if ($edits.Count -lt 1) { throw "$($mutation.Id): no edits." }
Write-Host "--- $($mutation.Id) ($($mutation.Mutant)): $($mutation.Rule)"
$texts = @{}
$files = @($edits | ForEach-Object { if ($_.File) { Normalize-TargetPath $_.File } elseif ($Target.Count -eq 1) { $Target[0] } else { throw "$($mutation.Id): an edit names no File and there is more than one target." } } | Select-Object -Unique)
foreach ($file in $files) { $texts[$file] = $utf8.GetString($originalBytes[$file]) }
foreach ($edit in $edits) {
$file = if ($edit.File) { Normalize-TargetPath $edit.File } else { $Target[0] }
$text = $texts[$file]
$eol = if ($text.Contains("`r`n")) { "`r`n" } else { "`n" }
$old = ($edit.Old -replace "`r`n", "`n")
$normalized = $text -replace "`r`n", "`n"
$count = 0
$at = $normalized.IndexOf($old, [System.StringComparison]::Ordinal)
while ($at -ge 0) { $count++; $at = $normalized.IndexOf($old, $at + 1, [System.StringComparison]::Ordinal) }
if ($count -ne 1) { throw "$($mutation.Id): anchor matches $count times in $file, expected exactly 1." }
$newValue = if ($null -ne $edit.New) { $edit.New } else { '' }
$mutatedNormalized = $normalized.Replace($old, ($newValue -replace "`r`n", "`n"))
$texts[$file] = $mutatedNormalized -replace "`n", $eol
}
foreach ($file in $files) { [System.IO.File]::WriteAllBytes((Get-SidecarPath $targets[$file]), $originalBytes[$file]) }
$verdict = 'NO VERDICT'
try {
foreach ($file in $files) { Write-Text $targets[$file] $texts[$file] }
$run = Run-Tests $mutation.Killer
if ($run.Crashed) { $verdict = 'NO VERDICT (test host crashed)' }
elseif (-not $run.Compiled) { $verdict = 'NO VERDICT (did not compile)' }
elseif ($run.Failed -ge 1) { $verdict = 'KILLED' }
else { $verdict = 'SURVIVED' }
}
catch {
# Not yet reached by Run-Tests itself (it no longer throws on a host crash), but a defensive
# backstop: any other exception during this entry's run is recorded and the campaign continues
# rather than aborting every remaining entry, never labeled "unreachable".
Write-Host "NO VERDICT: $($mutation.Id) - test run threw: $_"
$verdict = 'NO VERDICT (harness exception)'
}
finally {
foreach ($file in $files) {
[System.IO.File]::WriteAllBytes($targets[$file], $originalBytes[$file])
if ((Get-BytesHash (Read-Bytes $targets[$file])) -ne $originalHash[$file]) {
Write-Host "RESTORE FAILED: $($targets[$file]) not restored; original SHA-256 $($originalHash[$file]) kept in $(Get-SidecarPath $targets[$file])."
throw "restore failed for $($mutation.Id)"
}
Remove-Item -LiteralPath (Get-SidecarPath $targets[$file]) -Force
}
}
Write-Host "$($mutation.Id): $verdict"
if ($verdict -ne 'KILLED') { $exitCode = 1 }
}
$final = Run-Tests $ControlFilter
if (-not $final.Green) { throw "post-run sanity check: the suite is not green after restore (passed=$($final.Passed) failed=$($final.Failed))." }
Write-Host "Final: green, $($final.Passed) passed"
exit $exitCode