CultLib/scripts/mutate-dotnet.ps1
Meta Vens 568e8e356f CultNet selection cut1, commit 2 fix batch: door, shard-via-evaluator, Mesh v0 refusal, dead Reconcile code, mutation harness crash handling
Soul found commit 2 didn't pass (docs/cultnet-selection-cut.md, 2026-09-22):
empty/blank schemas-keys still reached the evaluator, the shard snapshot ran
its own loop instead of the evaluator, Mesh's v0 wire silently dropped v1-only
selection terms, the Mesh one-pass read could return the wrong schema's
record at a shared key, two decode rules for a foreign schema id disagreed,
dead identity-comparison code from the _projectRecord removal was still live,
and a disposed-socket race could abort a mutation run instead of recording
NO VERDICT. This is Self's fix batch for all of it.

- CultNetSelection.Validate now refuses a blank/whitespace entry inside
  schemas or keys, not just an empty array. CultNetSelectionEvaluator no
  longer treats [] as "no filter" (the door already refuses it before
  evaluation). CultNetV0SelectionLowering is the one place that lowers an
  empty or all-blank v0 list to null; CreateRawSnapshotResponse and
  CreateShardSnapshotResponse both route through it, and CultMesh's own
  Clean() now delegates to it instead of duplicating the rule.
- CreateShardSnapshotResponse no longer carries a private loop or its own
  schema/key matching: it builds a CultNetSelection and answers through
  CultNetDocumentRegistry.SelectPage (the same evaluator path as the
  non-shard snapshot), with the shard's own membership check as a row
  filter. Binding-alias expansion now runs on both paths identically.
- ResolveDefaultSelection decides "no schema filter" from the cleaned
  recordKeys, not the raw array length. OverlaySelection stops deciding what
  a missing override means: any explicit schemaIds/recordKeys override
  replaces the default selection wholesale; no override keeps the default
  untouched.
- CultMeshSnapshotRequestOptions.Selection is now checked by
  EnsureV0Compatible before every send: fields, cites, cited, limit, cursor,
  descending, and any non-default projection are refused typed
  (CultNetSelectionInvalidException), never silently dropped from the v0
  wire message. Mesh sending v1 is FU-Mesh-v1, not this cut.
- CultMeshHotBodySubscription.Selection returns a defensive copy on every
  read so its arrays can't be mutated out from under the stored selection.
- ReadDocumentFromSnapshotResponse now searches candidates at one recordKey
  in one order - exact schema, then alias, then a foreign schema id decoded
  from the payload - instead of one OR'd condition where a case-insensitive
  or structurally-lucky candidate could win first. The foreign-id decode
  reads the payload's embedded schema version with the registry's shared
  CultNetDocumentRegistry.TryReadSchemaVersion and alias-matches it against
  the caller's own descriptor directly, rather than resolving to "whichever
  registered type owns this schema string" - that resolution is ambiguous
  when two document types alias the same schema id (MeshNoteDocument and
  MeshNoteAliasDocument in the existing fixtures), which is what broke on
  the first pass; TryResolveDescriptorByPayloadSchema stays for its existing
  caller (ResolveDescriptorForRawDocument) with a note on when not to use it.
  DecodeSnapshotDocuments and the Mesh copy of the payload-schema decode
  (RawSnapshotPayloadMatchesSchema/TryReadSchemaVersion) are deleted in
  favour of the shared one.
- CultNetDatabaseSubscriptionServer: the dead RecordKey/SchemaId identity
  comparisons in Reconcile and ApplyProjectedChange, and the duplicate-key
  throw in CreateProjectedSnapshot, are deleted - they can no longer fire
  now that a projection's dictionary key is always the record's own key.
  A new test flips authorizeRecord per key while the whole request stays
  authorized (so DemandActive never toggles) and proves both halves of the
  remaining diff: the removal of a now-unauthorized key and the add loop
  picking up a newly-authorized one.
- Every NetworkingTests.cs test that starts an IsBackground poll thread now
  joins it before its server disposes (about 20 tests; one already did).
- scripts/mutate-dotnet.ps1: a dotnet-test-host crash writing to stderr no
  longer throws the whole run away under PowerShell 5.1's
  $ErrorActionPreference = 'Stop' (2>&1 on a native command wraps stderr in
  a terminating NativeCommandError); Run-Tests catches it and the entry loop
  records NO VERDICT instead of aborting every remaining entry.
- Two probe tests: CultNetDatabase_ShardAndNonShardSnapshot_Agree{On
  UnfilteredRequest,OnBindingSchemaIdOverride} prove the shard and non-shard
  snapshot paths agree, including shard-membership exclusion of a document
  outside the shard's key prefix and a binding schema-id override matching
  on both paths.
- Mutation entries updated/added to match: the stale NET-D4-FourthMatcher
  entry (its anchor was the private loop this batch deleted, not refactored)
  is replaced by NET-D4-ShardMembership-Revert; the Mesh entries file's
  header miscount ("two of three" loosening entries when there are only
  two) is corrected and both previously-surviving loosening mutants
  (MESH-DEFAULT-Loosening, MESH-KEY-Loosening) now have killing fixtures;
  scripts/mutate-cultnet-selection-fixbatch2.entries.ps1 is new, covering
  Reconcile's remaining two mutants.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-22 06:13:15 +02:00

221 lines
11 KiB
PowerShell

# Generic mutation runner for dotnet/NUnit suites - the CultNet selection campaign's counterpart to
# tools/eureka-mutations.ps1, whose Invoke-Cargo parses `cargo test` output and appends `-- --exact
# <test>`, and is therefore not usable against `dotnet test`. This is not scripts/mutate-cultmesh.mjs
# either - that file is the QUIC campaign's, live and dirty in this tree, and per
# docs/cultnet-selection-cut.md section 10 this campaign adds its entries in its own file rather than
# editing the QUIC campaign's targets.
#
# One copy serves every dotnet target in this campaign (and, once proven out, any future one): pass
# -Target, -TestProject and -Entries. Same discipline as the cargo harness: every entry is a byte-exact
# splice against one anchor that must match exactly once, restored from the original bytes (never from
# git) with a SHA-256 check per target, a sidecar (<target>.mutation-original) protects a hard kill
# mid-mutation, and M0 proves the read/write path round-trips and the unmutated suite is green before
# any entry runs.
#
# powershell -File scripts/mutate-dotnet.ps1 `
# -Target src\GameCult.Caching\CultCache.cs `
# -TestProject tests\GameCult.Caching.Tests\GameCult.Caching.Tests.csproj `
# -Entries scripts\mutate-cultnet-selection-caching.entries.ps1 `
# -ControlFilter 'FullyQualifiedName~CultDocumentSelectionSurfaceTests'
#
# -Target one or more files entries mutate, relative to the repo root. An entry with no `File`
# uses the sole target; an entry with more than one target names each edit's `File`.
# -TestProject the .csproj `dotnet test` runs. One project per invocation; entries needing another
# project are a separate run (as networking and caching already are).
# -Entries a .ps1 that returns an array of `@{ Id; Rule; Mutant; Killer; Old; New; File? }` (or
# `Edits = @(@{ File; Old; New }, ...)` for a multi-file entry). `Killer` is the
# `dotnet test --filter` value for the one test that must fail under this mutant.
# -ControlFilter the `dotnet test --filter` M0 runs to prove the suite is green unmutated. Defaults to
# running the whole project.
# -Repo repo root -Target/-TestProject/-Entries are relative to. Defaults to this script's
# parent's parent (scripts/..).
#
# Exit code: 0 when every entry's mutant was KILLED, non-zero otherwise (SURVIVED or NO VERDICT).
param(
[Parameter(Mandatory = $true)] [string[]] $Target,
[Parameter(Mandatory = $true)] [string] $TestProject,
[Parameter(Mandatory = $true)] [string] $Entries,
[string] $ControlFilter,
[string] $Repo
)
$ErrorActionPreference = 'Stop'
$repo = if ($Repo) { (Resolve-Path -LiteralPath $Repo).Path } else { (Resolve-Path (Join-Path $PSScriptRoot '..')).Path }
$testProjectPath = Join-Path $repo $TestProject
$utf8 = [System.Text.UTF8Encoding]::new($false)
function Read-Bytes([string]$path) { [System.IO.File]::ReadAllBytes($path) }
function Get-BytesHash([byte[]]$bytes) {
([System.Security.Cryptography.SHA256]::Create().ComputeHash($bytes) | ForEach-Object { $_.ToString('x2') }) -join ''
}
function Write-Text([string]$path, [string]$text) {
[System.IO.File]::WriteAllBytes($path, [byte[]]($utf8.GetPreamble() + $utf8.GetBytes($text)))
}
function Get-SidecarPath([string]$path) { "$path.mutation-original" }
function Run-Tests([string]$filter) {
$args = @('test', $testProjectPath, '-c', 'Debug', '--nologo')
if ($filter) { $args += @('--filter', $filter) }
# A test-host crash (e.g. S2-1's disposed-socket race on a background poll thread) can write to
# stderr; under PowerShell 5.1, `2>&1` on a native command wraps each stderr line in a
# NativeCommandError, and with the script's own $ErrorActionPreference = 'Stop' that becomes a
# terminating error that would otherwise throw the whole run away. Run this one call under
# 'Continue' and catch anything that still throws, so a host crash is captured as output and
# classified as NO VERDICT by the caller instead of aborting every remaining entry.
$previousPreference = $ErrorActionPreference
$ErrorActionPreference = 'Continue'
$crashed = $false
try {
$output = & dotnet @args 2>&1 | Out-String
}
catch {
$output = "$_"
$crashed = $true
}
finally {
$ErrorActionPreference = $previousPreference
}
Write-Host $output
$passedMatch = [regex]::Match($output, 'Passed:\s+(\d+)')
$failedMatch = [regex]::Match($output, 'Failed:\s+(\d+)')
$compiled = -not ($output -match 'error CS\d+')
$passedLine = $output -match 'Passed!'
$failedLine = $output -match 'Failed!'
[pscustomobject]@{
Compiled = $compiled
Crashed = $crashed
Green = (-not $crashed) -and $compiled -and $passedLine -and -not $failedLine -and $passedMatch.Success -and [int]$passedMatch.Groups[1].Value -ge 1
Passed = if ($passedMatch.Success) { [int]$passedMatch.Groups[1].Value } else { 0 }
Failed = if ($failedMatch.Success) { [int]$failedMatch.Groups[1].Value } else { 0 }
}
}
$targets = @{}
foreach ($file in $Target) {
$path = Join-Path $repo $file
if (-not (Test-Path -LiteralPath $path)) { throw "Target $file does not exist." }
$targets[$file] = $path
}
# --- Repair a run that died mid-mutation, for every target -------------------
foreach ($file in $Target) {
$sidecar = Get-SidecarPath $targets[$file]
if (Test-Path -LiteralPath $sidecar) {
$original = Read-Bytes $sidecar
[System.IO.File]::WriteAllBytes($targets[$file], $original)
Remove-Item -LiteralPath $sidecar -Force
Write-Host "Repaired: a previous run left $sidecar; $($targets[$file]) restored from it and the sidecar removed."
}
}
# --- M0: no-op control, every target -----------------------------------------
Write-Host '--- M0: no-op control'
$originalBytes = @{}
$originalHash = @{}
foreach ($file in $Target) {
$path = $targets[$file]
$bytes = Read-Bytes $path
$hash = Get-BytesHash $bytes
$text = $utf8.GetString($bytes)
$reencoded = [byte[]]($utf8.GetPreamble() + $utf8.GetBytes($text))
if ((Get-BytesHash $reencoded) -ne $hash) {
throw "harness broken: $file does not round-trip through the UTF-8 read/write path byte for byte. Nothing was written and no entry ran."
}
$sidecar = Get-SidecarPath $path
[System.IO.File]::WriteAllBytes($sidecar, $bytes)
try {
Write-Text $path $text
if ((Get-BytesHash (Read-Bytes $path)) -ne $hash) {
throw "harness broken: writing $file back through the harness path changed its bytes."
}
}
finally {
[System.IO.File]::WriteAllBytes($path, $bytes)
if ((Get-BytesHash (Read-Bytes $path)) -ne $hash) { throw "RESTORE FAILED after M0 for $file; original SHA-256 $hash is kept in $sidecar." }
Remove-Item -LiteralPath $sidecar -Force
}
$originalBytes[$file] = $bytes
$originalHash[$file] = $hash
Write-Host "M0: $file decoded, re-encoded and rewritten, bytes unchanged (SHA-256 $hash)"
}
$control = Run-Tests $ControlFilter
if (-not $control.Green) {
throw "harness broken: the suite is not green unmutated (compiled=$($control.Compiled) passed=$($control.Passed) failed=$($control.Failed)). No entry ran."
}
Write-Host "M0: green, $($control.Passed) passed"
# --- Load entries --------------------------------------------------------------
$entriesPath = Join-Path $repo $Entries
$mutations = & $entriesPath
if (-not $mutations -or @($mutations).Count -lt 1) { throw "$Entries`: no mutations." }
# --- Entries -----------------------------------------------------------------
$exitCode = 0
foreach ($mutation in $mutations) {
if (-not $mutation.Id) { throw 'An entry has no Id.' }
if (-not $mutation.Killer) { throw "$($mutation.Id): no Killer." }
$edits = if ($mutation.ContainsKey('Edits')) { @($mutation.Edits) } else { @(@{ File = $mutation.File; Old = $mutation.Old; New = $mutation.New }) }
if ($edits.Count -lt 1) { throw "$($mutation.Id): no edits." }
Write-Host "--- $($mutation.Id) ($($mutation.Mutant)): $($mutation.Rule)"
$texts = @{}
$files = @($edits | ForEach-Object { if ($_.File) { $_.File } elseif ($Target.Count -eq 1) { $Target[0] } else { throw "$($mutation.Id): an edit names no File and there is more than one target." } } | Select-Object -Unique)
foreach ($file in $files) { $texts[$file] = $utf8.GetString($originalBytes[$file]) }
foreach ($edit in $edits) {
$file = if ($edit.File) { $edit.File } else { $Target[0] }
$text = $texts[$file]
$eol = if ($text.Contains("`r`n")) { "`r`n" } else { "`n" }
$old = ($edit.Old -replace "`r`n", "`n")
$normalized = $text -replace "`r`n", "`n"
$count = 0
$at = $normalized.IndexOf($old, [System.StringComparison]::Ordinal)
while ($at -ge 0) { $count++; $at = $normalized.IndexOf($old, $at + 1, [System.StringComparison]::Ordinal) }
if ($count -ne 1) { throw "$($mutation.Id): anchor matches $count times in $file, expected exactly 1." }
$newValue = if ($null -ne $edit.New) { $edit.New } else { '' }
$mutatedNormalized = $normalized.Replace($old, ($newValue -replace "`r`n", "`n"))
$texts[$file] = $mutatedNormalized -replace "`n", $eol
}
foreach ($file in $files) { [System.IO.File]::WriteAllBytes((Get-SidecarPath $targets[$file]), $originalBytes[$file]) }
$verdict = 'NO VERDICT'
try {
foreach ($file in $files) { Write-Text $targets[$file] $texts[$file] }
$run = Run-Tests $mutation.Killer
if ($run.Crashed) { $verdict = 'NO VERDICT (test host crashed)' }
elseif (-not $run.Compiled) { $verdict = 'NO VERDICT (did not compile)' }
elseif ($run.Failed -ge 1) { $verdict = 'KILLED' }
else { $verdict = 'SURVIVED' }
}
catch {
# Not yet reached by Run-Tests itself (it no longer throws on a host crash), but a defensive
# backstop: any other exception during this entry's run is recorded and the campaign continues
# rather than aborting every remaining entry, never labeled "unreachable".
Write-Host "NO VERDICT: $($mutation.Id) - test run threw: $_"
$verdict = 'NO VERDICT (harness exception)'
}
finally {
foreach ($file in $files) {
[System.IO.File]::WriteAllBytes($targets[$file], $originalBytes[$file])
if ((Get-BytesHash (Read-Bytes $targets[$file])) -ne $originalHash[$file]) {
Write-Host "RESTORE FAILED: $($targets[$file]) not restored; original SHA-256 $($originalHash[$file]) kept in $(Get-SidecarPath $targets[$file])."
throw "restore failed for $($mutation.Id)"
}
Remove-Item -LiteralPath (Get-SidecarPath $targets[$file]) -Force
}
}
Write-Host "$($mutation.Id): $verdict"
if ($verdict -ne 'KILLED') { $exitCode = 1 }
}
$final = Run-Tests $ControlFilter
if (-not $final.Green) { throw "post-run sanity check: the suite is not green after restore (passed=$($final.Passed) failed=$($final.Failed))." }
Write-Host "Final: green, $($final.Passed) passed"
exit $exitCode