docs/cultnet-selection-cut.md section 2/4/6/7/8: CultNetSelection (schemas, keys, fields, cites, cited, projection, descending, limit, cursor) with its door validation (S1), and CultNetSelectionEvaluator: the single-row Matches fast path (D6) and the full Select (order, hop, cursor, paging). Neither reflects over a document; both read only through the cache's public surface (commit 0). Three v1 messages beside v0 (cultnet.snapshot_request.v1, cultnet.database_subscribe.v1, cultnet.snapshot_response_raw.v1), their schema files under contracts/cultnet, registry entries and dispatch arms. CultNetSchemaAliasMatching goes internal -> public (D4), the one alias matcher. CultNetDatabase gains LastWriteSequence(schemaId, key), confirmed at the one append site (AppendMutationLogEntry) that every committed write takes a shard-log sequence, including a local PutAsync outside any shard. Three engines deleted, one evaluator wired in their place: - CultNetDocumentRegistry: CreateRawSnapshotResponse's reflection loop over cache.AllEntries and MatchesRequestedSchema gone; cache.AllStoredDocuments replaces the reflection entirely. v0 lowers into a Selection and walks the evaluator's own pages so a v0 caller still sees its whole matching set (D3: v0 has no engine of its own). CreateSelectionResponse/SelectPage/ ToRawRecord/CreateSnapshotRequestV1 added. - CultNetDatabaseServer: CreateChangeMessage/CreateRawDocumentRecord/Matches/ MatchesSchema/InferSchemaName (122 lines) deleted; CreateSubscription lowers v0 into a Selection and matches through the evaluator. - CultNetDatabaseSubscriptionServer: CreateAuthorizedChange/CreateChange/ CreateChangeCore/ResolveWireSchemaId/MatchesRequestedSchema/CreateRawRecord (92 lines) deleted, replaced by CreateMatchedRecord. _projectRecord and CreateProjectedSnapshot's collapse are commit 2's, per the map's own deletion line, and are untouched here. Discrepancy found and fixed (not in the map): CultNetDocumentBinding can carry a schema id different from its descriptor's own (CultNetDocumentBinding.ForDocument's schemaId override), which the old per-engine matchers checked and a descriptor-only evaluator cannot see by design (it stays decoupled from CultNetDocumentRegistry). Fixed at the two call sites and in CultNetDocumentRegistry.SelectPage by resolving the binding's schema id into the effective selection before evaluation (CultNetSchemaAliasMatching.WithBindingSchemaAlias, CultNetDocumentRegistry.ExpandSchemaBindingAliases), not in the evaluator. Found by DatabaseSubscription_FiltersLiveChangesByWireSchemaBinding timing out; both call sites now pass. Second discrepancy, reported not fixed: CultNetDatabase.cs:1707 has a fourth, undocumented copy of schema-matching logic (MatchesRequestedSchema, private, serving CreateShardSnapshotResponse), which the map's section 1 audit did not name among "three separate selector engines". Left untouched this commit: it is out of the map's explicit deletes list and touching it risks shard snapshot behavior with no budgeted verification. Verification: dotnet test tests/GameCult.Networking.Tests - 162 passed (150 existing + 12 new), 0 failed; the seven map-named tests all still pass, two rewritten for the new signatures (CreateChangeMessage now takes CultNetSelection; CreateChange/CreateAuthorizedChange collapsed into CreateMatchedRecord, returning CultNetRawDocumentRecord? directly). tests/GameCult.Mesh.Tests - 252 passed, 1 pre-existing skip, 0 failed (no Mesh source touched this commit). tests/GameCult.Caching.Tests - unaffected (188, unchanged from commit 0). Negative greps: GetProperty|GetField|GetCustomAttribute in GameCult.Networking - only against the generic `object change` dispatch (CultNetDatabaseChange<T>) and CultRecordHandle<T> construction, pre-existing and out of this cut's scope, never against a document's declared values. _projectRecord survives in CultNetDatabaseSubscriptionServer.cs (expected, commit 2). CultNetSchemaAliasMatching is the only schema matcher outside the one reported discrepancy. scripts/mutate-dotnet.ps1: the caching mutation runner generalized into a shared dotnet/NUnit harness (Target/TestProject/Entries/ControlFilter), replacing scripts/mutate-cultnet-selection-caching.ps1 with an entries file of the same six mutations (re-verified: all six still KILL). It is now generic to any NUnit project in this repo, not only this campaign's two targets - a candidate to sit beside tools/eureka-mutations.ps1 as the dotnet-side counterpart. Nine new networking entries (S3, S5 x2, S7 x2, S16 x2, S18) all KILLED. Not yet mutation-run: S1, S2, S6, S23 (pinned by tests, not by a committed mutant); S9-S15 (subscription reconcile lowering, v0 no-engine-of-its-own); S17, S20 (pinned by tests only). S19/S21/S22 are cache-layer, covered in commit 0. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
194 lines
9.7 KiB
PowerShell
194 lines
9.7 KiB
PowerShell
# Generic mutation runner for dotnet/NUnit suites - the CultNet selection campaign's counterpart to
|
|
# tools/eureka-mutations.ps1, whose Invoke-Cargo parses `cargo test` output and appends `-- --exact
|
|
# <test>`, and is therefore not usable against `dotnet test`. This is not scripts/mutate-cultmesh.mjs
|
|
# either - that file is the QUIC campaign's, live and dirty in this tree, and per
|
|
# docs/cultnet-selection-cut.md section 10 this campaign adds its entries in its own file rather than
|
|
# editing the QUIC campaign's targets.
|
|
#
|
|
# One copy serves every dotnet target in this campaign (and, once proven out, any future one): pass
|
|
# -Target, -TestProject and -Entries. Same discipline as the cargo harness: every entry is a byte-exact
|
|
# splice against one anchor that must match exactly once, restored from the original bytes (never from
|
|
# git) with a SHA-256 check per target, a sidecar (<target>.mutation-original) protects a hard kill
|
|
# mid-mutation, and M0 proves the read/write path round-trips and the unmutated suite is green before
|
|
# any entry runs.
|
|
#
|
|
# powershell -File scripts/mutate-dotnet.ps1 `
|
|
# -Target src\GameCult.Caching\CultCache.cs `
|
|
# -TestProject tests\GameCult.Caching.Tests\GameCult.Caching.Tests.csproj `
|
|
# -Entries scripts\mutate-cultnet-selection-caching.entries.ps1 `
|
|
# -ControlFilter 'FullyQualifiedName~CultDocumentSelectionSurfaceTests'
|
|
#
|
|
# -Target one or more files entries mutate, relative to the repo root. An entry with no `File`
|
|
# uses the sole target; an entry with more than one target names each edit's `File`.
|
|
# -TestProject the .csproj `dotnet test` runs. One project per invocation; entries needing another
|
|
# project are a separate run (as networking and caching already are).
|
|
# -Entries a .ps1 that returns an array of `@{ Id; Rule; Mutant; Killer; Old; New; File? }` (or
|
|
# `Edits = @(@{ File; Old; New }, ...)` for a multi-file entry). `Killer` is the
|
|
# `dotnet test --filter` value for the one test that must fail under this mutant.
|
|
# -ControlFilter the `dotnet test --filter` M0 runs to prove the suite is green unmutated. Defaults to
|
|
# running the whole project.
|
|
# -Repo repo root -Target/-TestProject/-Entries are relative to. Defaults to this script's
|
|
# parent's parent (scripts/..).
|
|
#
|
|
# Exit code: 0 when every entry's mutant was KILLED, non-zero otherwise (SURVIVED or NO VERDICT).
|
|
|
|
param(
|
|
[Parameter(Mandatory = $true)] [string[]] $Target,
|
|
[Parameter(Mandatory = $true)] [string] $TestProject,
|
|
[Parameter(Mandatory = $true)] [string] $Entries,
|
|
[string] $ControlFilter,
|
|
[string] $Repo
|
|
)
|
|
|
|
$ErrorActionPreference = 'Stop'
|
|
|
|
$repo = if ($Repo) { (Resolve-Path -LiteralPath $Repo).Path } else { (Resolve-Path (Join-Path $PSScriptRoot '..')).Path }
|
|
$testProjectPath = Join-Path $repo $TestProject
|
|
$utf8 = [System.Text.UTF8Encoding]::new($false)
|
|
|
|
function Read-Bytes([string]$path) { [System.IO.File]::ReadAllBytes($path) }
|
|
function Get-BytesHash([byte[]]$bytes) {
|
|
([System.Security.Cryptography.SHA256]::Create().ComputeHash($bytes) | ForEach-Object { $_.ToString('x2') }) -join ''
|
|
}
|
|
function Write-Text([string]$path, [string]$text) {
|
|
[System.IO.File]::WriteAllBytes($path, [byte[]]($utf8.GetPreamble() + $utf8.GetBytes($text)))
|
|
}
|
|
function Get-SidecarPath([string]$path) { "$path.mutation-original" }
|
|
|
|
function Run-Tests([string]$filter) {
|
|
$args = @('test', $testProjectPath, '-c', 'Debug', '--nologo')
|
|
if ($filter) { $args += @('--filter', $filter) }
|
|
$output = & dotnet @args 2>&1 | Out-String
|
|
Write-Host $output
|
|
$passedMatch = [regex]::Match($output, 'Passed:\s+(\d+)')
|
|
$failedMatch = [regex]::Match($output, 'Failed:\s+(\d+)')
|
|
$compiled = -not ($output -match 'error CS\d+')
|
|
$passedLine = $output -match 'Passed!'
|
|
$failedLine = $output -match 'Failed!'
|
|
[pscustomobject]@{
|
|
Compiled = $compiled
|
|
Green = $compiled -and $passedLine -and -not $failedLine -and $passedMatch.Success -and [int]$passedMatch.Groups[1].Value -ge 1
|
|
Passed = if ($passedMatch.Success) { [int]$passedMatch.Groups[1].Value } else { 0 }
|
|
Failed = if ($failedMatch.Success) { [int]$failedMatch.Groups[1].Value } else { 0 }
|
|
}
|
|
}
|
|
|
|
$targets = @{}
|
|
foreach ($file in $Target) {
|
|
$path = Join-Path $repo $file
|
|
if (-not (Test-Path -LiteralPath $path)) { throw "Target $file does not exist." }
|
|
$targets[$file] = $path
|
|
}
|
|
|
|
# --- Repair a run that died mid-mutation, for every target -------------------
|
|
|
|
foreach ($file in $Target) {
|
|
$sidecar = Get-SidecarPath $targets[$file]
|
|
if (Test-Path -LiteralPath $sidecar) {
|
|
$original = Read-Bytes $sidecar
|
|
[System.IO.File]::WriteAllBytes($targets[$file], $original)
|
|
Remove-Item -LiteralPath $sidecar -Force
|
|
Write-Host "Repaired: a previous run left $sidecar; $($targets[$file]) restored from it and the sidecar removed."
|
|
}
|
|
}
|
|
|
|
# --- M0: no-op control, every target -----------------------------------------
|
|
|
|
Write-Host '--- M0: no-op control'
|
|
$originalBytes = @{}
|
|
$originalHash = @{}
|
|
foreach ($file in $Target) {
|
|
$path = $targets[$file]
|
|
$bytes = Read-Bytes $path
|
|
$hash = Get-BytesHash $bytes
|
|
$text = $utf8.GetString($bytes)
|
|
$reencoded = [byte[]]($utf8.GetPreamble() + $utf8.GetBytes($text))
|
|
if ((Get-BytesHash $reencoded) -ne $hash) {
|
|
throw "harness broken: $file does not round-trip through the UTF-8 read/write path byte for byte. Nothing was written and no entry ran."
|
|
}
|
|
$sidecar = Get-SidecarPath $path
|
|
[System.IO.File]::WriteAllBytes($sidecar, $bytes)
|
|
try {
|
|
Write-Text $path $text
|
|
if ((Get-BytesHash (Read-Bytes $path)) -ne $hash) {
|
|
throw "harness broken: writing $file back through the harness path changed its bytes."
|
|
}
|
|
}
|
|
finally {
|
|
[System.IO.File]::WriteAllBytes($path, $bytes)
|
|
if ((Get-BytesHash (Read-Bytes $path)) -ne $hash) { throw "RESTORE FAILED after M0 for $file; original SHA-256 $hash is kept in $sidecar." }
|
|
Remove-Item -LiteralPath $sidecar -Force
|
|
}
|
|
$originalBytes[$file] = $bytes
|
|
$originalHash[$file] = $hash
|
|
Write-Host "M0: $file decoded, re-encoded and rewritten, bytes unchanged (SHA-256 $hash)"
|
|
}
|
|
$control = Run-Tests $ControlFilter
|
|
if (-not $control.Green) {
|
|
throw "harness broken: the suite is not green unmutated (compiled=$($control.Compiled) passed=$($control.Passed) failed=$($control.Failed)). No entry ran."
|
|
}
|
|
Write-Host "M0: green, $($control.Passed) passed"
|
|
|
|
# --- Load entries --------------------------------------------------------------
|
|
|
|
$entriesPath = Join-Path $repo $Entries
|
|
$mutations = & $entriesPath
|
|
if (-not $mutations -or @($mutations).Count -lt 1) { throw "$Entries`: no mutations." }
|
|
|
|
# --- Entries -----------------------------------------------------------------
|
|
|
|
$exitCode = 0
|
|
foreach ($mutation in $mutations) {
|
|
if (-not $mutation.Id) { throw 'An entry has no Id.' }
|
|
if (-not $mutation.Killer) { throw "$($mutation.Id): no Killer." }
|
|
$edits = if ($mutation.ContainsKey('Edits')) { @($mutation.Edits) } else { @(@{ File = $mutation.File; Old = $mutation.Old; New = $mutation.New }) }
|
|
if ($edits.Count -lt 1) { throw "$($mutation.Id): no edits." }
|
|
|
|
Write-Host "--- $($mutation.Id) ($($mutation.Mutant)): $($mutation.Rule)"
|
|
$texts = @{}
|
|
$files = @($edits | ForEach-Object { if ($_.File) { $_.File } elseif ($Target.Count -eq 1) { $Target[0] } else { throw "$($mutation.Id): an edit names no File and there is more than one target." } } | Select-Object -Unique)
|
|
foreach ($file in $files) { $texts[$file] = $utf8.GetString($originalBytes[$file]) }
|
|
|
|
foreach ($edit in $edits) {
|
|
$file = if ($edit.File) { $edit.File } else { $Target[0] }
|
|
$text = $texts[$file]
|
|
$eol = if ($text.Contains("`r`n")) { "`r`n" } else { "`n" }
|
|
$old = ($edit.Old -replace "`r`n", "`n")
|
|
$normalized = $text -replace "`r`n", "`n"
|
|
$count = 0
|
|
$at = $normalized.IndexOf($old, [System.StringComparison]::Ordinal)
|
|
while ($at -ge 0) { $count++; $at = $normalized.IndexOf($old, $at + 1, [System.StringComparison]::Ordinal) }
|
|
if ($count -ne 1) { throw "$($mutation.Id): anchor matches $count times in $file, expected exactly 1." }
|
|
$newValue = if ($null -ne $edit.New) { $edit.New } else { '' }
|
|
$mutatedNormalized = $normalized.Replace($old, ($newValue -replace "`r`n", "`n"))
|
|
$texts[$file] = $mutatedNormalized -replace "`n", $eol
|
|
}
|
|
|
|
foreach ($file in $files) { [System.IO.File]::WriteAllBytes((Get-SidecarPath $targets[$file]), $originalBytes[$file]) }
|
|
$verdict = 'NO VERDICT'
|
|
try {
|
|
foreach ($file in $files) { Write-Text $targets[$file] $texts[$file] }
|
|
$run = Run-Tests $mutation.Killer
|
|
if (-not $run.Compiled) { $verdict = 'NO VERDICT (did not compile)' }
|
|
elseif ($run.Failed -ge 1) { $verdict = 'KILLED' }
|
|
else { $verdict = 'SURVIVED' }
|
|
}
|
|
finally {
|
|
foreach ($file in $files) {
|
|
[System.IO.File]::WriteAllBytes($targets[$file], $originalBytes[$file])
|
|
if ((Get-BytesHash (Read-Bytes $targets[$file])) -ne $originalHash[$file]) {
|
|
Write-Host "RESTORE FAILED: $($targets[$file]) not restored; original SHA-256 $($originalHash[$file]) kept in $(Get-SidecarPath $targets[$file])."
|
|
throw "restore failed for $($mutation.Id)"
|
|
}
|
|
Remove-Item -LiteralPath (Get-SidecarPath $targets[$file]) -Force
|
|
}
|
|
}
|
|
Write-Host "$($mutation.Id): $verdict"
|
|
if ($verdict -ne 'KILLED') { $exitCode = 1 }
|
|
}
|
|
|
|
$final = Run-Tests $ControlFilter
|
|
if (-not $final.Green) { throw "post-run sanity check: the suite is not green after restore (passed=$($final.Passed) failed=$($final.Failed))." }
|
|
Write-Host "Final: green, $($final.Passed) passed"
|
|
|
|
exit $exitCode
|