From cdb665bd5e7aa8ca77835b35552bca5b76dcc7e4 Mon Sep 17 00:00:00 2001 From: Meta Vens Date: Thu, 21 May 2026 08:40:06 +0100 Subject: [PATCH] Make full reservoir refresh deterministic --- research/rendering/fractal-reference-oracles.md | 8 ++++++++ .../Render/Shaders/D3D12FractalReservoirCompute.hlsl | 5 +++++ state/evidence.jsonl | 1 + state/map.yaml | 2 +- state/memory.json | 2 +- 5 files changed, 16 insertions(+), 2 deletions(-) diff --git a/research/rendering/fractal-reference-oracles.md b/research/rendering/fractal-reference-oracles.md index ff7b0e3..7711b12 100644 --- a/research/rendering/fractal-reference-oracles.md +++ b/research/rendering/fractal-reference-oracles.md @@ -225,6 +225,14 @@ the same gate to `43.13%` with `68.01%` occupancy overlap; pass that path. The contact sheet lives at `artifacts/fractal-flame-jwildfire-reference/julian-disc-baby-normal-cut-area-contact.png`. +Full reservoir refreshes must not use the stochastic hash indexer. When +`ReservoirUpdatesPerPass == SplatCount`, the compute shader now updates row +`updateIndex` directly for each reservoir layer. That keeps under-budget passes +stochastic while making full coverage actually full and memory-coherent. The +same `julian-disc-baby` full-refresh receipt stayed visually equivalent +(`43.09%`, `67.99%` occupancy overlap) but dropped from roughly `41 ms/frame` +to `7.508 ms/frame`, or `133.2 FPS` equivalent. + The remaining visible mismatch is now narrower: Aquarium draws the same family of form, but its density is still too compact and differently weighted. Fixing that belongs in exact JWildfire variation/camera parity, target density diff --git a/src/Aquarium.Engine/Render/Shaders/D3D12FractalReservoirCompute.hlsl b/src/Aquarium.Engine/Render/Shaders/D3D12FractalReservoirCompute.hlsl index 42a1d84..1bd34eb 100644 --- a/src/Aquarium.Engine/Render/Shaders/D3D12FractalReservoirCompute.hlsl +++ b/src/Aquarium.Engine/Render/Shaders/D3D12FractalReservoirCompute.hlsl @@ -304,6 +304,11 @@ float3 FractalPoint(uint index, out float radius) uint ReservoirIndex(uint updateIndex, uint passKind) { + if (ReservoirUpdatesPerPass >= SplatCount) + { + return updateIndex % SplatCount; + } + return Hash(updateIndex * 1664525u + FrameIndex * 1013904223u + passKind * 747796405u + Seed) % SplatCount; } diff --git a/state/evidence.jsonl b/state/evidence.jsonl index 07179a6..4eef3ce 100644 --- a/state/evidence.jsonl +++ b/state/evidence.jsonl @@ -196,3 +196,4 @@ {"timestamp":"2026-05-21T01:32:00+01:00","topic":"sdf-reservoir-density-parity","lesson":"The external flame parity candidate now comes from Aquarium's SDF envelope reservoir layer, not raw splat-center occupancy. tools/Aquarium.Fractal.Receipt stamps populated AquariumPackedSdfEnvelopeReservoir rows into shaped density histograms and reports the candidate source explicitly. PBR and radiosity remain separate reservoir layers: PBR should score appearance later, while radiance is out of this gate for now. With full 2M-row reservoir refresh on the GTX 1070, julian-disc-baby scored 22.14% against the FARender PPM at 256x256 bounds -2..2 and cost 42.053 ms/frame; the contact sheet shows a sparse, shape-shifted Aquarium SDF density rather than a valid rendered flame. The next work is dialect/camera/filter parity plus target-density and view-conditioned reservoir feedback, not raw occupancy scoring.","files":["tools/Aquarium.Fractal.Receipt/Program.cs","research/rendering/fractal-reference-oracles.md","state/map.yaml","state/memory.json"],"source":"user corrected the metric: SDF reservoirs express form density, PBR reservoirs express surface appearance, and raw occupancy is the wrong comparison"} {"timestamp":"2026-05-21T01:52:00+01:00","topic":"jwildfire-normal-is-not-linear","lesson":"JWildfire FARender does not treat `variationGroup normal=\"1\"` as an explicit linear variation. A normal-only FARender diagnostic collapsed to a constant point, proving Aquarium's `normal -> linear` approximation was inventing geometry. The importer now ignores JWildfire `normal` as variation-group metadata while preserving explicit `linear` for Apophysis/FLAM3 fixtures. The SDF-density receipt also enforces a minimum one-bin reconstruction footprint for subpixel envelopes so density parity no longer degenerates into point occupancy. The julian-disc-baby external score moved from 22.14% to 31.36% with full 2M-row reservoir refresh; remaining mismatch is compactness/weighting, not the previous fake affine contribution.","files":["src/Aquarium.Engine.Fractal/Grammar/FractalFlameFileParser.cs","tests/Aquarium.Engine.Fractal.Tests/ApophysisReferenceParityTests.cs","tools/Aquarium.Fractal.Receipt/Program.cs","research/rendering/fractal-reference-oracles.md","state/map.yaml","state/memory.json"],"source":"continued flame external parity push after discovering FARender normal-only behavior"} {"timestamp":"2026-05-21T02:00:00+01:00","topic":"flame-camera-bounds-receipt","lesson":"FARender maps flame points through camera bounds derived from flame `size / (scale * cam_zoom)` around `center`; using an ad hoc square histogram window was distorting external density parity. tools/Aquarium.Fractal.Receipt now accepts `--histogram-bounds flame-camera` after `--program-flame` and derives those bounds from the flame XML. On julian-disc-baby this moved the SDF-reservoir-density score from 31.36% in manual -2..2 bounds to 43.13% with 68.01% occupancy overlap. Remaining loss is now less camera-window stupidity and more exact variation/filter weighting plus reservoir targeting.","files":["tools/Aquarium.Fractal.Receipt/Program.cs","research/rendering/fractal-reference-oracles.md","state/map.yaml","state/memory.json"],"source":"continued flame external parity push after FARender source showed the camera size/scale mapping"} +{"timestamp":"2026-05-21T08:40:00+01:00","topic":"direct-full-reservoir-refresh","lesson":"The reservoir shader used the stochastic hash indexer even when ReservoirUpdatesPerPass equaled SplatCount, so a claimed full refresh was random-memory coupon-collector work. D3D12FractalReservoirCompute now maps full-refresh updates directly to updateIndex % SplatCount and keeps the hash path only for under-budget stochastic passes. On julian-disc-baby with flame-camera bounds, the visual score stayed essentially the same (43.09%, 67.99% occupancy overlap) but full SDF/PBR/radiosity reservoir refresh dropped to 7.508 ms/frame / 133.2 FPS equivalent instead of roughly 41 ms/frame.","files":["src/Aquarium.Engine/Render/Shaders/D3D12FractalReservoirCompute.hlsl","research/rendering/fractal-reference-oracles.md","state/map.yaml","state/memory.json"],"source":"continued hill climb after identifying the full-coverage reservoir pass as randomized memory confetti"} diff --git a/state/map.yaml b/state/map.yaml index 7a8dae3..6ab94d4 100644 --- a/state/map.yaml +++ b/state/map.yaml @@ -154,7 +154,7 @@ renderer_state: - View-conditioned visual parity now scores named camera windows instead of trusting one global histogram. For the depth-16 flame receipt, global -8..8 scores 97.09%, mid -1..1 scores 91.93%, close -0.5..0.5 scores 88.97%, and micro -0.25..0.25 scores 82.05% with 2,656 starved reference bins and 17.95% underrepresented mass. This is the current hard loss: the reservoir is not yet spending work based on camera-conditioned underrepresentation. - High-reference flame contact sheets should use at least 20M CPU oracle samples for visual judgment. The current artifact is artifacts/fractal-contact-sheet-hq/flame-parity-contact-sheet-hq.png, comparing 20M CPU oracle histograms against 250k Aquarium GPU readback samples for global frame 1/10 and zoom frame 1/10. The 1M-sample oracle was too noisy and should be treated as a scalar smoke check only. - External flame visual references are now reproducible from ignored local artifacts. scripts/install-flame-reference-tools.ps1 hydrates JWildfire 9.00 plus Temurin JRE 17 under artifacts/reference-tools; scripts/fractal-flame-jwildfire-reference.ps1 prefers JWildfire's bundled FARenderJWF GPU renderer for explicit PNG output. Local GTX 1070 smoke rendered FARender's selftest.flame at 1024x1024 quality 2000 in 7.69 sec to artifacts/fractal-flame-jwildfire-reference/selftest-farender-20260520-233638.png. JWildfire/FARender is a visual reference artifact lane, not Aquarium runtime code. - - tools/Aquarium.Fractal.Receipt now has an external density oracle: --reference-density-ppm compares Aquarium GPU SDF envelope reservoir density against luminance density from a binary PPM rendered by an external flame renderer. scripts/new-jwildfire-baby-fixture.ps1 derives a smaller FARender-compatible julian-disc-baby.flame from JWildfire's selftest by reducing julian power and muting gaussian/wfield amounts while preserving FARender boilerplate. First GTX 1070 external-density receipt against FARender's 768x768 quality-2000 baby render scored only 15.33% when Aquarium still binned raw splat centers; after fixing JWildfire coefs packing and FARender disc angle convention the raw-center score reached 26.87%. Switching to SDF-reservoir density with full 2M-row reservoir refresh scored 22.14% at 42.053 ms/frame. Cutting the false JWildfire `normal` -> `linear` import and enforcing a one-bin SDF density reconstruction footprint moved the score to 31.36% at 41.692 ms/frame in 256x256 bounds -2..2. Using `-HistogramBounds flame-camera` derives FARender-style bounds from flame size/scale/cam_zoom and raises the current score to 43.13% with 68.01% occupancy overlap. Aquarium still scores about 89.94% against its own CPU slice, so the remaining external gap is exact JWildfire variation/filter semantics, target density weighting, and view-conditioned reservoir feedback, not GPU sampling count. + - tools/Aquarium.Fractal.Receipt now has an external density oracle: --reference-density-ppm compares Aquarium GPU SDF envelope reservoir density against luminance density from a binary PPM rendered by an external flame renderer. scripts/new-jwildfire-baby-fixture.ps1 derives a smaller FARender-compatible julian-disc-baby.flame from JWildfire's selftest by reducing julian power and muting gaussian/wfield amounts while preserving FARender boilerplate. First GTX 1070 external-density receipt against FARender's 768x768 quality-2000 baby render scored only 15.33% when Aquarium still binned raw splat centers; after fixing JWildfire coefs packing and FARender disc angle convention the raw-center score reached 26.87%. Switching to SDF-reservoir density with full 2M-row reservoir refresh scored 22.14% at 42.053 ms/frame. Cutting the false JWildfire `normal` -> `linear` import and enforcing a one-bin SDF density reconstruction footprint moved the score to 31.36% at 41.692 ms/frame in 256x256 bounds -2..2. Using `-HistogramBounds flame-camera` derives FARender-style bounds from flame size/scale/cam_zoom and raises the current score to about 43.1% with about 68% occupancy overlap. Full reservoir refresh now uses direct row indexing instead of stochastic hashing, so the same full-refresh gate runs at 7.508 ms/frame / 133.2 FPS equivalent instead of roughly 41 ms/frame. Aquarium still scores about 89.94% against its own CPU slice, so the remaining external gap is exact JWildfire variation/filter semantics, target density weighting, and view-conditioned reservoir feedback, not GPU sampling count. - JWildfire selftest flame is the current honest faceplant target. After adding a first dialect slice for JWildfire coefficient layout, variationGroup, disc, julian, gaussian_blur, and post transforms, the depth-64 GTX 1070 receipt against Aquarium's CPU interpretation of artifacts/reference-tools/j-wildfire-9.00/lib/FARenderJWF/selftest.flame reaches 84.19% score in bounds -8..8 and 88.01% in bounds -128..128, but costs about 129 ms/frame. This is CPU/GPU slice parity, not external JWildfire visual parity. It proves the remaining hard work is richer dialect semantics plus persistent GPU iteration-state reservoirs; brute replaying deep flame iteration from zero per splat is not the final hot path. - FractalFlameFileParser now has ParseFirstWithReport for import accountability. Receipt runs emit fractal-flame-import-report-*.json beside flame histogram/GPU receipts. The JWildfire selftest report currently records 15 accepted fields, 3 approximations, 45 ignored render/presentation fields, and 11 rejected geometry/sampling fields: wfield_* weighting-field attributes. Successful XML parsing is not successful translation. - FractalFlameIterationState and FractalFlameIterationStepper define the persistent flame iteration invariant before GPU lowering: advancing a sample in chunks must match a one-shot advance from the same state. This is a parity/mock boundary for the future GPU state buffer, not a runtime CPU renderer. diff --git a/state/memory.json b/state/memory.json index 9364d99..a8a10bc 100644 --- a/state/memory.json +++ b/state/memory.json @@ -112,7 +112,7 @@ "Global flame parity is not enough. View-conditioned parity now scores named viewport bounds and reports GPU/reference hits, starved reference bins, and underrepresented mass. First depth-16 GTX 1070 receipt with 250k GPU readback splats vs 1M CPU samples: global -8..8 scores 97.09%, mid -1..1 scores 91.93%, close -0.5..0.5 scores 88.97%, micro -0.25..0.25 scores 82.05% with 2,656 starved bins and 17.95% underrepresented mass. This proves the current reservoir is fast but not view-adaptive; next work needs camera-conditioned target weights and underrepresented-region feedback.", "Do not use the quick 1M-sample flame oracle image for visual judgment; it is too noisy. The high-reference contact sheet uses 20,000,000 CPU oracle samples and 250,000 Aquarium GPU readback splats per panel at artifacts/fractal-contact-sheet-hq/flame-parity-contact-sheet-hq.png. It makes the zoom failure legible: the CPU oracle smooths out while Aquarium remains under-sampled after zoom because the current reservoir is not view-adaptive.", "Aquarium's CPU flame histogram is a math oracle, not visual ground truth. The visual reference ladder is: Chaotica as gold market-quality reference when available/imported, JWildfire/FARender as scriptable visual reference, FLAM3/Apophysis as portable baseline compatibility, Aquarium CPU as deterministic unit-test/debug oracle only. On 2026-05-20 Chaotica and flam3-render were not on PATH, but scripts/install-flame-reference-tools.ps1 hydrated JWildfire 9.00 plus Temurin JRE 17 under ignored artifacts, and scripts/fractal-flame-jwildfire-reference.ps1 rendered JWildfire's bundled FARenderJWF selftest flame at 1024x1024 quality 2000 in 7.69 sec on the GTX 1070. JWildfire/FARender is an external visual reference artifact lane, not Aquarium runtime code. Do not claim best-in-market flame parity until imported Chaotica renders or fully matched JWildfire flame dialect metrics are wired into residual scoring.", - "The useful flame oracle is now external density, not Aquarium CPU self-agreement. tools/Aquarium.Fractal.Receipt accepts --reference-density-ppm to compare Aquarium GPU SDF envelope reservoir density against luminance density from a binary PPM rendered by JWildfire/FARender. scripts/new-jwildfire-baby-fixture.ps1 derives a smaller FARender-compatible julian-disc-baby.flame from the bundled selftest while preserving required FARender boilerplate. First external receipt scored 15.33% in 256x256 bounds -2..2 while Aquarium still binned raw centers; after fixing JWildfire coefs packing (`a b d e c f`) and FARender's disc angle convention (`atan2(x,y)`), the raw-center score reached 26.87%. With the candidate switched to full SDF-reservoir density, the same target scored 22.14% at 42.053 ms/frame with all 2M reservoir rows refreshed. Cutting the false JWildfire `normal` -> `linear` import and resolving subpixel SDF envelopes as at least one histogram bin moved the score to 31.36% at 41.692 ms/frame. `-HistogramBounds flame-camera` derives FARender-style size/scale/cam_zoom bounds and moves the score to 43.13% with 68.01% occupancy overlap. Aquarium-vs-CPU still scores about 89.94%, so the live failure is exact JWildfire variation/filter semantics, target density weighting, and view-conditioned reservoir feedback, not sample count.", + "The useful flame oracle is now external density, not Aquarium CPU self-agreement. tools/Aquarium.Fractal.Receipt accepts --reference-density-ppm to compare Aquarium GPU SDF envelope reservoir density against luminance density from a binary PPM rendered by JWildfire/FARender. scripts/new-jwildfire-baby-fixture.ps1 derives a smaller FARender-compatible julian-disc-baby.flame from the bundled selftest while preserving required FARender boilerplate. First external receipt scored 15.33% in 256x256 bounds -2..2 while Aquarium still binned raw centers; after fixing JWildfire coefs packing (`a b d e c f`) and FARender's disc angle convention (`atan2(x,y)`), the raw-center score reached 26.87%. With the candidate switched to full SDF-reservoir density, the same target scored 22.14% at 42.053 ms/frame with all 2M reservoir rows refreshed. Cutting the false JWildfire `normal` -> `linear` import and resolving subpixel SDF envelopes as at least one histogram bin moved the score to 31.36% at 41.692 ms/frame. `-HistogramBounds flame-camera` derives FARender-style size/scale/cam_zoom bounds and moves the score to about 43.1% with about 68% occupancy overlap. Full reservoir refresh must use direct row indexing, not stochastic hashing; after that fix the same 2M-row full-refresh gate runs at 7.508 ms/frame / 133.2 FPS equivalent instead of roughly 41 ms/frame. Aquarium-vs-CPU still scores about 89.94%, so the live failure is exact JWildfire variation/filter semantics, target density weighting, and view-conditioned reservoir feedback, not sample count.", "JWildfire selftest is now the flame renderer faceplant target. The current dialect slice supports JWildfire coefficient layout, variationGroup, disc, julian, gaussian_blur, and post transforms. After post lowering, the depth-64 GTX 1070 receipt scores 84.19% against Aquarium's CPU slice oracle in bounds -8..8 and 88.01% in bounds -128..128, but costs about 129 ms/frame. This is not external JWildfire visual parity; it is CPU/GPU agreement on the supported slice. Aquarium still needs richer dialect semantics, palette/density/render-filter parity, weighting-field handling, and persistent per-sample GPU iteration-state reservoirs instead of replaying deep chaos-game ancestry from zero for every splat.", "Flame imports must report translation loss. FractalFlameFileParser.ParseFirstWithReport returns accepted, approximated, ignored, and rejected fields per transform, and receipt runs write fractal-flame-import-report-*.json. The JWildfire selftest report currently shows 15 accepted fields, 3 approximations, 45 ignored render/presentation fields, and 11 rejected geometry/sampling fields: wfield_* weighting-field attributes. Treat successful XML parsing as insufficient until rejected geometry fields are either implemented as DSL/operator nodes or explicitly accepted as out-of-scope diagnostics.", "Persistent flame iteration now has both a CPU parity contract and a GPU realization. FractalFlameIterationState plus FractalFlameIterationStepper prove chunked CPU advancement; AquariumPackedFractalFlameState mirrors the point/support/material/RNG/step packet in D3D12FractalReservoirCompute as a UAV so flame splats advance over frames. Keep the acquisition/maintenance split explicit in receipts and renderer budgets.",