From 2f440c44b2fa9e24d9f58c7dc6a71e8037b3c101 Mon Sep 17 00:00:00 2001 From: Meta Vens Date: Fri, 29 May 2026 20:04:27 +0100 Subject: [PATCH] Document fractal domain reservoir reorientation --- docs/README.md | 3 + docs/fractal-domain-reservoir.md | 126 +++++++++++++++++++ docs/reservoir-reconstruction-action-plan.md | 24 ++++ docs/temporal-spatial-evidence-reservoir.md | 33 +++++ state/evidence.jsonl | 1 + state/map.yaml | 12 ++ state/memory.json | 2 + state/scratch.md | 27 +++- 8 files changed, 222 insertions(+), 6 deletions(-) create mode 100644 docs/fractal-domain-reservoir.md diff --git a/docs/README.md b/docs/README.md index 954fb0d..825fccf 100644 --- a/docs/README.md +++ b/docs/README.md @@ -21,6 +21,9 @@ historical branch scar. - `temporal-spatial-evidence-reservoir.md`: stable-key temporal evidence cache for fractal rendering, Mimir-style sensor fusion, direct SDF contributors, and mesh-backed field evidence. +- `fractal-domain-reservoir.md`: Area ReSTIR/TSR reorientation for the Perfect + Machine reservoir as a domain-native evidence organ rather than a + screen-space texel cache. - `temporal-sdf-gaussian-field.md`: temporal Form-field evidence lane. - `tsr-inspired-taa-spec.md`: temporal resolve design and guide-buffer intent. - `stochastic-transparent-surface-pipeline.md`: density/extinction and diff --git a/docs/fractal-domain-reservoir.md b/docs/fractal-domain-reservoir.md new file mode 100644 index 0000000..317afeb --- /dev/null +++ b/docs/fractal-domain-reservoir.md @@ -0,0 +1,126 @@ +# Fractal Domain Reservoir + +## Objective + +Fensalir's Perfect Machine reservoir is not a screen-space texel cache. It is a +dynamic, hierarchical, domain-native evidence reservoir that can be projected +into pixels, history rows, field packets, denoiser inputs, residency decisions, +and debug views. + +The renderer may resolve through a per-pixel UAV row layout, but that row is an +execution surface. It is not the sampling foundation. + +## Reorientation + +TAA proves that a renderer can accumulate more than one pixel worth of +information over time by sampling different subpixel locations each frame. +ReSTIR generalizes that idea for path and light samples by storing a selected +representative sample plus the probability mass it represents. TSR generalizes +the reconstruction side by deciding when history is trustworthy, when it must +be rejected, and how rejected regions should be reconstructed without exposing +raw noise. + +Area ReSTIR exposes the missing invariant for Fensalir: when samples live over a +pixel footprint, lens aperture, filter footprint, tube support, SDF envelope, +quadtree tile, sensor cone, or nested fractal domain, the reservoir must store +the selected sample's actual domain coordinates and support. Reuse is legal +only when a source sample can be shifted or re-evaluated in the target domain +with valid support, target/source PDF, visibility, and MIS/Jacobian accounting +where required. + +Fensalir goes one step higher: its selected samples are not merely subpixel +rays. They are field and radiance claims lowered from semantic domains into a +dynamic cache, then into GPU-visible reservoirs under frame budget. + +```text +DSL / producer intent +-> semantic domain claims +-> ownership tree and conservative summaries +-> stochastic contribution cache / residency scheduler +-> bounded candidate proposals +-> fractal domain reservoir +-> temporal domain validation and shift +-> spatial/domain validation and shift +-> TSR-style rejection and reconstruction +-> denoiser-integrated resolved field/radiance output +``` + +## Invariant + +A reservoir sample must answer these questions before it can persist: + +```text +Which domain owns the sample? +Where inside that domain was it sampled? +What support/filter footprint does it represent? +What target says it matters? +What source PDF or proposal policy produced it? +What candidate count or subtree mass does it represent? +What shift mappings can legally reuse it? +What conservative bounds keep Form, Appearance, or Transport honest? +What denoiser/reconstruction features describe its confidence and variance? +``` + +If a sample only answers "which pixel wrote me," it is not a Perfect Machine +reservoir sample. It is a screen-space cache entry. + +## Architecture Consequences + +- Screen resolution is a consumer budget, not the native sampling domain. +- Pixel rows are a row-stage projection of selected domain evidence, not a + private top-k visibility list and not the owner of field identity. +- TubeField, SDF, mesh, volume, fractal probe, and sensor contributors must all + publish bounded domain proposals before reuse. +- A broad proxy raster footprint may discover candidates, but it does not define + selected support. The selected domain sample must carry its own subpixel and + producer-domain coordinates. +- Temporal reuse must reason over overlapping source and target domains, not + nearest previous texel ownership. +- Spatial reuse must shift or re-evaluate a neighbor's selected sample in the + target domain before merge. +- The occupancy/contribution tree is not separate from the reservoir. It is the + hierarchy that decides which domain nodes deserve probes, residency, child + expansion, or parent-summary fallback under the current compute budget. +- The denoiser is not downstream cosmetics. It consumes reservoir confidence, + selected support, contribution weight, variance/age, rejection reason, and + reconstruction features, and the sampling scheduler should respond to those + signals when choosing future probes. + +## Compute-Budget Contract + +Every stage must accept a budget without changing the truth model: + +```text +high budget: + more node probes, more local candidates, richer shifts, deeper tree cut, + better denoiser features + +low budget: + fewer probes, parent summaries, older but validated reservoirs, coarser + domains, stronger reconstruction +``` + +The frame target changes sample count, probe depth, candidate count, and +reconstruction aggressiveness. It must not change who owns visibility, +identity, support, or contribution mass. + +## Immediate Cut Line + +The live seven-lane GPU reservoir row can remain as an execution ABI only if it +gains a first-class domain/sample-coordinate contract, either by adding a lane +or by repacking an existing one deliberately. Without selected domain position +and support, the row-stage rebuild is still a pixel reservoir wearing field +language. + +Do not tune occlusion thresholds as the next answer to noisy TubeField leakage. +First make the selected sample's domain coordinates, support, and legal shift +mapping visible in the reservoir state and debug views. + +## Reference Anchors + +- Area ReSTIR: https://graphics.cs.utah.edu/research/projects/area-restir/ +- Area ReSTIR code: https://github.com/guiqi134/Area-ReSTIR +- ReSTIR DI: https://research.nvidia.com/labs/rtr/publication/bitterli2020spatiotemporal/ +- GRIS: https://research.nvidia.com/labs/rtr/publication/lin2022generalized/ +- Unreal TSR docs: https://dev.epicgames.com/documentation/unreal-engine/temporal-super-resolution-in-unreal-engine +- Perfect Machine article: E:\Projects\gamecult-site\GameCult\Blog\perfect-machine-fractal-reservoir-architecture.md diff --git a/docs/reservoir-reconstruction-action-plan.md b/docs/reservoir-reconstruction-action-plan.md index 2372052..4bd3aae 100644 --- a/docs/reservoir-reconstruction-action-plan.md +++ b/docs/reservoir-reconstruction-action-plan.md @@ -31,6 +31,24 @@ Reference anchors: increases current-frame authority when history is untrustworthy; history sample count controls minimum current-frame contribution. https://dev.epicgames.com/documentation/unreal-engine/temporal-super-resolution-frequently-asked-questions-for-unreal-engine +- Area ReSTIR: reservoir samples over pixel/lens/filter area domains must store + selected sample coordinates and support, then reuse by overlap, shifting, + visibility/PDF validation, and MIS/Jacobian accounting where required. + https://graphics.cs.utah.edu/research/projects/area-restir/ + +## Higher-Level Reorientation + +The rebuild target is not a better screen-space reservoir. The target is the +fractal domain reservoir described by the Perfect Machine architecture: +semantic field and radiance claims lower from DSL/producers into a dynamic +contribution tree, are probed under bounded stochastic budgets, become +reservoir proposals, and are then reprojected, pruned, shifted, resampled, and +reconstructed every frame. + +Per-pixel rows are an execution ABI for the current D3D12 resolve. They are not +the foundation. Sampling must remain decoupled from output resolution: lower +frame budgets reduce probes, node depth, candidate count, and reconstruction +quality, but they do not move authority back to texels. ## Previous Failure @@ -139,3 +157,9 @@ old occluded tubes leak through nearby missed tube samples. and visibility cuts. 12. Next: capture final color and row-3 debug views against the noisy/occlusion screenshot class and tune target/support policy from those diagnostics. +13. Next: add or repack a domain/sample-coordinate lane. TubeField and SDF + proposals must carry selected subpixel/domain coordinates plus support + before temporal/spatial reuse is tuned further. +14. Next: make debug modes expose selected domain sample position, support + overlap, shift legality, and whether rejection came from support miss, + occlusion/disocclusion, field mismatch, or invalid producer-domain mapping. diff --git a/docs/temporal-spatial-evidence-reservoir.md b/docs/temporal-spatial-evidence-reservoir.md index e2ce90b..0c249a7 100644 --- a/docs/temporal-spatial-evidence-reservoir.md +++ b/docs/temporal-spatial-evidence-reservoir.md @@ -61,6 +61,15 @@ belong to the shared spatiotemporal field reservoir. Sources: https://github.com/NVIDIA-RTX/RTXDI/blob/main/Doc/RestirGI.md https://github.com/NVIDIA-RTX/RTXDI/blob/main/Doc/RestirPT.md +- Area ReSTIR is the missing area-domain reference. It stores the selected + sample's actual pixel/lens coordinates and reuses history only when the + previous sample lies in, or can be shifted into, the target support with valid + target/PDF/visibility/Jacobian accounting. For Fensalir this generalizes to + TubeField, SDF, mesh, volume, sensor, and fractal-domain support: selected + domain coordinates are part of reservoir identity, not optional debug data. + Sources: + https://graphics.cs.utah.edu/research/projects/area-restir/ + https://github.com/guiqi134/Area-ReSTIR ## Fensalir Divergence @@ -87,6 +96,14 @@ becomes stable because a downstream pixel-history pass smears unstable pixels until they look acceptable, the field producer or reservoir validation is wrong. +The next divergence is more important: the reservoir is not fundamentally a +screen-space texel structure. Texel rows are the current GPU execution surface. +The actual reservoir domain is fractal and field-native: semantic claims lower +through ownership trees, conservative summaries, stochastic contribution +caches, and bounded proposals before a per-frame row projection exists. Pixel +resolution is therefore a budget and resolve target, not the source of sampling +truth. + ## Pipeline Map ```text @@ -114,6 +131,17 @@ producer observations - selected sample target value; - final contribution weight `weightSum / (candidateCount * selectedTarget)`. +The sample type owns selected domain identity: + +- domain key and producer kind; +- selected coordinate inside the domain, including subpixel or local support + coordinates when the producer has area support; +- support/filter footprint and conservative bounds; +- legal shift/reconnection metadata needed by temporal and spatial reuse. + +If those values are absent, the reservoir cannot prove area-domain reuse. It +may still be a useful cache, but it is not the Perfect Machine reservoir. + Candidate generators own proposal distributions. A fractal renderer may propose form probe candidates from projected error, node bounds, blue-noise screen tiles, or resident children. Mimir may propose visual/audio feature candidates @@ -236,6 +264,11 @@ LOD pressure, or route oversized fields through the reservoir path. - Reservoir math is a pure, testable core before it becomes HLSL. - A reservoir is not a dictionary of tracks. A track layer may use reservoirs, but it does not replace candidate resampling. +- A reservoir is not a texel dictionary either. Pixel rows are row-stage + projections of selected domain evidence; they do not define the native + sampling domain. +- Selected sample coordinates and support are part of reservoir identity for + any area, tube, SDF, volume, sensor, or fractal-domain contributor. - Reuse is invalid until a pass proves the shift/validation contract for the source and target domains. - Conservative bounds remain the safety authority. Learned or stochastic diff --git a/state/evidence.jsonl b/state/evidence.jsonl index 9d2629b..9eb47a6 100644 --- a/state/evidence.jsonl +++ b/state/evidence.jsonl @@ -288,3 +288,4 @@ {"timestamp":"2026-05-29T16:06:00+01:00","type":"decision","status":"ok","note":"Row-3 final reservoir color now applies the RIS contribution weight via fieldReservoirResolvedColor; intermediate current-frame proposal MRTs remain raw to avoid double-weighting before temporal/spatial reuse. Reservoir rejection debug now distinguishes unsupported final rows, temporal rejection, disocclusion/no previous support, spatial reuse, missing explicit motion, occlusion, and field-id mismatch. Verified with headless dev reload and captures artifacts/fensalir-captures/reservoir-rebuild-weighted-final.png, reservoir-rebuild-weighted-proposal.png, and reservoir-rebuild-rejection-codes.png."} {"timestamp":"2026-05-29T16:34:00+01:00","type":"decision","status":"ok","note":"Cut a surviving split reservoir owner: D3D12FieldReservoirResolvePS no longer calls currentFrameReservoirSample or imports TubeField row-0 proposals. It now forwards raw scene/SDF MRT evidence only, leaving D3D12ReservoirHistoryUpdateCS as the sole owner of row-0 RIS merge, temporal reuse, spatial reuse, row-3 final, and resolved HDR field output. Verified with dotnet build Fensalir.sln and Mimir synthetic spectrum captures artifacts/runtime/mimir-reservoir-rebuild-single-owner-headless-180.png plus mode13/mode14/mode15 debug captures; remaining noise is now visible as broad TubeField support/proposal mass rather than double-counted fullscreen resolve authority."} {"timestamp":"2026-05-29T16:45:00+01:00","type":"decision","status":"partial","note":"TubeField proposal emission now separates conservative proxy discovery from reservoir support, using narrow tube-core support for target/control/guide. Current-frame proposal assembly also gates structural visibility before RIS: materially farther samples at the same pixel are rejected as occluded, while near-equal-depth samples still use RIS. Verified shader compile/headless reload, Aquarium.Engine.Tests, Aquarium.Engine.Fractal.Tests, Mimir --mimir-spectrum-upload-smoke, and Mimir synthetic captures artifacts/runtime/mimir-reservoir-rebuild-visibility-gate-headless-180.png plus mode13/mode14/mode15. The visible result improves overlap slightly but still shows broad row-3 support bands, so the next cut needs row-stage debug probes or producer support semantics rather than another downstream clamp."} +{"timestamp":"2026-05-29T20:01:11+01:00","topic":"area-restir-fractal-domain-reservoir","lesson":"Area ReSTIR changes the reservoir invariant: selected sample coordinates and support are part of reservoir identity for any area-domain reuse. Fensalir must apply that lesson above screen space: the Perfect Machine reservoir is a fractal domain reservoir over semantic field/radiance claims, ownership-tree nodes, stochastic contribution probes, and bounded lowerings. Pixel rows are only the current D3D12 execution projection. Before more TubeField occlusion tuning, the live ABI or packing plan must carry selected subpixel/local domain coordinates and support so temporal/spatial reuse can validate overlap and legal shifts instead of guessing from nearest texels.","files":["docs/fractal-domain-reservoir.md","docs/temporal-spatial-evidence-reservoir.md","docs/reservoir-reconstruction-action-plan.md","state/map.yaml","state/scratch.md"],"sources":["https://graphics.cs.utah.edu/research/projects/area-restir/","https://github.com/guiqi134/Area-ReSTIR","E:\\Projects\\gamecult-site\\GameCult\\Blog\\perfect-machine-fractal-reservoir-architecture.md"]} diff --git a/state/map.yaml b/state/map.yaml index dc5d0f7..6f987ce 100644 --- a/state/map.yaml +++ b/state/map.yaml @@ -60,6 +60,14 @@ invariants: terrain should be a quadtree of brush-painted surface tiles, subdivided by projected footprint, curvature, brush/material detail, silhouettes, and temporal uncertainty before SDF probes emit detail-adaptive surface splats. + - The shared reservoir is a fractal domain reservoir, not a texel cache. + Pixel rows are the current D3D12 execution projection of selected domain + evidence. Area-domain contributors must carry selected subpixel or local + domain coordinates, support/filter footprint, proposal policy, and legal + shift/reconnection metadata before temporal or spatial reuse can claim + authority. Sampling budget may scale with target frame time and resolution, + but identity, support, contribution mass, and visibility authority remain + domain-native. - Rendering backends are lowerings, not truth owners. Heightfields, splines, direct SDF proxies, meshes, volumes, fractal probes, and Mimir sensor observations must publish bounded Form/Appearance/Transport field claims @@ -110,6 +118,10 @@ next_actions: - Keep `dotnet build Fensalir.sln` green after the split. - Cut the rendering rebuild through explicit field-domain/field-claim contracts before adding another backend-specific visual path. + - Before more occlusion-threshold tuning, update the live GPU reservoir ABI or + packing plan so TubeField and SDF proposals include selected + domain/sample-coordinate and support data. Then debug temporal/spatial reuse + in terms of support overlap and legal shifts, not nearest previous texels. - Build the planetary flow organ in staged cuts: provenance manifest, static relief and reanalysis cube-sphere bake pipeline, stochastic advection contracts, synthetic runtime slice, ERA5/OSCAR baseline learning, then diff --git a/state/memory.json b/state/memory.json index de46734..fda9587 100644 --- a/state/memory.json +++ b/state/memory.json @@ -53,6 +53,7 @@ "fractal_field_architecture": [ "The Perfect Machine field architecture lives in docs/perfect-machine-architecture.md. It is the design authority before expanding SDF splat backends, sensor-fusion adapters, reservoir guide buffers, occupancy graph state, resource residency, or learned priority.", "The consolidated fractal brush architecture lives in research/rendering/fractal-brush-architecture-plan.md. Pipeline: client intent -> domain roots -> semantic IFS brush grammar -> ownership tree -> node summaries -> contribution cache/residency scheduler -> backend packets -> renderer passes -> debug/evidence.", + "The fractal domain reservoir doctrine lives in docs/fractal-domain-reservoir.md. Area ReSTIR makes selected sample coordinates and support part of reservoir identity; TSR-style reconstruction then consumes that validated domain evidence. Pixel rows are execution projections, not the native sampling domain.", "The actionable implementation checklist lives in research/rendering/fractal-brush-implementation-roadmap.md.", "The .aquageo DSL carries semantic domains and Form claims before backend lowering. Height, SignedDistance, Density, and Extinction must not be collapsed into one payload lane.", "Fractal summaries carry FormEncodingMask so selected-cut structural probes do not silently default transparent form nodes back into SDF.", @@ -86,6 +87,7 @@ "research_notes": [ "Renderer notes live under docs/ and research/rendering/ in this repo after the split.", "docs/perfect-machine-architecture.md is the broad field/reservoir architecture authority.", + "docs/fractal-domain-reservoir.md records the Area ReSTIR/TSR reorientation: Fensalir's reservoir is domain-native and resolution-decoupled, with texel rows only as current D3D12 resolve ABI.", "docs/temporal-spatial-evidence-reservoir.md owns the stable-key temporal evidence cache contract.", "docs/temporal-sdf-gaussian-field.md is historically named but now represents temporal Form-field evidence, not only SDF surfaces.", "research/rendering/fractal-reference-oracles.md owns flame/IFS external reference strategy.", diff --git a/state/scratch.md b/state/scratch.md index b37b4ad..0fc232c 100644 --- a/state/scratch.md +++ b/state/scratch.md @@ -2,13 +2,28 @@ ## Current Slice -Fensalir has been split into its own adjacent repo. +Reorient the reservoir rebuild around the fractal domain reservoir, not +screen-space texel rows. -- This repo owns the native runtime, renderer, contracts, fractal machinery, - host scripts, and engine demo clients. -- `E:\Projects\Aquarium-Engine` now owns only the Epiphany Aquarium client. -- There is no remote configured yet; set it to the future `GameCult/Fensalir` - remote before pushing. +## Hot Lesson + +Area ReSTIR confirms that reuse over an area domain is invalid unless the +reservoir stores the selected sample's actual domain coordinates and support. +TSR confirms that temporal reconstruction must reject, reconstruct, and expose +history trust at the layer that owns visible output. Fensalir needs both, but +one level higher: selected samples are field and radiance claims lowered from +semantic domains and a dynamic contribution tree into bounded GPU reservoirs. + +The current seven-lane row ABI can remain only as an execution projection if it +gains first-class selected domain/sample-coordinate authority. A row keyed only +by pixel, travel, field id, guide, proposal, and stats is still too texel-like. + +## Next Bounded Move + +Before tuning occlusion thresholds, add or repack reservoir state so TubeField +and SDF proposals carry selected subpixel/domain coordinate plus support. Then +make temporal/spatial debug views show support overlap, shift legality, and +selected domain sample position. ## Verification