Fensalir/research/rendering/fractal-brush-implementation-roadmap.md

Fractal Brush Implementation Roadmap

This is the execution map for the fractal brush architecture in research/rendering/fractal-brush-architecture-plan.md.

The architecture is allowed to be ambitious. This roadmap is not. It exists to keep the work shippable, testable, and cuttable.

Objective

Build a reusable mapped fractal field system for planetary terrain, tileable surface detail, and later agent/body detail.

The first live proof is deliberately narrow:

cube-face address + projection harness
-> shaped 2D brush envelope
-> semantic claim/tree structs
-> flatten to current height brush pass
-> debug face/tile/brush/score overlays

No 3D recursive body form. No learned predictor. No streaming page store. The first pass proves address, envelope, semantic tree, flattening, tests, and debug visibility.

Standing Invariants

• The authored semantic tree is the source of truth.
• Backend packets are compiled output and may be replaced.
• Every domain, node, claim, and payload has a stable key.
• Every LOD subtree has a conservative parent summary.
• The cache may learn priority; it may not own field safety.
• Missing children render through parent summaries instead of stalling.
• Debug views are required before recursive detail density increases.
• Each module owns one invariant and exposes a mockable seam.

Module Targets

Aquarium.Engine.Contracts

Owns portable DTOs:

• AquariumFractalDomain
• AquariumBrushClaim
• AquariumFractalNode
• AquariumFractalSummary
• AquariumContributionState
• AquariumSelectedCut
• backend packet rows
• debug channel rows

Tests:

• serialization round trips;
• stable key equality;
• no renderer/Vortice dependency.

Aquarium.Engine.Fractal

Owns pure CPU algorithms:

• cube-face addressing;
• projection candidates;
• local frame/domain math;
• grammar expansion;
• ownership tree bounds;
• summary building;
• contribution scoring;
• stochastic estimator;
• selected-cut planning;
• residency decisions.

Tests:

• regular unit tests with deterministic fakes;
• no D3D12 boot required.

Aquarium.Engine.Render

Owns D3D12 lowering:

• GPU buffers;
• descriptor binding;
• shader packet decode;
• height/material/Form/Appearance/Transport backend passes;
• debug visualizations.

Tests:

• packet golden tests;
• CPU/HLSL parity tests;
• headless smoke renders.

Client Adapters

Aquarium.Zyphos owns planet domain roots and terrain grammar policy.

Aquarium.Epiphany later owns agent/body grammar selection and semantic state bindings.

Tests:

• clients emit contracts without renderer types;
• stable seeds emit stable keys.

Mock Boundaries

Add these before the modules need them:

• IFractalClock: deterministic frame/time.
• IFractalRandom: deterministic update lottery.
• IProjection: projection candidate swap.
• IContributionProbe: observed pixel/form/appearance/transport delta.
• IFractalPayloadStore: fake RAM/SSD payload availability.
• IFractalGpuBudget: fake GPU table/page limits.
• IFractalDebugSink: telemetry capture without UI.

Mock resources and observations, not private helper functions. The tests should pin module contracts, not puppet implementation details.

Phase 0: Test And Project Skeleton

Goal: create the module/testing skeleton before algorithm code spreads.

Tasks:

• Add src/Aquarium.Engine.Fractal.
• Add tests/Aquarium.Engine.Fractal.Tests.
• Add project references from tests to contracts/fractal core.
• Add deterministic fake clock/random/debug sink test helpers.
• Add a boundary test proving Aquarium.Engine.Fractal has no D3D12/Vortice dependency.

Exit gate:

• dotnet build Fensalir.sln
• dotnet test tests/Aquarium.Engine.Fractal.Tests

Cut line:

• If module wiring fights the existing solution, keep the new core smaller, not sneak fractal logic into D3D12Renderer.

Phase 1: Cube-Face Address And Projection Harness

Goal: establish stable planetary tile addresses and measured projection choices.

Tasks:

• Add CubeFace, CubeTileKey, and quadtree path helpers.
• Add local tile uv to cube-face coordinate conversion.
• Implement identity-normalize projection as baseline.
• Implement tangent projection candidate.
• Implement fifth-order odd polynomial projection candidate.
• Add placeholder/interface for COBE/QSC candidate.
• Add area distortion sampler.
• Add face-edge continuity tests.
• Add CPU benchmark or fixture reporting candidate costs.
• Add Zyphos debug mode that colors face/tile coordinates.

Tests:

• projection round trips where inverse exists;
• face-edge continuity cases;
• deterministic tile key formatting;
• sampled area error fixtures.

Exit gate:

• projection report names current default and measured error/cost;
• Zyphos can show face/tile debug colors.

Cut line:

• Ship tangent projection first if polynomial/COBE cost delays visible work. Keep the harness so the choice stays measurable.

Phase 2: Shaped 2D Brush Envelope

Goal: replace circular brush thinking with compact-support anisotropic fields.

Tasks:

• Add CPU FractalBrushEnvelope2D.
• Add HLSL packed envelope row.
• Implement compact-support ellipse evaluator.
• Add CPU/HLSL parity fixture.
• Add brush bounds and support rejection tests.
• Add a small adapter that lowers one claim into the current height brush path.
• Add debug overlay for envelope bounds and weight.

Tests:

• weight at center/edge/outside support;
• rotation parity;
• deterministic packet packing;
• old circular brush fixture still reproducible through a round envelope.

Exit gate:

• current height pass can render shaped 2D envelopes through a compatibility adapter.

Cut line:

• Do not add taper/skew/ridge controls until the basic envelope has debug views and parity tests.

Phase 3: Semantic Claims And Ownership Tree

Goal: preserve authoring meaning before backend flattening.

Tasks:

• Add contract DTOs for domains, claims, nodes, summaries, and selected cuts.
• Add stable key builder for planet tile + grammar path.
• Add in-memory ownership tree builder.
• Implement operations: Union, Subtract, Common, Repeat, Refine, Capture, Bind.
• Implement operation bounds.
• Add flattening compiler to shaped 2D brush packets.
• Add debug dump for authored tree and flattened packets.
• Preserve multiple .aquageo tile roots in the ownership tree instead of collapsing every claim under the final tile.
• Add compact flame grammar primitive for deterministic curled recursive brush emission.

Tests:

• seeded grammar determinism;
• stable keys across equivalent expansion;
• union/subtract/common bounds;
• flattening golden packets;
• client contracts contain no renderer type.

Exit gate:

• one toy terrain grammar emits semantic claims, an ownership tree, and flattened brush packets.

Cut line:

• Do not build a CAD CSG kernel. This tree adjudicates semantic ownership and lowering, not arbitrary mesh booleans.

Phase 4: Node Summaries

Goal: make every LOD subtree renderable without its children.

Tasks:

• Add summary builder for 2D brush tree nodes.
• Track max height/form error.
• Track max material delta placeholder.
• Track coverage and estimated cost.
• Add parent fallback payload.
• Add summary serialization.
• Add debug overlay for summary bounds/error.

Tests:

• parent summary conservatively covers child height contribution;
• child eviction leaves parent renderable;
• summary round trip preserves error bounds;
• summary cost monotonically covers descendants.

Exit gate:

• selected parent summary can render when children are absent.

Cut line:

• Use conservative amplitude/frequency bounds before clever exact summaries.

Phase 5: Contribution Cache And Selected Cut

Goal: choose a stable hierarchy cut without scoring every leaf.

Tasks:

• Add stable-keyed contribution table.
• Add projected error scorer.
• Add selected-cut builder.
• Add hysteresis fade weights.
• Add budget clamp.
• Add debug channels: score, depth, fade, estimated cost.
• Add explicit resource budget plan for CPU update count, GPU estimated packet cost, RAM residency, and SSD request pressure.

Tests:

• score decreases with distance for fixed error;
• score increases with error for fixed distance;
• small camera motion keeps cut stable;
• budget clamp chooses parent summaries;
• fade never changes conservative SDF/height safety fields.

Exit gate:

• toy terrain grammar selects parent at distance and children nearby.

Cut line:

• Keep scoring on CPU until data layout and debug channels are stable.

Phase 6: Probabilistic Online Updates

Goal: refresh contribution estimates under budget and converge without pretending an online smoother is ReSTIR.

Research authority:

• ReSTIR DI: candidate generation plus temporal and spatial reservoir reuse for real-time direct lighting. https://research.nvidia.com/labs/rtr/publication/bitterli2020spatiotemporal/
• ReSTIR GI: path reservoirs for indirect lighting, with final shading fed by a selected reservoir sample. https://research.nvidia.com/publication/2021-06_restir-gi-path-resampling-real-time-path-tracing
• GRIS: the generalized theory for correlated samples, unknown PDFs, varied domains, and shift mappings. This is the fractal/SDF/sensor-fusion door. https://research.nvidia.com/labs/rtr/publication/lin2022generalized/
• RTXDI integration docs: initial sampling, temporal reprojection, spatial reuse, validation, shift mapping, reservoir buffers, and denoiser guide buffers. https://github.com/NVIDIA-RTX/RTXDI/blob/main/Doc/RestirGI.md https://github.com/NVIDIA-RTX/RTXDI/blob/main/Doc/RestirPT.md

Tasks:

• Add pure ResampledImportanceReservoir<TSample> core with selected sample, target, weight sum, represented candidate count, merge, and contribution weight.
• Add estimator state: mean, variance/uncertainty, confidence, sample count, sample age.
• Add stochastic scheduler with deterministic random source.
• Add exploration budget.
• Add live contribution cache that observes only scheduled nodes each frame and feeds the next resource plan.
• Promote contribution state into FractalOccupancyGraph with visibility, update probability, confidence decay, and stale uncertainty growth.
• Convert contribution-cache refresh into a candidate generator that submits weighted candidates to the resampled-importance core.
• Add fractal SDF probe candidate records: domain key, local frame, bound, projected contribution target, source PDF, payload handle, and material delta.
• Add structural probe generator that lowers IFS node summaries into FractalProbeSample reservoir candidates.
• Add selected-cut structural probe reservoir and expose it through Zyphos' fractal render plan/debug dump.
• Add temporal reuse validation: camera motion, domain ancestry, local-frame error, bounds, and disocclusion/material compatibility. Domain ancestry and local-shift checks exist; camera/disocclusion/material validation still need renderer inputs.
• Add CPU temporal reuse validation contract with camera motion, disocclusion, material mismatch, visibility, domain ancestry, local-frame error, and bound rejection reasons.
• Add spatial reuse validation across screen tiles and cube-sphere neighbor domains.
• Feed reservoir confidence, selected target, candidate count, sample age, and invalidation reason into TAA guide/history buffers. Current reservoir confidence, sample age, domain validity, and invalidation code now have explicit scene/history reservoir guide targets; selected target and candidate count still need producer-side packing once GPU reservoirs exist.
• Add confidence decay and stale uncertainty growth.
• Add fake contribution probe.
• Add convergence telemetry.
• Add debug channels: uncertainty, sample age, update probability, reservoir weight sum, selected target, candidate count, reuse validity.

Tests:

• reservoir accepts candidates proportional to target/source PDF;
• reservoir merge preserves represented candidate count and weight sum;
• invalid zero-weight candidates do not corrupt reservoir state;
• contribution cache exposes scheduled updates as reservoir candidates;
• deterministic RNG fixture repeats schedule;
• visible nodes keep nonzero update probability;
• near-threshold nodes keep nonzero update probability;
• stale wrong score recovers after observations;
• fixed camera path converges within tolerance;
• temporal reuse rejects mismatched domain ancestry;
• spatial reuse rejects invalid local-frame shifts;
• temporal reuse rejects camera motion, disocclusion, material mismatch, and low visibility;
• TAA history confidence is modulated by current reservoir confidence;
• TAA history confidence drops when previous-frame reservoir/domain validity changes;
• update cost respects budget.

Exit gate:

• cache updates only a bounded subset per frame while selected cuts converge, and the selected detail path is driven by tested reservoir math rather than a frozen score table.

Cut line:

• No neural predictor until the ReSTIR/GRIS-shaped reservoir pipeline fails with captured telemetry. No consumer-owned temporal cache unless it clearly owns raw capture retention rather than resolved evidence.

Phase 7: Residency Simulation

Goal: prove page/key semantics before real SSD/GPU streaming.

Tasks:

• Add renderer-agnostic surface page key/metadata contracts for height, 2D signed-distance, material, and confidence pages.
• Add CPU surface page planner that lowers selected IFS cuts into stable page metadata.
• Add mockable surface page store boundary with resident/request/evict decisions under a byte budget.
• Add CPU surface page rasterizer for resident height, 2D signed-distance, material, and confidence pages.
• Add fake payload store with resident/missing states.
• Add child request queue.
• Add score/cost eviction policy.
• Add parent-summary fallback path.
• Clamp simulated RAM residency and SSD requests through the resource plan.
• Add debug channels: residency, requests, evictions.

Tests:

• missing child renders parent summary;
• high-score missing child enqueues request;
• low-score high-cost payload evicts first;
• frame path never blocks on fake store.

Exit gate:

• selected cut can tolerate missing children without frame stalls.

Cut line:

• Simulate SSD first. Real async file IO waits until page semantics are boring.

Phase 8: Zyphos Planet Proof

Goal: replace spherical terrain demo path with cube-sphere domain proof.

Tasks:

• Add Zyphos cube-sphere domain root.
• Add first tile grammar: ridges, craters, and material bands.
• Add first flame-style microdetail grammar line inside the Zyphos leaf domain.
• Lower selected cut to shaped 2D height/material brush packets.
• Feed planet shader from tile-local cube-face domains for post-hit material/detail evaluation.
• Route Zyphos render-plan construction through the resource budget planner.
• Update Zyphos contribution cache every render-plan request while reusing compiled brush packets for equivalent selected cuts.
• Feed traced planet geometry from a hard-budgeted top-influence subset of selected tile packets.
• Lower the structural probe reservoir into a first compact 3D SDF splat packet for inspection.
• Lower resident 2D SDF pages into bounded projected 3D splat packets through an explicit page-to-world mapping boundary.
• Add first face/tile/material influence debug view through render debug mode 11.
• Add LOD/cache debug overlays.
• Remove or quarantine old spherical noise path as baseline fixture.

Tests:

• face seams render continuously;
• tile keys remain stable under camera movement;
• debug modes show selected depth and cache state;
• headless Zyphos smoke render passes.

Exit gate:

• Zyphos renders terrain from semantic cube-sphere tile grammar.

Cut line:

• Do not start agent-body recursive form here. Planet terrain proves the shared address, envelope, summary, and cache spine first.

Phase 9: Agent/Body Pilot

Goal: prove the same grammar works for object-local detail.

Tasks:

• Pick one role/body with a stable base SDF.
• Add object-local domain root.
• Add 2D material/detail claims before traced form recursion.
• Add object-local summary tree.
• Integrate selected cut with SDF proxy packet.
• Add debug channels for body detail score and SDF cost.

Tests:

• silhouette stays coherent across LOD;
• material/detail fades by contribution;
• SDF step budget remains bounded;
• no scene-global mega-SDF path appears.

Exit gate:

• one body uses semantic detail grammar without violating proxy bounds.

Cut line:

• If form recursion destabilizes tracing, keep recursive detail in material payloads and leave traced form coarse.

Phase 10: Learned Scoring Research Gate

Goal: only add learned scoring after the estimator produces data.

Tasks:

• Add probe dataset writer.
• Capture camera paths, predicted scores, observed deltas, and costs.
• Analyze heuristic error offline.
• Try calibrated regression or tiny predictor.
• Add A/B mode against heuristic estimator.

Tests:

• held-out camera path comparison;
• learned mode beats heuristic fidelity/cost clearly;
• conservative summaries still own safety;
• fallback heuristic remains available.

Exit gate:

• learned scorer improves measured fidelity/cost without removing deterministic safety.

Cut line:

• If it does not beat the estimator clearly, delete it. No ornamental oracle.

Phase 11: GPU-Resident Splat And Reservoir Receipt

Goal: prove hot fractal splat and reservoir population belongs on the GPU, not in managed object graphs.

Tasks:

• Add packed GPU splat buffer contract with no per-splat strings or object ownership.
• Add native packed CPU buffer helper for the rare staging/readback paths that genuinely need C# memory.
• Add D3D12 compute receipt harness that writes millions of splats directly into a GPU-resident UAV.
• Add distinct GPU reservoir packet contracts for the first opaque slice: SDF envelopes, PBR material envelopes, and radiosity. These are backend packet names for Form, Appearance, and Transport, not the permanent conceptual architecture.
• Sample those three reservoir layers in independent compute passes under a stochastic per-frame update budget.
• Measure with GPU timestamp queries and read back only a tiny checksum sample.

Receipt:

• GTX 1070, 2,000,000 splats, 2,000,000 SDF/Form reservoirs, 2,000,000 PBR/Appearance reservoirs, and 2,000,000 radiosity/Transport reservoirs.
• Packet sizes: 80 bytes/splat, 64 bytes/reservoir.
• Budget: 50,000 updates per reservoir pass per frame, 2 candidates/update, full reservoir coverage in 40 frames.
• GPU time: 4.471 ms/frame.
• Equivalent throughput: 223.7 FPS, about 447.3M splats/sec.
• Command: .\scripts\fractal-splat-receipt.ps1 -Splats 2000000 -Frames 120 -Candidates 2 -ReservoirUpdates 50000.

Exit gate:

• Receipt must show plural-millions of GPU-resident splats and distinct reservoir layers populated or updated at hundreds of FPS equivalent on the target tired GTX 1070.

Cut line:

• This is compute-side population and cache-update throughput, not final shaded visibility. Do not sell it as renderer completion until the draw/raster/resolve path consumes the same packed buffers.
• The next coherent packet evolution is density/extinction Form plus participating-medium Appearance/Transport. Do not force flames or sensor confidence volumes through solid SDF semantics just because the first packet names said SDF.

First Work Packet

Start here:

1. Add Aquarium.Engine.Fractal and its test project.
2. Add cube-face/tile key types.
3. Add identity and tangent projection candidates.
4. Add area distortion sampler and face-edge tests.
5. Add a simple projection report fixture.

Definition of done:

• dotnet build Fensalir.sln
• new fractal tests pass;
• projection report is checked into test output or docs;
• state/evidence.jsonl records the chosen first projection default and why.