Aquarium/docs/tsr-inspired-taa-spec.md
2026-05-13 04:53:58 +01:00

288 lines
10 KiB
Markdown

# TSR-Inspired TAA Spec
Aquarium needs an owned temporal resolver, not a copied implementation. The
target is inspired by public TSR behavior: stable subpixel detail, aggressive
history validation, visible debug terms, and special handling for pixels whose
appearance is not fully described by motion vectors.
## Inputs
The resolver owns these signals:
- current HDR scene color
- current ray travel
- previous HDR history color
- previous ray travel
- current camera/Grid constants
- previous camera/Grid constants
- current and previous projection jitter
- stochastic coverage from diegetic Grid surfaces
Later gates add:
- field id
- normal
- explicit velocity
- reactive mask
- transparency/composition mask
- resurrection history
## Current Pass Contract
The scene pass renders into an HDR texture:
```text
rgb = linear scene color
a = ray travel in world units
```
The pass does not tonemap. Tonemapping belongs after temporal resolve.
## Reprojection
For the first implementation, motion is camera-derived:
1. Reconstruct the current world hit from current camera, current jittered ray,
and current travel.
2. Project that world hit into the previous camera basis.
3. Remove previous jitter to find the previous history UV.
4. Sample previous history if the UV is valid.
5. Compare previous history travel against the reprojected point's expected
distance from the previous camera, not against current-frame travel.
This is sufficient for camera motion and stochastic Grid coverage. It is not the
final answer for animated bodies or field animation.
## Rejection
Reject or weaken history when:
- previous UV is outside the frame
- travel differs too much
- current or previous travel is invalid
- current neighborhood color bounds do not contain the reprojected history
The first pass uses a 3x3 neighborhood min/max clamp and travel-difference
weighting. Later passes add field/material ids, normals, disocclusion masks, and
reactive masks.
## Resolve
The resolver outputs both:
- tonemapped final backbuffer color
- linear HDR history for the next frame
Current blend target:
```text
resolved = lerp(current, clamped_history, history_weight)
```
History weight starts conservative. Stochastic Grid coverage needs accumulation,
but ghosting is worse than shimmer while the renderer lacks full velocity,
material id, and reactive masks. The resolver should earn trust before taking
more history.
## Non-Negotiables
- Do not use Playdead-style TAA as the final model; it ghosts too easily for the
scene we are building.
- Do not vendor Unreal TSR code. Study behavior and build a clean-room resolver.
- Do not hide missing renderer signals with larger blend weights.
- Do not make the Grid a final-image HUD layer; it is diegetic scene UI and must
respect nearer solids.
## Implementation Gates
### Gate 1: Owned Temporal Pass
- HDR scene target
- history ping-pong
- camera jitter
- current/previous camera reprojection
- travel rejection
- 3x3 color clamp
- final tonemap after resolve
### Gate 2: Renderer Signals
- material/field id buffer
- normal buffer
- explicit velocity for animated SDF bodies
- coverage/reactive masks for stochastic Grid and future particles
Current Gate 2A implementation:
- scene metadata target stores stable field id plus surface normal
- history metadata ping-pongs alongside color/travel history
- field id mismatch rejects history
- normal mismatch weakens/rejects history
- Grid stochastic surfaces write Grid travel/normal even when the current
dither sample does not draw color, so the temporal resolver accumulates the
diegetic surface rather than reprojecting from the solid behind it
Current Gate 2B implementation:
- Self, Grid, and each orbiting planet have distinct field ids
- orbiting planet hits map their current world hit back to the planet's previous
center before camera reprojection
- this gives analytic SDF bodies object motion without a full velocity buffer
yet
Still missing:
- separate field ids once the SDF field registry exists
- reactive/coverage masks
- richer disocclusion classification
- a general velocity buffer for non-rigid fields and deformation
### Gate 3: Stochastic Event Control
Current Gate 3A implementation:
- scene pass writes a temporal-control target separate from color/travel and
field/normal metadata
- temporal control currently stores:
```text
x = reactive strength
y = stochastic/composition coverage
z = reserved
w = reserved
```
- Grid coverage feeds the coverage channel
- low-coverage Grid samples raise reactive strength and reduce history weight
- the resolve combines reactive and coverage weights with travel, field, normal,
and neighborhood-color validation
Current Gate 3B implementation:
- temporal-control history ping-pongs alongside color/travel and field/normal
history
- resolve samples previous-frame temporal control at the reprojected UV
- coverage discontinuity reduces history
- current temporal control is written into the history set for the next frame
Current Gate 3C implementation:
- temporal-control history `w` stores accepted history age in frames
- validation grows age when reprojected history survives the travel, field,
normal, color, and coverage checks
- history authority ramps with age, so newly accepted history can contribute but
does not immediately get the same weight as stable history
- age resets to zero on validation failure
Current Gate 3D implementation:
- travel validation uses the reprojected previous-world position's distance from
the previous camera
- raw current-frame travel is no longer compared directly to previous history
travel, because camera motion changes the ray distance even when the world hit
is stable
Current Gate 3E implementation:
- stochastic Grid history bypasses opaque-surface neighborhood color clamping
and current-frame color-delta rejection
- Grid validation still uses field id, travel, normal, coverage continuity,
reactive weight, and history age
- this prevents a current dither miss from declaring the true neighborhood to be
background and crushing the very history that should accumulate stochastic
coverage
Current Gate 3F implementation:
- stochastic Grid history uses a high-retention blend policy distinct from
opaque surfaces
- Grid history weight no longer scales down with current coverage or reactive
strength; low coverage is exactly where temporal accumulation is needed
- opaque surfaces keep the stricter color/reactive validation path
- debug mode `2` should now show a smoother history signal than raw scene mode
`1` when the Grid is stable
Current Gate 3G implementation:
- identity/control buffers are point-loaded, not linearly sampled
- field id equality is exact after point-loading, so Grid/Self/planet boundaries
hard-reject mismatched history
- Grid history color is sampled with the same nearest pixel used for metadata so
bright foreground silhouettes do not bilinearly bleed into background Grid
history
- Grid max history weight is reduced from the too-soft `0.975` to `0.94`
Current Gate 3H implementation:
- Grid temporal `coverage` is now line/contour/field-line support, not broad
overlay alpha
- broad weather/field tint can still render through stochastic coverage, but it
does not keep history age alive across the whole Grid surface
- Grid history is gated by current support, so stale stochastic hits cannot
billow away from the analytic line support
- Grid max history weight is reduced again to `0.90` with a lower fresh-history
scale for crisper line response
Current Gate 3I implementation:
- Grid resolve reconstructs the analytic Grid overlay at the current world hit
and uses premultiplied overlay color as the current color estimate
- the scene pass may still use stochastic coverage, but the temporal resolver
no longer treats the current binary hit/miss sample as the true radiance
- debug mode `1` can remain noisy because it shows the raw stochastic scene;
mode `0` should now use the analytic Grid estimate plus validated history
Current Gate 3J implementation:
- debug mode `1` is restored to raw current scene color, before Grid analytic
reconstruction
- Grid final color no longer blends previous color history; it uses the analytic
current Grid estimate and writes that estimate into history
- history/age/weight debug modes remain useful diagnostics, but stale Grid color
history no longer drives the final visible Grid
Current Gate 3L/3M implementation:
- projection jitter uses a small Halton sequence, currently scaled below half a
pixel so the resolve gets sample diversity without visible whole-frame wobble
- current and previous jitter are both uploaded through frame constants; previous
history projection removes the previous jitter before sampling history
- final presentation blends validated temporal color history in mode `0`
- SDF object highlights have a narrow hot-current path: when the current SDF
sample is much brighter than valid reprojected history, color rejection is
relaxed so history can damp transient specular fireflies while travel, field,
normal, coverage, and coverage-continuity validation still apply
- bloom presentation is attenuated when the resolved scene luminance is much
lower than the raw current scene luminance, so current-frame fireflies do not
keep blooming after temporal history has damped the underlying scene color
- bloom prefilter clamps unsupported single-pixel HDR spikes against local
neighborhood luminance before downsample/blur, so one-frame fireflies do not
spread into neighboring pixels before the temporal resolve sees them
- modes `2` through `4` show the sampled history, accepted history age, and
history weight
Still missing:
- reactive reset for lighting and density changes
- separate history policy for opaque surfaces and stochastic surfaces
### Gate 4: TSR Features Worth Stealing In Spirit
- flicker analysis for high-frequency linework and stochastic coverage
- history resurrection for reappearing stable details
- debug visualizations for rejection, clamp, velocity, travel, and history age
Current debug view controls:
- `F1` cycles temporal debug modes in the running window
- `0` final resolved color
- `1` raw current scene color
- `2` reprojected history color used by the resolver
- `3` accepted history age
- `4` history weight
- `5` current temporal control: reactive in red and Grid support/coverage in
green
- `6` current field identity palette for Grid, Self, and planets
- `7` bloom/veil contribution after exposure and before tonemapping
- `8` exposed luminance bands
- `--render-debug` and `AQUARIUM_RENDER_DEBUG_MODE` set the startup mode