Faset_Engine/docs/superpowers/specs/2026-09-24-p3-temporal-design.md

P3 temporal reconstruction design

This design implements the temporal half of PLAN P3 for Linux and Windows desktop games. The outcome is a selectable, observable 1:1 TAA path followed by a lower-internal-resolution temporal upscaler. The existing Direct renderer without temporal processing remains the reference and the default. P3 lighting and shadow work has its own design and can change the scene shader's lighting inputs without changing the temporal pass contract.

Temporal renderer and shader integration starts after the lighting plan's Task 2 checkpoint commit, Share typed multi-light shading across Direct and GPU paths. That commit establishes material descriptor set 0, frame-lighting set 1, and GPU graphics scene set 2; GPU cull/HZB compute descriptors remain in set 0. Temporal graphics variants consume the same material and lighting sets as fragmentMain, add prior-transform reads only from the GPU scene set 2, and leave the 96-byte Direct and 112-byte GPU graphics push constants intact. Temporal resolve/composite use their own checked pipeline layouts. The shared reflection and hot-reload validator must accept the combined ABI, not reconstruct the pre-lighting set 1 layout.

Boundaries and modes

RendererConfig selects TemporalMode::Off, TAA, or Upscale; Upscale also specifies an internal render scale in [0.5, 1). TAA uses scale 1. A runtime setter lets Editor and Player select the same modes without changing scene authoring files. Requested and effective modes are recorded separately: unsupported compute/format/extent capabilities produce an explicit Off fallback and a readable reason. Invalid scale is rejected as configuration input, rather than silently clamped. An exported Player contains cooked SPIR-V and reflection; Slang is not needed at runtime. Temporal processing is renderer-side and adds no live-game MCP endpoint.

The output extent remains Renderer::width()/height(), the swapchain/capture extent and the pixel coordinate system of UI. At scale below one, opaque and world-space raster targets use an internal extent, with integer rounding made deterministic. Snapshot::scene_rect remains in output pixels; it is converted once to the internal viewport. Camera aspect and projection remain those of the output scene rectangle. Shadow map dimensions and light-space matrices never depend on render scale. HZB dimensions, viewport metadata, culling projection, and post-cull depth refer to the actual internal scene target.

Frame data and history ownership

The caller supplies an unjittered Snapshot::view_projection and projection. Picking, gizmos, and authoring coordinates continue using them. The renderer chooses a deterministic Halton(2,3) subpixel offset, applies it only to the scene raster clip transform, and retains both current and previous jittered view-projection matrices. The offset is measured in internal scene pixels and enters clip space as clip.xy += 2 * jitter / internalViewportSize * clip.w; the sign is verified by an image/motion test for the Vulkan viewport convention. Culling and HZB projections must match the jittered geometry they test, with conservative edge handling. Snapshot::camera_cut is authoritative; a large discontinuity in camera position/orientation also rejects history when a caller fails to mark a teleport.

InstanceTracker already records previous_model, world bounds, a stable slot/generation, mesh identity and rendered-frame completion. That state becomes available to both Direct and GPU paths. HZB history and temporal color history have independent validity decisions; absence of HZB or a Direct visibility mode cannot invalidate an otherwise valid TAA history. Mesh/LOD replacement, reused slot, an anonymous draw, camera cut and changed view make that instance's velocity invalid. GPU InstanceRecord gains a previous model matrix while keeping the existing stable slot/generation lanes. The metadata.x low bit remains the HZB eligibility bit and a distinct bit marks temporal previous-transform eligibility; culling tests only the HZB bit. Dense candidate indices remain frame-local addresses, not temporal identities.

Temporal color/depth history is renderer-owned and double-buffered at output resolution. Each completed render commits one frame. A pending failed render must not promote history. The current active view may reuse history only if the previous frame has the same view_id, output/internal extents, scene rectangle, unjittered projection, temporal mode/scale, and compatible shader generation, and is not a cut. Switching among views safely resets history; retaining multiple cached histories is not required. Resize, shader reload, mode or scale change, and camera discontinuity reset it explicitly. The first valid frame uses current color only. The reset reason and whether a previous history was actually used are visible in FrameStats and Editor diagnostics.

Motion and reconstruction signal

Both opaque mesh paths write the same per-pixel motion contract:

• Direct path: the CPU-transformed vertex carries current clip position, prior clip position computed from InstanceUpdate::previous_model and prior jittered view-projection, plus a validity flag. The existing Off vertex format/shader remains a compatible path.
• GPU path: the vertex shader reads current/previous model matrices from the instance record and current/previous jittered view-projections from the view record. It emits current and prior clip coordinates plus validity. Both Main and Post raster use this contract.
• The scene fragment writes display-UV motion (currentUV - previousUV), expected depth in the previous projection, and validity/reactive state to a sampled floating-point MRT. A missing prior transform, invalid clip w, nonfinite projection, or non-opaque/reactive fragment marks motion invalid. UI never contributes motion. UVs are local to the scene rectangle so internal and output extents map consistently.

The scene image remains the current renderer's display-referred LDR shading for this stage. History uses a higher-precision floating-point color image so repeated accumulation does not quantize to 8 bits; the final composite writes the existing R8G8B8A8_UNORM output exactly once. There is no exposure adaptation or HDR claim in this temporal implementation. P3 lighting may later move scene shading to linear HDR behind the same sceneColor/depth/velocity -> resolvedColor boundary.

The signal checklist is informed by AMD's FSR 2 integration guide: it describes current color, depth, motion and reactive data, and explicit reset on camera cuts. Faset implements its own resolver and does not link FSR 2. Faset's current-minus-previous local UV motion sign and jitter-inclusive convention are its own contract; an FSR 2 adapter would have to convert to that API's documented motion and jitter conventions. NVIDIA's Adaptive TAA paper identifies blur and ghosting as temporal failure modes, which motivates the moving-image acceptance tests here; its adaptive ray-tracing algorithm is not part of this design.

The 1:1 TAA resolve samples current color/depth/velocity, dilates motion from the nearest depth in a 3×3 neighborhood at silhouettes, and computes previousUV = currentUV - motion. It rejects history when view history or fragment motion is invalid, coordinates leave the previous scene rectangle, or previous expected depth disagrees with sampled history depth beyond a depth-aware tolerance. Surviving history color is clipped to a current-color 3×3 neighborhood and blended with a bounded weight reduced by motion and reactive content. A newly exposed background pixel must use current color immediately; it may accumulate normally on following frames. The resolve writes color and depth to the next output-resolution history image. Sky/clear pixels use current color when reliable reprojection is unavailable.

Upscale reuses this validated resolve but samples current color/depth/velocity from a lower internal extent. It reconstructs the current signal at output-pixel positions, keeps output-resolution history, and uses the same rejection/dilation and first-frame spatial fallback. Switching render scale resets history. TAA must pass its moving-image tests before upscaling is enabled or described as complete; this path is a small, measured temporal upscaler, not a promise of Unreal TSR quality.

Render graph and scene/UI split

The temporal path adds internal sceneColor, sceneDepth, and sceneVelocity targets while retaining color as the full-resolution final image used by Readback and Presentation. Existing Off rendering and its one-target shader remain available. The temporal graph orders passes as follows:

1. ShadowMap and optional P2 MainCull.
2. Main scene raster clears scene color/depth/velocity and renders opaque meshes. Optional P2 BuildCurrentHZB and PostCull run from this scene depth, then PostRaster loads all three scene attachments and finishes deferred opaque meshes.
3. World transparency and sprites render against the same internal scene depth before resolve. Pixels they blend into color mark their velocity/reactive value invalid, so stale opaque history cannot leak through moving translucent content. They do not acquire synthetic rigid-mesh motion. The existing ordering and depth behavior are retained.
4. TemporalResolve reads the completed scene image/depth/velocity and previous history, writes the next color/depth history. TemporalComposite maps that resolved scene to the full-size color target.
5. A separate output-resolution UI pass renders quads, text, gizmo overlays and Editor chrome on color, without jitter or TAA. Readback and Presentation then consume color as before.

Only the temporal-enabled branch needs new scene targets and a second color attachment in opaque pipelines. A one-target UI pipeline remains valid. RenderGraph validates ordering by names; Vulkan layout transitions and compute→sampled/attachment barriers are recorded explicitly. The renderer's single queue, fence-per-frame model makes history lifetime simple but its synchronous capture/readback cost remains part of full-frame timings. The present 12-slot GPU timestamp pool must grow enough to time every new pass instead of silently omitting the end of the graph.

Shader package, controls and fallback

Temporal vertex/fragment, resolve and composite entry points have validated reflection, SPIR-V hash, descriptor types and record strides. Adding previousModel changes the GPU instance stride; the C++ static assertion, Slang record, reflection validator and GPU tests change together. Baseline Off shaders retain their existing entry points and layout fingerprints. Shader reload replaces a complete compatible temporal pipeline set atomically or preserves the working set; successful reload invalidates color history. BuildService copies every temporal .spv and .reflection.json into Linux/Windows exports and validates a relocated Player without Slang installed.

Editor diagnostics expose mode, scale, effective mode/fallback, internal extent, jitter, history accepted/reset reason, reject statistics when requested, GPU temporal pass time and memory. Player profiling records these same fields per frame. GPU readback for per-pixel diagnostics is optional and never required for the normal resolve. If temporal resources cannot be created on a supported Vulkan 1.3 device, the actual mode is Off, the frame still renders by the existing path, and the reason is reported. A failed temporal shader reload keeps the prior pipelines and does not corrupt history or change the visible mode.

Acceptance

Deterministic headless GPU sequences run for Direct, GPU frustum and GPU occlusion where supported. They cover static thin diagonals/wires at subpixel positions, slow camera pan, a moving rigid object, a doorway disocclusion, an explicit camera cut and unmarked teleport, projection/view/scene-rectangle/resize/mode/scale changes, anonymous and mesh-replaced instances, world transparency, sprites and pixel-exact UI. Tests compare current frames to Off and a high-resolution spatial reference, with fixed tolerance and documented image metrics rather than requiring bit-identical output across drivers. They assert zero Vulkan validation errors where the layer is present, no reappearance of old color after cut/reveal, correct actual mode and reset reason, and no UI softening. A clean fallback is exercised on a deliberately unsupported capability fixture.

Run CPU policy tests, reflection/package/tamper tests, Linux validation on physical GPU and software Vulkan, Windows software Vulkan CI and native builds, and relocated 2D/3D Player captures. Record the exact revision, driver/GPU, build mode, internal/output resolutions, temporal and full-frame GPU/CPU costs, memory, image differences, and limitations. These results gate any claim that TAA/upscaling improves image quality or frame time in a particular scene.