Skip to content

The thin feature's lock: lamps finer than a pixel keep their light, held still and in motion - #60

Merged
yujiaxu-ops merged 3 commits into
mainfrom
upscaler-motion
Nov 13, 2026
Merged

yujiaxu-ops merged 3 commits into
mainfrom
upscaler-motion

Conversation

@yujiaxu-ops

@yujiaxu-ops yujiaxu-ops commented Nov 13, 2026 •

Copy link
Copy Markdown
Collaborator

What was wrong

Dragging the view over Tokyo at night, the tower broke up: its legs and most of its lamps gone while the camera turned, back half a second after it stopped. Measured against offline renders, these are two faults of the upscaler, both older than #59.

  • Held still, a field of lamps a third of a pixel wide kept a third of its light. judge takes a surface for gone from in front when what was nearest among the nine samples a frame ago is nearer than what is nearest now, and the history's colour is not in this frame. A lamp the jitter hit last frame and missed this one is exactly that, so its history was dropped every frame it was missed.
  • In motion the history is clamped into the frame's nine samples, and a lamp none of them hit is outside them. The 4-sample cap in motion is not the cause: without it nothing changed.

The change

The remedy is the one FSR 2 (pixel locks) and UE's TSR (thin geometry detection) have: where the picture is a thin feature, the rules that hold the history to this frame's samples stand back.

Lock in live.wgsl is a half word per traced pixel, in space the fast mean's buffer had free:

  • compose makes it where the sample's emission is thin: no 2 × 2 block of the nine like it (FSR 2's rule), and above or below both of some pair of opposite neighbours.
  • It lasts a jitter sequence, is carried with the surface, and is never set on animated shading.
  • A locked pixel's history is not clamped in motion (the lock of any of its nine samples counts).
  • A locked sample's distance from the history's range is no finding.
  • Nothing is gone where one of the nine samples is locked.

The test runs on emission (what the camera ray saw directly) because that part of a traced sample has no noise. FSR 2's test on the composed sample called most of a night street "thin".

Measured

main this offline
lamps field held still, mean radiance 0.035 0.106 0.103
the same, the camera sliding 1.4 px a frame 0.027 0.112 0.103
Tokyo Tower in a turn, ΔE / SSIM / luminance against offline 2.81 / 0.867 / 0.77 2.23 / 0.928 / 1.05
the same, the whole frame 2.69 / 0.889 / 1.01 2.56 / 0.915 / 1.06
the tower half a second after the stop 1.89 / 0.948 / 1.07 1.47 / 0.964 / 1.00
the tower settled 1.48 / 0.962 / 0.94 1.39 / 0.965 / 0.99
  • test-render 190 pass, cargo test --release --locked 114 + 2, npm test 159, typecheck clean.
  • New tests upscale/lamps-still-live and upscale/lamps-in-motion-live: main fails both (0.021 and 0.027 of 0.1).
  • compose + judge + upscale: 1.73 → 1.80 ms at 1440p output.

What it costs

Where a view translates past emitters, the locked ones trail a little. bench-ghosting still passes on both scenes.

main this
Kabukichō signs, mid-strafe 6.42 6.67
Kabukichō signs, 0.1 s after the strafe (trail) 5.27 (26 px) 5.55 (31 px)
Kabukichō signs, at the dolly's stop 6.75 7.07
Bistro whole frame, 0.1 s after the strafe (trail) 2.49 (13 px) 2.63 (24 px)
Bistro slice, half a second after its explosion 5.28 5.41
Thames orbit, moving / held 2.24 / 1.34 2.29 / 1.24
Tokyo at dusk, skyline p99 at 6 s 11–14 16–18
  • Kabukichō's ground is the same to 0.01 in every shot; its whole frame is within 0.08.
  • Bistro's other shots are within 0.05.
  • Held still, the fire shot and the lighting-edit shot are as they were.

Left

  • Thin things that do not emit (a wire, a lattice by day) are still taken for gone under the jitter.
  • The locked lettering of a still sign comes back from a motion's blur by the blend alone.
  • A lamp that comes on arrives over a quarter of a second.

What was tried and left out is in docs/ROADMAP.md § 1a.

🤖 Generated with Claude Code


Note

Medium Risk
Changes core temporal upscaler behavior (compose, judge, upscale) with intentional ghosting tradeoffs on fast translation; validated by new tests but affects all live upscaled output.

Overview
Adds a thin-feature lock (Lock in live.wgsl) so sub-pixel emitters (lamps, lit windows) are not discarded by temporal upscaling when jitter misses them or motion clamps history into the nine-sample neighborhood.

compose detects thin emission (FSR2-like local similarity + opposite-neighbor contrast), persists lock state in spare ae_in f16 bits for a jitter sequence, and skips animated shading. judge no longer treats locked samples as disoccluded “gone” or penalizes history-range distance; upscale relaxes in-motion history clamping by LOCK_RELAX (0.75) when any contributing sample is locked; reject propagates max lock across the nine neighbors.

docs/ROADMAP.md documents the problem, tuning tradeoffs, and benchmark deltas (e.g. lamp field radiance 0.035→0.106 still, Tokyo Tower metrics improved). New render tests upscale/lamps-still-live and upscale/lamps-in-motion-live guard mean radiance (~0.1) for procedural sub-pixel lamp fields.

Reviewed by Cursor Bugbot for commit fa89a24. Bugbot is set up for automated code reviews on this repo. Configure here.

yujiaxu-ops and others added 3 commits November 13, 2026 16:36
…eld still and in motion (ROADMAP § 1a)

Two faults of the upscaler, measured against offline renders, both older than the flicker tolerance:

- Held still, a field of lamps a third of a pixel wide kept a third of its light. `judge` takes a surface for gone
  from in front when what was nearest among the nine samples a frame ago is nearer than what is nearest now and the
  history's colour is not in this frame: a lamp the jitter hit last frame and missed this one. Its history was
  dropped every frame it was missed.
- In motion the history is clamped into the frame's nine samples, and a lamp none of them hit is outside them: Tokyo
  Tower in a turn lost its legs and a quarter of its light.

`Lock` (FSR 2's pixel locks; UE's TSR loosens its clamp the same way for thin geometry): a half word per traced
pixel, made where the sample's emission is a thin feature, a jitter sequence long, carried with the surface, none on
animated shading. A locked pixel's history is not clamped in motion, a locked sample's distance from the history's
range is no finding, and nothing is gone where one of the nine samples is locked.

The lamps field: 0.035 to 0.106 still, 0.027 to 0.112 moving (offline 0.103). The tower in a turn against offline:
dE 2.81 to 2.23, SSIM 0.867 to 0.928, luminance 0.77 to 1.05. Cost: where a view translates past emitters the
locked ones trail a little (Kabukichō's signs 6.42 to 6.67 in a strafe, Bistro 2.49 to 2.63 a tenth of a second
after one, the slice's explosion 5.28 to 5.41); bench-ghosting passes on both scenes.

Tests: upscale/lamps-still-live, upscale/lamps-in-motion-live.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…75 a frame): no second copy of a lamp

The first form spared a locked pixel's history from the motion clamp entirely. In a quick orbit of the play window
(a degree a frame about a point 50 m ahead) a lamp the samples missed reprojected with what is behind it, and the
spared history became a second copy: Tokyo Tower's beacon doubled, its lamps drawn out sideways. TSR caps how far it
loosens its clamp for thin geometry; here 0.75 of the unclamped history is spared a frame and what no sample
supports fades by a quarter a frame.

Tokyo Tower against offline, dE / luminance: the quick orbit 4.86 / 1.20 to 4.15 / 0.86 (main 5.06 / 0.71); the
slow turn 2.23 / 1.05 to 2.17 / 0.80 (main 2.81 / 0.77). Kabukichō's signs a tenth of a second after a strafe 5.55
to 5.39 (main 5.27), Bistro 2.63 to 2.54 (main 2.49).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant