Repository navigation
The window's quality measurements into the repo: bench-window, render-identity, an orbit in bench-ghosting - #64
Conversation
…-identity, an orbit in bench-ghosting - tools/bench-window.ts: the play window against offline renders of the same moment, in the shots the window's quality work of 2026-11 was measured by: Bistro and Kabukichō held still at 1440p (detail, dE, SSIM, flicker), the Bistro slice half a second after its explosion, Tokyo at dusk (real time against the moment held), Tokyo turning and orbiting as the play window's left drag does (the tower and the whole frame, against offline of the last pose). Truths cached in out/cache/window-truth; the Tokyo shots need the city. - tools/render-identity.ts: test-render twice on each of two builds, every HDR output compared byte for byte; a test a build does not repeat passes within twice its run-to-run spread. Against itself: 172 byte-identical, 18 within their spread, 0 changed. - bench-ghosting: the play window's orbit (view.rs Orbit), reported, not yet guarded. - docs/WORKFLOW.md names both in the merge gate (R1: render-identity; R2 for the live passes: bench-window). On the way: upscale/lamps-in-motion-live never exercised the motion clamp (its lamps float before the black sky, which a sliding camera does not move). It changes with the fix of that cause; ROADMAP § 1a. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
Reviewed by Cursor Bugbot for commit a4f4d9e. Configure here.
| if (!existsSync(a) || !existsSync(b)) { | ||
| console.error("render-identity: --a and --b must name two threert-render binaries"); | ||
| process.exit(2); | ||
| } |
There was a problem hiding this comment.
Identity gate passes with no outputs
Medium Severity
render-identity exits 0 whenever changed is empty, including when it compared no HDR files. Missing --a/--b resolve to the working directory, which existsSync accepts, so test-render never produces outputs and the R1 byte-identity gate still reports success.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit a4f4d9e. Configure here.
There was a problem hiding this comment.
Fixed in ea1ada9: test-render takes --out DIR, each run of the gate writes a directory of its own and touches nothing else; --a / --b that are not files, or a run that leaves no outputs, exit 2.
| } | ||
| mkdirSync(dir, { recursive: true }); | ||
| for (const f of readdirSync(rendered)) if (f.endsWith(".raw")) cpSync(join(rendered, f), join(dir, f)); | ||
| } |
There was a problem hiding this comment.
Shared render dir mutated without lock
Medium Severity
run deletes and copies out/tests/render/*.raw without taking the GPU lock. Unlike bench-window, this tool never calls waitGpuLock, so it can wipe another test-render's outputs and copy files after that child has already released the lock.
Reviewed by Cursor Bugbot for commit a4f4d9e. Configure here.
There was a problem hiding this comment.
Fixed in ea1ada9: test-render takes --out DIR, each run of the gate writes a directory of its own and touches nothing else; --a / --b that are not files, or a run that leaves no outputs, exit 2.
…no pass on nothing compared Bugbot on #64: the gate passed when --a / --b were missing (they resolved to the working directory, test-render then left no outputs, and nothing compared counted as nothing changed); and it emptied and copied the shared out/tests/render without the GPU lock, so another test-render run could lose its outputs. test-render takes --out DIR (default out/tests/render); each run of the gate writes its own directory and touches nothing else. Binaries that are not files, or a run that leaves no outputs, exit 2. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Step 2 of the consolidation agreed on 2026-11-13. The measurements the play window's quality work was judged by lived in scratch scripts; they are tools now, so every later change runs the same gate. Tier T (tools and docs; the renderer is untouched).
What is added
tools/bench-window.ts: the play window against offline renders of the same moment. Offline truths are rendered once and cached inout/cache/window-truth.still-bistro,still-kabukichoslicetokyo-dusktokyo-turn,tokyo-orbitThe Tokyo shots need the city (
tools/tokyo); without it they are reported as not run.tools/render-identity.ts: the byte gate for changes that must not change a picture. It runs test-render twice on each of two builds and compares every HDR output byte for byte. A test that one build does not repeat to the byte passes when the two builds differ by no more than twice their own run-to-run spread;--toleranceadmits rounding (reported apart).bench-ghosting: a new move, the play window's orbit (view.rsOrbit). It is reported but not yet guarded.docs/WORKFLOW.md: render-identity joins the R1 gate; bench-window joins R2 for changes to the live passes.Checked on main's build
bench-window, all shots: 457 s on the first run (five truths rendered).
These are in line with the scratch tools' numbers.
render-identity, main against itself: 172 byte-identical, 18 within their run-to-run spread, 0 changed.
bench-ghosting
--quick --check, Bistro and Kabukichō: the guarded shots pass as before. The orbit, raw ΔE / trail:typecheck and
npm test159 pass.Found on the way
upscale/lamps-in-motion-live(from #60) never exercised the motion clamp. Its lamps float before the black sky, a sliding camera does not move the sky, and a lamp no sample hit takes the velocity of what is behind it: the upscaler saw no motion.With a black wall 1 cm behind, main keeps 0.045 of the lamps' 0.1 in motion. With no motion clamp at all it would be 0.105; before #60 it was 0.014.
The test will change together with the fix of the cause (a missed lamp reprojected by its own depth), so main does not carry a failing test meanwhile. Details in ROADMAP § 1a.
🤖 Generated with Claude Code
Note
Low Risk
Tier T: new TypeScript benchmarks and workflow docs only; no renderer or scene output changes in this PR.
Overview
Converts scratch play-window quality measurements into repo tools and wires them into the merge gate docs.
New tooling:
tools/bench-window.tsruns headless live sessions for the standard window shots (still Bistro/Kabukichō, Bistro slice, Tokyo dusk/turn/orbit), compares against cached offline truths inout/cache/window-truth, and records detail, ΔE, SSIM, flicker, etc.tools/render-identity.tsrunstest-renderon two renderer binaries, byte-compares every.rawHDR output, and allows drift within twice each build’s run-to-run spread (plus optional--tolerance).test-render.tsgains--outso identity runs can use isolated output dirs.bench-ghosting.tsadds an orbit camera move matching the play window’s left drag; it is measured and reported but not yet guarded.docs/WORKFLOW.mdanddocs/ROADMAP.mddocumentrender-identityfor R1 andbench-windowfor R2 live-pass changes.Reviewed by Cursor Bugbot for commit ea1ada9. Bugbot is set up for automated code reviews on this repo. Configure here.