Skip to content

fix(export): remap camera moves onto export time - #50

Merged
shreyaskarnik merged 3 commits into
shreyaskarnik:mainfrom
Joilence:pr/camera-move-remap
Sep 15, 2026
Merged

shreyaskarnik merged 3 commits into
shreyaskarnik:mainfrom
Joilence:pr/camera-move-remap

Conversation

@Joilence

@Joilence Joilence commented Sep 1, 2026 •

Copy link
Copy Markdown
Contributor

Why

  • Camera move times are recorded at wall clock, then handed to zoompan as between(in_time, ...) windows. The speed ramp rewrites the timeline upstream of that filter: the ramp's trim/setpts/concat runs first, and the camera filter reads its output. So in_time is the export timestamp while the window still describes the recording.

  • Placements already make that trip, through applySpeedRampToTimeline and adjustPlacementsForFreezes. Camera moves made neither: pipeline.ts:420 shifts them for head trim and nothing else. Every zoom fires late by the gap time removed ahead of it, so the last move in a video carries the whole accumulated error. Nothing errors; the filter renders, it just points at the wrong moment.

D-camera-move-side-by-side-orig.mp4

What

  • remapCameraMoves maps a move onto the export clock; exportTimelineRemap composes the two rewrites it has to survive, the ramp and any freezes. Wired through all four export paths per the CLAUDE.md requirement that every export feature reach all of them, sharing one mapping rather than four copies that can drift.
  • remapTimeMs now continues past its last segment at that segment's speed instead of saturating. Clamping suits a placement, which cannot outlive the timeline; it is wrong for measuring a rate, because a move's zoom-out tail can overhang the recording end and the saturated endpoint charged that overhang against the few ms left. A closing zoom in an uncompressed final scene rendered its 400ms ease in 67ms. No placement reaches the new branch.
  • Guards a fade rounding to zero: buildCameraMoveFilter divides by it, and (in_time-S)/0.0000 is a move ffmpeg renders as doing nothing.

Known approximation: the span ratio is uniform across zoom-in, hold and zoom-out, so a move whose hold alone spans a compressed gap has its fades compressed too. Remapping also shrinks the gaps detectChainedPairs measures, so moves that bounce on main can now chain into a pan. Both are bounded and much closer than no remap.

Test

  • 774 -> 783 tests

Joilence and others added 3 commits August 31, 2026 23:51
Camera move times are recorded at wall clock, then handed to zoompan as
`between(in_time, ...)` windows. The speed ramp and freezes both rewrite
the timeline upstream of that filter, so in_time is the export timestamp
while the window still describes the recording.

Placements already made that trip through applySpeedRampToTimeline and
adjustPlacementsForFreezes. Camera moves made neither, so every zoom
fired late by the total gap time removed ahead of it, stepping further
out at each compressed gap and leaving the last move in a video carrying
the whole accumulated error.

Both ends are remapped, not just the start. A move spanning a compressed
gap occupies less time on the output timeline than it did on the
recording, so shifting without shrinking would leave the zoom running
past the content it belongs to. The scale factor is exactly 1 for a move
inside one scene, which is the common case.

Wired through all four export paths, pipeline, CLI, preview and viewport
variants, per the requirement in CLAUDE.md that every export feature
reach all of them or the outputs diverge silently. They share one
exportTimelineRemap rather than four copies that can drift apart.

remapTimeMs now continues past its last segment at that segment's speed
instead of saturating there. Clamping is right for a placement, which
cannot outlive the timeline, and wrong for measuring a rate: a closing
zoom whose zoom-out tail overhangs the recording end had the whole
overhang charged against the few ms of timeline left, rendering a 400ms
ease in two frames. No placement reaches that branch, so the ramped
timing of everything else is unchanged.

Also guards a fade rounding to zero. buildCameraMoveFilter divides by
the fade duration to build its progress ramp, so a 0 yields
`(in_time-S)/0.0000`, which ffmpeg accepts without complaint and renders
as a move that silently does nothing.

remapCameraMoves and exportTimelineRemap are exported from the package
root so a consumer assembling its own export can make the same trip.

Known approximation: the span ratio is uniform across a move's zoom-in,
hold and zoom-out, so a move whose hold alone spans a compressed gap has
its fades compressed too, and a chained move is measured over a zoom-out
the filter never renders. Both are bounded and much closer than no remap.
`remapCameraMoves` scaled the zoom-in, hold and zoom-out by a single
start-to-end ratio. That is right for a uniform speed change and wrong for
a freeze, which inserts time at one instant: a freeze inside the hold
stretched both zooms in proportion to its own length.

  move 1000ms, 300 in / 1400 hold / 300 out; 5000ms freeze at 1500
  before: zoom-in ends 2050, zoom-out starts 6950   (should be 1300 / 7700)

A freeze starting exactly at a move's end also counted against the move,
stretching one that `main` left correct.

Each phase boundary is now remapped on its own. The end is remapped from
just inside the move, since the move covers [start, end) and content at
that instant belongs to what follows. A single `durationMs` still drives
both fades, so when they remap differently (a ramped gap under only one)
it takes their mean, which keeps the hold and the total span exact.

Tests for both freeze cases fail without this change; the existing
speed-ramp cases pass unchanged.
@shreyaskarnik

Copy link
Copy Markdown
Owner

Thanks @Joilence. This fixes a real problem: camera moves were matched against recording time while zoompan sees export time, so any speed ramp or freeze drifted them off their content. Putting both corrections in one exportTimelineRemap shared by every export path is the right structure. It's exactly the "wired into some paths and not others" divergence this repo keeps running into. The comments on why durationMs is never rounded to zero, and why a closing move keeps its authored speed, are very clear.

I pushed one commit to your branch (70cbf70) for the one case it didn't handle. remapCameraMoves scaled the zoom-in, hold and zoom-out by a single start-to-end ratio. That works for a uniform speed change, but a freeze inserts time at a single instant, so a freeze inside the hold stretched both zooms by an amount proportional to its own length:

move at 1000ms: 300 in / 1400 hold / 300 out, with a 5000ms freeze at 1500
before: zoom-in ends 2050, zoom-out starts 6950   (should be 1300 / 7700)

A freeze starting exactly at a move's end also stretched a move that main had left correct.

Each phase boundary is now remapped on its own, and the end is remapped from just inside the move, since a move covers [start, end). A single durationMs still drives both fades, so when they remap differently (a sped-up gap under only one of them) it takes their mean, which keeps the hold and the total span exact. Tests for both freeze cases fail without the change, and all of your speed-ramp tests pass unchanged.

Merging. Thanks again!

@shreyaskarnik
shreyaskarnik merged commit 5dd07e4 into shreyaskarnik:main Sep 15, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants