Skip to content

fix(core): resolve a relative data-start inside the clip's own composition - #5336

Open
jayhemnani9910 wants to merge 3 commits into
heygen-com:mainfrom
jayhemnani9910:fix/relative-start-own-composition
Open

jayhemnani9910 wants to merge 3 commits into
heygen-com:mainfrom
jayhemnani9910:fix/relative-start-own-composition

Conversation

@jayhemnani9910

Copy link
Copy Markdown

What

A relative data-start (data-start="first", "first + 0.5") now looks for the referenced clip inside the referencing clip's own composition first, and only falls back to the document-wide lookup when nothing there matches.

Why

The docs say references resolve inside the same composition, and hyperframes timeline and the audio mix already do that. The runtime start resolver used document.getElementById, so once two compositions in the rendered document contain the same id, every reference to it resolves against the first one in the document.

The common way to hit this is mounting one block twice. Each instance keeps the block's own ids, so the second instance's relative clips are timed off the first instance and drop out of the render.

Repro on 0.8.142 (also on current main):

compositions/card.html

<template id="card-template">
  <div data-composition-id="card" data-width="320" data-height="180" data-duration="2" data-no-timeline>
    <div id="first" class="clip" data-start="0" data-duration="1" style="position:absolute;inset:0;background:#00f"></div>
    <div id="second" class="clip" data-start="first" data-duration="1" style="position:absolute;inset:0;background:#0f0"></div>
    <audio id="beep" src="tone.wav" data-start="first" data-duration="1" data-media-start="3"></audio>
  </div>
</template>

index.html root:

<div id="root" data-composition-id="main" data-start="0" data-duration="4" data-width="320" data-height="180" data-no-timeline>
  <div id="card-a" data-composition-id="card" data-composition-src="compositions/card.html" data-no-timeline data-start="0" data-duration="2" data-track-index="1"></div>
  <div id="card-b" data-composition-id="card" data-composition-src="compositions/card.html" data-no-timeline data-start="2" data-duration="2" data-track-index="2"></div>
</div>

(tone.wav is any audio file longer than 4 s.)

hyperframes lint is clean, and hyperframes timeline shows second at 1-2 s in card-a and 3-4 s in card-b. In hyperframes render --fps 10, sampling the centre pixel per frame:

  • Expected: blue 0-1 s, green 1-2 s, blue 2-3 s, green 3-4 s.
  • Actual: blue 0-1 s, green 1-2 s, blue 2-3 s, black 3-4 s.

The beep plays at 1-2 s and 3-4 s as expected, so in card-b the sound plays over a frame where its clip is missing. card-b's second resolves to 1 s (the end of card-a's first), a time when card-b is not on screen yet.

Related work

No existing issue or PR found. #4481 fixed the same repeated-id problem for Studio's timeline rows. #4334 is a different startResolver problem (the walk-up from a wrapped mount) and does not overlap.

How

findReferenceTarget takes the referencing element. It finds that element's composition (for a composition host, its parent composition) and returns the first element with the referenced id whose own composition is the same one. If none matches, the existing getElementById and data-composition-id lookups run unchanged, so a reference that only exists elsewhere in the document resolves as before.

Test plan

  • Unit tests added/updated: startResolver.test.ts mounts one composition twice and checks the second instance's referenced clip starts at 3 s. It fails without the change (resolves to 1) and passes with it.
  • Manual testing performed: the repro above renders blue, green, blue, green with the beep under both green spans.
  • Package suites match main; oxlint, oxfmt, tsc, fallow and the comment checks pass.
  • Documentation updated (if applicable)
  • Comments follow CONTRIBUTING.md "Comments"

Copilot AI balanced review requested due to automatic review settings October 9, 2026 07:23

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@jayhemnani9910
jayhemnani9910 force-pushed the fix/relative-start-own-composition branch from ec8f7c9 to a92f48e Compare October 11, 2026 15:26

@jrusso1020 jrusso1020 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The runtime fix looks correct, and the test fails without it. Before this merges, the render-side resolver that claims to mirror this one should be scoped the same way.

What I checked:

  • The docs claim holds. docs/reference/html-schema.mdx says "References stay within one composition", and the runtime was resolving document-wide.
  • compositionScopeOf (startResolver.ts:49) treats a composition host as belonging to its parent composition. So data-start="card-a" on a sibling host still resolves in the parent scope, and the document-wide fallback (:61) keeps cross-composition references working as before.
  • Ran vitest run src/runtime/startResolver.test.ts: 33/33 pass. Mutation check: with startResolver.ts reverted and the new test kept, it fails with expected 1 to be 3.

Please address in this PR:

  • packages/engine/src/services/referenceResolver.ts:33 (findReferenceTargetEl) still does getElementById document-wide. Its header says it "Mirrors the browser runtime's startResolver so snapshot/render agree". The producer uses it to place video, image and audio in parseVideoElements/parseImageElements/parseAudioElements, through renderMediaCollector.ts. Your repro passes because the engine works in composition-local time and both mounts are the same block, so the first #first happens to have the same local timing. With two different compositions that share an id at different local times, the runtime now picks the in-scope element and the engine still picks the first one in the document. A <video data-start="intro"> would then get its visibility window from one clip and its extracted frames from another. Scoping the engine lookup the same way (nearest [data-composition-id], host counts as its parent's) keeps the two in step.
  • packages/sdk/src/session.ts:384 says "refId remains a bare global id, matching runtime resolution". That is no longer true after this change, so update the comment, or the SDK's lookup if it should follow.

No other open PR targets this. #4334 is a different startResolver bug, as the description says.

— Rames

…own composition

The engine's reference resolver still looked an id up document-wide, so
with two different compositions sharing an id the producer placed media
from the first one while the runtime used the clip's own. Scope it the
same way as the runtime's startResolver, with the same document-wide
fallback, and correct the SDK comments that called the runtime lookup global.
The scoped lookup needs parentElement, closest and querySelectorAll, but
RefResolverEl is exported from @hyperframes/engine. Make them optional and
fall back to the document-wide lookup when an adapter lacks them.
@jayhemnani9910

Copy link
Copy Markdown
Author

Both points are in 13eacd9, plus 9ac9d2b.

Engine. findReferenceTargetEl now follows the runtime: it looks inside the referencing clip's composition first (a host counts as its parent's), then falls back to the document-wide getElementById / data-composition-id lookup as before. New test in videoFrameExtractor.test.ts: scene-a and scene-b each have an #intro (2 s and 3 s), and a video in scene-b with data-start="intro" now starts at 3. Before the change it got 2. RefResolverEl is exported, so 9ac9d2b makes the new parentElement / closest / querySelectorAll members optional. An adapter with only getAttribute still works and gets the old document-wide lookup (test in referenceResolver.test.ts).

I also rendered it: scene-a at 0-4 s with a red 2 s #intro, scene-b at 4-8 s with a blue 3 s #intro and a green video at data-start="intro", which hyperframes timeline puts at 7-8 s. Centre pixel per second, before: red, red, black, black, blue, blue, green, black. After: red, red, black, black, blue, blue, blue, green.

SDK. I fixed the comment and left the lookup. getElementTimings resolves refId through resolveScoped, which matches data-hf-id, not id, and prefers the top-level match for a bare id, the same rule dispatch and undo use (pinned by "resolves a colliding bare id to the TOP-LEVEL match"). So it never matched the runtime: for <h1 id="intro" data-start="1" data-duration="3"> + <p data-start="intro">, getElementTimings puts the <p> at 0 and the runtime puts it at 4. Making the SDK follow the runtime would change how it addresses elements, so I'd rather do that separately if you want it. session.timings.test.ts had the same "the runtime is global too" comment; updated that too.

Left alone: a reference to a clip in a different composition (the fallback path) still comes out in the referencing composition's local time on the render side. With a at 10 s holding a 2 s #cue and a video in b (at 20 s) with data-start="cue", collectRenderMedia gives 22 and the runtime gives 12. That was the same before this PR, so I'd follow up on it separately.

Engine, SDK, producer and CLI suites give the same results as the PR head before these commits, plus the two new tests. tsc, oxlint, oxfmt, fallow and both comment checks pass.

@jayhemnani9910
jayhemnani9910 force-pushed the fix/relative-start-own-composition branch from a92f48e to 9ac9d2b Compare October 15, 2026 16:04

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants