Repository navigation
fix(producer): reject regression audio stream mismatches - #5228
user-github-me wants to merge 7 commits into
Conversation
jrusso1020
left a comment
There was a problem hiding this comment.
The change itself reads correctly, but the required regression check is red at this head, and the cause is this PR's new rule, not a flake. It failed the same way on a rerun.
What fails. Shard 2, pip-video-late-host:
✓ Visual quality: PASSED (0 failed frames, threshold: 5)
✗ Audio stream mismatch: rendered=missing, snapshot=present
✗ Audio quality: FAILED (correlation: -1.000, threshold: 0)
❌ Test FAILED: Pip video inside late-starting sub-composition host
Why. The fixture's only media element is a muted <video> (src/compositions/pip.html) with no data-has-audio. So a render with no audio stream is the correct output. The committed output/output.mp4, though, carries a real AAC track (volumedetect: mean -25.2 dB, max -4.5 dB). That snapshot was last regenerated in #3928. #4625 put this fixture into CI later and changed only the HTML, not the snapshot. Under the old rule a one-sided audio stream scored correlation 1, so the stale snapshot passed. Your stricter check now catches it, which is the behavior this PR is meant to add.
To land this, the PR needs to keep regression green:
- Regenerate
packages/producer/tests/pip-video-late-host/output/output.mp4with the Docker baseline flow, so the snapshot matches the muted composition (no audio stream). Include it in this PR. - Then check whether any other fixture with a muted-only composition has an audio-bearing snapshot. Only shard 2 reached a verdict here. Shards 1, 3, 4, 7 and 9 were cancelled when shard 2 failed, so this run doesn't tell you whether they're clean.
I'll approve once regression is green at the new head.
— Rames
12ccb43 to
920c432
Compare
A regression render can lose its audio track and still pass because the harness assigns correlation 1 when either side lacks audio. Require matching audio-stream presence before comparing envelopes. Both mismatch directions fail even with a zero correlation threshold; two silent renders still pass.
Preserve the reviewed silent
pip-video-late-hostbaseline: 144 H.264 frames at 24 fps, six seconds, no audio stream. Its LFS content hash is unchanged. The current-main integration retains newer integration-test classifications and refreshes their reachability fingerprint.Validation:
This PR shares the tested signed revision with #5297; the existing review history is preserved.