Repository navigation
test(webui): settle the shell's images before a baseline, and give the browser-frame wait its own budget - #16
Merged
Conversation
…aken The composer mascot is drawn from a <picture>: the browser starts on the poster PNG and upgrades to the WebP, and that upgrade aborts the first request. A frame taken inside that window shows the browser's broken-image placeholder instead of the picture, which reads as a few thousand pixels of difference and fails the baseline for reasons that have nothing to do with the layout under review. One spec already guarded against this. The guard now lives in the harness so every screenshot spec shares it, and it waits for the settled state rather than for a deadline: an image that never loads now fails with a legible message instead of a cryptic pixel diff.
The panel can only render a frame after the isolated Chromium has booted, opened the page and streamed its first picture back. That pipeline, not the product's responsiveness, is what the wait covers, and on a loaded CI runner it outlasts the 30s window the case was written against -- the same case failed that way five times across four commits, on macOS and Windows but never on Ubuntu. The failure now reports whether the panel is still showing its empty placeholder and whether a capability notice is on screen, so the next run says which of the two faults it hit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
Two flakes that redden
mainon the CI runners. Neither of them is a layout regression.1. A baseline that catches the composer mascot mid-swap
The shell draws the composer mascot from a
<picture>: the browser starts on theposter PNG and upgrades to the WebP, and that upgrade aborts the first request. A
frame taken inside that window shows the browser's broken-image placeholder instead
of the picture, which reads as a few thousand pixels of difference and fails the
baseline for reasons that have nothing to do with the layout under review.
On the latest
mainthis failedthe shell at 900pxon Windows: the difference was5614 pixels, every one of them inside a single 100x75 box at
(733, 671)-- themascot, rendered as the broken-image icon. The same run's mascot case in
composer.spec.mjspassed, so the image does load; the frame was simply takentoo early.
One spec already guarded against this (
visual-baselines.spec.mjs). The guard nowlives in the harness so every screenshot spec shares it, and it waits for the
settled state rather than for a deadline: an image that never loads now fails with
a legible message instead of a cryptic pixel diff.
2. The browser-frame wait outliving its 30s budget
agent-transcript.test.tsgives the browser panel 30 seconds to render its firstframe. That wait covers the whole pipeline -- the isolated Chromium has to boot,
open the page and stream its first picture back -- not the product's own
responsiveness. On a loaded runner that pipeline outlasts the budget: the same
case failed that way repeatedly, on macOS and Windows, and never on Ubuntu,
including on a commit that predates the other changes in this repository.
The wait gets its own budget, and the failure now reports whether the panel is
still showing its empty placeholder and whether a capability notice is on screen,
so the next run says which of the two faults it hit.
How this was checked
pnpm typecheck:webuipasses.check:webui-dependency,-presentation,-motion,-overlays,-feedback,-boundary) pass.same result (7 passed, 3 failed): the three failures are the known macOS
rasterisation difference between two machines, which the 8-pixel budget cannot
absorb, and they are unchanged by this pull request.