Skip to content

Fix flaky TestIsViteReady/vite_is_running - #7837

Merged
denik merged 1 commit into
mainfrom
fix-vite-ready-flake
Nov 20, 2026
Merged

denik merged 1 commit into
mainfrom
fix-vite-ready-flake

Conversation

@denik

@denik denik commented Nov 20, 2026

Copy link
Copy Markdown
Contributor

Why

TestIsViteReady/vite_is_running flakes, mostly on Windows, and passes on retry. It made a single isViteReady(5173) call, and that call allows only viteReadyCheckInterval (100ms) for the TCP dial to localhost. Every failure took 0.19–0.53s: the 50ms sleep plus a dial that ran out of time. On Windows, dialing localhost can try ::1 first, where nothing listens (the listener is on 127.0.0.1). A refused loopback connection on Windows is slow, so the 127.0.0.1 attempt gets little or none of the 100ms budget. On a loaded Linux runner a single dial can also miss it.

Changes

  • Listen on localhost:0 instead of the fixed 5173, so the check can't collide with anything else on that port.
  • Poll isViteReady with assert.Eventually (10s, 50ms tick), the same way startViteDevServer retries in production, instead of one sleep and one attempt.

Test-only; production code is unchanged.

Dashboard

link failure
error/3457 dev_test.go:43: Error: Should be true / --- FAIL: TestIsViteReady/vite_is_running (0.22s)
cmd/apps.TestIsViteReady/vite_is_running 5 flaky failures in 8 days (4 Windows, 1 Linux)

Tests

go test ./cmd/apps -run TestIsViteReady -count=20 passes locally (Linux).

AI agent (opus5.5, isaac_claude)

This pull request and its description were written by Isaac.

Poll isViteReady on a free port instead of a single 100ms dial on 5173.

Co-authored-by: Isaac <no-reply@databricks.com>
@denik denik added the ai-auto Deprecated: use factory-generated instead. label Nov 20, 2026
@github-actions github-actions Bot added the test-only-fix PRs that update failing of flaky tests without changing CLI code. label Nov 20, 2026
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: 9dbcde7

Run: 44700505295

Env 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 272 19 5:01
💚​ aws-windows-latest-4core-16gb 1 274 17 3:22
💚​ azure linux-2core-8gb 1 271 19 5:07
💚​ azure-windows-latest-4core-16gb 1 273 17 3:40
💚​ gcp linux-2core-8gb 1 272 19 4:49
💚​ gcp-windows-latest-4core-16gb 1 274 17 4:09
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 6 slowest tests (at least 2 minutes):
duration env testname
4:08 gcp-windows-latest-4core-16gb TestAccept
4:06 azure linux-2core-8gb TestAccept
3:41 aws linux-2core-8gb TestAccept
3:39 azure-windows-latest-4core-16gb TestAccept
3:36 gcp linux-2core-8gb TestAccept
3:21 aws-windows-latest-4core-16gb TestAccept

@denik
denik marked this pull request as ready for review November 20, 2026 08:36
@denik
denik requested review from a team as code owners November 20, 2026 08:36

@review-oss-bot review-oss-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review at 9dbcde7. Each finding was verified against the code.

No blocking issues found. 1 non-blocking comment(s) below.

This test-only PR fixes flakiness in TestIsViteReady/vite_is_running by switching from a fixed port to an ephemeral one and replacing the single-shot check with an assert.Eventually retry loop. The ephemeral-port change is clean. The retry loop is quite generous (10 s / 50 ms ≈ 200 attempts), which could mask a regression where isViteReady almost never succeeds on a healthy listener; tightening the budget to match the production retry window would keep the flake fix while preserving the test's diagnostic value.

Comment thread cmd/apps/dev_test.go
// unbound ::1 before 127.0.0.1, one attempt can time out. Poll like the
// production wait loop does.
port := listener.Addr().(*net.TCPAddr).Port
assert.Eventually(t, func() bool { return isViteReady(port) }, 10*time.Second, 50*time.Millisecond)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] assert.Eventually(..., 10*time.Second, 50*time.Millisecond) gives ~200 attempts, while the production loop in startViteDevServer budgets viteReadyMaxAttempts × viteReadyCheckInterval = 5 s with 50 attempts. If isViteReady regressed to succeeding only once every few seconds on a healthy local listener, this test would still pass. Consider capping at time.Duration(viteReadyMaxAttempts)*viteReadyCheckInterval (or even shorter — a handful of retries should be plenty for a loopback dial) so the test stays close to the production contract while still absorbing the dual-stack delay.

@denik
denik added this pull request to the merge queue Nov 20, 2026
Merged via the queue into main with commit 054a0bf Nov 20, 2026
32 checks passed
@denik
denik deleted the fix-vite-ready-flake branch November 20, 2026 10:47
@denik denik added factory-generated and removed ai-auto Deprecated: use factory-generated instead. labels Nov 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

factory-generated test-only-fix PRs that update failing of flaky tests without changing CLI code.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants