Skip to content

test: fix Stagehand binary cache race in plugin tests - #2460

Merged
vdusek merged 4 commits into
masterfrom
test/stagehand-binary-cache-race
Nov 18, 2026
Merged

vdusek merged 4 commits into
masterfrom
test/stagehand-binary-cache-race

Conversation

@vdusek

@vdusek vdusek commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

test_methods_raise_error_when_not_active creates a real AsyncStagehand, which copies the Stagehand binary into the per-user cache. It did not use the stagehand_binary fixture, so it skipped the FileLock added in #2418. With --dist worksteal, it can run on one worker while another worker is inside the locked fixture. Both write the same .tmp file and then rename it, and the rename fails with WinError 32 on Windows (failed run) or FileNotFoundError on Linux.

Changes:

  • Every test in test_stagehand_browser_plugin.py uses the stagehand_binary fixture through a module-level pytestmark, so a new test in the file can't skip the lock.
  • The lock file sits next to the per-user cache, not in the pytest session's temp directory. Two pytest sessions running at the same time (for example, in two worktrees) share one lock.
  • The fixture resolves the binary for stagehand.__version__, the version AsyncStagehand uses. Before, a set STAGEHAND_VERSION made it fill a different cache directory than the client.

Each change was checked locally with a 3 s delay patched in between the write and the rename, and a fresh cache for each run:

  • Plugin tests: 20 of 20 runs failed before, 0 of 100 after.
  • Two concurrent pytest sessions: 3 of 3 failed with the old lock path, 0 of 3 with the new one.
  • STAGEHAND_VERSION set: 3 of 3 failed before, 0 of 3 after.

✍️ Drafted by Claude Code

@vdusek vdusek added t-tooling Issues with this label are in the ownership of the tooling team. adhoc Ad-hoc unplanned task added during the sprint. labels Nov 17, 2026
@vdusek vdusek self-assigned this Nov 17, 2026
@codecov

codecov Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.14%. Comparing base (8670f7b) to head (ce8ebb9).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2460      +/-   ##
==========================================
- Coverage   94.25%   94.14%   -0.11%     
==========================================
  Files         184      184              
  Lines       13690    13690              
==========================================
- Hits        12904    12889      -15     
- Misses        786      801      +15     
Flag Coverage Δ
integration 76.56% <ø> (ø)
unit 85.64% <ø> (-0.11%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vdusek
vdusek marked this pull request as ready for review November 17, 2026 09:54
@vdusek
vdusek requested a review from Mantisus November 17, 2026 09:54

@Mantisus Mantisus left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's try 🙂

@vdusek
vdusek merged commit 91a79c7 into master Nov 18, 2026
52 checks passed
@vdusek
vdusek deleted the test/stagehand-binary-cache-race branch November 18, 2026 06:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

adhoc Ad-hoc unplanned task added during the sprint. t-tooling Issues with this label are in the ownership of the tooling team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants