Skip to content

Stop other workers promptly when --maxfail/-x is reached - #1395

Open
RonnyPfannschmidt wants to merge 5 commits into
masterfrom
claude/project-thread-v55eh6
Open

RonnyPfannschmidt wants to merge 5 commits into
masterfrom
claude/project-thread-v55eh6

Conversation

@RonnyPfannschmidt

@RonnyPfannschmidt RonnyPfannschmidt commented Oct 7, 2026 •

Copy link
Copy Markdown
Member

Requested by Ronny · project thread

🤖 Written by Claude Opus 5.5 via Claude Code for the pytest-xdist maintainers; I prompted it, it did the work, I read it.

Fixes #420.

Before: when -x/--maxfail is reached, the worker that hit the failure stops (since #1024), but every other worker keeps running all tests already queued to it. The controller's shutdown command only appends a stop marker to the end of each worker's queue.

After: the other workers stop after the test they are currently running, and do not start a new one.

How: triggershutdown passes the controller's shouldstop value (e.g. "stopping after 1 failures", or a keyboard-interrupt reason) along with the shutdown command. When it is set, the worker assigns it to session.shouldstop. The run loop already checks that after each test, and run_one_test now also checks it after waiting for the next item or the --ramp delay, so a stop that arrives during those waits doesn't start another test. Final teardown still happens in pytest_sessionfinish as with plain pytest -x. Because the worker reports shouldstop back, the controller's existing path in worker_workerfinished skips the pending-items crash check, so no special case is needed there. Plain shutdown sent by the schedulers when nothing is left to distribute is unchanged.

Limitation: tests that already started before the stop command arrives still run to completion.

  • Tests:
    • test_maxfail_stops_other_workers (--dist=loadfile, 40 slow tests on the other worker): 40 pass without the fix, fewer than 20 with it.
    • test_maxfail_stops_worker_during_ramp (-x -n2 --ramp=3): the delayed worker ran 1 test without the fix, 0 with it.
  • Changelog: changelog/420.bugfix.rst

Note: the TestLoadScope::test_workqueue_ordered_by_* tests are intermittently failing under full-suite load on master as well; unrelated to this change.

🤖 Generated with Claude Code

https://claude.ai/code/session_01S74uNrKyts6p5C9Mci4CmV

Previously the controller's shutdown command was queued behind the
tests already sent to each worker, so other workers kept running their
whole queue after the first failure. Now the controller sends
shutdown(immediately=True) when stopping early, and workers break out
after their current test.

Fixes #420

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S74uNrKyts6p5C9Mci4CmV

RonnyPfannschmidt commented Oct 7, 2026 •

Copy link
Copy Markdown
Member Author

🤖 Written by Claude Opus 5.5 via Claude Code for the pytest-xdist maintainers; I prompted it, it did the work, I read it.

The py311-pytestmain failures (5 × TestGroupScope) are not caused by this PR; they reproduce on master against pytest main (structured NodeId, pytest-dev/pytest@431f3e1) and #1388 addresses them.

Update: resolved. master now has the fix (it moved to 4616feb), and CI on db4efab is fully green, including pytestmain.

claude added 2 commits October 7, 2026 05:18
Setting shouldstop on the worker also makes the controller skip the
pending-items crash check in worker_workerfinished, so that special
case is no longer needed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S74uNrKyts6p5C9Mci4CmV
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S74uNrKyts6p5C9Mci4CmV
@RonnyPfannschmidt
RonnyPfannschmidt marked this pull request as ready for review October 7, 2026 05:37

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Workers can still start an unstarted test after receiving shutdown during lookahead or ramp-up waits.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
What changed in this PR

Addresses #420 by forwarding stop reasons to workers so they can stop without draining queued tests.

Changes:

  • Propagates controller stop reasons through shutdown commands.
  • Adds an acceptance test for stopping other workers.
  • Updates the test double and changelog.
File Description
testing/​test_dsession.py Updates the shutdown test double’s signature.
testing/​acceptance_test.py Tests early stopping across workers.
src/​xdist/​workermanage.py Includes stop reasons in shutdown commands.
src/​xdist/​remote.py Sets the worker session’s stop flag.
src/​xdist/​dsession.py Forwards the controller’s stop reason.
changelog/​420.bugfix.rst Documents the early-stop behavior.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/xdist/remote.py
The worker can receive the controller's shutdown while blocked on the
lookahead get() or during the --ramp delay; check shouldstop again
before running the protocol.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S74uNrKyts6p5C9Mci4CmV

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

xdist continues test execution in the background with '--exitfirst'

3 participants