Skip to content

feat: let a shard unit bundle several test classes - #1829

Open
maks-oleksyuk wants to merge 4 commits into
pestphp:5.xfrom
maks-oleksyuk:pr/shard-units
Open

maks-oleksyuk wants to merge 4 commits into
pestphp:5.xfrom
maks-oleksyuk:pr/shard-units

Conversation

@maks-oleksyuk

@maks-oleksyuk maks-oleksyuk commented Aug 7, 2026 •

Copy link
Copy Markdown

--shard splits test classes and weighs each one on its own. That holds while a test class is the whole unit of work, but not when the work a shard has to do spans several of them: splitting those classes across shards duplicates the work and leaves every shard with a partial result.

Mutation testing is the case that needs this. A mutation belongs to a source file, several test classes can cover that file, so the file is the unit and its covering test classes have to travel together. Related to #1691.

What changed

Shard::useTimings() accepts a richer value. Instead of a duration per test class, a unit may be ['time' => float, 'tests' => list<string>] under any key. A bare float still means a test class bundling only itself, so an existing shards.json and the behaviour of a plain --update-shards run are unchanged.

LPT partitioning now runs over units, and the --filter a shard receives is the union of the test classes its units bundle. Shard::selectedUnits() exposes the keys allocated to the current shard, so a plugin can narrow the work it generates to match.

Shard::useTimings() also takes optional metadata, stored next to the units and read back through Shard::metadata(). That lets a plugin carry a reference value from the unsharded run that produces the file into the sharded runs that consume it.

Three smaller fixes ride along. The timings file is written when a plugin supplied the units and the run exits non-zero, so a job whose only purpose is to record timings is not lost to an unrelated failure. The count reported by --update-shards now matches the number of entries actually written. The --list-tests subprocess no longer inherits --coverage arguments.

Testing

tests/Unit/Plugins/Shard.php covers unit normalisation, partitioning that keeps a bundle intact, both file shapes on read and write, precedence of plugin-supplied units, and the zero entries written for test classes that produced no work of their own.

Follow-ups

The plugin side that consumes this API lands in pestphp/pest-plugin-mutate#40. Together with #40 this closes the request in #1691.

@nunomaduro

Copy link
Copy Markdown
Member

thank you! can you rebase this?

@MrPunyapal

Copy link
Copy Markdown
Member

Please update this branch against the latest 5.x and resolve the conflicts so we can review and test the combined changes.

@MrPunyapal MrPunyapal left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address two regressions I reproduced with the sharding code on current 5.x.

  • Drop deleted test classes from legacy timings before partitioning. With three live classes and a stale 100-second entry, shard 1 runs nothing and shard 2 runs all three files. Add a test for a removed class in shards.json.
  • When removing --coverage-php from test discovery, consume its separate output path as well. --coverage-php path currently leaves path as a positional test file, so discovery fails. Cover the space-separated form alongside the equals form.

Please also resolve the conflicts in tests/.snapshots/success.txt and tests/Visual/Parallel.php against current 5.x.

@maks-oleksyuk

Copy link
Copy Markdown
Author

Rebased onto the latest 5.x and resolved the conflicts in tests/.snapshots/success.txt and tests/Visual/Parallel.php (snapshot and expected totals regenerated).

Both regressions are fixed:

  • Test classes removed from the suite are now dropped from the stored units before partitioning, so a stale entry in shards.json no longer skews the split. A unit left with no live test class is discarded, and a bundle keeps only its live classes.
  • The --list-tests discovery now consumes the separate value of --coverage-php (and the other value-taking coverage options), not only the = form.

Added tests for a removed class in the stored units and for the space-separated --coverage-php path form.

@MrPunyapal MrPunyapal left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original stale-class and --coverage-php path reproductions pass now. Please fix two remaining cases.

  • Preserve units saved with tests: []. The mutation plugin uses these units for ownership. Dropping them makes Shard::owns() accept the missing unit on every shard. Add a test that distinguishes an explicitly empty unit from a unit emptied by removing stale classes.
  • Consume the separate values of --coverage-jsonl and --coverage-filter in discovery too. I reproduced --coverage-jsonl path failing with “Test file not found.” Add cases for both flags next to the --coverage-php path test.

maks-oleksyuk and others added 4 commits November 20, 2026 16:09
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…option values

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…covery

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants