Skip to content

Mutate only what changed - #52

Open
fkrzski wants to merge 3 commits into
pestphp:5.xfrom
fkrzski:feat/changed-option
Open

fkrzski wants to merge 3 commits into
pestphp:5.xfrom
fkrzski:feat/changed-option

Conversation

@fkrzski

@fkrzski fkrzski commented Oct 28, 2026

Copy link
Copy Markdown

What:

  • Bug Fix
  • New Feature

Description:

Adds --changed[=<revision>] (and mutate()->changed()), which only generates mutations in the source files a change since git merge-base <revision> HEAD can affect. Without a revision it takes uncommitted changes.

./vendor/bin/pest --mutate --parallel --covered-only --changed=origin/main
  • A changed source file is mutated whole.
  • A changed test file adds what its covers()/mutates() name before and after the change, plus the source files its tests ran.
  • A change everything depends on (a fixture, composer.lock, phpunit.xml, PHP outside the source paths) mutates everything, and the output says why. A support file like tests/Pest.php that only gains functions doesn't count.

It narrows inside MutationTestRunner::run() rather than through --class, which lifts the __pest_mutate_only group and so runs different tests than a full run. A run with nothing to mutate passes --min.

On real pull requests of a 1,500-mutation suite, full runs take 27 s and --changed runs 8–12 s; a docs-only change takes 3 s.

Pull request What changed Full run --changed=HEAD^1
fkrzski/php-steam-api-sdk#174 1 source, 1 test file 27.6 s, 1522 mutations in 96 files 8.1 s, 188 mutations in 4 files
fkrzski/php-steam-api-sdk#175 4 source, 7 test files 26.2 s, 1457 mutations in 98 files 12.0 s, 469 mutations in 29 files
fkrzski/php-steam-api-sdk#176 17 source files 26.9 s, 1457 mutations in 98 files 8.6 s, 266 mutations in 11 files
fkrzski/php-steam-api-sdk#173 23 source, 6 test files, helpers added to tests/Pest.php 27.7 s, 1473 mutations in 97 files 11.8 s, 460 mutations in 30 files
— README.md only — 3.1 s, nothing to mutate

Each row replays the pull request's merge commit with --parallel --covered-only --no-cache on 16 threads, PHP 8.5 with PCOV and Pest 5.3.4; --changed times are the mean of two runs, and every run scored 100%. The same scope passed as --class took the same time, but on fkrzski/php-steam-api-sdk#176 it mutated 12 files instead of 11, because its initial run also counts coverage from tests without covers(). On GitHub's runners the full mutation job took a median 374 s per pull request.

It covers what --changed-only did before its removal in a6ec515, at file level and with test changes taken into account.

5.x already fails 3 visual snapshot tests and 2 PHPStan errors on a fresh install; this adds none.

Narrows inside the runner rather than through --class, which lifts the
__pest_mutate_only group and so runs different tests than a full run.
No unchanged test can call a new function, and the tests that do are
in the diff anyway.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant