Add .codecov.yml with threshold-based coverage status checks#529
Open
khatchad wants to merge 1 commit into
Open
Add .codecov.yml with threshold-based coverage status checks#529khatchad wants to merge 1 commit into
.codecov.yml with threshold-based coverage status checks#529khatchad wants to merge 1 commit into
Conversation
Without an explicit `.codecov.yml`, Codecov uses default `target: auto` with zero tolerance: any project-coverage decrease at all triggers `codecov/project: FAILURE`. Base-vs-head test-environment drift or flakiness can shave 1-2 lines off the project total and fail the gate even on PRs that don't touch Java code (e.g., POM-only fixes—see PR wala#516 where a -0.03% project delta fails the gate despite the patch having no Java changes to cover). Add a minimal `.codecov.yml` mirroring the configuration in `ponder-lab/ML`'s current state (commits `f7f610ff` / `fd19bb91` / `ee247f0b` / `3eac854a`): * `coverage.status.project.default.threshold: 1%`—tolerate up to 1% project-coverage decrease before failing. Real regressions still fail; noise doesn't. * `coverage.status.patch.default.threshold: 2%`—tolerate up to 2% patch-coverage decrease. * `github_checks.annotations: true`—surface uncovered changed lines as inline diff annotations in the PR's "Files changed" view (alongside the per-PR Codecov comment, which remains enabled by default). `informational: false` keeps the checks as real build signal rather than unconditionally-green—the thresholds are the absorptive layer.
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a repository-level Codecov configuration to make codecov/project and codecov/patch status checks tolerant to small coverage fluctuations, reducing CI noise while keeping coverage regressions as real failures.
Changes:
- Add
.codecov.ymlconfiguringprojectandpatchcoverage statuses to usetarget: autowiththresholdtolerances (1% / 2%). - Enable
github_checks.annotationsso uncovered changed lines appear as inline PR annotations.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #529 +/- ##
============================================
- Coverage 57.93% 57.90% -0.03%
+ Complexity 625 624 -1
============================================
Files 111 111
Lines 7671 7671
Branches 856 856
============================================
- Hits 4444 4442 -2
- Misses 3049 3051 +2
Partials 178 178 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a minimal
.codecov.ymlso Codecov'sproject/patchstatus checks tolerate small base-vs-head coverage drift instead of failing on every fractional drop. Mirrors the configuration inponder-lab/ML.Why
Without an explicit
.codecov.yml, Codecov uses defaulttarget: autowith zero tolerance: any project-coverage decrease at all triggerscodecov/project: FAILURE. Base-vs-head test-environment drift or flakiness can shave 1-2 lines off the project total and fail the gate even on PRs that don't touch Java code.Concrete case: #516 (the POM cleanup mirror of
ponder-lab/ML#272) hascodecov/patch: SUCCESS(no Java changed, so the patch is trivially 100% covered) butcodecov/project: FAILUREfrom a-0.03%project delta (-2hits /+2misses) that can't be attributed to the diff. The default Codecov policy fails on any non-zero negative delta; this configuration absorbs the noise.What This Changes
projectthreshold1%—tolerates project-coverage drops of up to 1% before failing. Real regressions still fail.patchthreshold2%—same idea for changed-line coverage.github_checks.annotations: true—surfaces uncovered changed lines as inline annotations in the PR's "Files changed" view.informational: falsekeeps the checks as real build signal rather than unconditionally green—the thresholds are the absorptive layer. The per-PR Codecov comment remains enabled by default (Codecov's default behavior withcommentunspecified).History Of The Equivalent
ponder-lab/MLConfigFor reference, the four commits that produced this state on the fork:
f7f610ff(#243): added.codecov.ymlwithinformational: true(unconditionally green).fd19bb91(#263): disabled per-PR comment, enabled inline annotations.ee247f0b(#264): flippedinformational: falsewith the1%/2%thresholds—"Now that there's no comment, we need to see the build failure."3eac854a(#266): re-enabled the per-PR comment.This PR ports the current combined state in one commit.
Test Plan
logging.propertiesinclude fromjython3.testPOM (#493 follow-up) #516 (or any past PR with a noisycodecov/project: FAILURE)—the threshold should absorb the drift and the gate should pass.pass/faildecision on the GitHub status check changes.🤖 Generated with Claude Code