Skip to content

ci: a Codecov outage fails every test job even though the tests pass #587

Open

Description

@gonzalesedwin1123

What happens

On 2026-10-05 Codecov had an outage (status.codecov.com: "Codecov app and services are down", 06:53–09:31 UTC). During it, every CI run on this repo went red although every module's tests passed. Example: run 37282633706 attempt 1 on PR #555, 19 test jobs failed, each log showing Failed: 0, Errors: 0 followed by:

Run codecov/codecov-action@v4
Error: write EPROTO ...:error:0A000410:SSL routines:ssl3_read_bytes:ssl/tls alert handshake failure

The Upload coverage step fails, that fails the job, and test-summary then fails because it requires every test job to be green. Dependabot PRs opened during the outage show the same pattern.

Why fail_ci_if_error: false does not help

.github/workflows/ci.yml (the Upload coverage step, around line 241) already sets fail_ci_if_error: false. That flag only covers errors the action reports itself. A transport error thrown by the action's Node runtime before the upload starts is not caught by it, so the step still fails.

Proposed fix

Mark the step as non-fatal at the workflow level so coverage upload can never block a merge:

      - name: Upload coverage
        if: always()
        continue-on-error: true
        uses: codecov/codecov-action@v4

Coverage would then be missing for that run (and the codecov PR comment absent), which is visible and recoverable by a rerun, instead of a red CI that needs a human to read 20 logs. Keep fail_ci_if_error: false as well.

Workaround used

Wait for the Codecov incident to resolve, then gh run rerun <run-id> --failed.

Found while merging #555.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions