Skip to content

ci: a corrected PR body cannot clear Validate conventional commit format #1383

Description

@KonstantinMirin

Validate conventional commit format reads the PR title and body from the
webhook payload captured when the workflow was triggered, and a body edit is not
a trigger. The combined effect is that a contributor who corrects their body
exactly as the error message instructs has no way to clear the check.

The two halves

scripts/check_main_policies.py reads the event from disk, not from the API:

event = json.loads(Path(os.environ["GITHUB_EVENT_PATH"]).read_text())   # line 154
...
pr = event["pull_request"]                                             # line 163
validate_message(pr["title"] + "\n\n" + (pr["body"] or ""))            # line 178

GITHUB_EVENT_PATH is the payload as it was when the event fired, so pr["body"]
is a snapshot. And .github/workflows/ci.yml declares no types::

on:
  pull_request:
    branches:
      - main

With types: omitted, the default is [opened, synchronize, reopened].
edited is not in that set, so editing a title or body does not start a run.

What a contributor experiences

  1. The check fails with breaking subject requires its BREAKING CHANGE footer in the actual commit.
  2. They add the footer to the PR body, which is what the message asks for.
  3. Nothing happens. The check does not re-run, because an edit is not a trigger.
  4. They press Re-run jobs, which is the obvious next move. It still fails:
    a re-run replays the same stored payload, so the checker sees the old body.
  5. The only thing that clears it is pushing a commit — a change unrelated to the
    problem, on a PR that may have been otherwise ready.

Observed concretely: a run created at 21:21:38Z had its job start at
22:31:45Z, 70 minutes later because of queueing, and still validated a body
that had been corrected at 22:05:43Z. The traceback is
check_main_policies.py:178, the PR-level validate_message, not the per-commit
loop — the commit in that PR carried a correct footer throughout.

Two remedies

1. Fetch the body from the API inside the checker. The script already has a
read-only GitHub client and a token; the PR-level validation would read current
title and body through it instead of from event. Scoped to the one check that
needs current data, and it makes Re-run jobs do what a maintainer expects.

2. Add types: [opened, synchronize, reopened, edited] to ci.yml. A
smaller diff, but it re-triggers the whole matrix on every title or body edit.
Given that the 3.12 job currently runs 48–56 minutes (#1381), that is an
expensive way to absorb a typo correction, and it re-runs 30-plus other checks
that have no interest in the body.

Remedy 1 looks clearly better: the data staleness is the defect, and fixing it
where the data is read costs one API call rather than a full matrix run.

Happy to send a PR for whichever you prefer.

Activity

  1. added
    claude-triagingTriage routine is actively working on this issue (1-3 min)
    on Oct 3, 2026
  2. bokelley commented on Oct 3, 2026

    @bokelley
    Contributor

    Triage

    Classification: Bug
    Bucket(s): CI tooling (no matching repo label)
    Status: ready-to-implement

    What the experts said:

    • code-reviewer: Remedy 1 is correct — add api: GitHub to conventional(), fetch live PR data, add head-SHA drift guard mirroring ipr() lines 261–263; CI-only, non-breaking.
    • dx-expert: Remedy 1 clearly dominates; the error string "in the actual commit" also misdirects contributors toward amending commits rather than editing the PR body — worth fixing in the same diff.

    My take: The defect is a data-staleness bug: conventional() reads the PR title/body from a frozen webhook snapshot while ipr() in the same file already fetches live data from the GitHub API. Remedy 1 is the surgical fix — it makes Re-run jobs do what contributors expect and is consistent with the existing pattern.

    Implementation scope:

    • Files: scripts/check_main_policies.py only — no public Python API, no Pydantic models, no .github/workflows/ changes.
    • Change: Add api: GitHub param to conventional(); in the PR branch replace pr = event["pull_request"] with api.repo(f"pulls/{expected['number']}") + head-SHA drift guard (mirror ipr() lines 261–263). Update main() to pass GitHub(os.environ["GH_TOKEN"]) to conventional(). Verify the conventional-commits CI job has GH_TOKEN set (the ipr job does; conventional may not).
    • Optional: Improve error at line 134 from "in the actual commit" to "in the PR body (for PRs) or commit body (after merge)" to match what the PR-level validation actually reads.
    • Checks: ruff check ., mypy src/, pytest on any script tests; no changeset or type regeneration needed.

    Triaged by Claude Code. Session: https://claude.ai/code/session_018xJ4crXY7B7cXz54ZVxZDq


    Generated by Claude Code

  3. added
    bugSomething isn't working
    and removed
    claude-triagingTriage routine is actively working on this issue (1-3 min)
    on Oct 3, 2026
  4. added a commit that references this issue on Oct 23, 2026
    41c2be7
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions