Skip to content

Commit bf05ad1

Browse files
committed
feat: final check on labelled PRs, posted as TechEngineBot
Adding the final-check label to a PR asks TechEngineBot for a last look. It posts one comment, edited in place on later pushes, and a final-check commit status that branch protection can require. It blocks drafts, merge conflicts, PRs into main that are not a release from develop or a hotfix, failed or cancelled checks, changed workflows that use a personal token, and commits no other check ran on. It waits up to 30 minutes for running checks and never runs the PR's code. The bot was suspended once for spam, so it only speaks when asked and only once per PR. Other repositories opt in by calling the reusable workflow.
1 parent 4999837 commit bf05ad1

5 files changed

Lines changed: 338 additions & 1 deletion

File tree

‎.github/workflows/final-check.yml‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
name: final-check
2+
3+
# TechEngineBot's final check on PRs labelled `final-check`: one comment per
4+
# PR, edited in place, plus a `final-check` commit status. Other repositories
5+
# call this as a reusable workflow (see README); TechMachine uses it directly.
6+
#
7+
# pull_request_target so the bot token is available, which is safe because
8+
# nothing from the PR is checked out or run — only TechMachine's own code,
9+
# reading what the GitHub API reports about the PR.
10+
on:
11+
pull_request_target:
12+
types: [labeled, synchronize, ready_for_review]
13+
workflow_call:
14+
secrets:
15+
TECHENGINEBOT_TOKEN:
16+
required: true
17+
18+
permissions:
19+
contents: read
20+
21+
concurrency:
22+
group: final-check-${{ github.repository }}-${{ github.event.pull_request.number }}
23+
cancel-in-progress: true
24+
25+
jobs:
26+
final-check:
27+
# Only a person adding the label starts it; later pushes refresh the same
28+
# comment while the label stays on.
29+
if: >-
30+
contains(github.event.pull_request.labels.*.name, 'final-check') &&
31+
(github.event.action != 'labeled' || github.event.label.name == 'final-check')
32+
runs-on: ubuntu-latest
33+
timeout-minutes: 40
34+
steps:
35+
- uses: actions/checkout@v4
36+
with:
37+
repository: GetTechAPI/TechMachine
38+
ref: main
39+
- uses: actions/setup-python@v5
40+
with:
41+
python-version: "3.12"
42+
- name: Judge and post as TechEngineBot
43+
env:
44+
GITHUB_TOKEN: ${{ secrets.TECHENGINEBOT_TOKEN }}
45+
run: >-
46+
python -m machine.final_check --publish --wait-minutes 30
47+
--repo "${{ github.repository }}" --pr "${{ github.event.pull_request.number }}"

‎README.md‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,45 @@ When the set of problems changes, the check comments on the issue and
4040
It stays quiet when the same failure recurs (the fingerprint is
4141
workflow + result, not the run link) and never pings for an all-clear.
4242

43+
## Final check
44+
45+
[TechEngineBot](https://github.com/TechEngineBot) gives important pull
46+
requests a last look before merge. A person asks for it by adding the
47+
`final-check` label; the bot then posts **one** comment (later pushes edit it,
48+
never add another) and a `final-check` commit status that branch protection
49+
can require.
50+
51+
It blocks a draft, a merge conflict, a PR into `main` that is not a release
52+
from `develop` or a `hotfix/`, any other check that failed or was cancelled,
53+
a changed workflow that uses a personal token, and a commit no other check
54+
ran on at all. It waits up to 30 minutes for running checks before judging.
55+
It never checks out or runs the PR's code.
56+
57+
The account was suspended once for spam, so this is its only job: it speaks
58+
only when asked, and only once per PR.
59+
60+
Another repository opts in with `.github/workflows/final-check.yml`:
61+
62+
```yaml
63+
name: final-check
64+
on:
65+
pull_request_target:
66+
types: [labeled, synchronize, ready_for_review]
67+
jobs:
68+
final-check:
69+
uses: GetTechAPI/TechMachine/.github/workflows/final-check.yml@main
70+
secrets: inherit
71+
```
72+
73+
It needs a `final-check` label in that repository and the
74+
`TECHENGINEBOT_TOKEN` secret: a fine-grained token of the TechEngineBot
75+
account with read access to contents and checks, and write access to pull
76+
requests (comments) and commit statuses — best as an organisation secret.
77+
78+
```bash
79+
python -m machine.final_check --repo GetTechAPI/TechAPI --pr 367 # print the verdict only
80+
```
81+
4382
## Satellite repositories
4483

4584
Categories that do not belong in TechAPI live in their own repository (games:

‎machine/final_check.py‎

Lines changed: 184 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,184 @@
1+
"""Final check on an important pull request, posted as TechEngineBot.
2+
3+
TechEngineBot is a real account and was suspended once for spam, so it keeps
4+
its standing by speaking rarely: only on PRs a person marks with the
5+
`final-check` label, as one comment per PR that later runs edit in place, plus
6+
one commit status. A status can be made a required check; neither edits nor
7+
statuses notify anyone.
8+
9+
The verdict only reads what GitHub already knows — it never checks out or
10+
runs the PR's code, so it is safe under `pull_request_target`.
11+
12+
Run with: python -m machine.final_check --repo OWNER/NAME --pr N [--publish]
13+
"""
14+
15+
from __future__ import annotations
16+
17+
import argparse
18+
import json
19+
import os
20+
import sys
21+
import time
22+
import urllib.request
23+
from dataclasses import dataclass, field
24+
from typing import Any
25+
26+
from machine.health import API, HEALTHY, _get, _get_text, foreign_secrets
27+
28+
MARKER = "<!-- techenginebot:final-check -->"
29+
CONTEXT = "final-check"
30+
# git-flow: main only moves through a release from develop, or a hotfix.
31+
RELEASE_SOURCES = ("develop", "hotfix/")
32+
33+
34+
@dataclass
35+
class Verdict:
36+
problems: list[str] = field(default_factory=list)
37+
pending: list[str] = field(default_factory=list)
38+
passed: list[str] = field(default_factory=list)
39+
40+
@property
41+
def state(self) -> str:
42+
if self.problems:
43+
return "failure"
44+
return "pending" if self.pending else "success"
45+
46+
47+
def evaluate(pr: dict[str, Any], check_runs: list[dict[str, Any]], statuses: list[dict[str, Any]],
48+
workflow_texts: dict[str, str], allowed: set[str] = frozenset()) -> Verdict:
49+
v = Verdict()
50+
51+
if pr.get("draft"):
52+
v.problems.append("the PR is still a draft")
53+
54+
base, head = pr["base"]["ref"], pr["head"]["ref"]
55+
if base == "main" and not head.startswith(RELEASE_SOURCES):
56+
v.problems.append(f"`{head}` → `main`: main only takes a release from develop or a hotfix")
57+
else:
58+
v.passed.append(f"`{head}` → `{base}` follows git-flow")
59+
60+
if pr.get("mergeable") is False:
61+
v.problems.append("merge conflicts with the base branch")
62+
63+
# Our own job and status would otherwise wait on themselves.
64+
runs = [r for r in check_runs if CONTEXT not in r.get("name", "")]
65+
others = [s for s in statuses if s.get("context") != CONTEXT]
66+
unsettled = len(v.problems) + len(v.pending)
67+
for run in runs:
68+
if run.get("status") != "completed":
69+
v.pending.append(f"check `{run['name']}` is still running")
70+
elif run.get("conclusion") not in HEALTHY:
71+
v.problems.append(f"check `{run['name']}`: {run.get('conclusion')}")
72+
for status in others:
73+
if status.get("state") == "pending":
74+
v.pending.append(f"status `{status['context']}` is pending")
75+
elif status.get("state") != "success":
76+
v.problems.append(f"status `{status['context']}`: {status.get('state')}")
77+
if not runs and not others:
78+
# Passing a commit nothing has looked at would be a silent failure of its own.
79+
v.problems.append("no other check ran on this commit — nothing to confirm")
80+
elif len(v.problems) + len(v.pending) == unsettled:
81+
v.passed.append(f"{len(runs) + len(others)} other check(s) passed")
82+
83+
for path, text in sorted(workflow_texts.items()):
84+
names = foreign_secrets(text, allowed)
85+
if names:
86+
v.problems.append(f"`{path}` uses {', '.join(names)} — automation must not run as a person")
87+
if workflow_texts and not any("automation must not" in p for p in v.problems):
88+
v.passed.append("changed workflows use only GITHUB_TOKEN")
89+
return v
90+
91+
92+
def render(v: Verdict, sha: str) -> str:
93+
title = {"success": "passed", "failure": "not ready", "pending": "waiting on checks"}[v.state]
94+
lines = [MARKER, f"### Final check: {title}", "", f"Checked `{sha[:7]}`.", ""]
95+
lines += [f"- ❌ {p}" for p in v.problems]
96+
lines += [f"- ⏳ {p}" for p in v.pending]
97+
lines += [f"- ✅ {p}" for p in v.passed]
98+
lines += ["", "<sub>Runs when the `final-check` label is on the PR; this comment is updated, not repeated.</sub>"]
99+
return "\n".join(lines) + "\n"
100+
101+
102+
def _call(method: str, path: str, token: str, payload: dict[str, Any]) -> Any:
103+
request = urllib.request.Request(f"{API}{path}", data=json.dumps(payload).encode("utf-8"),
104+
method=method, headers={
105+
"Accept": "application/vnd.github+json",
106+
"Authorization": f"Bearer {token}",
107+
"Content-Type": "application/json",
108+
})
109+
with urllib.request.urlopen(request, timeout=30) as response:
110+
return json.loads(response.read().decode("utf-8") or "null")
111+
112+
113+
def _paged(url: str, token: str | None, key: str | None = None) -> list[dict[str, Any]]:
114+
items: list[dict[str, Any]] = []
115+
for page in range(1, 11):
116+
data = _get(f"{url}{'&' if '?' in url else '?'}per_page=100&page={page}", token)
117+
batch = data[key] if key else data
118+
items.extend(batch)
119+
if len(batch) < 100:
120+
break
121+
return items
122+
123+
124+
def gather(repo: str, number: int, token: str | None, allowed: set[str]) -> tuple[dict[str, Any], Verdict]:
125+
pr = _get(f"{API}/repos/{repo}/pulls/{number}", token)
126+
sha = pr["head"]["sha"]
127+
runs = _paged(f"{API}/repos/{repo}/commits/{sha}/check-runs", token, "check_runs")
128+
statuses = _get(f"{API}/repos/{repo}/commits/{sha}/status", token).get("statuses", [])
129+
files = _paged(f"{API}/repos/{repo}/pulls/{number}/files", token)
130+
workflows = {
131+
f["filename"]: _get_text(f"{API}/repos/{repo}/contents/{f['filename']}?ref={sha}", token)
132+
for f in files
133+
if f["filename"].startswith(".github/workflows/") and f.get("status") != "removed"
134+
}
135+
return pr, evaluate(pr, runs, statuses, workflows, allowed)
136+
137+
138+
def publish(repo: str, number: int, sha: str, body: str, state: str, token: str) -> None:
139+
"""One comment per PR, edited in place, and one commit status."""
140+
comments = _paged(f"{API}/repos/{repo}/issues/{number}/comments", token)
141+
existing = next((c for c in comments if MARKER in (c.get("body") or "")), None)
142+
if existing:
143+
comment = _call("PATCH", f"/repos/{repo}/issues/comments/{existing['id']}", token, {"body": body})
144+
else:
145+
comment = _call("POST", f"/repos/{repo}/issues/{number}/comments", token, {"body": body})
146+
description = {"success": "Ready to merge", "failure": "Not ready — see the comment",
147+
"pending": "Waiting on other checks"}[state]
148+
_call("POST", f"/repos/{repo}/statuses/{sha}", token, {
149+
"state": state, "context": CONTEXT, "description": description,
150+
"target_url": comment.get("html_url", ""),
151+
})
152+
153+
154+
def main(argv: list[str] | None = None) -> int:
155+
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
156+
parser.add_argument("--repo", required=True, help="OWNER/NAME")
157+
parser.add_argument("--pr", type=int, required=True)
158+
parser.add_argument("--publish", action="store_true", help="comment and set the status")
159+
parser.add_argument("--wait-minutes", type=int, default=0,
160+
help="poll this long for other checks to finish before judging")
161+
parser.add_argument("--allowed-secret", action="append", default=[])
162+
args = parser.parse_args(argv)
163+
164+
token = os.environ.get("GITHUB_TOKEN")
165+
allowed = set(args.allowed_secret)
166+
deadline = time.monotonic() + args.wait_minutes * 60
167+
pr, verdict = gather(args.repo, args.pr, token, allowed)
168+
# Waiting here instead of posting "pending" keeps it to one meaningful update.
169+
while verdict.state == "pending" and time.monotonic() < deadline:
170+
time.sleep(60)
171+
pr, verdict = gather(args.repo, args.pr, token, allowed)
172+
173+
body = render(verdict, pr["head"]["sha"])
174+
print(body)
175+
if args.publish:
176+
if not token:
177+
raise SystemExit("--publish needs GITHUB_TOKEN (TechEngineBot's token)")
178+
publish(args.repo, args.pr, pr["head"]["sha"], body, verdict.state, token)
179+
# The status is the gate; a red job would only repeat it.
180+
return 0
181+
182+
183+
if __name__ == "__main__":
184+
sys.exit(main())

‎tests/test_final_check.py‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
"""TechEngineBot's verdict: block what is broken, wait on what is running."""
2+
3+
from __future__ import annotations
4+
5+
from machine.final_check import MARKER, evaluate, render
6+
7+
8+
def _pr(base: str = "develop", head: str = "feat/x", **extra) -> dict:
9+
return {"base": {"ref": base}, "head": {"ref": head, "sha": "abcdef1234"}, "draft": False,
10+
"mergeable": True, **extra}
11+
12+
13+
def _run(name: str, conclusion: str | None, status: str = "completed") -> dict:
14+
return {"name": name, "status": status, "conclusion": conclusion}
15+
16+
17+
def test_green_pr_passes():
18+
v = evaluate(_pr(), [_run("test", "success")], [], {})
19+
assert v.state == "success" and v.problems == []
20+
21+
22+
def test_failed_or_cancelled_check_blocks():
23+
assert evaluate(_pr(), [_run("test", "failure")], [], {}).state == "failure"
24+
assert evaluate(_pr(), [_run("validate", "cancelled")], [], {}).state == "failure"
25+
26+
27+
def test_running_check_waits():
28+
assert evaluate(_pr(), [_run("test", None, status="in_progress")], [], {}).state == "pending"
29+
30+
31+
def test_its_own_check_and_status_are_ignored():
32+
runs = [_run("final-check / final-check", None, status="in_progress")]
33+
statuses = [{"context": "final-check", "state": "pending"}]
34+
assert evaluate(_pr(), runs + [_run("test", "success")], statuses, {}).state == "success"
35+
36+
37+
def test_main_only_takes_a_release_or_hotfix():
38+
assert evaluate(_pr("main", "feat/x"), [_run("test", "success")], [], {}).state == "failure"
39+
assert evaluate(_pr("main", "develop"), [_run("test", "success")], [], {}).state == "success"
40+
assert evaluate(_pr("main", "hotfix/pages"), [_run("test", "success")], [], {}).state == "success"
41+
42+
43+
def test_nothing_checked_is_not_a_pass():
44+
# TechAPI's release PR after its workflows were removed: no evidence at all.
45+
v = evaluate(_pr("main", "develop"), [], [], {})
46+
assert v.state == "failure" and "nothing to confirm" in v.problems[0]
47+
48+
49+
def test_draft_and_conflicts_block():
50+
assert evaluate(_pr(draft=True), [], [], {}).state == "failure"
51+
assert evaluate(_pr(mergeable=False), [], [], {}).state == "failure"
52+
53+
54+
def test_workflow_with_personal_token_blocks():
55+
text = "token: ${{ secrets.TECHAPI_TOKEN }}"
56+
v = evaluate(_pr(), [_run("test", "success")], [], {".github/workflows/sync.yml": text})
57+
assert v.state == "failure" and "TECHAPI_TOKEN" in v.problems[0]
58+
59+
60+
def test_comment_carries_the_marker_so_it_is_edited_not_repeated():
61+
body = render(evaluate(_pr(), [_run("test", "success")], [], {}), "abcdef1234")
62+
assert body.startswith(MARKER) and "abcdef1" in body and "passed" in body

‎tests/test_health.py‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -109,6 +109,11 @@ def test_allowed_secret_is_fine():
109109
assert foreign_secrets("${{ secrets.SOME_API_KEY }}", {"SOME_API_KEY"}) == []
110110

111111

112+
# TechEngineBot is a machine account used only for the final check.
113+
BOT_SECRET = {"final-check.yml": {"TECHENGINEBOT_TOKEN"}}
114+
115+
112116
def test_techmachine_itself_uses_only_the_builtin_token():
113117
for path in sorted((ROOT / ".github" / "workflows").glob("*.yml")):
114-
assert foreign_secrets(path.read_text(encoding="utf-8")) == [], path
118+
allowed = BOT_SECRET.get(path.name, set())
119+
assert foreign_secrets(path.read_text(encoding="utf-8"), allowed) == [], path

0 commit comments

Comments
 (0)