Skip to content

Commit 77e5e2a

Browse files
Seungpyo1007Seungpyo1007
committed
ci: add TechEngineBot's label-triggered final check
Adding the final-check label to a PR runs TechMachine's checks (tests including a generated satellite repo, repos.json, syntax, SHA-pinned GitHub-owned actions) and TechEngineBot leaves one comment with the verdict, edited in place when the label is added again. It replaces Copilot as the reviewer. Same design as TechAPI's final check: the check job runs PR code with no secrets and a read-only token, and only the comment job sees TECHENGINEBOT_TOKEN, which must be a fine-grained token of TechEngineBot. Logs are fenced with backticks and @ removed so they cannot mention anyone.
1 parent 9f52bb9 commit 77e5e2a

3 files changed

Lines changed: 237 additions & 1 deletion

File tree

‎.github/workflows/final-check.yml‎

Lines changed: 216 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,216 @@
1+
name: final-check
2+
3+
# A maintainer adds the `final-check` label to an important PR. The check job
4+
# runs what TechMachine's CI checks (tests, repos.json, syntax, pinned actions)
5+
# and TechEngineBot leaves ONE comment with the verdict; adding the label again
6+
# edits that comment. Same design as TechAPI's final-check.
7+
#
8+
# The check job runs PR code, so it gets no secrets and a read-only token, and
9+
# nothing in it writes anywhere or reaches another repository. Only the comment
10+
# job sees TECHENGINEBOT_TOKEN, which must be a fine-grained token of
11+
# TechEngineBot with Pull requests: read and write; the job refuses any other
12+
# kind of token or account. Fork PRs never get a comment.
13+
on:
14+
pull_request:
15+
types: [labeled]
16+
branches: [develop, main]
17+
18+
permissions: {}
19+
20+
jobs:
21+
check:
22+
if: github.event.label.name == 'final-check'
23+
runs-on: ubuntu-24.04
24+
timeout-minutes: 15
25+
permissions:
26+
contents: read
27+
concurrency:
28+
group: final-check-${{ github.event.pull_request.number }}
29+
cancel-in-progress: true
30+
outputs:
31+
body: ${{ steps.report.outputs.body }}
32+
env:
33+
PYTHONIOENCODING: utf-8
34+
steps:
35+
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
36+
with:
37+
persist-credentials: false
38+
39+
- uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
40+
with:
41+
python-version: "3.12"
42+
43+
- name: Install pytest
44+
id: install
45+
continue-on-error: true
46+
run: |
47+
status=0
48+
python -m pip install --disable-pip-version-check pytest > "$RUNNER_TEMP/install.log" 2>&1 || status=$?
49+
tail -n 20 "$RUNNER_TEMP/install.log"
50+
exit "$status"
51+
52+
# Includes generating a satellite repository and running its own tests
53+
# and validator, so a template change that breaks new repos fails here.
54+
- name: Tests
55+
id: tests
56+
if: steps.install.outcome == 'success'
57+
continue-on-error: true
58+
run: |
59+
status=0
60+
python -m pytest -q -p no:cacheprovider > "$RUNNER_TEMP/tests.log" 2>&1 || status=$?
61+
cat "$RUNNER_TEMP/tests.log"
62+
exit "$status"
63+
64+
- name: Validate repos.json
65+
id: config
66+
continue-on-error: true
67+
run: |
68+
status=0
69+
python - > "$RUNNER_TEMP/config.log" 2>&1 <<'EOF' || status=$?
70+
import json, sys
71+
cfg = json.load(open("machine/repos.json", encoding="utf-8"))
72+
errors = []
73+
for repo in cfg.get("repos", []):
74+
name = repo.get("name", "")
75+
if not name.startswith("GetTechAPI/"):
76+
errors.append(f"repo {name!r} is not in GetTechAPI")
77+
if not repo.get("branches"):
78+
errors.append(f"{name}: no branches")
79+
if not all(isinstance(s, str) for s in repo.get("allowed_secrets", [])):
80+
errors.append(f"{name}: allowed_secrets must be strings")
81+
for endpoint in cfg.get("endpoints", []):
82+
if not endpoint.get("url", "").startswith("https://"):
83+
errors.append(f"{endpoint.get('name')}: url must be https")
84+
if not endpoint.get("expect"):
85+
errors.append(f"{endpoint.get('name')}: no expected field")
86+
if not cfg.get("repos"):
87+
errors.append("no repos")
88+
print("\n".join(errors) or f"{len(cfg['repos'])} repos, {len(cfg.get('endpoints', []))} endpoints")
89+
sys.exit(1 if errors else 0)
90+
EOF
91+
cat "$RUNNER_TEMP/config.log"
92+
exit "$status"
93+
94+
# The repository only allows GitHub-owned actions pinned by full SHA.
95+
- name: Syntax and pinned actions
96+
id: lint
97+
continue-on-error: true
98+
run: |
99+
status=0
100+
{
101+
python -m compileall -q machine tests || status=1
102+
unpinned=$(grep -nE '^\s*-?\s*uses:' .github/workflows/*.yml | grep -vE 'uses: actions/[a-z-]+@[0-9a-f]{40}( |$)' || true)
103+
if [ -n "$unpinned" ]; then
104+
echo "Not a GitHub-owned action pinned by SHA:"
105+
echo "$unpinned"
106+
status=1
107+
fi
108+
} > "$RUNNER_TEMP/lint.log" 2>&1
109+
cat "$RUNNER_TEMP/lint.log"
110+
exit "$status"
111+
112+
# The comment quotes tool output, which can carry text from the PR: logs
113+
# stay inside code fences with backticks and @ removed, so they can
114+
# neither break out into markdown nor mention anyone.
115+
- name: Report
116+
id: report
117+
if: always()
118+
env:
119+
INSTALL: ${{ steps.install.outcome }}
120+
TESTS: ${{ steps.tests.outcome }}
121+
CONFIG: ${{ steps.config.outcome }}
122+
LINT: ${{ steps.lint.outcome }}
123+
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
124+
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
125+
run: |
126+
clean() { tr -d '`@' | cut -c1-300; }
127+
fence() { # fence <log> <lines>
128+
echo '```text'
129+
tail -n "$2" "$1" 2>/dev/null | clean
130+
echo '```'
131+
}
132+
verdict='✅ Final check passed'
133+
[ "$INSTALL/$TESTS/$CONFIG/$LINT" = success/success/success/success ] || verdict='❌ Final check failed'
134+
{
135+
echo '<!-- techmachine-final-check -->'
136+
echo "### $verdict"
137+
echo
138+
echo "Head \`${HEAD_SHA::7}\`, checked as merged into the base · [run]($RUN_URL)"
139+
echo
140+
echo '| Step | Result |'
141+
echo '| --- | --- |'
142+
echo "| Install pytest | ${INSTALL:-skipped} |"
143+
echo "| Tests (incl. a generated satellite repo) | ${TESTS:-skipped} |"
144+
echo "| repos.json | ${CONFIG:-skipped} |"
145+
echo "| Syntax and pinned actions | ${LINT:-skipped} |"
146+
[ "$INSTALL" = failure ] && { echo; echo '<details><summary>Install log</summary>'; echo; fence "$RUNNER_TEMP/install.log" 20; echo '</details>'; }
147+
[ "$TESTS" = failure ] && { echo; echo '<details><summary>Test log</summary>'; echo; fence "$RUNNER_TEMP/tests.log" 40; echo '</details>'; }
148+
[ "$CONFIG" = failure ] && { echo; echo '<details><summary>repos.json log</summary>'; echo; fence "$RUNNER_TEMP/config.log" 20; echo '</details>'; }
149+
[ "$LINT" = failure ] && { echo; echo '<details><summary>Syntax and pinned actions log</summary>'; echo; fence "$RUNNER_TEMP/lint.log" 20; echo '</details>'; }
150+
true
151+
} > "$RUNNER_TEMP/body.md"
152+
cat "$RUNNER_TEMP/body.md" >> "$GITHUB_STEP_SUMMARY"
153+
delim="BODY_$(openssl rand -hex 16)"
154+
{ echo "body<<$delim"; cat "$RUNNER_TEMP/body.md"; echo "$delim"; } >> "$GITHUB_OUTPUT"
155+
[ "$verdict" = '✅ Final check passed' ]
156+
157+
comment:
158+
needs: check
159+
if: >-
160+
always() && github.event.label.name == 'final-check' &&
161+
(needs.check.result == 'success' || needs.check.result == 'failure') &&
162+
github.event.pull_request.head.repo.full_name == github.repository
163+
runs-on: ubuntu-24.04
164+
timeout-minutes: 5
165+
permissions: {}
166+
steps:
167+
- name: Check the token
168+
id: token
169+
env:
170+
TOKEN: ${{ secrets.TECHENGINEBOT_TOKEN }}
171+
run: |
172+
if [ -z "$TOKEN" ]; then
173+
echo "::warning::TECHENGINEBOT_TOKEN is not set; the verdict is in the check job's summary only."
174+
echo "ok=false" >> "$GITHUB_OUTPUT"
175+
exit 0
176+
fi
177+
case "$TOKEN" in
178+
github_pat_*) echo "ok=true" >> "$GITHUB_OUTPUT" ;;
179+
*) echo "::error::TECHENGINEBOT_TOKEN must be a fine-grained token (github_pat_...) of TechEngineBot with Pull requests: read and write."
180+
exit 1 ;;
181+
esac
182+
183+
- uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
184+
if: steps.token.outputs.ok == 'true'
185+
env:
186+
BODY: ${{ needs.check.outputs.body }}
187+
PR: ${{ github.event.pull_request.number }}
188+
with:
189+
github-token: ${{ secrets.TECHENGINEBOT_TOKEN }}
190+
script: |
191+
const marker = '<!-- techmachine-final-check -->';
192+
const body = process.env.BODY || '';
193+
if (!body.startsWith(marker)) {
194+
core.setFailed('The check job produced no report; nothing posted.');
195+
return;
196+
}
197+
const { data: me } = await github.rest.users.getAuthenticated();
198+
if (me.login !== 'TechEngineBot') {
199+
core.setFailed(`TECHENGINEBOT_TOKEN belongs to ${me.login}, not TechEngineBot; nothing posted.`);
200+
return;
201+
}
202+
const { owner, repo } = context.repo;
203+
const issue_number = Number(process.env.PR);
204+
const comments = await github.paginate(github.rest.issues.listComments, {
205+
owner, repo, issue_number, per_page: 100,
206+
});
207+
const mine = comments.find((c) => c.user?.login === me.login && c.body?.startsWith(marker));
208+
if (!mine) {
209+
const { data } = await github.rest.issues.createComment({ owner, repo, issue_number, body });
210+
core.info(`Posted ${data.html_url}`);
211+
} else if (mine.body !== body) {
212+
await github.rest.issues.updateComment({ owner, repo, comment_id: mine.id, body });
213+
core.info(`Updated ${mine.html_url}`);
214+
} else {
215+
core.info(`Unchanged: ${mine.html_url}`);
216+
}

‎README.md‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,21 @@ python -m pytest -q
4040

4141
Adding a repository or endpoint is one entry in `machine/repos.json`.
4242

43+
## Final check
44+
45+
Adding the `final-check` label to a pull request asks
46+
[TechEngineBot](https://github.com/TechEngineBot) for a last look, in place of
47+
Copilot. The check job runs the tests (including a generated satellite
48+
repository), validates `repos.json`, and checks syntax and that every action
49+
is GitHub-owned and pinned by SHA. TechEngineBot then leaves **one** comment
50+
with the verdict; adding the label again edits it.
51+
52+
The check job runs the PR's code with no secrets and a read-only token. Only
53+
the comment job sees `TECHENGINEBOT_TOKEN`, and it posts nothing unless that is
54+
a fine-grained token belonging to TechEngineBot. This is the only thing the
55+
bot account does here: it speaks when a person asks, once per PR. Same design
56+
as TechAPI's final check.
57+
4358
## Satellite repositories
4459

4560
Categories that do not belong in TechAPI live in their own repository (games:

‎tests/test_health.py‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,11 @@ def test_allowed_secret_is_fine():
7676
assert foreign_secrets("${{ secrets.SOME_API_KEY }}", {"SOME_API_KEY"}) == []
7777

7878

79+
# TechEngineBot's token is allowed in one place: the label-triggered final check.
80+
BOT_SECRET = {"final-check.yml": {"TECHENGINEBOT_TOKEN"}}
81+
82+
7983
def test_techmachine_itself_uses_only_the_builtin_token():
8084
for path in sorted((ROOT / ".github" / "workflows").glob("*.yml")):
81-
assert foreign_secrets(path.read_text(encoding="utf-8")) == [], path
85+
allowed = BOT_SECRET.get(path.name, set())
86+
assert foreign_secrets(path.read_text(encoding="utf-8"), allowed) == [], path

0 commit comments

Comments
 (0)