Skip to content

ci: split slow Verify jobs with exact file coverage and strict summaries - #355

Merged
Ancienttwo merged 2 commits into
mainfrom
codex/ci-verify-sharding
Oct 14, 2026
Merged

Ancienttwo merged 2 commits into
mainfrom
codex/ci-verify-sharding

Conversation

@Ancienttwo

@Ancienttwo Ancienttwo commented Oct 14, 2026 •

Copy link
Copy Markdown
Owner

Verify can exhaust the existing 20-minute macOS / 60-minute Windows caps while tests keep passing. Split only the four measured timeout targets—macOS15/Node24, macOS15/Node25, Windows/Node22.22 and Windows/Node25—into three isolated whole-file test jobs. The other five targets retain complete verifiers.

This CI delivery is independent of #351 / #354. #354 remains frozen at 468df532; no product commits are included. Base: d6788c4b565aa043530b2f6407234b3ed108fa00. Review HEAD: d0fcb921b5fe088cf4406ad7280d5bf441170236. Keep Draft; no merge or serial integration yet.

  • First-round evidence is retained: 38637184856, HEAD a10fafde, ended with 27 successful jobs, macOS25 and Windows22 execution timeouts/cancellations, and two correctly failed summaries. Neither cancelled execution log contained a (fail) marker. Windows22 finished main tests (3104 pass / 5 skip / 0 fail, 3352.16s), explorer and native smoke; the full packaged CLI smoke and later gates were unfinished. macOS25's main suite did not finish. This was not all-green.
  • Cost: first round measured 282.87 raw runner minutes, 302 in the per-job rounded model (not an invoice), and 19 installs / 6.68 minutes; the extra eight shard/post installs consumed 3.87 minutes. Four-target topology now has 25 execution jobs, 27 installs and 39 total jobs. The additional eight installs model 2.93 raw minutes at the observed 13s macOS25 / 31s Windows22 rates. Provisional total planning is 295–310 raw runner minutes, including an explicit allowance for more setup and previously unfinished work; it is not a proven bound or cost-saving claim. Actual second-round totals remain pending. No Governance reuse savings are assumed.
  • Order/state: pre → all shards → exact coverage proof → post. Local bun run verify remains byte-for-byte unchanged. Post reinstalls dependencies and rebuilds its native bundle/Git/daemon fixtures. Only run-bound discovery/inventory/JUnit metadata crosses machines; tests keep real isolated processes, protocols and every product assertion.
  • Discovery: pinned Bun supplies the actual current census, including empty files; weights never whitelist paths. Added/deleted paths, spaces, duplicate basenames and empty shards are checked with real Bun fixtures. Fresh native shard inventories must cover every discovered file exactly once for the same run/attempt/checkout.
  • Summary: always() with current-attempt paginated job evidence; every expected worker must uniquely be completed/success with matching subject. Missing, skipped, cancelled, failed, duplicate or stale jobs fail. All nine FG6 summary names, IPC artifact semantics, permissions and caps remain unchanged.
  • Governance is unchanged. The runbook records its 15-minute pressure and a deferred concrete receipt/dedup proposal. That authority change is not approved or implemented in this push; fix(runtime): preserve selected failure metadata (#328) #356/D6-A is untouched.

Second-round delta is limited to .github/verify-sharding.json, scripts/verify-sharding-workflow.test.ts and the active runbook. The approved policy expectation expands from two to four targets. A real topology CLI regression checks four pre/post targets, twelve shards, five unsplit workers and 25 execution jobs; strict rejection cases cover each expected worker.

Validation: 27 focused tests pass, 0 fail, 344 assertions; typecheck, package-boundary audit and diff checks pass. Workflow/controller/package bytes match the first candidate; actionlint1.7.12 passed on that unchanged workflow. The original native discovery census was 260 files, including 3 absent from old weights and excluding 1 deleted weighted path. No product/protocol assertions or historical verification artifacts changed.

Second-round CI: Verify 38647011093, queued when created; budget and results are pending. Detailed state/dependency audit, first-round failures, measured cost and next-run assumptions: verify sharding runbook.

Coordinator acceptance (2026-10-14 23:10 UTC): exact HEAD d0fcb92, Verify 38647011093 attempt 1 completed successfully; all 39 jobs and all nine named summaries passed. Independent static and incremental reviews found no blocking findings. Observed total: 273.23 raw runner minutes, 298 summed per-job-rounded minutes (model, not invoice), 27 installs / 12.28 minutes; sixteen extra shard/post installs / 7.22 minutes. First-to-last job wall clock 48m45s. One successful run supports acceptance, not a stable cost-saving claim. Prior pending language above describes the state when the candidate was first published.

@Ancienttwo
Ancienttwo marked this pull request as ready for review October 14, 2026 23:11
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-14T23:14:16.414364Z d0fcb92 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@Ancienttwo
Ancienttwo merged commit f4d87ae into main Oct 14, 2026
40 checks passed
@Ancienttwo
Ancienttwo deleted the codex/ci-verify-sharding branch October 15, 2026 15:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant