Skip to content

Give Delivery's nested Markdown suites independent CI deadlines - #1409

Merged
taras merged 1 commit into
mainfrom
codex/delivery-suite-topology-1408
Nov 18, 2026
Merged

taras merged 1 commit into
mainfrom
codex/delivery-suite-topology-1408

Conversation

@taras

@taras taras commented Nov 18, 2026 •

Copy link
Copy Markdown
Owner

Why

Delivery's Markdown suite exceeded its shared five-minute deadline on a slow Node CI runner, interrupting the final case after about 39 seconds of its 90-second allowance. Its earlier phases had also slowed; the saved trace does not demonstrate a blocked Prompt. Refs #1408.

What changes

The existing summary-repair and progress-completion groups become two independently discovered Markdown suites. Delivery keeps its 300-second document deadline. The new suites each use the existing 120-second adversarial suite class, while retaining their original wrapper and nested case timeouts.

How it works

CI discovers and shards DeliverySummaryRepair.test.md and DeliveryProgressCompletion.test.md directly. Their existing wrappers still launch the nested documents through the normal Deno fixture command under every outer runtime. Both suites are explicitly registered in the deadline map and discovery/launch regressions.

How to verify it

  • All nine focused launches passed: three suites under Deno, Node and Bun, using the real repository runner and Deno child profile.
  • Reduced Delivery took 162–170 seconds locally; each extracted suite took 27–30 seconds. These are local completion measurements, not a controlled hosted speedup.
  • Structural comparisons verify the retained Delivery prefix, moved wrappers, shared preambles, nested documents, titles, assertions and authored timeouts are unchanged.
  • Discovery/deadline regression, affected tests (22 tests, 85 steps), explicit runner/test typecheck and formatting passed.
  • Original Planner: PASS at signed 29bfc884418892117ac664c36ad524e2c2aae649. Independent Codex gpt-6-astra review: PASS on the same source revision, after the retained Fable quota-only refusal and documented provider reconciliation.

Scope

The change reorganizes existing test groups and registers their existing deadline class. It adds no product API, timeout increase, dependency change or weaker assertion.

Generated or mechanical changes

Hosted corpus remeasurement is mandatory and pending. The first run was stopped because it excluded the new test file in pending #1402. After #1402 and #1407 merge, the supported workflow will measure the final rebased source, covering the settled corpus. Its complete assembled artifact will be committed deliberately with truthful source/run/attempt provenance; measurements will not be edited manually. Source review and passing focused tests do not waive this delivery work.

Risks and limitations

Local runtime/host versions differ from hosted CI. Source reviews are complete. Required CI passed on the reviewed source commit. Hosted measurement, the generated weights commit and final exact-head review/CI remain pending. The known timeout issue stays open until its correction and delivery evidence are complete.

Scope confirmation

  • Every changed file supports the suite isolation.
  • Existing scenario assertions and authored case allowances are preserved.
  • Unrelated cleanup and formatting changes are excluded.
  • The description distinguishes completed evidence from pending delivery gates.

Delivery.test.md ran the nested summary-repair and progress-completion
documents inside its own 300-second whole-launch budget, and on a slow
Node runner the cumulative work reached that deadline (#1408). Each
wrapper now lives in its own directly discovered suite with the 120-second
adversarial deadline, keeping its Deno child command, authored case
timeout and nested document unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

taras commented Nov 18, 2026

Copy link
Copy Markdown
Owner Author

Hosted measurement started on the exact source commit 29bfc884418892117ac664c36ad524e2c2aae649: run 44232626922, attempt 1. Deno, Node and Bun are measured independently; only complete matching records may be assembled. Required PR CI is 44232417602.

Fable returned only its five-hour quota notice before producing a review; its provider scope joined and all failed records are retained. The existing repository policy and prior user direction authorize the independent Codex gpt-6-astra replacement. That review is being resumed through explicit recovery records on the same source/evidence, without changing the classifier, original identity or correction limits. No independent PASS is claimed yet.

taras commented Nov 18, 2026

Copy link
Copy Markdown
Owner Author

Independent review: PASS on exact signed 29bfc884418892117ac664c36ad524e2c2aae649.

Reviewer: Codex gpt-6-astra, actual independent native identity 01a1ecc6-86ad-7dc0-9ed3-6e546dc0409c. Fable's quota-only attempt was preserved, its provider joined, and its held wrapper was settled through supported reconciliation before the replacement started. Source and correction limits were unchanged.

The reviewer independently compared both moved wrappers/preambles and the unchanged nested documents against the base. All titles, variants, assertions and 120s/90s case allowances are preserved. Discovery, explicit 300s/120s document deadlines, fixture plugin/includes and Deno child execution were traced through the real runner. All nine candidate launch receipts match the reviewed source; focused formatting also passed. No blocking findings.

Receipt SHA-256: 72c6ffa95aed1895dcddedf90658f92cfef8c4873d2ed627e0c8bb6eb25b4197.

Hosted measurement 44232626922, required CI and exact-head review of the eventual generated weights remain delivery gates. This source PASS does not claim merge readiness.

taras commented Nov 18, 2026

Copy link
Copy Markdown
Owner Author

Required source CI passed on reviewed 29bfc884418892117ac664c36ad524e2c2aae649: run 44232417602, 62 successful jobs and one expected skip. Planner and independent Astra source reviews are PASS.

The PR remains draft because the required full-corpus measurement 44232626922 is still running. Its complete generated weights, truthful measured-source provenance and final exact-head review/CI are still required. No incomplete measurement or source-only PASS is being treated as merge readiness.

taras commented Nov 18, 2026

Copy link
Copy Markdown
Owner Author

Measurement sequencing correction: run 44232626922 was deliberately stopped. It began before the pending journal PR #1402's added test file was included, so its corpus would be obsolete once that PR merges. All three measurement jobs were still running without a failure when cancellation was requested. Partial records/logs will be retained as historical evidence.

The mandatory complete measurement will run after #1402 and #1407 merge and this PR is based on their actual final main. The resulting artifact must still pass same-source/run/attempt validation, supported reassembly, deliberate generated-file commitment and final exact-head review/CI. Source 29bfc884 remains Planner/Astra PASS with green required CI; the PR stays draft while delivery work is pending.

taras commented Nov 18, 2026

Copy link
Copy Markdown
Owner Author

The cancelled measurement's preserved partial records had already completed all three affected suites on source 29bfc884, using the pinned hosted toolchain (Deno 2.9.5, Node 22.23.4, Bun 1.4.0):

Outer runtime Delivery Progress completion Summary repair
Deno 117.077s 28.968s 29.034s
Node 179.538s 40.106s 40.244s
Bun 121.680s 28.535s 28.409s

Delivery's document limit remains 300s; each extracted suite's document limit is 120s. Authored inner case limits remain unchanged. These completed individual-file measurements are retained evidence of bounded hosted execution; they do not establish a controlled total-runtime speedup.

All three full-corpus records are explicitly partial, with no recorded failure. They are retained for evidence and cannot supply the final weights artifact. Final corpus measurement still follows the pending #1402/#1407 merges and base reconciliation.

@taras
taras marked this pull request as ready for review November 18, 2026 08:01
@taras
taras merged commit 6207d24 into main Nov 18, 2026
64 of 68 checks passed
@taras
taras deleted the codex/delivery-suite-topology-1408 branch November 18, 2026 08:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant