Skip to content

Preserve mounted REPL journals when tests fail - #1402

Merged
taras merged 6 commits into
mainfrom
codex/mounted-repl-failure-journals
Nov 18, 2026
Merged

taras merged 6 commits into
mainfrom
codex/mounted-repl-failure-journals

Conversation

@taras

@taras taras commented Nov 18, 2026 •

Copy link
Copy Markdown
Owner

Current delivery head: dd7e47bb7d85cc95ea175aaf1f521aa80db3cc16, the maintainer's merge of green main. The journal patch is unchanged. Retained Planner and selected Astra both PASS on this exact head; required CI passed.

Why

Mounted REPL tests write real session journals, then remove their temporary storage during cleanup. Failed cases therefore lose the entry and Sidekick histories before CI can upload them. This was the capture gap behind the missing journals from #1387.

Closes #1398. Continues the migration tracked in #1171.

What changes

Before: mounted fixture cleanup deletes the failed case's journals.

After: the shared case journal resource retains the mounted host's actual journal text before storage removal. A failed case saves it under XMD_TEST_ARTIFACTS, alongside metadata identifying the owning test, suite, execution root, session and journal file. A passing case discards the retained snapshot.

Mounted suites use ordinary beforeEach(journalBeforeEach(OWNER)) setup and keep their existing it declarations. Selection includes supported paired and legacy journals and excludes authored workspace assets and components.

Existing tests that deliberately hold an append also release it on exceptional paths. Their normal observation/release points stay intact, so an earlier assertion failure can reach cleanup and artifact saving.

How it works

command work unwinds
  -> outstanding fixture appends settle
  -> snapshot owned journal files
  -> remove temporary storage
  -> save attributed snapshots if the case failed

File-backed and in-memory roots share the existing per-case outcome resource. Snapshots retain original valid UTF-8 text, including Unicode and an unterminated final fragment; they do not parse or rewrite records. Capture adds no per-append or periodic diagnostic writes.

Review guide

Start with packages/cli/tests/support/mounted-repl.ts for file ownership and write settlement. Then review packages/test-support/failure-journal.ts for teardown ordering and case ownership, followed by packages/cli/src/test-diagnostics.ts for artifact placement and metadata.

The portable parent/child regression is packages/cli/tests/mounted-failure-journal.test.ts. Its expected failing children have captured output and a joined exit status, so they do not print misleading outer failures.

How to verify it

The regression covers successful cases producing no artifacts, a late assertion after successful execution, cooperative deadline/cancellation, multiple/nested roots with an earlier in-memory root, actual append settlement before snapshot/removal, authored JSONL exclusion, and valid UTF-8 preservation. A controlled failure before the normal append release must surface without an external rescue; removing its exceptional cleanup makes that control fail.

  • Final signed c4098d631d8a3bba7b01cfc2d7679a16fa472ec7: the layout-admission suite passed 13 tests / 37 steps, covering the final missing registration.
  • Capture/lifecycle evidence at 106d380b: Deno passed 12 tests / 44 steps; pinned Node 22.23.4 and Bun 1.4.0 passed 44 cases each. The final commit adds only that registration line.
  • The shared capture/resource implementation also passed its focused gate suites (47 tests / 196 steps), Deno typing and Node typing at 1bbe9dd6; those shared shapes are unchanged by the later owner-cleanup and registration fixes.

Planner: PASS on the final commit. Independent Astra (gpt-6-astra): PASS through the documented fallback after the retained Fable session could not resume. It carried forward Fable's full owner/capture audit and independently verified closure of the one missing-hook finding. Local Deno was 2.9.7 versus CI's 2.9.5. Required CI remains a delivery gate.

Scope

The change covers the shared mounted host and its ordinary test consumers. The driver-exit fixture deliberately fabricates replay history through savedRestoreFixture; that exception is recorded in the consumer inventory. Other remaining #1171 migrations stay open.

Generated or mechanical changes

The suite hook/import edits introduce per-case ownership without renaming test registration functions. The 17 owner-local exception guards reindent existing test bodies; review their acquisition/release scope with whitespace ignored.

Those 17 guards were initially inserted by an editing script, contrary to the task's no-codemod instruction. The provenance was preserved, and Planner and Fable subsequently inspected every owner manually. The later cleanup fixes and final registration used contextual manual edits. Existing successful-path release points and assertions remain.

Risks and limitations

Passing teardown reads the selected journal histories into memory until the case outcome is known. That cost grows with history size. Failure capture depends on cooperative teardown; it does not promise capture after a process is forcibly killed.

Scope confirmation

  • Every changed file supports mounted failure-journal capture.
  • Unrelated cleanup and formatting changes are excluded.
  • Hook/import and exception-guard changes are identified above.
  • The description matches the final diff and completed evidence.

taras and others added 5 commits November 17, 2026 18:38
A mounted REPL host stores its paired entry and Sidekick histories as real
files in a temporary root that is removed when the case's scope ends, so a
failed case's saver found nothing to keep. useMountedHost now registers the
root with the case's journal resource as soon as it owns it; the JSONL files
are read when that scope unwinds, before the root's removal, kept in memory,
and written beside the case identity only when the case fails.

Mounted suites opt into per-case attribution with beforeEach(journalBeforeEach)
and keep their plain it bodies. A child-process regression covers a passing
case, a late assertion, a case deadline that cancels a turn in flight, and a
nested and an outer root beside an in-memory journal, under each runtime.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A mounted host's failure snapshot copied every JSONL file beneath its root,
which included an execution's authored assets and components. The host now
supplies its own snapshot operation to retainJournalFiles: each paired
execution's entries and Sidekick files and each direct legacy history, read
as UTF-8 text.

Halting a task that asked for an append abandons its wait but not the write
already scheduled, so a held append could land after the histories were read
or the root removed. The mounted host now keeps its outstanding appends and its
teardown waits for each to settle before the snapshot and the removal. SS6
holds its never-written response with the operation gate, which a halt takes
back, rather than as a write the host would wait on.

The child fixture declares plain describe and it cases, with the witness set up
in beforeEach, and adds a held append whose release comes from outside the
waiting scope, authored files beside a pair, a second session and a legacy
history in a supplied root, and nested attribution.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Merges main efb666c (merged Round UI) into
the mounted failure-journal branch, keeping both Round behavior and
journalBeforeEach attribution.

The seventeen ordinary AppendGate holds that released only on their success
path now also release in an owner-local synchronous finally, so a failing
case's host drain finishes and its original error and journals surface. The
held-append witness tells its controller on every exit of its body, and a
held-message witness proves that the owner's own release, not a rescue,
lets the drain finish. The witnesses' entries run through Round Send.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…al exit

A failed assertion after Stop reached the held provider cancellation, or
after the contributed component's cleanup started, left that gate held, so
the host could not settle and the case's failure never surfaced. Each owner
now releases its gate in a finally while keeping its normal release point.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ntory cases

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

taras commented Nov 18, 2026

Copy link
Copy Markdown
Owner Author

Independent review: PASS on c4098d631d8a3bba7b01cfc2d7679a16fa472ec7, using the documented Codex Astra (gpt-6-astra) fallback after the retained Fable session could not resume. This is an Astra verdict, not a Fable re-review.

Fable had independently audited the complete capture path, all 17 append owners, PV3/R4f cleanup and consumer attribution. Its one blocking finding was the missing layout-suite registration. The final commit adds only that hook. Astra independently verified that it is installed before fixture acquisition and that the registered/unregistered failure probe distinguishes the repaired omission.

All five acceptance clauses passed: capture before deletion, case identity, no artifacts on success, observable timeout/cancellation cleanup, and portable runtime evidence. No blocking finding remains.

Evidence includes the final layout run (13 tests / 37 steps), the applicable Deno/Node/Bun lifecycle/capture checks, shared typing checks, and the preserved broken-cleanup controls. Astra reused that evidence without rerunning tests or inspecting CI. Review receipt SHA-256: 3beca2f19ec503cec214097d004221ea9fa52a2a8585c2c426122e0fa260e1db.

Limits remain explicit: valid UTF-8 text preservation, cooperative cleanup, and a passing-path snapshot read proportional to selected history size. Seven registrations involving separately owned journey roots remain outside this temporary-directory deletion repair; #1171 stays open.

Implementation provenance: the initial 17-guard transformation violated the no-codemod instruction. Its script and tool records were preserved, and both Planner and Fable manually inspected every changed owner afterward. The later cleanup and final hook were contextual manual edits. This correctness verdict does not rewrite the process history.

Required CI and exact-main health remain separate merge gates. The maintainer owns the merge.

taras commented Nov 18, 2026

Copy link
Copy Markdown
Owner Author

Delivery status: run 44222375137 finished with 59 successful jobs and one skipped job. The real test failure is Node 4/18: Delivery.test.md reached its outer 300-second deadline. Its final progress-completion scenario started about 262 seconds into the document and was interrupted at a Prompt. This is not yet diagnosed as either a stalled operation or cumulative runtime cost.

The first failure output and both uploaded journals are preserved before any rerun. A native Fable Problem Solver is inspecting them read-only. No timeout change, source edit or retry has been made for this failure.

The separate main-green failure reflects the genuine macOS History race on current main; #1406 is the focused repair. The final journal implementation remains at reviewed c4098d631d8a3bba7b01cfc2d7679a16fa472ec7.

@taras
taras marked this pull request as ready for review November 18, 2026 03:14

taras commented Nov 18, 2026

Copy link
Copy Markdown
Owner Author

The native Fable diagnosis has joined. The supported cause is cumulative Delivery-suite runtime exceeding its 300-second document budget. The final case had used only about 39 seconds of its 90-second allowance; its journal shows three completed variants before the fourth variant's opening Prompt was interrupted. Earlier phases were also slower, so the trace does not demonstrate a blocked Prompt.

The journal implementation does not execute on this pre-failure path. The unchanged exact revision passed a local Node probe in 219.76 seconds; host and runtime versions differ from CI, so this is not an identical-host CI retry.

The focused test-topology repair is tracked separately in #1408 and is entering the native role workflow. It preserves assertions and deadlines, extracts the two existing nested groups into separately discovered Markdown suites, and includes the required weight remeasurement. No further journal-source correction or budget reset is being made on this PR. #1406 has merged; its new main run is 44228908083.

taras commented Nov 18, 2026

Copy link
Copy Markdown
Owner Author

The single bounded failed-job retry attempt 2 passed on unchanged reviewed c4098d631d8a3bba7b01cfc2d7679a16fa472ec7; no source or deadline changed. The original failing and passing Node logs are retained. This confirms hosted intermittence for #1408, which stays open with its causal correction in #1409.

Main is also genuinely green at 1ce93ad5, and #1399 closed automatically. Before offering this PR for merge, its branch still needs mechanical base maintenance under the strict current-main rule and exact-head review reconciliation. That work will preserve the journal patch and exhausted correction record, and will be sequenced after #1407 to avoid repeating the same base update.

taras commented Nov 18, 2026

Copy link
Copy Markdown
Owner Author

The branch is already updated by maintainer merge dd7e47bb7d85cc95ea175aaf1f521aa80db3cc16 (parents: reviewed c4098d63 and current main 1ce93ad5). Its full 45-file journal patch is byte-for-byte identical to the previously reviewed patch, SHA-256 7ff6aca754847252598de286e69d7ac7ba4a1b36a0841992ed327b1a8e780df0. GitHub verifies the maintainer merge signature as valid.

The separate exact-head CI run 44229032148 passed. The retained Planner and selected Astra reviewer are reconciling against this actual head now. No further restack, new commit, journal-source correction or budget reset is needed. This supersedes the earlier sequencing note that assumed the hosted branch was still at c4098d63.

taras commented Nov 18, 2026

Copy link
Copy Markdown
Owner Author

Final delivery reconciliation: PASS on current maintainer head dd7e47bb7d85cc95ea175aaf1f521aa80db3cc16.

  • Retained Planner conversation: PASS.
  • Same selected independent Codex Astra reviewer 01a1ec6f: PASS, all five original acceptance clauses retained, no blocking findings.
  • The complete 45-file journal patch and every changed file remain byte-identical to reviewed c4098d63; incoming main changes were inspected. Earlier focused runtime/typing evidence is explicitly inherited, not described as rerun.
  • Exact-head CI 44229032148 passed, and its base is current green main 1ce93ad5.

Astra receipt SHA-256: 894470aaa54e5fb133cbb6a16fb668992fc3ecc1eaad0133652ae097c69db88c. Original Fable/Astra findings, first failure, identical-revision passing retry and recovery records remain preserved. No source amendment, new commit or reset of the exhausted 4/4 source allowance occurred.

The PR is already non-draft and is ready for maintainer merge. The separate Delivery reliability correction remains tracked in #1408/#1409; an unchanged retry did not close that issue.

@taras
taras merged commit 731194d into main Nov 18, 2026
64 checks passed
@taras
taras deleted the codex/mounted-repl-failure-journals branch November 18, 2026 07:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Preserve mounted REPL journals when tests fail

1 participant