Skip to content

fix: report a ghostty-web page crash as REPLAY_ERROR on a live session - #363

Merged
ThomasK33 merged 7 commits into
mainfrom
fix/ghostty-web-crash-replay-error
Nov 8, 2026
Merged

ThomasK33 merged 7 commits into
mainfrom
fix/ghostty-web-crash-replay-error

Conversation

@ThomasK33

@ThomasK33 ThomasK33 commented Nov 8, 2026 •

Copy link
Copy Markdown
Member

Why

ghostty-web 0.4.0 has a bug. A short program output crashes the Chromium page of the ghostty-web renderer. The pinned version is 0.4.0 and the newest prerelease (0.4.0-next.21) still has the bug. No release has a fix.

The shortest input has 12 bytes. They are a, U+0301 (a combining acute), ESC c (full reset), ESC [ ? 4 7 h (enter the alternate screen) and o. In printf form: printf 'a\xcc\x81\033c\033[?47ho'.

  • After a full reset, ghostty-web keeps the old cells on the alternate screen when ?47h or ?1047h enters it. ?1049h is not affected.
  • Output over a cell that held a combining mark or another grapheme cluster then makes the WASM module fault.
  • Chromium kills the page. Playwright reports page.evaluate: Target crashed.
  • The native libghostty-vt renderer shows a blank alternate screen and does not fault on the same bytes.

I found the input by delta-reduction of seeded random logs. The same input crashes in plain Node with only the WASM file, and 300 of 300 random logs agree between Node and Chromium. A random log needs all three parts (full reset, ?47h or ?1047h, grapheme cluster). Reverse index (ESC M) is not needed.

Upstream issue: coder/ghostty-web#215. When a ghostty-web release closes it and agent-tty upgrades to it, remove the error class and the tests that carry a KNOWN-BUG header, or invert them to expect a screenshot.

This PR does not fix the crash. It changes what agent-tty tells the user when it happens.

What

This PR has seven commits:

  1. Names the crash in messages and keeps every code.
  2. Changes the code of the live error.
  3. Corrects one sentence of the exit-code table.
  4. Links the upstream issue and adds negative cases for the crash classifier.
  5. Gives the crash behind an offline screenshot --renderer libghostty-vt the same REPLAY_ERROR (see below).
  6. Adds a classifier case that contains crashed without Target crashed.
  7. Docs for commit 5.
  • New GhosttyWebCrashedError in src/renderer/ghosttyWeb/crashError.ts. Its message names the crash and the known cause. It ends with the page error in parentheses. Its cause is the Playwright error.
  • GhosttyWebBackend.replayTo and replayWithTiming throw it when a page call fails with Target crashed. Every other error passes through unchanged. The two bodies are now replayToInner and replayWithTimingInner, and the public methods wrap them.
  • withOfflineReplayRenderer (exited sessions) and record export add the message to their fixed text. Before, REPLAY_ERROR and EXPORT_ERROR hid the cause in error.cause, which no CLI output prints.
  • Second commit: HostRendererManager.withBackend (src/host/renderer.ts) turns a GhosttyWebCrashedError into a CliError with code REPLAY_ERROR and the same message. A live screenshot, snapshot --renderer ghostty-web or render wait now fails like the same command on an exited session: exit 10, retryable: false. Before, the host sent a plain Error, which became RPC_ERROR (exit 9) with retryable: true. The crash is deterministic, so a retry crashes again. Every other error passes through.
  • withOfflineReplayRenderer (src/replay/offlineReplay.ts) also maps a GhosttyWebCrashedError that its callback throws. A libghostty-vt screenshot replays the log a second time in a ghostty-web page that it boots inside the callback, so on an exited session that crash was outside the replay try/catch. Before, it exited 1 with INTERNAL_ERROR. Now it gives the same REPLAY_ERROR as the replay (Failed to boot or replay renderer for session <id>. and the explanation). Every other callback failure keeps its code.
  • No schema changes. REPLAY_ERROR already exists in ERROR_CODES, its exit code (10) is in EXIT_CODE_BY_ERROR_CODE, and RpcErrorSchema.code is a free string. The retryable flag is not on the wire: the CLI derives it from the code. record export --format webm keeps EXPORT_ERROR (exit 7).
  • Docs: the 10 row of the exit-code table (it says that a crash in a screenshot, snapshot or wait replay exits 10, and that a WebM export exits 7) and a pointer in docs/USAGE.md, a new A Command Says ghostty-web crashed section in docs/TROUBLESHOOTING.md, and the READMEs of src/host, src/renderer/ghosttyWeb, src/replay and src/cli/commands.

What the user sees for the 12-byte log (real CLI, AGENT_TTY_HOME under /tmp):

Command Session origin/main This PR
screenshot, snapshot --renderer ghostty-web live exit 9, RPC_ERROR, retryable: true, message page.evaluate: Target crashed exit 10, REPLAY_ERROR, retryable: false. The message starts ghostty-web crashed while it replayed this session's output. and ends (page.evaluate: Target crashed)
wait --text o --renderer ghostty-web live exit 9 after 10 failed polls exit 10, REPLAY_ERROR after 10 failed polls (median 10.1 s, see below)
screenshot --renderer libghostty-vt live exit 9, RPC_ERROR, retryable: true, message page.evaluate: Target crashed exit 10, REPLAY_ERROR, retryable: false
screenshot --renderer libghostty-vt exited exit 1, INTERNAL_ERROR, message page.evaluate: Target crashed exit 10, REPLAY_ERROR, retryable: false, message Failed to boot or replay renderer for session <id>. ghostty-web crashed …
screenshot, snapshot --renderer ghostty-web, wait --renderer ghostty-web exited exit 10, REPLAY_ERROR, Failed to boot or replay renderer for session <id>. same code and exit. The same prefix, then the explanation
record export --format webm live or exited exit 7, EXPORT_ERROR, Failed to export session "<id>" as webm. same code and exit. The same prefix, then the explanation
snapshot, wait --text, inspect, record export --format svg any work work (unchanged)

The host and the session stay up after a crash. Later default commands on the same session work.

Costs and limits

  • Contract change. A script that matches exit 9 or RPC_ERROR for this failure sees exit 10 and REPLAY_ERROR now. I did not test an old CLI against a new host. The code string is one that old CLIs know, so I expect exit 10 there too.
  • No runtime cost. One try/catch around each of two replay methods, and one .catch in withBackend.
  • The host keeps the crashed headless Chromium (5 processes, about 313 MB RSS and 88 MB PSS in total) until the next render command or destroy. This PR does not change that.
  • The message names ghostty-web 0.4.0. Update crashError.ts when ghostty-web changes.
  • A page crash only inside snapshot() (the replay succeeds, snapshot() crashes) stays RPC_ERROR (exit 9, retryable: true), and the retry gives the same error. A retry gets REPLAY_ERROR only when the replay itself crashes. No known input crashes in snapshot().
  • The wait time of the live wait --renderer ghostty-web depends on host load. Each of the 10 failed polls boots a new Chromium. I measured it 5 times on c4f8d7b7, host load 38 to 61: 5.9, 11.0, 11.3, 8.9 and 10.1 s (median 10.1 s).
  • The new integration tests run on Linux only. I did not verify how Chromium on macOS reports the fault.

Validation

Run on commit c4f8d7b7, from a git archive copy of it, Node 26.8.1, Chromium headless shell 1248.

  • oxfmt . --check, oxlint src test vitest.config.ts (0 warnings, 0 errors), tsc -p tsconfig.json --noEmit and tsc -p tsconfig.build.json --noEmit pass. check.sh --drift-only says the covered code matches the sync record.
  • vitest run test/unit: 149 files, 2318 tests, all pass.
  • Integration and e2e, each file alone with --maxWorkers=1 --retry=0: 20 integration files (mouse, record-export, host-renderer-rpc, run, cli, empty-session-capture, wait-render, batch, slow-replay, backend-selection, event-log, screen-hash, ghostty-web-crash, screenshot-out, io-loop, artifact-manifest-race, screenshot-missing-glyphs, record-diff, renderer-backend, cross-backend-screen-hash) and all 10 test/e2e files. The offline screenshot paths are in ghostty-web-crash (4 tests), screenshot-out, empty-session-capture, record-export and renderer-backend. 299 tests passed in the full run, with 3 files red. lifecycle.test.ts has one test that reaches the renderer (failed session supports offline snapshot). It passes alone.
  • Tests that passed only on rerun, with the red assertion of the full run. The host load went above 200 during that run (other agents):
    • record-export exports animated svg with recorded timing: Hook timed out in 10000ms. Alone: 17 of 17.
    • slow-replay answers a run whose catch-up replay goes on for longer than the client waits: RPC request timed out after 13000ms and Hook timed out in 10000ms. Alone: 5 of 5.
    • backend-selection, 6 of 7 red with Test timed out in 5000ms. Alone at load 58 to 76 it failed 2 of 7 three times with the same 5 s timeouts. With --testTimeout=120000 it passed 7 of 7. Origin/main fails this file 3 of 7 the same way under load.
    • No test needed a code change.

New tests (they carry a KNOWN-BUG header with the upstream issue, so remove or invert them when ghostty-web has a fix):

  • test/integration/ghostty-web-crash.test.ts, 4 tests, Linux only, all on the 12-byte log through the real CLI. The two libghostty-vt tests skip when the native package is missing. On 3a5fad6c the test reports the crash behind a libghostty-vt screenshot of an exited session as REPLAY_ERROR is red (expected 1 to be 10). The test for the live variant is green there. It guards that the live path keeps exiting 10. On origin/main the first two tests are red (screenshot <id> --json: expected 9 to be 10).
  • test/unit/renderer/ghosttyWebCrash.test.ts, 10 tests, no browser. 5 messages that are not a crash must stay unchanged (ghostty-web bridge write() is unavailable, page.evaluate: Target page, context or browser has been closed, page.evaluate: Timeout 30000ms exceeded., page.evaluate: Error: crash reporter is unavailable and page.evaluate: Error: worker crashed).
  • test/unit/replay/offlineReplay.test.ts, 1 new test: a crash thrown by the callback becomes REPLAY_ERROR (retryable: false, message with the session id and the explanation), and the backend is disposed.
  • test/unit/host/renderer.test.ts, 2 tests: a crashed page becomes a REPLAY_ERROR that is not retryable, and every other error passes through.

Mutations (one change each to the committed tree, restored with git checkout). An identical-text control stays green.

Mutant Red test
offline callback crash is not mapped unit maps a ghostty-web crash inside the callback to REPLAY_ERROR and disposes the backend. At the CLI seam: integration reports the crash behind a libghostty-vt screenshot of an exited session as REPLAY_ERROR
offline callback maps every error unit propagates callback failures without wrapping and still disposes the backend
host mapper off (withBackend never calls it) unit reports a crashed ghostty-web page as a REPLAY_ERROR that is not retryable. At the CLI seam: integration names the crash when a live screenshot and a live ghostty-web snapshot fail and reports the crash behind a live libghostty-vt screenshot as REPLAY_ERROR
the mapper rethrows the error unchanged unit reports a crashed ghostty-web page as a REPLAY_ERROR that is not retryable
the mapper maps every Error unit passes every other renderer failure through unchanged
the mapped error is retryable: true unit reports a crashed ghostty-web page as a REPLAY_ERROR that is not retryable
the mapped error is retryable: true, at the CLI seam survives the integration test. The CLI derives retryable from the code, so the host flag never reaches it. The unit test above catches it
classifier pattern /Target/ unit returns the failure "page.evaluate: Target page, context or browser has been closed" as it is
classifier pattern /crash/i unit returns the failure "page.evaluate: Error: crash reporter is unavailable" as it is
classifier pattern /crashed/ unit returns the failure "page.evaluate: Error: worker crashed" as it is
classifier pattern also matches Target page, context or browser has been closed unit returns the failure "page.evaluate: Target page, context or browser has been closed" as it is
classifier pattern also matches Timeout unit returns the failure "page.evaluate: Timeout 30000ms exceeded." as it is
classifier pattern anchored (/^Target crashed/) unit turns a crashed page into a GhosttyWebCrashedError that keeps the cause, replayTo rejects with the crash error and replayWithTiming rejects with the crash error
classifier pattern /Target crashed/i survives. Playwright writes the message as Target crashed, so I do not pin the case

The offline-callback rows, the /crashed/ row and the live libghostty-vt test of the "host mapper off" row ran on 987518b2 (the last commit with code or test changes). The other host-mapper rows ran on a963cde5, and the host code is unchanged since then. The /Target/, /crash/i, closed-page, Timeout, anchored and /i rows ran on 3a5fad6c. The first three of them survived on a963cde5. The classifier code is unchanged since then. The mutants of the first commit (describeReplayFailure, the two wrappers and the two text sites) are red with its tests. I ran them on that commit (886f1c21) only.

Dogfood with the real CLI (node --import tsx src/cli/main.ts, own AGENT_TTY_HOME under /tmp, every session destroyed), on c4f8d7b7: a live session that prints the 12 bytes.

screenshot            exit=10 REPLAY_ERROR retryable=false "ghostty-web crashed while it replayed this session's output. One known cause is … (page.evaluate: Target crashed)"
screenshot (again)    exit=10 REPLAY_ERROR retryable=false (same)
snapshot --renderer ghostty-web  exit=10 (same)
screenshot --renderer libghostty-vt  exit=10 REPLAY_ERROR retryable=false (its fallback ghostty-web page crashes)
snapshot              exit=0  ok
record export webm    exit=7  EXPORT_ERROR "Failed to export session \"<id>\" as webm. ghostty-web crashed while it replayed …"
record export svg     exit=0  ok
exited session: snapshot/screenshot --renderer ghostty-web exit=10 REPLAY_ERROR "Failed to boot or replay renderer for session <id>. ghostty-web crashed …"
exited session: screenshot --renderer libghostty-vt  exit=10 REPLAY_ERROR "Failed to boot or replay renderer for session <id>. ghostty-web crashed …"   (3a5fad6c and origin/main: exit 1 INTERNAL_ERROR)
exited session: snapshot --renderer libghostty-vt  exit=0 ok (no ghostty-web page)
exited session: record export webm  exit=7 EXPORT_ERROR

wait --text o --renderer ghostty-web --timeout-ms 90000 on the live session exits 10 with REPLAY_ERROR. It took 5.9, 11.0, 11.3, 8.9 and 10.1 s in 5 runs on c4f8d7b7 at host load 38 to 61 (median 10.1 s). Each of the 10 failed polls boots a new Chromium, so the time depends on the load.

After each failing screenshot, the host has two children: the program and one headless Chromium with a crashed page (5 processes, about 313 MB RSS in total). The next render command replaces it.

Open PR #358 also changes src/renderer/ghosttyWeb/backend.ts. A trial merge of this tip with the current head of #358 (e98841cd, git merge-tree --write-tree) has no conflicts, and neither has a trial merge with origin/main (9c019706).


Generated with xum • Model: anthropic:claude-sonnet-5-5 • Thinking: xhigh

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-08T14:20:45.474648Z c4f8d7b Manual request
🔒 Security Review ✅ Completed 2026-11-08T14:21:46.623853Z c4f8d7b Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Bravo.

Reviewed commit: a963cde5e9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: a963cde5e9

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. More of your lovely PRs please.

Reviewed commit: 3a5fad6ca0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: 3a5fad6ca0

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Nice work!

Reviewed commit: c4f8d7b7a1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: c4f8d7b7a1

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@ThomasK33
ThomasK33 added this pull request to the merge queue Nov 8, 2026
Merged via the queue into main with commit bf1d284 Nov 8, 2026
28 checks passed
@ThomasK33
ThomasK33 deleted the fix/ghostty-web-crash-replay-error branch November 8, 2026 15:36
@github-actions github-actions Bot mentioned this pull request Nov 8, 2026
ThomasK33 added a commit that referenced this pull request Nov 9, 2026
#363 wraps replayWithTiming in an outer method that describes page crashes.
The outer method now passes onProgress on to the inner one.
ThomasK33 added a commit that referenced this pull request Nov 9, 2026
Main holds #358 as one squash commit. Its tree equals the e98841c tree plus
#363, and this branch already had both, so the merge keeps this branch's files.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant