From 025f41d8ef9e493f4cc16f8efb0a7861de141970 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 06:42:10 +0000 Subject: [PATCH 01/28] fix: let the webm replay deadline follow the recording length The timed replay ran in real time under a fixed 5-minute limit, so a recording of more than about 5 minutes failed at the limit, and the CLI hid the cause behind the generic export message. The deadline is now the planned duration of the replay plus a 5-minute margin. The plan is the recorded gaps that the timing mode keeps, plus the final frame hold. A replay past its deadline rejects with WebmReplayTimeoutError, and record export reports EXPORT_ERROR with the timeout and the planned duration in the message and in details. --- docs/USAGE.md | 2 + src/cli/commands/README.md | 2 +- src/cli/commands/record-export.ts | 16 +++ src/export/README.md | 6 +- src/export/webm.ts | 101 ++++++++++++++- test/unit/commands/record-export.test.ts | 52 +++++++- test/unit/export/webm.test.ts | 157 ++++++++++++++++++++++- 7 files changed, 326 insertions(+), 10 deletions(-) diff --git a/docs/USAGE.md b/docs/USAGE.md index e3d737c0..1fef60ed 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -294,6 +294,8 @@ agent-tty record export --format webm --json | jq -c '.result.missi WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. +The replay runs in real time, so a WebM export with recorded timing takes about as long as the recording. The export stops a replay that runs more than 5 minutes past its plan (the recorded gaps that the timing keeps, plus the hold on the last frame) and fails with `EXPORT_ERROR`. The message names the timeout and the planned duration, and `details` holds `replayTimeoutMs`, `plannedDurationMs` and `timingMode`. + SVG export renders styled grid frames from the event log through the native `libghostty-vt` backend with no browser and no ffmpeg, so it requires the optional `@coder/libghostty-vt-node` package (there is no `ghostty-web` fallback). The output is deterministic: exporting the same session twice produces byte-identical, diffable SVG. `--format svg` writes a still image of the final screen and ignores event timing, so it also works on a log whose clock stepped backward (`--animate` refuses such a log and names the event where the clock stepped back); add `--animate` for an animated SVG of de-duplicated frames replayed with recorded event-log timing (`--timing` is not supported with SVG). Animated SVG capture retains every distinct frame's styled grid in memory, so it is bounded by a total-cell budget (the V8 heap limit divided by 400 bytes, at most 20M cells or roughly 10,000 distinct 80x24 frames, adapting to terminal size); recordings that exceed it fail with a clear error suggesting a still SVG or WebM export instead. The SVG text is also bounded (the V8 heap limit divided by 8 bytes, at most 2^28 characters or about 268 MB of mostly ASCII text), so a dense, colorful animation that fits the cell budget can still fail with the same kind of error. diff --git a/src/cli/commands/README.md b/src/cli/commands/README.md index 92026ab8..d7213e3f 100644 --- a/src/cli/commands/README.md +++ b/src/cli/commands/README.md @@ -34,7 +34,7 @@ This directory holds one handler module per agent-tty CLI command. `src/cli/main - `wait` rejects a mix of legacy flags (`--exit`, `--idle-ms`) and render flags. `--text` and `--regex` are mutually exclusive. The default timeout is `DEFAULT_WAIT_TIMEOUT_MS` (30,000 ms), which the `main.ts` help text names. An `--idle-ms` at or above the effective timeout (explicit or default), or a `--screen-stable-ms` whose `screenStableMinimumMs` is not shorter than it, throws `INVALID_DURATION`. `--timeout 0` skips the check. [wait.ts:runWaitCommand] [wait.ts:assertWindowFitsTimeout] [main.ts:main] - A timed-out `wait` throws `WAIT_TIMEOUT` with the wait result attached, so its error envelope keeps `result`. Without `--json`, and only when the wait lasted its whole timeout (`timeoutMs > 0`, measured by the CLI), the message names the timeout and, for the default, how to change it. With `--json`, the message stays `Wait timed out before its condition was met.` [wait.ts:throwIfTimedOut] [wait.ts:timeoutMessage] - `snapshot` and `screenshot` try the live RPC only when the manifest status is `running`. They fall back to offline replay only on `HOST_UNREACHABLE` and rethrow other errors. [snapshot.ts:runSnapshotCommand] [screenshot.ts:runScreenshotCommand] -- `record export` accepts only the formats in `RecordExportFormatSchema` (`asciicast`, `webm`, `svg`). `--animate` needs `svg`, and `svg` refuses `--timing`. An event log over the size limit fails with `EXPORT_ERROR` in every format, a message that names the log size and the limit, and `eventLogBytes` and `maxEventLogBytes` in `details`. [record-export.ts:runRecordExportCommand] +- `record export` accepts only the formats in `RecordExportFormatSchema` (`asciicast`, `webm`, `svg`). `--animate` needs `svg`, and `svg` refuses `--timing`. An event log over the size limit fails with `EXPORT_ERROR` in every format, a message that names the log size and the limit, and `eventLogBytes` and `maxEventLogBytes` in `details`. A `webm` replay that passes its deadline fails with `EXPORT_ERROR`, a message that names the timeout and the planned duration, and `replayTimeoutMs`, `plannedDurationMs` and `timingMode` in `details`. [record-export.ts:runRecordExportCommand] - `batch` parses the whole plan with `parseBatchPlan` before it resolves the session target. [batch.ts:runBatchCommand] - `create` polls readiness up to `READINESS_MAX_ATTEMPTS` (50) times, `READINESS_POLL_INTERVAL_MS` (100 ms) apart. [create.ts:waitForReadyHost] - `doctor` reports the `node-runtime` check as failed when the Node version is outside the `engines.node` range of `package.json`. It keeps running the other checks, reports `ok: false` and sets `process.exitCode = 1`. [doctor.ts:runNodeRuntimeCheck] [doctor.ts:runDoctorCommand] diff --git a/src/cli/commands/record-export.ts b/src/cli/commands/record-export.ts index a8a205ab..9c0d8d32 100644 --- a/src/cli/commands/record-export.ts +++ b/src/cli/commands/record-export.ts @@ -13,6 +13,7 @@ import { generateAsciicast } from '../../export/asciicast.js'; import { renderGridFramesToSvg } from '../../export/svg.js'; import { generateWebmExport, + WebmReplayTimeoutError, type WebmExportResult, } from '../../export/webm.js'; import { captureGridFrames } from '../../replay/gridFrames.js'; @@ -596,6 +597,21 @@ export async function runRecordExportCommand( }); } + if (error instanceof WebmReplayTimeoutError) { + throw makeCliError(ERROR_CODES.EXPORT_ERROR, { + message: `Failed to export session "${options.sessionId}" as ${format}: the replay timed out after ${String(error.timeoutMs / 1000)} s. The recording plans ${String(error.plannedDurationMs / 1000)} s at ${error.timingMode} timing.`, + details: { + sessionId: options.sessionId, + format, + out: options.out, + replayTimeoutMs: error.timeoutMs, + plannedDurationMs: error.plannedDurationMs, + timingMode: error.timingMode, + }, + cause: error, + }); + } + throw makeCliError(ERROR_CODES.EXPORT_ERROR, { message: `Failed to export session "${options.sessionId}" as ${format}.`, details: { diff --git a/src/export/README.md b/src/export/README.md index 876ffdfa..89f7d789 100644 --- a/src/export/README.md +++ b/src/export/README.md @@ -10,9 +10,10 @@ This directory converts a session's event log into recording exports. `generateA - `AsciicastExport`: `contents`, `header`, `capturedAtSeq`, `durationMs`, `outputEventCount`, `resizeEventCount` and `markerCount` [asciicast.ts:AsciicastExport]. - `generateAsciicast(sessionId, manifest, events, toolVersion?)` returns an `AsciicastExport`. It runs synchronously [asciicast.ts:generateAsciicast]. - `WebmExportOptions`: `sessionId`, `sessionDir`, `manifest`, `events`, `outputPath`, and optional `targetSeq`, `profileName`, `timingMode` and `rendererName` [webm.ts:WebmExportOptions]. -- `WebmExportDeps`: optional `backendFactory` and `replayTimeoutMs`, for tests and callers that supply their own backend [webm.ts:WebmExportDeps]. +- `WebmExportDeps`: optional `backendFactory` and `replayTimeoutMs`, for tests and callers that supply their own backend. `replayTimeoutMs` replaces the replay deadline that is derived from the plan (see Invariants) [webm.ts:WebmExportDeps]. - `WebmExportResult`: `capturedAtSeq`, `durationMs`, `outputEventCount`, `resizeEventCount`, `cols`, `rows`, `profileName`, `timingMode` and `rendererBackend`, plus the optional `missingGlyphs` and `missingGlyphsTruncated` of `RecordedGlyphFields` [webm.ts:WebmExportResult]. - `generateWebmExport(options, deps?)` returns a `Promise` [webm.ts:generateWebmExport]. +- `WebmReplayTimeoutError` is the error that `generateWebmExport` rejects with when the timed replay passes its deadline. It carries `timeoutMs`, `plannedDurationMs` and `timingMode` [webm.ts:WebmReplayTimeoutError]. - `renderGridFramesToSvg({ profile, frames, animate, heapSizeLimit? })` returns SVG text from styled grid frames, which `captureGridFrames` in `src/replay/gridFrames.ts` builds. It runs synchronously and leaves the file write to the caller [svg.ts:renderGridFramesToSvg]. ## Invariants @@ -26,7 +27,8 @@ This directory converts a session's event log into recording exports. `generateA - A requested `libghostty-vt` renderer is replaced with `ghostty-web`. A code comment says `libghostty-vt` is semantic-only [webm.ts:generateWebmExport]. - The default backend factory accepts only `ghostty-web` [webm.ts:createDefaultBackend]. - Timing presets: `accelerated` caps gaps at 400 ms and holds frames at least 100 ms. `max-speed` holds frames at least 16 ms. Each mode holds the final frame (1000 ms, or 500 ms for `max-speed`) [webm.ts:buildReplayTimingOptions]. -- The timed replay fails after `REPLAY_TIMEOUT_MS` (5 minutes) [webm.ts:REPLAY_TIMEOUT_MS]. +- The timed replay fails with `WebmReplayTimeoutError` after its deadline: the planned duration plus `REPLAY_TIMEOUT_MARGIN_MS` (5 minutes), at most 2^31-1 ms (the longest delay that a timer runs). A long recording is not refused for its length [webm.ts:generateWebmExport, webm.ts:REPLAY_TIMEOUT_MARGIN_MS]. +- The planned duration is the schedule of the replay: the gaps between the events up to the target seq, as the timing mode keeps them (`recorded` keeps each gap, `accelerated` caps each at 400 ms, `max-speed` drops them), plus the hold on the final frame. It asserts that event timestamps do not decrease [webm.ts:plannedReplayDurationMs]. - `renderGridFramesToSvg` needs the profile font size `SVG_FONT_SIZE` (14) and draws fixed 8.4x18 px cells, so equal input gives byte-identical output [svg.ts:renderGridFramesToSvg]. It always embeds the JetBrains Mono subset, and embeds each other bundled face only when a glyph needs it [svg.ts:embeddedFontAssets]. Output past `defaultMaxSvgOutputLength` of the V8 heap limit (the heap limit divided by 8 bytes, at most `MAX_SVG_OUTPUT_LENGTH`) fails with `EXPORT_ERROR` [svg.ts:defaultMaxSvgOutputLength]. ## Side Effects diff --git a/src/export/webm.ts b/src/export/webm.ts index 326351d4..8767687f 100644 --- a/src/export/webm.ts +++ b/src/export/webm.ts @@ -25,7 +25,13 @@ import { resolveProfile } from '../renderer/profiles.js'; import type { RenderProfileConfig, ReplayState } from '../renderer/types.js'; import { invariant, unreachable } from '../util/assert.js'; -const REPLAY_TIMEOUT_MS = 5 * 60 * 1000; +// The replay may run this long past its plan. The margin covers bridge calls, +// painting, the glyph check and a loaded host. It equals the old fixed limit, +// so an export that fit before still fits. Boot and finalize run outside the +// replay deadline. +const REPLAY_TIMEOUT_MARGIN_MS = 5 * 60 * 1000; +// Node runs a timer with a longer delay after 1 ms. +const MAX_TIMER_DELAY_MS = 2 ** 31 - 1; // maxGapMs/minFrameHoldMs are tuned for watchability: shorter clamps make the // video flicker because every idle gap collapses to a near-instant cut. const ACCELERATED_TIMING: ReplayTimingOptions = Object.freeze({ @@ -50,6 +56,42 @@ function buildReplayTimingOptions(mode: ReplayTimingMode): ReplayTimingOptions { } } +function computeScheduledGapMs( + timing: ReplayTimingOptions, + recordedGapMs: number, +): number { + switch (timing.mode) { + case 'accelerated': + return Math.min(recordedGapMs, timing.maxGapMs); + case 'recorded': + return recordedGapMs; + case 'max-speed': + return 0; + default: + return unreachable(timing, 'unsupported replay timing mode'); + } +} + +export class WebmReplayTimeoutError extends Error { + public readonly timeoutMs: number; + public readonly plannedDurationMs: number; + public readonly timingMode: ReplayTimingMode; + + public constructor( + timeoutMs: number, + plannedDurationMs: number, + timingMode: ReplayTimingMode, + ) { + super( + `WebM replay timed out after ${String(timeoutMs)} ms (planned duration ${String(plannedDurationMs)} ms at ${timingMode} timing)`, + ); + this.name = 'WebmReplayTimeoutError'; + this.timeoutMs = timeoutMs; + this.plannedDurationMs = plannedDurationMs; + this.timingMode = timingMode; + } +} + export interface WebmExportOptions { sessionId: string; sessionDir: string; @@ -106,11 +148,46 @@ function parseTimestamp(timestamp: string, label: string): number { return parsedTimestamp; } +// The time that the replay waits on its schedule: the recorded gaps that the +// timing mode keeps, plus the hold on the final frame. It mirrors the schedule +// in GhosttyWebBackend.replayWithTiming. Frame holds run inside that schedule, +// so only the margin covers a replay that falls behind it. +function plannedReplayDurationMs( + replayInput: ReturnType, + timing: ReplayTimingOptions, +): number { + let scheduledMs = 0; + let previousTimestampMs: number | null = null; + + for (const event of replayInput.events) { + if (event.seq > replayInput.targetSeq) { + break; + } + + const timestampMs = parseTimestamp( + event.ts, + `replay event ${String(event.seq)} timestamp`, + ); + if (previousTimestampMs !== null) { + const recordedGapMs = timestampMs - previousTimestampMs; + invariant( + recordedGapMs >= 0, + 'replay event timestamps must be ordered non-decreasingly', + ); + scheduledMs += computeScheduledGapMs(timing, recordedGapMs); + } + previousTimestampMs = timestampMs; + } + + return scheduledMs + timing.finalFrameHoldMs; +} + async function replayWithTimeout( backend: VideoCapableRendererBackend, replayInput: ReturnType, timingOptions: ReplayTimingOptions, timeoutMs: number, + plannedDurationMs: number, ): Promise { let timeoutHandle: ReturnType | undefined; @@ -119,7 +196,13 @@ async function replayWithTimeout( backend.replayWithTiming(replayInput, timingOptions), new Promise((_, reject) => { timeoutHandle = setTimeout(() => { - reject(new Error('WebM replay timed out after 5 minutes')); + reject( + new WebmReplayTimeoutError( + timeoutMs, + plannedDurationMs, + timingOptions.mode, + ), + ); }, timeoutMs); }), ]); @@ -187,7 +270,18 @@ export async function generateWebmExport( requestedRendererName === 'libghostty-vt' ? 'ghostty-web' : requestedRendererName; - const replayTimeoutMs = deps?.replayTimeoutMs ?? REPLAY_TIMEOUT_MS; + // A long recording must not time out only because it is long: the + // deadline follows the plan, and the margin bounds a hung renderer. + const plannedDurationMs = plannedReplayDurationMs( + replayInput, + timingOptions, + ); + const replayTimeoutMs = + deps?.replayTimeoutMs ?? + Math.min( + plannedDurationMs + REPLAY_TIMEOUT_MARGIN_MS, + MAX_TIMER_DELAY_MS, + ); invariant( replayTimeoutMs > 0, @@ -207,6 +301,7 @@ export async function generateWebmExport( replayInput, timingOptions, replayTimeoutMs, + plannedDurationMs, ); const missingGlyphs = await backend.recordedMissingGlyphs(); await backend.finalizeVideo(options.outputPath); diff --git a/test/unit/commands/record-export.test.ts b/test/unit/commands/record-export.test.ts index 779703e9..9ef8d640 100644 --- a/test/unit/commands/record-export.test.ts +++ b/test/unit/commands/record-export.test.ts @@ -14,11 +14,13 @@ import process from 'node:process'; import type * as FsPromises from 'node:fs/promises'; +import type * as WebmModule from '../../../src/export/webm.js'; import type * as EventLogCodecModule from '../../../src/storage/eventLogCodec.js'; import type * as HomeModule from '../../../src/storage/home.js'; import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; +import { WebmReplayTimeoutError } from '../../../src/export/webm.js'; import { ERROR_CODES, makeCliError } from '../../../src/protocol/errors.js'; import { EventLogTooLargeError } from '../../../src/storage/eventLogCodec.js'; @@ -89,7 +91,8 @@ vi.mock('../../../src/util/packageMetadata.js', () => ({ loadPackageMetadata: mocks.loadPackageMetadata, })); -vi.mock('../../../src/export/webm.js', () => ({ +vi.mock('../../../src/export/webm.js', async (importOriginal) => ({ + ...(await importOriginal()), generateWebmExport: mocks.generateWebmExport, })); @@ -1659,6 +1662,53 @@ describe('record export command', () => { }, ); + it('names the timeout and the planned duration when the WebM replay times out', async () => { + mocks.recordingFilename.mockReturnValue('recording-1-webm.webm'); + mocks.generateWebmExport.mockRejectedValue( + new WebmReplayTimeoutError(661_000, 361_000, 'recorded'), + ); + + await expect( + runRecordExportCommand({ + context: TEST_CONTEXT, + json: true, + sessionId: 'session-01', + format: 'webm', + }), + ).rejects.toMatchObject({ + code: ERROR_CODES.EXPORT_ERROR, + message: + 'Failed to export session "session-01" as webm: the replay timed out after 661 s. The recording plans 361 s at recorded timing.', + details: { + sessionId: 'session-01', + format: 'webm', + replayTimeoutMs: 661_000, + plannedDurationMs: 361_000, + timingMode: 'recorded', + }, + }); + expect(mocks.generateWebmExport).toHaveBeenCalledTimes(1); + expect(mocks.emitSuccess).not.toHaveBeenCalled(); + }); + + it('keeps the generic message for other WebM export failures', async () => { + mocks.recordingFilename.mockReturnValue('recording-1-webm.webm'); + mocks.generateWebmExport.mockRejectedValue(new Error('replay failed')); + + await expect( + runRecordExportCommand({ + context: TEST_CONTEXT, + json: true, + sessionId: 'session-01', + format: 'webm', + }), + ).rejects.toMatchObject({ + code: ERROR_CODES.EXPORT_ERROR, + message: 'Failed to export session "session-01" as webm.', + }); + expect(mocks.generateWebmExport).toHaveBeenCalledTimes(1); + }); + it('keeps the generic message for other event log read failures', async () => { mocks.readEventLogRecords.mockRejectedValue(new Error('EIO')); diff --git a/test/unit/export/webm.test.ts b/test/unit/export/webm.test.ts index fb2e3a85..c5e00f7b 100644 --- a/test/unit/export/webm.test.ts +++ b/test/unit/export/webm.test.ts @@ -1,4 +1,4 @@ -import { beforeEach, describe, expect, it, vi } from 'vitest'; +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; import type { VideoCapableRendererBackend, @@ -23,7 +23,10 @@ vi.mock('../../../src/renderer/profiles.js', () => ({ resolveProfile: mocks.resolveProfile, })); -import { generateWebmExport } from '../../../src/export/webm.js'; +import { + generateWebmExport, + WebmReplayTimeoutError, +} from '../../../src/export/webm.js'; function createManifest(overrides: Partial = {}): SessionRecord { return { @@ -514,7 +517,9 @@ describe('generateWebmExport', () => { replayTimeoutMs: 1, }, ), - ).rejects.toThrow('WebM replay timed out after 5 minutes'); + ).rejects.toThrow( + 'WebM replay timed out after 1 ms (planned duration 2500 ms at recorded timing)', + ); expect(mockBackend.finalizeVideo).not.toHaveBeenCalled(); expect(mockBackend.dispose).toHaveBeenCalledTimes(1); @@ -544,3 +549,149 @@ describe('generateWebmExport', () => { expect(mockBackend.dispose).toHaveBeenCalledTimes(1); }); }); + +describe('generateWebmExport replay deadline', () => { + const FIVE_MINUTES_MS = 5 * 60 * 1000; + const BASE_TIME_MS = Date.parse('2026-03-19T12:00:00.000Z'); + + // Two output events, `spanMs` apart: the recording's whole length. + function useRecordingOfLength(spanMs: number): void { + mocks.buildReplayInput.mockReturnValue({ + sessionId: 'session-01', + initialCols: 80, + initialRows: 24, + events: [ + { + seq: 0, + type: 'output', + ts: new Date(BASE_TIME_MS).toISOString(), + payload: { data: 'start\n' }, + }, + { + seq: 1, + type: 'output', + ts: new Date(BASE_TIME_MS + spanMs).toISOString(), + payload: { data: 'end\n' }, + }, + ], + targetSeq: 1, + }); + } + + // Starts an export whose replay runs on fake timers. It takes `replayMs` of + // fake time, or never ends when `replayMs` is null. + async function startExport( + timingMode: 'recorded' | 'accelerated' | 'max-speed', + replayMs: number | null, + ) { + let markReplayStarted: () => void = () => undefined; + const replayStarted = new Promise((resolve) => { + markReplayStarted = resolve; + }); + const replayWithTiming = vi.fn( + (): Promise> => { + markReplayStarted(); + return new Promise((resolve) => { + if (replayMs !== null) { + setTimeout(() => { + resolve(createReplayState()); + }, replayMs); + } + }); + }, + ); + const mockBackend = createMockBackend({ replayWithTiming }); + let settled = false; + const outcome = generateWebmExport( + { + sessionId: 'session-01', + sessionDir: '/tmp/agent-tty/sessions/session-01', + manifest: createManifest(), + events: [], + outputPath: '/tmp/exports/recording-1-webm.webm', + timingMode, + }, + { backendFactory: () => mockBackend.backend }, + ).then( + (result) => { + settled = true; + return { result, error: undefined }; + }, + (error: unknown) => { + settled = true; + return { result: undefined, error }; + }, + ); + await replayStarted; + + return { outcome, mockBackend, isSettled: () => settled }; + } + + beforeEach(() => { + vi.useFakeTimers(); + }); + + afterEach(() => { + vi.useRealTimers(); + }); + + it('gives a recording of more than 5 minutes the time that its plan needs', async () => { + useRecordingOfLength(6 * 60 * 1000); + // The replay waits out the recorded gap, then holds the final frame. + const { outcome, mockBackend } = await startExport( + 'recorded', + 6 * 60 * 1000 + 1_000, + ); + + await vi.advanceTimersByTimeAsync(6 * 60 * 1000 + 1_000); + const { result, error } = await outcome; + + expect(error).toBeUndefined(); + expect(result?.durationMs).toBe(6 * 60 * 1000); + expect(mockBackend.finalizeVideo).toHaveBeenCalledTimes(1); + }); + + it('keeps the deadline within the longest delay that a timer runs', async () => { + useRecordingOfLength(30 * 24 * 60 * 60 * 1000); + const { outcome } = await startExport('recorded', 60_000); + + await vi.advanceTimersByTimeAsync(60_000); + const { error } = await outcome; + + expect(error).toBeUndefined(); + }); + + it.each([ + // The deadline is the planned duration plus 5 minutes. The plan is the + // recorded gap and the final frame hold, with the gap cut by the mode. + { timingMode: 'recorded', plannedDurationMs: 6 * 60 * 1000 + 1_000 }, + { timingMode: 'accelerated', plannedDurationMs: 400 + 1_000 }, + { timingMode: 'max-speed', plannedDurationMs: 500 }, + ] as const)( + 'stops a hung $timingMode replay once its plan and 5 minutes have passed', + async ({ timingMode, plannedDurationMs }) => { + useRecordingOfLength(6 * 60 * 1000); + const deadlineMs = plannedDurationMs + FIVE_MINUTES_MS; + const { outcome, mockBackend, isSettled } = await startExport( + timingMode, + null, + ); + + await vi.advanceTimersByTimeAsync(deadlineMs - 1); + expect(isSettled()).toBe(false); + + await vi.advanceTimersByTimeAsync(1); + const { error } = await outcome; + + expect(error).toBeInstanceOf(WebmReplayTimeoutError); + expect(error).toMatchObject({ + timeoutMs: deadlineMs, + plannedDurationMs, + timingMode, + message: `WebM replay timed out after ${String(deadlineMs)} ms (planned duration ${String(plannedDurationMs)} ms at ${timingMode} timing)`, + }); + expect(mockBackend.finalizeVideo).not.toHaveBeenCalled(); + expect(mockBackend.dispose).toHaveBeenCalledTimes(1); + }, + ); +}); From ff8f2d264c9e8f5b9a6a80bb04e2af3e96520b01 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 07:23:15 +0000 Subject: [PATCH 02/28] perf: wait for one frame per flush when replaying output in ghostty-web The harness awaited terminal.write(data, callback) for every output chunk. ghostty-web runs that callback one animation frame later, so a replay cost about 17-24 ms per event even though Node already sent up to 1000 chunks per page call. Add a harness writeBatch(chunks, waitForFrame) that writes all chunks at once and waits for one frame only when asked. The backend asks for the frame on the last page call of a flush. Screenshots and snapshots are byte-identical to before. In a WebM replay, the chunks of one flush now share one video frame. --- src/renderer/ghosttyWeb/README.md | 5 +- src/renderer/ghosttyWeb/backend.ts | 31 ++-- .../ghosttyWeb/embeddedHarnessHtml.ts | 33 +++++ test/integration/renderer-backend.test.ts | 73 ++++++++++ test/unit/renderer/ghosttyWebBackend.test.ts | 134 ++++++++++++++++++ 5 files changed, 264 insertions(+), 12 deletions(-) diff --git a/src/renderer/ghosttyWeb/README.md b/src/renderer/ghosttyWeb/README.md index 10ea8ac4..a48cc672 100644 --- a/src/renderer/ghosttyWeb/README.md +++ b/src/renderer/ghosttyWeb/README.md @@ -20,7 +20,7 @@ This directory holds the `ghostty-web` reference renderer. `GhosttyWebBackend` s - `stripTrailingAsciiSpaces(text)` and `assembleCanonicalLine(cols, readColumn)` build canonical line text [harnessDecoding.ts:assembleCanonicalLine]. - Assertion helpers `assertNonNegativeInteger`, `assertPositiveInteger`, `assertPositiveNumber` and `assertHexColor`, plus `normalizeError(error, prefix)` [harnessDecoding.ts:normalizeError]. - `loadHarnessHtml()` returns `EMBEDDED_HARNESS_HTML` [harnessDecoding.ts:loadHarnessHtml]. -- `EMBEDDED_HARNESS_HTML` is the harness page as a template string. In the browser it defines `window.__agentTty` with `write`, `getSnapshot`, `getCellGraphemes`, `getVisibleText`, `isReady`, `resize` and `setCursorVisible` [embeddedHarnessHtml.ts:EMBEDDED_HARNESS_HTML]. +- `EMBEDDED_HARNESS_HTML` is the harness page as a template string. In the browser it defines `window.__agentTty` with `write`, `writeBatch`, `getSnapshot`, `getCellGraphemes`, `getVisibleText`, `isReady`, `resize` and `setCursorVisible` [embeddedHarnessHtml.ts:EMBEDDED_HARNESS_HTML]. - `harness.html` is a standalone copy of the harness page. No code serves it, and it differs from `EMBEDDED_HARNESS_HTML` [harnessDecoding.ts:loadHarnessHtml]. - `assets/` (not documented separately): holds the bundled fonts `JetBrainsMono-Regular-latin.woff2`, `SymbolsNerdFontMono-Regular.ttf`, `NotoSansSymbols2-Regular.woff2` and `NotoSansMath-Regular.woff2`, plus `FONT-LICENSE.txt` and the OFL texts of the Noto fonts. `scripts/subset-symbol-fonts.sh` rebuilds the two Noto subsets. `src/renderer/bundledFont.ts` loads them, and `scripts/copy-renderer-assets.mjs` copies them into `dist/` during `npm run build`. @@ -30,6 +30,7 @@ This directory holds the `ghostty-web` reference renderer. `GhosttyWebBackend` s - Replay cannot rewind: `targetSeq` must be >= the last applied seq. The initial dimensions cannot change after the first replay. `input.sessionId` must match the backend [backend.ts:GhosttyWebBackend.replayTo]. - `replayWithTiming` asserts strictly increasing seqs and non-decreasing timestamps [backend.ts:GhosttyWebBackend.replayWithTiming]. - `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` flushes once per run of output events [backend.ts:GhosttyWebBackend.replayWithTiming]. +- `writeBatch(chunks, waitForFrame)` writes every chunk of a call at once. ghostty-web calls a write callback one animation frame (about 17 ms) after the write, and waiting on every chunk (`write`) cost one frame per output event. With `waitForFrame` the call resolves after one frame, and the backend sets it only on the last call of a flush, so a flush of any size waits for one frame. In `replayWithTiming`, the chunks of one flush share one video frame [embeddedHarnessHtml.ts:EMBEDDED_HARNESS_HTML]. - `snapshot()` and `screenshot()` throw until `replayTo()` or `replayWithTiming()` has run once and set the initial size. Both replays accept `targetSeq` -1 for an empty Event Log. A replay over zero events sets `lastAppliedSeq` to `targetSeq` [backend.ts:GhosttyWebBackend.snapshot]. - `missingGlyphs` lists a code point only when a bundled face that lacks it drew it. Chromium draws a cluster with one font: the first stack font that covers all of it, or else the first font of the stack, which is bundled in both built-in profiles and then draws `.notdef` boxes. CDP names the face that drew a probe by its PostScript name, so a family alias or several faces under one family resolve to the right cmap. When one bundled face drew the whole grapheme, every code point that face lacks is listed. A probe with no reported font is judged against every face of the first family. When the grapheme is split over several fonts, each uncovered code point is judged by its own probe, and a mark's probe on its base is compared with a probe of the base alone. Glyphs that a host font draws are not listed. The check runs after the PNG is written, removes its probe spans, and caches each probe text's result for the page, clearing the cache when it would pass `MAX_CACHED_GLYPH_PROBES` (16384) entries. It probes at most `MAX_GLYPH_PROBES` texts and lists at most `MAX_MISSING_GLYPH_CELLS` cells (4096 each). `probeUsedFonts` sends its CDP queries in batches of 256 and stops starting batches after `GLYPH_PROBE_BUDGET_MS` (1000 ms). The result sets `missingGlyphsTruncated` when any of these stops the check early. Family names match without regard to case. A web font whose PostScript name the profile does not bundle fails the check. A failed check is logged at debug level and leaves the field out, so the screenshot still succeeds. `replayWithTiming()` records the viewport graphemes in the page after each output batch and each resize (at most `MAX_CACHED_GLYPH_PROBES`), and the final frame hold judges them once, with the hold as the probe budget, so the check does not lengthen the video. `recordedMissingGlyphs()` returns that result. A failed note or check leaves the record export fields out. A profile whose font stack does not start with a bundled font gets no `missingGlyphs` field [missingGlyphs.ts:probeUsedFonts] [backend.ts:GhosttyWebBackend.screenshot]. - `screenshot()` requires an absolute `outputPath` whose parent directory exists. It checks that the output is a non-empty PNG with a valid signature and an `IHDR` first chunk [backend.ts:GhosttyWebBackend.screenshot]. @@ -49,7 +50,7 @@ This directory holds the `ghostty-web` reference renderer. `GhosttyWebBackend` s - Reads the `ghostty-web` package files (`ghostty-web.js`, its browser-external helper and `ghostty-vt.wasm`) with `fs/promises` once per process [backend.ts:getServedAssets]. - Starts a `node:http` server on `127.0.0.1` with an ephemeral port. It serves the harness, the `ghostty-web` assets and the bundled fonts for `GET` and `HEAD` [backend.ts:GhosttyWebBackend.startServer]. - Calls `ensurePlaywrightBrowsersPath()`, which can set `process.env.PLAYWRIGHT_BROWSERS_PATH`. Then it launches headless Chromium with `chromium.launch` [backend.ts:GhosttyWebBackend.bootInternal]. -- Calls `page.evaluate` on `window.__agentTty` to write output, resize and read snapshots [backend.ts:GhosttyWebBackend.writeBatchBridge]. +- Calls `page.evaluate` on `window.__agentTty` to write output, resize and read snapshots. One `writeBatch` call carries up to 1000 output chunks [backend.ts:GhosttyWebBackend.writeBatchBridge]. - `screenshot()` writes the PNG through Playwright `locator('#terminal').screenshot`, then reads it back to hash it [backend.ts:GhosttyWebBackend.screenshot]. - With `videoOptions`, Playwright records into `outputDir`. `finalizeVideo()` closes the page and context and calls `video.saveAs(outputPath)` [backend.ts:GhosttyWebBackend.finalizeVideo]. - `dispose()` closes the page, context, browser and HTTP server [backend.ts:GhosttyWebBackend.dispose]. diff --git a/src/renderer/ghosttyWeb/backend.ts b/src/renderer/ghosttyWeb/backend.ts index 0aecf144..e872106d 100644 --- a/src/renderer/ghosttyWeb/backend.ts +++ b/src/renderer/ghosttyWeb/backend.ts @@ -81,6 +81,10 @@ interface GhosttyServedAsset extends GhosttyRequestAsset { interface GhosttyBrowserBridge { isReady?: () => boolean; write?: (data: string) => Promise | void; + writeBatch?: ( + chunks: string[], + waitForFrame?: boolean, + ) => Promise | void; resize?: (cols: number, rows: number) => Promise | void; setCursorVisible?: (visible: boolean) => Promise | void; getSnapshot?: (options?: SnapshotOptions) => GhosttyHarnessSnapshot; @@ -1692,7 +1696,12 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { batch.length <= MAX_REPLAY_BATCH_SIZE, 'flushOutputBatch batch size must respect MAX_REPLAY_BATCH_SIZE', ); - await this.writeBatchBridge(page, batch); + // The page waits for a frame only after the last batch of the flush. + await this.writeBatchBridge( + page, + batch, + batchStart + MAX_REPLAY_BATCH_SIZE >= dataChunks.length, + ); await afterEachBatch?.(); } } @@ -1777,6 +1786,7 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { private async writeBatchBridge( page: Page, dataChunks: string[], + waitForFrame = true, ): Promise { invariant( dataChunks.length > 0, @@ -1790,16 +1800,17 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { assertString(chunk, 'bridge batch write chunk must be a string'); } - await page.evaluate(async (chunks: string[]) => { - const bridge = (globalThis as GhosttyBrowserGlobal).__agentTty; - if (bridge === undefined || typeof bridge.write !== 'function') { - throw new Error('ghostty-web bridge write() is unavailable'); - } + await page.evaluate( + async ([chunks, shouldWaitForFrame]: readonly [string[], boolean]) => { + const bridge = (globalThis as GhosttyBrowserGlobal).__agentTty; + if (bridge === undefined || typeof bridge.writeBatch !== 'function') { + throw new Error('ghostty-web bridge writeBatch() is unavailable'); + } - for (const chunk of chunks) { - await bridge.write(chunk); - } - }, dataChunks); + await bridge.writeBatch(chunks, shouldWaitForFrame); + }, + [dataChunks, waitForFrame] as const, + ); } private async writeBridge(page: Page, data: string): Promise { diff --git a/src/renderer/ghosttyWeb/embeddedHarnessHtml.ts b/src/renderer/ghosttyWeb/embeddedHarnessHtml.ts index 6ed36c1b..adb0c9e5 100644 --- a/src/renderer/ghosttyWeb/embeddedHarnessHtml.ts +++ b/src/renderer/ghosttyWeb/embeddedHarnessHtml.ts @@ -651,6 +651,39 @@ export const EMBEDDED_HARNESS_HTML = ` }); updateDocumentState(); }, + async writeBatch(chunks, waitForFrame = true) { + const terminal = getReadyTerminal(); + invariant( + Array.isArray(chunks) && chunks.length > 0, + 'writeBatch() chunks must be a non-empty array', + ); + invariant( + typeof waitForFrame === 'boolean', + 'writeBatch() waitForFrame must be a boolean', + ); + for (const chunk of chunks) { + assertStringValue(chunk, 'writeBatch() chunks must be strings'); + } + + // The wasm write is synchronous, but ghostty-web calls a write + // callback one animation frame later. Waiting on every chunk made a + // replay cost one frame per event. The callback on the last chunk + // keeps the contract of write(): the call resolves after a frame. + // A caller that writes a run of batches asks for that frame only + // after the last one. + const lastIndex = chunks.length - 1; + for (let index = 0; index < lastIndex; index += 1) { + terminal.write(chunks[index]); + } + if (waitForFrame) { + await new Promise((resolve) => { + terminal.write(chunks[lastIndex], resolve); + }); + } else { + terminal.write(chunks[lastIndex]); + } + updateDocumentState(); + }, getSnapshot(options) { return getSnapshotPayload(options); }, diff --git a/test/integration/renderer-backend.test.ts b/test/integration/renderer-backend.test.ts index 5d357bf9..471b2191 100644 --- a/test/integration/renderer-backend.test.ts +++ b/test/integration/renderer-backend.test.ts @@ -755,6 +755,79 @@ describe('GhosttyWebBackend integration', { timeout: 120_000 }, () => { } }); + // ghostty-web calls a write callback one animation frame (about 17 ms) + // after the write. A bridge call that waits on every chunk made a replay cost + // one frame per output event: 90 minutes for 250 000 events. + it.each([ + [ + 'replayTo', + (renderer: GhosttyWebBackend, input: ReplayInput) => + renderer.replayTo(input), + ], + [ + 'replayWithTiming', + (renderer: GhosttyWebBackend, input: ReplayInput) => + renderer.replayWithTiming(input, { + mode: 'max-speed', + minFrameHoldMs: 0, + finalFrameHoldMs: 0, + }), + ], + ])( + '%s waits for one animation frame per flush of output events, not per event', + async (_name, replay) => { + await backend.boot(); + const { page } = backend as unknown as { page: Page }; + // ghostty-web calls a write callback one frame after the write, so the + // number of writes that pass a callback is the number of frames that a + // replay waits for. The string keeps vitest from rewriting the import. + await page.evaluate(`(async () => { + const { Terminal } = await import('/assets/ghostty-web.js'); + const write = Terminal.prototype.write; + globalThis.__writeFrameWaits = 0; + Terminal.prototype.write = function (data, callback) { + if (callback !== undefined) { + globalThis.__writeFrameWaits += 1; + } + return write.call(this, data, callback); + }; + })()`); + + // One character per event, so a chunk that is lost or moved changes + // the text on the screen. 2501 characters fill 32 rows of 80 columns. + const characters = Array.from({ length: 2_501 }, (_, seq) => + 'abcdefghijklmnopqrstuvwxyz0123456789'.charAt((seq * 7) % 36), + ); + const input = createReplayInput( + characters.map((character, seq) => ({ + seq, + ts: timestampFor(0), + type: 'output' as const, + payload: { data: character }, + })), + { initialRows: 40 }, + ); + await replay(backend, input); + + // A bridge call holds at most 1000 chunks, so 2501 events take 3 calls. + // Only the last call waits for a frame. + expect( + await page.evaluate( + () => + (globalThis as unknown as { __writeFrameWaits: number }) + .__writeFrameWaits, + ), + ).toBe(1); + const expectedRows = Array.from({ length: 32 }, (_, row) => + characters.slice(row * 80, (row + 1) * 80).join(''), + ); + const screenRows = (await backend.snapshot()).visibleLines.map( + (line) => line.text, + ); + expect(screenRows.slice(0, 32)).toEqual(expectedRows); + }, + ); + it('allows dispose to be called more than once', async () => { await backend.boot(); diff --git a/test/unit/renderer/ghosttyWebBackend.test.ts b/test/unit/renderer/ghosttyWebBackend.test.ts index 723b321f..86a90e8a 100644 --- a/test/unit/renderer/ghosttyWebBackend.test.ts +++ b/test/unit/renderer/ghosttyWebBackend.test.ts @@ -426,6 +426,140 @@ describe('GhosttyWebBackend unit guards', () => { expect(evaluate).not.toHaveBeenCalled(); }); + // The page waits one animation frame (about 17 ms) after each bridge write. + // A bridge call that writes its chunks one by one makes a replay cost one + // frame per output event, so these tests run the page function of every + // call against a counting bridge. + async function withCountingBridge( + run: (context: { + page: { isClosed: () => boolean; evaluate: ReturnType }; + write: ReturnType; + writeBatchSizes: number[]; + frameWaits: boolean[]; + writtenChunks: string[]; + }) => Promise, + ): Promise { + const writeBatchSizes: number[] = []; + const frameWaits: boolean[] = []; + const writtenChunks: string[] = []; + const write = vi.fn(() => Promise.resolve()); + vi.stubGlobal('__agentTty', { + write, + writeBatch: (chunks: string[], waitForFrame: boolean) => { + writeBatchSizes.push(chunks.length); + frameWaits.push(waitForFrame); + writtenChunks.push(...chunks); + return Promise.resolve(); + }, + }); + const page = { + isClosed: () => false, + evaluate: vi.fn((pageFunction: (arg: unknown) => unknown, arg: unknown) => + Promise.resolve(pageFunction(arg)), + ), + }; + + try { + return await run({ + page, + write, + writeBatchSizes, + frameWaits, + writtenChunks, + }); + } finally { + vi.unstubAllGlobals(); + } + } + + function outputReplayInput(eventCount: number): ReplayInput { + return { + sessionId: 'renderer-unit-session', + initialCols: 80, + initialRows: 24, + targetSeq: eventCount - 1, + events: Array.from({ length: eventCount }, (_, seq) => ({ + seq, + ts: '2026-03-19T12:00:00.000Z', + type: 'output' as const, + payload: { data: `chunk-${String(seq)}` }, + })), + }; + } + + it('writes a bridge batch into the page in one call that waits for a frame', async () => { + const backend = createBackend(); + const chunks = Array.from({ length: 1000 }, (_, i) => `chunk-${String(i)}`); + + await withCountingBridge( + async ({ page, write, writeBatchSizes, frameWaits, writtenChunks }) => { + await ( + backend as unknown as { + writeBatchBridge: ( + page: object, + dataChunks: string[], + ) => Promise; + } + ).writeBatchBridge(page, chunks); + + expect(page.evaluate).toHaveBeenCalledTimes(1); + expect(writeBatchSizes).toEqual([1000]); + expect(frameWaits).toEqual([true]); + expect(writtenChunks).toEqual(chunks); + expect(write).not.toHaveBeenCalled(); + }, + ); + }); + + it.each([ + [ + 'replayTo', + (backend: GhosttyWebBackend, input: ReplayInput) => + backend.replayTo(input), + ], + [ + 'replayWithTiming', + (backend: GhosttyWebBackend, input: ReplayInput) => + backend.replayWithTiming(input, { + mode: 'max-speed', + minFrameHoldMs: 0, + finalFrameHoldMs: 0, + }), + ], + ])( + '%s calls the page once per 1000 output events, and waits for a frame after the last call', + async (_name, replay) => { + const backend = createBackend(); + const input = outputReplayInput(2_501); + + await withCountingBridge( + async ({ page, write, writeBatchSizes, frameWaits, writtenChunks }) => { + Object.assign(backend as object, { + isBooted: true, + page, + resizeBridge: () => Promise.resolve(), + notePaintedGraphemes: () => Promise.resolve(), + checkPaintedGlyphs: () => Promise.resolve({}), + readHarnessSnapshot: () => + Promise.resolve(createHarnessSnapshotPayload({ cols: 80 })), + }); + + await replay(backend, input); + + expect(page.evaluate).toHaveBeenCalledTimes(3); + expect(writeBatchSizes).toEqual([1000, 1000, 501]); + expect(frameWaits).toEqual([false, false, true]); + expect(writtenChunks).toEqual( + input.events.map((event) => + event.type === 'output' ? event.payload.data : '', + ), + ); + expect(write).not.toHaveBeenCalled(); + }, + ); + }, + ); + it('includes structured cell data only when snapshot includeCells is requested', async () => { const backend = createBackend(); const evaluate = vi.fn().mockResolvedValue( From 8d586e83a3a03898bbb6d9151b5166c6aa4e76db Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 07:47:28 +0000 Subject: [PATCH 03/28] fix: stop the webm replay on a stall instead of a deadline Replace the planned-duration deadline with an inactivity watchdog. The replay reports progress and the wait that it starts on the timing schedule. The export fails only when the renderer makes no progress for 5 minutes, and names the last event it reached. --- src/cli/commands/record-export.ts | 14 +- src/export/webm.ts | 175 +++---- src/renderer/backend.ts | 18 +- src/renderer/ghosttyWeb/backend.ts | 21 +- test/unit/commands/record-export.test.ts | 69 +-- test/unit/export/webm.test.ts | 475 ++++++++++++++++--- test/unit/renderer/ghosttyWebBackend.test.ts | 127 ++++- 7 files changed, 683 insertions(+), 216 deletions(-) diff --git a/src/cli/commands/record-export.ts b/src/cli/commands/record-export.ts index 9c0d8d32..45553520 100644 --- a/src/cli/commands/record-export.ts +++ b/src/cli/commands/record-export.ts @@ -13,7 +13,7 @@ import { generateAsciicast } from '../../export/asciicast.js'; import { renderGridFramesToSvg } from '../../export/svg.js'; import { generateWebmExport, - WebmReplayTimeoutError, + WebmReplayStallError, type WebmExportResult, } from '../../export/webm.js'; import { captureGridFrames } from '../../replay/gridFrames.js'; @@ -597,15 +597,19 @@ export async function runRecordExportCommand( }); } - if (error instanceof WebmReplayTimeoutError) { + if (error instanceof WebmReplayStallError) { + const position = + error.lastSeq >= 0 + ? `after event ${String(error.lastSeq)}` + : 'at the start of the replay'; throw makeCliError(ERROR_CODES.EXPORT_ERROR, { - message: `Failed to export session "${options.sessionId}" as ${format}: the replay timed out after ${String(error.timeoutMs / 1000)} s. The recording plans ${String(error.plannedDurationMs / 1000)} s at ${error.timingMode} timing.`, + message: `Failed to export session "${options.sessionId}" as ${format}: the renderer made no progress for ${String(error.stallWindowMs / 1000)} s ${position} (${error.timingMode} timing).`, details: { sessionId: options.sessionId, format, out: options.out, - replayTimeoutMs: error.timeoutMs, - plannedDurationMs: error.plannedDurationMs, + stallWindowMs: error.stallWindowMs, + lastSeq: error.lastSeq, timingMode: error.timingMode, }, cause: error, diff --git a/src/export/webm.ts b/src/export/webm.ts index 8767687f..a5ae918a 100644 --- a/src/export/webm.ts +++ b/src/export/webm.ts @@ -11,6 +11,7 @@ import { } from '../protocol/schemas.js'; import type { RecordedGlyphFields, + ReplayProgress, ReplayTimingOptions, VideoCapableRendererBackend, VideoRecordingOptions, @@ -25,11 +26,11 @@ import { resolveProfile } from '../renderer/profiles.js'; import type { RenderProfileConfig, ReplayState } from '../renderer/types.js'; import { invariant, unreachable } from '../util/assert.js'; -// The replay may run this long past its plan. The margin covers bridge calls, -// painting, the glyph check and a loaded host. It equals the old fixed limit, -// so an export that fit before still fits. Boot and finalize run outside the -// replay deadline. -const REPLAY_TIMEOUT_MARGIN_MS = 5 * 60 * 1000; +// The replay fails when the renderer makes no progress for this long. One +// bridge call writes at most 1000 output chunks, which takes seconds. The +// window is the old fixed limit on the whole replay, so an export that fit +// before still fits. Boot and finalize run outside the watchdog. +const REPLAY_STALL_WINDOW_MS = 5 * 60 * 1000; // Node runs a timer with a longer delay after 1 ms. const MAX_TIMER_DELAY_MS = 2 ** 31 - 1; // maxGapMs/minFrameHoldMs are tuned for watchability: shorter clamps make the @@ -56,38 +57,24 @@ function buildReplayTimingOptions(mode: ReplayTimingMode): ReplayTimingOptions { } } -function computeScheduledGapMs( - timing: ReplayTimingOptions, - recordedGapMs: number, -): number { - switch (timing.mode) { - case 'accelerated': - return Math.min(recordedGapMs, timing.maxGapMs); - case 'recorded': - return recordedGapMs; - case 'max-speed': - return 0; - default: - return unreachable(timing, 'unsupported replay timing mode'); - } -} - -export class WebmReplayTimeoutError extends Error { - public readonly timeoutMs: number; - public readonly plannedDurationMs: number; +export class WebmReplayStallError extends Error { + public readonly stallWindowMs: number; + // Seq of the last event that the replay reached, or -1 when it had reached + // none before it stalled. + public readonly lastSeq: number; public readonly timingMode: ReplayTimingMode; public constructor( - timeoutMs: number, - plannedDurationMs: number, + stallWindowMs: number, + lastSeq: number, timingMode: ReplayTimingMode, ) { super( - `WebM replay timed out after ${String(timeoutMs)} ms (planned duration ${String(plannedDurationMs)} ms at ${timingMode} timing)`, + `WebM replay stalled: the renderer made no progress for ${String(stallWindowMs)} ms (last event seq ${String(lastSeq)}, ${timingMode} timing)`, ); - this.name = 'WebmReplayTimeoutError'; - this.timeoutMs = timeoutMs; - this.plannedDurationMs = plannedDurationMs; + this.name = 'WebmReplayStallError'; + this.stallWindowMs = stallWindowMs; + this.lastSeq = lastSeq; this.timingMode = timingMode; } } @@ -111,7 +98,7 @@ export interface WebmExportDeps { profile: RenderProfileConfig, videoOptions: VideoRecordingOptions, ) => VideoCapableRendererBackend; - replayTimeoutMs?: number; + stallWindowMs?: number; } export interface WebmExportResult extends RecordedGlyphFields { @@ -148,68 +135,66 @@ function parseTimestamp(timestamp: string, label: string): number { return parsedTimestamp; } -// The time that the replay waits on its schedule: the recorded gaps that the -// timing mode keeps, plus the hold on the final frame. It mirrors the schedule -// in GhosttyWebBackend.replayWithTiming. Frame holds run inside that schedule, -// so only the margin covers a replay that falls behind it. -function plannedReplayDurationMs( - replayInput: ReturnType, - timing: ReplayTimingOptions, -): number { - let scheduledMs = 0; - let previousTimestampMs: number | null = null; - - for (const event of replayInput.events) { - if (event.seq > replayInput.targetSeq) { - break; - } - - const timestampMs = parseTimestamp( - event.ts, - `replay event ${String(event.seq)} timestamp`, - ); - if (previousTimestampMs !== null) { - const recordedGapMs = timestampMs - previousTimestampMs; - invariant( - recordedGapMs >= 0, - 'replay event timestamps must be ordered non-decreasingly', - ); - scheduledMs += computeScheduledGapMs(timing, recordedGapMs); - } - previousTimestampMs = timestampMs; - } - - return scheduledMs + timing.finalFrameHoldMs; -} - -async function replayWithTimeout( +// Fails the replay when the renderer makes no progress for `stallWindowMs`. +// The replay reports progress (see ReplayProgress) and each report restarts the +// window. A wait on the timing schedule is not a stall, so the window starts +// after that wait. A timer runs at most MAX_TIMER_DELAY_MS, so a longer span +// is a chain of timers. +async function replayWithStallWatchdog( backend: VideoCapableRendererBackend, replayInput: ReturnType, timingOptions: ReplayTimingOptions, - timeoutMs: number, - plannedDurationMs: number, + stallWindowMs: number, ): Promise { - let timeoutHandle: ReturnType | undefined; + let timer: ReturnType | undefined; + let lastSeq = -1; + let done = false; + let rejectStall: (error: WebmReplayStallError) => void = () => undefined; + const stalled = new Promise((_, reject) => { + rejectStall = reject; + }); + const arm = (remainingMs: number): void => { + const delayMs = Math.min(remainingMs, MAX_TIMER_DELAY_MS); + timer = setTimeout(() => { + if (remainingMs > delayMs) { + arm(remainingMs - delayMs); + return; + } + done = true; + rejectStall( + new WebmReplayStallError(stallWindowMs, lastSeq, timingOptions.mode), + ); + }, delayMs); + }; + const onProgress = (progress: ReplayProgress): void => { + // A replay that outlives its failure must not start a timer again. + if (done) { + return; + } + invariant( + Number.isInteger(progress.seq) && progress.seq >= -1, + 'replay progress seq must be an integer of at least -1', + ); + invariant( + Number.isFinite(progress.scheduledWaitMs) && + progress.scheduledWaitMs >= 0, + 'replay progress scheduledWaitMs must be a non-negative number', + ); + clearTimeout(timer); + lastSeq = progress.seq; + arm(progress.scheduledWaitMs + stallWindowMs); + }; + + arm(stallWindowMs); try { return await Promise.race([ - backend.replayWithTiming(replayInput, timingOptions), - new Promise((_, reject) => { - timeoutHandle = setTimeout(() => { - reject( - new WebmReplayTimeoutError( - timeoutMs, - plannedDurationMs, - timingOptions.mode, - ), - ); - }, timeoutMs); - }), + backend.replayWithTiming(replayInput, timingOptions, onProgress), + stalled, ]); } finally { - if (timeoutHandle !== undefined) { - clearTimeout(timeoutHandle); - } + done = true; + clearTimeout(timer); } } @@ -270,22 +255,11 @@ export async function generateWebmExport( requestedRendererName === 'libghostty-vt' ? 'ghostty-web' : requestedRendererName; - // A long recording must not time out only because it is long: the - // deadline follows the plan, and the margin bounds a hung renderer. - const plannedDurationMs = plannedReplayDurationMs( - replayInput, - timingOptions, - ); - const replayTimeoutMs = - deps?.replayTimeoutMs ?? - Math.min( - plannedDurationMs + REPLAY_TIMEOUT_MARGIN_MS, - MAX_TIMER_DELAY_MS, - ); + const stallWindowMs = deps?.stallWindowMs ?? REPLAY_STALL_WINDOW_MS; invariant( - replayTimeoutMs > 0, - 'replayTimeoutMs must be a positive number when provided', + Number.isFinite(stallWindowMs) && stallWindowMs > 0, + 'stallWindowMs must be a positive number when provided', ); backend = backendFactory( videoRendererName, @@ -296,12 +270,11 @@ export async function generateWebmExport( await backend.boot(); - const replayState = await replayWithTimeout( + const replayState = await replayWithStallWatchdog( backend, replayInput, timingOptions, - replayTimeoutMs, - plannedDurationMs, + stallWindowMs, ); const missingGlyphs = await backend.recordedMissingGlyphs(); await backend.finalizeVideo(options.outputPath); diff --git a/src/renderer/backend.ts b/src/renderer/backend.ts index b5cc08b5..b0ead18b 100644 --- a/src/renderer/backend.ts +++ b/src/renderer/backend.ts @@ -102,11 +102,27 @@ export type ReplayTimingOptions = | RecordedTimingOptions | MaxSpeedTimingOptions; +/** One report that a timed replay makes to its caller. */ +export interface ReplayProgress { + /** Seq of the last event that the replay reached, or -1 before the first. */ + seq: number; + /** + * How long the replay waits on its timing schedule right after this report. + * A caller that watches for stalls must not count that wait. + */ + scheduledWaitMs: number; +} + export interface VideoCapableRendererBackend extends RendererBackend { - /** Replay events with controlled timing for video capture. */ + /** + * Replay events with controlled timing for video capture. `onProgress` + * runs before each wait on the timing schedule and after each batch of + * output that the replay writes to the renderer. + */ replayWithTiming( input: ReplayInput, timing: ReplayTimingOptions, + onProgress?: (progress: ReplayProgress) => void, ): Promise; /** Finalize and save the video recording to the given path. */ diff --git a/src/renderer/ghosttyWeb/backend.ts b/src/renderer/ghosttyWeb/backend.ts index 0aecf144..b50a03b3 100644 --- a/src/renderer/ghosttyWeb/backend.ts +++ b/src/renderer/ghosttyWeb/backend.ts @@ -30,6 +30,7 @@ import { } from '../../util/resourceScope.js'; import type { RecordedGlyphFields, + ReplayProgress, ReplayTimingOptions, ScreenshotOptions, SnapshotOptions, @@ -508,6 +509,7 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { public async replayWithTiming( input: ReplayInput, timing: ReplayTimingOptions, + onProgress?: (progress: ReplayProgress) => void, ): Promise { const page = this.requireOperationalPage('replayWithTiming()'); @@ -664,9 +666,10 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { return; } - await this.flushOutputBatch(page, pendingOutputChunks, () => - this.notePaintedGraphemes(page), - ); + await this.flushOutputBatch(page, pendingOutputChunks, async () => { + await this.notePaintedGraphemes(page); + onProgress?.({ seq: highestProcessedSeq, scheduledWaitMs: 0 }); + }); pendingOutputChunks = []; pendingSinceTimestampMs = null; await waitForDelay(resolvedMinFrameHoldMs); @@ -719,14 +722,14 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { } // Waiting for the event's place on the schedule lets frame holds and // bridge calls use up time that the recording spent idle. - await waitForDelay( - Math.max( - 0, - Math.round( - scheduledElapsedMs - (performance.now() - scheduleStartedAtMs), - ), + const scheduledWaitMs = Math.max( + 0, + Math.round( + scheduledElapsedMs - (performance.now() - scheduleStartedAtMs), ), ); + onProgress?.({ seq: highestProcessedSeq, scheduledWaitMs }); + await waitForDelay(scheduledWaitMs); } previousProcessedEventTimestampMs = eventTimestampMs; diff --git a/test/unit/commands/record-export.test.ts b/test/unit/commands/record-export.test.ts index 9ef8d640..75b910ed 100644 --- a/test/unit/commands/record-export.test.ts +++ b/test/unit/commands/record-export.test.ts @@ -20,7 +20,7 @@ import type * as HomeModule from '../../../src/storage/home.js'; import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; -import { WebmReplayTimeoutError } from '../../../src/export/webm.js'; +import { WebmReplayStallError } from '../../../src/export/webm.js'; import { ERROR_CODES, makeCliError } from '../../../src/protocol/errors.js'; import { EventLogTooLargeError } from '../../../src/storage/eventLogCodec.js'; @@ -1662,34 +1662,47 @@ describe('record export command', () => { }, ); - it('names the timeout and the planned duration when the WebM replay times out', async () => { - mocks.recordingFilename.mockReturnValue('recording-1-webm.webm'); - mocks.generateWebmExport.mockRejectedValue( - new WebmReplayTimeoutError(661_000, 361_000, 'recorded'), - ); + it.each([ + { + lastSeq: 1_234, + timingMode: 'recorded', + position: 'after event 1234', + }, + { + lastSeq: -1, + timingMode: 'max-speed', + position: 'at the start of the replay', + }, + ] as const)( + 'names the stall window and the last event when the WebM replay stalls ($position)', + async ({ lastSeq, timingMode, position }) => { + mocks.recordingFilename.mockReturnValue('recording-1-webm.webm'); + mocks.generateWebmExport.mockRejectedValue( + new WebmReplayStallError(300_000, lastSeq, timingMode), + ); - await expect( - runRecordExportCommand({ - context: TEST_CONTEXT, - json: true, - sessionId: 'session-01', - format: 'webm', - }), - ).rejects.toMatchObject({ - code: ERROR_CODES.EXPORT_ERROR, - message: - 'Failed to export session "session-01" as webm: the replay timed out after 661 s. The recording plans 361 s at recorded timing.', - details: { - sessionId: 'session-01', - format: 'webm', - replayTimeoutMs: 661_000, - plannedDurationMs: 361_000, - timingMode: 'recorded', - }, - }); - expect(mocks.generateWebmExport).toHaveBeenCalledTimes(1); - expect(mocks.emitSuccess).not.toHaveBeenCalled(); - }); + await expect( + runRecordExportCommand({ + context: TEST_CONTEXT, + json: true, + sessionId: 'session-01', + format: 'webm', + }), + ).rejects.toMatchObject({ + code: ERROR_CODES.EXPORT_ERROR, + message: `Failed to export session "session-01" as webm: the renderer made no progress for 300 s ${position} (${timingMode} timing).`, + details: { + sessionId: 'session-01', + format: 'webm', + stallWindowMs: 300_000, + lastSeq, + timingMode, + }, + }); + expect(mocks.generateWebmExport).toHaveBeenCalledTimes(1); + expect(mocks.emitSuccess).not.toHaveBeenCalled(); + }, + ); it('keeps the generic message for other WebM export failures', async () => { mocks.recordingFilename.mockReturnValue('recording-1-webm.webm'); diff --git a/test/unit/export/webm.test.ts b/test/unit/export/webm.test.ts index c5e00f7b..8592887e 100644 --- a/test/unit/export/webm.test.ts +++ b/test/unit/export/webm.test.ts @@ -1,6 +1,7 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; import type { + ReplayProgress, VideoCapableRendererBackend, VideoRecordingOptions, } from '../../../src/renderer/backend.js'; @@ -25,8 +26,9 @@ vi.mock('../../../src/renderer/profiles.js', () => ({ import { generateWebmExport, - WebmReplayTimeoutError, + WebmReplayStallError, } from '../../../src/export/webm.js'; +import { GhosttyWebBackend } from '../../../src/renderer/ghosttyWeb/index.js'; function createManifest(overrides: Partial = {}): SessionRecord { return { @@ -259,6 +261,7 @@ describe('generateWebmExport', () => { mode: 'recorded', finalFrameHoldMs: 1_000, }, + expect.any(Function), ); expect(mockBackend.finalizeVideo).toHaveBeenCalledWith( '/tmp/exports/recording-1-webm.webm', @@ -381,6 +384,7 @@ describe('generateWebmExport', () => { minFrameHoldMs: 100, finalFrameHoldMs: 1_000, }, + expect.any(Function), ); expect(result.timingMode).toBe('accelerated'); }); @@ -403,6 +407,7 @@ describe('generateWebmExport', () => { expect(mockBackend.replayWithTiming).toHaveBeenCalledWith( expect.objectContaining({ sessionId: 'session-01' }), { mode: 'max-speed', minFrameHoldMs: 16, finalFrameHoldMs: 500 }, + expect.any(Function), ); expect(result.timingMode).toBe('max-speed'); }); @@ -424,6 +429,7 @@ describe('generateWebmExport', () => { expect(mockBackend.replayWithTiming).toHaveBeenCalledWith( expect.objectContaining({ sessionId: 'session-01' }), { mode: 'recorded', finalFrameHoldMs: 1_000 }, + expect.any(Function), ); expect(result.timingMode).toBe('recorded'); }); @@ -453,6 +459,7 @@ describe('generateWebmExport', () => { expect(mockBackend.replayWithTiming).toHaveBeenCalledWith( emptyInput, expect.anything(), + expect.any(Function), ); expect(result).toMatchObject({ capturedAtSeq: -1, @@ -496,7 +503,7 @@ describe('generateWebmExport', () => { expect(mockBackend.dispose).toHaveBeenCalledTimes(1); }); - it('times out replay and still disposes the backend', async () => { + it('stops a replay that makes no progress and still disposes the backend', async () => { const replayWithTiming = vi.fn( (): Promise> => new Promise(() => undefined), @@ -514,11 +521,11 @@ describe('generateWebmExport', () => { }, { backendFactory: () => mockBackend.backend, - replayTimeoutMs: 1, + stallWindowMs: 1, }, ), ).rejects.toThrow( - 'WebM replay timed out after 1 ms (planned duration 2500 ms at recorded timing)', + 'WebM replay stalled: the renderer made no progress for 1 ms (last event seq -1, recorded timing)', ); expect(mockBackend.finalizeVideo).not.toHaveBeenCalled(); @@ -550,54 +557,39 @@ describe('generateWebmExport', () => { }); }); -describe('generateWebmExport replay deadline', () => { - const FIVE_MINUTES_MS = 5 * 60 * 1000; - const BASE_TIME_MS = Date.parse('2026-03-19T12:00:00.000Z'); +describe('generateWebmExport stall watchdog', () => { + const WINDOW_MS = 5 * 60 * 1000; + const DAY_MS = 24 * 60 * 60 * 1000; + const MAX_TIMER_DELAY_MS = 2 ** 31 - 1; - // Two output events, `spanMs` apart: the recording's whole length. - function useRecordingOfLength(spanMs: number): void { - mocks.buildReplayInput.mockReturnValue({ - sessionId: 'session-01', - initialCols: 80, - initialRows: 24, - events: [ - { - seq: 0, - type: 'output', - ts: new Date(BASE_TIME_MS).toISOString(), - payload: { data: 'start\n' }, - }, - { - seq: 1, - type: 'output', - ts: new Date(BASE_TIME_MS + spanMs).toISOString(), - payload: { data: 'end\n' }, - }, - ], - targetSeq: 1, + type ReplayScript = ( + onProgress: (progress: ReplayProgress) => void, + ) => Promise>; + + function sleep(ms: number): Promise { + return new Promise((resolve) => { + setTimeout(resolve, ms); }); } - // Starts an export whose replay runs on fake timers. It takes `replayMs` of - // fake time, or never ends when `replayMs` is null. + // Starts an export whose replay is `script`, on fake timers. The script + // acts as the renderer: it reports progress and takes fake time. async function startExport( - timingMode: 'recorded' | 'accelerated' | 'max-speed', - replayMs: number | null, + script: ReplayScript, + timingMode: 'recorded' | 'accelerated' | 'max-speed' = 'recorded', ) { let markReplayStarted: () => void = () => undefined; const replayStarted = new Promise((resolve) => { markReplayStarted = resolve; }); const replayWithTiming = vi.fn( - (): Promise> => { + ( + _input: unknown, + _timing: unknown, + onProgress?: (progress: ReplayProgress) => void, + ) => { markReplayStarted(); - return new Promise((resolve) => { - if (replayMs !== null) { - setTimeout(() => { - resolve(createReplayState()); - }, replayMs); - } - }); + return script(onProgress ?? (() => undefined)); }, ); const mockBackend = createMockBackend({ replayWithTiming }); @@ -628,70 +620,411 @@ describe('generateWebmExport replay deadline', () => { } beforeEach(() => { + vi.clearAllMocks(); vi.useFakeTimers(); + mocks.resolveProfile.mockReturnValue({ + name: 'reference-dark', + theme: 'dark', + fontFamily: 'monospace', + fontSize: 16, + cursorStyle: 'block', + backgroundColor: '#000000', + foregroundColor: '#ffffff', + }); + mocks.buildReplayInput.mockReturnValue({ + sessionId: 'session-01', + initialCols: 80, + initialRows: 24, + events: createEvents(), + targetSeq: 1, + }); }); afterEach(() => { vi.useRealTimers(); }); - it('gives a recording of more than 5 minutes the time that its plan needs', async () => { - useRecordingOfLength(6 * 60 * 1000); - // The replay waits out the recorded gap, then holds the final frame. - const { outcome, mockBackend } = await startExport( - 'recorded', - 6 * 60 * 1000 + 1_000, - ); + it('keeps a replay going while it reports progress for longer than the window', async () => { + const { outcome, mockBackend } = await startExport(async (onProgress) => { + for (let seq = 0; seq < 5; seq += 1) { + await sleep(WINDOW_MS - 60_000); + onProgress({ seq, scheduledWaitMs: 0 }); + } + return createReplayState(); + }); - await vi.advanceTimersByTimeAsync(6 * 60 * 1000 + 1_000); - const { result, error } = await outcome; + await vi.advanceTimersByTimeAsync(5 * (WINDOW_MS - 60_000)); + const { error } = await outcome; expect(error).toBeUndefined(); - expect(result?.durationMs).toBe(6 * 60 * 1000); expect(mockBackend.finalizeVideo).toHaveBeenCalledTimes(1); }); - it('keeps the deadline within the longest delay that a timer runs', async () => { - useRecordingOfLength(30 * 24 * 60 * 60 * 1000); - const { outcome } = await startExport('recorded', 60_000); + it('does not count a multi-day wait on the timing schedule as a stall', async () => { + const { outcome, mockBackend } = await startExport(async (onProgress) => { + onProgress({ seq: 0, scheduledWaitMs: 20 * DAY_MS }); + await sleep(20 * DAY_MS); + onProgress({ seq: 1, scheduledWaitMs: 0 }); + return createReplayState(); + }); - await vi.advanceTimersByTimeAsync(60_000); + await vi.advanceTimersByTimeAsync(20 * DAY_MS); const { error } = await outcome; expect(error).toBeUndefined(); + expect(mockBackend.finalizeVideo).toHaveBeenCalledTimes(1); }); - it.each([ - // The deadline is the planned duration plus 5 minutes. The plan is the - // recorded gap and the final frame hold, with the gap cut by the mode. - { timingMode: 'recorded', plannedDurationMs: 6 * 60 * 1000 + 1_000 }, - { timingMode: 'accelerated', plannedDurationMs: 400 + 1_000 }, - { timingMode: 'max-speed', plannedDurationMs: 500 }, - ] as const)( - 'stops a hung $timingMode replay once its plan and 5 minutes have passed', - async ({ timingMode, plannedDurationMs }) => { - useRecordingOfLength(6 * 60 * 1000); - const deadlineMs = plannedDurationMs + FIVE_MINUTES_MS; + it('starts no timer that runs longer than a timer can, for a wait of 30 days', async () => { + const setTimeoutSpy = vi.spyOn(globalThis, 'setTimeout'); + const { outcome, mockBackend } = await startExport(async (onProgress) => { + onProgress({ seq: 0, scheduledWaitMs: 30 * DAY_MS }); + await sleep(15 * DAY_MS); + await sleep(15 * DAY_MS); + onProgress({ seq: 1, scheduledWaitMs: 0 }); + return createReplayState(); + }); + + await vi.advanceTimersByTimeAsync(30 * DAY_MS); + const { error } = await outcome; + + expect(error).toBeUndefined(); + expect(mockBackend.finalizeVideo).toHaveBeenCalledTimes(1); + const delays = setTimeoutSpy.mock.calls.map((call) => Number(call[1])); + expect(delays.length).toBeGreaterThan(0); + expect(Math.max(...delays)).toBeLessThanOrEqual(MAX_TIMER_DELAY_MS); + }); + + it.each(['recorded', 'accelerated', 'max-speed'] as const)( + 'stops a hung %s replay once the window has passed since its last progress', + async (timingMode) => { const { outcome, mockBackend, isSettled } = await startExport( + async (onProgress) => { + await sleep(1_000); + onProgress({ seq: 7, scheduledWaitMs: 0 }); + return new Promise(() => undefined); + }, timingMode, - null, ); - await vi.advanceTimersByTimeAsync(deadlineMs - 1); + await vi.advanceTimersByTimeAsync(1_000 + WINDOW_MS - 1); expect(isSettled()).toBe(false); await vi.advanceTimersByTimeAsync(1); const { error } = await outcome; - expect(error).toBeInstanceOf(WebmReplayTimeoutError); + expect(error).toBeInstanceOf(WebmReplayStallError); expect(error).toMatchObject({ - timeoutMs: deadlineMs, - plannedDurationMs, + stallWindowMs: WINDOW_MS, + lastSeq: 7, timingMode, - message: `WebM replay timed out after ${String(deadlineMs)} ms (planned duration ${String(plannedDurationMs)} ms at ${timingMode} timing)`, + message: `WebM replay stalled: the renderer made no progress for ${String(WINDOW_MS)} ms (last event seq 7, ${timingMode} timing)`, }); expect(mockBackend.finalizeVideo).not.toHaveBeenCalled(); expect(mockBackend.dispose).toHaveBeenCalledTimes(1); }, ); + + it('stops a replay that reports nothing, naming no event', async () => { + const { outcome, isSettled } = await startExport( + () => new Promise(() => undefined), + ); + + await vi.advanceTimersByTimeAsync(WINDOW_MS - 1); + expect(isSettled()).toBe(false); + + await vi.advanceTimersByTimeAsync(1); + const { error } = await outcome; + + expect(error).toBeInstanceOf(WebmReplayStallError); + expect(error).toMatchObject({ stallWindowMs: WINDOW_MS, lastSeq: -1 }); + }); + + it('starts the window after a scheduled wait, not before it', async () => { + const { outcome, isSettled } = await startExport((onProgress) => { + onProgress({ seq: 3, scheduledWaitMs: 2 * DAY_MS }); + return new Promise(() => undefined); + }); + + await vi.advanceTimersByTimeAsync(2 * DAY_MS + WINDOW_MS - 1); + expect(isSettled()).toBe(false); + + await vi.advanceTimersByTimeAsync(1); + const { error } = await outcome; + + expect(error).toBeInstanceOf(WebmReplayStallError); + expect(error).toMatchObject({ stallWindowMs: WINDOW_MS, lastSeq: 3 }); + }); + + it('leaves no timer behind when the replay succeeds', async () => { + const { outcome } = await startExport((onProgress) => { + onProgress({ seq: 0, scheduledWaitMs: DAY_MS }); + return Promise.resolve(createReplayState()); + }); + + const { error } = await outcome; + + expect(error).toBeUndefined(); + expect(vi.getTimerCount()).toBe(0); + }); + + it('leaves no timer behind when the replay fails', async () => { + const replayError = new Error('replay failed'); + const { outcome } = await startExport((onProgress) => { + onProgress({ seq: 0, scheduledWaitMs: DAY_MS }); + return Promise.reject(replayError); + }); + + const { error } = await outcome; + + expect(error).toBe(replayError); + expect(vi.getTimerCount()).toBe(0); + }); + + it('starts no timer when a replay that already stalled reports again', async () => { + let reportLate: (progress: ReplayProgress) => void = () => undefined; + const { outcome } = await startExport((onProgress) => { + reportLate = onProgress; + return new Promise(() => undefined); + }); + + await vi.advanceTimersByTimeAsync(WINDOW_MS); + const { error } = await outcome; + expect(error).toBeInstanceOf(WebmReplayStallError); + + reportLate({ seq: 9, scheduledWaitMs: DAY_MS }); + + expect(vi.getTimerCount()).toBe(0); + }); + + it.each([ + { seq: 0.5, scheduledWaitMs: 0, message: 'seq must be an integer' }, + { seq: -2, scheduledWaitMs: 0, message: 'seq must be an integer' }, + { seq: 0, scheduledWaitMs: -1, message: 'scheduledWaitMs must be' }, + { seq: 0, scheduledWaitMs: Number.NaN, message: 'scheduledWaitMs must be' }, + ])( + 'rejects a progress report of seq $seq and wait $scheduledWaitMs', + async ({ seq, scheduledWaitMs, message }) => { + const { outcome } = await startExport((onProgress) => { + onProgress({ seq, scheduledWaitMs }); + return Promise.resolve(createReplayState()); + }); + + const { error } = await outcome; + + expect(error).toBeInstanceOf(Error); + expect((error as Error).message).toContain(message); + expect(vi.getTimerCount()).toBe(0); + }, + ); +}); + +// These tests run the real GhosttyWebBackend replay loop with its renderer +// bridge replaced by a stub that takes fake time. They check that the loop +// reports progress where the watchdog needs it. +describe('generateWebmExport stall watchdog with the ghostty-web replay loop', () => { + const BASE_TIME_MS = Date.parse('2026-03-19T12:00:00.000Z'); + const DAY_MS = 24 * 60 * 60 * 1000; + const WINDOW_MS = 5 * 60 * 1000; + + type ReplayEvent = { + seq: number; + type: 'output' | 'input_text'; + ts: string; + payload: { data: string }; + }; + + function event( + seq: number, + offsetMs: number, + type: ReplayEvent['type'] = 'output', + ): ReplayEvent { + return { + seq, + type, + ts: new Date(BASE_TIME_MS + offsetMs).toISOString(), + payload: { data: `e${String(seq)}` }, + }; + } + + // Exports `events` with the real replay loop. Each bridge call takes + // `bridgeCostMs` of fake time. Returns how long the replay took in fake + // time, or the error that stopped it. + async function exportWithRealReplayLoop( + events: ReplayEvent[], + timingMode: 'recorded' | 'accelerated' | 'max-speed', + bridgeCostMs: number, + runForMs: number, + ): Promise<{ replayMs: number | undefined; error: unknown }> { + mocks.buildReplayInput.mockReturnValue({ + sessionId: 'session-01', + initialCols: 80, + initialRows: 24, + events, + targetSeq: events.length - 1, + }); + let markReplayStarted: () => void = () => undefined; + const replayStarted = new Promise((resolve) => { + markReplayStarted = resolve; + }); + let finalizedAtMs: number | undefined; + let startedAtMs = 0; + const outcome = generateWebmExport( + { + sessionId: 'session-01', + sessionDir: '/tmp/agent-tty/sessions/session-01', + manifest: createManifest(), + events: [], + outputPath: '/tmp/exports/recording-1-webm.webm', + timingMode, + }, + { + backendFactory: (_rendererName, sessionId, profile, videoOptions) => { + const backend = new GhosttyWebBackend( + sessionId, + profile, + videoOptions, + ); + Object.assign(backend as object, { + boot: () => Promise.resolve(), + isBooted: true, + page: { isClosed: () => false }, + resizeBridge: () => { + startedAtMs = Date.now(); + markReplayStarted(); + return Promise.resolve(); + }, + writeBatchBridge: () => + new Promise((resolve) => { + setTimeout(resolve, bridgeCostMs); + }), + notePaintedGraphemes: () => Promise.resolve(), + checkPaintedGlyphs: () => Promise.resolve({}), + readHarnessSnapshot: () => + Promise.resolve({ + cols: 80, + rows: 24, + cursorRow: 0, + cursorCol: 0, + isAltScreen: false, + visibleLines: [], + }), + recordedMissingGlyphs: () => Promise.resolve({}), + finalizeVideo: () => { + finalizedAtMs = Date.now(); + return Promise.resolve(); + }, + dispose: () => Promise.resolve(), + }); + return backend; + }, + }, + ).then( + () => undefined, + (error: unknown) => error, + ); + + await replayStarted; + await vi.advanceTimersByTimeAsync(runForMs); + const error = await outcome; + + return { + replayMs: + finalizedAtMs === undefined ? undefined : finalizedAtMs - startedAtMs, + error, + }; + } + + beforeEach(() => { + vi.clearAllMocks(); + vi.useFakeTimers(); + mocks.resolveProfile.mockReturnValue({ + name: 'reference-dark', + theme: 'dark', + fontFamily: 'monospace', + fontSize: 16, + cursorStyle: 'block', + backgroundColor: '#000000', + foregroundColor: '#ffffff', + }); + }); + + afterEach(() => { + vi.useRealTimers(); + }); + + it.each([ + { label: '6 minutes', gapMs: 6 * 60 * 1000 }, + { label: '20 days', gapMs: 20 * DAY_MS }, + ])( + 'exports a recording with a gap of $label at recorded timing', + async ({ gapMs }) => { + const { replayMs, error } = await exportWithRealReplayLoop( + [event(0, 0), event(1, gapMs)], + 'recorded', + 0, + gapMs + 60_000, + ); + + expect(error).toBeUndefined(); + expect(replayMs).toBeGreaterThan(gapMs); + }, + ); + + it.each([ + { timingMode: 'accelerated', eventCount: 7_401, bridgeCostMs: 50 }, + { timingMode: 'max-speed', eventCount: 39_001, bridgeCostMs: 15 }, + ] as const)( + 'exports a dense $timingMode recording whose replay runs far past its plan', + async ({ timingMode, eventCount, bridgeCostMs }) => { + // An input event after each output event forces a flush, and each flush + // holds the frame, so the replay takes minutes for a recording that + // spans seconds. + const events = Array.from({ length: eventCount }, (_, seq) => + event(seq, seq, seq % 2 === 0 ? 'output' : 'input_text'), + ); + const { replayMs, error } = await exportWithRealReplayLoop( + events, + timingMode, + bridgeCostMs, + 60 * 60 * 1000, + ); + + expect(error).toBeUndefined(); + expect(replayMs).toBeGreaterThan(WINDOW_MS); + }, + ); + + it('exports a max-speed recording whose output goes to the renderer in one flush', async () => { + // One flush is 40 bridge calls of 21 s each, 14 minutes in all. Each + // call is progress. + const events = Array.from({ length: 39_001 }, (_, seq) => event(seq, seq)); + const { replayMs, error } = await exportWithRealReplayLoop( + events, + 'max-speed', + 21_000, + 60 * 60 * 1000, + ); + + expect(error).toBeUndefined(); + expect(replayMs).toBeGreaterThan(WINDOW_MS); + }); + + it('stops a replay whose renderer call hangs and names the last event it reached', async () => { + const events = [event(0, 0), event(1, 10), event(2, 20, 'input_text')]; + const { replayMs, error } = await exportWithRealReplayLoop( + events, + 'recorded', + // A bridge call that outlasts the window. + 2 * WINDOW_MS, + 60 * 60 * 1000, + ); + + expect(replayMs).toBeUndefined(); + expect(error).toBeInstanceOf(WebmReplayStallError); + expect(error).toMatchObject({ + stallWindowMs: WINDOW_MS, + timingMode: 'recorded', + }); + }); }); diff --git a/test/unit/renderer/ghosttyWebBackend.test.ts b/test/unit/renderer/ghosttyWebBackend.test.ts index 723b321f..e8c299de 100644 --- a/test/unit/renderer/ghosttyWebBackend.test.ts +++ b/test/unit/renderer/ghosttyWebBackend.test.ts @@ -13,7 +13,10 @@ import { dirname, join } from 'node:path'; import { afterEach, describe, expect, it, vi } from 'vitest'; -import type { ReplayTimingOptions } from '../../../src/renderer/backend.js'; +import type { + ReplayProgress, + ReplayTimingOptions, +} from '../../../src/renderer/backend.js'; import type { ReplayInput } from '../../../src/renderer/types.js'; import { BUNDLED_FONT_ASSETS } from '../../../src/renderer/bundledFont.js'; import { hashProfile, resolveProfile } from '../../../src/renderer/profiles.js'; @@ -187,6 +190,78 @@ describe('GhosttyWebBackend unit guards', () => { expect(calls).toEqual(['write first', 'note', 'write second', 'note']); }); + it('reports progress after each bridge batch of a timed replay flush', async () => { + const backend = createBackend(); + const calls: string[] = []; + const reports: ReplayProgress[] = []; + Object.assign(backend as object, { + isBooted: true, + page: { isClosed: () => false }, + resizeBridge: () => Promise.resolve(), + writeBatchBridge: (_page: object, dataChunks: string[]) => { + calls.push(`write ${String(dataChunks.length)}`); + return Promise.resolve(); + }, + notePaintedGraphemes: () => { + calls.push('note'); + return Promise.resolve(); + }, + checkPaintedGlyphs: () => Promise.resolve({}), + readHarnessSnapshot: () => + Promise.resolve( + createHarnessSnapshotPayload({ + cols: 80, + rows: 24, + cursorRow: 0, + cursorCol: 0, + }), + ), + }); + const events: ReplayInput['events'] = Array.from( + { length: 2_501 }, + (_, seq) => ({ + seq, + ts: '2026-03-19T12:00:00.000Z', + type: 'output', + payload: { data: `out${String(seq)}` }, + }), + ); + + await backend.replayWithTiming( + { + sessionId: 'renderer-unit-session', + initialCols: 80, + initialRows: 24, + targetSeq: 2_500, + events, + }, + { mode: 'max-speed', minFrameHoldMs: 0, finalFrameHoldMs: 0 }, + (progress) => { + calls.push('report'); + reports.push(progress); + }, + ); + + // Max-speed sends the whole run of output in one flush of three bridge + // calls. A caller that watches for stalls sees progress after each call. + expect(calls.slice(calls.indexOf('write 1000'))).toEqual([ + 'write 1000', + 'note', + 'report', + 'write 1000', + 'note', + 'report', + 'write 501', + 'note', + 'report', + ]); + expect(reports.slice(-3)).toEqual([ + { seq: 2_500, scheduledWaitMs: 0 }, + { seq: 2_500, scheduledWaitMs: 0 }, + { seq: 2_500, scheduledWaitMs: 0 }, + ]); + }); + it('retries the painted-grapheme note after the backend reboots', async () => { const backend = createBackend(); const graphemes = vi.fn(() => Promise.resolve({ missingGlyphs: [] })); @@ -1291,6 +1366,56 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { ); }); + it('reports the wait on the timing schedule before it starts the wait', async () => { + vi.useFakeTimers(); + const backend = createBackend(); + Object.assign(backend as object, { + isBooted: true, + page: { isClosed: () => false }, + resizeBridge: () => Promise.resolve(), + flushOutputBatch: () => Promise.resolve(), + readHarnessSnapshot: () => + Promise.resolve( + createHarnessSnapshotPayload({ + cols: 80, + rows: 24, + visibleLines: [], + }), + ), + }); + const reports: (ReplayProgress & { atMs: number })[] = []; + const startedAtMs = performance.now(); + + const replay = backend.replayWithTiming( + { + sessionId: 'renderer-unit-session', + initialCols: 80, + initialRows: 24, + targetSeq: 2, + events: [outputEvent(0, 0), outputEvent(1, 200), outputEvent(2, 5_000)], + }, + RECORDED, + (progress) => { + reports.push({ ...progress, atMs: performance.now() - startedAtMs }); + }, + ); + await vi.runAllTimersAsync(); + await replay; + + // The first event sits at the first frame hold. Each later event sits + // at its recorded offset from it, so a report plus its wait ends at that + // place on the schedule. + expect( + reports.map((report) => ({ + seq: report.seq, + endsAtMs: report.atMs + report.scheduledWaitMs, + })), + ).toEqual([ + { seq: 0, endsAtMs: FIRST_EVENT_AT_MS + 200 }, + { seq: 1, endsAtMs: FIRST_EVENT_AT_MS + 5_000 }, + ]); + }); + it('keeps the recorded pace when continuous output fills every frame hold', async () => { const events = Array.from({ length: 101 }, (_, index) => outputEvent(index, index * 10), From b121f7562379473268c44e9b57a8dbd9ed645803 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 07:49:57 +0000 Subject: [PATCH 04/28] test: watch for the stall through the backend's dispose, not the export promise --- test/unit/export/webm.test.ts | 36 +++++++++++++---------------------- 1 file changed, 13 insertions(+), 23 deletions(-) diff --git a/test/unit/export/webm.test.ts b/test/unit/export/webm.test.ts index 8592887e..23b2285e 100644 --- a/test/unit/export/webm.test.ts +++ b/test/unit/export/webm.test.ts @@ -593,7 +593,6 @@ describe('generateWebmExport stall watchdog', () => { }, ); const mockBackend = createMockBackend({ replayWithTiming }); - let settled = false; const outcome = generateWebmExport( { sessionId: 'session-01', @@ -605,18 +604,12 @@ describe('generateWebmExport stall watchdog', () => { }, { backendFactory: () => mockBackend.backend }, ).then( - (result) => { - settled = true; - return { result, error: undefined }; - }, - (error: unknown) => { - settled = true; - return { result: undefined, error }; - }, + (result) => ({ result, error: undefined }), + (error: unknown) => ({ result: undefined, error }), ); await replayStarted; - return { outcome, mockBackend, isSettled: () => settled }; + return { outcome, mockBackend }; } beforeEach(() => { @@ -698,17 +691,14 @@ describe('generateWebmExport stall watchdog', () => { it.each(['recorded', 'accelerated', 'max-speed'] as const)( 'stops a hung %s replay once the window has passed since its last progress', async (timingMode) => { - const { outcome, mockBackend, isSettled } = await startExport( - async (onProgress) => { - await sleep(1_000); - onProgress({ seq: 7, scheduledWaitMs: 0 }); - return new Promise(() => undefined); - }, - timingMode, - ); + const { outcome, mockBackend } = await startExport(async (onProgress) => { + await sleep(1_000); + onProgress({ seq: 7, scheduledWaitMs: 0 }); + return new Promise(() => undefined); + }, timingMode); await vi.advanceTimersByTimeAsync(1_000 + WINDOW_MS - 1); - expect(isSettled()).toBe(false); + expect(mockBackend.dispose).not.toHaveBeenCalled(); await vi.advanceTimersByTimeAsync(1); const { error } = await outcome; @@ -726,12 +716,12 @@ describe('generateWebmExport stall watchdog', () => { ); it('stops a replay that reports nothing, naming no event', async () => { - const { outcome, isSettled } = await startExport( + const { outcome, mockBackend } = await startExport( () => new Promise(() => undefined), ); await vi.advanceTimersByTimeAsync(WINDOW_MS - 1); - expect(isSettled()).toBe(false); + expect(mockBackend.dispose).not.toHaveBeenCalled(); await vi.advanceTimersByTimeAsync(1); const { error } = await outcome; @@ -741,13 +731,13 @@ describe('generateWebmExport stall watchdog', () => { }); it('starts the window after a scheduled wait, not before it', async () => { - const { outcome, isSettled } = await startExport((onProgress) => { + const { outcome, mockBackend } = await startExport((onProgress) => { onProgress({ seq: 3, scheduledWaitMs: 2 * DAY_MS }); return new Promise(() => undefined); }); await vi.advanceTimersByTimeAsync(2 * DAY_MS + WINDOW_MS - 1); - expect(isSettled()).toBe(false); + expect(mockBackend.dispose).not.toHaveBeenCalled(); await vi.advanceTimersByTimeAsync(1); const { error } = await outcome; From 313eef9ed17fb58b65e519df976e1853c525ed6f Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 07:51:09 +0000 Subject: [PATCH 05/28] docs: describe the stall watchdog and the progress reports --- docs/USAGE.md | 2 +- src/cli/commands/README.md | 2 +- src/export/README.md | 7 +++---- src/renderer/README.md | 2 +- src/renderer/ghosttyWeb/README.md | 3 ++- 5 files changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/USAGE.md b/docs/USAGE.md index 1fef60ed..13b52d3a 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -294,7 +294,7 @@ agent-tty record export --format webm --json | jq -c '.result.missi WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. -The replay runs in real time, so a WebM export with recorded timing takes about as long as the recording. The export stops a replay that runs more than 5 minutes past its plan (the recorded gaps that the timing keeps, plus the hold on the last frame) and fails with `EXPORT_ERROR`. The message names the timeout and the planned duration, and `details` holds `replayTimeoutMs`, `plannedDurationMs` and `timingMode`. +The replay runs in real time, so a WebM export with recorded timing takes about as long as the recording, and a long recording takes a long time. The export has no total time limit. It stops only when the renderer makes no progress for 5 minutes, and then fails with `EXPORT_ERROR`. The message names the window and the last event that the replay reached, and `details` holds `stallWindowMs`, `lastSeq` and `timingMode`. Waiting out a recorded gap is not a stall. SVG export renders styled grid frames from the event log through the native `libghostty-vt` backend with no browser and no ffmpeg, so it requires the optional `@coder/libghostty-vt-node` package (there is no `ghostty-web` fallback). The output is deterministic: exporting the same session twice produces byte-identical, diffable SVG. `--format svg` writes a still image of the final screen and ignores event timing, so it also works on a log whose clock stepped backward (`--animate` refuses such a log and names the event where the clock stepped back); add `--animate` for an animated SVG of de-duplicated frames replayed with recorded event-log timing (`--timing` is not supported with SVG). diff --git a/src/cli/commands/README.md b/src/cli/commands/README.md index d7213e3f..f4828bae 100644 --- a/src/cli/commands/README.md +++ b/src/cli/commands/README.md @@ -34,7 +34,7 @@ This directory holds one handler module per agent-tty CLI command. `src/cli/main - `wait` rejects a mix of legacy flags (`--exit`, `--idle-ms`) and render flags. `--text` and `--regex` are mutually exclusive. The default timeout is `DEFAULT_WAIT_TIMEOUT_MS` (30,000 ms), which the `main.ts` help text names. An `--idle-ms` at or above the effective timeout (explicit or default), or a `--screen-stable-ms` whose `screenStableMinimumMs` is not shorter than it, throws `INVALID_DURATION`. `--timeout 0` skips the check. [wait.ts:runWaitCommand] [wait.ts:assertWindowFitsTimeout] [main.ts:main] - A timed-out `wait` throws `WAIT_TIMEOUT` with the wait result attached, so its error envelope keeps `result`. Without `--json`, and only when the wait lasted its whole timeout (`timeoutMs > 0`, measured by the CLI), the message names the timeout and, for the default, how to change it. With `--json`, the message stays `Wait timed out before its condition was met.` [wait.ts:throwIfTimedOut] [wait.ts:timeoutMessage] - `snapshot` and `screenshot` try the live RPC only when the manifest status is `running`. They fall back to offline replay only on `HOST_UNREACHABLE` and rethrow other errors. [snapshot.ts:runSnapshotCommand] [screenshot.ts:runScreenshotCommand] -- `record export` accepts only the formats in `RecordExportFormatSchema` (`asciicast`, `webm`, `svg`). `--animate` needs `svg`, and `svg` refuses `--timing`. An event log over the size limit fails with `EXPORT_ERROR` in every format, a message that names the log size and the limit, and `eventLogBytes` and `maxEventLogBytes` in `details`. A `webm` replay that passes its deadline fails with `EXPORT_ERROR`, a message that names the timeout and the planned duration, and `replayTimeoutMs`, `plannedDurationMs` and `timingMode` in `details`. [record-export.ts:runRecordExportCommand] +- `record export` accepts only the formats in `RecordExportFormatSchema` (`asciicast`, `webm`, `svg`). `--animate` needs `svg`, and `svg` refuses `--timing`. An event log over the size limit fails with `EXPORT_ERROR` in every format, a message that names the log size and the limit, and `eventLogBytes` and `maxEventLogBytes` in `details`. A `webm` replay whose renderer makes no progress for the stall window fails with `EXPORT_ERROR`, a message that names the window and the last event that the replay reached, and `stallWindowMs`, `lastSeq` and `timingMode` in `details`. [record-export.ts:runRecordExportCommand] - `batch` parses the whole plan with `parseBatchPlan` before it resolves the session target. [batch.ts:runBatchCommand] - `create` polls readiness up to `READINESS_MAX_ATTEMPTS` (50) times, `READINESS_POLL_INTERVAL_MS` (100 ms) apart. [create.ts:waitForReadyHost] - `doctor` reports the `node-runtime` check as failed when the Node version is outside the `engines.node` range of `package.json`. It keeps running the other checks, reports `ok: false` and sets `process.exitCode = 1`. [doctor.ts:runNodeRuntimeCheck] [doctor.ts:runDoctorCommand] diff --git a/src/export/README.md b/src/export/README.md index 89f7d789..db437a3e 100644 --- a/src/export/README.md +++ b/src/export/README.md @@ -10,10 +10,10 @@ This directory converts a session's event log into recording exports. `generateA - `AsciicastExport`: `contents`, `header`, `capturedAtSeq`, `durationMs`, `outputEventCount`, `resizeEventCount` and `markerCount` [asciicast.ts:AsciicastExport]. - `generateAsciicast(sessionId, manifest, events, toolVersion?)` returns an `AsciicastExport`. It runs synchronously [asciicast.ts:generateAsciicast]. - `WebmExportOptions`: `sessionId`, `sessionDir`, `manifest`, `events`, `outputPath`, and optional `targetSeq`, `profileName`, `timingMode` and `rendererName` [webm.ts:WebmExportOptions]. -- `WebmExportDeps`: optional `backendFactory` and `replayTimeoutMs`, for tests and callers that supply their own backend. `replayTimeoutMs` replaces the replay deadline that is derived from the plan (see Invariants) [webm.ts:WebmExportDeps]. +- `WebmExportDeps`: optional `backendFactory` and `stallWindowMs`, for tests and callers that supply their own backend. `stallWindowMs` replaces the 5-minute stall window (see Invariants) [webm.ts:WebmExportDeps]. - `WebmExportResult`: `capturedAtSeq`, `durationMs`, `outputEventCount`, `resizeEventCount`, `cols`, `rows`, `profileName`, `timingMode` and `rendererBackend`, plus the optional `missingGlyphs` and `missingGlyphsTruncated` of `RecordedGlyphFields` [webm.ts:WebmExportResult]. - `generateWebmExport(options, deps?)` returns a `Promise` [webm.ts:generateWebmExport]. -- `WebmReplayTimeoutError` is the error that `generateWebmExport` rejects with when the timed replay passes its deadline. It carries `timeoutMs`, `plannedDurationMs` and `timingMode` [webm.ts:WebmReplayTimeoutError]. +- `WebmReplayStallError` is the error that `generateWebmExport` rejects with when the renderer makes no progress for the stall window. It carries `stallWindowMs`, `lastSeq` (the last event that the replay reached, or -1 for none) and `timingMode` [webm.ts:WebmReplayStallError]. - `renderGridFramesToSvg({ profile, frames, animate, heapSizeLimit? })` returns SVG text from styled grid frames, which `captureGridFrames` in `src/replay/gridFrames.ts` builds. It runs synchronously and leaves the file write to the caller [svg.ts:renderGridFramesToSvg]. ## Invariants @@ -27,8 +27,7 @@ This directory converts a session's event log into recording exports. `generateA - A requested `libghostty-vt` renderer is replaced with `ghostty-web`. A code comment says `libghostty-vt` is semantic-only [webm.ts:generateWebmExport]. - The default backend factory accepts only `ghostty-web` [webm.ts:createDefaultBackend]. - Timing presets: `accelerated` caps gaps at 400 ms and holds frames at least 100 ms. `max-speed` holds frames at least 16 ms. Each mode holds the final frame (1000 ms, or 500 ms for `max-speed`) [webm.ts:buildReplayTimingOptions]. -- The timed replay fails with `WebmReplayTimeoutError` after its deadline: the planned duration plus `REPLAY_TIMEOUT_MARGIN_MS` (5 minutes), at most 2^31-1 ms (the longest delay that a timer runs). A long recording is not refused for its length [webm.ts:generateWebmExport, webm.ts:REPLAY_TIMEOUT_MARGIN_MS]. -- The planned duration is the schedule of the replay: the gaps between the events up to the target seq, as the timing mode keeps them (`recorded` keeps each gap, `accelerated` caps each at 400 ms, `max-speed` drops them), plus the hold on the final frame. It asserts that event timestamps do not decrease [webm.ts:plannedReplayDurationMs]. +- The timed replay has no total time limit. It fails with `WebmReplayStallError` only when the renderer makes no progress for `REPLAY_STALL_WINDOW_MS` (5 minutes). The backend reports progress through the `onProgress` argument of `replayWithTiming`: each report restarts the window. A report that comes before a wait on the timing schedule carries the length of that wait, and the window starts after it, so a long recorded gap is not a stall. No timer runs longer than 2^31-1 ms (the longest delay that a timer runs): a longer span is a chain of timers. The watchdog starts no timer once the replay has ended or stalled. Boot and finalize run outside the watchdog [webm.ts:replayWithStallWatchdog, webm.ts:REPLAY_STALL_WINDOW_MS]. - `renderGridFramesToSvg` needs the profile font size `SVG_FONT_SIZE` (14) and draws fixed 8.4x18 px cells, so equal input gives byte-identical output [svg.ts:renderGridFramesToSvg]. It always embeds the JetBrains Mono subset, and embeds each other bundled face only when a glyph needs it [svg.ts:embeddedFontAssets]. Output past `defaultMaxSvgOutputLength` of the V8 heap limit (the heap limit divided by 8 bytes, at most `MAX_SVG_OUTPUT_LENGTH`) fails with `EXPORT_ERROR` [svg.ts:defaultMaxSvgOutputLength]. ## Side Effects diff --git a/src/renderer/README.md b/src/renderer/README.md index a2154a14..2194da7a 100644 --- a/src/renderer/README.md +++ b/src/renderer/README.md @@ -10,7 +10,7 @@ This directory defines the renderer abstraction that turns replayed session even - [./libghosttyVt/README.md](./libghosttyVt/README.md): `LibghosttyVtBackend`, a native semantic renderer from `@coder/libghostty-vt-node`. Its screenshots go through a `ghostty-web` fallback. - `RendererBackend`: `boot`, `replayTo`, `snapshot`, `screenshot`, `getVisibleText`, `dispose`, `rendererBackend` and `isBooted` [backend.ts:RendererBackend]. Option types `SnapshotOptions`, `ScreenshotOptions` and `VideoRecordingOptions` [backend.ts:VideoRecordingOptions]. - `MouseEncodingBackend` adds `encodeMouse(input)`, which encodes one `MouseEncodingInput` with the mouse modes the child set and returns the bytes, empty when tracking suppresses the event. `supportsMouseEncoding(backend)` narrows a `RendererBackend` to it. Only `LibghosttyVtBackend` implements it [backend.ts:MouseEncodingBackend] [backend.ts:supportsMouseEncoding]. -- `VideoCapableRendererBackend` adds `replayWithTiming(input, timing)`, `finalizeVideo(outputPath)` and `recordedMissingGlyphs()`, which resolves to the `RecordedGlyphFields` of the replay and never rejects [backend.ts:VideoCapableRendererBackend]. `ReplayTimingOptions` is the union of the exported interfaces `AcceleratedTimingOptions`, `RecordedTimingOptions` and `MaxSpeedTimingOptions` (modes `accelerated`, `recorded` and `max-speed`) [backend.ts:ReplayTimingOptions]. +- `VideoCapableRendererBackend` adds `replayWithTiming(input, timing, onProgress?)`, `finalizeVideo(outputPath)` and `recordedMissingGlyphs()`, which resolves to the `RecordedGlyphFields` of the replay and never rejects [backend.ts:VideoCapableRendererBackend]. `onProgress` receives a `ReplayProgress`: the last `seq` that the replay reached, and the `scheduledWaitMs` that the replay waits right after the report [backend.ts:ReplayProgress]. `ReplayTimingOptions` is the union of the exported interfaces `AcceleratedTimingOptions`, `RecordedTimingOptions` and `MaxSpeedTimingOptions` (modes `accelerated`, `recorded` and `max-speed`) [backend.ts:ReplayTimingOptions]. - Schemas and inferred types: `ReplayEventSchema` (11 event types, discriminated on `type`), `ReplayInputSchema`, `ReplayStateSchema`, `SemanticSnapshotSchema`, `TextSnapshotSchema`, `ScreenshotResultSchema`, `RenderProfileBundledFontSchema` and `RenderProfileConfigSchema` [types.ts:ReplayEventSchema]. `types.ts` re-exports `VisibleLineSchema` from `src/protocol/schemas.ts`. - `RendererNameSchema` (`'ghostty-web'`, `'libghostty-vt'`), `resolveRendererName`, and the defaults `DEFAULT_RENDERER_NAME`, `DEFAULT_SEMANTIC_RENDERER_NAME` and `DEFAULT_VISUAL_RENDERER_NAME` [names.ts:RendererNameSchema]. - `createRendererBackend(rendererName, sessionId, profile, options?)` returns a backend instance. `RendererRegistryOptions` can override the `libghostty-vt` loader [registry.ts:createRendererBackend]. diff --git a/src/renderer/ghosttyWeb/README.md b/src/renderer/ghosttyWeb/README.md index 10ea8ac4..3e483a3c 100644 --- a/src/renderer/ghosttyWeb/README.md +++ b/src/renderer/ghosttyWeb/README.md @@ -8,7 +8,7 @@ This directory holds the `ghostty-web` reference renderer. `GhosttyWebBackend` s - `GhosttyWebBackend` [backend.ts:GhosttyWebBackend]: implements `VideoCapableRendererBackend` from `src/renderer/backend.ts`. Its `rendererBackend` is `'ghostty-web'`. It is re-exported by [index.ts:GhosttyWebBackend]. - `constructor(sessionId, profile, videoOptions?, logger?)` [backend.ts:GhosttyWebBackend.constructor]. - - `boot()`, `replayTo(input)` and `replayWithTiming(input, timing)` [backend.ts:GhosttyWebBackend.replayWithTiming]. Timing modes are `accelerated`, `recorded` and `max-speed`. + - `boot()`, `replayTo(input)` and `replayWithTiming(input, timing, onProgress?)` [backend.ts:GhosttyWebBackend.replayWithTiming]. Timing modes are `accelerated`, `recorded` and `max-speed`. - `snapshot(options?)` returns a `SemanticSnapshot` [backend.ts:GhosttyWebBackend.snapshot]. - `screenshot(outputPath, options?)` returns a `ScreenshotResult` with `sha256`, `renderProfileHash` and `missingGlyphs` [backend.ts:GhosttyWebBackend.screenshot]. - `recordedMissingGlyphs()` returns the boxed code points of every frame that `replayWithTiming()` painted [backend.ts:GhosttyWebBackend.recordedMissingGlyphs]. @@ -30,6 +30,7 @@ This directory holds the `ghostty-web` reference renderer. `GhosttyWebBackend` s - Replay cannot rewind: `targetSeq` must be >= the last applied seq. The initial dimensions cannot change after the first replay. `input.sessionId` must match the backend [backend.ts:GhosttyWebBackend.replayTo]. - `replayWithTiming` asserts strictly increasing seqs and non-decreasing timestamps [backend.ts:GhosttyWebBackend.replayWithTiming]. - `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` flushes once per run of output events [backend.ts:GhosttyWebBackend.replayWithTiming]. +- `replayWithTiming` calls the optional `onProgress` with a `ReplayProgress` in two places. Before each wait on the schedule, `scheduledWaitMs` is the length of that wait. After each bridge batch of a flush (and after `notePaintedGraphemes`), `scheduledWaitMs` is 0. `seq` is the last event that the replay had reached. `generateWebmExport` uses these reports to tell a stall from a wait [backend.ts:GhosttyWebBackend.replayWithTiming]. - `snapshot()` and `screenshot()` throw until `replayTo()` or `replayWithTiming()` has run once and set the initial size. Both replays accept `targetSeq` -1 for an empty Event Log. A replay over zero events sets `lastAppliedSeq` to `targetSeq` [backend.ts:GhosttyWebBackend.snapshot]. - `missingGlyphs` lists a code point only when a bundled face that lacks it drew it. Chromium draws a cluster with one font: the first stack font that covers all of it, or else the first font of the stack, which is bundled in both built-in profiles and then draws `.notdef` boxes. CDP names the face that drew a probe by its PostScript name, so a family alias or several faces under one family resolve to the right cmap. When one bundled face drew the whole grapheme, every code point that face lacks is listed. A probe with no reported font is judged against every face of the first family. When the grapheme is split over several fonts, each uncovered code point is judged by its own probe, and a mark's probe on its base is compared with a probe of the base alone. Glyphs that a host font draws are not listed. The check runs after the PNG is written, removes its probe spans, and caches each probe text's result for the page, clearing the cache when it would pass `MAX_CACHED_GLYPH_PROBES` (16384) entries. It probes at most `MAX_GLYPH_PROBES` texts and lists at most `MAX_MISSING_GLYPH_CELLS` cells (4096 each). `probeUsedFonts` sends its CDP queries in batches of 256 and stops starting batches after `GLYPH_PROBE_BUDGET_MS` (1000 ms). The result sets `missingGlyphsTruncated` when any of these stops the check early. Family names match without regard to case. A web font whose PostScript name the profile does not bundle fails the check. A failed check is logged at debug level and leaves the field out, so the screenshot still succeeds. `replayWithTiming()` records the viewport graphemes in the page after each output batch and each resize (at most `MAX_CACHED_GLYPH_PROBES`), and the final frame hold judges them once, with the hold as the probe budget, so the check does not lengthen the video. `recordedMissingGlyphs()` returns that result. A failed note or check leaves the record export fields out. A profile whose font stack does not start with a bundled font gets no `missingGlyphs` field [missingGlyphs.ts:probeUsedFonts] [backend.ts:GhosttyWebBackend.screenshot]. - `screenshot()` requires an absolute `outputPath` whose parent directory exists. It checks that the output is a non-empty PNG with a valid signature and an `IHDR` first chunk [backend.ts:GhosttyWebBackend.screenshot]. From 898cda96427b31608768859657c8189d9fc46193 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 07:54:40 +0000 Subject: [PATCH 06/28] docs: keep the progress bullet clear of the batching change --- src/renderer/ghosttyWeb/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/renderer/ghosttyWeb/README.md b/src/renderer/ghosttyWeb/README.md index 3e483a3c..e3b4bf39 100644 --- a/src/renderer/ghosttyWeb/README.md +++ b/src/renderer/ghosttyWeb/README.md @@ -29,8 +29,8 @@ This directory holds the `ghostty-web` reference renderer. `GhosttyWebBackend` s - The constructor asserts a non-empty `sessionId`, `profile.name` and `profile.fontFamily`, a positive `fontSize`, and hex colors. With `videoOptions`, `outputDir` must be absolute and the size must use positive integers [backend.ts:GhosttyWebBackend.constructor]. - Replay cannot rewind: `targetSeq` must be >= the last applied seq. The initial dimensions cannot change after the first replay. `input.sessionId` must match the backend [backend.ts:GhosttyWebBackend.replayTo]. - `replayWithTiming` asserts strictly increasing seqs and non-decreasing timestamps [backend.ts:GhosttyWebBackend.replayWithTiming]. -- `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` flushes once per run of output events [backend.ts:GhosttyWebBackend.replayWithTiming]. - `replayWithTiming` calls the optional `onProgress` with a `ReplayProgress` in two places. Before each wait on the schedule, `scheduledWaitMs` is the length of that wait. After each bridge batch of a flush (and after `notePaintedGraphemes`), `scheduledWaitMs` is 0. `seq` is the last event that the replay had reached. `generateWebmExport` uses these reports to tell a stall from a wait [backend.ts:GhosttyWebBackend.replayWithTiming]. +- `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` flushes once per run of output events [backend.ts:GhosttyWebBackend.replayWithTiming]. - `snapshot()` and `screenshot()` throw until `replayTo()` or `replayWithTiming()` has run once and set the initial size. Both replays accept `targetSeq` -1 for an empty Event Log. A replay over zero events sets `lastAppliedSeq` to `targetSeq` [backend.ts:GhosttyWebBackend.snapshot]. - `missingGlyphs` lists a code point only when a bundled face that lacks it drew it. Chromium draws a cluster with one font: the first stack font that covers all of it, or else the first font of the stack, which is bundled in both built-in profiles and then draws `.notdef` boxes. CDP names the face that drew a probe by its PostScript name, so a family alias or several faces under one family resolve to the right cmap. When one bundled face drew the whole grapheme, every code point that face lacks is listed. A probe with no reported font is judged against every face of the first family. When the grapheme is split over several fonts, each uncovered code point is judged by its own probe, and a mark's probe on its base is compared with a probe of the base alone. Glyphs that a host font draws are not listed. The check runs after the PNG is written, removes its probe spans, and caches each probe text's result for the page, clearing the cache when it would pass `MAX_CACHED_GLYPH_PROBES` (16384) entries. It probes at most `MAX_GLYPH_PROBES` texts and lists at most `MAX_MISSING_GLYPH_CELLS` cells (4096 each). `probeUsedFonts` sends its CDP queries in batches of 256 and stops starting batches after `GLYPH_PROBE_BUDGET_MS` (1000 ms). The result sets `missingGlyphsTruncated` when any of these stops the check early. Family names match without regard to case. A web font whose PostScript name the profile does not bundle fails the check. A failed check is logged at debug level and leaves the field out, so the screenshot still succeeds. `replayWithTiming()` records the viewport graphemes in the page after each output batch and each resize (at most `MAX_CACHED_GLYPH_PROBES`), and the final frame hold judges them once, with the hold as the probe budget, so the check does not lengthen the video. `recordedMissingGlyphs()` returns that result. A failed note or check leaves the record export fields out. A profile whose font stack does not start with a bundled font gets no `missingGlyphs` field [missingGlyphs.ts:probeUsedFonts] [backend.ts:GhosttyWebBackend.screenshot]. - `screenshot()` requires an absolute `outputPath` whose parent directory exists. It checks that the output is a non-empty PNG with a valid signature and an `IHDR` first chunk [backend.ts:GhosttyWebBackend.screenshot]. From ccd711977e7268575c97e50fde19f5dd07790868 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 07:55:56 +0000 Subject: [PATCH 07/28] test: keep a recording going through several waits that pass the longest timer together --- test/unit/export/webm.test.ts | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/test/unit/export/webm.test.ts b/test/unit/export/webm.test.ts index 23b2285e..3b33c174 100644 --- a/test/unit/export/webm.test.ts +++ b/test/unit/export/webm.test.ts @@ -668,6 +668,22 @@ describe('generateWebmExport stall watchdog', () => { expect(mockBackend.finalizeVideo).toHaveBeenCalledTimes(1); }); + it('keeps going through several waits that together pass the longest timer', async () => { + const { outcome, mockBackend } = await startExport(async (onProgress) => { + for (let seq = 0; seq < 3; seq += 1) { + onProgress({ seq, scheduledWaitMs: 10 * DAY_MS }); + await sleep(10 * DAY_MS); + } + return createReplayState(); + }); + + await vi.advanceTimersByTimeAsync(30 * DAY_MS); + const { error } = await outcome; + + expect(error).toBeUndefined(); + expect(mockBackend.finalizeVideo).toHaveBeenCalledTimes(1); + }); + it('starts no timer that runs longer than a timer can, for a wait of 30 days', async () => { const setTimeoutSpy = vi.spyOn(globalThis, 'setTimeout'); const { outcome, mockBackend } = await startExport(async (onProgress) => { From 3407142c950f98e863b95bafb0b586c3cc6e445e Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 08:11:28 +0000 Subject: [PATCH 08/28] docs: state the measured bridge call time next to the stall window --- src/export/webm.ts | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/src/export/webm.ts b/src/export/webm.ts index a5ae918a..9c105034 100644 --- a/src/export/webm.ts +++ b/src/export/webm.ts @@ -26,10 +26,11 @@ import { resolveProfile } from '../renderer/profiles.js'; import type { RenderProfileConfig, ReplayState } from '../renderer/types.js'; import { invariant, unreachable } from '../util/assert.js'; -// The replay fails when the renderer makes no progress for this long. One -// bridge call writes at most 1000 output chunks, which takes seconds. The -// window is the old fixed limit on the whole replay, so an export that fit -// before still fits. Boot and finalize run outside the watchdog. +// The replay fails when the renderer makes no progress for this long. A bridge +// call writes at most 1000 output chunks, and the longest one that a measured +// export made took 22 s on a loaded host. The window is the old fixed limit on +// the whole replay, so an export that fit before still fits. Boot and finalize +// run outside the watchdog. const REPLAY_STALL_WINDOW_MS = 5 * 60 * 1000; // Node runs a timer with a longer delay after 1 ms. const MAX_TIMER_DELAY_MS = 2 ** 31 - 1; From 3a8c8d9e1d1af993ef168a5a861f191c189cd1f9 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 09:45:51 +0000 Subject: [PATCH 09/28] fix: keep max-speed WebM frames per moment and paint each ghostty-web flush once Max-speed flushed once per run of output events, and a flush is one frame since the batched write. A WebM of an output-only recording then showed only its final state. Max-speed now flushes every frame hold (16 ms) of recorded time and holds the frame only after the last flush of a run, so it is never slower than one frame per output event. A flush of more than 1000 chunks takes several page calls. ghostty-web paints on every animation frame, so a state between two calls reached the video. The page now holds the chunks of a flush until the call that ends it and writes them in one task. Tests pin flushes of exactly 1000 and 2000 chunks and read the screencast frames. --- docs/USAGE.md | 2 + src/renderer/ghosttyWeb/README.md | 6 +- src/renderer/ghosttyWeb/backend.ts | 36 ++-- .../ghosttyWeb/embeddedHarnessHtml.ts | 53 ++++-- test/integration/renderer-backend.test.ts | 109 ++++++++++- test/unit/renderer/ghosttyWebBackend.test.ts | 170 +++++++++++++++++- 6 files changed, 332 insertions(+), 44 deletions(-) diff --git a/docs/USAGE.md b/docs/USAGE.md index e3d737c0..17e83909 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -294,6 +294,8 @@ agent-tty record export --format webm --json | jq -c '.result.missi WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. +A frame of the video shows the screen after all the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event. `max-speed` removes the idle time but gives each 16 ms of recorded output its own frame, so a progress bar that updates every 30 ms shows its steps and not only its last one. The video has 25 frames per second, so it can still skip a state that lasts less than one frame (40 ms). + SVG export renders styled grid frames from the event log through the native `libghostty-vt` backend with no browser and no ffmpeg, so it requires the optional `@coder/libghostty-vt-node` package (there is no `ghostty-web` fallback). The output is deterministic: exporting the same session twice produces byte-identical, diffable SVG. `--format svg` writes a still image of the final screen and ignores event timing, so it also works on a log whose clock stepped backward (`--animate` refuses such a log and names the event where the clock stepped back); add `--animate` for an animated SVG of de-duplicated frames replayed with recorded event-log timing (`--timing` is not supported with SVG). Animated SVG capture retains every distinct frame's styled grid in memory, so it is bounded by a total-cell budget (the V8 heap limit divided by 400 bytes, at most 20M cells or roughly 10,000 distinct 80x24 frames, adapting to terminal size); recordings that exceed it fail with a clear error suggesting a still SVG or WebM export instead. The SVG text is also bounded (the V8 heap limit divided by 8 bytes, at most 2^28 characters or about 268 MB of mostly ASCII text), so a dense, colorful animation that fits the cell budget can still fail with the same kind of error. diff --git a/src/renderer/ghosttyWeb/README.md b/src/renderer/ghosttyWeb/README.md index a48cc672..f98617e6 100644 --- a/src/renderer/ghosttyWeb/README.md +++ b/src/renderer/ghosttyWeb/README.md @@ -29,8 +29,8 @@ This directory holds the `ghostty-web` reference renderer. `GhosttyWebBackend` s - The constructor asserts a non-empty `sessionId`, `profile.name` and `profile.fontFamily`, a positive `fontSize`, and hex colors. With `videoOptions`, `outputDir` must be absolute and the size must use positive integers [backend.ts:GhosttyWebBackend.constructor]. - Replay cannot rewind: `targetSeq` must be >= the last applied seq. The initial dimensions cannot change after the first replay. `input.sessionId` must match the backend [backend.ts:GhosttyWebBackend.replayTo]. - `replayWithTiming` asserts strictly increasing seqs and non-decreasing timestamps [backend.ts:GhosttyWebBackend.replayWithTiming]. -- `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` flushes once per run of output events [backend.ts:GhosttyWebBackend.replayWithTiming]. -- `writeBatch(chunks, waitForFrame)` writes every chunk of a call at once. ghostty-web calls a write callback one animation frame (about 17 ms) after the write, and waiting on every chunk (`write`) cost one frame per output event. With `waitForFrame` the call resolves after one frame, and the backend sets it only on the last call of a flush, so a flush of any size waits for one frame. In `replayWithTiming`, the chunks of one flush share one video frame [embeddedHarnessHtml.ts:EMBEDDED_HARNESS_HTML]. +- `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` has no schedule to fall behind, so it flushes once the next event is at least one frame hold of recorded time after the batch's first output. Each moment that a viewer could tell apart gets a frame, and a replay costs at most one frame per output event. Events with the same timestamp share a flush when the hold is above 0 [backend.ts:GhosttyWebBackend.replayWithTiming]. +- `writeBatch(chunks, endsFlush)` takes the chunks of one bridge call. A flush that has more than 1000 chunks needs several calls, and ghostty-web paints on every animation frame, so the page holds the chunks of a call that does not end the flush. The call that ends it (`endsFlush`, set by the backend on the last call only) writes every held chunk in one task, so no frame paints a state between two calls, and then resolves after one frame. ghostty-web calls a write callback one animation frame (about 17 ms) after the write, and waiting on every chunk (`write`) cost one frame per output event. `write()` and `resize()` throw while chunks are held. In `replayWithTiming`, the chunks of one flush share one video frame [embeddedHarnessHtml.ts:EMBEDDED_HARNESS_HTML]. - `snapshot()` and `screenshot()` throw until `replayTo()` or `replayWithTiming()` has run once and set the initial size. Both replays accept `targetSeq` -1 for an empty Event Log. A replay over zero events sets `lastAppliedSeq` to `targetSeq` [backend.ts:GhosttyWebBackend.snapshot]. - `missingGlyphs` lists a code point only when a bundled face that lacks it drew it. Chromium draws a cluster with one font: the first stack font that covers all of it, or else the first font of the stack, which is bundled in both built-in profiles and then draws `.notdef` boxes. CDP names the face that drew a probe by its PostScript name, so a family alias or several faces under one family resolve to the right cmap. When one bundled face drew the whole grapheme, every code point that face lacks is listed. A probe with no reported font is judged against every face of the first family. When the grapheme is split over several fonts, each uncovered code point is judged by its own probe, and a mark's probe on its base is compared with a probe of the base alone. Glyphs that a host font draws are not listed. The check runs after the PNG is written, removes its probe spans, and caches each probe text's result for the page, clearing the cache when it would pass `MAX_CACHED_GLYPH_PROBES` (16384) entries. It probes at most `MAX_GLYPH_PROBES` texts and lists at most `MAX_MISSING_GLYPH_CELLS` cells (4096 each). `probeUsedFonts` sends its CDP queries in batches of 256 and stops starting batches after `GLYPH_PROBE_BUDGET_MS` (1000 ms). The result sets `missingGlyphsTruncated` when any of these stops the check early. Family names match without regard to case. A web font whose PostScript name the profile does not bundle fails the check. A failed check is logged at debug level and leaves the field out, so the screenshot still succeeds. `replayWithTiming()` records the viewport graphemes in the page after each output batch and each resize (at most `MAX_CACHED_GLYPH_PROBES`), and the final frame hold judges them once, with the hold as the probe budget, so the check does not lengthen the video. `recordedMissingGlyphs()` returns that result. A failed note or check leaves the record export fields out. A profile whose font stack does not start with a bundled font gets no `missingGlyphs` field [missingGlyphs.ts:probeUsedFonts] [backend.ts:GhosttyWebBackend.screenshot]. - `screenshot()` requires an absolute `outputPath` whose parent directory exists. It checks that the output is a non-empty PNG with a valid signature and an `IHDR` first chunk [backend.ts:GhosttyWebBackend.screenshot]. @@ -50,7 +50,7 @@ This directory holds the `ghostty-web` reference renderer. `GhosttyWebBackend` s - Reads the `ghostty-web` package files (`ghostty-web.js`, its browser-external helper and `ghostty-vt.wasm`) with `fs/promises` once per process [backend.ts:getServedAssets]. - Starts a `node:http` server on `127.0.0.1` with an ephemeral port. It serves the harness, the `ghostty-web` assets and the bundled fonts for `GET` and `HEAD` [backend.ts:GhosttyWebBackend.startServer]. - Calls `ensurePlaywrightBrowsersPath()`, which can set `process.env.PLAYWRIGHT_BROWSERS_PATH`. Then it launches headless Chromium with `chromium.launch` [backend.ts:GhosttyWebBackend.bootInternal]. -- Calls `page.evaluate` on `window.__agentTty` to write output, resize and read snapshots. One `writeBatch` call carries up to 1000 output chunks [backend.ts:GhosttyWebBackend.writeBatchBridge]. +- Calls `page.evaluate` on `window.__agentTty` to write output, resize and read snapshots. One `writeBatch` call carries up to 1000 output chunks, and the page holds them until the last call of the flush [backend.ts:GhosttyWebBackend.writeBatchBridge]. - `screenshot()` writes the PNG through Playwright `locator('#terminal').screenshot`, then reads it back to hash it [backend.ts:GhosttyWebBackend.screenshot]. - With `videoOptions`, Playwright records into `outputDir`. `finalizeVideo()` closes the page and context and calls `video.saveAs(outputPath)` [backend.ts:GhosttyWebBackend.finalizeVideo]. - `dispose()` closes the page, context, browser and HTTP server [backend.ts:GhosttyWebBackend.dispose]. diff --git a/src/renderer/ghosttyWeb/backend.ts b/src/renderer/ghosttyWeb/backend.ts index e872106d..1321b8cf 100644 --- a/src/renderer/ghosttyWeb/backend.ts +++ b/src/renderer/ghosttyWeb/backend.ts @@ -81,10 +81,7 @@ interface GhosttyServedAsset extends GhosttyRequestAsset { interface GhosttyBrowserBridge { isReady?: () => boolean; write?: (data: string) => Promise | void; - writeBatch?: ( - chunks: string[], - waitForFrame?: boolean, - ) => Promise | void; + writeBatch?: (chunks: string[], endsFlush?: boolean) => Promise | void; resize?: (cols: number, rows: number) => Promise | void; setCursorVisible?: (visible: boolean) => Promise | void; getSnapshot?: (options?: SnapshotOptions) => GhosttyHarnessSnapshot; @@ -663,7 +660,7 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { let scheduleStartedAtMs = 0; let scheduledElapsedMs = 0; - const flushOutputBatch = async (): Promise => { + const flushOutputBatch = async (holdFrame = true): Promise => { if (pendingOutputChunks.length === 0) { return; } @@ -673,7 +670,9 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { ); pendingOutputChunks = []; pendingSinceTimestampMs = null; - await waitForDelay(resolvedMinFrameHoldMs); + if (holdFrame) { + await waitForDelay(resolvedMinFrameHoldMs); + } }; for (const event of input.events) { @@ -710,16 +709,20 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { // Output shows once its batch is a frame hold old, unless the replay // runs behind the schedule: then batches grow until it catches up, so // slow flushes lower the frame rate instead of stretching the video. - // Max-speed keeps one batch per run of output events, so its frame - // holds cannot outlast the export timeout. + // Max-speed has no schedule to fall behind. It flushes every frame + // hold of recorded time, so each moment that a viewer could tell apart + // gets a frame, and a flush costs at most one frame per output event. + // The frame wait of such a flush is already the frame of its moment, + // so max-speed holds only after the last flush of a run of output: a + // hold after every flush would export 30 ms steps slower than they ran. if ( - timing.mode !== 'max-speed' && pendingSinceTimestampMs !== null && eventTimestampMs - pendingSinceTimestampMs >= resolvedMinFrameHoldMs && - performance.now() - scheduleStartedAtMs <= scheduledElapsedMs + (timing.mode === 'max-speed' || + performance.now() - scheduleStartedAtMs <= scheduledElapsedMs) ) { - await flushOutputBatch(); + await flushOutputBatch(timing.mode !== 'max-speed'); } // Waiting for the event's place on the schedule lets frame holds and // bridge calls use up time that the recording spent idle. @@ -1696,7 +1699,8 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { batch.length <= MAX_REPLAY_BATCH_SIZE, 'flushOutputBatch batch size must respect MAX_REPLAY_BATCH_SIZE', ); - // The page waits for a frame only after the last batch of the flush. + // The page holds the batches of a flush until the last one. Then it + // writes them all and waits for one frame. await this.writeBatchBridge( page, batch, @@ -1786,7 +1790,7 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { private async writeBatchBridge( page: Page, dataChunks: string[], - waitForFrame = true, + endsFlush = true, ): Promise { invariant( dataChunks.length > 0, @@ -1801,15 +1805,15 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { } await page.evaluate( - async ([chunks, shouldWaitForFrame]: readonly [string[], boolean]) => { + async ([chunks, batchEndsFlush]: readonly [string[], boolean]) => { const bridge = (globalThis as GhosttyBrowserGlobal).__agentTty; if (bridge === undefined || typeof bridge.writeBatch !== 'function') { throw new Error('ghostty-web bridge writeBatch() is unavailable'); } - await bridge.writeBatch(chunks, shouldWaitForFrame); + await bridge.writeBatch(chunks, batchEndsFlush); }, - [dataChunks, waitForFrame] as const, + [dataChunks, endsFlush] as const, ); } diff --git a/src/renderer/ghosttyWeb/embeddedHarnessHtml.ts b/src/renderer/ghosttyWeb/embeddedHarnessHtml.ts index adb0c9e5..84299ee5 100644 --- a/src/renderer/ghosttyWeb/embeddedHarnessHtml.ts +++ b/src/renderer/ghosttyWeb/embeddedHarnessHtml.ts @@ -262,6 +262,8 @@ export const EMBEDDED_HARNESS_HTML = ` const state = { errorMessage: null, + // Chunks of a flush that writeBatch() holds until the call that ends it. + heldChunks: [], ready: false, terminal: null, }; @@ -645,43 +647,54 @@ export const EMBEDDED_HARNESS_HTML = ` async write(data) { const terminal = getReadyTerminal(); assertStringValue(data, 'write() data must be a string'); + invariant( + state.heldChunks.length === 0, + 'write() cannot run while writeBatch() holds the chunks of a flush', + ); await new Promise((resolve) => { terminal.write(data, resolve); }); updateDocumentState(); }, - async writeBatch(chunks, waitForFrame = true) { + async writeBatch(chunks, endsFlush = true) { const terminal = getReadyTerminal(); invariant( Array.isArray(chunks) && chunks.length > 0, 'writeBatch() chunks must be a non-empty array', ); invariant( - typeof waitForFrame === 'boolean', - 'writeBatch() waitForFrame must be a boolean', + typeof endsFlush === 'boolean', + 'writeBatch() endsFlush must be a boolean', ); for (const chunk of chunks) { assertStringValue(chunk, 'writeBatch() chunks must be strings'); } - // The wasm write is synchronous, but ghostty-web calls a write - // callback one animation frame later. Waiting on every chunk made a - // replay cost one frame per event. The callback on the last chunk - // keeps the contract of write(): the call resolves after a frame. - // A caller that writes a run of batches asks for that frame only - // after the last one. - const lastIndex = chunks.length - 1; - for (let index = 0; index < lastIndex; index += 1) { - terminal.write(chunks[index]); + // A flush shows as one frame. A flush that needs several calls holds + // its chunks here until the call that ends it, so no frame paints + // the screen between two calls: ghostty-web renders on every animation + // frame, and a video records every paint. The call that ends the + // flush writes all chunks in one task. The wasm write is synchronous, + // but ghostty-web calls a write callback one animation frame later, + // so waiting on every chunk made a replay cost one frame per event. + // The callback on the last chunk makes the call resolve after a frame. + for (const chunk of chunks) { + state.heldChunks.push(chunk); + } + if (!endsFlush) { + return; } - if (waitForFrame) { - await new Promise((resolve) => { - terminal.write(chunks[lastIndex], resolve); - }); - } else { - terminal.write(chunks[lastIndex]); + + const flush = state.heldChunks; + state.heldChunks = []; + const lastIndex = flush.length - 1; + for (let index = 0; index < lastIndex; index += 1) { + terminal.write(flush[index]); } + await new Promise((resolve) => { + terminal.write(flush[lastIndex], resolve); + }); updateDocumentState(); }, getSnapshot(options) { @@ -700,6 +713,10 @@ export const EMBEDDED_HARNESS_HTML = ` }, resize(cols, rows) { const terminal = getReadyTerminal(); + invariant( + state.heldChunks.length === 0, + 'resize() cannot run while writeBatch() holds the chunks of a flush', + ); assertPositiveInteger( cols, 'resize() cols must be a positive integer', diff --git a/test/integration/renderer-backend.test.ts b/test/integration/renderer-backend.test.ts index 471b2191..7d76cd88 100644 --- a/test/integration/renderer-backend.test.ts +++ b/test/integration/renderer-backend.test.ts @@ -1,3 +1,4 @@ +import { createHash } from 'node:crypto'; import { mkdtemp, readFile, realpath, rm, stat } from 'node:fs/promises'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; @@ -769,7 +770,7 @@ describe('GhosttyWebBackend integration', { timeout: 120_000 }, () => { (renderer: GhosttyWebBackend, input: ReplayInput) => renderer.replayWithTiming(input, { mode: 'max-speed', - minFrameHoldMs: 0, + minFrameHoldMs: 16, finalFrameHoldMs: 0, }), ], @@ -828,6 +829,112 @@ describe('GhosttyWebBackend integration', { timeout: 120_000 }, () => { }, ); + // ghostty-web paints on every animation frame, and a WebM records every + // paint. A flush of more than 1000 chunks takes several page calls, so a + // state that exists between two calls can reach the video. + it.each([ + [ + 'replayTo', + (renderer: GhosttyWebBackend, input: ReplayInput) => + renderer.replayTo(input), + ], + [ + 'replayWithTiming', + (renderer: GhosttyWebBackend, input: ReplayInput) => + renderer.replayWithTiming(input, { + mode: 'max-speed', + minFrameHoldMs: 16, + finalFrameHoldMs: 200, + }), + ], + ])( + '%s paints a flush of several page calls as one frame', + async (_name, replay) => { + await backend.boot(); + const { page } = backend as unknown as { page: Page }; + // The screencast is the source of the frames that Playwright encodes into + // a video: it sends a frame for every paint. + const screencast = await page.context().newCDPSession(page); + const frames: string[] = []; + screencast.on('Page.screencastFrame', (frame) => { + const hash = createHash('sha256').update(frame.data).digest('hex'); + if (frames.at(-1) !== hash) { + frames.push(hash); + } + void screencast + .send('Page.screencastFrameAck', { sessionId: frame.sessionId }) + .catch(() => {}); + }); + await screencast.send('Page.startScreencast', { + format: 'png', + everyNthFrame: 1, + }); + + // Each bridge call of 1000 chunks fills the screen with its own colour + // (red, green, blue), so a paint between two calls shows a colour that is + // not the last. A pause after each call stands for a slow host. + const backgroundFor = (seq: number): number => + [41, 42, 44][Math.floor(seq / 1_000)] ?? 44; + const input = createReplayInput( + Array.from({ length: 2_501 }, (_, seq) => ({ + seq, + ts: timestampFor(0), + type: 'output' as const, + payload: { data: `\u001b[${String(backgroundFor(seq))}m\u001b[2J` }, + })), + ); + const writeBatchBridge = ( + backend as unknown as { + writeBatchBridge: (...args: unknown[]) => Promise; + } + ).writeBatchBridge.bind(backend); + Object.assign(backend as object, { + writeBatchBridge: async (...args: unknown[]) => { + await writeBatchBridge(...args); + await new Promise((resolve) => setTimeout(resolve, 100)); + }, + }); + // Wait for the first frame, so that it is the screen before the replay. + await expect + .poll(() => frames.length, { timeout: 10_000 }) + .toBeGreaterThan(0); + await replay(backend, input); + // Let every frame that the replay caused reach the listener. + await new Promise((resolve) => setTimeout(resolve, 500)); + await screencast.send('Page.stopScreencast'); + + // The frame before the replay, then the frame of the last colour. + expect(frames).toHaveLength(2); + }, + ); + + it('holds the chunks of a flush until the call that ends it', async () => { + await backend.boot(); + const { page } = backend as unknown as { page: Page }; + const visibleFirstRow = (): Promise => + page.evaluate( + `window.__agentTty.getVisibleText().split('\\n')[0]`, + ) as Promise; + await page.evaluate('window.__agentTty.resize(80, 24)'); + + await page.evaluate(`window.__agentTty.writeBatch(['ab'], false)`); + await page.evaluate(`window.__agentTty.writeBatch(['cd'], false)`); + expect(await visibleFirstRow()).toBe(''); + + // A direct write or a resize would move the held chunks behind its own + // output, so the page refuses both. + await expect(page.evaluate(`window.__agentTty.write('x')`)).rejects.toThrow( + 'holds the chunks of a flush', + ); + await expect( + page.evaluate(`window.__agentTty.resize(100, 30)`), + ).rejects.toThrow('holds the chunks of a flush'); + expect(await visibleFirstRow()).toBe(''); + + await page.evaluate(`window.__agentTty.writeBatch(['ef'], true)`); + expect(await visibleFirstRow()).toBe('abcdef'); + }); + it('allows dispose to be called more than once', async () => { await backend.boot(); diff --git a/test/unit/renderer/ghosttyWebBackend.test.ts b/test/unit/renderer/ghosttyWebBackend.test.ts index 86a90e8a..8f14e15a 100644 --- a/test/unit/renderer/ghosttyWebBackend.test.ts +++ b/test/unit/renderer/ghosttyWebBackend.test.ts @@ -487,7 +487,7 @@ describe('GhosttyWebBackend unit guards', () => { }; } - it('writes a bridge batch into the page in one call that waits for a frame', async () => { + it('writes a bridge batch into the page in one call that ends its flush', async () => { const backend = createBackend(); const chunks = Array.from({ length: 1000 }, (_, i) => `chunk-${String(i)}`); @@ -522,12 +522,12 @@ describe('GhosttyWebBackend unit guards', () => { (backend: GhosttyWebBackend, input: ReplayInput) => backend.replayWithTiming(input, { mode: 'max-speed', - minFrameHoldMs: 0, + minFrameHoldMs: 16, finalFrameHoldMs: 0, }), ], ])( - '%s calls the page once per 1000 output events, and waits for a frame after the last call', + '%s calls the page once per 1000 output events, and ends the flush with the last call', async (_name, replay) => { const backend = createBackend(); const input = outputReplayInput(2_501); @@ -560,6 +560,70 @@ describe('GhosttyWebBackend unit guards', () => { }, ); + // A bridge call holds at most 1000 chunks. The page writes a flush when the + // call that ends it arrives, so the flag must be true on exactly one call of + // a flush, the last: also when the chunk count is a multiple of 1000. + it.each([ + [1, [true]], + [999, [true]], + [1_000, [true]], + [1_001, [false, true]], + [1_999, [false, true]], + [2_000, [false, true]], + [2_001, [false, false, true]], + ])( + 'ends a flush of %i output chunks with its last bridge call only', + async (chunkCount, expectedEnds) => { + const backend = createBackend(); + + await withCountingBridge( + async ({ page, writeBatchSizes, frameWaits }) => { + Object.assign(backend as object, { + isBooted: true, + page, + resizeBridge: () => Promise.resolve(), + readHarnessSnapshot: () => + Promise.resolve(createHarnessSnapshotPayload({ cols: 80 })), + }); + + await backend.replayTo(outputReplayInput(chunkCount)); + + expect(frameWaits).toEqual(expectedEnds); + expect(writeBatchSizes.reduce((sum, size) => sum + size, 0)).toBe( + chunkCount, + ); + }, + ); + }, + ); + + it('ends the flush before a marker, so the next flush starts holding again', async () => { + const backend = createBackend(); + const input = outputReplayInput(3_001); + input.events[1_500] = { + seq: 1_500, + ts: '2026-03-19T12:00:00.000Z', + type: 'marker', + payload: { label: 'boundary' }, + }; + + await withCountingBridge(async ({ page, writeBatchSizes, frameWaits }) => { + Object.assign(backend as object, { + isBooted: true, + page, + resizeBridge: () => Promise.resolve(), + readHarnessSnapshot: () => + Promise.resolve(createHarnessSnapshotPayload({ cols: 80 })), + }); + + await backend.replayTo(input); + + // 1500 chunks, the marker, then 1500 chunks. + expect(writeBatchSizes).toEqual([1_000, 500, 1_000, 500]); + expect(frameWaits).toEqual([false, true, false, true]); + }); + }); + it('includes structured cell data only when snapshot includeCells is requested', async () => { const backend = createBackend(); const evaluate = vi.fn().mockResolvedValue( @@ -1400,15 +1464,109 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { ]); }); - it('keeps max-speed outputs in one flush so frame holds cannot outlast the export timeout', async () => { + it('flushes max-speed output every frame hold of recorded time, so each moment gets a frame', async () => { const flushes = await replayAndRecordFlushes( - [outputEvent(0, 0), outputEvent(1, 200), outputEvent(2, 400)], + [ + outputEvent(0, 0), + outputEvent(1, 5), + outputEvent(2, 10), + outputEvent(3, 16), + outputEvent(4, 20), + outputEvent(5, 40), + outputEvent(6, 41), + exitEvent(7, 500), + ], MAX_SPEED, ); + expect(flushes.map((flush) => flush.chunks)).toEqual([ + ['out0', 'out1', 'out2'], + ['out3', 'out4'], + ['out5', 'out6'], + ]); + }); + + it('keeps one max-speed flush per moment when the outputs are far apart', async () => { + const flushes = await replayAndRecordFlushes( + [ + outputEvent(0, 0), + outputEvent(1, 200), + outputEvent(2, 400), + exitEvent(3, 600), + ], + MAX_SPEED, + ); + + expect(flushes.map((flush) => flush.chunks)).toEqual([ + ['out0'], + ['out1'], + ['out2'], + ]); + }); + + // The frame wait of a flush is the frame of its moment. A hold after every + // flush of a run would make each 30 ms step cost 44 ms to export. + it('holds max-speed frames only after the last flush of a run of output', async () => { + const flushes = await replayAndRecordFlushes( + [ + outputEvent(0, 0), + outputEvent(1, 20), + outputEvent(2, 40), + exitEvent(3, 50), + outputEvent(4, 80), + ], + MAX_SPEED, + ); + + // out0 and out1 flush when the next output arrives, so no hold follows + // them. The exit ends the run: out2 flushes, and its hold delays out4. expect(flushes).toEqual([ - { atMs: FIRST_EVENT_AT_MS, chunks: ['out0', 'out1', 'out2'] }, + { atMs: FIRST_EVENT_AT_MS, chunks: ['out0'] }, + { atMs: FIRST_EVENT_AT_MS, chunks: ['out1'] }, + { atMs: FIRST_EVENT_AT_MS, chunks: ['out2'] }, + { atMs: FIRST_EVENT_AT_MS + 16, chunks: ['out4'] }, + ]); + }); + + it('does not merge max-speed moments when a flush costs more than a frame hold', async () => { + const events = Array.from({ length: 101 }, (_, index) => + outputEvent(index, index * 10), + ); + events.push(exitEvent(101, 1_000)); + const flushes = await replayAndRecordFlushes(events, MAX_SPEED, 30); + + // Recorded timing lets batches grow while the replay is behind its + // schedule. Max-speed has no schedule, so a slow flush must not hide a + // moment: every flush holds the outputs of one frame hold (two events). + expect(flushes.length).toBe(51); + expect(flushes.every((flush) => flush.chunks.length <= 2)).toBe(true); + expect(flushes.flatMap((flush) => flush.chunks)).toEqual( + events.slice(0, 101).map((event) => `out${String(event.seq)}`), + ); + }); + + it('keeps accelerated frames at least one frame hold of recorded time apart', async () => { + const accelerated: ReplayTimingOptions = { + mode: 'accelerated', + maxGapMs: 400, + minFrameHoldMs: 100, + finalFrameHoldMs: 1_000, + }; + const events = Array.from({ length: 41 }, (_, index) => + outputEvent(index, index * 30), + ); + events.push(exitEvent(41, 1_300)); + const flushes = await replayAndRecordFlushes(events, accelerated); + + // 30 ms steps, 100 ms holds: a flush takes the outputs of four steps. + expect(flushes.map((flush) => flush.chunks.length)).toEqual([ + 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 1, ]); + for (let index = 1; index < flushes.length; index += 1) { + const gapMs = + (flushes[index]?.atMs ?? 0) - (flushes[index - 1]?.atMs ?? 0); + expect(gapMs).toBeGreaterThanOrEqual(100); + } }); it('refreshes continuous output at least once per frame hold', async () => { From 4a65f4d4029d036eb000c24acdf1d37ad6f75caf Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 09:52:27 +0000 Subject: [PATCH 10/28] docs: keep the WebM frame explanation in the timing paragraph of USAGE.md --- docs/USAGE.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs/USAGE.md b/docs/USAGE.md index 17e83909..60eaa353 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -292,9 +292,7 @@ agent-tty record export --format webm --json | jq -c '.result.missi # [{"codePoint":"U+0378"}] ``` -WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. - -A frame of the video shows the screen after all the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event. `max-speed` removes the idle time but gives each 16 ms of recorded output its own frame, so a progress bar that updates every 30 ms shows its steps and not only its last one. The video has 25 frames per second, so it can still skip a state that lasts less than one frame (40 ms). +WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. A frame of the video shows the screen after all the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event. `max-speed` removes the idle time but gives each 16 ms of recorded output its own frame, so a progress bar that updates every 30 ms shows its steps and not only its last one. The video has 25 frames per second, so it can still skip a state that lasts less than one frame (40 ms). SVG export renders styled grid frames from the event log through the native `libghostty-vt` backend with no browser and no ffmpeg, so it requires the optional `@coder/libghostty-vt-node` package (there is no `ghostty-web` fallback). The output is deterministic: exporting the same session twice produces byte-identical, diffable SVG. `--format svg` writes a still image of the final screen and ignores event timing, so it also works on a log whose clock stepped backward (`--animate` refuses such a log and names the event where the clock stepped back); add `--animate` for an animated SVG of de-duplicated frames replayed with recorded event-log timing (`--timing` is not supported with SVG). From 9da052f692568d43c99d898ee3dd08a64e937b79 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 09:53:50 +0000 Subject: [PATCH 11/28] fix: keep events with one timestamp in one max-speed flush when the frame hold is 0 --- src/renderer/ghosttyWeb/backend.ts | 5 +++-- test/integration/renderer-backend.test.ts | 2 +- test/unit/renderer/ghosttyWebBackend.test.ts | 2 +- 3 files changed, 5 insertions(+), 4 deletions(-) diff --git a/src/renderer/ghosttyWeb/backend.ts b/src/renderer/ghosttyWeb/backend.ts index 1321b8cf..ae5262c2 100644 --- a/src/renderer/ghosttyWeb/backend.ts +++ b/src/renderer/ghosttyWeb/backend.ts @@ -706,7 +706,8 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { 'replay event timestamps must be ordered non-decreasingly', ); scheduledElapsedMs += computeInterEventDelay(interEventDelayMs); - // Output shows once its batch is a frame hold old, unless the replay + // Output shows once its batch is a frame hold old (and at least 1 ms: + // events with one timestamp are one moment), unless the replay // runs behind the schedule: then batches grow until it catches up, so // slow flushes lower the frame rate instead of stretching the video. // Max-speed has no schedule to fall behind. It flushes every frame @@ -718,7 +719,7 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { if ( pendingSinceTimestampMs !== null && eventTimestampMs - pendingSinceTimestampMs >= - resolvedMinFrameHoldMs && + Math.max(1, resolvedMinFrameHoldMs) && (timing.mode === 'max-speed' || performance.now() - scheduleStartedAtMs <= scheduledElapsedMs) ) { diff --git a/test/integration/renderer-backend.test.ts b/test/integration/renderer-backend.test.ts index 7d76cd88..fe956f17 100644 --- a/test/integration/renderer-backend.test.ts +++ b/test/integration/renderer-backend.test.ts @@ -770,7 +770,7 @@ describe('GhosttyWebBackend integration', { timeout: 120_000 }, () => { (renderer: GhosttyWebBackend, input: ReplayInput) => renderer.replayWithTiming(input, { mode: 'max-speed', - minFrameHoldMs: 16, + minFrameHoldMs: 0, finalFrameHoldMs: 0, }), ], diff --git a/test/unit/renderer/ghosttyWebBackend.test.ts b/test/unit/renderer/ghosttyWebBackend.test.ts index 8f14e15a..1593fe0d 100644 --- a/test/unit/renderer/ghosttyWebBackend.test.ts +++ b/test/unit/renderer/ghosttyWebBackend.test.ts @@ -522,7 +522,7 @@ describe('GhosttyWebBackend unit guards', () => { (backend: GhosttyWebBackend, input: ReplayInput) => backend.replayWithTiming(input, { mode: 'max-speed', - minFrameHoldMs: 16, + minFrameHoldMs: 0, finalFrameHoldMs: 0, }), ], From 060eed1df6e3a42d82157b8e8df7ec58267d8b02 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 10:24:17 +0000 Subject: [PATCH 12/28] test: wait for the last screencast frame before counting frames --- test/integration/renderer-backend.test.ts | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/test/integration/renderer-backend.test.ts b/test/integration/renderer-backend.test.ts index fe956f17..6d077211 100644 --- a/test/integration/renderer-backend.test.ts +++ b/test/integration/renderer-backend.test.ts @@ -899,7 +899,11 @@ describe('GhosttyWebBackend integration', { timeout: 120_000 }, () => { .poll(() => frames.length, { timeout: 10_000 }) .toBeGreaterThan(0); await replay(backend, input); - // Let every frame that the replay caused reach the listener. + // The last colour must reach the listener. Then wait a little more, so + // that a frame between two calls would have arrived too. + await expect + .poll(() => frames.length, { timeout: 10_000 }) + .toBeGreaterThan(1); await new Promise((resolve) => setTimeout(resolve, 500)); await screencast.send('Page.stopScreencast'); From 078fe21b79b35b4d1e8fce6498b82264cf8b33f4 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 13:55:00 +0000 Subject: [PATCH 13/28] perf: note painted graphemes in the page call of a flush, bound max-speed frames, drop held chunks after a failed flush --- src/renderer/ghosttyWeb/backend.ts | 222 +++++++++++----- .../ghosttyWeb/embeddedHarnessHtml.ts | 54 +++- test/integration/renderer-backend.test.ts | 116 +++++++++ test/unit/renderer/ghosttyWebBackend.test.ts | 237 ++++++++++++++++-- 4 files changed, 545 insertions(+), 84 deletions(-) diff --git a/src/renderer/ghosttyWeb/backend.ts b/src/renderer/ghosttyWeb/backend.ts index ae5262c2..724fffdd 100644 --- a/src/renderer/ghosttyWeb/backend.ts +++ b/src/renderer/ghosttyWeb/backend.ts @@ -81,7 +81,13 @@ interface GhosttyServedAsset extends GhosttyRequestAsset { interface GhosttyBrowserBridge { isReady?: () => boolean; write?: (data: string) => Promise | void; - writeBatch?: (chunks: string[], endsFlush?: boolean) => Promise | void; + writeBatch?: ( + chunks: string[], + endsFlush?: boolean, + paintedCap?: number | null, + ) => Promise | string | null; + discardHeldChunks?: () => void; + notePaintedGraphemes?: (cap: number) => void; resize?: (cols: number, rows: number) => Promise | void; setCursorVisible?: (visible: boolean) => Promise | void; getSnapshot?: (options?: SnapshotOptions) => GhosttyHarnessSnapshot; @@ -131,12 +137,43 @@ const HTML_CONTENT_TYPE = 'text/html; charset=utf-8'; const WASM_CONTENT_TYPE = 'application/wasm'; const MAX_REPLAY_BATCH_SIZE = 1000; +/** + * A max-speed replay flushes at most about this many times, however long the + * recording is. Every flush waits for one animation frame (17 to 25 ms) and + * one page call, so a replay takes about 2 minutes at the limit. That stays + * below the 5 minutes that a WebM export allows its replay, even on a slow + * host. The video has 25 frames per second, so 3,000 flushes fill 2 minutes + * of video: a viewer cannot tell more states apart. + */ +const MAX_SPEED_FRAME_BUDGET = 3_000; const RAF_TIMEOUT_MS = 5_000; let servedAssetsPromise: Promise< ReadonlyMap > | null = null; +/** + * Recorded time between the first and the last event that a replay applies: + * the seqs above `afterSeq`, up to the target. It is 0 when a timestamp does + * not parse. The replay loop rejects such a timestamp itself. + */ +function recordedSpanMs(input: ReplayInput, afterSeq: number): number { + let firstMs: number | undefined; + let lastMs = 0; + for (const event of input.events) { + if (event.seq <= afterSeq) { + continue; + } + if (event.seq > input.targetSeq) { + break; + } + lastMs = Date.parse(event.ts); + firstMs ??= lastMs; + } + const spanMs = lastMs - (firstMs ?? lastMs); + return Number.isFinite(spanMs) && spanMs > 0 ? spanMs : 0; +} + async function closeServer(server: Server): Promise { if (!server.listening) { return; @@ -649,6 +686,18 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { ); } + // Output shows once the next event is this far after the batch's first + // output. It is at least 1 ms: events with one timestamp are one moment. + // Max-speed has no schedule that bounds its replay time, so it also + // spreads a fixed number of flushes over the recorded span: a sparse + // recording then costs the same number of frames as a dense one. + const minSplitMs = Math.max( + 1, + resolvedMinFrameHoldMs, + timing.mode === 'max-speed' + ? recordedSpanMs(input, this.lastAppliedSeq) / MAX_SPEED_FRAME_BUDGET + : 0, + ); let outputEventCount = 0; let previousEventSeq = -1; let previousProcessedEventTimestampMs: number | null = null; @@ -665,9 +714,7 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { return; } - await this.flushOutputBatch(page, pendingOutputChunks, () => - this.notePaintedGraphemes(page), - ); + await this.flushOutputBatch(page, pendingOutputChunks, undefined, true); pendingOutputChunks = []; pendingSinceTimestampMs = null; if (holdFrame) { @@ -675,6 +722,16 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { } }; + // Events that change no pixel end a flush in recorded and accelerated + // mode, where the pause before them is part of the video. Max-speed has no + // pauses: a flush at each of them would cost one frame per event, and a + // viewer cannot tell apart what happens within one minimum split. + const flushBeforeNonVisualEvent = async (): Promise => { + if (timing.mode !== 'max-speed') { + await flushOutputBatch(); + } + }; + for (const event of input.events) { assertNonNegativeInteger( event.seq, @@ -706,20 +763,18 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { 'replay event timestamps must be ordered non-decreasingly', ); scheduledElapsedMs += computeInterEventDelay(interEventDelayMs); - // Output shows once its batch is a frame hold old (and at least 1 ms: - // events with one timestamp are one moment), unless the replay - // runs behind the schedule: then batches grow until it catches up, so - // slow flushes lower the frame rate instead of stretching the video. - // Max-speed has no schedule to fall behind. It flushes every frame - // hold of recorded time, so each moment that a viewer could tell apart - // gets a frame, and a flush costs at most one frame per output event. - // The frame wait of such a flush is already the frame of its moment, - // so max-speed holds only after the last flush of a run of output: a + // Output shows once its batch is a minimum split old, unless the + // replay runs behind the schedule: then batches grow until it catches + // up, so slow flushes lower the frame rate instead of stretching the + // video. Max-speed has no schedule to fall behind. It flushes every + // minimum split of recorded time, so each moment that a viewer could + // tell apart gets a frame, within the frame budget. The frame wait of + // such a flush is already the frame of its moment, so max-speed holds + // only after a flush that ends the replay or comes before a resize: a // hold after every flush would export 30 ms steps slower than they ran. if ( pendingSinceTimestampMs !== null && - eventTimestampMs - pendingSinceTimestampMs >= - Math.max(1, resolvedMinFrameHoldMs) && + eventTimestampMs - pendingSinceTimestampMs >= minSplitMs && (timing.mode === 'max-speed' || performance.now() - scheduleStartedAtMs <= scheduledElapsedMs) ) { @@ -764,7 +819,7 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { break; } case 'marker': { - await flushOutputBatch(); + await flushBeforeNonVisualEvent(); break; } case 'input_text': @@ -775,7 +830,7 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { case 'run_complete': case 'signal': case 'exit': { - await flushOutputBatch(); + await flushBeforeNonVisualEvent(); break; } default: { @@ -1007,40 +1062,39 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { return; } await page.evaluate((cap) => { - const scope = globalThis as GhosttyBrowserGlobal; - const bridge = scope.__agentTty; + const bridge = (globalThis as GhosttyBrowserGlobal).__agentTty; if ( bridge === undefined || - typeof bridge.getCellGraphemes !== 'function' + typeof bridge.notePaintedGraphemes !== 'function' ) { throw new Error( - 'ghostty-web bridge getCellGraphemes() is unavailable', + 'ghostty-web bridge notePaintedGraphemes() is unavailable', ); } - scope.__agentTtyPainted ??= { seen: new Set(), complete: true }; - const painted = scope.__agentTtyPainted; - for (const row of bridge.getCellGraphemes()) { - for (const grapheme of row) { - if (grapheme === '' || painted.seen.has(grapheme)) { - continue; - } - if (painted.seen.size < cap) { - painted.seen.add(grapheme); - } else { - painted.complete = false; - } - } - } + bridge.notePaintedGraphemes(cap); }, MAX_CACHED_GLYPH_PROBES); } catch (error) { - this.paintedNoteFailed = true; - this.logger.debug( - 'Missing-glyph note failed; the record export result omits missingGlyphs', + this.failPaintedNote( error instanceof Error ? error.message : String(error), ); } } + private failPaintedNote(message: string): void { + this.paintedNoteFailed = true; + this.logger.debug( + 'Missing-glyph note failed; the record export result omits missingGlyphs', + message, + ); + } + + /** The cap that a flush passes to the page to note its painted graphemes. */ + private paintedNoteCap(): number | null { + return this.paintedNoteFailed || this.glyphCheck() === null + ? null + : MAX_CACHED_GLYPH_PROBES; + } + public async recordedMissingGlyphs(): Promise { return ( this.recordedGlyphs ?? this.checkPaintedGlyphs(GLYPH_PROBE_BUDGET_MS) @@ -1676,38 +1730,70 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { ]); } + /** + * Writes the chunks of one flush into the page. With `notePainted`, the page + * also notes the graphemes on the screen once the flush is written, in the + * call that ends the flush. + */ private async flushOutputBatch( page: Page, dataChunks: string[], afterEachBatch?: () => Promise, + notePainted = false, ): Promise { invariant( dataChunks.length > 0, 'flushOutputBatch requires at least one data chunk', ); - for ( - let batchStart = 0; - batchStart < dataChunks.length; - batchStart += MAX_REPLAY_BATCH_SIZE - ) { - const batch = dataChunks.slice( - batchStart, - batchStart + MAX_REPLAY_BATCH_SIZE, - ); - invariant(batch.length > 0, 'flushOutputBatch batch must not be empty'); - invariant( - batch.length <= MAX_REPLAY_BATCH_SIZE, - 'flushOutputBatch batch size must respect MAX_REPLAY_BATCH_SIZE', - ); - // The page holds the batches of a flush until the last one. Then it - // writes them all and waits for one frame. - await this.writeBatchBridge( - page, - batch, - batchStart + MAX_REPLAY_BATCH_SIZE >= dataChunks.length, + try { + for ( + let batchStart = 0; + batchStart < dataChunks.length; + batchStart += MAX_REPLAY_BATCH_SIZE + ) { + const batch = dataChunks.slice( + batchStart, + batchStart + MAX_REPLAY_BATCH_SIZE, + ); + invariant(batch.length > 0, 'flushOutputBatch batch must not be empty'); + invariant( + batch.length <= MAX_REPLAY_BATCH_SIZE, + 'flushOutputBatch batch size must respect MAX_REPLAY_BATCH_SIZE', + ); + // The page holds the batches of a flush until the last one. Then it + // writes them all and waits for one frame. + const endsFlush = + batchStart + MAX_REPLAY_BATCH_SIZE >= dataChunks.length; + await this.writeBatchBridge( + page, + batch, + endsFlush, + endsFlush && notePainted ? this.paintedNoteCap() : null, + ); + await afterEachBatch?.(); + } + } catch (error) { + // The page still holds the chunks of the calls that did succeed. The + // backend stays in use after a failed replay, and a retry would meet + // them: its output would follow them, and its resize would be refused. + await this.discardHeldChunks(page); + throw error; + } + } + + private async discardHeldChunks(page: Page): Promise { + try { + await page.evaluate(() => { + const bridge = (globalThis as GhosttyBrowserGlobal).__agentTty; + bridge?.discardHeldChunks?.(); + }); + } catch (error) { + // A page that fails here is as broken as the call that failed first. + this.logger.debug( + 'Dropping the held chunks of a failed flush failed', + error instanceof Error ? error.message : String(error), ); - await afterEachBatch?.(); } } @@ -1792,6 +1878,7 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { page: Page, dataChunks: string[], endsFlush = true, + paintedCap: number | null = null, ): Promise { invariant( dataChunks.length > 0, @@ -1805,17 +1892,26 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { assertString(chunk, 'bridge batch write chunk must be a string'); } - await page.evaluate( - async ([chunks, batchEndsFlush]: readonly [string[], boolean]) => { + const noteError = await page.evaluate( + async ([chunks, batchEndsFlush, batchPaintedCap]: readonly [ + string[], + boolean, + number | null, + ]) => { const bridge = (globalThis as GhosttyBrowserGlobal).__agentTty; if (bridge === undefined || typeof bridge.writeBatch !== 'function') { throw new Error('ghostty-web bridge writeBatch() is unavailable'); } - await bridge.writeBatch(chunks, batchEndsFlush); + return await bridge.writeBatch(chunks, batchEndsFlush, batchPaintedCap); }, - [dataChunks, endsFlush] as const, + [dataChunks, endsFlush, paintedCap] as const, ); + // The page notes the painted graphemes in this call and reports a failed + // note by its message, so that the note cannot fail the flush. + if (typeof noteError === 'string') { + this.failPaintedNote(noteError); + } } private async writeBridge(page: Page, data: string): Promise { diff --git a/src/renderer/ghosttyWeb/embeddedHarnessHtml.ts b/src/renderer/ghosttyWeb/embeddedHarnessHtml.ts index 84299ee5..666368d2 100644 --- a/src/renderer/ghosttyWeb/embeddedHarnessHtml.ts +++ b/src/renderer/ghosttyWeb/embeddedHarnessHtml.ts @@ -572,6 +572,29 @@ export const EMBEDDED_HARNESS_HTML = ` return grid; } + // Adds the graphemes on the screen to the set that the missing-glyph + // check of a recorded replay probes. The cap bounds the set. + function notePaintedGraphemes(cap) { + assertPositiveInteger( + cap, + 'notePaintedGraphemes() cap must be a positive integer', + ); + window.__agentTtyPainted ??= { seen: new Set(), complete: true }; + const painted = window.__agentTtyPainted; + for (const row of getCellGraphemes()) { + for (const grapheme of row) { + if (grapheme === '' || painted.seen.has(grapheme)) { + continue; + } + if (painted.seen.size < cap) { + painted.seen.add(grapheme); + } else { + painted.complete = false; + } + } + } + } + function getSnapshotPayload(options) { invariant( options === undefined || @@ -657,7 +680,7 @@ export const EMBEDDED_HARNESS_HTML = ` }); updateDocumentState(); }, - async writeBatch(chunks, endsFlush = true) { + async writeBatch(chunks, endsFlush = true, paintedCap = null) { const terminal = getReadyTerminal(); invariant( Array.isArray(chunks) && chunks.length > 0, @@ -667,6 +690,10 @@ export const EMBEDDED_HARNESS_HTML = ` typeof endsFlush === 'boolean', 'writeBatch() endsFlush must be a boolean', ); + invariant( + paintedCap === null || Number.isInteger(paintedCap), + 'writeBatch() paintedCap must be null or an integer', + ); for (const chunk of chunks) { assertStringValue(chunk, 'writeBatch() chunks must be strings'); } @@ -683,7 +710,7 @@ export const EMBEDDED_HARNESS_HTML = ` state.heldChunks.push(chunk); } if (!endsFlush) { - return; + return null; } const flush = state.heldChunks; @@ -696,6 +723,29 @@ export const EMBEDDED_HARNESS_HTML = ` terminal.write(flush[lastIndex], resolve); }); updateDocumentState(); + + // The caller asks to note the painted graphemes of a recorded + // replay here, so a flush costs one page call and not two. A failed + // note must not fail the flush: the call returns its message, and + // the caller then drops the missing-glyph fields of the export. + if (paintedCap === null) { + return null; + } + try { + notePaintedGraphemes(paintedCap); + return null; + } catch (error) { + return error instanceof Error ? error.message : String(error); + } + }, + // A flush that failed in the middle leaves its earlier calls' chunks + // held. The caller drops them, so that a retry starts from the screen + // as the failed flush found it. + discardHeldChunks() { + state.heldChunks = []; + }, + notePaintedGraphemes(cap) { + notePaintedGraphemes(cap); }, getSnapshot(options) { return getSnapshotPayload(options); diff --git a/test/integration/renderer-backend.test.ts b/test/integration/renderer-backend.test.ts index 6d077211..c181af2d 100644 --- a/test/integration/renderer-backend.test.ts +++ b/test/integration/renderer-backend.test.ts @@ -939,6 +939,122 @@ describe('GhosttyWebBackend integration', { timeout: 120_000 }, () => { expect(await visibleFirstRow()).toBe('abcdef'); }); + // HostRendererManager keeps the backend after a failed replay and retries + // from the same seq. The page held the chunks of the calls before the one + // that failed, so a retry that started with a resize was refused for good. + it('runs a retry that starts with a resize after a page call failed in the middle of a flush', async () => { + await backend.boot(); + // One character per event, so a chunk that is lost, repeated or moved + // changes the text on the screen. 2501 characters fill 32 rows of 80 + // columns. + const characters = Array.from({ length: 2_501 }, (_, index) => + 'abcdefghijklmnopqrstuvwxyz0123456789'.charAt((index * 7) % 36), + ); + const outputs = (firstSeq: number): ReplayInput['events'] => + characters.map((character, index) => ({ + seq: firstSeq + index, + ts: timestampFor(0), + type: 'output' as const, + payload: { data: character }, + })); + const writeBatchBridge = ( + backend as unknown as { + writeBatchBridge: (...args: unknown[]) => Promise; + } + ).writeBatchBridge.bind(backend); + let pageCalls = 0; + Object.assign(backend as object, { + writeBatchBridge: async (...args: unknown[]) => { + pageCalls += 1; + if (pageCalls === 2) { + throw new Error('page call failed'); + } + await writeBatchBridge(...args); + }, + }); + + await expect( + backend.replayTo(createReplayInput(outputs(0))), + ).rejects.toThrow('page call failed'); + + Object.assign(backend as object, { writeBatchBridge }); + await backend.replayTo( + createReplayInput( + [ + { + seq: 0, + ts: timestampFor(0), + type: 'resize', + payload: { cols: 80, rows: 40 }, + }, + ...outputs(1), + ], + { initialRows: 24 }, + ), + ); + + const expectedRows = Array.from({ length: 32 }, (_, row) => + characters.slice(row * 80, (row + 1) * 80).join(''), + ); + const screenRows = (await backend.snapshot()).visibleLines.map( + (line) => line.text, + ); + expect(screenRows.slice(0, 32)).toEqual(expectedRows); + }); + + // A flush of 1001 chunks takes two page calls, and the glyph that it paints + // comes with its last chunk. The next flush erases it. + it('records the glyphs that a flush painted, in the page call that ends the flush', async () => { + await backend.boot(); + const { page } = backend as unknown as { page: Page }; + const pageFunctions: string[] = []; + const evaluate = page.evaluate.bind(page) as ( + ...args: unknown[] + ) => Promise; + Object.assign(page, { + evaluate: (pageFunction: unknown, ...rest: unknown[]) => { + pageFunctions.push(String(pageFunction)); + return evaluate(pageFunction, ...rest); + }, + }); + + await backend.replayWithTiming( + createReplayInput([ + ...Array.from({ length: 1_000 }, (_, seq) => ({ + seq, + ts: timestampFor(0), + type: 'output' as const, + payload: { data: 'x' }, + })), + { + seq: 1_000, + ts: timestampFor(0), + type: 'output' as const, + payload: { data: '\u0378' }, + }, + { + seq: 1_001, + ts: timestampFor(1), + type: 'output' as const, + payload: { data: '\u001b[2J\u001b[Hok' }, + }, + ]), + { mode: 'max-speed', minFrameHoldMs: 16, finalFrameHoldMs: 200 }, + ); + + await expect(backend.recordedMissingGlyphs()).resolves.toEqual({ + missingGlyphs: [{ codePoint: 'U+0378' }], + }); + // 1001 chunks take 2 calls, the next flush takes 1. None of them has a + // call of its own for the note. + expect( + pageFunctions.filter((source) => source.includes('writeBatch')), + ).toHaveLength(3); + expect( + pageFunctions.filter((source) => source.includes('notePaintedGraphemes')), + ).toHaveLength(0); + }); + it('allows dispose to be called more than once', async () => { await backend.boot(); diff --git a/test/unit/renderer/ghosttyWebBackend.test.ts b/test/unit/renderer/ghosttyWebBackend.test.ts index 1593fe0d..5a12a0dc 100644 --- a/test/unit/renderer/ghosttyWebBackend.test.ts +++ b/test/unit/renderer/ghosttyWebBackend.test.ts @@ -16,6 +16,7 @@ import { afterEach, describe, expect, it, vi } from 'vitest'; import type { ReplayTimingOptions } from '../../../src/renderer/backend.js'; import type { ReplayInput } from '../../../src/renderer/types.js'; import { BUNDLED_FONT_ASSETS } from '../../../src/renderer/bundledFont.js'; +import { MAX_CACHED_GLYPH_PROBES } from '../../../src/renderer/ghosttyWeb/missingGlyphs.js'; import { hashProfile, resolveProfile } from '../../../src/renderer/profiles.js'; import { GhosttyWebBackend } from '../../../src/renderer/ghosttyWeb/index.js'; import { Logger } from '../../../src/util/logger.js'; @@ -88,7 +89,7 @@ describe('GhosttyWebBackend unit guards', () => { expect(recordedChunks).toEqual(chunks); }); - it('notes painted graphemes after each bridge batch of a split flush', async () => { + it('runs the after-batch hook after each bridge call of a split flush', async () => { const backend = createBackend(); const calls: string[] = []; ( @@ -113,29 +114,111 @@ describe('GhosttyWebBackend unit guards', () => { ) => Promise; } ).flushOutputBatch({}, chunks, () => { - calls.push('note'); + calls.push('hook'); return Promise.resolve(); }); expect(calls).toEqual([ 'write 1000', - 'note', + 'hook', 'write 1000', - 'note', + 'hook', 'write 501', - 'note', + 'hook', ]); }); - it('notes painted graphemes after each output flush of a timed replay', async () => { + // The page holds the chunks of the calls before the one that failed. + // HostRendererManager keeps the backend after a failed replay, so a retry + // would meet them. + describe.each([ + [ + 'a page call', + (calls: number) => (calls === 2 ? new Error('page call failed') : null), + undefined, + ], + [ + 'the hook between two calls', + () => null, + () => Promise.reject(new Error('page call failed')), + ], + ])('when %s of a flush fails', (_name, failure, afterEachBatch) => { + function failFlush(discard: () => unknown): { + rejection: Promise; + evaluate: ReturnType; + } { + const backend = createBackend(); + vi.stubGlobal('__agentTty', { discardHeldChunks: discard }); + const evaluate = vi.fn((pageFunction: () => unknown) => + Promise.resolve(pageFunction()), + ); + let calls = 0; + ( + backend as unknown as { + writeBatchBridge: () => Promise; + } + ).writeBatchBridge = () => { + calls += 1; + const error = failure(calls); + return error === null ? Promise.resolve() : Promise.reject(error); + }; + const rejection = ( + backend as unknown as { + flushOutputBatch: ( + page: object, + dataChunks: string[], + afterEachBatch?: () => Promise, + ) => Promise; + } + ).flushOutputBatch( + { evaluate }, + Array.from({ length: 2_501 }, () => 'x'), + afterEachBatch, + ); + return { rejection, evaluate }; + } + + afterEach(() => { + vi.unstubAllGlobals(); + }); + + it('drops the held chunks and rejects with the original error', async () => { + const discard = vi.fn(); + const { rejection } = failFlush(discard); + + await expect(rejection).rejects.toThrow('page call failed'); + expect(discard).toHaveBeenCalledTimes(1); + }); + + it('rejects with the original error when the page cannot drop the chunks either', async () => { + const { rejection, evaluate } = failFlush(() => { + throw new Error('page crashed'); + }); + + await expect(rejection).rejects.toThrow('page call failed'); + expect(evaluate).toHaveBeenCalledTimes(1); + }); + }); + + // A separate page call costs a round trip behind a frame, 5 to 60 ms + // depending on the terminal size, so the page notes the graphemes inside the + // call that ends the flush. + it('notes painted graphemes in the page call that ends each output flush of a timed replay', async () => { const backend = createBackend(); const calls: string[] = []; Object.assign(backend as object, { isBooted: true, page: { isClosed: () => false }, resizeBridge: () => Promise.resolve(), - writeBatchBridge: (_page: object, dataChunks: string[]) => { - calls.push(`write ${dataChunks.join(',')}`); + writeBatchBridge: ( + _page: object, + dataChunks: string[], + endsFlush: boolean, + paintedCap: number | null, + ) => { + calls.push( + `write ${dataChunks.join(',')} ${String(endsFlush)} ${String(paintedCap)}`, + ); return Promise.resolve(); }, notePaintedGraphemes: () => { @@ -184,7 +267,10 @@ describe('GhosttyWebBackend unit guards', () => { { mode: 'max-speed', minFrameHoldMs: 0, finalFrameHoldMs: 0 }, ); - expect(calls).toEqual(['write first', 'note', 'write second', 'note']); + expect(calls).toEqual([ + `write first true ${String(MAX_CACHED_GLYPH_PROBES)}`, + `write second true ${String(MAX_CACHED_GLYPH_PROBES)}`, + ]); }); it('retries the painted-grapheme note after the backend reboots', async () => { @@ -436,20 +522,27 @@ describe('GhosttyWebBackend unit guards', () => { write: ReturnType; writeBatchSizes: number[]; frameWaits: boolean[]; + paintedCaps: (number | null)[]; writtenChunks: string[]; }) => Promise, ): Promise { const writeBatchSizes: number[] = []; const frameWaits: boolean[] = []; + const paintedCaps: (number | null)[] = []; const writtenChunks: string[] = []; const write = vi.fn(() => Promise.resolve()); vi.stubGlobal('__agentTty', { write, - writeBatch: (chunks: string[], waitForFrame: boolean) => { + writeBatch: ( + chunks: string[], + waitForFrame: boolean, + paintedCap: number | null, + ) => { writeBatchSizes.push(chunks.length); frameWaits.push(waitForFrame); + paintedCaps.push(paintedCap); writtenChunks.push(...chunks); - return Promise.resolve(); + return Promise.resolve(null); }, }); const page = { @@ -465,6 +558,7 @@ describe('GhosttyWebBackend unit guards', () => { write, writeBatchSizes, frameWaits, + paintedCaps, writtenChunks, }); } finally { @@ -511,11 +605,15 @@ describe('GhosttyWebBackend unit guards', () => { ); }); + // Only a recorded replay probes glyphs, and it notes the painted graphemes + // in the call that ends the flush: no call of its own, and none in a call + // that only holds chunks. it.each([ [ 'replayTo', (backend: GhosttyWebBackend, input: ReplayInput) => backend.replayTo(input), + [null, null, null], ], [ 'replayWithTiming', @@ -525,20 +623,27 @@ describe('GhosttyWebBackend unit guards', () => { minFrameHoldMs: 0, finalFrameHoldMs: 0, }), + [null, null, MAX_CACHED_GLYPH_PROBES], ], ])( '%s calls the page once per 1000 output events, and ends the flush with the last call', - async (_name, replay) => { + async (_name, replay, expectedPaintedCaps) => { const backend = createBackend(); const input = outputReplayInput(2_501); await withCountingBridge( - async ({ page, write, writeBatchSizes, frameWaits, writtenChunks }) => { + async ({ + page, + write, + writeBatchSizes, + frameWaits, + paintedCaps, + writtenChunks, + }) => { Object.assign(backend as object, { isBooted: true, page, resizeBridge: () => Promise.resolve(), - notePaintedGraphemes: () => Promise.resolve(), checkPaintedGlyphs: () => Promise.resolve({}), readHarnessSnapshot: () => Promise.resolve(createHarnessSnapshotPayload({ cols: 80 })), @@ -549,6 +654,7 @@ describe('GhosttyWebBackend unit guards', () => { expect(page.evaluate).toHaveBeenCalledTimes(3); expect(writeBatchSizes).toEqual([1000, 1000, 501]); expect(frameWaits).toEqual([false, false, true]); + expect(paintedCaps).toEqual(expectedPaintedCaps); expect(writtenChunks).toEqual( input.events.map((event) => event.type === 'output' ? event.payload.data : '', @@ -1313,17 +1419,38 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { }; } + function inputEvent( + seq: number, + offsetMs: number, + ): ReplayInput['events'][number] { + return { + seq, + ts: new Date(BASE_TIMESTAMP_MS + offsetMs).toISOString(), + type: 'input_text', + payload: { data: 'x' }, + }; + } + async function replayAndRecordFlushes( events: ReplayInput['events'], timing: ReplayTimingOptions, flushCostMs = 0, onFlush: () => void = () => {}, + // Seq that an earlier replay applied: the backend starts after it. + appliedSeq = -1, ): Promise<{ atMs: number; chunks: string[] }[]> { vi.useFakeTimers(); const backend = createBackend(); const flushes: { atMs: number; chunks: string[] }[] = []; let replayStartedAtMs = 0; + if (appliedSeq >= 0) { + Object.assign(backend as object, { + lastAppliedSeq: appliedSeq, + initialReplayCols: 80, + initialReplayRows: 24, + }); + } Object.assign(backend as object, { isBooted: true, page: { isClosed: () => false }, @@ -1505,8 +1632,8 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { }); // The frame wait of a flush is the frame of its moment. A hold after every - // flush of a run would make each 30 ms step cost 44 ms to export. - it('holds max-speed frames only after the last flush of a run of output', async () => { + // flush would make each 30 ms step cost 44 ms to export. + it('holds max-speed frames only after the last flush of the replay', async () => { const flushes = await replayAndRecordFlushes( [ outputEvent(0, 0), @@ -1518,16 +1645,88 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { MAX_SPEED, ); - // out0 and out1 flush when the next output arrives, so no hold follows - // them. The exit ends the run: out2 flushes, and its hold delays out4. + // Every flush starts when the next event arrives, so no hold follows one. expect(flushes).toEqual([ { atMs: FIRST_EVENT_AT_MS, chunks: ['out0'] }, { atMs: FIRST_EVENT_AT_MS, chunks: ['out1'] }, { atMs: FIRST_EVENT_AT_MS, chunks: ['out2'] }, - { atMs: FIRST_EVENT_AT_MS + 16, chunks: ['out4'] }, + { atMs: FIRST_EVENT_AT_MS, chunks: ['out4'] }, ]); }); + // A flush at every input, marker or exit event would cost one frame for each + // of them, and a recording of typing has an input for every output. + it('does not end a max-speed flush at an event that changes no pixel', async () => { + const flushes = await replayAndRecordFlushes( + [ + outputEvent(0, 0), + inputEvent(1, 5), + outputEvent(2, 10), + exitEvent(3, 500), + ], + MAX_SPEED, + ); + + expect(flushes.map((flush) => flush.chunks)).toEqual([['out0', 'out2']]); + }); + + it('still ends a recorded flush at an event that changes no pixel', async () => { + const flushes = await replayAndRecordFlushes( + [ + outputEvent(0, 0), + inputEvent(1, 5), + outputEvent(2, 10), + exitEvent(3, 500), + ], + RECORDED, + ); + + expect(flushes.map((flush) => flush.chunks)).toEqual([['out0'], ['out2']]); + }); + + // 10,000 outputs with an input between each pair, 40 ms apart: 400 s of + // recording. Without a budget, every output is a moment of its own and the + // replay makes 10,000 flushes of a frame each, more than 5 minutes. + it('spreads max-speed flushes over a long recording, one per 133 ms of recorded time', async () => { + const events: ReplayInput['events'] = []; + for (let index = 0; index < 10_000; index += 1) { + events.push(outputEvent(2 * index, 40 * index)); + events.push(inputEvent(2 * index + 1, 40 * index + 20)); + } + events.push(exitEvent(20_000, 400_000)); + const flushes = await replayAndRecordFlushes(events, MAX_SPEED); + + // A flush ends at the first event 133 ms after the first output of the + // batch, so each one takes 4 outputs (160 ms): about 2,500 flushes. + expect(flushes.length).toBeGreaterThan(2_000); + expect(flushes.length).toBeLessThanOrEqual(3_001); + // No output is lost or moved. + expect(flushes.flatMap((flush) => flush.chunks)).toEqual( + events + .filter((event) => event.type === 'output') + .map((event) => `out${String(event.seq)}`), + ); + }); + + it('measures the frame budget over the events that the replay applies', async () => { + // 20 ms apart: the whole log spans 200 s, but the last 10 events span 180 ms. + const events = Array.from({ length: 10_000 }, (_, index) => + outputEvent(index, index * 20), + ); + const flushes = await replayAndRecordFlushes( + events, + MAX_SPEED, + 0, + () => {}, + 9_989, + ); + + // 20 ms apart is above the frame hold of 16 ms: a moment each. + expect(flushes.map((flush) => flush.chunks)).toEqual( + events.slice(9_990).map((event) => [`out${String(event.seq)}`]), + ); + }); + it('does not merge max-speed moments when a flush costs more than a frame hold', async () => { const events = Array.from({ length: 101 }, (_, index) => outputEvent(index, index * 10), From 0663f72651feb3720ce15bdc307ead7b1bc89e6a Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 13:55:51 +0000 Subject: [PATCH 14/28] test: pin a failed painted-grapheme note --- test/integration/renderer-backend.test.ts | 18 ++++++++++ test/unit/renderer/ghosttyWebBackend.test.ts | 38 ++++++++++++++++++++ 2 files changed, 56 insertions(+) diff --git a/test/integration/renderer-backend.test.ts b/test/integration/renderer-backend.test.ts index c181af2d..ccff5f78 100644 --- a/test/integration/renderer-backend.test.ts +++ b/test/integration/renderer-backend.test.ts @@ -1055,6 +1055,24 @@ describe('GhosttyWebBackend integration', { timeout: 120_000 }, () => { ).toHaveLength(0); }); + it('writes a flush even when its painted-grapheme note fails, and reports the failure', async () => { + await backend.boot(); + const { page } = backend as unknown as { page: Page }; + await page.evaluate('window.__agentTty.resize(80, 24)'); + + // A cap of 0 makes the note throw after the chunks are on the screen. + const noteError = await page.evaluate( + `window.__agentTty.writeBatch(['ab', 'cd'], true, 0)`, + ); + + expect(noteError).toEqual( + expect.stringContaining('cap must be a positive integer'), + ); + expect( + await page.evaluate(`window.__agentTty.getVisibleText().split('\\n')[0]`), + ).toBe('abcd'); + }); + it('allows dispose to be called more than once', async () => { await backend.boot(); diff --git a/test/unit/renderer/ghosttyWebBackend.test.ts b/test/unit/renderer/ghosttyWebBackend.test.ts index 5a12a0dc..9284f9a0 100644 --- a/test/unit/renderer/ghosttyWebBackend.test.ts +++ b/test/unit/renderer/ghosttyWebBackend.test.ts @@ -273,6 +273,44 @@ describe('GhosttyWebBackend unit guards', () => { ]); }); + it('omits the record export glyph fields, and asks for no more notes, once the page reports a failed note', async () => { + const logger = new Logger('debug'); + const debugSpy = vi.spyOn(logger, 'debug').mockImplementation(() => {}); + const backend = new GhosttyWebBackend( + 'renderer-unit-session', + PROFILE, + undefined, + logger, + ); + const evaluate = vi.fn(() => Promise.resolve('grapheme read failed')); + const page = { isClosed: () => false, evaluate }; + Object.assign(backend as object, { isBooted: true, page }); + const flush = (cap: number | null): Promise => + ( + backend as unknown as { + writeBatchBridge: ( + page: object, + dataChunks: string[], + endsFlush: boolean, + paintedCap: number | null, + ) => Promise; + } + ).writeBatchBridge(page, ['x'], true, cap); + + await expect(flush(MAX_CACHED_GLYPH_PROBES)).resolves.toBeUndefined(); + + expect(debugSpy).toHaveBeenCalledWith( + 'Missing-glyph note failed; the record export result omits missingGlyphs', + 'grapheme read failed', + ); + await expect(backend.recordedMissingGlyphs()).resolves.toEqual({}); + expect( + ( + backend as unknown as { paintedNoteCap: () => number | null } + ).paintedNoteCap(), + ).toBeNull(); + }); + it('retries the painted-grapheme note after the backend reboots', async () => { const backend = createBackend(); const graphemes = vi.fn(() => Promise.resolve({ missingGlyphs: [] })); From e1a727818bc1a6ff7fe8470e8e668dcecb5f8c40 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 14:06:57 +0000 Subject: [PATCH 15/28] docs: describe the max-speed frame budget and the in-call painted-grapheme note --- docs/USAGE.md | 2 +- src/renderer/ghosttyWeb/README.md | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/USAGE.md b/docs/USAGE.md index bf4722b4..d7b60d91 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -294,7 +294,7 @@ agent-tty record export --format webm --json | jq -c '.result.missi # [{"codePoint":"U+0378"}] ``` -WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. A frame of the video shows the screen after all the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event. `max-speed` removes the idle time but gives each 16 ms of recorded output its own frame, so a progress bar that updates every 30 ms shows its steps and not only its last one. The video has 25 frames per second, so it can still skip a state that lasts less than one frame (40 ms). +WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. A frame of the video shows the screen after all the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event. `max-speed` removes the idle time but gives each 16 ms of recorded output its own frame, so a progress bar that updates every 30 ms shows its steps and not only its last one. It makes at most about 3,000 frames, so a recording longer than 48 s gets one frame per 1/3,000 of its recorded time instead (a 1-hour recording: one per 1.2 s), and its export time depends on its length and not on its number of events. Input, marker and exit events do not end a `max-speed` frame by themselves. The video has 25 frames per second, so it can still skip a state that lasts less than one frame (40 ms). SVG export renders styled grid frames from the event log through the native `libghostty-vt` backend with no browser and no ffmpeg, so it requires the optional `@coder/libghostty-vt-node` package (there is no `ghostty-web` fallback). The output is deterministic: exporting the same session twice produces byte-identical, diffable SVG. `--format svg` writes a still image of the final screen and ignores event timing, so it also works on a log whose clock stepped backward (`--animate` refuses such a log and names the event where the clock stepped back); add `--animate` for an animated SVG of de-duplicated frames replayed with recorded event-log timing (`--timing` is not supported with SVG). diff --git a/src/renderer/ghosttyWeb/README.md b/src/renderer/ghosttyWeb/README.md index f98617e6..9100806b 100644 --- a/src/renderer/ghosttyWeb/README.md +++ b/src/renderer/ghosttyWeb/README.md @@ -20,7 +20,7 @@ This directory holds the `ghostty-web` reference renderer. `GhosttyWebBackend` s - `stripTrailingAsciiSpaces(text)` and `assembleCanonicalLine(cols, readColumn)` build canonical line text [harnessDecoding.ts:assembleCanonicalLine]. - Assertion helpers `assertNonNegativeInteger`, `assertPositiveInteger`, `assertPositiveNumber` and `assertHexColor`, plus `normalizeError(error, prefix)` [harnessDecoding.ts:normalizeError]. - `loadHarnessHtml()` returns `EMBEDDED_HARNESS_HTML` [harnessDecoding.ts:loadHarnessHtml]. -- `EMBEDDED_HARNESS_HTML` is the harness page as a template string. In the browser it defines `window.__agentTty` with `write`, `writeBatch`, `getSnapshot`, `getCellGraphemes`, `getVisibleText`, `isReady`, `resize` and `setCursorVisible` [embeddedHarnessHtml.ts:EMBEDDED_HARNESS_HTML]. +- `EMBEDDED_HARNESS_HTML` is the harness page as a template string. In the browser it defines `window.__agentTty` with `write`, `writeBatch`, `discardHeldChunks`, `notePaintedGraphemes`, `getSnapshot`, `getCellGraphemes`, `getVisibleText`, `isReady`, `resize` and `setCursorVisible` [embeddedHarnessHtml.ts:EMBEDDED_HARNESS_HTML]. - `harness.html` is a standalone copy of the harness page. No code serves it, and it differs from `EMBEDDED_HARNESS_HTML` [harnessDecoding.ts:loadHarnessHtml]. - `assets/` (not documented separately): holds the bundled fonts `JetBrainsMono-Regular-latin.woff2`, `SymbolsNerdFontMono-Regular.ttf`, `NotoSansSymbols2-Regular.woff2` and `NotoSansMath-Regular.woff2`, plus `FONT-LICENSE.txt` and the OFL texts of the Noto fonts. `scripts/subset-symbol-fonts.sh` rebuilds the two Noto subsets. `src/renderer/bundledFont.ts` loads them, and `scripts/copy-renderer-assets.mjs` copies them into `dist/` during `npm run build`. @@ -29,8 +29,8 @@ This directory holds the `ghostty-web` reference renderer. `GhosttyWebBackend` s - The constructor asserts a non-empty `sessionId`, `profile.name` and `profile.fontFamily`, a positive `fontSize`, and hex colors. With `videoOptions`, `outputDir` must be absolute and the size must use positive integers [backend.ts:GhosttyWebBackend.constructor]. - Replay cannot rewind: `targetSeq` must be >= the last applied seq. The initial dimensions cannot change after the first replay. `input.sessionId` must match the backend [backend.ts:GhosttyWebBackend.replayTo]. - `replayWithTiming` asserts strictly increasing seqs and non-decreasing timestamps [backend.ts:GhosttyWebBackend.replayWithTiming]. -- `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` has no schedule to fall behind, so it flushes once the next event is at least one frame hold of recorded time after the batch's first output. Each moment that a viewer could tell apart gets a frame, and a replay costs at most one frame per output event. Events with the same timestamp share a flush when the hold is above 0 [backend.ts:GhosttyWebBackend.replayWithTiming]. -- `writeBatch(chunks, endsFlush)` takes the chunks of one bridge call. A flush that has more than 1000 chunks needs several calls, and ghostty-web paints on every animation frame, so the page holds the chunks of a call that does not end the flush. The call that ends it (`endsFlush`, set by the backend on the last call only) writes every held chunk in one task, so no frame paints a state between two calls, and then resolves after one frame. ghostty-web calls a write callback one animation frame (about 17 ms) after the write, and waiting on every chunk (`write`) cost one frame per output event. `write()` and `resize()` throw while chunks are held. In `replayWithTiming`, the chunks of one flush share one video frame [embeddedHarnessHtml.ts:EMBEDDED_HARNESS_HTML]. +- `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` has no schedule to fall behind, so it flushes once the next event is at least one frame hold of recorded time after the batch's first output. It also spreads at most `MAX_SPEED_FRAME_BUDGET` (3,000) flushes over the recorded span of the events that it applies: the minimum distance between two flushes is the larger of the frame hold and the span divided by 3,000. So a `max-speed` replay costs about one frame per moment for a recording shorter than 48 s and at most about 3,000 frames for a longer one, whatever its number of events. Input, marker, run-complete, signal and exit events do not end a `max-speed` flush, because a flush at each of them would cost a frame per event (a resize still does, to keep the order). Events with the same timestamp share a flush at any hold, 0 included, because a flush needs at least 1 ms of recorded time [backend.ts:GhosttyWebBackend.replayWithTiming]. +- `writeBatch(chunks, endsFlush, paintedCap)` takes the chunks of one bridge call. A flush that has more than 1000 chunks needs several calls, and ghostty-web paints on every animation frame, so the page holds the chunks of a call that does not end the flush. The call that ends it (`endsFlush`, set by the backend on the last call only) writes every held chunk in one task, so no frame paints a state between two calls, and then resolves after one frame. ghostty-web calls a write callback one animation frame (about 17 ms) after the write, and waiting on every chunk (`write`) cost one frame per output event. `write()` and `resize()` throw while chunks are held. `discardHeldChunks()` drops them: the backend calls it when a flush fails, because `HostRendererManager` keeps the backend and a retry would otherwise meet them. A `paintedCap` that is not null makes the call that ends the flush add the graphemes on the screen to the set that the missing-glyph check of `replayWithTiming` probes, so a flush is one page call. The call returns the message of a failed note and not an error, because a failed note must not fail the flush. In `replayWithTiming`, the chunks of one flush share one video frame [embeddedHarnessHtml.ts:EMBEDDED_HARNESS_HTML]. - `snapshot()` and `screenshot()` throw until `replayTo()` or `replayWithTiming()` has run once and set the initial size. Both replays accept `targetSeq` -1 for an empty Event Log. A replay over zero events sets `lastAppliedSeq` to `targetSeq` [backend.ts:GhosttyWebBackend.snapshot]. - `missingGlyphs` lists a code point only when a bundled face that lacks it drew it. Chromium draws a cluster with one font: the first stack font that covers all of it, or else the first font of the stack, which is bundled in both built-in profiles and then draws `.notdef` boxes. CDP names the face that drew a probe by its PostScript name, so a family alias or several faces under one family resolve to the right cmap. When one bundled face drew the whole grapheme, every code point that face lacks is listed. A probe with no reported font is judged against every face of the first family. When the grapheme is split over several fonts, each uncovered code point is judged by its own probe, and a mark's probe on its base is compared with a probe of the base alone. Glyphs that a host font draws are not listed. The check runs after the PNG is written, removes its probe spans, and caches each probe text's result for the page, clearing the cache when it would pass `MAX_CACHED_GLYPH_PROBES` (16384) entries. It probes at most `MAX_GLYPH_PROBES` texts and lists at most `MAX_MISSING_GLYPH_CELLS` cells (4096 each). `probeUsedFonts` sends its CDP queries in batches of 256 and stops starting batches after `GLYPH_PROBE_BUDGET_MS` (1000 ms). The result sets `missingGlyphsTruncated` when any of these stops the check early. Family names match without regard to case. A web font whose PostScript name the profile does not bundle fails the check. A failed check is logged at debug level and leaves the field out, so the screenshot still succeeds. `replayWithTiming()` records the viewport graphemes in the page after each output batch and each resize (at most `MAX_CACHED_GLYPH_PROBES`), and the final frame hold judges them once, with the hold as the probe budget, so the check does not lengthen the video. `recordedMissingGlyphs()` returns that result. A failed note or check leaves the record export fields out. A profile whose font stack does not start with a bundled font gets no `missingGlyphs` field [missingGlyphs.ts:probeUsedFonts] [backend.ts:GhosttyWebBackend.screenshot]. - `screenshot()` requires an absolute `outputPath` whose parent directory exists. It checks that the output is a non-empty PNG with a valid signature and an `IHDR` first chunk [backend.ts:GhosttyWebBackend.screenshot]. From 028a1ab32db4bb00ae2b51c4115351160c1bd4fd Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 15:30:24 +0000 Subject: [PATCH 16/28] perf: size the max-speed frame budget from the measured flush cost --- docs/USAGE.md | 2 +- src/renderer/ghosttyWeb/README.md | 2 +- src/renderer/ghosttyWeb/backend.ts | 13 +++++++------ test/unit/renderer/ghosttyWebBackend.test.ts | 10 +++++----- 4 files changed, 14 insertions(+), 13 deletions(-) diff --git a/docs/USAGE.md b/docs/USAGE.md index d7b60d91..000e381f 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -294,7 +294,7 @@ agent-tty record export --format webm --json | jq -c '.result.missi # [{"codePoint":"U+0378"}] ``` -WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. A frame of the video shows the screen after all the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event. `max-speed` removes the idle time but gives each 16 ms of recorded output its own frame, so a progress bar that updates every 30 ms shows its steps and not only its last one. It makes at most about 3,000 frames, so a recording longer than 48 s gets one frame per 1/3,000 of its recorded time instead (a 1-hour recording: one per 1.2 s), and its export time depends on its length and not on its number of events. Input, marker and exit events do not end a `max-speed` frame by themselves. The video has 25 frames per second, so it can still skip a state that lasts less than one frame (40 ms). +WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. A frame of the video shows the screen after all the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event. `max-speed` removes the idle time but gives each 16 ms of recorded output its own frame, so a progress bar that updates every 30 ms shows its steps and not only its last one. It makes at most about 2,400 frames, so a recording longer than 38 s gets one frame per 1/2,400 of its recorded time instead (a 1-hour recording: one per 1.5 s), and its export time depends on its length and not on its number of events. Input, marker and exit events do not end a `max-speed` frame by themselves. The video has 25 frames per second, so it can still skip a state that lasts less than one frame (40 ms). SVG export renders styled grid frames from the event log through the native `libghostty-vt` backend with no browser and no ffmpeg, so it requires the optional `@coder/libghostty-vt-node` package (there is no `ghostty-web` fallback). The output is deterministic: exporting the same session twice produces byte-identical, diffable SVG. `--format svg` writes a still image of the final screen and ignores event timing, so it also works on a log whose clock stepped backward (`--animate` refuses such a log and names the event where the clock stepped back); add `--animate` for an animated SVG of de-duplicated frames replayed with recorded event-log timing (`--timing` is not supported with SVG). diff --git a/src/renderer/ghosttyWeb/README.md b/src/renderer/ghosttyWeb/README.md index 9100806b..dc5a5ee9 100644 --- a/src/renderer/ghosttyWeb/README.md +++ b/src/renderer/ghosttyWeb/README.md @@ -29,7 +29,7 @@ This directory holds the `ghostty-web` reference renderer. `GhosttyWebBackend` s - The constructor asserts a non-empty `sessionId`, `profile.name` and `profile.fontFamily`, a positive `fontSize`, and hex colors. With `videoOptions`, `outputDir` must be absolute and the size must use positive integers [backend.ts:GhosttyWebBackend.constructor]. - Replay cannot rewind: `targetSeq` must be >= the last applied seq. The initial dimensions cannot change after the first replay. `input.sessionId` must match the backend [backend.ts:GhosttyWebBackend.replayTo]. - `replayWithTiming` asserts strictly increasing seqs and non-decreasing timestamps [backend.ts:GhosttyWebBackend.replayWithTiming]. -- `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` has no schedule to fall behind, so it flushes once the next event is at least one frame hold of recorded time after the batch's first output. It also spreads at most `MAX_SPEED_FRAME_BUDGET` (3,000) flushes over the recorded span of the events that it applies: the minimum distance between two flushes is the larger of the frame hold and the span divided by 3,000. So a `max-speed` replay costs about one frame per moment for a recording shorter than 48 s and at most about 3,000 frames for a longer one, whatever its number of events. Input, marker, run-complete, signal and exit events do not end a `max-speed` flush, because a flush at each of them would cost a frame per event (a resize still does, to keep the order). Events with the same timestamp share a flush at any hold, 0 included, because a flush needs at least 1 ms of recorded time [backend.ts:GhosttyWebBackend.replayWithTiming]. +- `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` has no schedule to fall behind, so it flushes once the next event is at least one frame hold of recorded time after the batch's first output. It also spreads at most `MAX_SPEED_FRAME_BUDGET` (2,400) flushes over the recorded span of the events that it applies: the minimum distance between two flushes is the larger of the frame hold and the span divided by 2,400. So a `max-speed` replay costs about one frame per moment for a recording shorter than 38 s and at most about 2,400 frames for a longer one, whatever its number of events. Input, marker, run-complete, signal and exit events do not end a `max-speed` flush, because a flush at each of them would cost a frame per event (a resize still does, to keep the order). Events with the same timestamp share a flush at any hold, 0 included, because a flush needs at least 1 ms of recorded time [backend.ts:GhosttyWebBackend.replayWithTiming]. - `writeBatch(chunks, endsFlush, paintedCap)` takes the chunks of one bridge call. A flush that has more than 1000 chunks needs several calls, and ghostty-web paints on every animation frame, so the page holds the chunks of a call that does not end the flush. The call that ends it (`endsFlush`, set by the backend on the last call only) writes every held chunk in one task, so no frame paints a state between two calls, and then resolves after one frame. ghostty-web calls a write callback one animation frame (about 17 ms) after the write, and waiting on every chunk (`write`) cost one frame per output event. `write()` and `resize()` throw while chunks are held. `discardHeldChunks()` drops them: the backend calls it when a flush fails, because `HostRendererManager` keeps the backend and a retry would otherwise meet them. A `paintedCap` that is not null makes the call that ends the flush add the graphemes on the screen to the set that the missing-glyph check of `replayWithTiming` probes, so a flush is one page call. The call returns the message of a failed note and not an error, because a failed note must not fail the flush. In `replayWithTiming`, the chunks of one flush share one video frame [embeddedHarnessHtml.ts:EMBEDDED_HARNESS_HTML]. - `snapshot()` and `screenshot()` throw until `replayTo()` or `replayWithTiming()` has run once and set the initial size. Both replays accept `targetSeq` -1 for an empty Event Log. A replay over zero events sets `lastAppliedSeq` to `targetSeq` [backend.ts:GhosttyWebBackend.snapshot]. - `missingGlyphs` lists a code point only when a bundled face that lacks it drew it. Chromium draws a cluster with one font: the first stack font that covers all of it, or else the first font of the stack, which is bundled in both built-in profiles and then draws `.notdef` boxes. CDP names the face that drew a probe by its PostScript name, so a family alias or several faces under one family resolve to the right cmap. When one bundled face drew the whole grapheme, every code point that face lacks is listed. A probe with no reported font is judged against every face of the first family. When the grapheme is split over several fonts, each uncovered code point is judged by its own probe, and a mark's probe on its base is compared with a probe of the base alone. Glyphs that a host font draws are not listed. The check runs after the PNG is written, removes its probe spans, and caches each probe text's result for the page, clearing the cache when it would pass `MAX_CACHED_GLYPH_PROBES` (16384) entries. It probes at most `MAX_GLYPH_PROBES` texts and lists at most `MAX_MISSING_GLYPH_CELLS` cells (4096 each). `probeUsedFonts` sends its CDP queries in batches of 256 and stops starting batches after `GLYPH_PROBE_BUDGET_MS` (1000 ms). The result sets `missingGlyphsTruncated` when any of these stops the check early. Family names match without regard to case. A web font whose PostScript name the profile does not bundle fails the check. A failed check is logged at debug level and leaves the field out, so the screenshot still succeeds. `replayWithTiming()` records the viewport graphemes in the page after each output batch and each resize (at most `MAX_CACHED_GLYPH_PROBES`), and the final frame hold judges them once, with the hold as the probe budget, so the check does not lengthen the video. `recordedMissingGlyphs()` returns that result. A failed note or check leaves the record export fields out. A profile whose font stack does not start with a bundled font gets no `missingGlyphs` field [missingGlyphs.ts:probeUsedFonts] [backend.ts:GhosttyWebBackend.screenshot]. diff --git a/src/renderer/ghosttyWeb/backend.ts b/src/renderer/ghosttyWeb/backend.ts index 724fffdd..d6e16465 100644 --- a/src/renderer/ghosttyWeb/backend.ts +++ b/src/renderer/ghosttyWeb/backend.ts @@ -139,13 +139,14 @@ const WASM_CONTENT_TYPE = 'application/wasm'; const MAX_REPLAY_BATCH_SIZE = 1000; /** * A max-speed replay flushes at most about this many times, however long the - * recording is. Every flush waits for one animation frame (17 to 25 ms) and - * one page call, so a replay takes about 2 minutes at the limit. That stays - * below the 5 minutes that a WebM export allows its replay, even on a slow - * host. The video has 25 frames per second, so 3,000 flushes fill 2 minutes - * of video: a viewer cannot tell more states apart. + * recording is. A flush costs one animation frame, one page call and one scan + * of the screen: 26 ms at 80x24, 33 ms at 120x40 and 97 ms at 200x60 in the + * median of the sparse-log runs of PR 358 (host load 60 to 90). The limit then + * takes 1 to 4 minutes, under the 5 minutes that a WebM export allows its + * replay. The video has 25 frames per second, so a viewer cannot tell more + * than this many states apart in a video of that length. */ -const MAX_SPEED_FRAME_BUDGET = 3_000; +const MAX_SPEED_FRAME_BUDGET = 2_400; const RAF_TIMEOUT_MS = 5_000; let servedAssetsPromise: Promise< diff --git a/test/unit/renderer/ghosttyWebBackend.test.ts b/test/unit/renderer/ghosttyWebBackend.test.ts index 9284f9a0..98174f15 100644 --- a/test/unit/renderer/ghosttyWebBackend.test.ts +++ b/test/unit/renderer/ghosttyWebBackend.test.ts @@ -1725,7 +1725,7 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { // 10,000 outputs with an input between each pair, 40 ms apart: 400 s of // recording. Without a budget, every output is a moment of its own and the // replay makes 10,000 flushes of a frame each, more than 5 minutes. - it('spreads max-speed flushes over a long recording, one per 133 ms of recorded time', async () => { + it('spreads max-speed flushes over a long recording, one per 167 ms of recorded time', async () => { const events: ReplayInput['events'] = []; for (let index = 0; index < 10_000; index += 1) { events.push(outputEvent(2 * index, 40 * index)); @@ -1734,10 +1734,10 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { events.push(exitEvent(20_000, 400_000)); const flushes = await replayAndRecordFlushes(events, MAX_SPEED); - // A flush ends at the first event 133 ms after the first output of the - // batch, so each one takes 4 outputs (160 ms): about 2,500 flushes. - expect(flushes.length).toBeGreaterThan(2_000); - expect(flushes.length).toBeLessThanOrEqual(3_001); + // A flush ends at the first event 167 ms after the first output of the + // batch, so each one takes 5 outputs (200 ms): about 2,000 flushes. + expect(flushes.length).toBeGreaterThan(1_800); + expect(flushes.length).toBeLessThanOrEqual(2_401); // No output is lost or moved. expect(flushes.flatMap((flush) => flush.chunks)).toEqual( events From dfde02d942cbb7059c249a166a31f762aa18f282 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 15:55:38 +0000 Subject: [PATCH 17/28] docs: update the max-speed frame budget comment with the measured flush costs --- src/renderer/ghosttyWeb/backend.ts | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/src/renderer/ghosttyWeb/backend.ts b/src/renderer/ghosttyWeb/backend.ts index d6e16465..70eaea1f 100644 --- a/src/renderer/ghosttyWeb/backend.ts +++ b/src/renderer/ghosttyWeb/backend.ts @@ -140,11 +140,11 @@ const MAX_REPLAY_BATCH_SIZE = 1000; /** * A max-speed replay flushes at most about this many times, however long the * recording is. A flush costs one animation frame, one page call and one scan - * of the screen: 26 ms at 80x24, 33 ms at 120x40 and 97 ms at 200x60 in the - * median of the sparse-log runs of PR 358 (host load 60 to 90). The limit then - * takes 1 to 4 minutes, under the 5 minutes that a WebM export allows its - * replay. The video has 25 frames per second, so a viewer cannot tell more - * than this many states apart in a video of that length. + * of the screen: 26 ms at 80x24, 33 ms at 120x40 and 112 ms at 200x60 in the + * median of five sparse-log runs (host load 40 to 100). The limit then takes + * 1 to 4.5 minutes, under the 5 minutes that a WebM export allows its replay. + * The video has 25 frames per second, so a viewer cannot tell more than this + * many states apart in a video of that length. */ const MAX_SPEED_FRAME_BUDGET = 2_400; const RAF_TIMEOUT_MS = 5_000; From e64b021674e7be3cdc5c73b0f0d991dcdf513f56 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 21:16:11 +0000 Subject: [PATCH 18/28] test: pin the max-speed frame budget, resize hold and note, painted-set cap, and fix the max-speed docs --- docs/USAGE.md | 2 +- src/export/README.md | 2 +- src/renderer/ghosttyWeb/README.md | 2 +- src/renderer/ghosttyWeb/backend.ts | 24 +-- test/integration/renderer-backend.test.ts | 93 ++++++++ test/unit/renderer/ghosttyWebBackend.test.ts | 214 ++++++++++++++++--- 6 files changed, 292 insertions(+), 45 deletions(-) diff --git a/docs/USAGE.md b/docs/USAGE.md index 000e381f..15a41f49 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -294,7 +294,7 @@ agent-tty record export --format webm --json | jq -c '.result.missi # [{"codePoint":"U+0378"}] ``` -WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. A frame of the video shows the screen after all the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event. `max-speed` removes the idle time but gives each 16 ms of recorded output its own frame, so a progress bar that updates every 30 ms shows its steps and not only its last one. It makes at most about 2,400 frames, so a recording longer than 38 s gets one frame per 1/2,400 of its recorded time instead (a 1-hour recording: one per 1.5 s), and its export time depends on its length and not on its number of events. Input, marker and exit events do not end a `max-speed` frame by themselves. The video has 25 frames per second, so it can still skip a state that lasts less than one frame (40 ms). +WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. A frame of the video shows the screen after the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event, and a state that the next output of the frame replaces is not shown. In `recorded` and `accelerated` mode, an event that changes no pixel (an input, a marker, a run-complete, a signal or the exit) also ends a frame. In `max-speed` mode it does not: only recorded time ends a frame, so output on both sides of an input event that is less than 16 ms apart shares one frame. `max-speed` removes the idle time. It makes a frame at most once per 16 ms of recorded time, so a progress bar that updates every 30 ms shows its steps and not only its last one. It makes at most about 2,400 frames, so a recording longer than 38 s gets one frame per 1/2,400 of its recorded time instead (a 1-hour recording: one per 1.5 s), and its export time depends on its length and not on its number of events. `max-speed` waits for one animation frame after each frame but holds no further time between two frames, so two frames that follow each other can end up in one video frame. The video has 25 frames per second, so it can skip a state that lasts less than one video frame (40 ms). SVG export renders styled grid frames from the event log through the native `libghostty-vt` backend with no browser and no ffmpeg, so it requires the optional `@coder/libghostty-vt-node` package (there is no `ghostty-web` fallback). The output is deterministic: exporting the same session twice produces byte-identical, diffable SVG. `--format svg` writes a still image of the final screen and ignores event timing, so it also works on a log whose clock stepped backward (`--animate` refuses such a log and names the event where the clock stepped back); add `--animate` for an animated SVG of de-duplicated frames replayed with recorded event-log timing (`--timing` is not supported with SVG). diff --git a/src/export/README.md b/src/export/README.md index 876ffdfa..7733b406 100644 --- a/src/export/README.md +++ b/src/export/README.md @@ -25,7 +25,7 @@ This directory converts a session's event log into recording exports. `generateA - Defaults: timing mode `recorded` and profile `reference-dark` [webm.ts:generateWebmExport]. - A requested `libghostty-vt` renderer is replaced with `ghostty-web`. A code comment says `libghostty-vt` is semantic-only [webm.ts:generateWebmExport]. - The default backend factory accepts only `ghostty-web` [webm.ts:createDefaultBackend]. -- Timing presets: `accelerated` caps gaps at 400 ms and holds frames at least 100 ms. `max-speed` holds frames at least 16 ms. Each mode holds the final frame (1000 ms, or 500 ms for `max-speed`) [webm.ts:buildReplayTimingOptions]. +- Timing presets: `accelerated` caps gaps at 400 ms and holds frames at least 100 ms. `max-speed` uses a frame hold of 16 ms, but holds it only after the last flush of the replay and before a resize (see the `ghostty-web` README). Each mode holds the final frame (1000 ms, or 500 ms for `max-speed`) [webm.ts:buildReplayTimingOptions]. - The timed replay fails after `REPLAY_TIMEOUT_MS` (5 minutes) [webm.ts:REPLAY_TIMEOUT_MS]. - `renderGridFramesToSvg` needs the profile font size `SVG_FONT_SIZE` (14) and draws fixed 8.4x18 px cells, so equal input gives byte-identical output [svg.ts:renderGridFramesToSvg]. It always embeds the JetBrains Mono subset, and embeds each other bundled face only when a glyph needs it [svg.ts:embeddedFontAssets]. Output past `defaultMaxSvgOutputLength` of the V8 heap limit (the heap limit divided by 8 bytes, at most `MAX_SVG_OUTPUT_LENGTH`) fails with `EXPORT_ERROR` [svg.ts:defaultMaxSvgOutputLength]. diff --git a/src/renderer/ghosttyWeb/README.md b/src/renderer/ghosttyWeb/README.md index dc5a5ee9..18b39f20 100644 --- a/src/renderer/ghosttyWeb/README.md +++ b/src/renderer/ghosttyWeb/README.md @@ -29,7 +29,7 @@ This directory holds the `ghostty-web` reference renderer. `GhosttyWebBackend` s - The constructor asserts a non-empty `sessionId`, `profile.name` and `profile.fontFamily`, a positive `fontSize`, and hex colors. With `videoOptions`, `outputDir` must be absolute and the size must use positive integers [backend.ts:GhosttyWebBackend.constructor]. - Replay cannot rewind: `targetSeq` must be >= the last applied seq. The initial dimensions cannot change after the first replay. `input.sessionId` must match the backend [backend.ts:GhosttyWebBackend.replayTo]. - `replayWithTiming` asserts strictly increasing seqs and non-decreasing timestamps [backend.ts:GhosttyWebBackend.replayWithTiming]. -- `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` has no schedule to fall behind, so it flushes once the next event is at least one frame hold of recorded time after the batch's first output. It also spreads at most `MAX_SPEED_FRAME_BUDGET` (2,400) flushes over the recorded span of the events that it applies: the minimum distance between two flushes is the larger of the frame hold and the span divided by 2,400. So a `max-speed` replay costs about one frame per moment for a recording shorter than 38 s and at most about 2,400 frames for a longer one, whatever its number of events. Input, marker, run-complete, signal and exit events do not end a `max-speed` flush, because a flush at each of them would cost a frame per event (a resize still does, to keep the order). Events with the same timestamp share a flush at any hold, 0 included, because a flush needs at least 1 ms of recorded time [backend.ts:GhosttyWebBackend.replayWithTiming]. +- `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` has no schedule to fall behind, so only recorded time ends its flushes. A flush ends at the first event that is at least one frame hold of recorded time after the batch's first output. An event that changes no pixel does not end it: input, marker, run-complete, signal and exit events leave the batch as it is, because a flush at each of them would cost a frame per event. A resize and the end of the replay still end it. `max-speed` also spreads at most `MAX_SPEED_FRAME_BUDGET` (2,400) flushes over the recorded span of the events that it applies: the distance between the first output of a flush and the event that ends it is at least the larger of the frame hold and the span divided by 2,400. So a `max-speed` replay makes about one flush per frame hold of recorded time for a recording shorter than 38 s and at most about 2,400 flushes for a longer one, whatever its number of events. Output that arrives less than that distance after the first output of a flush shares the flush, also when an input or a marker sits between, so a state that the next output of the flush replaces is never painted. A flush waits for one animation frame. `max-speed` holds the frame hold only after the last flush of the replay and before a resize, so two flushes that follow each other can paint within one video frame. In `recorded` and `accelerated` mode, every event that changes no pixel ends a flush, and a hold follows every flush. Events with the same timestamp share a flush at any hold, 0 included, because a flush needs at least 1 ms of recorded time [backend.ts:GhosttyWebBackend.replayWithTiming]. - `writeBatch(chunks, endsFlush, paintedCap)` takes the chunks of one bridge call. A flush that has more than 1000 chunks needs several calls, and ghostty-web paints on every animation frame, so the page holds the chunks of a call that does not end the flush. The call that ends it (`endsFlush`, set by the backend on the last call only) writes every held chunk in one task, so no frame paints a state between two calls, and then resolves after one frame. ghostty-web calls a write callback one animation frame (about 17 ms) after the write, and waiting on every chunk (`write`) cost one frame per output event. `write()` and `resize()` throw while chunks are held. `discardHeldChunks()` drops them: the backend calls it when a flush fails, because `HostRendererManager` keeps the backend and a retry would otherwise meet them. A `paintedCap` that is not null makes the call that ends the flush add the graphemes on the screen to the set that the missing-glyph check of `replayWithTiming` probes, so a flush is one page call. The call returns the message of a failed note and not an error, because a failed note must not fail the flush. In `replayWithTiming`, the chunks of one flush share one video frame [embeddedHarnessHtml.ts:EMBEDDED_HARNESS_HTML]. - `snapshot()` and `screenshot()` throw until `replayTo()` or `replayWithTiming()` has run once and set the initial size. Both replays accept `targetSeq` -1 for an empty Event Log. A replay over zero events sets `lastAppliedSeq` to `targetSeq` [backend.ts:GhosttyWebBackend.snapshot]. - `missingGlyphs` lists a code point only when a bundled face that lacks it drew it. Chromium draws a cluster with one font: the first stack font that covers all of it, or else the first font of the stack, which is bundled in both built-in profiles and then draws `.notdef` boxes. CDP names the face that drew a probe by its PostScript name, so a family alias or several faces under one family resolve to the right cmap. When one bundled face drew the whole grapheme, every code point that face lacks is listed. A probe with no reported font is judged against every face of the first family. When the grapheme is split over several fonts, each uncovered code point is judged by its own probe, and a mark's probe on its base is compared with a probe of the base alone. Glyphs that a host font draws are not listed. The check runs after the PNG is written, removes its probe spans, and caches each probe text's result for the page, clearing the cache when it would pass `MAX_CACHED_GLYPH_PROBES` (16384) entries. It probes at most `MAX_GLYPH_PROBES` texts and lists at most `MAX_MISSING_GLYPH_CELLS` cells (4096 each). `probeUsedFonts` sends its CDP queries in batches of 256 and stops starting batches after `GLYPH_PROBE_BUDGET_MS` (1000 ms). The result sets `missingGlyphsTruncated` when any of these stops the check early. Family names match without regard to case. A web font whose PostScript name the profile does not bundle fails the check. A failed check is logged at debug level and leaves the field out, so the screenshot still succeeds. `replayWithTiming()` records the viewport graphemes in the page after each output batch and each resize (at most `MAX_CACHED_GLYPH_PROBES`), and the final frame hold judges them once, with the hold as the probe budget, so the check does not lengthen the video. `recordedMissingGlyphs()` returns that result. A failed note or check leaves the record export fields out. A profile whose font stack does not start with a bundled font gets no `missingGlyphs` field [missingGlyphs.ts:probeUsedFonts] [backend.ts:GhosttyWebBackend.screenshot]. diff --git a/src/renderer/ghosttyWeb/backend.ts b/src/renderer/ghosttyWeb/backend.ts index 70eaea1f..eef422e8 100644 --- a/src/renderer/ghosttyWeb/backend.ts +++ b/src/renderer/ghosttyWeb/backend.ts @@ -138,13 +138,12 @@ const WASM_CONTENT_TYPE = 'application/wasm'; const MAX_REPLAY_BATCH_SIZE = 1000; /** - * A max-speed replay flushes at most about this many times, however long the + * A max-speed replay makes at most about this many flushes, however long the * recording is. A flush costs one animation frame, one page call and one scan * of the screen: 26 ms at 80x24, 33 ms at 120x40 and 112 ms at 200x60 in the * median of five sparse-log runs (host load 40 to 100). The limit then takes * 1 to 4.5 minutes, under the 5 minutes that a WebM export allows its replay. - * The video has 25 frames per second, so a viewer cannot tell more than this - * many states apart in a video of that length. + * The video has 25 frames per second, so this many flushes fill 96 s of video. */ const MAX_SPEED_FRAME_BUDGET = 2_400; const RAF_TIMEOUT_MS = 5_000; @@ -688,7 +687,7 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { } // Output shows once the next event is this far after the batch's first - // output. It is at least 1 ms: events with one timestamp are one moment. + // output. It is at least 1 ms: events with one timestamp share a flush. // Max-speed has no schedule that bounds its replay time, so it also // spreads a fixed number of flushes over the recorded span: a sparse // recording then costs the same number of frames as a dense one. @@ -725,8 +724,9 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { // Events that change no pixel end a flush in recorded and accelerated // mode, where the pause before them is part of the video. Max-speed has no - // pauses: a flush at each of them would cost one frame per event, and a - // viewer cannot tell apart what happens within one minimum split. + // pauses: a flush at each of them would cost one frame per event. So only + // recorded time ends a max-speed flush, and output on both sides of such + // an event shares a flush when it is less than one minimum split apart. const flushBeforeNonVisualEvent = async (): Promise => { if (timing.mode !== 'max-speed') { await flushOutputBatch(); @@ -767,12 +767,12 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { // Output shows once its batch is a minimum split old, unless the // replay runs behind the schedule: then batches grow until it catches // up, so slow flushes lower the frame rate instead of stretching the - // video. Max-speed has no schedule to fall behind. It flushes every - // minimum split of recorded time, so each moment that a viewer could - // tell apart gets a frame, within the frame budget. The frame wait of - // such a flush is already the frame of its moment, so max-speed holds - // only after a flush that ends the replay or comes before a resize: a - // hold after every flush would export 30 ms steps slower than they ran. + // video. Max-speed has no schedule to fall behind. It flushes once per + // minimum split of recorded time at most, within the frame budget. + // The frame wait of such a flush is already its frame, so max-speed + // holds only after a flush that ends the replay or comes before a + // resize: a hold after every flush would export 30 ms steps slower + // than they ran. if ( pendingSinceTimestampMs !== null && eventTimestampMs - pendingSinceTimestampMs >= minSplitMs && diff --git a/test/integration/renderer-backend.test.ts b/test/integration/renderer-backend.test.ts index ccff5f78..f36a82f9 100644 --- a/test/integration/renderer-backend.test.ts +++ b/test/integration/renderer-backend.test.ts @@ -1073,6 +1073,99 @@ describe('GhosttyWebBackend integration', { timeout: 120_000 }, () => { ).toBe('abcd'); }); + // A resize can bring graphemes on the screen that no flush showed: growing + // the screen pulls lines back from the scrollback. The note that follows the + // resize records them, and the next flush then erases the screen. + it('records a glyph that a resize brings back from the scrollback', async () => { + await backend.boot(); + + await backend.replayWithTiming( + createReplayInput([ + { + seq: 0, + ts: timestampFor(0), + type: 'output', + payload: { + data: `\u0378\r\n${Array.from({ length: 30 }, (_, line) => `line ${String(line)}\r\n`).join('')}`, + }, + }, + { + seq: 1, + ts: timestampFor(1), + type: 'resize', + payload: { cols: 80, rows: 40 }, + }, + { + seq: 2, + ts: timestampFor(2), + type: 'output', + payload: { data: '\u001b[2J\u001b[Hok' }, + }, + ]), + { mode: 'max-speed', minFrameHoldMs: 16, finalFrameHoldMs: 200 }, + ); + + await expect(backend.recordedMissingGlyphs()).resolves.toEqual({ + missingGlyphs: [{ codePoint: 'U+0378' }], + }); + }); + + // The page keeps the size attributes of the body (data-cols, data-rows) in + // step with the terminal after every write. The call that ends a flush does + // it too, as write() does. + it('refreshes the size attributes of the document when a flush ends', async () => { + await backend.boot(); + const { page } = backend as unknown as { page: Page }; + await page.evaluate('window.__agentTty.resize(80, 24)'); + const sizeAttributes = (): Promise => + page.evaluate( + `document.body.dataset.cols + 'x' + document.body.dataset.rows`, + ) as Promise; + expect(await sizeAttributes()).toBe('80x24'); + await page.evaluate( + `document.body.dataset.cols = 'stale'; document.body.dataset.rows = 'stale'`, + ); + + await page.evaluate(`window.__agentTty.writeBatch(['ab'], false)`); + expect(await sizeAttributes()).toBe('stalexstale'); + await page.evaluate(`window.__agentTty.writeBatch(['cd'], true)`); + + expect(await sizeAttributes()).toBe('80x24'); + }); + + // The set of painted graphemes feeds a probe of every grapheme in it. The + // cap that the backend passes bounds the set, and a set that hit the cap is + // reported as incomplete, so the export result says that its list is cut. + it('bounds the painted graphemes by the cap of the call that ends a flush', async () => { + await backend.boot(); + const { page } = backend as unknown as { page: Page }; + await page.evaluate('window.__agentTty.resize(80, 24)'); + const painted = (): Promise<{ seen: string[]; complete: boolean } | null> => + page.evaluate( + `(() => { + const state = window.__agentTtyPainted; + return state === undefined ? null : { seen: [...state.seen], complete: state.complete }; + })()`, + ) as Promise<{ seen: string[]; complete: boolean } | null>; + + // Nothing is noted while the call does not ask for it. + await page.evaluate(`window.__agentTty.writeBatch(['abc'], true, null)`); + expect(await painted()).toBeNull(); + + await page.evaluate( + `window.__agentTty.writeBatch(['\\x1b[2J\\x1b[Habc'], true, 5)`, + ); + expect(await painted()).toEqual({ seen: ['a', 'b', 'c'], complete: true }); + + await page.evaluate( + `window.__agentTty.writeBatch(['\\x1b[2J\\x1b[Hdefgh'], true, 5)`, + ); + expect(await painted()).toEqual({ + seen: ['a', 'b', 'c', 'd', 'e'], + complete: false, + }); + }); + it('allows dispose to be called more than once', async () => { await backend.boot(); diff --git a/test/unit/renderer/ghosttyWebBackend.test.ts b/test/unit/renderer/ghosttyWebBackend.test.ts index 98174f15..9b37cdb5 100644 --- a/test/unit/renderer/ghosttyWebBackend.test.ts +++ b/test/unit/renderer/ghosttyWebBackend.test.ts @@ -1532,6 +1532,82 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { })); } + function resizeEvent( + seq: number, + offsetMs: number, + ): ReplayInput['events'][number] { + return { + seq, + ts: new Date(BASE_TIMESTAMP_MS + offsetMs).toISOString(), + type: 'resize', + payload: { cols: 100, rows: 30 }, + }; + } + + // The steps of a replay with the time of each, counted from the resize that + // sets the initial size: `flush` is a write of pending output, `resize` and + // `note` follow a resize event, `snapshot` is the read of the final state. + async function replayAndRecordTimeline( + events: ReplayInput['events'], + timing: ReplayTimingOptions, + ): Promise<{ step: string; atMs: number }[]> { + vi.useFakeTimers(); + const backend = createBackend(); + const timeline: { step: string; atMs: number }[] = []; + let replayStartedAtMs: number | undefined; + const record = (step: string): void => { + timeline.push({ + step, + atMs: performance.now() - (replayStartedAtMs ?? performance.now()), + }); + }; + + Object.assign(backend as object, { + isBooted: true, + page: { isClosed: () => false }, + resizeBridge: (_page: object, cols: number) => { + if (replayStartedAtMs === undefined) { + replayStartedAtMs = performance.now(); + } else { + record(`resize ${String(cols)}`); + } + return Promise.resolve(); + }, + flushOutputBatch: (_page: object, dataChunks: string[]) => { + record(`flush ${dataChunks.join(',')}`); + return Promise.resolve(); + }, + notePaintedGraphemes: () => { + record('note'); + return Promise.resolve(); + }, + readHarnessSnapshot: () => { + record('snapshot'); + return Promise.resolve( + createHarnessSnapshotPayload({ + cols: 80, + rows: 24, + visibleLines: [], + }), + ); + }, + }); + + const replay = backend.replayWithTiming( + { + sessionId: 'renderer-unit-session', + initialCols: 80, + initialRows: 24, + targetSeq: Math.max(...events.map((event) => event.seq)), + events, + }, + timing, + ); + await vi.runAllTimersAsync(); + await replay; + return timeline; + } + it('shows each output at its recorded time when the outputs are far apart', async () => { const flushes = await replayAndRecordFlushes( [ @@ -1629,7 +1705,7 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { ]); }); - it('flushes max-speed output every frame hold of recorded time, so each moment gets a frame', async () => { + it('ends a max-speed flush at the first event that is a frame hold after the first output of the batch', async () => { const flushes = await replayAndRecordFlushes( [ outputEvent(0, 0), @@ -1651,7 +1727,7 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { ]); }); - it('keeps one max-speed flush per moment when the outputs are far apart', async () => { + it('keeps one max-speed flush per output when the outputs are far apart', async () => { const flushes = await replayAndRecordFlushes( [ outputEvent(0, 0), @@ -1669,7 +1745,7 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { ]); }); - // The frame wait of a flush is the frame of its moment. A hold after every + // The frame wait of a flush is its frame. A hold after every // flush would make each 30 ms step cost 44 ms to export. it('holds max-speed frames only after the last flush of the replay', async () => { const flushes = await replayAndRecordFlushes( @@ -1692,6 +1768,72 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { ]); }); + // The page paints a flush on its next animation frame. The hold gives that + // paint time to reach the video before the resize changes the canvas. + it('holds one frame hold between the last flush and a resize', async () => { + const timeline = await replayAndRecordTimeline( + [ + outputEvent(0, 0), + resizeEvent(1, 5), + outputEvent(2, 10), + exitEvent(3, 12), + ], + { mode: 'max-speed', minFrameHoldMs: 16, finalFrameHoldMs: 0 }, + ); + + expect(timeline.map((entry) => entry.step)).toEqual([ + 'flush out0', + 'resize 100', + 'note', + 'flush out2', + 'snapshot', + ]); + const atMs = (step: string): number | undefined => + timeline.find((entry) => entry.step === step)?.atMs; + expect((atMs('resize 100') ?? 0) - (atMs('flush out0') ?? 0)).toBe(16); + }); + + it.each([ + ['recorded', RECORDED], + ['max-speed', MAX_SPEED], + ])( + 'notes the painted graphemes right after a resize event, in %s mode', + async (_name, timing) => { + const timeline = await replayAndRecordTimeline( + [ + outputEvent(0, 0), + resizeEvent(1, 200), + outputEvent(2, 400), + exitEvent(3, 600), + ], + timing, + ); + + const steps = timeline.map((entry) => entry.step); + const resizeAt = steps.indexOf('resize 100'); + expect(resizeAt).toBeGreaterThan(0); + expect(steps[resizeAt + 1]).toBe('note'); + expect(timeline[resizeAt + 1]?.atMs).toBe(timeline[resizeAt]?.atMs); + }, + ); + + it('holds one frame hold after a resize, and one after the last flush of the replay', async () => { + const timeline = await replayAndRecordTimeline( + [ + outputEvent(0, 0), + resizeEvent(1, 5), + outputEvent(2, 10), + exitEvent(3, 12), + ], + { mode: 'max-speed', minFrameHoldMs: 16, finalFrameHoldMs: 0 }, + ); + + const atMs = (step: string): number => + timeline.find((entry) => entry.step === step)?.atMs ?? Number.NaN; + expect(atMs('flush out2') - atMs('note')).toBe(16); + expect(atMs('snapshot') - atMs('flush out2')).toBe(16); + }); + // A flush at every input, marker or exit event would cost one frame for each // of them, and a recording of typing has an input for every output. it('does not end a max-speed flush at an event that changes no pixel', async () => { @@ -1722,29 +1864,41 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { expect(flushes.map((flush) => flush.chunks)).toEqual([['out0'], ['out2']]); }); - // 10,000 outputs with an input between each pair, 40 ms apart: 400 s of - // recording. Without a budget, every output is a moment of its own and the - // replay makes 10,000 flushes of a frame each, more than 5 minutes. - it('spreads max-speed flushes over a long recording, one per 167 ms of recorded time', async () => { - const events: ReplayInput['events'] = []; - for (let index = 0; index < 10_000; index += 1) { - events.push(outputEvent(2 * index, 40 * index)); - events.push(inputEvent(2 * index + 1, 40 * index + 20)); - } - events.push(exitEvent(20_000, 400_000)); - const flushes = await replayAndRecordFlushes(events, MAX_SPEED); - - // A flush ends at the first event 167 ms after the first output of the - // batch, so each one takes 5 outputs (200 ms): about 2,000 flushes. - expect(flushes.length).toBeGreaterThan(1_800); - expect(flushes.length).toBeLessThanOrEqual(2_401); - // No output is lost or moved. - expect(flushes.flatMap((flush) => flush.chunks)).toEqual( - events - .filter((event) => event.type === 'output') - .map((event) => `out${String(event.seq)}`), - ); - }); + // 9,000 outputs over 180 s or 360 s. Without a budget, every output is a + // flush of its own and the replay makes 9,000 flushes of a frame each. With + // a budget of 2,400 flushes, a flush ends at the first event that is 75 ms or + // 150 ms (span / 2,400) after the first output of its batch. That is the + // fourth output in both cases: 2,250 flushes. A budget of 2,000 flushes gives + // 1,800, and one of 3,000 flushes gives 3,000. + it.each([ + { label: 'outputs 20 ms apart', outputGapMs: 20, withInputs: false }, + { + label: 'outputs 40 ms apart with an input between', + outputGapMs: 40, + withInputs: true, + }, + ])( + 'ends a max-speed flush every 1/2,400 of the recorded span: 9,000 $label make 2,250 flushes', + async ({ outputGapMs, withInputs }) => { + const events: ReplayInput['events'] = []; + for (let index = 0; index < 9_000; index += 1) { + events.push(outputEvent(events.length, outputGapMs * index)); + if (withInputs) { + events.push(inputEvent(events.length, outputGapMs * index + 20)); + } + } + events.push(exitEvent(events.length, 180_000 * (withInputs ? 2 : 1))); + const flushes = await replayAndRecordFlushes(events, MAX_SPEED); + + expect(flushes.length).toBe(2_250); + // No output is lost or moved. + expect(flushes.flatMap((flush) => flush.chunks)).toEqual( + events + .filter((event) => event.type === 'output') + .map((event) => `out${String(event.seq)}`), + ); + }, + ); it('measures the frame budget over the events that the replay applies', async () => { // 20 ms apart: the whole log spans 200 s, but the last 10 events span 180 ms. @@ -1759,13 +1913,13 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { 9_989, ); - // 20 ms apart is above the frame hold of 16 ms: a moment each. + // 20 ms apart is above the frame hold of 16 ms: a flush each. expect(flushes.map((flush) => flush.chunks)).toEqual( events.slice(9_990).map((event) => [`out${String(event.seq)}`]), ); }); - it('does not merge max-speed moments when a flush costs more than a frame hold', async () => { + it('does not merge max-speed flushes when a flush costs more than a frame hold', async () => { const events = Array.from({ length: 101 }, (_, index) => outputEvent(index, index * 10), ); @@ -1773,8 +1927,8 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { const flushes = await replayAndRecordFlushes(events, MAX_SPEED, 30); // Recorded timing lets batches grow while the replay is behind its - // schedule. Max-speed has no schedule, so a slow flush must not hide a - // moment: every flush holds the outputs of one frame hold (two events). + // schedule. Max-speed has no schedule, so a slow flush must not merge two + // flushes: every flush holds the outputs of one frame hold (two events). expect(flushes.length).toBe(51); expect(flushes.every((flush) => flush.chunks.length <= 2)).toBe(true); expect(flushes.flatMap((flush) => flush.chunks)).toEqual( From 3da5a5785b8aa736919f003c1d5ccc91c4183b6e Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 22:23:43 +0000 Subject: [PATCH 19/28] docs: say that max-speed shows most steps of a 30 ms progress bar --- docs/USAGE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/USAGE.md b/docs/USAGE.md index 15a41f49..5cbc9d55 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -294,7 +294,7 @@ agent-tty record export --format webm --json | jq -c '.result.missi # [{"codePoint":"U+0378"}] ``` -WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. A frame of the video shows the screen after the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event, and a state that the next output of the frame replaces is not shown. In `recorded` and `accelerated` mode, an event that changes no pixel (an input, a marker, a run-complete, a signal or the exit) also ends a frame. In `max-speed` mode it does not: only recorded time ends a frame, so output on both sides of an input event that is less than 16 ms apart shares one frame. `max-speed` removes the idle time. It makes a frame at most once per 16 ms of recorded time, so a progress bar that updates every 30 ms shows its steps and not only its last one. It makes at most about 2,400 frames, so a recording longer than 38 s gets one frame per 1/2,400 of its recorded time instead (a 1-hour recording: one per 1.5 s), and its export time depends on its length and not on its number of events. `max-speed` waits for one animation frame after each frame but holds no further time between two frames, so two frames that follow each other can end up in one video frame. The video has 25 frames per second, so it can skip a state that lasts less than one video frame (40 ms). +WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. A frame of the video shows the screen after the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event, and a state that the next output of the frame replaces is not shown. In `recorded` and `accelerated` mode, an event that changes no pixel (an input, a marker, a run-complete, a signal or the exit) also ends a frame. In `max-speed` mode it does not: only recorded time ends a frame, so output on both sides of an input event that is less than 16 ms apart shares one frame. `max-speed` removes the idle time. It makes a frame at most once per 16 ms of recorded time, so a progress bar that updates every 30 ms shows most of its steps and not only its last one. It makes at most about 2,400 frames, so a recording longer than 38 s gets one frame per 1/2,400 of its recorded time instead (a 1-hour recording: one per 1.5 s), and its export time depends on its length and not on its number of events. `max-speed` waits for one animation frame after each frame but holds no further time between two frames, so two frames that follow each other can end up in one video frame. The video has 25 frames per second, so it can skip a state that lasts less than one video frame (40 ms). SVG export renders styled grid frames from the event log through the native `libghostty-vt` backend with no browser and no ffmpeg, so it requires the optional `@coder/libghostty-vt-node` package (there is no `ghostty-web` fallback). The output is deterministic: exporting the same session twice produces byte-identical, diffable SVG. `--format svg` writes a still image of the final screen and ignores event timing, so it also works on a log whose clock stepped backward (`--animate` refuses such a log and names the event where the clock stepped back); add `--animate` for an animated SVG of de-duplicated frames replayed with recorded event-log timing (`--timing` is not supported with SVG). From bc68da9ef2749c39dccae7f8194d9647144dfe87 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 23:27:52 +0000 Subject: [PATCH 20/28] test: wait for a quiet first screen before counting screencast frames of a replay --- test/integration/renderer-backend.test.ts | 26 +++++++++++++++++------ 1 file changed, 20 insertions(+), 6 deletions(-) diff --git a/test/integration/renderer-backend.test.ts b/test/integration/renderer-backend.test.ts index f36a82f9..8a1ee507 100644 --- a/test/integration/renderer-backend.test.ts +++ b/test/integration/renderer-backend.test.ts @@ -894,21 +894,35 @@ describe('GhosttyWebBackend integration', { timeout: 120_000 }, () => { await new Promise((resolve) => setTimeout(resolve, 100)); }, }); - // Wait for the first frame, so that it is the screen before the replay. + // Wait until the screen before the replay is quiet: a slow host can + // deliver a late paint of the first screen, which is not a frame of the + // replay. The screen before the replay counts as one frame. + let framesSeen = 0; + let quietSinceMs = Date.now(); await expect - .poll(() => frames.length, { timeout: 10_000 }) - .toBeGreaterThan(0); + .poll( + () => { + if (frames.length !== framesSeen) { + framesSeen = frames.length; + quietSinceMs = Date.now(); + } + return framesSeen > 0 && Date.now() - quietSinceMs >= 300; + }, + { timeout: 10_000 }, + ) + .toBe(true); + const framesBefore = frames.length; await replay(backend, input); // The last colour must reach the listener. Then wait a little more, so // that a frame between two calls would have arrived too. await expect .poll(() => frames.length, { timeout: 10_000 }) - .toBeGreaterThan(1); + .toBeGreaterThan(framesBefore); await new Promise((resolve) => setTimeout(resolve, 500)); await screencast.send('Page.stopScreencast'); - // The frame before the replay, then the frame of the last colour. - expect(frames).toHaveLength(2); + // The screen before the replay, then the frame of the last colour. + expect(frames).toHaveLength(framesBefore + 1); }, ); From e39bd15135905aa93505402a4a14cdbd1b988735 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Sat, 7 Nov 2026 00:52:03 +0000 Subject: [PATCH 21/28] docs: say that max-speed also holds a frame after a resize --- src/export/README.md | 2 +- src/renderer/ghosttyWeb/README.md | 2 +- src/renderer/ghosttyWeb/backend.ts | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/src/export/README.md b/src/export/README.md index 7733b406..5a471594 100644 --- a/src/export/README.md +++ b/src/export/README.md @@ -25,7 +25,7 @@ This directory converts a session's event log into recording exports. `generateA - Defaults: timing mode `recorded` and profile `reference-dark` [webm.ts:generateWebmExport]. - A requested `libghostty-vt` renderer is replaced with `ghostty-web`. A code comment says `libghostty-vt` is semantic-only [webm.ts:generateWebmExport]. - The default backend factory accepts only `ghostty-web` [webm.ts:createDefaultBackend]. -- Timing presets: `accelerated` caps gaps at 400 ms and holds frames at least 100 ms. `max-speed` uses a frame hold of 16 ms, but holds it only after the last flush of the replay and before a resize (see the `ghostty-web` README). Each mode holds the final frame (1000 ms, or 500 ms for `max-speed`) [webm.ts:buildReplayTimingOptions]. +- Timing presets: `accelerated` caps gaps at 400 ms and holds frames at least 100 ms. `max-speed` uses a frame hold of 16 ms, but holds it only after the last flush of the replay and before and after a resize (see the `ghostty-web` README). Each mode holds the final frame (1000 ms, or 500 ms for `max-speed`) [webm.ts:buildReplayTimingOptions]. - The timed replay fails after `REPLAY_TIMEOUT_MS` (5 minutes) [webm.ts:REPLAY_TIMEOUT_MS]. - `renderGridFramesToSvg` needs the profile font size `SVG_FONT_SIZE` (14) and draws fixed 8.4x18 px cells, so equal input gives byte-identical output [svg.ts:renderGridFramesToSvg]. It always embeds the JetBrains Mono subset, and embeds each other bundled face only when a glyph needs it [svg.ts:embeddedFontAssets]. Output past `defaultMaxSvgOutputLength` of the V8 heap limit (the heap limit divided by 8 bytes, at most `MAX_SVG_OUTPUT_LENGTH`) fails with `EXPORT_ERROR` [svg.ts:defaultMaxSvgOutputLength]. diff --git a/src/renderer/ghosttyWeb/README.md b/src/renderer/ghosttyWeb/README.md index 18b39f20..8abfa834 100644 --- a/src/renderer/ghosttyWeb/README.md +++ b/src/renderer/ghosttyWeb/README.md @@ -29,7 +29,7 @@ This directory holds the `ghostty-web` reference renderer. `GhosttyWebBackend` s - The constructor asserts a non-empty `sessionId`, `profile.name` and `profile.fontFamily`, a positive `fontSize`, and hex colors. With `videoOptions`, `outputDir` must be absolute and the size must use positive integers [backend.ts:GhosttyWebBackend.constructor]. - Replay cannot rewind: `targetSeq` must be >= the last applied seq. The initial dimensions cannot change after the first replay. `input.sessionId` must match the backend [backend.ts:GhosttyWebBackend.replayTo]. - `replayWithTiming` asserts strictly increasing seqs and non-decreasing timestamps [backend.ts:GhosttyWebBackend.replayWithTiming]. -- `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` has no schedule to fall behind, so only recorded time ends its flushes. A flush ends at the first event that is at least one frame hold of recorded time after the batch's first output. An event that changes no pixel does not end it: input, marker, run-complete, signal and exit events leave the batch as it is, because a flush at each of them would cost a frame per event. A resize and the end of the replay still end it. `max-speed` also spreads at most `MAX_SPEED_FRAME_BUDGET` (2,400) flushes over the recorded span of the events that it applies: the distance between the first output of a flush and the event that ends it is at least the larger of the frame hold and the span divided by 2,400. So a `max-speed` replay makes about one flush per frame hold of recorded time for a recording shorter than 38 s and at most about 2,400 flushes for a longer one, whatever its number of events. Output that arrives less than that distance after the first output of a flush shares the flush, also when an input or a marker sits between, so a state that the next output of the flush replaces is never painted. A flush waits for one animation frame. `max-speed` holds the frame hold only after the last flush of the replay and before a resize, so two flushes that follow each other can paint within one video frame. In `recorded` and `accelerated` mode, every event that changes no pixel ends a flush, and a hold follows every flush. Events with the same timestamp share a flush at any hold, 0 included, because a flush needs at least 1 ms of recorded time [backend.ts:GhosttyWebBackend.replayWithTiming]. +- `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` has no schedule to fall behind, so only recorded time ends its flushes. A flush ends at the first event that is at least one frame hold of recorded time after the batch's first output. An event that changes no pixel does not end it: input, marker, run-complete, signal and exit events leave the batch as it is, because a flush at each of them would cost a frame per event. A resize and the end of the replay still end it. `max-speed` also spreads at most `MAX_SPEED_FRAME_BUDGET` (2,400) flushes over the recorded span of the events that it applies: the distance between the first output of a flush and the event that ends it is at least the larger of the frame hold and the span divided by 2,400. So a `max-speed` replay makes about one flush per frame hold of recorded time for a recording shorter than 38 s and at most about 2,400 flushes for a longer one, whatever its number of events. Output that arrives less than that distance after the first output of a flush shares the flush, also when an input or a marker sits between, so a state that the next output of the flush replaces is never painted. A flush waits for one animation frame. `max-speed` holds the frame hold only after the last flush of the replay and before and after a resize, so two flushes that follow each other can paint within one video frame. In `recorded` and `accelerated` mode, every event that changes no pixel ends a flush, and a hold follows every flush. Events with the same timestamp share a flush at any hold, 0 included, because a flush needs at least 1 ms of recorded time [backend.ts:GhosttyWebBackend.replayWithTiming]. - `writeBatch(chunks, endsFlush, paintedCap)` takes the chunks of one bridge call. A flush that has more than 1000 chunks needs several calls, and ghostty-web paints on every animation frame, so the page holds the chunks of a call that does not end the flush. The call that ends it (`endsFlush`, set by the backend on the last call only) writes every held chunk in one task, so no frame paints a state between two calls, and then resolves after one frame. ghostty-web calls a write callback one animation frame (about 17 ms) after the write, and waiting on every chunk (`write`) cost one frame per output event. `write()` and `resize()` throw while chunks are held. `discardHeldChunks()` drops them: the backend calls it when a flush fails, because `HostRendererManager` keeps the backend and a retry would otherwise meet them. A `paintedCap` that is not null makes the call that ends the flush add the graphemes on the screen to the set that the missing-glyph check of `replayWithTiming` probes, so a flush is one page call. The call returns the message of a failed note and not an error, because a failed note must not fail the flush. In `replayWithTiming`, the chunks of one flush share one video frame [embeddedHarnessHtml.ts:EMBEDDED_HARNESS_HTML]. - `snapshot()` and `screenshot()` throw until `replayTo()` or `replayWithTiming()` has run once and set the initial size. Both replays accept `targetSeq` -1 for an empty Event Log. A replay over zero events sets `lastAppliedSeq` to `targetSeq` [backend.ts:GhosttyWebBackend.snapshot]. - `missingGlyphs` lists a code point only when a bundled face that lacks it drew it. Chromium draws a cluster with one font: the first stack font that covers all of it, or else the first font of the stack, which is bundled in both built-in profiles and then draws `.notdef` boxes. CDP names the face that drew a probe by its PostScript name, so a family alias or several faces under one family resolve to the right cmap. When one bundled face drew the whole grapheme, every code point that face lacks is listed. A probe with no reported font is judged against every face of the first family. When the grapheme is split over several fonts, each uncovered code point is judged by its own probe, and a mark's probe on its base is compared with a probe of the base alone. Glyphs that a host font draws are not listed. The check runs after the PNG is written, removes its probe spans, and caches each probe text's result for the page, clearing the cache when it would pass `MAX_CACHED_GLYPH_PROBES` (16384) entries. It probes at most `MAX_GLYPH_PROBES` texts and lists at most `MAX_MISSING_GLYPH_CELLS` cells (4096 each). `probeUsedFonts` sends its CDP queries in batches of 256 and stops starting batches after `GLYPH_PROBE_BUDGET_MS` (1000 ms). The result sets `missingGlyphsTruncated` when any of these stops the check early. Family names match without regard to case. A web font whose PostScript name the profile does not bundle fails the check. A failed check is logged at debug level and leaves the field out, so the screenshot still succeeds. `replayWithTiming()` records the viewport graphemes in the page after each output batch and each resize (at most `MAX_CACHED_GLYPH_PROBES`), and the final frame hold judges them once, with the hold as the probe budget, so the check does not lengthen the video. `recordedMissingGlyphs()` returns that result. A failed note or check leaves the record export fields out. A profile whose font stack does not start with a bundled font gets no `missingGlyphs` field [missingGlyphs.ts:probeUsedFonts] [backend.ts:GhosttyWebBackend.screenshot]. diff --git a/src/renderer/ghosttyWeb/backend.ts b/src/renderer/ghosttyWeb/backend.ts index eef422e8..8fab2484 100644 --- a/src/renderer/ghosttyWeb/backend.ts +++ b/src/renderer/ghosttyWeb/backend.ts @@ -771,8 +771,8 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { // minimum split of recorded time at most, within the frame budget. // The frame wait of such a flush is already its frame, so max-speed // holds only after a flush that ends the replay or comes before a - // resize: a hold after every flush would export 30 ms steps slower - // than they ran. + // resize (a resize is followed by a hold too): a hold after every + // flush would export 30 ms steps slower than they ran. if ( pendingSinceTimestampMs !== null && eventTimestampMs - pendingSinceTimestampMs >= minSplitMs && From e98841cdc0babe67f8d8178b1a01725fac72c73a Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Sun, 8 Nov 2026 12:04:14 +0000 Subject: [PATCH 22/28] docs: do not claim that max-speed shows most steps of a 30 ms progress bar --- docs/USAGE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/USAGE.md b/docs/USAGE.md index c1b0087d..9cbb14fb 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -296,7 +296,7 @@ agent-tty record export --format webm --json | jq -c '.result.missi # [{"codePoint":"U+0378"}] ``` -WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. A frame of the video shows the screen after the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event, and a state that the next output of the frame replaces is not shown. In `recorded` and `accelerated` mode, an event that changes no pixel (an input, a marker, a run-complete, a signal or the exit) also ends a frame. In `max-speed` mode it does not: only recorded time ends a frame, so output on both sides of an input event that is less than 16 ms apart shares one frame. `max-speed` removes the idle time. It makes a frame at most once per 16 ms of recorded time, so a progress bar that updates every 30 ms shows most of its steps and not only its last one. It makes at most about 2,400 frames, so a recording longer than 38 s gets one frame per 1/2,400 of its recorded time instead (a 1-hour recording: one per 1.5 s), and its export time depends on its length and not on its number of events. `max-speed` waits for one animation frame after each frame but holds no further time between two frames, so two frames that follow each other can end up in one video frame. The video has 25 frames per second, so it can skip a state that lasts less than one video frame (40 ms). +WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. A frame of the video shows the screen after the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event, and a state that the next output of the frame replaces is not shown. In `recorded` and `accelerated` mode, an event that changes no pixel (an input, a marker, a run-complete, a signal or the exit) also ends a frame. In `max-speed` mode it does not: only recorded time ends a frame, so output on both sides of an input event that is less than 16 ms apart shares one frame. `max-speed` removes the idle time. It makes a frame at most once per 16 ms of recorded time, so a progress bar that updates every 30 ms shows many of its steps and not only its last one. It makes at most about 2,400 frames, so a recording longer than 38 s gets one frame per 1/2,400 of its recorded time instead (a 1-hour recording: one per 1.5 s), and its export time depends on its length and not on its number of events. `max-speed` waits for one animation frame after each frame but holds no further time between two frames, so two frames that follow each other can end up in one video frame. The video has 25 frames per second, so it can skip a state that lasts less than one video frame (40 ms). A step of 30 ms is shorter than that, and the number of steps that survive depends on how fast the host runs the export. SVG export renders styled grid frames from the event log through the native `libghostty-vt` backend with no browser and no ffmpeg, so it requires the optional `@coder/libghostty-vt-node` package (there is no `ghostty-web` fallback). The output is deterministic: exporting the same session twice produces byte-identical, diffable SVG. `--format svg` writes a still image of the final screen and ignores event timing, so it also works on a log whose clock stepped backward (`--animate` refuses such a log and names the event where the clock stepped back); add `--animate` for an animated SVG of de-duplicated frames replayed with recorded event-log timing (`--timing` is not supported with SVG). From 21b0ac13659a9f092648fb080cb517bce4026ea8 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Sun, 8 Nov 2026 13:35:52 +0000 Subject: [PATCH 23/28] fix: report progress after the resize call of a timed WebM replay A resize event is two page calls: the resize and the note of the painted graphemes. A report between them keeps each call a stretch of its own for the stall watchdog. The stall watchdog tests also follow the batched replay of #358 now, and new tests cover the survivors of the earlier mutation runs: the "after event 0" message, the cause and output path of a stall error, an infinite wait, the stall window check and a late report after the replay ended. --- src/renderer/backend.ts | 5 +- src/renderer/ghosttyWeb/README.md | 2 +- src/renderer/ghosttyWeb/backend.ts | 7 +- test/unit/commands/record-export.test.ts | 32 +++++ test/unit/export/webm.test.ts | 144 ++++++++++++++++--- test/unit/renderer/ghosttyWebBackend.test.ts | 83 ++++++++++- 6 files changed, 240 insertions(+), 33 deletions(-) diff --git a/src/renderer/backend.ts b/src/renderer/backend.ts index aaf68780..25a5f183 100644 --- a/src/renderer/backend.ts +++ b/src/renderer/backend.ts @@ -123,8 +123,9 @@ export interface ReplayProgress { export interface VideoCapableRendererBackend extends RendererBackend { /** * Replay events with controlled timing for video capture. `onProgress` - * runs before each wait on the timing schedule and after each batch of - * output that the replay writes to the renderer. + * runs before each wait on the timing schedule, after each batch of + * output that the replay writes to the renderer and after the renderer + * resizes for a resize event. */ replayWithTiming( input: ReplayInput, diff --git a/src/renderer/ghosttyWeb/README.md b/src/renderer/ghosttyWeb/README.md index 0de91728..54ca1be6 100644 --- a/src/renderer/ghosttyWeb/README.md +++ b/src/renderer/ghosttyWeb/README.md @@ -29,7 +29,7 @@ This directory holds the `ghostty-web` reference renderer. `GhosttyWebBackend` s - The constructor asserts a non-empty `sessionId`, `profile.name` and `profile.fontFamily`, a positive `fontSize`, and hex colors. With `videoOptions`, `outputDir` must be absolute and the size must use positive integers [backend.ts:GhosttyWebBackend.constructor]. - Replay cannot rewind: `targetSeq` must be >= the last applied seq. The initial dimensions cannot change after the first replay. `input.sessionId` must match the backend [backend.ts:GhosttyWebBackend.replayTo]. - `replayWithTiming` asserts strictly increasing seqs and non-decreasing timestamps [backend.ts:GhosttyWebBackend.replayWithTiming]. -- `replayWithTiming` calls the optional `onProgress` with a `ReplayProgress` in two places. Before each wait on the schedule, `scheduledWaitMs` is the length of that wait. After each bridge batch of a flush (and after `notePaintedGraphemes`), `scheduledWaitMs` is 0. `seq` is the last event that the replay had reached. `generateWebmExport` uses these reports to tell a stall from a wait [backend.ts:GhosttyWebBackend.replayWithTiming]. +- `replayWithTiming` calls the optional `onProgress` with a `ReplayProgress` in three places. Before each wait on the schedule, `scheduledWaitMs` is the length of that wait. After each bridge call of a flush, and after the resize call of a resize event, `scheduledWaitMs` is 0. `seq` is the last event that the replay had reached. `generateWebmExport` uses these reports to tell a stall from a wait [backend.ts:GhosttyWebBackend.replayWithTiming]. - `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` has no schedule to fall behind, so only recorded time ends its flushes. A flush ends at the first event that is at least one frame hold of recorded time after the batch's first output. An event that changes no pixel does not end it: input, marker, run-complete, signal and exit events leave the batch as it is, because a flush at each of them would cost a frame per event. A resize and the end of the replay still end it. `max-speed` also spreads at most `MAX_SPEED_FRAME_BUDGET` (2,400) flushes over the recorded span of the events that it applies: the distance between the first output of a flush and the event that ends it is at least the larger of the frame hold and the span divided by 2,400. So a `max-speed` replay makes about one flush per frame hold of recorded time for a recording shorter than 38 s and at most about 2,400 flushes for a longer one, whatever its number of events. Output that arrives less than that distance after the first output of a flush shares the flush, also when an input or a marker sits between, so a state that the next output of the flush replaces is never painted. A flush waits for one animation frame. `max-speed` holds the frame hold only after the last flush of the replay and before and after a resize, so two flushes that follow each other can paint within one video frame. In `recorded` and `accelerated` mode, every event that changes no pixel ends a flush, and a hold follows every flush. Events with the same timestamp share a flush at any hold, 0 included, because a flush needs at least 1 ms of recorded time [backend.ts:GhosttyWebBackend.replayWithTiming]. - `writeBatch(chunks, endsFlush, paintedCap)` takes the chunks of one bridge call. A flush that has more than 1000 chunks needs several calls, and ghostty-web paints on every animation frame, so the page holds the chunks of a call that does not end the flush. The call that ends it (`endsFlush`, set by the backend on the last call only) writes every held chunk in one task, so no frame paints a state between two calls, and then resolves after one frame. ghostty-web calls a write callback one animation frame (about 17 ms) after the write, and waiting on every chunk (`write`) cost one frame per output event. `write()` and `resize()` throw while chunks are held. `discardHeldChunks()` drops them: the backend calls it when a flush fails, because `HostRendererManager` keeps the backend and a retry would otherwise meet them. A `paintedCap` that is not null makes the call that ends the flush add the graphemes on the screen to the set that the missing-glyph check of `replayWithTiming` probes, so a flush is one page call. The call returns the message of a failed note and not an error, because a failed note must not fail the flush. In `replayWithTiming`, the chunks of one flush share one video frame [embeddedHarnessHtml.ts:EMBEDDED_HARNESS_HTML]. - `snapshot()` and `screenshot()` throw until `replayTo()` or `replayWithTiming()` has run once and set the initial size. Both replays accept `targetSeq` -1 for an empty Event Log. A replay over zero events sets `lastAppliedSeq` to `targetSeq` [backend.ts:GhosttyWebBackend.snapshot]. diff --git a/src/renderer/ghosttyWeb/backend.ts b/src/renderer/ghosttyWeb/backend.ts index c7dcee5d..96269add 100644 --- a/src/renderer/ghosttyWeb/backend.ts +++ b/src/renderer/ghosttyWeb/backend.ts @@ -822,6 +822,9 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { this.currentCols = event.payload.cols; this.currentRows = event.payload.rows; resizeEventCount += 1; + // The resize and the note that follows are two page calls. A + // report between them keeps each call a stretch of its own. + onProgress?.({ seq: event.seq, scheduledWaitMs: 0 }); await this.notePaintedGraphemes(page); await waitForDelay(resolvedMinFrameHoldMs); break; @@ -1741,7 +1744,7 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { /** * Writes the chunks of one flush into the page. With `notePainted`, the page * also notes the graphemes on the screen once the flush is written, in the - * call that ends the flush. + * call that ends the flush. `afterEachBatch` runs after each page call. */ private async flushOutputBatch( page: Page, @@ -1779,7 +1782,7 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { endsFlush, endsFlush && notePainted ? this.paintedNoteCap() : null, ); - await afterEachBatch?.(); + afterEachBatch?.(); } } catch (error) { // The page still holds the chunks of the calls that did succeed. The diff --git a/test/unit/commands/record-export.test.ts b/test/unit/commands/record-export.test.ts index 75b910ed..0d366c00 100644 --- a/test/unit/commands/record-export.test.ts +++ b/test/unit/commands/record-export.test.ts @@ -1668,6 +1668,11 @@ describe('record export command', () => { timingMode: 'recorded', position: 'after event 1234', }, + { + lastSeq: 0, + timingMode: 'accelerated', + position: 'after event 0', + }, { lastSeq: -1, timingMode: 'max-speed', @@ -1704,6 +1709,33 @@ describe('record export command', () => { }, ); + it('keeps the requested output path and the stall error in the error of a stalled WebM replay', async () => { + const outputDirectory = await createTemporaryDirectory( + 'agent-tty-record-export-stall-', + ); + const out = join(outputDirectory, 'stalled.webm'); + const stall = new WebmReplayStallError(300_000, 12, 'recorded'); + mocks.recordingFilename.mockReturnValue('recording-1-webm.webm'); + mocks.generateWebmExport.mockRejectedValue(stall); + + const failure: unknown = await runRecordExportCommand({ + context: TEST_CONTEXT, + json: true, + sessionId: 'session-01', + format: 'webm', + out, + }).then( + () => undefined, + (error: unknown) => error, + ); + + expect(failure).toMatchObject({ + code: ERROR_CODES.EXPORT_ERROR, + details: { out }, + }); + expect((failure as { cause?: unknown }).cause).toBe(stall); + }); + it('keeps the generic message for other WebM export failures', async () => { mocks.recordingFilename.mockReturnValue('recording-1-webm.webm'); mocks.generateWebmExport.mockRejectedValue(new Error('replay failed')); diff --git a/test/unit/export/webm.test.ts b/test/unit/export/webm.test.ts index 3b33c174..c31c01f3 100644 --- a/test/unit/export/webm.test.ts +++ b/test/unit/export/webm.test.ts @@ -532,6 +532,32 @@ describe('generateWebmExport', () => { expect(mockBackend.dispose).toHaveBeenCalledTimes(1); }); + it.each([0, -1, Number.NaN, Number.POSITIVE_INFINITY])( + 'refuses a stall window of %s before it creates a backend', + async (stallWindowMs) => { + const mockBackend = createMockBackend(); + const backendFactory = vi.fn(() => mockBackend.backend); + + await expect( + generateWebmExport( + { + sessionId: 'session-01', + sessionDir: '/tmp/agent-tty/sessions/session-01', + manifest: createManifest(), + events: createEvents(), + outputPath: '/tmp/exports/recording-1-webm.webm', + }, + { backendFactory, stallWindowMs }, + ), + ).rejects.toThrow( + 'stallWindowMs must be a positive number when provided', + ); + + expect(backendFactory).not.toHaveBeenCalled(); + expect(mockBackend.replayWithTiming).not.toHaveBeenCalled(); + }, + ); + it('disposes the backend when finalizeVideo throws', async () => { const finalizeError = new Error('finalize failed'); const finalizeVideo = vi.fn().mockRejectedValue(finalizeError); @@ -803,11 +829,34 @@ describe('generateWebmExport stall watchdog', () => { expect(vi.getTimerCount()).toBe(0); }); + it.each(['succeeded', 'failed'] as const)( + 'starts no timer when a replay that already %s reports again', + async (ending) => { + let reportLate: (progress: ReplayProgress) => void = () => undefined; + const { outcome } = await startExport((onProgress) => { + reportLate = onProgress; + return ending === 'succeeded' + ? Promise.resolve(createReplayState()) + : Promise.reject(new Error('replay failed')); + }); + + await outcome; + reportLate({ seq: 9, scheduledWaitMs: DAY_MS }); + + expect(vi.getTimerCount()).toBe(0); + }, + ); + it.each([ { seq: 0.5, scheduledWaitMs: 0, message: 'seq must be an integer' }, { seq: -2, scheduledWaitMs: 0, message: 'seq must be an integer' }, { seq: 0, scheduledWaitMs: -1, message: 'scheduledWaitMs must be' }, { seq: 0, scheduledWaitMs: Number.NaN, message: 'scheduledWaitMs must be' }, + { + seq: 0, + scheduledWaitMs: Number.POSITIVE_INFINITY, + message: 'scheduledWaitMs must be', + }, ])( 'rejects a progress report of seq $seq and wait $scheduledWaitMs', async ({ seq, scheduledWaitMs, message }) => { @@ -833,17 +882,24 @@ describe('generateWebmExport stall watchdog with the ghostty-web replay loop', ( const DAY_MS = 24 * 60 * 60 * 1000; const WINDOW_MS = 5 * 60 * 1000; - type ReplayEvent = { - seq: number; - type: 'output' | 'input_text'; - ts: string; - payload: { data: string }; - }; + type ReplayEvent = + | { + seq: number; + type: 'output' | 'input_text'; + ts: string; + payload: { data: string }; + } + | { + seq: number; + type: 'resize'; + ts: string; + payload: { cols: number; rows: number }; + }; function event( seq: number, offsetMs: number, - type: ReplayEvent['type'] = 'output', + type: 'output' | 'input_text' = 'output', ): ReplayEvent { return { seq, @@ -853,14 +909,25 @@ describe('generateWebmExport stall watchdog with the ghostty-web replay loop', ( }; } - // Exports `events` with the real replay loop. Each bridge call takes - // `bridgeCostMs` of fake time. Returns how long the replay took in fake - // time, or the error that stopped it. + function resizeEvent(seq: number, offsetMs: number): ReplayEvent { + return { + seq, + type: 'resize', + ts: new Date(BASE_TIME_MS + offsetMs).toISOString(), + payload: { cols: 100 + seq, rows: 30 }, + }; + } + + // Exports `events` with the real replay loop. Each output bridge call takes + // `bridgeCostMs` of fake time, each resize call `resizeCostMs` and each note + // of the painted graphemes after a resize `noteCostMs`. Returns how long the + // replay took in fake time, or the error that stopped it. async function exportWithRealReplayLoop( events: ReplayEvent[], timingMode: 'recorded' | 'accelerated' | 'max-speed', bridgeCostMs: number, runForMs: number, + { resizeCostMs = 0, noteCostMs = 0 } = {}, ): Promise<{ replayMs: number | undefined; error: unknown }> { mocks.buildReplayInput.mockReturnValue({ sessionId: 'session-01', @@ -875,6 +942,7 @@ describe('generateWebmExport stall watchdog with the ghostty-web replay loop', ( }); let finalizedAtMs: number | undefined; let startedAtMs = 0; + let resizeCalls = 0; const outcome = generateWebmExport( { sessionId: 'session-01', @@ -896,15 +964,25 @@ describe('generateWebmExport stall watchdog with the ghostty-web replay loop', ( isBooted: true, page: { isClosed: () => false }, resizeBridge: () => { - startedAtMs = Date.now(); - markReplayStarted(); - return Promise.resolve(); + resizeCalls += 1; + if (resizeCalls === 1) { + // The first call sizes the terminal before the replay starts. + startedAtMs = Date.now(); + markReplayStarted(); + return Promise.resolve(); + } + return new Promise((resolve) => { + setTimeout(resolve, resizeCostMs); + }); }, writeBatchBridge: () => new Promise((resolve) => { setTimeout(resolve, bridgeCostMs); }), - notePaintedGraphemes: () => Promise.resolve(), + notePaintedGraphemes: () => + new Promise((resolve) => { + setTimeout(resolve, noteCostMs); + }), checkPaintedGlyphs: () => Promise.resolve({}), readHarnessSnapshot: () => Promise.resolve({ @@ -979,13 +1057,15 @@ describe('generateWebmExport stall watchdog with the ghostty-web replay loop', ( it.each([ { timingMode: 'accelerated', eventCount: 7_401, bridgeCostMs: 50 }, - { timingMode: 'max-speed', eventCount: 39_001, bridgeCostMs: 15 }, + { timingMode: 'max-speed', eventCount: 39_001, bridgeCostMs: 200 }, ] as const)( 'exports a dense $timingMode recording whose replay runs far past its plan', async ({ timingMode, eventCount, bridgeCostMs }) => { - // An input event after each output event forces a flush, and each flush - // holds the frame, so the replay takes minutes for a recording that - // spans seconds. + // In accelerated mode an input event after each output event forces a + // flush, and each flush holds the frame. In max-speed mode the flushes + // are spread over the recorded span, about 2,300 of them here, and each + // one costs a slow bridge call. Either way the replay takes minutes for + // a recording that spans seconds. const events = Array.from({ length: eventCount }, (_, seq) => event(seq, seq, seq % 2 === 0 ? 'output' : 'input_text'), ); @@ -1002,9 +1082,9 @@ describe('generateWebmExport stall watchdog with the ghostty-web replay loop', ( ); it('exports a max-speed recording whose output goes to the renderer in one flush', async () => { - // One flush is 40 bridge calls of 21 s each, 14 minutes in all. Each - // call is progress. - const events = Array.from({ length: 39_001 }, (_, seq) => event(seq, seq)); + // Events with one timestamp share a flush. This flush is 40 bridge calls + // of 21 s each, 14 minutes in all. Each call is progress. + const events = Array.from({ length: 39_001 }, (_, seq) => event(seq, 0)); const { replayMs, error } = await exportWithRealReplayLoop( events, 'max-speed', @@ -1016,6 +1096,28 @@ describe('generateWebmExport stall watchdog with the ghostty-web replay loop', ( expect(replayMs).toBeGreaterThan(WINDOW_MS); }); + it('exports a recording that resizes often while no page call outlasts the window', async () => { + // A resize is two page calls, the resize and the note of the graphemes + // on the screen. Each takes 160 s, so a resize takes longer than the + // window, and the report between the two calls keeps both under it. + const events = [ + event(0, 0), + resizeEvent(1, 10), + resizeEvent(2, 20), + resizeEvent(3, 30), + ]; + const { replayMs, error } = await exportWithRealReplayLoop( + events, + 'recorded', + 0, + 60 * 60 * 1000, + { resizeCostMs: 160_000, noteCostMs: 160_000 }, + ); + + expect(error).toBeUndefined(); + expect(replayMs).toBeGreaterThan(3 * WINDOW_MS); + }); + it('stops a replay whose renderer call hangs and names the last event it reached', async () => { const events = [event(0, 0), event(1, 10), event(2, 20, 'input_text')]; const { replayMs, error } = await exportWithRealReplayLoop( diff --git a/test/unit/renderer/ghosttyWebBackend.test.ts b/test/unit/renderer/ghosttyWebBackend.test.ts index 9aa2db3a..c5112b4c 100644 --- a/test/unit/renderer/ghosttyWebBackend.test.ts +++ b/test/unit/renderer/ghosttyWebBackend.test.ts @@ -326,10 +326,6 @@ describe('GhosttyWebBackend unit guards', () => { calls.push(`write ${String(dataChunks.length)}`); return Promise.resolve(); }, - notePaintedGraphemes: () => { - calls.push('note'); - return Promise.resolve(); - }, checkPaintedGlyphs: () => Promise.resolve({}), readHarnessSnapshot: () => Promise.resolve( @@ -370,13 +366,10 @@ describe('GhosttyWebBackend unit guards', () => { // calls. A caller that watches for stalls sees progress after each call. expect(calls.slice(calls.indexOf('write 1000'))).toEqual([ 'write 1000', - 'note', 'report', 'write 1000', - 'note', 'report', 'write 501', - 'note', 'report', ]); expect(reports.slice(-3)).toEqual([ @@ -386,6 +379,82 @@ describe('GhosttyWebBackend unit guards', () => { ]); }); + it('reports progress between the resize call and the note that follow a resize event', async () => { + const backend = createBackend(); + const calls: string[] = []; + Object.assign(backend as object, { + isBooted: true, + page: { isClosed: () => false }, + resizeBridge: (_page: object, cols: number, rows: number) => { + calls.push(`resize ${String(cols)}x${String(rows)}`); + return Promise.resolve(); + }, + writeBatchBridge: (_page: object, dataChunks: string[]) => { + calls.push(`write ${dataChunks.join(',')}`); + return Promise.resolve(); + }, + notePaintedGraphemes: () => { + calls.push('note'); + return Promise.resolve(); + }, + checkPaintedGlyphs: () => Promise.resolve({}), + readHarnessSnapshot: () => + Promise.resolve( + createHarnessSnapshotPayload({ + cols: 100, + rows: 30, + cursorRow: 0, + cursorCol: 0, + }), + ), + }); + + await backend.replayWithTiming( + { + sessionId: 'renderer-unit-session', + initialCols: 80, + initialRows: 24, + targetSeq: 2, + events: [ + { + seq: 0, + ts: '2026-03-19T12:00:00.000Z', + type: 'output', + payload: { data: 'first' }, + }, + { + seq: 1, + ts: '2026-03-19T12:00:00.000Z', + type: 'resize', + payload: { cols: 100, rows: 30 }, + }, + { + seq: 2, + ts: '2026-03-19T12:00:00.000Z', + type: 'output', + payload: { data: 'second' }, + }, + ], + }, + { mode: 'max-speed', minFrameHoldMs: 0, finalFrameHoldMs: 0 }, + (progress) => { + calls.push( + `report ${String(progress.seq)}/${String(progress.scheduledWaitMs)}`, + ); + }, + ); + + expect(calls.slice(calls.indexOf('resize 100x30') - 1)).toEqual([ + 'report 0/0', + 'resize 100x30', + 'report 1/0', + 'note', + 'report 1/0', + 'write second', + 'report 2/0', + ]); + }); + it('retries the painted-grapheme note after the backend reboots', async () => { const backend = createBackend(); const graphemes = vi.fn(() => Promise.resolve({ missingGlyphs: [] })); From 211e73b8d41d09eee1518df2c692b3aa603ff773 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Sun, 8 Nov 2026 13:53:21 +0000 Subject: [PATCH 24/28] fix: keep awaiting the hook between two calls of a flush The hook is a promise again, as the flush tests of #358 expect: a failing hook must drop the held chunks and reject with its own error. --- src/renderer/ghosttyWeb/backend.ts | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/src/renderer/ghosttyWeb/backend.ts b/src/renderer/ghosttyWeb/backend.ts index 96269add..347170aa 100644 --- a/src/renderer/ghosttyWeb/backend.ts +++ b/src/renderer/ghosttyWeb/backend.ts @@ -719,7 +719,10 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { await this.flushOutputBatch( page, pendingOutputChunks, - () => onProgress?.({ seq: highestProcessedSeq, scheduledWaitMs: 0 }), + () => { + onProgress?.({ seq: highestProcessedSeq, scheduledWaitMs: 0 }); + return Promise.resolve(); + }, true, ); pendingOutputChunks = []; @@ -1749,7 +1752,7 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { private async flushOutputBatch( page: Page, dataChunks: string[], - afterEachBatch?: () => void, + afterEachBatch?: () => Promise, notePainted = false, ): Promise { invariant( @@ -1782,7 +1785,7 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { endsFlush, endsFlush && notePainted ? this.paintedNoteCap() : null, ); - afterEachBatch?.(); + await afterEachBatch?.(); } } catch (error) { // The page still holds the chunks of the calls that did succeed. The From b7ed95f4f9eab98b8963a9061acf2f938f3b131a Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Sun, 8 Nov 2026 14:19:15 +0000 Subject: [PATCH 25/28] docs: say how long a healthy WebM replay stays silent --- src/export/README.md | 2 +- src/export/webm.ts | 11 ++++++----- 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/src/export/README.md b/src/export/README.md index 0e4f5ab1..108911dc 100644 --- a/src/export/README.md +++ b/src/export/README.md @@ -27,7 +27,7 @@ This directory converts a session's event log into recording exports. `generateA - A requested `libghostty-vt` renderer is replaced with `ghostty-web`. A code comment says `libghostty-vt` is semantic-only [webm.ts:generateWebmExport]. - The default backend factory accepts only `ghostty-web` [webm.ts:createDefaultBackend]. - Timing presets: `accelerated` caps gaps at 400 ms and holds frames at least 100 ms. `max-speed` uses a frame hold of 16 ms, but holds it only after the last flush of the replay and before and after a resize (see the `ghostty-web` README). Each mode holds the final frame (1000 ms, or 500 ms for `max-speed`) [webm.ts:buildReplayTimingOptions]. -- The timed replay has no total time limit. It fails with `WebmReplayStallError` only when the renderer makes no progress for `REPLAY_STALL_WINDOW_MS` (5 minutes). The backend reports progress through the `onProgress` argument of `replayWithTiming`: each report restarts the window. A report that comes before a wait on the timing schedule carries the length of that wait, and the window starts after it, so a long recorded gap is not a stall. No timer runs longer than 2^31-1 ms (the longest delay that a timer runs): a longer span is a chain of timers. The watchdog starts no timer once the replay has ended or stalled. Boot and finalize run outside the watchdog [webm.ts:replayWithStallWatchdog, webm.ts:REPLAY_STALL_WINDOW_MS]. +- The timed replay has no total time limit. It fails with `WebmReplayStallError` only when the renderer makes no progress for `REPLAY_STALL_WINDOW_MS` (5 minutes). The backend reports progress through the `onProgress` argument of `replayWithTiming`: each report restarts the window. A report that comes before a wait on the timing schedule carries the length of that wait, and the window starts after it, so a long recorded gap is not a stall. A healthy replay is silent for one page call at most. The call that ends a flush writes the chunks of the whole flush, and the event log limit bounds that. No timer runs longer than 2^31-1 ms (the longest delay that a timer runs): a longer span is a chain of timers. The watchdog starts no timer once the replay has ended, failed or stalled. Boot and finalize run outside the watchdog [webm.ts:replayWithStallWatchdog, webm.ts:REPLAY_STALL_WINDOW_MS]. - `renderGridFramesToSvg` needs the profile font size `SVG_FONT_SIZE` (14) and draws fixed 8.4x18 px cells, so equal input gives byte-identical output [svg.ts:renderGridFramesToSvg]. It always embeds the JetBrains Mono subset, and embeds each other bundled face only when a glyph needs it [svg.ts:embeddedFontAssets]. Output past `defaultMaxSvgOutputLength` of the V8 heap limit (the heap limit divided by 8 bytes, at most `MAX_SVG_OUTPUT_LENGTH`) fails with `EXPORT_ERROR` [svg.ts:defaultMaxSvgOutputLength]. ## Side Effects diff --git a/src/export/webm.ts b/src/export/webm.ts index 9c105034..3b903f38 100644 --- a/src/export/webm.ts +++ b/src/export/webm.ts @@ -26,11 +26,12 @@ import { resolveProfile } from '../renderer/profiles.js'; import type { RenderProfileConfig, ReplayState } from '../renderer/types.js'; import { invariant, unreachable } from '../util/assert.js'; -// The replay fails when the renderer makes no progress for this long. A bridge -// call writes at most 1000 output chunks, and the longest one that a measured -// export made took 22 s on a loaded host. The window is the old fixed limit on -// the whole replay, so an export that fit before still fits. Boot and finalize -// run outside the watchdog. +// The replay fails when the renderer makes no progress for this long. A healthy +// replay is silent for the length of one page call at most. The call that ends +// a flush writes the chunks of the whole flush, and the event log limit (50 +// MiB) bounds that. The window is the old fixed limit on the whole replay, so +// an export that fit before still fits. Boot and finalize run outside the +// watchdog. const REPLAY_STALL_WINDOW_MS = 5 * 60 * 1000; // Node runs a timer with a longer delay after 1 ms. const MAX_TIMER_DELAY_MS = 2 ** 31 - 1; From 00cb184263a275dd3e922f655f7052b843df1183 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Mon, 9 Nov 2026 10:45:48 +0000 Subject: [PATCH 26/28] fix: wait a long recorded gap in full and report the last event of a WebM replay A recorded gap above 2^31-1 ms (24.8 days) ran as one timer that Node cuts to 1 ms, so a 30-day gap exported at once. The wait is a chain of timers now, and each timer after the first reports the rest of the wait. The report that precedes an event comes before the event is applied. When the last event changes no pixel, no flush reports it, so a stall in the final calls to the renderer named the event before it, or none for a one-event recording. The replay reports its last event before those calls. --- src/renderer/backend.ts | 7 +- src/renderer/ghosttyWeb/README.md | 2 +- src/renderer/ghosttyWeb/backend.ts | 43 ++++++++--- test/unit/export/webm.test.ts | 78 +++++++++++++++++--- test/unit/renderer/ghosttyWebBackend.test.ts | 70 +++++++++++++++++- 5 files changed, 174 insertions(+), 26 deletions(-) diff --git a/src/renderer/backend.ts b/src/renderer/backend.ts index 25a5f183..ceab162f 100644 --- a/src/renderer/backend.ts +++ b/src/renderer/backend.ts @@ -123,9 +123,10 @@ export interface ReplayProgress { export interface VideoCapableRendererBackend extends RendererBackend { /** * Replay events with controlled timing for video capture. `onProgress` - * runs before each wait on the timing schedule, after each batch of - * output that the replay writes to the renderer and after the renderer - * resizes for a resize event. + * runs before each wait on the timing schedule (and between the timers of a + * long wait), after each batch of output that the replay writes to the + * renderer, after the renderer resizes for a resize event and once the replay + * has reached its last event. */ replayWithTiming( input: ReplayInput, diff --git a/src/renderer/ghosttyWeb/README.md b/src/renderer/ghosttyWeb/README.md index d8d0104c..9db18d3d 100644 --- a/src/renderer/ghosttyWeb/README.md +++ b/src/renderer/ghosttyWeb/README.md @@ -30,7 +30,7 @@ This directory holds the `ghostty-web` reference renderer. `GhosttyWebBackend` s - The constructor asserts a non-empty `sessionId`, `profile.name` and `profile.fontFamily`, a positive `fontSize`, and hex colors. With `videoOptions`, `outputDir` must be absolute and the size must use positive integers [backend.ts:GhosttyWebBackend.constructor]. - Replay cannot rewind: `targetSeq` must be >= the last applied seq. The initial dimensions cannot change after the first replay. `input.sessionId` must match the backend [backend.ts:GhosttyWebBackend.replayTo]. - `replayWithTiming` asserts strictly increasing seqs and non-decreasing timestamps [backend.ts:GhosttyWebBackend.replayWithTiming]. -- `replayWithTiming` calls the optional `onProgress` with a `ReplayProgress` in three places. Before each wait on the schedule, `scheduledWaitMs` is the length of that wait. After each bridge call of a flush, and after the resize call of a resize event, `scheduledWaitMs` is 0. `seq` is the last event that the replay had reached. `generateWebmExport` uses these reports to tell a stall from a wait [backend.ts:GhosttyWebBackend.replayWithTiming]. +- `replayWithTiming` calls the optional `onProgress` with a `ReplayProgress` in four places. Before each wait on the schedule, `scheduledWaitMs` is the length of that wait. A wait longer than a timer runs (2^31-1 ms) is a chain of timers, so a recorded gap of 30 days takes 30 days, and each timer after the first is followed by a report that carries the rest of the wait. After each bridge call of a flush, and after the resize call of a resize event, `scheduledWaitMs` is 0. After the last event and its flush, and before the final calls to the renderer, `scheduledWaitMs` is 0 too, so a stall in those calls names the last event, also when that event changed no pixel. `seq` is the last event that the replay had reached. `generateWebmExport` uses these reports to tell a stall from a wait [backend.ts:GhosttyWebBackend.replayWithTiming]. - `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` has no schedule to fall behind, so only recorded time ends its flushes. A flush ends at the first event that is at least one frame hold of recorded time after the batch's first output. An event that changes no pixel does not end it: input, marker, run-complete, signal and exit events leave the batch as it is, because a flush at each of them would cost a frame per event. A resize and the end of the replay still end it. `max-speed` also spreads at most `MAX_SPEED_FRAME_BUDGET` (2,400) flushes over the recorded span of the events that it applies: the distance between the first output of a flush and the event that ends it is at least the larger of the frame hold and the span divided by 2,400. So a `max-speed` replay makes about one flush per frame hold of recorded time for a recording shorter than 38 s and at most about 2,400 flushes for a longer one, whatever its number of events. Output that arrives less than that distance after the first output of a flush shares the flush, also when an input or a marker sits between, so a state that the next output of the flush replaces is never painted. A flush waits for one animation frame. `max-speed` holds the frame hold only after the last flush of the replay and before and after a resize, so two flushes that follow each other can paint within one video frame. In `recorded` and `accelerated` mode, every event that changes no pixel ends a flush, and a hold follows every flush. Events with the same timestamp share a flush at any hold, 0 included, because a flush needs at least 1 ms of recorded time [backend.ts:GhosttyWebBackend.replayWithTiming]. - `writeBatch(chunks, endsFlush, paintedCap)` takes the chunks of one bridge call. A flush that has more than 1000 chunks needs several calls, and ghostty-web paints on every animation frame, so the page holds the chunks of a call that does not end the flush. The call that ends it (`endsFlush`, set by the backend on the last call only) writes every held chunk in one task, so no frame paints a state between two calls, and then resolves after one frame. ghostty-web calls a write callback one animation frame (about 17 ms) after the write, and waiting on every chunk (`write`) cost one frame per output event. `write()` and `resize()` throw while chunks are held. `discardHeldChunks()` drops them: the backend calls it when a flush fails, because `HostRendererManager` keeps the backend and a retry would otherwise meet them. A `paintedCap` that is not null makes the call that ends the flush add the graphemes on the screen to the set that the missing-glyph check of `replayWithTiming` probes, so a flush is one page call. The call returns the message of a failed note and not an error, because a failed note must not fail the flush. In `replayWithTiming`, the chunks of one flush share one video frame [embeddedHarnessHtml.ts:EMBEDDED_HARNESS_HTML]. - `snapshot()` and `screenshot()` throw until `replayTo()` or `replayWithTiming()` has run once and set the initial size. Both replays accept `targetSeq` -1 for an empty Event Log. A replay over zero events sets `lastAppliedSeq` to `targetSeq` [backend.ts:GhosttyWebBackend.snapshot]. diff --git a/src/renderer/ghosttyWeb/backend.ts b/src/renderer/ghosttyWeb/backend.ts index 3f9ed2a1..589dc740 100644 --- a/src/renderer/ghosttyWeb/backend.ts +++ b/src/renderer/ghosttyWeb/backend.ts @@ -139,13 +139,16 @@ const HTML_CONTENT_TYPE = 'text/html; charset=utf-8'; const WASM_CONTENT_TYPE = 'application/wasm'; const MAX_REPLAY_BATCH_SIZE = 1000; +// Node runs a timer with a longer delay after 1 ms, so a longer wait is a chain +// of timers. +const MAX_TIMER_DELAY_MS = 2 ** 31 - 1; /** * A max-speed replay makes at most about this many flushes, however long the * recording is. A flush costs one animation frame, one page call and one scan * of the screen: 26 ms at 80x24, 33 ms at 120x40 and 112 ms at 200x60 in the * median of five sparse-log runs (host load 40 to 100). The limit then takes - * 1 to 4.5 minutes, under the 5 minutes that a WebM export allows its replay. - * The video has 25 frames per second, so this many flushes fill 96 s of video. + * 1 to 4.5 minutes. The video has 25 frames per second, so this many flushes + * fill 96 s of video. */ const MAX_SPEED_FRAME_BUDGET = 2_400; const RAF_TIMEOUT_MS = 5_000; @@ -665,18 +668,29 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { } }; - const waitForDelay = async (delayMs: number): Promise => { + // A wait longer than a timer runs is a chain of timers, so a recorded gap of + // 30 days takes 30 days. `onRemaining` runs between two timers with the rest + // of the wait. + const waitForDelay = async ( + delayMs: number, + onRemaining?: (remainingMs: number) => void, + ): Promise => { assertNonNegativeInteger( delayMs, 'replayWithTiming delayMs must be a non-negative integer', ); - if (delayMs === 0) { - return; - } - await new Promise((resolvePromise) => { - setTimeout(resolvePromise, delayMs); - }); + let remainingMs = delayMs; + while (remainingMs > 0) { + const stepMs = Math.min(remainingMs, MAX_TIMER_DELAY_MS); + await new Promise((resolvePromise) => { + setTimeout(resolvePromise, stepMs); + }); + remainingMs -= stepMs; + if (remainingMs > 0) { + onRemaining?.(remainingMs); + } + } }; const parseEventTimestampMs = ( eventTs: string, @@ -821,7 +835,12 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { ), ); onProgress?.({ seq: highestProcessedSeq, scheduledWaitMs }); - await waitForDelay(scheduledWaitMs); + await waitForDelay(scheduledWaitMs, (remainingMs) => { + onProgress?.({ + seq: highestProcessedSeq, + scheduledWaitMs: remainingMs, + }); + }); } previousProcessedEventTimestampMs = eventTimestampMs; @@ -878,6 +897,10 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { } await flushOutputBatch(); + // The last event can change no pixel, so no flush has reported it. The final + // calls to the renderer follow, and a stall in one of them is a stall after + // this event. + onProgress?.({ seq: highestProcessedSeq, scheduledWaitMs: 0 }); invariant( outputEventCount + resizeEventCount <= processedEventCount, diff --git a/test/unit/export/webm.test.ts b/test/unit/export/webm.test.ts index c31c01f3..a254829b 100644 --- a/test/unit/export/webm.test.ts +++ b/test/unit/export/webm.test.ts @@ -927,7 +927,15 @@ describe('generateWebmExport stall watchdog with the ghostty-web replay loop', ( timingMode: 'recorded' | 'accelerated' | 'max-speed', bridgeCostMs: number, runForMs: number, - { resizeCostMs = 0, noteCostMs = 0 } = {}, + { + resizeCostMs = 0, + noteCostMs = 0, + hangingCall, + }: { + resizeCostMs?: number; + noteCostMs?: number; + hangingCall?: 'checkPaintedGlyphs' | 'readHarnessSnapshot'; + } = {}, ): Promise<{ replayMs: number | undefined; error: unknown }> { mocks.buildReplayInput.mockReturnValue({ sessionId: 'session-01', @@ -983,16 +991,21 @@ describe('generateWebmExport stall watchdog with the ghostty-web replay loop', ( new Promise((resolve) => { setTimeout(resolve, noteCostMs); }), - checkPaintedGlyphs: () => Promise.resolve({}), + checkPaintedGlyphs: () => + hangingCall === 'checkPaintedGlyphs' + ? new Promise(() => undefined) + : Promise.resolve({}), readHarnessSnapshot: () => - Promise.resolve({ - cols: 80, - rows: 24, - cursorRow: 0, - cursorCol: 0, - isAltScreen: false, - visibleLines: [], - }), + hangingCall === 'readHarnessSnapshot' + ? new Promise(() => undefined) + : Promise.resolve({ + cols: 80, + rows: 24, + cursorRow: 0, + cursorCol: 0, + isAltScreen: false, + visibleLines: [], + }), recordedMissingGlyphs: () => Promise.resolve({}), finalizeVideo: () => { finalizedAtMs = Date.now(); @@ -1055,6 +1068,51 @@ describe('generateWebmExport stall watchdog with the ghostty-web replay loop', ( }, ); + it('waits a recorded gap of 30 days in full, with no timer longer than a timer can run', async () => { + const gapMs = 30 * DAY_MS; + const setTimeoutSpy = vi.spyOn(globalThis, 'setTimeout'); + try { + const { replayMs, error } = await exportWithRealReplayLoop( + [event(0, 0), event(1, gapMs)], + 'recorded', + 0, + gapMs + 60_000, + ); + + expect(error).toBeUndefined(); + expect(replayMs).toBeGreaterThanOrEqual(gapMs); + const delays = setTimeoutSpy.mock.calls.map((call) => Number(call[1])); + expect(Math.max(...delays)).toBeLessThanOrEqual(2 ** 31 - 1); + } finally { + setTimeoutSpy.mockRestore(); + } + }); + + it.each([ + { calls: 'checkPaintedGlyphs', eventCount: 1, lastSeq: 0 }, + { calls: 'readHarnessSnapshot', eventCount: 1, lastSeq: 0 }, + { calls: 'checkPaintedGlyphs', eventCount: 3, lastSeq: 2 }, + ] as const)( + 'names event $lastSeq when $calls hangs after a recording of $eventCount events that ends with input', + async ({ calls, eventCount, lastSeq }) => { + // The last event is input, so it changes no pixel and no flush reports it. + const events = Array.from({ length: eventCount }, (_, seq) => + event(seq, seq * 10, seq === eventCount - 1 ? 'input_text' : 'output'), + ); + const { replayMs, error } = await exportWithRealReplayLoop( + events, + 'recorded', + 0, + 60 * 60 * 1000, + { hangingCall: calls }, + ); + + expect(replayMs).toBeUndefined(); + expect(error).toBeInstanceOf(WebmReplayStallError); + expect(error).toMatchObject({ lastSeq, timingMode: 'recorded' }); + }, + ); + it.each([ { timingMode: 'accelerated', eventCount: 7_401, bridgeCostMs: 50 }, { timingMode: 'max-speed', eventCount: 39_001, bridgeCostMs: 200 }, diff --git a/test/unit/renderer/ghosttyWebBackend.test.ts b/test/unit/renderer/ghosttyWebBackend.test.ts index c5112b4c..778a89d1 100644 --- a/test/unit/renderer/ghosttyWebBackend.test.ts +++ b/test/unit/renderer/ghosttyWebBackend.test.ts @@ -363,7 +363,8 @@ describe('GhosttyWebBackend unit guards', () => { ); // Max-speed sends the whole run of output in one flush of three bridge - // calls. A caller that watches for stalls sees progress after each call. + // calls. A caller that watches for stalls sees progress after each call, + // and once more when the replay has reached its last event. expect(calls.slice(calls.indexOf('write 1000'))).toEqual([ 'write 1000', 'report', @@ -371,8 +372,10 @@ describe('GhosttyWebBackend unit guards', () => { 'report', 'write 501', 'report', + 'report', ]); - expect(reports.slice(-3)).toEqual([ + expect(reports.slice(-4)).toEqual([ + { seq: 2_500, scheduledWaitMs: 0 }, { seq: 2_500, scheduledWaitMs: 0 }, { seq: 2_500, scheduledWaitMs: 0 }, { seq: 2_500, scheduledWaitMs: 0 }, @@ -452,6 +455,7 @@ describe('GhosttyWebBackend unit guards', () => { 'report 1/0', 'write second', 'report 2/0', + 'report 2/0', ]); }); @@ -2165,9 +2169,71 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { ).toEqual([ { seq: 0, endsAtMs: FIRST_EVENT_AT_MS + 200 }, { seq: 1, endsAtMs: FIRST_EVENT_AT_MS + 5_000 }, + // The report of the last event follows the frame hold of the last flush. + { seq: 2, endsAtMs: FIRST_EVENT_AT_MS + 5_000 + 16 }, ]); }); + it('runs a wait longer than a timer can run as a chain and reports what is left', async () => { + vi.useFakeTimers(); + const MAX_TIMER_DELAY_MS = 2 ** 31 - 1; + const gapMs = 30 * 24 * 60 * 60 * 1000; + const backend = createBackend(); + Object.assign(backend as object, { + isBooted: true, + page: { isClosed: () => false }, + resizeBridge: () => Promise.resolve(), + flushOutputBatch: () => Promise.resolve(), + readHarnessSnapshot: () => + Promise.resolve( + createHarnessSnapshotPayload({ + cols: 80, + rows: 24, + visibleLines: [], + }), + ), + }); + const setTimeoutSpy = vi.spyOn(globalThis, 'setTimeout'); + const reports: (ReplayProgress & { atMs: number })[] = []; + const startedAtMs = performance.now(); + + try { + const replay = backend.replayWithTiming( + { + sessionId: 'renderer-unit-session', + initialCols: 80, + initialRows: 24, + targetSeq: 1, + events: [outputEvent(0, 0), outputEvent(1, gapMs)], + }, + RECORDED, + (progress) => { + reports.push({ ...progress, atMs: performance.now() - startedAtMs }); + }, + ); + await vi.runAllTimersAsync(); + await replay; + const delays = setTimeoutSpy.mock.calls.map((call) => Number(call[1])); + expect(Math.max(...delays)).toBeLessThanOrEqual(MAX_TIMER_DELAY_MS); + } finally { + setTimeoutSpy.mockRestore(); + } + + // The first report carries the whole wait. The second comes after the first + // timer and carries the rest, so the window of a watchdog follows the wait. + const [first, second] = reports; + if (first === undefined) { + throw new Error('the replay made no report'); + } + expect(first.scheduledWaitMs).toBeGreaterThan(MAX_TIMER_DELAY_MS); + expect(first.atMs + first.scheduledWaitMs).toBe(FIRST_EVENT_AT_MS + gapMs); + expect(second).toEqual({ + seq: 0, + scheduledWaitMs: first.scheduledWaitMs - MAX_TIMER_DELAY_MS, + atMs: first.atMs + MAX_TIMER_DELAY_MS, + }); + }); + it('keeps the recorded pace when continuous output fills every frame hold', async () => { const events = Array.from({ length: 101 }, (_, index) => outputEvent(index, index * 10), From 1b29fb36067fd4f5a73a04ded1744e3cb5e1c10f Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Mon, 9 Nov 2026 12:03:03 +0000 Subject: [PATCH 27/28] test: stop a hung replay after a wait or a window longer than a timer; state the silent stretches exactly The watchdog chains its timers for a wait plus window above 2^31-1 ms. The new tests stop a hung replay after a scheduled wait of 30 days and with a stall window of 2^31+5 ms. A watchdog that re-arms the whole remaining time on each step never fires, and now fails them. The webm.ts comment and the export README name the three places where a healthy replay is silent, with measured times. USAGE no longer says that an export takes about as long as the recording. --- docs/USAGE.md | 2 +- src/export/README.md | 2 +- src/export/webm.ts | 14 +++++++---- test/unit/export/webm.test.ts | 44 ++++++++++++++++++++++++++++++++++- 4 files changed, 54 insertions(+), 8 deletions(-) diff --git a/docs/USAGE.md b/docs/USAGE.md index 2fa4fdc5..22c9b3a9 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -298,7 +298,7 @@ agent-tty record export --format webm --json | jq -c '.result.missi WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. A frame of the video shows the screen after the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event, and a state that the next output of the frame replaces is not shown. In `recorded` and `accelerated` mode, an event that changes no pixel (an input, a marker, a run-complete, a signal or the exit) also ends a frame. In `max-speed` mode it does not: only recorded time ends a frame, so output on both sides of an input event that is less than 16 ms apart shares one frame. `max-speed` removes the idle time. It makes a frame at most once per 16 ms of recorded time, so a progress bar that updates every 30 ms shows many of its steps and not only its last one. It makes at most about 2,400 frames, so a recording longer than 38 s gets one frame per 1/2,400 of its recorded time instead (a 1-hour recording: one per 1.5 s), and its export time depends on its length and not on its number of events. `max-speed` waits for one animation frame after each frame but holds no further time between two frames, so two frames that follow each other can end up in one video frame. The video has 25 frames per second, so it can skip a state that lasts less than one video frame (40 ms). A step of 30 ms is shorter than that, and the number of steps that survive depends on how fast the host runs the export. -The replay runs in real time, so a WebM export with recorded timing takes about as long as the recording, and a long recording takes a long time. The export has no total time limit. It stops only when the renderer makes no progress for 5 minutes, and then fails with `EXPORT_ERROR`. The message names the window and the last event that the replay reached, and `details` holds `stallWindowMs`, `lastSeq` and `timingMode`. Waiting out a recorded gap is not a stall. +With recorded timing the replay waits out every recorded gap, so an export takes at least as long as the recording. It takes longer when the recording is dense or resizes a large terminal, because each frame costs time in the browser and each resize of a large terminal costs seconds. On a shared host, a recording of 7.4 s with 7,401 events took 473 s at `--timing accelerated`, and a recording of 0.8 s with 400 resizes of a 600x150 terminal took 910 s at recorded timing. The export has no total time limit. It stops only when the renderer makes no progress for 5 minutes, and then fails with `EXPORT_ERROR`. The message names the window and the last event that the replay reached, and `details` holds `stallWindowMs`, `lastSeq` and `timingMode`. Waiting out a recorded gap is not a stall. SVG export renders styled grid frames from the event log through the native `libghostty-vt` backend with no browser and no ffmpeg, so it requires the optional `@coder/libghostty-vt-node` package (there is no `ghostty-web` fallback). The output is deterministic: exporting the same session twice produces byte-identical, diffable SVG. `--format svg` writes a still image of the final screen and ignores event timing, so it also works on a log whose clock stepped backward (`--animate` refuses such a log and names the event where the clock stepped back); add `--animate` for an animated SVG of de-duplicated frames replayed with recorded event-log timing (`--timing` is not supported with SVG). diff --git a/src/export/README.md b/src/export/README.md index 108911dc..e93a745e 100644 --- a/src/export/README.md +++ b/src/export/README.md @@ -27,7 +27,7 @@ This directory converts a session's event log into recording exports. `generateA - A requested `libghostty-vt` renderer is replaced with `ghostty-web`. A code comment says `libghostty-vt` is semantic-only [webm.ts:generateWebmExport]. - The default backend factory accepts only `ghostty-web` [webm.ts:createDefaultBackend]. - Timing presets: `accelerated` caps gaps at 400 ms and holds frames at least 100 ms. `max-speed` uses a frame hold of 16 ms, but holds it only after the last flush of the replay and before and after a resize (see the `ghostty-web` README). Each mode holds the final frame (1000 ms, or 500 ms for `max-speed`) [webm.ts:buildReplayTimingOptions]. -- The timed replay has no total time limit. It fails with `WebmReplayStallError` only when the renderer makes no progress for `REPLAY_STALL_WINDOW_MS` (5 minutes). The backend reports progress through the `onProgress` argument of `replayWithTiming`: each report restarts the window. A report that comes before a wait on the timing schedule carries the length of that wait, and the window starts after it, so a long recorded gap is not a stall. A healthy replay is silent for one page call at most. The call that ends a flush writes the chunks of the whole flush, and the event log limit bounds that. No timer runs longer than 2^31-1 ms (the longest delay that a timer runs): a longer span is a chain of timers. The watchdog starts no timer once the replay has ended, failed or stalled. Boot and finalize run outside the watchdog [webm.ts:replayWithStallWatchdog, webm.ts:REPLAY_STALL_WINDOW_MS]. +- The timed replay has no total time limit. It fails with `WebmReplayStallError` only when the renderer makes no progress for `REPLAY_STALL_WINDOW_MS` (5 minutes). The backend reports progress through the `onProgress` argument of `replayWithTiming`: each report restarts the window. A report that comes before a wait on the timing schedule carries the length of that wait, and the window starts after it, so a long recorded gap is not a stall. A healthy replay is silent in three places only: from the start to the first report (the replay sizes the terminal and holds one frame), inside one page call (the call that ends a flush writes the chunks of the whole flush, and the event log limit bounds that), and from the last report to the end (the final frame hold of 1000 ms, or 500 ms at `max-speed`, with the glyph probes inside it, and then a read of the screen). On a loaded host at 600x150 the start took 1.5 s at most and the end 2.3 s, and a flush of 52 MB took 2.8 s. No timer runs longer than 2^31-1 ms (the longest delay that a timer runs): a longer span is a chain of timers. The watchdog starts no timer once the replay has ended, failed or stalled. Boot and finalize run outside the watchdog [webm.ts:replayWithStallWatchdog, webm.ts:REPLAY_STALL_WINDOW_MS]. - `renderGridFramesToSvg` needs the profile font size `SVG_FONT_SIZE` (14) and draws fixed 8.4x18 px cells, so equal input gives byte-identical output [svg.ts:renderGridFramesToSvg]. It always embeds the JetBrains Mono subset, and embeds each other bundled face only when a glyph needs it [svg.ts:embeddedFontAssets]. Output past `defaultMaxSvgOutputLength` of the V8 heap limit (the heap limit divided by 8 bytes, at most `MAX_SVG_OUTPUT_LENGTH`) fails with `EXPORT_ERROR` [svg.ts:defaultMaxSvgOutputLength]. ## Side Effects diff --git a/src/export/webm.ts b/src/export/webm.ts index 3b903f38..98e04c2c 100644 --- a/src/export/webm.ts +++ b/src/export/webm.ts @@ -27,11 +27,15 @@ import type { RenderProfileConfig, ReplayState } from '../renderer/types.js'; import { invariant, unreachable } from '../util/assert.js'; // The replay fails when the renderer makes no progress for this long. A healthy -// replay is silent for the length of one page call at most. The call that ends -// a flush writes the chunks of the whole flush, and the event log limit (50 -// MiB) bounds that. The window is the old fixed limit on the whole replay, so -// an export that fit before still fits. Boot and finalize run outside the -// watchdog. +// replay is silent for a few seconds at most, in three places: from the start +// to the first report (it sizes the terminal and holds one frame), inside one +// page call (the call that ends a flush writes the chunks of the whole flush, +// which the event log limit of 50 MiB bounds), and from the last report to the +// end (the final frame hold, which also covers the glyph probes, and then a +// read of the screen). On a loaded host at 600x150 the start took 1.5 s at most +// and the end 2.3 s, and a flush of 52 MB took 2.8 s. The window is the old +// fixed limit on the whole replay, so an export that fit before still fits. +// Boot and finalize run outside the watchdog. const REPLAY_STALL_WINDOW_MS = 5 * 60 * 1000; // Node runs a timer with a longer delay after 1 ms. const MAX_TIMER_DELAY_MS = 2 ** 31 - 1; diff --git a/test/unit/export/webm.test.ts b/test/unit/export/webm.test.ts index a254829b..48c099e4 100644 --- a/test/unit/export/webm.test.ts +++ b/test/unit/export/webm.test.ts @@ -603,6 +603,7 @@ describe('generateWebmExport stall watchdog', () => { async function startExport( script: ReplayScript, timingMode: 'recorded' | 'accelerated' | 'max-speed' = 'recorded', + stallWindowMs?: number, ) { let markReplayStarted: () => void = () => undefined; const replayStarted = new Promise((resolve) => { @@ -628,7 +629,10 @@ describe('generateWebmExport stall watchdog', () => { outputPath: '/tmp/exports/recording-1-webm.webm', timingMode, }, - { backendFactory: () => mockBackend.backend }, + { + backendFactory: () => mockBackend.backend, + ...(stallWindowMs === undefined ? {} : { stallWindowMs }), + }, ).then( (result) => ({ result, error: undefined }), (error: unknown) => ({ result: undefined, error }), @@ -788,6 +792,44 @@ describe('generateWebmExport stall watchdog', () => { expect(error).toMatchObject({ stallWindowMs: WINDOW_MS, lastSeq: 3 }); }); + it('stops a replay that hangs after a scheduled wait of 30 days, which with the window is longer than a timer runs', async () => { + // The wait and the window together are 30 days and 5 minutes: more than + // the longest timer, so the stall is the end of a chain of two timers. + const { outcome, mockBackend } = await startExport((onProgress) => { + onProgress({ seq: 3, scheduledWaitMs: 30 * DAY_MS }); + return new Promise(() => undefined); + }); + + await vi.advanceTimersByTimeAsync(30 * DAY_MS + WINDOW_MS - 1); + expect(mockBackend.dispose).not.toHaveBeenCalled(); + + await vi.advanceTimersByTimeAsync(1); + expect(mockBackend.dispose).toHaveBeenCalledTimes(1); + const { error } = await outcome; + + expect(error).toBeInstanceOf(WebmReplayStallError); + expect(error).toMatchObject({ stallWindowMs: WINDOW_MS, lastSeq: 3 }); + }); + + it('stops a hung replay once a stall window that outlasts a timer has passed', async () => { + const stallWindowMs = MAX_TIMER_DELAY_MS + 5; + const { outcome, mockBackend } = await startExport( + () => new Promise(() => undefined), + 'recorded', + stallWindowMs, + ); + + await vi.advanceTimersByTimeAsync(stallWindowMs - 1); + expect(mockBackend.dispose).not.toHaveBeenCalled(); + + await vi.advanceTimersByTimeAsync(1); + expect(mockBackend.dispose).toHaveBeenCalledTimes(1); + const { error } = await outcome; + + expect(error).toBeInstanceOf(WebmReplayStallError); + expect(error).toMatchObject({ stallWindowMs, lastSeq: -1 }); + }); + it('leaves no timer behind when the replay succeeds', async () => { const { outcome } = await startExport((onProgress) => { onProgress({ seq: 0, scheduledWaitMs: DAY_MS }); From 4f3886794e150e928c1ac76595a450a5a2e40a8b Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Mon, 9 Nov 2026 15:07:58 +0000 Subject: [PATCH 28/28] fix: say exactly where a WebM replay reports and how long it stays silent The report of the rest of a long wait follows each timer except the last, not each timer after the first. The silent stretches are measured examples, not bounds, and each is one page call or a fixed hold. A test pins a recorded gap of 1 ms. --- src/export/README.md | 2 +- src/export/webm.ts | 24 ++++++++++++-------- src/renderer/ghosttyWeb/README.md | 2 +- test/unit/renderer/ghosttyWebBackend.test.ts | 11 +++++++++ 4 files changed, 27 insertions(+), 12 deletions(-) diff --git a/src/export/README.md b/src/export/README.md index e93a745e..2a30546d 100644 --- a/src/export/README.md +++ b/src/export/README.md @@ -27,7 +27,7 @@ This directory converts a session's event log into recording exports. `generateA - A requested `libghostty-vt` renderer is replaced with `ghostty-web`. A code comment says `libghostty-vt` is semantic-only [webm.ts:generateWebmExport]. - The default backend factory accepts only `ghostty-web` [webm.ts:createDefaultBackend]. - Timing presets: `accelerated` caps gaps at 400 ms and holds frames at least 100 ms. `max-speed` uses a frame hold of 16 ms, but holds it only after the last flush of the replay and before and after a resize (see the `ghostty-web` README). Each mode holds the final frame (1000 ms, or 500 ms for `max-speed`) [webm.ts:buildReplayTimingOptions]. -- The timed replay has no total time limit. It fails with `WebmReplayStallError` only when the renderer makes no progress for `REPLAY_STALL_WINDOW_MS` (5 minutes). The backend reports progress through the `onProgress` argument of `replayWithTiming`: each report restarts the window. A report that comes before a wait on the timing schedule carries the length of that wait, and the window starts after it, so a long recorded gap is not a stall. A healthy replay is silent in three places only: from the start to the first report (the replay sizes the terminal and holds one frame), inside one page call (the call that ends a flush writes the chunks of the whole flush, and the event log limit bounds that), and from the last report to the end (the final frame hold of 1000 ms, or 500 ms at `max-speed`, with the glyph probes inside it, and then a read of the screen). On a loaded host at 600x150 the start took 1.5 s at most and the end 2.3 s, and a flush of 52 MB took 2.8 s. No timer runs longer than 2^31-1 ms (the longest delay that a timer runs): a longer span is a chain of timers. The watchdog starts no timer once the replay has ended, failed or stalled. Boot and finalize run outside the watchdog [webm.ts:replayWithStallWatchdog, webm.ts:REPLAY_STALL_WINDOW_MS]. +- The timed replay has no total time limit. It fails with `WebmReplayStallError` only when the renderer makes no progress for `REPLAY_STALL_WINDOW_MS` (5 minutes). The backend reports progress through the `onProgress` argument of `replayWithTiming`: each report restarts the window. A report that comes before a wait on the timing schedule carries the length of that wait, and the window starts after it, so a long recorded gap is not a stall. A stretch without a report is one page call or a fixed hold, never the whole replay: a call carries at most 1000 chunks, and a report follows it. The stretches are the start to the first report (the replay sizes the terminal and holds one frame), one page call (the call that ends a flush also writes the chunks that the earlier calls of the flush held, so the event log limit of 50 MiB bounds it), and the last report to the end (the final frame hold of 1000 ms, or 500 ms at `max-speed`, with the glyph probes inside it, and then a read of the screen). Examples that I measured at 600x150 on a shared host under load: the start took 1.3 to 2.0 s, the end 1.8 to 2.8 s, a flush of 11,000 events of 4 KB 8.4 s, and one stretch inside the loop 4.4 s. These are examples and not bounds. The longest, 8.4 s, is under 3% of the window. No timer runs longer than 2^31-1 ms (the longest delay that a timer runs): a longer span is a chain of timers. The watchdog starts no timer once the replay has ended, failed or stalled. Boot and finalize run outside the watchdog [webm.ts:replayWithStallWatchdog, webm.ts:REPLAY_STALL_WINDOW_MS]. - `renderGridFramesToSvg` needs the profile font size `SVG_FONT_SIZE` (14) and draws fixed 8.4x18 px cells, so equal input gives byte-identical output [svg.ts:renderGridFramesToSvg]. It always embeds the JetBrains Mono subset, and embeds each other bundled face only when a glyph needs it [svg.ts:embeddedFontAssets]. Output past `defaultMaxSvgOutputLength` of the V8 heap limit (the heap limit divided by 8 bytes, at most `MAX_SVG_OUTPUT_LENGTH`) fails with `EXPORT_ERROR` [svg.ts:defaultMaxSvgOutputLength]. ## Side Effects diff --git a/src/export/webm.ts b/src/export/webm.ts index 98e04c2c..f9f7e5cf 100644 --- a/src/export/webm.ts +++ b/src/export/webm.ts @@ -26,16 +26,20 @@ import { resolveProfile } from '../renderer/profiles.js'; import type { RenderProfileConfig, ReplayState } from '../renderer/types.js'; import { invariant, unreachable } from '../util/assert.js'; -// The replay fails when the renderer makes no progress for this long. A healthy -// replay is silent for a few seconds at most, in three places: from the start -// to the first report (it sizes the terminal and holds one frame), inside one -// page call (the call that ends a flush writes the chunks of the whole flush, -// which the event log limit of 50 MiB bounds), and from the last report to the -// end (the final frame hold, which also covers the glyph probes, and then a -// read of the screen). On a loaded host at 600x150 the start took 1.5 s at most -// and the end 2.3 s, and a flush of 52 MB took 2.8 s. The window is the old -// fixed limit on the whole replay, so an export that fit before still fits. -// Boot and finalize run outside the watchdog. +// The replay fails when the renderer makes no progress for this long. A stretch +// without a report is one page call or a fixed hold, never the whole replay: a +// call carries at most 1000 chunks, and a report follows it. The stretches are +// the start to the first report (the replay sizes the terminal and holds one +// frame), one page call (the call that ends a flush also writes the chunks that +// the earlier calls of the flush held, so the event log limit of 50 MiB bounds +// it), and the last report to the end (the final frame hold, which also covers +// the glyph probes, and then a read of the screen). Examples that I measured at +// 600x150 on a shared host under load: the start took 1.3 to 2.0 s, the end 1.8 +// to 2.8 s, a flush of 11,000 events of 4 KB 8.4 s, and one stretch inside the +// loop 4.4 s. These are examples and not bounds. The longest, 8.4 s, is under +// 3% of the window. The window is the old fixed limit on the whole replay, so +// an export that fit before still fits. Boot and finalize run outside the +// watchdog. const REPLAY_STALL_WINDOW_MS = 5 * 60 * 1000; // Node runs a timer with a longer delay after 1 ms. const MAX_TIMER_DELAY_MS = 2 ** 31 - 1; diff --git a/src/renderer/ghosttyWeb/README.md b/src/renderer/ghosttyWeb/README.md index 9db18d3d..d3880f4c 100644 --- a/src/renderer/ghosttyWeb/README.md +++ b/src/renderer/ghosttyWeb/README.md @@ -30,7 +30,7 @@ This directory holds the `ghostty-web` reference renderer. `GhosttyWebBackend` s - The constructor asserts a non-empty `sessionId`, `profile.name` and `profile.fontFamily`, a positive `fontSize`, and hex colors. With `videoOptions`, `outputDir` must be absolute and the size must use positive integers [backend.ts:GhosttyWebBackend.constructor]. - Replay cannot rewind: `targetSeq` must be >= the last applied seq. The initial dimensions cannot change after the first replay. `input.sessionId` must match the backend [backend.ts:GhosttyWebBackend.replayTo]. - `replayWithTiming` asserts strictly increasing seqs and non-decreasing timestamps [backend.ts:GhosttyWebBackend.replayWithTiming]. -- `replayWithTiming` calls the optional `onProgress` with a `ReplayProgress` in four places. Before each wait on the schedule, `scheduledWaitMs` is the length of that wait. A wait longer than a timer runs (2^31-1 ms) is a chain of timers, so a recorded gap of 30 days takes 30 days, and each timer after the first is followed by a report that carries the rest of the wait. After each bridge call of a flush, and after the resize call of a resize event, `scheduledWaitMs` is 0. After the last event and its flush, and before the final calls to the renderer, `scheduledWaitMs` is 0 too, so a stall in those calls names the last event, also when that event changed no pixel. `seq` is the last event that the replay had reached. `generateWebmExport` uses these reports to tell a stall from a wait [backend.ts:GhosttyWebBackend.replayWithTiming]. +- `replayWithTiming` calls the optional `onProgress` with a `ReplayProgress` in four places. Before each wait on the schedule, `scheduledWaitMs` is the length of that wait. A wait longer than a timer runs (2^31-1 ms) is a chain of timers, so a recorded gap of 30 days takes 30 days, and a report that carries the rest of the wait follows each timer except the last. After each bridge call of a flush, and after the resize call of a resize event, `scheduledWaitMs` is 0. After the last event and its flush, and before the final calls to the renderer, `scheduledWaitMs` is 0 too, so a stall in those calls names the last event, also when that event changed no pixel. `seq` is the last event that the replay had reached. `generateWebmExport` uses these reports to tell a stall from a wait [backend.ts:GhosttyWebBackend.replayWithTiming]. - `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` has no schedule to fall behind, so only recorded time ends its flushes. A flush ends at the first event that is at least one frame hold of recorded time after the batch's first output. An event that changes no pixel does not end it: input, marker, run-complete, signal and exit events leave the batch as it is, because a flush at each of them would cost a frame per event. A resize and the end of the replay still end it. `max-speed` also spreads at most `MAX_SPEED_FRAME_BUDGET` (2,400) flushes over the recorded span of the events that it applies: the distance between the first output of a flush and the event that ends it is at least the larger of the frame hold and the span divided by 2,400. So a `max-speed` replay makes about one flush per frame hold of recorded time for a recording shorter than 38 s and at most about 2,400 flushes for a longer one, whatever its number of events. Output that arrives less than that distance after the first output of a flush shares the flush, also when an input or a marker sits between, so a state that the next output of the flush replaces is never painted. A flush waits for one animation frame. `max-speed` holds the frame hold only after the last flush of the replay and before and after a resize, so two flushes that follow each other can paint within one video frame. In `recorded` and `accelerated` mode, every event that changes no pixel ends a flush, and a hold follows every flush. Events with the same timestamp share a flush at any hold, 0 included, because a flush needs at least 1 ms of recorded time [backend.ts:GhosttyWebBackend.replayWithTiming]. - `writeBatch(chunks, endsFlush, paintedCap)` takes the chunks of one bridge call. A flush that has more than 1000 chunks needs several calls, and ghostty-web paints on every animation frame, so the page holds the chunks of a call that does not end the flush. The call that ends it (`endsFlush`, set by the backend on the last call only) writes every held chunk in one task, so no frame paints a state between two calls, and then resolves after one frame. ghostty-web calls a write callback one animation frame (about 17 ms) after the write, and waiting on every chunk (`write`) cost one frame per output event. `write()` and `resize()` throw while chunks are held. `discardHeldChunks()` drops them: the backend calls it when a flush fails, because `HostRendererManager` keeps the backend and a retry would otherwise meet them. A `paintedCap` that is not null makes the call that ends the flush add the graphemes on the screen to the set that the missing-glyph check of `replayWithTiming` probes, so a flush is one page call. The call returns the message of a failed note and not an error, because a failed note must not fail the flush. In `replayWithTiming`, the chunks of one flush share one video frame [embeddedHarnessHtml.ts:EMBEDDED_HARNESS_HTML]. - `snapshot()` and `screenshot()` throw until `replayTo()` or `replayWithTiming()` has run once and set the initial size. Both replays accept `targetSeq` -1 for an empty Event Log. A replay over zero events sets `lastAppliedSeq` to `targetSeq` [backend.ts:GhosttyWebBackend.snapshot]. diff --git a/test/unit/renderer/ghosttyWebBackend.test.ts b/test/unit/renderer/ghosttyWebBackend.test.ts index 778a89d1..4f0e66a6 100644 --- a/test/unit/renderer/ghosttyWebBackend.test.ts +++ b/test/unit/renderer/ghosttyWebBackend.test.ts @@ -2234,6 +2234,17 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { }); }); + it('waits a recorded gap of 1 ms', async () => { + // The input event ends the flush of the first output, so the flush is + // the mark of the end of the wait before the input event. + const flushes = await replayAndRecordFlushes( + [outputEvent(0, 0), inputEvent(1, 1)], + RECORDED, + ); + + expect(flushes.map((flush) => flush.atMs)).toEqual([FIRST_EVENT_AT_MS + 1]); + }); + it('keeps the recorded pace when continuous output fills every frame hold', async () => { const events = Array.from({ length: 101 }, (_, index) => outputEvent(index, index * 10),