From ff8f2d264c9e8f5b9a6a80bb04e2af3e96520b01 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 07:23:15 +0000 Subject: [PATCH 01/26] perf: wait for one frame per flush when replaying output in ghostty-web The harness awaited terminal.write(data, callback) for every output chunk. ghostty-web runs that callback one animation frame later, so a replay cost about 17-24 ms per event even though Node already sent up to 1000 chunks per page call. Add a harness writeBatch(chunks, waitForFrame) that writes all chunks at once and waits for one frame only when asked. The backend asks for the frame on the last page call of a flush. Screenshots and snapshots are byte-identical to before. In a WebM replay, the chunks of one flush now share one video frame. --- src/renderer/ghosttyWeb/README.md | 5 +- src/renderer/ghosttyWeb/backend.ts | 31 ++-- .../ghosttyWeb/embeddedHarnessHtml.ts | 33 +++++ test/integration/renderer-backend.test.ts | 73 ++++++++++ test/unit/renderer/ghosttyWebBackend.test.ts | 134 ++++++++++++++++++ 5 files changed, 264 insertions(+), 12 deletions(-) diff --git a/src/renderer/ghosttyWeb/README.md b/src/renderer/ghosttyWeb/README.md index 10ea8ac4..a48cc672 100644 --- a/src/renderer/ghosttyWeb/README.md +++ b/src/renderer/ghosttyWeb/README.md @@ -20,7 +20,7 @@ This directory holds the `ghostty-web` reference renderer. `GhosttyWebBackend` s - `stripTrailingAsciiSpaces(text)` and `assembleCanonicalLine(cols, readColumn)` build canonical line text [harnessDecoding.ts:assembleCanonicalLine]. - Assertion helpers `assertNonNegativeInteger`, `assertPositiveInteger`, `assertPositiveNumber` and `assertHexColor`, plus `normalizeError(error, prefix)` [harnessDecoding.ts:normalizeError]. - `loadHarnessHtml()` returns `EMBEDDED_HARNESS_HTML` [harnessDecoding.ts:loadHarnessHtml]. -- `EMBEDDED_HARNESS_HTML` is the harness page as a template string. In the browser it defines `window.__agentTty` with `write`, `getSnapshot`, `getCellGraphemes`, `getVisibleText`, `isReady`, `resize` and `setCursorVisible` [embeddedHarnessHtml.ts:EMBEDDED_HARNESS_HTML]. +- `EMBEDDED_HARNESS_HTML` is the harness page as a template string. In the browser it defines `window.__agentTty` with `write`, `writeBatch`, `getSnapshot`, `getCellGraphemes`, `getVisibleText`, `isReady`, `resize` and `setCursorVisible` [embeddedHarnessHtml.ts:EMBEDDED_HARNESS_HTML]. - `harness.html` is a standalone copy of the harness page. No code serves it, and it differs from `EMBEDDED_HARNESS_HTML` [harnessDecoding.ts:loadHarnessHtml]. - `assets/` (not documented separately): holds the bundled fonts `JetBrainsMono-Regular-latin.woff2`, `SymbolsNerdFontMono-Regular.ttf`, `NotoSansSymbols2-Regular.woff2` and `NotoSansMath-Regular.woff2`, plus `FONT-LICENSE.txt` and the OFL texts of the Noto fonts. `scripts/subset-symbol-fonts.sh` rebuilds the two Noto subsets. `src/renderer/bundledFont.ts` loads them, and `scripts/copy-renderer-assets.mjs` copies them into `dist/` during `npm run build`. @@ -30,6 +30,7 @@ This directory holds the `ghostty-web` reference renderer. `GhosttyWebBackend` s - Replay cannot rewind: `targetSeq` must be >= the last applied seq. The initial dimensions cannot change after the first replay. `input.sessionId` must match the backend [backend.ts:GhosttyWebBackend.replayTo]. - `replayWithTiming` asserts strictly increasing seqs and non-decreasing timestamps [backend.ts:GhosttyWebBackend.replayWithTiming]. - `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` flushes once per run of output events [backend.ts:GhosttyWebBackend.replayWithTiming]. +- `writeBatch(chunks, waitForFrame)` writes every chunk of a call at once. ghostty-web calls a write callback one animation frame (about 17 ms) after the write, and waiting on every chunk (`write`) cost one frame per output event. With `waitForFrame` the call resolves after one frame, and the backend sets it only on the last call of a flush, so a flush of any size waits for one frame. In `replayWithTiming`, the chunks of one flush share one video frame [embeddedHarnessHtml.ts:EMBEDDED_HARNESS_HTML]. - `snapshot()` and `screenshot()` throw until `replayTo()` or `replayWithTiming()` has run once and set the initial size. Both replays accept `targetSeq` -1 for an empty Event Log. A replay over zero events sets `lastAppliedSeq` to `targetSeq` [backend.ts:GhosttyWebBackend.snapshot]. - `missingGlyphs` lists a code point only when a bundled face that lacks it drew it. Chromium draws a cluster with one font: the first stack font that covers all of it, or else the first font of the stack, which is bundled in both built-in profiles and then draws `.notdef` boxes. CDP names the face that drew a probe by its PostScript name, so a family alias or several faces under one family resolve to the right cmap. When one bundled face drew the whole grapheme, every code point that face lacks is listed. A probe with no reported font is judged against every face of the first family. When the grapheme is split over several fonts, each uncovered code point is judged by its own probe, and a mark's probe on its base is compared with a probe of the base alone. Glyphs that a host font draws are not listed. The check runs after the PNG is written, removes its probe spans, and caches each probe text's result for the page, clearing the cache when it would pass `MAX_CACHED_GLYPH_PROBES` (16384) entries. It probes at most `MAX_GLYPH_PROBES` texts and lists at most `MAX_MISSING_GLYPH_CELLS` cells (4096 each). `probeUsedFonts` sends its CDP queries in batches of 256 and stops starting batches after `GLYPH_PROBE_BUDGET_MS` (1000 ms). The result sets `missingGlyphsTruncated` when any of these stops the check early. Family names match without regard to case. A web font whose PostScript name the profile does not bundle fails the check. A failed check is logged at debug level and leaves the field out, so the screenshot still succeeds. `replayWithTiming()` records the viewport graphemes in the page after each output batch and each resize (at most `MAX_CACHED_GLYPH_PROBES`), and the final frame hold judges them once, with the hold as the probe budget, so the check does not lengthen the video. `recordedMissingGlyphs()` returns that result. A failed note or check leaves the record export fields out. A profile whose font stack does not start with a bundled font gets no `missingGlyphs` field [missingGlyphs.ts:probeUsedFonts] [backend.ts:GhosttyWebBackend.screenshot]. - `screenshot()` requires an absolute `outputPath` whose parent directory exists. It checks that the output is a non-empty PNG with a valid signature and an `IHDR` first chunk [backend.ts:GhosttyWebBackend.screenshot]. @@ -49,7 +50,7 @@ This directory holds the `ghostty-web` reference renderer. `GhosttyWebBackend` s - Reads the `ghostty-web` package files (`ghostty-web.js`, its browser-external helper and `ghostty-vt.wasm`) with `fs/promises` once per process [backend.ts:getServedAssets]. - Starts a `node:http` server on `127.0.0.1` with an ephemeral port. It serves the harness, the `ghostty-web` assets and the bundled fonts for `GET` and `HEAD` [backend.ts:GhosttyWebBackend.startServer]. - Calls `ensurePlaywrightBrowsersPath()`, which can set `process.env.PLAYWRIGHT_BROWSERS_PATH`. Then it launches headless Chromium with `chromium.launch` [backend.ts:GhosttyWebBackend.bootInternal]. -- Calls `page.evaluate` on `window.__agentTty` to write output, resize and read snapshots [backend.ts:GhosttyWebBackend.writeBatchBridge]. +- Calls `page.evaluate` on `window.__agentTty` to write output, resize and read snapshots. One `writeBatch` call carries up to 1000 output chunks [backend.ts:GhosttyWebBackend.writeBatchBridge]. - `screenshot()` writes the PNG through Playwright `locator('#terminal').screenshot`, then reads it back to hash it [backend.ts:GhosttyWebBackend.screenshot]. - With `videoOptions`, Playwright records into `outputDir`. `finalizeVideo()` closes the page and context and calls `video.saveAs(outputPath)` [backend.ts:GhosttyWebBackend.finalizeVideo]. - `dispose()` closes the page, context, browser and HTTP server [backend.ts:GhosttyWebBackend.dispose]. diff --git a/src/renderer/ghosttyWeb/backend.ts b/src/renderer/ghosttyWeb/backend.ts index 0aecf144..e872106d 100644 --- a/src/renderer/ghosttyWeb/backend.ts +++ b/src/renderer/ghosttyWeb/backend.ts @@ -81,6 +81,10 @@ interface GhosttyServedAsset extends GhosttyRequestAsset { interface GhosttyBrowserBridge { isReady?: () => boolean; write?: (data: string) => Promise | void; + writeBatch?: ( + chunks: string[], + waitForFrame?: boolean, + ) => Promise | void; resize?: (cols: number, rows: number) => Promise | void; setCursorVisible?: (visible: boolean) => Promise | void; getSnapshot?: (options?: SnapshotOptions) => GhosttyHarnessSnapshot; @@ -1692,7 +1696,12 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { batch.length <= MAX_REPLAY_BATCH_SIZE, 'flushOutputBatch batch size must respect MAX_REPLAY_BATCH_SIZE', ); - await this.writeBatchBridge(page, batch); + // The page waits for a frame only after the last batch of the flush. + await this.writeBatchBridge( + page, + batch, + batchStart + MAX_REPLAY_BATCH_SIZE >= dataChunks.length, + ); await afterEachBatch?.(); } } @@ -1777,6 +1786,7 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { private async writeBatchBridge( page: Page, dataChunks: string[], + waitForFrame = true, ): Promise { invariant( dataChunks.length > 0, @@ -1790,16 +1800,17 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { assertString(chunk, 'bridge batch write chunk must be a string'); } - await page.evaluate(async (chunks: string[]) => { - const bridge = (globalThis as GhosttyBrowserGlobal).__agentTty; - if (bridge === undefined || typeof bridge.write !== 'function') { - throw new Error('ghostty-web bridge write() is unavailable'); - } + await page.evaluate( + async ([chunks, shouldWaitForFrame]: readonly [string[], boolean]) => { + const bridge = (globalThis as GhosttyBrowserGlobal).__agentTty; + if (bridge === undefined || typeof bridge.writeBatch !== 'function') { + throw new Error('ghostty-web bridge writeBatch() is unavailable'); + } - for (const chunk of chunks) { - await bridge.write(chunk); - } - }, dataChunks); + await bridge.writeBatch(chunks, shouldWaitForFrame); + }, + [dataChunks, waitForFrame] as const, + ); } private async writeBridge(page: Page, data: string): Promise { diff --git a/src/renderer/ghosttyWeb/embeddedHarnessHtml.ts b/src/renderer/ghosttyWeb/embeddedHarnessHtml.ts index 6ed36c1b..adb0c9e5 100644 --- a/src/renderer/ghosttyWeb/embeddedHarnessHtml.ts +++ b/src/renderer/ghosttyWeb/embeddedHarnessHtml.ts @@ -651,6 +651,39 @@ export const EMBEDDED_HARNESS_HTML = ` }); updateDocumentState(); }, + async writeBatch(chunks, waitForFrame = true) { + const terminal = getReadyTerminal(); + invariant( + Array.isArray(chunks) && chunks.length > 0, + 'writeBatch() chunks must be a non-empty array', + ); + invariant( + typeof waitForFrame === 'boolean', + 'writeBatch() waitForFrame must be a boolean', + ); + for (const chunk of chunks) { + assertStringValue(chunk, 'writeBatch() chunks must be strings'); + } + + // The wasm write is synchronous, but ghostty-web calls a write + // callback one animation frame later. Waiting on every chunk made a + // replay cost one frame per event. The callback on the last chunk + // keeps the contract of write(): the call resolves after a frame. + // A caller that writes a run of batches asks for that frame only + // after the last one. + const lastIndex = chunks.length - 1; + for (let index = 0; index < lastIndex; index += 1) { + terminal.write(chunks[index]); + } + if (waitForFrame) { + await new Promise((resolve) => { + terminal.write(chunks[lastIndex], resolve); + }); + } else { + terminal.write(chunks[lastIndex]); + } + updateDocumentState(); + }, getSnapshot(options) { return getSnapshotPayload(options); }, diff --git a/test/integration/renderer-backend.test.ts b/test/integration/renderer-backend.test.ts index 5d357bf9..471b2191 100644 --- a/test/integration/renderer-backend.test.ts +++ b/test/integration/renderer-backend.test.ts @@ -755,6 +755,79 @@ describe('GhosttyWebBackend integration', { timeout: 120_000 }, () => { } }); + // ghostty-web calls a write callback one animation frame (about 17 ms) + // after the write. A bridge call that waits on every chunk made a replay cost + // one frame per output event: 90 minutes for 250 000 events. + it.each([ + [ + 'replayTo', + (renderer: GhosttyWebBackend, input: ReplayInput) => + renderer.replayTo(input), + ], + [ + 'replayWithTiming', + (renderer: GhosttyWebBackend, input: ReplayInput) => + renderer.replayWithTiming(input, { + mode: 'max-speed', + minFrameHoldMs: 0, + finalFrameHoldMs: 0, + }), + ], + ])( + '%s waits for one animation frame per flush of output events, not per event', + async (_name, replay) => { + await backend.boot(); + const { page } = backend as unknown as { page: Page }; + // ghostty-web calls a write callback one frame after the write, so the + // number of writes that pass a callback is the number of frames that a + // replay waits for. The string keeps vitest from rewriting the import. + await page.evaluate(`(async () => { + const { Terminal } = await import('/assets/ghostty-web.js'); + const write = Terminal.prototype.write; + globalThis.__writeFrameWaits = 0; + Terminal.prototype.write = function (data, callback) { + if (callback !== undefined) { + globalThis.__writeFrameWaits += 1; + } + return write.call(this, data, callback); + }; + })()`); + + // One character per event, so a chunk that is lost or moved changes + // the text on the screen. 2501 characters fill 32 rows of 80 columns. + const characters = Array.from({ length: 2_501 }, (_, seq) => + 'abcdefghijklmnopqrstuvwxyz0123456789'.charAt((seq * 7) % 36), + ); + const input = createReplayInput( + characters.map((character, seq) => ({ + seq, + ts: timestampFor(0), + type: 'output' as const, + payload: { data: character }, + })), + { initialRows: 40 }, + ); + await replay(backend, input); + + // A bridge call holds at most 1000 chunks, so 2501 events take 3 calls. + // Only the last call waits for a frame. + expect( + await page.evaluate( + () => + (globalThis as unknown as { __writeFrameWaits: number }) + .__writeFrameWaits, + ), + ).toBe(1); + const expectedRows = Array.from({ length: 32 }, (_, row) => + characters.slice(row * 80, (row + 1) * 80).join(''), + ); + const screenRows = (await backend.snapshot()).visibleLines.map( + (line) => line.text, + ); + expect(screenRows.slice(0, 32)).toEqual(expectedRows); + }, + ); + it('allows dispose to be called more than once', async () => { await backend.boot(); diff --git a/test/unit/renderer/ghosttyWebBackend.test.ts b/test/unit/renderer/ghosttyWebBackend.test.ts index 723b321f..86a90e8a 100644 --- a/test/unit/renderer/ghosttyWebBackend.test.ts +++ b/test/unit/renderer/ghosttyWebBackend.test.ts @@ -426,6 +426,140 @@ describe('GhosttyWebBackend unit guards', () => { expect(evaluate).not.toHaveBeenCalled(); }); + // The page waits one animation frame (about 17 ms) after each bridge write. + // A bridge call that writes its chunks one by one makes a replay cost one + // frame per output event, so these tests run the page function of every + // call against a counting bridge. + async function withCountingBridge( + run: (context: { + page: { isClosed: () => boolean; evaluate: ReturnType }; + write: ReturnType; + writeBatchSizes: number[]; + frameWaits: boolean[]; + writtenChunks: string[]; + }) => Promise, + ): Promise { + const writeBatchSizes: number[] = []; + const frameWaits: boolean[] = []; + const writtenChunks: string[] = []; + const write = vi.fn(() => Promise.resolve()); + vi.stubGlobal('__agentTty', { + write, + writeBatch: (chunks: string[], waitForFrame: boolean) => { + writeBatchSizes.push(chunks.length); + frameWaits.push(waitForFrame); + writtenChunks.push(...chunks); + return Promise.resolve(); + }, + }); + const page = { + isClosed: () => false, + evaluate: vi.fn((pageFunction: (arg: unknown) => unknown, arg: unknown) => + Promise.resolve(pageFunction(arg)), + ), + }; + + try { + return await run({ + page, + write, + writeBatchSizes, + frameWaits, + writtenChunks, + }); + } finally { + vi.unstubAllGlobals(); + } + } + + function outputReplayInput(eventCount: number): ReplayInput { + return { + sessionId: 'renderer-unit-session', + initialCols: 80, + initialRows: 24, + targetSeq: eventCount - 1, + events: Array.from({ length: eventCount }, (_, seq) => ({ + seq, + ts: '2026-03-19T12:00:00.000Z', + type: 'output' as const, + payload: { data: `chunk-${String(seq)}` }, + })), + }; + } + + it('writes a bridge batch into the page in one call that waits for a frame', async () => { + const backend = createBackend(); + const chunks = Array.from({ length: 1000 }, (_, i) => `chunk-${String(i)}`); + + await withCountingBridge( + async ({ page, write, writeBatchSizes, frameWaits, writtenChunks }) => { + await ( + backend as unknown as { + writeBatchBridge: ( + page: object, + dataChunks: string[], + ) => Promise; + } + ).writeBatchBridge(page, chunks); + + expect(page.evaluate).toHaveBeenCalledTimes(1); + expect(writeBatchSizes).toEqual([1000]); + expect(frameWaits).toEqual([true]); + expect(writtenChunks).toEqual(chunks); + expect(write).not.toHaveBeenCalled(); + }, + ); + }); + + it.each([ + [ + 'replayTo', + (backend: GhosttyWebBackend, input: ReplayInput) => + backend.replayTo(input), + ], + [ + 'replayWithTiming', + (backend: GhosttyWebBackend, input: ReplayInput) => + backend.replayWithTiming(input, { + mode: 'max-speed', + minFrameHoldMs: 0, + finalFrameHoldMs: 0, + }), + ], + ])( + '%s calls the page once per 1000 output events, and waits for a frame after the last call', + async (_name, replay) => { + const backend = createBackend(); + const input = outputReplayInput(2_501); + + await withCountingBridge( + async ({ page, write, writeBatchSizes, frameWaits, writtenChunks }) => { + Object.assign(backend as object, { + isBooted: true, + page, + resizeBridge: () => Promise.resolve(), + notePaintedGraphemes: () => Promise.resolve(), + checkPaintedGlyphs: () => Promise.resolve({}), + readHarnessSnapshot: () => + Promise.resolve(createHarnessSnapshotPayload({ cols: 80 })), + }); + + await replay(backend, input); + + expect(page.evaluate).toHaveBeenCalledTimes(3); + expect(writeBatchSizes).toEqual([1000, 1000, 501]); + expect(frameWaits).toEqual([false, false, true]); + expect(writtenChunks).toEqual( + input.events.map((event) => + event.type === 'output' ? event.payload.data : '', + ), + ); + expect(write).not.toHaveBeenCalled(); + }, + ); + }, + ); + it('includes structured cell data only when snapshot includeCells is requested', async () => { const backend = createBackend(); const evaluate = vi.fn().mockResolvedValue( From 3a8c8d9e1d1af993ef168a5a861f191c189cd1f9 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 09:45:51 +0000 Subject: [PATCH 02/26] fix: keep max-speed WebM frames per moment and paint each ghostty-web flush once Max-speed flushed once per run of output events, and a flush is one frame since the batched write. A WebM of an output-only recording then showed only its final state. Max-speed now flushes every frame hold (16 ms) of recorded time and holds the frame only after the last flush of a run, so it is never slower than one frame per output event. A flush of more than 1000 chunks takes several page calls. ghostty-web paints on every animation frame, so a state between two calls reached the video. The page now holds the chunks of a flush until the call that ends it and writes them in one task. Tests pin flushes of exactly 1000 and 2000 chunks and read the screencast frames. --- docs/USAGE.md | 2 + src/renderer/ghosttyWeb/README.md | 6 +- src/renderer/ghosttyWeb/backend.ts | 36 ++-- .../ghosttyWeb/embeddedHarnessHtml.ts | 53 ++++-- test/integration/renderer-backend.test.ts | 109 ++++++++++- test/unit/renderer/ghosttyWebBackend.test.ts | 170 +++++++++++++++++- 6 files changed, 332 insertions(+), 44 deletions(-) diff --git a/docs/USAGE.md b/docs/USAGE.md index e3d737c0..17e83909 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -294,6 +294,8 @@ agent-tty record export --format webm --json | jq -c '.result.missi WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. +A frame of the video shows the screen after all the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event. `max-speed` removes the idle time but gives each 16 ms of recorded output its own frame, so a progress bar that updates every 30 ms shows its steps and not only its last one. The video has 25 frames per second, so it can still skip a state that lasts less than one frame (40 ms). + SVG export renders styled grid frames from the event log through the native `libghostty-vt` backend with no browser and no ffmpeg, so it requires the optional `@coder/libghostty-vt-node` package (there is no `ghostty-web` fallback). The output is deterministic: exporting the same session twice produces byte-identical, diffable SVG. `--format svg` writes a still image of the final screen and ignores event timing, so it also works on a log whose clock stepped backward (`--animate` refuses such a log and names the event where the clock stepped back); add `--animate` for an animated SVG of de-duplicated frames replayed with recorded event-log timing (`--timing` is not supported with SVG). Animated SVG capture retains every distinct frame's styled grid in memory, so it is bounded by a total-cell budget (the V8 heap limit divided by 400 bytes, at most 20M cells or roughly 10,000 distinct 80x24 frames, adapting to terminal size); recordings that exceed it fail with a clear error suggesting a still SVG or WebM export instead. The SVG text is also bounded (the V8 heap limit divided by 8 bytes, at most 2^28 characters or about 268 MB of mostly ASCII text), so a dense, colorful animation that fits the cell budget can still fail with the same kind of error. diff --git a/src/renderer/ghosttyWeb/README.md b/src/renderer/ghosttyWeb/README.md index a48cc672..f98617e6 100644 --- a/src/renderer/ghosttyWeb/README.md +++ b/src/renderer/ghosttyWeb/README.md @@ -29,8 +29,8 @@ This directory holds the `ghostty-web` reference renderer. `GhosttyWebBackend` s - The constructor asserts a non-empty `sessionId`, `profile.name` and `profile.fontFamily`, a positive `fontSize`, and hex colors. With `videoOptions`, `outputDir` must be absolute and the size must use positive integers [backend.ts:GhosttyWebBackend.constructor]. - Replay cannot rewind: `targetSeq` must be >= the last applied seq. The initial dimensions cannot change after the first replay. `input.sessionId` must match the backend [backend.ts:GhosttyWebBackend.replayTo]. - `replayWithTiming` asserts strictly increasing seqs and non-decreasing timestamps [backend.ts:GhosttyWebBackend.replayWithTiming]. -- `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` flushes once per run of output events [backend.ts:GhosttyWebBackend.replayWithTiming]. -- `writeBatch(chunks, waitForFrame)` writes every chunk of a call at once. ghostty-web calls a write callback one animation frame (about 17 ms) after the write, and waiting on every chunk (`write`) cost one frame per output event. With `waitForFrame` the call resolves after one frame, and the backend sets it only on the last call of a flush, so a flush of any size waits for one frame. In `replayWithTiming`, the chunks of one flush share one video frame [embeddedHarnessHtml.ts:EMBEDDED_HARNESS_HTML]. +- `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` has no schedule to fall behind, so it flushes once the next event is at least one frame hold of recorded time after the batch's first output. Each moment that a viewer could tell apart gets a frame, and a replay costs at most one frame per output event. Events with the same timestamp share a flush when the hold is above 0 [backend.ts:GhosttyWebBackend.replayWithTiming]. +- `writeBatch(chunks, endsFlush)` takes the chunks of one bridge call. A flush that has more than 1000 chunks needs several calls, and ghostty-web paints on every animation frame, so the page holds the chunks of a call that does not end the flush. The call that ends it (`endsFlush`, set by the backend on the last call only) writes every held chunk in one task, so no frame paints a state between two calls, and then resolves after one frame. ghostty-web calls a write callback one animation frame (about 17 ms) after the write, and waiting on every chunk (`write`) cost one frame per output event. `write()` and `resize()` throw while chunks are held. In `replayWithTiming`, the chunks of one flush share one video frame [embeddedHarnessHtml.ts:EMBEDDED_HARNESS_HTML]. - `snapshot()` and `screenshot()` throw until `replayTo()` or `replayWithTiming()` has run once and set the initial size. Both replays accept `targetSeq` -1 for an empty Event Log. A replay over zero events sets `lastAppliedSeq` to `targetSeq` [backend.ts:GhosttyWebBackend.snapshot]. - `missingGlyphs` lists a code point only when a bundled face that lacks it drew it. Chromium draws a cluster with one font: the first stack font that covers all of it, or else the first font of the stack, which is bundled in both built-in profiles and then draws `.notdef` boxes. CDP names the face that drew a probe by its PostScript name, so a family alias or several faces under one family resolve to the right cmap. When one bundled face drew the whole grapheme, every code point that face lacks is listed. A probe with no reported font is judged against every face of the first family. When the grapheme is split over several fonts, each uncovered code point is judged by its own probe, and a mark's probe on its base is compared with a probe of the base alone. Glyphs that a host font draws are not listed. The check runs after the PNG is written, removes its probe spans, and caches each probe text's result for the page, clearing the cache when it would pass `MAX_CACHED_GLYPH_PROBES` (16384) entries. It probes at most `MAX_GLYPH_PROBES` texts and lists at most `MAX_MISSING_GLYPH_CELLS` cells (4096 each). `probeUsedFonts` sends its CDP queries in batches of 256 and stops starting batches after `GLYPH_PROBE_BUDGET_MS` (1000 ms). The result sets `missingGlyphsTruncated` when any of these stops the check early. Family names match without regard to case. A web font whose PostScript name the profile does not bundle fails the check. A failed check is logged at debug level and leaves the field out, so the screenshot still succeeds. `replayWithTiming()` records the viewport graphemes in the page after each output batch and each resize (at most `MAX_CACHED_GLYPH_PROBES`), and the final frame hold judges them once, with the hold as the probe budget, so the check does not lengthen the video. `recordedMissingGlyphs()` returns that result. A failed note or check leaves the record export fields out. A profile whose font stack does not start with a bundled font gets no `missingGlyphs` field [missingGlyphs.ts:probeUsedFonts] [backend.ts:GhosttyWebBackend.screenshot]. - `screenshot()` requires an absolute `outputPath` whose parent directory exists. It checks that the output is a non-empty PNG with a valid signature and an `IHDR` first chunk [backend.ts:GhosttyWebBackend.screenshot]. @@ -50,7 +50,7 @@ This directory holds the `ghostty-web` reference renderer. `GhosttyWebBackend` s - Reads the `ghostty-web` package files (`ghostty-web.js`, its browser-external helper and `ghostty-vt.wasm`) with `fs/promises` once per process [backend.ts:getServedAssets]. - Starts a `node:http` server on `127.0.0.1` with an ephemeral port. It serves the harness, the `ghostty-web` assets and the bundled fonts for `GET` and `HEAD` [backend.ts:GhosttyWebBackend.startServer]. - Calls `ensurePlaywrightBrowsersPath()`, which can set `process.env.PLAYWRIGHT_BROWSERS_PATH`. Then it launches headless Chromium with `chromium.launch` [backend.ts:GhosttyWebBackend.bootInternal]. -- Calls `page.evaluate` on `window.__agentTty` to write output, resize and read snapshots. One `writeBatch` call carries up to 1000 output chunks [backend.ts:GhosttyWebBackend.writeBatchBridge]. +- Calls `page.evaluate` on `window.__agentTty` to write output, resize and read snapshots. One `writeBatch` call carries up to 1000 output chunks, and the page holds them until the last call of the flush [backend.ts:GhosttyWebBackend.writeBatchBridge]. - `screenshot()` writes the PNG through Playwright `locator('#terminal').screenshot`, then reads it back to hash it [backend.ts:GhosttyWebBackend.screenshot]. - With `videoOptions`, Playwright records into `outputDir`. `finalizeVideo()` closes the page and context and calls `video.saveAs(outputPath)` [backend.ts:GhosttyWebBackend.finalizeVideo]. - `dispose()` closes the page, context, browser and HTTP server [backend.ts:GhosttyWebBackend.dispose]. diff --git a/src/renderer/ghosttyWeb/backend.ts b/src/renderer/ghosttyWeb/backend.ts index e872106d..1321b8cf 100644 --- a/src/renderer/ghosttyWeb/backend.ts +++ b/src/renderer/ghosttyWeb/backend.ts @@ -81,10 +81,7 @@ interface GhosttyServedAsset extends GhosttyRequestAsset { interface GhosttyBrowserBridge { isReady?: () => boolean; write?: (data: string) => Promise | void; - writeBatch?: ( - chunks: string[], - waitForFrame?: boolean, - ) => Promise | void; + writeBatch?: (chunks: string[], endsFlush?: boolean) => Promise | void; resize?: (cols: number, rows: number) => Promise | void; setCursorVisible?: (visible: boolean) => Promise | void; getSnapshot?: (options?: SnapshotOptions) => GhosttyHarnessSnapshot; @@ -663,7 +660,7 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { let scheduleStartedAtMs = 0; let scheduledElapsedMs = 0; - const flushOutputBatch = async (): Promise => { + const flushOutputBatch = async (holdFrame = true): Promise => { if (pendingOutputChunks.length === 0) { return; } @@ -673,7 +670,9 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { ); pendingOutputChunks = []; pendingSinceTimestampMs = null; - await waitForDelay(resolvedMinFrameHoldMs); + if (holdFrame) { + await waitForDelay(resolvedMinFrameHoldMs); + } }; for (const event of input.events) { @@ -710,16 +709,20 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { // Output shows once its batch is a frame hold old, unless the replay // runs behind the schedule: then batches grow until it catches up, so // slow flushes lower the frame rate instead of stretching the video. - // Max-speed keeps one batch per run of output events, so its frame - // holds cannot outlast the export timeout. + // Max-speed has no schedule to fall behind. It flushes every frame + // hold of recorded time, so each moment that a viewer could tell apart + // gets a frame, and a flush costs at most one frame per output event. + // The frame wait of such a flush is already the frame of its moment, + // so max-speed holds only after the last flush of a run of output: a + // hold after every flush would export 30 ms steps slower than they ran. if ( - timing.mode !== 'max-speed' && pendingSinceTimestampMs !== null && eventTimestampMs - pendingSinceTimestampMs >= resolvedMinFrameHoldMs && - performance.now() - scheduleStartedAtMs <= scheduledElapsedMs + (timing.mode === 'max-speed' || + performance.now() - scheduleStartedAtMs <= scheduledElapsedMs) ) { - await flushOutputBatch(); + await flushOutputBatch(timing.mode !== 'max-speed'); } // Waiting for the event's place on the schedule lets frame holds and // bridge calls use up time that the recording spent idle. @@ -1696,7 +1699,8 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { batch.length <= MAX_REPLAY_BATCH_SIZE, 'flushOutputBatch batch size must respect MAX_REPLAY_BATCH_SIZE', ); - // The page waits for a frame only after the last batch of the flush. + // The page holds the batches of a flush until the last one. Then it + // writes them all and waits for one frame. await this.writeBatchBridge( page, batch, @@ -1786,7 +1790,7 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { private async writeBatchBridge( page: Page, dataChunks: string[], - waitForFrame = true, + endsFlush = true, ): Promise { invariant( dataChunks.length > 0, @@ -1801,15 +1805,15 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { } await page.evaluate( - async ([chunks, shouldWaitForFrame]: readonly [string[], boolean]) => { + async ([chunks, batchEndsFlush]: readonly [string[], boolean]) => { const bridge = (globalThis as GhosttyBrowserGlobal).__agentTty; if (bridge === undefined || typeof bridge.writeBatch !== 'function') { throw new Error('ghostty-web bridge writeBatch() is unavailable'); } - await bridge.writeBatch(chunks, shouldWaitForFrame); + await bridge.writeBatch(chunks, batchEndsFlush); }, - [dataChunks, waitForFrame] as const, + [dataChunks, endsFlush] as const, ); } diff --git a/src/renderer/ghosttyWeb/embeddedHarnessHtml.ts b/src/renderer/ghosttyWeb/embeddedHarnessHtml.ts index adb0c9e5..84299ee5 100644 --- a/src/renderer/ghosttyWeb/embeddedHarnessHtml.ts +++ b/src/renderer/ghosttyWeb/embeddedHarnessHtml.ts @@ -262,6 +262,8 @@ export const EMBEDDED_HARNESS_HTML = ` const state = { errorMessage: null, + // Chunks of a flush that writeBatch() holds until the call that ends it. + heldChunks: [], ready: false, terminal: null, }; @@ -645,43 +647,54 @@ export const EMBEDDED_HARNESS_HTML = ` async write(data) { const terminal = getReadyTerminal(); assertStringValue(data, 'write() data must be a string'); + invariant( + state.heldChunks.length === 0, + 'write() cannot run while writeBatch() holds the chunks of a flush', + ); await new Promise((resolve) => { terminal.write(data, resolve); }); updateDocumentState(); }, - async writeBatch(chunks, waitForFrame = true) { + async writeBatch(chunks, endsFlush = true) { const terminal = getReadyTerminal(); invariant( Array.isArray(chunks) && chunks.length > 0, 'writeBatch() chunks must be a non-empty array', ); invariant( - typeof waitForFrame === 'boolean', - 'writeBatch() waitForFrame must be a boolean', + typeof endsFlush === 'boolean', + 'writeBatch() endsFlush must be a boolean', ); for (const chunk of chunks) { assertStringValue(chunk, 'writeBatch() chunks must be strings'); } - // The wasm write is synchronous, but ghostty-web calls a write - // callback one animation frame later. Waiting on every chunk made a - // replay cost one frame per event. The callback on the last chunk - // keeps the contract of write(): the call resolves after a frame. - // A caller that writes a run of batches asks for that frame only - // after the last one. - const lastIndex = chunks.length - 1; - for (let index = 0; index < lastIndex; index += 1) { - terminal.write(chunks[index]); + // A flush shows as one frame. A flush that needs several calls holds + // its chunks here until the call that ends it, so no frame paints + // the screen between two calls: ghostty-web renders on every animation + // frame, and a video records every paint. The call that ends the + // flush writes all chunks in one task. The wasm write is synchronous, + // but ghostty-web calls a write callback one animation frame later, + // so waiting on every chunk made a replay cost one frame per event. + // The callback on the last chunk makes the call resolve after a frame. + for (const chunk of chunks) { + state.heldChunks.push(chunk); + } + if (!endsFlush) { + return; } - if (waitForFrame) { - await new Promise((resolve) => { - terminal.write(chunks[lastIndex], resolve); - }); - } else { - terminal.write(chunks[lastIndex]); + + const flush = state.heldChunks; + state.heldChunks = []; + const lastIndex = flush.length - 1; + for (let index = 0; index < lastIndex; index += 1) { + terminal.write(flush[index]); } + await new Promise((resolve) => { + terminal.write(flush[lastIndex], resolve); + }); updateDocumentState(); }, getSnapshot(options) { @@ -700,6 +713,10 @@ export const EMBEDDED_HARNESS_HTML = ` }, resize(cols, rows) { const terminal = getReadyTerminal(); + invariant( + state.heldChunks.length === 0, + 'resize() cannot run while writeBatch() holds the chunks of a flush', + ); assertPositiveInteger( cols, 'resize() cols must be a positive integer', diff --git a/test/integration/renderer-backend.test.ts b/test/integration/renderer-backend.test.ts index 471b2191..7d76cd88 100644 --- a/test/integration/renderer-backend.test.ts +++ b/test/integration/renderer-backend.test.ts @@ -1,3 +1,4 @@ +import { createHash } from 'node:crypto'; import { mkdtemp, readFile, realpath, rm, stat } from 'node:fs/promises'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; @@ -769,7 +770,7 @@ describe('GhosttyWebBackend integration', { timeout: 120_000 }, () => { (renderer: GhosttyWebBackend, input: ReplayInput) => renderer.replayWithTiming(input, { mode: 'max-speed', - minFrameHoldMs: 0, + minFrameHoldMs: 16, finalFrameHoldMs: 0, }), ], @@ -828,6 +829,112 @@ describe('GhosttyWebBackend integration', { timeout: 120_000 }, () => { }, ); + // ghostty-web paints on every animation frame, and a WebM records every + // paint. A flush of more than 1000 chunks takes several page calls, so a + // state that exists between two calls can reach the video. + it.each([ + [ + 'replayTo', + (renderer: GhosttyWebBackend, input: ReplayInput) => + renderer.replayTo(input), + ], + [ + 'replayWithTiming', + (renderer: GhosttyWebBackend, input: ReplayInput) => + renderer.replayWithTiming(input, { + mode: 'max-speed', + minFrameHoldMs: 16, + finalFrameHoldMs: 200, + }), + ], + ])( + '%s paints a flush of several page calls as one frame', + async (_name, replay) => { + await backend.boot(); + const { page } = backend as unknown as { page: Page }; + // The screencast is the source of the frames that Playwright encodes into + // a video: it sends a frame for every paint. + const screencast = await page.context().newCDPSession(page); + const frames: string[] = []; + screencast.on('Page.screencastFrame', (frame) => { + const hash = createHash('sha256').update(frame.data).digest('hex'); + if (frames.at(-1) !== hash) { + frames.push(hash); + } + void screencast + .send('Page.screencastFrameAck', { sessionId: frame.sessionId }) + .catch(() => {}); + }); + await screencast.send('Page.startScreencast', { + format: 'png', + everyNthFrame: 1, + }); + + // Each bridge call of 1000 chunks fills the screen with its own colour + // (red, green, blue), so a paint between two calls shows a colour that is + // not the last. A pause after each call stands for a slow host. + const backgroundFor = (seq: number): number => + [41, 42, 44][Math.floor(seq / 1_000)] ?? 44; + const input = createReplayInput( + Array.from({ length: 2_501 }, (_, seq) => ({ + seq, + ts: timestampFor(0), + type: 'output' as const, + payload: { data: `\u001b[${String(backgroundFor(seq))}m\u001b[2J` }, + })), + ); + const writeBatchBridge = ( + backend as unknown as { + writeBatchBridge: (...args: unknown[]) => Promise; + } + ).writeBatchBridge.bind(backend); + Object.assign(backend as object, { + writeBatchBridge: async (...args: unknown[]) => { + await writeBatchBridge(...args); + await new Promise((resolve) => setTimeout(resolve, 100)); + }, + }); + // Wait for the first frame, so that it is the screen before the replay. + await expect + .poll(() => frames.length, { timeout: 10_000 }) + .toBeGreaterThan(0); + await replay(backend, input); + // Let every frame that the replay caused reach the listener. + await new Promise((resolve) => setTimeout(resolve, 500)); + await screencast.send('Page.stopScreencast'); + + // The frame before the replay, then the frame of the last colour. + expect(frames).toHaveLength(2); + }, + ); + + it('holds the chunks of a flush until the call that ends it', async () => { + await backend.boot(); + const { page } = backend as unknown as { page: Page }; + const visibleFirstRow = (): Promise => + page.evaluate( + `window.__agentTty.getVisibleText().split('\\n')[0]`, + ) as Promise; + await page.evaluate('window.__agentTty.resize(80, 24)'); + + await page.evaluate(`window.__agentTty.writeBatch(['ab'], false)`); + await page.evaluate(`window.__agentTty.writeBatch(['cd'], false)`); + expect(await visibleFirstRow()).toBe(''); + + // A direct write or a resize would move the held chunks behind its own + // output, so the page refuses both. + await expect(page.evaluate(`window.__agentTty.write('x')`)).rejects.toThrow( + 'holds the chunks of a flush', + ); + await expect( + page.evaluate(`window.__agentTty.resize(100, 30)`), + ).rejects.toThrow('holds the chunks of a flush'); + expect(await visibleFirstRow()).toBe(''); + + await page.evaluate(`window.__agentTty.writeBatch(['ef'], true)`); + expect(await visibleFirstRow()).toBe('abcdef'); + }); + it('allows dispose to be called more than once', async () => { await backend.boot(); diff --git a/test/unit/renderer/ghosttyWebBackend.test.ts b/test/unit/renderer/ghosttyWebBackend.test.ts index 86a90e8a..8f14e15a 100644 --- a/test/unit/renderer/ghosttyWebBackend.test.ts +++ b/test/unit/renderer/ghosttyWebBackend.test.ts @@ -487,7 +487,7 @@ describe('GhosttyWebBackend unit guards', () => { }; } - it('writes a bridge batch into the page in one call that waits for a frame', async () => { + it('writes a bridge batch into the page in one call that ends its flush', async () => { const backend = createBackend(); const chunks = Array.from({ length: 1000 }, (_, i) => `chunk-${String(i)}`); @@ -522,12 +522,12 @@ describe('GhosttyWebBackend unit guards', () => { (backend: GhosttyWebBackend, input: ReplayInput) => backend.replayWithTiming(input, { mode: 'max-speed', - minFrameHoldMs: 0, + minFrameHoldMs: 16, finalFrameHoldMs: 0, }), ], ])( - '%s calls the page once per 1000 output events, and waits for a frame after the last call', + '%s calls the page once per 1000 output events, and ends the flush with the last call', async (_name, replay) => { const backend = createBackend(); const input = outputReplayInput(2_501); @@ -560,6 +560,70 @@ describe('GhosttyWebBackend unit guards', () => { }, ); + // A bridge call holds at most 1000 chunks. The page writes a flush when the + // call that ends it arrives, so the flag must be true on exactly one call of + // a flush, the last: also when the chunk count is a multiple of 1000. + it.each([ + [1, [true]], + [999, [true]], + [1_000, [true]], + [1_001, [false, true]], + [1_999, [false, true]], + [2_000, [false, true]], + [2_001, [false, false, true]], + ])( + 'ends a flush of %i output chunks with its last bridge call only', + async (chunkCount, expectedEnds) => { + const backend = createBackend(); + + await withCountingBridge( + async ({ page, writeBatchSizes, frameWaits }) => { + Object.assign(backend as object, { + isBooted: true, + page, + resizeBridge: () => Promise.resolve(), + readHarnessSnapshot: () => + Promise.resolve(createHarnessSnapshotPayload({ cols: 80 })), + }); + + await backend.replayTo(outputReplayInput(chunkCount)); + + expect(frameWaits).toEqual(expectedEnds); + expect(writeBatchSizes.reduce((sum, size) => sum + size, 0)).toBe( + chunkCount, + ); + }, + ); + }, + ); + + it('ends the flush before a marker, so the next flush starts holding again', async () => { + const backend = createBackend(); + const input = outputReplayInput(3_001); + input.events[1_500] = { + seq: 1_500, + ts: '2026-03-19T12:00:00.000Z', + type: 'marker', + payload: { label: 'boundary' }, + }; + + await withCountingBridge(async ({ page, writeBatchSizes, frameWaits }) => { + Object.assign(backend as object, { + isBooted: true, + page, + resizeBridge: () => Promise.resolve(), + readHarnessSnapshot: () => + Promise.resolve(createHarnessSnapshotPayload({ cols: 80 })), + }); + + await backend.replayTo(input); + + // 1500 chunks, the marker, then 1500 chunks. + expect(writeBatchSizes).toEqual([1_000, 500, 1_000, 500]); + expect(frameWaits).toEqual([false, true, false, true]); + }); + }); + it('includes structured cell data only when snapshot includeCells is requested', async () => { const backend = createBackend(); const evaluate = vi.fn().mockResolvedValue( @@ -1400,15 +1464,109 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { ]); }); - it('keeps max-speed outputs in one flush so frame holds cannot outlast the export timeout', async () => { + it('flushes max-speed output every frame hold of recorded time, so each moment gets a frame', async () => { const flushes = await replayAndRecordFlushes( - [outputEvent(0, 0), outputEvent(1, 200), outputEvent(2, 400)], + [ + outputEvent(0, 0), + outputEvent(1, 5), + outputEvent(2, 10), + outputEvent(3, 16), + outputEvent(4, 20), + outputEvent(5, 40), + outputEvent(6, 41), + exitEvent(7, 500), + ], MAX_SPEED, ); + expect(flushes.map((flush) => flush.chunks)).toEqual([ + ['out0', 'out1', 'out2'], + ['out3', 'out4'], + ['out5', 'out6'], + ]); + }); + + it('keeps one max-speed flush per moment when the outputs are far apart', async () => { + const flushes = await replayAndRecordFlushes( + [ + outputEvent(0, 0), + outputEvent(1, 200), + outputEvent(2, 400), + exitEvent(3, 600), + ], + MAX_SPEED, + ); + + expect(flushes.map((flush) => flush.chunks)).toEqual([ + ['out0'], + ['out1'], + ['out2'], + ]); + }); + + // The frame wait of a flush is the frame of its moment. A hold after every + // flush of a run would make each 30 ms step cost 44 ms to export. + it('holds max-speed frames only after the last flush of a run of output', async () => { + const flushes = await replayAndRecordFlushes( + [ + outputEvent(0, 0), + outputEvent(1, 20), + outputEvent(2, 40), + exitEvent(3, 50), + outputEvent(4, 80), + ], + MAX_SPEED, + ); + + // out0 and out1 flush when the next output arrives, so no hold follows + // them. The exit ends the run: out2 flushes, and its hold delays out4. expect(flushes).toEqual([ - { atMs: FIRST_EVENT_AT_MS, chunks: ['out0', 'out1', 'out2'] }, + { atMs: FIRST_EVENT_AT_MS, chunks: ['out0'] }, + { atMs: FIRST_EVENT_AT_MS, chunks: ['out1'] }, + { atMs: FIRST_EVENT_AT_MS, chunks: ['out2'] }, + { atMs: FIRST_EVENT_AT_MS + 16, chunks: ['out4'] }, + ]); + }); + + it('does not merge max-speed moments when a flush costs more than a frame hold', async () => { + const events = Array.from({ length: 101 }, (_, index) => + outputEvent(index, index * 10), + ); + events.push(exitEvent(101, 1_000)); + const flushes = await replayAndRecordFlushes(events, MAX_SPEED, 30); + + // Recorded timing lets batches grow while the replay is behind its + // schedule. Max-speed has no schedule, so a slow flush must not hide a + // moment: every flush holds the outputs of one frame hold (two events). + expect(flushes.length).toBe(51); + expect(flushes.every((flush) => flush.chunks.length <= 2)).toBe(true); + expect(flushes.flatMap((flush) => flush.chunks)).toEqual( + events.slice(0, 101).map((event) => `out${String(event.seq)}`), + ); + }); + + it('keeps accelerated frames at least one frame hold of recorded time apart', async () => { + const accelerated: ReplayTimingOptions = { + mode: 'accelerated', + maxGapMs: 400, + minFrameHoldMs: 100, + finalFrameHoldMs: 1_000, + }; + const events = Array.from({ length: 41 }, (_, index) => + outputEvent(index, index * 30), + ); + events.push(exitEvent(41, 1_300)); + const flushes = await replayAndRecordFlushes(events, accelerated); + + // 30 ms steps, 100 ms holds: a flush takes the outputs of four steps. + expect(flushes.map((flush) => flush.chunks.length)).toEqual([ + 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 1, ]); + for (let index = 1; index < flushes.length; index += 1) { + const gapMs = + (flushes[index]?.atMs ?? 0) - (flushes[index - 1]?.atMs ?? 0); + expect(gapMs).toBeGreaterThanOrEqual(100); + } }); it('refreshes continuous output at least once per frame hold', async () => { From 4a65f4d4029d036eb000c24acdf1d37ad6f75caf Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 09:52:27 +0000 Subject: [PATCH 03/26] docs: keep the WebM frame explanation in the timing paragraph of USAGE.md --- docs/USAGE.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs/USAGE.md b/docs/USAGE.md index 17e83909..60eaa353 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -292,9 +292,7 @@ agent-tty record export --format webm --json | jq -c '.result.missi # [{"codePoint":"U+0378"}] ``` -WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. - -A frame of the video shows the screen after all the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event. `max-speed` removes the idle time but gives each 16 ms of recorded output its own frame, so a progress bar that updates every 30 ms shows its steps and not only its last one. The video has 25 frames per second, so it can still skip a state that lasts less than one frame (40 ms). +WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. A frame of the video shows the screen after all the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event. `max-speed` removes the idle time but gives each 16 ms of recorded output its own frame, so a progress bar that updates every 30 ms shows its steps and not only its last one. The video has 25 frames per second, so it can still skip a state that lasts less than one frame (40 ms). SVG export renders styled grid frames from the event log through the native `libghostty-vt` backend with no browser and no ffmpeg, so it requires the optional `@coder/libghostty-vt-node` package (there is no `ghostty-web` fallback). The output is deterministic: exporting the same session twice produces byte-identical, diffable SVG. `--format svg` writes a still image of the final screen and ignores event timing, so it also works on a log whose clock stepped backward (`--animate` refuses such a log and names the event where the clock stepped back); add `--animate` for an animated SVG of de-duplicated frames replayed with recorded event-log timing (`--timing` is not supported with SVG). From 9da052f692568d43c99d898ee3dd08a64e937b79 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 09:53:50 +0000 Subject: [PATCH 04/26] fix: keep events with one timestamp in one max-speed flush when the frame hold is 0 --- src/renderer/ghosttyWeb/backend.ts | 5 +++-- test/integration/renderer-backend.test.ts | 2 +- test/unit/renderer/ghosttyWebBackend.test.ts | 2 +- 3 files changed, 5 insertions(+), 4 deletions(-) diff --git a/src/renderer/ghosttyWeb/backend.ts b/src/renderer/ghosttyWeb/backend.ts index 1321b8cf..ae5262c2 100644 --- a/src/renderer/ghosttyWeb/backend.ts +++ b/src/renderer/ghosttyWeb/backend.ts @@ -706,7 +706,8 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { 'replay event timestamps must be ordered non-decreasingly', ); scheduledElapsedMs += computeInterEventDelay(interEventDelayMs); - // Output shows once its batch is a frame hold old, unless the replay + // Output shows once its batch is a frame hold old (and at least 1 ms: + // events with one timestamp are one moment), unless the replay // runs behind the schedule: then batches grow until it catches up, so // slow flushes lower the frame rate instead of stretching the video. // Max-speed has no schedule to fall behind. It flushes every frame @@ -718,7 +719,7 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { if ( pendingSinceTimestampMs !== null && eventTimestampMs - pendingSinceTimestampMs >= - resolvedMinFrameHoldMs && + Math.max(1, resolvedMinFrameHoldMs) && (timing.mode === 'max-speed' || performance.now() - scheduleStartedAtMs <= scheduledElapsedMs) ) { diff --git a/test/integration/renderer-backend.test.ts b/test/integration/renderer-backend.test.ts index 7d76cd88..fe956f17 100644 --- a/test/integration/renderer-backend.test.ts +++ b/test/integration/renderer-backend.test.ts @@ -770,7 +770,7 @@ describe('GhosttyWebBackend integration', { timeout: 120_000 }, () => { (renderer: GhosttyWebBackend, input: ReplayInput) => renderer.replayWithTiming(input, { mode: 'max-speed', - minFrameHoldMs: 16, + minFrameHoldMs: 0, finalFrameHoldMs: 0, }), ], diff --git a/test/unit/renderer/ghosttyWebBackend.test.ts b/test/unit/renderer/ghosttyWebBackend.test.ts index 8f14e15a..1593fe0d 100644 --- a/test/unit/renderer/ghosttyWebBackend.test.ts +++ b/test/unit/renderer/ghosttyWebBackend.test.ts @@ -522,7 +522,7 @@ describe('GhosttyWebBackend unit guards', () => { (backend: GhosttyWebBackend, input: ReplayInput) => backend.replayWithTiming(input, { mode: 'max-speed', - minFrameHoldMs: 16, + minFrameHoldMs: 0, finalFrameHoldMs: 0, }), ], From 060eed1df6e3a42d82157b8e8df7ec58267d8b02 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 10:24:17 +0000 Subject: [PATCH 05/26] test: wait for the last screencast frame before counting frames --- test/integration/renderer-backend.test.ts | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/test/integration/renderer-backend.test.ts b/test/integration/renderer-backend.test.ts index fe956f17..6d077211 100644 --- a/test/integration/renderer-backend.test.ts +++ b/test/integration/renderer-backend.test.ts @@ -899,7 +899,11 @@ describe('GhosttyWebBackend integration', { timeout: 120_000 }, () => { .poll(() => frames.length, { timeout: 10_000 }) .toBeGreaterThan(0); await replay(backend, input); - // Let every frame that the replay caused reach the listener. + // The last colour must reach the listener. Then wait a little more, so + // that a frame between two calls would have arrived too. + await expect + .poll(() => frames.length, { timeout: 10_000 }) + .toBeGreaterThan(1); await new Promise((resolve) => setTimeout(resolve, 500)); await screencast.send('Page.stopScreencast'); From 078fe21b79b35b4d1e8fce6498b82264cf8b33f4 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 13:55:00 +0000 Subject: [PATCH 06/26] perf: note painted graphemes in the page call of a flush, bound max-speed frames, drop held chunks after a failed flush --- src/renderer/ghosttyWeb/backend.ts | 222 +++++++++++----- .../ghosttyWeb/embeddedHarnessHtml.ts | 54 +++- test/integration/renderer-backend.test.ts | 116 +++++++++ test/unit/renderer/ghosttyWebBackend.test.ts | 237 ++++++++++++++++-- 4 files changed, 545 insertions(+), 84 deletions(-) diff --git a/src/renderer/ghosttyWeb/backend.ts b/src/renderer/ghosttyWeb/backend.ts index ae5262c2..724fffdd 100644 --- a/src/renderer/ghosttyWeb/backend.ts +++ b/src/renderer/ghosttyWeb/backend.ts @@ -81,7 +81,13 @@ interface GhosttyServedAsset extends GhosttyRequestAsset { interface GhosttyBrowserBridge { isReady?: () => boolean; write?: (data: string) => Promise | void; - writeBatch?: (chunks: string[], endsFlush?: boolean) => Promise | void; + writeBatch?: ( + chunks: string[], + endsFlush?: boolean, + paintedCap?: number | null, + ) => Promise | string | null; + discardHeldChunks?: () => void; + notePaintedGraphemes?: (cap: number) => void; resize?: (cols: number, rows: number) => Promise | void; setCursorVisible?: (visible: boolean) => Promise | void; getSnapshot?: (options?: SnapshotOptions) => GhosttyHarnessSnapshot; @@ -131,12 +137,43 @@ const HTML_CONTENT_TYPE = 'text/html; charset=utf-8'; const WASM_CONTENT_TYPE = 'application/wasm'; const MAX_REPLAY_BATCH_SIZE = 1000; +/** + * A max-speed replay flushes at most about this many times, however long the + * recording is. Every flush waits for one animation frame (17 to 25 ms) and + * one page call, so a replay takes about 2 minutes at the limit. That stays + * below the 5 minutes that a WebM export allows its replay, even on a slow + * host. The video has 25 frames per second, so 3,000 flushes fill 2 minutes + * of video: a viewer cannot tell more states apart. + */ +const MAX_SPEED_FRAME_BUDGET = 3_000; const RAF_TIMEOUT_MS = 5_000; let servedAssetsPromise: Promise< ReadonlyMap > | null = null; +/** + * Recorded time between the first and the last event that a replay applies: + * the seqs above `afterSeq`, up to the target. It is 0 when a timestamp does + * not parse. The replay loop rejects such a timestamp itself. + */ +function recordedSpanMs(input: ReplayInput, afterSeq: number): number { + let firstMs: number | undefined; + let lastMs = 0; + for (const event of input.events) { + if (event.seq <= afterSeq) { + continue; + } + if (event.seq > input.targetSeq) { + break; + } + lastMs = Date.parse(event.ts); + firstMs ??= lastMs; + } + const spanMs = lastMs - (firstMs ?? lastMs); + return Number.isFinite(spanMs) && spanMs > 0 ? spanMs : 0; +} + async function closeServer(server: Server): Promise { if (!server.listening) { return; @@ -649,6 +686,18 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { ); } + // Output shows once the next event is this far after the batch's first + // output. It is at least 1 ms: events with one timestamp are one moment. + // Max-speed has no schedule that bounds its replay time, so it also + // spreads a fixed number of flushes over the recorded span: a sparse + // recording then costs the same number of frames as a dense one. + const minSplitMs = Math.max( + 1, + resolvedMinFrameHoldMs, + timing.mode === 'max-speed' + ? recordedSpanMs(input, this.lastAppliedSeq) / MAX_SPEED_FRAME_BUDGET + : 0, + ); let outputEventCount = 0; let previousEventSeq = -1; let previousProcessedEventTimestampMs: number | null = null; @@ -665,9 +714,7 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { return; } - await this.flushOutputBatch(page, pendingOutputChunks, () => - this.notePaintedGraphemes(page), - ); + await this.flushOutputBatch(page, pendingOutputChunks, undefined, true); pendingOutputChunks = []; pendingSinceTimestampMs = null; if (holdFrame) { @@ -675,6 +722,16 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { } }; + // Events that change no pixel end a flush in recorded and accelerated + // mode, where the pause before them is part of the video. Max-speed has no + // pauses: a flush at each of them would cost one frame per event, and a + // viewer cannot tell apart what happens within one minimum split. + const flushBeforeNonVisualEvent = async (): Promise => { + if (timing.mode !== 'max-speed') { + await flushOutputBatch(); + } + }; + for (const event of input.events) { assertNonNegativeInteger( event.seq, @@ -706,20 +763,18 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { 'replay event timestamps must be ordered non-decreasingly', ); scheduledElapsedMs += computeInterEventDelay(interEventDelayMs); - // Output shows once its batch is a frame hold old (and at least 1 ms: - // events with one timestamp are one moment), unless the replay - // runs behind the schedule: then batches grow until it catches up, so - // slow flushes lower the frame rate instead of stretching the video. - // Max-speed has no schedule to fall behind. It flushes every frame - // hold of recorded time, so each moment that a viewer could tell apart - // gets a frame, and a flush costs at most one frame per output event. - // The frame wait of such a flush is already the frame of its moment, - // so max-speed holds only after the last flush of a run of output: a + // Output shows once its batch is a minimum split old, unless the + // replay runs behind the schedule: then batches grow until it catches + // up, so slow flushes lower the frame rate instead of stretching the + // video. Max-speed has no schedule to fall behind. It flushes every + // minimum split of recorded time, so each moment that a viewer could + // tell apart gets a frame, within the frame budget. The frame wait of + // such a flush is already the frame of its moment, so max-speed holds + // only after a flush that ends the replay or comes before a resize: a // hold after every flush would export 30 ms steps slower than they ran. if ( pendingSinceTimestampMs !== null && - eventTimestampMs - pendingSinceTimestampMs >= - Math.max(1, resolvedMinFrameHoldMs) && + eventTimestampMs - pendingSinceTimestampMs >= minSplitMs && (timing.mode === 'max-speed' || performance.now() - scheduleStartedAtMs <= scheduledElapsedMs) ) { @@ -764,7 +819,7 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { break; } case 'marker': { - await flushOutputBatch(); + await flushBeforeNonVisualEvent(); break; } case 'input_text': @@ -775,7 +830,7 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { case 'run_complete': case 'signal': case 'exit': { - await flushOutputBatch(); + await flushBeforeNonVisualEvent(); break; } default: { @@ -1007,40 +1062,39 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { return; } await page.evaluate((cap) => { - const scope = globalThis as GhosttyBrowserGlobal; - const bridge = scope.__agentTty; + const bridge = (globalThis as GhosttyBrowserGlobal).__agentTty; if ( bridge === undefined || - typeof bridge.getCellGraphemes !== 'function' + typeof bridge.notePaintedGraphemes !== 'function' ) { throw new Error( - 'ghostty-web bridge getCellGraphemes() is unavailable', + 'ghostty-web bridge notePaintedGraphemes() is unavailable', ); } - scope.__agentTtyPainted ??= { seen: new Set(), complete: true }; - const painted = scope.__agentTtyPainted; - for (const row of bridge.getCellGraphemes()) { - for (const grapheme of row) { - if (grapheme === '' || painted.seen.has(grapheme)) { - continue; - } - if (painted.seen.size < cap) { - painted.seen.add(grapheme); - } else { - painted.complete = false; - } - } - } + bridge.notePaintedGraphemes(cap); }, MAX_CACHED_GLYPH_PROBES); } catch (error) { - this.paintedNoteFailed = true; - this.logger.debug( - 'Missing-glyph note failed; the record export result omits missingGlyphs', + this.failPaintedNote( error instanceof Error ? error.message : String(error), ); } } + private failPaintedNote(message: string): void { + this.paintedNoteFailed = true; + this.logger.debug( + 'Missing-glyph note failed; the record export result omits missingGlyphs', + message, + ); + } + + /** The cap that a flush passes to the page to note its painted graphemes. */ + private paintedNoteCap(): number | null { + return this.paintedNoteFailed || this.glyphCheck() === null + ? null + : MAX_CACHED_GLYPH_PROBES; + } + public async recordedMissingGlyphs(): Promise { return ( this.recordedGlyphs ?? this.checkPaintedGlyphs(GLYPH_PROBE_BUDGET_MS) @@ -1676,38 +1730,70 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { ]); } + /** + * Writes the chunks of one flush into the page. With `notePainted`, the page + * also notes the graphemes on the screen once the flush is written, in the + * call that ends the flush. + */ private async flushOutputBatch( page: Page, dataChunks: string[], afterEachBatch?: () => Promise, + notePainted = false, ): Promise { invariant( dataChunks.length > 0, 'flushOutputBatch requires at least one data chunk', ); - for ( - let batchStart = 0; - batchStart < dataChunks.length; - batchStart += MAX_REPLAY_BATCH_SIZE - ) { - const batch = dataChunks.slice( - batchStart, - batchStart + MAX_REPLAY_BATCH_SIZE, - ); - invariant(batch.length > 0, 'flushOutputBatch batch must not be empty'); - invariant( - batch.length <= MAX_REPLAY_BATCH_SIZE, - 'flushOutputBatch batch size must respect MAX_REPLAY_BATCH_SIZE', - ); - // The page holds the batches of a flush until the last one. Then it - // writes them all and waits for one frame. - await this.writeBatchBridge( - page, - batch, - batchStart + MAX_REPLAY_BATCH_SIZE >= dataChunks.length, + try { + for ( + let batchStart = 0; + batchStart < dataChunks.length; + batchStart += MAX_REPLAY_BATCH_SIZE + ) { + const batch = dataChunks.slice( + batchStart, + batchStart + MAX_REPLAY_BATCH_SIZE, + ); + invariant(batch.length > 0, 'flushOutputBatch batch must not be empty'); + invariant( + batch.length <= MAX_REPLAY_BATCH_SIZE, + 'flushOutputBatch batch size must respect MAX_REPLAY_BATCH_SIZE', + ); + // The page holds the batches of a flush until the last one. Then it + // writes them all and waits for one frame. + const endsFlush = + batchStart + MAX_REPLAY_BATCH_SIZE >= dataChunks.length; + await this.writeBatchBridge( + page, + batch, + endsFlush, + endsFlush && notePainted ? this.paintedNoteCap() : null, + ); + await afterEachBatch?.(); + } + } catch (error) { + // The page still holds the chunks of the calls that did succeed. The + // backend stays in use after a failed replay, and a retry would meet + // them: its output would follow them, and its resize would be refused. + await this.discardHeldChunks(page); + throw error; + } + } + + private async discardHeldChunks(page: Page): Promise { + try { + await page.evaluate(() => { + const bridge = (globalThis as GhosttyBrowserGlobal).__agentTty; + bridge?.discardHeldChunks?.(); + }); + } catch (error) { + // A page that fails here is as broken as the call that failed first. + this.logger.debug( + 'Dropping the held chunks of a failed flush failed', + error instanceof Error ? error.message : String(error), ); - await afterEachBatch?.(); } } @@ -1792,6 +1878,7 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { page: Page, dataChunks: string[], endsFlush = true, + paintedCap: number | null = null, ): Promise { invariant( dataChunks.length > 0, @@ -1805,17 +1892,26 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { assertString(chunk, 'bridge batch write chunk must be a string'); } - await page.evaluate( - async ([chunks, batchEndsFlush]: readonly [string[], boolean]) => { + const noteError = await page.evaluate( + async ([chunks, batchEndsFlush, batchPaintedCap]: readonly [ + string[], + boolean, + number | null, + ]) => { const bridge = (globalThis as GhosttyBrowserGlobal).__agentTty; if (bridge === undefined || typeof bridge.writeBatch !== 'function') { throw new Error('ghostty-web bridge writeBatch() is unavailable'); } - await bridge.writeBatch(chunks, batchEndsFlush); + return await bridge.writeBatch(chunks, batchEndsFlush, batchPaintedCap); }, - [dataChunks, endsFlush] as const, + [dataChunks, endsFlush, paintedCap] as const, ); + // The page notes the painted graphemes in this call and reports a failed + // note by its message, so that the note cannot fail the flush. + if (typeof noteError === 'string') { + this.failPaintedNote(noteError); + } } private async writeBridge(page: Page, data: string): Promise { diff --git a/src/renderer/ghosttyWeb/embeddedHarnessHtml.ts b/src/renderer/ghosttyWeb/embeddedHarnessHtml.ts index 84299ee5..666368d2 100644 --- a/src/renderer/ghosttyWeb/embeddedHarnessHtml.ts +++ b/src/renderer/ghosttyWeb/embeddedHarnessHtml.ts @@ -572,6 +572,29 @@ export const EMBEDDED_HARNESS_HTML = ` return grid; } + // Adds the graphemes on the screen to the set that the missing-glyph + // check of a recorded replay probes. The cap bounds the set. + function notePaintedGraphemes(cap) { + assertPositiveInteger( + cap, + 'notePaintedGraphemes() cap must be a positive integer', + ); + window.__agentTtyPainted ??= { seen: new Set(), complete: true }; + const painted = window.__agentTtyPainted; + for (const row of getCellGraphemes()) { + for (const grapheme of row) { + if (grapheme === '' || painted.seen.has(grapheme)) { + continue; + } + if (painted.seen.size < cap) { + painted.seen.add(grapheme); + } else { + painted.complete = false; + } + } + } + } + function getSnapshotPayload(options) { invariant( options === undefined || @@ -657,7 +680,7 @@ export const EMBEDDED_HARNESS_HTML = ` }); updateDocumentState(); }, - async writeBatch(chunks, endsFlush = true) { + async writeBatch(chunks, endsFlush = true, paintedCap = null) { const terminal = getReadyTerminal(); invariant( Array.isArray(chunks) && chunks.length > 0, @@ -667,6 +690,10 @@ export const EMBEDDED_HARNESS_HTML = ` typeof endsFlush === 'boolean', 'writeBatch() endsFlush must be a boolean', ); + invariant( + paintedCap === null || Number.isInteger(paintedCap), + 'writeBatch() paintedCap must be null or an integer', + ); for (const chunk of chunks) { assertStringValue(chunk, 'writeBatch() chunks must be strings'); } @@ -683,7 +710,7 @@ export const EMBEDDED_HARNESS_HTML = ` state.heldChunks.push(chunk); } if (!endsFlush) { - return; + return null; } const flush = state.heldChunks; @@ -696,6 +723,29 @@ export const EMBEDDED_HARNESS_HTML = ` terminal.write(flush[lastIndex], resolve); }); updateDocumentState(); + + // The caller asks to note the painted graphemes of a recorded + // replay here, so a flush costs one page call and not two. A failed + // note must not fail the flush: the call returns its message, and + // the caller then drops the missing-glyph fields of the export. + if (paintedCap === null) { + return null; + } + try { + notePaintedGraphemes(paintedCap); + return null; + } catch (error) { + return error instanceof Error ? error.message : String(error); + } + }, + // A flush that failed in the middle leaves its earlier calls' chunks + // held. The caller drops them, so that a retry starts from the screen + // as the failed flush found it. + discardHeldChunks() { + state.heldChunks = []; + }, + notePaintedGraphemes(cap) { + notePaintedGraphemes(cap); }, getSnapshot(options) { return getSnapshotPayload(options); diff --git a/test/integration/renderer-backend.test.ts b/test/integration/renderer-backend.test.ts index 6d077211..c181af2d 100644 --- a/test/integration/renderer-backend.test.ts +++ b/test/integration/renderer-backend.test.ts @@ -939,6 +939,122 @@ describe('GhosttyWebBackend integration', { timeout: 120_000 }, () => { expect(await visibleFirstRow()).toBe('abcdef'); }); + // HostRendererManager keeps the backend after a failed replay and retries + // from the same seq. The page held the chunks of the calls before the one + // that failed, so a retry that started with a resize was refused for good. + it('runs a retry that starts with a resize after a page call failed in the middle of a flush', async () => { + await backend.boot(); + // One character per event, so a chunk that is lost, repeated or moved + // changes the text on the screen. 2501 characters fill 32 rows of 80 + // columns. + const characters = Array.from({ length: 2_501 }, (_, index) => + 'abcdefghijklmnopqrstuvwxyz0123456789'.charAt((index * 7) % 36), + ); + const outputs = (firstSeq: number): ReplayInput['events'] => + characters.map((character, index) => ({ + seq: firstSeq + index, + ts: timestampFor(0), + type: 'output' as const, + payload: { data: character }, + })); + const writeBatchBridge = ( + backend as unknown as { + writeBatchBridge: (...args: unknown[]) => Promise; + } + ).writeBatchBridge.bind(backend); + let pageCalls = 0; + Object.assign(backend as object, { + writeBatchBridge: async (...args: unknown[]) => { + pageCalls += 1; + if (pageCalls === 2) { + throw new Error('page call failed'); + } + await writeBatchBridge(...args); + }, + }); + + await expect( + backend.replayTo(createReplayInput(outputs(0))), + ).rejects.toThrow('page call failed'); + + Object.assign(backend as object, { writeBatchBridge }); + await backend.replayTo( + createReplayInput( + [ + { + seq: 0, + ts: timestampFor(0), + type: 'resize', + payload: { cols: 80, rows: 40 }, + }, + ...outputs(1), + ], + { initialRows: 24 }, + ), + ); + + const expectedRows = Array.from({ length: 32 }, (_, row) => + characters.slice(row * 80, (row + 1) * 80).join(''), + ); + const screenRows = (await backend.snapshot()).visibleLines.map( + (line) => line.text, + ); + expect(screenRows.slice(0, 32)).toEqual(expectedRows); + }); + + // A flush of 1001 chunks takes two page calls, and the glyph that it paints + // comes with its last chunk. The next flush erases it. + it('records the glyphs that a flush painted, in the page call that ends the flush', async () => { + await backend.boot(); + const { page } = backend as unknown as { page: Page }; + const pageFunctions: string[] = []; + const evaluate = page.evaluate.bind(page) as ( + ...args: unknown[] + ) => Promise; + Object.assign(page, { + evaluate: (pageFunction: unknown, ...rest: unknown[]) => { + pageFunctions.push(String(pageFunction)); + return evaluate(pageFunction, ...rest); + }, + }); + + await backend.replayWithTiming( + createReplayInput([ + ...Array.from({ length: 1_000 }, (_, seq) => ({ + seq, + ts: timestampFor(0), + type: 'output' as const, + payload: { data: 'x' }, + })), + { + seq: 1_000, + ts: timestampFor(0), + type: 'output' as const, + payload: { data: '\u0378' }, + }, + { + seq: 1_001, + ts: timestampFor(1), + type: 'output' as const, + payload: { data: '\u001b[2J\u001b[Hok' }, + }, + ]), + { mode: 'max-speed', minFrameHoldMs: 16, finalFrameHoldMs: 200 }, + ); + + await expect(backend.recordedMissingGlyphs()).resolves.toEqual({ + missingGlyphs: [{ codePoint: 'U+0378' }], + }); + // 1001 chunks take 2 calls, the next flush takes 1. None of them has a + // call of its own for the note. + expect( + pageFunctions.filter((source) => source.includes('writeBatch')), + ).toHaveLength(3); + expect( + pageFunctions.filter((source) => source.includes('notePaintedGraphemes')), + ).toHaveLength(0); + }); + it('allows dispose to be called more than once', async () => { await backend.boot(); diff --git a/test/unit/renderer/ghosttyWebBackend.test.ts b/test/unit/renderer/ghosttyWebBackend.test.ts index 1593fe0d..5a12a0dc 100644 --- a/test/unit/renderer/ghosttyWebBackend.test.ts +++ b/test/unit/renderer/ghosttyWebBackend.test.ts @@ -16,6 +16,7 @@ import { afterEach, describe, expect, it, vi } from 'vitest'; import type { ReplayTimingOptions } from '../../../src/renderer/backend.js'; import type { ReplayInput } from '../../../src/renderer/types.js'; import { BUNDLED_FONT_ASSETS } from '../../../src/renderer/bundledFont.js'; +import { MAX_CACHED_GLYPH_PROBES } from '../../../src/renderer/ghosttyWeb/missingGlyphs.js'; import { hashProfile, resolveProfile } from '../../../src/renderer/profiles.js'; import { GhosttyWebBackend } from '../../../src/renderer/ghosttyWeb/index.js'; import { Logger } from '../../../src/util/logger.js'; @@ -88,7 +89,7 @@ describe('GhosttyWebBackend unit guards', () => { expect(recordedChunks).toEqual(chunks); }); - it('notes painted graphemes after each bridge batch of a split flush', async () => { + it('runs the after-batch hook after each bridge call of a split flush', async () => { const backend = createBackend(); const calls: string[] = []; ( @@ -113,29 +114,111 @@ describe('GhosttyWebBackend unit guards', () => { ) => Promise; } ).flushOutputBatch({}, chunks, () => { - calls.push('note'); + calls.push('hook'); return Promise.resolve(); }); expect(calls).toEqual([ 'write 1000', - 'note', + 'hook', 'write 1000', - 'note', + 'hook', 'write 501', - 'note', + 'hook', ]); }); - it('notes painted graphemes after each output flush of a timed replay', async () => { + // The page holds the chunks of the calls before the one that failed. + // HostRendererManager keeps the backend after a failed replay, so a retry + // would meet them. + describe.each([ + [ + 'a page call', + (calls: number) => (calls === 2 ? new Error('page call failed') : null), + undefined, + ], + [ + 'the hook between two calls', + () => null, + () => Promise.reject(new Error('page call failed')), + ], + ])('when %s of a flush fails', (_name, failure, afterEachBatch) => { + function failFlush(discard: () => unknown): { + rejection: Promise; + evaluate: ReturnType; + } { + const backend = createBackend(); + vi.stubGlobal('__agentTty', { discardHeldChunks: discard }); + const evaluate = vi.fn((pageFunction: () => unknown) => + Promise.resolve(pageFunction()), + ); + let calls = 0; + ( + backend as unknown as { + writeBatchBridge: () => Promise; + } + ).writeBatchBridge = () => { + calls += 1; + const error = failure(calls); + return error === null ? Promise.resolve() : Promise.reject(error); + }; + const rejection = ( + backend as unknown as { + flushOutputBatch: ( + page: object, + dataChunks: string[], + afterEachBatch?: () => Promise, + ) => Promise; + } + ).flushOutputBatch( + { evaluate }, + Array.from({ length: 2_501 }, () => 'x'), + afterEachBatch, + ); + return { rejection, evaluate }; + } + + afterEach(() => { + vi.unstubAllGlobals(); + }); + + it('drops the held chunks and rejects with the original error', async () => { + const discard = vi.fn(); + const { rejection } = failFlush(discard); + + await expect(rejection).rejects.toThrow('page call failed'); + expect(discard).toHaveBeenCalledTimes(1); + }); + + it('rejects with the original error when the page cannot drop the chunks either', async () => { + const { rejection, evaluate } = failFlush(() => { + throw new Error('page crashed'); + }); + + await expect(rejection).rejects.toThrow('page call failed'); + expect(evaluate).toHaveBeenCalledTimes(1); + }); + }); + + // A separate page call costs a round trip behind a frame, 5 to 60 ms + // depending on the terminal size, so the page notes the graphemes inside the + // call that ends the flush. + it('notes painted graphemes in the page call that ends each output flush of a timed replay', async () => { const backend = createBackend(); const calls: string[] = []; Object.assign(backend as object, { isBooted: true, page: { isClosed: () => false }, resizeBridge: () => Promise.resolve(), - writeBatchBridge: (_page: object, dataChunks: string[]) => { - calls.push(`write ${dataChunks.join(',')}`); + writeBatchBridge: ( + _page: object, + dataChunks: string[], + endsFlush: boolean, + paintedCap: number | null, + ) => { + calls.push( + `write ${dataChunks.join(',')} ${String(endsFlush)} ${String(paintedCap)}`, + ); return Promise.resolve(); }, notePaintedGraphemes: () => { @@ -184,7 +267,10 @@ describe('GhosttyWebBackend unit guards', () => { { mode: 'max-speed', minFrameHoldMs: 0, finalFrameHoldMs: 0 }, ); - expect(calls).toEqual(['write first', 'note', 'write second', 'note']); + expect(calls).toEqual([ + `write first true ${String(MAX_CACHED_GLYPH_PROBES)}`, + `write second true ${String(MAX_CACHED_GLYPH_PROBES)}`, + ]); }); it('retries the painted-grapheme note after the backend reboots', async () => { @@ -436,20 +522,27 @@ describe('GhosttyWebBackend unit guards', () => { write: ReturnType; writeBatchSizes: number[]; frameWaits: boolean[]; + paintedCaps: (number | null)[]; writtenChunks: string[]; }) => Promise, ): Promise { const writeBatchSizes: number[] = []; const frameWaits: boolean[] = []; + const paintedCaps: (number | null)[] = []; const writtenChunks: string[] = []; const write = vi.fn(() => Promise.resolve()); vi.stubGlobal('__agentTty', { write, - writeBatch: (chunks: string[], waitForFrame: boolean) => { + writeBatch: ( + chunks: string[], + waitForFrame: boolean, + paintedCap: number | null, + ) => { writeBatchSizes.push(chunks.length); frameWaits.push(waitForFrame); + paintedCaps.push(paintedCap); writtenChunks.push(...chunks); - return Promise.resolve(); + return Promise.resolve(null); }, }); const page = { @@ -465,6 +558,7 @@ describe('GhosttyWebBackend unit guards', () => { write, writeBatchSizes, frameWaits, + paintedCaps, writtenChunks, }); } finally { @@ -511,11 +605,15 @@ describe('GhosttyWebBackend unit guards', () => { ); }); + // Only a recorded replay probes glyphs, and it notes the painted graphemes + // in the call that ends the flush: no call of its own, and none in a call + // that only holds chunks. it.each([ [ 'replayTo', (backend: GhosttyWebBackend, input: ReplayInput) => backend.replayTo(input), + [null, null, null], ], [ 'replayWithTiming', @@ -525,20 +623,27 @@ describe('GhosttyWebBackend unit guards', () => { minFrameHoldMs: 0, finalFrameHoldMs: 0, }), + [null, null, MAX_CACHED_GLYPH_PROBES], ], ])( '%s calls the page once per 1000 output events, and ends the flush with the last call', - async (_name, replay) => { + async (_name, replay, expectedPaintedCaps) => { const backend = createBackend(); const input = outputReplayInput(2_501); await withCountingBridge( - async ({ page, write, writeBatchSizes, frameWaits, writtenChunks }) => { + async ({ + page, + write, + writeBatchSizes, + frameWaits, + paintedCaps, + writtenChunks, + }) => { Object.assign(backend as object, { isBooted: true, page, resizeBridge: () => Promise.resolve(), - notePaintedGraphemes: () => Promise.resolve(), checkPaintedGlyphs: () => Promise.resolve({}), readHarnessSnapshot: () => Promise.resolve(createHarnessSnapshotPayload({ cols: 80 })), @@ -549,6 +654,7 @@ describe('GhosttyWebBackend unit guards', () => { expect(page.evaluate).toHaveBeenCalledTimes(3); expect(writeBatchSizes).toEqual([1000, 1000, 501]); expect(frameWaits).toEqual([false, false, true]); + expect(paintedCaps).toEqual(expectedPaintedCaps); expect(writtenChunks).toEqual( input.events.map((event) => event.type === 'output' ? event.payload.data : '', @@ -1313,17 +1419,38 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { }; } + function inputEvent( + seq: number, + offsetMs: number, + ): ReplayInput['events'][number] { + return { + seq, + ts: new Date(BASE_TIMESTAMP_MS + offsetMs).toISOString(), + type: 'input_text', + payload: { data: 'x' }, + }; + } + async function replayAndRecordFlushes( events: ReplayInput['events'], timing: ReplayTimingOptions, flushCostMs = 0, onFlush: () => void = () => {}, + // Seq that an earlier replay applied: the backend starts after it. + appliedSeq = -1, ): Promise<{ atMs: number; chunks: string[] }[]> { vi.useFakeTimers(); const backend = createBackend(); const flushes: { atMs: number; chunks: string[] }[] = []; let replayStartedAtMs = 0; + if (appliedSeq >= 0) { + Object.assign(backend as object, { + lastAppliedSeq: appliedSeq, + initialReplayCols: 80, + initialReplayRows: 24, + }); + } Object.assign(backend as object, { isBooted: true, page: { isClosed: () => false }, @@ -1505,8 +1632,8 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { }); // The frame wait of a flush is the frame of its moment. A hold after every - // flush of a run would make each 30 ms step cost 44 ms to export. - it('holds max-speed frames only after the last flush of a run of output', async () => { + // flush would make each 30 ms step cost 44 ms to export. + it('holds max-speed frames only after the last flush of the replay', async () => { const flushes = await replayAndRecordFlushes( [ outputEvent(0, 0), @@ -1518,16 +1645,88 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { MAX_SPEED, ); - // out0 and out1 flush when the next output arrives, so no hold follows - // them. The exit ends the run: out2 flushes, and its hold delays out4. + // Every flush starts when the next event arrives, so no hold follows one. expect(flushes).toEqual([ { atMs: FIRST_EVENT_AT_MS, chunks: ['out0'] }, { atMs: FIRST_EVENT_AT_MS, chunks: ['out1'] }, { atMs: FIRST_EVENT_AT_MS, chunks: ['out2'] }, - { atMs: FIRST_EVENT_AT_MS + 16, chunks: ['out4'] }, + { atMs: FIRST_EVENT_AT_MS, chunks: ['out4'] }, ]); }); + // A flush at every input, marker or exit event would cost one frame for each + // of them, and a recording of typing has an input for every output. + it('does not end a max-speed flush at an event that changes no pixel', async () => { + const flushes = await replayAndRecordFlushes( + [ + outputEvent(0, 0), + inputEvent(1, 5), + outputEvent(2, 10), + exitEvent(3, 500), + ], + MAX_SPEED, + ); + + expect(flushes.map((flush) => flush.chunks)).toEqual([['out0', 'out2']]); + }); + + it('still ends a recorded flush at an event that changes no pixel', async () => { + const flushes = await replayAndRecordFlushes( + [ + outputEvent(0, 0), + inputEvent(1, 5), + outputEvent(2, 10), + exitEvent(3, 500), + ], + RECORDED, + ); + + expect(flushes.map((flush) => flush.chunks)).toEqual([['out0'], ['out2']]); + }); + + // 10,000 outputs with an input between each pair, 40 ms apart: 400 s of + // recording. Without a budget, every output is a moment of its own and the + // replay makes 10,000 flushes of a frame each, more than 5 minutes. + it('spreads max-speed flushes over a long recording, one per 133 ms of recorded time', async () => { + const events: ReplayInput['events'] = []; + for (let index = 0; index < 10_000; index += 1) { + events.push(outputEvent(2 * index, 40 * index)); + events.push(inputEvent(2 * index + 1, 40 * index + 20)); + } + events.push(exitEvent(20_000, 400_000)); + const flushes = await replayAndRecordFlushes(events, MAX_SPEED); + + // A flush ends at the first event 133 ms after the first output of the + // batch, so each one takes 4 outputs (160 ms): about 2,500 flushes. + expect(flushes.length).toBeGreaterThan(2_000); + expect(flushes.length).toBeLessThanOrEqual(3_001); + // No output is lost or moved. + expect(flushes.flatMap((flush) => flush.chunks)).toEqual( + events + .filter((event) => event.type === 'output') + .map((event) => `out${String(event.seq)}`), + ); + }); + + it('measures the frame budget over the events that the replay applies', async () => { + // 20 ms apart: the whole log spans 200 s, but the last 10 events span 180 ms. + const events = Array.from({ length: 10_000 }, (_, index) => + outputEvent(index, index * 20), + ); + const flushes = await replayAndRecordFlushes( + events, + MAX_SPEED, + 0, + () => {}, + 9_989, + ); + + // 20 ms apart is above the frame hold of 16 ms: a moment each. + expect(flushes.map((flush) => flush.chunks)).toEqual( + events.slice(9_990).map((event) => [`out${String(event.seq)}`]), + ); + }); + it('does not merge max-speed moments when a flush costs more than a frame hold', async () => { const events = Array.from({ length: 101 }, (_, index) => outputEvent(index, index * 10), From 0663f72651feb3720ce15bdc307ead7b1bc89e6a Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 13:55:51 +0000 Subject: [PATCH 07/26] test: pin a failed painted-grapheme note --- test/integration/renderer-backend.test.ts | 18 ++++++++++ test/unit/renderer/ghosttyWebBackend.test.ts | 38 ++++++++++++++++++++ 2 files changed, 56 insertions(+) diff --git a/test/integration/renderer-backend.test.ts b/test/integration/renderer-backend.test.ts index c181af2d..ccff5f78 100644 --- a/test/integration/renderer-backend.test.ts +++ b/test/integration/renderer-backend.test.ts @@ -1055,6 +1055,24 @@ describe('GhosttyWebBackend integration', { timeout: 120_000 }, () => { ).toHaveLength(0); }); + it('writes a flush even when its painted-grapheme note fails, and reports the failure', async () => { + await backend.boot(); + const { page } = backend as unknown as { page: Page }; + await page.evaluate('window.__agentTty.resize(80, 24)'); + + // A cap of 0 makes the note throw after the chunks are on the screen. + const noteError = await page.evaluate( + `window.__agentTty.writeBatch(['ab', 'cd'], true, 0)`, + ); + + expect(noteError).toEqual( + expect.stringContaining('cap must be a positive integer'), + ); + expect( + await page.evaluate(`window.__agentTty.getVisibleText().split('\\n')[0]`), + ).toBe('abcd'); + }); + it('allows dispose to be called more than once', async () => { await backend.boot(); diff --git a/test/unit/renderer/ghosttyWebBackend.test.ts b/test/unit/renderer/ghosttyWebBackend.test.ts index 5a12a0dc..9284f9a0 100644 --- a/test/unit/renderer/ghosttyWebBackend.test.ts +++ b/test/unit/renderer/ghosttyWebBackend.test.ts @@ -273,6 +273,44 @@ describe('GhosttyWebBackend unit guards', () => { ]); }); + it('omits the record export glyph fields, and asks for no more notes, once the page reports a failed note', async () => { + const logger = new Logger('debug'); + const debugSpy = vi.spyOn(logger, 'debug').mockImplementation(() => {}); + const backend = new GhosttyWebBackend( + 'renderer-unit-session', + PROFILE, + undefined, + logger, + ); + const evaluate = vi.fn(() => Promise.resolve('grapheme read failed')); + const page = { isClosed: () => false, evaluate }; + Object.assign(backend as object, { isBooted: true, page }); + const flush = (cap: number | null): Promise => + ( + backend as unknown as { + writeBatchBridge: ( + page: object, + dataChunks: string[], + endsFlush: boolean, + paintedCap: number | null, + ) => Promise; + } + ).writeBatchBridge(page, ['x'], true, cap); + + await expect(flush(MAX_CACHED_GLYPH_PROBES)).resolves.toBeUndefined(); + + expect(debugSpy).toHaveBeenCalledWith( + 'Missing-glyph note failed; the record export result omits missingGlyphs', + 'grapheme read failed', + ); + await expect(backend.recordedMissingGlyphs()).resolves.toEqual({}); + expect( + ( + backend as unknown as { paintedNoteCap: () => number | null } + ).paintedNoteCap(), + ).toBeNull(); + }); + it('retries the painted-grapheme note after the backend reboots', async () => { const backend = createBackend(); const graphemes = vi.fn(() => Promise.resolve({ missingGlyphs: [] })); From e1a727818bc1a6ff7fe8470e8e668dcecb5f8c40 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 14:06:57 +0000 Subject: [PATCH 08/26] docs: describe the max-speed frame budget and the in-call painted-grapheme note --- docs/USAGE.md | 2 +- src/renderer/ghosttyWeb/README.md | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/USAGE.md b/docs/USAGE.md index bf4722b4..d7b60d91 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -294,7 +294,7 @@ agent-tty record export --format webm --json | jq -c '.result.missi # [{"codePoint":"U+0378"}] ``` -WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. A frame of the video shows the screen after all the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event. `max-speed` removes the idle time but gives each 16 ms of recorded output its own frame, so a progress bar that updates every 30 ms shows its steps and not only its last one. The video has 25 frames per second, so it can still skip a state that lasts less than one frame (40 ms). +WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. A frame of the video shows the screen after all the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event. `max-speed` removes the idle time but gives each 16 ms of recorded output its own frame, so a progress bar that updates every 30 ms shows its steps and not only its last one. It makes at most about 3,000 frames, so a recording longer than 48 s gets one frame per 1/3,000 of its recorded time instead (a 1-hour recording: one per 1.2 s), and its export time depends on its length and not on its number of events. Input, marker and exit events do not end a `max-speed` frame by themselves. The video has 25 frames per second, so it can still skip a state that lasts less than one frame (40 ms). SVG export renders styled grid frames from the event log through the native `libghostty-vt` backend with no browser and no ffmpeg, so it requires the optional `@coder/libghostty-vt-node` package (there is no `ghostty-web` fallback). The output is deterministic: exporting the same session twice produces byte-identical, diffable SVG. `--format svg` writes a still image of the final screen and ignores event timing, so it also works on a log whose clock stepped backward (`--animate` refuses such a log and names the event where the clock stepped back); add `--animate` for an animated SVG of de-duplicated frames replayed with recorded event-log timing (`--timing` is not supported with SVG). diff --git a/src/renderer/ghosttyWeb/README.md b/src/renderer/ghosttyWeb/README.md index f98617e6..9100806b 100644 --- a/src/renderer/ghosttyWeb/README.md +++ b/src/renderer/ghosttyWeb/README.md @@ -20,7 +20,7 @@ This directory holds the `ghostty-web` reference renderer. `GhosttyWebBackend` s - `stripTrailingAsciiSpaces(text)` and `assembleCanonicalLine(cols, readColumn)` build canonical line text [harnessDecoding.ts:assembleCanonicalLine]. - Assertion helpers `assertNonNegativeInteger`, `assertPositiveInteger`, `assertPositiveNumber` and `assertHexColor`, plus `normalizeError(error, prefix)` [harnessDecoding.ts:normalizeError]. - `loadHarnessHtml()` returns `EMBEDDED_HARNESS_HTML` [harnessDecoding.ts:loadHarnessHtml]. -- `EMBEDDED_HARNESS_HTML` is the harness page as a template string. In the browser it defines `window.__agentTty` with `write`, `writeBatch`, `getSnapshot`, `getCellGraphemes`, `getVisibleText`, `isReady`, `resize` and `setCursorVisible` [embeddedHarnessHtml.ts:EMBEDDED_HARNESS_HTML]. +- `EMBEDDED_HARNESS_HTML` is the harness page as a template string. In the browser it defines `window.__agentTty` with `write`, `writeBatch`, `discardHeldChunks`, `notePaintedGraphemes`, `getSnapshot`, `getCellGraphemes`, `getVisibleText`, `isReady`, `resize` and `setCursorVisible` [embeddedHarnessHtml.ts:EMBEDDED_HARNESS_HTML]. - `harness.html` is a standalone copy of the harness page. No code serves it, and it differs from `EMBEDDED_HARNESS_HTML` [harnessDecoding.ts:loadHarnessHtml]. - `assets/` (not documented separately): holds the bundled fonts `JetBrainsMono-Regular-latin.woff2`, `SymbolsNerdFontMono-Regular.ttf`, `NotoSansSymbols2-Regular.woff2` and `NotoSansMath-Regular.woff2`, plus `FONT-LICENSE.txt` and the OFL texts of the Noto fonts. `scripts/subset-symbol-fonts.sh` rebuilds the two Noto subsets. `src/renderer/bundledFont.ts` loads them, and `scripts/copy-renderer-assets.mjs` copies them into `dist/` during `npm run build`. @@ -29,8 +29,8 @@ This directory holds the `ghostty-web` reference renderer. `GhosttyWebBackend` s - The constructor asserts a non-empty `sessionId`, `profile.name` and `profile.fontFamily`, a positive `fontSize`, and hex colors. With `videoOptions`, `outputDir` must be absolute and the size must use positive integers [backend.ts:GhosttyWebBackend.constructor]. - Replay cannot rewind: `targetSeq` must be >= the last applied seq. The initial dimensions cannot change after the first replay. `input.sessionId` must match the backend [backend.ts:GhosttyWebBackend.replayTo]. - `replayWithTiming` asserts strictly increasing seqs and non-decreasing timestamps [backend.ts:GhosttyWebBackend.replayWithTiming]. -- `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` has no schedule to fall behind, so it flushes once the next event is at least one frame hold of recorded time after the batch's first output. Each moment that a viewer could tell apart gets a frame, and a replay costs at most one frame per output event. Events with the same timestamp share a flush when the hold is above 0 [backend.ts:GhosttyWebBackend.replayWithTiming]. -- `writeBatch(chunks, endsFlush)` takes the chunks of one bridge call. A flush that has more than 1000 chunks needs several calls, and ghostty-web paints on every animation frame, so the page holds the chunks of a call that does not end the flush. The call that ends it (`endsFlush`, set by the backend on the last call only) writes every held chunk in one task, so no frame paints a state between two calls, and then resolves after one frame. ghostty-web calls a write callback one animation frame (about 17 ms) after the write, and waiting on every chunk (`write`) cost one frame per output event. `write()` and `resize()` throw while chunks are held. In `replayWithTiming`, the chunks of one flush share one video frame [embeddedHarnessHtml.ts:EMBEDDED_HARNESS_HTML]. +- `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` has no schedule to fall behind, so it flushes once the next event is at least one frame hold of recorded time after the batch's first output. It also spreads at most `MAX_SPEED_FRAME_BUDGET` (3,000) flushes over the recorded span of the events that it applies: the minimum distance between two flushes is the larger of the frame hold and the span divided by 3,000. So a `max-speed` replay costs about one frame per moment for a recording shorter than 48 s and at most about 3,000 frames for a longer one, whatever its number of events. Input, marker, run-complete, signal and exit events do not end a `max-speed` flush, because a flush at each of them would cost a frame per event (a resize still does, to keep the order). Events with the same timestamp share a flush at any hold, 0 included, because a flush needs at least 1 ms of recorded time [backend.ts:GhosttyWebBackend.replayWithTiming]. +- `writeBatch(chunks, endsFlush, paintedCap)` takes the chunks of one bridge call. A flush that has more than 1000 chunks needs several calls, and ghostty-web paints on every animation frame, so the page holds the chunks of a call that does not end the flush. The call that ends it (`endsFlush`, set by the backend on the last call only) writes every held chunk in one task, so no frame paints a state between two calls, and then resolves after one frame. ghostty-web calls a write callback one animation frame (about 17 ms) after the write, and waiting on every chunk (`write`) cost one frame per output event. `write()` and `resize()` throw while chunks are held. `discardHeldChunks()` drops them: the backend calls it when a flush fails, because `HostRendererManager` keeps the backend and a retry would otherwise meet them. A `paintedCap` that is not null makes the call that ends the flush add the graphemes on the screen to the set that the missing-glyph check of `replayWithTiming` probes, so a flush is one page call. The call returns the message of a failed note and not an error, because a failed note must not fail the flush. In `replayWithTiming`, the chunks of one flush share one video frame [embeddedHarnessHtml.ts:EMBEDDED_HARNESS_HTML]. - `snapshot()` and `screenshot()` throw until `replayTo()` or `replayWithTiming()` has run once and set the initial size. Both replays accept `targetSeq` -1 for an empty Event Log. A replay over zero events sets `lastAppliedSeq` to `targetSeq` [backend.ts:GhosttyWebBackend.snapshot]. - `missingGlyphs` lists a code point only when a bundled face that lacks it drew it. Chromium draws a cluster with one font: the first stack font that covers all of it, or else the first font of the stack, which is bundled in both built-in profiles and then draws `.notdef` boxes. CDP names the face that drew a probe by its PostScript name, so a family alias or several faces under one family resolve to the right cmap. When one bundled face drew the whole grapheme, every code point that face lacks is listed. A probe with no reported font is judged against every face of the first family. When the grapheme is split over several fonts, each uncovered code point is judged by its own probe, and a mark's probe on its base is compared with a probe of the base alone. Glyphs that a host font draws are not listed. The check runs after the PNG is written, removes its probe spans, and caches each probe text's result for the page, clearing the cache when it would pass `MAX_CACHED_GLYPH_PROBES` (16384) entries. It probes at most `MAX_GLYPH_PROBES` texts and lists at most `MAX_MISSING_GLYPH_CELLS` cells (4096 each). `probeUsedFonts` sends its CDP queries in batches of 256 and stops starting batches after `GLYPH_PROBE_BUDGET_MS` (1000 ms). The result sets `missingGlyphsTruncated` when any of these stops the check early. Family names match without regard to case. A web font whose PostScript name the profile does not bundle fails the check. A failed check is logged at debug level and leaves the field out, so the screenshot still succeeds. `replayWithTiming()` records the viewport graphemes in the page after each output batch and each resize (at most `MAX_CACHED_GLYPH_PROBES`), and the final frame hold judges them once, with the hold as the probe budget, so the check does not lengthen the video. `recordedMissingGlyphs()` returns that result. A failed note or check leaves the record export fields out. A profile whose font stack does not start with a bundled font gets no `missingGlyphs` field [missingGlyphs.ts:probeUsedFonts] [backend.ts:GhosttyWebBackend.screenshot]. - `screenshot()` requires an absolute `outputPath` whose parent directory exists. It checks that the output is a non-empty PNG with a valid signature and an `IHDR` first chunk [backend.ts:GhosttyWebBackend.screenshot]. From 028a1ab32db4bb00ae2b51c4115351160c1bd4fd Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 15:30:24 +0000 Subject: [PATCH 09/26] perf: size the max-speed frame budget from the measured flush cost --- docs/USAGE.md | 2 +- src/renderer/ghosttyWeb/README.md | 2 +- src/renderer/ghosttyWeb/backend.ts | 13 +++++++------ test/unit/renderer/ghosttyWebBackend.test.ts | 10 +++++----- 4 files changed, 14 insertions(+), 13 deletions(-) diff --git a/docs/USAGE.md b/docs/USAGE.md index d7b60d91..000e381f 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -294,7 +294,7 @@ agent-tty record export --format webm --json | jq -c '.result.missi # [{"codePoint":"U+0378"}] ``` -WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. A frame of the video shows the screen after all the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event. `max-speed` removes the idle time but gives each 16 ms of recorded output its own frame, so a progress bar that updates every 30 ms shows its steps and not only its last one. It makes at most about 3,000 frames, so a recording longer than 48 s gets one frame per 1/3,000 of its recorded time instead (a 1-hour recording: one per 1.2 s), and its export time depends on its length and not on its number of events. Input, marker and exit events do not end a `max-speed` frame by themselves. The video has 25 frames per second, so it can still skip a state that lasts less than one frame (40 ms). +WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. A frame of the video shows the screen after all the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event. `max-speed` removes the idle time but gives each 16 ms of recorded output its own frame, so a progress bar that updates every 30 ms shows its steps and not only its last one. It makes at most about 2,400 frames, so a recording longer than 38 s gets one frame per 1/2,400 of its recorded time instead (a 1-hour recording: one per 1.5 s), and its export time depends on its length and not on its number of events. Input, marker and exit events do not end a `max-speed` frame by themselves. The video has 25 frames per second, so it can still skip a state that lasts less than one frame (40 ms). SVG export renders styled grid frames from the event log through the native `libghostty-vt` backend with no browser and no ffmpeg, so it requires the optional `@coder/libghostty-vt-node` package (there is no `ghostty-web` fallback). The output is deterministic: exporting the same session twice produces byte-identical, diffable SVG. `--format svg` writes a still image of the final screen and ignores event timing, so it also works on a log whose clock stepped backward (`--animate` refuses such a log and names the event where the clock stepped back); add `--animate` for an animated SVG of de-duplicated frames replayed with recorded event-log timing (`--timing` is not supported with SVG). diff --git a/src/renderer/ghosttyWeb/README.md b/src/renderer/ghosttyWeb/README.md index 9100806b..dc5a5ee9 100644 --- a/src/renderer/ghosttyWeb/README.md +++ b/src/renderer/ghosttyWeb/README.md @@ -29,7 +29,7 @@ This directory holds the `ghostty-web` reference renderer. `GhosttyWebBackend` s - The constructor asserts a non-empty `sessionId`, `profile.name` and `profile.fontFamily`, a positive `fontSize`, and hex colors. With `videoOptions`, `outputDir` must be absolute and the size must use positive integers [backend.ts:GhosttyWebBackend.constructor]. - Replay cannot rewind: `targetSeq` must be >= the last applied seq. The initial dimensions cannot change after the first replay. `input.sessionId` must match the backend [backend.ts:GhosttyWebBackend.replayTo]. - `replayWithTiming` asserts strictly increasing seqs and non-decreasing timestamps [backend.ts:GhosttyWebBackend.replayWithTiming]. -- `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` has no schedule to fall behind, so it flushes once the next event is at least one frame hold of recorded time after the batch's first output. It also spreads at most `MAX_SPEED_FRAME_BUDGET` (3,000) flushes over the recorded span of the events that it applies: the minimum distance between two flushes is the larger of the frame hold and the span divided by 3,000. So a `max-speed` replay costs about one frame per moment for a recording shorter than 48 s and at most about 3,000 frames for a longer one, whatever its number of events. Input, marker, run-complete, signal and exit events do not end a `max-speed` flush, because a flush at each of them would cost a frame per event (a resize still does, to keep the order). Events with the same timestamp share a flush at any hold, 0 included, because a flush needs at least 1 ms of recorded time [backend.ts:GhosttyWebBackend.replayWithTiming]. +- `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` has no schedule to fall behind, so it flushes once the next event is at least one frame hold of recorded time after the batch's first output. It also spreads at most `MAX_SPEED_FRAME_BUDGET` (2,400) flushes over the recorded span of the events that it applies: the minimum distance between two flushes is the larger of the frame hold and the span divided by 2,400. So a `max-speed` replay costs about one frame per moment for a recording shorter than 38 s and at most about 2,400 frames for a longer one, whatever its number of events. Input, marker, run-complete, signal and exit events do not end a `max-speed` flush, because a flush at each of them would cost a frame per event (a resize still does, to keep the order). Events with the same timestamp share a flush at any hold, 0 included, because a flush needs at least 1 ms of recorded time [backend.ts:GhosttyWebBackend.replayWithTiming]. - `writeBatch(chunks, endsFlush, paintedCap)` takes the chunks of one bridge call. A flush that has more than 1000 chunks needs several calls, and ghostty-web paints on every animation frame, so the page holds the chunks of a call that does not end the flush. The call that ends it (`endsFlush`, set by the backend on the last call only) writes every held chunk in one task, so no frame paints a state between two calls, and then resolves after one frame. ghostty-web calls a write callback one animation frame (about 17 ms) after the write, and waiting on every chunk (`write`) cost one frame per output event. `write()` and `resize()` throw while chunks are held. `discardHeldChunks()` drops them: the backend calls it when a flush fails, because `HostRendererManager` keeps the backend and a retry would otherwise meet them. A `paintedCap` that is not null makes the call that ends the flush add the graphemes on the screen to the set that the missing-glyph check of `replayWithTiming` probes, so a flush is one page call. The call returns the message of a failed note and not an error, because a failed note must not fail the flush. In `replayWithTiming`, the chunks of one flush share one video frame [embeddedHarnessHtml.ts:EMBEDDED_HARNESS_HTML]. - `snapshot()` and `screenshot()` throw until `replayTo()` or `replayWithTiming()` has run once and set the initial size. Both replays accept `targetSeq` -1 for an empty Event Log. A replay over zero events sets `lastAppliedSeq` to `targetSeq` [backend.ts:GhosttyWebBackend.snapshot]. - `missingGlyphs` lists a code point only when a bundled face that lacks it drew it. Chromium draws a cluster with one font: the first stack font that covers all of it, or else the first font of the stack, which is bundled in both built-in profiles and then draws `.notdef` boxes. CDP names the face that drew a probe by its PostScript name, so a family alias or several faces under one family resolve to the right cmap. When one bundled face drew the whole grapheme, every code point that face lacks is listed. A probe with no reported font is judged against every face of the first family. When the grapheme is split over several fonts, each uncovered code point is judged by its own probe, and a mark's probe on its base is compared with a probe of the base alone. Glyphs that a host font draws are not listed. The check runs after the PNG is written, removes its probe spans, and caches each probe text's result for the page, clearing the cache when it would pass `MAX_CACHED_GLYPH_PROBES` (16384) entries. It probes at most `MAX_GLYPH_PROBES` texts and lists at most `MAX_MISSING_GLYPH_CELLS` cells (4096 each). `probeUsedFonts` sends its CDP queries in batches of 256 and stops starting batches after `GLYPH_PROBE_BUDGET_MS` (1000 ms). The result sets `missingGlyphsTruncated` when any of these stops the check early. Family names match without regard to case. A web font whose PostScript name the profile does not bundle fails the check. A failed check is logged at debug level and leaves the field out, so the screenshot still succeeds. `replayWithTiming()` records the viewport graphemes in the page after each output batch and each resize (at most `MAX_CACHED_GLYPH_PROBES`), and the final frame hold judges them once, with the hold as the probe budget, so the check does not lengthen the video. `recordedMissingGlyphs()` returns that result. A failed note or check leaves the record export fields out. A profile whose font stack does not start with a bundled font gets no `missingGlyphs` field [missingGlyphs.ts:probeUsedFonts] [backend.ts:GhosttyWebBackend.screenshot]. diff --git a/src/renderer/ghosttyWeb/backend.ts b/src/renderer/ghosttyWeb/backend.ts index 724fffdd..d6e16465 100644 --- a/src/renderer/ghosttyWeb/backend.ts +++ b/src/renderer/ghosttyWeb/backend.ts @@ -139,13 +139,14 @@ const WASM_CONTENT_TYPE = 'application/wasm'; const MAX_REPLAY_BATCH_SIZE = 1000; /** * A max-speed replay flushes at most about this many times, however long the - * recording is. Every flush waits for one animation frame (17 to 25 ms) and - * one page call, so a replay takes about 2 minutes at the limit. That stays - * below the 5 minutes that a WebM export allows its replay, even on a slow - * host. The video has 25 frames per second, so 3,000 flushes fill 2 minutes - * of video: a viewer cannot tell more states apart. + * recording is. A flush costs one animation frame, one page call and one scan + * of the screen: 26 ms at 80x24, 33 ms at 120x40 and 97 ms at 200x60 in the + * median of the sparse-log runs of PR 358 (host load 60 to 90). The limit then + * takes 1 to 4 minutes, under the 5 minutes that a WebM export allows its + * replay. The video has 25 frames per second, so a viewer cannot tell more + * than this many states apart in a video of that length. */ -const MAX_SPEED_FRAME_BUDGET = 3_000; +const MAX_SPEED_FRAME_BUDGET = 2_400; const RAF_TIMEOUT_MS = 5_000; let servedAssetsPromise: Promise< diff --git a/test/unit/renderer/ghosttyWebBackend.test.ts b/test/unit/renderer/ghosttyWebBackend.test.ts index 9284f9a0..98174f15 100644 --- a/test/unit/renderer/ghosttyWebBackend.test.ts +++ b/test/unit/renderer/ghosttyWebBackend.test.ts @@ -1725,7 +1725,7 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { // 10,000 outputs with an input between each pair, 40 ms apart: 400 s of // recording. Without a budget, every output is a moment of its own and the // replay makes 10,000 flushes of a frame each, more than 5 minutes. - it('spreads max-speed flushes over a long recording, one per 133 ms of recorded time', async () => { + it('spreads max-speed flushes over a long recording, one per 167 ms of recorded time', async () => { const events: ReplayInput['events'] = []; for (let index = 0; index < 10_000; index += 1) { events.push(outputEvent(2 * index, 40 * index)); @@ -1734,10 +1734,10 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { events.push(exitEvent(20_000, 400_000)); const flushes = await replayAndRecordFlushes(events, MAX_SPEED); - // A flush ends at the first event 133 ms after the first output of the - // batch, so each one takes 4 outputs (160 ms): about 2,500 flushes. - expect(flushes.length).toBeGreaterThan(2_000); - expect(flushes.length).toBeLessThanOrEqual(3_001); + // A flush ends at the first event 167 ms after the first output of the + // batch, so each one takes 5 outputs (200 ms): about 2,000 flushes. + expect(flushes.length).toBeGreaterThan(1_800); + expect(flushes.length).toBeLessThanOrEqual(2_401); // No output is lost or moved. expect(flushes.flatMap((flush) => flush.chunks)).toEqual( events From dfde02d942cbb7059c249a166a31f762aa18f282 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 15:55:38 +0000 Subject: [PATCH 10/26] docs: update the max-speed frame budget comment with the measured flush costs --- src/renderer/ghosttyWeb/backend.ts | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/src/renderer/ghosttyWeb/backend.ts b/src/renderer/ghosttyWeb/backend.ts index d6e16465..70eaea1f 100644 --- a/src/renderer/ghosttyWeb/backend.ts +++ b/src/renderer/ghosttyWeb/backend.ts @@ -140,11 +140,11 @@ const MAX_REPLAY_BATCH_SIZE = 1000; /** * A max-speed replay flushes at most about this many times, however long the * recording is. A flush costs one animation frame, one page call and one scan - * of the screen: 26 ms at 80x24, 33 ms at 120x40 and 97 ms at 200x60 in the - * median of the sparse-log runs of PR 358 (host load 60 to 90). The limit then - * takes 1 to 4 minutes, under the 5 minutes that a WebM export allows its - * replay. The video has 25 frames per second, so a viewer cannot tell more - * than this many states apart in a video of that length. + * of the screen: 26 ms at 80x24, 33 ms at 120x40 and 112 ms at 200x60 in the + * median of five sparse-log runs (host load 40 to 100). The limit then takes + * 1 to 4.5 minutes, under the 5 minutes that a WebM export allows its replay. + * The video has 25 frames per second, so a viewer cannot tell more than this + * many states apart in a video of that length. */ const MAX_SPEED_FRAME_BUDGET = 2_400; const RAF_TIMEOUT_MS = 5_000; From e64b021674e7be3cdc5c73b0f0d991dcdf513f56 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 21:16:11 +0000 Subject: [PATCH 11/26] test: pin the max-speed frame budget, resize hold and note, painted-set cap, and fix the max-speed docs --- docs/USAGE.md | 2 +- src/export/README.md | 2 +- src/renderer/ghosttyWeb/README.md | 2 +- src/renderer/ghosttyWeb/backend.ts | 24 +-- test/integration/renderer-backend.test.ts | 93 ++++++++ test/unit/renderer/ghosttyWebBackend.test.ts | 214 ++++++++++++++++--- 6 files changed, 292 insertions(+), 45 deletions(-) diff --git a/docs/USAGE.md b/docs/USAGE.md index 000e381f..15a41f49 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -294,7 +294,7 @@ agent-tty record export --format webm --json | jq -c '.result.missi # [{"codePoint":"U+0378"}] ``` -WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. A frame of the video shows the screen after all the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event. `max-speed` removes the idle time but gives each 16 ms of recorded output its own frame, so a progress bar that updates every 30 ms shows its steps and not only its last one. It makes at most about 2,400 frames, so a recording longer than 38 s gets one frame per 1/2,400 of its recorded time instead (a 1-hour recording: one per 1.5 s), and its export time depends on its length and not on its number of events. Input, marker and exit events do not end a `max-speed` frame by themselves. The video has 25 frames per second, so it can still skip a state that lasts less than one frame (40 ms). +WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. A frame of the video shows the screen after the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event, and a state that the next output of the frame replaces is not shown. In `recorded` and `accelerated` mode, an event that changes no pixel (an input, a marker, a run-complete, a signal or the exit) also ends a frame. In `max-speed` mode it does not: only recorded time ends a frame, so output on both sides of an input event that is less than 16 ms apart shares one frame. `max-speed` removes the idle time. It makes a frame at most once per 16 ms of recorded time, so a progress bar that updates every 30 ms shows its steps and not only its last one. It makes at most about 2,400 frames, so a recording longer than 38 s gets one frame per 1/2,400 of its recorded time instead (a 1-hour recording: one per 1.5 s), and its export time depends on its length and not on its number of events. `max-speed` waits for one animation frame after each frame but holds no further time between two frames, so two frames that follow each other can end up in one video frame. The video has 25 frames per second, so it can skip a state that lasts less than one video frame (40 ms). SVG export renders styled grid frames from the event log through the native `libghostty-vt` backend with no browser and no ffmpeg, so it requires the optional `@coder/libghostty-vt-node` package (there is no `ghostty-web` fallback). The output is deterministic: exporting the same session twice produces byte-identical, diffable SVG. `--format svg` writes a still image of the final screen and ignores event timing, so it also works on a log whose clock stepped backward (`--animate` refuses such a log and names the event where the clock stepped back); add `--animate` for an animated SVG of de-duplicated frames replayed with recorded event-log timing (`--timing` is not supported with SVG). diff --git a/src/export/README.md b/src/export/README.md index 876ffdfa..7733b406 100644 --- a/src/export/README.md +++ b/src/export/README.md @@ -25,7 +25,7 @@ This directory converts a session's event log into recording exports. `generateA - Defaults: timing mode `recorded` and profile `reference-dark` [webm.ts:generateWebmExport]. - A requested `libghostty-vt` renderer is replaced with `ghostty-web`. A code comment says `libghostty-vt` is semantic-only [webm.ts:generateWebmExport]. - The default backend factory accepts only `ghostty-web` [webm.ts:createDefaultBackend]. -- Timing presets: `accelerated` caps gaps at 400 ms and holds frames at least 100 ms. `max-speed` holds frames at least 16 ms. Each mode holds the final frame (1000 ms, or 500 ms for `max-speed`) [webm.ts:buildReplayTimingOptions]. +- Timing presets: `accelerated` caps gaps at 400 ms and holds frames at least 100 ms. `max-speed` uses a frame hold of 16 ms, but holds it only after the last flush of the replay and before a resize (see the `ghostty-web` README). Each mode holds the final frame (1000 ms, or 500 ms for `max-speed`) [webm.ts:buildReplayTimingOptions]. - The timed replay fails after `REPLAY_TIMEOUT_MS` (5 minutes) [webm.ts:REPLAY_TIMEOUT_MS]. - `renderGridFramesToSvg` needs the profile font size `SVG_FONT_SIZE` (14) and draws fixed 8.4x18 px cells, so equal input gives byte-identical output [svg.ts:renderGridFramesToSvg]. It always embeds the JetBrains Mono subset, and embeds each other bundled face only when a glyph needs it [svg.ts:embeddedFontAssets]. Output past `defaultMaxSvgOutputLength` of the V8 heap limit (the heap limit divided by 8 bytes, at most `MAX_SVG_OUTPUT_LENGTH`) fails with `EXPORT_ERROR` [svg.ts:defaultMaxSvgOutputLength]. diff --git a/src/renderer/ghosttyWeb/README.md b/src/renderer/ghosttyWeb/README.md index dc5a5ee9..18b39f20 100644 --- a/src/renderer/ghosttyWeb/README.md +++ b/src/renderer/ghosttyWeb/README.md @@ -29,7 +29,7 @@ This directory holds the `ghostty-web` reference renderer. `GhosttyWebBackend` s - The constructor asserts a non-empty `sessionId`, `profile.name` and `profile.fontFamily`, a positive `fontSize`, and hex colors. With `videoOptions`, `outputDir` must be absolute and the size must use positive integers [backend.ts:GhosttyWebBackend.constructor]. - Replay cannot rewind: `targetSeq` must be >= the last applied seq. The initial dimensions cannot change after the first replay. `input.sessionId` must match the backend [backend.ts:GhosttyWebBackend.replayTo]. - `replayWithTiming` asserts strictly increasing seqs and non-decreasing timestamps [backend.ts:GhosttyWebBackend.replayWithTiming]. -- `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` has no schedule to fall behind, so it flushes once the next event is at least one frame hold of recorded time after the batch's first output. It also spreads at most `MAX_SPEED_FRAME_BUDGET` (2,400) flushes over the recorded span of the events that it applies: the minimum distance between two flushes is the larger of the frame hold and the span divided by 2,400. So a `max-speed` replay costs about one frame per moment for a recording shorter than 38 s and at most about 2,400 frames for a longer one, whatever its number of events. Input, marker, run-complete, signal and exit events do not end a `max-speed` flush, because a flush at each of them would cost a frame per event (a resize still does, to keep the order). Events with the same timestamp share a flush at any hold, 0 included, because a flush needs at least 1 ms of recorded time [backend.ts:GhosttyWebBackend.replayWithTiming]. +- `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` has no schedule to fall behind, so only recorded time ends its flushes. A flush ends at the first event that is at least one frame hold of recorded time after the batch's first output. An event that changes no pixel does not end it: input, marker, run-complete, signal and exit events leave the batch as it is, because a flush at each of them would cost a frame per event. A resize and the end of the replay still end it. `max-speed` also spreads at most `MAX_SPEED_FRAME_BUDGET` (2,400) flushes over the recorded span of the events that it applies: the distance between the first output of a flush and the event that ends it is at least the larger of the frame hold and the span divided by 2,400. So a `max-speed` replay makes about one flush per frame hold of recorded time for a recording shorter than 38 s and at most about 2,400 flushes for a longer one, whatever its number of events. Output that arrives less than that distance after the first output of a flush shares the flush, also when an input or a marker sits between, so a state that the next output of the flush replaces is never painted. A flush waits for one animation frame. `max-speed` holds the frame hold only after the last flush of the replay and before a resize, so two flushes that follow each other can paint within one video frame. In `recorded` and `accelerated` mode, every event that changes no pixel ends a flush, and a hold follows every flush. Events with the same timestamp share a flush at any hold, 0 included, because a flush needs at least 1 ms of recorded time [backend.ts:GhosttyWebBackend.replayWithTiming]. - `writeBatch(chunks, endsFlush, paintedCap)` takes the chunks of one bridge call. A flush that has more than 1000 chunks needs several calls, and ghostty-web paints on every animation frame, so the page holds the chunks of a call that does not end the flush. The call that ends it (`endsFlush`, set by the backend on the last call only) writes every held chunk in one task, so no frame paints a state between two calls, and then resolves after one frame. ghostty-web calls a write callback one animation frame (about 17 ms) after the write, and waiting on every chunk (`write`) cost one frame per output event. `write()` and `resize()` throw while chunks are held. `discardHeldChunks()` drops them: the backend calls it when a flush fails, because `HostRendererManager` keeps the backend and a retry would otherwise meet them. A `paintedCap` that is not null makes the call that ends the flush add the graphemes on the screen to the set that the missing-glyph check of `replayWithTiming` probes, so a flush is one page call. The call returns the message of a failed note and not an error, because a failed note must not fail the flush. In `replayWithTiming`, the chunks of one flush share one video frame [embeddedHarnessHtml.ts:EMBEDDED_HARNESS_HTML]. - `snapshot()` and `screenshot()` throw until `replayTo()` or `replayWithTiming()` has run once and set the initial size. Both replays accept `targetSeq` -1 for an empty Event Log. A replay over zero events sets `lastAppliedSeq` to `targetSeq` [backend.ts:GhosttyWebBackend.snapshot]. - `missingGlyphs` lists a code point only when a bundled face that lacks it drew it. Chromium draws a cluster with one font: the first stack font that covers all of it, or else the first font of the stack, which is bundled in both built-in profiles and then draws `.notdef` boxes. CDP names the face that drew a probe by its PostScript name, so a family alias or several faces under one family resolve to the right cmap. When one bundled face drew the whole grapheme, every code point that face lacks is listed. A probe with no reported font is judged against every face of the first family. When the grapheme is split over several fonts, each uncovered code point is judged by its own probe, and a mark's probe on its base is compared with a probe of the base alone. Glyphs that a host font draws are not listed. The check runs after the PNG is written, removes its probe spans, and caches each probe text's result for the page, clearing the cache when it would pass `MAX_CACHED_GLYPH_PROBES` (16384) entries. It probes at most `MAX_GLYPH_PROBES` texts and lists at most `MAX_MISSING_GLYPH_CELLS` cells (4096 each). `probeUsedFonts` sends its CDP queries in batches of 256 and stops starting batches after `GLYPH_PROBE_BUDGET_MS` (1000 ms). The result sets `missingGlyphsTruncated` when any of these stops the check early. Family names match without regard to case. A web font whose PostScript name the profile does not bundle fails the check. A failed check is logged at debug level and leaves the field out, so the screenshot still succeeds. `replayWithTiming()` records the viewport graphemes in the page after each output batch and each resize (at most `MAX_CACHED_GLYPH_PROBES`), and the final frame hold judges them once, with the hold as the probe budget, so the check does not lengthen the video. `recordedMissingGlyphs()` returns that result. A failed note or check leaves the record export fields out. A profile whose font stack does not start with a bundled font gets no `missingGlyphs` field [missingGlyphs.ts:probeUsedFonts] [backend.ts:GhosttyWebBackend.screenshot]. diff --git a/src/renderer/ghosttyWeb/backend.ts b/src/renderer/ghosttyWeb/backend.ts index 70eaea1f..eef422e8 100644 --- a/src/renderer/ghosttyWeb/backend.ts +++ b/src/renderer/ghosttyWeb/backend.ts @@ -138,13 +138,12 @@ const WASM_CONTENT_TYPE = 'application/wasm'; const MAX_REPLAY_BATCH_SIZE = 1000; /** - * A max-speed replay flushes at most about this many times, however long the + * A max-speed replay makes at most about this many flushes, however long the * recording is. A flush costs one animation frame, one page call and one scan * of the screen: 26 ms at 80x24, 33 ms at 120x40 and 112 ms at 200x60 in the * median of five sparse-log runs (host load 40 to 100). The limit then takes * 1 to 4.5 minutes, under the 5 minutes that a WebM export allows its replay. - * The video has 25 frames per second, so a viewer cannot tell more than this - * many states apart in a video of that length. + * The video has 25 frames per second, so this many flushes fill 96 s of video. */ const MAX_SPEED_FRAME_BUDGET = 2_400; const RAF_TIMEOUT_MS = 5_000; @@ -688,7 +687,7 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { } // Output shows once the next event is this far after the batch's first - // output. It is at least 1 ms: events with one timestamp are one moment. + // output. It is at least 1 ms: events with one timestamp share a flush. // Max-speed has no schedule that bounds its replay time, so it also // spreads a fixed number of flushes over the recorded span: a sparse // recording then costs the same number of frames as a dense one. @@ -725,8 +724,9 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { // Events that change no pixel end a flush in recorded and accelerated // mode, where the pause before them is part of the video. Max-speed has no - // pauses: a flush at each of them would cost one frame per event, and a - // viewer cannot tell apart what happens within one minimum split. + // pauses: a flush at each of them would cost one frame per event. So only + // recorded time ends a max-speed flush, and output on both sides of such + // an event shares a flush when it is less than one minimum split apart. const flushBeforeNonVisualEvent = async (): Promise => { if (timing.mode !== 'max-speed') { await flushOutputBatch(); @@ -767,12 +767,12 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { // Output shows once its batch is a minimum split old, unless the // replay runs behind the schedule: then batches grow until it catches // up, so slow flushes lower the frame rate instead of stretching the - // video. Max-speed has no schedule to fall behind. It flushes every - // minimum split of recorded time, so each moment that a viewer could - // tell apart gets a frame, within the frame budget. The frame wait of - // such a flush is already the frame of its moment, so max-speed holds - // only after a flush that ends the replay or comes before a resize: a - // hold after every flush would export 30 ms steps slower than they ran. + // video. Max-speed has no schedule to fall behind. It flushes once per + // minimum split of recorded time at most, within the frame budget. + // The frame wait of such a flush is already its frame, so max-speed + // holds only after a flush that ends the replay or comes before a + // resize: a hold after every flush would export 30 ms steps slower + // than they ran. if ( pendingSinceTimestampMs !== null && eventTimestampMs - pendingSinceTimestampMs >= minSplitMs && diff --git a/test/integration/renderer-backend.test.ts b/test/integration/renderer-backend.test.ts index ccff5f78..f36a82f9 100644 --- a/test/integration/renderer-backend.test.ts +++ b/test/integration/renderer-backend.test.ts @@ -1073,6 +1073,99 @@ describe('GhosttyWebBackend integration', { timeout: 120_000 }, () => { ).toBe('abcd'); }); + // A resize can bring graphemes on the screen that no flush showed: growing + // the screen pulls lines back from the scrollback. The note that follows the + // resize records them, and the next flush then erases the screen. + it('records a glyph that a resize brings back from the scrollback', async () => { + await backend.boot(); + + await backend.replayWithTiming( + createReplayInput([ + { + seq: 0, + ts: timestampFor(0), + type: 'output', + payload: { + data: `\u0378\r\n${Array.from({ length: 30 }, (_, line) => `line ${String(line)}\r\n`).join('')}`, + }, + }, + { + seq: 1, + ts: timestampFor(1), + type: 'resize', + payload: { cols: 80, rows: 40 }, + }, + { + seq: 2, + ts: timestampFor(2), + type: 'output', + payload: { data: '\u001b[2J\u001b[Hok' }, + }, + ]), + { mode: 'max-speed', minFrameHoldMs: 16, finalFrameHoldMs: 200 }, + ); + + await expect(backend.recordedMissingGlyphs()).resolves.toEqual({ + missingGlyphs: [{ codePoint: 'U+0378' }], + }); + }); + + // The page keeps the size attributes of the body (data-cols, data-rows) in + // step with the terminal after every write. The call that ends a flush does + // it too, as write() does. + it('refreshes the size attributes of the document when a flush ends', async () => { + await backend.boot(); + const { page } = backend as unknown as { page: Page }; + await page.evaluate('window.__agentTty.resize(80, 24)'); + const sizeAttributes = (): Promise => + page.evaluate( + `document.body.dataset.cols + 'x' + document.body.dataset.rows`, + ) as Promise; + expect(await sizeAttributes()).toBe('80x24'); + await page.evaluate( + `document.body.dataset.cols = 'stale'; document.body.dataset.rows = 'stale'`, + ); + + await page.evaluate(`window.__agentTty.writeBatch(['ab'], false)`); + expect(await sizeAttributes()).toBe('stalexstale'); + await page.evaluate(`window.__agentTty.writeBatch(['cd'], true)`); + + expect(await sizeAttributes()).toBe('80x24'); + }); + + // The set of painted graphemes feeds a probe of every grapheme in it. The + // cap that the backend passes bounds the set, and a set that hit the cap is + // reported as incomplete, so the export result says that its list is cut. + it('bounds the painted graphemes by the cap of the call that ends a flush', async () => { + await backend.boot(); + const { page } = backend as unknown as { page: Page }; + await page.evaluate('window.__agentTty.resize(80, 24)'); + const painted = (): Promise<{ seen: string[]; complete: boolean } | null> => + page.evaluate( + `(() => { + const state = window.__agentTtyPainted; + return state === undefined ? null : { seen: [...state.seen], complete: state.complete }; + })()`, + ) as Promise<{ seen: string[]; complete: boolean } | null>; + + // Nothing is noted while the call does not ask for it. + await page.evaluate(`window.__agentTty.writeBatch(['abc'], true, null)`); + expect(await painted()).toBeNull(); + + await page.evaluate( + `window.__agentTty.writeBatch(['\\x1b[2J\\x1b[Habc'], true, 5)`, + ); + expect(await painted()).toEqual({ seen: ['a', 'b', 'c'], complete: true }); + + await page.evaluate( + `window.__agentTty.writeBatch(['\\x1b[2J\\x1b[Hdefgh'], true, 5)`, + ); + expect(await painted()).toEqual({ + seen: ['a', 'b', 'c', 'd', 'e'], + complete: false, + }); + }); + it('allows dispose to be called more than once', async () => { await backend.boot(); diff --git a/test/unit/renderer/ghosttyWebBackend.test.ts b/test/unit/renderer/ghosttyWebBackend.test.ts index 98174f15..9b37cdb5 100644 --- a/test/unit/renderer/ghosttyWebBackend.test.ts +++ b/test/unit/renderer/ghosttyWebBackend.test.ts @@ -1532,6 +1532,82 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { })); } + function resizeEvent( + seq: number, + offsetMs: number, + ): ReplayInput['events'][number] { + return { + seq, + ts: new Date(BASE_TIMESTAMP_MS + offsetMs).toISOString(), + type: 'resize', + payload: { cols: 100, rows: 30 }, + }; + } + + // The steps of a replay with the time of each, counted from the resize that + // sets the initial size: `flush` is a write of pending output, `resize` and + // `note` follow a resize event, `snapshot` is the read of the final state. + async function replayAndRecordTimeline( + events: ReplayInput['events'], + timing: ReplayTimingOptions, + ): Promise<{ step: string; atMs: number }[]> { + vi.useFakeTimers(); + const backend = createBackend(); + const timeline: { step: string; atMs: number }[] = []; + let replayStartedAtMs: number | undefined; + const record = (step: string): void => { + timeline.push({ + step, + atMs: performance.now() - (replayStartedAtMs ?? performance.now()), + }); + }; + + Object.assign(backend as object, { + isBooted: true, + page: { isClosed: () => false }, + resizeBridge: (_page: object, cols: number) => { + if (replayStartedAtMs === undefined) { + replayStartedAtMs = performance.now(); + } else { + record(`resize ${String(cols)}`); + } + return Promise.resolve(); + }, + flushOutputBatch: (_page: object, dataChunks: string[]) => { + record(`flush ${dataChunks.join(',')}`); + return Promise.resolve(); + }, + notePaintedGraphemes: () => { + record('note'); + return Promise.resolve(); + }, + readHarnessSnapshot: () => { + record('snapshot'); + return Promise.resolve( + createHarnessSnapshotPayload({ + cols: 80, + rows: 24, + visibleLines: [], + }), + ); + }, + }); + + const replay = backend.replayWithTiming( + { + sessionId: 'renderer-unit-session', + initialCols: 80, + initialRows: 24, + targetSeq: Math.max(...events.map((event) => event.seq)), + events, + }, + timing, + ); + await vi.runAllTimersAsync(); + await replay; + return timeline; + } + it('shows each output at its recorded time when the outputs are far apart', async () => { const flushes = await replayAndRecordFlushes( [ @@ -1629,7 +1705,7 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { ]); }); - it('flushes max-speed output every frame hold of recorded time, so each moment gets a frame', async () => { + it('ends a max-speed flush at the first event that is a frame hold after the first output of the batch', async () => { const flushes = await replayAndRecordFlushes( [ outputEvent(0, 0), @@ -1651,7 +1727,7 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { ]); }); - it('keeps one max-speed flush per moment when the outputs are far apart', async () => { + it('keeps one max-speed flush per output when the outputs are far apart', async () => { const flushes = await replayAndRecordFlushes( [ outputEvent(0, 0), @@ -1669,7 +1745,7 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { ]); }); - // The frame wait of a flush is the frame of its moment. A hold after every + // The frame wait of a flush is its frame. A hold after every // flush would make each 30 ms step cost 44 ms to export. it('holds max-speed frames only after the last flush of the replay', async () => { const flushes = await replayAndRecordFlushes( @@ -1692,6 +1768,72 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { ]); }); + // The page paints a flush on its next animation frame. The hold gives that + // paint time to reach the video before the resize changes the canvas. + it('holds one frame hold between the last flush and a resize', async () => { + const timeline = await replayAndRecordTimeline( + [ + outputEvent(0, 0), + resizeEvent(1, 5), + outputEvent(2, 10), + exitEvent(3, 12), + ], + { mode: 'max-speed', minFrameHoldMs: 16, finalFrameHoldMs: 0 }, + ); + + expect(timeline.map((entry) => entry.step)).toEqual([ + 'flush out0', + 'resize 100', + 'note', + 'flush out2', + 'snapshot', + ]); + const atMs = (step: string): number | undefined => + timeline.find((entry) => entry.step === step)?.atMs; + expect((atMs('resize 100') ?? 0) - (atMs('flush out0') ?? 0)).toBe(16); + }); + + it.each([ + ['recorded', RECORDED], + ['max-speed', MAX_SPEED], + ])( + 'notes the painted graphemes right after a resize event, in %s mode', + async (_name, timing) => { + const timeline = await replayAndRecordTimeline( + [ + outputEvent(0, 0), + resizeEvent(1, 200), + outputEvent(2, 400), + exitEvent(3, 600), + ], + timing, + ); + + const steps = timeline.map((entry) => entry.step); + const resizeAt = steps.indexOf('resize 100'); + expect(resizeAt).toBeGreaterThan(0); + expect(steps[resizeAt + 1]).toBe('note'); + expect(timeline[resizeAt + 1]?.atMs).toBe(timeline[resizeAt]?.atMs); + }, + ); + + it('holds one frame hold after a resize, and one after the last flush of the replay', async () => { + const timeline = await replayAndRecordTimeline( + [ + outputEvent(0, 0), + resizeEvent(1, 5), + outputEvent(2, 10), + exitEvent(3, 12), + ], + { mode: 'max-speed', minFrameHoldMs: 16, finalFrameHoldMs: 0 }, + ); + + const atMs = (step: string): number => + timeline.find((entry) => entry.step === step)?.atMs ?? Number.NaN; + expect(atMs('flush out2') - atMs('note')).toBe(16); + expect(atMs('snapshot') - atMs('flush out2')).toBe(16); + }); + // A flush at every input, marker or exit event would cost one frame for each // of them, and a recording of typing has an input for every output. it('does not end a max-speed flush at an event that changes no pixel', async () => { @@ -1722,29 +1864,41 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { expect(flushes.map((flush) => flush.chunks)).toEqual([['out0'], ['out2']]); }); - // 10,000 outputs with an input between each pair, 40 ms apart: 400 s of - // recording. Without a budget, every output is a moment of its own and the - // replay makes 10,000 flushes of a frame each, more than 5 minutes. - it('spreads max-speed flushes over a long recording, one per 167 ms of recorded time', async () => { - const events: ReplayInput['events'] = []; - for (let index = 0; index < 10_000; index += 1) { - events.push(outputEvent(2 * index, 40 * index)); - events.push(inputEvent(2 * index + 1, 40 * index + 20)); - } - events.push(exitEvent(20_000, 400_000)); - const flushes = await replayAndRecordFlushes(events, MAX_SPEED); - - // A flush ends at the first event 167 ms after the first output of the - // batch, so each one takes 5 outputs (200 ms): about 2,000 flushes. - expect(flushes.length).toBeGreaterThan(1_800); - expect(flushes.length).toBeLessThanOrEqual(2_401); - // No output is lost or moved. - expect(flushes.flatMap((flush) => flush.chunks)).toEqual( - events - .filter((event) => event.type === 'output') - .map((event) => `out${String(event.seq)}`), - ); - }); + // 9,000 outputs over 180 s or 360 s. Without a budget, every output is a + // flush of its own and the replay makes 9,000 flushes of a frame each. With + // a budget of 2,400 flushes, a flush ends at the first event that is 75 ms or + // 150 ms (span / 2,400) after the first output of its batch. That is the + // fourth output in both cases: 2,250 flushes. A budget of 2,000 flushes gives + // 1,800, and one of 3,000 flushes gives 3,000. + it.each([ + { label: 'outputs 20 ms apart', outputGapMs: 20, withInputs: false }, + { + label: 'outputs 40 ms apart with an input between', + outputGapMs: 40, + withInputs: true, + }, + ])( + 'ends a max-speed flush every 1/2,400 of the recorded span: 9,000 $label make 2,250 flushes', + async ({ outputGapMs, withInputs }) => { + const events: ReplayInput['events'] = []; + for (let index = 0; index < 9_000; index += 1) { + events.push(outputEvent(events.length, outputGapMs * index)); + if (withInputs) { + events.push(inputEvent(events.length, outputGapMs * index + 20)); + } + } + events.push(exitEvent(events.length, 180_000 * (withInputs ? 2 : 1))); + const flushes = await replayAndRecordFlushes(events, MAX_SPEED); + + expect(flushes.length).toBe(2_250); + // No output is lost or moved. + expect(flushes.flatMap((flush) => flush.chunks)).toEqual( + events + .filter((event) => event.type === 'output') + .map((event) => `out${String(event.seq)}`), + ); + }, + ); it('measures the frame budget over the events that the replay applies', async () => { // 20 ms apart: the whole log spans 200 s, but the last 10 events span 180 ms. @@ -1759,13 +1913,13 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { 9_989, ); - // 20 ms apart is above the frame hold of 16 ms: a moment each. + // 20 ms apart is above the frame hold of 16 ms: a flush each. expect(flushes.map((flush) => flush.chunks)).toEqual( events.slice(9_990).map((event) => [`out${String(event.seq)}`]), ); }); - it('does not merge max-speed moments when a flush costs more than a frame hold', async () => { + it('does not merge max-speed flushes when a flush costs more than a frame hold', async () => { const events = Array.from({ length: 101 }, (_, index) => outputEvent(index, index * 10), ); @@ -1773,8 +1927,8 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { const flushes = await replayAndRecordFlushes(events, MAX_SPEED, 30); // Recorded timing lets batches grow while the replay is behind its - // schedule. Max-speed has no schedule, so a slow flush must not hide a - // moment: every flush holds the outputs of one frame hold (two events). + // schedule. Max-speed has no schedule, so a slow flush must not merge two + // flushes: every flush holds the outputs of one frame hold (two events). expect(flushes.length).toBe(51); expect(flushes.every((flush) => flush.chunks.length <= 2)).toBe(true); expect(flushes.flatMap((flush) => flush.chunks)).toEqual( From 3da5a5785b8aa736919f003c1d5ccc91c4183b6e Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 22:23:43 +0000 Subject: [PATCH 12/26] docs: say that max-speed shows most steps of a 30 ms progress bar --- docs/USAGE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/USAGE.md b/docs/USAGE.md index 15a41f49..5cbc9d55 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -294,7 +294,7 @@ agent-tty record export --format webm --json | jq -c '.result.missi # [{"codePoint":"U+0378"}] ``` -WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. A frame of the video shows the screen after the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event, and a state that the next output of the frame replaces is not shown. In `recorded` and `accelerated` mode, an event that changes no pixel (an input, a marker, a run-complete, a signal or the exit) also ends a frame. In `max-speed` mode it does not: only recorded time ends a frame, so output on both sides of an input event that is less than 16 ms apart shares one frame. `max-speed` removes the idle time. It makes a frame at most once per 16 ms of recorded time, so a progress bar that updates every 30 ms shows its steps and not only its last one. It makes at most about 2,400 frames, so a recording longer than 38 s gets one frame per 1/2,400 of its recorded time instead (a 1-hour recording: one per 1.5 s), and its export time depends on its length and not on its number of events. `max-speed` waits for one animation frame after each frame but holds no further time between two frames, so two frames that follow each other can end up in one video frame. The video has 25 frames per second, so it can skip a state that lasts less than one video frame (40 ms). +WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. A frame of the video shows the screen after the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event, and a state that the next output of the frame replaces is not shown. In `recorded` and `accelerated` mode, an event that changes no pixel (an input, a marker, a run-complete, a signal or the exit) also ends a frame. In `max-speed` mode it does not: only recorded time ends a frame, so output on both sides of an input event that is less than 16 ms apart shares one frame. `max-speed` removes the idle time. It makes a frame at most once per 16 ms of recorded time, so a progress bar that updates every 30 ms shows most of its steps and not only its last one. It makes at most about 2,400 frames, so a recording longer than 38 s gets one frame per 1/2,400 of its recorded time instead (a 1-hour recording: one per 1.5 s), and its export time depends on its length and not on its number of events. `max-speed` waits for one animation frame after each frame but holds no further time between two frames, so two frames that follow each other can end up in one video frame. The video has 25 frames per second, so it can skip a state that lasts less than one video frame (40 ms). SVG export renders styled grid frames from the event log through the native `libghostty-vt` backend with no browser and no ffmpeg, so it requires the optional `@coder/libghostty-vt-node` package (there is no `ghostty-web` fallback). The output is deterministic: exporting the same session twice produces byte-identical, diffable SVG. `--format svg` writes a still image of the final screen and ignores event timing, so it also works on a log whose clock stepped backward (`--animate` refuses such a log and names the event where the clock stepped back); add `--animate` for an animated SVG of de-duplicated frames replayed with recorded event-log timing (`--timing` is not supported with SVG). From bc68da9ef2749c39dccae7f8194d9647144dfe87 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Fri, 6 Nov 2026 23:27:52 +0000 Subject: [PATCH 13/26] test: wait for a quiet first screen before counting screencast frames of a replay --- test/integration/renderer-backend.test.ts | 26 +++++++++++++++++------ 1 file changed, 20 insertions(+), 6 deletions(-) diff --git a/test/integration/renderer-backend.test.ts b/test/integration/renderer-backend.test.ts index f36a82f9..8a1ee507 100644 --- a/test/integration/renderer-backend.test.ts +++ b/test/integration/renderer-backend.test.ts @@ -894,21 +894,35 @@ describe('GhosttyWebBackend integration', { timeout: 120_000 }, () => { await new Promise((resolve) => setTimeout(resolve, 100)); }, }); - // Wait for the first frame, so that it is the screen before the replay. + // Wait until the screen before the replay is quiet: a slow host can + // deliver a late paint of the first screen, which is not a frame of the + // replay. The screen before the replay counts as one frame. + let framesSeen = 0; + let quietSinceMs = Date.now(); await expect - .poll(() => frames.length, { timeout: 10_000 }) - .toBeGreaterThan(0); + .poll( + () => { + if (frames.length !== framesSeen) { + framesSeen = frames.length; + quietSinceMs = Date.now(); + } + return framesSeen > 0 && Date.now() - quietSinceMs >= 300; + }, + { timeout: 10_000 }, + ) + .toBe(true); + const framesBefore = frames.length; await replay(backend, input); // The last colour must reach the listener. Then wait a little more, so // that a frame between two calls would have arrived too. await expect .poll(() => frames.length, { timeout: 10_000 }) - .toBeGreaterThan(1); + .toBeGreaterThan(framesBefore); await new Promise((resolve) => setTimeout(resolve, 500)); await screencast.send('Page.stopScreencast'); - // The frame before the replay, then the frame of the last colour. - expect(frames).toHaveLength(2); + // The screen before the replay, then the frame of the last colour. + expect(frames).toHaveLength(framesBefore + 1); }, ); From e39bd15135905aa93505402a4a14cdbd1b988735 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Sat, 7 Nov 2026 00:52:03 +0000 Subject: [PATCH 14/26] docs: say that max-speed also holds a frame after a resize --- src/export/README.md | 2 +- src/renderer/ghosttyWeb/README.md | 2 +- src/renderer/ghosttyWeb/backend.ts | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/src/export/README.md b/src/export/README.md index 7733b406..5a471594 100644 --- a/src/export/README.md +++ b/src/export/README.md @@ -25,7 +25,7 @@ This directory converts a session's event log into recording exports. `generateA - Defaults: timing mode `recorded` and profile `reference-dark` [webm.ts:generateWebmExport]. - A requested `libghostty-vt` renderer is replaced with `ghostty-web`. A code comment says `libghostty-vt` is semantic-only [webm.ts:generateWebmExport]. - The default backend factory accepts only `ghostty-web` [webm.ts:createDefaultBackend]. -- Timing presets: `accelerated` caps gaps at 400 ms and holds frames at least 100 ms. `max-speed` uses a frame hold of 16 ms, but holds it only after the last flush of the replay and before a resize (see the `ghostty-web` README). Each mode holds the final frame (1000 ms, or 500 ms for `max-speed`) [webm.ts:buildReplayTimingOptions]. +- Timing presets: `accelerated` caps gaps at 400 ms and holds frames at least 100 ms. `max-speed` uses a frame hold of 16 ms, but holds it only after the last flush of the replay and before and after a resize (see the `ghostty-web` README). Each mode holds the final frame (1000 ms, or 500 ms for `max-speed`) [webm.ts:buildReplayTimingOptions]. - The timed replay fails after `REPLAY_TIMEOUT_MS` (5 minutes) [webm.ts:REPLAY_TIMEOUT_MS]. - `renderGridFramesToSvg` needs the profile font size `SVG_FONT_SIZE` (14) and draws fixed 8.4x18 px cells, so equal input gives byte-identical output [svg.ts:renderGridFramesToSvg]. It always embeds the JetBrains Mono subset, and embeds each other bundled face only when a glyph needs it [svg.ts:embeddedFontAssets]. Output past `defaultMaxSvgOutputLength` of the V8 heap limit (the heap limit divided by 8 bytes, at most `MAX_SVG_OUTPUT_LENGTH`) fails with `EXPORT_ERROR` [svg.ts:defaultMaxSvgOutputLength]. diff --git a/src/renderer/ghosttyWeb/README.md b/src/renderer/ghosttyWeb/README.md index 18b39f20..8abfa834 100644 --- a/src/renderer/ghosttyWeb/README.md +++ b/src/renderer/ghosttyWeb/README.md @@ -29,7 +29,7 @@ This directory holds the `ghostty-web` reference renderer. `GhosttyWebBackend` s - The constructor asserts a non-empty `sessionId`, `profile.name` and `profile.fontFamily`, a positive `fontSize`, and hex colors. With `videoOptions`, `outputDir` must be absolute and the size must use positive integers [backend.ts:GhosttyWebBackend.constructor]. - Replay cannot rewind: `targetSeq` must be >= the last applied seq. The initial dimensions cannot change after the first replay. `input.sessionId` must match the backend [backend.ts:GhosttyWebBackend.replayTo]. - `replayWithTiming` asserts strictly increasing seqs and non-decreasing timestamps [backend.ts:GhosttyWebBackend.replayWithTiming]. -- `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` has no schedule to fall behind, so only recorded time ends its flushes. A flush ends at the first event that is at least one frame hold of recorded time after the batch's first output. An event that changes no pixel does not end it: input, marker, run-complete, signal and exit events leave the batch as it is, because a flush at each of them would cost a frame per event. A resize and the end of the replay still end it. `max-speed` also spreads at most `MAX_SPEED_FRAME_BUDGET` (2,400) flushes over the recorded span of the events that it applies: the distance between the first output of a flush and the event that ends it is at least the larger of the frame hold and the span divided by 2,400. So a `max-speed` replay makes about one flush per frame hold of recorded time for a recording shorter than 38 s and at most about 2,400 flushes for a longer one, whatever its number of events. Output that arrives less than that distance after the first output of a flush shares the flush, also when an input or a marker sits between, so a state that the next output of the flush replaces is never painted. A flush waits for one animation frame. `max-speed` holds the frame hold only after the last flush of the replay and before a resize, so two flushes that follow each other can paint within one video frame. In `recorded` and `accelerated` mode, every event that changes no pixel ends a flush, and a hold follows every flush. Events with the same timestamp share a flush at any hold, 0 included, because a flush needs at least 1 ms of recorded time [backend.ts:GhosttyWebBackend.replayWithTiming]. +- `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` has no schedule to fall behind, so only recorded time ends its flushes. A flush ends at the first event that is at least one frame hold of recorded time after the batch's first output. An event that changes no pixel does not end it: input, marker, run-complete, signal and exit events leave the batch as it is, because a flush at each of them would cost a frame per event. A resize and the end of the replay still end it. `max-speed` also spreads at most `MAX_SPEED_FRAME_BUDGET` (2,400) flushes over the recorded span of the events that it applies: the distance between the first output of a flush and the event that ends it is at least the larger of the frame hold and the span divided by 2,400. So a `max-speed` replay makes about one flush per frame hold of recorded time for a recording shorter than 38 s and at most about 2,400 flushes for a longer one, whatever its number of events. Output that arrives less than that distance after the first output of a flush shares the flush, also when an input or a marker sits between, so a state that the next output of the flush replaces is never painted. A flush waits for one animation frame. `max-speed` holds the frame hold only after the last flush of the replay and before and after a resize, so two flushes that follow each other can paint within one video frame. In `recorded` and `accelerated` mode, every event that changes no pixel ends a flush, and a hold follows every flush. Events with the same timestamp share a flush at any hold, 0 included, because a flush needs at least 1 ms of recorded time [backend.ts:GhosttyWebBackend.replayWithTiming]. - `writeBatch(chunks, endsFlush, paintedCap)` takes the chunks of one bridge call. A flush that has more than 1000 chunks needs several calls, and ghostty-web paints on every animation frame, so the page holds the chunks of a call that does not end the flush. The call that ends it (`endsFlush`, set by the backend on the last call only) writes every held chunk in one task, so no frame paints a state between two calls, and then resolves after one frame. ghostty-web calls a write callback one animation frame (about 17 ms) after the write, and waiting on every chunk (`write`) cost one frame per output event. `write()` and `resize()` throw while chunks are held. `discardHeldChunks()` drops them: the backend calls it when a flush fails, because `HostRendererManager` keeps the backend and a retry would otherwise meet them. A `paintedCap` that is not null makes the call that ends the flush add the graphemes on the screen to the set that the missing-glyph check of `replayWithTiming` probes, so a flush is one page call. The call returns the message of a failed note and not an error, because a failed note must not fail the flush. In `replayWithTiming`, the chunks of one flush share one video frame [embeddedHarnessHtml.ts:EMBEDDED_HARNESS_HTML]. - `snapshot()` and `screenshot()` throw until `replayTo()` or `replayWithTiming()` has run once and set the initial size. Both replays accept `targetSeq` -1 for an empty Event Log. A replay over zero events sets `lastAppliedSeq` to `targetSeq` [backend.ts:GhosttyWebBackend.snapshot]. - `missingGlyphs` lists a code point only when a bundled face that lacks it drew it. Chromium draws a cluster with one font: the first stack font that covers all of it, or else the first font of the stack, which is bundled in both built-in profiles and then draws `.notdef` boxes. CDP names the face that drew a probe by its PostScript name, so a family alias or several faces under one family resolve to the right cmap. When one bundled face drew the whole grapheme, every code point that face lacks is listed. A probe with no reported font is judged against every face of the first family. When the grapheme is split over several fonts, each uncovered code point is judged by its own probe, and a mark's probe on its base is compared with a probe of the base alone. Glyphs that a host font draws are not listed. The check runs after the PNG is written, removes its probe spans, and caches each probe text's result for the page, clearing the cache when it would pass `MAX_CACHED_GLYPH_PROBES` (16384) entries. It probes at most `MAX_GLYPH_PROBES` texts and lists at most `MAX_MISSING_GLYPH_CELLS` cells (4096 each). `probeUsedFonts` sends its CDP queries in batches of 256 and stops starting batches after `GLYPH_PROBE_BUDGET_MS` (1000 ms). The result sets `missingGlyphsTruncated` when any of these stops the check early. Family names match without regard to case. A web font whose PostScript name the profile does not bundle fails the check. A failed check is logged at debug level and leaves the field out, so the screenshot still succeeds. `replayWithTiming()` records the viewport graphemes in the page after each output batch and each resize (at most `MAX_CACHED_GLYPH_PROBES`), and the final frame hold judges them once, with the hold as the probe budget, so the check does not lengthen the video. `recordedMissingGlyphs()` returns that result. A failed note or check leaves the record export fields out. A profile whose font stack does not start with a bundled font gets no `missingGlyphs` field [missingGlyphs.ts:probeUsedFonts] [backend.ts:GhosttyWebBackend.screenshot]. diff --git a/src/renderer/ghosttyWeb/backend.ts b/src/renderer/ghosttyWeb/backend.ts index eef422e8..8fab2484 100644 --- a/src/renderer/ghosttyWeb/backend.ts +++ b/src/renderer/ghosttyWeb/backend.ts @@ -771,8 +771,8 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { // minimum split of recorded time at most, within the frame budget. // The frame wait of such a flush is already its frame, so max-speed // holds only after a flush that ends the replay or comes before a - // resize: a hold after every flush would export 30 ms steps slower - // than they ran. + // resize (a resize is followed by a hold too): a hold after every + // flush would export 30 ms steps slower than they ran. if ( pendingSinceTimestampMs !== null && eventTimestampMs - pendingSinceTimestampMs >= minSplitMs && From e98841cdc0babe67f8d8178b1a01725fac72c73a Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Sun, 8 Nov 2026 12:04:14 +0000 Subject: [PATCH 15/26] docs: do not claim that max-speed shows most steps of a 30 ms progress bar --- docs/USAGE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/USAGE.md b/docs/USAGE.md index c1b0087d..9cbb14fb 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -296,7 +296,7 @@ agent-tty record export --format webm --json | jq -c '.result.missi # [{"codePoint":"U+0378"}] ``` -WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. A frame of the video shows the screen after the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event, and a state that the next output of the frame replaces is not shown. In `recorded` and `accelerated` mode, an event that changes no pixel (an input, a marker, a run-complete, a signal or the exit) also ends a frame. In `max-speed` mode it does not: only recorded time ends a frame, so output on both sides of an input event that is less than 16 ms apart shares one frame. `max-speed` removes the idle time. It makes a frame at most once per 16 ms of recorded time, so a progress bar that updates every 30 ms shows most of its steps and not only its last one. It makes at most about 2,400 frames, so a recording longer than 38 s gets one frame per 1/2,400 of its recorded time instead (a 1-hour recording: one per 1.5 s), and its export time depends on its length and not on its number of events. `max-speed` waits for one animation frame after each frame but holds no further time between two frames, so two frames that follow each other can end up in one video frame. The video has 25 frames per second, so it can skip a state that lasts less than one video frame (40 ms). +WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. A frame of the video shows the screen after the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event, and a state that the next output of the frame replaces is not shown. In `recorded` and `accelerated` mode, an event that changes no pixel (an input, a marker, a run-complete, a signal or the exit) also ends a frame. In `max-speed` mode it does not: only recorded time ends a frame, so output on both sides of an input event that is less than 16 ms apart shares one frame. `max-speed` removes the idle time. It makes a frame at most once per 16 ms of recorded time, so a progress bar that updates every 30 ms shows many of its steps and not only its last one. It makes at most about 2,400 frames, so a recording longer than 38 s gets one frame per 1/2,400 of its recorded time instead (a 1-hour recording: one per 1.5 s), and its export time depends on its length and not on its number of events. `max-speed` waits for one animation frame after each frame but holds no further time between two frames, so two frames that follow each other can end up in one video frame. The video has 25 frames per second, so it can skip a state that lasts less than one video frame (40 ms). A step of 30 ms is shorter than that, and the number of steps that survive depends on how fast the host runs the export. SVG export renders styled grid frames from the event log through the native `libghostty-vt` backend with no browser and no ffmpeg, so it requires the optional `@coder/libghostty-vt-node` package (there is no `ghostty-web` fallback). The output is deterministic: exporting the same session twice produces byte-identical, diffable SVG. `--format svg` writes a still image of the final screen and ignores event timing, so it also works on a log whose clock stepped backward (`--animate` refuses such a log and names the event where the clock stepped back); add `--animate` for an animated SVG of de-duplicated frames replayed with recorded event-log timing (`--timing` is not supported with SVG). From 9fee92d05da9b60f00b28416d3e8a19e721b40dc Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Sun, 8 Nov 2026 13:23:18 +0000 Subject: [PATCH 16/26] fix: hold a frame before a resize when the timestamp check ended the last max-speed flush --- src/renderer/ghosttyWeb/backend.ts | 10 +++++ test/unit/renderer/ghosttyWebBackend.test.ts | 46 ++++++++++++++++++++ 2 files changed, 56 insertions(+) diff --git a/src/renderer/ghosttyWeb/backend.ts b/src/renderer/ghosttyWeb/backend.ts index 8fab2484..631ad223 100644 --- a/src/renderer/ghosttyWeb/backend.ts +++ b/src/renderer/ghosttyWeb/backend.ts @@ -708,6 +708,8 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { let pendingSinceTimestampMs: number | null = null; let scheduleStartedAtMs = 0; let scheduledElapsedMs = 0; + // True after a flush that no hold followed. The next resize pays that hold. + let lastFlushUnheld = false; const flushOutputBatch = async (holdFrame = true): Promise => { if (pendingOutputChunks.length === 0) { @@ -720,6 +722,7 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { if (holdFrame) { await waitForDelay(resolvedMinFrameHoldMs); } + lastFlushUnheld = !holdFrame; }; // Events that change no pixel end a flush in recorded and accelerated @@ -803,6 +806,13 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { } case 'resize': { await flushOutputBatch(); + // A flush that the timestamp check made earlier has no hold, and + // the resize then finds nothing pending. The screen before the + // resize still needs its frame hold to reach the video. + if (lastFlushUnheld) { + await waitForDelay(resolvedMinFrameHoldMs); + lastFlushUnheld = false; + } assertPositiveInteger( event.payload.cols, 'resize event cols must be a positive integer', diff --git a/test/unit/renderer/ghosttyWebBackend.test.ts b/test/unit/renderer/ghosttyWebBackend.test.ts index 9b37cdb5..4c89a60a 100644 --- a/test/unit/renderer/ghosttyWebBackend.test.ts +++ b/test/unit/renderer/ghosttyWebBackend.test.ts @@ -1793,6 +1793,52 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { expect((atMs('resize 100') ?? 0) - (atMs('flush out0') ?? 0)).toBe(16); }); + // The timestamp check ends a max-speed flush without a hold. When that flush + // comes before a resize, the resize finds nothing pending, so the hold of the + // resize case is the only one left. Without it the resize changes the canvas + // before the video has recorded the screen before the resize. + it.each([ + { + label: 'the resize itself ends the flush', + events: [ + outputEvent(0, 0), + resizeEvent(1, 20), + outputEvent(2, 40), + exitEvent(3, 42), + ], + }, + { + label: 'an input before the resize ends the flush', + events: [ + outputEvent(0, 0), + inputEvent(1, 20), + resizeEvent(2, 25), + outputEvent(3, 40), + exitEvent(4, 42), + ], + }, + ])( + 'holds one frame hold between the last flush and a resize when $label', + async ({ events }) => { + const timeline = await replayAndRecordTimeline(events, { + mode: 'max-speed', + minFrameHoldMs: 16, + finalFrameHoldMs: 0, + }); + + expect(timeline.map((entry) => entry.step)).toEqual([ + 'flush out0', + 'resize 100', + 'note', + 'flush out' + String(events.length - 2), + 'snapshot', + ]); + const atMs = (step: string): number => + timeline.find((entry) => entry.step === step)?.atMs ?? Number.NaN; + expect(atMs('resize 100') - atMs('flush out0')).toBe(16); + }, + ); + it.each([ ['recorded', RECORDED], ['max-speed', MAX_SPEED], From ed329a5e8956ea585ac88cd87684ef5c65713d02 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Sun, 8 Nov 2026 13:23:47 +0000 Subject: [PATCH 17/26] test: hold the frame before a resize once, not at every resize --- test/unit/renderer/ghosttyWebBackend.test.ts | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/test/unit/renderer/ghosttyWebBackend.test.ts b/test/unit/renderer/ghosttyWebBackend.test.ts index 4c89a60a..150b774b 100644 --- a/test/unit/renderer/ghosttyWebBackend.test.ts +++ b/test/unit/renderer/ghosttyWebBackend.test.ts @@ -1839,6 +1839,24 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { }, ); + it('holds that frame once, for the first of two resizes that follow the flush', async () => { + const timeline = await replayAndRecordTimeline( + [ + outputEvent(0, 0), + resizeEvent(1, 20), + resizeEvent(2, 21), + exitEvent(3, 22), + ], + { mode: 'max-speed', minFrameHoldMs: 16, finalFrameHoldMs: 0 }, + ); + + const resizes = timeline.filter((entry) => entry.step === 'resize 100'); + expect(resizes).toHaveLength(2); + // The hold after the first resize is the only time between the two. + expect((resizes[1]?.atMs ?? 0) - (resizes[0]?.atMs ?? 0)).toBe(16); + expect((resizes[0]?.atMs ?? 0) - (timeline[0]?.atMs ?? 0)).toBe(16); + }); + it.each([ ['recorded', RECORDED], ['max-speed', MAX_SPEED], From 08a8c13acf3b0fbb978426320bf696ea90d58d95 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Sun, 8 Nov 2026 13:28:00 +0000 Subject: [PATCH 18/26] refactor: name the hold after an unheld max-speed flush --- src/renderer/ghosttyWeb/backend.ts | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/src/renderer/ghosttyWeb/backend.ts b/src/renderer/ghosttyWeb/backend.ts index 631ad223..6df2d54f 100644 --- a/src/renderer/ghosttyWeb/backend.ts +++ b/src/renderer/ghosttyWeb/backend.ts @@ -724,6 +724,12 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { } lastFlushUnheld = !holdFrame; }; + const holdAfterUnheldFlush = async (): Promise => { + if (lastFlushUnheld) { + lastFlushUnheld = false; + await waitForDelay(resolvedMinFrameHoldMs); + } + }; // Events that change no pixel end a flush in recorded and accelerated // mode, where the pause before them is part of the video. Max-speed has no @@ -809,10 +815,7 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { // A flush that the timestamp check made earlier has no hold, and // the resize then finds nothing pending. The screen before the // resize still needs its frame hold to reach the video. - if (lastFlushUnheld) { - await waitForDelay(resolvedMinFrameHoldMs); - lastFlushUnheld = false; - } + await holdAfterUnheldFlush(); assertPositiveInteger( event.payload.cols, 'resize event cols must be a positive integer', From d5d3540b46d3e0467dcd415d6d89ec00fbb46842 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Sun, 8 Nov 2026 15:27:27 +0000 Subject: [PATCH 19/26] docs: state what the max-speed budget comment measured and where resizes end a flush --- docs/USAGE.md | 2 +- src/renderer/ghosttyWeb/README.md | 2 +- src/renderer/ghosttyWeb/backend.ts | 24 ++++++++++++++++-------- 3 files changed, 18 insertions(+), 10 deletions(-) diff --git a/docs/USAGE.md b/docs/USAGE.md index 9cbb14fb..1f687ab4 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -296,7 +296,7 @@ agent-tty record export --format webm --json | jq -c '.result.missi # [{"codePoint":"U+0378"}] ``` -WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. A frame of the video shows the screen after the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event, and a state that the next output of the frame replaces is not shown. In `recorded` and `accelerated` mode, an event that changes no pixel (an input, a marker, a run-complete, a signal or the exit) also ends a frame. In `max-speed` mode it does not: only recorded time ends a frame, so output on both sides of an input event that is less than 16 ms apart shares one frame. `max-speed` removes the idle time. It makes a frame at most once per 16 ms of recorded time, so a progress bar that updates every 30 ms shows many of its steps and not only its last one. It makes at most about 2,400 frames, so a recording longer than 38 s gets one frame per 1/2,400 of its recorded time instead (a 1-hour recording: one per 1.5 s), and its export time depends on its length and not on its number of events. `max-speed` waits for one animation frame after each frame but holds no further time between two frames, so two frames that follow each other can end up in one video frame. The video has 25 frames per second, so it can skip a state that lasts less than one video frame (40 ms). A step of 30 ms is shorter than that, and the number of steps that survive depends on how fast the host runs the export. +WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. A frame of the video shows the screen after the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event, and a state that the next output of the frame replaces is not shown. In `recorded` and `accelerated` mode, an event that changes no pixel (an input, a marker, a run-complete, a signal or the exit) also ends a frame. In `max-speed` mode it does not: recorded time, a resize and the end of the recording end a frame, so output on both sides of an input event that is less than 16 ms apart shares one frame. `max-speed` removes the idle time. It makes a frame at most once per 16 ms of recorded time, so a progress bar that updates every 30 ms shows many of its steps and not only its last one. Recorded time makes at most about 2,400 frames, so a recording longer than 38 s gets one frame per 1/2,400 of its recorded time instead (a 1-hour recording: one per 1.5 s). Each resize adds one frame to that, so the export time of a recording follows its length and its number of resizes, and not its number of output events. `max-speed` waits for one animation frame after each frame but holds no further time between two frames, so two frames that follow each other can end up in one video frame. The video has 25 frames per second, so it can skip a state that lasts less than one video frame (40 ms). A step of 30 ms is shorter than that, and the number of steps that survive depends on how fast the host runs the export. SVG export renders styled grid frames from the event log through the native `libghostty-vt` backend with no browser and no ffmpeg, so it requires the optional `@coder/libghostty-vt-node` package (there is no `ghostty-web` fallback). The output is deterministic: exporting the same session twice produces byte-identical, diffable SVG. `--format svg` writes a still image of the final screen and ignores event timing, so it also works on a log whose clock stepped backward (`--animate` refuses such a log and names the event where the clock stepped back); add `--animate` for an animated SVG of de-duplicated frames replayed with recorded event-log timing (`--timing` is not supported with SVG). diff --git a/src/renderer/ghosttyWeb/README.md b/src/renderer/ghosttyWeb/README.md index 8abfa834..c2770e54 100644 --- a/src/renderer/ghosttyWeb/README.md +++ b/src/renderer/ghosttyWeb/README.md @@ -29,7 +29,7 @@ This directory holds the `ghostty-web` reference renderer. `GhosttyWebBackend` s - The constructor asserts a non-empty `sessionId`, `profile.name` and `profile.fontFamily`, a positive `fontSize`, and hex colors. With `videoOptions`, `outputDir` must be absolute and the size must use positive integers [backend.ts:GhosttyWebBackend.constructor]. - Replay cannot rewind: `targetSeq` must be >= the last applied seq. The initial dimensions cannot change after the first replay. `input.sessionId` must match the backend [backend.ts:GhosttyWebBackend.replayTo]. - `replayWithTiming` asserts strictly increasing seqs and non-decreasing timestamps [backend.ts:GhosttyWebBackend.replayWithTiming]. -- `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` has no schedule to fall behind, so only recorded time ends its flushes. A flush ends at the first event that is at least one frame hold of recorded time after the batch's first output. An event that changes no pixel does not end it: input, marker, run-complete, signal and exit events leave the batch as it is, because a flush at each of them would cost a frame per event. A resize and the end of the replay still end it. `max-speed` also spreads at most `MAX_SPEED_FRAME_BUDGET` (2,400) flushes over the recorded span of the events that it applies: the distance between the first output of a flush and the event that ends it is at least the larger of the frame hold and the span divided by 2,400. So a `max-speed` replay makes about one flush per frame hold of recorded time for a recording shorter than 38 s and at most about 2,400 flushes for a longer one, whatever its number of events. Output that arrives less than that distance after the first output of a flush shares the flush, also when an input or a marker sits between, so a state that the next output of the flush replaces is never painted. A flush waits for one animation frame. `max-speed` holds the frame hold only after the last flush of the replay and before and after a resize, so two flushes that follow each other can paint within one video frame. In `recorded` and `accelerated` mode, every event that changes no pixel ends a flush, and a hold follows every flush. Events with the same timestamp share a flush at any hold, 0 included, because a flush needs at least 1 ms of recorded time [backend.ts:GhosttyWebBackend.replayWithTiming]. +- `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` has no schedule to fall behind, so recorded time ends most of its flushes. A flush ends at the first event that is at least one frame hold of recorded time after the batch's first output. An event that changes no pixel does not end it: input, marker, run-complete, signal and exit events leave the batch as it is, because a flush at each of them would cost a frame per event. A resize and the end of the replay still end it. `max-speed` also spreads at most `MAX_SPEED_FRAME_BUDGET` (2,400) flushes over the recorded span of the events that it applies: the distance between the first output of a flush and the event that ends it is at least the larger of the frame hold and the span divided by 2,400. So recorded time makes about one flush per frame hold for a recording shorter than 38 s and at most about 2,400 flushes for a longer one, whatever its number of output events. A resize ends a flush too, and these flushes do not count against the 2,400, so a recording with many resizes makes about one more flush for each of them. Output that arrives less than that distance after the first output of a flush shares the flush, also when an input or a marker sits between, so a state that the next output of the flush replaces is never painted. A flush waits for one animation frame. `max-speed` holds the frame hold only after the last flush of the replay and before and after a resize, so two flushes that follow each other can paint within one video frame. In `recorded` and `accelerated` mode, every event that changes no pixel ends a flush, and a hold follows every flush. Output events with the same timestamp share a flush at any hold, 0 included, because the time check needs at least 1 ms of recorded time. An event between them that ends a flush splits them anyway: a resize in every mode, and in `recorded` and `accelerated` mode also an input, a marker, a run-complete, a signal or an exit [backend.ts:GhosttyWebBackend.replayWithTiming]. - `writeBatch(chunks, endsFlush, paintedCap)` takes the chunks of one bridge call. A flush that has more than 1000 chunks needs several calls, and ghostty-web paints on every animation frame, so the page holds the chunks of a call that does not end the flush. The call that ends it (`endsFlush`, set by the backend on the last call only) writes every held chunk in one task, so no frame paints a state between two calls, and then resolves after one frame. ghostty-web calls a write callback one animation frame (about 17 ms) after the write, and waiting on every chunk (`write`) cost one frame per output event. `write()` and `resize()` throw while chunks are held. `discardHeldChunks()` drops them: the backend calls it when a flush fails, because `HostRendererManager` keeps the backend and a retry would otherwise meet them. A `paintedCap` that is not null makes the call that ends the flush add the graphemes on the screen to the set that the missing-glyph check of `replayWithTiming` probes, so a flush is one page call. The call returns the message of a failed note and not an error, because a failed note must not fail the flush. In `replayWithTiming`, the chunks of one flush share one video frame [embeddedHarnessHtml.ts:EMBEDDED_HARNESS_HTML]. - `snapshot()` and `screenshot()` throw until `replayTo()` or `replayWithTiming()` has run once and set the initial size. Both replays accept `targetSeq` -1 for an empty Event Log. A replay over zero events sets `lastAppliedSeq` to `targetSeq` [backend.ts:GhosttyWebBackend.snapshot]. - `missingGlyphs` lists a code point only when a bundled face that lacks it drew it. Chromium draws a cluster with one font: the first stack font that covers all of it, or else the first font of the stack, which is bundled in both built-in profiles and then draws `.notdef` boxes. CDP names the face that drew a probe by its PostScript name, so a family alias or several faces under one family resolve to the right cmap. When one bundled face drew the whole grapheme, every code point that face lacks is listed. A probe with no reported font is judged against every face of the first family. When the grapheme is split over several fonts, each uncovered code point is judged by its own probe, and a mark's probe on its base is compared with a probe of the base alone. Glyphs that a host font draws are not listed. The check runs after the PNG is written, removes its probe spans, and caches each probe text's result for the page, clearing the cache when it would pass `MAX_CACHED_GLYPH_PROBES` (16384) entries. It probes at most `MAX_GLYPH_PROBES` texts and lists at most `MAX_MISSING_GLYPH_CELLS` cells (4096 each). `probeUsedFonts` sends its CDP queries in batches of 256 and stops starting batches after `GLYPH_PROBE_BUDGET_MS` (1000 ms). The result sets `missingGlyphsTruncated` when any of these stops the check early. Family names match without regard to case. A web font whose PostScript name the profile does not bundle fails the check. A failed check is logged at debug level and leaves the field out, so the screenshot still succeeds. `replayWithTiming()` records the viewport graphemes in the page after each output batch and each resize (at most `MAX_CACHED_GLYPH_PROBES`), and the final frame hold judges them once, with the hold as the probe budget, so the check does not lengthen the video. `recordedMissingGlyphs()` returns that result. A failed note or check leaves the record export fields out. A profile whose font stack does not start with a bundled font gets no `missingGlyphs` field [missingGlyphs.ts:probeUsedFonts] [backend.ts:GhosttyWebBackend.screenshot]. diff --git a/src/renderer/ghosttyWeb/backend.ts b/src/renderer/ghosttyWeb/backend.ts index 6df2d54f..fbf19d26 100644 --- a/src/renderer/ghosttyWeb/backend.ts +++ b/src/renderer/ghosttyWeb/backend.ts @@ -138,12 +138,19 @@ const WASM_CONTENT_TYPE = 'application/wasm'; const MAX_REPLAY_BATCH_SIZE = 1000; /** - * A max-speed replay makes at most about this many flushes, however long the - * recording is. A flush costs one animation frame, one page call and one scan - * of the screen: 26 ms at 80x24, 33 ms at 120x40 and 112 ms at 200x60 in the - * median of five sparse-log runs (host load 40 to 100). The limit then takes - * 1 to 4.5 minutes, under the 5 minutes that a WebM export allows its replay. - * The video has 25 frames per second, so this many flushes fill 96 s of video. + * Recorded time ends at most about this many flushes of a max-speed replay, + * however long the recording is. A resize ends one more flush, and this limit + * does not count it, so a log with many resizes makes more. A flush costs one + * animation frame, one page call and one scan of the screen, and that cost + * grows with the terminal size and the load of the host. Measured on PR 358 + * (host CPU pressure 15 to 44 %): 400 outputs 20 ms apart took 18 to 20 ms per + * flush at 80x24. A worst-case log of 2,400 full-screen repaints 17 ms apart + * at 200x60 took 287 s (120 ms per flush) in two runs, and 368 s (153 ms per + * flush) in a run of a verifier at pressure 23 to 38 %. A WebM export allows + * its replay 5 minutes (REPLAY_TIMEOUT_MS), so a loaded host can pass that + * limit at a large terminal size and fail the export. PR 356 replaces that + * limit with a stall window. The video has 25 frames per second, so this many + * flushes fill 96 s of video. */ const MAX_SPEED_FRAME_BUDGET = 2_400; const RAF_TIMEOUT_MS = 5_000; @@ -689,8 +696,9 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { // Output shows once the next event is this far after the batch's first // output. It is at least 1 ms: events with one timestamp share a flush. // Max-speed has no schedule that bounds its replay time, so it also - // spreads a fixed number of flushes over the recorded span: a sparse - // recording then costs the same number of frames as a dense one. + // spreads a fixed number of its flushes over the recorded span: a sparse + // recording then costs about the same number of frames as a dense one. A + // resize ends one more flush, and the budget does not count it. const minSplitMs = Math.max( 1, resolvedMinFrameHoldMs, From 099817a4d1588ec775c28c66a49dbaafebd48aae Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Sun, 8 Nov 2026 15:35:05 +0000 Subject: [PATCH 20/26] docs: give each measured flush cost of the max-speed budget comment its own host load --- src/renderer/ghosttyWeb/backend.ts | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/src/renderer/ghosttyWeb/backend.ts b/src/renderer/ghosttyWeb/backend.ts index fbf19d26..ac476d33 100644 --- a/src/renderer/ghosttyWeb/backend.ts +++ b/src/renderer/ghosttyWeb/backend.ts @@ -142,11 +142,12 @@ const MAX_REPLAY_BATCH_SIZE = 1000; * however long the recording is. A resize ends one more flush, and this limit * does not count it, so a log with many resizes makes more. A flush costs one * animation frame, one page call and one scan of the screen, and that cost - * grows with the terminal size and the load of the host. Measured on PR 358 - * (host CPU pressure 15 to 44 %): 400 outputs 20 ms apart took 18 to 20 ms per - * flush at 80x24. A worst-case log of 2,400 full-screen repaints 17 ms apart - * at 200x60 took 287 s (120 ms per flush) in two runs, and 368 s (153 ms per - * flush) in a run of a verifier at pressure 23 to 38 %. A WebM export allows + * grows with the terminal size and the load of the host. Measured on PR 358: + * 400 outputs 20 ms apart took 18 to 20 ms per flush at 80x24 (host CPU + * pressure 4 to 22 %). A worst-case log of 2,400 full-screen repaints 17 ms + * apart at 200x60 took 287 s (120 ms per flush) in two runs at pressure 15 to + * 44 %, and 368 s (153 ms per flush) in a run of a verifier at pressure 23 to + * 38 %. A WebM export allows * its replay 5 minutes (REPLAY_TIMEOUT_MS), so a loaded host can pass that * limit at a large terminal size and fail the export. PR 356 replaces that * limit with a stall window. The video has 25 frames per second, so this many From 112d34daa5415efb37ba15889eea07b5e83ab39e Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Sun, 8 Nov 2026 15:35:17 +0000 Subject: [PATCH 21/26] docs: rewrap the max-speed budget comment --- src/renderer/ghosttyWeb/backend.ts | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/src/renderer/ghosttyWeb/backend.ts b/src/renderer/ghosttyWeb/backend.ts index ac476d33..dbe49797 100644 --- a/src/renderer/ghosttyWeb/backend.ts +++ b/src/renderer/ghosttyWeb/backend.ts @@ -147,11 +147,10 @@ const MAX_REPLAY_BATCH_SIZE = 1000; * pressure 4 to 22 %). A worst-case log of 2,400 full-screen repaints 17 ms * apart at 200x60 took 287 s (120 ms per flush) in two runs at pressure 15 to * 44 %, and 368 s (153 ms per flush) in a run of a verifier at pressure 23 to - * 38 %. A WebM export allows - * its replay 5 minutes (REPLAY_TIMEOUT_MS), so a loaded host can pass that - * limit at a large terminal size and fail the export. PR 356 replaces that - * limit with a stall window. The video has 25 frames per second, so this many - * flushes fill 96 s of video. + * 38 %. A WebM export allows its replay 5 minutes (REPLAY_TIMEOUT_MS), so a + * loaded host can pass that limit at a large terminal size and fail the + * export. PR 356 replaces that limit with a stall window. The video has 25 + * frames per second, so this many flushes fill 96 s of video. */ const MAX_SPEED_FRAME_BUDGET = 2_400; const RAF_TIMEOUT_MS = 5_000; From a9b80836dc86a7b56278454826911a3cbde79514 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Mon, 9 Nov 2026 10:03:29 +0000 Subject: [PATCH 22/26] docs: leave the open stall-window change out of the max-speed budget comment --- src/renderer/ghosttyWeb/backend.ts | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/renderer/ghosttyWeb/backend.ts b/src/renderer/ghosttyWeb/backend.ts index 0c5c683c..ca00baff 100644 --- a/src/renderer/ghosttyWeb/backend.ts +++ b/src/renderer/ghosttyWeb/backend.ts @@ -150,8 +150,8 @@ const MAX_REPLAY_BATCH_SIZE = 1000; * 44 %, and 368 s (153 ms per flush) in a run of a verifier at pressure 23 to * 38 %. A WebM export allows its replay 5 minutes (REPLAY_TIMEOUT_MS), so a * loaded host can pass that limit at a large terminal size and fail the - * export. PR 356 replaces that limit with a stall window. The video has 25 - * frames per second, so this many flushes fill 96 s of video. + * export. The video has 25 frames per second, so this many flushes fill 96 s + * of video. */ const MAX_SPEED_FRAME_BUDGET = 2_400; const RAF_TIMEOUT_MS = 5_000; From 8a4236c8e0f20eeff99cf8ca5dce8107db0826a5 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Mon, 9 Nov 2026 13:32:35 +0000 Subject: [PATCH 23/26] fix: hold one video frame before a resize in a max-speed export The 16 ms frame hold before a resize is shorter than one video frame (40 ms at 25 frames per second), so the recorder missed the screen before a resize in 20 of 720 resizes in an interleaved test (719 of 720 seen with a hold of one video frame). A max-speed replay now holds the larger of the frame hold and one video frame before a resize. Recorded and accelerated replays keep their frame hold. The context sets the 25 frames per second that the hold is built on. A new test pins one hold when a timed flush, pending output and a resize follow each other. The tests for the sticky flag and for a hold before the resize's own flush turn red. --- src/renderer/ghosttyWeb/backend.ts | 64 ++++++--- test/unit/renderer/ghosttyWebBackend.test.ts | 129 ++++++++++++++++++- 2 files changed, 166 insertions(+), 27 deletions(-) diff --git a/src/renderer/ghosttyWeb/backend.ts b/src/renderer/ghosttyWeb/backend.ts index ca00baff..39beaaa1 100644 --- a/src/renderer/ghosttyWeb/backend.ts +++ b/src/renderer/ghosttyWeb/backend.ts @@ -140,20 +140,31 @@ const WASM_CONTENT_TYPE = 'application/wasm'; const MAX_REPLAY_BATCH_SIZE = 1000; /** * Recorded time ends at most about this many flushes of a max-speed replay, - * however long the recording is. A resize ends one more flush, and this limit - * does not count it, so a log with many resizes makes more. A flush costs one + * however long the recording is. A resize that finds output waiting ends one + * more flush, and this limit does not count it, so a log with many resizes + * makes more (at most one more for each resize). A flush costs one * animation frame, one page call and one scan of the screen, and that cost - * grows with the terminal size and the load of the host. Measured on PR 358: - * 400 outputs 20 ms apart took 18 to 20 ms per flush at 80x24 (host CPU - * pressure 4 to 22 %). A worst-case log of 2,400 full-screen repaints 17 ms - * apart at 200x60 took 287 s (120 ms per flush) in two runs at pressure 15 to - * 44 %, and 368 s (153 ms per flush) in a run of a verifier at pressure 23 to - * 38 %. A WebM export allows its replay 5 minutes (REPLAY_TIMEOUT_MS), so a - * loaded host can pass that limit at a large terminal size and fail the - * export. The video has 25 frames per second, so this many flushes fill 96 s - * of video. + * grows with the terminal size and the load of the host. At 80x24, 400 + * outputs 20 ms apart took 18 to 20 ms per flush at host CPU pressure 4 to + * 22 %, and 400 outputs 40 ms apart took 27 to 54 ms per flush at pressure 38 + * to 70 %. A worst-case log of 2,400 full-screen repaints 17 ms apart at + * 200x60 took 287 s (120 ms per flush) in two runs at pressure 15 to 44 %, and + * 368 s to 405 s (153 to 169 ms per flush) in three runs of a verifier at + * pressure 23 to 45 %. A WebM export allows its replay 5 minutes + * (REPLAY_TIMEOUT_MS), so a loaded host can pass that limit at a large + * terminal size and fail the export. The video has 25 frames per second and + * records the replay as it runs, so this many flushes of 40 ms or more make a + * video of 96 s or more. */ const MAX_SPEED_FRAME_BUDGET = 2_400; +/** + * The frames per second of the video. Playwright records 25 when the context + * sets no `recordVideo.fps`, and the context here sets this value, so the + * number does not depend on that default. A state of the page that stays on it + * for less than one video frame can be missing from the video. + */ +const VIDEO_FPS = 25; +const VIDEO_FRAME_MS = 1000 / VIDEO_FPS; const RAF_TIMEOUT_MS = 5_000; let servedAssetsPromise: Promise< @@ -718,7 +729,8 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { // Max-speed has no schedule that bounds its replay time, so it also // spreads a fixed number of its flushes over the recorded span: a sparse // recording then costs about the same number of frames as a dense one. A - // resize ends one more flush, and the budget does not count it. + // resize that finds output waiting ends one more flush, and the budget + // does not count it. const minSplitMs = Math.max( 1, resolvedMinFrameHoldMs, @@ -736,10 +748,21 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { let pendingSinceTimestampMs: number | null = null; let scheduleStartedAtMs = 0; let scheduledElapsedMs = 0; + // Max-speed has no pauses, so the screen before a resize reaches the video + // only if it stays on the page for one video frame. A frame hold of 16 ms + // is shorter than that, and the recorder then misses the screen in some + // runs. Recorded and accelerated replays keep their pauses and their hold. + const preResizeHoldMs = + timing.mode === 'max-speed' + ? Math.max(resolvedMinFrameHoldMs, VIDEO_FRAME_MS) + : resolvedMinFrameHoldMs; // True after a flush that no hold followed. The next resize pays that hold. let lastFlushUnheld = false; - const flushOutputBatch = async (holdFrame = true): Promise => { + const flushOutputBatch = async ( + holdFrame = true, + holdMs = resolvedMinFrameHoldMs, + ): Promise => { if (pendingOutputChunks.length === 0) { return; } @@ -748,14 +771,14 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { pendingOutputChunks = []; pendingSinceTimestampMs = null; if (holdFrame) { - await waitForDelay(resolvedMinFrameHoldMs); + await waitForDelay(holdMs); } lastFlushUnheld = !holdFrame; }; const holdAfterUnheldFlush = async (): Promise => { if (lastFlushUnheld) { lastFlushUnheld = false; - await waitForDelay(resolvedMinFrameHoldMs); + await waitForDelay(preResizeHoldMs); } }; @@ -809,7 +832,8 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { // The frame wait of such a flush is already its frame, so max-speed // holds only after a flush that ends the replay or comes before a // resize (a resize is followed by a hold too): a hold after every - // flush would export 30 ms steps slower than they ran. + // flush would export 30 ms steps slower than they ran. The hold + // before a resize lasts one video frame, see preResizeHoldMs. if ( pendingSinceTimestampMs !== null && eventTimestampMs - pendingSinceTimestampMs >= minSplitMs && @@ -839,10 +863,9 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { break; } case 'resize': { - await flushOutputBatch(); - // A flush that the timestamp check made earlier has no hold, and - // the resize then finds nothing pending. The screen before the - // resize still needs its frame hold to reach the video. + // The flush here holds before the resize. A flush that the timestamp + // check made earlier has no hold, and the resize pays it now, once. + await flushOutputBatch(true, preResizeHoldMs); await holdAfterUnheldFlush(); assertPositiveInteger( event.payload.cols, @@ -1343,6 +1366,7 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { recordVideo: { dir: this.videoOptions.outputDir, size: this.videoOptions.size, + fps: VIDEO_FPS, }, } : {}), diff --git a/test/unit/renderer/ghosttyWebBackend.test.ts b/test/unit/renderer/ghosttyWebBackend.test.ts index 150b774b..11fcdb07 100644 --- a/test/unit/renderer/ghosttyWebBackend.test.ts +++ b/test/unit/renderer/ghosttyWebBackend.test.ts @@ -12,6 +12,7 @@ import { tmpdir } from 'node:os'; import { dirname, join } from 'node:path'; import { afterEach, describe, expect, it, vi } from 'vitest'; +import { chromium, type Browser } from 'playwright'; import type { ReplayTimingOptions } from '../../../src/renderer/backend.js'; import type { ReplayInput } from '../../../src/renderer/types.js'; @@ -924,6 +925,43 @@ describe('GhosttyWebBackend unit guards', () => { } }, 45_000); + // The max-speed hold before a resize lasts one video frame, which is 40 ms + // only at 25 frames per second. The context sets that rate, so a change of + // the Playwright default does not move the hold. + it('records the video at 25 frames per second', async () => { + const outputDir = await mkdtemp(join(tmpdir(), 'agent-tty-video-fps-')); + const size = { width: 760, height: 536 }; + const newContext = vi.fn(() => + Promise.reject(new Error('stop at the context options')), + ); + const launch = vi.spyOn(chromium, 'launch').mockResolvedValue({ + newContext, + on: vi.fn(), + close: vi.fn(() => Promise.resolve()), + } as unknown as Browser); + const backend = new GhosttyWebBackend('renderer-unit-session', PROFILE, { + outputDir, + size, + }); + + try { + await expect(backend.boot()).rejects.toThrow( + 'stop at the context options', + ); + + expect(newContext).toHaveBeenCalledTimes(1); + expect(newContext.mock.calls[0]).toEqual([ + expect.objectContaining({ + recordVideo: { dir: outputDir, size, fps: 25 }, + }), + ]); + } finally { + launch.mockRestore(); + await backend.dispose(); + await rm(outputDir, { recursive: true, force: true }); + } + }); + it('times out screenshot paint waits after 5 seconds', async () => { vi.useFakeTimers(); const backend = createBackend(); @@ -1768,9 +1806,10 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { ]); }); - // The page paints a flush on its next animation frame. The hold gives that - // paint time to reach the video before the resize changes the canvas. - it('holds one frame hold between the last flush and a resize', async () => { + // The page paints a flush on its next animation frame, and the video has 25 + // frames per second. The hold gives that paint one video frame (40 ms) to + // reach the video before the resize changes the canvas. + it('holds one video frame between the last flush and a resize', async () => { const timeline = await replayAndRecordTimeline( [ outputEvent(0, 0), @@ -1790,7 +1829,7 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { ]); const atMs = (step: string): number | undefined => timeline.find((entry) => entry.step === step)?.atMs; - expect((atMs('resize 100') ?? 0) - (atMs('flush out0') ?? 0)).toBe(16); + expect((atMs('resize 100') ?? 0) - (atMs('flush out0') ?? 0)).toBe(40); }); // The timestamp check ends a max-speed flush without a hold. When that flush @@ -1818,7 +1857,7 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { ], }, ])( - 'holds one frame hold between the last flush and a resize when $label', + 'holds one video frame between the last flush and a resize when $label', async ({ events }) => { const timeline = await replayAndRecordTimeline(events, { mode: 'max-speed', @@ -1835,7 +1874,7 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { ]); const atMs = (step: string): number => timeline.find((entry) => entry.step === step)?.atMs ?? Number.NaN; - expect(atMs('resize 100') - atMs('flush out0')).toBe(16); + expect(atMs('resize 100') - atMs('flush out0')).toBe(40); }, ); @@ -1854,9 +1893,85 @@ describe('GhosttyWebBackend replayWithTiming output pacing', () => { expect(resizes).toHaveLength(2); // The hold after the first resize is the only time between the two. expect((resizes[1]?.atMs ?? 0) - (resizes[0]?.atMs ?? 0)).toBe(16); - expect((resizes[0]?.atMs ?? 0) - (timeline[0]?.atMs ?? 0)).toBe(16); + expect((resizes[0]?.atMs ?? 0) - (timeline[0]?.atMs ?? 0)).toBe(40); + }); + + // A timed flush has no hold, then output arrives and waits for the resize. + // The resize case flushes that output, and the one hold that follows is all + // the screen before the resize needs. A second hold would show that screen + // for two video frames instead of one. + it('holds one video frame, not two, when a timed flush is followed by pending output and a resize', async () => { + const timeline = await replayAndRecordTimeline( + [ + outputEvent(0, 0), + outputEvent(1, 100), + resizeEvent(2, 101), + exitEvent(3, 102), + ], + { mode: 'max-speed', minFrameHoldMs: 16, finalFrameHoldMs: 0 }, + ); + + expect(timeline.map((entry) => entry.step)).toEqual([ + 'flush out0', + 'flush out1', + 'resize 100', + 'note', + 'snapshot', + ]); + const atMs = (step: string): number => + timeline.find((entry) => entry.step === step)?.atMs ?? Number.NaN; + // The timed flush has no hold, so the next flush follows at once. + expect(atMs('flush out1') - atMs('flush out0')).toBe(0); + // One hold sits between the last flush and the resize. + expect(atMs('resize 100') - atMs('flush out1')).toBe(40); + }); + + // A frame hold longer than a video frame already gives the screen before the + // resize more than one video frame. + it('holds a max-speed frame hold that is longer than a video frame before a resize', async () => { + const timeline = await replayAndRecordTimeline( + [outputEvent(0, 0), resizeEvent(1, 5), exitEvent(2, 6)], + { mode: 'max-speed', minFrameHoldMs: 100, finalFrameHoldMs: 0 }, + ); + + const atMs = (step: string): number => + timeline.find((entry) => entry.step === step)?.atMs ?? Number.NaN; + expect(atMs('resize 100') - atMs('flush out0')).toBe(100); }); + // Recorded and accelerated replays keep their pauses, so the screen before a + // resize is on the page for as long as the recording shows it. Their frame + // hold before a resize does not change. + it.each([ + [ + 'recorded', + { mode: 'recorded', finalFrameHoldMs: 0 } satisfies ReplayTimingOptions, + 16, + ], + [ + 'accelerated', + { + mode: 'accelerated', + maxGapMs: 400, + minFrameHoldMs: 100, + finalFrameHoldMs: 0, + } satisfies ReplayTimingOptions, + 100, + ], + ])( + 'holds the frame hold of %s mode before a resize', + async (_name, timing, holdMs) => { + const timeline = await replayAndRecordTimeline( + [outputEvent(0, 0), resizeEvent(1, 5), exitEvent(2, 6)], + timing, + ); + + const atMs = (step: string): number => + timeline.find((entry) => entry.step === step)?.atMs ?? Number.NaN; + expect(atMs('resize 100') - atMs('flush out0')).toBe(holdMs); + }, + ); + it.each([ ['recorded', RECORDED], ['max-speed', MAX_SPEED], From 426998cc30f6785489e2445f906041b71b0a5483 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Mon, 9 Nov 2026 13:33:17 +0000 Subject: [PATCH 24/26] docs: correct the max-speed frame count, resize and pre-resize hold text The USAGE sentence that said max-speed makes a frame at most once per 16 ms ignored resizes, and a resize adds at most one frame, not one each. The renderer and export READMEs now describe the one-video-frame hold before a resize. The budget comment lists the measured flush costs. --- docs/USAGE.md | 2 +- src/export/README.md | 2 +- src/renderer/ghosttyWeb/README.md | 2 +- src/renderer/ghosttyWeb/backend.ts | 9 +++++---- 4 files changed, 8 insertions(+), 7 deletions(-) diff --git a/docs/USAGE.md b/docs/USAGE.md index 65022239..2822b6d9 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -296,7 +296,7 @@ agent-tty record export --format webm --json | jq -c '.result.missi # [{"codePoint":"U+0378"}] ``` -WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. A frame of the video shows the screen after the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event, and a state that the next output of the frame replaces is not shown. In `recorded` and `accelerated` mode, an event that changes no pixel (an input, a marker, a run-complete, a signal or the exit) also ends a frame. In `max-speed` mode it does not: recorded time, a resize and the end of the recording end a frame, so output on both sides of an input event that is less than 16 ms apart shares one frame. `max-speed` removes the idle time. It makes a frame at most once per 16 ms of recorded time, so a progress bar that updates every 30 ms shows many of its steps and not only its last one. Recorded time makes at most about 2,400 frames, so a recording longer than 38 s gets one frame per 1/2,400 of its recorded time instead (a 1-hour recording: one per 1.5 s). Each resize adds one frame to that, so the export time of a recording follows its length and its number of resizes, and not its number of output events. `max-speed` waits for one animation frame after each frame but holds no further time between two frames, so two frames that follow each other can end up in one video frame. The video has 25 frames per second, so it can skip a state that lasts less than one video frame (40 ms). A step of 30 ms is shorter than that, and the number of steps that survive depends on how fast the host runs the export. +WebM export replays with recorded wall-clock timing by default. Pass `--timing accelerated` (idle gaps clamped to 400ms) or `--timing max-speed` for a time-compressed video. A frame of the video shows the screen after the output that arrived within one frame hold of the first output of that frame: 16 ms in `recorded` and `max-speed` mode, 100 ms in `accelerated` mode. Output events that close together share one frame, so a burst of output does not cost one frame per event, and a state that the next output of the frame replaces is not shown. In `recorded` and `accelerated` mode, an event that changes no pixel (an input, a marker, a run-complete, a signal or the exit) also ends a frame. In `max-speed` mode it does not: recorded time, a resize and the end of the recording end a frame, so output on both sides of an input event that is less than 16 ms apart shares one frame. `max-speed` removes the idle time. Recorded time ends a frame at most once per 16 ms, so a progress bar that updates every 30 ms shows many of its steps and not only its last one. Recorded time ends at most about 2,400 frames, so a recording longer than 38 s gets one frame per 1/2,400 of its recorded time instead (a 1-hour recording: one per 1.5 s). A resize that finds output waiting ends one more frame, and the limits above do not count it: 100,000 outputs that each have a resize 1 ms after them make 100,000 frames. A resize adds at most one frame, and none when no output is waiting. So the export time of a recording follows its length and its number of resizes, and not its number of output events. `max-speed` waits for one animation frame after each frame but holds no further time between two frames, so two frames that follow each other can end up in one video frame. The video has 25 frames per second, so it can skip a state that lasts less than one video frame (40 ms). A step of 30 ms is shorter than that, and the number of steps that survive depends on how fast the host runs the export. The screen before a resize is the exception: `max-speed` keeps it on the page for one video frame (40 ms) before the resize, so that the video shows it. The video can still miss it in rare cases. SVG export renders styled grid frames from the event log through the native `libghostty-vt` backend with no browser and no ffmpeg, so it requires the optional `@coder/libghostty-vt-node` package (there is no `ghostty-web` fallback). The output is deterministic: exporting the same session twice produces byte-identical, diffable SVG. `--format svg` writes a still image of the final screen and ignores event timing, so it also works on a log whose clock stepped backward (`--animate` refuses such a log and names the event where the clock stepped back); add `--animate` for an animated SVG of de-duplicated frames replayed with recorded event-log timing (`--timing` is not supported with SVG). diff --git a/src/export/README.md b/src/export/README.md index 5a471594..859b64de 100644 --- a/src/export/README.md +++ b/src/export/README.md @@ -25,7 +25,7 @@ This directory converts a session's event log into recording exports. `generateA - Defaults: timing mode `recorded` and profile `reference-dark` [webm.ts:generateWebmExport]. - A requested `libghostty-vt` renderer is replaced with `ghostty-web`. A code comment says `libghostty-vt` is semantic-only [webm.ts:generateWebmExport]. - The default backend factory accepts only `ghostty-web` [webm.ts:createDefaultBackend]. -- Timing presets: `accelerated` caps gaps at 400 ms and holds frames at least 100 ms. `max-speed` uses a frame hold of 16 ms, but holds it only after the last flush of the replay and before and after a resize (see the `ghostty-web` README). Each mode holds the final frame (1000 ms, or 500 ms for `max-speed`) [webm.ts:buildReplayTimingOptions]. +- Timing presets: `accelerated` caps gaps at 400 ms and holds frames at least 100 ms. `max-speed` uses a frame hold of 16 ms, but holds it only after the last flush of the replay and after a resize, and holds one video frame (40 ms) before a resize (see the `ghostty-web` README). Each mode holds the final frame (1000 ms, or 500 ms for `max-speed`) [webm.ts:buildReplayTimingOptions]. - The timed replay fails after `REPLAY_TIMEOUT_MS` (5 minutes) [webm.ts:REPLAY_TIMEOUT_MS]. - `renderGridFramesToSvg` needs the profile font size `SVG_FONT_SIZE` (14) and draws fixed 8.4x18 px cells, so equal input gives byte-identical output [svg.ts:renderGridFramesToSvg]. It always embeds the JetBrains Mono subset, and embeds each other bundled face only when a glyph needs it [svg.ts:embeddedFontAssets]. Output past `defaultMaxSvgOutputLength` of the V8 heap limit (the heap limit divided by 8 bytes, at most `MAX_SVG_OUTPUT_LENGTH`) fails with `EXPORT_ERROR` [svg.ts:defaultMaxSvgOutputLength]. diff --git a/src/renderer/ghosttyWeb/README.md b/src/renderer/ghosttyWeb/README.md index faf0446d..1755a247 100644 --- a/src/renderer/ghosttyWeb/README.md +++ b/src/renderer/ghosttyWeb/README.md @@ -30,7 +30,7 @@ This directory holds the `ghostty-web` reference renderer. `GhosttyWebBackend` s - The constructor asserts a non-empty `sessionId`, `profile.name` and `profile.fontFamily`, a positive `fontSize`, and hex colors. With `videoOptions`, `outputDir` must be absolute and the size must use positive integers [backend.ts:GhosttyWebBackend.constructor]. - Replay cannot rewind: `targetSeq` must be >= the last applied seq. The initial dimensions cannot change after the first replay. `input.sessionId` must match the backend [backend.ts:GhosttyWebBackend.replayTo]. - `replayWithTiming` asserts strictly increasing seqs and non-decreasing timestamps [backend.ts:GhosttyWebBackend.replayWithTiming]. -- `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` has no schedule to fall behind, so recorded time ends most of its flushes. A flush ends at the first event that is at least one frame hold of recorded time after the batch's first output. An event that changes no pixel does not end it: input, marker, run-complete, signal and exit events leave the batch as it is, because a flush at each of them would cost a frame per event. A resize and the end of the replay still end it. `max-speed` also spreads at most `MAX_SPEED_FRAME_BUDGET` (2,400) flushes over the recorded span of the events that it applies: the distance between the first output of a flush and the event that ends it is at least the larger of the frame hold and the span divided by 2,400. So recorded time makes about one flush per frame hold for a recording shorter than 38 s and at most about 2,400 flushes for a longer one, whatever its number of output events. A resize ends a flush too, and these flushes do not count against the 2,400, so a recording with many resizes makes about one more flush for each of them. Output that arrives less than that distance after the first output of a flush shares the flush, also when an input or a marker sits between, so a state that the next output of the flush replaces is never painted. A flush waits for one animation frame. `max-speed` holds the frame hold only after the last flush of the replay and before and after a resize, so two flushes that follow each other can paint within one video frame. In `recorded` and `accelerated` mode, every event that changes no pixel ends a flush, and a hold follows every flush. Output events with the same timestamp share a flush at any hold, 0 included, because the time check needs at least 1 ms of recorded time. An event between them that ends a flush splits them anyway: a resize in every mode, and in `recorded` and `accelerated` mode also an input, a marker, a run-complete, a signal or an exit [backend.ts:GhosttyWebBackend.replayWithTiming]. +- `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` has no schedule to fall behind, so recorded time ends most of its flushes. A flush ends at the first event that is at least one frame hold of recorded time after the batch's first output. An event that changes no pixel (input, marker, run-complete, signal or exit) ends a flush only through that rule, because a flush at each of them would cost a frame per event. A resize that finds output waiting and the end of the replay still end it. `max-speed` also spreads at most `MAX_SPEED_FRAME_BUDGET` (2,400) flushes over the recorded span of the events that it applies: the distance between the first output of a flush and the event that ends it is at least the larger of the frame hold and the span divided by 2,400. So recorded time makes about one flush per frame hold for a recording shorter than 38 s and at most about 2,400 flushes for a longer one, whatever its number of output events. A resize that finds output waiting ends a flush too. These flushes do not count against the 2,400, so each resize adds at most one flush, and none when no output is waiting: 100,000 outputs that each have a resize 1 ms after them make 100,000 flushes. Output that arrives less than that distance after the first output of a flush shares the flush, also when an input or a marker sits between, so a state that the next output of the flush replaces is never painted. A flush waits for one animation frame. `max-speed` holds only after the last flush of the replay and around a resize. After the last flush and after a resize it holds the frame hold. Before a resize it holds one video frame (40 ms, from `VIDEO_FPS` = 25), or the frame hold when that is longer, because a screen that stays on the page for less than one video frame can be missing from the video. Two flushes that follow each other can paint within one video frame. In `recorded` and `accelerated` mode, every event that changes no pixel ends a flush, and a hold follows every flush. Output events with the same timestamp share a flush at any hold, 0 included, because the time check needs at least 1 ms of recorded time. An event between them that ends a flush splits them anyway: a resize in every mode, and in `recorded` and `accelerated` mode also an input, a marker, a run-complete, a signal or an exit [backend.ts:GhosttyWebBackend.replayWithTiming]. - `writeBatch(chunks, endsFlush, paintedCap)` takes the chunks of one bridge call. A flush that has more than 1000 chunks needs several calls, and ghostty-web paints on every animation frame, so the page holds the chunks of a call that does not end the flush. The call that ends it (`endsFlush`, set by the backend on the last call only) writes every held chunk in one task, so no frame paints a state between two calls, and then resolves after one frame. ghostty-web calls a write callback one animation frame (about 17 ms) after the write, and waiting on every chunk (`write`) cost one frame per output event. `write()` and `resize()` throw while chunks are held. `discardHeldChunks()` drops them: the backend calls it when a flush fails, because `HostRendererManager` keeps the backend and a retry would otherwise meet them. A `paintedCap` that is not null makes the call that ends the flush add the graphemes on the screen to the set that the missing-glyph check of `replayWithTiming` probes, so a flush is one page call. The call returns the message of a failed note and not an error, because a failed note must not fail the flush. In `replayWithTiming`, the chunks of one flush share one video frame [embeddedHarnessHtml.ts:EMBEDDED_HARNESS_HTML]. - `snapshot()` and `screenshot()` throw until `replayTo()` or `replayWithTiming()` has run once and set the initial size. Both replays accept `targetSeq` -1 for an empty Event Log. A replay over zero events sets `lastAppliedSeq` to `targetSeq` [backend.ts:GhosttyWebBackend.snapshot]. - `missingGlyphs` lists a code point only when a bundled face that lacks it drew it. Chromium draws a cluster with one font: the first stack font that covers all of it, or else the first font of the stack, which is bundled in both built-in profiles and then draws `.notdef` boxes. CDP names the face that drew a probe by its PostScript name, so a family alias or several faces under one family resolve to the right cmap. When one bundled face drew the whole grapheme, every code point that face lacks is listed. A probe with no reported font is judged against every face of the first family. When the grapheme is split over several fonts, each uncovered code point is judged by its own probe, and a mark's probe on its base is compared with a probe of the base alone. Glyphs that a host font draws are not listed. The check runs after the PNG is written, removes its probe spans, and caches each probe text's result for the page, clearing the cache when it would pass `MAX_CACHED_GLYPH_PROBES` (16384) entries. It probes at most `MAX_GLYPH_PROBES` texts and lists at most `MAX_MISSING_GLYPH_CELLS` cells (4096 each). `probeUsedFonts` sends its CDP queries in batches of 256 and stops starting batches after `GLYPH_PROBE_BUDGET_MS` (1000 ms). The result sets `missingGlyphsTruncated` when any of these stops the check early. Family names match without regard to case. A web font whose PostScript name the profile does not bundle fails the check. A failed check is logged at debug level and leaves the field out, so the screenshot still succeeds. `replayWithTiming()` records the viewport graphemes in the page after each output batch and each resize (at most `MAX_CACHED_GLYPH_PROBES`), and the final frame hold judges them once, with the hold as the probe budget, so the check does not lengthen the video. `recordedMissingGlyphs()` returns that result. A failed note or check leaves the record export fields out. A profile whose font stack does not start with a bundled font gets no `missingGlyphs` field [missingGlyphs.ts:probeUsedFonts] [backend.ts:GhosttyWebBackend.screenshot]. diff --git a/src/renderer/ghosttyWeb/backend.ts b/src/renderer/ghosttyWeb/backend.ts index 39beaaa1..e4cd7224 100644 --- a/src/renderer/ghosttyWeb/backend.ts +++ b/src/renderer/ghosttyWeb/backend.ts @@ -748,10 +748,11 @@ export class GhosttyWebBackend implements VideoCapableRendererBackend { let pendingSinceTimestampMs: number | null = null; let scheduleStartedAtMs = 0; let scheduledElapsedMs = 0; - // Max-speed has no pauses, so the screen before a resize reaches the video - // only if it stays on the page for one video frame. A frame hold of 16 ms - // is shorter than that, and the recorder then misses the screen in some - // runs. Recorded and accelerated replays keep their pauses and their hold. + // Max-speed has no pauses, so a screen that stays on the page for less + // than one video frame before a resize can be missing from the video. A + // frame hold of 16 ms is shorter than that, and the recorder misses the + // screen in some runs. Recorded and accelerated replays keep their pauses + // and their hold. const preResizeHoldMs = timing.mode === 'max-speed' ? Math.max(resolvedMinFrameHoldMs, VIDEO_FRAME_MS) From a702ebf660703433f50d202b8d11d68cb8703ae3 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Mon, 9 Nov 2026 16:04:34 +0000 Subject: [PATCH 25/26] fix: assert that a video frame lasts a whole number of milliseconds The max-speed wait before a resize passes VIDEO_FRAME_MS to waitForDelay, which needs a whole number. A VIDEO_FPS that does not divide 1000 now fails when the module loads, with a message that names the cause. --- src/renderer/ghosttyWeb/backend.ts | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/renderer/ghosttyWeb/backend.ts b/src/renderer/ghosttyWeb/backend.ts index e4cd7224..61accee7 100644 --- a/src/renderer/ghosttyWeb/backend.ts +++ b/src/renderer/ghosttyWeb/backend.ts @@ -165,6 +165,11 @@ const MAX_SPEED_FRAME_BUDGET = 2_400; */ const VIDEO_FPS = 25; const VIDEO_FRAME_MS = 1000 / VIDEO_FPS; +// waitForDelay takes whole milliseconds, and the wait before a resize uses this. +invariant( + Number.isInteger(VIDEO_FRAME_MS), + 'VIDEO_FPS must divide 1000 so that a video frame is a whole number of ms', +); const RAF_TIMEOUT_MS = 5_000; let servedAssetsPromise: Promise< From c8345e640e723390a03ffb3c92a0f735c7e685a8 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Mon, 9 Nov 2026 16:04:34 +0000 Subject: [PATCH 26/26] docs: name the mode of each flush rule in the ghostty-web README An event that changes no pixel ends a flush only through the time rule in max-speed mode. In recorded and accelerated mode it ends a flush directly. --- src/renderer/ghosttyWeb/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/renderer/ghosttyWeb/README.md b/src/renderer/ghosttyWeb/README.md index 1755a247..5f94a4bf 100644 --- a/src/renderer/ghosttyWeb/README.md +++ b/src/renderer/ghosttyWeb/README.md @@ -30,7 +30,7 @@ This directory holds the `ghostty-web` reference renderer. `GhosttyWebBackend` s - The constructor asserts a non-empty `sessionId`, `profile.name` and `profile.fontFamily`, a positive `fontSize`, and hex colors. With `videoOptions`, `outputDir` must be absolute and the size must use positive integers [backend.ts:GhosttyWebBackend.constructor]. - Replay cannot rewind: `targetSeq` must be >= the last applied seq. The initial dimensions cannot change after the first replay. `input.sessionId` must match the backend [backend.ts:GhosttyWebBackend.replayTo]. - `replayWithTiming` asserts strictly increasing seqs and non-decreasing timestamps [backend.ts:GhosttyWebBackend.replayWithTiming]. -- `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` has no schedule to fall behind, so recorded time ends most of its flushes. A flush ends at the first event that is at least one frame hold of recorded time after the batch's first output. An event that changes no pixel (input, marker, run-complete, signal or exit) ends a flush only through that rule, because a flush at each of them would cost a frame per event. A resize that finds output waiting and the end of the replay still end it. `max-speed` also spreads at most `MAX_SPEED_FRAME_BUDGET` (2,400) flushes over the recorded span of the events that it applies: the distance between the first output of a flush and the event that ends it is at least the larger of the frame hold and the span divided by 2,400. So recorded time makes about one flush per frame hold for a recording shorter than 38 s and at most about 2,400 flushes for a longer one, whatever its number of output events. A resize that finds output waiting ends a flush too. These flushes do not count against the 2,400, so each resize adds at most one flush, and none when no output is waiting: 100,000 outputs that each have a resize 1 ms after them make 100,000 flushes. Output that arrives less than that distance after the first output of a flush shares the flush, also when an input or a marker sits between, so a state that the next output of the flush replaces is never painted. A flush waits for one animation frame. `max-speed` holds only after the last flush of the replay and around a resize. After the last flush and after a resize it holds the frame hold. Before a resize it holds one video frame (40 ms, from `VIDEO_FPS` = 25), or the frame hold when that is longer, because a screen that stays on the page for less than one video frame can be missing from the video. Two flushes that follow each other can paint within one video frame. In `recorded` and `accelerated` mode, every event that changes no pixel ends a flush, and a hold follows every flush. Output events with the same timestamp share a flush at any hold, 0 included, because the time check needs at least 1 ms of recorded time. An event between them that ends a flush splits them anyway: a resize in every mode, and in `recorded` and `accelerated` mode also an input, a marker, a run-complete, a signal or an exit [backend.ts:GhosttyWebBackend.replayWithTiming]. +- `replayWithTiming` waits until each event's place on a schedule built from the recorded gaps: `recorded` keeps them, `accelerated` caps each at `maxGapMs`, and `max-speed` sets them to 0. It measures that schedule with `performance.now()`, so frame holds and bridge calls use up idle time instead of adding to it. In `recorded` and `accelerated` mode it flushes pending output once the next event is at least one frame hold after the batch's first output, unless the replay runs behind the schedule, in which case the batch grows until it catches up. `max-speed` has no schedule to fall behind, so recorded time ends most of its flushes. In `max-speed` mode, a flush ends at the first event that is at least one frame hold of recorded time after the batch's first output. Also in `max-speed` mode, an event that changes no pixel (input, marker, run-complete, signal or exit) ends a flush only through that rule, because a flush at each of them would cost a frame per event. In `recorded` and `accelerated` mode such an event ends a flush directly (see below). A resize that finds output waiting and the end of the replay end a flush in every mode. `max-speed` also spreads at most `MAX_SPEED_FRAME_BUDGET` (2,400) flushes over the recorded span of the events that it applies: the distance between the first output of a flush and the event that ends it is at least the larger of the frame hold and the span divided by 2,400. So in `max-speed` mode, recorded time makes about one flush per frame hold for a recording shorter than 38 s and at most about 2,400 flushes for a longer one, whatever its number of output events. A resize that finds output waiting ends a flush too. These flushes do not count against the 2,400, so each resize adds at most one flush, and none when no output is waiting: 100,000 outputs that each have a resize 1 ms after them make 100,000 flushes. In `max-speed` mode, output that arrives less than that distance after the first output of a flush shares the flush, also when an input or a marker sits between, so a state that the next output of the flush replaces is never painted. A flush waits for one animation frame. `max-speed` holds only after the last flush of the replay and around a resize. After the last flush and after a resize it holds the frame hold. Before a resize it holds one video frame (40 ms, from `VIDEO_FPS` = 25), or the frame hold when that is longer, because a screen that stays on the page for less than one video frame can be missing from the video. Two flushes that follow each other can paint within one video frame. In `recorded` and `accelerated` mode, every event that changes no pixel ends a flush, and a hold follows every flush. Output events with the same timestamp share a flush at any hold, 0 included, because the time check needs at least 1 ms of recorded time. An event between them that ends a flush splits them anyway: a resize in every mode, and in `recorded` and `accelerated` mode also an input, a marker, a run-complete, a signal or an exit [backend.ts:GhosttyWebBackend.replayWithTiming]. - `writeBatch(chunks, endsFlush, paintedCap)` takes the chunks of one bridge call. A flush that has more than 1000 chunks needs several calls, and ghostty-web paints on every animation frame, so the page holds the chunks of a call that does not end the flush. The call that ends it (`endsFlush`, set by the backend on the last call only) writes every held chunk in one task, so no frame paints a state between two calls, and then resolves after one frame. ghostty-web calls a write callback one animation frame (about 17 ms) after the write, and waiting on every chunk (`write`) cost one frame per output event. `write()` and `resize()` throw while chunks are held. `discardHeldChunks()` drops them: the backend calls it when a flush fails, because `HostRendererManager` keeps the backend and a retry would otherwise meet them. A `paintedCap` that is not null makes the call that ends the flush add the graphemes on the screen to the set that the missing-glyph check of `replayWithTiming` probes, so a flush is one page call. The call returns the message of a failed note and not an error, because a failed note must not fail the flush. In `replayWithTiming`, the chunks of one flush share one video frame [embeddedHarnessHtml.ts:EMBEDDED_HARNESS_HTML]. - `snapshot()` and `screenshot()` throw until `replayTo()` or `replayWithTiming()` has run once and set the initial size. Both replays accept `targetSeq` -1 for an empty Event Log. A replay over zero events sets `lastAppliedSeq` to `targetSeq` [backend.ts:GhosttyWebBackend.snapshot]. - `missingGlyphs` lists a code point only when a bundled face that lacks it drew it. Chromium draws a cluster with one font: the first stack font that covers all of it, or else the first font of the stack, which is bundled in both built-in profiles and then draws `.notdef` boxes. CDP names the face that drew a probe by its PostScript name, so a family alias or several faces under one family resolve to the right cmap. When one bundled face drew the whole grapheme, every code point that face lacks is listed. A probe with no reported font is judged against every face of the first family. When the grapheme is split over several fonts, each uncovered code point is judged by its own probe, and a mark's probe on its base is compared with a probe of the base alone. Glyphs that a host font draws are not listed. The check runs after the PNG is written, removes its probe spans, and caches each probe text's result for the page, clearing the cache when it would pass `MAX_CACHED_GLYPH_PROBES` (16384) entries. It probes at most `MAX_GLYPH_PROBES` texts and lists at most `MAX_MISSING_GLYPH_CELLS` cells (4096 each). `probeUsedFonts` sends its CDP queries in batches of 256 and stops starting batches after `GLYPH_PROBE_BUDGET_MS` (1000 ms). The result sets `missingGlyphsTruncated` when any of these stops the check early. Family names match without regard to case. A web font whose PostScript name the profile does not bundle fails the check. A failed check is logged at debug level and leaves the field out, so the screenshot still succeeds. `replayWithTiming()` records the viewport graphemes in the page after each output batch and each resize (at most `MAX_CACHED_GLYPH_PROBES`), and the final frame hold judges them once, with the hold as the probe budget, so the check does not lengthen the video. `recordedMissingGlyphs()` returns that result. A failed note or check leaves the record export fields out. A profile whose font stack does not start with a bundled font gets no `missingGlyphs` field [missingGlyphs.ts:probeUsedFonts] [backend.ts:GhosttyWebBackend.screenshot].