diff --git a/docs/USAGE.md b/docs/USAGE.md index 582a98cb..57fc024c 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -264,6 +264,8 @@ Semantic `snapshot`, screen-hash, and render-backed `wait` paths prefer `libghos On a session that has printed nothing yet, `snapshot` returns an empty screen of the session's size and `screenshot` returns a blank image. `record export` gives a header-only asciicast, a blank WebM or a blank SVG of the session's size, with `durationMs: 0`. An animated SVG of such a session reports `durationMs: 1000`, the hold of its one blank frame. All of them report `capturedAtSeq: -1`, and the default files are named `screenshot--1-.png`, `recording--1-.` and `recording--1-svg-[-animated].svg`. +A live `screenshot`, and a live `snapshot --renderer ghostty-web`, first replay the session's event log into a renderer of the host. With `--renderer libghostty-vt`, the screenshot replays the log a second time, in a `ghostty-web` renderer that it boots for the screenshot. The more events the log holds, the longer that takes. The host reports progress when a renderer has booted and after each slice of a replay, and a slice aims at half a second. The command waits for as long as the replay moves forward, also when the replay belongs to a command that started earlier. It fails with `HOST_TIMEOUT` (exit 5) after 5 s in which the host has reported no progress, for example when the renderer hangs. + The screenshot hides the cursor unless you pass `--show-cursor`. `--hide-cursor` is the default, and the result reports `cursorVisible` for the choice. ```bash diff --git a/src/host/README.md b/src/host/README.md index ef4345cd..10ad7d13 100644 --- a/src/host/README.md +++ b/src/host/README.md @@ -2,7 +2,7 @@ ## Overview -`src/host` holds the per-session host and the CLI-side code that starts, reaches and cleans up hosts. `lifecycle.ts` allocates a session, spawns a detached host process and destroys, lists or reconciles sessions. `manifestWriter.ts` orders the host's manifest writes. `programGroup.ts` records the program's session token in `program.json` and decides before each kill of the program's process group whether the group is still the program's. `hostMain.ts` is the host process: it spawns the PTY, appends all eleven event types (`output`, `input_text`, `input_paste`, `input_keys`, `input_mouse`, `input_run`, `run_complete`, `resize`, `signal`, `exit` and `marker`) to `events.jsonl` through `EventLog`, serves RPC on a Unix socket and renders through `HostRendererManager`. `mouseRpc.ts` serves the `mouse` RPC, and `mouseInput.ts` stages its held-button state. `runCompletionCoordinator.ts` and `runCompletionSentinel.ts` detect when a command sent with `run` has finished. The host is an internal detail. The CLI JSON envelope in `src/protocol/` is the stable surface. +`src/host` holds the per-session host and the CLI-side code that starts, reaches and cleans up hosts. `lifecycle.ts` allocates a session, spawns a detached host process and destroys, lists or reconciles sessions. `manifestWriter.ts` orders the host's manifest writes. `programGroup.ts` records the program's session token in `program.json` and decides before each kill of the program's process group whether the group is still the program's. `hostMain.ts` is the host process: it spawns the PTY, appends all eleven event types (`output`, `input_text`, `input_paste`, `input_keys`, `input_mouse`, `input_run`, `run_complete`, `resize`, `signal`, `exit` and `marker`) to `events.jsonl` through `EventLog`, serves RPC on a Unix socket and renders through `HostRendererManager`. `HostRendererManager` replays a long log in slices with `replayInSlices` from `src/renderer/replaySlices.ts`, so that a slow replay shows progress. `mouseRpc.ts` serves the `mouse` RPC, and `mouseInput.ts` stages its held-button state. `runCompletionCoordinator.ts` and `runCompletionSentinel.ts` detect when a command sent with `run` has finished. The host is an internal detail. The CLI JSON envelope in `src/protocol/` is the stable surface. ## Interface @@ -35,9 +35,9 @@ - `createMouseRpcHandler(deps)` returns the `mouse` handler `(request, rendererName)`, which resolves a `MouseResult` (`reported`, `bytesWritten`, `seq`). `MouseRpcDependencies` injects the renderer manager (in `runHost` a second `HostRendererManager` that only mouse input uses, so a render on another renderer or profile never replaces the encoder's backend and its same-cell motion state), the profile, the libghostty-vt probe, the replay loader, the commandable check, the PTY writer and the `input_mouse` append. The handler keeps the held buttons between requests. [mouseRpc.ts:createMouseRpcHandler] [mouseRpc.ts:MouseRpcDependencies] - `prepareHeldMouseButtons(currentHeld, input)` returns the next held set, `anyButtonPressed` and the `dragButton` that a button-free `move` reports, without changing `currentHeld`. [mouseInput.ts:prepareHeldMouseButtons] - `buildReplayInput(sessionId, manifest, events, ...)` builds a validated `ReplayInput` (type from `src/renderer/types.ts`) with the initial size and target sequence number. [replay.ts:buildReplayInput] -- `RpcServer(socketPath, handlers)` serves newline-delimited JSON RPC on a Unix socket, one request per connection, with `listen()` and `close()`. `MethodHandler` and `MethodContext` type the handlers. `MethodContext.signal` aborts when the client socket closes. [rpcServer.ts:RpcServer] [rpcServer.ts:MethodHandler] [rpcServer.ts:MethodContext] +- `RpcServer(socketPath, handlers)` serves newline-delimited JSON RPC on a Unix socket, one request per connection, with `listen()` and `close()`. `MethodHandler` and `MethodContext` type the handlers. `MethodContext.signal` aborts when the client socket closes. `MethodContext.keepAlive()` writes one space ahead of the response line, which tells the client that the handler still makes progress, and does nothing once the client has gone or the response is sent. [rpcServer.ts:RpcServer] [rpcServer.ts:MethodHandler] [rpcServer.ts:MethodContext] - `sendRpc(socketPath, method, params?, timeoutMs?, signal?)` sends one request to a host and returns the validated result. For a path under the socket root, it connects only after `socketRootExists` read the root as private, so no other user can have put the socket there. When the root is missing or not private, or the socket is missing (`HOST_UNREACHABLE` with errno `ENOENT`), it sends the request to `legacySocketPath(socketPath)`, where hosts of older versions listen, if `isOwnLegacySocket` accepts that path. Otherwise it throws the root's `STORAGE_WRITE_ERROR` or the missing socket's error. Transport and response errors are `CliError` values (class from `src/cli/errors.ts`). An aborted `signal` rejects with the error from `makeAbortError`: the signal's `reason` when it is an `Error`, else a plain `Error` named `AbortError`. A failed `invariant` throws an `AssertionError`. [rpcClient.ts:sendRpc] [src/util/abort.ts:makeAbortError] -- `HostRendererManager` keeps one current renderer backend. It disposes and replaces that backend when the renderer name or profile changes. Methods: `getBackend`, `withBackend` (replays to a `ReplayInput` when the input is non-null, including an empty one at `targetSeq` -1, then runs the operation), `screenshotPath`, `dispose`, `isBooted`, `isBootInFlight`, `getCurrentRendererName` and `getCurrentProfileName`. [renderer.ts:HostRendererManager] +- `HostRendererManager` keeps one current renderer backend. It disposes and replaces that backend when the renderer name or profile changes. Methods: `getBackend`, `withBackend(rendererName, profile, loadReplayInput, operation, onProgress?)` (replays to a `ReplayInput` when the input is non-null, including an empty one at `targetSeq` -1, then runs the operation, and calls `onProgress` once the backend has booted and after each call of `replayTo`, also for a replay of another call that this one waits for), `screenshotPath`, `dispose`, `isBooted`, `isBootInFlight`, `getCurrentRendererName` and `getCurrentProfileName`. [renderer.ts:HostRendererManager] - `RunCompletionCoordinator(appender)` tracks waited runs. Methods: `prepareWaitedRun`, `registerWaitedRun` (returns the shell `postamble`, the `sentinel` and a single-use `wait(timeoutMs, options?)`), `ingestPtyData`, `flushPtyDataOnExit` and `resetForExit`. Related types: `RunCompletionEventAppender`, `PreparedWaitedRun`, `RegisteredWaitedRunCompletion`, `RunCompletionWaitOptions` and `TimedRunCompletionWaitResult` (`completed`, `exited` or `timeout`). [runCompletionCoordinator.ts:RunCompletionCoordinator] [runCompletionCoordinator.ts:RunCompletionEventAppender] [runCompletionCoordinator.ts:TimedRunCompletionWaitResult] - `RunCompletionSentinelScanner` splits PTY output into `SentinelPiece` values (`output` or `run_complete`) by finding registered sentinel frames. `RunCompletionPostambleEchoSanitizer` removes the shell's echo of the injected postamble from the output. [runCompletionSentinel.ts:RunCompletionSentinelScanner] [runCompletionSentinel.ts:RunCompletionPostambleEchoSanitizer] [runCompletionSentinel.ts:SentinelPiece] - `RUN_COMPLETE_SENTINEL_PREFIX`, `RUN_COMPLETE_SENTINEL_SUFFIX`, `RUN_MARKER_PATTERN`, `assertRunMarker`, `buildRunCompleteSentinel` and `buildRunCompleteSignalSentinel` define and build the sentinel frames and run markers. [runCompletionSentinel.ts:RUN_MARKER_PATTERN] [runCompletionSentinel.ts:buildRunCompleteSentinel] [runCompletionSentinel.ts:buildRunCompleteSignalSentinel] @@ -63,9 +63,11 @@ - `allocateSession` throws `INVALID_DIMENSIONS` for non-positive `cols` or `rows`, `STORAGE_READ_ERROR` when `cwd` is not a directory and `INVALID_INPUT` when the shell path is not a file. An empty `command` defaults to the shell path. It creates the session directory and calls `launch` only after these checks pass. [lifecycle.ts:allocateSession] - `buildReplayInput` validates the manifest with `SessionRecordSchema` and requires its `sessionId` to match. It validates the events with `validateEventRecords` unless `trustValidated` is set. `targetSeq` is -1 for no events, else from 0 to the last `seq`. [replay.ts:buildReplayInput] - `RpcServer.listen` removes a stale socket file (no answer within 1,000 ms, `ECONNREFUSED` or `ENOENT`) and throws when another server is listening. It asserts a socket path of at most 104 characters. [rpcServer.ts:RpcServer] -- `RpcServer` caps a request at 1,048,576 characters. `sendRpc` caps a response at `MAX_RPC_RESPONSE_CHARS` (67,108,864 characters, from `src/protocol/messages.ts`), so a structured snapshot with cells of a 500x200 screen (about 5.6 million characters when every cell is styled) fits. The server answers a success response above that cap with `PROTOCOL_ERROR` "RPC response takes N characters, over the 67108864-character limit.", with `responseChars` and `maxResponseChars` in `details`. `sendRpc` stops reading a response from an older host, which has no such check, at the cap and reports `PROTOCOL_ERROR` "RPC response is over the 67108864-character limit." with `maxResponseChars` in `details`. The server validates the request with `RpcRequestSchema`, the params and result with `RpcMethodSchemas`, and the response with `RpcResponseSchema` (all from `src/protocol/messages.ts`). A handler that throws a `CliError` gets an error response with its code, message and, when it has any, its `details`. It leaves `details` out when they do not serialize to a JSON object (a `BigInt`, a circular object, a `toJSON` that returns something else) or take more than 65,536 characters. It cuts an error message to 65,536 characters, without splitting a surrogate pair, and ends it with `[message truncated]`. A request id over 128 characters fails `RpcRequestSchema`, and the error response uses the id `unknown`. So an error response stays under 1,048,576 characters, which a CLI from before the larger response cap also reads, and keeps the error's code. `sendRpc` rebuilds that `CliError` with the same details. Any other thrown error becomes `RPC_ERROR` without details. If building or sending the response throws, the server logs the error and answers `RPC_ERROR` "RPC request handling failed unexpectedly.", so the host keeps running. Unknown methods get an error response. [rpcServer.ts:RpcServer] [rpcClient.ts:sendRpc] -- `sendRpc` waits 5,000 ms by default, as a socket inactivity timeout. The `snapshot` command passes 60,000 ms when it asks for cells, because the host sends nothing until it has built every cell. `sendRpc` scans each response chunk once for the newline. It requires the response `id` to match the request. It maps `ECONNREFUSED`, `ENOENT` and `ECONNRESET` to `HOST_UNREACHABLE`, a socket timeout to `HOST_TIMEOUT` and other transport errors to `RPC_ERROR`. [rpcClient.ts:sendRpc] +- `RpcServer` caps a request at 1,048,576 characters. `sendRpc` caps a response at `MAX_RPC_RESPONSE_CHARS` (67,108,864 characters, from `src/protocol/messages.ts`), so a structured snapshot with cells of a 500x200 screen (about 5.6 million characters when every cell is styled) fits. The server answers a success response above that cap with `PROTOCOL_ERROR` "RPC response takes N characters, over the 67108864-character limit.", with `responseChars` and `maxResponseChars` in `details`. `sendRpc` stops reading a response from an older host, which has no such check, at the cap and reports `PROTOCOL_ERROR` "RPC response is over the 67108864-character limit." with `maxResponseChars` in `details`. The cap counts the response line alone, as the server does: `sendRpc` leaves out the spaces that a host writes ahead of it as keep-alives, so a response at the cap still arrives after them. The server validates the request with `RpcRequestSchema`, the params and result with `RpcMethodSchemas`, and the response with `RpcResponseSchema` (all from `src/protocol/messages.ts`). A handler that throws a `CliError` gets an error response with its code, message and, when it has any, its `details`. It leaves `details` out when they do not serialize to a JSON object (a `BigInt`, a circular object, a `toJSON` that returns something else) or take more than 65,536 characters. It cuts an error message to 65,536 characters, without splitting a surrogate pair, and ends it with `[message truncated]`. A request id over 128 characters fails `RpcRequestSchema`, and the error response uses the id `unknown`. So an error response stays under 1,048,576 characters, which a CLI from before the larger response cap also reads, and keeps the error's code. `sendRpc` rebuilds that `CliError` with the same details. Any other thrown error becomes `RPC_ERROR` without details. If building or sending the response throws, the server logs the error and answers `RPC_ERROR` "RPC request handling failed unexpectedly.", so the host keeps running. Unknown methods get an error response. [rpcServer.ts:RpcServer] [rpcClient.ts:sendRpc] +- `sendRpc` waits 5,000 ms by default, as a socket inactivity timeout: each byte from the host starts the wait over. So a host that reports progress with `keepAlive` can take longer than 5,000 ms to answer, and a host that sends nothing for 5,000 ms ends in `HOST_TIMEOUT`. The `snapshot` command passes 60,000 ms when it asks for cells, because the host sends nothing while it builds every cell. `sendRpc` scans each response chunk once for the newline. It requires the response `id` to match the request. It maps `ECONNREFUSED`, `ENOENT` and `ECONNRESET` to `HOST_UNREACHABLE`, a socket timeout to `HOST_TIMEOUT` and other transport errors to `RPC_ERROR`. [rpcClient.ts:sendRpc] - `HostRendererManager` runs backend operations one at a time. `withBackend` requires the replay input `sessionId` to match, and recreates the backend when the initial size changes. [renderer.ts:HostRendererManager] +- `withBackend` replays through `replayInSlices` (`src/renderer/replaySlices.ts`) when more than one slice of events is left. Each slice is a `replayTo` call with the events of the slice and the target of the whole replay, so a backend ends at the last event of the slice and does not rewind. A slice holds at most as many events as the size that the time of the last slice gives, which starts at 32 (`FIRST_SLICE_EVENTS`), aims at 500 ms (`TARGET_SLICE_MS`) and changes by at most four times, and at most 1 MiB of output (`MAX_SLICE_BYTES`, except that one event larger than that is a slice alone). A slice also holds at most 16 events that follow output (`MAX_SLICE_FLUSHES`): a renderer flushes its output at each of them and waits for a frame, so a slice of thousands of such pairs would stay silent for minutes. Only a slice that the count of events ended changes the size. The last call gets the whole input, because a backend keeps its last input (the libghostty-vt screenshot replays it in a second renderer), and it replays at most one slice of events. `HostRendererManager` keeps the position that the last replay reported, and a backend that is further on, after a `mouse` request, makes the first slice skip its events. Any split of the events gives the same backend state. [src/renderer/replaySlices.ts:replayInSlices] [src/renderer/replaySlices.ts:sliceEnd] [renderer.ts:HostRendererManager] +- The `snapshot`, `screenshot` and `run` handlers pass `MethodContext.keepAlive` to `withBackend` as `onProgress`, so a client waits for a replay that moves forward, also when the replay belongs to a request ahead of it in the queue. `keepAlive` is never called on a timer, so a renderer that hangs ends the client's wait with `HOST_TIMEOUT` 5 s after its last slice. A response line may start with spaces, which JSON allows and `sendRpc` of older versions parses. `withBackend` also gives its operation a `reportProgress` function, which does what the replay does after each slice: it calls the `onProgress` of every call that waits, the caller's own included. The `screenshot` handler passes it as `onReplayProgress` of the screenshot options, so the libghostty-vt backend reports the boot and the slices of the replay in its second renderer to the screenshot and to every command that waits behind it in the renderer queue. `waitForRender` and `mouse` pass nothing: a render wait answers at its own timeout, and `mouse` replays libghostty-vt only. The time of a boot and of one slice is still silent. [hostMain.ts:runHost] [rpcServer.ts:RpcServer] - A waited `run` calls `EventLog.startAppend`, `registerWaitedRun` and `pty.write` with no `await` in between, then awaits `written`. When `registerWaitedRun` or `pty.write` throws, it waits for `written` to settle and rethrows that error, so a failed write never becomes an unhandled rejection. So no other input reaches the PTY or the log between the command's `seq` and its write, and the marker is registered before the command can print. [hostMain.ts:runHost] - `RUN_MARKER_PATTERN` is `/^__AT_MARKER_([0-9a-f]{32})__$/u`. `registerWaitedRun` requires a unique marker and a non-negative integer `inputRunSeq`. `buildRunCompletePostamble` limits the postamble to 64 characters before its newline, and the postamble must not contain the marker. [runCompletionSentinel.ts:RUN_MARKER_PATTERN] [runCompletionCoordinator.ts:RunCompletionCoordinator] [runCompletionCoordinator.ts:buildRunCompletePostamble] - A timed-out `wait` returns `{ kind: 'timeout' }`. An aborted `wait` rejects with an abort error. Both keep the sentinel registered, so the later completion bytes stay out of the output. [runCompletionCoordinator.ts:RunCompletionCoordinator] @@ -87,7 +89,7 @@ - `EventLog.open` opens the file in append mode and runs `chmod 0o600`. `append` writes with `fileHandle.appendFile`, one call at a time, and never takes the file past its size limit. `close` calls `fileHandle.sync()` and then closes the file. [eventLog.ts:EventLog] - `RpcServer.listen` uses `net.createServer`, listens on the socket path and runs `chmod 0o600` on the socket. `close` closes the server and unlinks the socket. [rpcServer.ts:RpcServer] - `sendRpc` connects with `net.connect` and writes one JSON line. [rpcClient.ts:sendRpc] -- `HostRendererManager` boots, replays and disposes backends from the injected factory. In `runHost` that factory is `createRendererBackend` (`src/renderer/registry.ts`), with the session's creation size as the libghostty-vt initial size. `screenshotPath` creates `/screenshots` with `mkdirSync`. [renderer.ts:HostRendererManager] +- `HostRendererManager` boots, replays (in slices) and disposes backends from the injected factory. In `runHost` that factory is `createRendererBackend` (`src/renderer/registry.ts`), with the session's creation size as the libghostty-vt initial size. `screenshotPath` creates `/screenshots` with `mkdirSync`. [renderer.ts:HostRendererManager] - `RunCompletionCoordinator` uses `crypto.randomUUID` and `crypto.randomBytes`. It appends `output` and `run_complete` events only through its injected appender. [runCompletionCoordinator.ts:RunCompletionCoordinator] - `createSerialManifestWriter` does I/O only through its `write` argument. [manifestWriter.ts:createSerialManifestWriter] - `createMouseRpcHandler` does I/O only through its dependencies. `mouseInput.ts` does no I/O. [mouseRpc.ts:createMouseRpcHandler] [mouseInput.ts:prepareHeldMouseButtons] @@ -96,5 +98,5 @@ ## Verification - Run: `npx vitest run test/unit/host` -- Run: `npx vitest run --maxWorkers=1 test/integration/lifecycle.test.ts test/integration/host-renderer-rpc.test.ts test/integration/wait-render.test.ts test/integration/run.test.ts test/integration/mouse.test.ts` +- Run: `npx vitest run --maxWorkers=1 test/integration/lifecycle.test.ts test/integration/host-renderer-rpc.test.ts test/integration/wait-render.test.ts test/integration/run.test.ts test/integration/mouse.test.ts test/integration/slow-replay.test.ts` - Run: `npm run typecheck` diff --git a/src/host/hostMain.ts b/src/host/hostMain.ts index e1b5857d..e7acae24 100644 --- a/src/host/hostMain.ts +++ b/src/host/hostMain.ts @@ -393,7 +393,10 @@ export async function runHost( appendMouseEvent: (payload) => eventLog.append('input_mouse', payload), }); - const replayRendererThroughSeq = async (targetSeq: number): Promise => { + const replayRendererThroughSeq = async ( + targetSeq: number, + keepAlive: () => void, + ): Promise => { invariant( Number.isInteger(targetSeq) && targetSeq >= 0, 'targetSeq must be a non-negative integer', @@ -421,6 +424,7 @@ export async function runHost( 'renderer snapshot must include the run-complete event sequence', ); }, + keepAlive, ); }; @@ -638,7 +642,7 @@ export async function runHost( rendererBootInFlight, }; }, - snapshot: async (params: unknown) => { + snapshot: async (params: unknown, context) => { const { format: requestedFormat, includeScrollback: requestedIncludeScrollback, @@ -672,6 +676,7 @@ export async function runHost( }), rendererBackend: backend.rendererBackend, }), + context.keepAlive, ); return await captureSnapshotResult({ sessionDir: sessDir, @@ -681,7 +686,7 @@ export async function runHost( expectedSessionId: sessionId, }); }, - screenshot: async (params: unknown) => { + screenshot: async (params: unknown, context) => { const { profile: requestedProfileName, rendererName: requestedRendererName, @@ -713,14 +718,18 @@ export async function runHost( rendererName, profile, () => loadReplayInput(), - async (backend) => + async (backend, _replayInput, reportProgress) => await captureScreenshotResult({ backend, sessionDir: sessDir, profileName: profile.name, expectedSessionId: sessionId, ...(showCursor === undefined ? {} : { showCursor }), + // Not `context.keepAlive`: a command that waits behind this + // screenshot in the renderer queue must hear of the progress too. + onReplayProgress: reportProgress, }), + context.keepAlive, ); }, type: async (params: unknown) => { @@ -822,7 +831,7 @@ export async function runHost( if (waitResult.kind === 'completed') { try { - await replayRendererThroughSeq(waitResult.seq); + await replayRendererThroughSeq(waitResult.seq, context.keepAlive); } catch { // The run already completed and was committed to the event log. Do not // turn a best-effort renderer catch-up failure into a command retry diff --git a/src/host/renderer.ts b/src/host/renderer.ts index a0478e3c..f1e0ed8c 100644 --- a/src/host/renderer.ts +++ b/src/host/renderer.ts @@ -4,6 +4,7 @@ import { dirname, isAbsolute, relative, resolve, sep } from 'node:path'; import type { RendererBackend } from '../renderer/backend.js'; import type { RendererName } from '../renderer/names.js'; import { hashProfile } from '../renderer/profiles.js'; +import { replayInSlices } from '../renderer/replaySlices.js'; import type { RenderProfileConfig, ReplayInput } from '../renderer/types.js'; import { invariant } from '../util/assert.js'; @@ -45,6 +46,11 @@ export class HostRendererManager { private bootPromise: Promise | null = null; private lifecyclePromise: Promise = Promise.resolve(); private screenshotsDirectoryCreated = false; + // Where the last replay left the current backend. A mouse request can move + // the backend further, so this is where it was at least. + private appliedSeq = -1; + // One entry for each `withBackend` call that waits or runs. + private readonly progressListeners = new Set<{ notify: () => void }>(); constructor(options: HostRendererManagerOptions) { assertNonEmptyString(options.sessionId, 'sessionId'); @@ -72,6 +78,19 @@ export class HostRendererManager { ); } + /** + * Run `operation` on the current backend, after the replay to the input. + * + * `onProgress` runs each time a replay moves forward, the replay of this + * call or of an earlier call that this one waits for. A request passes its + * `keepAlive`, so that it does not time out while the work it needs goes on. + * + * `operation` gets `reportProgress` as its third argument. It tells every + * call that waits, this one included, that work moves forward, for an + * operation that is slow by itself (a libghostty-vt screenshot replays the + * log in a second renderer). A call that waits behind the operation hears + * of it only this way. + */ async withBackend( rendererName: RendererName, profile: RenderProfileConfig, @@ -79,7 +98,9 @@ export class HostRendererManager { operation: ( backend: RendererBackend, replayInput: ReplayInput | null, + reportProgress: () => void, ) => T | Promise, + onProgress?: () => void, ): Promise { assertNonEmptyString(rendererName, 'rendererName'); assertNonEmptyString(profile.name, 'profile name'); @@ -91,7 +112,11 @@ export class HostRendererManager { // Load inside the lock: work queued earlier, such as a mouse request that // catches up to the whole log, can leave the backend past any input // captured before the wait, and a stateful backend cannot rewind. - return this.runExclusive(async () => { + const listener = onProgress === undefined ? null : { notify: onProgress }; + if (listener !== null) { + this.progressListeners.add(listener); + } + const queued = this.runExclusive(async () => { const replayInput = loadReplayInput(); if (replayInput !== null) { invariant( @@ -111,17 +136,35 @@ export class HostRendererManager { } const backend = await this.ensureBackend(rendererName, profile); + // A booted backend is progress too: a replay starts only after the boot. + this.reportProgress(); if (replayInput !== null) { - await backend.replayTo(replayInput); + await replayInSlices(backend, replayInput, this.appliedSeq, (state) => { + this.appliedSeq = state.lastSeq; + this.reportProgress(); + }); this.cachedInitialCols = replayInput.initialCols; this.cachedInitialRows = replayInput.initialRows; } - return await operation(backend, replayInput); + return await operation(backend, replayInput, () => { + this.reportProgress(); + }); + }); + return queued.finally(() => { + if (listener !== null) { + this.progressListeners.delete(listener); + } }); } + private reportProgress(): void { + for (const listener of this.progressListeners) { + listener.notify(); + } + } + screenshotPath(profileName: string): string { assertNonEmptyString(profileName, 'profileName'); invariant( @@ -248,6 +291,7 @@ export class HostRendererManager { this.currentProfileName = null; this.cachedInitialCols = null; this.cachedInitialRows = null; + this.appliedSeq = -1; this.bootPromise = null; if (backend === null) { diff --git a/src/host/rpcClient.ts b/src/host/rpcClient.ts index 58c291f8..994cc12a 100644 --- a/src/host/rpcClient.ts +++ b/src/host/rpcClient.ts @@ -26,6 +26,7 @@ import { ResourceScope } from '../util/resourceScope.js'; import { isOwnLegacySocket, socketRootExists } from './socketRoot.js'; const DEFAULT_TIMEOUT_MS = 5_000; +const LEADING_SPACES = /^ +/; const HOST_UNREACHABLE_SOCKET_CODES = new Set([ 'ECONNREFUSED', 'ENOENT', @@ -228,11 +229,21 @@ async function sendRpcAt( rejectWithTransportError(error); }); - socket.on('data', (chunk: string) => { + socket.on('data', (received: string) => { if (responseHandled) { return; } + // A host may write spaces ahead of the response as keep-alives. The + // response cap counts the response line alone, as the host does, so a + // response at the cap does not fail after keep-alives. Only a space is + // a keep-alive: a newline ends the line. + const chunk = + buffer.length === 0 ? received.replace(LEADING_SPACES, '') : received; + if (chunk.length === 0) { + return; + } + if (buffer.length + chunk.length > MAX_RPC_RESPONSE_CHARS) { // Only an older host, without the server's response cap, gets here. // A retry gets the same size, so this is not RPC_ERROR (retryable). diff --git a/src/host/rpcServer.ts b/src/host/rpcServer.ts index 788413e9..b603bb2b 100644 --- a/src/host/rpcServer.ts +++ b/src/host/rpcServer.ts @@ -44,9 +44,19 @@ const UNKNOWN_REQUEST_ID = 'unknown'; * * `signal` aborts when the client socket closes, indicating the caller is no * longer waiting for a response. + * + * `keepAlive` tells the client that the handler is still making progress. It + * writes one space ahead of the response line. JSON allows whitespace before + * a value, so a client that does not know about it still parses the response, + * and every client restarts its inactivity timer on the byte (`sendRpc` waits + * for that long without a byte before it gives up). Call it only when work + * moved forward, never on a timer: a handler that hangs must go silent so + * that its client fails with HOST_TIMEOUT. It does nothing once the client is + * gone or the response is on its way. */ export interface MethodContext { readonly signal: AbortSignal; + readonly keepAlive: () => void; } export type MethodHandler = ( @@ -491,9 +501,16 @@ export class RpcServer { socket.off('close', abortRequest); }); + const keepAlive = (): void => { + if (socket.writable) { + socket.write(' '); + } + }; + try { const result = await handler(paramsResult.data, { signal: requestAbortController.signal, + keepAlive, }); if (requestAbortController.signal.aborted) { return; diff --git a/src/protocol/README.md b/src/protocol/README.md index 74476e42..a4f6879a 100644 --- a/src/protocol/README.md +++ b/src/protocol/README.md @@ -34,7 +34,7 @@ - `InputRunEventPayloadSchema` requires `marker` when `noWait` is false and rejects it when `noWait` is true. [schemas.ts:InputRunEventPayloadSchema] - `WaitForRenderParamsSchema` requires at least one of `text`, `regex`, `screenStableMs`, `cursorRow` or `cursorCol`. `text` and `regex` are mutually exclusive, and scope `cursor-line` needs `text` or `regex`. [schemas.ts:WaitForRenderParamsSchema] - `RecordDiffResultSchema` checks that `identical` matches the equality of the two screen hashes. An identical result must have an empty diff and equal side row counts. A side with `capturedAtSeq` -1 must hash to a blank screen. For a non-identical result, the rows rebuilt from the diff must hash to each side's `screenHash`. [schemas.ts:RecordDiffResultSchema] -- `RpcRequestSchema` requires non-empty `id` and `method`. The `id` takes at most `MAX_RPC_REQUEST_ID_CHARS` (128) characters, so the id that the server echoes in a response stays small. `MAX_RPC_RESPONSE_CHARS` (67,108,864) is the largest response line, newline included, that `sendRpc` reads and `RpcServer` sends. It does not restrict `method` to `RpcMethodSchema`. [messages.ts:RpcRequestSchema] +- `RpcRequestSchema` requires non-empty `id` and `method`. The `id` takes at most `MAX_RPC_REQUEST_ID_CHARS` (128) characters, so the id that the server echoes in a response stays small. `MAX_RPC_RESPONSE_CHARS` (67,108,864) is the largest response line, newline included, that `sendRpc` reads and `RpcServer` sends. The spaces that a host writes ahead of the line as keep-alives are not part of it. It does not restrict `method` to `RpcMethodSchema`. [messages.ts:RpcRequestSchema] - `RpcErrorSchema` requires non-empty `code` and `message`, and allows an optional `details` record. A CLI from before `details` was added rejects an error response that has it, so that CLI reports `RPC_ERROR` (`retryable: true`) for such an error from a newer host, and retrying does not help. A newer CLI reads an older host's error responses. [messages.ts:RpcErrorSchema] - `RpcMethodSchemas` uses `satisfies Record`, so TypeScript requires one entry per method. [messages.ts:RpcMethodSchemas] - `RunParamsSchema` defaults `noWait` to `false`. [messages.ts:RunParamsSchema] diff --git a/src/renderer/README.md b/src/renderer/README.md index a2154a14..5e6a9279 100644 --- a/src/renderer/README.md +++ b/src/renderer/README.md @@ -8,7 +8,7 @@ This directory defines the renderer abstraction that turns replayed session even - [./ghosttyWeb/README.md](./ghosttyWeb/README.md): `GhosttyWebBackend`, the reference renderer. It runs `ghostty-web` in headless Chromium through Playwright and supports snapshots, PNG screenshots and WebM video. - [./libghosttyVt/README.md](./libghosttyVt/README.md): `LibghosttyVtBackend`, a native semantic renderer from `@coder/libghostty-vt-node`. Its screenshots go through a `ghostty-web` fallback. -- `RendererBackend`: `boot`, `replayTo`, `snapshot`, `screenshot`, `getVisibleText`, `dispose`, `rendererBackend` and `isBooted` [backend.ts:RendererBackend]. Option types `SnapshotOptions`, `ScreenshotOptions` and `VideoRecordingOptions` [backend.ts:VideoRecordingOptions]. +- `RendererBackend`: `boot`, `replayTo`, `snapshot`, `screenshot`, `getVisibleText`, `dispose`, `rendererBackend` and `isBooted` [backend.ts:RendererBackend]. Option types `SnapshotOptions`, `ScreenshotOptions` and `VideoRecordingOptions` [backend.ts:VideoRecordingOptions]. `ScreenshotOptions.onReplayProgress` is called when a backend that replays the log inside `screenshot()` has booted its second renderer or finished a slice of that replay [backend.ts:ScreenshotOptions]. - `MouseEncodingBackend` adds `encodeMouse(input)`, which encodes one `MouseEncodingInput` with the mouse modes the child set and returns the bytes, empty when tracking suppresses the event. `supportsMouseEncoding(backend)` narrows a `RendererBackend` to it. Only `LibghosttyVtBackend` implements it [backend.ts:MouseEncodingBackend] [backend.ts:supportsMouseEncoding]. - `VideoCapableRendererBackend` adds `replayWithTiming(input, timing)`, `finalizeVideo(outputPath)` and `recordedMissingGlyphs()`, which resolves to the `RecordedGlyphFields` of the replay and never rejects [backend.ts:VideoCapableRendererBackend]. `ReplayTimingOptions` is the union of the exported interfaces `AcceleratedTimingOptions`, `RecordedTimingOptions` and `MaxSpeedTimingOptions` (modes `accelerated`, `recorded` and `max-speed`) [backend.ts:ReplayTimingOptions]. - Schemas and inferred types: `ReplayEventSchema` (11 event types, discriminated on `type`), `ReplayInputSchema`, `ReplayStateSchema`, `SemanticSnapshotSchema`, `TextSnapshotSchema`, `ScreenshotResultSchema`, `RenderProfileBundledFontSchema` and `RenderProfileConfigSchema` [types.ts:ReplayEventSchema]. `types.ts` re-exports `VisibleLineSchema` from `src/protocol/schemas.ts`. @@ -16,6 +16,7 @@ This directory defines the renderer abstraction that turns replayed session even - `createRendererBackend(rendererName, sessionId, profile, options?)` returns a backend instance. `RendererRegistryOptions` can override the `libghostty-vt` loader [registry.ts:createRendererBackend]. - `BUILTIN_PROFILE_NAMES` (`reference-dark`, `reference-light`), `REFERENCE_PROFILE_FONT_STACK`, `getBuiltinProfile`, `resolveProfile` and `hashProfile` [profiles.ts:resolveProfile]. - `canonicalVisibleLines`, `canonicalVisibleText` and `computeScreenHash` derive the canonical visible text and its SHA-256 Screen Hash [canonicalScreen.ts:computeScreenHash]. +- `replayInSlices(backend, input, appliedSeq, onReplayed)` replays `input` to a backend in slices when many events are left. A slice holds at most a number of events that follows the time of the last slice (`FIRST_SLICE_EVENTS` first, `TARGET_SLICE_MS` each) at most `MAX_SLICE_BYTES` of output, except one event that is larger, and at most `MAX_SLICE_FLUSHES` events that follow output. It calls `onReplayed` with the state after each `replayTo`, and the last call gets the whole input [replaySlices.ts:replayInSlices] [replaySlices.ts:sliceEnd]. `HostRendererManager` and the libghostty-vt screenshot use it. - `iterateInRangeReplayEvents(input, lastAppliedSeq)` yields the events of a `ReplayInput` in `(lastAppliedSeq, input.targetSeq]` [replayEvents.ts:iterateInRangeReplayEvents]. - `resolvePlaywrightBrowsersPath`, `resolveDefaultPlaywrightBrowsersPath` and `ensurePlaywrightBrowsersPath` locate the Playwright browser cache [browserPath.ts:ensurePlaywrightBrowsersPath]. Types: `PlaywrightBrowsersPathSource` (`env` or `captured-home`), `PlaywrightBrowsersPathResolution` and `ResolvePlaywrightBrowsersPathOptions` [browserPath.ts:PlaywrightBrowsersPathResolution]. - Bundled font constants and `getBundledFontAssetByIdentity`: `BUNDLED_FONT_ASSETS`, `BUNDLED_FONT_FAMILY`, `BUNDLED_SYMBOLS_FONT_FAMILY`, `BUNDLED_FONT_ROUTE`, `BUNDLED_PRIMARY_FONT_ASSET` and others [bundledFont.ts:getBundledFontAssetByIdentity]. @@ -23,7 +24,7 @@ This directory defines the renderer abstraction that turns replayed session even - `discoverCapabilities(mode, deps?)` returns one `CapabilityEntry` per capability name. `mode` is `'quick'` or `'full'` [capabilities.ts:discoverCapabilities]. The exported types are `DiscoveryMode` and `CapabilityDiscoveryDependencies`, which lets callers replace the probes [capabilities.ts:CapabilityDiscoveryDependencies]. Callers are `src/cli/commands/version.ts` and `src/cli/commands/doctor.ts`. - Capability schemas: `CapabilityNameSchema` (`snapshot`, `wait`, `screenshot`, `record-export-asciicast`, `record-export-webm`, `dashboard`, `mouse-input`), `CapabilityStatusSchema` (`available`, `unavailable`, `degraded`, `unknown`), `CapabilityEntrySchema`, `RendererRuntimeModeSchema`, `RendererRuntimeStatusSchema` and `RendererRuntimeSummarySchema` [capabilities.ts:CapabilityNameSchema]. - `probeLibghosttyVt`, `assertDashboardRendererAvailable`, `buildDashboardCapability`, `LIBGHOSTTY_VT_PACKAGE` and `DASHBOARD_RENDERER_UNAVAILABLE_MESSAGE` [readiness.ts:probeLibghosttyVt]. Types: `LibghosttyVtProbe` and `LibghosttyVtLoader` [readiness.ts:LibghosttyVtProbe]. The probe sets `mouseInputAvailable` from the package's `supportsMouseInput` export [readiness.ts:probeLibghosttyVt]. -- `index.ts` is a barrel for the profile, schema, name and registry exports, the `RendererBackend` and `SnapshotOptions` types, and `GhosttyWebBackend`. It does not re-export `browserPath`, `bundledFont`, `canonicalScreen`, `replayEvents`, `capabilities` or `readiness` [index.ts:createRendererBackend]. +- `index.ts` is a barrel for the profile, schema, name and registry exports, the `RendererBackend` and `SnapshotOptions` types, and `GhosttyWebBackend`. It does not re-export `browserPath`, `bundledFont`, `canonicalScreen`, `replayEvents`, `replaySlices`, `capabilities` or `readiness` [index.ts:createRendererBackend]. ## Invariants diff --git a/src/renderer/backend.ts b/src/renderer/backend.ts index b5cc08b5..8513b8e7 100644 --- a/src/renderer/backend.ts +++ b/src/renderer/backend.ts @@ -18,6 +18,13 @@ export interface SnapshotOptions { export interface ScreenshotOptions { showCursor?: boolean; + /** + * Called each time a backend that replays the log inside `screenshot()` (the + * libghostty-vt one, in a second renderer) has booted that renderer or + * finished a slice of the replay. It lets a caller that waits for the + * screenshot tell that the replay moves forward. + */ + onReplayProgress?: () => void; } export interface RendererBackend { diff --git a/src/renderer/libghosttyVt/README.md b/src/renderer/libghosttyVt/README.md index 6fce59a3..630c4027 100644 --- a/src/renderer/libghosttyVt/README.md +++ b/src/renderer/libghosttyVt/README.md @@ -11,7 +11,7 @@ This directory holds the `libghostty-vt` renderer backend. `LibghosttyVtBackend` - `boot()` loads the native module on first call. Concurrent calls share one boot promise [backend.ts:LibghosttyVtBackend.boot]. - `replayTo(input)` feeds `output` events and applies `resize` events. It skips input, marker, run, signal and exit events [backend.ts:LibghosttyVtBackend.replayTo]. - `snapshot(options?)` returns a `SemanticSnapshot` parsed by `SemanticSnapshotSchema` [backend.ts:LibghosttyVtBackend.snapshot]. - - `screenshot(outputPath, options?)` boots a fallback backend, replays the last replay input into it, takes the screenshot, and disposes the fallback [backend.ts:LibghosttyVtBackend.screenshot]. + - `screenshot(outputPath, options?)` boots a fallback backend, replays the last replay input into it in slices (`replayInSlices`), takes the screenshot, and disposes the fallback. It calls `options.onReplayProgress` after the fallback has booted and after each `replayTo` call [backend.ts:LibghosttyVtBackend.screenshot]. - `encodeMouse(input)` maps a cell, button and modifiers to the native `encodeMouse` with a pixel geometry of `cellWidth` and `cellHeight` (default 1) per cell, and returns the bytes [backend.ts:LibghosttyVtBackend.encodeMouse]. - `getVisibleText()` and `dispose()` [backend.ts:LibghosttyVtBackend.dispose]. - `LibghosttyVtBackendOptions`: `initialCols`, `initialRows`, `scrollbackLimit`, `loadNative`, `fallbackFactory` and `logger` [backend.ts:LibghosttyVtBackendOptions]. Re-exported as a type by `index.ts`. diff --git a/src/renderer/libghosttyVt/backend.ts b/src/renderer/libghosttyVt/backend.ts index 244315ba..cb66dd0d 100644 --- a/src/renderer/libghosttyVt/backend.ts +++ b/src/renderer/libghosttyVt/backend.ts @@ -28,6 +28,7 @@ import type { } from '../types.js'; import { SemanticSnapshotSchema } from '../types.js'; import { iterateInRangeReplayEvents } from '../replayEvents.js'; +import { replayInSlices } from '../replaySlices.js'; import { DEFAULT_COLS, DEFAULT_ROWS } from '../../config/defaults.js'; import { invariant, assertString, unreachable } from '../../util/assert.js'; import { Logger, createProcessLogger } from '../../util/logger.js'; @@ -669,7 +670,12 @@ export class LibghosttyVtBackend implements MouseEncodingBackend { const fallback = await this.createFallbackBackend(); try { - await fallback.replayTo(this.latestReplayInput); + options?.onReplayProgress?.(); + // A cold renderer replays the whole log, which can take minutes: slice + // it, so that the caller hears of each slice. + await replayInSlices(fallback, this.latestReplayInput, -1, () => { + options?.onReplayProgress?.(); + }); return await fallback.screenshot(outputPath, options); } finally { await fallback.dispose(); diff --git a/src/renderer/replaySlices.ts b/src/renderer/replaySlices.ts new file mode 100644 index 00000000..f23861cd --- /dev/null +++ b/src/renderer/replaySlices.ts @@ -0,0 +1,175 @@ +import type { RendererBackend } from './backend.js'; +import type { ReplayEvent, ReplayInput, ReplayState } from './types.js'; +import { invariant } from '../util/assert.js'; + +// A replay of a long log can take minutes, and a caller sees nothing of it +// until `replayTo` returns. So the host replays a long backlog in slices and +// reports each finished slice as progress, which keeps `sendRpc` (it gives up +// after 5 s without a byte from the host) waiting while the replay moves and +// lets it fail when the replay stops. A backend that is stuck finishes no +// slice. Any split of the events gives the same backend state, because every +// live render already replays only the events since the one before. +// +// Slice size follows the measured time of the last slice, because replay +// speed differs by 1,000 times between backends and logs. A slice that takes +// 0.5 s leaves 10 times the 5 s of `sendRpc` for a host that runs late. The +// first slice is small, because nothing is known about the speed yet: at the +// 21 ms per event that ghostty-web once took, it is 0.7 s. +export const FIRST_SLICE_EVENTS = 32; +export const TARGET_SLICE_MS = 500; +// A count of events does not bound the time of a slice when the speed depends +// on the bytes that a slice writes. After thousands of tiny events, the size +// has grown, and 500 events of 64 KB would then run for seconds in silence. +// So a slice also holds at most this many bytes of output. One event that is +// larger than this makes a slice alone. +export const MAX_SLICE_BYTES = 1024 * 1024; +// Output waits in a batch that a renderer writes at once. An input, a marker, +// a resize or any other event that follows output makes the renderer write the +// batch first and wait for a frame, which takes about 34 ms. Tiny outputs cost +// almost nothing, so the size grows over them, and thousands of such pairs +// after them would then run for minutes in silence. So a slice also holds at +// most this many of these flushes. At 34 ms, it stays near the target time. +export const MAX_SLICE_FLUSHES = 16; +// A slice may grow or shrink by this factor at most, so one cheap or costly +// stretch of the log does not make the next slice far too long. +const MAX_SLICE_SCALE = 4; + +interface SeqEvent { + readonly seq: number; +} + +/** The number of events with a seq of at most `seq` (events are in seq order). */ +export function countEventsThrough( + events: readonly SeqEvent[], + seq: number, +): number { + let low = 0; + let high = events.length; + while (low < high) { + const middle = (low + high) >>> 1; + const event = events[middle]; + invariant(event !== undefined, 'replay event must exist'); + if (event.seq <= seq) { + low = middle + 1; + } else { + high = middle; + } + } + return low; +} + +function outputBytes(event: ReplayEvent): number { + return event.type === 'output' ? Buffer.byteLength(event.payload.data) : 0; +} + +/** + * The index after the last event of the slice that starts at `start`, at most + * `end`. The slice holds at most `maxEvents` events, and after its first event + * at most `MAX_SLICE_BYTES` of output and `MAX_SLICE_FLUSHES` events that + * follow output (the replay of the slice flushes its output at each of them). + */ +export function sliceEnd( + events: readonly ReplayEvent[], + start: number, + end: number, + maxEvents: number, +): number { + invariant( + Number.isInteger(maxEvents) && maxEvents > 0, + 'slice event limit must be a positive integer', + ); + const limit = Math.min(end, start + maxEvents); + let bytes = 0; + let flushes = 0; + let outputPending = false; + let index = start; + while (index < limit) { + const event = events[index]; + invariant(event !== undefined, 'replay event must exist'); + bytes += outputBytes(event); + if (event.type === 'output') { + outputPending = true; + } else if (outputPending) { + outputPending = false; + flushes += 1; + } + if ( + index > start && + (bytes > MAX_SLICE_BYTES || flushes > MAX_SLICE_FLUSHES) + ) { + break; + } + index += 1; + } + return index; +} + +export function nextSliceSize(size: number, elapsedMs: number): number { + invariant( + Number.isInteger(size) && size > 0, + 'slice size must be a positive integer', + ); + const scaled = Math.round((size * TARGET_SLICE_MS) / Math.max(elapsedMs, 1)); + return Math.min( + size * MAX_SLICE_SCALE, + Math.max(Math.ceil(size / MAX_SLICE_SCALE), scaled), + ); +} + +/** + * Replay `input` to `backend` like `backend.replayTo(input)`, but in slices + * when many events are left, and call `onReplayed` after each `replayTo`. + * + * `appliedSeq` is where the caller last saw the backend. It may be behind the + * backend, which skips events it holds, but never ahead of it. + */ +export async function replayInSlices( + backend: Pick, + input: ReplayInput, + appliedSeq: number, + onReplayed: (state: ReplayState) => void, + now: () => number = () => performance.now(), +): Promise { + invariant( + Number.isInteger(appliedSeq) && appliedSeq >= -1, + 'appliedSeq must be an integer of at least -1', + ); + const { events } = input; + // Events past the target are never replayed. + const end = countEventsThrough(events, input.targetSeq); + let next = countEventsThrough(events, appliedSeq); + let size = FIRST_SLICE_EVENTS; + let stop = sliceEnd(events, next, end, size); + + // The rest fits in one slice when `stop` reaches `end`. + while (stop < end) { + const startedAt = now(); + // The slice keeps the target of the whole replay. A lower target would + // fail on a backend that is ahead of `appliedSeq`, which cannot rewind. + // The backend ends at the last event that it replays, not at the target. + const state = await backend.replayTo({ + ...input, + events: events.slice(next, stop), + }); + onReplayed(state); + const reached = countEventsThrough(events, state.lastSeq); + invariant( + reached >= stop && reached <= end, + 'a replay slice must end at its last event or at the backend position', + ); + // Only a slice that the count of events ended says how many events take + // the target time. The byte limit ended the others, and they hold events + // of a different cost. + if (stop - next === size) { + size = nextSliceSize(size, now() - startedAt); + } + // The backend can be ahead of `appliedSeq`, so take its own position. + next = reached; + stop = sliceEnd(events, next, end, size); + } + + // The last call gets the whole input: a backend keeps its last input (the + // libghostty-vt screenshot replays it in a second renderer), and it skips + // the events it already holds. The events that are left fit in one slice. + onReplayed(await backend.replayTo(input)); +} diff --git a/src/screenshot/README.md b/src/screenshot/README.md index 30dc3ebf..797a185b 100644 --- a/src/screenshot/README.md +++ b/src/screenshot/README.md @@ -8,7 +8,7 @@ This directory turns a renderer screenshot into a stored session artifact. `capt - `CaptureScreenshotBackend`: `Pick`, so callers can only reach `screenshot()` [capture.ts:CaptureScreenshotBackend]. - `CapturedScreenshotResult`: `ScreenshotResult` (from `src/protocol/messages.ts`) with a required `sha256` [capture.ts:CapturedScreenshotResult]. -- `CaptureScreenshotResultOptions`: `backend`, `sessionDir`, `profileName`, `expectedSessionId` and optional `showCursor` [capture.ts:CaptureScreenshotResultOptions]. +- `CaptureScreenshotResultOptions`: `backend`, `sessionDir`, `profileName`, `expectedSessionId`, and optional `showCursor` and `onReplayProgress`, which go to the backend as screenshot options [capture.ts:CaptureScreenshotResultOptions]. - `parseScreenshotResult(rawResult, message?)` validates a value with `ScreenshotResultSchema` from `src/protocol/schemas.ts` through `parseValidatedResult` [capture.ts:parseScreenshotResult]. - `captureScreenshotResult(options)` captures, stores and records one screenshot and returns the `CapturedScreenshotResult` [capture.ts:captureScreenshotResult]. diff --git a/src/screenshot/capture.ts b/src/screenshot/capture.ts index 074055da..4eb9632e 100644 --- a/src/screenshot/capture.ts +++ b/src/screenshot/capture.ts @@ -3,7 +3,10 @@ import { rm } from 'node:fs/promises'; import { ulid } from 'ulid'; import type { ScreenshotResult } from '../protocol/messages.js'; -import type { RendererBackend } from '../renderer/backend.js'; +import type { + RendererBackend, + ScreenshotOptions, +} from '../renderer/backend.js'; import { ScreenshotResultSchema } from '../protocol/schemas.js'; import { parseValidatedResult } from '../protocol/validation.js'; @@ -41,6 +44,8 @@ export interface CaptureScreenshotResultOptions { profileName: string; expectedSessionId: string; showCursor?: boolean; + /** See `ScreenshotOptions.onReplayProgress`. */ + onReplayProgress?: () => void; } export function parseScreenshotResult( @@ -67,11 +72,19 @@ export async function captureScreenshotResult( ); try { + const screenshotOptions: ScreenshotOptions = { + ...(options.showCursor === undefined + ? {} + : { showCursor: options.showCursor }), + ...(options.onReplayProgress === undefined + ? {} + : { onReplayProgress: options.onReplayProgress }), + }; const rendererResult = await options.backend.screenshot( temporaryOutputPath, - options.showCursor === undefined + Object.keys(screenshotOptions).length === 0 ? undefined - : { showCursor: options.showCursor }, + : screenshotOptions, ); invariant( diff --git a/test/fixtures/slow-replay.ts b/test/fixtures/slow-replay.ts new file mode 100644 index 00000000..104e6629 --- /dev/null +++ b/test/fixtures/slow-replay.ts @@ -0,0 +1,37 @@ +import process from 'node:process'; + +// Makes the ghostty-web backend of the host replay as slowly as it did when it +// took 21 ms for each event: every `replayTo` first waits +// AGENT_TTY_TEST_REPLAY_MS_PER_EVENT ms for each event that it will replay. +// With AGENT_TTY_TEST_REPLAY_STALL_AFTER_CALLS=n, the call after the first n +// calls never ends. +if (process.argv.includes('_host')) { + const { register } = await import('tsx/esm/api'); + register(); + const { GhosttyWebBackend } = + await import('../../src/renderer/ghosttyWeb/backend.js'); + const msPerEvent = Number( + process.env.AGENT_TTY_TEST_REPLAY_MS_PER_EVENT ?? '0', + ); + const stallAfterCalls = Number( + process.env.AGENT_TTY_TEST_REPLAY_STALL_AFTER_CALLS ?? 'Infinity', + ); + // oxlint-disable-next-line typescript/unbound-method -- the patch below calls it with replayTo.call(this, ...). + const replayTo = GhosttyWebBackend.prototype.replayTo; + let calls = 0; + GhosttyWebBackend.prototype.replayTo = async function ( + this: { lastAppliedSeq: number }, + input: Parameters[0], + ) { + calls += 1; + if (calls > stallAfterCalls) { + await new Promise(() => undefined); + } + const pending = input.events.filter( + (event) => + event.seq > this.lastAppliedSeq && event.seq <= input.targetSeq, + ).length; + await new Promise((resolve) => setTimeout(resolve, pending * msPerEvent)); + return replayTo.call(this as never, input); + } as typeof replayTo; +} diff --git a/test/helpers/fakeBackend.ts b/test/helpers/fakeBackend.ts index 4b1a18bd..d3031ff1 100644 --- a/test/helpers/fakeBackend.ts +++ b/test/helpers/fakeBackend.ts @@ -2,7 +2,10 @@ import { writeFile } from 'node:fs/promises'; import { vi } from 'vitest'; -import type { RendererBackend } from '../../src/renderer/backend.js'; +import type { + RendererBackend, + ScreenshotOptions, +} from '../../src/renderer/backend.js'; import type { ReplayInput, ReplayState, @@ -20,10 +23,7 @@ export interface FakeBackendOptions { resultOverrides?: Partial; writePng?: boolean; fail?: Error; - onScreenshot?: ( - outputPath: string, - options?: { showCursor?: boolean }, - ) => void; + onScreenshot?: (outputPath: string, options?: ScreenshotOptions) => void; } export type FakeRendererBackend = RendererBackend & { @@ -79,7 +79,7 @@ export function createFakeBackend( const screenshotMock = vi.fn( async ( outputPath: string, - screenshotOptions?: { showCursor?: boolean }, + screenshotOptions?: ScreenshotOptions, ): Promise => { options.onScreenshot?.(outputPath, screenshotOptions); if (options.fail !== undefined) { diff --git a/test/integration/slow-replay.test.ts b/test/integration/slow-replay.test.ts new file mode 100644 index 00000000..f7717fc3 --- /dev/null +++ b/test/integration/slow-replay.test.ts @@ -0,0 +1,249 @@ +import { execFile } from 'node:child_process'; +import { mkdtemp, realpath } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join, resolve } from 'node:path'; +import process from 'node:process'; +import { pathToFileURL } from 'node:url'; + +import { afterEach, beforeEach, describe, expect, it } from 'vitest'; + +import { + cleanupHome, + createSession, + destroySession, + readEvents, + sleep, +} from '../helpers.js'; + +// The host client gives up after 5 s without a byte from the host. +const CLIENT_WAIT_MS = 5_000; +const PRINTED_LINES = 200; +// With 50 ms for each event, the replay of the lines takes 10 s or more. +const REPLAY_MS_PER_EVENT = 50; + +interface CliOutcome { + readonly code: number | null; + readonly stdout: string; + readonly elapsedMs: number; +} + +function runCliAsync( + testHome: string, + args: string[], + timeoutMs: number, +): Promise { + const startedAt = performance.now(); + return new Promise((resolvePromise) => { + execFile( + process.execPath, + ['--import', 'tsx', './src/cli/main.ts', ...args, '--json'], + { + cwd: process.cwd(), + env: { ...process.env, AGENT_TTY_HOME: testHome }, + timeout: timeoutMs, + }, + (error, stdout) => { + const code = (error as { code?: unknown } | null)?.code; + resolvePromise({ + code: error === null ? 0 : typeof code === 'number' ? code : null, + stdout, + elapsedMs: performance.now() - startedAt, + }); + }, + ); + }); +} + +describe( + 'live render of a log that the renderer replays slowly', + { timeout: 180_000 }, + () => { + let testHome = ''; + let sessionId = ''; + + beforeEach(async () => { + // oxfmt-ignore + testHome = await realpath(await mkdtemp(join(tmpdir(), 'agent-tty-slow-replay-'))); + }); + + afterEach(async () => { + destroySession(testHome, sessionId); + await cleanupHome(testHome); + sessionId = ''; + testHome = ''; + }); + + // A session whose host replays with the slow-replay fixture and has + // printed the lines, one event for each line. + async function createSlowSession( + command: string[], + env: Record, + printedEvents: number, + ): Promise { + const preload = pathToFileURL(resolve('test/fixtures/slow-replay.ts')); + sessionId = createSession(testHome, command, { + env: { + NODE_OPTIONS: [ + process.env.NODE_OPTIONS ?? '', + `--import=${preload.href}`, + ].join(' '), + AGENT_TTY_TEST_REPLAY_MS_PER_EVENT: String(REPLAY_MS_PER_EVENT), + ...env, + }, + }); + await waitForEvents(printedEvents); + } + + async function waitForEvents(count: number): Promise { + const deadline = Date.now() + 60_000; + while (Date.now() < deadline) { + const events = await readEvents(testHome, sessionId).catch(() => []); + if (events.length >= count) { + return; + } + await sleep(250); + } + throw new Error('timed out waiting for the printed lines'); + } + + // Prints a line every 20 ms, so that each line is its own event. + const PRINT_LINES_COMMAND = [ + '/bin/sh', + '-c', + `i=0; while [ $i -lt ${String(PRINTED_LINES)} ]; do echo line-$i; i=$((i+1)); sleep 0.02; done; exec cat`, + ]; + + it('answers a screenshot, and a snapshot that waits behind it, while the replay goes on for longer than the client waits', async () => { + await createSlowSession(PRINT_LINES_COMMAND, {}, PRINTED_LINES); + + const screenshot = runCliAsync( + testHome, + ['screenshot', sessionId], + 90_000, + ); + // Starts while the first replay runs, so it waits for it. + await sleep(1_000); + const snapshot = runCliAsync( + testHome, + ['snapshot', sessionId, '--renderer', 'ghostty-web'], + 90_000, + ); + const [screenshotOutcome, snapshotOutcome] = await Promise.all([ + screenshot, + snapshot, + ]); + + expect(screenshotOutcome.code, screenshotOutcome.stdout).toBe(0); + expect(screenshotOutcome.elapsedMs).toBeGreaterThan(CLIENT_WAIT_MS); + expect(snapshotOutcome.code, snapshotOutcome.stdout).toBe(0); + expect(snapshotOutcome.elapsedMs).toBeGreaterThan(CLIENT_WAIT_MS); + }); + + it('answers a libghostty-vt screenshot, which replays the log in a second renderer, for longer than the client waits', async () => { + await createSlowSession(PRINT_LINES_COMMAND, {}, PRINTED_LINES); + + const outcome = await runCliAsync( + testHome, + ['screenshot', sessionId, '--renderer', 'libghostty-vt'], + 90_000, + ); + + expect(outcome.code, outcome.stdout).toBe(0); + expect(JSON.parse(outcome.stdout)).toMatchObject({ + ok: true, + result: { rendererBackend: 'ghostty-web' }, + }); + expect(outcome.elapsedMs).toBeGreaterThan(CLIENT_WAIT_MS); + }); + + it('answers a snapshot that waits behind a libghostty-vt screenshot, which replays the log in a second renderer, for longer than the client waits', async () => { + await createSlowSession(PRINT_LINES_COMMAND, {}, PRINTED_LINES); + + const screenshot = runCliAsync( + testHome, + ['screenshot', sessionId, '--renderer', 'libghostty-vt'], + 90_000, + ); + // Starts while the screenshot replays the log in its second renderer, + // so it waits behind the screenshot in the queue of the host. + await sleep(1_000); + const snapshot = runCliAsync( + testHome, + ['snapshot', sessionId, '--renderer', 'libghostty-vt'], + 90_000, + ); + const [screenshotOutcome, snapshotOutcome] = await Promise.all([ + screenshot, + snapshot, + ]); + + expect(screenshotOutcome.code, screenshotOutcome.stdout).toBe(0); + expect(snapshotOutcome.code, snapshotOutcome.stdout).toBe(0); + expect(snapshotOutcome.elapsedMs).toBeGreaterThan(CLIENT_WAIT_MS); + }); + + it('still ends with HOST_TIMEOUT when the replay stops moving', async () => { + // The renderer finishes two slices and then stops. + await createSlowSession( + PRINT_LINES_COMMAND, + { AGENT_TTY_TEST_REPLAY_STALL_AFTER_CALLS: '2' }, + PRINTED_LINES, + ); + + const outcome = await runCliAsync( + testHome, + ['screenshot', sessionId], + 90_000, + ); + + expect(outcome.code, outcome.stdout).toBe(5); + expect(JSON.parse(outcome.stdout)).toMatchObject({ + ok: false, + error: { code: 'HOST_TIMEOUT' }, + }); + // It waited for the slices, then for 5 s without a byte. + expect(outcome.elapsedMs).toBeGreaterThan(CLIENT_WAIT_MS); + }); + + it('answers a run whose catch-up replay goes on for longer than the client waits', async () => { + // The host default renderer is ghostty-web, so that the catch-up replay + // after the run uses the slow backend. + await createSlowSession( + ['/bin/bash'], + { + AGENT_TTY_RENDERER: 'ghostty-web', + AGENT_TTY_TEST_REPLAY_MS_PER_EVENT: '100', + HISTFILE: '/dev/null', + }, + 1, + ); + const printed = await runCliAsync( + testHome, + [ + 'run', + sessionId, + `for i in $(seq 1 ${String(PRINTED_LINES)}); do echo line-$i; sleep 0.02; done`, + '--no-wait', + ], + 30_000, + ); + expect(printed.code, printed.stdout).toBe(0); + await waitForEvents(PRINTED_LINES); + + // The command ends at once. The client waits 3 s for it and 10 s more + // for the host, and the catch-up replay takes 20 s or more. + const outcome = await runCliAsync( + testHome, + ['run', sessionId, 'true', '--timeout', '3000'], + 90_000, + ); + + expect(outcome.code, outcome.stdout).toBe(0); + expect(JSON.parse(outcome.stdout)).toMatchObject({ + ok: true, + result: { completed: true }, + }); + expect(outcome.elapsedMs).toBeGreaterThan(13_000); + }); + }, +); diff --git a/test/unit/host/renderer.test.ts b/test/unit/host/renderer.test.ts index 2b369bad..1191119b 100644 --- a/test/unit/host/renderer.test.ts +++ b/test/unit/host/renderer.test.ts @@ -7,9 +7,11 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; import { HostRendererManager } from '../../../src/host/renderer.js'; import type { RendererBackend } from '../../../src/renderer/backend.js'; +import { iterateInRangeReplayEvents } from '../../../src/renderer/replayEvents.js'; import type { RenderProfileConfig, ReplayInput, + ReplayState, } from '../../../src/renderer/types.js'; import { @@ -476,3 +478,377 @@ describe('HostRendererManager', () => { ).toThrow('backendFactory must be a function'); }); }); + +describe('HostRendererManager replay progress', () => { + let sessionDir: string; + let backends: FakeRendererBackend[]; + let appliedByBackend: number[][]; + // One entry for each `replayTo` call that waits for the test to let it go. + let pendingReplays: ReturnType>[]; + let holdReplays: boolean; + let backendFactory: ReturnType>; + + function createLongInput(eventCount: number): ReplayInput { + return createReplayInput({ + events: Array.from({ length: eventCount }, (_, seq) => ({ + seq, + ts: '2026-03-20T12:00:00.000Z', + type: 'output' as const, + payload: { data: 'x' }, + })), + targetSeq: eventCount - 1, + }); + } + + // A backend that replays as the real ones do: it holds a position, skips the + // events that it holds and ends at the last event that it replays. + function createReplayingBackend(): FakeRendererBackend { + const backend = createFakeBackend(); + const applied: number[] = []; + appliedByBackend.push(applied); + let lastAppliedSeq = -1; + const replay = (input: ReplayInput): ReplayState => { + for (const event of iterateInRangeReplayEvents(input, lastAppliedSeq)) { + applied.push(event.seq); + lastAppliedSeq = event.seq; + } + return { + lastSeq: lastAppliedSeq, + cols: input.initialCols, + rows: input.initialRows, + cursorRow: 0, + cursorCol: 0, + }; + }; + const replayToMock = vi.fn((input: ReplayInput): Promise => { + if (!holdReplays) { + return Promise.resolve(replay(input)); + } + const hold = createDeferred(); + pendingReplays.push(hold); + return hold.promise.then(() => replay(input)); + }); + backend.replayTo = replayToMock; + backend.replayToMock = replayToMock; + return backend; + } + + // Let the replays go that wait now, one by one, until `done` reads true. + async function releaseReplaysUntil(done: () => boolean): Promise { + for (let round = 0; round < 1_000 && !done(); round += 1) { + await flushAsyncQueue(); + pendingReplays.shift()?.resolve(undefined); + } + expect(done()).toBe(true); + } + + function createManager(): HostRendererManager { + return new HostRendererManager({ + sessionId: 'session-01', + sessionDir, + backendFactory, + }); + } + + beforeEach(async () => { + // oxfmt-ignore + sessionDir = await realpath(await mkdtemp(join(tmpdir(), 'agent-tty-renderer-'))); + backends = []; + appliedByBackend = []; + pendingReplays = []; + holdReplays = false; + backendFactory = vi.fn(() => { + const backend = createReplayingBackend(); + backends.push(backend); + return backend; + }); + }); + + afterEach(async () => { + await rm(sessionDir, { recursive: true, force: true }); + }); + + it('reports progress each time a replay of a long log moves forward', async () => { + const manager = createManager(); + const onProgress = vi.fn(); + + await manager.withBackend( + 'ghostty-web', + createProfile(), + () => createLongInput(300), + () => undefined, + onProgress, + ); + + const replayCalls = getCreatedBackend(backends, 0).replayToMock.mock.calls; + expect(replayCalls.length).toBeGreaterThan(1); + // The boot, and each call of `replayTo`. + expect(onProgress).toHaveBeenCalledTimes(1 + replayCalls.length); + expect(appliedByBackend[0]).toEqual( + Array.from({ length: 300 }, (_, seq) => seq), + ); + }); + + it('reports progress when the backend has booted, before its first slice is done', async () => { + const manager = createManager(); + holdReplays = true; + const onProgress = vi.fn(); + + const call = manager.withBackend( + 'ghostty-web', + createProfile(), + () => createLongInput(300), + () => undefined, + onProgress, + ); + await flushAsyncQueue(); + await flushAsyncQueue(); + + // The first slice waits, and the boot is the only progress so far. + expect(pendingReplays).toHaveLength(1); + expect(onProgress).toHaveBeenCalledTimes(1); + + let finished = false; + void call.then(() => { + finished = true; + }); + await releaseReplaysUntil(() => finished); + }); + + it('reports the progress of the replay ahead to a call that waits for it', async () => { + const manager = createManager(); + holdReplays = true; + const aheadProgress = vi.fn(); + const waitingProgress = vi.fn(); + const waitingOperation = vi.fn(); + + const ahead = manager.withBackend( + 'ghostty-web', + createProfile(), + () => createLongInput(300), + () => undefined, + aheadProgress, + ); + const waiting = manager.withBackend( + 'ghostty-web', + createProfile(), + () => createLongInput(300), + waitingOperation, + waitingProgress, + ); + await flushAsyncQueue(); + await flushAsyncQueue(); + expect(pendingReplays).toHaveLength(1); + + pendingReplays.shift()?.resolve(undefined); + await flushAsyncQueue(); + await flushAsyncQueue(); + + // The call ahead has booted the backend and replayed its first slice. The + // other call has not begun, and it hears of both. + expect(aheadProgress).toHaveBeenCalledTimes(2); + expect(waitingProgress).toHaveBeenCalledTimes(2); + expect(waitingOperation).not.toHaveBeenCalled(); + + let finished = false; + void Promise.all([ahead, waiting]).then(() => { + finished = true; + }); + await releaseReplaysUntil(() => finished); + }); + + it('reports the progress that an operation reports to every call that waits, its own too', async () => { + const manager = createManager(); + const gate = createDeferred(); + const aheadProgress = vi.fn(); + const waitingProgress = vi.fn(); + const waitingOperation = vi.fn(); + const heard = { ahead: 0, waiting: 0 }; + + // No input: nothing replays, so the only progress is the boot and what + // the operation reports (a libghostty-vt screenshot reports each slice of + // the replay in its second renderer). + const ahead = manager.withBackend( + 'libghostty-vt', + createProfile(), + () => null, + async (_backend, _replayInput, reportProgress) => { + const before = { + ahead: aheadProgress.mock.calls.length, + waiting: waitingProgress.mock.calls.length, + }; + reportProgress(); + reportProgress(); + heard.ahead = aheadProgress.mock.calls.length - before.ahead; + heard.waiting = waitingProgress.mock.calls.length - before.waiting; + await gate.promise; + }, + aheadProgress, + ); + const waiting = manager.withBackend( + 'libghostty-vt', + createProfile(), + () => null, + waitingOperation, + waitingProgress, + ); + await flushAsyncQueue(); + await flushAsyncQueue(); + + expect(heard).toEqual({ ahead: 2, waiting: 2 }); + expect(waitingOperation).not.toHaveBeenCalled(); + + gate.resolve(undefined); + await Promise.all([ahead, waiting]); + expect(waitingOperation).toHaveBeenCalledTimes(1); + }); + + it('reports nothing once a replay stops, and a call that finished hears nothing more', async () => { + const manager = createManager(); + holdReplays = true; + const onProgress = vi.fn(); + + const stuck = manager.withBackend( + 'ghostty-web', + createProfile(), + () => createLongInput(300), + () => undefined, + onProgress, + ); + for (let slice = 0; slice < 2; slice += 1) { + await flushAsyncQueue(); + pendingReplays.shift()?.resolve(undefined); + } + await flushAsyncQueue(); + await flushAsyncQueue(); + // The boot and two slices. + expect(onProgress).toHaveBeenCalledTimes(3); + + // The third slice never ends. + await new Promise((resolve) => setTimeout(resolve, 30)); + expect(onProgress).toHaveBeenCalledTimes(3); + expect(pendingReplays).toHaveLength(1); + + let finished = false; + void stuck.then(() => { + finished = true; + }); + await releaseReplaysUntil(() => finished); + const finalCount = onProgress.mock.calls.length; + + // Another call replays more events. The finished call is not told. + holdReplays = false; + await manager.withBackend( + 'ghostty-web', + createProfile(), + () => createLongInput(600), + () => undefined, + ); + expect(onProgress).toHaveBeenCalledTimes(finalCount); + }); + + it('replays only the new events when the backend already holds the earlier ones', async () => { + const manager = createManager(); + + await manager.withBackend( + 'ghostty-web', + createProfile(), + () => createLongInput(300), + () => undefined, + ); + const replayCalls = getCreatedBackend(backends, 0).replayToMock.mock.calls; + const callsBefore = replayCalls.length; + await manager.withBackend( + 'ghostty-web', + createProfile(), + () => createLongInput(400), + () => undefined, + ); + + const secondFirstSlice = replayCalls[callsBefore]?.[0] as ReplayInput; + expect(secondFirstSlice.events[0]?.seq).toBe(300); + expect(appliedByBackend[0]).toEqual( + Array.from({ length: 400 }, (_, seq) => seq), + ); + }); + + it('goes on in slices from the last slice that finished, when a slice failed', async () => { + // The third call of `replayTo` fails once, as when the page crashes. + backendFactory = vi.fn(() => { + const backend = createReplayingBackend(); + const replay = backend.replayTo.bind(backend); + let calls = 0; + const replayToMock = vi.fn((input: ReplayInput): Promise => { + calls += 1; + return calls === 3 + ? Promise.reject(new Error('page crashed')) + : replay(input); + }); + backend.replayTo = replayToMock; + backend.replayToMock = replayToMock; + backends.push(backend); + return backend; + }); + const manager = createManager(); + const profile = createProfile(); + + await expect( + manager.withBackend( + 'ghostty-web', + profile, + () => createLongInput(300), + () => undefined, + ), + ).rejects.toThrow('page crashed'); + const replayCalls = getCreatedBackend(backends, 0).replayToMock.mock.calls; + expect(replayCalls).toHaveLength(3); + const replayedBeforeRetry = appliedByBackend[0]?.length ?? 0; + expect(replayedBeforeRetry).toBeGreaterThan(0); + expect(replayedBeforeRetry).toBeLessThan(300); + + await manager.withBackend( + 'ghostty-web', + profile, + () => createLongInput(300), + () => undefined, + ); + + // The retry does not start over, and it is not one call that replays all + // the events that are left: it has slices, and the last has the input. + const retryInputs = replayCalls + .slice(3) + .map(([input]) => input as ReplayInput); + expect(retryInputs.length).toBeGreaterThan(1); + expect(retryInputs[0]?.events[0]?.seq).toBe(replayedBeforeRetry); + expect(retryInputs[0]?.events.length).toBeLessThan( + 300 - replayedBeforeRetry, + ); + expect(retryInputs.at(-1)?.events).toHaveLength(300); + expect(appliedByBackend[0]).toEqual( + Array.from({ length: 300 }, (_, seq) => seq), + ); + }); + + it('replays a new backend from the first event', async () => { + const manager = createManager(); + + await manager.withBackend( + 'ghostty-web', + createProfile('first'), + () => createLongInput(300), + () => undefined, + ); + // A change of profile replaces the backend, which holds nothing yet. + await manager.withBackend( + 'ghostty-web', + createProfile('second'), + () => createLongInput(400), + () => undefined, + ); + + expect(backends).toHaveLength(2); + expect(appliedByBackend[1]).toEqual( + Array.from({ length: 400 }, (_, seq) => seq), + ); + }); +}); diff --git a/test/unit/host/rpcClient.test.ts b/test/unit/host/rpcClient.test.ts index 1a7d79a7..dbe5cce9 100644 --- a/test/unit/host/rpcClient.test.ts +++ b/test/unit/host/rpcClient.test.ts @@ -3,6 +3,7 @@ import { tmpdir } from 'node:os'; import { dirname, join } from 'node:path'; import net from 'node:net'; import process from 'node:process'; +import { setTimeout } from 'node:timers/promises'; import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; @@ -127,6 +128,89 @@ describe('sendRpc abort handling', () => { }); }); +describe('sendRpc wait for the host', () => { + const WAIT_MS = 800; + const SHORT_WAIT_MS = 100; + const BYTE_INTERVAL_MS = 250; + + beforeEach(async () => { + tempDir = await mkdtemp(join(tmpdir(), 'agent-tty-rpc-client-')); + }); + + afterEach(async () => { + await rm(tempDir, { recursive: true, force: true }); + tempDir = ''; + }); + + // A host that answers `ping` after `bytes` spaces, one every 250 ms. + async function listen(bytes: number): Promise { + const socketFile = join(tempDir, 'rpc.sock'); + const server = net.createServer((socket) => { + socket.on('error', () => undefined); + socket.once('data', (request: Buffer) => { + const { id } = JSON.parse(request.toString('utf8')) as { id: string }; + void (async () => { + for (let sent = 0; sent < bytes; sent += 1) { + await setTimeout(BYTE_INTERVAL_MS); + socket.write(' '); + } + await setTimeout(BYTE_INTERVAL_MS); + socket.end( + `${JSON.stringify({ id, ok: true, result: { ok: 1 } })}\n`, + ); + })(); + }); + }); + await new Promise((resolve, reject) => { + server.once('error', reject); + server.listen(socketFile, () => { + server.off('error', reject); + resolve(); + }); + }); + return server; + } + + async function close(server: net.Server): Promise { + await new Promise((resolve) => { + server.close(() => { + resolve(); + }); + }); + } + + it('waits again after each byte of the host, so a slow host that sends bytes can take longer than the wait', async () => { + // The answer comes after 1,500 ms, and a byte comes every 250 ms. + const server = await listen(5); + try { + await expect( + sendRpc(join(tempDir, 'rpc.sock'), 'ping', {}, WAIT_MS), + ).resolves.toEqual({ ok: 1 }); + } finally { + await close(server); + } + }); + + it('fails with HOST_TIMEOUT after the wait without a byte', async () => { + // The answer comes after 250 ms with no byte before it. + const server = await listen(0); + try { + const error = await sendRpc( + join(tempDir, 'rpc.sock'), + 'ping', + {}, + SHORT_WAIT_MS, + ).catch((rejection: unknown) => rejection); + expect(error).toMatchObject({ + code: 'HOST_TIMEOUT', + details: { method: 'ping', timeoutMs: SHORT_WAIT_MS }, + }); + } finally { + await close(server); + } + }); +}); + describe('sendRpc response cap', () => { beforeEach(async () => { tempDir = await mkdtemp(join(tmpdir(), 'agent-tty-rpc-client-')); @@ -186,6 +270,188 @@ describe('sendRpc response cap', () => { ); }); +describe('sendRpc response cap with keep-alives', () => { + const MAX_CHARS = 67_108_864; + + beforeEach(async () => { + tempDir = await mkdtemp(join(tmpdir(), 'agent-tty-rpc-client-')); + }); + + afterEach(async () => { + await rm(tempDir, { recursive: true, force: true }); + tempDir = ''; + }); + + // A host that writes each chunk that `chunksFor(id)` returns in a write of + // its own, 20 ms apart, once it has seen the request. + async function listenChunks( + chunksFor: (id: string) => string[], + ): Promise { + const server = net.createServer((socket) => { + socket.on('error', () => undefined); + socket.once('data', (request: Buffer) => { + const { id } = JSON.parse(request.toString('utf8')) as { id: string }; + void (async () => { + for (const chunk of chunksFor(id)) { + socket.write(chunk); + await setTimeout(20); + } + socket.end(); + })(); + }); + }); + await new Promise((resolve, reject) => { + server.once('error', reject); + server.listen(join(tempDir, 'rpc.sock'), () => { + server.off('error', reject); + resolve(); + }); + }); + return server; + } + + it('keeps the spaces at the start of a chunk inside the response', async () => { + const server = await listenChunks((id) => [ + ' ', + ' ', + `{"id":${JSON.stringify(id)},"ok":true,"result":"a`, + ' b"}\n', + ]); + try { + const result = await sendRpc( + join(tempDir, 'rpc.sock'), + 'ping', + {}, + 5_000, + ); + + expect(result).toBe('a b'); + } finally { + await close(server); + } + }); + + it('takes a newline ahead of the response as an empty response line', async () => { + // Only a space is a keep-alive. A newline ends the line, so the response + // that follows it is never read. + const server = await listenChunks((id) => [ + ' ', + '\n', + `{"id":${JSON.stringify(id)},"ok":true,"result":"a"}\n`, + ]); + try { + const error = await sendRpc( + join(tempDir, 'rpc.sock'), + 'ping', + {}, + 5_000, + ).catch((rejection: unknown) => rejection); + + expect(error).toMatchObject({ code: 'RPC_ERROR' }); + } finally { + await close(server); + } + }); + + // A host that writes `keepAlives` spaces and then a success response line of + // `lineChars` characters, newline included. `sent.resultChars` is the length + // of the result in that line, once the host has seen the request. + async function listen( + keepAlives: number, + lineChars: number, + sameWrite: boolean, + ): Promise<{ server: net.Server; sent: { resultChars: number } }> { + const sent = { resultChars: 0 }; + const server = net.createServer((socket) => { + socket.on('error', () => undefined); + socket.once('data', (request: Buffer) => { + const { id } = JSON.parse(request.toString('utf8')) as { id: string }; + const head = `{"id":${JSON.stringify(id)},"ok":true,"result":"`; + const tail = '"}\n'; + sent.resultChars = lineChars - head.length - tail.length; + const line = `${head}${'x'.repeat(sent.resultChars)}${tail}`; + const spaces = ' '.repeat(keepAlives); + void (async () => { + if (sameWrite) { + socket.end(spaces + line); + return; + } + for (const space of spaces) { + socket.write(space); + await setTimeout(20); + } + socket.end(line); + })(); + }); + }); + await new Promise((resolve, reject) => { + server.once('error', reject); + server.listen(join(tempDir, 'rpc.sock'), () => { + server.off('error', reject); + resolve(); + }); + }); + return { server, sent }; + } + + async function close(server: net.Server): Promise { + await new Promise((resolve) => { + server.close(() => { + resolve(); + }); + }); + } + + it.each([ + { name: 'in chunks of their own', sameWrite: false }, + { name: 'in the same write as the response', sameWrite: true }, + ])( + 'takes a response of exactly 64 MiB after keep-alives that come $name', + { timeout: 30_000 }, + async ({ sameWrite }) => { + // The host checks the response line alone, and the line is at its cap. + const { server, sent } = await listen(3, MAX_CHARS, sameWrite); + try { + const result = await sendRpc( + join(tempDir, 'rpc.sock'), + 'ping', + {}, + 20_000, + ); + + expect(sent.resultChars).toBeGreaterThan(MAX_CHARS - 1_000); + expect(result).toHaveLength(sent.resultChars); + } finally { + await close(server); + } + }, + ); + + it( + 'refuses a response one character over 64 MiB after keep-alives', + { timeout: 30_000 }, + async () => { + const { server } = await listen(3, MAX_CHARS + 1, false); + try { + const error = await sendRpc( + join(tempDir, 'rpc.sock'), + 'ping', + {}, + 20_000, + ).catch((rejection: unknown) => rejection); + + expect(error).toMatchObject({ + code: 'PROTOCOL_ERROR', + retryable: false, + details: { method: 'ping', maxResponseChars: MAX_CHARS }, + }); + } finally { + await close(server); + } + }, + ); +}); + describe.skipIf(process.platform === 'win32')( 'sendRpc to a host that an older version started', () => { diff --git a/test/unit/host/rpcServer.test.ts b/test/unit/host/rpcServer.test.ts index 0c9b7775..4965d235 100644 --- a/test/unit/host/rpcServer.test.ts +++ b/test/unit/host/rpcServer.test.ts @@ -2,6 +2,7 @@ import { mkdtemp, rm } from 'node:fs/promises'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; import net from 'node:net'; +import { setTimeout } from 'node:timers/promises'; import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; @@ -371,6 +372,168 @@ describe('RpcServer large success responses', () => { ); }); +describe('RpcServer keep-alive', () => { + // A success result of the `snapshot` method, which is the smallest. + const snapshotResult = { + format: 'text', + sessionId: 'session-1', + capturedAtSeq: 1, + cols: 80, + rows: 24, + cursorRow: 0, + cursorCol: 0, + text: 'done', + }; + const WAIT_FOR_KEEP_ALIVE_MS = 800; + const KEEP_ALIVE_INTERVAL_MS = 250; + + beforeEach(async () => { + tempDir = await mkdtemp(join(tmpdir(), 'agent-tty-rpc-server-')); + }); + + afterEach(async () => { + await rm(tempDir, { recursive: true, force: true }); + tempDir = ''; + }); + + it('keeps a client waiting for as long as the handler reports progress', async () => { + const socketFile = join(tempDir, 'rpc.sock'); + const server = new RpcServer(socketFile, { + snapshot: async (_params, context) => { + // 1,250 ms in all, which is more than the 800 ms that the client waits + // for a byte, with a byte every 250 ms. + for (let round = 0; round < 5; round += 1) { + await setTimeout(KEEP_ALIVE_INTERVAL_MS); + context.keepAlive(); + } + return snapshotResult; + }, + }); + await server.listen(); + + try { + await expect( + sendRpc( + socketFile, + 'snapshot', + { format: 'text' }, + WAIT_FOR_KEEP_ALIVE_MS, + ), + ).resolves.toMatchObject({ format: 'text', text: 'done' }); + } finally { + await server.close(); + } + }); + + it('fails a client with HOST_TIMEOUT once the handler stops reporting progress', async () => { + const socketFile = join(tempDir, 'rpc.sock'); + const server = new RpcServer(socketFile, { + snapshot: async (_params, context) => { + context.keepAlive(); + await setTimeout(KEEP_ALIVE_INTERVAL_MS); + context.keepAlive(); + // The handler hangs until the client gives up. + await new Promise((resolve) => { + context.signal.addEventListener('abort', () => resolve(), { + once: true, + }); + }); + return snapshotResult; + }, + }); + await server.listen(); + + try { + const startedAt = performance.now(); + const error = await sendRpc( + socketFile, + 'snapshot', + { format: 'text' }, + WAIT_FOR_KEEP_ALIVE_MS, + ).catch((rejection: unknown) => rejection); + const elapsedMs = performance.now() - startedAt; + + expect(error).toMatchObject({ code: 'HOST_TIMEOUT' }); + // The wait started over at the last byte, 250 ms in. + expect(elapsedMs).toBeGreaterThanOrEqual( + KEEP_ALIVE_INTERVAL_MS + WAIT_FOR_KEEP_ALIVE_MS - 50, + ); + } finally { + await server.close(); + } + }); + + it('does not cut a large response when the handler reports progress after it returned', async () => { + const socketFile = join(tempDir, 'rpc.sock'); + const text = 'x'.repeat(4_000_000); + const server = new RpcServer(socketFile, { + snapshot: (_params, context) => { + // The response is on its way when this runs. A byte now would be a + // write after the end of the socket, which destroys the socket with + // part of the response still unsent. + setImmediate(() => { + context.keepAlive(); + }); + return Promise.resolve({ ...snapshotResult, text }); + }, + }); + await server.listen(); + + try { + await expect( + sendRpc(socketFile, 'snapshot', { format: 'text' }, 20_000), + ).resolves.toMatchObject({ format: 'text', text }); + } finally { + await server.close(); + } + }); + + it('sends only spaces ahead of the response line, so that an older client parses it', async () => { + const socketFile = join(tempDir, 'rpc.sock'); + const server = new RpcServer(socketFile, { + snapshot: async (_params, context) => { + context.keepAlive(); + await setTimeout(20); + context.keepAlive(); + await setTimeout(20); + context.keepAlive(); + return snapshotResult; + }, + }); + await server.listen(); + + try { + const client = net.connect({ path: socketFile }); + client.setEncoding('utf8'); + let received = ''; + client.on('data', (chunk: string) => { + received += chunk; + }); + const ended = new Promise((resolve, reject) => { + client.once('end', resolve); + client.once('error', reject); + }); + client.write( + `${JSON.stringify({ id: 'request-1', method: 'snapshot', params: { format: 'text' } })}\n`, + ); + await ended; + + // The first line of the bytes is what a client before the keep-alive + // gave to JSON.parse. + const [line, rest] = received.split('\n'); + expect(line?.startsWith(' {')).toBe(true); + expect(rest).toBe(''); + expect(JSON.parse(line ?? '')).toMatchObject({ + id: 'request-1', + ok: true, + result: { text: 'done' }, + }); + } finally { + await server.close(); + } + }); +}); + describe('RpcServer waitForRender results', () => { beforeEach(async () => { tempDir = await mkdtemp(join(tmpdir(), 'agent-tty-rpc-server-')); diff --git a/test/unit/renderer/libghosttyVtBackend.test.ts b/test/unit/renderer/libghosttyVtBackend.test.ts index a43af21c..02fa7514 100644 --- a/test/unit/renderer/libghosttyVtBackend.test.ts +++ b/test/unit/renderer/libghosttyVtBackend.test.ts @@ -1,10 +1,13 @@ import { describe, expect, it, vi } from 'vitest'; import { LibghosttyVtBackend } from '../../../src/renderer/libghosttyVt/backend.js'; +import type { ScreenshotOptions } from '../../../src/renderer/backend.js'; import type { LibghosttyVtNativeModule } from '../../../src/renderer/libghosttyVt/backend.js'; +import { iterateInRangeReplayEvents } from '../../../src/renderer/replayEvents.js'; import type { RenderProfileConfig, ReplayInput, + ReplayState, } from '../../../src/renderer/types.js'; import { createLogger } from '../../../src/util/logger.js'; @@ -665,6 +668,89 @@ describe('LibghosttyVtBackend', () => { expect(result.rendererBackend).toBe('ghostty-web'); }); + it('replays the fallback of a screenshot in slices, and reports its boot and each slice', async () => { + const fixture = createNativeFixture(); + const fallback = createFakeBackend({ + rendererBackend: 'ghostty-web', + writePng: false, + }); + // The fallback replays as a real backend does: it holds a position and + // ends at the last event that it replays. + const steps: string[] = []; + const replayedSeqs: number[] = []; + let lastSeq = -1; + const bootMock = vi.fn((): Promise => { + steps.push('boot'); + return Promise.resolve(); + }); + const replayToMock = vi.fn((input: ReplayInput): Promise => { + steps.push('replay'); + for (const event of iterateInRangeReplayEvents(input, lastSeq)) { + replayedSeqs.push(event.seq); + lastSeq = event.seq; + } + return Promise.resolve({ + lastSeq, + cols: input.initialCols, + rows: input.initialRows, + cursorRow: 0, + cursorCol: 0, + }); + }); + const screenshot = fallback.screenshot.bind(fallback); + const screenshotMock = vi.fn( + (outputPath: string, options?: ScreenshotOptions) => { + steps.push('screenshot'); + expect(options?.onReplayProgress).toBeTypeOf('function'); + return screenshot(outputPath, options); + }, + ); + fallback.boot = bootMock; + fallback.replayTo = replayToMock; + fallback.screenshot = screenshotMock; + const backend = createBackend(fixture, { fallbackFactory: () => fallback }); + const replayInput = createReplayInput({ + targetSeq: 299, + events: Array.from({ length: 300 }, (_, seq) => ({ + seq, + ts: '2026-03-20T12:00:00.000Z', + type: 'output' as const, + payload: { data: 'x' }, + })), + }); + await backend.boot(); + await backend.replayTo(replayInput); + + await backend.screenshot('/tmp/screenshot.png', { + onReplayProgress: () => { + steps.push('progress'); + }, + }); + + // The first report follows the boot. Each `replayTo` call has one. + const replayCalls = replayToMock.mock.calls.length; + expect(replayCalls).toBeGreaterThan(2); + expect(steps).toEqual([ + 'boot', + 'progress', + ...Array.from({ length: replayCalls }, () => [ + 'replay', + 'progress', + ]).flat(), + 'screenshot', + ]); + // The slices hold the events once, and the last call has the input. + const inputs = replayToMock.mock.calls.map(([input]) => input); + expect(inputs.at(-1)).toBe(replayInput); + expect( + inputs.slice(0, -1).every((input) => input.events.length < 300), + ).toBe(true); + // The fallback is cold, so it gets every event once, the first one too. + expect(replayedSeqs).toEqual(Array.from({ length: 300 }, (_, seq) => seq)); + expect(lastSeq).toBe(299); + expect(fallback.disposeMock).toHaveBeenCalledTimes(1); + }); + it('disposes native resources idempotently after screenshot fallback cleanup', async () => { const fixture = createNativeFixture(); const fallback = createFakeBackend({ diff --git a/test/unit/renderer/replaySlices.test.ts b/test/unit/renderer/replaySlices.test.ts new file mode 100644 index 00000000..8c758d3f --- /dev/null +++ b/test/unit/renderer/replaySlices.test.ts @@ -0,0 +1,620 @@ +import { describe, expect, it } from 'vitest'; + +import { + FIRST_SLICE_EVENTS, + MAX_SLICE_BYTES, + MAX_SLICE_FLUSHES, + TARGET_SLICE_MS, + countEventsThrough, + nextSliceSize, + replayInSlices, + sliceEnd, +} from '../../../src/renderer/replaySlices.js'; +import { iterateInRangeReplayEvents } from '../../../src/renderer/replayEvents.js'; +import type { + ReplayEvent, + ReplayInput, + ReplayState, +} from '../../../src/renderer/types.js'; + +function makeInput(eventCount: number, targetSeq?: number): ReplayInput { + return { + sessionId: 'session-01', + initialCols: 80, + initialRows: 24, + events: Array.from({ length: eventCount }, (_, seq) => ({ + seq, + ts: '2026-03-20T12:00:00.000Z', + type: 'output' as const, + payload: { data: 'x' }, + })), + targetSeq: targetSeq ?? eventCount - 1, + }; +} + +// Output events of the given sizes in bytes (ASCII, so one byte for each +// character), one event for each size. +function makeOutputEvents(sizes: readonly number[]): ReplayEvent[] { + return sizes.map((size, seq) => ({ + seq, + ts: '2026-03-20T12:00:00.000Z', + type: 'output' as const, + payload: { data: 'x'.repeat(size) }, + })); +} + +function makeInputOf(events: readonly ReplayEvent[]): ReplayInput { + return { + sessionId: 'session-01', + initialCols: 80, + initialRows: 24, + events: [...events], + targetSeq: events.length - 1, + }; +} + +type FlushingType = 'input_text' | 'marker' | 'resize'; + +// An event of a type that makes a renderer flush its output. +function flushingEvent(type: FlushingType, seq: number): ReplayEvent { + const ts = '2026-03-20T12:00:00.000Z'; + switch (type) { + case 'input_text': + return { seq, ts, type, payload: { data: 'a' } }; + case 'marker': + return { seq, ts, type, payload: { label: 'm' } }; + case 'resize': + return { seq, ts, type, payload: { cols: 80, rows: 24 } }; + } +} + +// `pairs` tiny outputs, each followed by an event of the given type. +function makeOutputInputPairs( + type: FlushingType, + pairs: number, + firstSeq = 0, +): ReplayEvent[] { + return Array.from({ length: 2 * pairs }, (_, index) => { + const seq = firstSeq + index; + return index % 2 === 0 + ? { + seq, + ts: '2026-03-20T12:00:00.000Z', + type: 'output' as const, + payload: { data: 'x' }, + } + : flushingEvent(type, seq); + }); +} + +interface Call { + readonly input: ReplayInput; + readonly appliedSeqs: readonly number[]; + // The events that the call replays, and the time that it takes. + readonly applied: readonly ReplayEvent[]; + readonly elapsedMs: number; +} + +// A backend with the rules of the real ones: it holds a position, skips the +// events that it holds, ends at the last event that it replays, and cannot go +// back. Each event takes `costPerEvent` ms of the clock that the test passes +// to `replayInSlices`, or `costOf(event)` ms when the test gives that. +function makeBackend( + options: { + startSeq?: number; + costPerEvent?: number; + costOf?: (event: ReplayEvent) => number; + } = {}, +) { + let lastAppliedSeq = options.startSeq ?? -1; + let clock = 0; + const calls: Call[] = []; + return { + calls, + now: () => clock, + get lastAppliedSeq() { + return lastAppliedSeq; + }, + replayTo(input: ReplayInput): Promise { + if (input.targetSeq < lastAppliedSeq) { + throw new Error( + `cannot rewind from seq ${String(lastAppliedSeq)} to ${String(input.targetSeq)}`, + ); + } + const appliedSeqs: number[] = []; + const applied: ReplayEvent[] = []; + let highestProcessedSeq = lastAppliedSeq; + let elapsedMs = 0; + for (const event of iterateInRangeReplayEvents(input, lastAppliedSeq)) { + appliedSeqs.push(event.seq); + applied.push(event); + highestProcessedSeq = event.seq; + elapsedMs += options.costOf?.(event) ?? options.costPerEvent ?? 0; + } + if (highestProcessedSeq < 0) { + highestProcessedSeq = input.targetSeq; + } + lastAppliedSeq = highestProcessedSeq; + clock += elapsedMs; + calls.push({ input, appliedSeqs, applied, elapsedMs }); + return Promise.resolve({ + lastSeq: lastAppliedSeq, + cols: 80, + rows: 24, + cursorRow: 0, + cursorCol: 0, + }); + }, + }; +} + +function allApplied(calls: readonly Call[]): number[] { + return calls.flatMap((call) => [...call.appliedSeqs]); +} + +describe('countEventsThrough', () => { + it('counts the events with a seq of at most the given seq', () => { + const events = [{ seq: 0 }, { seq: 1 }, { seq: 4 }, { seq: 9 }]; + + expect(countEventsThrough(events, -1)).toBe(0); + expect(countEventsThrough(events, 0)).toBe(1); + expect(countEventsThrough(events, 3)).toBe(2); + expect(countEventsThrough(events, 4)).toBe(3); + expect(countEventsThrough(events, 100)).toBe(4); + expect(countEventsThrough([], 5)).toBe(0); + }); +}); + +describe('sliceEnd', () => { + const KIB = 1024; + + it('ends at the event limit, and at the end of the events', () => { + const events = makeOutputEvents(Array.from({ length: 100 }, () => 1)); + + expect(sliceEnd(events, 0, 100, 32)).toBe(32); + expect(sliceEnd(events, 40, 100, 32)).toBe(72); + expect(sliceEnd(events, 90, 100, 32)).toBe(100); + expect(sliceEnd(events, 0, 10, 32)).toBe(10); + expect(sliceEnd(events, 100, 100, 32)).toBe(100); + }); + + it('ends before the event that would take the slice past the byte limit', () => { + const events = makeOutputEvents( + Array.from({ length: 10 }, () => 400 * KIB), + ); + + // Two events are 800 KiB. A third would make 1,200 KiB, past 1 MiB. + expect(sliceEnd(events, 0, 10, 1_000)).toBe(2); + expect(sliceEnd(events, 2, 10, 1_000)).toBe(4); + }); + + it('takes events up to exactly the byte limit', () => { + const events = makeOutputEvents( + Array.from({ length: 10 }, () => 512 * KIB), + ); + + expect(MAX_SLICE_BYTES).toBe(1024 * KIB); + expect(sliceEnd(events, 0, 10, 1_000)).toBe(2); + }); + + it('makes a slice of one event of its own when that event is past the byte limit', () => { + const events = makeOutputEvents([ + 2 * MAX_SLICE_BYTES, + 2 * MAX_SLICE_BYTES, + 10, + 10, + ]); + + expect(sliceEnd(events, 0, 4, 1_000)).toBe(1); + expect(sliceEnd(events, 1, 4, 1_000)).toBe(2); + expect(sliceEnd(events, 2, 4, 1_000)).toBe(4); + }); + + it('counts bytes of UTF-8, not characters', () => { + // Each event has 200,000 characters of 3 bytes, which are 600,000 bytes. + const events = makeOutputEvents([]); + for (let seq = 0; seq < 4; seq += 1) { + events.push({ + seq, + ts: '2026-03-20T12:00:00.000Z', + type: 'output', + payload: { data: '€'.repeat(200_000) }, + }); + } + + expect(sliceEnd(events, 0, 4, 1_000)).toBe(1); + }); +}); + +describe('sliceEnd with events that follow output', () => { + it.each(['input_text', 'marker', 'resize'])( + 'ends before the %s that would flush output for more than the limit', + (type) => { + const events = makeOutputInputPairs(type, 100); + + expect(MAX_SLICE_FLUSHES).toBe(16); + // The events at the odd indexes flush: the 16th is event 31. Event 32 + // is output, and event 33 would be the 17th flush. + expect(sliceEnd(events, 0, 200, 1_000)).toBe(33); + // The next slice starts with the event 33, which follows no output of + // the slice, so it flushes nothing. The flushes are the events 35 to 65, + // and event 67 would be the 17th. + expect(sliceEnd(events, 33, 200, 1_000)).toBe(67); + }, + ); + + it('does not count an event that follows no output, or one that follows another such event', () => { + const lone = Array.from({ length: 1_000 }, (_, seq) => + flushingEvent('marker', seq), + ); + const afterOneOutput = [ + ...makeOutputEvents([1]), + ...Array.from({ length: 1_000 }, (_, index) => + flushingEvent('input_text', index + 1), + ), + ]; + + expect(sliceEnd(lone, 0, 1_000, 2_000)).toBe(1_000); + // One flush, at the first input. + expect(sliceEnd(afterOneOutput, 0, 1_001, 2_000)).toBe(1_001); + }); + + it('always takes the first event of a slice', () => { + const events = makeOutputInputPairs('marker', 100); + + // Each slice holds the event that it starts with, whatever it is. + for (let start = 0; start < 200; start += 1) { + expect(sliceEnd(events, start, 200, 1_000)).toBeGreaterThan(start); + } + }); +}); + +describe('nextSliceSize', () => { + it('aims at the target time with the measured speed', () => { + expect(nextSliceSize(100, 250)).toBe(200); + expect(nextSliceSize(100, 1000)).toBe(50); + expect(nextSliceSize(100, 500)).toBe(100); + }); + + it('grows or shrinks by at most four times', () => { + expect(nextSliceSize(32, 1)).toBe(128); + expect(nextSliceSize(32, 0)).toBe(128); + expect(nextSliceSize(32, 60_000)).toBe(8); + expect(nextSliceSize(1, 60_000)).toBe(1); + }); +}); + +describe('replayInSlices', () => { + it('replays every event once, in order, and ends at the target', async () => { + const input = makeInput(1_000); + const backend = makeBackend({ costPerEvent: 5 }); + const states: ReplayState[] = []; + + await replayInSlices( + backend, + input, + -1, + (state) => states.push(state), + backend.now, + ); + + expect(allApplied(backend.calls)).toEqual(input.events.map((e) => e.seq)); + expect(backend.lastAppliedSeq).toBe(999); + expect(backend.calls.length).toBeGreaterThan(2); + // Each call reports one state, and the states move forward. + expect(states).toHaveLength(backend.calls.length); + const reached = states.map((state) => state.lastSeq); + expect(reached).toEqual(reached.toSorted((a, b) => a - b)); + }); + + it('makes the last call with the whole input, which a backend keeps', async () => { + const input = makeInput(500); + const backend = makeBackend({ costPerEvent: 5 }); + + await replayInSlices(backend, input, -1, () => undefined, backend.now); + + expect(backend.calls.length).toBeGreaterThan(1); + expect(backend.calls.at(-1)?.input).toBe(input); + for (const call of backend.calls.slice(0, -1)) { + expect(call.input.events.length).toBeLessThan(input.events.length); + expect(call.input.targetSeq).toBe(input.targetSeq); + } + }); + + it('keeps the first slice small and sizes the next from its time', async () => { + const input = makeInput(2_000); + // 15.625 ms for each event, so that the first slice of 32 events takes + // 500 ms and the next has the same size. + const slow = makeBackend({ costPerEvent: 15.625 }); + await replayInSlices(slow, input, -1, () => undefined, slow.now); + expect(slow.calls[0]?.appliedSeqs).toHaveLength(FIRST_SLICE_EVENTS); + expect(slow.calls[1]?.appliedSeqs).toHaveLength(FIRST_SLICE_EVENTS); + + // A backend that takes no time makes each slice four times larger. + const fast = makeBackend(); + await replayInSlices(fast, input, -1, () => undefined, fast.now); + expect(fast.calls.map((call) => call.appliedSeqs.length)).toEqual([ + 32, 128, 512, 1_328, + ]); + }); + + it('replays a backlog of at most one slice in one call, with the input as it is', async () => { + const input = makeInput(FIRST_SLICE_EVENTS); + const backend = makeBackend(); + + await replayInSlices(backend, input, -1, () => undefined, backend.now); + + expect(backend.calls).toHaveLength(1); + expect(backend.calls[0]?.input).toBe(input); + }); + + it('takes the position of a backend that is ahead of the caller', async () => { + const input = makeInput(1_000); + const backend = makeBackend({ startSeq: 600, costPerEvent: 5 }); + + // The caller thinks that the backend has nothing. A slice with a target + // below the backend would fail to rewind it. + await replayInSlices(backend, input, -1, () => undefined, backend.now); + + expect(allApplied(backend.calls)).toEqual( + input.events.slice(601).map((event) => event.seq), + ); + expect(backend.lastAppliedSeq).toBe(999); + // The first slice finds the backend at 600, and the next goes on from it. + expect(backend.calls[0]?.appliedSeqs).toEqual([]); + expect(backend.calls[1]?.appliedSeqs[0]).toBe(601); + }); + + it('replays only the events after the position that the caller gives', async () => { + const input = makeInput(1_000); + const backend = makeBackend({ startSeq: 399, costPerEvent: 5 }); + + await replayInSlices(backend, input, 399, () => undefined, backend.now); + + expect(backend.calls[0]?.appliedSeqs[0]).toBe(400); + expect(allApplied(backend.calls)).toHaveLength(600); + }); + + it('does not replay events past the target', async () => { + const input = makeInput(1_000, 449); + const backend = makeBackend({ costPerEvent: 5 }); + + await replayInSlices(backend, input, -1, () => undefined, backend.now); + + expect(allApplied(backend.calls)).toEqual( + input.events.slice(0, 450).map((event) => event.seq), + ); + expect(backend.lastAppliedSeq).toBe(449); + }); + + it('stops at a slice that fails and leaves the earlier slices replayed', async () => { + const input = makeInput(500); + const backend = makeBackend({ costPerEvent: 5 }); + const original = backend.replayTo.bind(backend); + let calls = 0; + backend.replayTo = (replayInput) => { + calls += 1; + if (calls === 3) { + return Promise.reject(new Error('page crashed')); + } + return original(replayInput); + }; + const reported: number[] = []; + + await expect( + replayInSlices( + backend, + input, + -1, + (state) => reported.push(state.lastSeq), + backend.now, + ), + ).rejects.toThrow('page crashed'); + + // Two slices finished and were reported before the failure. + expect(reported).toHaveLength(2); + expect(backend.lastAppliedSeq).toBe(reported.at(-1)); + }); + describe('with a limit on the bytes of a slice', () => { + // Replay at the speed of #358: a tiny event is cheap, and 64 KiB of + // output takes 12 ms. + const TINY_MS = 0.02; + const BIG_EVENT_BYTES = 64 * 1024; + const BIG_EVENT_MS = 12; + + function costOf(event: ReplayEvent): number { + return event.type === 'output' && event.payload.data.length > 1_000 + ? BIG_EVENT_MS + : TINY_MS; + } + + function bytesOf(events: readonly ReplayEvent[]): number { + return events.reduce( + (sum, event) => + sum + + (event.type === 'output' ? Buffer.byteLength(event.payload.data) : 0), + 0, + ); + } + + it('never makes a call that takes long, after thousands of tiny events and then big ones', async () => { + const input = makeInputOf( + makeOutputEvents([ + ...Array.from({ length: 9_000 }, () => 4), + ...Array.from({ length: 500 }, () => BIG_EVENT_BYTES), + ]), + ); + const backend = makeBackend({ costOf }); + + await replayInSlices(backend, input, -1, () => undefined, backend.now); + + // The size of a slice has grown to thousands of events by the time of + // the big ones, and the last call is no exception. + expect( + Math.max(...backend.calls.map((call) => call.elapsedMs)), + ).toBeLessThan(2 * TARGET_SLICE_MS); + expect(backend.calls.at(-1)?.input).toBe(input); + expect(allApplied(backend.calls)).toEqual(input.events.map((e) => e.seq)); + for (const call of backend.calls) { + expect( + call.applied.length <= 1 || bytesOf(call.applied) <= MAX_SLICE_BYTES, + ).toBe(true); + } + }); + + it('replays a backlog of a few events in slices when their output is past the limit', async () => { + // 20 events are fewer than the first slice of 32 events. + const input = makeInputOf( + makeOutputEvents(Array.from({ length: 20 }, () => BIG_EVENT_BYTES)), + ); + const backend = makeBackend({ costOf }); + + await replayInSlices(backend, input, -1, () => undefined, backend.now); + + expect(backend.calls.map((call) => call.applied.length)).toEqual([16, 4]); + expect(backend.calls.at(-1)?.input).toBe(input); + }); + + it('replays an event that is past the limit by itself, and goes on', async () => { + const input = makeInputOf( + makeOutputEvents([ + 2 * MAX_SLICE_BYTES, + 10, + 3 * MAX_SLICE_BYTES, + 2 * MAX_SLICE_BYTES, + 10, + ]), + ); + const backend = makeBackend(); + + await replayInSlices(backend, input, -1, () => undefined, backend.now); + + // A big event leaves no room in its slice, and no slice takes it as its + // second event, so each event has a call of its own. + expect(backend.calls.map((call) => call.appliedSeqs)).toEqual([ + [0], + [1], + [2], + [3], + [4], + ]); + }); + + it('does not take a slice of big events for the speed of the events that follow', async () => { + // Big events that cost nothing make 16-event slices. Their time says + // nothing about 5,000 events that follow, each of which takes 5 ms. + const input = makeInputOf( + makeOutputEvents([ + ...Array.from({ length: 500 }, () => BIG_EVENT_BYTES), + ...Array.from({ length: 5_000 }, () => 4), + ]), + ); + const backend = makeBackend({ + costOf: (event) => (event.seq < 500 ? 0 : 5), + }); + + await replayInSlices(backend, input, -1, () => undefined, backend.now); + + expect( + Math.max(...backend.calls.map((call) => call.elapsedMs)), + ).toBeLessThan(2 * TARGET_SLICE_MS); + }); + }); + + describe('with a limit on the flushes of a slice', () => { + // Replay at the speed of #358: a tiny output is cheap, and an event that + // follows output flushes it and waits for a frame, which takes 34 ms. + const TINY_MS = 0.005; + const FLUSH_MS = 34; + + function costOf(event: ReplayEvent): number { + return event.type === 'output' ? TINY_MS : FLUSH_MS; + } + + it.each(['input_text', 'marker', 'resize'])( + 'never makes a call that takes long, after thousands of tiny outputs and then pairs of output and %s', + async (type) => { + const tiny = makeOutputEvents(Array.from({ length: 30_000 }, () => 1)); + const input = makeInputOf([ + ...tiny, + ...makeOutputInputPairs(type, 3_000, tiny.length), + ]); + const backend = makeBackend({ costOf }); + + await replayInSlices(backend, input, -1, () => undefined, backend.now); + + // Without the limit, the size has grown to tens of thousands of events + // by the time of the pairs, and one slice takes minutes. + expect( + Math.max(...backend.calls.map((call) => call.elapsedMs)), + ).toBeLessThan(2 * TARGET_SLICE_MS); + expect(backend.calls.at(-1)?.input).toBe(input); + expect(allApplied(backend.calls)).toEqual( + input.events.map((event) => event.seq), + ); + }, + ); + + it('never makes a call that takes long, when the pairs come after thousands of cheap events of other types', async () => { + // Markers that follow markers flush nothing, so they cost nothing and + // make the size grow, as tiny outputs do. + const markers = Array.from({ length: 30_000 }, (_, seq) => + flushingEvent('marker', seq), + ); + const input = makeInputOf([ + ...markers, + ...makeOutputInputPairs('input_text', 3_000, markers.length), + ]); + const backend = makeBackend({ + costOf: (event) => (event.seq < markers.length ? 0 : costOf(event)), + }); + + await replayInSlices(backend, input, -1, () => undefined, backend.now); + + expect( + Math.max(...backend.calls.map((call) => call.elapsedMs)), + ).toBeLessThan(2 * TARGET_SLICE_MS); + }); + }); + + describe('with a backend that reports a position that does not fit', () => { + function makeFakeBackend(lastSeq: number) { + const calls: ReplayInput[] = []; + return { + calls, + replayTo(input: ReplayInput): Promise { + calls.push(input); + if (calls.length > 20) { + return Promise.reject(new Error('runaway replay')); + } + return Promise.resolve({ + lastSeq, + cols: 80, + rows: 24, + cursorRow: 0, + cursorCol: 0, + }); + }, + }; + } + + it('fails when a slice does not end at its last event', async () => { + const backend = makeFakeBackend(5); + + await expect( + replayInSlices(backend, makeInput(500), -1, () => undefined), + ).rejects.toThrow(/a replay slice must end at its last event/); + expect(backend.calls).toHaveLength(1); + }); + + it('fails when a slice ends past the target', async () => { + // The target is event 299, and the backend reports event 450. + const backend = makeFakeBackend(450); + + await expect( + replayInSlices(backend, makeInput(500, 299), -1, () => undefined), + ).rejects.toThrow(/a replay slice must end at its last event/); + expect(backend.calls).toHaveLength(1); + }); + }); +}); diff --git a/test/unit/screenshot/capture.test.ts b/test/unit/screenshot/capture.test.ts index f80ee3a7..0044039d 100644 --- a/test/unit/screenshot/capture.test.ts +++ b/test/unit/screenshot/capture.test.ts @@ -7,6 +7,7 @@ import { describe, expect, it } from 'vitest'; import { createTemporarySessionDir } from '../../helpers.js'; +import type { ScreenshotOptions } from '../../../src/renderer/backend.js'; import type { ScreenshotResult } from '../../../src/renderer/types.js'; import { @@ -216,6 +217,36 @@ describe('screenshot capture', () => { expect(falseResult.cursorVisible).toBe(false); }); + it('hands the replay progress callback to the backend, with or without showCursor', async () => { + const observed: ScreenshotOptions[] = []; + const onReplayProgress = (): void => undefined; + + for (const [index, showCursor] of [undefined, true].entries()) { + const sessionId = `session-progress-${String(index)}`; + const backend = createFakeBackend({ + onScreenshot: (_outputPath, screenshotOptions) => { + observed.push(screenshotOptions ?? {}); + }, + resultOverrides: { sessionId }, + }); + + await captureScreenshotResult({ + backend, + sessionDir: await createSessionDir(sessionId), + profileName: 'reference-dark', + expectedSessionId: sessionId, + onReplayProgress, + ...(showCursor === undefined ? {} : { showCursor }), + }); + } + + expect(observed).toEqual([ + { onReplayProgress }, + { showCursor: true, onReplayProgress }, + ]); + expect(observed[0]?.onReplayProgress).toBe(onReplayProgress); + }); + it('rejects empty sessionDir, profileName, or expectedSessionId before invoking the backend', async () => { const sessionDirectory = await createSessionDir(); const observed: Array = [];