diff --git a/docs/internal/EVENTS.md b/docs/internal/EVENTS.md index 9cefb77da..f783d44b6 100644 --- a/docs/internal/EVENTS.md +++ b/docs/internal/EVENTS.md @@ -124,7 +124,7 @@ consumer, just operator diagnostics, so it's a `warn` log line (`daemon.driver-d |---|---|---|---|---| | `baseline.saved` | name, version, platform, model, OS version, mode, image tag (when the device has one), size in bytes (when it could be counted), lease id (the lease the save ended), requester id (the admin principal that ran `baseline save`) | once per save, after the `baselines` row is committed and before the saved device is deleted | BaselineService | implemented | | `baseline.removed` | name, version, reason (`operator` for `baseline remove`, `nuke` for `nuke --delete-devices`, `replaced` when a newer save of the name dropped it), requester id (the principal that ran the command; absent for a removal nobody asked for) | once per version deleted, after its row is deleted | BaselineService | implemented | -| `baseline.stale` | name, version, reason (the driver's words: the runtime, image or emulator that changed) | when a check finds a baseline stale that had no stale reason before (again after it was fresh in between). The check runs at start, after `component.removed`, and before `baseline list` and `doctor` answer | BaselineService | implemented | +| `baseline.stale` | name, version, reason (the driver's words: the runtime, image or emulator that changed, or an iOS baseline saved without a lineage record) | when a check finds a baseline stale that had no stale reason before (again after it was fresh in between). The check runs at start, after `component.removed`, after a save, and before `baseline list` and `doctor` answer | BaselineService | implemented | `BaselineService` (`src/core/baselines.ts`) is the only emitter; drivers keep and destroy the sources but never emit (architecture rule 5). diff --git a/docs/internal/adr/0076-an-ios-clone-names-only-its-own-folders-before-its-first-boot.md b/docs/internal/adr/0076-an-ios-clone-names-only-its-own-folders-before-its-first-boot.md index 2a632bfef..f2006aa57 100644 --- a/docs/internal/adr/0076-an-ios-clone-names-only-its-own-folders-before-its-first-boot.md +++ b/docs/internal/adr/0076-an-ios-clone-names-only-its-own-folders-before-its-first-boot.md @@ -1,6 +1,6 @@ # 0076. An iOS clone names only its own folders before its first boot -- **Status:** Accepted — not yet implemented +- **Status:** Accepted - **Date:** 2026-11-15, revised 2026-11-16 - **Issue:** [#1561](https://github.com/callstackincubator/simlock/issues/1561), bug [#1548](https://github.com/callstackincubator/simlock/issues/1548) - **Narrows:** [ADR 0061](0061-every-virtual-device-is-a-copy-of-a-source-and-a-baseline-is-a-named-source.md) diff --git a/docs/internal/adr/README.md b/docs/internal/adr/README.md index aa0d55fae..b413bc3e1 100644 --- a/docs/internal/adr/README.md +++ b/docs/internal/adr/README.md @@ -110,7 +110,7 @@ the status is stale. | [0073](0073-a-worker-may-enrol-the-phones-plugged-into-it.md) | A worker may enrol the phones plugged into it | Accepted; narrows 0022 §3, §5 and §6 | | [0074](0074-an-admission-command-ranks-or-refuses-a-gateway-request-and-an-ended-chosen-lease-id-is-remembered.md) | An admission command ranks or refuses a gateway request, and an ended chosen lease id is remembered | Accepted; narrows 0005 items 10 and 11, decision 5 and requirements 14 and 30; supersedes 0020 §1 in part | | [0075](0075-a-foldable-simulator-streams-the-panel-in-use-and-a-stream-may-change-size.md) | A foldable simulator streams the panel in use, and a stream may change size while it runs | Accepted; narrows 0031's Consequences and 0052 §4 | -| [0076](0076-an-ios-clone-names-only-its-own-folders-before-its-first-boot.md) | An iOS clone names only its own folders before its first boot | Accepted — not yet implemented; narrows 0061 §1, §2 and §4 | +| [0076](0076-an-ios-clone-names-only-its-own-folders-before-its-first-boot.md) | An iOS clone names only its own folders before its first boot | Accepted; narrows 0061 §1, §2 and §4 | | [0077](0077-the-repository-is-a-workspace-and-its-packages-release-together-under-one-version.md) | The repository is a workspace, and its packages release together under one version | Accepted | | [0078](0078-an-android-worker-tries-its-default-image-tag-first-and-falls-back-to-none.md) | An Android worker tries its default image tag first, and a lease falls back to none | Accepted | | [0079](0079-one-capture-per-device-is-shared-by-every-viewer-and-every-capture-sends-a-key-frame-every-5-s.md) | One capture per device is shared by every viewer, and every capture sends a key frame at least every 5 s | Accepted; narrows 0045 §5 | diff --git a/docs/internal/flows/startup.md b/docs/internal/flows/startup.md index 6c09e3d67..e71155fbf 100644 --- a/docs/internal/flows/startup.md +++ b/docs/internal/flows/startup.md @@ -68,7 +68,8 @@ Before step 16, a driver with `deviceTcp` sweeps the reverse rules into 47000-47 `device.shutdown` (initiator `startup-interrupted-reclaim`). 3. Spent deletes: `shutdown` devices that may not be granted again are deleted inline. A failed delete is logged; the device stays `shutdown`. - 4. Warm pool and capacity observer start. The pool closes the lease rows of its learned-kind + 4. Baselines `checkStale` (a failure is logged), so a baseline the driver calls stale is known before the first lease. + 5. Warm pool and capacity observer start. The pool closes the lease rows of its learned-kind record whose lease the registry no longer holds, and its first pass boots the recorded kinds. 22. `queue.changed` with the first depth. Converge done → health `running`. diff --git a/e2e/helpers/env.test.ts b/e2e/helpers/env.test.ts index 14423231b..bde30bb51 100644 --- a/e2e/helpers/env.test.ts +++ b/e2e/helpers/env.test.ts @@ -212,3 +212,69 @@ describe("withSharedDaemon with keepSeeds", () => { } }); }); + +describe("withSharedDaemon with keepSeeds and an iOS seed", () => { + it("gives the next env the iOS seed's lineage and its file rows, even when its simulator set is outside the home", async () => { + const laneHome = join(tmpdir(), `simlock-lane-${randomBytes(3).toString("hex")}`); + const sources = `${laneHome}-sources`; + process.env.SIMLOCK_E2E_LANE_HOME = laneHome; + const udid = "11111111-2222-3333-4444-555555555555"; + const lineage = JSON.stringify(["AAAAAAAA-1111-2222-3333-BBBBBBBBBBBB"]); + const files = [ + { kind: "file", relative_path: "data/Library/a.plist" }, + { kind: "link", relative_path: "data/Library/b" }, + { kind: "unreadable", relative_path: "data/Library/c" }, + ]; + try { + await mkdir(join(sources, udid), { recursive: true }); + const first = await laneEnv(); + try { + const database = new DatabaseSync(join(first.home, "state.db")); + try { + database.exec("PRAGMA busy_timeout=5000"); + database + .prepare( + `INSERT INTO ios_source_data (source_id, kind, state, spec_key, udid, set_path, + device_type_id, runtime_id, model, os_version, maker, lineage, created_at) + VALUES ('seed_ios', 'seed', 'ready', ?, ?, ?, 'type', 'runtime', 'iPhone 16', '26.5', 'maker', ?, 1)`, + ) + .run("iPhone 16\u0000runtime", udid, sources, lineage); + for (const file of files) { + database + .prepare( + "INSERT INTO ios_source_files (source_id, relative_path, kind) VALUES ('seed_ios', ?, ?)", + ) + .run(file.relative_path, file.kind); + } + } finally { + database.close(); + } + } finally { + await first.teardown(); + } + + const second = await laneEnv(); + try { + const database = new DatabaseSync(join(second.home, "state.db")); + try { + expect(database.prepare("SELECT source_id, lineage FROM ios_source_data").all()).toEqual([ + { lineage, source_id: "seed_ios" }, + ]); + expect( + database + .prepare("SELECT kind, relative_path FROM ios_source_files ORDER BY relative_path") + .all(), + ).toEqual(files); + } finally { + database.close(); + } + } finally { + await second.teardown(); + } + } finally { + delete process.env.SIMLOCK_E2E_LANE_HOME; + await rm(laneHome, { force: true, recursive: true }); + await rm(sources, { force: true, recursive: true }); + } + }); +}); diff --git a/e2e/helpers/lane-home.ts b/e2e/helpers/lane-home.ts index c0001ac7b..579d61032 100644 --- a/e2e/helpers/lane-home.ts +++ b/e2e/helpers/lane-home.ts @@ -13,6 +13,8 @@ type Row = Record; interface KeptSeed { readonly table: string; readonly row: Row; + /** why: ADR 0076: an iOS seed's file record is its lineage's other half, so it is kept with the row. */ + readonly files?: readonly Row[]; } interface SeedTable { @@ -77,13 +79,21 @@ export async function restoreKeptSeeds(home: string): Promise { const seeds = JSON.parse(await readFile(join(home, KEPT_SEEDS_FILE), "utf8")) as KeptSeed[]; const database = new DatabaseSync(join(home, "state.db")); try { - for (const { table, row } of seeds) { + for (const { table, row, files } of seeds) { const columns = Object.keys(row); database .prepare( `INSERT INTO ${table} (${columns.join(", ")}) VALUES (${columns.map(() => "?").join(", ")})`, ) .run(...columns.map((column) => row[column] ?? null)); + for (const file of files ?? []) { + const fileColumns = Object.keys(file); + database + .prepare( + `INSERT INTO ios_source_files (${fileColumns.join(", ")}) VALUES (${fileColumns.map(() => "?").join(", ")})`, + ) + .run(...fileColumns.map((column) => file[column] ?? null)); + } } } finally { database.close(); @@ -138,7 +148,13 @@ async function readySeeds(home: string): Promise { const database = new DatabaseSync(path, { readOnly: true }); try { return SEED_TABLES.flatMap(({ table }) => - readyRows(database, table).map((row) => ({ row, table })), + readyRows(database, table).map((row) => ({ + row, + table, + ...(table === "ios_source_data" + ? { files: textRows(database, "ios_source_files", "source_id", String(row["source_id"])) } + : {}), + })), ).filter((seed) => existsSync(folderOf(seed))); } finally { database.close(); @@ -151,6 +167,17 @@ async function readySeeds(home: string): Promise { * is read as bytes and decoded here. */ function readyRows(database: DatabaseSync, table: string): Row[] { + return textRows(database, table, undefined, undefined, "kind = 'seed' AND state = 'ready'"); +} + +/** why: the rows of a table with every text column read as bytes (see `readyRows`); a table the driver never made has none. */ +function textRows( + database: DatabaseSync, + table: string, + column?: string, + value?: string, + where = "1 = 1", +): Row[] { const columns = database.prepare(`PRAGMA table_info(${table})`).all() as { name: string; type: string; @@ -159,9 +186,10 @@ function readyRows(database: DatabaseSync, table: string): Row[] { const selected = columns.map(({ name, type }) => type === "TEXT" ? `CAST(${name} AS BLOB) AS ${name}` : name, ); + const filter = column === undefined ? where : `${column} = ?`; return database - .prepare(`SELECT ${selected.join(", ")} FROM ${table} WHERE kind = 'seed' AND state = 'ready'`) - .all() + .prepare(`SELECT ${selected.join(", ")} FROM ${table} WHERE ${filter}`) + .all(...(value === undefined ? [] : [value])) .map((row) => Object.fromEntries( Object.entries(row).map(([name, value]) => [ diff --git a/e2e/slow-ios-source-isolation.test.ts b/e2e/slow-ios-source-isolation.test.ts new file mode 100644 index 000000000..a0a8d68b2 --- /dev/null +++ b/e2e/slow-ios-source-isolation.test.ts @@ -0,0 +1,198 @@ +import { createHash } from "node:crypto"; +import { existsSync } from "node:fs"; +import { mkdir, readdir, readFile, readlink, stat, writeFile } from "node:fs/promises"; +import { homedir } from "node:os"; +import { join } from "node:path"; +import { DatabaseSync } from "node:sqlite"; +import { describe, expect, it } from "vitest"; + +import { withDaemon } from "./helpers/index.js"; +import type { TestEnv } from "./helpers/env.js"; +import { sweepStaleDeviceSets } from "./helpers/ios-device-set.js"; +import { DEVICE_READY_MS } from "./helpers/wait.js"; + +interface Grant { + readonly deviceSet: string; + readonly leaseId: string; + readonly udid: string; +} + +const LOG_ROOT = join(homedir(), "Library", "Logs", "CoreSimulator"); +const BASELINE = "isolation"; + +async function lease(env: TestEnv, extra: readonly string[] = []): Promise { + const granted = await env.cli( + ["lease", "--platform", "ios", "--agent-id", "isolation", "--detach", ...extra], + { timeout: DEVICE_READY_MS }, + ); + expect(granted.code, `lease failed: ${granted.stderr}`).toBe(0); + const grant = granted.json as { + device: { driverDeviceId: string }; + environment: Record; + lease: { id: string }; + }; + return { + deviceSet: grant.environment.SIMLOCK_IOS_DEVICE_SET as string, + leaseId: grant.lease.id, + udid: grant.device.driverDeviceId, + }; +} + +async function release(env: TestEnv, grant: Grant): Promise { + const released = await env.cli(["release", grant.leaseId]); + expect(released.code, `release failed: ${released.stderr}`).toBe(0); +} + +/** why: a file's size, modification time and hash are what "unchanged after the row turned ready" compares (ADR 0076). */ +async function fingerprint(folder: string): Promise> { + const result: Record = {}; + for (const entry of await readdir(folder, { recursive: true, withFileTypes: true })) { + const path = join(entry.parentPath, entry.name); + if (entry.isSymbolicLink()) result[path] = `link ${await readlink(path)}`; + else if (entry.isFile()) { + const { mtimeMs, size } = await stat(path); + // why: a system service's cache file inside a source can be unreadable to the test (EPERM); the product records it as unreadable too, so its size and time stand for it. + const hash = await readFile(path).then( + (bytes) => createHash("sha256").update(bytes).digest("hex"), + (error: NodeJS.ErrnoException) => { + if (error.code === "EPERM" || error.code === "EACCES") return "unreadable"; + throw error; + }, + ); + result[path] = `${size} ${mtimeMs} ${hash}`; + } + } + return result; +} + +function sourceFolder(home: string, kind: "seed" | "baseline"): string { + const database = new DatabaseSync(join(home, "state.db"), { readOnly: true }); + try { + database.exec("PRAGMA busy_timeout=5000"); + const row = database + .prepare( + "SELECT set_path, udid FROM ios_source_data WHERE kind = ? AND state = 'ready' ORDER BY created_at DESC", + ) + .get(kind) as { set_path: string; udid: string } | undefined; + if (row === undefined) throw new Error(`no ready ${kind} source in state.db`); + return join(row.set_path, row.udid); + } finally { + database.close(); + } +} + +function seedThrowaway(home: string): string { + const database = new DatabaseSync(join(home, "state.db"), { readOnly: true }); + try { + const row = database + .prepare("SELECT lineage FROM ios_source_data WHERE kind = 'seed' AND state = 'ready'") + .get() as { lineage: string } | undefined; + const [throwaway] = JSON.parse(row?.lineage ?? "[]") as string[]; + if (throwaway === undefined) throw new Error("the ready seed has no lineage"); + return throwaway; + } finally { + database.close(); + } +} + +/** why: ADR 0076: a copy's `Logs` link names the copy's own folder, never the throwaway's or the saved-from lease's. */ +async function expectOwnLogs(grant: Grant, strangers: readonly string[]): Promise { + const target = await readlink(join(grant.deviceSet, grant.udid, "data", "Library", "Logs")); + expect(target.toUpperCase()).toContain(grant.udid.toUpperCase()); + for (const stranger of strangers) expect(target.toUpperCase()).not.toContain(stranger); +} + +async function expectNoFolderNamedAfter( + names: readonly string[], + deviceSet: string, +): Promise { + for (const root of [deviceSet, LOG_ROOT]) { + const entries = existsSync(root) ? await readdir(root) : []; + for (const name of names) { + expect( + entries.filter((entry) => entry.toUpperCase() === name), + `${root} holds a folder named after ${name}`, + ).toEqual([]); + } + } +} + +async function swapMilliseconds(env: TestEnv, udid: string): Promise { + const log = await readFile(env.logPath, "utf8"); + for (const line of log.split("\n")) { + const match = new RegExp( + `swapped \\d+ files and \\d+ links in (\\d+) ms for ${udid}`, + "i", + ).exec(line); + if (match !== null) return Number(match[1]); + } + throw new Error(`the daemon log has no swap line for ${udid}`); +} + +// why: this lane needs the real simctl toolchain on Xcode 27, hence darwin-only; it never installs a runtime. +describe.skipIf(process.platform !== "darwin")( + "iOS source isolation (real simctl)", + { tags: ["slow", "ios"] }, + () => { + it( + "leaves the seed unchanged after three leases and a saved baseline's source unchanged after two, gives every copy a Logs link to its own folder, brings back no folder named after the throwaway or the saved-from lease, and swaps a baseline's copy in under 1 s", + { timeout: 2_400_000 }, + async () => { + await sweepStaleDeviceSets(); + const env = await withDaemon({ + configOverrides: { log: { level: "debug" } }, + driver: "real", + }); + + // why: case 1, the seed, after three leases of one kind. + const first = await lease(env); + const seedFolder = sourceFolder(env.home, "seed"); + const throwaway = seedThrowaway(env.home).toUpperCase(); + const seedBefore = await fingerprint(seedFolder); + await expectOwnLogs(first, [throwaway]); + await release(env, first); + for (let more = 0; more < 2; more += 1) { + const next = await lease(env); + await expectOwnLogs(next, [throwaway]); + await release(env, next); + } + expect(await fingerprint(seedFolder), "the seed changed after its row was ready").toEqual( + seedBefore, + ); + + // why: case 2, a baseline saved from a used lease, after two leases of it. + const saved = await lease(env); + const marker = join(saved.deviceSet, saved.udid, "data", "Documents", "isolation.txt"); + await mkdir(join(saved.deviceSet, saved.udid, "data", "Documents"), { recursive: true }); + await writeFile(marker, "used"); + const baseline = await env.cli(["baseline", "save", BASELINE, "--lease", saved.leaseId], { + timeout: 600_000, + }); + expect(baseline.code, `baseline save failed: ${baseline.stderr}`).toBe(0); + const baselineFolder = sourceFolder(env.home, "baseline"); + const baselineBefore = await fingerprint(baselineFolder); + const strangers = [throwaway, saved.udid.toUpperCase()]; + const copies: Grant[] = []; + for (let n = 0; n < 2; n += 1) { + const copy = await lease(env, ["--baseline", BASELINE]); + copies.push(copy); + // why: case 3, each copy's Logs link names its own folder. + await expectOwnLogs(copy, strangers); + await release(env, copy); + } + expect( + await fingerprint(baselineFolder), + "the baseline's source changed after its row was ready", + ).toEqual(baselineBefore); + + // why: case 4, no folder named after the throwaway or the saved-from lease comes back. + await expectNoFolderNamedAfter(strangers, first.deviceSet); + + // why: case 5, a baseline's copy swaps in under 1 s, so a fall back to a full read is caught. + for (const copy of copies) { + expect(await swapMilliseconds(env, copy.udid)).toBeLessThan(1000); + } + }, + ); + }, +); diff --git a/src/auth/admin-secret.test.ts b/src/auth/admin-secret.test.ts index 85dd1effc..84272bb3b 100644 --- a/src/auth/admin-secret.test.ts +++ b/src/auth/admin-secret.test.ts @@ -102,6 +102,15 @@ describe("AdminSecretManager", () => { directorySize: () => { throw new Error("verify() must not touch the filesystem"); }, + writeAt: () => { + throw new Error("verify() must not touch the filesystem"); + }, + readlink: () => { + throw new Error("verify() must not touch the filesystem"); + }, + replaceSymlink: () => { + throw new Error("verify() must not touch the filesystem"); + }, writeFileExclusive: () => { throw new Error("verify() must not touch the filesystem"); }, @@ -189,6 +198,15 @@ describe("AdminSecretManager", () => { directorySize: () => { throw new Error("must not touch the filesystem"); }, + writeAt: () => { + throw new Error("must not touch the filesystem"); + }, + readlink: () => { + throw new Error("must not touch the filesystem"); + }, + replaceSymlink: () => { + throw new Error("must not touch the filesystem"); + }, writeFileExclusive: () => { throw new Error("must not touch the filesystem"); }, diff --git a/src/core/AGENTS.md b/src/core/AGENTS.md index 4f77c216a..32bf432c7 100644 --- a/src/core/AGENTS.md +++ b/src/core/AGENTS.md @@ -58,7 +58,7 @@ Baselines (ADR 0061 §4): - One save or remove at a time. A save: leasing's `releaseToSave` ends the lease, then `ReclaimCoordinator.retireSaved` calls `driver.saveSource` under the `reclaim` claim. A driver failure quarantines the device and answers `BASELINE_SAVE_FAILED`. - A replace keeps the leased devices of older versions. An older version is dropped only when no device record and no provision (`hold`) has it; that is checked after a save, on every `device.deleted` and at startup. - `remove` refuses a baseline a lease uses, and destroys each source before its row. `removeAll` (nuke) deletes rows first. -- `checkStale` runs before `list` and `doctor` and after `component.removed`. +- `checkStale` runs before `list` and `doctor`, after `component.removed`, after a save, and at startup. A failure is logged and goes on, except before `list`, which fails with it. ## Known gaps diff --git a/src/core/baselines.ts b/src/core/baselines.ts index 69aef6550..d5b45636c 100644 --- a/src/core/baselines.ts +++ b/src/core/baselines.ts @@ -1,6 +1,6 @@ import type { EventBus } from "../bus/index.js"; import type { BaselineEntry } from "../contract/index.js"; -import type { Clock } from "../ports/index.js"; +import type { Clock, Logger } from "../ports/index.js"; import type { StateDatabase } from "../state/index.js"; import { assertVirtual, type Platform, type VirtualDevice } from "./domain.js"; import type { Driver, DriverSource } from "./driver.js"; @@ -105,6 +105,7 @@ export interface BaselineServiceOptions { readonly decisions: SerializedDecision; readonly eventBus: Pick; readonly lifecycle: Pick; + readonly logger?: Logger | undefined; readonly reclaim: Pick; readonly registry: { readonly snapshot: Pick }; readonly state: StateDatabase; @@ -170,6 +171,13 @@ export class BaselineService { }); await this.#destroyIdleOlderDevices(baseline.name, baseline.version); await this.#dropReplaced(request.requesterId); + try { + await this.checkStale(); + } catch (error: unknown) { + this.options.logger?.warn( + `could not check baselines after a save: ${error instanceof Error ? error.message : String(error)}`, + ); + } return { baseline, leaseId: request.leaseId }; } diff --git a/src/core/create-core.ts b/src/core/create-core.ts index 78b3ed19e..d6a224895 100644 --- a/src/core/create-core.ts +++ b/src/core/create-core.ts @@ -259,6 +259,7 @@ export function createCore(options: CoreOptions): Core { eventBus: options.eventBus, leaseSave: () => port("leaseSave"), lifecycle: deviceLifecycle, + logger: options.logger, reclaim, registry, state: options.state, @@ -444,6 +445,13 @@ export function createCore(options: CoreOptions): Core { await startup.converge(read); await dropReplacedBaselines(); await listStartupSources(options.drivers, options.logger); + try { + await baselines.checkStale(); + } catch (error: unknown) { + options.logger?.warn( + `could not check baselines at startup: ${error instanceof Error ? error.message : String(error)}`, + ); + } warmPool.start(); capacityObserver.start(); }, diff --git a/src/daemon/dispatcher-baseline-lease.test.ts b/src/daemon/dispatcher-baseline-lease.test.ts index ec43a2321..234c108e1 100644 --- a/src/daemon/dispatcher-baseline-lease.test.ts +++ b/src/daemon/dispatcher-baseline-lease.test.ts @@ -1,6 +1,6 @@ -import { describe, expect, it } from "vitest"; +import { describe, expect, it, vi } from "vitest"; import { FakeDriver } from "../core/testing.js"; -import { FakeClock } from "../ports/index.js"; +import { FakeClock, JsonLinesLogger, MemoryLogSink } from "../ports/index.js"; import { DispatchError } from "./dispatcher.js"; import { classifyError } from "./error-code.js"; import { buildDispatcher, gibibyte, session } from "./test-dispatcher.js"; @@ -269,3 +269,104 @@ describe("Dispatcher: a baseline lease is released", () => { }, ); }); + +describe("Dispatcher: a baseline the driver calls stale", () => { + const reason = "saved before simulators stopped sharing folders; save it again"; + + async function built(logger?: JsonLinesLogger) { + const clock = new FakeClock(1_000); + const harness = await buildDispatcher({ + clock, + driverOptions: { knownModels: ["iPhone 17 Pro"] }, + ...(logger === undefined ? {} : { engine: { logger } }), + warmPool: { baselineLearnAfter: 1 }, + }); + const real = harness.driver.listSources.bind(harness.driver); + return { + ...harness, + clock, + markStale: () => + vi + .spyOn(harness.driver, "listSources") + .mockImplementation(async () => + (await real()).map((source) => ({ ...source, staleReason: reason })), + ), + provisions: () => + harness.driver.calls.filter((call) => call.operation === "provision").length, + request: () => + harness.dispatcher + .dispatch("lease.request", { baseline: "checkout", platform: "ios" }, session()) + .catch((error: unknown) => error), + restart: async () => harness.engine.core.converge(await harness.engine.core.readStartup()), + save: async () => { + const first = await harness.dispatcher.dispatch( + "lease.request", + { model: "iPhone 17 Pro", osVersion: "26.5", platform: "ios" }, + session(), + ); + await harness.dispatcher.dispatch( + "baseline.save", + { leaseId: first.lease.id, name: "checkout" }, + session({ role: "admin" }), + ); + }, + }; + } + + async function learned(harness: Awaited>): Promise { + await harness.restart(); + await harness.save(); + const granted = await harness.request(); + await harness.dispatcher.dispatch( + "lease.release", + { leaseId: (granted as { lease: { id: string } }).lease.id }, + session(), + ); + } + + it("answers BASELINE_STALE from the first request after the worker starts or after its save, and warms no copy of it", async () => { + const control = await built(); + await learned(control); + const warmed = control.provisions(); + await control.engine.core.passWarmPool(); + expect(control.provisions(), "a fresh baseline's learned kind is warmed").toBeGreaterThan( + warmed, + ); + + const restarted = await built(); + await learned(restarted); + restarted.markStale(); + await restarted.restart(); + const provisioned = restarted.provisions(); + await restarted.engine.core.passWarmPool(); + expect(restarted.provisions(), "no copy of a stale baseline is warmed").toBe(provisioned); + expect(await restarted.request()).toMatchObject({ + code: "BASELINE_STALE", + message: `baseline checkout is stale: ${reason}`, + }); + + const saved = await built(); + saved.markStale(); + await saved.save(); + expect(await saved.request()).toMatchObject({ + code: "BASELINE_STALE", + message: `baseline checkout is stale: ${reason}`, + }); + }); + + it("goes on, logging it, when the check at startup or after a save fails", async () => { + const sink = new MemoryLogSink(); + const harness = await built(new JsonLinesLogger({ clock: new FakeClock(0), sink })); + vi.spyOn(harness.engine.core.baselines, "checkStale").mockRejectedValue( + new Error("catalog unreadable"), + ); + + await expect(harness.restart()).resolves.toBeUndefined(); + await expect(harness.save()).resolves.toBeUndefined(); + + expect(sink.records.map((record) => record.message)).toEqual([ + "could not check baselines at startup: catalog unreadable", + "could not check baselines after a save: catalog unreadable", + ]); + }); +}); diff --git a/src/daemon/driver-discovery.ts b/src/daemon/driver-discovery.ts index 3687c43dd..97d570bcd 100644 --- a/src/daemon/driver-discovery.ts +++ b/src/daemon/driver-discovery.ts @@ -36,6 +36,7 @@ import { type IdGenerator, type Logger, NodeBytePipeLauncher, + NodeSqliteOpener, NodeTcpListenerFactory, NodeTcpStreamConnector, type ProcessRunner, @@ -203,6 +204,7 @@ async function discoverIosDriver( onSilenceFailed: logSilenceFailure(logger), onWindowFailed: logWindowFailure(logger), onSlimmed: emitSlimDiagnostic(options.eventBus), + sqliteOpener: new NodeSqliteOpener(), screenCapture: { baseEnvironment: process.env, command: PACKAGED_SCREEN_CAPTURE, diff --git a/src/daemon/test-dispatcher.ts b/src/daemon/test-dispatcher.ts index 65e459870..add8d1a90 100644 --- a/src/daemon/test-dispatcher.ts +++ b/src/daemon/test-dispatcher.ts @@ -191,7 +191,7 @@ export async function buildDispatcher( readonly otherStallDrivers?: readonly FakeDriver[]; /** why: a second platform's driver for the leasing engine, for a test that leases on both. */ readonly extraDrivers?: readonly FakeDriver[]; - readonly engine?: Pick[0], "defaultModes">; + readonly engine?: Pick[0], "defaultModes" | "logger">; readonly physicalDevices?: readonly SeededPhysicalDevice[]; /** why: rows a test writes to `state.db` before the registry loads. */ readonly seedState?: (state: StateDatabase) => void; diff --git a/src/drivers/ios/AGENTS.md b/src/drivers/ios/AGENTS.md index 0365b214b..52a027220 100644 --- a/src/drivers/ios/AGENTS.md +++ b/src/drivers/ios/AGENTS.md @@ -23,16 +23,16 @@ Resolve and boot: - The overrides folder lives outside the device set (`launchdOverridesRoot`), so `destroy` removes it itself after `simctl delete`. - After every boot and before the readback, `runAfterBoot` (`boot-courtesies.ts`) runs the volume step, then the window step. Each only reports its failure (`onSilenceFailed`, `onWindowFailed`). - Volume (`ios.simulator.audio: false`): apps play at volume 0. A baseline source is not silenced. -- Window (`ios.simulator.headless: false`): DeviceHub is opened in the background (the front app stays in front) and told not to shut down started devices when it quits. Only a non-zero exit of that open means DeviceHub is missing: Simulator.app is then opened instead, detached, so quitting it leaves the simulator running. +- Window (`ios.simulator.headless: false`): DeviceHub is opened in the background and told not to shut down started devices when it quits. Only a non-zero exit of that open means DeviceHub is missing: Simulator.app is then opened instead, detached, so quitting it leaves the simulator running. Seeds (ADR 0061): - A simulator is a clone of its kind's seed. A seed is known by its `ios_source_data` row, never by name, and lives in the sources set: `sources` in the owned set, an owned root of its own under `default`. It is in no `listManaged` or `simlock simctl list`. -- A seed is made from a throwaway (ADR 0076): `simlock-seed-` is booted once, shut down, cloned unbooted into the sources set, then deleted with its log folder (`logsRoot`); `saveSource` deletes the saved-from lease's log folder too. A log folder that cannot be deleted is logged and the make or save goes on; a throwaway that cannot be deleted fails the make. Only its `making` row (`throwaway_udid`) deletes a throwaway, at the first `listSources` of a later process or the next make of its kind. -- Every clone (a seed's copy, `saveSource`'s own clone, a `provision` with `source`) has its `data/var/db/lsd/com.apple.LaunchServices-*.csstore` deleted before its first boot: it records each app under the original device's folder. A clone whose drop fails is deleted. +- A seed is made from a throwaway (ADR 0076): `simlock-seed-` is booted once, shut down, cloned unbooted into the sources set, then deleted with its log folder; `saveSource` deletes the saved-from lease's log folder too. A log folder that cannot be deleted is logged and the make or save goes on; a throwaway that cannot be deleted fails the make. +- A source records its lineage and, in `ios_source_files`, the files and links naming it, after flushing its WALs. A clone keeps its source's lineage and, before its first boot, has its `com.apple.LaunchServices-*.csstore` deleted and those files and links swapped to its own UDID; a failed drop or swap deletes it. No lineage: a seed is remade, a baseline stale. - Two provisions of a kind make one seed; clones of a seed and its deletion take turns. - A `clone` or `create` waits `ios.simulator.createTimeoutMs`. At the limit it fails but `simctl` runs on: the UDID it later prints is deleted, the seed's deletion waits for it, and one silent 5 minutes more is killed and logged. -- A `making` row of another process is swept at the first `listSources`; one of this process, once older than any make takes, at the next make of its kind. +- A `making` row of another process is swept at the first `listSources`, with its throwaway; one of this process, once older than any make takes, at the next make of its kind. - A seed that `simctl delete` calls `Invalid device` is already gone: its row is dropped. `removeComponent` deletes the runtime's seeds first. Physical, passthrough, screen: diff --git a/src/drivers/ios/device-data.test.ts b/src/drivers/ios/device-data.test.ts index 40ad35a18..142312594 100644 --- a/src/drivers/ios/device-data.test.ts +++ b/src/drivers/ios/device-data.test.ts @@ -13,6 +13,7 @@ describeDeviceDataTable("iosDeviceDataTable", iosDeviceDataTable, [ base, { ...base, slimMarkToken: "tok-1", slimSignature: "sig-1" }, { ...base, slimSignature: "sig-2" }, + { ...base, lineage: ["9F2C0000-0000-0000-0000-000000000002", base.udid] }, ]); describe("iosDeviceDataTable write", () => { @@ -23,6 +24,7 @@ describe("iosDeviceDataTable write", () => { "required key of the wrong type": { ...base, udid: 5 }, "optional key of the wrong type": { ...base, slimSignature: 5 }, "extra key": { ...base, extra: "x" }, + "lineage entry that is not a UDID": { ...base, lineage: ["not-a-udid"] }, }; const state = await openMemoryState([iosDeviceDataTable]); for (const [label, value] of Object.entries(cases)) { diff --git a/src/drivers/ios/device-data.ts b/src/drivers/ios/device-data.ts index a65f897b4..94134f279 100644 --- a/src/drivers/ios/device-data.ts +++ b/src/drivers/ios/device-data.ts @@ -1,4 +1,5 @@ import type { DeviceDataTable } from "../../core/index.js"; +import { UDID_PATTERN } from "./device-set-leftovers.js"; interface IosDeviceDataRow { readonly udid: string; @@ -7,6 +8,7 @@ interface IosDeviceDataRow { readonly runtime_id: string; readonly slim_signature: string | null; readonly slim_mark_token: string | null; + readonly lineage: string | null; } interface IosDriverData { @@ -16,19 +18,23 @@ interface IosDriverData { readonly slimMarkToken?: string; readonly slimSignature?: string; readonly udid: string; + readonly lineage?: readonly string[]; } const REQUIRED_KEYS = ["deviceTypeId", "name", "runtimeId", "udid"] as const; const OPTIONAL_KEYS = ["slimMarkToken", "slimSignature"] as const; +const KNOWN_KEYS = new Set([...REQUIRED_KEYS, ...OPTIONAL_KEYS, "lineage"]); function isIosDriverData(value: unknown): value is IosDriverData { if (typeof value !== "object" || value === null) return false; const record = value as Record; - const known = new Set([...REQUIRED_KEYS, ...OPTIONAL_KEYS]); return ( - Object.keys(record).every((key) => known.has(key)) && + Object.keys(record).every((key) => KNOWN_KEYS.has(key)) && REQUIRED_KEYS.every((key) => typeof record[key] === "string") && - OPTIONAL_KEYS.every((key) => record[key] === undefined || typeof record[key] === "string") + OPTIONAL_KEYS.every((key) => record[key] === undefined || typeof record[key] === "string") && + (record["lineage"] === undefined || + (Array.isArray(record["lineage"]) && + record["lineage"].every((udid) => typeof udid === "string" && UDID_PATTERN.test(udid)))) ); } @@ -43,12 +49,14 @@ export const iosDeviceDataTable: DeviceDataTable = { slim_signature TEXT, slim_mark_token TEXT ) STRICT`, + // why: ADR 0076: the lineage of the source this device was copied from, kept so a save knows it without reading the copy; null on a device made before. + "ALTER TABLE ios_device_data ADD COLUMN lineage TEXT", ], name: "ios-device-data", read(database, deviceId) { const row = database .prepare( - "SELECT udid, name, device_type_id, runtime_id, slim_signature, slim_mark_token FROM ios_device_data WHERE device_id = ?", + "SELECT udid, name, device_type_id, runtime_id, slim_signature, slim_mark_token, lineage FROM ios_device_data WHERE device_id = ?", ) .get(deviceId) as IosDeviceDataRow | undefined; if (row === undefined) throw new Error(`No ios driver data for device ${deviceId}`); @@ -58,10 +66,11 @@ export const iosDeviceDataTable: DeviceDataTable = { runtimeId: row.runtime_id, ...(row.slim_mark_token === null ? {} : { slimMarkToken: row.slim_mark_token }), ...(row.slim_signature === null ? {} : { slimSignature: row.slim_signature }), + ...(row.lineage === null ? {} : { lineage: JSON.parse(row.lineage) as string[] }), udid: row.udid, }; }, - version: 1, + version: 2, write(database, deviceId, driverData) { if (!isIosDriverData(driverData)) { throw new Error(`Invalid ios driver data for device ${deviceId}`); @@ -69,11 +78,12 @@ export const iosDeviceDataTable: DeviceDataTable = { const data = driverData; database .prepare( - `INSERT INTO ios_device_data (device_id, udid, name, device_type_id, runtime_id, slim_signature, slim_mark_token) - VALUES (?, ?, ?, ?, ?, ?, ?) + `INSERT INTO ios_device_data (device_id, udid, name, device_type_id, runtime_id, slim_signature, slim_mark_token, lineage) + VALUES (?, ?, ?, ?, ?, ?, ?, ?) ON CONFLICT (device_id) DO UPDATE SET udid = excluded.udid, name = excluded.name, device_type_id = excluded.device_type_id, runtime_id = excluded.runtime_id, - slim_signature = excluded.slim_signature, slim_mark_token = excluded.slim_mark_token`, + slim_signature = excluded.slim_signature, slim_mark_token = excluded.slim_mark_token, + lineage = excluded.lineage`, ) .run( deviceId, @@ -83,6 +93,7 @@ export const iosDeviceDataTable: DeviceDataTable = { data.runtimeId, data.slimSignature ?? null, data.slimMarkToken ?? null, + data.lineage === undefined ? null : JSON.stringify(data.lineage), ); }, }; diff --git a/src/drivers/ios/driver-options.ts b/src/drivers/ios/driver-options.ts index 10da15860..9c969ad8b 100644 --- a/src/drivers/ios/driver-options.ts +++ b/src/drivers/ios/driver-options.ts @@ -8,6 +8,7 @@ import type { Logger, ProcessRunner, SqliteDatabase, + SqliteOpener, TcpListenerFactory, TcpProbe, TcpStreamConnector, @@ -35,6 +36,8 @@ export interface IosSimctlDriverOptions { readonly coreSimulatorRoot?: string; // why: where CoreSimulator keeps each simulator's log folder, which `simctl delete` leaves behind; with none given the driver deletes no log folder. readonly logsRoot?: string; + // why: how a source's WALs are flushed when it is made; with none given the driver flushes nothing. + readonly sqliteOpener?: SqliteOpener; // why: where launchd_sim keeps each simulator's overrides folder; a test points it elsewhere. readonly launchdOverridesRoot?: string; readonly onSlimmed?: (fact: SlimmedFact) => void; diff --git a/src/drivers/ios/index-device-set.test.ts b/src/drivers/ios/index-device-set.test.ts index c0eea0a21..243f91a42 100644 --- a/src/drivers/ios/index-device-set.test.ts +++ b/src/drivers/ios/index-device-set.test.ts @@ -122,6 +122,13 @@ describe("IosSimctlDriver device set", () => { : []), { match: { args: argvOf(step), command: "xcrun" }, + ...(step.args[0] === "clone" && step.args[1] === throwawayUdid + ? { + onSpawn: () => { + void filesystem.mkdirp(`${sourcesSet}/${seedUdid}`); + }, + } + : {}), ...(step.stdout === undefined ? {} : { result: { code: 0, stderr: "", stdout: step.stdout } }), diff --git a/src/drivers/ios/index-save-source-lifecycle.test.ts b/src/drivers/ios/index-save-source-lifecycle.test.ts index 1beb0023e..8e1d87ca1 100644 --- a/src/drivers/ios/index-save-source-lifecycle.test.ts +++ b/src/drivers/ios/index-save-source-lifecycle.test.ts @@ -66,7 +66,7 @@ describe("IosSimctlDriver", () => { const device = { address: driverData.udid, deviceId: driverData.udid, - driverData, + driverData: { ...driverData, lineage: ["AAAAAAAA-1111-2222-3333-BBBBBBBBBBBB"] }, physical: false, } as const; const cloneInvocation = simctl("clone", driverData.udid, baselineName, sourcesSet); @@ -274,7 +274,7 @@ describe("IosSimctlDriver", () => { await expect(driver.provision({ ...spec, mode: "slim" })).resolves.toEqual({ address: driverData.udid, deviceId: driverData.udid, - driverData, + driverData: { ...driverData, lineage: [] }, }); }); diff --git a/src/drivers/ios/index-seed-provision.test.ts b/src/drivers/ios/index-seed-provision.test.ts index dd74be977..0d856ec74 100644 --- a/src/drivers/ios/index-seed-provision.test.ts +++ b/src/drivers/ios/index-seed-provision.test.ts @@ -26,6 +26,7 @@ import { listFixture, listInvocation, scopedRunners, + plantSeedClone, seedUdid, seededDatabase, sourcesSimctl, @@ -76,6 +77,7 @@ describe("IosSimctlDriver", () => { full, ); await plantManagedDevice(filesystem); + await plantSeedClone(filesystem); await driver.provision(spec); @@ -116,12 +118,13 @@ describe("IosSimctlDriver", () => { const filesystem = new ModeRecordingFilesystem(); const driver = await createDriver(runner, new FakeClock(), filesystem, database); await plantManagedDevice(filesystem); + await plantSeedClone(filesystem); await filesystem.mkdirp(`${deviceRoot}/${copyUdid}/data`); await expect(driver.provision(spec)).resolves.toEqual({ address: driverData.udid, deviceId: driverData.udid, - driverData, + driverData: { ...driverData, lineage: [throwawayUdid.toUpperCase()] }, }); expect( rowsWhenCreated, @@ -163,6 +166,7 @@ describe("IosSimctlDriver", () => { ]); const filesystem = new MemoryFilesystem(); await plantManagedDevice(filesystem); + await plantSeedClone(filesystem); const driver = await createDriver(runner, new FakeClock(), filesystem, database); const error = await driver.provision(spec).catch((caught: unknown) => caught); @@ -189,6 +193,7 @@ describe("IosSimctlDriver", () => { const clock = new FakeClock(); const driver = await createDriver(runner, clock, filesystem); await plantManagedDevice(filesystem); + await plantSeedClone(filesystem); await filesystem.mkdirp(`${deviceRoot}/${copyUdid}/data`); const clones = (): number => runner.calls.filter((call) => call.args[3] === "clone" && call.args[4] === seedUdid).length; @@ -266,6 +271,7 @@ describe("IosSimctlDriver", () => { scopedRunners.push(runner); const filesystem = new MemoryFilesystem(); await plantManagedDevice(filesystem); + await plantSeedClone(filesystem); const clock = new FakeClock(); const config = { simulator: { audio: true, createTimeoutMs: 120_000, headless: true } }; const driver = await IosSimctlDriver.create({ @@ -393,6 +399,7 @@ describe("IosSimctlDriver", () => { }; const filesystem = new MemoryFilesystem(); await plantManagedDevice(filesystem); + await plantSeedClone(filesystem); const clock = new FakeClock(); const driver = await IosSimctlDriver.create({ database: seeded ? await seededDatabase() : await memorySourceDatabase(), diff --git a/src/drivers/ios/index-seed-record.test.ts b/src/drivers/ios/index-seed-record.test.ts new file mode 100644 index 000000000..ee7e400a6 --- /dev/null +++ b/src/drivers/ios/index-seed-record.test.ts @@ -0,0 +1,311 @@ +import { afterEach, describe, expect, it } from "vitest"; +import { + FakeClock, + FakeSqliteOpener, + MemoryFilesystem, + ScriptedProcessRunner, + type SqliteDatabase, +} from "../../ports/index.js"; +import { memorySourceDatabase, plantBytes } from "./testing.js"; +import { deviceRoot, driverData, plantManagedDevice, simctl } from "./test-slim.js"; +import { + RecordingLogger, + createDriver, + listFixture, + listInvocation, + logsRoot, + scopedRunners, + seedUdid, + sourcesSimctl, + spec, + throwawaySeedSteps, + throwawayUdid, +} from "./test-index.js"; + +afterEach(() => { + scopedRunners.length = 0; +}); + +const copyUdid = "CCCCCCCC-4444-5555-6666-DDDDDDDDDDDD"; +const lineageUdid = throwawayUdid.toUpperCase(); +const sourcesSet = `${deviceRoot}/sources`; +const seedFolder = `${sourcesSet}/${seedUdid}`; +const SQLITE_HEADER = Buffer.from("SQLite format 3\u0000"); +const MEBIBYTE = 1_048_576; + +function utf16be(text: string): Buffer { + return Buffer.from(text, "utf16le").swap16(); +} + +class UnreadableFiles extends MemoryFilesystem { + readonly unreadable = new Set(); + + override async openFile(path: string): ReturnType { + if (this.unreadable.has(path)) { + throw Object.assign(new Error(`EACCES: permission denied, open '${path}'`), { + code: "EACCES", + }); + } + return super.openFile(path); + } +} + +interface Made { + readonly database: SqliteDatabase; + readonly error: unknown; + readonly filesystem: UnreadableFiles; + readonly logger: RecordingLogger; + readonly opener: FakeSqliteOpener; + readonly runner: ScriptedProcessRunner; +} + +/** why: the first provision of a kind, with `plant` writing what the seed's clone holds and `fail` naming what the fake filesystem and opener refuse. */ +async function provisionNewSeed(options: { + readonly plant: (filesystem: UnreadableFiles) => Promise; + readonly failingPaths?: readonly string[]; + readonly unreadablePaths?: readonly string[]; + readonly failingDatabases?: readonly string[]; + readonly keepingFrames?: readonly string[]; +}): Promise { + const database = await memorySourceDatabase(); + const filesystem = new UnreadableFiles(); + await plantManagedDevice(filesystem); + await filesystem.mkdirp(`${logsRoot}/${throwawayUdid}`); + await options.plant(filesystem); + // why: `simctl clone` gives the copy what the seed holds, and the scripted runner cannot. + await filesystem.mkdirp(seedFolder); + await filesystem.copyTree(seedFolder, `${deviceRoot}/${copyUdid}`, { timeoutMs: 1 }); + for (const path of options.failingPaths ?? []) filesystem.defineFailure(path, "EACCES"); + for (const path of options.unreadablePaths ?? []) filesystem.unreadable.add(path); + const opener = new FakeSqliteOpener(filesystem, { + failing: options.failingDatabases ?? [], + keepingFrames: options.keepingFrames ?? [], + }); + const logger = new RecordingLogger(); + const runner = new ScriptedProcessRunner([ + { match: listInvocation, result: { code: 0, stderr: "", stdout: listFixture } }, + ...throwawaySeedSteps(), + { + match: sourcesSimctl("clone", seedUdid, driverData.name, deviceRoot), + result: { code: 0, stderr: "", stdout: `${copyUdid}\n` }, + }, + ...[sourcesSimctl("shutdown", seedUdid), sourcesSimctl("delete", seedUdid)].map((match) => ({ + match, + })), + ]); + const driver = await createDriver( + runner, + new FakeClock(), + filesystem, + database, + logger, + undefined, + { + sqliteOpener: opener, + }, + ); + const error = await driver.provision(spec).then( + () => undefined, + (caught: unknown) => caught, + ); + return { database, error, filesystem, logger, opener, runner }; +} + +const baselineUdid = "EEEEEEEE-7777-8888-9999-FFFFFFFFFFFF"; + +/** why: a save of the lease `driverData` made, with the clone's files planted, so the baseline's record can be read back. */ +async function saveBaseline(options: { + readonly lineage?: readonly string[]; + readonly plant: (filesystem: UnreadableFiles) => Promise; +}): Promise { + const database = await memorySourceDatabase(); + const filesystem = new UnreadableFiles(); + await plantManagedDevice(filesystem); + await options.plant(filesystem); + const opener = new FakeSqliteOpener(filesystem); + const logger = new RecordingLogger(); + const runner = new ScriptedProcessRunner([ + { match: listInvocation, result: { code: 0, stderr: "", stdout: listFixture } }, + { match: simctl("shutdown", driverData.udid) }, + { + match: simctl("clone", driverData.udid, "simlock-baseline-device-1", sourcesSet), + result: { code: 0, stderr: "", stdout: `${baselineUdid}\n` }, + }, + ]); + const driver = await createDriver( + runner, + new FakeClock(), + filesystem, + database, + logger, + undefined, + { + sqliteOpener: opener, + }, + ); + const error = await driver + .saveSource({ + address: driverData.udid, + deviceId: driverData.udid, + driverData: { + ...driverData, + ...(options.lineage === undefined ? {} : { lineage: options.lineage }), + }, + physical: false, + }) + .then( + () => undefined, + (caught: unknown) => caught, + ); + return { database, error, filesystem, logger, opener, runner }; +} + +function recorded(database: SqliteDatabase): unknown[] { + return database + .prepare("SELECT relative_path, kind FROM ios_source_files ORDER BY relative_path") + .all(); +} + +function storedLineage(database: SqliteDatabase): unknown { + return database.prepare("SELECT lineage, saved_without_lineage FROM ios_source_data").get(); +} + +describe("IosSimctlDriver source records", () => { + it("a source made is flushed, its lineage stored and every file or link naming the lineage recorded before it turns ready", async () => { + const naming = Buffer.from(`root /Devices/${lineageUdid}/data`); + const big = Buffer.alloc(64 * MEBIBYTE + 100); + naming.copy(big, big.length - naming.length); + + const seed = await provisionNewSeed({ + plant: async (filesystem) => { + await plantBytes(filesystem, `${seedFolder}/data/a.plist`, naming); + await plantBytes(filesystem, `${seedFolder}/data/big.bin`, big); + await plantBytes(filesystem, `${seedFolder}/data/wide.bin`, utf16be(lineageUdid)); + await plantBytes(filesystem, `${seedFolder}/data/none.txt`, Buffer.from("nothing")); + await plantBytes(filesystem, `${seedFolder}/data/one.db`, SQLITE_HEADER); + await plantBytes(filesystem, `${seedFolder}/data/one.db-wal`, naming); + await plantBytes(filesystem, `${seedFolder}/data/one.db-shm`, Buffer.from("index")); + await plantBytes(filesystem, `${seedFolder}/data/two.db`, SQLITE_HEADER); + await plantBytes(filesystem, `${seedFolder}/data/two.db-wal`, naming); + filesystem.defineSymlink(`${seedFolder}/data/Logs`, `${logsRoot}/${lineageUdid}`); + }, + }); + + expect(seed.error).toBeUndefined(); + expect(seed.opener.checkpoints.sort()).toEqual([ + `${seedFolder}/data/one.db`, + `${seedFolder}/data/two.db`, + ]); + expect(storedLineage(seed.database)).toEqual({ + lineage: JSON.stringify([lineageUdid]), + saved_without_lineage: 0, + }); + expect(recorded(seed.database)).toEqual([ + { kind: "link", relative_path: "data/Logs" }, + { kind: "file", relative_path: "data/a.plist" }, + { kind: "file", relative_path: "data/big.bin" }, + { kind: "file", relative_path: "data/one.db" }, + { kind: "file", relative_path: "data/two.db" }, + { kind: "file", relative_path: "data/wide.bin" }, + ]); + expect(seed.logger.warnings).toEqual([]); + + const unflushed = await provisionNewSeed({ + failingDatabases: [`${seedFolder}/data/one.db`], + keepingFrames: [`${seedFolder}/data/two.db`], + plant: async (filesystem) => { + await plantBytes(filesystem, `${seedFolder}/data/one.db`, SQLITE_HEADER); + await plantBytes(filesystem, `${seedFolder}/data/one.db-wal`, naming); + await plantBytes(filesystem, `${seedFolder}/data/two.db`, SQLITE_HEADER); + await plantBytes(filesystem, `${seedFolder}/data/two.db-wal`, naming); + }, + }); + expect(unflushed.error).toBeUndefined(); + expect(unflushed.logger.warnings).toEqual([ + `could not flush ${seedFolder}/data/one.db: unable to open database file ${seedFolder}/data/one.db`, + `WAL ${seedFolder}/data/two.db-wal still holds frames after the flush; left unpatched`, + ]); + expect( + recorded(unflushed.database), + "a WAL is never recorded, and a database whose frames stayed names nothing", + ).toEqual([]); + + const unreadable = await provisionNewSeed({ + plant: async (filesystem) => { + await plantBytes(filesystem, `${seedFolder}/data/secret.db`, naming); + await plantBytes(filesystem, `${seedFolder}/data/other.txt`, Buffer.from("nothing")); + }, + unreadablePaths: [`${seedFolder}/data/secret.db`], + }); + expect(unreadable.error).toBeUndefined(); + expect(recorded(unreadable.database)).toEqual([ + { kind: "unreadable", relative_path: "data/secret.db" }, + ]); + expect(unreadable.logger.warnings).toEqual([ + `could not read ${seedFolder}/data/secret.db in source ${seedUdid}: EACCES: permission denied, open '${seedFolder}/data/secret.db'; recorded as unreadable`, + ]); + + const namingNothing = await provisionNewSeed({ + plant: (filesystem) => + plantBytes(filesystem, `${seedFolder}/data/none.txt`, Buffer.from("nothing")), + }); + expect(namingNothing.error).toBeUndefined(); + expect(recorded(namingNothing.database)).toEqual([]); + expect(storedLineage(namingNothing.database)).toEqual({ + lineage: JSON.stringify([lineageUdid]), + saved_without_lineage: 0, + }); + + const walkFails = await provisionNewSeed({ + failingPaths: [`${seedFolder}/data/sealed`], + plant: async (filesystem) => { + await plantBytes(filesystem, `${seedFolder}/data/sealed/a.plist`, naming); + await plantBytes(filesystem, `${seedFolder}/data/b.plist`, naming); + }, + }); + expect(walkFails.error).toMatchObject({ code: "EACCES" }); + expect( + walkFails.runner.calls.some( + (call) => call.args.join(" ") === `simctl --set ${sourcesSet} delete ${seedUdid}`, + ), + "the seed's clone is deleted", + ).toBe(true); + expect(walkFails.database.prepare("SELECT source_id FROM ios_source_data").all()).toEqual([]); + expect(recorded(walkFails.database)).toEqual([]); + + const baselineFolder = `${sourcesSet}/${baselineUdid}`; + const lease = driverData.udid.toUpperCase(); + const fromUsedLease = await saveBaseline({ + lineage: [lineageUdid], + plant: async (filesystem) => { + await plantBytes(filesystem, `${baselineFolder}/data/lease.plist`, Buffer.from(lease)); + await plantBytes(filesystem, `${baselineFolder}/data/seed.plist`, naming); + await plantBytes(filesystem, `${baselineFolder}/data/none.txt`, Buffer.from("nothing")); + await plantBytes(filesystem, `${baselineFolder}/data/one.db`, SQLITE_HEADER); + await plantBytes(filesystem, `${baselineFolder}/data/one.db-wal`, naming); + }, + }); + expect(fromUsedLease.error).toBeUndefined(); + expect(fromUsedLease.opener.checkpoints).toEqual([`${baselineFolder}/data/one.db`]); + expect(storedLineage(fromUsedLease.database)).toEqual({ + lineage: JSON.stringify([lease, lineageUdid]), + saved_without_lineage: 0, + }); + expect(recorded(fromUsedLease.database)).toEqual([ + { kind: "file", relative_path: "data/lease.plist" }, + { kind: "file", relative_path: "data/one.db" }, + { kind: "file", relative_path: "data/seed.plist" }, + ]); + + const fromOldDevice = await saveBaseline({ + plant: (filesystem) => + plantBytes(filesystem, `${baselineFolder}/data/lease.plist`, Buffer.from(lease)), + }); + expect(fromOldDevice.error).toBeUndefined(); + expect(storedLineage(fromOldDevice.database)).toEqual({ + lineage: null, + saved_without_lineage: 1, + }); + expect(recorded(fromOldDevice.database)).toEqual([]); + }); +}); diff --git a/src/drivers/ios/index-seed-sources.test.ts b/src/drivers/ios/index-seed-sources.test.ts index 145896e29..d923e9e0f 100644 --- a/src/drivers/ios/index-seed-sources.test.ts +++ b/src/drivers/ios/index-seed-sources.test.ts @@ -2,7 +2,13 @@ import { afterEach, expect, it, describe } from "vitest"; import { DriverCrashError, BootTimeoutError } from "../../core/index.js"; import { ScriptedProcessRunner, FakeClock, MemoryFilesystem } from "../../ports/index.js"; import { promiseState } from "../../test-support/promise-state.js"; -import { plantSource, memorySourceDatabase } from "./testing.js"; +import { + type PlantedSource, + plantBytes, + plantSource, + memorySourceDatabase, + readBytes, +} from "./testing.js"; import { deviceRoot, driverData, @@ -20,6 +26,7 @@ import { listInvocation, plantLaunchServices, scopedRunners, + plantSeedClone, seedUdid, seededDatabase, sourcesSimctl, @@ -122,6 +129,7 @@ describe("IosSimctlDriver", () => { it("provision deletes the clone and throws when the LaunchServices database cannot be dropped", async () => { const filesystem = new FaultyFilesystem(); await plantManagedDevice(filesystem); + await plantSeedClone(filesystem); await plantLaunchServices(filesystem, `${deviceRoot}/${driverData.udid}`); filesystem.failing = "rm"; const runner = new ScriptedProcessRunner([ @@ -141,25 +149,46 @@ describe("IosSimctlDriver", () => { }); it.each([ - { name: "a named source is copied instead of the seed", id: "planted-named", cloned: true }, + { + name: "a named source is copied instead of the seed, with its lineage swapped for the copy's own UDID", + id: "planted-named", + cloned: true, + }, { name: "an unknown source is refused", id: "nope", cloned: false }, { name: "a stale source is refused", id: "planted-old", cloned: false }, + { name: "a source with no lineage is refused", id: "planted-unlineaged", cloned: false }, ])("provision with a source: $name", async ({ id, cloned }) => { const namedUdid = "00000000-0000-0000-0000-0000000000bb"; const oldUdid = "00000000-0000-0000-0000-0000000000cc"; + const unlineagedUdid = "00000000-0000-0000-0000-0000000000dd"; const database = await seededDatabase(); const kind = { deviceTypeId: driverData.deviceTypeId, runtimeId: driverData.runtimeId }; - const planted = plantSource(database, { ...kind, kind: "baseline", udid: namedUdid }); + const namedLineage = "AAAAAAAA-1111-2222-3333-BBBBBBBBBBBB"; + const planted = plantSource(database, { + ...kind, + files: [{ kind: "file", relativePath: "data/note.plist" }], + kind: "baseline", + lineage: [namedLineage], + udid: namedUdid, + }); database .prepare("UPDATE ios_source_data SET source_id = 'planted-named' WHERE udid = ?") .run(namedUdid); + database.prepare("UPDATE ios_source_files SET source_id = 'planted-named'").run(); plantSource(planted, { ...kind, kind: "baseline", runtimeBuild: "23F77", udid: oldUdid }); database .prepare("UPDATE ios_source_data SET source_id = 'planted-old' WHERE udid = ?") .run(oldUdid); + plantSource(database, { ...kind, kind: "baseline", lineage: null, udid: unlineagedUdid }); + database + .prepare("UPDATE ios_source_data SET source_id = 'planted-unlineaged' WHERE udid = ?") + .run(unlineagedUdid); const filesystem = new MemoryFilesystem(); await plantManagedDevice(filesystem); + await plantSeedClone(filesystem); await plantLaunchServices(filesystem, `${deviceRoot}/${driverData.udid}`); + const note = `${deviceRoot}/${driverData.udid}/data/note.plist`; + await plantBytes(filesystem, note, Buffer.from(`root /Devices/${namedLineage}/data`)); const catalog = JSON.parse(listFixture) as { runtimes: { buildversion?: string; identifier: string }[]; }; @@ -180,8 +209,15 @@ describe("IosSimctlDriver", () => { const provisioned = driver.provision(spec, { source: id }); if (cloned) { - await expect(provisioned).resolves.toMatchObject({ deviceId: driverData.udid }); + await expect(provisioned).resolves.toMatchObject({ + deviceId: driverData.udid, + driverData: { lineage: [namedLineage] }, + }); expect(runner.calls.map((call) => call.args[3])).toEqual(["list", "list", "clone"]); + expect( + (await readBytes(filesystem, note)).toString(), + "the copy of a baseline names its own folder in place of the baseline's lineage", + ).toBe(`root /Devices/${driverData.udid.toUpperCase()}/data`); expect( await launchServicesFiles(filesystem, `${deviceRoot}/${driverData.udid}`), "the copy of a source loses the LaunchServices database that names the source's folder", @@ -225,6 +261,7 @@ describe("IosSimctlDriver", () => { ]); const filesystem = new MemoryFilesystem(); await plantManagedDevice(filesystem); + await plantSeedClone(filesystem); await filesystem.mkdirp(`${deviceRoot}/${copyUdid}/data`); const driver = await createDriver(runner, new FakeClock(now), filesystem, database); @@ -387,9 +424,29 @@ describe("IosSimctlDriver", () => { runtimeBuild: undefined, remade: false, }, - ])( + { + name: "it has no lineage", + lineage: null, + seedBuild: "23F79", + runtimeBuild: "23F79", + remade: true, + }, + { + name: "its lineage is empty", + lineage: [], + seedBuild: "23F79", + runtimeBuild: "23F79", + remade: false, + }, + ] as readonly { + name: string; + lineage?: readonly string[] | null; + seedBuild: string | undefined; + runtimeBuild: string | undefined; + remade: boolean; + }[])( "makes the seed again before copying it only when it is stale: $name -> remade $remade", - async ({ seedBuild, runtimeBuild, remade }) => { + async ({ lineage, seedBuild, runtimeBuild, remade }) => { const staleUdid = "00000000-0000-0000-0000-0000000000bb"; const catalog = JSON.parse(listFixture) as { runtimes: { buildversion?: string; identifier: string }[]; @@ -418,12 +475,14 @@ describe("IosSimctlDriver", () => { ]); const filesystem = new MemoryFilesystem(); await plantManagedDevice(filesystem); + await plantSeedClone(filesystem); const driver = await createDriver( runner, new FakeClock(), filesystem, await seededDatabase({ udid: planted, + ...(lineage === undefined ? {} : { lineage }), ...(seedBuild === undefined ? {} : { runtimeBuild: seedBuild }), }), ); @@ -450,8 +509,41 @@ describe("IosSimctlDriver", () => { }, ); - it.each([ + it.each<{ + name: string; + edit: (catalog: Catalog) => void; + planted?: Partial; + persists?: true; + reason: string | undefined; + }>([ { name: "healthy", edit: () => undefined, reason: undefined }, + { + name: "a seed has no lineage, which is remade and shows no reason", + edit: () => undefined, + planted: { lineage: null }, + reason: undefined, + }, + { + name: "a baseline was saved before lineages were kept", + edit: () => undefined, + planted: { kind: "baseline", lineage: null }, + persists: true, + reason: "saved before simulators stopped sharing folders; save it again", + }, + { + name: "a baseline was saved from a simulator made before lineages were kept", + edit: () => undefined, + planted: { kind: "baseline", lineage: null, savedWithoutLineage: true }, + persists: true, + reason: + "saved from a simulator made before simulators stopped sharing folders; save it again from a new lease", + }, + { + name: "a baseline has a lineage", + edit: () => undefined, + planted: { kind: "baseline", lineage: ["AAAAAAAA-1111-2222-3333-BBBBBBBBBBBB"] }, + reason: undefined, + }, { name: "its runtime is gone", edit: (catalog: Catalog) => { @@ -480,8 +572,8 @@ describe("IosSimctlDriver", () => { reason: `device type ${driverData.deviceTypeId} is not available`, }, ])( - "lists a ready seed as a source with its model, version and size, and its staleReason when $name, which clears once it is back", - async ({ edit, reason }) => { + "lists a ready source with its model, version and size, and its staleReason when $name, which clears once it is back", + async ({ edit, persists, planted, reason }) => { const catalogOf = (change: (catalog: Catalog) => void) => { const catalog = JSON.parse(listFixture) as Catalog; for (const entry of catalog.runtimes) { @@ -503,19 +595,23 @@ describe("IosSimctlDriver", () => { runtimeId: driverData.runtimeId, sizeBytes: 1_073_741_824, udid: seedUdid, + ...planted, }); const driver = await createDriver(runner, new FakeClock(), undefined, database); expect(await driver.listSources()).toMatchObject([ { - kind: "seed", + kind: planted?.kind ?? "seed", model: "iPhone 16", osVersion: "26.5", sizeBytes: 1_073_741_824, ...(reason === undefined ? {} : { staleReason: reason }), }, ]); - expect((await driver.listSources())[0]).not.toHaveProperty("staleReason"); + // why: a lineage the source never had does not come back with the catalog, so its reason stays. + const again = (await driver.listSources())[0]; + if (persists === true) expect(again).toHaveProperty("staleReason", reason); + else expect(again).not.toHaveProperty("staleReason"); expect(runner.calls.map((call) => call.args[3])).toEqual(["list", "list"]); }, ); @@ -567,6 +663,7 @@ describe("IosSimctlDriver", () => { const orphanUdid = "00000000-0000-0000-0000-0000000000ee"; const filesystem = new MemoryFilesystem(); await plantManagedDevice(filesystem); + await plantSeedClone(filesystem); await filesystem.mkdirp(`${deviceRoot}/sources/${seedUdid}`); await filesystem.mkdirp(`${deviceRoot}/sources/${orphanUdid}`); const database = await seededDatabase(); @@ -615,6 +712,7 @@ describe("IosSimctlDriver", () => { ])("destroySource $name", async ({ deleteCode, stderr, rows }) => { const database = plantSource(await memorySourceDatabase(), { deviceTypeId: driverData.deviceTypeId, + files: [{ kind: "file", relativePath: "data/a.plist" }], runtimeId: driverData.runtimeId, udid: seedUdid, }); @@ -637,6 +735,9 @@ describe("IosSimctlDriver", () => { expect(database.prepare("SELECT COUNT(*) AS n FROM ios_source_data").get()).toEqual({ n: rows, }); + expect(database.prepare("SELECT COUNT(*) AS n FROM ios_source_files").get()).toEqual({ + n: rows, + }); }); it("destroySource refuses an id that names no source, without running simctl", async () => { diff --git a/src/drivers/ios/index-seed-swap.test.ts b/src/drivers/ios/index-seed-swap.test.ts new file mode 100644 index 000000000..3acd83383 --- /dev/null +++ b/src/drivers/ios/index-seed-swap.test.ts @@ -0,0 +1,215 @@ +import { afterEach, describe, expect, it } from "vitest"; +import { DriverCrashError } from "../../core/index.js"; +import { FakeClock, MemoryFilesystem, ScriptedProcessRunner } from "../../ports/index.js"; +import { plantBytes, readBytes } from "./testing.js"; +import { deviceRoot, driverData, plantManagedDevice, simctl } from "./test-slim.js"; +import { + RecordingLogger, + createDriver, + listFixture, + listInvocation, + plantLaunchServices, + scopedRunners, + seedUdid, + seededDatabase, + sourcesSimctl, + spec, +} from "./test-index.js"; + +afterEach(() => { + scopedRunners.length = 0; +}); + +const lineageUdid = "AAAAAAAA-1111-2222-3333-BBBBBBBBBBBB"; +const copyUdid = "CCCCCCCC-4444-5555-6666-DDDDDDDDDDDD"; + +describe("IosSimctlDriver copy swap", () => { + it("a copy keeps its source's lineage, swaps the lineage's UDIDs in every recorded file and link for its own before its first boot and logs the swap", async () => { + class TickingWrites extends MemoryFilesystem { + constructor(private readonly clock: FakeClock) { + super(); + } + + override async writeAt(path: string, offset: number, bytes: Uint8Array): Promise { + this.clock.advance(5); + await super.writeAt(path, offset, bytes); + } + } + const utf16le = (text: string): Buffer => Buffer.from(text, "utf16le"); + const utf16be = (text: string): Buffer => utf16le(text).swap16(); + const ascii = (text: string): Buffer => Buffer.from(text, "latin1"); + const lower = lineageUdid.toLowerCase(); + const ownLower = copyUdid.toLowerCase(); + const window = 1_048_576; + const spanning = (udid: Buffer, at: number): Buffer => { + const bytes = Buffer.alloc(window + 200, 0x20); + udid.copy(bytes, at); + return bytes; + }; + const big = Buffer.alloc(64 * window + 100); + const bigTail = ascii(`/${lineageUdid}/`); + bigTail.copy(big, big.length - bigTail.length); + const bigSwapped = Buffer.from(big); + ascii(`/${copyUdid}/`).copy(bigSwapped, big.length - bigTail.length); + const files = [ + { + name: "ascii", + before: ascii(`path /Devices/${lineageUdid}/data end`), + after: ascii(`path /Devices/${copyUdid}/data end`), + }, + { name: "lower", before: ascii(`/logs/${lower}/x`), after: ascii(`/logs/${ownLower}/x`) }, + { name: "le", before: utf16le(`a ${lineageUdid} b`), after: utf16le(`a ${copyUdid} b`) }, + { name: "be", before: utf16be(`a ${lower} b`), after: utf16be(`a ${ownLower} b`) }, + { + name: "span-ascii", + before: spanning(ascii(lineageUdid), window - 18), + after: spanning(ascii(copyUdid), window - 18), + }, + { + name: "span-utf16", + before: spanning(utf16le(lineageUdid), window - 36), + after: spanning(utf16le(copyUdid), window - 36), + }, + { + name: "edge-utf16", + before: spanning(utf16le(lineageUdid), window - 1), + after: spanning(utf16le(copyUdid), window - 1), + }, + { name: "big", before: big, after: bigSwapped }, + { name: "own", before: ascii(`own ${copyUdid}`), after: ascii(`own ${copyUdid}`) }, + { + name: "unreadable", + before: ascii(`x ${lineageUdid}`), + after: ascii(`x ${lineageUdid}`), + kind: "unreadable" as const, + }, + ]; + const linkTarget = (udid: string): string => `/Users/x/Library/Logs/CoreSimulator/${udid}`; + const database = await seededDatabase({ + files: [ + ...files.map((file) => ({ + kind: file.kind ?? ("file" as const), + relativePath: `data/${file.name}`, + })), + { kind: "link", relativePath: "data/Logs" }, + ], + lineage: [lineageUdid], + }); + const runner = new ScriptedProcessRunner([ + { match: listInvocation, result: { code: 0, stderr: "", stdout: listFixture } }, + { + match: sourcesSimctl("clone", seedUdid, driverData.name, deviceRoot), + result: { code: 0, stderr: "", stdout: `${copyUdid}\n` }, + }, + ]); + const clock = new FakeClock(); + const filesystem = new TickingWrites(clock); + const logger = new RecordingLogger(); + await plantManagedDevice(filesystem); + for (const file of files) { + await plantBytes(filesystem, `${deviceRoot}/${copyUdid}/data/${file.name}`, file.before); + } + filesystem.defineSymlink(`${deviceRoot}/${copyUdid}/data/Logs`, linkTarget(lineageUdid)); + const driver = await createDriver(runner, clock, filesystem, database, logger); + + const device = await driver.provision(spec); + + for (const file of files) { + const bytes = await readBytes(filesystem, `${deviceRoot}/${copyUdid}/data/${file.name}`); + expect(bytes.equals(file.after), file.name).toBe(true); + } + expect(await filesystem.readlink(`${deviceRoot}/${copyUdid}/data/Logs`)).toBe( + linkTarget(copyUdid), + ); + expect(device.driverData).toMatchObject({ lineage: [lineageUdid] }); + expect(logger.debugs).toEqual([ + `swapped 9 files and 1 links in 40 ms for ${copyUdid}; 1 unreadable`, + ]); + }); + + it("a copy whose swap fails is deleted and the provision fails", async () => { + class FailingWrites extends MemoryFilesystem { + override async writeAt(): Promise { + throw new Error("disk full"); + } + } + const file = `${deviceRoot}/${copyUdid}/data/Library/note.plist`; + const cases = [ + { label: "control: a swap that works", error: undefined }, + { label: "a recorded file gone", gone: true, error: /ENOENT/ }, + { + label: "a write that fails", + filesystem: new FailingWrites(), + error: /disk full/, + }, + { + label: "a result of another length", + lineage: ["AAAAAAAA-1111-2222-3333-BBBBBBBBBBB"], + crash: true, + error: new RegExp(`${file}.*length`), + }, + ]; + for (const attempt of cases) { + const database = await seededDatabase({ + files: [{ kind: "file", relativePath: "data/Library/note.plist" }], + lineage: attempt.lineage ?? [lineageUdid], + }); + const runner = new ScriptedProcessRunner([ + { match: listInvocation, result: { code: 0, stderr: "", stdout: listFixture } }, + { + match: sourcesSimctl("clone", seedUdid, driverData.name, deviceRoot), + result: { code: 0, stderr: "", stdout: `${copyUdid}\n` }, + }, + { match: simctl("delete", copyUdid) }, + ]); + const filesystem = attempt.filesystem ?? new MemoryFilesystem(); + await plantManagedDevice(filesystem); + await plantBytes( + filesystem as MemoryFilesystem, + attempt.gone === true ? `${deviceRoot}/${copyUdid}/data/other` : file, + Buffer.from(`/Devices/${lineageUdid}/ AAAAAAAA-1111-2222-3333-BBBBBBBBBBB`), + ); + const driver = await createDriver(runner, new FakeClock(), filesystem, database); + + const outcome = await driver.provision(spec).then( + () => undefined, + (error: unknown) => error as Error, + ); + + const deleted = runner.calls.some((call) => + call.args.join(" ").endsWith(`delete ${copyUdid}`), + ); + if (attempt.error === undefined) { + expect(outcome, attempt.label).toBeUndefined(); + expect(deleted, attempt.label).toBe(false); + } else { + expect(outcome?.message ?? "no error", attempt.label).toMatch(attempt.error); + expect(deleted, attempt.label).toBe(true); + if (attempt.crash === true) expect(outcome, attempt.label).toBeInstanceOf(DriverCrashError); + } + } + }); + + it("a LaunchServices database the seed recorded is dropped from the copy and is not swapped", async () => { + const csstore = "data/var/db/lsd/com.apple.LaunchServices-33554481-v2.csstore"; + const database = await seededDatabase({ + files: [{ kind: "file", relativePath: csstore }], + lineage: [lineageUdid], + }); + const runner = new ScriptedProcessRunner([ + { match: listInvocation, result: { code: 0, stderr: "", stdout: listFixture } }, + { + match: sourcesSimctl("clone", seedUdid, driverData.name, deviceRoot), + result: { code: 0, stderr: "", stdout: `${copyUdid}\n` }, + }, + ]); + const filesystem = new MemoryFilesystem(); + await plantManagedDevice(filesystem); + await plantLaunchServices(filesystem, `${deviceRoot}/${copyUdid}`); + const driver = await createDriver(runner, new FakeClock(), filesystem, database); + + await driver.provision(spec); + + expect(await filesystem.exists(`${deviceRoot}/${copyUdid}/${csstore}`)).toBe(false); + }); +}); diff --git a/src/drivers/ios/index-seed-throwaway.test.ts b/src/drivers/ios/index-seed-throwaway.test.ts index bbaa78b37..7bd5d6ffb 100644 --- a/src/drivers/ios/index-seed-throwaway.test.ts +++ b/src/drivers/ios/index-seed-throwaway.test.ts @@ -16,6 +16,7 @@ import { logsRoot, scopedRunners, listInvocation, + plantSeedClone, seedUdid, sourcesSimctl, spec, @@ -107,7 +108,12 @@ describe("IosSimctlDriver seed throwaway", () => { step(inSet("bootstatus", throwawayUdid, "-b")), step(inSet("listapps", throwawayUdid)), step(inSet("shutdown", throwawayUdid)), - step(inSet("clone", throwawayUdid, name, world.sourcesSet), `${seedUdid}\n`), + { + ...step(inSet("clone", throwawayUdid, name, world.sourcesSet), `${seedUdid}\n`), + onSpawn: () => { + void filesystem.mkdirp(`${world.sourcesSet}/${seedUdid}`); + }, + }, step(inSet("delete", throwawayUdid)), ...hub(`Delete :DevicePreferences:${throwawayUdid}`), step( @@ -233,6 +239,7 @@ describe("IosSimctlDriver seed throwaway", () => { ]); scopedRunners.push(runner); await plantManagedDevice(filesystem); + await plantSeedClone(filesystem); await filesystem.mkdirp(`${deviceRoot}/${copyUdid}/data`); if (failure.absent !== true) await filesystem.mkdirp(`${logsRoot}/${throwawayUdid}`); const driver = await IosSimctlDriver.create({ diff --git a/src/drivers/ios/index.ts b/src/drivers/ios/index.ts index 5a5360730..e5f82bae2 100644 --- a/src/drivers/ios/index.ts +++ b/src/drivers/ios/index.ts @@ -225,6 +225,7 @@ export class IosSimctlDriver implements Driver { seedTurns: this.#seedTurns, simctl: this.#simctl, sourcesSet: sourcesSetFor(options.simlockHome, deviceRoot, rootOptions !== undefined), + sqliteOpener: options.sqliteOpener, }); this.screenCapture = createScreenCaptureFor( options, diff --git a/src/drivers/ios/lineage-files.ts b/src/drivers/ios/lineage-files.ts new file mode 100644 index 000000000..c7cf6b485 --- /dev/null +++ b/src/drivers/ios/lineage-files.ts @@ -0,0 +1,194 @@ +import { DriverCrashError } from "../../core/index.js"; +import { + isMissingPathError, + type Filesystem, + type Logger, + type SqliteOpener, +} from "../../ports/index.js"; +import { errorMessage } from "./command-run.js"; +import type { SourceFile } from "./source-data.js"; +import { SWAP_OVERLAP_BYTES, namesLineage, swapLineage, type LineageSwap } from "./udid-swap.js"; + +/** why: a window is read whole, so this bounds memory at any file size. */ +const WINDOW_BYTES = 1_048_576; + +const WAL_SUFFIX = "-wal"; + +export interface SwapCounts { + readonly files: number; + readonly links: number; + readonly unreadable: number; +} + +/** why: ADR 0076 §3: before its first boot a copy's recorded files and links name its own UDID in place of the lineage's, each keeping its length; an unreadable file is left. */ +export async function swapCopyFiles( + filesystem: Filesystem, + folder: string, + files: readonly SourceFile[], + swap: LineageSwap, +): Promise { + const counts = { files: 0, links: 0, unreadable: 0 }; + for (const file of files) { + const path = `${folder}/${file.relativePath}`; + if (file.kind === "unreadable") { + counts.unreadable += 1; + } else if (file.kind === "link") { + counts.links += 1; + const target = Buffer.from(await filesystem.readlink(path)); + const swapped = swapLineage(target, swap); + checkLength(path, target, swapped); + if (!target.equals(swapped)) { + await filesystem.replaceSymlink(path, Buffer.from(swapped).toString()); + } + } else { + counts.files += 1; + await eachWindow(filesystem, path, async (window, start) => { + const swapped = swapLineage(window, swap); + checkLength(path, window, swapped); + if (!window.equals(swapped)) await filesystem.writeAt(path, start, swapped); + return false; + }); + } + } + return counts; +} + +function checkLength(path: string, before: Uint8Array, after: Uint8Array): void { + if (before.length !== after.length) { + throw new DriverCrashError(`swapping UDIDs in ${path} changed its length`); + } +} + +export interface RecordOptions { + readonly filesystem: Filesystem; + /** why: the source's UDID folder; recorded paths are relative to it. */ + readonly folder: string; + readonly lineage: readonly string[]; + readonly logger: Logger | undefined; + readonly sqlite: SqliteOpener | undefined; + readonly udid: string; +} + +/** + * why: ADR 0076 §2: when a source is made every WAL in it is flushed into its database, and the files and links that name a lineage UDID are found once and recorded, so a copy patches only those. + */ +export async function recordLineageFiles(options: RecordOptions): Promise { + const files: SourceFile[] = []; + await visitFolder(options, options.folder, "", files); + return files; +} + +async function visitFolder( + options: RecordOptions, + directory: string, + relative: string, + files: SourceFile[], +): Promise { + // why: a database and its WAL are siblings, and a database is read only after its WAL is flushed into it (ADR 0076 §2); a flush closes SQLite, which deletes the `-shm`, so the folder is listed again once every WAL in it is flushed. + for (const name of (await options.filesystem.readdir(directory)).sort()) { + if (!name.endsWith(WAL_SUFFIX)) continue; + const path = `${directory}/${name}`; + if ((await options.filesystem.lstat(path)).kind === "file") await flushWal(options, path); + } + const names = (await options.filesystem.readdir(directory)).sort(); + for (const name of names) { + const path = `${directory}/${name}`; + const relativePath = relative === "" ? name : `${relative}/${name}`; + const details = await options.filesystem.lstat(path); + if (details.kind === "directory") { + await visitFolder(options, path, relativePath, files); + continue; + } + const kind = await recordedKind(options, path, details.kind); + if (kind !== undefined) files.push({ kind, relativePath }); + } +} + +/** why: a link is recorded when its target names the lineage; a WAL is never recorded; any other file is recorded when its bytes name it or cannot be read. */ +async function recordedKind( + options: RecordOptions, + path: string, + entry: string, +): Promise { + if (entry === "symlink") { + const target = await options.filesystem.readlink(path); + return namesLineage(Buffer.from(target), options.lineage) ? "link" : undefined; + } + if (entry !== "file") return undefined; + if (path.endsWith(WAL_SUFFIX)) return undefined; + return classify(options, path); +} + +async function flushWal(options: RecordOptions, wal: string): Promise { + if (options.sqlite === undefined) return; + const database = wal.slice(0, -WAL_SUFFIX.length); + try { + const opened = await options.sqlite.open(database, { readOnly: false }); + try { + opened.exec("PRAGMA wal_checkpoint(TRUNCATE)"); + } finally { + opened.close(); + } + } catch (error: unknown) { + options.logger?.warn(`could not flush ${database}: ${errorMessage(error)}`); + return; + } + const left = await options.filesystem.stat(wal).then( + (found) => found.size, + (error: unknown) => { + if (isMissingPathError(error)) return 0; + throw error; + }, + ); + if (left > 0) { + options.logger?.warn(`WAL ${wal} still holds frames after the flush; left unpatched`); + } +} + +async function classify( + options: RecordOptions, + path: string, +): Promise<"file" | "unreadable" | undefined> { + try { + const names = await eachWindow(options.filesystem, path, (window) => + Promise.resolve(namesLineage(window, options.lineage)), + ); + return names ? "file" : undefined; + } catch (error: unknown) { + options.logger?.warn( + `could not read ${path} in source ${options.udid}: ${errorMessage(error)}; recorded as unreadable`, + ); + return "unreadable"; + } +} + +/** why: windows overlap by one UDID in UTF-16 less a byte, so a UDID spanning two reads is whole in one; `visit` answers true to stop. */ +async function eachWindow( + filesystem: Filesystem, + path: string, + visit: (window: Buffer, start: number) => Promise, +): Promise { + const { size } = await filesystem.stat(path); + for (let start = 0; start < size; start += WINDOW_BYTES) { + const end = Math.min(start + WINDOW_BYTES + SWAP_OVERLAP_BYTES, size) - 1; + if (await visit(await readRange(filesystem, path, start, end), start)) return true; + } + return false; +} + +async function readRange( + filesystem: Filesystem, + path: string, + start: number, + end: number, +): Promise { + const file = await filesystem.openFile(path); + const chunks: Uint8Array[] = []; + const reader = file.read({ end, start }).getReader(); + for (;;) { + const piece = await reader.read(); + if (piece.done) break; + chunks.push(piece.value); + } + return Buffer.concat(chunks); +} diff --git a/src/drivers/ios/seeds.ts b/src/drivers/ios/seeds.ts index 673093bc9..11e982be1 100644 --- a/src/drivers/ios/seeds.ts +++ b/src/drivers/ios/seeds.ts @@ -20,18 +20,26 @@ import type { ResolvedIosSpec } from "./catalog.js"; import { UDID_PATTERN } from "./device-set-leftovers.js"; -import { IosSourceTable, type ReadySeed, type SourceRow } from "./source-data.js"; +import { + IosSourceTable, + type ReadySeed, + type SourceRecord, + type SourceRow, +} from "./source-data.js"; + +import { recordLineageFiles, swapCopyFiles } from "./lineage-files.js"; import { type SimctlCatalog } from "./simctl-parse.js"; import { type IosDriverData, iosDriverData } from "./simulator-data.js"; -import type { SqliteDatabase } from "../../ports/index.js"; +import type { SqliteDatabase, SqliteOpener } from "../../ports/index.js"; /** why: ADR 0061 §1: a `making` row of this process older than this is a make that never ended. */ const STALE_MAKING_MS = 300_000; const LAUNCH_SERVICES_DATABASE = /^com\.apple\.LaunchServices-.*\.csstore$/; +const LAUNCH_SERVICES_PATH = /^data\/var\/db\/lsd\/com\.apple\.LaunchServices-[^/]*\.csstore$/; interface SeedsOptions { readonly boot: SimulatorBoot; @@ -51,6 +59,7 @@ interface SeedsOptions { readonly seedTurns: SeedTurns; readonly simctl: Simctl; readonly sourcesSet: string; + readonly sqliteOpener: SqliteOpener | undefined; } /** why: the part of IosSimctlDriver (ADR 0063) that makes, keeps and deletes the seeds every provision copies (ADR 0061), and lists them as sources. */ @@ -73,6 +82,7 @@ export class Seeds { readonly #simctl: Simctl; readonly #sources: IosSourceTable; readonly #sourcesSet: string; + readonly #sqliteOpener: SqliteOpener | undefined; #sourceMaker: string | undefined; #swept = false; @@ -94,6 +104,7 @@ export class Seeds { this.#simctl = options.simctl; this.#sources = new IosSourceTable(options.database); this.#sourcesSet = options.sourcesSet; + this.#sqliteOpener = options.sqliteOpener; } async listSources(): Promise { @@ -160,8 +171,9 @@ export class Seeds { this.#sources.setUdid(sourceId, cloneUdid); await this.#slimBoot.copyOverrides(data.udid, cloneUdid); await this.#dropLaunchServicesDatabase(this.#sourcesSet, cloneUdid); + const record = await this.#baselineRecord(data, cloneUdid); const sizeBytes = await this.#sizeOf(cloneUdid); - this.#sources.markReady(sourceId, sizeBytes); + this.#sources.markReady(sourceId, sizeBytes, record); // why: ADR 0076: the lease is destroyed next, and `simctl delete` leaves its log folder behind for the baseline's copies to inherit. await this.#deleteLogFolder(data.udid); return { sourceId, ...(sizeBytes === undefined ? {} : { sizeBytes }) }; @@ -174,6 +186,14 @@ export class Seeds { } } + /** why: ADR 0076 §2: the baseline's lineage is its lease's UDID and the lineage the lease was copied with; a lease made before lineages were kept has none, so the baseline is never copied. */ + async #baselineRecord(data: IosDriverData, cloneUdid: string): Promise { + const lineage = + data.lineage === undefined ? undefined : [data.udid.toUpperCase(), ...data.lineage]; + const files = lineage === undefined ? [] : await this.#recordFiles(cloneUdid, lineage); + return { files, lineage, savedWithoutLineage: lineage === undefined }; + } + async destroySource(sourceId: string): Promise { const row = this.#sources.findRow(sourceId); if (row === undefined) { @@ -219,6 +239,7 @@ export class Seeds { try { await this.#marks.writeMark(udid); await this.#dropLaunchServicesDatabase(this.#deviceRoot, udid); + if (seed.lineage !== undefined) await this.#swapLineage(seed.sourceId, seed.lineage, udid); } catch (error: unknown) { // why: a clone whose mark or database drop failed is not a device Simlock hands out, and // in Xcode's set an unmarked one would never be listed, reclaimed or deleted again. @@ -235,6 +256,7 @@ export class Seeds { name, runtimeId: resolved.runtime.identifier, udid, + ...(seed.lineage === undefined ? {} : { lineage: seed.lineage }), } satisfies IosDriverData, }; } @@ -268,14 +290,19 @@ export class Seeds { } } - async #namedSource(sourceId: string): Promise<{ readonly udid: string }> { + async #namedSource(sourceId: string): Promise { const row = this.#sources.findRow(sourceId); if (row?.state !== "ready" || row.udid === undefined) { throw new DriverCrashError(`iOS keeps no ready source ${sourceId}`); } const reason = sourceStaleReason(row, await this.#simctl.loadCatalog()); if (reason !== undefined) throw new DriverCrashError(`source ${sourceId} is stale: ${reason}`); - return { udid: row.udid }; + return { + lineage: row.lineage, + runtimeBuild: row.runtimeBuild, + sourceId: row.sourceId, + udid: row.udid, + }; } /** why: two provisions of one kind at once make one seed, the second waiting for the first's make. */ @@ -301,6 +328,16 @@ export class Seeds { return this.#makeSeed(resolved, specKey); } + async #bootThrowaway(throwaway: string, sourceId: string): Promise { + try { + await this.#boot.bootAndWait(throwaway, sourceId, this.#fullBootTimeoutMs); + } catch (error: unknown) { + // why: a seed that ran out of boot time is a boot timeout, as an Android seed's is. + if (error instanceof BootTimeoutError) throw error; + throw new DriverCrashError(`seed ${throwaway} did not boot: ${errorMessage(error)}`); + } + } + async #makeSeed(resolved: ResolvedIosSpec, specKey: string): Promise { // why: a `making` row of this process older than the longest a make can take belongs to a make that never ended. for (const row of this.#sources.makingRowsOfKindBefore( @@ -350,13 +387,7 @@ export class Seeds { await this.#marks.writeMark(throwaway); await this.#deviceHub.disablePasteboardSync(throwaway); } - try { - await this.#boot.bootAndWait(throwaway, sourceId, this.#fullBootTimeoutMs); - } catch (error: unknown) { - // why: a seed that ran out of boot time is a boot timeout, as an Android seed's is. - if (error instanceof BootTimeoutError) throw error; - throw new DriverCrashError(`seed ${throwaway} did not boot: ${errorMessage(error)}`); - } + await this.#bootThrowaway(throwaway, sourceId); await this.#simctl.shutdown(throwaway); const cloned = await this.#simctl.createSimulator( ["clone", throwaway, `simlock-seed-${sourceId}`, this.#sourcesSet], @@ -365,12 +396,14 @@ export class Seeds { seedClone = cloned.stdout.trim(); this.#sources.setUdid(sourceId, seedClone); const done = throwaway; + const lineage = [done.toUpperCase()]; throwaway = undefined; keepRow = true; await this.#deleteThrowaway(done, false); keepRow = false; - this.#sources.markReady(sourceId, await this.#sizeOf(seedClone)); - return { sourceId, udid: seedClone, runtimeBuild: resolved.runtime.build }; + const files = await this.#recordFiles(seedClone, lineage); + this.#sources.markReady(sourceId, await this.#sizeOf(seedClone), { files, lineage }); + return { sourceId, udid: seedClone, runtimeBuild: resolved.runtime.build, lineage }; } catch (error: unknown) { keepRow = await this.#undoFailedMake(seedClone, throwaway, keepRow); if (!keepRow) this.#sources.deleteRow(sourceId); @@ -472,6 +505,33 @@ export class Seeds { }); } + async #swapLineage(sourceId: string, lineage: readonly string[], udid: string): Promise { + const started = this.#clock.now(); + const counts = await swapCopyFiles( + this.#filesystem, + join(this.#deviceRoot, udid), + // why: the copy's LaunchServices database was dropped just before, so a seed that recorded one has nothing of it left to swap. + this.#sources + .filesOf(sourceId) + .filter((file) => !LAUNCH_SERVICES_PATH.test(file.relativePath)), + { lineage, own: udid }, + ); + this.#logger?.debug( + `swapped ${String(counts.files)} files and ${String(counts.links)} links in ${String(this.#clock.now() - started)} ms for ${udid}; ${String(counts.unreadable)} unreadable`, + ); + } + + #recordFiles(udid: string, lineage: readonly string[]): ReturnType { + return recordLineageFiles({ + filesystem: this.#filesystem, + folder: join(this.#sourcesSet, udid), + lineage, + logger: this.#logger, + sqlite: this.#sqliteOpener, + udid, + }); + } + async #sizeOf(udid: string): Promise { try { return await this.#filesystem.directorySize(join(this.#sourcesSet, udid)); @@ -514,12 +574,18 @@ export class Seeds { /** why: a seed built on a runtime whose build changed no longer matches it; with no build on either side only the runtime's presence is checked, and a provision always has the runtime. */ function seedIsStale(seed: ReadySeed, resolved: ResolvedIosSpec): boolean { + if (seed.lineage === undefined) return true; const current = resolved.runtime.build; return seed.runtimeBuild !== undefined && current !== undefined && seed.runtimeBuild !== current; } /** why: the phrase reads after "is stale:" in `doctor`, and is what a person acts on, so it names the runtime or device type by id. */ function sourceStaleReason(row: SourceRow, catalog: SimctlCatalog): string | undefined { + if (row.kind === "baseline" && row.lineage === undefined) { + return row.savedWithoutLineage + ? "saved from a simulator made before simulators stopped sharing folders; save it again from a new lease" + : "saved before simulators stopped sharing folders; save it again"; + } const runtime = catalog.runtimes.find((entry) => entry.identifier === row.runtimeId); if (runtime === undefined) return `runtime ${row.runtimeId} is not installed`; if ( diff --git a/src/drivers/ios/simulator-data.ts b/src/drivers/ios/simulator-data.ts index 89acb6fe3..58b9de717 100644 --- a/src/drivers/ios/simulator-data.ts +++ b/src/drivers/ios/simulator-data.ts @@ -6,6 +6,7 @@ export interface IosDriverData { readonly name: string; readonly runtimeId: string; readonly udid: string; + readonly lineage?: readonly string[]; } export function iosDriverData(device: DriverDevice): IosDriverData { @@ -29,6 +30,7 @@ export function iosDriverData(device: DriverDevice): IosDriverData { // an older daemon stamped here is not read: the spec's mode is the only record of it (ADR 0007 §7). ...(typeof value.slimSignature === "string" ? { slimSignature: value.slimSignature } : {}), ...(typeof value.slimMarkToken === "string" ? { slimMarkToken: value.slimMarkToken } : {}), + ...(isUdidList(value.lineage) ? { lineage: value.lineage } : {}), }; } @@ -37,3 +39,7 @@ export function requireIosPlatform(platform: string): void { throw new DriverCrashError(`iOS simctl driver cannot handle ${platform} device requests`); } } + +function isUdidList(value: unknown): value is readonly string[] { + return Array.isArray(value) && value.every((entry) => typeof entry === "string"); +} diff --git a/src/drivers/ios/source-data.ts b/src/drivers/ios/source-data.ts index 674ed8170..c49efebe9 100644 --- a/src/drivers/ios/source-data.ts +++ b/src/drivers/ios/source-data.ts @@ -1,5 +1,5 @@ import { join } from "node:path"; -import type { SqliteDatabase, SqliteSchemaOwner } from "../../ports/index.js"; +import { inTransaction, type SqliteDatabase, type SqliteSchemaOwner } from "../../ports/index.js"; export const iosSourceDataOwner: SqliteSchemaOwner = { migrations: [ @@ -31,9 +31,18 @@ export const iosSourceDataOwner: SqliteSchemaOwner = { WHERE kind = 'seed' AND state = 'ready';`, // why: ADR 0076: the throwaway a seed was copied from, kept so a restart mid-make can delete it through the row. "ALTER TABLE ios_source_data ADD COLUMN throwaway_udid TEXT", + // why: ADR 0076: the UDIDs a source was copied from (null: made before lineages were kept), the files that name them, and whether a baseline was saved from a device that had no lineage. + `ALTER TABLE ios_source_data ADD COLUMN lineage TEXT; + ALTER TABLE ios_source_data ADD COLUMN saved_without_lineage INTEGER NOT NULL DEFAULT 0; + CREATE TABLE ios_source_files ( + source_id TEXT NOT NULL, + relative_path TEXT NOT NULL CHECK (length(CAST(relative_path AS BLOB)) <= 4096), + kind TEXT NOT NULL CHECK (kind IN ('link', 'file', 'unreadable')), + PRIMARY KEY (source_id, relative_path) + ) STRICT;`, ], name: "ios-source-data", - version: 3, + version: 4, }; export interface NewSourceRow { @@ -63,12 +72,27 @@ export interface SourceRow { readonly state: "making" | "ready"; readonly maker: string; readonly sizeBytes: number | undefined; + readonly lineage: readonly string[] | undefined; + readonly savedWithoutLineage: boolean; } export interface ReadySeed { readonly sourceId: string; readonly udid: string; readonly runtimeBuild: string | undefined; + readonly lineage: readonly string[] | undefined; +} + +export interface SourceFile { + readonly relativePath: string; + readonly kind: "link" | "file" | "unreadable"; +} + +/** why: what a source records when it is made: the UDIDs it came from and the files that name them. */ +export interface SourceRecord { + readonly lineage: readonly string[] | undefined; + readonly files: readonly SourceFile[]; + readonly savedWithoutLineage?: boolean; } /** why: the one place that reads or writes `ios_source_data`; nothing outside the iOS driver does (ADR 0036 §2). */ @@ -113,20 +137,47 @@ export class IosSourceTable { .run(udid, sourceId); } - markReady(sourceId: string, sizeBytes: number | undefined): void { - this.#database - .prepare("UPDATE ios_source_data SET state = 'ready', size_bytes = ? WHERE source_id = ?") - .run(sizeBytes ?? null, sourceId); + markReady(sourceId: string, sizeBytes: number | undefined, record: SourceRecord): void { + inTransaction(this.#database, () => { + this.#database + .prepare( + `UPDATE ios_source_data SET state = 'ready', size_bytes = ?, lineage = ?, saved_without_lineage = ? + WHERE source_id = ?`, + ) + .run( + sizeBytes ?? null, + record.lineage === undefined ? null : JSON.stringify(record.lineage), + record.savedWithoutLineage === true ? 1 : 0, + sourceId, + ); + const insert = this.#database.prepare( + "INSERT INTO ios_source_files (source_id, relative_path, kind) VALUES (?, ?, ?)", + ); + for (const file of record.files) insert.run(sourceId, file.relativePath, file.kind); + }); + } + + filesOf(sourceId: string): readonly SourceFile[] { + return this.#database + .prepare("SELECT relative_path, kind FROM ios_source_files WHERE source_id = ?") + .all(sourceId) + .map((row) => ({ + relativePath: row["relative_path"] as string, + kind: row["kind"] as SourceFile["kind"], + })); } deleteRow(sourceId: string): void { - this.#database.prepare("DELETE FROM ios_source_data WHERE source_id = ?").run(sourceId); + inTransaction(this.#database, () => { + this.#database.prepare("DELETE FROM ios_source_files WHERE source_id = ?").run(sourceId); + this.#database.prepare("DELETE FROM ios_source_data WHERE source_id = ?").run(sourceId); + }); } findReadySeed(specKey: string): ReadySeed | undefined { const row = this.#database .prepare( - "SELECT source_id, udid, runtime_build FROM ios_source_data WHERE kind = 'seed' AND state = 'ready' AND spec_key = ?", + "SELECT source_id, udid, runtime_build, lineage FROM ios_source_data WHERE kind = 'seed' AND state = 'ready' AND spec_key = ?", ) .get(specKey); if (row === undefined) return undefined; @@ -134,6 +185,7 @@ export class IosSourceTable { sourceId: row["source_id"] as string, udid: row["udid"] as string, runtimeBuild: (row["runtime_build"] as string | null) ?? undefined, + lineage: parseLineage(row["lineage"]), }; } @@ -181,7 +233,7 @@ export class IosSourceTable { return this.#database .prepare( `SELECT source_id, kind, udid, throwaway_udid, device_type_id, runtime_id, runtime_build, model, os_version, - state, maker, size_bytes FROM ios_source_data ${where}`, + state, maker, size_bytes, lineage, saved_without_lineage FROM ios_source_data ${where}`, ) .all(...parameters) .map((row) => ({ @@ -197,10 +249,16 @@ export class IosSourceTable { state: row["state"] as "making" | "ready", maker: row["maker"] as string, sizeBytes: (row["size_bytes"] as number | null) ?? undefined, + lineage: parseLineage(row["lineage"]), + savedWithoutLineage: row["saved_without_lineage"] === 1, })); } } +function parseLineage(value: unknown): readonly string[] | undefined { + return typeof value === "string" ? (JSON.parse(value) as string[]) : undefined; +} + export function sourcesSetFor(simlockHome: string, deviceRoot: string, owned: boolean): string { return owned ? join(deviceRoot, "sources") : join(simlockHome, "devices", "ios-sources"); } diff --git a/src/drivers/ios/test-index.ts b/src/drivers/ios/test-index.ts index 0eacab7df..bf4990a75 100644 --- a/src/drivers/ios/test-index.ts +++ b/src/drivers/ios/test-index.ts @@ -8,8 +8,8 @@ import { MemoryFilesystem, type Logger, } from "../../ports/index.js"; -import { IosSimctlDriver, SLIM_CATEGORIES } from "./index.js"; -import { plantReadySeed, memorySourceDatabase } from "./testing.js"; +import { IosSimctlDriver, type IosSimctlDriverOptions, SLIM_CATEGORIES } from "./index.js"; +import { plantReadySeed, memorySourceDatabase, type PlantedSource } from "./testing.js"; import { simctl, deviceRoot, @@ -94,7 +94,7 @@ export function sourcesSimctl(...args: readonly string[]): { /** why: A database that already holds the seed of the kind `spec` names, so a test of the copy step starts at the clone. */ export async function seededDatabase( - seed: { readonly runtimeBuild?: string; readonly udid?: string } = {}, + seed: Partial> = {}, ): Promise { return plantReadySeed(await memorySourceDatabase(), { deviceTypeId: driverData.deviceTypeId, @@ -218,9 +218,11 @@ export async function createDriver( database?: SqliteDatabase, logger?: Logger, fullBootTimeoutMs?: number, + extra: Partial = {}, ): Promise { scopedRunners.push(runner); return IosSimctlDriver.create({ + ...extra, database: database ?? (await memorySourceDatabase()), clock, driverConfig: { deviceRoot }, @@ -260,8 +262,11 @@ export async function launchServicesFiles( /** why: Records every `warn` message, so the destroy's log line is asserted on its text. */ export class RecordingLogger implements Logger { readonly warnings: string[] = []; + readonly debugs: string[] = []; - debug(): void {} + debug(message: string): void { + this.debugs.push(message); + } info(): void {} warn(message: string): void { this.warnings.push(message); @@ -395,3 +400,12 @@ export function throwawaySeedSteps( { match: simctl("delete", throwawayUdid) }, ]; } + +/** why: `simctl clone` made the seed's folder in the sources set, which the scripted runner cannot do, and a make walks it before the seed turns ready. */ +export async function plantSeedClone( + filesystem: Filesystem, + sourcesSet: string = `${deviceRoot}/sources`, + udid: string = seedUdid, +): Promise { + await filesystem.mkdirp(`${sourcesSet}/${udid}`); +} diff --git a/src/drivers/ios/testing.ts b/src/drivers/ios/testing.ts index 6e074517e..522df47ac 100644 --- a/src/drivers/ios/testing.ts +++ b/src/drivers/ios/testing.ts @@ -1,5 +1,5 @@ -import { NodeSqliteOpener, type SqliteDatabase } from "../../ports/index.js"; -import { iosSourceDataOwner, IosSourceTable } from "./source-data.js"; +import { NodeSqliteOpener, type MemoryFilesystem, type SqliteDatabase } from "../../ports/index.js"; +import { iosSourceDataOwner, IosSourceTable, type SourceFile } from "./source-data.js"; export async function memorySourceDatabase(): Promise { const database = await new NodeSqliteOpener().open(":memory:", { readOnly: false }); @@ -9,6 +9,11 @@ export async function memorySourceDatabase(): Promise { export interface PlantedSource { readonly deviceTypeId: string; + /** why: the files that name the lineage, as a source's make recorded them. */ + readonly files?: readonly SourceFile[]; + /** why: `null` is a source made before lineages were kept; absent is a lineage of no simulator. */ + readonly lineage?: readonly string[] | null; + readonly savedWithoutLineage?: boolean; readonly kind?: "seed" | "baseline"; readonly maker?: string; readonly model?: string; @@ -40,7 +45,13 @@ export function plantSource(database: SqliteDatabase, source: PlantedSource): Sq }); if (source.udid !== undefined) table.setUdid(sourceId, source.udid); if (source.throwawayUdid !== undefined) table.setThrowaway(sourceId, source.throwawayUdid); - if (source.state !== "making") table.markReady(sourceId, source.sizeBytes); + if (source.state !== "making") { + table.markReady(sourceId, source.sizeBytes, { + files: source.files ?? [], + lineage: source.lineage === null ? undefined : (source.lineage ?? []), + savedWithoutLineage: source.savedWithoutLineage ?? false, + }); + } return database; } @@ -50,3 +61,34 @@ export function plantReadySeed( ): SqliteDatabase { return plantSource(database, seed); } + +/** why: a file with the exact bytes a test names, which `writeFileAtomic` cannot hold for a UTF-16 or binary one. */ +export async function plantBytes( + filesystem: MemoryFilesystem, + path: string, + bytes: Uint8Array, +): Promise { + await filesystem.mkdirp(path.slice(0, path.lastIndexOf("/"))); + await filesystem.writeFileStream( + path, + new ReadableStream({ + start(controller) { + controller.enqueue(bytes); + controller.close(); + }, + }), + ); +} + +export async function readBytes(filesystem: MemoryFilesystem, path: string): Promise { + const file = await filesystem.openFile(path); + const chunks: Uint8Array[] = []; + const reader = file.read().getReader(); + for (;;) { + const piece = await reader.read(); + if (piece.done) break; + chunks.push(piece.value); + } + await file.close(); + return Buffer.concat(chunks); +} diff --git a/src/drivers/ios/udid-swap.ts b/src/drivers/ios/udid-swap.ts new file mode 100644 index 000000000..462241b66 --- /dev/null +++ b/src/drivers/ios/udid-swap.ts @@ -0,0 +1,47 @@ +/** why: a UDID is 36 characters, so its UTF-16 form is 72 bytes; a window overlaps the next by one byte less, and a match straddling two windows lies whole in one of them (ADR 0076 §3). */ +export const SWAP_OVERLAP_BYTES = 71; + +export interface LineageSwap { + readonly lineage: readonly string[]; + readonly own: string; +} + +/** why: ADR 0076 §3: a UDID is stored upper or lower case, as ASCII or as UTF-16 of either byte order, and no other way is looked for. */ +function forms(udid: string): readonly Buffer[] { + return [udid.toUpperCase(), udid.toLowerCase()].flatMap((text) => [ + Buffer.from(text, "latin1"), + Buffer.from(text, "utf16le"), + Buffer.from(text, "utf16le").swap16(), + ]); +} + +/** why: the finder of ADR 0076 §2: whether any lineage UDID is in the bytes in any of its forms. */ +export function namesLineage(chunk: Uint8Array, lineage: readonly string[]): boolean { + const bytes = Buffer.from(chunk.buffer, chunk.byteOffset, chunk.byteLength); + return lineage.some((udid) => forms(udid).some((form) => bytes.includes(form))); +} + +/** why: the UDID swap (ADR 0076 §3) replaces each lineage UDID with the copy's own in the same case and encoding; a UDID has one length, so the output has the input's length, and a caller checks it. */ +export function swapLineage(chunk: Uint8Array, swap: LineageSwap): Uint8Array { + let out = Buffer.from(chunk); + const ownForms = forms(swap.own); + for (const udid of swap.lineage) { + if (udid.toUpperCase() === swap.own.toUpperCase()) continue; + const replacements = forms(udid).map((form, index) => ({ + form, + own: ownForms[index] ?? form, + })); + for (const { form, own } of replacements) { + if (form.length === 0) continue; + const pieces: Buffer[] = []; + let from = 0; + for (let at = out.indexOf(form); at !== -1; at = out.indexOf(form, from)) { + pieces.push(out.subarray(from, at), own); + from = at + form.length; + } + pieces.push(out.subarray(from)); + out = Buffer.concat(pieces); + } + } + return out; +} diff --git a/src/ports/filesystem.test.ts b/src/ports/filesystem.test.ts index 61ea824a2..932f08414 100644 --- a/src/ports/filesystem.test.ts +++ b/src/ports/filesystem.test.ts @@ -235,6 +235,27 @@ describe.each(implementations)("Filesystem contract: $name", ({ create, link }) ); }); + it("reads where a link points, repoints it, and writes bytes into a file at an offset without truncating it or touching its modification time, and refuses to read a file as a link or write into a file that is not there", async () => { + const filesystem = create(); + const dir = temporaryDirectory; + await filesystem.mkdirp(dir); + await filesystem.writeFileAtomic(`${dir}/file`, "abcdef"); + // why: the memory filesystem has no disk file to age, and its write never touches the time. + if (filesystem instanceof NodeFilesystem) await utimes(`${dir}/file`, 1_000_000, 1_000_000); + const modifiedBefore = (await filesystem.stat(`${dir}/file`)).modifiedAtMs; + await link(filesystem, `${dir}/link`, `${dir}/old`); + + await expect(filesystem.readlink(`${dir}/link`)).resolves.toBe(`${dir}/old`); + await filesystem.replaceSymlink(`${dir}/link`, `${dir}/new`); + await expect(filesystem.readlink(`${dir}/link`)).resolves.toBe(`${dir}/new`); + await filesystem.writeAt(`${dir}/file`, 2, new TextEncoder().encode("XY")); + await expect(filesystem.readFile(`${dir}/file`)).resolves.toBe("abXYef"); + expect((await filesystem.stat(`${dir}/file`)).modifiedAtMs).toBe(modifiedBefore); + + await expect(filesystem.readlink(`${dir}/file`)).rejects.toThrow(); + await expect(filesystem.writeAt(`${dir}/missing`, 0, new Uint8Array(1))).rejects.toThrow(); + }); + it("refuses to answer for a path that runs through a file", async () => { // why: A mistyped device root -- `/notes.txt/ios` -- is a different problem from one // that is simply not there yet, and callers can only tell them apart by the errno. diff --git a/src/ports/filesystem.ts b/src/ports/filesystem.ts index 835a32b7d..b5046c63c 100644 --- a/src/ports/filesystem.ts +++ b/src/ports/filesystem.ts @@ -9,11 +9,13 @@ import { open, readdir, readFile, + readlink, realpath, rename, rm, stat, statfs, + symlink, writeFile, } from "node:fs/promises"; import { basename, dirname, join } from "node:path"; @@ -121,6 +123,12 @@ export interface Filesystem { diskFree(path: string): Promise; diskSpace(path: string): Promise<{ readonly freeBytes: number; readonly totalBytes: number }>; directorySize(path: string): Promise; + /** why: what a symbolic link points at, without following it. */ + readlink(path: string): Promise; + /** why: the link at `path` points at `target` from one step to the next, with no moment when it is missing. */ + replaceSymlink(path: string, target: string): Promise; + /** why: `bytes` written into an existing file at `offset`, with no truncation, keeping its mode and modification time. */ + writeAt(path: string, offset: number, bytes: Uint8Array): Promise; } // why: only the directories that are absent get the mode, so one that is already there keeps its own. @@ -406,6 +414,32 @@ export class NodeFilesystem implements Filesystem { } } + async readlink(path: string): Promise { + return readlink(path); + } + + async replaceSymlink(path: string, target: string): Promise { + const staged = `${path}.simlock-swap-${String(process.pid)}`; + await symlink(target, staged); + try { + await rename(staged, path); + } catch (error: unknown) { + await rm(staged, { force: true }); + throw error; + } + } + + async writeAt(path: string, offset: number, bytes: Uint8Array): Promise { + const handle = await open(path, "r+"); + try { + const { atime, mtime } = await handle.stat(); + await handle.write(bytes, 0, bytes.length, offset); + await handle.utimes(atime, mtime); + } finally { + await handle.close(); + } + } + async directorySize(path: string): Promise { let total = 0; for (const entry of await readdir(path, { withFileTypes: true })) { @@ -766,6 +800,33 @@ export class MemoryFilesystem implements Filesystem { return total; } + async readlink(path: string): Promise { + this.#failIfDefined(path); + const entry = this.#rawEntryAt(this.#underResolvedParent(path)); + if (entry.kind !== "symlink") throw errnoError("EINVAL", `Not a symbolic link: ${path}`); + return entry.target; + } + + async replaceSymlink(path: string, target: string): Promise { + this.#failIfDefined(path); + const at = this.#underResolvedParent(path); + const entry = this.#rawEntryAt(at); + if (entry.kind !== "symlink") throw errnoError("EINVAL", `Not a symbolic link: ${path}`); + this.#entries.set(at, { ...this.#newEntry({ kind: "symlink", target }), uid: entry.uid }); + } + + async writeAt(path: string, offset: number, bytes: Uint8Array): Promise { + this.#failIfDefined(path); + const entry = this.#entryAt(path); + if (entry.kind !== "file") throw errnoError("EISDIR", `Is a directory: ${path}`); + const old = fileBytes(entry); + const next = new Uint8Array(Math.max(old.byteLength, offset + bytes.byteLength)); + next.set(old); + next.set(bytes, offset); + entry.bytes = next; + entry.contents = Buffer.from(next).toString("utf8"); + } + /** * why: Test-only: makes every operation on `path` fail with `code`, which is the only way to * reach the branches that turn an unexpected filesystem failure into a typed rejection. diff --git a/src/ports/index.ts b/src/ports/index.ts index 869a4d0bf..821ff3ebf 100644 --- a/src/ports/index.ts +++ b/src/ports/index.ts @@ -161,6 +161,7 @@ export { WindowsHostLoadProbe, } from "./host-load.js"; export { + FakeSqliteOpener, filterSqliteExperimentalWarning, inTransaction, isLockError, diff --git a/src/ports/sqlite.ts b/src/ports/sqlite.ts index dabff6a28..38d641d3b 100644 --- a/src/ports/sqlite.ts +++ b/src/ports/sqlite.ts @@ -1,5 +1,7 @@ import { closeSync, openSync } from "node:fs"; +import type { MemoryFilesystem } from "./filesystem.js"; + export interface SqliteRunResult { readonly changes: number | bigint; readonly lastInsertRowid: number | bigint; @@ -46,6 +48,62 @@ export class NodeSqliteOpener implements SqliteOpener { } } +/** + * why: a fake for a caller that opens a database only to flush its WAL: it records each database + * a checkpoint ran on, can fail one, and a checkpoint moves the `-wal` file's bytes onto the end of + * its database in a memory filesystem and empties the WAL, unless told to leave its frames. + */ +export class FakeSqliteOpener implements SqliteOpener { + readonly checkpoints: string[] = []; + readonly #filesystem: MemoryFilesystem; + readonly #failing: ReadonlySet; + readonly #keepingFrames: ReadonlySet; + + constructor( + filesystem: MemoryFilesystem, + options: { + readonly failing?: readonly string[]; + readonly keepingFrames?: readonly string[]; + } = {}, + ) { + this.#filesystem = filesystem; + this.#failing = new Set(options.failing); + this.#keepingFrames = new Set(options.keepingFrames); + } + + async open(path: string): Promise { + if (this.#failing.has(path)) throw new Error(`unable to open database file ${path}`); + // why: `exec` is synchronous, so what a checkpoint moves is read here, and the filesystem calls + // below run their bodies at once. + const wal = `${path}-wal`; + const frames = await this.#filesystem.readFile(wal).then( + (text) => Buffer.from(text), + () => undefined, + ); + const size = frames === undefined ? 0 : (await this.#filesystem.stat(path)).size; + let emptied = false; + const checkpoint = (): void => { + this.checkpoints.push(path); + if (this.#keepingFrames.has(path)) return; + if (frames !== undefined) void this.#filesystem.writeAt(path, size, frames); + void this.#filesystem.rm(wal); + emptied = true; + }; + return { + // why: like SQLite, the last connection to close after a full checkpoint removes the `-shm` beside the emptied WAL. + close: () => { + if (emptied) void this.#filesystem.rm(`${path}-shm`); + }, + exec: (sql) => { + if (sql === "PRAGMA wal_checkpoint(TRUNCATE)") checkpoint(); + }, + prepare: () => { + throw new Error("FakeSqliteOpener runs no statements"); + }, + }; + } +} + const SQLITE_WARNING_PREFIX = "SQLite is an experimental feature"; export function filterSqliteExperimentalWarning(): void { diff --git a/website/src/content/docs/docs/reference/cli.mdx b/website/src/content/docs/docs/reference/cli.mdx index 042b20b44..88e8ba622 100644 --- a/website/src/content/docs/docs/reference/cli.mdx +++ b/website/src/content/docs/docs/reference/cli.mdx @@ -453,7 +453,7 @@ simlock doctor [--fix] [--purge-orphans] [--yes] | `--purge-orphans` | Destroy every orphaned device: a device in Simlock's device folders with no record. Also destroy every orphan source: a seed simulator with no record, or a saved baseline's copy that no baseline names. Needs the admin role. Asks for confirmation. | Off | | `--yes` | Skip the confirmation for `--purge-orphans`. Required when there is no terminal to ask in. | Off | -The report also lists, under `sources`, the copies on disk that new devices are made from. Each has `platform`, `kind` (`seed` or `baseline`), `model`, `osVersion`, `sizeBytes` when known, and `staleReason` when it no longer matches the installed runtime. A baseline source also has `baselineName` and `baselineVersion`. A seed is a simulator Simlock copies to make each iOS simulator a lease gets: it takes disk space but is never leased. A seed with no record, or a baseline copy no baseline names, shows `orphan: true`. That is not a finding; only `--purge-orphans` deletes it. +The report also lists, under `sources`, the copies on disk that new devices are made from. Each has `platform`, `kind` (`seed` or `baseline`), `model`, `osVersion`, `sizeBytes` when known, and `staleReason` when it no longer matches the installed runtime, or when an iOS baseline was saved before simulators stopped sharing folders, or from a lease made before that. A baseline source also has `baselineName` and `baselineVersion`. A seed is a simulator Simlock copies to make each iOS simulator a lease gets: it takes disk space but is never leased. A seed with no record, or a baseline copy no baseline names, shows `orphan: true`. That is not a finding; only `--purge-orphans` deletes it. On a worker whose login service names a Node or `main.js` that is gone, the report holds a `service-broken` finding and stderr gets `Service broken: () names missing . Run `simlock daemon install` again.` (`, ` for two). diff --git a/website/src/content/docs/docs/reference/errors.mdx b/website/src/content/docs/docs/reference/errors.mdx index f57ae212e..4d2f54ff1 100644 --- a/website/src/content/docs/docs/reference/errors.mdx +++ b/website/src/content/docs/docs/reference/errors.mdx @@ -69,7 +69,7 @@ In a script, check `code`, not the message: the wording of a message can change, | `BASELINE_SAVE_FAILED` | 1 | 500 | `baseline save` ended the lease, but Simlock could not keep a copy of the device. The message gives the reason. No baseline was saved, and the device is set aside for cleanup. | Lease a fresh device, prepare it again and save again. | | `UNKNOWN_BASELINE` | 12 | 404 | `baseline remove`, or a lease request with `--baseline`, naming a baseline this worker does not have. Through a gateway: no worker holds the baseline, or the worker `--worker` names does not. | List the baselines with `simlock baseline list`. | | `BASELINE_MISMATCH` | 2 | 422 | A lease request names a baseline and also a platform, model, OS, mode or image tag that differs from it. The message names the first field that differs. | Drop the field, or change it to the baseline's own value. | -| `BASELINE_STALE` | 12 | 422 | A lease request names a baseline whose runtime or image has been removed since it was saved. The message gives the reason. | Install the runtime or image again, or save the baseline again from a new device. | +| `BASELINE_STALE` | 12 | 422 | A lease request names a baseline whose runtime or image has been removed since it was saved, or an iOS baseline saved before simulators stopped sharing folders, or from a lease made before that. The message gives the reason. | Install the runtime or image again, or save the baseline again from a new device. | | `BASELINE_IN_USE` | 12 | 409 | `baseline remove` of a baseline a lease still uses. The message names the lease. | End that lease, or remove the baseline after it. | | `COMPONENT_BUSY` | 11 | 409 | `component remove` while an install or removal runs or waits on that platform. | Try again once it ends. | diff --git a/website/src/content/docs/docs/reference/events.mdx b/website/src/content/docs/docs/reference/events.mdx index f262e5246..5dcbe32e3 100644 --- a/website/src/content/docs/docs/reference/events.mdx +++ b/website/src/content/docs/docs/reference/events.mdx @@ -168,7 +168,7 @@ release and records nothing. | --- | --- | --- | --- | | `baseline.saved` | `simlock baseline save` kept a device as a baseline. | `name`, `version`, `platform`, `model`, `osVersion`, `mode`, `imageTag` when the device has one, `sizeBytes` when known, `leaseId`, `requesterId` | Relayed | | `baseline.removed` | One version of a baseline was deleted, by `baseline remove`, by `nuke --delete-devices` or because a newer save of the name replaced it. | `name`, `version`, `reason` (`operator`, `nuke` or `replaced`), `requesterId` when someone asked | Relayed | -| `baseline.stale` | A baseline became stale: the runtime, system image or emulator it was made with changed after the save. | `name`, `version`, `reason` | Relayed | +| `baseline.stale` | A baseline became stale: the runtime, system image or emulator it was made with changed, or an iOS baseline was saved in a way that cannot be copied safely and must be saved again. A check runs after a save too. | `name`, `version`, `reason` | Relayed | ## Workers diff --git a/website/src/content/docs/docs/using-devices/errors.mdx b/website/src/content/docs/docs/using-devices/errors.mdx index ef2ce6f30..d4f2f2242 100644 --- a/website/src/content/docs/docs/using-devices/errors.mdx +++ b/website/src/content/docs/docs/using-devices/errors.mdx @@ -45,7 +45,7 @@ In a script, check the exit code or `code`, never the message: the message is fo | `RUNTIME_MISSING` | 12 | The OS version, or the Android image, isn't installed. An OS range or `--image-tag` never downloads. | Pass `--allow-download` for an exact version, or ask an admin to install it. See [Choosing a device](/simlock/docs/using-devices/choosing-a-device#when-the-runtime-is-not-installed). | | `UNKNOWN_BASELINE` | 12 | `--baseline` names a baseline this machine doesn't have. | Check the name with `simlock baseline list`. | | `BASELINE_MISMATCH` | 2 | `--baseline` came with a platform, model, OS, mode or image tag that differs from the baseline. The message names the first one that differs. | Drop that option: the baseline already decides it. | -| `BASELINE_STALE` | 12 | The OS or image the baseline was saved on has been removed. | Ask an admin to install it again, or save the baseline again. | +| `BASELINE_STALE` | 12 | The OS or image the baseline was saved on has been removed, or an iOS baseline was saved before simulators stopped sharing folders, or from a lease made before that. | Ask an admin to install it again, or save the baseline again. | | `LICENSE_NOT_ACCEPTED` | 12 | An Android download needs the SDK licences accepted, and this machine hasn't accepted them. | An admin sets `downloads.acceptAndroidLicenses` to `true`. | | `INSUFFICIENT_DISK_SPACE` | 12 | The download doesn't fit on the disk. The message says how much is needed and free. | Free disk space on that machine, or ask for an installed version. | | `DOWNLOAD_TIMEOUT` | 10 | The download, including time waiting behind another download, took longer than the machine allows. | Retry; the download may already be further along. Or ask an admin to install the runtime. | diff --git a/website/src/content/docs/docs/worker/baselines.mdx b/website/src/content/docs/docs/worker/baselines.mdx index c477df398..683273eab 100644 --- a/website/src/content/docs/docs/worker/baselines.mdx +++ b/website/src/content/docs/docs/worker/baselines.mdx @@ -72,7 +72,7 @@ login 1 ios iPhone 17 27.0 slim - 3.1 GB 2026-11 stale: runtime com.apple.CoreSimulator.SimRuntime.iOS-27-0 is not installed ``` -A baseline is stale when the iOS runtime, Android system image or emulator it was saved with has changed or been removed. The reason is on the line under it. A lease that names a stale baseline fails with `BASELINE_STALE`. Install that runtime or image again and the baseline works again; or save a new version from a device on the runtime you have now. +A baseline is stale when the iOS runtime, Android system image or emulator it was saved with has changed or been removed. An iOS baseline is stale too when it was saved before Simlock kept each simulator in its own folders, or from a lease made before that; save it again from a new lease. The reason is on the line under it. A lease that names a stale baseline fails with `BASELINE_STALE`. Install that runtime or image again and the baseline works again; or save a new version from a device on the runtime you have now. ## Remove a baseline @@ -95,7 +95,7 @@ This deletes every version of `login` and the idle copies made from it. If an ag | --- | --- | --- | | `BASELINE_SAVE_FAILED` | The lease has ended, but Simlock could not copy the device. The message says why. No baseline was saved, and the device is kept aside for `simlock doctor`. | Lease a new device, prepare it again and save again. | | `UNKNOWN_BASELINE` | No baseline on this worker has that name. | Check the name with `simlock baseline list`. | -| `BASELINE_STALE` | The runtime or image the baseline needs is gone. | Install it again, or save a new version. | +| `BASELINE_STALE` | The runtime or image the baseline needs is gone, or an iOS baseline was saved before simulators stopped sharing folders, or from a lease made before that. | Install it again, or save a new version. | | `BASELINE_IN_USE` | `remove` was refused because a lease uses a copy. | Wait for that lease to end, then remove. | | `DEVICE_BUSY` | `remove` was refused because a copy is being cleaned up or another operation holds it. | Try again in a moment. | | `FORBIDDEN` | The session does not have the admin role. | Run the command as an admin. |