Skip to content

The e2e device provider leases, installs, renews and releases, with a Run tests from CI page #1567

Closed

Description

@V3RON

Part of #1562.

Scope

The provider works. After this PR a team installs @simlock/e2e, passes simlockDevices({...}) to the e2e runner, and each worker gets a device from the farm with the app installed: the provider waits on a full farm, retries a cut upload, swaps a device that cannot be set up, renews while the run lives and releases at the end. The website gains the "Run tests from CI" page.

If this goes wrong, a user notices a run that fails at device setup on a healthy farm, or devices that stay leased after a run ended.

Technical spec

Commit d8fadd1.

Rows

  • Behaviours: B2, B3, B4, B5, B6, B7, B8, B9, B13
  • Stored state: S2
  • Interfaces: I2, I3 (uses I1, I4, I6, I7)
  • Existing code: X15, X16, X17, X18, X19, X20, X21, X22, X23, X24

Facts

# Fact Source
F1 The slow-lane helpers make an agent token and load agent-device's client library from the pinned install. e2e/helpers/agent-device.ts:101, e2e/helpers/agent-device.ts:197
F2 The slow lane installs the Appduct fixture app from target.app. e2e/slow-agent-device.test.ts:226
F3 A slow-lane file is run with scripts/slow-e2e.sh <file>. scripts/slow-e2e.sh:4
F4 The agent-device client's allocate sends its options and returns the lease normalised: leaseId, tenantId, runId, backend, and deviceKey, clientId when present. v0.21.36:packages/client/src/lease-client.ts lines 20-29 and 80-90
F5 The website sidebar lists Using devices pages one doc(...) line each. website/astro.config.mjs:98-104
F6 The token steps a CI page links to exist: simlock token create --role agent --label ci-runner-3. src/cli/commands/token.ts:14; shown to users at website/src/content/docs/docs/using-devices/remote-agents.mdx:27
F7 The agent-device page documents the 330 s wait and the error reasons the new page refers to. website/src/content/docs/docs/using-devices/agent-device.mdx:131, website/src/content/docs/docs/using-devices/agent-device.mdx:212-224

Modules touched

Searches: rg -n "createProvider|simlockDevices" packages e2e, rg -n "using-devices/" website/astro.config.mjs.

  • packages/e2e/src/provider.ts and new files beside it for the queue, the install and the renewal, with their tests
  • e2e/slow-e2e-provider.test.ts (new), e2e/helpers/agent-device.ts (export the library loader for the provider)
  • website/src/content/docs/docs/using-devices/ci.mdx (new), website/astro.config.mjs (sidebar), links from using-devices/remote-agents.mdx and using-devices/agent-device.mdx
  • packages/e2e/README.md (link to the page)
  • .agents/scripts/docs-surface-check.mjs, .agents/scripts/surface-paths.mjs and their tests (src/docs-surface-check.test.ts)

Rules in play

  • ADR 0077 §3: the provider imports nothing from src/; the slow-lane test under e2e/ imports the provider, the one allowed direction.
  • ADR 0070 §1, §2, §4: no boot or close handling in the provider; retainOnClose: true on every allocate; retry decisions read retriable, never message text.
  • docs/internal/agent-rules/documentation.md rule 9: the page ships in this PR with the provider.
  • docs/internal/agent-rules/testing.md: fake timers for the queue and the renewal; no real sleep in the fast lane.
  • docs/internal/agent-rules/toolchain.md (slow lane): B9 needs the "agent-device through simlock" machine checks with iOS.

Tests

The seam is createProvider(options, loadClient) with a fake client that records calls and answers from a script; model the fake on the injected client in rnw-community/mobile-ci packages/e2e-simlock/test/provider.test.ts, and the slow-lane test on e2e/slow-agent-device.test.ts.

  • B2: the runner calls acquire: iOS UDID from deviceKey; Android serial; optional fields sent only when set; platform mismatch throws; no deviceKey releases and throws; agent-device cannot be imported throws.
  • B3: the farm is full or cut the request: retriable error asks again under a new id after releasing the old; no answer does the same; not retriable throws at once; abort during a pause or an ask; queueMs over during an ask; two instant retriable errors are 10 s apart; second ask succeeds.
  • B4: acquire installs: appPath with app; appPath without app; no appPath but extraApps; five slots with installConcurrency: 3 never exceed three in flight.
  • B5: a cut install is sent again: cut then fine; three cuts. A cut is recognised the way B3 recognises "no answer from simlock": by the error's shape (no simlock details.reason on it), never by its message text, so an upload whose answer is not JSON is retried and a test where the client error says something else is still retried (decided in delivery, pending the maintainer).
  • B6: a failed install swaps once: fails on one, fine on two; fails on both; three cuts or a retriable farm error do not swap. An error with no simlock details.reason (a sign-in error, or an upload simlock refused, whose reason the client drops) counts as a cut: sent three times, then thrown with no swap; the comments in packages/e2e/src/errors.ts and install.ts name these cases (decided in delivery, pending the maintainer).
  • B7: the provider renews: 180000 gives 60 s; 30000 gives 10 s; a failure then success; ended by the farm; none after release.
  • B8: the runner calls release: held lease; failed release resolves and logs; unknown lease.
  • B9: on a Mac with an iOS runtime, e2e/slow-e2e-provider.test.ts: acquire with the fixture app; close; lease still listed; release; lease gone.
  • B13: the docs page: the docs surface check fails for a change to packages/e2e/src/index.ts with no website change (a case in the check's own test).

Done when

  • pnpm check passes.
  • On a Mac with an iOS runtime: scripts/slow-e2e.sh e2e/slow-e2e-provider.test.ts passes.
  • pnpm --dir website build passes and the built site has docs/using-devices/ci/.
  • The page lists every option of I1 with its default; grep -c "installConcurrency\|queueMs\|ttlMs\|deviceBaseline\|deviceMode\|imageTag\|osVersion\|extraApps" website/src/content/docs/docs/using-devices/ci.mdx prints at least 8.

Out of scope

Depends on

Written by an agent.

Activity

  1. added
    task:draftScope written; technical spec or its spec check missing.
    on Nov 15, 2026
  2. added
    task:readySpec checked. An agent may implement it once its Depends on allow.
    and removed
    task:draftScope written; technical spec or its spec check missing.
    on Nov 15, 2026
  3. self-assigned this
    on Nov 15, 2026
  4. 10 remaining items

  5. V3RON commented on Nov 15, 2026

    @V3RON
    ContributorAuthor

    Handoff

    Done: #1572 is built, audited, and passed review: round 1, round 2 and a final fix-check, with all six blocking findings fixed. The PR is ready and tops the feature stack, #1565 → #1568 → #1572. Three decisions are pending the maintainer: #1565's half-published-release recovery, #1568's text-pinned workflow tests, and #1572's no-reason-means-cut rule, each in a ## Decision taken comment.

    Not done: Done when line 2, scripts/slow-e2e.sh e2e/slow-e2e-provider.test.ts, and the matching #1562 completion condition. The audit could not probe that test, and the probe needs a slow-lane run: make release in packages/e2e/src/provider.ts return early, then check that the final listLeases assertion fails.

    Findings: none open. One review note is posted on #1572.

    Blocked on: a Mac with an iOS runtime and SIMLOCK_E2E_AGENT_DEVICE set, or .slow-lane.env. Run the test there, or re-run deliver 1562 on that machine. The stack then merges with merge-pr.sh 1572 after ADR 0077 flips to Accepted.

    Written by an agent.

  6. added
    task:blockedAgent stopped on a blocker in its handoff. Maintainer re-adds the label it came from.
    and removed
    task:readySpec checked. An agent may implement it once its Depends on allow.
    on Nov 15, 2026
  7. removed their assignment
    on Nov 15, 2026
  8. added
    task:readySpec checked. An agent may implement it once its Depends on allow.
    and removed
    task:blockedAgent stopped on a blocker in its handoff. Maintainer re-adds the label it came from.
    on Nov 15, 2026
  9. self-assigned this
    on Nov 15, 2026
  10. added a commit that references this issue on Nov 15, 2026
    6db7d04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

task:readySpec checked. An agent may implement it once its Depends on allow.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions