Skip to content

feat: extract AWS Device Farm provider plugin - #4617

Open
thymikee wants to merge 13 commits into
fix/a1-extraction-unused-import-20261116from
feat/0.22-aws-device-farm-plugin
Open

thymikee wants to merge 13 commits into
fix/a1-extraction-unused-import-20261116from
feat/0.22-aws-device-farm-plugin

Conversation

@thymikee

@thymikee thymikee commented Nov 16, 2026 •

Copy link
Copy Markdown
Member

Summary

Extract AWS Device Farm into a private provider plugin so the core package does not include AWS-specific integration. Plugin commands retain the supplied host cwd and environment, including request overrides. This coherent layer changes 99 files and includes shared registry, replay, and connection host propagation.

Depends on #4579; targets its branch. Related: #3310, #4591.

Validation

At 98dfddb46ff137ceadf18560ab8a6aba10619613, local exact-commit affected and full gates passed (42 and 41 checks respectively; 25 skipped each). Released-wire comparison passed against v0.21.38. The local packed private-store fixture passed its failure contract, isolated install, and CLI connect; core SHA-256 6aa05df5df0ad3cace29a2d319c9a9964c71608d127e5619f950d0ed5a5c5dee, plugin SHA-256 91c602afaebabb7402e047660f9bf1df6f1bf1743c27557a737011c386b1ce1b.

Independent source reviews covered the extraction at f9a0b34067 and the host-context correction through this head; neither independently reran checks. Hosted CI is pending. Supported manager installation and live AWS acceptance remain unverified. Before public release, a maintainer must publish an empty placeholder and configure the trusted publisher; the package remains private. Integration into 0.22 depends on #4579 and resolution of these holds.


Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.

View guided diff Turn on auto-fix

@github-actions

Copy link
Copy Markdown
Contributor

Size Report

Metric Base Current Diff
Installed (including dependencies) 6.12 MB 6.11 MB -12.6 kB
Package (unpacked) 6.12 MB 6.11 MB -12.6 kB
Package (download) 1.86 MB 1.85 MB -4.3 kB

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 27.7 ms 32.2 ms +4.5 ms
CLI --help 50.7 ms 60.9 ms +10.1 ms

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

17 issues found across 99 files

You’re at about 98% of the monthly reviewed-line limit. You may want to disable incremental reviews to conserve quota. Reviews will continue until that limit is exceeded. If you need help avoiding interruptions, please contact contact@cubic.dev.

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="test/integration/provider-scenarios/cloud-webdriver-provider-regressions.test.ts">

<violation number="1" location="test/integration/provider-scenarios/cloud-webdriver-provider-regressions.test.ts:40">
P3: These tests now duplicate the provider ID instead of using `AWS_DEVICE_FARM_PROVIDER` from `./aws-runtime.ts`; use the shared constant for every runtime lookup and lease so provider identity cannot drift.

(Based on your team's feedback about aligning provider-specific test identities.)</violation>
</file>

<file name="website/docs/docs/device-clouds.md">

<violation number="1" location="website/docs/docs/device-clouds.md:9">
P1: These public docs now direct users to install `@agent-device/aws-device-farm`, but the package is still private and cannot be installed from npm. Remove or gate these references until the package is published; otherwise the documented AWS flow is blocked immediately.</violation>
</file>

<file name="src/__tests__/provider-device-runtimes.test.ts">

<violation number="1" location="src/__tests__/provider-device-runtimes.test.ts:34">
P3: This expectation removes the last active built-in from the test, so the test no longer covers built-in precedence despite its title. Rename the test to describe the remaining Limrun/store behavior, or configure a built-in and retain an explicit precedence assertion.</violation>
</file>

<file name="website/docs/docs/plugins.md">

<violation number="1" location="website/docs/docs/plugins.md:16">
P2: This command is documented as immediately installable, but the AWS package remains private and unpublished in this PR. Qualify the instruction with “after publication” or omit AWS until the package is available.</violation>

<violation number="2" location="website/docs/docs/plugins.md:16">
P3: The missing-plugin recovery instructions still enumerate only BrowserStack and TestMu AI, so AWS users are not told to expect the installation command and retry guidance. Include `aws-device-farm` in that recovery example and explanation.</violation>
</file>

<file name="scripts/layering/package-boundaries.ts">

<violation number="1" location="scripts/layering/package-boundaries.ts:592">
P2: This filter drops real relative imports from the script-test boundary scan, including relative tunnels into `packages/*/src`. Pass all parsed sites; `checkRootSites` already permits ordinary script-relative imports and rejects only package-boundary tunnels.</violation>

<violation number="2" location="scripts/layering/package-boundaries.ts:595">
P2: This marks every `scripts/*.test.ts` import as compiler-owned, so R11 no longer rejects unknown workspace packages or unexported subpaths in these real test imports. Pass the script file’s actual compiled classification so scripts remain fail-closed.</violation>
</file>

<file name="packages/provider-aws-device-farm/package.json">

<violation number="1" location="packages/provider-aws-device-farm/package.json:35">
P2: The published package drops the declared `./connection-verification` export, so consumers cannot import this subpath after installation. Add a bundled connection-verification entry and retain it in `publishConfig.exports`, or remove the development export if it is not part of the package API.</violation>
</file>

<file name="website/docs/docs/aws-device-farm.md">

<violation number="1" location="website/docs/docs/aws-device-farm.md:10">
P2: This setup directs embedded hosts to the CLI plugin store, which embedded runtimes ignore, and `daemon stop` cannot replace a live embedded runtime. Add an embedded-host example using `awsDeviceFarm()` in `startAgentDevice({ plugins: [...] })` and instruct users to close and recreate that runtime after changes.</violation>

<violation number="2" location="website/docs/docs/aws-device-farm.md:16">
P2: Installing a missing plugin does not update an already-running daemon, so retrying immediately still reports the provider as unavailable. Tell store-backed users to finish sessions, stop the matching daemon, and then retry.</violation>
</file>

<file name="packages/provider-aws-device-farm/README.md">

<violation number="1" location="packages/provider-aws-device-farm/README.md:6">
P2: This installation command cannot work for users while `@agent-device/aws-device-farm` remains private and unpublished. Keep the README explicitly unreleased until publication, or publish the package through the required release sequence before documenting this as the install path.</violation>
</file>

<file name=".changeset/aws-device-farm-plugin.md">

<violation number="1" location=".changeset/aws-device-farm-plugin.md:5">
P2: This upgrade instruction is misleading for both current availability and embedded SDK hosts: the package is still unpublished, and embedded runtimes neither read the CLI plugin store nor support `daemon stop`. State the publication prerequisite, then give CLI hosts the store/restart path and embedded hosts the `startAgentDevice({ plugins })` plus close/recreate path.</violation>
</file>

<file name="packages/plugins/src/host.ts">

<violation number="1" location="packages/plugins/src/host.ts:34">
P2: This `maxBuffer` setting does not bound `runHostCommand`: asynchronous `runCmd` ignores it and accumulates stdout/stderr without a limit. Add enforcement to the async executor (or use an async bounded command API) before relying on this 10 MiB cap for provider-controlled output.</violation>
</file>

<file name="packages/provider-aws-device-farm/test/package-smoke.mjs">

<violation number="1" location="packages/provider-aws-device-farm/test/package-smoke.mjs:11">
P2: These smoke-test arguments are not valid AWS Device Farm ARNs, so the PATH fixture masks a malformed real `connect` request. Use the colon resource separators (`:project:`, `:device:`, and `:upload:`) so this package smoke test exercises realistic selectors.</violation>
</file>

<file name="src/builtin-provider-plugins.ts">

<violation number="1" location="src/builtin-provider-plugins.ts:19">
P2: This removes AWS Device Farm from the built-in registry, but the plugin guide still tells users that daemon status reports it as bundled. Update that status documentation to classify AWS Device Farm as an installed plugin so users do not misread its source or omit the required plugin installation.</violation>
</file>

<file name="README.md">

<violation number="1" location="README.md:154">
P3: This parenthetical makes `agent-device plugins add` apply only to TestMu AI, leaving BrowserStack and AWS Device Farm's plugin requirement unclear. Say that all three providers require plugin installation.</violation>
</file>

<file name="src/commands/provider-plugin-catalog.ts">

<violation number="1" location="src/commands/provider-plugin-catalog.ts:10">
P2: The catalog directs users to install `@agent-device/aws-device-farm`, but that package is still marked `private: true`, so `plugins add` cannot install it from the registry. Remove this catalog entry until the package is published, or point it to an installable package.</violation>
</file>

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic


- [BrowserStack](/docs/browserstack): Android and iOS App Automate sessions over WebDriver. Install the `@agent-device/browserstack` plugin first.
- [AWS Device Farm](/docs/aws-device-farm): Android and iOS remote-access sessions through AWS.
- [AWS Device Farm](/docs/aws-device-farm): Android and iOS remote-access sessions through AWS. Install the `@agent-device/aws-device-farm` plugin first.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: These public docs now direct users to install @agent-device/aws-device-farm, but the package is still private and cannot be installed from npm. Remove or gate these references until the package is published; otherwise the documented AWS flow is blocked immediately.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At website/docs/docs/device-clouds.md, line 9:

<comment>These public docs now direct users to install `@agent-device/aws-device-farm`, but the package is still private and cannot be installed from npm. Remove or gate these references until the package is published; otherwise the documented AWS flow is blocked immediately.</comment>

<file context>
@@ -6,7 +6,7 @@ description: Choose a hosted device provider for agent and CI workflows.
 
 - [BrowserStack](/docs/browserstack): Android and iOS App Automate sessions over WebDriver. Install the `@agent-device/browserstack` plugin first.
-- [AWS Device Farm](/docs/aws-device-farm): Android and iOS remote-access sessions through AWS.
+- [AWS Device Farm](/docs/aws-device-farm): Android and iOS remote-access sessions through AWS. Install the `@agent-device/aws-device-farm` plugin first.
 - [TestMu AI](/docs/testmu): Android emulator, iOS simulator, and real-device sessions over WebDriver.
 - [Limrun](/docs/limrun): direct iOS simulator and Android emulator instances.
</file context>

`agent-device help connect` and `agent-device help remote` identify known optional providers as
`plugin, install separately` and show their installation commands, even before you install them.
For example, BrowserStack requires `agent-device plugins add @agent-device/browserstack`, and TestMu AI requires `agent-device plugins add @agent-device/testmu`.
For example, BrowserStack requires `agent-device plugins add @agent-device/browserstack`, AWS Device Farm requires `agent-device plugins add @agent-device/aws-device-farm`, and TestMu AI requires `agent-device plugins add @agent-device/testmu`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: This command is documented as immediately installable, but the AWS package remains private and unpublished in this PR. Qualify the instruction with “after publication” or omit AWS until the package is available.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At website/docs/docs/plugins.md, line 16:

<comment>This command is documented as immediately installable, but the AWS package remains private and unpublished in this PR. Qualify the instruction with “after publication” or omit AWS until the package is available.</comment>

<file context>
@@ -13,7 +13,7 @@ agent-device plugins remove @example/agent-device-provider
 `agent-device help connect` and `agent-device help remote` identify known optional providers as
 `plugin, install separately` and show their installation commands, even before you install them.
-For example, BrowserStack requires `agent-device plugins add @agent-device/browserstack`, and TestMu AI requires `agent-device plugins add @agent-device/testmu`.
+For example, BrowserStack requires `agent-device plugins add @agent-device/browserstack`, AWS Device Farm requires `agent-device plugins add @agent-device/aws-device-farm`, and TestMu AI requires `agent-device plugins add @agent-device/testmu`.
 
 If you run `agent-device connect browserstack` or `agent-device connect testmu` without its plugin, the error gives that installation
</file context>
Suggested change
For example, BrowserStack requires `agent-device plugins add @agent-device/browserstack`, AWS Device Farm requires `agent-device plugins add @agent-device/aws-device-farm`, and TestMu AI requires `agent-device plugins add @agent-device/testmu`.
For example, BrowserStack requires `agent-device plugins add @agent-device/browserstack`, AWS Device Farm will require `agent-device plugins add @agent-device/aws-device-farm` after publication, and TestMu AI requires `agent-device plugins add @agent-device/testmu`.

// still need root declarations; fixture strings are not module records.
violations.push(
...checkRootSites(
sites.filter((site) => !site.specifier.startsWith('.')),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: This filter drops real relative imports from the script-test boundary scan, including relative tunnels into packages/*/src. Pass all parsed sites; checkRootSites already permits ordinary script-relative imports and rejects only package-boundary tunnels.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At scripts/layering/package-boundaries.ts, line 592:

<comment>This filter drops real relative imports from the script-test boundary scan, including relative tunnels into `packages/*/src`. Pass all parsed sites; `checkRootSites` already permits ordinary script-relative imports and rejects only package-boundary tunnels.</comment>

<file context>
@@ -582,18 +582,23 @@ export function checkPackageBoundaries(repoRoot: string): PackageBoundaryViolati
+        // still need root declarations; fixture strings are not module records.
+        violations.push(
+          ...checkRootSites(
+            sites.filter((site) => !site.specifier.startsWith('.')),
+            packages,
+            rootDependencies,
</file context>
Suggested change
sites.filter((site) => !site.specifier.startsWith('.')),
sites,

sites.filter((site) => !site.specifier.startsWith('.')),
packages,
rootDependencies,
true,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: This marks every scripts/*.test.ts import as compiler-owned, so R11 no longer rejects unknown workspace packages or unexported subpaths in these real test imports. Pass the script file’s actual compiled classification so scripts remain fail-closed.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At scripts/layering/package-boundaries.ts, line 595:

<comment>This marks every `scripts/*.test.ts` import as compiler-owned, so R11 no longer rejects unknown workspace packages or unexported subpaths in these real test imports. Pass the script file’s actual compiled classification so scripts remain fail-closed.</comment>

<file context>
@@ -582,18 +582,23 @@ export function checkPackageBoundaries(repoRoot: string): PackageBoundaryViolati
+            sites.filter((site) => !site.specifier.startsWith('.')),
+            packages,
+            rootDependencies,
+            true,
+          ),
+        );
</file context>
Suggested change
true,
insideCompiledSources(file),

"@agent-device/provider-webdriver": "workspace:*",
"agent-device": "workspace:*"
},
"publishConfig": {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The published package drops the declared ./connection-verification export, so consumers cannot import this subpath after installation. Add a bundled connection-verification entry and retain it in publishConfig.exports, or remove the development export if it is not part of the package API.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/provider-aws-device-farm/package.json, line 35:

<comment>The published package drops the declared `./connection-verification` export, so consumers cannot import this subpath after installation. Add a bundled connection-verification entry and retain it in `publishConfig.exports`, or remove the development export if it is not part of the package API.</comment>

<file context>
@@ -0,0 +1,53 @@
+    "@agent-device/provider-webdriver": "workspace:*",
+    "agent-device": "workspace:*"
+  },
+  "publishConfig": {
+    "exports": {
+      ".": {
</file context>

const PROVIDER_PLUGIN_CATALOG: readonly ProviderPluginDeclaration[] = [
{
provider: 'aws-device-farm',
package: '@agent-device/aws-device-farm',

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The catalog directs users to install @agent-device/aws-device-farm, but that package is still marked private: true, so plugins add cannot install it from the registry. Remove this catalog entry until the package is published, or point it to an installable package.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At src/commands/provider-plugin-catalog.ts, line 10:

<comment>The catalog directs users to install `@agent-device/aws-device-farm`, but that package is still marked `private: true`, so `plugins add` cannot install it from the registry. Remove this catalog entry until the package is published, or point it to an installable package.</comment>

<file context>
@@ -5,6 +5,11 @@ type ProviderPluginDeclaration = Readonly<{
 const PROVIDER_PLUGIN_CATALOG: readonly ProviderPluginDeclaration[] = [
+  {
+    provider: 'aws-device-farm',
+    package: '@agent-device/aws-device-farm',
+    label: 'AWS Device Farm',
+  },
</file context>

runHostCommand: host.run,
}),
CLOUD_WEBDRIVER_PROVIDERS.awsDeviceFarm,
'aws-device-farm',

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: These tests now duplicate the provider ID instead of using AWS_DEVICE_FARM_PROVIDER from ./aws-runtime.ts; use the shared constant for every runtime lookup and lease so provider identity cannot drift.

(Based on your team's feedback about aligning provider-specific test identities.)

View Feedback

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At test/integration/provider-scenarios/cloud-webdriver-provider-regressions.test.ts, line 40:

<comment>These tests now duplicate the provider ID instead of using `AWS_DEVICE_FARM_PROVIDER` from `./aws-runtime.ts`; use the shared constant for every runtime lookup and lease so provider identity cannot drift.

(Based on your team's feedback about aligning provider-specific test identities.) </comment>

<file context>
@@ -38,9 +37,9 @@ test('AWS Device Farm endpoint selection skips live-control WebSocket URLs', asy
         runHostCommand: host.run,
       }),
-      CLOUD_WEBDRIVER_PROVIDERS.awsDeviceFarm,
+      'aws-device-farm',
     );
-    const lease = providerRegressionLease(CLOUD_WEBDRIVER_PROVIDERS.awsDeviceFarm);
</file context>

assert.deepEqual(
composition.runtimes.map((runtime) => runtime.provider),
['aws-device-farm', 'example'],
['example'],

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: This expectation removes the last active built-in from the test, so the test no longer covers built-in precedence despite its title. Rename the test to describe the remaining Limrun/store behavior, or configure a built-in and retain an explicit precedence assertion.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At src/__tests__/provider-device-runtimes.test.ts, line 34:

<comment>This expectation removes the last active built-in from the test, so the test no longer covers built-in precedence despite its title. Rename the test to describe the remaining Limrun/store behavior, or configure a built-in and retain an explicit precedence assertion.</comment>

<file context>
@@ -31,14 +31,11 @@ test('built-in providers register ahead of the store and an unconfigured Limrun
   assert.deepEqual(
     composition.runtimes.map((runtime) => runtime.provider),
-    ['aws-device-farm', 'example'],
+    ['example'],
   );
   assert.deepEqual(
</file context>

`agent-device help connect` and `agent-device help remote` identify known optional providers as
`plugin, install separately` and show their installation commands, even before you install them.
For example, BrowserStack requires `agent-device plugins add @agent-device/browserstack`, and TestMu AI requires `agent-device plugins add @agent-device/testmu`.
For example, BrowserStack requires `agent-device plugins add @agent-device/browserstack`, AWS Device Farm requires `agent-device plugins add @agent-device/aws-device-farm`, and TestMu AI requires `agent-device plugins add @agent-device/testmu`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The missing-plugin recovery instructions still enumerate only BrowserStack and TestMu AI, so AWS users are not told to expect the installation command and retry guidance. Include aws-device-farm in that recovery example and explanation.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At website/docs/docs/plugins.md, line 16:

<comment>The missing-plugin recovery instructions still enumerate only BrowserStack and TestMu AI, so AWS users are not told to expect the installation command and retry guidance. Include `aws-device-farm` in that recovery example and explanation.</comment>

<file context>
@@ -13,7 +13,7 @@ agent-device plugins remove @example/agent-device-provider
 `agent-device help connect` and `agent-device help remote` identify known optional providers as
 `plugin, install separately` and show their installation commands, even before you install them.
-For example, BrowserStack requires `agent-device plugins add @agent-device/browserstack`, and TestMu AI requires `agent-device plugins add @agent-device/testmu`.
+For example, BrowserStack requires `agent-device plugins add @agent-device/browserstack`, AWS Device Farm requires `agent-device plugins add @agent-device/aws-device-farm`, and TestMu AI requires `agent-device plugins add @agent-device/testmu`.
 
 If you run `agent-device connect browserstack` or `agent-device connect testmu` without its plugin, the error gives that installation
</file context>

Comment thread README.md
| Local | Trying commands and debugging apps on simulators, emulators, physical devices, macOS, and Linux. | Follow the Quick Start. |
| CI/CD | Automated pull request and merge validation with replay scripts and captured artifacts. | Try the [EAS workflow template](https://github.com/callstackincubator/eas-agent-device/blob/main/.eas/workflows/agent-qa-mobile.yml). |
| Cloud / remote | Linux runners, managed devices, and remote jobs. | Set up a [remote proxy](https://agent-device.dev/docs/remote-proxy), connect a [device cloud](https://agent-device.dev/docs/device-clouds) (BrowserStack and TestMu AI through `agent-device plugins add`, AWS Device Farm, Limrun), or [contact Callstack](mailto:hello@callstack.com) for team QA. |
| Cloud / remote | Linux runners, managed devices, and remote jobs. | Set up a [remote proxy](https://agent-device.dev/docs/remote-proxy), connect a [device cloud](https://agent-device.dev/docs/device-clouds) (BrowserStack, AWS Device Farm, and TestMu AI through `agent-device plugins add`, Limrun), or [contact Callstack](mailto:hello@callstack.com) for team QA. |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: This parenthetical makes agent-device plugins add apply only to TestMu AI, leaving BrowserStack and AWS Device Farm's plugin requirement unclear. Say that all three providers require plugin installation.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At README.md, line 154:

<comment>This parenthetical makes `agent-device plugins add` apply only to TestMu AI, leaving BrowserStack and AWS Device Farm's plugin requirement unclear. Say that all three providers require plugin installation.</comment>

<file context>
@@ -151,7 +151,7 @@ The same session and evidence model works at every step: the agent explores the
 | Local | Trying commands and debugging apps on simulators, emulators, physical devices, macOS, and Linux. | Follow the Quick Start. |
 | CI/CD | Automated pull request and merge validation with replay scripts and captured artifacts. | Try the [EAS workflow template](https://github.com/callstackincubator/eas-agent-device/blob/main/.eas/workflows/agent-qa-mobile.yml). |
-| Cloud / remote | Linux runners, managed devices, and remote jobs. | Set up a [remote proxy](https://agent-device.dev/docs/remote-proxy), connect a [device cloud](https://agent-device.dev/docs/device-clouds) (BrowserStack and TestMu AI through `agent-device plugins add`, AWS Device Farm, Limrun), or [contact Callstack](mailto:hello@callstack.com) for team QA. |
+| Cloud / remote | Linux runners, managed devices, and remote jobs. | Set up a [remote proxy](https://agent-device.dev/docs/remote-proxy), connect a [device cloud](https://agent-device.dev/docs/device-clouds) (BrowserStack, AWS Device Farm, and TestMu AI through `agent-device plugins add`, Limrun), or [contact Callstack](mailto:hello@callstack.com) for team QA. |
 
 ## How it works
</file context>
Suggested change
| Cloud / remote | Linux runners, managed devices, and remote jobs. | Set up a [remote proxy](https://agent-device.dev/docs/remote-proxy), connect a [device cloud](https://agent-device.dev/docs/device-clouds) (BrowserStack, AWS Device Farm, and TestMu AI through `agent-device plugins add`, Limrun), or [contact Callstack](mailto:hello@callstack.com) for team QA. |
| Cloud / remote | Linux runners, managed devices, and remote jobs. | Set up a [remote proxy](https://agent-device.dev/docs/remote-proxy), connect a [device cloud](https://agent-device.dev/docs/device-clouds) (BrowserStack, AWS Device Farm, and TestMu AI, each installed with `agent-device plugins add`, or Limrun), or [contact Callstack](mailto:hello@callstack.com) for team QA. |

@apex-sentinel-staging

Copy link
Copy Markdown

Apex Sentinel · ✅ Approve

2 finding(s): 2 minor.

🟡 Minor (2)

Where Finding
packages/provider-aws-device-farm/package.json:36 Published aws-device-farm manifest omits the connection-verification subpath the sibling plugins publish
packages/provider-aws-device-farm/tsdown.config.ts:1 aws-device-farm tsdown.config duplicates build-config instead of reusing it
Not in the reviewers' prompt (1 file)
  • pnpm-lock.yaml: generated or a lockfile, so its lines were left out

Reviewed with this repository's own Sentinel configuration (a30accf38081).

💬 Talking to Sentinel

In a finding's thread, reply with:

Command What it does
@apex-sentinel-staging note <context> Adds context; Sentinel re-checks the cited code
@apex-sentinel-staging pivot <why> Says the finding is wrong; Sentinel re-checks the cited code
@apex-sentinel-staging like Marks the finding as useful
@apex-sentinel-staging dislike <why> Marks it as not useful, so it is not raised again

On this conversation, comment @apex-sentinel-staging review to run a full review once again.

@apex-sentinel-staging apex-sentinel-staging Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apex Sentinel: 2 new findings on 98dfddb. The verdict and the full list are in the summary comment.

Comment on lines +36 to +46
"exports": {
".": {
"import": "./dist/agent-device.mjs"
},
"./agent-device": {
"import": "./dist/agent-device.mjs"
}
},
"access": "public"
},
"license": "MIT",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Minor · Published aws-device-farm manifest omits the connection-verification subpath the sibling plugins publish

publishConfig.exports lists only "." and "./agent-device" and tsdown.config.ts builds only the agent-device entry, so the published tarball will not expose @agent-device/aws-device-farm/connection-verification—the substitution seam that @agent-device/browserstack ships via its dist/connection-verification.mjs. The workspace exports map declares the subpath but the publish map does not, and the gap surfaces at publish, not today (the package is still private).

⚠️ Check this patch before you commit it: the review doubted it would be right as written.

Suggested change
"exports": {
".": {
"import": "./dist/agent-device.mjs"
},
"./agent-device": {
"import": "./dist/agent-device.mjs"
}
},
"access": "public"
},
"license": "MIT",
"publishConfig": {
"exports": {
".": {
"import": "./dist/agent-device.mjs"
},
"./agent-device": {
"import": "./dist/agent-device.mjs"
},
"./connection-verification": {
"import": "./dist/connection-verification.mjs"
},
"./package.json": "./package.json"
},
"access": "public"
},

Suggested change

Add the ./connection-verification (and ./package.json) entries to publishConfig.exports and build a second tsdown entry for src/aws-device-farm-connection-verification.ts, matching packages/provider-browserstack, before the package is published.

Details

Also raised here

Also raised at this location by persona senior-developer: publishConfig drops ./connection-verification subpath from published plugin — packages/provider-aws-device-farm/package.json declares "./connection-verification" in exports (line 25-28) but its publishConfig.exports lists only . and ./agent-device, so the published tarball's exports map has no ./connection-verification. Any host that imports that subpath — the pattern the removed @agent-device/provider-webdriver/connection-verification export existed for, documented there as "Published as one subpath so a host can substitute every bundled verifier together" — gets ERR_PACKAGE_PATH_NOT_EXPORTED on an installed plugin while working against the workspace. Also note check-provider-plugin.mjs cannot catch this: it asserts installed.exports equals {...fixture.additionalExports, '.', './agent-device'} and the new test/package-smoke.mjs names no additionalExports, so the subpath's disappearance is asserted as correct.

persona: senior-developer, javascript

Useful? Reply @apex-sentinel-staging like, or @apex-sentinel-staging dislike <why> if it is not.

Reply @apex-sentinel-staging note ... to add context, or @apex-sentinel-staging pivot ... if it is wrong — I will re-check the cited code.

Comment on lines +1 to +19
import { defineConfig } from 'tsdown';

export default defineConfig({
entry: { 'agent-device': 'src/index.ts' },
outDir: 'dist',
format: 'esm',
platform: 'node',
target: 'es2022',
minify: true,
dts: false,
hash: false,
deps: { alwaysBundle: [/^@agent-device\//] },
inputOptions: {
onLog(level, log, handler) {
if (log.code === 'UNRESOLVED_IMPORT') throw new Error(log.message);
handler(level, log);
},
},
});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Minor · aws-device-farm tsdown.config duplicates build-config instead of reusing it

packages/provider-aws-device-farm/tsdown.config.ts re-implements providerPluginBuildConfig entry by entry, while its sibling packages/provider-browserstack/tsdown.config.ts simply imports it from @agent-device/build-config. It also declares neither the dependency nor the tsconfig project reference that sibling plugins have, so future changes to the shared config silently skip this package's published artifact. Import and reuse providerPluginBuildConfig instead.

Details

persona: javascript, senior-developer

Useful? Reply @apex-sentinel-staging like, or @apex-sentinel-staging dislike <why> if it is not.

Reply @apex-sentinel-staging note ... to add context, or @apex-sentinel-staging pivot ... if it is wrong — I will re-check the cited code.

@thymikee

Copy link
Copy Markdown
Member Author

I reviewed 98dfddb and found two problems to fix before merge. CI is green (21 checks, none failing), but no CI job runs the packed AWS plugin smoke, so green CI does not cover the install-from-store route. No conflicts.

This PR changes the production route that allocates and releases AWS Device Farm sessions (setUpSlot -> plugin.setup -> createAwsDeviceFarmWebDriver -> host.runHostCommand, now bound to the daemon host cwd/env instead of the ambient process env) in aws-device-farm-webdriver.ts. Issue #3310 requires one live Device Farm session with the packed plugin, or the exact blocker named, and the PR body only says live AWS acceptance is unverified. The aws CLI may see a different env or cwd in the daemon (credential chain, AWS_PROFILE, relative config), and the packed plugin may fail to load from the store at daemon start. Every test injects runHostCommand or uses the source-exported workspace package, so users would hit this first. Please record one run with the packed tarball installed through agent-device plugins add <tarball> into a fresh AGENT_DEVICE_HOME. It should show connect aws-device-farm --platform android|ios --aws-project-arn ... --aws-device-arn ... --aws-app-arn ... printing the verification, daemon status --json listing aws-device-farm with source store and state active, open <bundle-id> creating a remote-access session (session ARN in output or logs), one snapshot, and close stopping the session. If no AWS account is available, name that blocker in the PR body instead.

run-host-command.ts re-declares HostCommandResult and RunHostCommand verbatim. This same PR adds those types to packages/plugins/src/host.ts and exports them from agent-device/plugins, which the plugin already imports ProviderPluginHost from. If the host contract changes (for example, to add stderr or a signal), the plugin's copy will drift silently. The rule is that the plugin SDK owns host-capability types and plugins import them. Please delete the local file and import type RunHostCommand from agent-device/plugins (or use ProviderPluginHost['runHostCommand']) at every site that imports the local copy.

The open inline threads on the unqualified install docs still stand: device-clouds.md, plugins.md install, embedded runtimes, daemon stop retry, package README install, changeset file, maxBuffer no-op, stale bundled list and tsdown config copy, plus four lower-priority threads. These do not apply, so please resolve them: script-test skip (the head scans more than base, so nothing is newly dropped), R11 on script tests (it never ran there at base, and root declarations are still enforced), connection-verification export and the same point again (the only importer is a core workspace test, so dropping the dev export is cleaner), ARN format (only the region is parsed), catalog package name (BrowserStack is listed the same way) and README wording (it already scopes the install to all three plugins).

Not blocking, and fine to take or leave: missingAttachedAppRecovery in src/cli/commands/connection-presentation.ts:282 fires for any provider but prints a hard-coded connect aws-device-farm ... command, so the plugin's verify result could carry the recovery command and core could print it generically; the AWS line at src/commands/schema/cli-help.ts:475 could read AWS Device Farm (plugin, install separately) like its siblings; the capabilities object in the plugin's src/index.ts:21 equals HOSTED_WEBDRIVER_CONNECTION except requiresAppAttachment: true, so it can spread that; .changeset/aws-device-farm-plugin.md is gitignored and unused, so it can go; the R11 scan change and the @agent-device/build-config root devDependency removal look unrelated to this extraction and could be a separate PR; and the new SDK HostCommandResult in packages/plugins/src/host.ts:6 shares a name with the core platform-runtime-host type of different shape, so renaming it or exporting only RunHostCommand would avoid confusion.

Before merge, this needs the live AWS run (or a named blocker), the type and tsdown config copies removed, and the install docs corrected.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant