Skip to content

test(testing): run the e2e suite on two workers by default - #1892

Merged
joshunrau merged 1 commit into
DouglasNeuroInformatics:mainfrom
joshunrau:test/e2e-default-workers
Nov 9, 2026
Merged

joshunrau merged 1 commit into
DouglasNeuroInformatics:mainfrom
joshunrau:test/e2e-default-workers

Conversation

@joshunrau

Copy link
Copy Markdown
Collaborator

Problem

Outside CI, testing/playwright.config.ts left workers undefined, so Playwright used half the cores. That caused two problems:

  • The full suite fails locally at five workers. The API allows 50 logins a minute. From mail.spec.ts:257 onwards tests fail with Login failed … (429): ThrottlerException: Too Many Requests, and each failure restarts a worker that logs in again, so the failures cascade through password-reset, permission-sets and remote-assignments. Three full runs failed at the same test.
  • It takes most of a machine. Five browser workers alongside the api, gateway, web and playground dev servers leave little for other work running in parallel.

Change

  • The default is now two workers, and still one when CI is set.
  • E2E_WORKERS in the root .env overrides both. testing/src/support/env.ts reads it and throws while the config loads on anything but a positive integer.
  • .env.template carries the key blank, and testing/AGENTS.md §Running documents the default, the reason and the override. testing-strategy.md points there.

One thing to know about the override: the test:* scripts load .env through env-cmd, whose values win over the shell. So once .env carries the key, even blank, a shell export of E2E_WORKERS is ignored. For a single run, --workers=<n> after the script still works.

Verification

  • pnpm test:e2e from the root, with no flags, passes in about 6 minutes. Before this change the same command failed on this machine in three of three runs.
  • The resolved workers value, checked by loading the config under each combination: 2 with the key absent, 1 with CI set, 4 with E2E_WORKERS=4 (also with CI set), 2 with a blank key and a shell export of 4, and a thrown error for abc and 0.
  • pnpm lint passes and pnpm test:coverage is at 100%.

Exceptions:

  • No unit test and no new e2e test. testing/ has no unit tier, and the change is the suite's own configuration; the suite passing at the default is its test.
  • CI behaviour is unchanged, so CI does not exercise the new default.

Co-Authored-By: Claude Fable 5.2 noreply@anthropic.com

Outside CI the suite ran on Playwright's default of half the cores. At five
workers it logs in faster than the API's login throttle allows (50 a minute):
from mail.spec.ts onwards tests fail with "Login failed (429)", and each
failure restarts a worker that logs in again. That many workers also leave
the machine little for anything else running.

The default is now two workers, and still one on CI. E2E_WORKERS in .env
overrides both, and anything but a positive integer fails while the config
loads.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@joshunrau joshunrau closed this Nov 9, 2026
@joshunrau joshunrau reopened this Nov 9, 2026
@joshunrau
joshunrau merged commit 32b22c1 into DouglasNeuroInformatics:main Nov 9, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant