Skip to content

ci(e2e): compare auth tests on playwright and endform - #1

Open
ostenbom wants to merge 1 commit into
mainfrom
codex/endform-auth-comparison
Open

ostenbom wants to merge 1 commit into
mainfrom
codex/endform-auth-comparison

Conversation

@ostenbom

@ostenbom ostenbom commented Sep 29, 2026 •

Copy link
Copy Markdown

Description

Compare the existing 29 auth scenarios on Playwright and Endform in CI, using the same production Studio build and GitHub runner type. The auth suite mocks its APIs, so this fork can establish a baseline without Sanity staging credentials or a Vercel deployment.

What to review

The auth job runs a Playwright/Endform matrix, with browser installation only for Playwright and verified GitHub OIDC authentication for Endform organization 2G1ZCj7X. Both runners retain failure traces and keep the existing one retry. Test files and documentation are unchanged.

Testing

Both CI jobs passed on commit fcd88e5:

Runner Result Test step Whole job
Playwright (2 workers) 28 passed, 1 passed on retry 75s 3m 00s
Endform 29 passed 50s 2m 05s

Endform CI dashboard. Endform's own suite timer reported 42.8s and 8.5 billable runner minutes. GitHub step durations above include CLI startup and server setup. Playwright also spent 23s installing Chromium in this first run.

The Playwright retry was cookieAuth.spec.ts:52 (login after logout syncs across tabs): the first attempt timed out clicking a hidden Sign out item at line 70. It passed on retry; the test was left unchanged. These are single-run measurements, and the retry affects the timing comparison.

Locally, both runners passed all 29 tests with matching trace settings: Playwright took 13.92s total command time and Endform took 35.52s while forwarding to the Mac's preview server. Endform local run.

Formatting, lint, dependency checks, export tests and workflow security audit passed in CI. The main Studio/embedded workflows are outside this comparison and still need the fork's Sanity/Vercel configuration.

Notes for release

N/A – Internal only.


@ostenbom ostenbom added the 🤖 bot Pull requests created by AI agents label Sep 29, 2026
@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

📦 Bundle Stats — sanity

Compared against main (89aa947a) · v6.16.0 (npm)

sanity

Metric Value vs main (89aa947) vs v6.16.0
Internal (raw) 4.49 MB - +56.0 KB, +1.2%
Internal (gzip) 1.12 MB - +19.5 KB, +1.7%
Bundled (raw) 13.34 MB - +73.1 KB, +0.5%
Bundled (gzip) 3.16 MB - +22.9 KB, +0.7%
Import time 1.75s +10ms, +0.5% +22ms, +1.3%

bin:sanity

Metric Value vs main (89aa947) vs v6.16.0
Internal (raw) 7.1 KB - -
Internal (gzip) 2.9 KB - -
Bundled (raw) 7.1 KB - -
Bundled (gzip) 2.8 KB - -
Import time 5ms -0ms, -0.4% +0ms, +4.0%

🗺️ View treemap · Artifacts

Details
  • Import time regressions over 10% are flagged with ⚠️
  • Sizes shown as raw / gzip 🗜️. Internal bytes = own code only. Total bytes = with all dependencies. Import time = Node.js cold-start median.

@ostenbom
ostenbom marked this pull request as ready for review September 29, 2026 13:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🤖 bot Pull requests created by AI agents

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant