Skip to content

test: establish the native Playwright baseline on portable runners - #1

Open
ostenbom wants to merge 1 commit into
mainfrom
codex/playwright-baseline
Open

ostenbom wants to merge 1 commit into
mainfrom
codex/playwright-baseline

Conversation

@ostenbom

@ostenbom ostenbom commented Oct 1, 2026 •

Copy link
Copy Markdown

The existing product browser suite cannot run in this fork because its build, test, and report jobs require upstream private runners. Use GitHub-hosted Ubuntu runners while preserving the native Playwright runner, Chromium installation, full manifest selection, per-shard application instances, fixtures, retries, and assertions.

The server uses a debug Cargo build as required by the repository instructions, with debug symbols and incremental compilation disabled to fit hosted runners. Both benchmark configurations will use this same build setup. The build timeout is increased for the smaller runner; it is not a measured duration. Optional upstream Kinora uploads require explicit opt-in outside upstream. No product or test source changes.

Validation: all 22 matrix-selection contract tests pass locally; required cargo formatting and full workspace/all-targets clippy passed. Full native Playwright attempt 1 completed with failures: 1,906 passed, 14 flaky passes, 139 skipped, and 3 failed (2,062 total), across all 29 groups. The failures persisted through four attempts each: two schema-drawer visibility checks in Streams/schema.spec.js (lines 156 and 229) and the streaming join-query refresh-button timeout in Streams/streaming.spec.js (line 58). Assertions and tests are unchanged. The existing auto-rerun workflow subsequently retried the failed group; that partial rerun is separate from the full benchmark sample. Attempt 2 subsequently passed, recovering all 29 groups without source changes. The initial three failures are therefore transient under this setup, despite exhausting the original in-process retries. This PR remains open as the reproducible baseline for a separate stacked Endform migration; neither PR will be merged.

Evidence: baseline commit 6a7114856cb7de1cb532602a56b4ceba062de2cb; completed attempt 1, merged-report-1 artifact. Whole workflow: 57m44s; test-stage wall-clock critical path: 35m43s, including later groups waiting for GitHub runners. Build job: 18m18s (binary/frontend build step: 12m25s). Queue times for test jobs: 2s–27m55s. One full completed sample; no stable-performance claim.

Passing recovery attempt 2. This reruns failed jobs only and is not another full-suite timing sample.

@ostenbom ostenbom added ready-for-ci Run CI for this PR e2e-full Run the full existing UI matrix labels Oct 1, 2026
@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown

🔎 OpenObserve Code Review

Caution

⛔ Not reviewed

The review API key is not set. OpenObserve Code Review did not run for this PR — this is a CI misconfiguration, not a skip. Please confirm the review secrets are provisioned.

1 similar comment
@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown

🔎 OpenObserve Code Review

Caution

⛔ Not reviewed

The review API key is not set. OpenObserve Code Review did not run for this PR — this is a CI misconfiguration, not a skip. Please confirm the review secrets are provisioned.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

e2e-full Run the full existing UI matrix ready-for-ci Run CI for this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant