Repository navigation
Conversation
ostenbom
added
ready-for-ci
🔎 OpenObserve Code ReviewCaution ⛔ Not reviewedThe review API key is not set. OpenObserve Code Review did not run for this PR — this is a CI misconfiguration, not a skip. Please confirm the review secrets are provisioned. |
🔎 OpenObserve Code ReviewCaution ⛔ Not reviewedThe review API key is not set. OpenObserve Code Review did not run for this PR — this is a CI misconfiguration, not a skip. Please confirm the review secrets are provisioned. |
🔎 OpenObserve Code ReviewCaution ⛔ Not reviewedThe review API key is not set. OpenObserve Code Review did not run for this PR — this is a CI misconfiguration, not a skip. Please confirm the review secrets are provisioned. |
🔎 OpenObserve Code ReviewCaution ⛔ Not reviewedThe review API key is not set. OpenObserve Code Review did not run for this PR — this is a CI misconfiguration, not a skip. Please confirm the review secrets are provisioned. |
ostenbom
changed the title
🔎 OpenObserve Code ReviewCaution ⛔ Not reviewedThe review API key is not set. OpenObserve Code Review did not run for this PR — this is a CI misconfiguration, not a skip. Please confirm the review secrets are provisioned. |
🔎 OpenObserve Code ReviewCaution ⛔ Not reviewedThe review API key is not set. OpenObserve Code Review did not run for this PR — this is a CI misconfiguration, not a skip. Please confirm the review secrets are provisioned. |
🔎 OpenObserve Code ReviewCaution ⛔ Not reviewedThe review API key is not set. OpenObserve Code Review did not run for this PR — this is a CI misconfiguration, not a skip. Please confirm the review secrets are provisioned. |
🔎 OpenObserve Code ReviewCaution ⛔ Not reviewedThe review API key is not set. OpenObserve Code Review did not run for this PR — this is a CI misconfiguration, not a skip. Please confirm the review secrets are provisioned. |
🔎 OpenObserve Code ReviewCaution ⛔ Not reviewedThe review API key is not set. OpenObserve Code Review did not run for this PR — this is a CI misconfiguration, not a skip. Please confirm the review secrets are provisioned. |
1 participant
Run the selected product browser suite in one Endform test job against one CI-local OpenObserve process. The existing manifest still selects all 2,062 Chromium tests in 231 files; the old 29-job test matrix is removed. Build artifacts and existing changed-path selection remain in use.
The main invocation distributes 2,057 tests using Endform's native
concurrentTestLimits: forty total remote executions; one RUM file at a time (including the token-reset file); one SLO file at a time (protecting the backfill scheduler); one preferences file; two Reports files; three Alerts files; and at most three tests carrying the existing@pipelinestag. These constraints overlap within the same distributed run, so independent domains can continue while a constrained resource is busy. Actual serial describe groups stay on one worker; independent cases in those files remain distributed. Reports/RUM/Alerts files requiring shared module state stay together. Chromium browser options and test assertions are unchanged; project names identify resource pools.The same backend uses a 240-hour ingestion window and a seven-day SLO backfill chunk. The final five tests require a process-start quick-mode flag, so the job restarts the backend once with fresh data and the original quick-mode profile, then runs those five. Both invocations execute even if earlier tests fail, and both reports merge into one artifact. Backend CPU/RSS samples are attached. There is no test matrix or broadly sequential second batch.
This PR is stacked on the native baseline PR, branch
codex/playwright-baseline, commit6a7114856cb7de1cb532602a56b4ceba062de2cb. Keep both PRs unmerged for the comparison; retarget this PR after the baseline is eventually merged.Endform uses GitHub OIDC without an API-key secret. Its port proxy reaches the backend, mail sink, and CI-hosted callback receiver, while worker-hosted RUM fixture servers retain their own ports. Runtime assets are explicitly transferred; the locked RUM npm fixture builds locally before upload. Authentication and ingestion reuse the existing application endpoints and seed helpers without installing browsers. Named helper exports work in Endform's module loader. Test bodies and assertions are unchanged.
Validation: native and consolidated test listings match exactly, with zero missing or additional cases; 22 existing selection contract tests pass; JavaScript syntax checks, workflow YAML parsing, and
cargo fmt --allpass. The required workspace clippy command passed earlier on the unchanged Rust source. Targeted local runs verified serial Reports state, named helper exports, the npm RUM fixture, and real alert webhook delivery. A local five-test quick-mode run passed after validating remote packaging of the serial-file metadata. Local resource-limit checks passed: CDN dataflow plus token reset (6 passes, 1 existing skip), and both SLO measurement files (25 passes). The consolidated CI attempt completed with a failed test job; exact main-phase completeness is being checked. Its final quick-mode invocation passed all five tests. The next full CI run targets commit0361fc5d98f5c9c9818f46fe8ecfe92eb1861e23, including the narrower serial grouping and stable pipeline header target.The native first full attempt completed in 57m44s, including a 35m43s test-stage critical path: 1,906 passed, 14 flaky passes, 139 skipped, and three Streams failures. Its existing automatic rerun recovered those failures. Baseline first attempt.
An earlier Endform matrix attempt completed with 1,857 passed, 12 flaky passes, 142 skipped, and 51 failures, in 63m00s. That exposed integration problems and is not evidence of successful acceleration. The consolidated architecture and its timing comparison require the pending full run. Both use the same application/test revision; the backend consolidation, concurrency, serial execution granularity, and SLO server settings are explicit resource/scheduling differences.
The first consolidated attempt at ten concurrent executions hit its two-hour job limit with 785 cases unfinished and produced no complete report. It is excluded from the timing comparison. Backend samples peaked at 3,341 MiB RSS and 241% process-average CPU (four available cores). The next full run uses forty total remote executions, retains every resource-specific limit, and allows three hours for completion. The job timeout is a guard, not a measured suite runtime.
Interactive pipeline investigation identified a Playwright runtime difference: Endform runs Playwright 1.60.0 / Chromium 148, whereas the baseline used Playwright 1.55.1 / Chromium 140. An unscoped Name table header is exposed as a columnheader in the newer runtime and as a cell in 1.55.1. The integration now selects its existing
data-test="o2-table-th-name"target in the pipeline page helper. A native probe against the real application confirmed that the original role selector and this stable selector resolve to exactly the same element. Test bodies and assertions remain unchanged. The unchanged pipeline core case passed on Endform, one attempt. The runtime/browser difference will remain explicit in the timing comparison.Run 36886959428 was cancelled before completion after finding and fixing a quick-mode restart readiness bug (native Fetch uses the
okproperty). A native Fetch Response probe now verifies restart readiness. That incomplete run is excluded from the comparison. The corrected full run still covers main + quick mode and retains forty total executions plus all resource-specific limits. No active Endform quarantine rules apply to this suite.A committed local refinement is ready for the next CI run: only actual serial describe groups are co-located, while independent tests in the same file remain distributed.
dashboard-multi-sql.spec.jshas six serial cases and forty independent cases; placing all forty-six on one worker was too coarse (native summed case time is about 24 minutes, beyond Endform’s 15-minute file limit). The partition selects exactly the same 2,062 cases with no additions, omissions, or duplicates. The full mixed multi-SQL file passed locally: 46 passes, one attempt each, 4.9 minutes. This verifies the serial grouping refinement and is not a comparable CI timing sample. The prior full CI attempt used the earlier whole-file grouping. The refinement is now pushed and its full CI run is pending.