Repository navigation
Conversation
Coverage report
This PR does not seem to contain any modification to coverable code. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Run Baserow's existing product Playwright suite on Endform with the same test selection and application services. One Endform submission replaces four native shards and the separate report merger.
Depends on #1 (
codex/endform-baseline), which remains open againstdevelop. This PR targets that baseline branch; retarget todevelopafter the baseline is reviewed/merged. Both branches use Playwright 1.63.0. Test bodies, assertions, fixtures, product source, serial groups, 30-second timeout and three CI retries are unchanged.chrome, excluding@slow; Firefox remains configured but is not selected by native CI either.id-token: write; organization2G1ZCj7Xis connected to GitHub. No API key created or stored. Endform CLI resolves to 0.81.3.BASE_URLkeeps the existing frontend fallback and explicit builder preview override. Failure traces are retained; existing retry policy preserved.playwright.endform.config.tsreuses the existing Playwright config so the monorepo import stays inside the upload root.additionalFilestransfers only the runtime-read upload image and the frontend protocol module. A private E2E-only root Yarn workspace hoists one Playwright instance and retains the existing dependency resolutions. Original E2E package and lock match the baseline.Validation: version-aligned native CI at
13fb6ee328d3159d93d97383b5962b0365eac8d7completed green: 311 first-attempt passes + 2 retry passes + 3 existing skips. Test-step wall span 980s (staggered runner queues); longest shard 377s; summed shard runner time 1194s; whole workflow 1785s. The original 1.48 native run also passed, whole workflow 1779s and test-stage wall 464s. Endform discovery hung on 1.48; 1.63 preserves all 316 selected tests.The first completed full Endform CI run at
33920bddaa370e4005f795515d7f784f1af4c4a9completed failed: 286 first-attempt passes + 10 retry passes + 17 failures + 3 skips (full dashboard). Suite duration 2122.863s; GitHub test step 2164s includes an initial single-test diagnostic; whole workflow 2474s. Seven failures were missing frontend module imports; ten were runtime assertions/timeouts. No quarantine or assertion changes; no successful speedup claim.Adding
../web-frontend/...to additionalFiles caused upload failure before test execution. The root layout now loads that exact module remotely: local transfer preflight reaches the test's API setup; it fails there because the local app is unavailable. It is not a suite benchmark. Discovery remains 316 tests / 41 files, frozen workspace install, YAML parsing and diff checks pass.Final clean head
aed4edfeaf467007c2b21eb00a5274a913c9e67ccompleted in CI 36865776622, Endform dashboard: 292 first-attempt passes + 10 retry passes + 11 failures + 3 existing skips = 316 selected. All 313 non-skipped tests were executed. The CLI recorded Endform 0.81.3. The diagnostic is removed. Every other CI Pipeline job passed; E2E and its aggregate status check failed. No file-upload or missing-module errors remain; six replay tests pass and the seventh reaches a runtime assertion.*The first completed Endform test step includes the initial login diagnostic. Its full-suite execution duration alone was 2122.863s. Final full-suite execution duration is 2273.952s; the GitHub test step also includes CLI startup, transfer, result download and report generation. Native's longest shard is 377s, and its summed shard test-runner time is 1194s; neither is the 980s wall-clock test span. Queue/install/build/startup rows overlap across jobs and should not be added to reconstruct workflow duration. Builds were warm-cache in these three runs.
Using measured GitHub test-step wall spans: first-completed Endform speedup = 980/2164 = 0.453×, time reduction = 1−2164/980 = −120.8%. Whole-workflow speedup = 1785/2474 = 0.722×, reduction = −38.6%. For final Endform, test-stage speedup = 980/2290 = 0.428×, reduction = −133.7%; whole-workflow speedup = 1785/2470 = 0.723×, reduction = −38.4%. These are slower, failing full-suite runs, not successful speedup claims.
One aligned native sample and one sample of each Endform configuration; no medians or stability claim. A second native run used 1.48.0, and the cancelled initial Endform run is excluded from completed-run ratios. Further repeated full pipelines are expensive. Runner queues, trace policy (native on-first-retry, Endform retain-on-failure), and application isolation/resources differ. Observed GitHub hosts: Ubuntu 24.04 image 20260927.320.1, 4 CPUs/~15.6 GiB each; Node 24.21.0, Yarn 1.22.22. Native system Google Chrome 154.0.8037.57; Endform managed browser/worker version, CPU and memory are unverified. Native has four hosts/stacks (12 Gunicorn workers + 4 Celery workers total), Endform one host/stack (3 + 1), with at most four simultaneous managed tests. Same application image contents, dump, migrations, services and test bodies; this is not a resource-matched stable benchmark. No dollar cost is claimed; final CLI reports 160.2 billable runner minutes.
Remaining failures (each exhausted three retries):
automation/automationNode.spec.ts:103builder/builderPreview.spec.ts:93dashboard/dashboardWidgetGrid.spec.ts:307dashboard/dashboardWidgetGrid.spec.ts:609database/button_field.spec.ts:1222database/button_field.spec.ts:1288database/button_field.spec.ts:1311database/button_field.spec.ts:1328database/button_field.spec.ts:1631database/grid/grid_row_crud.spec.ts:1205database/realtimeReplay.spec.ts:835These passed in the aligned native run. They are unresolved Endform-only observations; no claim that they are pre-existing product failures. Assertions, bodies, grouping, timeouts and retry policy remain unchanged. No quarantine rules were created. Permanent skips remain image upload/download (
database/file_field.spec.ts:9), template regression baserow#1606 (enterprise/regressions.spec.ts:18), and builder template (builder/builderTemplate.spec.ts:5).The demonstrated operational change is one suite submission with no matrix/report-merger orchestration. Endform traces helped locate the Axios setup stall, and error stacks identified the missing cross-package import. The demonstrated outcome does not support a faster-feedback sales claim. Further failure investigation and comparable resource/isolation trials are needed before presenting this as a passing Endform migration.
Local full execution remains blocked:
just e2e buildbuilt the backend but Nuxt exited 137 / cannot allocate memory under Docker's ~8 GB memory limit. No local suite pass is claimed.Separate inherited Database Team PR Automation fails because
DATABASE_PROJECT_WORKFLOW_TOKENis absent (Input required and not supplied: github-token). That unrelated project-board workflow remains unchanged.