CONSOLE-5197: Collect Playwright test artifacts for Prow CI

[stefanonardo]

Analysis / Root cause:
The Playwright CI test runner (test-playwright-e2e.sh) was not copying test results to $ARTIFACT_DIR, so Prow had no artifacts to display in the job results page. Unlike the Cypress runner which has a trap "copyArtifacts; generateReport" EXIT, the Playwright script had no equivalent. Additionally, using exec to launch Playwright meant any trap would never fire.

Solution description:

• Add an EXIT trap in test-playwright-e2e.sh that copies the test-results/ directory (JUnit XML, traces, screenshots, videos) to $ARTIFACT_DIR/playwright-test-results
• Replace exec "$playwright_bin" test "$@" with a direct invocation so the EXIT trap actually fires

Screenshots / screen recording:
N/A — CI script change, no visual impact.

Test setup:
Run a Playwright E2E job in Prow and verify artifacts appear in the job results page.

Test cases:

• Prow job artifacts include playwright-test-results/ directory
• JUnit XML is present and parsed by Prow UI
• On test failure, traces/screenshots/videos are collected

Browser conformance:
N/A — CI infrastructure change.

Additional info:
Related Prow job with missing artifacts: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_release/78021/rehearse-78021-pull-ci-openshift-console-main-e2e-playwright/2055259618824687616

Summary by CodeRabbit

• Chores
  • Enhanced Playwright E2E test runner to automatically capture and archive test results, improving the artifact collection process for better debugging and test reporting.

[openshift-ci-robot]

@stefanonardo: This pull request references CONSOLE-5197 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Analysis / Root cause:
The Playwright CI test runner (test-playwright-e2e.sh) was not copying test results to $ARTIFACT_DIR, so Prow had no artifacts to display in the job results page. Unlike the Cypress runner which has a trap "copyArtifacts; generateReport" EXIT, the Playwright script had no equivalent. Additionally, using exec to launch Playwright meant any trap would never fire.

Solution description:

• Add an EXIT trap in test-playwright-e2e.sh that copies the test-results/ directory (JUnit XML, traces, screenshots, videos) to $ARTIFACT_DIR/playwright-test-results
• Replace exec "$playwright_bin" test "$@" with a direct invocation so the EXIT trap actually fires

Screenshots / screen recording:
N/A — CI script change, no visual impact.

Test setup:
Run a Playwright E2E job in Prow and verify artifacts appear in the job results page.

Test cases:

• Prow job artifacts include playwright-test-results/ directory
• JUnit XML is present and parsed by Prow UI
• On test failure, traces/screenshots/videos are collected

Browser conformance:
N/A — CI infrastructure change.

Additional info:
Related Prow job with missing artifacts: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_release/78021/rehearse-78021-pull-ci-openshift-console-main-e2e-playwright/2055259618824687616

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: stefanonardo
Once this PR has been reviewed and has the lgtm label, please assign spadgett for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• frontend/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[coderabbitai]

📝 Walkthrough

Walkthrough

The Playwright E2E test runner script gains artifact preservation. A new copyArtifacts function copies the test-results directory into ARTIFACT_DIR/playwright-test-results when both paths exist. The function is registered to run on script exit via trap. The Playwright invocation is changed from exec "$playwright_bin" test "$@" to direct invocation, allowing the EXIT trap to execute instead of replacing the shell process.

Suggested reviewers

• rhamilto
• spadgett
• logonoff

🚥 Pre-merge checks | ✅ 11 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (11 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely summarizes the main change: adding artifact collection for Playwright tests in Prow CI.
Description check	✅ Passed	The description comprehensively covers analysis, solution, test setup, and test cases; all critical sections are well-populated with relevant details.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names	✅ Passed	Custom check is not applicable. PR modifies Playwright E2E test runner infrastructure (shell script), not Ginkgo test definitions. Check targets Ginkgo test titles; PR involves no test title changes.
Test Structure And Quality	✅ Passed	Not applicable. PR modifies bash test runner script only. Custom check assesses Ginkgo test code (It blocks, BeforeEach/AfterEach, timeouts, assertions), which don't apply here.
Microshift Test Compatibility	✅ Passed	This PR modifies a Playwright test runner script (Bash), not Ginkgo e2e tests. The check applies only to new Ginkgo tests, which are not present here.
Single Node Openshift (Sno) Test Compatibility	✅ Passed	Custom check is not applicable. The PR modifies only a Bash test runner script for Playwright E2E tests. No Ginkgo e2e tests are added, so SNO compatibility verification is not required.
Topology-Aware Scheduling Compatibility	✅ Passed	Custom check not applicable. PR modifies a test utility script, not deployment manifests, operator code, or controllers. No scheduling constraints present.
Ote Binary Stdout Contract	✅ Passed	Not applicable. The check targets Go binaries with JSON stdout contracts. This PR modifies a Bash script that invokes Playwright (Node.js), which has separate test result handling.
Ipv6 And Disconnected Network Test Compatibility	✅ Passed	Check not applicable: PR modifies only Bash test runner script, not Ginkgo e2e tests. No Ginkgo patterns present.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@frontend/integration-tests/test-playwright-e2e.sh`:
- Around line 83-90: The EXIT trap currently runs copyArtifacts and any failure
there can overwrite the original Playwright exit code; modify copyArtifacts to
capture the incoming exit status at the start (e.g., exit_code=$?), perform the
artifact copy as a best-effort (suppress or log cp errors rather than returning
non-zero), and at the end restore the original status by exiting with exit_code
so the Playwright test exit code (from the process that triggered the trap) is
preserved; reference the copyArtifacts function, ARTIFACT_DIR and the cp of
test-results invoked by the trap copyArtifacts EXIT.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 4b5f9cf4-20ec-4bcf-a561-bbb12a57e09b

📥 Commits

Reviewing files that changed from the base of the PR and between ee38b13 and b89c32a.

📒 Files selected for processing (1)

• frontend/integration-tests/test-playwright-e2e.sh

📜 Review details

🧰 Additional context used

📓 Path-based instructions (1)

**/*

📄 CodeRabbit inference engine (STYLEGUIDE.md)

Use lowercase dash-separated names for all files (to avoid git issues with case-insensitive file systems)

Files:

• frontend/integration-tests/test-playwright-e2e.sh

🔇 Additional comments (1)

frontend/integration-tests/test-playwright-e2e.sh (1)

101-101: LGTM!

[openshift-ci]

@stefanonardo: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.