Skip to content

ci(frontend): upload Playwright artifacts when E2E tests fail#7774

Merged
talissoncosta merged 1 commit into
mainfrom
ci/e2e-upload-failure-artifacts
Jun 15, 2026
Merged

ci(frontend): upload Playwright artifacts when E2E tests fail#7774
talissoncosta merged 1 commit into
mainfrom
ci/e2e-upload-failure-artifacts

Conversation

@talissoncosta

Copy link
Copy Markdown
Contributor

Thanks for submitting a PR! Please check the boxes below:

  • I have read the Contributing Guide.
  • I have added information to docs/ if required so people know about the feature.
  • I have filled in the "Changes" section below.
  • I have filled in the "How did you test this code" section below.

Changes

When the E2E gate fails on the production deploy (or staging/ECS runs), the Playwright traces, screenshots and DOM snapshots die with the runner, so failures like the ones blocking today's deploy can only be debugged by reproducing locally.

This uploads e2e/test-results and the Playwright report as a run artifact when the E2E step fails:

  • 7-day retention, since traces capture network payloads including the (throwaway, reseeded-per-run) e2e users' auth tokens.
  • Artifact name includes github.run_attempt to avoid the immutable-name collision in upload-artifact@v4 when a failed job is re-run.
  • No-op on green runs (if: failure()), and if-no-files-found: ignore covers failures before Playwright starts.

How did you test this code?

Validated the action YAML parses. The step only runs on E2E failure in CI; the paths match Playwright's configured outputDir (e2e/test-results) and report directory, verified against a local failing run.

E2E failures on the production deploy gate currently leave nothing to
debug with - traces, screenshots and DOM snapshots die with the runner.
Upload e2e/test-results and the Playwright report as a run artifact on
failure.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@talissoncosta talissoncosta requested a review from a team as a code owner June 12, 2026 17:54
@talissoncosta talissoncosta requested review from emyller and removed request for a team June 12, 2026 17:54
@vercel

vercel Bot commented Jun 12, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

3 Skipped Deployments
Project Deployment Actions Updated (UTC)
docs Ignored Ignored Jun 12, 2026 5:54pm
flagsmith-frontend-preview Ignored Ignored Jun 12, 2026 5:54pm
flagsmith-frontend-staging Ignored Ignored Jun 12, 2026 5:54pm

Request Review

@github-actions

github-actions Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Docker builds report

Image Build Status Security report
ghcr.io/flagsmith/flagsmith-api-test:pr-7774 Finished ✅ Skipped
ghcr.io/flagsmith/flagsmith-e2e:pr-7774 Finished ✅ Skipped
ghcr.io/flagsmith/flagsmith-frontend:pr-7774 Finished ✅ Results
ghcr.io/flagsmith/flagsmith-api:pr-7774 Finished ✅ Results
ghcr.io/flagsmith/flagsmith:pr-7774 Finished ✅ Results
ghcr.io/flagsmith/flagsmith-private-cloud:pr-7774 Finished ✅ Results

@github-actions github-actions Bot added the ci-cd Build, test and deployment related label Jun 12, 2026
@codecov

codecov Bot commented Jun 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.57%. Comparing base (56f2ad6) to head (322fe31).
⚠️ Report is 6 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7774      +/-   ##
==========================================
+ Coverage   98.56%   98.57%   +0.01%     
==========================================
  Files        1459     1460       +1     
  Lines       56134    57082     +948     
==========================================
+ Hits        55327    56271     +944     
- Misses        807      811       +4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Playwright Test Results (oss - depot-ubuntu-latest-arm-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  42.2 seconds
commit  322fe31
info  🔄 Run: #17471 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  40.5 seconds
commit  322fe31
info  🔄 Run: #17471 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  1 minute, 13 seconds
commit  322fe31
info  🔄 Run: #17471 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-16)

passed  2 passed

Details

stats  2 tests across 2 suites
duration  37.2 seconds
commit  322fe31
info  🔄 Run: #17471 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-arm-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  38.6 seconds
commit  322fe31
info  🔄 Run: #17501 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  40.8 seconds
commit  322fe31
info  🔄 Run: #17501 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-16)

passed  3 passed

Details

stats  3 tests across 3 suites
duration  33 seconds
commit  322fe31
info  🔄 Run: #17501 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  41.8 seconds
commit  322fe31
info  🔄 Run: #17501 (attempt 1)

@github-actions

github-actions Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Visual Regression

19 screenshots compared. See report for details.
View full report

@emyller emyller left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I'm finally happier since #6023 was closed.

@talissoncosta talissoncosta reopened this Jun 15, 2026
@github-actions github-actions Bot added ci-cd Build, test and deployment related and removed ci-cd Build, test and deployment related labels Jun 15, 2026
@talissoncosta talissoncosta merged commit 9fd0eb3 into main Jun 15, 2026
58 checks passed
@talissoncosta talissoncosta deleted the ci/e2e-upload-failure-artifacts branch June 15, 2026 12:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-cd Build, test and deployment related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants