Skip to content

ci: write a swift test summary to the job summary - #598

Open
kieran-osgood-shopify wants to merge 1 commit into
swift-artifact-uploadfrom
kieran-osgood/swift-artifact-upload/xcresult-step-summary
Open

ci: write a swift test summary to the job summary#598
kieran-osgood-shopify wants to merge 1 commit into
swift-artifact-uploadfrom
kieran-osgood/swift-artifact-upload/xcresult-step-summary

Conversation

@kieran-osgood-shopify

@kieran-osgood-shopify kieran-osgood-shopify commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Stack Context

Stacked on #595, which writes an .xcresult bundle on CI and uploads it when a Swift job fails.

This PR renders the test results directly on the run page

What?

Read it at Checks → CI → Summary, under the Run Package Tests job. No log expansion, no artifact download.
https://github.com/Shopify/checkout-kit/actions/runs/31108425595?pr=596

Why?

xcode_run already annotates failures at file and line through xcbeautify. That covers a single failure well. It does not scale:

  • GitHub caps annotations at 10 errors per step and 50 per job.
  • Every Swift test runs in one step, Run Tests.
  • Past 10 failures GitHub drops the rest, picks an arbitrary subset, and shows no sign that it truncated.

The job summary has no such cap, so a wide breakage stays legible.

Why package tests only?

xcresultparser only supports test output, not build output

Notes for review

Decision Reason
--output-format md, not github The github format emits unbalanced LaTeX ($\textcolor{... never closes) and GitHub renders a math error.
sed 's/\\"/"/g' The tool escapes its own quotes, which makes style=\"color:red\" malformed HTML.
cat -s The raw output carries long runs of blank lines.
working-directory: ${{ github.workspace }} defaults.run.working-directory is platforms/swift, but .xcresults/ sits at the repository root.
shopt -s nullglob An empty .xcresults/ writes nothing instead of a literal glob.

Verified

The step body ran against the failed bundle from CI run 31104402707
https://github.com/Shopify/checkout-kit/actions/runs/31108425595?pr=596

## ShopifyCheckoutKit-Package

Errors: 0; Warnings: 2; Tests: 883; Failed: 1; Skipped: 0; Execution time = 313.939s

### LockedValueTests (1s)
* 🔴 testGetReturnsInitialValue() (1s)
  * LockedValueTests.swift:8: XCTAssertEqual failed: ("initial") is not equal to (...)
image

End-to-end CI proof runs through the stacked verification PR #596.

🤖 Generated with Claude Code

Copy link
Copy Markdown
Contributor Author

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@github-actions github-actions Bot added the #gsd:50662 Rebase Checkout Kit on UCP label Aug 6, 2026
@kieran-osgood-shopify
kieran-osgood-shopify marked this pull request as ready for review August 7, 2026 10:39
@kieran-osgood-shopify
kieran-osgood-shopify requested a review from a team as a code owner August 7, 2026 10:39
@bitrise

bitrise Bot commented Aug 7, 2026

Copy link
Copy Markdown

Install this build

Open Tophat, select your target device, then click Install. Links open on the Mac running Tophat.

SDK Install
Swift Install with Tophat

Checkout Kit E2E results

Status Suite Target Platform OS version tag Device
tests/shared/launch-smoke.yaml swift ios latest iPhone 15
iOS 27 Beta

@kieran-osgood-shopify
kieran-osgood-shopify changed the base branch from swift-artifact-upload to graphite-base/598 August 7, 2026 14:18
@kieran-osgood-shopify
kieran-osgood-shopify force-pushed the kieran-osgood/swift-artifact-upload/xcresult-step-summary branch from 6f54314 to 8dfab78 Compare August 7, 2026 14:18
@kieran-osgood-shopify
kieran-osgood-shopify changed the base branch from graphite-base/598 to swift-artifact-upload August 7, 2026 14:19
@kieran-osgood-shopify
kieran-osgood-shopify force-pushed the kieran-osgood/swift-artifact-upload/xcresult-step-summary branch from 8dfab78 to c79264c Compare August 7, 2026 14:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

#gsd:50662 Rebase Checkout Kit on UCP

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant