Skip to content

Run the conformance suite at release-v5.3.4 in CI - #361

Merged
cicnavi merged 4 commits into
simplesamlphp:masterfrom
ioigoume:ci/use-latest-conformance-suite
Nov 4, 2026
Merged

cicnavi merged 4 commits into
simplesamlphp:masterfrom
ioigoume:ci/use-latest-conformance-suite

Conversation

@ioigoume

@ioigoume ioigoume commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

No description provided.

@codecov

codecov Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 54.01%. Comparing base (869eea6) to head (83a5f8c).
⚠️ Report is 4 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff              @@
##             master     #361      +/-   ##
============================================
+ Coverage     53.99%   54.01%   +0.01%     
  Complexity     1512     1512              
============================================
  Files           141      141              
  Lines          6078     6080       +2     
============================================
+ Hits           3282     3284       +2     
  Misses         2796     2796              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ioigoume
ioigoume force-pushed the ci/use-latest-conformance-suite branch from f059093 to 1f9ef82 Compare October 8, 2026 16:30
add failure diagnostic logging
…for back-channel logout client endpoints in conformance testing fixtures.
@ioigoume
ioigoume force-pushed the ci/use-latest-conformance-suite branch from cc85c32 to a669d3d Compare October 8, 2026 16:54
Following the suite's latest release at run time lets a new release turn
CI red with no change here, and makes the same commit pass or fail
depending on when it runs. The suite had eight releases between August
and October 2026. Pin it instead, as wip-version-7 does, so moving to a
new release is a deliberate commit.

This removes the step resolving the latest release and the jq install.
The runner image already has jq.

The failure dump now uses the step from wip-version-7, which shows the
last 2000 lines of the OP's log rather than 100 lines per container. The
/api/log part is gone: it printed nothing in a failed run.

The local instructions in docs/5-oidc-conformance.md name the same
release.

Assisted-by: Claude Code (claude-opus-5-5, implementation)
@cicnavi cicnavi changed the title fetch the latest OpenID Conformance Suite release Run the conformance suite at release-v5.3.4 in CI Nov 4, 2026
@cicnavi
cicnavi changed the base branch from version-6 to master November 4, 2026 09:54
@cicnavi

cicnavi commented Nov 4, 2026

Copy link
Copy Markdown
Collaborator

Thanks for this! I pushed a commit on top: the suite is now pinned at release-v5.3.4 instead of following latest, so a new suite release can't turn CI red with no change on our side. The version moves in deliberate commits, as on wip-version-7. Your host rename stays as it was, and the failure dump now uses the same OP log step as wip-version-7.

@cicnavi
cicnavi merged commit 83d0a25 into simplesamlphp:master Nov 4, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants