Skip to content

Pin the conformance CLI to the alpha dist-tag - #497

Merged
koic merged 1 commit into
modelcontextprotocol:mainfrom
koic:pin_conformance_cli_to_the_alpha_dist_tag
Aug 8, 2026
Merged

Pin the conformance CLI to the alpha dist-tag#497
koic merged 1 commit into
modelcontextprotocol:mainfrom
koic:pin_conformance_cli_to_the_alpha_dist_tag

Conversation

@koic

@koic koic commented Aug 8, 2026

Copy link
Copy Markdown
Member

Motivation and Context

The npm latest dist-tag of @modelcontextprotocol/conformance points at 0.1.16, the old 0.1 line, while the 2026-07-28 requirement revisions and the frozen-requirements framework live in the 0.2.0 alpha line (alpha currently resolves 0.2.0-alpha.11). The runners invoked the bare package name, so rake conformance and the conformance CI job resolve whatever latest points at: a fresh machine silently downgrades to the old suite while a warm npx cache may keep serving a newer one, and the suite actually in use depends on the environment. Pinning the alpha dist-tag puts every run on the line the frozen conformance requirements are anchored to (0.2.0-alpha.10, published 2026-07-27).

Running the real alpha suite surfaced the actual client-leg state, so the expected-failures baseline is updated to match: initialize now passes (the harness accepts 2026-07-28, fulfilling the TODO on the old entry), and the scenarios pending client-side work are recorded with their reasons: the 2026-07-28 stateless lifecycle and the SEP-2322 MRTR driver, SEP-2243 custom-header mirroring, client-side JSON Schema preservation, the DPoP and WIF authorization extensions, and one auth/authorization-server-migration check added to the suite after the alpha.10 anchor.

The manual npx examples in conformance/README.md carry the same pin so local reproduction runs the same suite as the rake task.

How Has This Been Tested?

npx --yes @modelcontextprotocol/conformance@alpha --version resolves 0.2.0-alpha.11, and bundle exec rake conformance passes the baseline check on both legs: the server leg reports 73 passed with 0 failed and no expected failures, and the client leg fails exactly the scenarios the baseline records.

Breaking Changes

None. The change is confined to the conformance tooling; the shipped gem is untouched.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update

Checklist

  • I have read the MCP Documentation
  • My code follows the repository's style guidelines
  • New and existing tests pass locally
  • I have added appropriate error handling
  • I have added or updated documentation as needed

## Motivation and Context

The npm `latest` dist-tag of `@modelcontextprotocol/conformance` points at 0.1.16, the old 0.1 line,
while the 2026-07-28 requirement revisions and the frozen-requirements framework live in the 0.2.0 alpha line
(`alpha` currently resolves 0.2.0-alpha.11). The runners invoked the bare package name, so `rake conformance`
and the conformance CI job resolve whatever `latest` points at: a fresh machine silently downgrades to
the old suite while a warm npx cache may keep serving a newer one, and the suite actually in use depends
on the environment. Pinning the `alpha` dist-tag puts every run on the line the frozen conformance requirements
are anchored to (0.2.0-alpha.10, published 2026-07-27).

Running the real alpha suite surfaced the actual client-leg state, so the expected-failures baseline is
updated to match: `initialize` now passes (the harness accepts 2026-07-28, fulfilling the TODO on the old entry),
and the scenarios pending client-side work are recorded with their reasons: the 2026-07-28 stateless lifecycle
and the SEP-2322 MRTR driver, SEP-2243 custom-header mirroring, client-side JSON Schema preservation,
the DPoP and WIF authorization extensions, and one `auth/authorization-server-migration` check added to
the suite after the alpha.10 anchor.

The manual `npx` examples in `conformance/README.md` carry the same pin so local reproduction runs
the same suite as the rake task.

## How Has This Been Tested?

`npx --yes @modelcontextprotocol/conformance@alpha --version` resolves 0.2.0-alpha.11,
and `bundle exec rake conformance` passes the baseline check on both legs: the server leg reports 73 passed with 0 failed
and no expected failures, and the client leg fails exactly the scenarios the baseline records.

## Breaking Changes

None. The change is confined to the conformance tooling; the shipped gem is untouched.
@koic

koic commented Aug 8, 2026

Copy link
Copy Markdown
Member Author

Merging via bypass to use this as the baseline for verifying 2026-07-28 specification coverage.

@koic
koic merged commit 4fac357 into modelcontextprotocol:main Aug 8, 2026
11 checks passed
@koic
koic deleted the pin_conformance_cli_to_the_alpha_dist_tag branch August 8, 2026 09:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant