Skip to content

Pin the conformance CLI to the alpha dist-tag #497

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
koic merged 1 commit into from
Aug 8, 2026
+23 −10
Merged

Pin the conformance CLI to the alpha dist-tag #497

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions conformance/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,10 +47,10 @@ bundle exec rake conformance_server PORT=3000
bundle exec rake conformance_server

# Terminal 2: run all scenarios
npx @modelcontextprotocol/conformance server --url http://localhost:9292/mcp
npx @modelcontextprotocol/conformance@alpha server --url http://localhost:9292/mcp

# Terminal 2: run a single scenario
npx @modelcontextprotocol/conformance server --url http://localhost:9292/mcp --scenario ping
npx @modelcontextprotocol/conformance@alpha server --url http://localhost:9292/mcp --scenario ping
```

Keeps the server alive between test runs, which avoids the startup overhead when iterating
Expand Down
4 changes: 2 additions & 2 deletions conformance/client_runner.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# frozen_string_literal: true

# Runs `npx @modelcontextprotocol/conformance client` against the conformance client script.
# Runs `npx @modelcontextprotocol/conformance@alpha client` against the conformance client script.
require "English"

module Conformance
Expand Down Expand Up @@ -29,7 +29,7 @@ def build_command
npx_command = [
"npx",
"--yes",
"@modelcontextprotocol/conformance",
"@modelcontextprotocol/conformance@alpha",
"client",
"--command",
"bundle exec ruby #{client_script}",
Expand Down
21 changes: 17 additions & 4 deletions conformance/expected_failures.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,19 @@
server: []
client:
# TODO: The conformance framework's client-initialization check pins the expected requested version to 2025-11-25,
# while this SDK's latest protocol version is 2026-07-28. Remove once the @modelcontextprotocol/conformance package
# accepts 2026-07-28.
- initialize
# Pending client-side support for the 2026-07-28 stateless lifecycle: the conformance client
# still connects through the legacy handshake and has no SEP-2322 MRTR driver yet.
- request-metadata
- sep-2322-client-request-state
# SEP-2243 client-side custom-header mirroring and invalid-tool header gating
# are not implemented in `MCP::Client::HTTP` yet.
- http-custom-headers
- http-invalid-tool-headers
# Client-side JSON Schema preservation gaps: `$ref` values must reach the caller unresolved.
- json-schema-ref-no-deref
- json-schema-2020-12-preservation
# Unimplemented authorization extensions.
- auth/dpop
- auth/dpop-nonce
- auth/wif-jwt-bearer
# Fails one check added to the suite after the 0.2.0-alpha.10 requirements anchor.
- auth/authorization-server-migration
4 changes: 2 additions & 2 deletions conformance/server_runner.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# frozen_string_literal: true

# Starts the conformance server and runs `npx @modelcontextprotocol/conformance` against it.
# Starts the conformance server and runs `npx @modelcontextprotocol/conformance@alpha` against it.
require "English"
require "net/http"
require_relative "server"
Expand Down Expand Up @@ -31,7 +31,7 @@ def run
def build_command
expected_failures_yml = File.expand_path("expected_failures.yml", __dir__)

npx_command = ["npx", "--yes", "@modelcontextprotocol/conformance", "server", "--url", "http://localhost:#{@port}/mcp"]
npx_command = ["npx", "--yes", "@modelcontextprotocol/conformance@alpha", "server", "--url", "http://localhost:#{@port}/mcp"]
npx_command += ["--scenario", @scenario] if @scenario
npx_command += ["--spec-version", @spec_version] if @spec_version
npx_command += ["--verbose"] if @verbose
Expand Down