Skip to content

fix: invalidate cached MCP session on server-side session loss - #7061

Open
sachiantany wants to merge 1 commit into
google:mainfrom
sachiantany:fix/mcp-stale-session-reconnect
Open

fix: invalidate cached MCP session on server-side session loss#7061
sachiantany wants to merge 1 commit into
google:mainfrom
sachiantany:fix/mcp-stale-session-reconnect

Conversation

@sachiantany

Copy link
Copy Markdown

When an MCP server scales to zero and back, its in-memory sessions are lost. The existing @retry_on_errors decorator retries the call, but create_session() returns the same cached (now dead) session because _is_session_disconnected() only checks transport-level stream closure — the transport is still alive, only the server-side session is gone.

Add MCPSessionManager.invalidate_session() so callers can mark a session key as stale. _is_session_disconnected() now consults an _invalidated_sessions set before falling back to the stream check. McpToolset._execute_with_session() calls invalidate_session() on any exception, so the next retry from @retry_on_errors builds a fresh session instead of reusing the dead one.

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

1. Link to an existing issue (if applicable):

Problem:
When an MCP server (e.g. on Cloud Run) scales to zero and restarts, its in-memory sessions are lost. The agent's @retry_on_errors decorator retries the failed call, but MCPSessionManager.create_session() returns the same cached dead session because _is_session_disconnected() only checks transport-level stream closure — the transport is still alive, only the server-side session is gone. Every retry reuses the stale session and fails, until all retries are exhausted.

Solution:

  • Add MCPSessionManager.invalidate_session() method to mark a session key as stale
  • Extend _is_session_disconnected() to check an _invalidated_sessions set before falling back to the transport stream check
  • Call invalidate_session() from McpToolset._execute_with_session() on any exception, before re-raising
  • On the next create_session() call, the invalidated session is replaced with a fresh one and the flag is cleared

This lets the existing @retry_on_errors mechanism work correctly for server-side session loss, not just transport failures.

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.

105 tests passed, 0 failures in tests/unittests/tools/mcp_tool/test_mcp_session_manager.py (5.30s). New tests added:

  • test_is_session_disconnected_with_invalidated_key — invalidated key returns disconnected, other keys unaffected
  • test_invalidate_session_marks_key — correct session key added to _invalidated_sessions
  • test_invalidated_session_is_replaced_on_next_create — full integration: inreturns new session → flag cleared
  • Updated existing pickle test to verify _invalidated_sessions round-trips as empty set

Manual End-to-End (E2E) Tests:

Tested with an ADK agent connected to an MCP server deployed on Cloud Run (Stith scale-to-zero enabled. After ~15 minutes idle, the server scaled to zero. On the next tool call, the server scaled back up — the agent detected the stale session, invalidated it, and successfully reconnected with a fresh session on retry instead of failing with repeated stale session errors.

Checklist

  • I have read the [CONTRIBUTING.md](https://github.com/google/adk-python/bdocument.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • I have manually tested my changes end-to-end.
  • Any dependent changes have been merged and published in downstream modul

Additional context

3 files changed, 125 insertions(+), 3 deletions(-):

  • src/google/adk/tools/mcp_tool/mcp_session_manager.py — 45 additions
  • src/google/adk/tools/mcp_tool/mcp_toolset.py — 6 additions
  • tests/unittests/tools/mcp_tool/test_mcp_session_manager.py — 77 additions (3 new tests + 1 updated)

When an MCP server scales to zero and back, its in-memory sessions are
lost.  The existing @retry_on_errors decorator retries the call, but
create_session() returns the same cached (now dead) session because
_is_session_disconnected() only checks transport-level stream closure
— the transport is still alive, only the server-side session is gone.

Add MCPSessionManager.invalidate_session() so callers can mark a
session key as stale.  _is_session_disconnected() now consults an
_invalidated_sessions set before falling back to the stream check.
McpToolset._execute_with_session() calls invalidate_session() on any
exception, so the next retry from @retry_on_errors builds a fresh
session instead of reusing the dead one.
@google-cla

google-cla Bot commented Sep 8, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MCP session stale after server scale-to-zero - @retry_on_errors reuses dead cached session

2 participants