Agent Merge: recover from a checks fragment the host refuses

[Benjamin Christopher Simmonds (benibenj)]

Agent Merge could hang indefinitely on a pull request whose checks fragment GitHub refuses, showing the user nothing at all. I hit this on a real session and traced it from the agent host logs.

What happens

When an organization enforces SAML SSO and the signed-in token is not authorized for it, the checks GraphQL query is refused with HTTP 200 and a FORBIDDEN entry in the body (hence the odd authorization:200 in the logs). microsoft/vscode is public, so core, mergeability, review threads and comments all load fine — only checks fails, because it reaches into GitHub Actions data that is org-protected even on a public repo.

Verified by introspection: CheckRun.checkSuite is CheckSuite! (NON_NULL), so a refusal on the Actions data behind checkSuite { workflowRun { workflow { name } } } null-propagates and destroys the entire check node — the whole fragment fails, not just that one informational field.

From there, five things compounded into a silent, unbounded loop:

1. checksPending(undefined) returns true, so a fragment that never loaded read as "checks still running" and held the fast poll cadence — a measured steady ~66s re-request of a permanently refused query, forever, with no backoff.
2. _scheduleAfterFailure treated authorization as "just reschedule": no failure counting, no backoff.
3. checks therefore never satisfied isCompleteHeadFragment, so the gate was permanently indeterminate.
4. indeterminate was the only gate outcome with neither an action nor a budget — prompt has repeat/total caps and terminal disables, but indeterminate just rescheduled the backstop unconditionally.
5. Only kind === 'authentication' raised an auth requirement, so an authorization refusal never prompted the user. Agent Merge looked enabled and did nothing.

The generic reason string made it hard to diagnose too: one "Pull request state is incomplete or stale" covered five different fragments and never said which one, or why.

The fix

• Gate the workflow-name subselection and drop it for a repository whose host refuses it, keeping the checks themselves. workflowName is purely informational (one consumer: repair-agent context in agentMergeTools.ts). The decision is remembered per repository so later polls don't re-pay for a rejected request.
  • Only the rollup request is retried. _fetchExpectedCheckSuites reads the same protected Actions surface and Agent Merge always requests it (checks: { required: true }), so a refusal there would otherwise be misattributed — re-running the whole paginated query for nothing and disabling workflow names on a repo where they were never the problem. That request now degrades to absent-and-incomplete instead.
• Raise an auth requirement for authorization as well as authentication, naming the organization and calling out SSO when GitHub reports it — turning a silent hang into an actionable prompt.
• Give indeterminate a budget, measured over continuously observed time. Evaluation is suspended while a turn runs and stops entirely while the host sleeps, so a gap between observations restarts the window rather than counting toward it. Keyed on a stable cause (e.g. checks:authorization) rather than the volatile reason text, so an oscillating reason can't defeat it.
• Name the fragment and its error in the indeterminate reason. FragmentState.error already carried this; it was simply never read.
• Back off persistent authorization failures, and stop an errored checks fragment from holding the fast cadence — _setError preserves the previous value, so this covers both the never-loaded and the stale-value cases.

Safety

Partial check data is still never accepted. throwGraphQLErrors continues to throw on any non-empty errors array, so blocked nodes (which arrive as null because of the NON_NULL propagation) can never reach toCheck. Silently dropping refused checks could let Agent Merge conclude required checks passed and merge unsafely — the existing behaviour fails closed, which is correct; what was missing was visibility, not permissiveness. classifyAgentMergeRequiredChecks ignores expectedSuites entirely, so degrading it does not affect any merge decision.

Validation

• npm run typecheck-client clean.
• 68 tests pass across the affected suites, including new coverage for the refusal fallback (with the Agent-Merge-shaped { checks: { required: true } } config), the expected-suites refusal, and the fragment-naming reasons.
• The rewritten budget state machine was verified directly against the compiled method: continuous observation fires at 30 minutes, a 120-minute unobserved gap does not, a changed cause restarts, and it still fires after a restart.

[Copilot]

Pull request overview

Improves Agent Merge recovery when GitHub refuses protected checks data.

Changes:

• Retries checks without workflow names and degrades unavailable expected suites.
• Adds authorization diagnostics, polling backoff, and indeterminate-state budgeting.
• Adds tests for query fallback and fragment diagnostics.

Show a summary per file

File	Description
src/vs/platform/github/test/node/pullRequestQueryService.test.ts	Tests checks-query fallbacks.
src/vs/platform/github/common/pullRequestResourceService.ts	Adjusts failure polling cadence.
src/vs/platform/github/common/pullRequestQueryService.ts	Implements authorization fallbacks.
src/vs/platform/github/common/githubService.ts	Supplies logging to query service.
src/vs/platform/agentHost/test/common/agentMerge.test.ts	Tests fragment-specific reasons.
src/vs/platform/agentHost/node/agentMergeController.ts	Adds authorization handling and timeout budget.
src/vs/platform/agentHost/common/agentMerge.ts	Adds stable indeterminate causes.

Review details

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Suppressed comments (3)

src/vs/platform/agentHost/node/agentMergeController.ts:442

• Please add an automated controller test for this new timeout state machine. The committed tests do not cover the 30-minute threshold, observation-gap reset, changed-cause reset, or eventual disable behavior, so the central guard against another unbounded loop can regress despite the one-off compiled-method verification described in the PR.

				if (this._isIndeterminateBudgetExhausted(session, runtime, gate.cause)) {
					this._disable(session, agentMerge, `the pull request state could not be evaluated for ${Math.round(maximumIndeterminateDuration / 60_000)} minutes: ${gate.reason}`);
					return;

src/vs/platform/github/common/pullRequestResourceService.ts:701

• Please cover this cadence branch in pullRequestResourceService.test.ts for both a never-loaded checks fragment and a stale pending value. Existing polling tests do not assert that status === 'error' switches either case from the pending interval to checksBackstop, which is one of the failure-loop fixes introduced here.

				const checks = entry.snapshot.get().checks;
				return checks.status !== 'error' && checksPending(checks.value)
					? visible ? this._policy.checksPendingVisible : this._policy.checksPendingBackground
					: this._policy.checksBackstop;

src/vs/platform/agentHost/node/agentMergeController.ts:811

• This JSDoc exceeds the repository's 1–2 sentence limit and duplicates the surrounding control flow. Reduce it to the method's timing contract.

	/**
	 * Reports whether one unchanged indeterminate cause has persisted past its
	 * budget.

• Files reviewed: 7/7 changed files
• Comments generated: 5
• Review effort level: Balanced