feat(mcp): disclose when search_text was bound by limit, and make the ceiling configurable

[tiendungdev]

Closes #674, and the ceiling half of #672.

The zero-cost failure

search_text is described as the alt grep backbone, and a result bound by limit was byte-indistinguishable from a complete one: count is len(enriched), so the array and the count corroborated each other at the ceiling. On the repo in #672 that is count: 1000 against a git grep truth of 8963, with no flag, no cursor, and nothing else in the payload to doubt.

The byte budget has always disclosed its own truncation (_truncated_by_budget). The limit path had no equivalent.

What a truncated response now carries

{
  "count": 1000,
  "_truncated_by_limit": true,
  "_limit_applied": 1000,
  "count_is_exact": false,
  "_limit_requested": 100000,     // only when the ceiling, not the caller, chose it
  "truncation_note": "…"
}

The note follows the contract_tier_unbuilt shape that already works elsewhere in this codebase: say what the number means, say how to widen, and rule out the widening that looks obvious but does not work. Here that last part is path: the filter runs over what survived truncation rather than over the corpus, so slicing a subtree returns whatever was left of the global cut. #672 lists that as one of three workarounds it had to rule out by hand.

The ordering is the fix

rawMatches is counted before the path and scope filters, and that is the substance rather than an implementation detail:

matches = <searcher>(query, …, limit)   // stops at limit
rawMatches := len(matches)              // ← here
matches = filterTextMatchesByPath(…)    // runs over what survived
matches = s.filterTextMatchesByResolvedScope(…)

A response holding 952 matches can be a truncated 1000. Measuring after the filters would miss exactly the case a caller cannot detect from the outside — and the 952 / 998 rows in #672's table are that shape, which is why the issue notes that even len(matches) == 1000 is not a reliable client-side detector.

TestSearchText_TruncationIsMeasuredBeforeThePathFilter pins it with a fixture where the count necessarily lands below the limit: four matching files, limit: 3, filtered to a directory holding two. Whichever three the searcher returns, at most two survive, and the disclosure must still fire.

Landing on the effective limit is the signal, so this can fire on a corpus holding exactly limit matches. A spurious "verify this" is the safe direction to be wrong in.

The ceiling

GORTEX_SEARCH_TEXT_MAX_LIMIT, defaulting to 1000 so no existing caller's response changes shape. An unset, unparseable or non-positive value keeps the default rather than lifting the bound — a typo must not become an unbounded scan.

Raising the ceiling without the disclosure only moves the silent cliff, which is why they land together. This is why I marked #674 as the one this closes and #672 as refs: the flag is the fix, the override is the affordance.

Tests

Five end-to-end cases through the handler and tables over both pure helpers. Ten mutants, each failing its test and passing restored: the flag never published · count_is_exact always true · landing exactly on the limit not counted as truncation · truncation measured after the filters · the clamped request not disclosed · the disclosure riding on every response · the ceiling override ignored · an unusable ceiling value honored · the note dropping the recovery that does not work · the applied limit not reported.

Three of those initially came back as "nothing ran" rather than as failures — each mutation orphaned a variable and the package stopped compiling, which is a void result, not a dead mutant. They were re-run in compile-clean form (if _ = rawMatches; …) so all ten are real.

Verification

go build ./... clean; the SearchText suite passes.

Two measurements worth reporting rather than asserting:

The schema addition spends 41 of the 62 bytes the core preset has left. My first version spent 233 and turned CI red on TestToolsListByteCeilings (core at 97671 against a 97500 baseline; main sits at 97438). The description now carries only the part an agent needs before calling — that a bound result sets _truncated_by_limit — and the recovery detail rides on the response in truncation_note instead. Both ceiling tests pass at 97479.

I had originally checked TestCompactToolsListIsStaticAndBudgeted instead, measured 14984 bytes on both trees, and reported that as evidence the addition was free. That number is about the facade surface, which search_text is not on; it was the wrong ceiling. Headroom as measured on main: 62 bytes on the core preset, 16 bytes on the facade list — both recorded on #685.

One unrelated test is order-dependent. TestSearchTextRefusalIsTheCapabilityEvaluationsRefusal failed for me, and I did not call it flaky until I had the mechanism: on a cold ref-view build it gets view_building: … retry after 2s instead of the capability_unavailable refusal it asserts, and on the next run — the build now cached under a content-hashed id that survives the per-test t.TempDir() — it passes. I reproduced both states on my branch and confirmed the pristine tree passes only in the warm state. Nothing in this change touches view building. Filing it separately.

🤖 Generated with Claude Code

[tiendungdev]

CI was red on me, fixed in f2ff364c. Two things I got wrong are worth stating plainly, because the second one invalidates a claim in my PR description.

What failed

TestToolsListByteCeilings — the core preset, not the facade surface:

core   mode=defer   bytes=97671  (baseline 97500)
core preset must shrink below its pre-diet baseline (97500), got 97671

macOS failed on it and ubuntu was cancelled by fail-fast, which is why both test jobs read as failures.

Mistake 1 — I checked the wrong ceiling

Before pushing I did go looking for the #685 hazard, found TestCompactToolsListIsStaticAndBudgeted, measured 14984 bytes on both trees, and concluded the cost was zero. That number is real and it is irrelevant: it covers the facade surface, where search_text is not present at all. The budget this change actually spends is the core preset's, and the test that guards it is the one I did not run.

The PR description says "the compact tools/list is byte-identical at 14984 on both trees" as evidence the addition was free. It was evidence about a surface my change cannot reach. I have corrected the body.

Mistake 2 — my isolation measured my own code

When I tried to confirm the cost after the CI failure, I reverted the schema file and re-ran:

git checkout -- internal/mcp/tools_analysis.go

and got 97671 again — identical to my branch. I nearly reported that as "the failure is pre-existing on main". It is not: git checkout -- restores from the index, and the change was already committed, so I had re-measured my own edit and read the matching numbers as proof of innocence.

Checking out upstream/main itself settled it:

main        core  97438  PASS
my branch   core  97671  FAIL   (+233)

Worth recording because the earlier isolation runs in this PR were sound only by accident — those changes were still uncommitted, where git checkout -- does revert. The technique quietly stops working the moment the work is committed, and it fails in the direction that exonerates the diff.

The fix

Available headroom on main is 62 bytes. The description now carries only what an agent cannot learn anywhere else before calling:

Max matching lines to return (default 100, capped at 1000). A bound result sets _truncated_by_limit.

41 bytes, landing at 97479. The rest of what I had put there — how to widen, and that a path filter cannot recover the remainder — already rides on the response in truncation_note, which is where a caller reads it after seeing the flag. Both ceiling tests and the SearchText suite pass.

If you would rather not spend 41 of the 62 remaining bytes on this at all, say so and I will drop the schema line entirely; the flag is discoverable from the response either way. That is a budget call, not a correctness one.

For #685

Fresh numbers, since this is the second time in one day I have had to reverse-engineer a budget from a log line:

• core preset: 97438 / 97500 → 62 bytes of headroom on main
• facade tools/list: 14984 / 15000 → 16 bytes

Nothing surfaces either number until CI is red, and neither failure message names the headroom — only a total that has to be subtracted by hand against a baseline stated in the message. I have added this measurement to #685.