fix: avoid intercepting quoted TOOL_NAME syntax in system tool parser

[MumuTW]

Summary

• gate non-standard TOOL_NAME: start detection so it is only allowed at the beginning of assistant output
• keep standard ```tool fenced detection unchanged
• add regression coverage to ensure explanatory/quoted tool syntax is not intercepted

Tests

• npm run vitest -- tools/systemMessageTools/toolCodeblocks/detectToolCallStart.vitest.ts tools/systemMessageTools/toolCodeblocks/interceptSystemToolCalls.vitest.ts

Closes #11070.

---

Continue Tasks: 🔄 7 running — View all

---

Summary by cubic

Prevented the system tools parser from intercepting quoted “TOOL_NAME:” syntax by only allowing loose tool starts at the very beginning of assistant output. Standard ```tool fenced detection remains unchanged. Addresses Linear #11070.

• Bug Fixes
  • Add allowAlternateStarts option to detectToolCallStart and disable it after any non-whitespace assistant text.
  • Track non-whitespace assistant text to confine loose "TOOL_NAME:" detection to the output start.
  • Add tests to skip non-standard starts when alternates are disabled and ensure quoted/explanatory tool syntax is emitted as plain text.

^{Written for commit 71083c6. Summary will update on new commits.}

[cubic-dev-ai]

No issues found across 4 files

[MumuTW]

Pushed a formatting follow-up to address the failure.\n\n- Ran repository-configured Prettier on all PR-changed files; one test file required formatting updates.\n- Committed the fix in .\n- Re-ran targeted tests:  ERR_PNPM_NO_IMPORTER_MANIFEST_FOUND  No package.json (or package.yaml, or package.json5) was found in "/home/opc/.paperclip/instances/default/workspaces/7948d02f-b91e-4189-b9eb-32bf0b5923d2". (pass).

[MumuTW]

Pushed a formatting follow-up to address the prettier-check failure.

• Ran repository-configured Prettier on all PR-changed files; one test file required formatting updates.
• Committed the fix in 71083c6.
• Re-ran targeted tests: pnpm vitest tools/systemMessageTools/toolCodeblocks/detectToolCallStart.vitest.ts tools/systemMessageTools/toolCodeblocks/interceptSystemToolCalls.vitest.ts (pass).

[RomneyDa]

@MumuTW this one is interesting, I think the way the system prompt is formatted we would sometimes expect to see text before the tool call starts (interleaving text with tool call for lower-performance models was actually a major motivation for system tool calls). Could you provide an example for where this goes wrong or more thoughts on it?

[MumuTW]

@RomneyDa Great question — here's a concrete scenario where the current parser misfires.

The problem

The alternate start pattern (tool_name:, index 1) matches on a per-line basis because splitAtCodeblocksAndNewLines splits at newlines, resetting the buffer each time. So any line that starts with TOOL_NAME: is treated as a tool invocation — even if the model is just explaining the syntax to the user.

Example input stream (model explaining how to use system tools):

Chunk 1: "Here is the syntax:\n"
Chunk 2: "TOOL_NAME: read_file\n"
Chunk 3: "BEGIN_ARG: filepath\n"
Chunk 4: "path/to/the_file.txt\n"
Chunk 5: "END_ARG\n"

Current (pre-fix) behavior:

• Chunk 1 is yielded as assistant text, buffer resets to ""
• Chunk 2 starts a fresh buffer → matches the loose "tool_name:" pattern (case-insensitive, index 1)
• Parser enters tool-call mode → produces a toolCalls delta with function.name = "read_file" and arguments = {"filepath": "path/to/the_file.txt"}
• The user never sees the explanatory text — it's silently swallowed into a phantom tool invocation

Post-fix behavior:

• After yielding chunk 1 (contains non-whitespace text), sawAssistantNonWhitespaceText becomes true
• Chunk 2 arrives → detectToolCallStart is called with allowAlternateStarts: false → the loose pattern at index 1 is skipped
• All 5 chunks pass through as normal assistant text

Why the standard format is unaffected

The fix only suppresses alternate starts (index > 0) after the model has already produced conversational text. The standard code-fenced format (```tool\n, index 0) is always allowed regardless of the flag. So a well-formed model that uses code fences can still freely interleave text with tool calls — only the fallback TOOL_NAME: pattern (designed for lower-performance models that skip code fences) is restricted to appearing at the very start of assistant output.

This means the interleaving use case you mentioned is fully preserved for the standard format. The alternate format is the one that's problematic because it's too easy to match accidentally in explanatory text.

Let me know if you'd like me to adjust the approach!

[MumuTW]

@RomneyDa Friendly ping — the detailed explanation above addresses the concern about interleaving. To summarize:

• The fix only restricts the loose TOOL_NAME: alternate start (index > 0) after non-whitespace assistant text has been emitted.
• The standard code-fenced format (\``tool`) at index 0 is always allowed, preserving the interleaving use case for models that use it.
• All CI checks are green and all 47 tests pass.

Happy to adjust the approach if you'd prefer a different direction!

[RomneyDa]

@MumuTW thanks for the explanation. I think the downsides of preventing model from interleaving with only TOOL_NAME and potentially preventing it from calling tools when it does want to probably outweigh the upside of allowing models to explain the syntax, since that's an edge case and most users won't worry about the syntax. I don't have evals on this but do you think people ask about the syntax often?