Skip to content

add ToolSearchToolset and ToolProxyToolset for dynamic tool discovery#5140

Open
longcw wants to merge 11 commits intolongc/improve-toolsetsfrom
longc/tool-search-toolset
Open

add ToolSearchToolset and ToolProxyToolset for dynamic tool discovery#5140
longcw wants to merge 11 commits intolongc/improve-toolsetsfrom
longc/tool-search-toolset

Conversation

@longcw
Copy link
Contributor

@longcw longcw commented Mar 18, 2026

When agents have many tools (10+), loading all tool definitions into the LLM context upfront degrades accuracy and wastes tokens. This PR adds two toolsets that enable on-demand tool discovery and loading.

ToolSearchToolset — exposes a tool_search function. Matched tools are added directly to the LLM's tool list on the next turn, so the model uses native tool calls. Simpler for the model.

ToolProxyToolset — exposes exactly two fixed tools: tool_search (returns schemas) and call_tool (executes by name). The tool list remains unchanged, preserving prompt cache across turns for providers like OpenAI and Anthropic.

Both support:

  • Nested toolsets (MCPToolset), standalone tools, ProviderTool, and RawFunctionTool
  • Atomic toolset loading: if a tool picked belongs to a nested toolset, the whole toolset is loaded.
  • Pluggable search via SearchStrategy protocol
  • Built-in keywod and BM25 search strategy

need to merge #5138 first

References

@longcw longcw changed the title add ToolSearchToolset add ToolSearchToolset and ToolProxyToolset for dynamic tool discovery Mar 19, 2026
@longcw longcw marked this pull request as ready for review March 19, 2026 08:26
Copy link
Contributor

@devin-ai-integration devin-ai-integration bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

@longcw longcw force-pushed the longc/tool-search-toolset branch from 2575393 to d6c9e1a Compare March 20, 2026 03:16
devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant