Skip to content

Conversation

@keugenek
Copy link
Contributor

Summary

  • Add new MCP tool databricks_query_sdk_docs that allows LLM agents to search Databricks SDK documentation
  • Addresses the problem where LLMs struggle with SDK APIs and guess incorrectly (e.g., using requests.get(...) instead of proper SDK methods)
  • Generated documentation index from annotations_openapi.yml with 7 core services, 277 types, and 3 enums

Features

  • Fuzzy/keyword search across services, methods, types, and enums
  • Filtering by category (services, methods, types, enums) and service name
  • Score-based ranking for relevant results
  • LLM-friendly output with method signatures, parameters, return types, and examples

Example Usage

Query: "how to create a job"
Response: Jobs.Create method with signature, parameters, return type, and example code

Test plan

  • Unit tests pass: go test ./experimental/aitools/lib/providers/sdkdocs/...
  • Linter passes: make lint
  • CLI builds successfully: go build -o cli .
  • Manual testing with MCP client

Related

Discussion in #db-agent-builder about APX MCP providing SDK documentation indexing for better LLM interactions.

🤖 Generated with Claude Code

Add a new MCP tool `databricks_query_sdk_docs` that allows LLM agents
to search Databricks SDK documentation for methods, types, and examples.

This addresses the problem where LLMs struggle with the Databricks SDK
because they lack indexed documentation. Instead of guessing API calls,
agents can now query for proper method signatures, parameters, and usage.

Features:
- Fuzzy/keyword search across services, methods, types, and enums
- Category and service filtering
- Score-based result ranking
- LLM-friendly markdown output with signatures and examples

Implementation:
- New sdkdocs provider with embedded JSON documentation index
- Index generator tool that parses annotations_openapi.yml
- Generated index includes 7 core services, 277 types, and 3 enums
- Full unit test coverage for search and index loading

Co-Authored-By: Claude Opus 4.5 <[email protected]>
@eng-dev-ecosystem-bot
Copy link
Collaborator

eng-dev-ecosystem-bot commented Jan 12, 2026

Commit: 142b8c9

Run: 20926060084

Env 🟨​KNOWN 🔄​flaky 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
🟨​ aws linux 3 21 2 380 685 24:38
🟨​ aws windows 6 18 2 382 683 20:49
🟨​ aws-ucws linux 3 21 2 531 562 34:59
🟨​ aws-ucws windows 6 18 2 533 560 31:24
🔄​ azure linux 2 17 3 380 684 25:47
🟨​ azure windows 4 1 14 3 382 682 22:17
💚​ azure-ucws linux 19 3 527 561 34:57
🟨​ azure-ucws windows 4 15 3 529 559 28:43
💚​ gcp linux 19 3 369 690 20:57
🟨​ gcp windows 4 15 3 371 688 18:31
26 interesting tests: 18 RECOVERED, 6 KNOWN, 1 SKIP, 1 flaky
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
🟨​ TestAccept 🟨​K 🟨​K 🟨​K 🟨​K 🔄​f 🟨​K 💚​R 🟨​K 💚​R 🟨​K
💚​ TestAccept/bundle/deployment/bind/alert 🙈​S 🙈​S 🙈​S 🙈​S 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🟨​ TestAccept/bundle/generate/alert 💚​R 🟨​K 💚​R 🟨​K 💚​R 🟨​K 💚​R 🟨​K 💚​R 🟨​K
🟨​ TestAccept/bundle/generate/alert/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 🟨​K 💚​R 🟨​K 💚​R 🟨​K 💚​R 🟨​K 💚​R 🟨​K
🟨​ TestAccept/bundle/generate/alert/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 🟨​K 💚​R 🟨​K 💚​R 🟨​K 💚​R 🟨​K 💚​R 🟨​K
💚​ TestAccept/bundle/resources/alerts/basic 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/alerts/basic/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/alerts/basic/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/alerts/with_file 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/alerts/with_file/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/alerts/with_file/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/resources/permissions 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions 🟨​K 🟨​K 🟨​K 🟨​K 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 🟨​K 🟨​K
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions 💚​R 💚​R 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R
🔄​ TestAccept/ssh/connection 💚​R 💚​R 💚​R 💚​R 🔄​f 🔄​f 💚​R 💚​R 💚​R 💚​R
💚​ TestExport 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestExportWithFileFlag 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestImportDir 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestImportDirDoesNotOverwrite 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestImportDirWithOverwriteFlag 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestImportFileFormatAuto 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestImportFileFormatSource 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 25 slowest tests (at least 2 minutes):
duration env testname
6:44 aws-ucws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:56 aws-ucws linux TestAccept/bundle/resources/synced_database_tables/basic
5:55 aws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:48 aws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:46 aws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:36 gcp windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:30 gcp linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:25 gcp linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:23 aws-ucws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:12 gcp windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:12 aws-ucws windows TestAccept/bundle/resources/synced_database_tables/basic
4:45 azure windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
4:39 azure-ucws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
4:31 azure linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
4:23 azure-ucws linux TestAccept/bundle/resources/synced_database_tables/basic
4:02 azure-ucws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
4:01 azure windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
3:27 azure-ucws windows TestAccept/bundle/resources/synced_database_tables/basic
2:29 azure-ucws linux TestAccept
2:26 aws-ucws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
2:24 aws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
2:16 gcp linux TestAccept
2:13 aws-ucws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
2:05 azure linux TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=no/NBOOK=no/PY=no/READPLAN=
2:00 gcp linux TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=no/NBOOK=no/PY=no/READPLAN=

Adds a skill that helps Claude Code users discover and use
the databricks_query_sdk_docs MCP tool effectively when
asking about SDK methods, types, and parameters.

Co-Authored-By: Claude Opus 4.5 <[email protected]>
@keugenek keugenek changed the title aitools: Add SDK documentation query tool for MCP server [PoC] aitools: Add SDK documentation query tool for MCP server Jan 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants