Update Databricks CLI to v1.17.0

[eng-dev-ecosystem-bot]

Update Databricks CLI to v1.17.0

[github-actions]

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/vscode

Inputs:

• PR number: 2200
• Commit SHA: bfa4af5c5a3d2a41f8c5a39f68f7f9c353061f81

Checks will be approved automatically on success.

[rugpanov]

Executive summary — Databricks CLI v1.16.1 → v1.17.0

Compatibility note (action-relevant): the direct-engine deployment state format moves to version 3 — bundles deployed with this CLI are rejected by clients older than v1.8.0 (#6713). The extension always shells out to its bundled binary (./bin/databricks, resolved in CliWrapper.cliPath at packages/databricks-vscode/src/cli/CliWrapper.ts:517), so it moves forward as one unit; the only exposure is a user whose CI or teammates run a pre‑1.8.0 CLI against the same bundle state.

CLI

• New INVALID_REFRESH_TOKEN error code on databricks auth token --output json failures (#6684).
• Experimental databricks auth docker configure / databricks auth docker token for Databricks Artifact Registry credential-helper access (#6700, #6699).
• environments setup-local now returns the distinct E_PROVISION_CONFLICT (instead of generic E_PROVISION) when uv sync hits a dependency conflict (#6666).
• SSH sessions now survive temporary tunnel disconnects, with bounded replay and backpressure for large transfers (#6650).
• OAuth U2M login can override the client ID via --client-id, profile client_id, or DATABRICKS_CLIENT_ID (#6594).

Bundles

• Deploy/plan accuracy (direct engine): permissions/grants/secret-ACL removals and already-deleted resources are no longer reported as deletions (#6647, #6675); phantom drift eliminated for cluster‑policy policy_id, server-populated nested fields, and trailing‑slash storage_root (#6531, #6618, #6622).
• Deploy UX: direct-engine deploy now reports each resource as it lands (and what a partial deploy did apply) instead of a single end-of-run dump (#6361).
• Correctness fixes: bundle run no longer fails with expected an int, found a string on unrelated undeployed refs (#6690); pipeline recreation on ingestion_definition add/remove (#6589); state lineage/cleanup on destroy-then-redeploy (#6210, #6685); destroy leaves triggered job runs alone (#6672); vector-search→catalog/schema ordering (#6655).
• New resource support: AI Gateway model_service, mcp_service, model_provider_service incl. grants (#6525, #6633, #6634, #6635); PyDABs for cluster policies, dashboards, Genie spaces (#6585); job_runs lifecycle.triggers.on_file_change (#6309); bundle generate app preserves Git-backed app config (#6656).
• Summary/perf: dashboard serialized_dashboard stored as a content hash (#6105); Postgres branches/endpoints/databases/roles now show a name in bundle summary (#6663); faster config load for many-file bundles (#6195); clearer "not deployed" messaging for URL-less resource types (#6583).

Dependency updates

• Databricks SDK for Go v0.177.0 → v0.178.0 (#6673), Terraform provider v1.131.0 → v1.132.0 (#6671), and CVE bumps (#6695). Bundled into the binary — no extension-visible surface.

What could be integrated into the VS Code extension experience

1. E_PROVISION_CONFLICT — already plumbed; this bump is what turns it on (zero code change).
The extension already has a full, dedicated code path for E_PROVISION_CONFLICT: the code is in the union at packages/databricks-vscode/src/python-setup/models/PythonSetupResult.ts:58-59, with its own cluster‑vs‑local conflict message (errorMessages.ts:233), its own error action (errorMessages.ts:316), and report-worthiness handling (errorMessages.ts:496). Against v1.16.1 the CLI only ever emitted generic E_PROVISION for a uv sync conflict, so this branch was dead. v1.17.0 (#6666) is precisely the change that makes environments setup-local emit the distinct code, so the extension's tailored copy starts firing on the bump alone. This is a flag-flip on existing plumbing — highest value, no work. Worth a quick manual confirmation that the CLI's wire spelling matches the enum string, since the whole path hinges on that exact token.

2. SSH tunnel resilience — transparent no-op, high user value.
The extension launches databricks ssh connect --ide=… --auto-approve in a transient terminal (CliWrapper.getSshConnectCommand at CliWrapper.ts:141-155; SshCommands.launchSshTunnel at SshCommands.ts:485-506). The reconnect/replay work (#6650) lives entirely inside that subprocess, so users of the "Start SSH Tunnel" flow (databricks.ssh.startTunnel, extension.ts:446/:1501) get disconnect resilience for free with no extension change. This is the CLI half of the same regression surface the repo just added e2e coverage for (recent commits bc07008, 4850897) — no action needed beyond letting the bundled binary carry it.

3. Incremental deploy reporting — transparent no-op, better live log for free.
bundleDeploy runs bundle deploy --target … --verbose and streams stderr line-by-line into the bundle output channel (CliWrapper.ts:1056; the streaming happens via runBundleCommand's onStderr → logger.info, CliWrapper.ts:446-476). Because the extension already renders the CLI's streamed output verbatim, the per-resource progress from #6361 improves the "Deploying the bundle…" log with no code change. Same story for the plan/drift accuracy fixes (#6647, #6675, #6531, #6618, #6622): the extension does not run bundle plan anywhere (verified — no bundle plan invocation exists), so these only make deploy logs less noisy.

4. INVALID_REFRESH_TOKEN structured error — net-new, and not on a path the extension uses today.
The extension never invokes databricks auth token --output json (verified — no auth token call in src/cli or src/configuration). Its "session expired, re-authenticate" detection is string matching on setup-local stderr: isReauthRequiredError in python-setup/utils/authErrors.ts:25 looks for "refresh token is invalid" + a databricks auth login remediation. Consuming the new machine-readable INVALID_REFRESH_TOKEN code (#6684) would make that detection robust to wording drift — but it is net-new work on a command the extension doesn't currently call, and the existing heuristic is deliberately tuned for precision, so this is a low-priority hardening idea, not a quick win.

5. auth login --client-id override — net-new, no current need.
The extension's login shells out with a fixed arg list (["auth", "login", "--host", host] plus --profile/--timeout) at configuration/auth/DatabricksCliCheck.ts:157-163. #6594 makes it possible to pass a custom --client-id, but there is no setting or flow that wants one today; wiring it up would be net-new and speculative. Flagging only for completeness.

No-ops for the extension (called out explicitly)

• auth docker configure/token (#6700, #6699): the extension has no Artifact Registry / Docker credential surface (the only docker references are docker_image fields in the generated BundleSchema.ts). Nothing to integrate.
• AI Gateway resources, PyDABs, on_file_change, bundle generate app, Postgres summary names (#6525/#6633/#6634/#6635, #6585, #6309, #6656, #6663): the bundle resource explorer only renders jobs and pipelines as interactive resource types (ui/bundle-resource-explorer/types.ts:36-39), and the extension runs neither bundle generate nor bundle plan. New resource kinds fall through to the generic/unknown tree node at best — no dedicated UI benefits without net-new explorer support, which is out of scope for a version bump.
• Terraform-engine deprecation reinforced by state v3 (#6713): the extension already warns once-per-workspace when bundle validate reports engine === "terraform" and links the direct-engine migration guide (bundle/BundleEngineManager.ts:67-100). The state-version note strengthens the case for that existing nudge but needs no change.

_{🤖 Auto-generated executive summary of the CLI v1.16.1 → v1.17.0 changelog. Integration notes are opportunities, not commitments — verify before acting.}