Skip to content

AIR CLI Integration: authenticate air get up front and fail fast#5729

Merged
riddhibhagwat-db merged 4 commits into
air-clifrom
air-get-auth-failfast
Jul 8, 2026
Merged

AIR CLI Integration: authenticate air get up front and fail fast#5729
riddhibhagwat-db merged 4 commits into
air-clifrom
air-get-auth-failfast

Conversation

@riddhibhagwat-db

@riddhibhagwat-db riddhibhagwat-db commented Jun 25, 2026

Copy link
Copy Markdown

Changes

Authenticate before any run status or config is fetched or printed:

  • PreRunE maps a MustWorkspaceClient failure to an actionable auth error:
    • no default profile set and --profile/-p not passed (config.ErrCannotConfigureDefault) → "no default profile is set: pass --profile (-p) or configure a default profile in your .databrickscfg"
    • otherwise → "authentication was not successful: "
  • RunE calls CurrentUser.Me before fetching/rendering anything, so a credential that resolves locally but is rejected by the workspace also fails fast with the same clear message — and no partial status/config is shown.

Both errors are permanent (not retryable) and, in -o json mode, render as the standard error envelope (code: UNAUTHENTICATED).

Why

air get JOB_RUN_ID validated authentication only lazily. MustWorkspaceClient (PreRunE) calls Config.Authenticate, which merely attaches credentials (for a PAT it does no server-side check), so an invalid credential or missing profile surfaced as a confusing, generic failure partway through — after a run's config had already started rendering — instead of a clear, up-front error.

Testing

  • TestGetRunAuthFailed — a rejected CurrentUser.Me short-circuits before GetRun (no run fetched, nothing rendered).
  • TestAuthError — verifies the no-profile vs generic-auth message mapping (and that the cause is preserved).
  • Existing not-found tests updated to stub the up-front Me success.
  • go build ./..., the air unit tests, the air acceptance suite, and ./task lint-q (0 issues) all pass.

Based on air-cli (post-#5685, so the command is air get JOB_RUN_ID).

This pull request and its description were written by Isaac.

@riddhibhagwat-db riddhibhagwat-db changed the title experimental/air: authenticate air get up front and fail fast AIR CLI Integration: authenticate air get up front and fail fast Jun 25, 2026
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Jun 25, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 3eabc54

Run: 28917657084

Env ❌​FAIL 🟨​KNOWN 🔄​flaky 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
❌​ aws linux 6 1 3 4 261 1027 7:46
❌​ aws windows 6 1 1 3 4 262 1025 11:27
💚​ aws-ucws linux 4 4 363 941 5:57
💚​ aws-ucws windows 4 4 365 939 7:33
💚​ azure linux 4 4 264 1027 5:57
💚​ azure windows 4 4 266 1025 6:41
💚​ azure-ucws linux 4 4 362 939 9:11
💚​ azure-ucws windows 4 4 364 937 7:57
💚​ gcp linux 4 4 260 1030 6:39
💚​ gcp windows 4 4 262 1028 6:01
15 interesting tests: 6 FAIL, 4 SKIP, 3 RECOVERED, 1 KNOWN, 1 flaky
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
🟨​ TestAccept 🟨​K 🟨​K 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/invariant/no_drift 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
❌​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions ❌​F ❌​F ✅​p ✅​p 🙈​s 🙈​s 🙈​s 🙈​s 🙈​s 🙈​s
❌​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct ❌​F ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform ❌​F ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions ❌​F ❌​F ✅​p ✅​p 🙈​s 🙈​s 🙈​s 🙈​s 🙈​s 🙈​s
❌​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct ❌​F ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform ❌​F ❌​F ✅​p ✅​p
🙈​ TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/ssh/connection 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🔄​ TestSyncIncrementalFileOverwritesFolder ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
💚​ TestFetchRepositoryInfoAPI_FromRepo 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestFetchRepositoryInfoAPI_FromRepo/root 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestFetchRepositoryInfoAPI_FromRepo/subdir 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 23 slowest tests (at least 2 minutes):
duration env testname
5:59 azure-ucws linux TestSecretsPutSecretStringValue
4:42 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:18 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:15 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
4:06 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:38 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:26 aws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:25 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:17 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:14 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:07 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:05 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:02 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:56 gcp linux TestSecretsPutSecretStringValue
2:55 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:51 aws linux TestSecretsPutSecretStringValue
2:47 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:44 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:42 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:40 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:38 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:34 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:23 aws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct

@riddhibhagwat-db riddhibhagwat-db force-pushed the air-get-auth-failfast branch 2 times, most recently from 2fb4758 to 9b8fd81 Compare July 6, 2026 15:27
@riddhibhagwat-db riddhibhagwat-db changed the base branch from air-cli to air-integration-m3-1 July 6, 2026 15:27
@riddhibhagwat-db riddhibhagwat-db force-pushed the air-integration-m3-1 branch 2 times, most recently from 1cf6da6 to 827ff43 Compare July 6, 2026 18:46
@riddhibhagwat-db riddhibhagwat-db force-pushed the air-get-auth-failfast branch from 9b8fd81 to 387c976 Compare July 6, 2026 18:47
@riddhibhagwat-db riddhibhagwat-db changed the base branch from air-integration-m3-1 to air-cli July 6, 2026 18:48
// mid-render instead of a clear "not authenticated" error here.
if _, err := w.CurrentUser.Me(ctx, iam.MeRequest{}); err != nil {
return authError(ctx, cmd, err)
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any failure from this Me() probe is routed through authError, which unconditionally emits UNAUTHENTICATED / PERMANENT / retryable=false (L82–87). So a transient failure reaching /me (429, 5xx, network blip) gets reported as a non-retryable auth error.

Consider only mapping to UNAUTHENTICATED when the error is actually authshaped (e.g. an apierr 401/403), and falling back to the transient classification otherwise.

Comment thread experimental/air/cmd/get.go Outdated
data.Sweep = buildSweepInfo(ctx, w, task)
} else if genAIComputeTask(run) == nil {
// The typed SDK drops ai_runtime_task, so read it from the raw run.
enrichFromRawRun(ctx, w, runID, &data)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

w.Jobs.GetRun at L133 already fetched this run; fetchJobRun hits the same runs/get endpoint again. Can we reuse that response (or parse the raw payload once) to skip the second roundtrip?

The get command validated authentication only lazily: Config.Authenticate
(in PreRunE) merely attaches credentials, so a bad PAT or missing profile
surfaced as a confusing error partway through rendering, after a run's
config had already been printed.

Validate the workspace client up front instead:

- PreRunE maps MustWorkspaceClient failures to an actionable auth error: a
  missing-profile hint when no default profile is set and --profile (-p)
  was not passed (ErrCannotConfigureDefault), otherwise "authentication was
  not successful".
- RunE calls CurrentUser.Me before fetching or rendering anything, so a
  credential that resolves locally but is rejected by the workspace fails
  fast with the same clear message and no partial status/config output.

Co-authored-by: Isaac
…_task runs

The typed SDK's jobs.RunTask has no ai_runtime_task field, so w.Jobs.GetRun
drops the task for runs submitted by the current `air run` (which emits
ai_runtime_task). This left the Configuration box empty and Experiment /
Accelerators as N/A for those runs.

Mirror the Python CLI: for a single run with no gen_ai_compute_task, read the
raw runs/get payload, take the ai_runtime_task command_path, and fill the config
box from the sibling training_config.yaml plus the experiment and accelerator
cells. Legacy gen_ai_compute_task runs keep their existing path.

Co-authored-by: Isaac
Two fixes from review of the fail-fast/render commits:

- authError only maps genuinely auth-shaped failures (missing profile, or
  an apierr 401/403 via ErrUnauthenticated/ErrPermissionDenied) to
  UNAUTHENTICATED/PERMANENT. A transient failure at the Me() probe (429,
  5xx, network blip) is now reported as INTERNAL_ERROR/TRANSIENT so it
  stays retryable instead of a misleading non-retryable auth error.

- `air get` fetched the run twice on the ai_runtime_task path: w.Jobs.GetRun
  (typed) then a second runs/get in enrichFromRawRun. Add fetchRun, which
  makes one runs/get call and unmarshals the body into both the typed
  jobs.Run (display path) and the raw jobRun (preserves ai_runtime_task).
  enrichFromRawRun now takes the already-fetched raw run — no second
  roundtrip.

Tests: command-level not-found tests move to a real runs/get HTTP server
(the run fetch is no longer the typed, mockable API); add fetchRun coverage
and a transient-auth case.

Co-authored-by: Isaac
yamlfmt normalizes the block scalar to `command: |-` (strip trailing
newline); the CLI echoes the config back, so regenerate the acceptance
golden to match. Fixes the `task fmt` CI gate.

Co-authored-by: Isaac
@riddhibhagwat-db riddhibhagwat-db merged commit a3492b8 into air-cli Jul 8, 2026
18 of 21 checks passed
@riddhibhagwat-db riddhibhagwat-db deleted the air-get-auth-failfast branch July 8, 2026 04:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants