Skip to content

feat(cli): actionable errors for missing models and credentials (phase 1 of #3442)#3452

Merged
dgageot merged 1 commit into
mainfrom
feat/3442-actionable-model-credential-errors
Jul 6, 2026
Merged

feat(cli): actionable errors for missing models and credentials (phase 1 of #3442)#3452
dgageot merged 1 commit into
mainfrom
feat/3442-actionable-model-credential-errors

Conversation

@Sayt-0

@Sayt-0 Sayt-0 commented Jul 3, 2026

Copy link
Copy Markdown
Member

Part of #3442 (Phase 1: actionable errors). Phases 2-4 (doctor, guided setup, docs tutorial) are independently shippable and left to follow-up PRs, as laid out in the issue.

Changes mapped to the issue scenarios

# Scenario Before After
3 Bare docker agent, non-TTY, empty stdin (CI) exit 0, zero output error naming the next steps (pass a message, pipe one, or use a terminal), exit 1; piped prompts keep working
2 DMR endpoint reachable, docker model CLI broken, model not pulled TUI starts, first message fails with raw HTTP 404: POST .../chat/completions fails at client creation: "model X is not pulled in Docker Model Runner" + docker model pull X
1/2 "Not installed" detection exact string match on the full docker usage text content match on unknown flag: --json, shared between NewClient and ListModels
4 Missing model credentials (RequiredEnvError, first_available variant) "use one of the built-in environment variable providers" (never named) the secret sources able to hold arbitrary keys are named with a one-line example each (export, --env-from-file, pass, macOS Keychain), plus the docs URL and the local-model alternative (only when the missing vars are model credentials)
6 run --env-from-file typo.env missing/malformed file logged via slog (invisible without --debug), run continues run aborts: --env-from-file: open typo.env: no such file or directory; parse errors name the file and expected format
5 Exec-mode failure printed twice (CLI error printer + cobra Error: ...) printed once by cobra, on stderr

Sample output (missing model credentials)

Error: The following environment variables must be set:
 - ANTHROPIC_API_KEY

Provide them using any of these sources:
 - Shell environment:  export ANTHROPIC_API_KEY=<value>
 - Env file:           docker agent run --env-from-file <file> ...
 - pass:               pass insert ANTHROPIC_API_KEY
 - macOS Keychain:     security add-generic-password -a "$USER" -s ANTHROPIC_API_KEY -w

See https://docs.docker.com/ai/docker-agent/guides/secrets/ for details.

No API key? Run a local model instead: docker agent run --model dmr/ai/qwen3 ...
(the model is pulled on first use; `docker model ls` shows models already pulled)

Message coherence pass

Follow-up to the review comment about message consistency. Every claim made by the new messages is now backed by the code path it describes:

Message Problem Fix
ModelNotAvailableError "is not available" in the full message but "is not pulled" in the summary shown under AutoModelFallbackError; final bullet diverged from PullFailedError's both say "is not pulled in Docker Model Runner"; final bullet identical to PullFailedError's
Secret sources help suggested Docker Desktop and the credential helper, but DockerDesktopProvider and CredentialHelperProvider only resolve DOCKER_EMAIL/DOCKER_USERNAME/DOCKER_TOKEN and can never satisfy the reported missing keys both lines removed; a NotContains test locks the exclusion
Local-model hint claimed pulled models are "detected automatically by the default auto model" (the built-in agent uses first_available, and pinned agentfile models are never re-selected) and pointed at docker agent models for local models (that command deliberately skips DMR discovery and lists the catalog) now: "the model is pulled on first use; docker model ls shows models already pulled"
Empty-stdin errors the two sibling messages introduced their next steps with different separators aligned on ": "

Pre-existing divergences left out of this PR, candidates for follow-ups:

  • docs/guides/secrets/ presents Docker Desktop and the credential helper as generic secret sources (same conflation as above, docs side)
  • "To fix this, you can:" (auto.go) vs "To resolve this, you can:" (pull.go, available.go)
  • ErrNotInstalled lowercase "docker model runner" branding
  • lowercase fallback branch "no first_available candidate has credentials configured (tried: ...)" vs the capitalized main branch

Implementation notes

docker agent run (dmr model, no BaseURL / MODEL_RUNNER_HOST)
  |- docker model status ok        -> inspect + pull prompt (unchanged)
  |- status fails, "unknown flag"  -> ErrNotInstalled (content match, was exact match)
  `- status fails otherwise        -> GET <dmr>/models at client creation
       |- model listed             -> proceed
       |- model absent             -> ModelNotAvailableError ("docker model pull X")
       `- endpoint unqueryable     -> "cannot query Docker Model Runner at <url>" + install link
  • ModelNotAvailableError passes through the teamloader unwrapped (same treatment as PullFailedError) and exposes a one-line summary when nested under AutoModelFallbackError, so guidance blocks do not stack.
  • The non-TTY check lives at input time in cli.Run rather than at bare-command dispatch: a dispatch-level isatty(stdout) check would break the working echo "prompt" | docker agent > out.txt flow. The acceptance criterion (actionable error, exit != 0, never a silent no-op) is met either way; if the dispatch-level check seems preferable, it can be changed.
  • --env-from-file validation reuses the provider chain built by RuntimeConfig.EnvProvider() (new EnvFilesError() accessor) and aborts in the shared command pre-run, so every command taking the flag is covered.
  • RequiredEnvError.MissingModelCredentials is computed in gatherMissingEnvVars so the local-model hint is not shown for missing tool secrets (e.g. GITHUB_PERSONAL_ACCESS_TOKEN).

Testing

Area Coverage
pkg/environment message content for both flag states, sources that cannot hold arbitrary keys are excluded, parse error names the file
pkg/model/provider/dmr errIndicatesNotInstalled, modelAvailable (tag/repo matching), checkModelAvailable via httptest (pulled / not pulled / unqueryable), NewClient shim test with variant docker usage text
pkg/cli empty stdin (bare and -) fails, piped stdin still works, ErrorEvent returned once and not printed
pkg/config EnvFilesError (missing, malformed, valid, clone), MissingModelCredentials wiring, updated first_available assertions
cmd/root pre-run aborts on missing and malformed --env-from-file

All scenarios also verified end to end against a built binary (including a live reproduction of scenario 2: reachable DMR, broken docker model CLI, not-pulled model). The message coherence pass only changes user-facing strings; the updated unit tests assert the new content. Existing e2e expectations (environment variables must be set, OPENAI_API_KEY on stderr) are preserved. Remaining suite failures (TestURLSource_Read_RejectsLocalAddresses, SSRF tests, TestLoadExamples DMR pull) reproduce on main and are environment-related.

@Sayt-0 Sayt-0 requested a review from a team as a code owner July 3, 2026 14:58
@aheritier aheritier added kind/feat PR adds a new feature (maps to feat:). Use on PRs only. area/cli CLI commands, flags, output formatting area/config For configuration parsing, YAML, environment variables area/providers/docker-model-runner Docker Model Runner (DMR) local inference labels Jul 3, 2026
@dgageot

dgageot commented Jul 3, 2026

Copy link
Copy Markdown
Member

@Sayt-0 we need to review some of the messages

Phase 1 of #3442:

- Bare `docker agent` with empty non-TTY stdin fails with next steps
  instead of exiting 0 silently; piped prompts keep working
- DMR: "not installed" detection no longer depends on the exact docker
  CLI usage text; when `docker model status` fails, model availability
  is verified through the DMR HTTP API at client creation so a
  not-pulled model reports "docker model pull X" instead of a raw
  HTTP 404 at message time
- RequiredEnvError and the first_available variant name the secret
  sources able to supply arbitrary keys (shell environment, env file,
  pass, macOS Keychain) with a one-line example each and a link to
  the secrets guide; Docker Desktop and the credential helper are
  deliberately absent, they only resolve fixed Docker variables
  (DOCKER_TOKEN) and can never satisfy the reported missing keys
- When model credentials are missing, the error suggests running a
  local model instead (--model dmr/ai/qwen3, pulled on first use,
  `docker model ls` lists models already pulled)
- --env-from-file with an unreadable or malformed file aborts the run
  instead of being logged and skipped; parse errors name the file
- Exec mode prints the final error once (cobra), not twice
@Sayt-0 Sayt-0 force-pushed the feat/3442-actionable-model-credential-errors branch from f5243ab to 549b076 Compare July 6, 2026 16:22
@dgageot dgageot merged commit 0caaa26 into main Jul 6, 2026
12 checks passed
@dgageot dgageot deleted the feat/3442-actionable-model-credential-errors branch July 6, 2026 16:41
pull Bot pushed a commit to TheTechOddBug/cagent that referenced this pull request Jul 7, 2026
…and credentials

Adds a troubleshooting section covering the new error messages fired when
model credentials are absent or DMR is unreachable, directing users to
`docker agent doctor` for diagnosis.

Source: docker#3452
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/cli CLI commands, flags, output formatting area/config For configuration parsing, YAML, environment variables area/providers/docker-model-runner Docker Model Runner (DMR) local inference kind/feat PR adds a new feature (maps to feat:). Use on PRs only.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants