AIR CLI Integration: fast air list via active run index #5814
Merged
Conversation
riddhibhagwat-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
riddhibhagwat-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
Collaborator
Integration test reportCommit: cb3b2f8
8 interesting tests: 4 RECOVERED, 4 SKIP
Top 20 slowest tests (at least 2 minutes):
|
vinchenzo-db
approved these changes
Jul 6, 2026
vinchenzo-db
left a comment
vinchenzo-db
left a comment
There was a problem hiding this comment.
generally lgtm, make sure u understand how are list works because it's a little non-trivial now.
| if err != nil || id == 0 { | ||
| continue | ||
| } | ||
| refs = append(refs, workflowRef{jobRunID: id, submitTimeMs: parseSubmitTimeMs(wf.SubmitTime)}) |
There was a problem hiding this comment.
Please make sure you de-duplicate job_run_ids... wouldn't want list to return 20 runs, but 2 are duplicates so user only gets 19.
…nal-run cache Ports the Python AIR CLI's fast `air list`. Three changes, matching Python: - Default to active-only; add `--all-status` (replaces `--active`). Plain `air list` no longer scans every run of every state, which was the main reason it lagged the Python CLI. - For `--all-status` scoped to yourself, fetch cheap (job_run_id, submit_time) pairs from the AiTrainingService index (/api/2.0/ai-training/workflows), order by submit time, keep the newest --limit, and hydrate only those via Jobs runs/get (concurrent, per-run ACL-enforced: 403/404 drops the id, other errors propagate). If the index is unavailable, silently fall back to the Jobs scan. --all-users and other-user filters always use the scan. - Cache hydrated terminal runs on disk (libs/cache, 60-day TTL) so repeat --all-status calls skip runs/get + get-output + MLflow for those ids. The runFetcher now wraps a listStrategy (jobsScanStrategy | indexStrategy) behind the same next(want)/exhausted contract, so the TUI, JSON, and one-shot paths are unchanged. Also converts listModel's fetch helpers to value receivers (fixes a recvcheck lint and a stale-`loading` guard bug). Co-authored-by: Isaac
The index can return the same job_run_id on more than one page. Without id-level dedup the newest-`limit` truncation counts duplicates, so `air list --limit 20` could yield fewer than 20 unique runs. Track seen ids and skip repeats (separate from the existing page-token loop guard). Co-authored-by: Isaac
riddhibhagwat-db
force-pushed
the
air-list-fast
branch
from
7a64fc1 to
cb3b2f8
Compare
riddhibhagwat-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
riddhibhagwat-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Default to active-only; add --all-status (replaces --active). Plain air list previously scanned every run of every state through Jobs runs/list. It now lists only active runs by default;
--all-statusopts into all states.AiTrainingService index fast path for --all-status scoped to yourself. Instead of scanning the Jobs firehose, it fetches cheap (job_run_id, submit_time) pairs from GET /api/2.0/ai-training/workflows, orders by submit time, keeps the newest --limit, and surfaces only those via concurrent Jobs runs/get. If the index is unavailable, it silently falls back to the Jobs scan so the command never hard-fails. --all-users and other-user filters always use the scan (the index is per-user only).
Terminal runs are immutable, so once hydrated, their row is cached; repeat
--all-statuscalls skip runs/get + get-output + MLflow for those ids.The runFetcher now wraps a listStrategy (jobsScanStrategy | indexStrategy) behind the same next(want)/exhausted contract, so the interactive table, JSON, and one-shot output paths are unchanged.
This PR also fixes a pre-existing recvcheck lint failure and a latent stale-loading guard bug in list_tui.go (fetch helpers converted to value receivers). The index path over-fetches (skips the newest-N truncation) when a --filter on task fields is active, so a filtered-out run can't shrink the result below --limit.
Why
air list in the Go CLI was noticeably slower than the Python AIR CLI — in both plain and --limit modes. The Python CLI's speed comes from three architectural choices, not a faster scan; this PR ports all three to reach parity.
Testing