feat(agents): Support fileset configuration in optimize agent job

[steramae-nvidia]

Signed-off-by: Sean Teramae steramae@nvidia.com

Summary by CodeRabbit

• New Features
  • Optimize agent jobs can now source the optimization YAML and related inputs from a platform fileset via optimize_config_fileset.
  • Output destination now supports clearer output semantics, including fileset-based output staging.
• Bug Fixes
  • Improved validation for optimize_config and destination references, including safer relative-path handling.
  • Optimization output is only uploaded on successful completion; failures skip upload.
  • Endpoint handling for the underlying optimizer is now applied only in the explicit endpoint mode.
• Documentation
  • OpenAPI schema descriptions updated for optimize_config, optimize_config_fileset, and output.
• Tests
  • Expanded unit coverage for spec validation and staged download/upload behaviors.

[steramae-nvidia]

This change is part of the following stack:

• feat(agents): Support fileset configuration in optimize agent job #822 ◀

_{Change managed by git-spice.}

[coderabbitai]

📝 Walkthrough

Walkthrough

Optimize agent jobs now support optimization YAML and output artifacts through local paths or platform filesets. Configuration staging, validation, temporary injection, subprocess execution, and success-only output uploads use the resolved paths.

Changes

Optimize fileset staging and execution

Layer / File(s)	Summary
Configuration contract and compilation plugins/nemo-agents/src/nemo_agents_plugin/jobs/optimize_agent.py, plugins/nemo-agents/openapi/openapi.yaml, plugins/nemo-agents/tests/unit/test_optimize_agent_job.py	OptimizeAgentSpec adds and documents fileset-backed configuration and output fields, validates references, and permits relative configuration paths when a fileset is configured.
Shared configuration and output staging plugins/nemo-agents/src/nemo_agents_plugin/jobs/fileset_io.py, plugins/nemo-agents/tests/unit/test_fileset_io.py	Shared helpers preserve local paths, download staged configuration, reject invalid or escaping paths, and upload fileset outputs only on successful completion.
Resolved optimization execution and validation plugins/nemo-agents/src/nemo_agents_plugin/jobs/optimize_agent.py, plugins/nemo-agents/tests/unit/test_optimize_agent_job.py	run uses resolved configuration and output paths for validation, temporary injection, nat optimize, and failure handling; tests cover staging, uploads, and skipped uploads on subprocess failure.

Sequence Diagram(s)

sequenceDiagram
  participant OptimizeAgentJob
  participant resolve_staged_config
  participant sdk.files.download
  participant resolve_output
  participant sdk.files.upload
  OptimizeAgentJob->>resolve_staged_config: resolve optimize YAML
  resolve_staged_config->>sdk.files.download: download configuration fileset
  sdk.files.download-->>resolve_staged_config: staged local files
  OptimizeAgentJob->>resolve_output: create output staging directory
  resolve_output->>sdk.files.upload: upload output fileset after success

Loading

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 26.67% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: adding fileset configuration support to the optimize agent job.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch steramae/optimize-fileset

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@plugins/nemo-agents/src/nemo_agents_plugin/jobs/optimize_agent.py`:
- Around line 172-173: Update the compilation validation around
spec.optimize_config_fileset to reject absolute optimize_config paths when a
fileset is configured: check Path(spec.optimize_config).is_absolute() and fail
compilation, while preserving the existing _require_absolute behavior for
non-fileset configurations.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 93a482f1-9ed5-4054-bbed-924bbf72fcc9

📥 Commits

Reviewing files that changed from the base of the PR and between 732b1cd and e50aa40.

📒 Files selected for processing (2)

• plugins/nemo-agents/src/nemo_agents_plugin/jobs/optimize_agent.py
• plugins/nemo-agents/tests/unit/test_optimize_agent_job.py

[github-actions]

Suite	Lines Covered	Line Rate	Branch Rate
Unit Tests	27123/34832	77.9%	62.1%
Integration Tests	15921/33544	47.5%	19.9%

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@plugins/nemo-agents/src/nemo_agents_plugin/jobs/optimize_agent.py`:
- Around line 336-338: Update the logging around the optimization command in the
relevant job function to avoid emitting the raw endpoint, including both the
“Optimizing against agent endpoint” and “Running” messages. Sanitize or omit
endpoint-derived values before passing them to logger.info, while preserving
useful non-sensitive execution context such as the command and working
directory.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 04c83515-8f4b-49ec-ad9c-1773b0b5be38

📥 Commits

Reviewing files that changed from the base of the PR and between 48969ba and d9b852c.

📒 Files selected for processing (3)

• plugins/nemo-agents/openapi/openapi.yaml
• plugins/nemo-agents/src/nemo_agents_plugin/jobs/optimize_agent.py
• plugins/nemo-agents/tests/unit/test_optimize_agent_job.py

🚧 Files skipped from review as they are similar to previous changes (2)

• plugins/nemo-agents/openapi/openapi.yaml
• plugins/nemo-agents/tests/unit/test_optimize_agent_job.py