docs(local-models): add vllm hidden-state serving notes

[sabhatinas]

Summary

• Add internal local vLLM serving notes for hidden-state extraction.
• Document both the Docker launch path and the equivalent direct vllm serve command.
• Include a one-request safetensors smoke test and local serving troubleshooting notes.

Validation

• uv run --only-group docs mkdocs build --strict

Summary by CodeRabbit

• Documentation
  • Added a new guide for serving a local vLLM model with hidden-state extraction enabled.
  • Included setup options for Docker and direct CLI usage, plus example launch commands.
  • Added steps to verify output files and inspect saved hidden states, along with troubleshooting tips.
  • Updated the documentation navigation to point to the new guide.

[github-actions]

PR Preview Action v1.8.1
🚀 View preview at https://NVIDIA-NeMo.github.io/Switchyard/pr-preview/pr-13/
Built to branch gh-pages at 2026-07-22 23:43 UTC. Preview will be ready when the GitHub Pages deployment is complete.

[nv-msvoboda]

@sir-merge-a-lot groom

[nv-sir-merge-a-lot]

🤖 sir-merge-a-lot

Rebase succeeded

Rebased sabhatinas/switch-876-vllm-hidden-state-docs (client-side). Pipeline running on the new HEAD.

Why now: this PR needed to be rebased onto main; server-side rebase was unavailable or failed, so the bot performed a client-side rebase and force-pushed the result.
What I checked: server-side rebase status; client-side rebase preconditions (source_branch_allows_force_push, not a fork, signing not required, fetch succeeded); no merge conflicts emerged; settled platform status: blocked.
What I changed: rebased sabhatinas/switch-876-vllm-hidden-state-docs locally and force-pushed. New HEAD: ab7ed14dd6c1b1369cb51a20bcb08800a730b088.
Recovery: previous tip preserved at refs/sir-merge-a-lot/backup/sabhatinas/switch-876-vllm-hidden-state-docs on the remote; pushed with --force-with-lease=sabhatinas/switch-876-vllm-hidden-state-docs:ab7ed14dd6c1b1369cb51a20bcb08800a730b088. To revert: git push origin refs/sir-merge-a-lot/backup/sabhatinas/switch-876-vllm-hidden-state-docs:refs/heads/sabhatinas/switch-876-vllm-hidden-state-docs.

📖 How rebases run · Force-with-lease and backup refs

🤖 sir-merge-a-lot · why this happened? · pause / opt out · Updated 2026-07-09 18:41:04 UTC

[coderabbitai]

Walkthrough

This PR adds a new documentation page, docs/vllm-serve-hidden-state.md, describing how to serve vLLM with hidden-state extraction enabled, covering Docker and CLI launch methods, output verification, and troubleshooting. The mkdocs.yml nav is updated to reference this new page.

Changes

Hidden-State Serving Docs

Layer / File(s)	Summary
Launch instructions docs/vllm-serve-hidden-state.md	Introduces the workflow's purpose and prerequisites, and documents both Docker-based and direct CLI launch methods for hidden-state extraction, including required flags and mount paths.
Verification and troubleshooting docs/vllm-serve-hidden-state.md	Adds a verification procedure using a sample Chat Completions request and Python snippet to inspect .safetensors output, plus a troubleshooting section listing common failures and fixes.
Navigation update mkdocs.yml	Replaces the local_models.md nav entry with vllm-serve-hidden-state.md.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Poem

A rabbit hops through docs anew,
Hidden states now in plain view,
Safetensors saved, paths well laid,
Docker or CLI, both displayed,
Nav updated, hop hop hooray! 🐇📄

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: new vLLM hidden-state serving documentation under the local-models docs area.

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/vllm-serve-hidden-state.md`:
- Around line 81-97: The smoke test still selects the newest .safetensors file
instead of using the hidden-state path returned by vLLM. Update the Python
snippet in the hidden-state docs to read and open the actual path from
kv_transfer_params.hidden_states_path, and use that value consistently when
printing tensor metadata so the test validates the correct run.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 91b5ffaf-496b-478c-92bd-16fd32ed21cd

📥 Commits

Reviewing files that changed from the base of the PR and between 072ae3e and 1977ff4.

📒 Files selected for processing (2)

• docs/vllm-serve-hidden-state.md
• mkdocs.yml