[executorch][cuda] Lower off-graph KV storage through AOTI

[Gasoonjia]

Stack from ghstack (oldest at bottom):

• [executorch][cuda] Isolate off-graph KV cache by session #22820
• [executorch][cuda] Recapture off-graph KV decode graphs #22819
• [executorch][muse] Wire dynamic KV cache into the runtime #22818
• [executorch][muse] Add dynamic off-graph KV export #22817
• [executorch][cuda] Add dynamic off-graph KV runtime manager #22816
• -> [executorch][cuda] Lower off-graph KV storage through AOTI #22815
• [executorch][cuda] Add off-graph KV Triton primitives #22814

Lower the neutral kvcache::update_and_attend contract to runtime-bound private tensors, retain their logical metadata without serializing payloads, and allow the weight loader to defer those bindings. Add validation for manifests and storage omission. Authored with Codex.

Differential Revision: D119571985

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22815

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• B200 dgxb200-03 in bad state

❌ 3 Unclassified Failures

As of commit 9bea6b3 with merge base e37af23 ():

UNCLASSIFIED FAILURES - DrCI could not classify the following jobs because the workflow did not run on the merge base. The failures may be pre-existing on trunk or introduced by this PR:

• Test CUDA Builds / test-model-cuda-e2e-Qwen3.5-35B-A3B-HQQ-INT4-quantized-int4-tile-packed / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• Test CUDA Builds / unittest-cuda / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  backends/cuda/tests/test_offgraph_kv.py::OffGraphKVTest::test_flat_split_k_decode_matches_reference
• Test ROCm AOTI / unittest-rocm-gfx950-rocm7.2 / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  backends/cuda/tests/test_offgraph_kv.py::OffGraphKVTest::test_flat_split_k_decode_matches_reference

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.