MODEL-NEMOTRON-H-ABI-A2P's A3 token gate now reads TOKEN MATCH: 96/96 over 3 prompt(s) (full rows=3, short rows=0, mode=decode) STRICT PASS on GB10 sm_121a under the #1157 fix (PR #1221), against the released nemotron-3.5-lightning-30b-nvfp4 at revision 29f2d1746d8f41e316523194b19018707749b1b1, with the fix reverted reading 4/24 on the same binary. Correctness is therefore established on the device, and no speed number exists for this model on any axis.
docs/BENCHMARKS.md carries the row as "No speed number, by the unit's own rule". This issue owns producing the first ones: a same-workload comparison against the pinned vLLM oracle (vllm_commit = 5559679229bc961848b121ccdeaa8fa5d79bec98) on dgx:gpu0, inside an rc lease, on the identical checkpoint, prompts, token counts, batching, concurrency and sampling, with vLLM in its production configuration as the denominator and never --enforce-eager.
The gate run already says where the number will land and why, and the causes are named rather than guessed:
The gate legs measured 264.4 s engine load and 342.61 / 328.19 / 327.51 s for 32 tokens each, i.e. about 10.3 s per output token at batch 1. That is a measurement to record honestly with its causes, not a ceiling.
Two comparability facts this issue has to control for. --gpu-memory-utilization 0.92 does NOT size our KV pool -- the profile run is unimplemented (#83) and the pool falls back to 256 blocks -- so the pool is sized explicitly on our side and the oracle's KV budget is matched to it, or the memory axis is not like-for-like. And the oracle side is unproven for a MODEL RUN inside a lease: #1185 established build, install, import and torch.cuda.is_available() only, and the recorded failure mode of the step after torch.compile on this host is a reboot of the box. If the oracle cannot run, the numerator-only figures are recorded as ungated and never as a ratio.
MODEL-NEMOTRON-H-ABI-A2P's A3 token gate now readsTOKEN MATCH: 96/96 over 3 prompt(s) (full rows=3, short rows=0, mode=decode)STRICT PASSon GB10sm_121aunder the #1157 fix (PR #1221), against the releasednemotron-3.5-lightning-30b-nvfp4at revision29f2d1746d8f41e316523194b19018707749b1b1, with the fix reverted reading 4/24 on the same binary. Correctness is therefore established on the device, and no speed number exists for this model on any axis.docs/BENCHMARKS.mdcarries the row as "No speed number, by the unit's own rule". This issue owns producing the first ones: a same-workload comparison against the pinned vLLM oracle (vllm_commit = 5559679229bc961848b121ccdeaa8fa5d79bec98) ondgx:gpu0, inside anrclease, on the identical checkpoint, prompts, token counts, batching, concurrency and sampling, with vLLM in its production configuration as the denominator and never--enforce-eager.The gate run already says where the number will land and why, and the causes are named rather than guessed:
nemotron_h.cpp:1031-1034refuses the NVFP4lm_headon a non-CPU queue, so the last step of every forward is a HOST projection. Owned by A2-Q2b, not landed.in_proj/out_projprojections still execute host-side, 36.6% of decode bytes and 27.6% of GEMM FLOPs. Owned by A2-Q1, The FP8 W8A8 linear path is not a shared seam: residency and GEMM entry points live inside qwen3_5.cpp, so a second model cannot reach them without hand-rolling a parallel path #940, not landed.The gate legs measured 264.4 s engine load and 342.61 / 328.19 / 327.51 s for 32 tokens each, i.e. about 10.3 s per output token at batch 1. That is a measurement to record honestly with its causes, not a ceiling.
Two comparability facts this issue has to control for.
--gpu-memory-utilization 0.92does NOT size our KV pool -- the profile run is unimplemented (#83) and the pool falls back to 256 blocks -- so the pool is sized explicitly on our side and the oracle's KV budget is matched to it, or the memory axis is not like-for-like. And the oracle side is unproven for a MODEL RUN inside a lease: #1185 established build, install, import andtorch.cuda.is_available()only, and the recorded failure mode of the step aftertorch.compileon this host is a reboot of the box. If the oracle cannot run, the numerator-only figures are recorded as ungated and never as a ratio.