gate(ROCM): M4 -- pinned vLLM-ROCm oracle on gfx1100 + ROCm device-golden lane in the Qwen3-0.6B near-tie gate by VikashLoomba · Pull Request #549 · mudler/vllm.cpp

VikashLoomba · 2026-08-12T23:25:29Z

M4 on gfx1100 — the pinned vLLM-ROCm oracle + the ROCm near-tie gate lane (issue #41)

This PR lands the M4 milestone for BACKEND-ROCM: a pinned vLLM-ROCm oracle built on the same box (the maintainer's assignment — this is the only board that can host one) and the ROCm arm of the existing device-golden lane in the Qwen3-0.6B near-tie gate.

The oracle

Upstream vLLM at the parity pin 555967922, runtime identity 0.23.1rc1.dev1511+g555967922 (verified == .agents/upstream-sync.md parity-pin block)
Built from source as a ROCm wheel for gfx1100 in rocm/vllm-dev:base (2,696 files; _C/_rocm_C/_moe_C ABI extensions present)
Committed image vllm-rocm-oracle:555967922-gfx1100; full recipe in .agents/specs/rocm-m4-oracle.md
K=10 per-prompt greedy capture: deterministic in every (prompt,pos) cell — a well-posed strict gate on this board

The gate result (Qwen3-0.6B, 4x RX 7900 XTX, ROCm 7.14)

correctness gate: 16/16 prompts PASS  (STRICT token-exact: 11/16;
near-tie-band only: 5/16; max gap 0.125 nats @ prompt[3] tok=1;
0 forward-divergent) — 125/125 assertions
BACKEND PROOF — Qwen3-dense ops on device type 5 with 0 declines
(kPagedAttention selections=7168)

All 28 token-divergent positions vs the oracle's own greedy are inside the 0.5-nat band. The known p0 France/Italy flip is a literal 0.0000-nat tie in the oracle's own logits — the Metal M3b signature, reproduced on ROCm: vLLM's teacher-forced argmax on our exact prefix IS 15344 (" Italy").

Change shape (additive only)

tests/parity/test_qwen3_paged_engine.cpp: the existing Metal/Tenstorrent device-golden branch gains the kROCM arm — identical anchor + near-tie-band logic, ROCm's own oracle-backed golden pair; CUDA/CPU/Metal/TT behaviour byte-unchanged
New goldens: our_ids_rocm.npy, neartie_gap_mnats_rocm.npy (the lane pair) + greedy_ids_rocm.npy/greedy_dist_rocm.npy (oracle capture evidence). dgx base goldens untouched
.agents/specs/rocm-m4-oracle.md: the reproducible recipe
Records: parity-ledger row, owning spec, and STATUS.md. Derived NOW/claim surfaces are intentionally untouched.

Stacked follow-ups

#570 fixes the 0.8B output-dtype dispatch; #559 adds its green oracle gate; #523 reaches the 35B keep-quant production path. Qwen3-4B strict deterministic gating still needs the checkpoint.

Current-main integration cutoff (2026-08-18)

Rebased from 17892e7ab7e111a36d70e050bd3b74d984dbc7d3 to 822f4bdece31eedecf71a5fc0fc571e79faa3037 on recorded current-main cutoff 65d6cdaed3e20e9bc70b4f9374fccafefefa7bd0. Exact M4 0.6B commit replay. Fresh cutoff-head builds, focused gates, full repository preflight, and independent mutation review all pass.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: pi:kimi-k3 [pi]
Assisted-by: codex:gpt-5.6-sol [codex]
Assisted-by: pi:gpt-5.6-sol [pi]

VikashLoomba · 2026-08-18T06:42:59Z

Current-main rebase and fresh review

Rebased from c3edd8bb to ce9ab4ff on current-main pin f22c6cc8 and force-updated with an explicit lease.

Range-diff: exact M4 0.6B patch replay.
Qwen3-0.6B gate ran on ROCm, not skipped: 16/16 prompts, 125/125 assertions; 11 strict, 5 ratified near-tie, maximum 0.125 nats, zero forward-divergent.
M4 topology is exact: f22c6cc8 → this PR → fix(ROCM): AttnQkNormRopeGate output-dtype dispatch -- the 0.8B GDN forward divergence root-caused and fixed (RED anchor turns GREEN) #570 → gate(ROCM): M4 for Qwen3.5-0.8B GDN -- oracle-backed green near-tie gate #559.
Derived .agents/NOW.md and .agents/coordination.md are untouched; current-main head_dim=128 status remains.
Repository preflight: All gates green.
Fresh independent integration review: PASS; final tree clean.

…lden lane in the Qwen3-0.6B near-tie gate Issue mudler#41 M4 milestone. The oracle is upstream vLLM AT THE PIN (555967922, runtime 0.23.1rc1.dev1511+g555967922 -- verified == the parity-pin block) built from source as a ROCm wheel for gfx1100 in rocm/vllm-dev:base and committed as the image vllm-rocm-oracle:555967922-gfx1100 (recipe: .agents/specs/rocm-m4-oracle.md). Gate result on this box (4x RX 7900 XTX, ROCm 7.14, enforce_eager, batch=1, flock ~/gpu.lock): oracle K=10 per-prompt greedy deterministic in every cell; our ROCm engine 16/16 prompts PASS -- 11/16 strict token-exact vs base greedy, 5/16 near-tie band, max teacher-forced gap 0.125 nats, 0 forward-divergent, 125/125 assertions, backend proof all 8 Qwen3-dense ops selections>0 and declines==0 on device 5. The p0 France/Italy flip is a literal 0.0000-nat tie in the oracle's own logits (the Metal M3b signature, reproduced on ROCm). Change shape: the parity gate's Metal/Tenstorrent device-golden lane gains the kROCM arm (identical anchor+band logic, ROCm's own oracle-backed golden pair); dgx base goldens byte-untouched; the *_rocm goldens are additive. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: pi:kimi-k3 [pi]

VikashLoomba · 2026-08-18T12:24:13Z

Final integration cutoff

Final head: 822f4bdece31eedecf71a5fc0fc571e79faa3037, exact replay on cutoff 65d6cdae.

Qwen3-0.6B gate ran on ROCm through ROCM_ATTN: 16/16 prompts, 125 assertions; 11 strict, 5 ratified near-tie, max 0.125 nats, zero forward-divergent.
Final topology is cutoff → this PR → fix(ROCM): AttnQkNormRopeGate output-dtype dispatch -- the 0.8B GDN forward divergence root-caused and fixed (RED anchor turns GREEN) #570 → gate(ROCM): M4 for Qwen3.5-0.8B GDN -- oracle-backed green near-tie gate #559.
Full preflight and independent cutoff review: PASS.

… gate (#559) ## M4 for Qwen3.5-0.8B (GDN) — the oracle-backed gate, landed green-shaped (issue #41) Stacked: #549 (0.6B lane + oracle) ← **the dispatch fix** (#570) ← this gate. The fix lands *below* the gate, so every commit in the stack is individually green on the gate host and the committed goldens are ones the tree can actually reproduce. ### Gate `test_qwen35_paged_engine.cpp` — the GDN sibling of the Qwen3-dense SACRED gate: standard 16-prompt battery through the full paged engine, hard anchor + ≤0.5-nat near-tie band, backend proof over the 15-op GDN set. Goldens captured from the pinned vLLM-ROCm oracle on this box (K=10 per-prompt, **deterministic in every cell**) with `manifest.json` recording the oracle identity; the checkpoint resolves via `parity::HfSnapshot` at the pinned HF revision `2fc06364` (check-snapshot-pins clean). **Verdict: 16/16 PASS** (15/16 strict token-exact vs the oracle, 1/16 near-tie band, max gap 0.125 nats, 0 forward-divergent). ### Review-sweep rework (all three findings accepted) 1. **CI-red manifest fixed**: `qwen35_0_8b_greedy` is in `PendingRunnerOps()` (the `qwen36_gguf_greedy` precedent) — the manifest itself stays; its oracle identity is the point. 2. **Fail-safe by device**: the gate exits 77 (CTest Skipped) on any non-ROCm device and when the pinned checkpoint is absent — never a false green, never a foreign engine compared against ROCm-derived goldens. 3. **Green-shaped landing**: the committed goldens are the FIXED engine's oracle-re-derived pair. The pre-fix capture (13/16 forward-divergent; first-token divergences on 6 prompts at 0.375–1.062 nats, 5 of 6 over band) is kept as evidence in the spec + parity ledger, not as goldens no code can pass. The anchor-drift REQUIRE now reads "REGRESSION SUSPECTED — bisect first"; golden re-derivation is the last step of a justified re-capture, not the response to a failure. ### Prose corrections from review First-divergence maximum is 1.062 nats (not 1.188), 5 of the 6 token-0 divergences are over the band (not all), and the headline gap is at the first over-band position rather than a post-divergence maximum. ## In-flow gate repair Issue #1222 found three missing-artifact paths that returned normally and let doctest report false success. The gate now routes absent greedy, anchor, and gap artifacts through exit 77. A no-GPU subprocess regression drives the real executable and asserts `77/77/77`; its complete-artifact control requires hard sentinel exit 86 so probe mode can never bypass snapshot/model/device checks as zero-assertion success. Bootstrap writes its token IDs and then exits skipped until correctness artifacts exist. Malformed present arrays remain hard failures. Fixes #1222. ## Current-main integration cutoff (2026-08-18) Rebased from `5b31bc30349d527ef0b4c1c8fcd7c86749b4cfa3` to `4673a35dd47087cae3f6f9808e4fcf64667bcbcf` on recorded current-main cutoff `65d6cdaed3e20e9bc70b4f9374fccafefefa7bd0`. Exact six-commit replay of #549 → #570 → the green 0.8B gate → the three #1222 spec/repair commits. Fresh cutoff-head builds, focused gates, full repository preflight, and independent mutation review all pass. ## Landing note (maintainer) This branch is the tip of the 0.6B/0.8B ROCm stack and carries six commits: the #549 M4 oracle and 0.6B gate lane, the #570 `AttnQkNormRopeGate` output-dtype dispatch fix, the green 0.8B GDN gate, and the three #1222 gate repairs. Landing it lands that whole stack, so #549 and #570 are closed against this merge rather than squashed separately. Merging the tip is deliberate: squash-merging a stack in order rewrites each ancestor's history and destroys the merge base of the branches above it. `agent-record` is red on the pull request and the merge result is green. The branch predates the #1229 gate-commands repair now on `main`, so it ran the 41-test version of `tests/scripts/test_check_gate_commands.py` against its own tree. The actual merge was constructed locally and gated: 42 tests OK, and `check-agent-record`, `check-doc-checkpoint`, `check-issue-index-append-only` and `check-readme-structure` all pass on the merged tree. `windows-msvc-cpu` and `windows-msvc-vulkan` are red as they are on every pull request here and carry no verdict. The remaining 17 gates passed. The ROCm correctness evidence is the contributor's and was not re-run by the maintainer: it needs the gfx1100 board, which this session did not hold. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: pi:kimi-k3 [pi] Assisted-by: codex:gpt-5.6-sol [codex] Assisted-by: pi:gpt-5.6-sol [pi]

localai-bot · 2026-08-18T19:58:12Z

Landed via #559 as 7b89cf307, not superseded.

#559 is the tip of this stack and carried this branch's commit 822f4bdec (the M4 oracle plus the 0.6B device-golden lane) verbatim. Verified on main: tests/parity/goldens/qwen3_greedy_0_6b/ carries greedy_ids_rocm.npy, greedy_dist_rocm.npy, neartie_gap_mnats_rocm.npy and our_ids_rocm.npy.

The tip was merged rather than each branch in turn because this repository is squash-only, and squash-merging a stack in order rewrites each ancestor and destroys the merge base of the branches above it. Authorship is preserved on the landed commit. Thanks for the oracle work, and for building the stack so each commit was individually green on the gate host.

This was referenced Aug 12, 2026

ROCm (AMD GPU) backend #41

Open

gate(ROCM): M4 for Qwen3.5-0.8B GDN -- oracle-backed green near-tie gate #559

Merged

VikashLoomba force-pushed the row/ROCM-M4-GATE branch from bc1e825 to 1b33265 Compare August 13, 2026 02:12

VikashLoomba mentioned this pull request Aug 13, 2026

fix(ROCM): AttnQkNormRopeGate output-dtype dispatch -- the 0.8B GDN forward divergence root-caused and fixed (RED anchor turns GREEN) #570

Closed

VikashLoomba force-pushed the row/ROCM-M4-GATE branch 5 times, most recently from c3edd8b to ce9ab4f Compare August 18, 2026 06:41

VikashLoomba marked this pull request as ready for review August 18, 2026 06:43

VikashLoomba force-pushed the row/ROCM-M4-GATE branch from ce9ab4f to 822f4bd Compare August 18, 2026 12:22

localai-bot closed this Aug 18, 2026

localai-bot mentioned this pull request Aug 18, 2026

feat(rocm): grouped + non-grouped keep-quant expert GEMM — the MoE-path terminus (issue #41) #523

Open

1 task

Navigation Menu

gate(ROCM): M4 -- pinned vLLM-ROCm oracle on gfx1100 + ROCm device-golden lane in the Qwen3-0.6B near-tie gate - #549

gate(ROCM): M4 -- pinned vLLM-ROCm oracle on gfx1100 + ROCm device-golden lane in the Qwen3-0.6B near-tie gate#549
VikashLoomba wants to merge 1 commit into
mudler:mainfrom
VikashLoomba:row/ROCM-M4-GATE

VikashLoomba commented Aug 12, 2026 •

edited

Loading

Uh oh!

VikashLoomba commented Aug 18, 2026

Uh oh!

VikashLoomba commented Aug 18, 2026

Uh oh!

localai-bot commented Aug 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

VikashLoomba commented Aug 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

M4 on gfx1100 — the pinned vLLM-ROCm oracle + the ROCm near-tie gate lane (issue #41)

The oracle

The gate result (Qwen3-0.6B, 4x RX 7900 XTX, ROCm 7.14)

Change shape (additive only)

Stacked follow-ups

Current-main integration cutoff (2026-08-18)

Uh oh!

VikashLoomba commented Aug 18, 2026

Current-main rebase and fresh review

Uh oh!

VikashLoomba commented Aug 18, 2026

Final integration cutoff

Uh oh!

localai-bot commented Aug 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

VikashLoomba commented Aug 12, 2026 •

edited

Loading