Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .agents/claims/CLAIM-SPEC-DSPARK-QWEN3-ROUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# CLAIM-SPEC-DSPARK-QWEN3-ROUTING

| Claim | Row IDs | Agent | Worktree / remote dir | Branch | Owned scope | State | Last update |
|---|---|---|---|---|---|---|---|
| `CLAIM-SPEC-DSPARK-QWEN3-ROUTING` | `SPEC-DSPARK-QWEN3-ROUTING` (`ACTIVE`) | Claude Code (claude-opus-5), fresh implementer repairing a fresh review | worktree `dspark-qwen3-routing`, CPU only (no `nvcc`, no GPU lease) | `row/DSPARK-QWEN3-ROUTING-IMPL`, issue [#1193](https://github.com/mudler/vllm.cpp/issues/1193), spec [dspark-qwen3-routing.md](../specs/dspark-qwen3-routing.md). The branch name predates the row ID and does NOT match the `row/<ID>` form; it is recorded in the spec's `Role / claim` field rather than renamed, because renaming mid-flight strands the review already anchored to it | Owns ONLY: `SpeculativeConfig::IsDsparkDraft` and `SpeculativeConfig::ResolveDsparkArchitecture` in `include/vllm/config/speculative.h`; `ReadDsparkDraftIdentity` and the dspark branch of `LoadedEngine::ResolveSpecConfig` in `src/vllm/entrypoints/model_loader.cpp`; `tests/vllm/entrypoints/test_dspark_draft_routing.cpp` and the DSpark-routing cases of `tests/vllm/config/test_speculative_dspark.cpp`; this row, this claim file, the spec, and the `DSparkDraftModel` model-matrix row's routing note. **NON-COLLISION:** disjoint from `row/DSPARK-BLOCK-SIZE-GUARD` ([#1225](https://github.com/mudler/vllm.cpp/issues/1225)), which a parallel agent owns and which works on `n_predict` and `dspark_block_size` inside `ResolveDspark`; this claim touches neither key and neither `ResolveDspark` call site. EXCLUDED: the DSpark speculator, the draft weight path, the `--speculative-config` parser, and the second classification call site inside `LoadDsparkDraft` | `ACTIVE` | 2026-08-18 — W1-W4 landed, reviewed once, repaired once. G1 green (`test_speculative_dspark` 12/40, `test_dspark_draft_routing` 7/19), G2/G3 mutation-proven in a scratch copy, G4 green on the branch. Merged `origin/main` `b626be75a` on 2026-08-18, whose hoist of the DSpark resolution (#1225) makes this classification RUN FIRST on the `FromModelDir` path and retires the second call site this claim had excluded. Owed and named in the spec's `## Owed`: G5 (token-exact run gate, needs the 2.53 GiB draft and a GPU lease), G6 (spec-off byte identity), and the reading that would settle the empty-`architectures` narrowing |
6 changes: 3 additions & 3 deletions .agents/engine-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,12 +43,12 @@ forensics: roadmap_v1.md and the parity ledger.
| Parallelism | 6 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 5 |
| Sampling and generation | 15 | 4 | 2 | 0 | 0 | 4 | 0 | 1 | 4 |
| Structured output and tools | 7 | 0 | 4 | 0 | 0 | 2 | 0 | 0 | 1 |
| Speculative decoding | 24 | 0 | 0 | 1 | 1 | 7 | 0 | 4 | 10 |
| Speculative decoding | 24 | 0 | 0 | 1 | 0 | 8 | 0 | 4 | 10 |
| Serving, API, CLI, library | 35 | 10 | 2 | 1 | 3 | 10 | 2 | 3 | 4 |
| LoRA and adapters | 2 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 1 |
| Long context and attention | 10 | 5 | 0 | 0 | 1 | 0 | 1 | 0 | 3 |
| Loading, tokenizer, config | 10 | 3 | 3 | 0 | 0 | 1 | 1 | 1 | 1 |
| **Total** | **163** | **35** | **17** | **5** | **12** | **36** | **8** | **11** | **38** |
| **Total** | **163** | **35** | **17** | **5** | **11** | **37** | **8** | **11** | **38** |

## Engine core and scheduling

Expand Down Expand Up @@ -197,7 +197,7 @@ claims it.
| `SPEC-CUSTOM-CLASS` | Pluggable custom proposer (`method="custom_class"`): the proposer class path is read from `speculative_config.model` and must expose a `propose` method — serves the extensibility-first priority (drop-in third-party speculators). | T2 | `vllm/v1/spec_decode/custom_class_proposer.py:12`; dispatch `vllm/v1/worker/gpu_model_runner.py:596-599` | - | - | [specs/spec-decode-inventory.md](specs/spec-decode-inventory.md) | `INVENTORIED` | - |
| `SPEC-EXTRACT-HIDDEN` | Hidden-state extraction proposer (`method="extract_hidden_states"`): an offline / data-collection path that runs the target with aux hidden-state outputs (for training EAGLE/MTP drafts) rather than accelerating decode; model row `MODEL-SPEC-extract-hidden-states-extract-hidden-states-model`. | T2 | `vllm/v1/spec_decode/extract_hidden_states.py:29`; dispatch `vllm/v1/worker/gpu_model_runner.py:646-650` | - | - | [specs/spec-decode-inventory.md](specs/spec-decode-inventory.md) | `INVENTORIED` | - |
| `SPEC-MLP-SPECULATOR` | MLP speculator (`method="mlp_speculator"`): a small MLP head predicting K tokens. UPSTREAM-DEPRECATED at the pin — the enum still resolves it but the V1 dispatch has NO branch, so it raises `ValueError` (`gpu_model_runner.py:651-654`); V0-only. Rowed for enumeration completeness, lowest priority. | T2 | `vllm/model_executor/models/mlp_speculator.py`; enum `vllm/config/speculative.py:69,890` | - | - | [specs/spec-decode-inventory.md](specs/spec-decode-inventory.md) | `INVENTORIED` | - |
| `SPEC-DSPARK-QWEN3-ROUTING` | DSpark draft-architecture routing: a draft whose config declares `architectures=["DSparkDraftModel"]` together with `model_type` `qwen3` must resolve to the landed Qwen3 DSpark lane, and the predicate that decides it must be reached from the loader. At the pin, upstream forces EVERY DSpark draft that is not `Qwen3DSparkModel` or `Gemma4DSparkModel` onto `model_type` `deepseek_v4` plus `architectures` `["DSparkDraftModel"]` (`speculative.py:934-944`); **BEYOND-PIN** vLLM PR vllm#52197, merged 2026-08-17 at `7075ddac`, puts a leading branch in front of that catch-all which normalizes the pair to `Qwen3DSparkModel`, and adds the same pair to the `method = "dspark"` auto-detection at `:882-887`. We diverge from BOTH ends: the forced rewrite was NEVER ported, so no code reads a draft config's `architectures` key (the three reads in `model_loader.cpp` at `:1247`, `:1249`, `:1692` are the TARGET model), and `SpeculativeConfig::IsDsparkDraft` (`speculative.h:120-136`, arch list `:131`) has NO production caller — every reference outside its header is in `tests/vllm/config/test_speculative_dspark.cpp:132-140`, while `ResolveSpecConfig` branches on `cli.method` alone (`model_loader.cpp:844`, `:862`, `:875`, `:886`). Net effect today: the checkpoint loads, by never having ported the pinned behavior rather than by decision, and the DeepSeek-V4 arm we do not implement has no named refusal. GATEABLE ON THIS HOST: `RadixArk/Qwen3.8-27B-DSpark` @ `85ef153be924f17ce4bf62726954eeaa4a73e854` carries exactly the shape (one 2718576122-byte shard, sha256 `9d26d5e637551c244d543c67c790bd0947f360e005c569e5851a185ffe692786`, 5 layers against a 64-layer Qwen3.8-27B target); the 2.4T lane named by the upstream PR stays memory-infeasible and is NOT claimed. Scoped to classification only: the speculator, the weight path and the `--speculative-config` parser are excluded | T1 | `vllm/config/speculative.py:882-887,934-944` @ `555967922`; **BEYOND-PIN** PR vllm#52197 @ `7075ddac28c25d4fd2b84bc2a9a6c5ffde0345c8` | - | - | [dspark-qwen3-routing.md](specs/dspark-qwen3-routing.md) ([#1193](https://github.com/mudler/vllm.cpp/issues/1193)) | `READY` | - |
| `SPEC-DSPARK-QWEN3-ROUTING` | DSpark draft-architecture routing: a draft whose config declares `architectures=["DSparkDraftModel"]` together with `model_type` `qwen3` must resolve to the landed Qwen3 DSpark lane, and the predicate that decides it must be reached from the loader. At the pin, upstream forces EVERY DSpark draft that is not `Qwen3DSparkModel` or `Gemma4DSparkModel` onto `model_type` `deepseek_v4` plus `architectures` `["DSparkDraftModel"]` (`speculative.py:934-944`); **BEYOND-PIN** vLLM PR vllm#52197, merged 2026-08-17 at `7075ddac`, puts a leading branch in front of that catch-all which normalizes the pair to `Qwen3DSparkModel`, and adds the same pair to the `method = "dspark"` auto-detection at `:882-887`. We diverged from BOTH ends until W1-W4 landed. **Before:** the forced rewrite was NEVER ported, so no code read a draft config's `architectures` key (the three reads in `src/vllm/entrypoints/model_loader.cpp` were the TARGET model's), and `include/vllm/config/speculative.h::IsDsparkDraft` had NO production caller — every reference outside its header was in `tests/vllm/config/test_speculative_dspark.cpp`, while `LoadedEngine::ResolveSpecConfig` branched on `cli.method` alone. So the checkpoint loaded by never having ported the pinned behavior rather than by decision, and the DeepSeek-V4 arm we do not implement had no named refusal. **Now (W1-W4 landed):** the dspark branch of `src/vllm/entrypoints/model_loader.cpp::ResolveSpecConfig` reads the draft's own `config.json` through `src/vllm/entrypoints/model_loader.cpp::ReadDsparkDraftIdentity` — which translates the Speculators layout first — and classifies it with `include/vllm/config/speculative.h::IsDsparkDraft` and then `include/vllm/config/speculative.h::ResolveDsparkArchitecture`. The predicate therefore HAS a production caller, the `DSparkDraftModel` + `qwen3` pair routes to the landed Qwen3 lane, and a draft that resolves to the DeepSeek-V4 lane is refused BY NAME rather than rewritten into the `DeepseekV4Model` stub (TWO tracked divergences from upstream: that refusal, and the Gemma4 collapse onto one implemented lane). A draft that DECLARES no architecture is deliberately NOT classified, which is narrower than upstream's catch-all and is recorded under `## Owed`. REACHED, as of the merge of `SPEC-DSPARK-BLOCK-SIZE-GUARD` ([#1225](https://github.com/mudler/vllm.cpp/issues/1225)): that row hoists the DSpark resolution to the top of `LoadedEngine::FromModelDir`, above the target-directory check and far above `maybe_load_dflash`, so `ResolveSpecConfig` and therefore this classification now run BEFORE `LoadDsparkDraft`. A user arriving through `include/vllm.h` or the server at a DeepSeek-V4 DSpark draft gets the NAMED refusal, not the draft loader's missing-key message. The second call site inside `LoadDsparkDraft` that this row previously owed is therefore no longer owed; only two of the resolution's own earlier messages still precede the classification (#1225's missing-`k` refusal, and the GGUF branch's named refusal). Still owed: the G5 token-exact run gate (2.53 GiB draft + GPU authority) and the G6 spec-off gate. GATEABLE ON THIS HOST: `RadixArk/Qwen3.8-27B-DSpark` @ `85ef153be924f17ce4bf62726954eeaa4a73e854` carries exactly the shape (one 2718576122-byte shard, sha256 `9d26d5e637551c244d543c67c790bd0947f360e005c569e5851a185ffe692786`, 5 layers against a 64-layer Qwen3.8-27B target); the 2.4T lane named by the upstream PR stays memory-infeasible and is NOT claimed. Scoped to classification only: the speculator, the weight path and the `--speculative-config` parser are excluded | T1 | `vllm/config/speculative.py:882-887,934-944` @ `555967922`; **BEYOND-PIN** PR vllm#52197 @ `7075ddac28c25d4fd2b84bc2a9a6c5ffde0345c8` | [speculative.h:134-232](../include/vllm/config/speculative.h#L134-L232) (`IsDsparkDraft` + `ResolveDsparkArchitecture`); [model_loader.cpp:466-502](../src/vllm/entrypoints/model_loader.cpp#L466-L502) (`ReadDsparkDraftIdentity`) called from [model_loader.cpp:1017-1086](../src/vllm/entrypoints/model_loader.cpp#L1017-L1086), the dspark branch of `LoadedEngine::ResolveSpecConfig` | [test_speculative_dspark.cpp:130-216](../tests/vllm/config/test_speculative_dspark.cpp#L130-L216) 12 cases / 40 assertions (the predicate); [test_dspark_draft_routing.cpp:368-460](../tests/vllm/entrypoints/test_dspark_draft_routing.cpp#L368-L460) 7 cases / 19 assertions (REACHABILITY, entered through the `LoadedEngine` constructor, both published draft layouts). Mutation-proven 2026-08-18 in a scratch copy: deleting the `IsDsparkDraft` call, making it return constant true, and deleting the `ResolveDsparkArchitecture` call each turn the routing suite RED (rc=1). G5 run gate and G6 spec-off OWED, GPU/authority-blocked | [dspark-qwen3-routing.md](specs/dspark-qwen3-routing.md) ([#1193](https://github.com/mudler/vllm.cpp/issues/1193)) | `ACTIVE` | `CLAIM-SPEC-DSPARK-QWEN3-ROUTING` |
| `SPEC-DSPARK-BLOCK-SIZE-GUARD` | DSpark block floor, reached. `SpeculativeConfig::ResolveDspark` implements upstream's `k >= dspark_block_size` hard error, and BOTH production resolutions USED TO pass `std::nullopt` for `n_predict` AND for `dspark_block_size`, so the floor could not fire from any user path — the unpassed-parameter shape of `.agents/reachability.md`, exercised only by `tests/vllm/config/test_speculative_dspark.cpp` with hand-written values. The failure was SILENT: our draft step is sized only by `k` (`DsparkBlockLayout::num_speculative_steps` in `include/vllm/v1/worker/gpu/spec_decode/dspark/speculator.h`), nothing under `src/vllm/v1/worker/gpu/spec_decode/dspark/` reads the draft's block key, and no weight is shaped by the block, so a short `k` tripped no `VT_CHECK` and drafted a structurally wrong block while the tokens kept flowing. **A literal port would still not fire:** upstream reads `dspark_block_size`, an identifier that appears nowhere in the pinned checkout except `speculative.py`, and NEITHER published Qwen3 draft carries it — `deepseek-ai/dspark_qwen3_4b_block7` (`Qwen3DSparkModel`) and `RadixArk/Qwen3.8-27B-DSpark` @ `85ef153b` (`DSparkDraftModel`, `model_type` `qwen3`) both carry `block_size: 7` and no `n_predict`, and upstream's `block_size` normalization at `:945-961` is Gemma4-only, so `k=6` is accepted upstream on both sides of vllm#52197. LANDED: a file-local `ReadDsparkDraftKeys` threads the draft `config.json`'s real values into the ONE resolution that remains. `LoadedEngine::FromModelDir` no longer resolves a second time with its own argument list — it delegates to `LoadedEngine::ResolveSpecConfig` before it touches the model directory and hands the result to the draft load — so one resolution applies the floor, defaults `k` from `n_predict`, and owns the messages. TWO recorded divergences, both argued in the spec: `block_size` supplies the floor when `dspark_block_size` is absent (§2), and the refusal runs BEFORE the model-directory error where upstream builds `ModelConfig` first (§2a). Upstream's threshold, wording and `n_predict` rules are otherwise unchanged, and the refusal now names the key the value was read from. `test_dspark_block_size_guard` is 14/14 with 39 assertions; the reachability mutations red it 8/14 and 2/14, and the spec's §6c records the one claim the tests do NOT carry. G3 (run) and G4 (spec-off) owed, both needing a GPU lease. Excludes the speculator, `IsDsparkDraft` and the architecture classification (`SPEC-DSPARK-QWEN3-ROUTING`) | T1 | `vllm/config/speculative.py:945-961,973-994,1003-1027` @ `555967922` | `src/vllm/entrypoints/model_loader.cpp` `ReadDsparkDraftKeys` + the `ResolveDspark` call in `LoadedEngine::ResolveSpecConfig`, which `FromModelDir` now delegates to | `tests/vllm/entrypoints/test_dspark_block_size_guard.cpp` | [dspark-block-size-guard.md](specs/dspark-block-size-guard.md) ([#1225](https://github.com/mudler/vllm.cpp/issues/1225)) | `ACTIVE` | `CLAIM-SPEC-DSPARK-BLOCK-SIZE-GUARD` |

## Serving surface, CLI, and library
Expand Down
Loading
Loading