policy(porting): a model port documents the WEIGHTS it was fed, not just the code (#672) by localai-bot · Pull Request #951 · mudler/vllm.cpp

localai-bot · 2026-08-15T19:27:21Z

Developer-directed. Records only — no src/, include/ or tests/ change.

MiniMax-Music3 is the case that made it concrete. The port landed complete, and a
reader still could not run it without being told that the 57.4 GB repository is
~28.5 GB in the arm we actually load, split across six component directories,
with a native .pth layout we refuse by name and thirteen community
quantized repositories we do not implement.

None of that is inferable from a header file. The code is the easy half to
publish; saying what to feed it is the half that makes the port usable — and it
was nowhere in the checklist.

The rule

.agents/porting-a-model.md §2.1, with a checkbox in §2 and a pointer from
AGENTS.md's quantized-arms paragraph: every ported model documents the
checkpoints it was built and gated against in docs/USAGE.md, in the same
change that makes the capability reachable, mirroring the table MiniMax-H3
already carries — file name, size, and a link to the exact HuggingFace repo,
grouped by arm.

What a row owes, and why each clause exists

Clause	The thing that bit us
repo id and revision; sha256 for a quantized artifact	`unsloth/Qwen3.6-27B-NVFP4` was silently re-quantized in place under an unchanged name; Music3's GGUF lineage is published four ways under one model (`audiocpp`, `mm3`, `qwen3`, `wan`) and `wan` collides with genuine Wan video GGUFs
which arms are refused	same polarity as the refuse-by-name rule one paragraph above: a user should read the refusal in the docs, not discover it at load time
the resident size	it is not the repository's size, and it is what decides whether a reader can run it at all
whether a third party published it	community quant repos are the arm most users reach for, and carry different provenance from a first-party release

Placement

In porting-a-model.md rather than verification.md or benchmarking.md,
because it is a property of shipping a model rather than of gating or
measuring one — and because that file is the checklist an implementer already has
open. AGENTS.md carries it in prose so the rule is reachable without opening the
guide.

Gate: check-agent-record OK, check-doc-checkpoint OK.

🤖 Generated with Claude Code

…ust the code (#672) Developer-directed. MiniMax-Music3 is the case that made it concrete: the port landed complete, and a reader still could not run it without being told that the 57.4 GB repository is ~28.5 GB in the arm we actually load, split across six component directories, with a native `.pth` layout we refuse by name and thirteen community quantized repositories we do not implement. None of that is inferable from a header file. The code is the easy half to publish; saying WHAT TO FEED IT is the half that makes the port usable, and it was nowhere in the checklist. THE RULE, in `.agents/porting-a-model.md` §2.1 with a checkbox in §2 and a pointer from AGENTS.md's "Shared seams": every ported model documents the checkpoints it was built and gated against in `docs/USAGE.md`, in the SAME change that makes the capability reachable, mirroring the table MiniMax-H3 already carries — file name, size, and a link to the exact HuggingFace repo, grouped by arm. WHAT A ROW OWES, and each clause is there because something bit us: * The repo id AND the revision. A repo id alone is not a pin -- `unsloth/Qwen3.6-27B-NVFP4` was silently re-quantized in place under an unchanged name, and Music3's GGUF lineage is published four different ways under one model (`audiocpp`, `mm3`, `qwen3`, `wan`) with `wan` colliding with genuine Wan video GGUFs. A quantized artifact records its sha256 too. * Which arms are REFUSED, so a user reads the refusal in the docs rather than discovering it at load time -- the same polarity as the refuse-by-name rule one paragraph above it. * The total RESIDENT size, because that is what decides whether a reader can run it at all, and it is not the repository's size. * Whether a THIRD PARTY published it. Community quant repos are the arm most users actually reach for and they carry different provenance from a first-party release. Placed in `porting-a-model.md` rather than in `verification.md` or `benchmarking.md` because it is a property of SHIPPING a model, not of gating or measuring one, and because that file is the checklist an implementer already has open. AGENTS.md carries the rule in prose so it is reachable without opening the guide. Issue: #672 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

Routine integration ahead of the PR. No conflicts: this branch touches only AGENTS.md and .agents/porting-a-model.md, which main did not move. FOLLOWING_AGENTS_PROTOCOL Issue: #672 Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

Routine integration. This branch touches only AGENTS.md and .agents/porting-a-model.md. FOLLOWING_AGENTS_PROTOCOL Issue: #672 Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

…the first, and it came from main (#968, #672) With #965's `C4456 'loaded' shadow` removed, `windows-msvc-cpu` and `windows-msvc-vulkan` failed again on this row's pull request — and on a different cause: include\vector(1461,29): error C2220: the following warning is treated as an error include\vector(1461,29): warning C4244: '=': conversion from 'const double' to 'float', possible loss of data raised from `src/vllm/multimodal/ltx2_video.cpp:203,214`, the two narrowing `positions.assign` calls that `c7cb59fbb` (#964, LTX25-TOKEN-APPEND) landed on `main` while this row was in flight. `StreamState::positions` and `Ltx2LatentState::positions` differ in element type; GCC and Clang narrow silently, MSVC diagnoses and the build treats it as an error. NOT FIXED HERE, deliberately. #964's own comment at `ltx2_video.cpp:129-132` reasons that "double -> float -> double reproduces the bits", so the narrowing is intentional and a silencing `static_cast` would be a claim about that reasoning rather than a formatting repair. It belongs to the lane that owns the round trip. Filed as #968 with the evidence rather than papered over. THE MATCHED ARM SPLITS EXACTLY ON THE MERGE BASE, which is what makes it inherited rather than mine. Grepping each `windows-msvc-cpu` job log for the warning: #966 and #951, both on `c7cb59fbb`, hit it twice each; #967, #956, #950, #939 and #938, all based before it, do not hit it at all. This row's diff touches zero LTX2 files. THE FINDING WORTH CARRYING is not either warning. It is that TWO INDEPENDENT CAUSES WERE STACKED BEHIND ONE HABITUALLY-RED JOB NAME, and the first hid the second — and that neither was #645, the `M_PI` regression both jobs are usually attributed to. A known-red list tells you a job is often red. It never tells you that today's red is the same one. Only reading the log does. Issue: #968 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

localai-bot · 2026-08-16T01:19:55Z

Superseded by #970 — identical diff, rebuilt on a clean branch.

This PR's commit-protocol-tag red was a force-push artifact rather than a real
trailer defect: I amended a merge commit to add the trailers it was missing, which
left github.event.before pointing at a commit no longer on the branch, so the
push-lane range was unresolvable. Every commit in PR_BASE..PR_HEAD carries
FOLLOWING_AGENTS_PROTOCOL — verified by walking them individually — and all
three local checker invocations pass.

The lesson is the one #954 hit independently in the same campaign: this gate walks
every commit in the range, merge commits included, so a branch that integrates
main by merging inherits that obligation for its own merge commits. Rebuilding on
a fresh branch is cheaper than rewriting history under an open PR.

… heard, and the five keys upstream refuses that we dropped (#672, #953, #965) (#966) feat(MODEL-MUSIC-MUSIC3): a music-only server, an example that can be heard, and the five keys upstream refuses that we dropped (#672, #953) Developer directive: parity on what upstream supports — "we want to be a good reference" — usage docs for MiniMax-Music3, and in those docs the weights. Then, mid-flight: "we should allow to load only the music model" and "we need to have an e2e test working". FOLLOWING_AGENTS_PROTOCOL ## The upstream surface, enumerated rather than summarized SGLang-Omni `748a0b43` `models/minimax_music3/` and diffusers `c6da9936` `modular_pipelines/minimax_music3/`, read field by field and recorded with `file:line` in spec §10.1 so the next reader re-derives nothing. **Closed here:** the music-only server, the missing example, and five refusals. **Owed and named:** the non-`wav` response formats, request batching and `/v1/audio/speech/batch`, the 32 kHz delivery resample. **Permanently refused rather than owed:** streaming — neither upstream arm has it (`supports_streaming_vocoder=False`). **One place we are ahead of both arms:** `guidance_scale` is a real per-request control here, where diffusers freezes it at 1.7 into the guider component (`denoise.py:180`) and SGLang exposes it only as a serve-time knob. ## `--model` is optional when `--speech-model` is given Serving a 28.5 GB music model also forced loading an unrelated text model, and on this box the smallest text checkpoint is 35B — so the recipe this project documented was effectively unrunnable. Upstream's own is `sgl-omni serve --model MiniMaxAI/MiniMax-Music3`, no text tower anywhere. vllm-server --speech-model /path/to/minimax-music3 Third instance of a shape already in `server_main.cpp`: a pooling checkpoint serves `/v1/embeddings` alone, a Parakeet checkpoint serves `/v1/audio/transcriptions` alone. It mirrors vLLM's task-conditional registration (`api_server.py:255-265`). **Additive, and proved rather than argued.** The only case whose verdict changes is BOTH flags absent, which was an error and remains one with a message naming both options. The route table is gated in both directions over a real socket, because a handler-dispatch test cannot see route registration at all. ## The example the music family did not have `examples/minimax_music3_gen` — a thin client of `include/vllm.h` and nothing else, like `parakeet-transcribe` and `vllm-cli`. Hearing this model previously needed a running server plus a `curl`, or a C ABI caller nobody had written. ## Five keys upstream refuses by name were SILENT here (#953) `temperature`, `top_p`, `top_k`, `repetition_penalty` — refused upstream at `request_builders.py:14-19,109-114`, because this model's autoregressive stage has ONE sampler, a fixed top-50 draw (`encoders.py:48,94-103`). And `max_new_tokens`, upstream's LENGTH spelling in 25 Hz frames rather than seconds (`request_builders.py:56-68`), so a 250-frame request silently became the family's 60 s default. That is the #925 class exactly, in the same file that already carries #925's refusal one paragraph above. Fixed in flow. ## The e2e gate no longer reports a skip wearing a pass It read 5 cases / 5 passed with **`assertions: 0`** whenever the checkpoint was absent — the same shape that fooled this project on `test_qwen3_paged_engine`. Split into a checkpoint-free half that runs unconditionally in CI (request contract, both ceilings, the speech-only route table over a real socket with a stub synthesizer) and the env-gated half, whose HTTP case now drives the real engine over a real socket against the music-only server shape. A coverage-report case prints which arms ran, every run. The full arm was run: `POST /v1/audio/speech -> 200 audio/wav, 12332 bytes in 518 s wall`, 2 AR frames -> 6 latent frames -> 3072 samples/channel, 6144 int16 samples all non-zero, 0 clipped, 2818 of 3072 positions differing between L and R, and `/v1/completions` + `/v1/chat/completions` both 404 from the route table. `checkpoint_arms_run=5`. | arm | cases | assertions | |---|---|---| | `test_minimax_music3_e2e_real`, no env vars | 9 | 37 (was 5 / **0**) | | `test_minimax_music3_e2e_real`, checkpoint only | 9 | 86 | | `test_minimax_music3_e2e_real`, checkpoint + `VLLM_CPP_MUSIC3_DIT=1` | 9 | **582** | | `test_speech_api` | 6 | 67 | | `test_openai_api_server` | 62 | 733 | | `test_openai_conformance` | 23 | 252 | | `test_minimax_h3` (unchanged) | 79 | 57395 | | server flag ctest cases | 7 passed | | ## The weights are documented (porting-a-model.md §2.1) `docs/USAGE.md` gains component-by-component tables: the diffusers arm at `MiniMaxAI/MiniMax-Music3`@`fbdf52fbaaca799592917417eb05f1899f1255ec`, **28.5 GB resident** (28 517 617 303 B, measured) out of a 57.4 GB repository and why they differ; the native `.pth` arm we refuse and SGLang-Omni serves; the one implemented GGUF Q4_K artifact with its sha256; and the fourteen third-party quantized repositories in five formats, each marked refused. The revision is verified rather than copied — `condition_encoder/diffusion_pytorch_model.safetensors` on disk hashes to that revision's own LFS record. ## A sample a human can hear 2.0 s of 44100 Hz stereo from this engine in 3286 s of wall clock: RMS 0.03169, peak 0.97437 with 0 clipped samples, 84 073 of 88 064 positions differing between left and right. **Its samples are compared to nothing** — §5 withdrew the token gate — so it shows the pipeline runs, not that the music is right. It is not committed: `check-pr-size.py` classifies every path and none takes a `.wav` outside `tests/`, where a file compared to nothing would sit beside the goldens and imply it was one. ## The four asks, answered directly **1. Music-only server.** `vllm-server --speech-model <dir>` with NO `--model` starts and serves, observed live rather than inferred: server: speech/music-only model (family=minimax-music3, 44100 Hz, text-only synthesis, family DETECTED); serving /v1/audio/speech server: listening on http://0.0.0.0:18923 (model 'minimax-music3') `--model` alone and `--model` + `--speech-model` are **byte-identical in behaviour**. The whole change is one new early branch, `if (args.model_dir.empty())`, which loads the speech engine and `return 0`s before reaching a single line of the existing path; nothing downstream of it was touched. The only case whose verdict changes is BOTH flags absent, which was an error and remains one. Server suites: **7 of 7** ctest cases (4 pre-existing + 3 new — neither flag is still an error and now names both options; `--speech-model` alone reaches the speech LOAD; `--speech-family` alone still demands a checkpoint), `test_openai_api_server` **62 cases / 733 assertions** (+1 case / +24 assertions, the speech-only route table over a real socket), `test_openai_conformance` **23 / 252** unchanged. **2. e2e, three arms.** What a bare CI run executes unconditionally: the request contract on the exact body the real case posts, the near-miss and sampling refusals, the duration arithmetic including both ceilings, and the speech-only route table over a real socket with a stub synthesizer. What stays env-gated: everything needing the 28.5 GB checkpoint, plus the two 2.4B-DiT arms behind `VLLM_CPP_MUSIC3_DIT`. | arm | cases | assertions | checkpoint arms run | |---|---|---|---| | no env vars (what CI runs) | 9 | **37** | 0 — was 5 cases / **0 assertions** | | `VLLM_CPP_MUSIC3_CHECKPOINT` | 9 | **86** | 3 | | + `VLLM_CPP_MUSIC3_DIT=1` | 9 | **582** | 5 | **3. The five keys.** All five were **accepted and silently dropped**; all five are **now refused by name**. | key | upstream anchor | why it cannot be honoured | |---|---|---| | `temperature` | `request_builders.py:14-19,109-114` | the AR stage's only sampler is a fixed top-50 draw, `encoders.py:48,94-103` | | `top_p` | same | no nucleus branch exists | | `top_k` | same | `_AR_SAMPLING_TOP_K` is a module constant of 50 | | `repetition_penalty` | same | no penalty is applied anywhere in the loop | | `max_new_tokens` | `request_builders.py:56-68`, `constants.py:4-5` | upstream's LENGTH, in 25 Hz frames not seconds; the refusal names `audio_duration` and the /25 conversion | **4. The weights table** (`docs/USAGE.md`, "MiniMax-Music3: the exact weights"). It carries: repo **and** revision — `MiniMaxAI/MiniMax-Music3`@`fbdf52fbaaca799592917417eb05f1899f1255ec`, verified rather than copied, since `condition_encoder/diffusion_pytorch_model.safetensors` on disk hashes to `83179c5e…a202c2a4d`, that revision's own LFS record; the Q4_K artifact's sha256 `4c5d41b2…c70cbdd0` at revision `c36aaeed…` with its exact byte count; **28.5 GB resident (28 517 617 303 B, measured) versus 57.4 GB repository**, with the reason they differ; the refused native `.pth` arm (`qwen_7B/`, `flowmatching_vae.pth`, `dav.pth`) and that SGLang-Omni serves it; and all fourteen community quant repositories across five formats, each marked refused and each marked **third-party** rather than first-party. This is the first application of `.agents/porting-a-model.md` §2.1 (landing as #951). ## Two reds stacked behind one habitually-red job name `windows-msvc-cpu`/`windows-msvc-vulkan` are usually attributed to #645 (`M_PI` in three LTX2 sources). **Neither of the two causes here was #645**, and the first hid the second. **#965, fixed in flow.** `C4456: declaration of 'loaded' hides previous local declaration` at `server_main.cpp:1315` — W6's own speech-attach block declaring `loaded` inside the text engine's `loaded`. The only warning in the job, on `main` since W6 landed. Matched arm: #956, #950 and #939, none touching the speech surface, fail identically. Renamed; nothing suppressed. **#968, filed and NOT fixed here.** With the shadow gone the same jobs failed again on `C4244: conversion from 'const double' to 'float'`, raised inside MSVC's `<vector>` from `ltx2_video.cpp:203,214` — two narrowing `positions.assign` calls that `c7cb59fbb` (#964) landed on `main` while this row was in flight. **This branch touches zero LTX2 files.** The matched arm splits exactly on the merge base: #966 and #951 (on `c7cb59fbb`) hit it, #967/#956/ #950/#939/#938 (before it) do not. It is deliberately left to the LTX-2.5 lane — #964's own comment reasons that "double -> float -> double reproduces the bits", so a silencing cast is a claim about that reasoning rather than a formatting fix. **The finding, which outlives both:** a known-red list tells you a job is often red. It never tells you that today's red is the same one. Only reading the log does — and here it took two readings, because removing the first cause is what made the second visible.  ### The first of the two, in detail (#965) `windows-msvc-cpu`/`windows-msvc-vulkan` failed here, and they are **not** #645 (`M_PI` in three LTX2 sources). They were W6's own `C4456: declaration of 'loaded' hides previous local declaration` at `server_main.cpp:1315` — the only warning in the job, on `main` since W6 landed. The matched-arm check is what separated it from my diff: #956, #950 and #939, all touching no speech surface, fail identically. `main` has no baseline because `windows-msvc-*` are PR-only (#584), so it presents to each author in turn as their own red. Filed and fixed in flow by renaming the inner declaration; nothing suppressed. ## Mutations Four run, four fire: sampling refusal neutered (5 assertions red), `max_new_tokens` refusal neutered (2 red), `--model` made mandatory again (2 ctest cases red), generate routes registered unconditionally (3 cases / 6 assertions red in the api-server suite, 1 / 2 in the e2e suite). Sources restored and verified sha256-identical. Supersedes #954 (untrailered merge commits) and #963 (a `server_main.cpp` commit that owed `docs/USAGE.md` under the per-commit documentation checkpoint). Same tree, linear history, every commit green on `check-doc-checkpoint`, `check-commit-trailers` and `check-commit-style` locally before pushing. Every source file is byte-identical to the one built and gated. Issue: #672 Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io>

…ust the code (#672) (#970) Records only — no `src/`, `include/` or `tests/` change. Supersedes #951, which carried a force-push artifact in its commit range; this is the identical diff rebuilt on a clean branch with no rewritten history. Developer-directed. MiniMax-Music3 is the case that made it concrete: the port landed complete, and a reader still could not run it without being told that the 57.4 GB repository is **~28.5 GB in the arm we actually load**, split across six component directories, with a native `.pth` layout we refuse by name and **fourteen** community quantized repositories we do not implement. None of that is inferable from a header file. The code is the easy half to publish; saying *what to feed it* is the half that makes the port usable — and it was nowhere in the checklist. ## The rule `.agents/porting-a-model.md` §2.1, with a checkbox in §2 and a pointer from AGENTS.md's quantized-arms paragraph: every ported model documents the checkpoints it was built and gated against in `docs/USAGE.md`, **in the same change that makes the capability reachable**, mirroring the table MiniMax-H3 already carries — file name, size, and a link to the exact HuggingFace repo, grouped by arm. ## What a row owes, and why each clause exists | Clause | The thing that bit us | |---|---| | repo id **and revision**; sha256 for a quantized artifact | `unsloth/Qwen3.6-27B-NVFP4` was silently re-quantized in place under an unchanged name; Music3's GGUF lineage is published four ways under one model (`audiocpp`, `mm3`, `qwen3`, `wan`) and `wan` collides with genuine Wan video GGUFs | | which arms are **refused** | same polarity as the refuse-by-name rule one paragraph above: a user should read the refusal in the docs, not discover it at load time | | the **resident** size | it is not the repository's size, and it is what decides whether a reader can run it at all | | whether a **third party** published it | community quant repos are the arm most users reach for, and carry different provenance from a first-party release | ## Placement In `porting-a-model.md` rather than `verification.md` or `benchmarking.md`, because it is a property of **shipping** a model rather than of gating or measuring one — and because that file is the checklist an implementer already has open. AGENTS.md carries it in prose so the rule is reachable without opening the guide. Its first application is the MiniMax-Music3 weights table in #966. Gate: `check-agent-record` OK, `check-doc-checkpoint` OK, `check-commit-trailers` OK over the PR range. 🤖 Generated with [Claude Code](https://claude.com/claude-code) FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io>

mudler added 3 commits August 15, 2026 19:26

merge: origin/main into row/MUSIC3-DEMO-SAMPLE

af71fb7

Routine integration. This branch touches only AGENTS.md and .agents/porting-a-model.md. FOLLOWING_AGENTS_PROTOCOL Issue: #672 Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

localai-bot force-pushed the row/MUSIC3-DEMO-SAMPLE branch from 4bbcea1 to af71fb7 Compare August 16, 2026 00:38

localai-bot mentioned this pull request Aug 16, 2026

policy(porting): a model port documents the WEIGHTS it was fed, not just the code (#672) #970

Merged

localai-bot closed this Aug 16, 2026

Navigation Menu

policy(porting): a model port documents the WEIGHTS it was fed, not just the code (#672) - #951

policy(porting): a model port documents the WEIGHTS it was fed, not just the code (#672)#951
localai-bot wants to merge 3 commits into
mainfrom
row/MUSIC3-DEMO-SAMPLE

localai-bot commented Aug 15, 2026

Uh oh!

localai-bot commented Aug 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

localai-bot commented Aug 15, 2026

The rule

What a row owes, and why each clause exists

Placement

Uh oh!

localai-bot commented Aug 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants