test(BACKEND-ROCM): the block-size contract is enforced at its production call site - #1273
Open
tbrasser wants to merge 1 commit into
Open
test(BACKEND-ROCM): the block-size contract is enforced at its production call site#1273tbrasser wants to merge 1 commit into
tbrasser wants to merge 1 commit into
Conversation
…tion call site `CheckKvCacheShape` is well tested in isolation (test_attn_backend_registry / test_common_attn_metadata), but its install inside `GPUModelRunner::initialize_kv_cache` had no test: deleting it left every gate green (the mudler#1065 Owed item). A runner built with a non-multiple-of-16 block size now asserts the throw at construction, from the FLASH_ATTN backend's own `get_kv_cache_shape` — the executable statement of the contract the server's `--block-size` validation and the bench rounding exist to prevent at the entry points. Validated on gfx1151 (Strix Halo) as part of the M3 battery: test_runner 20/20, 544 assertions. Issue: mudler#41 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Buffy:assistant [Freebuff]
Collaborator
|
@mudler This is good to merge. The test reaches the production |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the "Owed" item from #1065: CheckKvCacheShape's production call site in GPUModelRunner::initialize_kv_cache now has a test. Building the runner with a non-multiple-of-16 block size (kBlockSize = 8) asserts the throw at construction, from the FLASH_ATTN backend's own get_kv_cache_shape — the executable statement of the contract the server's --block-size validation and the bench rounding exist to prevent at the entry points.
Validated on gfx1151 (Strix Halo, ROCm 7.2.3) as part of the #41 M3 battery: test_runner 20/20 cases, 544 assertions, on the current-main build with #1056 + #1065 merged.
Issue: #41
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: DeepSeekV4:Flash [Freebuff]