bench(concurrency): add ragged paired benchmark

[Graffioh]

Dependency

Depends on #595 (packed ragged prefill), transitively on #594.

Scope

Benchmark-only changes under harness/benchmarks/concurrency/. This PR does not change server runtime behavior.

• paired Luce/llama.cpp synthetic workloads with arbitrary concurrency levels; the default matrix is C2/C4/C8/C16;
• deterministic, mean-matched ragged cohorts with cumulative offsets and no prompt wrapping or reuse within a level;
• HumanEval, GSM8K, and math canonical service workloads in complete waves;
• exact fixed-token validation, warmup isolation, request/cohort provenance, and repeat stability checks;
• end-to-end TTFT/goodput metrics plus optional Luce usage.timings native-prefill metrics;
• fail-closed Strix identity and running-process GPU/offload proof;
• neutral passthrough and metadata for latency-policy knobs used by downstream experiments.

The synthetic runner supports C32 and larger cohorts when the server has enough slots and the prompt generator is asked for those levels. Canonical C16 is intentionally not fabricated from ten-case corpora: C2/C4/C8 use eight unique cases in complete waves.

Metric semantics

• Output goodput includes admission, prefill, and decode.
• Output-window tok/s starts at the earliest visible output and is decode-facing, but can include staggered prefill.
• Prompt tok/s to first is an end-to-end cross-engine service metric.
• Native prefill tok/s comes from Luce usage.timings; it is optional, Luce-only, and is not a kernel-only measurement.

Partial native telemetry, incomplete fixed-token responses, prompt reuse, missing terminal SSE markers, GPU identity mismatches, and partial llama GPU offload all fail closed.

Strix Halo screen

One-repeat screening was run exclusively on gfx1151 / Radeon 8060S with Qwen3.6-27B-Q4_K_M, temperature 0, seed 1, EOS ignored, and exactly 64 output tokens per successful request. Luce process logs prove gfx1151; llama.cpp reports Radeon 8060S Graphics and offloaded 65/65 layers to GPU.

At C16:

Workload	Luce goodput	llama.cpp goodput	Luce max TTFT	llama max TTFT
short	30.96 tok/s	22.67 tok/s	23.404 s	29.561 s
medium*	16.78 tok/s	12.06 tok/s	51.110 s	69.125 s
long	6.08 tok/s	5.19 tok/s	158.331 s	180.800 s

* The medium-C16 llama value is a separate fresh-server retry over the identical selected prompt set. The original paired warmup lost one terminal SSE marker. Long-C8 llama likewise produced only 7/8 valid streams, so the harness excludes that partial row.

Luce native C16 prefill throughput stayed broadly flat as context grew: 349.5 tok/s short, 363.1 tok/s medium, and 356.9 tok/s long. The native window grew with total prompt work, from 21.47 s to 151.73 s.

These are screening measurements, not variance-backed publication results. Use at least five repeats for final claims.

Running

MODEL=/path/to/Qwen3.6-27B-Q4_K_M.gguf \
GPU_DEVICE=1 EXPECTED_GPU_ARCH=gfx1151 \
CLIENTS=2,4,8,16 REPEATS=5 \
harness/benchmarks/concurrency/run_qwen36_concurrency.sh

MODEL=/path/to/Qwen3.6-27B-Q4_K_M.gguf \
GPU_DEVICE=1 EXPECTED_GPU_ARCH=gfx1151 \
SUITES=he,gsm,math CLIENTS=2,4,8 CASE_LIMIT=8 REPEATS=5 \
harness/benchmarks/concurrency/run_qwen36_canonical_concurrency.sh

Both runners document all options with --help and save exact commands, hashes, GPU proof, warmups, JSON results, and summaries under OUT.

Validation

• full concurrency harness: 60/60 tests;
• both runners pass shell syntax checks;
• git diff --check;
• amended delta from the prior bench(concurrency): add ragged paired benchmark #596 head contains only the 12 benchmark/harness files.

[cubic-dev-ai]

All reported issues were addressed across 75 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

All reported issues were addressed across 75 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

All reported issues were addressed

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

All reported issues were addressed across 86 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[Graffioh]

Added canonical workload concurrency coverage in b98d3c3:

• exact raw 10-prompt HumanEval corpus from server/scripts/bench_he.py with identity templating for blog parity
• checked-in HumanEval-chat, GSM8K, Math500, and agent suites with message roles preserved
• full-suite fixed-width waves (10-case: C=1/2/5/10; agent: C=1/2/3/6), with scheduler-retirement barriers so nominal C does not accumulate hidden live requests
• AR vs blog-DDTree variants (Q8_0 draft, SWA=2048, budget=22, fast rollback, adaptive disabled, 128 forced tokens)
• fail-closed response-ID telemetry proof for DDTree steps, target forwards, mean accepted length, and acceptance rate
• consolidated summary plus 25/25 unit tests

Single-repeat Strix Halo screening on the exact raw HumanEval corpus (server binary SHA-256 31664e6f184b7cc37df9e719179d34c2b527ab36e02fd518cf08aef45aa65389):

C	AR goodput	DDTree goodput	DDTree vs AR	DDTree AL	Acceptance
1	11.59	15.17	+30.9%	5.40	33.8%
2	16.66	22.32	+34.0%	5.60	35.0%
5	37.50	25.05	-33.2%	5.59	34.9%
10	56.13	24.48	-56.4%	5.59	34.9%

All rows are 10/10 requests with exactly 128 output tokens. This is a screen, not a publication result; use REPEATS=5 on a clean target PR build for final numbers. The flat acceptance makes the C>=5 crossover an implementation-cost signal rather than a draft-quality failure.

[cubic-dev-ai]

All reported issues were addressed across 12 files (changes from recent commits).

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[Graffioh]

C=3/C=4 follow-up (Strix Halo, Qwen3.6-27B Q4_K_M target + Q8_0/SWA=2048 blog drafter, budget 22, 96 forced output tokens, five repeats):

Suite	C	Cases	AR goodput	Fixed DDTree	Median paired delta	AL	Acceptance
GSM8K	3	9	18.88 tok/s	24.46 tok/s	+29.4%	5.92	37.0%
Math500	3	9	19.03 tok/s	28.71 tok/s	+50.9%	7.00	43.8%
GSM8K	4	8	31.73 tok/s	27.84 tok/s	-12.3%	5.57	34.8%
Math500	4	8	31.81 tok/s	32.92 tok/s	+3.1%	7.10	44.4%

An additional five-repeat C=4 adaptive-DDTree screen measured 30.36 tok/s on GSM8K (-4.4% vs AR) and 31.53 tok/s on Math500 (-1.0% vs AR). It reduces the low-yield GSM penalty, but did not beat AR in either C=4 suite.

Practical result: fixed blog-DDTree is clearly best through C=3 in these reasoning workloads. At C=4, fixed DDTree is worthwhile only for a known high-acceptance Math-like workload; AR is the safer choice for GSM8K or unknown/mixed traffic. Existing HumanEval screening favors AR at C>=4.

All requests completed with exact 96-token accounting and no failures. Output hashes varied across repeats for both AR and DDTree, so this is explicitly reported as a concurrent reproducibility warning rather than hidden. These remain screening measurements because the measured local PR605 binary/worktree was not a clean publication build.

Commit 4527e85 adds CASE_LIMIT for exact full C=3 waves, adaptive-ddtree as an independently selectable variant, robust interrupt cleanup, output-stability reporting, tests, and the measured guidance.

[cubic-dev-ai]

All reported issues were addressed across 5 files (changes from recent commits).

<sub>Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic</sub>

[cubic-dev-ai]

All reported issues were addressed across 12 files (changes from recent commits).

<sub>Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic</sub>