Qwen3.8-27Bbenchedno guard
②Verdict
A 27-billion-parameter dense model with a hybrid attention layout that makes long context nearly free: the KV cache costs 64 KiB per token, 2.00 GiB at 32k, and both quants fit the full 262k declared context with more than 75 GiB of headroom clm-0056. The capability case pre-registered for it was met: 1.000 on the five-task τ² smoke against the 0.80 bar recorded before measurement, and 0.577 over the 26 tasks its capped full run completed clm-0057. Speed is the limitation, not competence: the dense floor decodes at 7.8 t/s, speculation via the MTP head is therefore mandatory, and its 2.33x lift to roughly 18 t/s is hard-capped at n_max 3 because higher settings corrupt generations outright in live sessions — a failure this lab characterised and staged for upstream clm-0055. Backend choice is not free either: decode is backend-indifferent, but Vulkan prefill halves at working depth and the device is lost entirely at 131k, so serving this model at depth requires ROCm clm-0054. The shallow probe picked the other backend — depth reversed it, which is the standing per-model, per-phase rule doing its work clm-0050.
③Best configuration
| model | Qwen3.8-27B-Q8_0.gguf · Q8_0 |
| engine | ggml-org/llama.cpp 3653e6d · rocm · host aihydra (igpu) |
| flags | -ngl 999 -fa on -c 32768 -dev ROCm0 --spec-type draft-mtp --spec-draft-n-max 3 --jinja --reasoning-format deepseek --chat-template-kwargs {"reasoning_effort":"medium"} -n 4096 --parallel 1 --load-mode none --slots |
| sampling | temperature 1 · top_p 0.95 · max_tokens 4096 |
| template | not recorded at test time |
| tree | upstream — stock |
config record cfg-0066
every cell generated from the record at build time · throughput cells from cfg-0062 (same build 3653e6d, fa on, f16 KV) · provenance: measured-here throughout
③aDecode against context depth
④Other configurations tested — each as a delta against best
| variant | Δ decode | Δ turns (paired tasks) | Δ energy | note | records |
|---|---|---|---|---|---|
| UD-Q4_K_XL quant · same backend | +42% @32k | — | — | The comparability arm decodes faster, as a dense model reading fewer bytes per token must — the capability screen ran on Q8_0, so the quant's own agentic score is unmeasured, not inherited. | clm-0054 |
| Vulkan backend · same quant | -0.2% @32k | — | — | Decode is a wash — the delta that matters is not in this column. Vulkan prefill collapses to roughly half of ROCm's at working depth, and at the deepest cell Vulkan lost the device on every attempt while ROCm completed the matrix. | clm-0054 run-0265 run-0266 |
| draft-mtp · n_max 3 | +133% | — | — | Not optional on this model — the floor is unusably slow — and not safe above this setting either; the ceiling is a correctness bound, not a tuning preference. | run-0268 clm-0055 |
| draft-mtp · n_max above 3 hazard | — | generations collapse to 1 token after accumulated session volume, and the broken cells report 1,000,000 t/s | The EOS cliff: the speculative path drives the target's own next-token distribution onto end-of-turn in live varied-prompt sessions. Sweeps that average unfiltered throughput flatter exactly the broken settings. | clm-0055 | |
deltas computed at build time from the named runs, paired-task traces and metered windows — a hazard row shows words where a delta would mislead
⑤Open questions
- The EOS cliff's root cause — the target's forward pass is demonstrably corrupted at n_max ≥ 4, the MTP head's internal state is the suspect, and the upstream issue is not yet filed clm-0055.
- ROCm at n_max ≥ 4 — the symptom was characterised on Vulkan; the clean ROCm n_max 3 cell is not evidence either way clm-0055.
- The full 50-task τ² mean — the wall-bound run completed 26 tasks; the remaining tasks and an uncapped-output arm would make the score comparable to the historical series clm-0057.
- Wh per correct answer — unmeasured; the screen recorded no energy windows.
- d204800 throughput — the matrix ended at 131,072; ROCm's lead there is extrapolated from other models, not measured on this one clm-0054.
- UD-Q5_K_XL — the community's preferred quant for this model was not screened here; the bracket around it (Q8_0 above, UD-Q4_K_XL below) is clm-0054.