HALOBENCH_

The topology — what this lab actually runs

Four measurements

Field snapshot → full comparison

best measured configuration per model · every row's full fingerprint on /compare · exclusions stay visible: an excluded row is data, not an omission

Start with a question → all questions

From the log → the full notebook

• 2026-08-15claimOn gfx1151 at f16 KV, Qwen3.8-27B (dense) is a fourth model measured under the per-model, per-phase backend rule (clm-0050), and it lands emphatically on the prefill-favours-ROCm side: DECODE is backend-independent on this model (every matched cell within 5%), but Vulkan PREFILL collapses with depth — 0.55x ROCm at d32768 on Q8_0 (87.82 vs 159.98 t/s) and 0.48x on UD-Q4_K_XL (99.80 vs 207.61) — and at d131072 stock Vulkan cannot complete the cell at all: 2 of 2 reps in BOTH quants aborted with vk::DeviceLostError, the kernel logging an amdgpu ring timeout and recovering the device by ring reset, while ROCm completed every cell it was offered (83.44 pp / 6.11 tg Q8_0, 95.59 pp / 8.14 tg UD-Q4_K_XL at d131072). clm-0054
• 2026-08-15claimdraft-mtp speculation on Qwen3.8-27B (stock 3653e6d, gfx1151) peaks at spec-draft-n-max=3 over CLEAN cells — Vulkan Q8_0 17.75 t/s (2.26x its 7.86 no-speculation floor, acceptance 0.626), Vulkan UD-Q4_K_XL 26.68 t/s (2.23x, 0.623), ROCm Q8_0 18.25 t/s (2.33x, 0.6235) — and at n_max >= 4 the feature is BROKEN on this model: after accumulated generation volume in a live session (sequential VARIED prompts at full length; not fresh servers, not one repeated prompt, not short generations), generations start terminating at 1 token with <|im_end|> (id 248046). clm-0055
• 2026-08-15claimQwen3.8-27B's KV cache on llama.cpp costs exactly 64.00 KiB per token — the hybrid-attention allocation working as designed, measured byte-exact on this box: only 16 of the 64 layers carry a conventional KV cache (the 3:1 linear-to-full-attention layout), and 16 layers x 4 KV heads x 256 head_dim x 2 (K+V) x 2 bytes (f16) = 65,536 bytes/token. clm-0056
• 2026-08-15claimQwen3.8-27B met its registered agentic prediction: on the standing 5-task tau2 airline smoke subset it scored 1.000 (5/5, 21 tool-call messages, 0 empty assistant turns, VALID under the smoke gate) against the >= 0.80 bar recorded in the candidate record on 2026-08-14, BEFORE any measurement — and against the incumbent qwen36-27b-mtp's 0.80 on the identical tasks under the identical pinned-simulator protocol, where the incumbent failed task 2 and this model passed it. clm-0057
• 2026-08-15gateqwen38-27b: listed → screened — Screen tier passed on release day, on-box. clm-0055 clm-0056 clm-0057 run-0267
• 2026-08-15gateqwen38-27b: screened → benched — DAY-ONE SCREEN COMPLETE, all four phases, overnight on release day. clm-0054 clm-0055 clm-0056 clm-0057 run-0271 run-0267 run-0268
• 2026-08-15runs1 run landed on cfg-0066 (eos-cliffguard) run-0267
• 2026-08-15runs1 run landed on cfg-0067 (mtp-probe) run-0268
• 2026-08-15runs1 run landed on cfg-0066 (tau2-bench-airline) run-0271
• 2026-08-14claimAMENDED 2026-08-14 — the rule below is PER-MODEL AND PER-PHASE, not fleet-wide; see the correction history and the amendment note. clm-0050
• 2026-08-14claimOn gfx1151, stock ROCm 3653e6d's own BF16 KV decode is SLOWER than its own F16 KV decode: 29.26 t/s vs 42.35 t/s at 32,768 depth (Qwen3.6-35B-A3B, fa on) — a 31% penalty for switching to the KV type the community recommends for its quality, with prefill roughly unchanged (587.4 vs 595.0). stew675's rdna-boosts branch (commit ed89854) fixes it with a native BF16 flash-attention tile kernel: 47.31 t/s BF16 decode, +61.7% over stock's own BF16 and stock's best config on this chip. clm-0052
• 2026-08-14claimA benchmarking-harness defect, not a hardware limitation, produced this lab's earlier "Nemotron-3-Super cannot allocate on ROCm at any depth" verdict (clm-0050's first amendment, corrected 2026-08-14). clm-0053
• 2026-08-14gate4 candidates → screened: npu-embeddinggemma, npu-lfm2, npu-qwen3-4b-thinking, npu-whisper
• 2026-08-14gateqwen38-27b: listed → listed — RELEASED and CONFIRMED runnable.

How to read our numbers → method