Qwen3.8-27B DFlash 2: 3.43x Faster Or 1.01x — Their Own Table Says Both

Описание: One setting is supposed to take Qwen3.8-27B from about 69 tokens a second to 236. That number is real, it is published, and the hardware is named. Further down the same table the same setting returns 1.01x - no gain at all. Both numbers are correct, and which one arrives has nothing to do with your graphics card.

WHAT DFLASH 2 ACTUALLY IS
Not a faster Qwen. A second, much smaller model that guesses ahead, from z-lab / Inco AI, apache-2.0, published 2026-08-15.

From its config.json:
num_hidden_layers 5, against the target's 64 - 7.8% of the depth.
block_size 8, and is_causal FALSE - the whole block is drafted in ONE parallel pass, not eight passes.
target_layer_ids [5, 19, 33, 47, 61] - it taps five of the target's hidden states.
selector_top_k 16 keeps sixteen candidates per position; a two-tap dynamic convolution stops the draft decaying toward the end of the block.
All five layers are sliding_attention with sliding_window 2048, so the DRAFT'S OWN KV CACHE NEVER GROWS - 42 MB, fixed, at any context length.

THE CEILING, AS ARITHMETIC
One verification step reads the draft plus the target: 1.14 + 13.9 = 15.04 GB, and returns up to 8 tokens. Those 8 tokens decoded one at a time would have read 8 x 13.9 = 111.2 GB.
111.2 / 15.04 = a ceiling of 5.05x.

WHAT EATS IT: ACCEPTANCE
A guess can be wrong, and a wrong guess is thrown away. Measured acceptance length, out of 8:
GSM8K 5.46 · MATH-500 5.28 · HumanEval 4.39 · MBPP 4.79 · MT-Bench 4.10
Recomputed with real acceptance: 5.05x on grade-school arithmetic, 3.79x on open conversation.
They measured 3.43x and 2.67x - 68% and 70% of ceiling, which is the same 60-80% band real decoding always lands in.

THE PART EVERYONE SKIPPED: CONCURRENCY
The published table has fifteen cells, not one. Across its columns exactly one thing changes - how many conversations the server holds at once.

Speedup vs autoregressive, per their table:
concurrency 1 concurrency 8 concurrency 32
GSM8K 3.43x 2.84x 1.45x
MATH-500 3.34x 2.85x 1.30x
HumanEval 3.11x 2.67x 1.16x
MBPP 3.29x 2.78x 1.25x
MT-Bench 2.67x 2.27x 1.01x

The widely-quoted 3x is GSM8K at concurrency 1 - the single most favourable cell of fifteen.

AND QWEN'S OWN BUILT-IN MTP GOES BELOW 1.0 AT CONCURRENCY 32: 1.04 / 0.94 / 0.84 / 0.87 / 0.77. On a busy server the free speedup is a slowdown. DSpark behaves the same way (0.74-1.13x).

WHY: speculation does not make the card faster. It spends bandwidth that was already being wasted. Alone on the machine, the card reads 13.9 GB and emits ONE token - almost all of that reading is spent on nobody. Hold 32 conversations and the same read emits 32 tokens. The waste is already gone, so there is nothing left for a guess to reclaim. It is a LATENCY trick, not a THROUGHPUT trick.

THE PRICE IS NOT QUALITY
Decoding is lossless - greedy output matches the target model exactly, and sampling preserves its distribution.

THE PRICE IS CONTEXT
The draft sits on the card too. On a 24 GB 3090:
without it, 13.90 GB of weights leaves 9.10 GB of cache room - about 138,854 tokens.
with the Q4_K_M draft, 15.04 GB leaves 7.96 GB - about 121,414 tokens.
About 17,440 tokens of context, traded for the latency. Q8_0 costs more (107,476), BF16 far more (79,951).

WHAT THIS DOES NOT CLAIM
Every speedup above is z-lab's OWN measurement on ONE H200 with SGLang and FlashAttention 3, block size 8, temperature 1.0 / top-p 0.95 / top-k 20, xhigh reasoning, 4096 max new tokens. This channel has no GPU and measures nothing. Every 3090 figure here is arithmetic - a CEILING, not a benchmark.

SOURCES
DFlash 2 draft model and eval tables: https://huggingface.co/z-lab/Qwen3.8-...
Mirror: https://huggingface.co/incoai/Qwen3.8...
GGUF file sizes (Q4_K_M 1.143 GB, Q8_0 2.056 GB, BF16 3.860 GB): https://huggingface.co/z-lab/Qwen3.8-...
Qwen3.8-27B config.json: https://huggingface.co/Qwen/Qwen3.8-2...
RTX 3090 memory bandwidth 936 GB/s, manufacturer specification

Where the 13.9 GB and 64 KiB figures come from:
Qwen3.8-27B Doesn't Need 64 GB Of Cache. One config.json Line Says 16. -    • Qwen3.8-27B Doesn't Need 64 GB Of Cache. O...  
Qwen3.8-27B Drops 67 to 42 tok/s As You Talk (The KV Cache Is Read Every Token) -    • Qwen3.8-27B Drops 67 → 42 tok/s As You Tal...  
Can a $600 3090 Run Qwen3.8-27B? (Full VRAM Math) -    • Can a $600 3090 Run Qwen3.8-27B? (Full VRA...