Qwen 3.8's Speed Trick Has a Catch — What "Multi-Token Prediction" Actually Costs

Описание: Two people download the same model file. Same GPU. Same tool. One is generating noticeably faster than the other — and the difference is one thing switched on.

Here's the part that makes this worth your time: most people who think they have it switched on actually don't. The way you enable it changed, and the old instructions still circulating now silently do nothing. Your command still runs, still loads the model, still works — it just isn't doing the thing you think it's doing.

This is multi-token prediction, explained mechanism-first: why a local model is memory-bound rather than compute-bound, what the draft-and-verify trick actually does, why the depth of the guess is its own dial, and how it stacks with Qwen 3.8's hybrid architecture (three of every four layers aren't traditional attention).

Then the part nobody covering this is honest about: the two dials that decide whether you win — sampling randomness and memory headroom — and the specific, common situation where turning this ON makes your model SLOWER. On a card that's already nearly full, off can be faster than on. I show exactly where that line sits.

I read llama.cpp's actual argument definitions this morning, and the real story is more useful than the one going around: the old draft flags weren't silently renamed, they were REMOVED with a migration message you never see because it's swallowed by app layers and wrappers. And there's a second, separate flag that downloads the prediction head — which means there are two independent ways to think you've enabled this and be wrong.

No hero number here. You get a range with its conditions attached, because "3× faster" is true somewhere and false somewhere else.

⏱️ Chapters:
00:00 Same Model, Same GPU, Different Speed
00:22 Most People Think It's On — It Isn't
00:49 What This Video Covers
01:12 Earning the Number Before Giving It
01:38 Three Things
02:02 Standards Note
02:27 Why a Model Is Slow
02:45 Memory-Bound, Not Compute-Bound
03:13 One Token, One Trip to the Warehouse
03:29 The Trip Is Expensive, the Cargo Is Cheap
03:41 What Multi-Token Prediction Actually Is
03:48 "The Cat Sat on the ___"
04:14 Draft, Then Verify in Bulk
04:47 The Whole Trick in One Sentence
05:00 Why It's Called Speculative Decoding
05:22 The Depth Dial
05:54 It's a Bet on Predictability
06:17 The Architecture Wrinkle
06:34 Three of Four Layers Aren't Attention
06:56 Two Teams, One Bottleneck
07:29 The Silent-Off Problem
07:39 What the Source Actually Says
08:03 "The Argument Has Been Removed"
08:20 Why You Never See the Message
08:34 The Head Is a Separate Download
09:07 The Abstraction That Hides It
09:50 The Catch: Two Dials
10:09 Dial One — Sampling Randomness
10:51 Dial Two — Memory Headroom
11:22 Where It Flips Negative
11:43 The Number, With Its Conditions
12:03 Why a Range, Not a Headline
12:37 What to Actually Do
13:02 If Your Card Is Nearly Full
13:20 If You Do Creative Work
13:34 Honest Limits
15:09 Step Back: The Loop, Not the Weights
15:52 A Number Without Its Conditions
16:11 Run It Both Ways — Post Your Two Numbers
16:38 Every Link Below

🔗 Sources — the exact files I read:
The flag definitions (search this file for --mtp and --draft-max): https://github.com/ggml-org/llama.cpp...
Open MTP work in progress: https://github.com/ggml-org/llama.cpp...
Speculative decoding in llama.cpp: https://github.com/ggml-org/llama.cpp...
The model repo (now live): https://github.com/QwenLM/Qwen3.8
Model card: https://huggingface.co/Qwen/Qwen3.8-2...
Gated DeltaNet (the architecture, referenced not re-read): https://arxiv.org/abs/2412.06464

Our Playlists:
   • Run Massive AI Locally (No GPU Needed)  

#Qwen #LocalLLM #llamacpp #AIPerformance #SignalCoders

👇 Subscribe — a new plain-English breakdown of AI engineering, open source tools, and local LLMs every single day.
YouTube: @SignalCoders
Twitter/X: @SignalCoders



⚠️ Disclaimer: This video is for educational purposes only. Signal Coders is not affiliated with, or sponsored by, Qwen/Alibaba, the llama.cpp project, or any company mentioned. The flags, the removal message, the separate download flag and the architecture description were read at source on the recording date and quoted verbatim; llama.cpp's master branch moves quickly and exact flag spellings may change — the mechanism will not. NO benchmark in this video was reproduced on my own hardware: the speed figures circulating publicly are community and vendor numbers, deliberately presented as a RANGE with conditions rather than as a single multiplier, and at least one widely shared "before" figure appears to start from an already-slowed baseline. Real-world results depend entirely on your specific hardware, memory headroom, sampling settings and task type. Nothing here is a recommendation to change settings you don't understand — measure your own tokens per second before and after.