DeepSeek Did What Other Labs Won’t Even Try

Описание: DeepSeek V4 costs about 43 cents per million input tokens. The same request runs dramatically more on Claude Opus 5 and Gemini 3.1 Pro.

This is the animated breakdown of how DeepSeek got the price of long context that low: compressed attention. We cover why long prompts cost a fortune in the first place, how Compressed Sparse Attention and Heavily Compressed Attention split your prompt into different views, how DeepSeek compresses the KV cache, and the workloads where the whole trick breaks without warning.

Get the hotter takes in your inbox every Tuesday: [https://devsplainers.com/takeouts/](https://devsplainers.com/takeouts/)

CHAPTERS
00:00 The $19 API bill that should have cost four figures
00:36 Why long prompts cost a fortune (attention and the memory bill)
01:48 The skip: most attention math returns nothing
02:54 How DeepSeek built it (summaries, compressed attention, the skim lane)
04:28 Compressed Sparse Attention and the skeptic's six-month deadline
05:12 The rest of the bill: MoE, 4-bit weights, and the rate card
06:31 The catch: where compressed attention answers wrong with confidence
07:43 What cheap long context changes for you

WHAT IS COMPRESSED ATTENTION?
Standard attention compares every word in your prompt against every other word, so doubling the prompt quadruples the work. Measure where that attention actually lands and a handful of earlier words carry nearly all the weight while the rest sit near zero. Compressed attention predicts which words matter, cheaply, then spends the expensive math only on those. The hard part is doing the prediction without burning the savings, and without missing the one line that mattered. DeepSeek's answer, published in the V4 paper, uses Compressed Sparse Attention and Heavily Compressed Attention to create different views of your prompt: compressed summaries, selective retrieval, and a cheap view of the entire context. Send ten thousand tokens or a million and the expensive part reads roughly the same small pile.

COVERED IN THIS VIDEO
Why attention cost grows with the square of the prompt length
The KV cache memory bill and why it raises the price per customer
Attention concentration: the empirical result compressed attention is built on
Why fixed-window shortcuts and post-hoc pruning both failed
Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA)
How DeepSeek compresses the KV cache
How selective retrieval keeps the important information
How the full-context skim lane prevents information from disappearing
DeepSeek V4 pricing: $0.435 per million input tokens
How that compares to GPT-5.6 Luna, Claude Sonnet 5, and Gemini 3.1 Pro
Mixture of experts, 4-bit weights, and a draft model on top of the attention win
Where compressed attention fails: diffuse aggregation, exact strings, short prompts
What cheap long context does to the way developers actually work

RELATED DEVSPLAINERS VIDEOS
KV cache, explained: • KV Cache: The Real Reason Your AI Bill Is ...
Context windows, explained: • Context Window Explained: Why 1M Tokens St...
Speculative decoding, explained: • Speculative Decoding: How a Dumb Model Mak...
What AI coding actually costs: • Why Your AI Coding Bill Is About to Explode

SOURCES
DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence (arXiv:2606.19348)
Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention (arXiv:2502.11089)
Efficient Streaming Language Models with Attention Sinks (arXiv:2309.17453)
The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs (arXiv:2504.17768)
DeepSeek API pricing and the DeepSeek-V4-Flash-0731 model card
OpenAI, Anthropic, and Google Gemini public rate cards
OpenRouter model rankings
Artificial Analysis AA-Omniscience hallucination benchmark
Reuters on DeepSeek's May 2026 price cut
vLLM and SGLang sparse attention implementations and profiling

#DeepSeek #DeepSeekV4 #CompressedAttention #SparseAttention #LLM #AIEngineering