LLM’s Billion Dollar Problem

Описание: Check out Inngest and let your AI agents wear a harness now https://innge.st/yt-bycl-1

This video originally was going to be about the Linear Attention Saga that happened between June and November last year, but it turned out I needed quite some build up to explain what the significance of Linear Attention, 1 million context window, and compute scaling are. So it accidentally became a 17 mins video...


my latest project: Intuitive AI Academy
https://intuitiveai.academy/
limited time code "NYNM" for 50% off forever (only a 25 spots left!)

My Newsletter
https://mail.bycloud.ai/

my project: find, discover & explain AI research semantically
https://findmypapers.ai/

My Patreon
  / bycloud  

Sauces (will try to be in order of apperance)

GPT-OSS
[Code] https://github.com/openai/gpt-oss
[Paper] https://arxiv.org/abs/2508.10925
(they did not mention it is sliding window attention explicitly in its paper)

DeepSeek V3.2
[Paper] https://arxiv.org/pdf/2512.02556

DeepSeek V2 (Multi-head Latent Attention)
[Paper] https://arxiv.org/abs/2405.04434

Kimi K2.5
[Paper] https://arxiv.org/abs/2602.02276

MiniMax
[Text-01] https://arxiv.org/abs/2501.08313
[M1] https://arxiv.org/abs/2506.13585
[M2] https://huggingface.co/MiniMaxAI/Mini...
[Zhihu Blog] https://www.zhihu.com/question/196530...

Qwen-3 Next
[Project Page] https://qwen.ai/blog?id=4074cca803931...

Hunyuan T1
[Project Page] https://tencent.github.io/llm.hunyuan...

Kimi Linear
[Paper] https://arxiv.org/abs/2510.26692

Context Arena (used for comparing long context performance)
[Project Page] https://contextarena.ai/

Gemini 3 Flash
[Blog] https://blog.google/products-and-plat...

Claude 4.6
[Blog] https://www.anthropic.com/news/claude...


Try out my new fav place to learn how to code https://scrimba.com/?via=bycloudAI

This video is supported by the kind Patrons & YouTube Members:
🙏Spam Maj, Alex, Chris LeDoux, DX Research Group, Poof N' Inu, Deagan, Robert Zawiasa, Ryszard Warzocha, Midwstmakr, Tobe2d, Louis Muk, Akkusativ, Kevin Tai, Mark Buckler, NO U, Tony Jimenez, Ângelo Fonseca, jiye, Anushka, Asad Dhamani, Binnie Yiu, Calvin Yan, Clayton Ford, Diego Silva, Etrotta, Gonzalo Fidalgo, Handenon, Hector, Jake Disco very, Michael Brenner, Nilly K, OlegWock, Daddy Wen, Shuhong Chen, Sid_Cipher, Stefan Lorenz, Sup, tantan assawade, Thipok Tham, Thomas Di Martino, Thomas Lin, Richárd Nagyfi, Paperboy, mika, Leo, Berhane-Meskel, Kadhai Pesalam, mayssam, Bill Mangrum, nyaa, Toru Mon, Lame Plane, Matej Macak, thechoephix


[Discord]   / discord  
[Twitter]   / bycloudai  
[Patreon]   / bycloud  
[Business Inquiries] [email protected]
[Music] ‪@IraStoria‬
[Profile & Banner Art]   / pygm7  
[Video Editor] Abhay and ‪@Booga04‬
[Ko-fi] https://ko-fi.com/bycloudai