How ChatGPT Actually Thinks (It Doesn't Know Its Next Word)

Описание: Every answer ChatGPT gives is built one tiny piece at a time, by a machine that never knows what it will say next. In this video we open the blueprint: tokens, embeddings, attention, next-token prediction, training, fine-tuning, and why AI sometimes makes things up.

Chapters
0:00 Intro
0:15 The whole machine in one line
0:33 Step 1: Tokens
0:55 Step 2: Embeddings
1:17 Step 3: Attention
1:43 Step 4: Prediction
2:04 The loop and temperature
2:27 Training
2:52 Fine-tuning
3:16 Hallucinations
3:44 Does ChatGPT think?

Blueprint Lab: the blueprints behind the tech you use every day. Subscribe for more.

Source for the token rule of thumb: OpenAI Help Center, "What are tokens and how to count them"

This video uses AI-generated narration.