One Letter at a Time: The Granular Math of Bigram Language Models
Автор: Ramit Agarwal
Загружено: 2026-06-27
Просмотров: 105
Описание:
Have you ever wondered how AI actually learns to write? Before diving into massive Large Language Models with billions of parameters, it helps to understand the most basic question of AI: given what we have seen so far, what should come next?
In this video, we are getting granular. We will build a character-level bigram language model completely from scratch using a dataset of over 32,000 names.
By focusing on two-character sequences (bigrams), we will uncover the exact math and code needed to teach a model how to predict the next letter.
In this tutorial, we cover:
The 27x27 Matrix: How to extract bigrams and store character counts in a matrix.
Probability & Smoothing: Converting raw counts into probability distributions using "add-one smoothing".
Coding the Neural Network: Replicating our count-based model using PyTorch, one-hot encoding, and a single-layer neural network.
Gradient Descent & NLL: Training the network by minimizing the Negative Log-Likelihood (NLL).
Generating Names: Watching our AI generate brand new names by
sampling from the probability distributions it learned.
Whether you prefer the simple math of counting or the structure of a PyTorch neural network, this video will show you that both methods are essentially learning the exact same 27x27 table of probabilities!
Повторяем попытку...
Доступные форматы для скачивания:
Скачать видео
-
Информация по загрузке: