Build a Small Language Model (SLM) From Scratch

Описание: In this video, Dr. Raj Dandekar (MIT PhD) teaches you how to build a production level SLM entirely from scratch.

You will learn the following:

(1) Creating the dataset
(2) Tokenizing the dataset
(3) Creating input-target pairs
(4) Creating the entire SLM architecture
(5) Setup the SLM for pre-training
(6) Pre-training the SLM
(7) Inference

Google Colab Notebook: https://colab.research.google.com/dri...