Multi-Token Prediction Introduction

Описание: In this lecture, we learn the intuition behind Multi-Token Prediction (MTP), which was introduced by Meta in this paper: https://arxiv.org/pdf/2404.19737

MTP was later modified by DeepSeek in their architecture.

In this lecture, we learn about:

(1) What is Multi-Token Prediction (MTP)?
(2) What are the uses of MTP?