Knowledge Distillation in Deep Learning - DistilBERT Explained

Описание: In this video, i try to explain how distilBERT model was trained to create a smaller faster version of the famous BERT model using knowledge distillation technique.

Previous Video on the Basics of Knowledge Distillation :    • Knowledge Distillation in Deep Learning - ...  

Cross Entropy Loss :    • Why do we need Cross Entropy Loss? (Visual...