The efficiency misnomer | Size does not matter | What does the number of parameters mean in a model?

Описание: How important is the number of parameters in deep learning models? But what about other measures like FLOPs or speed/throughput?
► Check out our sponsor Aleph Alpha 👉 https://www.aleph-alpha.de/ !
Follow them on Twitter: Aleph__Alpha

Paper 📜:
Dehghani, Mostafa, Anurag Arnab, Lucas Beyer, Ashish Vaswani, and Yi Tay. "The Efficiency Misnomer." arXiv preprint arXiv:2110.12894 (2021). https://arxiv.org/abs/2110.12894

🔗 Megatron-Turing NLG 530B: https://www.microsoft.com/en-us/resea...

Thanks to our Patrons who support us in Tier 2, 3, 4: 🙏
donor, Dres. Trost GbR, Yannik Schneider
➡️ AI Coffee Break Merch! 🛍️ https://aicoffeebreak.creator-spring....

Outline:
00:00 Model efficiency comparison
02:51 FLOPs
03:55 Number of parameters: means what?
06:31 Speed / throughput
09:39 Aleph Alpha (Sponsor)

▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
🔥 Optionally, pay us a coffee to help with our Coffee Bean production! ☕
Patreon:   / aicoffeebreak  
Ko-fi: https://ko-fi.com/aicoffeebreak
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀

🔗 Links:
AICoffeeBreakQuiz:    / aicoffeebreak  
Twitter:   / aicoffeebreak  
Reddit:   / aicoffeebreak  
YouTube:    / aicoffeebreak  

#AICoffeeBreak #MsCoffeeBean #MachineLearning #AI #research​