Lecture 03 - Pruning and Sparsity (Part I) | MIT 6.S965

Описание: Lecture 3 gives an introduction to the basics of neural network pruning which can reduce the parameter counts of neural networks by more than 90%, decreasing the storage requirements and improving the computation efficiency of neural networks. In this lecture, we go through all steps of pruning and introduce different granularities and criteria of neural network pruning.

Keywords: Neural Network Pruning, Pruning, Magnitude-based Pruning, Channel Pruning, Fine-grained Pruning

Slides: https://efficientml.ai/schedule/

--------------------------------------------------------------------------------------

TinyML and Efficient Deep Learning Computing

Instructors:
Song Han: https://songhan.mit.edu

Have you found it difficult to deploy neural networks on mobile devices and IoT devices? Have you ever found it too slow to train neural networks? This course is a deep dive into efficient machine learning techniques that enable powerful deep learning applications on resource-constrained devices. Topics cover efficient inference techniques, including model compression, pruning, quantization, neural architecture search, and distillation; and efficient training techniques, including gradient compression and on-device transfer learning; followed by application-specific model optimization techniques for videos, point cloud, and NLP; and efficient quantum machine learning. Students will get hands-on experience implementing deep learning applications on microcontrollers, mobile phones, and quantum machines with an open-ended design project related to mobile AI.

Website:
http://efficientml.ai/