Score Matching Explained - The Key Idea Behind Diffusion Models

Описание: In this second video of the diffusion model series, we look at score matching. We see how score prediction can be combined with Langevin dynamics for generative modelling, and how the DDPM's noise matching loss is equivalent to score matching.

Chapters
---------------
00:00 Introduction
01:32 Generative modelling
03:27 Score matching
07:38 Denoising autoencoders
11:16 Denoising = score matching
14:39 Langevin sampling
20:16 Noise conditional score networks
23:30 NCSN sampling algorithm
24:37 Connection with DDPMs
27:34 Summary

Useful links
-------------------
Paper that introduced score matching (Hyvärinen, 2005): https://jmlr.org/papers/volume6/hyvar...
Paper that connected score matching to denoising (Vincent, 2001): https://www.iro.umontreal.ca/~vincent...
NCSN paper (Song & Ermon, 2019): https://arxiv.org/pdf/1907.05600
Lecture notes on Langevin dynamics: https://www.damtp.cam.ac.uk/user/tong...
Nice blog post on Langevin sampling: https://abdulfatir.com/blog/2020/Lang...
Paper on Langevin sampling for parameter estimation: https://www.stats.ox.ac.uk/~teh/resea...
Official NCSN Github (PyTorch): https://github.com/ermongroup/ncsn