Energy-Based Models using Non-Equilibrium Thermodynamics - Davide Carbone - Young Seminars SIFS

Описание: Energy-Based Models using Non-Equilibrium Thermodynamics
Davide Carbone, Politecnico di Torino

Abstract: In recent years, generative diffusion models (GDM) have emerged as a powerful class of models for generating high-quality data across various domains, even in the context of scientific machine learning. These models operate by gradually transforming a simple, tractable distribution into a complex data distribution through a series of diffusion steps. In this talk I will firstly give a summary of the strict relation between physics and GDM. Then I will show some results about energy-based models (EBMs), which have gained significant attention due to their ability to model complex data distributions and to provide an interpretation of energy landscapes. I will show how is possible to leverage tools from nonequilibrium statistical physics to improve the training of EBMs, usually performed via Contrastive Learning or modification.

References:
[1] Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N., and Ganguli, S. (2015). Deep unsupervised learning using nonequilibrium thermodynamics. In: International conference on machine learning (pp. 2256-2265).
[2] Carbone, D., Hua, M., Coste, S., and Vanden-Eijnden, E. (2023). Efficient training of energy-based models using jarzynski equality. In:Advances in Neural Information Processing Systems, 36, 52583-52614.
[3] Carbone, D., Hua, M., Coste, S., and Vanden-Eijnden, E. (2024). Generative Models as Out-of-Equilibrium Particle Systems: Training of Energy-Based Models Using Non-equilibrium Thermodynamics. In: International Conference on Nonlinear Dynamics and Applications (pp. 287-311)

#YSSIFS