Diffusion Transformers (DiT) Explained: Replacing U-Nets with Transformers

Описание: Transformers revolutionized NLP and computer vision — but can they replace U-Nets in diffusion models?

In this video, we break down the DiT (Diffusion Transformer) paper by William Peebles and Saining Xie, covering:

How diffusion models work
Why latent diffusion matters
Patchifying latent representations
Conditioning methods:
In-context tokens
Cross-attention
adaLN / adaLN-Zero
Why adaLN-Zero works so well
Scaling laws in diffusion transformers
Why GFlops matter more than parameter count
State-of-the-art ImageNet results

We also compare DiT against traditional U-Net diffusion architectures and explain why Transformers scale so effectively for image generation.

Slides based on:
“Scalable Diffusion Models with Transformers”