Diffusion Models: DDPM | Generative AI Animated

The first 500 people to use my link skl.sh/deepia05251 will get a 1 month free trial of Skillshare!

In this video you'll learn everything about the DDPM formulation of diffusion models. We go over how this paper simplified the training objective to a simple L2, then how we implement this in practice.
This video was sponsored by Skillshare.

The code to produce the Manim animations, as well as the implementation of the training and sampling codes, is available here:
github.com/ytdeepia/DDPM

Chapters:
00:00 Intro
00:32 General principles
03:28 Forward process
05:54 Variance preserving forward process
09:07 Reverse process
11:26 The ELBO
14:06 Simplifying the ELBO
16:05 From ELBO to L2
19:37 Simplifying the L2
23:04 Training implementation
23:44 Sponsor
24:39 Training implementation
28:14 Sampling implementation
31:24 Conclusion

Shout out to the viewers who contributed to the subtitles:
Vietnamese: Huy Mai
Chinese: Sakura (bili_sakura@zju.edu.cn), MouRen Sun, MZhao Tong
Bahasa Indonesia: Ananda SRW

If you want to dive deeper into this topic here are a few very good ressources on diffusion models.
Blog post:
sander.ai/2023/07/20/perspectives.html (perspectives on diffusion models)
learnopencv.com/denoising-diffusion-probabilistic-… (in depth guide)
nn.labml.ai/diffusion/ddpm/index.html (implementation with equations)
Papers:
arxiv.org/abs/1312.6114 (VAEs)
arxiv.org/abs/1503.03585 (2015 foundational paper on diffusion)
arxiv.org/abs/2006.11239 (DDPM)
arxiv.org/abs/2206.00364 (Elucidating Diffusion Models)
arxiv.org/abs/2011.13456 (Score-Diffusion formulation)
Videos:
   • History of Diffusion -  Sander Dieleman   (Podcast with Sander Dieleman)
   • Why Does Diffusion Work Better than Auto-R...   (Diffusion vs Autoregression)

This video features animations created with Manim, inspired by Grant Sanderson's work at @3blue1brown.
If you enjoyed the content, please like, comment, and subscribe to support the channel!