Diffusion Models on Chen Kai Blog

PDE and ML (7): Diffusion Models and Score Matching

Tue, 30 Jul 2024 09:00:00 +0000

The output side of a diffusion model is familiar: a high-quality image. The training objective, on the other hand, looks counter-intuitive at first sight — add noise to the data until it is fully Gaussian, then learn to denoise step by step. Why is this detour more effective than learning the data distribution directly?

The answer is hidden in PDEs. The forward noising process is a heat equation (or, more generally, a Fokker–Planck equation), and it admits a reverse-time version — provided we know the score (the gradient of the log-density) at every time. Score matching is the standard way to learn that score. From this angle, DDPM, DDIM, and score-based SDEs are not three different algorithms but three discretizations of the same PDE story.

Ordinary Differential Equations (18): Frontiers and Series Finale

Mon, 15 Apr 2024 09:00:00 +0000

The journey ends here. Eighteen chapters ago we picked up a falling apple. Today we’re going to finish in the same vein in which we began — by treating ODEs as the universal language of change — but standing on a much taller mountain.

This chapter does three things. First, it surveys four active research frontiers that are reshaping how we model dynamical systems: Neural ODEs, delay equations, stochastic differential equations, and fractional calculus. Second, it reviews the entire series with a problem-solving flowchart and a chapter-by-chapter map. Third, it draws explicit connections from the classical theory you have just mastered to modern machine learning — the place where ODEs are most alive in 2025.