https://www.chenk.top/en/tags/expectation-maximization/Recent content in Expectation Maximization on Chen Kai BlogHugoenSun, 01 Feb 2026 09:00:00 +0000https://www.chenk.top/en/ml-math-derivations/13-em-algorithm-and-gmm/Sun, 01 Feb 2026 09:00:00 +0000https://www.chenk.top/en/ml-math-derivations/13-em-algorithm-and-gmm/<p>When data has hidden structure — like an unobserved cluster label, a missing feature, or an unseen topic — maximum likelihood becomes challenging. The log of a sum has no closed form, and gradient methods get entangled with the latent variables. The <strong>EM algorithm</strong> sidesteps the difficulty with a deceptively simple idea: alternate between <em>guessing</em> the hidden variables under a posterior (E-step) and <em>fitting</em> the parameters as if those guesses were true (M-step). Each iteration is mathematically guaranteed to push the likelihood up. This post derives EM from first principles, proves the monotone-ascent property using Jensen&rsquo;s inequality, and explores its most famous application: <strong>Gaussian Mixture Models (GMM)</strong> — the soft, elliptical generalization of K-means.</p>