https://www.chenk.top/en/tags/ensemble-learning/Recent content in Ensemble Learning on Chen Kai BlogHugoenFri, 30 Jan 2026 09:00:00 +0000https://www.chenk.top/en/ml-math-derivations/11-ensemble-learning/Fri, 30 Jan 2026 09:00:00 +0000https://www.chenk.top/en/ml-math-derivations/11-ensemble-learning/<p>Why do mediocre classifiers in a committee outperform a single brilliant one? The answer is straightforward: averaging reduces variance, sequential reweighting reduces bias, and a bit of randomization breaks the correlation that would otherwise negate these benefits. This post delves into the math behind this — bias-variance decomposition, bootstrap aggregating, AdaBoost as forward stagewise minimization of exponential loss, and gradient boosting as gradient descent in function space.</p> <p>By the end, you should be able to look at any ensemble method and say what it reduces, why it works, and when it fails.</p>