ML Math Derivations (9): Naive Bayes

Wed, 28 Jan 2026 09:00:00 +0000

Hook: A spam filter that trains in milliseconds, scales to a million features, has no hyperparameters worth tuning, and still beats much fancier models on short-text problems. Naive Bayes pulls this off by making one outrageous assumption — every feature is independent given the class — and refusing to apologise for it. The assumption is wrong on essentially every real dataset, yet the classifier works. Understanding why is a tour through generative modelling, MAP estimation, Dirichlet priors, and the bias–variance tradeoff. This article walks the entire path.

Probability and Statistics (1): Probability Spaces — Why We Need Axioms (But Won't Overdo It)

Sun, 18 Aug 2024 09:00:00 +0000

Every time you check the weather forecast, run an A/B test, or train a neural network, you are standing on a foundation laid in 1933 by a Russian mathematician named Andrey Kolmogorov. Before him, probability was a grab bag of tricks for gamblers and actuaries. After him, it became a branch of mathematics as rigorous as calculus or algebra.

The good news: you don’t need to become a measure theorist to understand modern probability. The axioms are simple. What takes work is building the right intuitions around them — and learning to recognize when those intuitions fail.

Bayes Theorem on Chen Kai Blog

ML Math Derivations (9): Naive Bayes

Probability and Statistics (1): Probability Spaces — Why We Need Axioms (But Won't Overdo It)