Kernel Methods (6): Gaussian Processes — When Kernels Meet Bayesian Inference

Sun, 19 Dec 2021 09:00:00 +0000

Kernel ridge regression gives you a number. You feed it $$x_*$$ , it returns $\hat{y}_* = 23.7$ . End of story. But you wanted to act on that prediction — maybe schedule a delivery, dose a patient, place a bet — and the single number is not enough. Tomorrow’s temperature being “25°C” is useful; “very likely 25°C, 95% chance between 22 and 28” is actionable. Every decision under uncertainty needs the second one. Gaussian Processes are the cleanest way to upgrade a kernel method from “point predictor” to “distribution predictor”, and they do it without abandoning a single line of the kernel math from the previous five parts.

Kernel Methods (4): Common Kernel Families — RBF, Matern, Polynomial, Periodic, and More

Thu, 09 Dec 2021 09:00:00 +0000

You type SVC(kernel='rbf') in scikit-learn for the first time. What did you set gamma to? 'scale'? 'auto'? You scrolled past those defaults without thinking. Three months later your model is overfitting, your Gram matrix looks like the identity, and you have no idea which knob is wrong. Most “kernel tuning” debt is really kernel choice debt — you picked the default kernel for the wrong reason, and now no amount of grid search will save you.

Gaussian Processes on Chen Kai Blog

Kernel Methods (6): Gaussian Processes — When Kernels Meet Bayesian Inference

Kernel Methods (4): Common Kernel Families — RBF, Matern, Polynomial, Periodic, and More