PCA

Feb 5, 2026 ML Math Derivations 26 min read

ML Math Derivations (17): Dimensionality Reduction and PCA

High-dimensional spaces are hostile to distance-based learning. This article derives PCA from two equivalent angles (max variance and min reconstruction error), and extends to kernel PCA, LDA, t-SNE, and ICA -- with …

Apr 9, 2025 Linear Algebra 38 min read

Essence of Linear Algebra (15): Linear Algebra in Machine Learning

Machine learning speaks linear algebra as its native language. From PCA to SVMs, from matrix factorization in recommender systems to gradient descent optimization -- see how vectors, matrices, and decompositions power …

Feb 26, 2025 Linear Algebra 30 min read

Essence of Linear Algebra (9): Singular Value Decomposition — The Crown Jewel of Linear Algebra

SVD decomposes any matrix -- not just square or symmetric ones. From image compression to Netflix recommendations, from face recognition to gene analysis, SVD is the most powerful and most universal decomposition in …

Dec 14, 2021 Kernel Methods 44 min read

Kernel Methods (5): Kernel SVM, Kernel PCA, and Kernel Ridge Regression

The classic algorithms, kernelized — SVM's dual form, Kernel PCA's eigendecomposition in feature space, and Kernel Ridge's closed-form solution. With sklearn code and worked examples.