https://www.chenk.top/en/tags/ica/Recent content in ICA on Chen Kai BlogHugoenThu, 05 Feb 2026 09:00:00 +0000https://www.chenk.top/en/ml-math-derivations/17-dimensionality-reduction-and-pca/Thu, 05 Feb 2026 09:00:00 +0000https://www.chenk.top/en/ml-math-derivations/17-dimensionality-reduction-and-pca/<p><figure class="article-figure"> <img src="https://blog-pic-ck.oss-cn-beijing.aliyuncs.com/posts/en/ml-math-derivations/17-Dimensionality-Reduction-and-PCA/illustration_1.png" alt="ML Math Derivations (17): Dimensionality Reduction and PCA — Chapter overview" loading="lazy" decoding="async" class="content-image"> </figure> </p> <hr> <h2 id="what-you-will-learn" class="heading-anchor">What You Will Learn<a href="#what-you-will-learn" class="heading-link" aria-label="Permalink to this section" title="Copy link to this section">#</a> </h2><p>Feed a clustering algorithm <span class="math-inline">$10{,}000$</span> -dimensional data and it will most likely fail — not because the algorithm is broken, but because <strong>high-dimensional space is a hostile environment for distance-based learning</strong>. Volumes evaporate into thin shells, the ratio of nearest- to farthest-neighbour distances tends to <span class="math-inline">$1$</span> , and &ldquo;closeness&rdquo; stops carrying information. Dimensionality reduction is the response: project the data into a lower-dimensional space while keeping the structure that actually matters.</p>