https://www.chenk.top/en/tags/optimization-theory/Recent content in Optimization Theory on Chen Kai BlogHugoenFri, 31 May 2024 09:00:00 +0000https://www.chenk.top/en/pde-ml/03-variational-principles/Fri, 31 May 2024 09:00:00 +0000https://www.chenk.top/en/pde-ml/03-variational-principles/<p>What is the essence of neural-network training? When we run gradient descent in a high-dimensional parameter space, is there a deeper continuous-time dynamics at work? As the network width tends to infinity, does discrete parameter updating converge to some elegant partial differential equation? The answers live at the intersection of the calculus of variations, optimal transport, and PDE theory.</p> <p>The last decade of deep-learning success has rested mostly on engineering intuition. Recently, however, mathematicians have made a striking observation: <strong>viewing a neural network as a particle system on the space of probability measures</strong>, and studying its evolution under Wasserstein geometry, exposes the global structure of training — convergence guarantees, the role of over-parameterization, the meaning of initialization. The tool that makes this visible is <strong>the variational principle</strong> — from least action in physics, through the JKO scheme of modern optimal transport, to the mean-field limit of neural networks.</p>