Stochastic Methods on Chen Kai Bloghttps://www.chenk.top/zh/tags/stochastic-methods/Recent content in Stochastic Methods on Chen Kai BlogHugozh-CNTue, 27 Sep 2022 09:00:00 +0000优化理论（十）：随机优化与方差缩减https://www.chenk.top/zh/optimization-theory/10-stochastic-variance-reduction/Tue, 27 Sep 2022 09:00:00 +0000https://www.chenk.top/zh/optimization-theory/10-stochastic-variance-reduction/<p>随机梯度下降（SGD）每步只采样单个分量梯度，计算代价远低于全梯度方法——但噪声的代价是什么？能否在保持随机采样优势的同时获得确定性方法的快速收敛？本文从「噪声预算」视角出发，量化这一权衡，并推导解决方案。</p>