arXiv:2611.24562v1 Announce Type: cross
Abstract: In this paper, we study stochastic gradient descent in the smooth interpolation regime, where the population objective shares a common minimizer with the sample losses. In contrast to prior work, which has largely focused on in-expectation analysis, we obtain high-probability convergence guarantees under the same assumptions. In the convex case, we prove a fast rate of order $\log(\beta^{-1}) \cdot n^{-1}$ after $n$ iterations, with failure probability at most $\beta$. In the strongly convex case, we derive a family of moment-based bounds of order $\beta^{-p} \cdot \exp(-\mathcal{O}(p \cdot n))$ for $p \in (0,1]$, and complement them with a matching lower bound showing that the corresponding deterioration in the exponential rate is essentially unavoidable. In both settings, our theoretical results are unified across a broad class of step-size schedules. Building on this analysis, we extend our results to a low-noise regime and to nonconvex objectives satisfying the Polyak-Lojasiewicz condition.
Trending
- The most misunderstood graph in AI just got a major update
- OpenAI Brings Its Collaborative Space Hubs To Plus Users
- Adaptive-View Self-Distillation by Balancing Consensus and Teacher-Specific Privileged Signals
- Internet Archive Staffers Are Unionizing
- Improving Imitation Policies for Multi-Phase Tasks through Residual Learning Allocation
- Nothing’s AI Device Debuts December 8, And It Looks Like Two Vapes Stuck Together
- Why We Should Rethink the Sequential SFT-RL Paradigm
- Chips can survive the space environment. Can their financing?
