Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox


Error Lower Bounds of Constant Step-size Stochastic Gradient Descent

Oct 18, 2019
Zhiyan Ding, Yiding Chen, Qin Li, Xiaojin Zhu


Share this with someone who'll enjoy it:


Stochastic Gradient Descent (SGD) plays a central role in modern machine learning. While there is extensive work on providing error upper bound for SGD, not much is known about SGD error lower bound. In this paper, we study the convergence of constant step-size SGD. We provide error lower bound of SGD for potentially non-convex objective functions with Lipschitz gradients. To our knowledge, this is the first analysis for SGD error lower bound without the strong convexity assumption. We use experiments to illustrate our theoretical results.



   Access Paper Source



Share this with someone who'll enjoy it: