Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Nesterov's Accelerated Gradient and Momentum as approximations to Regularised Update Descent

Jul 11, 2016
Aleksandar Botev, Guy Lever, David Barber

Share this with someone who'll enjoy it:

We present a unifying framework for adapting the update direction in gradient-based iterative optimization methods. As natural special cases we re-derive classical momentum and Nesterov's accelerated gradient method, lending a new intuitive interpretation to the latter algorithm. We show that a new algorithm, which we term Regularised Gradient Descent, can converge more quickly than either Nesterov's algorithm or the classical momentum algorithm.

   Access Paper Source

Share this with someone who'll enjoy it: