Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Aitor Lewkowycz

How to decay your learning rate


Mar 23, 2021
Aitor Lewkowycz

* 9 + 14 pages, 5 + 11 figures 

  Access Paper or Ask Questions

On the training dynamics of deep networks with $L_2$ regularization


Jun 15, 2020
Aitor Lewkowycz, Guy Gur-Ari

* 10+12 pages, 5+10 figures 

  Access Paper or Ask Questions

The large learning rate phase of deep learning: the catapult mechanism


Mar 04, 2020
Aitor Lewkowycz, Yasaman Bahri, Ethan Dyer, Jascha Sohl-Dickstein, Guy Gur-Ari

* 25 pages, 19 figures 

  Access Paper or Ask Questions