Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Ethan Dyer

Explaining Neural Scaling Laws


Feb 12, 2021
Yasaman Bahri, Ethan Dyer, Jared Kaplan, Jaehoon Lee, Utkarsh Sharma

* 11 pages, 5 figures + Supplement 

  Access Paper or Ask Questions

When Do Curricula Work?


Dec 05, 2020
Xiaoxia Wu, Ethan Dyer, Behnam Neyshabur

* ICLR 2021 

  Access Paper or Ask Questions

Whitening and second order optimization both destroy information about the dataset, and can make generalization impossible


Aug 25, 2020
Neha S. Wadia, Daniel Duckworth, Samuel S. Schoenholz, Ethan Dyer, Jascha Sohl-Dickstein

* 15+7 pages, 7 figures; added references, edited model descriptions for clarity, results unchanged 

  Access Paper or Ask Questions

Asymptotics of Wide Convolutional Neural Networks


Aug 19, 2020
Anders Andreassen, Ethan Dyer

* 23 pages, 12 figures 

  Access Paper or Ask Questions

Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semantics


Jul 14, 2020
Vinay V. Ramasesh, Ethan Dyer, Maithra Raghu


  Access Paper or Ask Questions

The large learning rate phase of deep learning: the catapult mechanism


Mar 04, 2020
Aitor Lewkowycz, Yasaman Bahri, Ethan Dyer, Jascha Sohl-Dickstein, Guy Gur-Ari

* 25 pages, 19 figures 

  Access Paper or Ask Questions

Affinity and Diversity: Quantifying Mechanisms of Data Augmentation


Feb 20, 2020
Raphael Gontijo-Lopes, Sylvia J. Smullin, Ekin D. Cubuk, Ethan Dyer

* 10 pages, 7 figures 

  Access Paper or Ask Questions

Asymptotics of Wide Networks from Feynman Diagrams


Sep 25, 2019
Ethan Dyer, Guy Gur-Ari

* 10 pages, 3 figures, 1 Table + Appendices 

  Access Paper or Ask Questions

Gradient Descent Happens in a Tiny Subspace


Dec 12, 2018
Guy Gur-Ari, Daniel A. Roberts, Ethan Dyer

* 9 pages + appendices, 12 figures 

  Access Paper or Ask Questions