Get our free extension to see links to code for papers anywhere online!

Chrome logo  Add to Chrome

Firefox logo Add to Firefox

The Slingshot Mechanism: An Empirical Study of Adaptive Optimizers and the Grokking Phenomenon


Jun 13, 2022
Vimal Thilak, Etai Littwin, Shuangfei Zhai, Omid Saremi, Roni Paiss, Joshua Susskind

Add code

* Removed Tex formatting commands in title Title and Abstract 

   Access Paper or Ask Questions

  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

Learning Representation from Neural Fisher Kernel with Low-rank Approximation


Feb 04, 2022
Ruixiang Zhang, Shuangfei Zhai, Etai Littwin, Josh Susskind

Add code


   Access Paper or Ask Questions

  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

Implicit Greedy Rank Learning in Autoencoders via Overparameterized Linear Networks


Jul 02, 2021
Shih-Yu Sun, Vimal Thilak, Etai Littwin, Omid Saremi, Joshua M. Susskind

Add code


   Access Paper or Ask Questions

  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

Implicit Acceleration and Feature Learning in Infinitely Wide Neural Networks with Bottlenecks


Jul 02, 2021
Etai Littwin, Omid Saremi, Shuangfei Zhai, Vimal Thilak, Hanlin Goh, Joshua M. Susskind, Greg Yang

Add code


   Access Paper or Ask Questions

  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

Tensor Programs IIb: Architectural Universality of Neural Tangent Kernel Training Dynamics


May 08, 2021
Greg Yang, Etai Littwin

Add code

* ICML 2021 

   Access Paper or Ask Questions

  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

Collegial Ensembles


Jun 17, 2020
Etai Littwin, Ben Myara, Sima Sabah, Joshua Susskind, Shuangfei Zhai, Oren Golan

Add code


   Access Paper or Ask Questions

  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

On the Optimization Dynamics of Wide Hypernetworks


Apr 05, 2020
Etai Littwin, Tomer Galanti, Lior Wolf

Add code

* The first two authors contributed equally 

   Access Paper or Ask Questions

  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

Residual Tangent Kernels


Feb 18, 2020
Etai Littwin, Lior Wolf

Add code


   Access Paper or Ask Questions

  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email
1
2
>>