Get our free extension to see links to code for papers anywhere online!

Chrome logo  Add to Chrome

Firefox logo Add to Firefox

Rethinking the limiting dynamics of SGD: modified loss, phase space oscillations, and anomalous diffusion


Jul 19, 2021
Daniel Kunin , Javier Sagastuy-Brena , Lauren Gillespie , Eshed Margalit , Hidenori Tanaka , Surya Ganguli , Daniel L. K. Yamins

* 30 pages, 8 figures 

   Access Paper or Ask Questions

  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

Beyond BatchNorm: Towards a General Understanding of Normalization in Deep Learning


Jul 08, 2021
Ekdeep Singh Lubana , Robert P. Dick , Hidenori Tanaka


   Access Paper or Ask Questions

  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

Noether's Learning Dynamics: The Role of Kinetic Symmetry Breaking in Deep Learning


May 06, 2021
Hidenori Tanaka , Daniel Kunin


   Access Paper or Ask Questions

  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics


Dec 08, 2020
Daniel Kunin , Javier Sagastuy-Brena , Surya Ganguli , Daniel L. K. Yamins , Hidenori Tanaka

* 28 pages, 17 figures 

   Access Paper or Ask Questions

  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

Pruning neural networks without any data by iteratively conserving synaptic flow


Jun 09, 2020
Hidenori Tanaka , Daniel Kunin , Daniel L. K. Yamins , Surya Ganguli


   Access Paper or Ask Questions

  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

From deep learning to mechanistic understanding in neuroscience: the structure of retinal prediction


Dec 12, 2019
Hidenori Tanaka , Aran Nayebi , Niru Maheswaranathan , Lane McIntosh , Stephen A. Baccus , Surya Ganguli

* Neural Information Processing Systems (NeurIPS), 2019 

   Access Paper or Ask Questions

  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email