Get our free extension to see links to code for papers anywhere online!

Chrome logo  Add to Chrome

Firefox logo Add to Firefox

Beyond neural scaling laws: beating power law scaling via data pruning


Jun 29, 2022
Ben Sorscher, Robert Geirhos, Shashank Shekhar, Surya Ganguli, Ari S. Morcos


   Access Paper or Ask Questions

  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

Lottery Tickets on a Data Diet: Finding Initializations with Sparse Trainable Networks


Jun 02, 2022
Mansheej Paul, Brett W. Larsen, Surya Ganguli, Jonathan Frankle, Gintare Karolina Dziugaite

* The first two authors contributed equally 

   Access Paper or Ask Questions

  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

MetaMorph: Learning Universal Controllers with Transformers


Mar 22, 2022
Agrim Gupta, Linxi Fan, Surya Ganguli, Li Fei-Fei

* ICLR 2022 

   Access Paper or Ask Questions

  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

Rethinking the limiting dynamics of SGD: modified loss, phase space oscillations, and anomalous diffusion


Jul 19, 2021
Daniel Kunin, Javier Sagastuy-Brena, Lauren Gillespie, Eshed Margalit, Hidenori Tanaka, Surya Ganguli, Daniel L. K. Yamins

* 30 pages, 8 figures 

   Access Paper or Ask Questions

  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

Deep Learning on a Data Diet: Finding Important Examples Early in Training


Jul 15, 2021
Mansheej Paul, Surya Ganguli, Gintare Karolina Dziugaite

* 18 pages, 16 figures 

   Access Paper or Ask Questions

  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

How many degrees of freedom do we need to train deep networks: a loss landscape perspective


Jul 13, 2021
Brett W. Larsen, Stanislav Fort, Nic Becker, Surya Ganguli


   Access Paper or Ask Questions

  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

Understanding self-supervised Learning Dynamics without Contrastive Pairs


Feb 12, 2021
Yuandong Tian, Xinlei Chen, Surya Ganguli


   Access Paper or Ask Questions

  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

Embodied Intelligence via Learning and Evolution


Feb 03, 2021
Agrim Gupta, Silvio Savarese, Surya Ganguli, Li Fei-Fei

* Video available at https://youtu.be/MMrIiNavkuY 

   Access Paper or Ask Questions

  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics


Dec 08, 2020
Daniel Kunin, Javier Sagastuy-Brena, Surya Ganguli, Daniel L. K. Yamins, Hidenori Tanaka

* 28 pages, 17 figures 

   Access Paper or Ask Questions

  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernel


Oct 28, 2020
Stanislav Fort, Gintare Karolina Dziugaite, Mansheej Paul, Sepideh Kharaghani, Daniel M. Roy, Surya Ganguli

* 19 pages, 19 figures, In Advances in Neural Information Processing Systems 34 (NeurIPS 2020) 

   Access Paper or Ask Questions

  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email
1
2
3
4
5
6
>>