Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Surya Ganguli

Rethinking the limiting dynamics of SGD: modified loss, phase space oscillations, and anomalous diffusion


Jul 19, 2021
Daniel Kunin, Javier Sagastuy-Brena, Lauren Gillespie, Eshed Margalit, Hidenori Tanaka, Surya Ganguli, Daniel L. K. Yamins

* 30 pages, 8 figures 

  Access Paper or Ask Questions

Deep Learning on a Data Diet: Finding Important Examples Early in Training


Jul 15, 2021
Mansheej Paul, Surya Ganguli, Gintare Karolina Dziugaite

* 18 pages, 16 figures 

  Access Paper or Ask Questions

How many degrees of freedom do we need to train deep networks: a loss landscape perspective


Jul 13, 2021
Brett W. Larsen, Stanislav Fort, Nic Becker, Surya Ganguli


  Access Paper or Ask Questions

Understanding self-supervised Learning Dynamics without Contrastive Pairs


Feb 12, 2021
Yuandong Tian, Xinlei Chen, Surya Ganguli


  Access Paper or Ask Questions

Embodied Intelligence via Learning and Evolution


Feb 03, 2021
Agrim Gupta, Silvio Savarese, Surya Ganguli, Li Fei-Fei

* Video available at https://youtu.be/MMrIiNavkuY 

  Access Paper or Ask Questions

Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics


Dec 08, 2020
Daniel Kunin, Javier Sagastuy-Brena, Surya Ganguli, Daniel L. K. Yamins, Hidenori Tanaka

* 28 pages, 17 figures 

  Access Paper or Ask Questions

Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernel


Oct 28, 2020
Stanislav Fort, Gintare Karolina Dziugaite, Mansheej Paul, Sepideh Kharaghani, Daniel M. Roy, Surya Ganguli

* 19 pages, 19 figures, In Advances in Neural Information Processing Systems 34 (NeurIPS 2020) 

  Access Paper or Ask Questions

Understanding Self-supervised Learning with Dual Deep Networks


Oct 22, 2020
Yuandong Tian, Lantao Yu, Xinlei Chen, Surya Ganguli


  Access Paper or Ask Questions

Identifying Learning Rules From Neural Network Observables


Oct 22, 2020
Aran Nayebi, Sanjana Srivastava, Surya Ganguli, Daniel L. K. Yamins

* NeurIPS 2020 Camera Ready Version, 21 pages including supplementary information, 13 figures 

  Access Paper or Ask Questions

RNNs can generate bounded hierarchical languages with optimal memory


Oct 15, 2020
John Hewitt, Michael Hahn, Surya Ganguli, Percy Liang, Christopher D. Manning

* EMNLP2020 + appendix typo fixes 

  Access Paper or Ask Questions

Predictive coding in balanced neural networks with noise, chaos and delays


Jun 25, 2020
Jonathan Kadmon, Jonathan Timcheck, Surya Ganguli


  Access Paper or Ask Questions

Pruning neural networks without any data by iteratively conserving synaptic flow


Jun 09, 2020
Hidenori Tanaka, Daniel Kunin, Daniel L. K. Yamins, Surya Ganguli


  Access Paper or Ask Questions

Two Routes to Scalable Credit Assignment without Weight Symmetry


Feb 28, 2020
Daniel Kunin, Aran Nayebi, Javier Sagastuy-Brena, Surya Ganguli, Jon Bloom, Daniel L. K. Yamins

* 19 pages including supplementary information, 10 figures 

  Access Paper or Ask Questions

From deep learning to mechanistic understanding in neuroscience: the structure of retinal prediction


Dec 12, 2019
Hidenori Tanaka, Aran Nayebi, Niru Maheswaranathan, Lane McIntosh, Stephen A. Baccus, Surya Ganguli

* Neural Information Processing Systems (NeurIPS), 2019 

  Access Paper or Ask Questions

Emergent properties of the local geometry of neural loss landscapes


Oct 14, 2019
Stanislav Fort, Surya Ganguli

* 10 pages, 8 figures 

  Access Paper or Ask Questions

Universality and individuality in neural dynamics across large populations of recurrent networks


Jul 19, 2019
Niru Maheswaranathan, Alex H. Williams, Matthew D. Golub, Surya Ganguli, David Sussillo


  Access Paper or Ask Questions

Fast Convolutive Nonnegative Matrix Factorization Through Coordinate and Block Coordinate Updates


Jun 29, 2019
Anthony Degleris, Ben Antin, Surya Ganguli, Alex H Williams

* 10 pages, 5 figures 

  Access Paper or Ask Questions

Reverse engineering recurrent networks for sentiment classification reveals line attractor dynamics


Jun 25, 2019
Niru Maheswaranathan, Alex Williams, Matthew D. Golub, Surya Ganguli, David Sussillo


  Access Paper or Ask Questions

A Unified Theory of Early Visual Representations from Retina to Cortex through Anatomically Constrained Deep CNNs


Jan 03, 2019
Jack Lindsey, Samuel A. Ocko, Surya Ganguli, Stephane Deny

* International Conference on Learning Representations, 2019 https://openreview.net/forum?id=S1xq3oR5tQ 

  Access Paper or Ask Questions

Task-Driven Convolutional Recurrent Models of the Visual System


Oct 27, 2018
Aran Nayebi, Daniel Bear, Jonas Kubilius, Kohitij Kar, Surya Ganguli, David Sussillo, James J. DiCarlo, Daniel L. K. Yamins

* NIPS 2018 Camera Ready Version, 16 pages including supplementary information, 6 figures 

  Access Paper or Ask Questions

A mathematical theory of semantic development in deep neural networks


Oct 23, 2018
Andrew M. Saxe, James L. McClelland, Surya Ganguli


  Access Paper or Ask Questions

Statistical mechanics of low-rank tensor decomposition


Oct 23, 2018
Jonathan Kadmon, Surya Ganguli

* 27 pages, 3 figures 

  Access Paper or Ask Questions

An analytic theory of generalization dynamics and transfer learning in deep linear networks


Sep 27, 2018
Andrew K. Lampinen, Surya Ganguli

* Under review at ICLR 2019, 20 pages 

  Access Paper or Ask Questions

The Emergence of Spectral Universality in Deep Networks


Feb 27, 2018
Jeffrey Pennington, Samuel S. Schoenholz, Surya Ganguli

* 17 pages, 4 figures. Appearing at the 21st International Conference on Artificial Intelligence and Statistics (AISTATS) 2018 

  Access Paper or Ask Questions

Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice


Nov 13, 2017
Jeffrey Pennington, Samuel S. Schoenholz, Surya Ganguli

* 13 pages, 6 figures. Appearing at the 31st Conference on Neural Information Processing Systems (NIPS 2017) 

  Access Paper or Ask Questions

Variational Walkback: Learning a Transition Operator as a Stochastic Recurrent Net


Nov 07, 2017
Anirudh Goyal, Nan Rosemary Ke, Surya Ganguli, Yoshua Bengio

* To appear at NIPS 2017 

  Access Paper or Ask Questions

SuperSpike: Supervised learning in multi-layer spiking neural networks


Oct 14, 2017
Friedemann Zenke, Surya Ganguli


  Access Paper or Ask Questions

On the Expressive Power of Deep Neural Networks


Jun 18, 2017
Maithra Raghu, Ben Poole, Jon Kleinberg, Surya Ganguli, Jascha Sohl-Dickstein

* Accepted to ICML 2017 

  Access Paper or Ask Questions