On the Heavy-Tailed Theory of Stochastic Gradient Descent for Deep Neural Networks

Nov 29, 2019
Umut Şimşekli, Mert Gürbüzbalaban, Thanh Huy Nguyen, Gaël Richard, Levent Sagun

* 32 pages. arXiv admin note: substantial text overlap with arXiv:1901.06053 

  Access Model/Code and Paper
Finding the Needle in the Haystack with Convolutions: on the benefits of architectural bias

Jun 16, 2019
Stéphane d'Ascoli, Levent Sagun, Joan Bruna, Giulio Biroli


  Access Model/Code and Paper
Scaling description of generalization with number of parameters in deep learning

Jan 18, 2019
Mario Geiger, Arthur Jacot, Stefano Spigler, Franck Gabriel, Levent Sagun, Stéphane d'Ascoli, Giulio Biroli, Clément Hongler, Matthieu Wyart


  Access Model/Code and Paper
A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networks

Jan 18, 2019
Umut Simsekli, Levent Sagun, Mert Gurbuzbalaban


  Access Model/Code and Paper
A jamming transition from under- to over-parametrization affects loss landscape and generalization

Oct 22, 2018
Stefano Spigler, Mario Geiger, Stéphane d'Ascoli, Levent Sagun, Giulio Biroli, Matthieu Wyart

* 11 pages, 6 figures, submitted to NIPS workshop "Integration of Deep Learning Theories". arXiv admin note: substantial text overlap with arXiv:1809.09349 

  Access Model/Code and Paper
The jamming transition as a paradigm to understand the loss landscape of deep neural networks

Oct 03, 2018
Mario Geiger, Stefano Spigler, Stéphane d'Ascoli, Levent Sagun, Marco Baity-Jesi, Giulio Biroli, Matthieu Wyart


  Access Model/Code and Paper
Empirical Analysis of the Hessian of Over-Parametrized Neural Networks

May 07, 2018
Levent Sagun, Utku Evci, V. Ugur Guney, Yann Dauphin, Leon Bottou

* Minor update for ICLR 2018 Workshop Track presentation 

  Access Model/Code and Paper
Eigenvalues of the Hessian in Deep Learning: Singularity and Beyond

Oct 05, 2017
Levent Sagun, Leon Bottou, Yann LeCun

* ICLR submission, 2016 - updated to match the openreview.net version 

  Access Model/Code and Paper
SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine

Jun 11, 2017
Matthew Dunn, Levent Sagun, Mike Higgins, V. Ugur Guney, Volkan Cirik, Kyunghyun Cho


  Access Model/Code and Paper
Entropy-SGD: Biasing Gradient Descent Into Wide Valleys

Apr 21, 2017
Pratik Chaudhari, Anna Choromanska, Stefano Soatto, Yann LeCun, Carlo Baldassi, Christian Borgs, Jennifer Chayes, Levent Sagun, Riccardo Zecchina

* ICLR '17 

  Access Model/Code and Paper
Perspective: Energy Landscapes for Machine Learning

Mar 23, 2017
Andrew J. Ballard, Ritankar Das, Stefano Martiniani, Dhagash Mehta, Levent Sagun, Jacob D. Stevenson, David J. Wales

* 41 pages, 25 figures. Accepted for publication in Physical Chemistry Chemical Physics, 2017 

  Access Model/Code and Paper
Universal halting times in optimization and machine learning

Feb 21, 2017
Levent Sagun, Thomas Trogdon, Yann LeCun


  Access Model/Code and Paper
Explorations on high dimensional landscapes

Apr 06, 2015
Levent Sagun, V. Ugur Guney, Gerard Ben Arous, Yann LeCun

* 11 pages, 8 figures, workshop contribution at ICLR 2015 

  Access Model/Code and Paper