Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Levent Sagun

Transformed CNNs: recasting pre-trained convolutional layers with self-attention

Jun 10, 2021
St├ęphane d'Ascoli, Levent Sagun, Giulio Biroli, Ari Morcos

  Access Paper or Ask Questions

ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases

Mar 19, 2021
St├ęphane d'Ascoli, Hugo Touvron, Matthew Leavitt, Ari Morcos, Giulio Biroli, Levent Sagun

  Access Paper or Ask Questions

More data or more parameters? Investigating the effect of data structure on generalization

Mar 09, 2021
St├ęphane d'Ascoli, Marylou Gabri├ę, Levent Sagun, Giulio Biroli

  Access Paper or Ask Questions

Triple descent and the two kinds of overfitting: Where & why do they appear?

Jun 05, 2020
St├ęphane d'Ascoli, Levent Sagun, Giulio Biroli

  Access Paper or Ask Questions

On the Heavy-Tailed Theory of Stochastic Gradient Descent for Deep Neural Networks

Nov 29, 2019
Umut ┼×im┼čekli, Mert G├╝rb├╝zbalaban, Thanh Huy Nguyen, Ga├źl Richard, Levent Sagun

* 32 pages. arXiv admin note: substantial text overlap with arXiv:1901.06053 

  Access Paper or Ask Questions

Finding the Needle in the Haystack with Convolutions: on the benefits of architectural bias

Jun 16, 2019
St├ęphane d'Ascoli, Levent Sagun, Joan Bruna, Giulio Biroli

  Access Paper or Ask Questions

Scaling description of generalization with number of parameters in deep learning

Jan 18, 2019
Mario Geiger, Arthur Jacot, Stefano Spigler, Franck Gabriel, Levent Sagun, St├ęphane d'Ascoli, Giulio Biroli, Cl├ęment Hongler, Matthieu Wyart

  Access Paper or Ask Questions

A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networks

Jan 18, 2019
Umut Simsekli, Levent Sagun, Mert Gurbuzbalaban

  Access Paper or Ask Questions

A jamming transition from under- to over-parametrization affects loss landscape and generalization

Oct 22, 2018
Stefano Spigler, Mario Geiger, St├ęphane d'Ascoli, Levent Sagun, Giulio Biroli, Matthieu Wyart

* 11 pages, 6 figures, submitted to NIPS workshop "Integration of Deep Learning Theories". arXiv admin note: substantial text overlap with arXiv:1809.09349 

  Access Paper or Ask Questions

The jamming transition as a paradigm to understand the loss landscape of deep neural networks

Oct 03, 2018
Mario Geiger, Stefano Spigler, St├ęphane d'Ascoli, Levent Sagun, Marco Baity-Jesi, Giulio Biroli, Matthieu Wyart

  Access Paper or Ask Questions

Empirical Analysis of the Hessian of Over-Parametrized Neural Networks

May 07, 2018
Levent Sagun, Utku Evci, V. Ugur Guney, Yann Dauphin, Leon Bottou

* Minor update for ICLR 2018 Workshop Track presentation 

  Access Paper or Ask Questions

Eigenvalues of the Hessian in Deep Learning: Singularity and Beyond

Oct 05, 2017
Levent Sagun, Leon Bottou, Yann LeCun

* ICLR submission, 2016 - updated to match the version 

  Access Paper or Ask Questions

SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine

Jun 11, 2017
Matthew Dunn, Levent Sagun, Mike Higgins, V. Ugur Guney, Volkan Cirik, Kyunghyun Cho

  Access Paper or Ask Questions

Entropy-SGD: Biasing Gradient Descent Into Wide Valleys

Apr 21, 2017
Pratik Chaudhari, Anna Choromanska, Stefano Soatto, Yann LeCun, Carlo Baldassi, Christian Borgs, Jennifer Chayes, Levent Sagun, Riccardo Zecchina

* ICLR '17 

  Access Paper or Ask Questions

Perspective: Energy Landscapes for Machine Learning

Mar 23, 2017
Andrew J. Ballard, Ritankar Das, Stefano Martiniani, Dhagash Mehta, Levent Sagun, Jacob D. Stevenson, David J. Wales

* 41 pages, 25 figures. Accepted for publication in Physical Chemistry Chemical Physics, 2017 

  Access Paper or Ask Questions

Universal halting times in optimization and machine learning

Feb 21, 2017
Levent Sagun, Thomas Trogdon, Yann LeCun

  Access Paper or Ask Questions

Explorations on high dimensional landscapes

Apr 06, 2015
Levent Sagun, V. Ugur Guney, Gerard Ben Arous, Yann LeCun

* 11 pages, 8 figures, workshop contribution at ICLR 2015 

  Access Paper or Ask Questions