Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Giulio Biroli

Transformed CNNs: recasting pre-trained convolutional layers with self-attention


Jun 10, 2021
Stéphane d'Ascoli, Levent Sagun, Giulio Biroli, Ari Morcos


  Access Paper or Ask Questions

Sifting out the features by pruning: Are convolutional networks the winning lottery ticket of fully connected ones?


May 14, 2021
Franco Pellegrini, Giulio Biroli

* 25 pages, 18 figures; typos corrected, references added 

  Access Paper or Ask Questions

ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases


Mar 19, 2021
Stéphane d'Ascoli, Hugo Touvron, Matthew Leavitt, Ari Morcos, Giulio Biroli, Levent Sagun


  Access Paper or Ask Questions

More data or more parameters? Investigating the effect of data structure on generalization


Mar 09, 2021
Stéphane d'Ascoli, Marylou Gabrié, Levent Sagun, Giulio Biroli


  Access Paper or Ask Questions

An analytic theory of shallow networks dynamics for hinge loss classification


Jun 19, 2020
Franco Pellegrini, Giulio Biroli

* 16 pages, 6 figures 

  Access Paper or Ask Questions

Complex Dynamics in Simple Neural Networks: Understanding Gradient Flow in Phase Retrieval


Jun 12, 2020
Stefano Sarao Mannelli, Giulio Biroli, Chiara Cammarota, Florent Krzakala, Pierfrancesco Urbani, Lenka Zdeborová

* 9 pages, 5 figures + appendix 

  Access Paper or Ask Questions

Triple descent and the two kinds of overfitting: Where & why do they appear?


Jun 05, 2020
Stéphane d'Ascoli, Levent Sagun, Giulio Biroli


  Access Paper or Ask Questions

Double Trouble in Double Descent : Bias and Variance(s) in the Lazy Regime


Apr 03, 2020
Stéphane d'Ascoli, Maria Refinetti, Giulio Biroli, Florent Krzakala

* 29 pages, 12 figures 

  Access Paper or Ask Questions

Landscape Complexity for the Empirical Risk of Generalized Linear Models


Dec 04, 2019
Antoine Maillard, GĂ©rard Ben Arous, Giulio Biroli

* 17 pages and 18 pages appendix 

  Access Paper or Ask Questions

Who is Afraid of Big Bad Minima? Analysis of Gradient-Flow in a Spiked Matrix-Tensor Model


Jul 22, 2019
Stefano Sarao Mannelli, Giulio Biroli, Chiara Cammarota, Florent Krzakala, Lenka Zdeborová

* 9 pages, 4 figures + appendix 

  Access Paper or Ask Questions

Finding the Needle in the Haystack with Convolutions: on the benefits of architectural bias


Jun 16, 2019
Stéphane d'Ascoli, Levent Sagun, Joan Bruna, Giulio Biroli


  Access Paper or Ask Questions

Attractive vs. truncated repulsive supercooled liquids : dynamics is encoded in the pair correlation function


Jun 03, 2019
François P. Landes, Giulio Biroli, Olivier Dauchot, Andrea J. Liu, David R. Reichman

* 5 pages, 5 figures 

  Access Paper or Ask Questions

How to iron out rough landscapes and get optimal performances: Replicated Gradient Descent and its application to tensor PCA


May 29, 2019
Giulio Biroli, Chiara Cammarota, Federico Ricci-Tersenghi

* 12 pages, 6 figures, Supplementary Material included 

  Access Paper or Ask Questions

Scaling description of generalization with number of parameters in deep learning


Jan 18, 2019
Mario Geiger, Arthur Jacot, Stefano Spigler, Franck Gabriel, Levent Sagun, Stéphane d'Ascoli, Giulio Biroli, Clément Hongler, Matthieu Wyart


  Access Paper or Ask Questions

Marvels and Pitfalls of the Langevin Algorithm in Noisy High-dimensional Inference


Dec 21, 2018
Stefano Sarao Mannelli, Giulio Biroli, Chiara Cammarota, Florent Krzakala, Pierfrancesco Urbani, Lenka Zdeborová

* 10 pages and 5 figures + appendix 

  Access Paper or Ask Questions

A jamming transition from under- to over-parametrization affects loss landscape and generalization


Oct 22, 2018
Stefano Spigler, Mario Geiger, Stéphane d'Ascoli, Levent Sagun, Giulio Biroli, Matthieu Wyart

* 11 pages, 6 figures, submitted to NIPS workshop "Integration of Deep Learning Theories". arXiv admin note: substantial text overlap with arXiv:1809.09349 

  Access Paper or Ask Questions

The jamming transition as a paradigm to understand the loss landscape of deep neural networks


Oct 03, 2018
Mario Geiger, Stefano Spigler, Stéphane d'Ascoli, Levent Sagun, Marco Baity-Jesi, Giulio Biroli, Matthieu Wyart


  Access Paper or Ask Questions

Complex energy landscapes in spiked-tensor and simple glassy models: ruggedness, arrangements of local minima and phase transitions


Apr 24, 2018
Valentina Ros, Gerard Ben Arous, Giulio Biroli, Chiara Cammarota

* v2 with references added, typos corrected 

  Access Paper or Ask Questions