Alert button
Picture for Samy Bengio

Samy Bengio

Alert button

What Algorithms can Transformers Learn? A Study in Length Generalization

Oct 24, 2023
Hattie Zhou, Arwen Bradley, Etai Littwin, Noam Razin, Omid Saremi, Josh Susskind, Samy Bengio, Preetum Nakkiran

Viaarxiv icon

When can transformers reason with abstract symbols?

Oct 15, 2023
Enric Boix-Adsera, Omid Saremi, Emmanuel Abbe, Samy Bengio, Etai Littwin, Joshua Susskind

Figure 1 for When can transformers reason with abstract symbols?
Figure 2 for When can transformers reason with abstract symbols?
Figure 3 for When can transformers reason with abstract symbols?
Figure 4 for When can transformers reason with abstract symbols?
Viaarxiv icon

Adaptivity and Modularity for Efficient Generalization Over Task Complexity

Oct 13, 2023
Samira Abnar, Omid Saremi, Laurent Dinh, Shantel Wilson, Miguel Angel Bautista, Chen Huang, Vimal Thilak, Etai Littwin, Jiatao Gu, Josh Susskind, Samy Bengio

Figure 1 for Adaptivity and Modularity for Efficient Generalization Over Task Complexity
Figure 2 for Adaptivity and Modularity for Efficient Generalization Over Task Complexity
Figure 3 for Adaptivity and Modularity for Efficient Generalization Over Task Complexity
Figure 4 for Adaptivity and Modularity for Efficient Generalization Over Task Complexity
Viaarxiv icon

Boolformer: Symbolic Regression of Logic Functions with Transformers

Sep 21, 2023
Stéphane d'Ascoli, Samy Bengio, Josh Susskind, Emmanuel Abbé

Viaarxiv icon

Transformers learn through gradual rank increase

Jun 12, 2023
Enric Boix-Adsera, Etai Littwin, Emmanuel Abbe, Samy Bengio, Joshua Susskind

Figure 1 for Transformers learn through gradual rank increase
Figure 2 for Transformers learn through gradual rank increase
Figure 3 for Transformers learn through gradual rank increase
Figure 4 for Transformers learn through gradual rank increase
Viaarxiv icon

Generalization on the Unseen, Logic Reasoning and Degree Curriculum

Jan 30, 2023
Emmanuel Abbe, Samy Bengio, Aryo Lotfi, Kevin Rizk

Figure 1 for Generalization on the Unseen, Logic Reasoning and Degree Curriculum
Figure 2 for Generalization on the Unseen, Logic Reasoning and Degree Curriculum
Figure 3 for Generalization on the Unseen, Logic Reasoning and Degree Curriculum
Figure 4 for Generalization on the Unseen, Logic Reasoning and Degree Curriculum
Viaarxiv icon

Continuous Soft Pseudo-Labeling in ASR

Nov 11, 2022
Tatiana Likhomanenko, Ronan Collobert, Navdeep Jaitly, Samy Bengio

Figure 1 for Continuous Soft Pseudo-Labeling in ASR
Figure 2 for Continuous Soft Pseudo-Labeling in ASR
Figure 3 for Continuous Soft Pseudo-Labeling in ASR
Figure 4 for Continuous Soft Pseudo-Labeling in ASR
Viaarxiv icon

Continuous Pseudo-Labeling from the Start

Oct 17, 2022
Dan Berrebbi, Ronan Collobert, Samy Bengio, Navdeep Jaitly, Tatiana Likhomanenko

Figure 1 for Continuous Pseudo-Labeling from the Start
Figure 2 for Continuous Pseudo-Labeling from the Start
Figure 3 for Continuous Pseudo-Labeling from the Start
Figure 4 for Continuous Pseudo-Labeling from the Start
Viaarxiv icon

Learning to Reason with Neural Networks: Generalization, Unseen Data and Boolean Measures

May 26, 2022
Emmanuel Abbe, Samy Bengio, Elisabetta Cornacchia, Jon Kleinberg, Aryo Lotfi, Maithra Raghu, Chiyuan Zhang

Figure 1 for Learning to Reason with Neural Networks: Generalization, Unseen Data and Boolean Measures
Figure 2 for Learning to Reason with Neural Networks: Generalization, Unseen Data and Boolean Measures
Figure 3 for Learning to Reason with Neural Networks: Generalization, Unseen Data and Boolean Measures
Figure 4 for Learning to Reason with Neural Networks: Generalization, Unseen Data and Boolean Measures
Viaarxiv icon

Pointer Value Retrieval: A new benchmark for understanding the limits of neural network generalization

Jul 27, 2021
Chiyuan Zhang, Maithra Raghu, Jon Kleinberg, Samy Bengio

Figure 1 for Pointer Value Retrieval: A new benchmark for understanding the limits of neural network generalization
Figure 2 for Pointer Value Retrieval: A new benchmark for understanding the limits of neural network generalization
Figure 3 for Pointer Value Retrieval: A new benchmark for understanding the limits of neural network generalization
Figure 4 for Pointer Value Retrieval: A new benchmark for understanding the limits of neural network generalization
Viaarxiv icon