Alert button
Picture for Carlos Riquelme

Carlos Riquelme

Alert button

Stable LM 2 1.6B Technical Report

Feb 27, 2024
Marco Bellagente, Jonathan Tow, Dakota Mahan, Duy Phung, Maksym Zhuravinskyi, Reshinth Adithyan, James Baicoianu, Ben Brooks, Nathan Cooper, Ashish Datta, Meng Lee, Emad Mostaque, Michael Pieler, Nikhil Pinnaparju, Paulo Rocha, Harry Saini, Hannah Teufel, Niccolo Zanichelli, Carlos Riquelme

Viaarxiv icon

Routers in Vision Mixture of Experts: An Empirical Study

Jan 29, 2024
Tianlin Liu, Mathieu Blondel, Carlos Riquelme, Joan Puigcerver

Viaarxiv icon

Scaling Laws for Sparsely-Connected Foundation Models

Sep 15, 2023
Elias Frantar, Carlos Riquelme, Neil Houlsby, Dan Alistarh, Utku Evci

Figure 1 for Scaling Laws for Sparsely-Connected Foundation Models
Figure 2 for Scaling Laws for Sparsely-Connected Foundation Models
Figure 3 for Scaling Laws for Sparsely-Connected Foundation Models
Figure 4 for Scaling Laws for Sparsely-Connected Foundation Models
Viaarxiv icon

From Sparse to Soft Mixtures of Experts

Aug 02, 2023
Joan Puigcerver, Carlos Riquelme, Basil Mustafa, Neil Houlsby

Figure 1 for From Sparse to Soft Mixtures of Experts
Figure 2 for From Sparse to Soft Mixtures of Experts
Figure 3 for From Sparse to Soft Mixtures of Experts
Figure 4 for From Sparse to Soft Mixtures of Experts
Viaarxiv icon

Scaling Vision Transformers to 22 Billion Parameters

Feb 10, 2023
Mostafa Dehghani, Josip Djolonga, Basil Mustafa, Piotr Padlewski, Jonathan Heek, Justin Gilmer, Andreas Steiner, Mathilde Caron, Robert Geirhos, Ibrahim Alabdulmohsin, Rodolphe Jenatton, Lucas Beyer, Michael Tschannen, Anurag Arnab, Xiao Wang, Carlos Riquelme, Matthias Minderer, Joan Puigcerver, Utku Evci, Manoj Kumar, Sjoerd van Steenkiste, Gamaleldin F. Elsayed, Aravindh Mahendran, Fisher Yu, Avital Oliver, Fantine Huot, Jasmijn Bastings, Mark Patrick Collier, Alexey Gritsenko, Vighnesh Birodkar, Cristina Vasconcelos, Yi Tay, Thomas Mensink, Alexander Kolesnikov, Filip Pavetić, Dustin Tran, Thomas Kipf, Mario Lučić, Xiaohua Zhai, Daniel Keysers, Jeremiah Harmsen, Neil Houlsby

Figure 1 for Scaling Vision Transformers to 22 Billion Parameters
Figure 2 for Scaling Vision Transformers to 22 Billion Parameters
Figure 3 for Scaling Vision Transformers to 22 Billion Parameters
Figure 4 for Scaling Vision Transformers to 22 Billion Parameters
Viaarxiv icon

On the Adversarial Robustness of Mixture of Experts

Oct 19, 2022
Joan Puigcerver, Rodolphe Jenatton, Carlos Riquelme, Pranjal Awasthi, Srinadh Bhojanapalli

Figure 1 for On the Adversarial Robustness of Mixture of Experts
Figure 2 for On the Adversarial Robustness of Mixture of Experts
Figure 3 for On the Adversarial Robustness of Mixture of Experts
Figure 4 for On the Adversarial Robustness of Mixture of Experts
Viaarxiv icon

PaLI: A Jointly-Scaled Multilingual Language-Image Model

Sep 16, 2022
Xi Chen, Xiao Wang, Soravit Changpinyo, AJ Piergiovanni, Piotr Padlewski, Daniel Salz, Sebastian Goodman, Adam Grycner, Basil Mustafa, Lucas Beyer, Alexander Kolesnikov, Joan Puigcerver, Nan Ding, Keran Rong, Hassan Akbari, Gaurav Mishra, Linting Xue, Ashish Thapliyal, James Bradbury, Weicheng Kuo, Mojtaba Seyedhosseini, Chao Jia, Burcu Karagol Ayan, Carlos Riquelme, Andreas Steiner, Anelia Angelova, Xiaohua Zhai, Neil Houlsby, Radu Soricut

Figure 1 for PaLI: A Jointly-Scaled Multilingual Language-Image Model
Figure 2 for PaLI: A Jointly-Scaled Multilingual Language-Image Model
Figure 3 for PaLI: A Jointly-Scaled Multilingual Language-Image Model
Figure 4 for PaLI: A Jointly-Scaled Multilingual Language-Image Model
Viaarxiv icon

Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts

Jun 06, 2022
Basil Mustafa, Carlos Riquelme, Joan Puigcerver, Rodolphe Jenatton, Neil Houlsby

Figure 1 for Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts
Figure 2 for Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts
Figure 3 for Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts
Figure 4 for Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts
Viaarxiv icon

Learning to Merge Tokens in Vision Transformers

Feb 24, 2022
Cedric Renggli, André Susano Pinto, Neil Houlsby, Basil Mustafa, Joan Puigcerver, Carlos Riquelme

Figure 1 for Learning to Merge Tokens in Vision Transformers
Figure 2 for Learning to Merge Tokens in Vision Transformers
Figure 3 for Learning to Merge Tokens in Vision Transformers
Figure 4 for Learning to Merge Tokens in Vision Transformers
Viaarxiv icon

Scaling Vision with Sparse Mixture of Experts

Jun 10, 2021
Carlos Riquelme, Joan Puigcerver, Basil Mustafa, Maxim Neumann, Rodolphe Jenatton, André Susano Pinto, Daniel Keysers, Neil Houlsby

Figure 1 for Scaling Vision with Sparse Mixture of Experts
Figure 2 for Scaling Vision with Sparse Mixture of Experts
Figure 3 for Scaling Vision with Sparse Mixture of Experts
Figure 4 for Scaling Vision with Sparse Mixture of Experts
Viaarxiv icon