Alert button
Picture for Giri Anantharaman

Giri Anantharaman

Alert button

Efficient Large Scale Language Modeling with Mixtures of Experts

Dec 20, 2021
Mikel Artetxe, Shruti Bhosale, Naman Goyal, Todor Mihaylov, Myle Ott, Sam Shleifer, Xi Victoria Lin, Jingfei Du, Srinivasan Iyer, Ramakanth Pasunuru, Giri Anantharaman, Xian Li, Shuohui Chen, Halil Akin, Mandeep Baines, Louis Martin, Xing Zhou, Punit Singh Koura, Brian O'Horo, Jeff Wang, Luke Zettlemoyer, Mona Diab, Zornitsa Kozareva, Ves Stoyanov

Figure 1 for Efficient Large Scale Language Modeling with Mixtures of Experts
Figure 2 for Efficient Large Scale Language Modeling with Mixtures of Experts
Figure 3 for Efficient Large Scale Language Modeling with Mixtures of Experts
Figure 4 for Efficient Large Scale Language Modeling with Mixtures of Experts
Viaarxiv icon

Larger-Scale Transformers for Multilingual Masked Language Modeling

May 02, 2021
Naman Goyal, Jingfei Du, Myle Ott, Giri Anantharaman, Alexis Conneau

Figure 1 for Larger-Scale Transformers for Multilingual Masked Language Modeling
Figure 2 for Larger-Scale Transformers for Multilingual Masked Language Modeling
Figure 3 for Larger-Scale Transformers for Multilingual Masked Language Modeling
Figure 4 for Larger-Scale Transformers for Multilingual Masked Language Modeling
Viaarxiv icon