Get our free extension to see links to code for papers anywhere online!

Chrome logo  Add to Chrome

Firefox logo Add to Firefox

Building Machine Translation Systems for the Next Thousand Languages



Ankur Bapna , Isaac Caswell , Julia Kreutzer , Orhan Firat , Daan van Esch , Aditya Siddhant , Mengmeng Niu , Pallavi Baljekar , Xavier Garcia , Wolfgang Macherey , Theresa Breiner , Vera Axelrod , Jason Riesa , Yuan Cao , Mia Xu Chen , Klaus Macherey , Maxim Krikun , Pidong Wang , Alexander Gutkin , Apurva Shah , Yanping Huang , Zhifeng Chen , Yonghui Wu , Macduff Hughes

* V2: updated with some details from 24-language Google Translate launch in May 2022 

   Access Paper or Ask Questions

Mixture-of-Experts with Expert Choice Routing



Yanqi Zhou , Tao Lei , Hanxiao Liu , Nan Du , Yanping Huang , Vincent Zhao , Andrew Dai , Zhifeng Chen , Quoc Le , James Laudon


   Access Paper or Ask Questions

Designing Effective Sparse Expert Models



Barret Zoph , Irwan Bello , Sameer Kumar , Nan Du , Yanping Huang , Jeff Dean , Noam Shazeer , William Fedus

* 25 pages main text, 39 pages overall 

   Access Paper or Ask Questions

LaMDA: Language Models for Dialog Applications



Romal Thoppilan , Daniel De Freitas , Jamie Hall , Noam Shazeer , Apoorv Kulshreshtha , Heng-Tze Cheng , Alicia Jin , Taylor Bos , Leslie Baker , Yu Du , YaGuang Li , Hongrae Lee , Huaixiu Steven Zheng , Amin Ghafouri , Marcelo Menegali , Yanping Huang , Maxim Krikun , Dmitry Lepikhin , James Qin , Dehao Chen , Yuanzhong Xu , Zhifeng Chen , Adam Roberts , Maarten Bosma , Vincent Zhao , Yanqi Zhou , Chung-Ching Chang , Igor Krivokon , Will Rusch , Marc Pickett , Pranesh Srinivasan , Laichee Man , Kathleen Meier-Hellstern , Meredith Ringel Morris , Tulsee Doshi , Renelito Delos Santos , Toju Duke , Johnny Soraker , Ben Zevenbergen , Vinodkumar Prabhakaran , Mark Diaz , Ben Hutchinson , Kristen Olson , Alejandra Molina , Erin Hoffman-John , Josh Lee , Lora Aroyo , Ravi Rajakumar , Alena Butryna , Matthew Lamm , Viktoriya Kuzmina , Joe Fenton , Aaron Cohen , Rachel Bernstein , Ray Kurzweil , Blaise Aguera-Arcas , Claire Cui , Marian Croak , Ed Chi , Quoc Le


   Access Paper or Ask Questions

Alpa: Automating Inter- and Intra-Operator Parallelism for Distributed Deep Learning



Lianmin Zheng , Zhuohan Li , Hao Zhang , Yonghao Zhuang , Zhifeng Chen , Yanping Huang , Yida Wang , Yuanzhong Xu , Danyang Zhuo , Joseph E. Gonzalez , Ion Stoica


   Access Paper or Ask Questions

GLaM: Efficient Scaling of Language Models with Mixture-of-Experts



Nan Du , Yanping Huang , Andrew M. Dai , Simon Tong , Dmitry Lepikhin , Yuanzhong Xu , Maxim Krikun , Yanqi Zhou , Adams Wei Yu , Orhan Firat , Barret Zoph , Liam Fedus , Maarten Bosma , Zongwei Zhou , Tao Wang , Yu Emma Wang , Kellie Webster , Marie Pellat , Kevin Robinson , Kathy Meier-Hellstern , Toju Duke , Lucas Dixon , Kun Zhang , Quoc V Le , Yonghui Wu , Zhifeng Chen , Claire Cui


   Access Paper or Ask Questions

BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition



Yu Zhang , Daniel S. Park , Wei Han , James Qin , Anmol Gulati , Joel Shor , Aren Jansen , Yuanzhong Xu , Yanping Huang , Shibo Wang , Zongwei Zhou , Bo Li , Min Ma , William Chan , Jiahui Yu , Yongqiang Wang , Liangliang Cao , Khe Chai Sim , Bhuvana Ramabhadran , Tara N. Sainath , Françoise Beaufays , Zhifeng Chen , Quoc V. Le , Chung-Cheng Chiu , Ruoming Pang , Yonghui Wu

* 14 pages, 7 figures, 13 tables; v2: minor corrections, reference baselines and bibliography updated 

   Access Paper or Ask Questions

Beyond Distillation: Task-level Mixture-of-Experts for Efficient Inference



Sneha Kudugunta , Yanping Huang , Ankur Bapna , Maxim Krikun , Dmitry Lepikhin , Minh-Thang Luong , Orhan Firat

* EMNLP Findings 2021 

   Access Paper or Ask Questions

GSPMD: General and Scalable Parallelization for ML Computation Graphs



Yuanzhong Xu , HyoukJoong Lee , Dehao Chen , Blake Hechtman , Yanping Huang , Rahul Joshi , Maxim Krikun , Dmitry Lepikhin , Andy Ly , Marcello Maggioni , Ruoming Pang , Noam Shazeer , Shibo Wang , Tao Wang , Yonghui Wu , Zhifeng Chen


   Access Paper or Ask Questions

1
2
>>