Get our free extension to see links to code for papers anywhere online!

Chrome logo  Add to Chrome

Firefox logo Add to Firefox

Mixture-of-Experts with Expert Choice Routing



Yanqi Zhou , Tao Lei , Hanxiao Liu , Nan Du , Yanping Huang , Vincent Zhao , Andrew Dai , Zhifeng Chen , Quoc Le , James Laudon


   Access Paper or Ask Questions

LaMDA: Language Models for Dialog Applications



Romal Thoppilan , Daniel De Freitas , Jamie Hall , Noam Shazeer , Apoorv Kulshreshtha , Heng-Tze Cheng , Alicia Jin , Taylor Bos , Leslie Baker , Yu Du , YaGuang Li , Hongrae Lee , Huaixiu Steven Zheng , Amin Ghafouri , Marcelo Menegali , Yanping Huang , Maxim Krikun , Dmitry Lepikhin , James Qin , Dehao Chen , Yuanzhong Xu , Zhifeng Chen , Adam Roberts , Maarten Bosma , Vincent Zhao , Yanqi Zhou , Chung-Ching Chang , Igor Krivokon , Will Rusch , Marc Pickett , Pranesh Srinivasan , Laichee Man , Kathleen Meier-Hellstern , Meredith Ringel Morris , Tulsee Doshi , Renelito Delos Santos , Toju Duke , Johnny Soraker , Ben Zevenbergen , Vinodkumar Prabhakaran , Mark Diaz , Ben Hutchinson , Kristen Olson , Alejandra Molina , Erin Hoffman-John , Josh Lee , Lora Aroyo , Ravi Rajakumar , Alena Butryna , Matthew Lamm , Viktoriya Kuzmina , Joe Fenton , Aaron Cohen , Rachel Bernstein , Ray Kurzweil , Blaise Aguera-Arcas , Claire Cui , Marian Croak , Ed Chi , Quoc Le


   Access Paper or Ask Questions

GLaM: Efficient Scaling of Language Models with Mixture-of-Experts



Nan Du , Yanping Huang , Andrew M. Dai , Simon Tong , Dmitry Lepikhin , Yuanzhong Xu , Maxim Krikun , Yanqi Zhou , Adams Wei Yu , Orhan Firat , Barret Zoph , Liam Fedus , Maarten Bosma , Zongwei Zhou , Tao Wang , Yu Emma Wang , Kellie Webster , Marie Pellat , Kevin Robinson , Kathy Meier-Hellstern , Toju Duke , Lucas Dixon , Kun Zhang , Quoc V Le , Yonghui Wu , Zhifeng Chen , Claire Cui


   Access Paper or Ask Questions

A Transferable Approach for Partitioning Machine Learning Models on Multi-Chip-Modules



Xinfeng Xie , Prakash Prabhu , Ulysse Beaugnon , Phitchaya Mangpo Phothilimthana , Sudip Roy , Azalia Mirhoseini , Eugene Brevdo , James Laudon , Yanqi Zhou


   Access Paper or Ask Questions

Do Transformer Modifications Transfer Across Implementations and Applications?



Sharan Narang , Hyung Won Chung , Yi Tay , William Fedus , Thibault Fevry , Michael Matena , Karishma Malkan , Noah Fiedel , Noam Shazeer , Zhenzhong Lan , Yanqi Zhou , Wei Li , Nan Ding , Jake Marcus , Adam Roberts , Colin Raffel


   Access Paper or Ask Questions

Rethinking Co-design of Neural Architectures and Hardware Accelerators



Yanqi Zhou , Xuanyi Dong , Berkin Akin , Mingxing Tan , Daiyi Peng , Tianjian Meng , Amir Yazdanbakhsh , Da Huang , Ravi Narayanaswami , James Laudon


   Access Paper or Ask Questions

Apollo: Transferable Architecture Exploration



Amir Yazdanbakhsh , Christof Angermueller , Berkin Akin , Yanqi Zhou , Albin Jones , Milad Hashemi , Kevin Swersky , Satrajit Chatterjee , Ravi Narayanaswami , James Laudon

* 10 pages, 5 figures, Accepted to Workshop on ML for Systems at the 34th Conference on Neural Information Processing Systems (NeurIPS 2020) 

   Access Paper or Ask Questions

Transferable Graph Optimizers for ML Compilers



Yanqi Zhou , Sudip Roy , Amirali Abdolrashidi , Daniel Wong , Peter Ma , Qiumin Xu , Hanxiao Liu , Mangpo Phitchaya Phothilimtha , Shen Wang , Anna Goldie , Azalia Mirhoseini , James Laudon

* NeurIPS 2020 
* arXiv admin note: text overlap with arXiv:1910.01578 

   Access Paper or Ask Questions

A Learned Performance Model for the Tensor Processing Unit



Samuel J. Kaufman , Phitchaya Mangpo Phothilimthana , Yanqi Zhou , Mike Burrows


   Access Paper or Ask Questions

1
2
>>