Get our free extension to see links to code for papers anywhere online!

Chrome logo  Add to Chrome

Firefox logo Add to Firefox

PaLM: Scaling Language Modeling with Pathways



Aakanksha Chowdhery , Sharan Narang , Jacob Devlin , Maarten Bosma , Gaurav Mishra , Adam Roberts , Paul Barham , Hyung Won Chung , Charles Sutton , Sebastian Gehrmann , Parker Schuh , Kensen Shi , Sasha Tsvyashchenko , Joshua Maynez , Abhishek Rao , Parker Barnes , Yi Tay , Noam Shazeer , Vinodkumar Prabhakaran , Emily Reif , Nan Du , Ben Hutchinson , Reiner Pope , James Bradbury , Jacob Austin , Michael Isard , Guy Gur-Ari , Pengcheng Yin , Toju Duke , Anselm Levskaya , Sanjay Ghemawat , Sunipa Dev , Henryk Michalewski , Xavier Garcia , Vedant Misra , Kevin Robinson , Liam Fedus , Denny Zhou , Daphne Ippolito , David Luan , Hyeontaek Lim , Barret Zoph , Alexander Spiridonov , Ryan Sepassi , David Dohan , Shivani Agrawal , Mark Omernick , Andrew M. Dai , Thanumalayan Sankaranarayana Pillai , Marie Pellat , Aitor Lewkowycz , Erica Moreira , Rewon Child , Oleksandr Polozov , Katherine Lee , Zongwei Zhou , Xuezhi Wang , Brennan Saeta , Mark Diaz , Orhan Firat , Michele Catasta , Jason Wei , Kathy Meier-Hellstern , Douglas Eck , Jeff Dean , Slav Petrov , Noah Fiedel


   Access Paper or Ask Questions

Mixture-of-Experts with Expert Choice Routing



Yanqi Zhou , Tao Lei , Hanxiao Liu , Nan Du , Yanping Huang , Vincent Zhao , Andrew Dai , Zhifeng Chen , Quoc Le , James Laudon


   Access Paper or Ask Questions

Designing Effective Sparse Expert Models



Barret Zoph , Irwan Bello , Sameer Kumar , Nan Du , Yanping Huang , Jeff Dean , Noam Shazeer , William Fedus

* 25 pages main text, 39 pages overall 

   Access Paper or Ask Questions

GLaM: Efficient Scaling of Language Models with Mixture-of-Experts



Nan Du , Yanping Huang , Andrew M. Dai , Simon Tong , Dmitry Lepikhin , Yuanzhong Xu , Maxim Krikun , Yanqi Zhou , Adams Wei Yu , Orhan Firat , Barret Zoph , Liam Fedus , Maarten Bosma , Zongwei Zhou , Tao Wang , Yu Emma Wang , Kellie Webster , Marie Pellat , Kevin Robinson , Kathy Meier-Hellstern , Toju Duke , Lucas Dixon , Kun Zhang , Quoc V Le , Yonghui Wu , Zhifeng Chen , Claire Cui


   Access Paper or Ask Questions

Finetuned Language Models Are Zero-Shot Learners



Jason Wei , Maarten Bosma , Vincent Y. Zhao , Kelvin Guu , Adams Wei Yu , Brian Lester , Nan Du , Andrew M. Dai , Quoc V. Le


   Access Paper or Ask Questions

R2D2: Relational Text Decoding with Transformers



Aryan Arbabi , Mingqiu Wang , Laurent El Shafey , Nan Du , Izhak Shafran


   Access Paper or Ask Questions

The Medical Scribe: Corpus Development and Model Performance Analyses



Izhak Shafran , Nan Du , Linh Tran , Amanda Perry , Lauren Keyes , Mark Knichel , Ashley Domin , Lei Huang , Yuhui Chen , Gang Li , Mingqiu Wang , Laurent El Shafey , Hagen Soltau , Justin S. Paul

* Proceedings of Language Resources and Evaluation, 2020 
* Extended version of the paper accepted at LREC 2020 

   Access Paper or Ask Questions

Deep Physiological State Space Model for Clinical Forecasting



Yuan Xue , Denny Zhou , Nan Du , Andrew Dai , Zhen Xu , Kun Zhang , Claire Cui


   Access Paper or Ask Questions

Learning to Infer Entities, Properties and their Relations from Clinical Conversations



Nan Du , Mingqiu Wang , Linh Tran , Gang Li , Izhak Shafran

* Proc. Empirical Methods in Natural Language Processing, 2019 

   Access Paper or Ask Questions

1
2
3
>>