Get our free extension to see links to code for papers anywhere online!

Chrome logo  Add to Chrome

Firefox logo Add to Firefox

PaLM: Scaling Language Modeling with Pathways



Aakanksha Chowdhery , Sharan Narang , Jacob Devlin , Maarten Bosma , Gaurav Mishra , Adam Roberts , Paul Barham , Hyung Won Chung , Charles Sutton , Sebastian Gehrmann , Parker Schuh , Kensen Shi , Sasha Tsvyashchenko , Joshua Maynez , Abhishek Rao , Parker Barnes , Yi Tay , Noam Shazeer , Vinodkumar Prabhakaran , Emily Reif , Nan Du , Ben Hutchinson , Reiner Pope , James Bradbury , Jacob Austin , Michael Isard , Guy Gur-Ari , Pengcheng Yin , Toju Duke , Anselm Levskaya , Sanjay Ghemawat , Sunipa Dev , Henryk Michalewski , Xavier Garcia , Vedant Misra , Kevin Robinson , Liam Fedus , Denny Zhou , Daphne Ippolito , David Luan , Hyeontaek Lim , Barret Zoph , Alexander Spiridonov , Ryan Sepassi , David Dohan , Shivani Agrawal , Mark Omernick , Andrew M. Dai , Thanumalayan Sankaranarayana Pillai , Marie Pellat , Aitor Lewkowycz , Erica Moreira , Rewon Child , Oleksandr Polozov , Katherine Lee , Zongwei Zhou , Xuezhi Wang , Brennan Saeta , Mark Diaz , Orhan Firat , Michele Catasta , Jason Wei , Kathy Meier-Hellstern , Douglas Eck , Jeff Dean , Slav Petrov , Noah Fiedel


   Access Paper or Ask Questions

Scaling Up Models and Data with $\texttt{t5x}$ and $\texttt{seqio}$



Adam Roberts , Hyung Won Chung , Anselm Levskaya , Gaurav Mishra , James Bradbury , Daniel Andor , Sharan Narang , Brian Lester , Colin Gaffney , Afroz Mohiuddin , Curtis Hawthorne , Aitor Lewkowycz , Alex Salcianu , Marc van Zee , Jacob Austin , Sebastian Goodman , Livio Baldini Soares , Haitang Hu , Sasha Tsvyashchenko , Aakanksha Chowdhery , Jasmijn Bastings , Jannis Bulian , Xavier Garcia , Jianmo Ni , Andrew Chen , Kathleen Kenealy , Jonathan H. Clark , Stephan Lee , Dan Garrette , James Lee-Thorp , Colin Raffel , Noam Shazeer , Marvin Ritter , Maarten Bosma , Alexandre Passos , Jeremy Maitin-Shepard , Noah Fiedel , Mark Omernick , Brennan Saeta , Ryan Sepassi , Alexander Spiridonov , Joshua Newlan , Andrea Gesmundo


   Access Paper or Ask Questions

Scaling Language Models: Methods, Analysis & Insights from Training Gopher



Jack W. Rae , Sebastian Borgeaud , Trevor Cai , Katie Millican , Jordan Hoffmann , Francis Song , John Aslanides , Sarah Henderson , Roman Ring , Susannah Young , Eliza Rutherford , Tom Hennigan , Jacob Menick , Albin Cassirer , Richard Powell , George van den Driessche , Lisa Anne Hendricks , Maribeth Rauh , Po-Sen Huang , Amelia Glaese , Johannes Welbl , Sumanth Dathathri , Saffron Huang , Jonathan Uesato , John Mellor , Irina Higgins , Antonia Creswell , Nat McAleese , Amy Wu , Erich Elsen , Siddhant Jayakumar , Elena Buchatskaya , David Budden , Esme Sutherland , Karen Simonyan , Michela Paganini , Laurent Sifre , Lena Martens , Xiang Lorraine Li , Adhiguna Kuncoro , Aida Nematzadeh , Elena Gribovskaya , Domenic Donato , Angeliki Lazaridou , Arthur Mensch , Jean-Baptiste Lespiau , Maria Tsimpoukelli , Nikolai Grigorev , Doug Fritz , Thibault Sottiaux , Mantas Pajarskas , Toby Pohlen , Zhitao Gong , Daniel Toyama , Cyprien de Masson d'Autume , Yujia Li , Tayfun Terzi , Vladimir Mikulik , Igor Babuschkin , Aidan Clark , Diego de Las Casas , Aurelia Guy , Chris Jones , James Bradbury , Matthew Johnson , Blake Hechtman , Laura Weidinger , Iason Gabriel , William Isaac , Ed Lockhart , Simon Osindero , Laura Rimell , Chris Dyer , Oriol Vinyals , Kareem Ayoub , Jeff Stanway , Lorrayne Bennett , Demis Hassabis , Koray Kavukcuoglu , Geoffrey Irving

* 118 pages 

   Access Paper or Ask Questions

Exploring the limits of Concurrency in ML Training on Google TPUs



Sameer Kumar , James Bradbury , Cliff Young , Yu Emma Wang , Anselm Levskaya , Blake Hechtman , Dehao Chen , HyoukJoong Lee , Mehmet Deveci , Naveen Kumar , Pankaj Kanwar , Shibo Wang , Skye Wanderman-Milne , Steve Lacy , Tao Wang , Tayo Oguntebi , Yazhou Zu , Yuanzhong Xu , Andy Swing


   Access Paper or Ask Questions

A High-Quality Multilingual Dataset for Structured Documentation Translation



Kazuma Hashimoto , Raffaella Buschiazzo , James Bradbury , Teresa Marshall , Richard Socher , Caiming Xiong

* Published at WMT2019; the draft has been updated with our dataset's URL: https://github.com/salesforce/localization-xml-mt 

   Access Paper or Ask Questions

PyTorch: An Imperative Style, High-Performance Deep Learning Library



Adam Paszke , Sam Gross , Francisco Massa , Adam Lerer , James Bradbury , Gregory Chanan , Trevor Killeen , Zeming Lin , Natalia Gimelshein , Luca Antiga , Alban Desmaison , Andreas Köpf , Edward Yang , Zach DeVito , Martin Raison , Alykhan Tejani , Sasank Chilamkurthy , Benoit Steiner , Lu Fang , Junjie Bai , Soumith Chintala

* 12 pages, 3 figures, NeurIPS 2019 

   Access Paper or Ask Questions

OpenSpiel: A Framework for Reinforcement Learning in Games



Marc Lanctot , Edward Lockhart , Jean-Baptiste Lespiau , Vinicius Zambaldi , Satyaki Upadhyay , Julien Pérolat , Sriram Srinivasan , Finbarr Timbers , Karl Tuyls , Shayegan Omidshafiei , Daniel Hennes , Dustin Morrill , Paul Muller , Timo Ewalds , Ryan Faulkner , János Kramár , Bart De Vylder , Brennan Saeta , James Bradbury , David Ding , Sebastian Borgeaud , Matthew Lai , Julian Schrittwieser , Thomas Anthony , Edward Hughes , Ivo Danihelka , Jonah Ryan-Davis


   Access Paper or Ask Questions

1
2
>>