Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Lukasz Kaiser

Rethinking Attention with Performers


Sep 30, 2020
Krzysztof Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreea Gane, Tamas Sarlos, Peter Hawkins, Jared Davis, Afroz Mohiuddin, Lukasz Kaiser, David Belanger, Lucy Colwell, Adrian Weller

* 36 pages. This is an updated version of a previous submission which can be found at arXiv:2006.03555. See https://github.com/google-research/google-research/tree/master/protein_lm for protein language model code, and https://github.com/google-research/google-research/tree/master/performer for Performer code 

  Access Paper or Ask Questions

Parallel Scheduled Sampling


Jun 11, 2019
Daniel Duckworth, Arvind Neelakantan, Ben Goodrich, Lukasz Kaiser, Samy Bengio

* Initial submission 

  Access Paper or Ask Questions

Sample Efficient Text Summarization Using a Single Pre-Trained Transformer


May 21, 2019
Urvashi Khandelwal, Kevin Clark, Dan Jurafsky, Lukasz Kaiser


  Access Paper or Ask Questions

Model-Based Reinforcement Learning for Atari


Mar 05, 2019
Lukasz Kaiser, Mohammad Babaeizadeh, Piotr Milos, Blazej Osinski, Roy H Campbell, Konrad Czechowski, Dumitru Erhan, Chelsea Finn, Piotr Kozakowski, Sergey Levine, Ryan Sepassi, George Tucker, Henryk Michalewski


  Access Paper or Ask Questions

Area Attention


Oct 30, 2018
Yang Li, Lukasz Kaiser, Samy Bengio, Si Si


  Access Paper or Ask Questions

Generating Wikipedia by Summarizing Long Sequences


Jan 30, 2018
Peter J. Liu, Mohammad Saleh, Etienne Pot, Ben Goodrich, Ryan Sepassi, Lukasz Kaiser, Noam Shazeer

* Published as a conference paper at ICLR 2018 

  Access Paper or Ask Questions

Unsupervised Cipher Cracking Using Discrete GANs


Jan 15, 2018
Aidan N. Gomez, Sicong Huang, Ivan Zhang, Bryan M. Li, Muhammad Osama, Lukasz Kaiser


  Access Paper or Ask Questions

Attention Is All You Need


Dec 06, 2017
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin

* 15 pages, 5 figures 

  Access Paper or Ask Questions

One Model To Learn Them All


Jun 16, 2017
Lukasz Kaiser, Aidan N. Gomez, Noam Shazeer, Ashish Vaswani, Niki Parmar, Llion Jones, Jakob Uszkoreit


  Access Paper or Ask Questions

Depthwise Separable Convolutions for Neural Machine Translation


Jun 16, 2017
Lukasz Kaiser, Aidan N. Gomez, Francois Chollet


  Access Paper or Ask Questions

TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems


Mar 16, 2016
Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dan Mane, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viegas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, Xiaoqiang Zheng

* Version 2 updates only the metadata, to correct the formatting of Mart\'in Abadi's name 

  Access Paper or Ask Questions

Multi-task Sequence to Sequence Learning


Mar 01, 2016
Minh-Thang Luong, Quoc V. Le, Ilya Sutskever, Oriol Vinyals, Lukasz Kaiser

* 10 pages, 4 figures, ICLR 2016 camera-ready, added parsing SOTA results 

  Access Paper or Ask Questions

Adding Gradient Noise Improves Learning for Very Deep Networks


Nov 21, 2015
Arvind Neelakantan, Luke Vilnis, Quoc V. Le, Ilya Sutskever, Lukasz Kaiser, Karol Kurach, James Martens


  Access Paper or Ask Questions

Grammar as a Foreign Language


Jun 09, 2015
Oriol Vinyals, Lukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, Geoffrey Hinton


  Access Paper or Ask Questions