Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Ashish Vaswani

Simple and Efficient ways to Improve REALM


Apr 18, 2021
Vidhisha Balachandran, Ashish Vaswani, Yulia Tsvetkov, Niki Parmar


  Access Paper or Ask Questions

Scaling Local Self-Attention for Parameter Efficient Visual Backbones


Mar 30, 2021
Ashish Vaswani, Prajit Ramachandran, Aravind Srinivas, Niki Parmar, Blake Hechtman, Jonathon Shlens

* CVPR 2021 Oral 

  Access Paper or Ask Questions

Bottleneck Transformers for Visual Recognition


Jan 27, 2021
Aravind Srinivas, Tsung-Yi Lin, Niki Parmar, Jonathon Shlens, Pieter Abbeel, Ashish Vaswani

* Technical Report, 20 pages, 13 figures, 19 tables 

  Access Paper or Ask Questions

Efficient Content-Based Sparse Attention with Routing Transformers


Mar 12, 2020
Aurko Roy, Mohammad Saffar, Ashish Vaswani, David Grangier


  Access Paper or Ask Questions

Stand-Alone Self-Attention in Vision Models


Jun 13, 2019
Prajit Ramachandran, Niki Parmar, Ashish Vaswani, Irwan Bello, Anselm Levskaya, Jonathon Shlens


  Access Paper or Ask Questions

Stay on the Path: Instruction Fidelity in Vision-and-Language Navigation


Jun 04, 2019
Vihan Jain, Gabriel Magalhaes, Alexander Ku, Ashish Vaswani, Eugene Ie, Jason Baldridge

* Accepted at ACL 2019 as long paper 

  Access Paper or Ask Questions

Attention Augmented Convolutional Networks


Apr 22, 2019
Irwan Bello, Barret Zoph, Ashish Vaswani, Jonathon Shlens, Quoc V. Le


  Access Paper or Ask Questions

Mesh-TensorFlow: Deep Learning for Supercomputers


Nov 05, 2018
Noam Shazeer, Youlong Cheng, Niki Parmar, Dustin Tran, Ashish Vaswani, Penporn Koanantakool, Peter Hawkins, HyoukJoong Lee, Mingsheng Hong, Cliff Young, Ryan Sepassi, Blake Hechtman


  Access Paper or Ask Questions

Relational inductive biases, deep learning, and graph networks


Oct 17, 2018
Peter W. Battaglia, Jessica B. Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, Caglar Gulcehre, Francis Song, Andrew Ballard, Justin Gilmer, George Dahl, Ashish Vaswani, Kelsey Allen, Charles Nash, Victoria Langston, Chris Dyer, Nicolas Heess, Daan Wierstra, Pushmeet Kohli, Matt Botvinick, Oriol Vinyals, Yujia Li, Razvan Pascanu


  Access Paper or Ask Questions

Music Transformer


Oct 10, 2018
Cheng-Zhi Anna Huang, Ashish Vaswani, Jakob Uszkoreit, Noam Shazeer, Ian Simon, Curtis Hawthorne, Andrew M. Dai, Matthew D. Hoffman, Monica Dinculescu, Douglas Eck

* Rewrote many sections to clarify the work, and extended relative attention to the local case. Previous title is "An Improved Relative Self-Attention Mechanism for Transformer with Application to Music Generation" 

  Access Paper or Ask Questions

Theory and Experiments on Vector Quantized Autoencoders


Jul 20, 2018
Aurko Roy, Ashish Vaswani, Arvind Neelakantan, Niki Parmar


  Access Paper or Ask Questions

Image Transformer


Jun 15, 2018
Niki Parmar, Ashish Vaswani, Jakob Uszkoreit, Łukasz Kaiser, Noam Shazeer, Alexander Ku, Dustin Tran

* Appears in International Conference on Machine Learning, 2018. Code available at https://github.com/tensorflow/tensor2tensor 

  Access Paper or Ask Questions

Fast Decoding in Sequence Models using Discrete Latent Variables


Jun 07, 2018
Łukasz Kaiser, Aurko Roy, Ashish Vaswani, Niki Parmar, Samy Bengio, Jakob Uszkoreit, Noam Shazeer

* ICML 2018 

  Access Paper or Ask Questions

Self-Attention with Relative Position Representations


Apr 12, 2018
Peter Shaw, Jakob Uszkoreit, Ashish Vaswani

* NAACL 2018 

  Access Paper or Ask Questions

Tensor2Tensor for Neural Machine Translation


Mar 16, 2018
Ashish Vaswani, Samy Bengio, Eugene Brevdo, Francois Chollet, Aidan N. Gomez, Stephan Gouws, Llion Jones, Łukasz Kaiser, Nal Kalchbrenner, Niki Parmar, Ryan Sepassi, Noam Shazeer, Jakob Uszkoreit

* arXiv admin note: text overlap with arXiv:1706.03762 

  Access Paper or Ask Questions

Attention Is All You Need


Dec 06, 2017
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin

* 15 pages, 5 figures 

  Access Paper or Ask Questions

One Model To Learn Them All


Jun 16, 2017
Lukasz Kaiser, Aidan N. Gomez, Noam Shazeer, Ashish Vaswani, Niki Parmar, Llion Jones, Jakob Uszkoreit


  Access Paper or Ask Questions

Unsupervised Neural Hidden Markov Models


Sep 28, 2016
Ke Tran, Yonatan Bisk, Ashish Vaswani, Daniel Marcu, Kevin Knight

* accepted at EMNLP 2016, Workshop on Structured Prediction for NLP. Oral presentation 

  Access Paper or Ask Questions