Alert button
Picture for Ashish Vaswani

Ashish Vaswani

Alert button

The Efficiency Misnomer

Oct 25, 2021
Mostafa Dehghani, Anurag Arnab, Lucas Beyer, Ashish Vaswani, Yi Tay

Figure 1 for The Efficiency Misnomer
Figure 2 for The Efficiency Misnomer
Figure 3 for The Efficiency Misnomer
Viaarxiv icon

Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers

Sep 22, 2021
Yi Tay, Mostafa Dehghani, Jinfeng Rao, William Fedus, Samira Abnar, Hyung Won Chung, Sharan Narang, Dani Yogatama, Ashish Vaswani, Donald Metzler

Figure 1 for Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers
Figure 2 for Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers
Figure 3 for Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers
Figure 4 for Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers
Viaarxiv icon

Simple and Efficient ways to Improve REALM

Apr 18, 2021
Vidhisha Balachandran, Ashish Vaswani, Yulia Tsvetkov, Niki Parmar

Figure 1 for Simple and Efficient ways to Improve REALM
Figure 2 for Simple and Efficient ways to Improve REALM
Figure 3 for Simple and Efficient ways to Improve REALM
Figure 4 for Simple and Efficient ways to Improve REALM
Viaarxiv icon

Scaling Local Self-Attention for Parameter Efficient Visual Backbones

Mar 30, 2021
Ashish Vaswani, Prajit Ramachandran, Aravind Srinivas, Niki Parmar, Blake Hechtman, Jonathon Shlens

Figure 1 for Scaling Local Self-Attention for Parameter Efficient Visual Backbones
Figure 2 for Scaling Local Self-Attention for Parameter Efficient Visual Backbones
Figure 3 for Scaling Local Self-Attention for Parameter Efficient Visual Backbones
Figure 4 for Scaling Local Self-Attention for Parameter Efficient Visual Backbones
Viaarxiv icon

Bottleneck Transformers for Visual Recognition

Jan 27, 2021
Aravind Srinivas, Tsung-Yi Lin, Niki Parmar, Jonathon Shlens, Pieter Abbeel, Ashish Vaswani

Figure 1 for Bottleneck Transformers for Visual Recognition
Figure 2 for Bottleneck Transformers for Visual Recognition
Figure 3 for Bottleneck Transformers for Visual Recognition
Figure 4 for Bottleneck Transformers for Visual Recognition
Viaarxiv icon

Efficient Content-Based Sparse Attention with Routing Transformers

Mar 12, 2020
Aurko Roy, Mohammad Saffar, Ashish Vaswani, David Grangier

Figure 1 for Efficient Content-Based Sparse Attention with Routing Transformers
Figure 2 for Efficient Content-Based Sparse Attention with Routing Transformers
Figure 3 for Efficient Content-Based Sparse Attention with Routing Transformers
Figure 4 for Efficient Content-Based Sparse Attention with Routing Transformers
Viaarxiv icon

Stand-Alone Self-Attention in Vision Models

Jun 13, 2019
Prajit Ramachandran, Niki Parmar, Ashish Vaswani, Irwan Bello, Anselm Levskaya, Jonathon Shlens

Figure 1 for Stand-Alone Self-Attention in Vision Models
Figure 2 for Stand-Alone Self-Attention in Vision Models
Figure 3 for Stand-Alone Self-Attention in Vision Models
Figure 4 for Stand-Alone Self-Attention in Vision Models
Viaarxiv icon

Stay on the Path: Instruction Fidelity in Vision-and-Language Navigation

Jun 04, 2019
Vihan Jain, Gabriel Magalhaes, Alexander Ku, Ashish Vaswani, Eugene Ie, Jason Baldridge

Figure 1 for Stay on the Path: Instruction Fidelity in Vision-and-Language Navigation
Figure 2 for Stay on the Path: Instruction Fidelity in Vision-and-Language Navigation
Figure 3 for Stay on the Path: Instruction Fidelity in Vision-and-Language Navigation
Figure 4 for Stay on the Path: Instruction Fidelity in Vision-and-Language Navigation
Viaarxiv icon

Attention Augmented Convolutional Networks

Apr 22, 2019
Irwan Bello, Barret Zoph, Ashish Vaswani, Jonathon Shlens, Quoc V. Le

Figure 1 for Attention Augmented Convolutional Networks
Figure 2 for Attention Augmented Convolutional Networks
Figure 3 for Attention Augmented Convolutional Networks
Figure 4 for Attention Augmented Convolutional Networks
Viaarxiv icon