Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Yi Tay

The Benchmark Lottery


Jul 14, 2021
Mostafa Dehghani, Yi Tay, Alexey A. Gritsenko, Zhe Zhao, Neil Houlsby, Fernando Diaz, Donald Metzler, Oriol Vinyals


  Access Paper or Ask Questions

Charformer: Fast Character Transformers via Gradient-based Subword Tokenization


Jul 02, 2021
Yi Tay, Vinh Q. Tran, Sebastian Ruder, Jai Gupta, Hyung Won Chung, Dara Bahri, Zhen Qin, Simon Baumgartner, Cong Yu, Donald Metzler


  Access Paper or Ask Questions

SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption


Jun 29, 2021
Dara Bahri, Heinrich Jiang, Yi Tay, Donald Metzler


  Access Paper or Ask Questions

How Reliable are Model Diagnostics?


May 12, 2021
Vamsi Aribandi, Yi Tay, Donald Metzler

* ACL 2021 Findings 

  Access Paper or Ask Questions

Are Pre-trained Convolutions Better than Pre-trained Transformers?


May 07, 2021
Yi Tay, Mostafa Dehghani, Jai Gupta, Dara Bahri, Vamsi Aribandi, Zhen Qin, Donald Metzler

* Accepted to ACL 2021 

  Access Paper or Ask Questions

Rethinking Search: Making Experts out of Dilettantes


May 05, 2021
Donald Metzler, Yi Tay, Dara Bahri, Marc Najork


  Access Paper or Ask Questions

OmniNet: Omnidirectional Representations from Transformers


Mar 01, 2021
Yi Tay, Mostafa Dehghani, Vamsi Aribandi, Jai Gupta, Philip Pham, Zhen Qin, Dara Bahri, Da-Cheng Juan, Donald Metzler


  Access Paper or Ask Questions

Do Transformer Modifications Transfer Across Implementations and Applications?


Feb 23, 2021
Sharan Narang, Hyung Won Chung, Yi Tay, William Fedus, Thibault Fevry, Michael Matena, Karishma Malkan, Noah Fiedel, Noam Shazeer, Zhenzhong Lan, Yanqi Zhou, Wei Li, Nan Ding, Jake Marcus, Adam Roberts, Colin Raffel


  Access Paper or Ask Questions

Switch Spaces: Learning Product Spaces with Sparse Gating


Feb 17, 2021
Shuai Zhang, Yi Tay, Wenqi Jiang, Da-cheng Juan, Ce Zhang


  Access Paper or Ask Questions

Beyond Fully-Connected Layers with Quaternions: Parameterization of Hypercomplex Multiplications with $1/n$ Parameters


Feb 17, 2021
Aston Zhang, Yi Tay, Shuai Zhang, Alvin Chan, Anh Tuan Luu, Siu Cheung Hui, Jie Fu

* Published as a conference paper at the 9th International Conference on Learning Representations (ICLR 2021) 

  Access Paper or Ask Questions

Label Smoothed Embedding Hypothesis for Out-of-Distribution Detection


Feb 09, 2021
Dara Bahri, Heinrich Jiang, Yi Tay, Donald Metzler


  Access Paper or Ask Questions

StructFormer: Joint Unsupervised Induction of Dependency and Constituency Structure from Masked Language Modeling


Dec 15, 2020
Yikang Shen, Yi Tay, Che Zheng, Dara Bahri, Donald Metzler, Aaron Courville


  Access Paper or Ask Questions

Long Range Arena: A Benchmark for Efficient Transformers


Nov 08, 2020
Yi Tay, Mostafa Dehghani, Samira Abnar, Yikang Shen, Dara Bahri, Philip Pham, Jinfeng Rao, Liu Yang, Sebastian Ruder, Donald Metzler


  Access Paper or Ask Questions

Surprise: Result List Truncation via Extreme Value Theory


Oct 19, 2020
Dara Bahri, Che Zheng, Yi Tay, Donald Metzler, Andrew Tomkins


  Access Paper or Ask Questions

Poison Attacks against Text Datasets with Conditional Adversarially Regularized Autoencoder


Oct 06, 2020
Alvin Chan, Yi Tay, Yew-Soon Ong, Aston Zhang

* Accepted in EMNLP-Findings 2020, Camera Ready Version 

  Access Paper or Ask Questions

Efficient Transformers: A Survey


Sep 16, 2020
Yi Tay, Mostafa Dehghani, Dara Bahri, Donald Metzler


  Access Paper or Ask Questions

Generative Models are Unsupervised Predictors of Page Quality: A Colossal-Scale Study


Aug 17, 2020
Dara Bahri, Yi Tay, Che Zheng, Donald Metzler, Cliff Brunk, Andrew Tomkins


  Access Paper or Ask Questions

HyperGrid: Efficient Multi-Task Transformers with Grid-wise Decomposable Hyper Projections


Jul 12, 2020
Yi Tay, Zhe Zhao, Dara Bahri, Donald Metzler, Da-Cheng Juan


  Access Paper or Ask Questions

Synthesizer: Rethinking Self-Attention in Transformer Models


May 02, 2020
Yi Tay, Dara Bahri, Donald Metzler, Da-Cheng Juan, Zhe Zhao, Che Zheng


  Access Paper or Ask Questions

Choppy: Cut Transformer For Ranked List Truncation


Apr 26, 2020
Dara Bahri, Yi Tay, Che Zheng, Donald Metzler, Andrew Tomkins

* SIGIR 2020 

  Access Paper or Ask Questions

Reverse Engineering Configurations of Neural Text Generation Models


Apr 13, 2020
Yi Tay, Dara Bahri, Che Zheng, Clifford Brunk, Donald Metzler, Andrew Tomkins

* ACL 2020 

  Access Paper or Ask Questions

Sparse Sinkhorn Attention


Feb 26, 2020
Yi Tay, Dara Bahri, Liu Yang, Donald Metzler, Da-Cheng Juan


  Access Paper or Ask Questions

Jacobian Adversarially Regularized Networks for Robustness


Jan 29, 2020
Alvin Chan, Yi Tay, Yew Soon Ong, Jie Fu

* ICLR 2020 Camera Ready 

  Access Paper or Ask Questions

Multi-level Head-wise Match and Aggregation in Transformer for Textual Sequence Matching


Jan 20, 2020
Shuohang Wang, Yunshi Lan, Yi Tay, Jing Jiang, Jingjing Liu

* AAAI 2020, 8 pages 

  Access Paper or Ask Questions

What it Thinks is Important is Important: Robustness Transfers through Input Gradients


Dec 11, 2019
Alvin Chan, Yi Tay, Yew-Soon Ong


  Access Paper or Ask Questions

Interactive Machine Comprehension with Information Seeking Agents


Sep 04, 2019
Xingdi Yuan, Jie Fu, Marc-Alexandre Cote, Yi Tay, Christopher Pal, Adam Trischler


  Access Paper or Ask Questions

Lightweight and Efficient Neural Natural Language Processing with Quaternion Networks


Jun 11, 2019
Yi Tay, Aston Zhang, Luu Anh Tuan, Jinfeng Rao, Shuai Zhang, Shuohang Wang, Jie Fu, Siu Cheung Hui

* ACL 2019 

  Access Paper or Ask Questions