Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Donald Metzler

ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning


Nov 22, 2021
Vamsi Aribandi, Yi Tay, Tal Schuster, Jinfeng Rao, Huaixiu Steven Zheng, Sanket Vaibhav Mehta, Honglei Zhuang, Vinh Q. Tran, Dara Bahri, Jianmo Ni, Jai Gupta, Kai Hui, Sebastian Ruder, Donald Metzler


  Access Paper or Ask Questions

Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers


Sep 22, 2021
Yi Tay, Mostafa Dehghani, Jinfeng Rao, William Fedus, Samira Abnar, Hyung Won Chung, Sharan Narang, Dani Yogatama, Ashish Vaswani, Donald Metzler


  Access Paper or Ask Questions

The Benchmark Lottery


Jul 14, 2021
Mostafa Dehghani, Yi Tay, Alexey A. Gritsenko, Zhe Zhao, Neil Houlsby, Fernando Diaz, Donald Metzler, Oriol Vinyals


  Access Paper or Ask Questions

Charformer: Fast Character Transformers via Gradient-based Subword Tokenization


Jul 02, 2021
Yi Tay, Vinh Q. Tran, Sebastian Ruder, Jai Gupta, Hyung Won Chung, Dara Bahri, Zhen Qin, Simon Baumgartner, Cong Yu, Donald Metzler


  Access Paper or Ask Questions

SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption


Jun 29, 2021
Dara Bahri, Heinrich Jiang, Yi Tay, Donald Metzler


  Access Paper or Ask Questions

How Reliable are Model Diagnostics?


May 12, 2021
Vamsi Aribandi, Yi Tay, Donald Metzler

* ACL 2021 Findings 

  Access Paper or Ask Questions

Are Pre-trained Convolutions Better than Pre-trained Transformers?


May 07, 2021
Yi Tay, Mostafa Dehghani, Jai Gupta, Dara Bahri, Vamsi Aribandi, Zhen Qin, Donald Metzler

* Accepted to ACL 2021 

  Access Paper or Ask Questions

Rethinking Search: Making Experts out of Dilettantes


May 05, 2021
Donald Metzler, Yi Tay, Dara Bahri, Marc Najork


  Access Paper or Ask Questions

OmniNet: Omnidirectional Representations from Transformers


Mar 01, 2021
Yi Tay, Mostafa Dehghani, Vamsi Aribandi, Jai Gupta, Philip Pham, Zhen Qin, Dara Bahri, Da-Cheng Juan, Donald Metzler


  Access Paper or Ask Questions

Label Smoothed Embedding Hypothesis for Out-of-Distribution Detection


Feb 09, 2021
Dara Bahri, Heinrich Jiang, Yi Tay, Donald Metzler


  Access Paper or Ask Questions

StructFormer: Joint Unsupervised Induction of Dependency and Constituency Structure from Masked Language Modeling


Dec 15, 2020
Yikang Shen, Yi Tay, Che Zheng, Dara Bahri, Donald Metzler, Aaron Courville


  Access Paper or Ask Questions

Long Range Arena: A Benchmark for Efficient Transformers


Nov 08, 2020
Yi Tay, Mostafa Dehghani, Samira Abnar, Yikang Shen, Dara Bahri, Philip Pham, Jinfeng Rao, Liu Yang, Sebastian Ruder, Donald Metzler


  Access Paper or Ask Questions

Surprise: Result List Truncation via Extreme Value Theory


Oct 19, 2020
Dara Bahri, Che Zheng, Yi Tay, Donald Metzler, Andrew Tomkins


  Access Paper or Ask Questions

Efficient Transformers: A Survey


Sep 16, 2020
Yi Tay, Mostafa Dehghani, Dara Bahri, Donald Metzler


  Access Paper or Ask Questions

Generative Models are Unsupervised Predictors of Page Quality: A Colossal-Scale Study


Aug 17, 2020
Dara Bahri, Yi Tay, Che Zheng, Donald Metzler, Cliff Brunk, Andrew Tomkins


  Access Paper or Ask Questions

HyperGrid: Efficient Multi-Task Transformers with Grid-wise Decomposable Hyper Projections


Jul 12, 2020
Yi Tay, Zhe Zhao, Dara Bahri, Donald Metzler, Da-Cheng Juan


  Access Paper or Ask Questions

Synthesizer: Rethinking Self-Attention in Transformer Models


May 02, 2020
Yi Tay, Dara Bahri, Donald Metzler, Da-Cheng Juan, Zhe Zhao, Che Zheng


  Access Paper or Ask Questions

Choppy: Cut Transformer For Ranked List Truncation


Apr 26, 2020
Dara Bahri, Yi Tay, Che Zheng, Donald Metzler, Andrew Tomkins

* SIGIR 2020 

  Access Paper or Ask Questions

Reverse Engineering Configurations of Neural Text Generation Models


Apr 13, 2020
Yi Tay, Dara Bahri, Che Zheng, Clifford Brunk, Donald Metzler, Andrew Tomkins

* ACL 2020 

  Access Paper or Ask Questions

Sparse Sinkhorn Attention


Feb 26, 2020
Yi Tay, Dara Bahri, Liu Yang, Donald Metzler, Da-Cheng Juan


  Access Paper or Ask Questions

Separate and Attend in Personal Email Search


Nov 21, 2019
Yu Meng, Maryam Karimzadehgan, Honglei Zhuang, Donald Metzler

* WSDM 2020 

  Access Paper or Ask Questions

Domain Adaptation for Enterprise Email Search


Jun 19, 2019
Brandon Tran, Maryam Karimzadehgan, Rama Kumar Pasumarthi, Michael Bendersky, Donald Metzler

* Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, 2019 
* Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval 

  Access Paper or Ask Questions