Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Xiaohua Zhai

A Simple Single-Scale Vision Transformer for Object Localization and Instance Segmentation

Dec 17, 2021
Wuyang Chen, Xianzhi Du, Fan Yang, Lucas Beyer, Xiaohua Zhai, Tsung-Yi Lin, Huizhong Chen, Jing Li, Xiaodan Song, Zhangyang Wang, Denny Zhou

  Access Paper or Ask Questions

LiT: Zero-Shot Transfer with Locked-image Text Tuning

Nov 15, 2021
Xiaohua Zhai, Xiao Wang, Basil Mustafa, Andreas Steiner, Daniel Keysers, Alexander Kolesnikov, Lucas Beyer

* Xiaohua, Xiao, Basil, Andreas and Lucas contributed equally 

  Access Paper or Ask Questions

How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers

Jun 18, 2021
Andreas Steiner, Alexander Kolesnikov, Xiaohua Zhai, Ross Wightman, Jakob Uszkoreit, Lucas Beyer

* Andreas, Alex, Xiaohua and Lucas contributed equally. We release more than 50'000 ViT models trained under diverse settings on various datasets. We believe this to be a treasure trove for model analysis. Available at and 

  Access Paper or Ask Questions

Revisiting the Calibration of Modern Neural Networks

Jun 15, 2021
Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, Mario Lucic

  Access Paper or Ask Questions

Knowledge distillation: A good teacher is patient and consistent

Jun 09, 2021
Lucas Beyer, Xiaohua Zhai, Amélie Royer, Larisa Markeeva, Rohan Anil, Alexander Kolesnikov

* Lucas, Xiaohua, Am\'elie, Larisa, and Alex contributed equally 

  Access Paper or Ask Questions

Scaling Vision Transformers

Jun 08, 2021
Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, Lucas Beyer

* Xiaohua, Alex, and Lucas contributed equally 

  Access Paper or Ask Questions

MLP-Mixer: An all-MLP Architecture for Vision

May 17, 2021
Ilya Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, Mario Lucic, Alexey Dosovitskiy

* Fixed parameter counts in Table 1 

  Access Paper or Ask Questions

SI-Score: An image dataset for fine-grained analysis of robustness to object location, rotation and size

Apr 09, 2021
Jessica Yung, Rob Romijnders, Alexander Kolesnikov, Lucas Beyer, Josip Djolonga, Neil Houlsby, Sylvain Gelly, Mario Lucic, Xiaohua Zhai

* 4 pages (10 pages including references and appendix), 10 figures. Accepted at the ICLR 2021 RobustML Workshop. arXiv admin note: text overlap with arXiv:2007.08558 

  Access Paper or Ask Questions

Comparing Transfer and Meta Learning Approaches on a Unified Few-Shot Classification Benchmark

Apr 06, 2021
Vincent Dumoulin, Neil Houlsby, Utku Evci, Xiaohua Zhai, Ross Goroshin, Sylvain Gelly, Hugo Larochelle

  Access Paper or Ask Questions

Underspecification Presents Challenges for Credibility in Modern Machine Learning

Nov 06, 2020
Alexander D'Amour, Katherine Heller, Dan Moldovan, Ben Adlam, Babak Alipanahi, Alex Beutel, Christina Chen, Jonathan Deaton, Jacob Eisenstein, Matthew D. Hoffman, Farhad Hormozdiari, Neil Houlsby, Shaobo Hou, Ghassen Jerfel, Alan Karthikesalingam, Mario Lucic, Yian Ma, Cory McLean, Diana Mincu, Akinori Mitani, Andrea Montanari, Zachary Nado, Vivek Natarajan, Christopher Nielson, Thomas F. Osborne, Rajiv Raman, Kim Ramasamy, Rory Sayres, Jessica Schrouff, Martin Seneviratne, Shannon Sequeira, Harini Suresh, Victor Veitch, Max Vladymyrov, Xuezhi Wang, Kellie Webster, Steve Yadlowsky, Taedong Yun, Xiaohua Zhai, D. Sculley

  Access Paper or Ask Questions

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Oct 22, 2020
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, Neil Houlsby

* Fine-tuning code and pre-trained models are available at 

  Access Paper or Ask Questions

Training general representations for remote sensing using in-domain knowledge

Sep 30, 2020
Maxim Neumann, André Susano Pinto, Xiaohua Zhai, Neil Houlsby

* Accepted at the IEEE International Geoscience and Remote Sensing Symposium (IGARSS) 2020. arXiv admin note: substantial text overlap with arXiv:1911.06721 

  Access Paper or Ask Questions

On Robustness and Transferability of Convolutional Neural Networks

Jul 16, 2020
Josip Djolonga, Jessica Yung, Michael Tschannen, Rob Romijnders, Lucas Beyer, Alexander Kolesnikov, Joan Puigcerver, Matthias Minderer, Alexander D'Amour, Dan Moldovan, Sylvan Gelly, Neil Houlsby, Xiaohua Zhai, Mario Lucic

  Access Paper or Ask Questions

Are we done with ImageNet?

Jun 12, 2020
Lucas Beyer, Olivier J. Hénaff, Alexander Kolesnikov, Xiaohua Zhai, Aäron van den Oord

* All five authors contributed equally. New labels at 

  Access Paper or Ask Questions

Large Scale Learning of General Visual Representations for Transfer

Dec 24, 2019
Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Joan Puigcerver, Jessica Yung, Sylvain Gelly, Neil Houlsby

  Access Paper or Ask Questions

In-domain representation learning for remote sensing

Nov 15, 2019
Maxim Neumann, Andre Susano Pinto, Xiaohua Zhai, Neil Houlsby

  Access Paper or Ask Questions

The Visual Task Adaptation Benchmark

Oct 01, 2019
Xiaohua Zhai, Joan Puigcerver, Alexander Kolesnikov, Pierre Ruyssen, Carlos Riquelme, Mario Lucic, Josip Djolonga, Andre Susano Pinto, Maxim Neumann, Alexey Dosovitskiy, Lucas Beyer, Olivier Bachem, Michael Tschannen, Marcin Michalski, Olivier Bousquet, Sylvain Gelly, Neil Houlsby

  Access Paper or Ask Questions

S$^\mathbf{4}$L: Self-Supervised Semi-Supervised Learning

May 09, 2019
Xiaohua Zhai, Avital Oliver, Alexander Kolesnikov, Lucas Beyer

* All four authors contributed equally 

  Access Paper or Ask Questions

High-Fidelity Image Generation With Fewer Labels

Mar 06, 2019
Mario Lucic, Michael Tschannen, Marvin Ritter, Xiaohua Zhai, Olivier Bachem, Sylvain Gelly

* Mario Lucic, Michael Tschannen, and Marvin Ritter contributed equally to this work 

  Access Paper or Ask Questions

Revisiting Self-Supervised Visual Representation Learning

Jan 25, 2019
Alexander Kolesnikov, Xiaohua Zhai, Lucas Beyer

* All three authors contributed equally. Code is available at 

  Access Paper or Ask Questions

Self-Supervised Generative Adversarial Networks

Nov 27, 2018
Ting Chen, Xiaohua Zhai, Marvin Ritter, Mario Lucic, Neil Houlsby

  Access Paper or Ask Questions

Self-Supervised GAN to Counter Forgetting

Oct 27, 2018
Ting Chen, Xiaohua Zhai, Neil Houlsby

  Access Paper or Ask Questions

The GAN Landscape: Losses, Architectures, Regularization, and Normalization

Oct 26, 2018
Karol Kurach, Mario Lucic, Xiaohua Zhai, Marcin Michalski, Sylvain Gelly

* Changed formatting from ICML workshop to ICLR. We added additional resnet ablation studies, hinge loss, and an empirical comparison between KID and FID 

  Access Paper or Ask Questions

MemGEN: Memory is All You Need

Mar 29, 2018
Sylvain Gelly, Karol Kurach, Marcin Michalski, Xiaohua Zhai

  Access Paper or Ask Questions