Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Xiaohua Zhai

How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers


Jun 18, 2021
Andreas Steiner, Alexander Kolesnikov, Xiaohua Zhai, Ross Wightman, Jakob Uszkoreit, Lucas Beyer

* Andreas, Alex, Xiaohua and Lucas contributed equally. We release more than 50'000 ViT models trained under diverse settings on various datasets. We believe this to be a treasure trove for model analysis. Available at https://github.com/google-research/vision_transformer and https://github.com/rwightman/pytorch-image-models 

  Access Paper or Ask Questions

Revisiting the Calibration of Modern Neural Networks


Jun 15, 2021
Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, Mario Lucic


  Access Paper or Ask Questions

Knowledge distillation: A good teacher is patient and consistent


Jun 09, 2021
Lucas Beyer, Xiaohua Zhai, Amélie Royer, Larisa Markeeva, Rohan Anil, Alexander Kolesnikov

* Lucas, Xiaohua, Am\'elie, Larisa, and Alex contributed equally 

  Access Paper or Ask Questions

Scaling Vision Transformers


Jun 08, 2021
Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, Lucas Beyer

* Xiaohua, Alex, and Lucas contributed equally 

  Access Paper or Ask Questions

MLP-Mixer: An all-MLP Architecture for Vision


May 17, 2021
Ilya Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, Mario Lucic, Alexey Dosovitskiy

* Fixed parameter counts in Table 1 

  Access Paper or Ask Questions

SI-Score: An image dataset for fine-grained analysis of robustness to object location, rotation and size


Apr 09, 2021
Jessica Yung, Rob Romijnders, Alexander Kolesnikov, Lucas Beyer, Josip Djolonga, Neil Houlsby, Sylvain Gelly, Mario Lucic, Xiaohua Zhai

* 4 pages (10 pages including references and appendix), 10 figures. Accepted at the ICLR 2021 RobustML Workshop. arXiv admin note: text overlap with arXiv:2007.08558 

  Access Paper or Ask Questions

Comparing Transfer and Meta Learning Approaches on a Unified Few-Shot Classification Benchmark


Apr 06, 2021
Vincent Dumoulin, Neil Houlsby, Utku Evci, Xiaohua Zhai, Ross Goroshin, Sylvain Gelly, Hugo Larochelle


  Access Paper or Ask Questions

Underspecification Presents Challenges for Credibility in Modern Machine Learning


Nov 06, 2020
Alexander D'Amour, Katherine Heller, Dan Moldovan, Ben Adlam, Babak Alipanahi, Alex Beutel, Christina Chen, Jonathan Deaton, Jacob Eisenstein, Matthew D. Hoffman, Farhad Hormozdiari, Neil Houlsby, Shaobo Hou, Ghassen Jerfel, Alan Karthikesalingam, Mario Lucic, Yian Ma, Cory McLean, Diana Mincu, Akinori Mitani, Andrea Montanari, Zachary Nado, Vivek Natarajan, Christopher Nielson, Thomas F. Osborne, Rajiv Raman, Kim Ramasamy, Rory Sayres, Jessica Schrouff, Martin Seneviratne, Shannon Sequeira, Harini Suresh, Victor Veitch, Max Vladymyrov, Xuezhi Wang, Kellie Webster, Steve Yadlowsky, Taedong Yun, Xiaohua Zhai, D. Sculley


  Access Paper or Ask Questions

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale


Oct 22, 2020
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, Neil Houlsby

* Fine-tuning code and pre-trained models are available at https://github.com/google-research/vision_transformer 

  Access Paper or Ask Questions

Training general representations for remote sensing using in-domain knowledge


Sep 30, 2020
Maxim Neumann, André Susano Pinto, Xiaohua Zhai, Neil Houlsby

* Accepted at the IEEE International Geoscience and Remote Sensing Symposium (IGARSS) 2020. arXiv admin note: substantial text overlap with arXiv:1911.06721 

  Access Paper or Ask Questions

On Robustness and Transferability of Convolutional Neural Networks


Jul 16, 2020
Josip Djolonga, Jessica Yung, Michael Tschannen, Rob Romijnders, Lucas Beyer, Alexander Kolesnikov, Joan Puigcerver, Matthias Minderer, Alexander D'Amour, Dan Moldovan, Sylvan Gelly, Neil Houlsby, Xiaohua Zhai, Mario Lucic


  Access Paper or Ask Questions

Are we done with ImageNet?


Jun 12, 2020
Lucas Beyer, Olivier J. Hénaff, Alexander Kolesnikov, Xiaohua Zhai, Aäron van den Oord

* All five authors contributed equally. New labels at https://github.com/google-research/reassessed-imagenet 

  Access Paper or Ask Questions

Large Scale Learning of General Visual Representations for Transfer


Dec 24, 2019
Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Joan Puigcerver, Jessica Yung, Sylvain Gelly, Neil Houlsby


  Access Paper or Ask Questions

In-domain representation learning for remote sensing


Nov 15, 2019
Maxim Neumann, Andre Susano Pinto, Xiaohua Zhai, Neil Houlsby


  Access Paper or Ask Questions

The Visual Task Adaptation Benchmark


Oct 01, 2019
Xiaohua Zhai, Joan Puigcerver, Alexander Kolesnikov, Pierre Ruyssen, Carlos Riquelme, Mario Lucic, Josip Djolonga, Andre Susano Pinto, Maxim Neumann, Alexey Dosovitskiy, Lucas Beyer, Olivier Bachem, Michael Tschannen, Marcin Michalski, Olivier Bousquet, Sylvain Gelly, Neil Houlsby


  Access Paper or Ask Questions

S$^\mathbf{4}$L: Self-Supervised Semi-Supervised Learning


May 09, 2019
Xiaohua Zhai, Avital Oliver, Alexander Kolesnikov, Lucas Beyer

* All four authors contributed equally 

  Access Paper or Ask Questions

High-Fidelity Image Generation With Fewer Labels


Mar 06, 2019
Mario Lucic, Michael Tschannen, Marvin Ritter, Xiaohua Zhai, Olivier Bachem, Sylvain Gelly

* Mario Lucic, Michael Tschannen, and Marvin Ritter contributed equally to this work 

  Access Paper or Ask Questions

Revisiting Self-Supervised Visual Representation Learning


Jan 25, 2019
Alexander Kolesnikov, Xiaohua Zhai, Lucas Beyer

* All three authors contributed equally. Code is available at https://github.com/google/revisiting-self-supervised 

  Access Paper or Ask Questions

Self-Supervised Generative Adversarial Networks


Nov 27, 2018
Ting Chen, Xiaohua Zhai, Marvin Ritter, Mario Lucic, Neil Houlsby


  Access Paper or Ask Questions

Self-Supervised GAN to Counter Forgetting


Oct 27, 2018
Ting Chen, Xiaohua Zhai, Neil Houlsby


  Access Paper or Ask Questions

The GAN Landscape: Losses, Architectures, Regularization, and Normalization


Oct 26, 2018
Karol Kurach, Mario Lucic, Xiaohua Zhai, Marcin Michalski, Sylvain Gelly

* Changed formatting from ICML workshop to ICLR. We added additional resnet ablation studies, hinge loss, and an empirical comparison between KID and FID 

  Access Paper or Ask Questions

MemGEN: Memory is All You Need


Mar 29, 2018
Sylvain Gelly, Karol Kurach, Marcin Michalski, Xiaohua Zhai


  Access Paper or Ask Questions