Alexander Kolesnikov

Sigmoid Loss for Language Image Pre-Training

Mar 27, 2023
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, Lucas Beyer

* Xiaohua and Lucas contributed equally

Access Paper or Ask Questions

Tuning computer vision models with task rewards

Feb 16, 2023
André Susano Pinto, Alexander Kolesnikov, Yuge Shi, Lucas Beyer, Xiaohua Zhai

Figure 2 for Tuning computer vision models with task rewards

Figure 3 for Tuning computer vision models with task rewards

Figure 4 for Tuning computer vision models with task rewards

* 11 pages

Access Paper or Ask Questions

Scaling Vision Transformers to 22 Billion Parameters

Feb 10, 2023
Mostafa Dehghani, Josip Djolonga, Basil Mustafa, Piotr Padlewski, Jonathan Heek, Justin Gilmer, Andreas Steiner, Mathilde Caron, Robert Geirhos, Ibrahim Alabdulmohsin, Rodolphe Jenatton, Lucas Beyer, Michael Tschannen, Anurag Arnab, Xiao Wang, Carlos Riquelme, Matthias Minderer, Joan Puigcerver, Utku Evci, Manoj Kumar, Sjoerd van Steenkiste, Gamaleldin F. Elsayed, Aravindh Mahendran, Fisher Yu, Avital Oliver, Fantine Huot, Jasmijn Bastings, Mark Patrick Collier, Alexey Gritsenko, Vighnesh Birodkar, Cristina Vasconcelos, Yi Tay, Thomas Mensink, Alexander Kolesnikov, Filip Pavetić, Dustin Tran, Thomas Kipf, Mario Lučić, Xiaohua Zhai, Daniel Keysers, Jeremiah Harmsen, Neil Houlsby

Figure 2 for Scaling Vision Transformers to 22 Billion Parameters

Figure 3 for Scaling Vision Transformers to 22 Billion Parameters

Figure 4 for Scaling Vision Transformers to 22 Billion Parameters

Access Paper or Ask Questions

FlexiViT: One Model for All Patch Sizes

Dec 15, 2022
Lucas Beyer, Pavel Izmailov, Alexander Kolesnikov, Mathilde Caron, Simon Kornblith, Xiaohua Zhai, Matthias Minderer, Michael Tschannen, Ibrahim Alabdulmohsin, Filip Pavetic

Figure 2 for FlexiViT: One Model for All Patch Sizes

Figure 3 for FlexiViT: One Model for All Patch Sizes

Figure 4 for FlexiViT: One Model for All Patch Sizes

* Code and pre-trained models available at https://github.com/google-research/big_vision. All authors made significant technical contributions

Access Paper or Ask Questions

PaLI: A Jointly-Scaled Multilingual Language-Image Model

Sep 16, 2022
Xi Chen, Xiao Wang, Soravit Changpinyo, AJ Piergiovanni, Piotr Padlewski, Daniel Salz, Sebastian Goodman, Adam Grycner, Basil Mustafa, Lucas Beyer, Alexander Kolesnikov, Joan Puigcerver, Nan Ding, Keran Rong, Hassan Akbari, Gaurav Mishra, Linting Xue, Ashish Thapliyal, James Bradbury, Weicheng Kuo, Mojtaba Seyedhosseini, Chao Jia, Burcu Karagol Ayan, Carlos Riquelme, Andreas Steiner, Anelia Angelova, Xiaohua Zhai, Neil Houlsby, Radu Soricut

Figure 2 for PaLI: A Jointly-Scaled Multilingual Language-Image Model

Figure 3 for PaLI: A Jointly-Scaled Multilingual Language-Image Model

Figure 4 for PaLI: A Jointly-Scaled Multilingual Language-Image Model

Access Paper or Ask Questions

UViM: A Unified Modeling Approach for Vision with Learned Guiding Codes

May 27, 2022
Alexander Kolesnikov, André Susano Pinto, Lucas Beyer, Xiaohua Zhai, Jeremiah Harmsen, Neil Houlsby

Figure 2 for UViM: A Unified Modeling Approach for Vision with Learned Guiding Codes

Figure 3 for UViM: A Unified Modeling Approach for Vision with Learned Guiding Codes

Figure 4 for UViM: A Unified Modeling Approach for Vision with Learned Guiding Codes

* Alexander and Andr\'e share the first authorship, all authors made significant technical contributions to this work

Access Paper or Ask Questions

Better plain ViT baselines for ImageNet-1k

May 03, 2022
Lucas Beyer, Xiaohua Zhai, Alexander Kolesnikov

Figure 2 for Better plain ViT baselines for ImageNet-1k

Figure 3 for Better plain ViT baselines for ImageNet-1k

* Code available at https://github.com/google-research/big_vision

Access Paper or Ask Questions

LiT: Zero-Shot Transfer with Locked-image Text Tuning

Nov 15, 2021
Xiaohua Zhai, Xiao Wang, Basil Mustafa, Andreas Steiner, Daniel Keysers, Alexander Kolesnikov, Lucas Beyer

Figure 2 for LiT: Zero-Shot Transfer with Locked-image Text Tuning

Figure 3 for LiT: Zero-Shot Transfer with Locked-image Text Tuning

Figure 4 for LiT: Zero-Shot Transfer with Locked-image Text Tuning

* Xiaohua, Xiao, Basil, Andreas and Lucas contributed equally

Access Paper or Ask Questions

How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers

Jun 18, 2021
Andreas Steiner, Alexander Kolesnikov, Xiaohua Zhai, Ross Wightman, Jakob Uszkoreit, Lucas Beyer

Figure 2 for How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers

Figure 3 for How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers

Figure 4 for How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers

* Andreas, Alex, Xiaohua and Lucas contributed equally. We release more than 50'000 ViT models trained under diverse settings on various datasets. We believe this to be a treasure trove for model analysis. Available at https://github.com/google-research/vision_transformer and https://github.com/rwightman/pytorch-image-models

Access Paper or Ask Questions