Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Neil Houlsby

Sparse MoEs meet Efficient Ensembles


Oct 07, 2021
James Urquhart Allingham, Florian Wenzel, Zelda E Mariet, Basil Mustafa, Joan Puigcerver, Neil Houlsby, Ghassen Jerfel, Vincent Fortuin, Balaji Lakshminarayanan, Jasper Snoek, Dustin Tran, Carlos Riquelme Ruiz, Rodolphe Jenatton

* 44 pages, 19 figures, 24 tables 

  Access Paper or Ask Questions

The Benchmark Lottery


Jul 14, 2021
Mostafa Dehghani, Yi Tay, Alexey A. Gritsenko, Zhe Zhao, Neil Houlsby, Fernando Diaz, Donald Metzler, Oriol Vinyals


  Access Paper or Ask Questions

Revisiting the Calibration of Modern Neural Networks


Jun 15, 2021
Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, Mario Lucic


  Access Paper or Ask Questions

Scaling Vision with Sparse Mixture of Experts


Jun 10, 2021
Carlos Riquelme, Joan Puigcerver, Basil Mustafa, Maxim Neumann, Rodolphe Jenatton, André Susano Pinto, Daniel Keysers, Neil Houlsby

* 44 pages, 38 figures 

  Access Paper or Ask Questions

Scaling Vision Transformers


Jun 08, 2021
Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, Lucas Beyer

* Xiaohua, Alex, and Lucas contributed equally 

  Access Paper or Ask Questions

MLP-Mixer: An all-MLP Architecture for Vision


May 17, 2021
Ilya Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, Mario Lucic, Alexey Dosovitskiy

* Fixed parameter counts in Table 1 

  Access Paper or Ask Questions

SI-Score: An image dataset for fine-grained analysis of robustness to object location, rotation and size


Apr 09, 2021
Jessica Yung, Rob Romijnders, Alexander Kolesnikov, Lucas Beyer, Josip Djolonga, Neil Houlsby, Sylvain Gelly, Mario Lucic, Xiaohua Zhai

* 4 pages (10 pages including references and appendix), 10 figures. Accepted at the ICLR 2021 RobustML Workshop. arXiv admin note: text overlap with arXiv:2007.08558 

  Access Paper or Ask Questions

Comparing Transfer and Meta Learning Approaches on a Unified Few-Shot Classification Benchmark


Apr 06, 2021
Vincent Dumoulin, Neil Houlsby, Utku Evci, Xiaohua Zhai, Ross Goroshin, Sylvain Gelly, Hugo Larochelle


  Access Paper or Ask Questions

Supervised Transfer Learning at Scale for Medical Imaging


Jan 21, 2021
Basil Mustafa, Aaron Loh, Jan Freyberg, Patricia MacWilliams, Megan Wilson, Scott Mayer McKinney, Marcin Sieniek, Jim Winkens, Yuan Liu, Peggy Bui, Shruthi Prabhakara, Umesh Telang, Alan Karthikesalingam, Neil Houlsby, Vivek Natarajan


  Access Paper or Ask Questions

Underspecification Presents Challenges for Credibility in Modern Machine Learning


Nov 06, 2020
Alexander D'Amour, Katherine Heller, Dan Moldovan, Ben Adlam, Babak Alipanahi, Alex Beutel, Christina Chen, Jonathan Deaton, Jacob Eisenstein, Matthew D. Hoffman, Farhad Hormozdiari, Neil Houlsby, Shaobo Hou, Ghassen Jerfel, Alan Karthikesalingam, Mario Lucic, Yian Ma, Cory McLean, Diana Mincu, Akinori Mitani, Andrea Montanari, Zachary Nado, Vivek Natarajan, Christopher Nielson, Thomas F. Osborne, Rajiv Raman, Kim Ramasamy, Rory Sayres, Jessica Schrouff, Martin Seneviratne, Shannon Sequeira, Harini Suresh, Victor Veitch, Max Vladymyrov, Xuezhi Wang, Kellie Webster, Steve Yadlowsky, Taedong Yun, Xiaohua Zhai, D. Sculley


  Access Paper or Ask Questions

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale


Oct 22, 2020
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, Neil Houlsby

* Fine-tuning code and pre-trained models are available at https://github.com/google-research/vision_transformer 

  Access Paper or Ask Questions

Deep Ensembles for Low-Data Transfer Learning


Oct 19, 2020
Basil Mustafa, Carlos Riquelme, Joan Puigcerver, André Susano Pinto, Daniel Keysers, Neil Houlsby


  Access Paper or Ask Questions

Representation learning from videos in-the-wild: An object-centric approach


Oct 06, 2020
Rob Romijnders, Aravindh Mahendran, Michael Tschannen, Josip Djolonga, Marvin Ritter, Neil Houlsby, Mario Lucic


  Access Paper or Ask Questions

Training general representations for remote sensing using in-domain knowledge


Sep 30, 2020
Maxim Neumann, André Susano Pinto, Xiaohua Zhai, Neil Houlsby

* Accepted at the IEEE International Geoscience and Remote Sensing Symposium (IGARSS) 2020. arXiv admin note: substantial text overlap with arXiv:1911.06721 

  Access Paper or Ask Questions

Scalable Transfer Learning with Expert Models


Sep 28, 2020
Joan Puigcerver, Carlos Riquelme, Basil Mustafa, Cedric Renggli, André Susano Pinto, Sylvain Gelly, Daniel Keysers, Neil Houlsby


  Access Paper or Ask Questions

On Robustness and Transferability of Convolutional Neural Networks


Jul 16, 2020
Josip Djolonga, Jessica Yung, Michael Tschannen, Rob Romijnders, Lucas Beyer, Alexander Kolesnikov, Joan Puigcerver, Matthias Minderer, Alexander D'Amour, Dan Moldovan, Sylvan Gelly, Neil Houlsby, Xiaohua Zhai, Mario Lucic


  Access Paper or Ask Questions

Automatic Shortcut Removal for Self-Supervised Representation Learning


Feb 21, 2020
Matthias Minderer, Olivier Bachem, Neil Houlsby, Michael Tschannen


  Access Paper or Ask Questions

Large Scale Learning of General Visual Representations for Transfer


Dec 24, 2019
Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Joan Puigcerver, Jessica Yung, Sylvain Gelly, Neil Houlsby


  Access Paper or Ask Questions

Self-Supervised Learning of Video-Induced Visual Invariances


Dec 05, 2019
Michael Tschannen, Josip Djolonga, Marvin Ritter, Aravindh Mahendran, Neil Houlsby, Sylvain Gelly, Mario Lucic


  Access Paper or Ask Questions

In-domain representation learning for remote sensing


Nov 15, 2019
Maxim Neumann, Andre Susano Pinto, Xiaohua Zhai, Neil Houlsby


  Access Paper or Ask Questions

The Visual Task Adaptation Benchmark


Oct 01, 2019
Xiaohua Zhai, Joan Puigcerver, Alexander Kolesnikov, Pierre Ruyssen, Carlos Riquelme, Mario Lucic, Josip Djolonga, Andre Susano Pinto, Maxim Neumann, Alexey Dosovitskiy, Lucas Beyer, Olivier Bachem, Michael Tschannen, Marcin Michalski, Olivier Bousquet, Sylvain Gelly, Neil Houlsby


  Access Paper or Ask Questions

Parameter-Efficient Transfer Learning for NLP


Feb 02, 2019
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, Sylvain Gelly


  Access Paper or Ask Questions

Neural Architecture Search Over a Graph Search Space


Dec 27, 2018
Quentin de Laroussilhe, Stanisław Jastrzębski, Neil Houlsby, Andrea Gesmundo


  Access Paper or Ask Questions

Self-Supervised Generative Adversarial Networks


Nov 27, 2018
Ting Chen, Xiaohua Zhai, Marvin Ritter, Mario Lucic, Neil Houlsby


  Access Paper or Ask Questions

Self-Supervised GAN to Counter Forgetting


Oct 27, 2018
Ting Chen, Xiaohua Zhai, Neil Houlsby


  Access Paper or Ask Questions

On Self Modulation for Generative Adversarial Networks


Oct 02, 2018
Ting Chen, Mario Lucic, Neil Houlsby, Sylvain Gelly


  Access Paper or Ask Questions

Transfer Learning with Neural AutoML


Sep 27, 2018
Catherine Wong, Neil Houlsby, Yifeng Lu, Andrea Gesmundo


  Access Paper or Ask Questions