Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Sebastian Ruder

ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning


Nov 22, 2021
Vamsi Aribandi, Yi Tay, Tal Schuster, Jinfeng Rao, Huaixiu Steven Zheng, Sanket Vaibhav Mehta, Honglei Zhuang, Vinh Q. Tran, Dara Bahri, Jianmo Ni, Jai Gupta, Kai Hui, Sebastian Ruder, Donald Metzler


  Access Paper or Ask Questions

Balancing Average and Worst-case Accuracy in Multitask Learning


Oct 12, 2021
Paul Michel, Sebastian Ruder, Dani Yogatama

* Under review 

  Access Paper or Ask Questions

FewNLU: Benchmarking State-of-the-Art Methods for Few-Shot Natural Language Understanding


Sep 27, 2021
Yanan Zheng, Jing Zhou, Yujie Qian, Ming Ding, Jian Li, Ruslan Salakhutdinov, Jie Tang, Sebastian Ruder, Zhilin Yang


  Access Paper or Ask Questions

Efficient Test Time Adapter Ensembling for Low-resource Language Varieties


Sep 10, 2021
Xinyi Wang, Yulia Tsvetkov, Sebastian Ruder, Graham Neubig

* EMNLP 2021 Findings 

  Access Paper or Ask Questions

Charformer: Fast Character Transformers via Gradient-based Subword Tokenization


Jul 02, 2021
Yi Tay, Vinh Q. Tran, Sebastian Ruder, Jai Gupta, Hyung Won Chung, Dara Bahri, Zhen Qin, Simon Baumgartner, Cong Yu, Donald Metzler


  Access Paper or Ask Questions

Compacter: Efficient Low-Rank Hypercomplex Adapter Layers


Jun 08, 2021
Rabeeh Karimi Mahabadi, James Henderson, Sebastian Ruder


  Access Paper or Ask Questions

Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworks


Jun 08, 2021
Rabeeh Karimi Mahabadi, Sebastian Ruder, Mostafa Dehghani, James Henderson

* accepted in ACL, 2021 

  Access Paper or Ask Questions

BERT memorisation and pitfalls in low-resource scenarios


Apr 16, 2021
Michael Tänzer, Sebastian Ruder, Marek Rei

* 14 pages, 24 figures 

  Access Paper or Ask Questions

IndoNLG: Benchmark and Resources for Evaluating Indonesian Natural Language Generation


Apr 16, 2021
Samuel Cahyawijaya, Genta Indra Winata, Bryan Wilie, Karissa Vincentio, Xiaohong Li, Adhiguna Kuncoro, Sebastian Ruder, Zhi Yuan Lim, Syafri Bahar, Masayu Leylia Khodra, Ayu Purwarianti, Pascale Fung

* Work in progress, 10 pages 

  Access Paper or Ask Questions

XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluation


Apr 15, 2021
Sebastian Ruder, Noah Constant, Jan Botha, Aditya Siddhant, Orhan Firat, Jinlan Fu, Pengfei Liu, Junjie Hu, Graham Neubig, Melvin Johnson


  Access Paper or Ask Questions

Multi-view Subword Regularization


Apr 06, 2021
Xinyi Wang, Sebastian Ruder, Graham Neubig

* Accepted at NAACL 2021 

  Access Paper or Ask Questions

MasakhaNER: Named Entity Recognition for African Languages


Mar 22, 2021
David Ifeoluwa Adelani, Jade Abbott, Graham Neubig, Daniel D'souza, Julia Kreutzer, Constantine Lignos, Chester Palen-Michel, Happy Buzaaba, Shruti Rijhwani, Sebastian Ruder, Stephen Mayhew, Israel Abebe Azime, Shamsuddeen Muhammad, Chris Chinenye Emezue, Joyce Nakatumba-Nabende, Perez Ogayo, Anuoluwapo Aremu, Catherine Gitau, Derguene Mbaye, Jesujoba Alabi, Seid Muhie Yimam, Tajuddeen Gwadabe, Ignatius Ezeani, Rubungo Andre Niyongabo, Jonathan Mukiibi, Verrah Otiende, Iroro Orife, Davis David, Samba Ngom, Tosin Adewumi, Paul Rayson, Mofetoluwa Adeyemi, Gerald Muriuki, Emmanuel Anebi, Chiamaka Chukwuneke, Nkiruka Odu, Eric Peter Wairagala, Samuel Oyerinde, Clemencia Siro, Tobius Saul Bateesa, Temilola Oloyede, Yvonne Wambui, Victor Akinode, Deborah Nabagereka, Maurice Katusiime, Ayodele Awokoya, Mouhamadane MBOUP, Dibora Gebreyohannes, Henok Tilaye, Kelechi Nwaike, Degaga Wolde, Abdoulaye Faye, Blessing Sibanda, Orevaoghene Ahia, Bonaventure F. P. Dossou, Kelechi Ogueji, Thierno Ibrahima DIOP, Abdoulaye Diallo, Adewale Akinfaderin, Tendai Marengereke, Salomey Osei

* Accepted at the AfricaNLP Workshop @EACL 2021 

  Access Paper or Ask Questions

Pitfalls of Static Language Modelling


Feb 03, 2021
Angeliki Lazaridou, Adhiguna Kuncoro, Elena Gribovskaya, Devang Agrawal, Adam Liska, Tayfun Terzi, Mai Gimenez, Cyprien de Masson d'Autume, Sebastian Ruder, Dani Yogatama, Kris Cao, Tomas Kocisky, Susannah Young, Phil Blunsom


  Access Paper or Ask Questions

How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models


Dec 31, 2020
Phillip Rust, Jonas Pfeiffer, Ivan Vulić, Sebastian Ruder, Iryna Gurevych


  Access Paper or Ask Questions

UNKs Everywhere: Adapting Multilingual Language Models to New Scripts


Dec 31, 2020
Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych, Sebastian Ruder


  Access Paper or Ask Questions

Morphologically Aware Word-Level Translation


Nov 15, 2020
Paula Czarnowska, Sebastian Ruder, Ryan Cotterell, Ann Copestake

* COLING 2020 

  Access Paper or Ask Questions

Long Range Arena: A Benchmark for Efficient Transformers


Nov 08, 2020
Yi Tay, Mostafa Dehghani, Samira Abnar, Yikang Shen, Dara Bahri, Philip Pham, Jinfeng Rao, Liu Yang, Sebastian Ruder, Donald Metzler


  Access Paper or Ask Questions

Rethinking embedding coupling in pre-trained language models


Oct 24, 2020
Hyung Won Chung, Thibault Févry, Henry Tsai, Melvin Johnson, Sebastian Ruder


  Access Paper or Ask Questions

AdapterHub: A Framework for Adapting Transformers


Jul 15, 2020
Jonas Pfeiffer, Andreas Rücklé, Clifton Poth, Aishwarya Kamath, Ivan Vulić, Sebastian Ruder, Kyunghyun Cho, Iryna Gurevych


  Access Paper or Ask Questions

MAD-X: An Adapter-based Framework for Multi-task Cross-lingual Transfer


Apr 30, 2020
Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych, Sebastian Ruder


  Access Paper or Ask Questions

A Call for More Rigor in Unsupervised Cross-lingual Learning


Apr 30, 2020
Mikel Artetxe, Sebastian Ruder, Dani Yogatama, Gorka Labaka, Eneko Agirre

* ACL 2020 

  Access Paper or Ask Questions

AxCell: Automatic Extraction of Results from Machine Learning Papers


Apr 29, 2020
Marcin Kardas, Piotr Czapla, Pontus Stenetorp, Sebastian Ruder, Sebastian Riedel, Ross Taylor, Robert Stojnic


  Access Paper or Ask Questions

XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization


Apr 10, 2020
Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, Melvin Johnson


  Access Paper or Ask Questions

Are All Good Word Vector Spaces Isomorphic?


Apr 08, 2020
Ivan Vulić, Sebastian Ruder, Anders Søgaard

* Equal contribution from all three authors 

  Access Paper or Ask Questions

On the Cross-lingual Transferability of Monolingual Representations


Oct 25, 2019
Mikel Artetxe, Sebastian Ruder, Dani Yogatama


  Access Paper or Ask Questions

MultiFiT: Efficient Multi-lingual Language Model Fine-tuning


Sep 10, 2019
Julian Eisenschlos, Sebastian Ruder, Piotr Czapla, Marcin Kardas, Sylvain Gugger, Jeremy Howard

* Proceedings of EMNLP-IJCNLP 2019 

  Access Paper or Ask Questions