Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Sebastian Ruder

BERT memorisation and pitfalls in low-resource scenarios


Apr 16, 2021
Michael Tänzer, Sebastian Ruder, Marek Rei

* 14 pages, 24 figures 

  Access Paper or Ask Questions

IndoNLG: Benchmark and Resources for Evaluating Indonesian Natural Language Generation


Apr 16, 2021
Samuel Cahyawijaya, Genta Indra Winata, Bryan Wilie, Karissa Vincentio, Xiaohong Li, Adhiguna Kuncoro, Sebastian Ruder, Zhi Yuan Lim, Syafri Bahar, Masayu Leylia Khodra, Ayu Purwarianti, Pascale Fung

* Work in progress, 10 pages 

  Access Paper or Ask Questions

XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluation


Apr 15, 2021
Sebastian Ruder, Noah Constant, Jan Botha, Aditya Siddhant, Orhan Firat, Jinlan Fu, Pengfei Liu, Junjie Hu, Graham Neubig, Melvin Johnson


  Access Paper or Ask Questions

Multi-view Subword Regularization


Apr 06, 2021
Xinyi Wang, Sebastian Ruder, Graham Neubig

* Accepted at NAACL 2021 

  Access Paper or Ask Questions

MasakhaNER: Named Entity Recognition for African Languages


Mar 22, 2021
David Ifeoluwa Adelani, Jade Abbott, Graham Neubig, Daniel D'souza, Julia Kreutzer, Constantine Lignos, Chester Palen-Michel, Happy Buzaaba, Shruti Rijhwani, Sebastian Ruder, Stephen Mayhew, Israel Abebe Azime, Shamsuddeen Muhammad, Chris Chinenye Emezue, Joyce Nakatumba-Nabende, Perez Ogayo, Anuoluwapo Aremu, Catherine Gitau, Derguene Mbaye, Jesujoba Alabi, Seid Muhie Yimam, Tajuddeen Gwadabe, Ignatius Ezeani, Rubungo Andre Niyongabo, Jonathan Mukiibi, Verrah Otiende, Iroro Orife, Davis David, Samba Ngom, Tosin Adewumi, Paul Rayson, Mofetoluwa Adeyemi, Gerald Muriuki, Emmanuel Anebi, Chiamaka Chukwuneke, Nkiruka Odu, Eric Peter Wairagala, Samuel Oyerinde, Clemencia Siro, Tobius Saul Bateesa, Temilola Oloyede, Yvonne Wambui, Victor Akinode, Deborah Nabagereka, Maurice Katusiime, Ayodele Awokoya, Mouhamadane MBOUP, Dibora Gebreyohannes, Henok Tilaye, Kelechi Nwaike, Degaga Wolde, Abdoulaye Faye, Blessing Sibanda, Orevaoghene Ahia, Bonaventure F. P. Dossou, Kelechi Ogueji, Thierno Ibrahima DIOP, Abdoulaye Diallo, Adewale Akinfaderin, Tendai Marengereke, Salomey Osei

* Accepted at the AfricaNLP Workshop @EACL 2021 

  Access Paper or Ask Questions

Pitfalls of Static Language Modelling


Feb 03, 2021
Angeliki Lazaridou, Adhiguna Kuncoro, Elena Gribovskaya, Devang Agrawal, Adam Liska, Tayfun Terzi, Mai Gimenez, Cyprien de Masson d'Autume, Sebastian Ruder, Dani Yogatama, Kris Cao, Tomas Kocisky, Susannah Young, Phil Blunsom


  Access Paper or Ask Questions

How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models


Dec 31, 2020
Phillip Rust, Jonas Pfeiffer, Ivan Vulić, Sebastian Ruder, Iryna Gurevych


  Access Paper or Ask Questions

UNKs Everywhere: Adapting Multilingual Language Models to New Scripts


Dec 31, 2020
Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych, Sebastian Ruder


  Access Paper or Ask Questions

Morphologically Aware Word-Level Translation


Nov 15, 2020
Paula Czarnowska, Sebastian Ruder, Ryan Cotterell, Ann Copestake

* COLING 2020 

  Access Paper or Ask Questions

Long Range Arena: A Benchmark for Efficient Transformers


Nov 08, 2020
Yi Tay, Mostafa Dehghani, Samira Abnar, Yikang Shen, Dara Bahri, Philip Pham, Jinfeng Rao, Liu Yang, Sebastian Ruder, Donald Metzler


  Access Paper or Ask Questions

Rethinking embedding coupling in pre-trained language models


Oct 24, 2020
Hyung Won Chung, Thibault Févry, Henry Tsai, Melvin Johnson, Sebastian Ruder


  Access Paper or Ask Questions

AdapterHub: A Framework for Adapting Transformers


Jul 15, 2020
Jonas Pfeiffer, Andreas Rücklé, Clifton Poth, Aishwarya Kamath, Ivan Vulić, Sebastian Ruder, Kyunghyun Cho, Iryna Gurevych


  Access Paper or Ask Questions

MAD-X: An Adapter-based Framework for Multi-task Cross-lingual Transfer


Apr 30, 2020
Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych, Sebastian Ruder


  Access Paper or Ask Questions

A Call for More Rigor in Unsupervised Cross-lingual Learning


Apr 30, 2020
Mikel Artetxe, Sebastian Ruder, Dani Yogatama, Gorka Labaka, Eneko Agirre

* ACL 2020 

  Access Paper or Ask Questions

AxCell: Automatic Extraction of Results from Machine Learning Papers


Apr 29, 2020
Marcin Kardas, Piotr Czapla, Pontus Stenetorp, Sebastian Ruder, Sebastian Riedel, Ross Taylor, Robert Stojnic


  Access Paper or Ask Questions

XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization


Apr 10, 2020
Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, Melvin Johnson


  Access Paper or Ask Questions

Are All Good Word Vector Spaces Isomorphic?


Apr 08, 2020
Ivan Vulić, Sebastian Ruder, Anders Søgaard

* Equal contribution from all three authors 

  Access Paper or Ask Questions

On the Cross-lingual Transferability of Monolingual Representations


Oct 25, 2019
Mikel Artetxe, Sebastian Ruder, Dani Yogatama


  Access Paper or Ask Questions

MultiFiT: Efficient Multi-lingual Language Model Fine-tuning


Sep 10, 2019
Julian Eisenschlos, Sebastian Ruder, Piotr Czapla, Marcin Kardas, Sylvain Gugger, Jeremy Howard

* Proceedings of EMNLP-IJCNLP 2019 

  Access Paper or Ask Questions

What do Deep Networks Like to Read?


Sep 10, 2019
Jonas Pfeiffer, Aishwarya Kamath, Iryna Gurevych, Sebastian Ruder


  Access Paper or Ask Questions

Don't Forget the Long Tail! A Comprehensive Analysis of Morphological Generalization in Bilingual Lexicon Induction


Sep 06, 2019
Paula Czarnowska, Sebastian Ruder, Edouard Grave, Ryan Cotterell, Ann Copestake

* EMNLP 2019 

  Access Paper or Ask Questions

Episodic Memory in Lifelong Language Learning


Jun 03, 2019
Cyprien de Masson d'Autume, Sebastian Ruder, Lingpeng Kong, Dani Yogatama


  Access Paper or Ask Questions

To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Tasks


Mar 14, 2019
Matthew Peters, Sebastian Ruder, Noah A. Smith


  Access Paper or Ask Questions

How to (Properly) Evaluate Cross-Lingual Word Embeddings: On Strong Baselines, Comparative Analyses, and Some Misconceptions


Feb 01, 2019
Goran Glavas, Robert Litschko, Sebastian Ruder, Ivan Vulic


  Access Paper or Ask Questions

A Hierarchical Multi-task Approach for Learning Embeddings from Semantic Tasks


Nov 26, 2018
Victor Sanh, Thomas Wolf, Sebastian Ruder

* 8 pages, 1 figure, To appear in Proceedings of AAAI 2019 

  Access Paper or Ask Questions

Off-the-Shelf Unsupervised NMT


Nov 06, 2018
Chris Hokamp, Sebastian Ruder, John Glover


  Access Paper or Ask Questions

A Discriminative Latent-Variable Model for Bilingual Lexicon Induction


Oct 25, 2018
Sebastian Ruder, Ryan Cotterell, Yova Kementchedjhieva, Anders Søgaard

* Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing 

  Access Paper or Ask Questions