Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Orhan Firat

Towards the Next 1000 Languages in Multilingual Machine Translation: Exploring the Synergy Between Supervised and Self-Supervised Learning


Jan 13, 2022
Aditya Siddhant, Ankur Bapna, Orhan Firat, Yuan Cao, Mia Xu Chen, Isaac Caswell, Xavier Garcia


  Access Paper or Ask Questions

GLaM: Efficient Scaling of Language Models with Mixture-of-Experts


Dec 13, 2021
Nan Du, Yanping Huang, Andrew M. Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, Barret Zoph, Liam Fedus, Maarten Bosma, Zongwei Zhou, Tao Wang, Yu Emma Wang, Kellie Webster, Marie Pellat, Kevin Robinson, Kathy Meier-Hellstern, Toju Duke, Lucas Dixon, Kun Zhang, Quoc V Le, Yonghui Wu, Zhifeng Chen, Claire Cui


  Access Paper or Ask Questions

A Loss Curvature Perspective on Training Instability in Deep Learning


Oct 08, 2021
Justin Gilmer, Behrooz Ghorbani, Ankush Garg, Sneha Kudugunta, Behnam Neyshabur, David Cardoze, George Dahl, Zachary Nado, Orhan Firat

* 20 pages, 16 figures 

  Access Paper or Ask Questions

Beyond Distillation: Task-level Mixture-of-Experts for Efficient Inference


Sep 24, 2021
Sneha Kudugunta, Yanping Huang, Ankur Bapna, Maxim Krikun, Dmitry Lepikhin, Minh-Thang Luong, Orhan Firat

* EMNLP Findings 2021 

  Access Paper or Ask Questions

Multilingual Document-Level Translation Enables Zero-Shot Transfer From Sentences to Documents


Sep 21, 2021
Biao Zhang, Ankur Bapna, Melvin Johnson, Ali Dabirmoghaddam, Naveen Arivazhagan, Orhan Firat


  Access Paper or Ask Questions

Towards Zero-Label Language Learning


Sep 19, 2021
Zirui Wang, Adams Wei Yu, Orhan Firat, Yuan Cao


  Access Paper or Ask Questions

Scaling Laws for Neural Machine Translation


Sep 16, 2021
Behrooz Ghorbani, Orhan Firat, Markus Freitag, Ankur Bapna, Maxim Krikun, Xavier Garcia, Ciprian Chelba, Colin Cherry

* 31 pages, 23 figures 

  Access Paper or Ask Questions

Evaluating Multiway Multilingual NMT in the Turkic Languages


Sep 13, 2021
Jamshidbek Mirzakhalov, Anoop Babu, Aigiz Kunafin, Ahsan Wahab, Behzod Moydinboyev, Sardana Ivanova, Mokhiyakhon Uzokova, Shaxnoza Pulatova, Duygu Ataman, Julia Kreutzer, Francis Tyers, Orhan Firat, John Licato, Sriram Chellappan

* 9 pages, 3 figures, 7 tables. To be presented at WMT 2021 

  Access Paper or Ask Questions

A Large-Scale Study of Machine Translation in the Turkic Languages


Sep 09, 2021
Jamshidbek Mirzakhalov, Anoop Babu, Duygu Ataman, Sherzod Kariev, Francis Tyers, Otabek Abduraufov, Mammad Hajili, Sardana Ivanova, Abror Khaytbaev, Antonio Laverghetta Jr., Behzodbek Moydinboyev, Esra Onal, Shaxnoza Pulatova, Ahsan Wahab, Orhan Firat, Sriram Chellappan

* 9 pages, 1 figure, 8 tables. Main proceedings of EMNLP 2021 

  Access Paper or Ask Questions

Towards Universality in Multilingual Text Rewriting


Jul 30, 2021
Xavier Garcia, Noah Constant, Mandy Guo, Orhan Firat


  Access Paper or Ask Questions

XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluation


Apr 15, 2021
Sebastian Ruder, Noah Constant, Jan Botha, Aditya Siddhant, Orhan Firat, Jinlan Fu, Pengfei Liu, Junjie Hu, Graham Neubig, Melvin Johnson


  Access Paper or Ask Questions

Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets


Mar 22, 2021
Isaac Caswell, Julia Kreutzer, Lisa Wang, Ahsan Wahab, Daan van Esch, Nasanbayar Ulzii-Orshikh, Allahsera Tapo, Nishant Subramani, Artem Sokolov, Claytone Sikasote, Monang Setyawan, Supheakmungkol Sarin, Sokhar Samb, Benoît Sagot, Clara Rivera, Annette Rios, Isabel Papadimitriou, Salomey Osei, Pedro Javier Ortiz Suárez, Iroro Orife, Kelechi Ogueji, Rubungo Andre Niyongabo, Toan Q. Nguyen, Mathias Müller, André Müller, Shamsuddeen Hassan Muhammad, Nanda Muhammad, Ayanda Mnyakeni, Jamshidbek Mirzakhalov, Tapiwanashe Matangira, Colin Leong, Nze Lawson, Sneha Kudugunta, Yacine Jernite, Mathias Jenny, Orhan Firat, Bonaventure F. P. Dossou, Sakhile Dlamini, Nisansa de Silva, Sakine Çabuk Ballı, Stella Biderman, Alessia Battisti, Ahmed Baruwa, Ankur Bapna, Pallavi Baljekar, Israel Abebe Azime, Ayodele Awokoya, Duygu Ataman, Orevaoghene Ahia, Oghenefego Ahia, Sweta Agrawal, Mofetoluwa Adeyemi

* 10 pages paper; 10 pages appendix; AfricaNLP 2021 

  Access Paper or Ask Questions

Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution


Mar 11, 2021
Xavier Garcia, Noah Constant, Ankur P. Parikh, Orhan Firat

* Accepted at NAACL 2021 

  Access Paper or Ask Questions

Gradient-guided Loss Masking for Neural Machine Translation


Feb 26, 2021
Xinyi Wang, Ankur Bapna, Melvin Johnson, Orhan Firat


  Access Paper or Ask Questions

Rapid Domain Adaptation for Machine Translation with Monolingual Data


Oct 23, 2020
Mahdis Mahdieh, Mia Xu Chen, Yuan Cao, Orhan Firat


  Access Paper or Ask Questions

Towards End-to-End In-Image Neural Machine Translation


Oct 20, 2020
Elman Mansimov, Mitchell Stern, Mia Chen, Orhan Firat, Jakob Uszkoreit, Puneet Jain

* Accepted as an oral presentation at EMNLP, NLP Beyond Text workshop, 2020 

  Access Paper or Ask Questions

Complete Multilingual Neural Machine Translation


Oct 20, 2020
Markus Freitag, Orhan Firat

* Accepted at WMT 2020 

  Access Paper or Ask Questions

Explicit Alignment Objectives for Multilingual Bidirectional Encoders


Oct 15, 2020
Junjie Hu, Melvin Johnson, Orhan Firat, Aditya Siddhant, Graham Neubig


  Access Paper or Ask Questions

Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models


Oct 12, 2020
Zirui Wang, Yulia Tsvetkov, Orhan Firat, Yuan Cao


  Access Paper or Ask Questions

Harnessing Multilinguality in Unsupervised Machine Translation for Rare Languages


Sep 23, 2020
Xavier Garcia, Aditya Siddhant, Orhan Firat, Ankur P. Parikh


  Access Paper or Ask Questions

GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding


Jun 30, 2020
Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, Zhifeng Chen


  Access Paper or Ask Questions

Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine Translation


May 11, 2020
Aditya Siddhant, Ankur Bapna, Yuan Cao, Orhan Firat, Mia Chen, Sneha Kudugunta, Naveen Arivazhagan, Yonghui Wu

* ACL 2020 

  Access Paper or Ask Questions

XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization


Apr 10, 2020
Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, Melvin Johnson


  Access Paper or Ask Questions

On the Discrepancy between Density Estimation and Sequence Generation


Feb 17, 2020
Jason Lee, Dustin Tran, Orhan Firat, Kyunghyun Cho


  Access Paper or Ask Questions

Controlling Computation versus Quality for Neural Sequence Models


Feb 17, 2020
Ankur Bapna, Naveen Arivazhagan, Orhan Firat


  Access Paper or Ask Questions

Fill in the Blanks: Imputing Missing Sentences for Larger-Context Neural Machine Translation


Oct 30, 2019
Sébastien Jean, Ankur Bapna, Orhan Firat


  Access Paper or Ask Questions

On the Importance of Word Boundaries in Character-level Neural Machine Translation


Oct 21, 2019
Duygu Ataman, Orhan Firat, Mattia A. Di Gangi, Marcello Federico, Alexandra Birch

* To appear at the 3rd Workshop on Neural Generation and Translation (WNGT 2019) 

  Access Paper or Ask Questions