Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Li Dong

Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment


Jun 11, 2021
Zewen Chi, Li Dong, Bo Zheng, Shaohan Huang, Xian-Ling Mao, Heyan Huang, Furu Wei

* ACL-2021 

  Access Paper or Ask Questions

Memory-Efficient Differentiable Transformer Architecture Search


May 31, 2021
Yuekai Zhao, Li Dong, Yelong Shen, Zhihua Zhang, Furu Wei, Weizhu Chen

* Accepted by Findings of ACL 2021 

  Access Paper or Ask Questions

Zero-shot Cross-lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoders


Apr 18, 2021
Guanhua Chen, Shuming Ma, Yun Chen, Li Dong, Dongdong Zhang, Jia Pan, Wenping Wang, Furu Wei

* Preprint 

  Access Paper or Ask Questions

Knowledge Neurons in Pretrained Transformers


Apr 18, 2021
Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Furu Wei


  Access Paper or Ask Questions

mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs


Apr 18, 2021
Zewen Chi, Li Dong, Shuming Ma, Shaohan Huang Xian-Ling Mao, Heyan Huang, Furu Wei


  Access Paper or Ask Questions

Differentiable Logic Machines


Feb 24, 2021
Matthieu Zimmer, Xuening Feng, Claire Glanois, Zhaohui Jiang, Jianyi Zhang, Paul Weng, Hao Jianye, Li Dong, Liu Wulong


  Access Paper or Ask Questions

MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers


Dec 31, 2020
Wenhui Wang, Hangbo Bao, Shaohan Huang, Li Dong, Furu Wei


  Access Paper or Ask Questions

XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders


Dec 31, 2020
Shuming Ma, Jian Yang, Haoyang Huang, Zewen Chi, Li Dong, Dongdong Zhang, Hany Hassan Awadalla, Alexandre Muzio, Akiko Eriguchi, Saksham Singhal, Xia Song, Arul Menezes, Furu Wei


  Access Paper or Ask Questions

InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training


Jul 15, 2020
Zewen Chi, Li Dong, Furu Wei, Nan Yang, Saksham Singhal, Wenhui Wang, Xia Song, Xian-Ling Mao, Heyan Huang, Ming Zhou

* 11 pages 

  Access Paper or Ask Questions

Distributed Resource Scheduling for Large-Scale MEC Systems: A Multi-Agent Ensemble Deep Reinforcement Learning with Imitation Acceleration


May 21, 2020
Feibo Jiang, Li Dong, Kezhi Wang, Kun Yang, Cunhua Pan

* Submitted for Review 

  Access Paper or Ask Questions

Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks


May 18, 2020
Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, Yejin Choi, Jianfeng Gao

* Code and pre-trained models are released: https://github.com/microsoft/Oscar 

  Access Paper or Ask Questions

Harvesting and Refining Question-Answer Pairs for Unsupervised QA


May 06, 2020
Zhongli Li, Wenhui Wang, Li Dong, Furu Wei, Ke Xu

* Accepted by ACL-20 

  Access Paper or Ask Questions

Self-Attention Attribution: Interpreting Information Interactions Inside Transformer


Apr 23, 2020
Yaru Hao, Li Dong, Furu Wei, Ke Xu

* 11 pages 

  Access Paper or Ask Questions

UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training


Feb 28, 2020
Hangbo Bao, Li Dong, Furu Wei, Wenhui Wang, Nan Yang, Xiaodong Liu, Yu Wang, Songhao Piao, Jianfeng Gao, Ming Zhou, Hsiao-Wuen Hon

* 11 pages 

  Access Paper or Ask Questions

MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers


Feb 25, 2020
Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, Ming Zhou

* Code and models: https://github.com/microsoft/unilm/tree/master/minilm 

  Access Paper or Ask Questions

Stacked Auto Encoder Based Deep Reinforcement Learning for Online Resource Scheduling in Large-Scale MEC Networks


Jan 24, 2020
Feibo Jiang, Kezhi Wang, Li Dong, Cunhua Pan, Kun Yang

* Submitted for review 

  Access Paper or Ask Questions

Can Monolingual Pretrained Models Help Cross-Lingual Classification?


Nov 10, 2019
Zewen Chi, Li Dong, Furu Wei, Xian-Ling Mao, Heyan Huang

* 5 pages 

  Access Paper or Ask Questions

Transforming Wikipedia into Augmented Data for Query-Focused Summarization


Nov 08, 2019
Haichao Zhu, Li Dong, Furu Wei, Bing Qin, Ting Liu


  Access Paper or Ask Questions

Learning A Unified Named Entity Tagger From Multiple Partially Annotated Corpora For Efficient Adaptation


Oct 04, 2019
Xiao Huang, Li Dong, Elizabeth Boschee, Nanyun Peng

* 9 pages of main content + 4 pages of references and appendix. 4 figures and 2 tables in the main content. Accepted by CoNLL 2019 

  Access Paper or Ask Questions

Cross-Lingual Natural Language Generation via Pre-Training


Sep 24, 2019
Zewen Chi, Li Dong, Furu Wei, Wenhui Wang, Xian-Ling Mao, Heyan Huang

* 10 pages 

  Access Paper or Ask Questions

Visualizing and Understanding the Effectiveness of BERT


Aug 15, 2019
Yaru Hao, Li Dong, Furu Wei, Ke Xu

* Accepted by EMNLP-19 

  Access Paper or Ask Questions

Learning to Ask Unanswerable Questions for Machine Reading Comprehension


Jun 14, 2019
Haichao Zhu, Li Dong, Furu Wei, Wenhui Wang, Bing Qin, Ting Liu

* ACL 2019 (long paper) 

  Access Paper or Ask Questions

Data-to-text Generation with Entity Modeling


Jun 07, 2019
Ratish Puduppully, Li Dong, Mirella Lapata

* ACL 2019 

  Access Paper or Ask Questions

Unified Language Model Pre-training for Natural Language Understanding and Generation


May 08, 2019
Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, Hsiao-Wuen Hon


  Access Paper or Ask Questions

An Evaluation of Transfer Learning for Classifying Sales Engagement Emails at Large Scale


Apr 19, 2019
Yong Liu, Pavel Dmitriev, Yifei Huang, Andrew Brooks, Li Dong


  Access Paper or Ask Questions

Data-to-Text Generation with Content Selection and Planning


Sep 03, 2018
Ratish Puduppully, Li Dong, Mirella Lapata


  Access Paper or Ask Questions