Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox
Picture for Li Dong

Li Dong

Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment


Jun 11, 2021
Zewen Chi, Li Dong, Bo Zheng, Shaohan Huang, Xian-Ling Mao, Heyan Huang, Furu Wei

Figure 1 for Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment
Figure 2 for Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment
Figure 3 for Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment
Figure 4 for Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment
* ACL-2021 
  Access Paper or Ask Questions

Memory-Efficient Differentiable Transformer Architecture Search


May 31, 2021
Yuekai Zhao, Li Dong, Yelong Shen, Zhihua Zhang, Furu Wei, Weizhu Chen

Figure 1 for Memory-Efficient Differentiable Transformer Architecture Search
Figure 2 for Memory-Efficient Differentiable Transformer Architecture Search
Figure 3 for Memory-Efficient Differentiable Transformer Architecture Search
Figure 4 for Memory-Efficient Differentiable Transformer Architecture Search
* Accepted by Findings of ACL 2021 
  Access Paper or Ask Questions

Zero-shot Cross-lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoders


Apr 18, 2021
Guanhua Chen, Shuming Ma, Yun Chen, Li Dong, Dongdong Zhang, Jia Pan, Wenping Wang, Furu Wei

Figure 1 for Zero-shot Cross-lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoders
Figure 2 for Zero-shot Cross-lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoders
Figure 3 for Zero-shot Cross-lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoders
Figure 4 for Zero-shot Cross-lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoders
* Preprint 
  Access Paper or Ask Questions

Knowledge Neurons in Pretrained Transformers


Apr 18, 2021
Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Furu Wei

Figure 1 for Knowledge Neurons in Pretrained Transformers
Figure 2 for Knowledge Neurons in Pretrained Transformers
Figure 3 for Knowledge Neurons in Pretrained Transformers
Figure 4 for Knowledge Neurons in Pretrained Transformers

  Access Paper or Ask Questions

mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs


Apr 18, 2021
Zewen Chi, Li Dong, Shuming Ma, Shaohan Huang Xian-Ling Mao, Heyan Huang, Furu Wei

Figure 1 for mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs
Figure 2 for mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs
Figure 3 for mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs
Figure 4 for mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs

  Access Paper or Ask Questions

Differentiable Logic Machines


Feb 24, 2021
Matthieu Zimmer, Xuening Feng, Claire Glanois, Zhaohui Jiang, Jianyi Zhang, Paul Weng, Hao Jianye, Li Dong, Liu Wulong

Figure 1 for Differentiable Logic Machines
Figure 2 for Differentiable Logic Machines
Figure 3 for Differentiable Logic Machines
Figure 4 for Differentiable Logic Machines

  Access Paper or Ask Questions

MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers


Dec 31, 2020
Wenhui Wang, Hangbo Bao, Shaohan Huang, Li Dong, Furu Wei

Figure 1 for MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers
Figure 2 for MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers
Figure 3 for MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers
Figure 4 for MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers

  Access Paper or Ask Questions

XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders


Dec 31, 2020
Shuming Ma, Jian Yang, Haoyang Huang, Zewen Chi, Li Dong, Dongdong Zhang, Hany Hassan Awadalla, Alexandre Muzio, Akiko Eriguchi, Saksham Singhal, Xia Song, Arul Menezes, Furu Wei

Figure 1 for XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders
Figure 2 for XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders
Figure 3 for XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders
Figure 4 for XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders

  Access Paper or Ask Questions

InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training


Jul 15, 2020
Zewen Chi, Li Dong, Furu Wei, Nan Yang, Saksham Singhal, Wenhui Wang, Xia Song, Xian-Ling Mao, Heyan Huang, Ming Zhou

Figure 1 for InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training
Figure 2 for InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training
Figure 3 for InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training
Figure 4 for InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training
* 11 pages 
  Access Paper or Ask Questions

Distributed Resource Scheduling for Large-Scale MEC Systems: A Multi-Agent Ensemble Deep Reinforcement Learning with Imitation Acceleration


May 21, 2020
Feibo Jiang, Li Dong, Kezhi Wang, Kun Yang, Cunhua Pan

Figure 1 for Distributed Resource Scheduling for Large-Scale MEC Systems: A Multi-Agent Ensemble Deep Reinforcement Learning with Imitation Acceleration
Figure 2 for Distributed Resource Scheduling for Large-Scale MEC Systems: A Multi-Agent Ensemble Deep Reinforcement Learning with Imitation Acceleration
Figure 3 for Distributed Resource Scheduling for Large-Scale MEC Systems: A Multi-Agent Ensemble Deep Reinforcement Learning with Imitation Acceleration
Figure 4 for Distributed Resource Scheduling for Large-Scale MEC Systems: A Multi-Agent Ensemble Deep Reinforcement Learning with Imitation Acceleration
* Submitted for Review 
  Access Paper or Ask Questions

Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks


May 18, 2020
Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, Yejin Choi, Jianfeng Gao

Figure 1 for Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks
Figure 2 for Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks
Figure 3 for Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks
Figure 4 for Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks
* Code and pre-trained models are released: https://github.com/microsoft/Oscar 
  Access Paper or Ask Questions

Harvesting and Refining Question-Answer Pairs for Unsupervised QA


May 06, 2020
Zhongli Li, Wenhui Wang, Li Dong, Furu Wei, Ke Xu

Figure 1 for Harvesting and Refining Question-Answer Pairs for Unsupervised QA
Figure 2 for Harvesting and Refining Question-Answer Pairs for Unsupervised QA
Figure 3 for Harvesting and Refining Question-Answer Pairs for Unsupervised QA
Figure 4 for Harvesting and Refining Question-Answer Pairs for Unsupervised QA
* Accepted by ACL-20 
  Access Paper or Ask Questions

Self-Attention Attribution: Interpreting Information Interactions Inside Transformer


Apr 23, 2020
Yaru Hao, Li Dong, Furu Wei, Ke Xu

Figure 1 for Self-Attention Attribution: Interpreting Information Interactions Inside Transformer
Figure 2 for Self-Attention Attribution: Interpreting Information Interactions Inside Transformer
Figure 3 for Self-Attention Attribution: Interpreting Information Interactions Inside Transformer
Figure 4 for Self-Attention Attribution: Interpreting Information Interactions Inside Transformer
* 11 pages 
  Access Paper or Ask Questions

UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training


Feb 28, 2020
Hangbo Bao, Li Dong, Furu Wei, Wenhui Wang, Nan Yang, Xiaodong Liu, Yu Wang, Songhao Piao, Jianfeng Gao, Ming Zhou, Hsiao-Wuen Hon

Figure 1 for UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training
Figure 2 for UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training
Figure 3 for UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training
Figure 4 for UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training
* 11 pages 
  Access Paper or Ask Questions

MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers


Feb 25, 2020
Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, Ming Zhou

Figure 1 for MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
Figure 2 for MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
Figure 3 for MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
Figure 4 for MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
* Code and models: https://github.com/microsoft/unilm/tree/master/minilm 
  Access Paper or Ask Questions

Stacked Auto Encoder Based Deep Reinforcement Learning for Online Resource Scheduling in Large-Scale MEC Networks


Jan 24, 2020
Feibo Jiang, Kezhi Wang, Li Dong, Cunhua Pan, Kun Yang

Figure 1 for Stacked Auto Encoder Based Deep Reinforcement Learning for Online Resource Scheduling in Large-Scale MEC Networks
Figure 2 for Stacked Auto Encoder Based Deep Reinforcement Learning for Online Resource Scheduling in Large-Scale MEC Networks
Figure 3 for Stacked Auto Encoder Based Deep Reinforcement Learning for Online Resource Scheduling in Large-Scale MEC Networks
Figure 4 for Stacked Auto Encoder Based Deep Reinforcement Learning for Online Resource Scheduling in Large-Scale MEC Networks
* Submitted for review 
  Access Paper or Ask Questions

Can Monolingual Pretrained Models Help Cross-Lingual Classification?


Nov 10, 2019
Zewen Chi, Li Dong, Furu Wei, Xian-Ling Mao, Heyan Huang

Figure 1 for Can Monolingual Pretrained Models Help Cross-Lingual Classification?
Figure 2 for Can Monolingual Pretrained Models Help Cross-Lingual Classification?
Figure 3 for Can Monolingual Pretrained Models Help Cross-Lingual Classification?
Figure 4 for Can Monolingual Pretrained Models Help Cross-Lingual Classification?
* 5 pages 
  Access Paper or Ask Questions

Transforming Wikipedia into Augmented Data for Query-Focused Summarization


Nov 08, 2019
Haichao Zhu, Li Dong, Furu Wei, Bing Qin, Ting Liu

Figure 1 for Transforming Wikipedia into Augmented Data for Query-Focused Summarization
Figure 2 for Transforming Wikipedia into Augmented Data for Query-Focused Summarization
Figure 3 for Transforming Wikipedia into Augmented Data for Query-Focused Summarization
Figure 4 for Transforming Wikipedia into Augmented Data for Query-Focused Summarization

  Access Paper or Ask Questions

Learning A Unified Named Entity Tagger From Multiple Partially Annotated Corpora For Efficient Adaptation


Oct 04, 2019
Xiao Huang, Li Dong, Elizabeth Boschee, Nanyun Peng

Figure 1 for Learning A Unified Named Entity Tagger From Multiple Partially Annotated Corpora For Efficient Adaptation
Figure 2 for Learning A Unified Named Entity Tagger From Multiple Partially Annotated Corpora For Efficient Adaptation
Figure 3 for Learning A Unified Named Entity Tagger From Multiple Partially Annotated Corpora For Efficient Adaptation
Figure 4 for Learning A Unified Named Entity Tagger From Multiple Partially Annotated Corpora For Efficient Adaptation
* 9 pages of main content + 4 pages of references and appendix. 4 figures and 2 tables in the main content. Accepted by CoNLL 2019 
  Access Paper or Ask Questions

Cross-Lingual Natural Language Generation via Pre-Training


Sep 24, 2019
Zewen Chi, Li Dong, Furu Wei, Wenhui Wang, Xian-Ling Mao, Heyan Huang

Figure 1 for Cross-Lingual Natural Language Generation via Pre-Training
Figure 2 for Cross-Lingual Natural Language Generation via Pre-Training
Figure 3 for Cross-Lingual Natural Language Generation via Pre-Training
Figure 4 for Cross-Lingual Natural Language Generation via Pre-Training
* 10 pages 
  Access Paper or Ask Questions

Visualizing and Understanding the Effectiveness of BERT


Aug 15, 2019
Yaru Hao, Li Dong, Furu Wei, Ke Xu

Figure 1 for Visualizing and Understanding the Effectiveness of BERT
Figure 2 for Visualizing and Understanding the Effectiveness of BERT
Figure 3 for Visualizing and Understanding the Effectiveness of BERT
Figure 4 for Visualizing and Understanding the Effectiveness of BERT
* Accepted by EMNLP-19 
  Access Paper or Ask Questions

Learning to Ask Unanswerable Questions for Machine Reading Comprehension


Jun 14, 2019
Haichao Zhu, Li Dong, Furu Wei, Wenhui Wang, Bing Qin, Ting Liu

Figure 1 for Learning to Ask Unanswerable Questions for Machine Reading Comprehension
Figure 2 for Learning to Ask Unanswerable Questions for Machine Reading Comprehension
Figure 3 for Learning to Ask Unanswerable Questions for Machine Reading Comprehension
Figure 4 for Learning to Ask Unanswerable Questions for Machine Reading Comprehension
* ACL 2019 (long paper) 
  Access Paper or Ask Questions

Data-to-text Generation with Entity Modeling


Jun 07, 2019
Ratish Puduppully, Li Dong, Mirella Lapata

Figure 1 for Data-to-text Generation with Entity Modeling
Figure 2 for Data-to-text Generation with Entity Modeling
Figure 3 for Data-to-text Generation with Entity Modeling
Figure 4 for Data-to-text Generation with Entity Modeling
* ACL 2019 
  Access Paper or Ask Questions

Unified Language Model Pre-training for Natural Language Understanding and Generation


May 08, 2019
Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, Hsiao-Wuen Hon

Figure 1 for Unified Language Model Pre-training for Natural Language Understanding and Generation
Figure 2 for Unified Language Model Pre-training for Natural Language Understanding and Generation
Figure 3 for Unified Language Model Pre-training for Natural Language Understanding and Generation
Figure 4 for Unified Language Model Pre-training for Natural Language Understanding and Generation

  Access Paper or Ask Questions

An Evaluation of Transfer Learning for Classifying Sales Engagement Emails at Large Scale


Apr 19, 2019
Yong Liu, Pavel Dmitriev, Yifei Huang, Andrew Brooks, Li Dong

Figure 1 for An Evaluation of Transfer Learning for Classifying Sales Engagement Emails at Large Scale
Figure 2 for An Evaluation of Transfer Learning for Classifying Sales Engagement Emails at Large Scale
Figure 3 for An Evaluation of Transfer Learning for Classifying Sales Engagement Emails at Large Scale

  Access Paper or Ask Questions

Data-to-Text Generation with Content Selection and Planning


Sep 03, 2018
Ratish Puduppully, Li Dong, Mirella Lapata

Figure 1 for Data-to-Text Generation with Content Selection and Planning
Figure 2 for Data-to-Text Generation with Content Selection and Planning
Figure 3 for Data-to-Text Generation with Content Selection and Planning
Figure 4 for Data-to-Text Generation with Content Selection and Planning

  Access Paper or Ask Questions