Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox
Picture for Furu Wei

Furu Wei

LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understanding


Apr 18, 2021
Yiheng Xu, Tengchao Lv, Lei Cui, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Furu Wei

Figure 1 for LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understanding
Figure 2 for LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understanding
Figure 3 for LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understanding
Figure 4 for LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understanding
* Work in progress 
  Access Paper or Ask Questions

Zero-shot Cross-lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoders


Apr 18, 2021
Guanhua Chen, Shuming Ma, Yun Chen, Li Dong, Dongdong Zhang, Jia Pan, Wenping Wang, Furu Wei

Figure 1 for Zero-shot Cross-lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoders
Figure 2 for Zero-shot Cross-lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoders
Figure 3 for Zero-shot Cross-lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoders
Figure 4 for Zero-shot Cross-lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoders
* Preprint 
  Access Paper or Ask Questions

Knowledge Neurons in Pretrained Transformers


Apr 18, 2021
Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Furu Wei

Figure 1 for Knowledge Neurons in Pretrained Transformers
Figure 2 for Knowledge Neurons in Pretrained Transformers
Figure 3 for Knowledge Neurons in Pretrained Transformers
Figure 4 for Knowledge Neurons in Pretrained Transformers

  Access Paper or Ask Questions

mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs


Apr 18, 2021
Zewen Chi, Li Dong, Shuming Ma, Shaohan Huang Xian-Ling Mao, Heyan Huang, Furu Wei

Figure 1 for mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs
Figure 2 for mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs
Figure 3 for mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs
Figure 4 for mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs

  Access Paper or Ask Questions

Blow the Dog Whistle: A Chinese Dataset for Cant Understanding with Common Sense and World Knowledge


Apr 06, 2021
Canwen Xu, Wangchunshu Zhou, Tao Ge, Ke Xu, Julian McAuley, Furu Wei

Figure 1 for Blow the Dog Whistle: A Chinese Dataset for Cant Understanding with Common Sense and World Knowledge
Figure 2 for Blow the Dog Whistle: A Chinese Dataset for Cant Understanding with Common Sense and World Knowledge
Figure 3 for Blow the Dog Whistle: A Chinese Dataset for Cant Understanding with Common Sense and World Knowledge
Figure 4 for Blow the Dog Whistle: A Chinese Dataset for Cant Understanding with Common Sense and World Knowledge
* NAACL 2021 
  Access Paper or Ask Questions

UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data


Jan 19, 2021
Chengyi Wang, Yu Wu, Yao Qian, Kenichi Kumatani, Shujie Liu, Furu Wei, Michael Zeng, Xuedong Huang

Figure 1 for UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data
Figure 2 for UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data
Figure 3 for UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data
Figure 4 for UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data

  Access Paper or Ask Questions

Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting


Jan 02, 2021
Wangchunshu Zhou, Tao Ge, Ke Xu, Furu Wei

Figure 1 for Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting
Figure 2 for Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting
Figure 3 for Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting
Figure 4 for Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting

  Access Paper or Ask Questions

MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers


Dec 31, 2020
Wenhui Wang, Hangbo Bao, Shaohan Huang, Li Dong, Furu Wei

Figure 1 for MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers
Figure 2 for MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers
Figure 3 for MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers
Figure 4 for MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers

  Access Paper or Ask Questions

XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders


Dec 31, 2020
Shuming Ma, Jian Yang, Haoyang Huang, Zewen Chi, Li Dong, Dongdong Zhang, Hany Hassan Awadalla, Alexandre Muzio, Akiko Eriguchi, Saksham Singhal, Xia Song, Arul Menezes, Furu Wei

Figure 1 for XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders
Figure 2 for XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders
Figure 3 for XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders
Figure 4 for XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders

  Access Paper or Ask Questions

LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding


Dec 29, 2020
Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, Min Zhang, Lidong Zhou

Figure 1 for LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding
Figure 2 for LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding
Figure 3 for LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding
Figure 4 for LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding
* Work in progress 
  Access Paper or Ask Questions

Unsupervised Extractive Summarization by Pre-training Hierarchical Transformers


Oct 16, 2020
Shusheng Xu, Xingxing Zhang, Yi Wu, Furu Wei, Ming Zhou

Figure 1 for Unsupervised Extractive Summarization by Pre-training Hierarchical Transformers
Figure 2 for Unsupervised Extractive Summarization by Pre-training Hierarchical Transformers
Figure 3 for Unsupervised Extractive Summarization by Pre-training Hierarchical Transformers
Figure 4 for Unsupervised Extractive Summarization by Pre-training Hierarchical Transformers
* 9 pages, 4 figures 
  Access Paper or Ask Questions

Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction


Oct 07, 2020
Mengyun Chen, Tao Ge, Xingxing Zhang, Furu Wei, Ming Zhou

Figure 1 for Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction
Figure 2 for Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction
Figure 3 for Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction
Figure 4 for Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction
* Accepted by EMNLP 2020 
  Access Paper or Ask Questions

Generating Commonsense Explanation by Extracting Bridge Concepts from Reasoning Paths


Sep 24, 2020
Haozhe Ji, Pei Ke, Shaohan Huang, Furu Wei, Minlie Huang

Figure 1 for Generating Commonsense Explanation by Extracting Bridge Concepts from Reasoning Paths
Figure 2 for Generating Commonsense Explanation by Extracting Bridge Concepts from Reasoning Paths
Figure 3 for Generating Commonsense Explanation by Extracting Bridge Concepts from Reasoning Paths
Figure 4 for Generating Commonsense Explanation by Extracting Bridge Concepts from Reasoning Paths
* Accepted by AACL-IJCNLP 2020 
  Access Paper or Ask Questions

Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph


Sep 24, 2020
Haozhe Ji, Pei Ke, Shaohan Huang, Furu Wei, Xiaoyan Zhu, Minlie Huang

Figure 1 for Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph
Figure 2 for Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph
Figure 3 for Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph
Figure 4 for Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph
* accepted by EMNLP 2020 
  Access Paper or Ask Questions

InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training


Jul 15, 2020
Zewen Chi, Li Dong, Furu Wei, Nan Yang, Saksham Singhal, Wenhui Wang, Xia Song, Xian-Ling Mao, Heyan Huang, Ming Zhou

Figure 1 for InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training
Figure 2 for InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training
Figure 3 for InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training
Figure 4 for InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training
* 11 pages 
  Access Paper or Ask Questions

BERT Loses Patience: Fast and Robust Inference with Early Exit


Jun 29, 2020
Wangchunshu Zhou, Canwen Xu, Tao Ge, Julian McAuley, Ke Xu, Furu Wei

Figure 1 for BERT Loses Patience: Fast and Robust Inference with Early Exit
Figure 2 for BERT Loses Patience: Fast and Robust Inference with Early Exit
Figure 3 for BERT Loses Patience: Fast and Robust Inference with Early Exit
Figure 4 for BERT Loses Patience: Fast and Robust Inference with Early Exit
* 13 pages 
  Access Paper or Ask Questions

DocBank: A Benchmark Dataset for Document Layout Analysis


Jun 01, 2020
Minghao Li, Yiheng Xu, Lei Cui, Shaohan Huang, Furu Wei, Zhoujun Li, Ming Zhou

Figure 1 for DocBank: A Benchmark Dataset for Document Layout Analysis
Figure 2 for DocBank: A Benchmark Dataset for Document Layout Analysis
Figure 3 for DocBank: A Benchmark Dataset for Document Layout Analysis
Figure 4 for DocBank: A Benchmark Dataset for Document Layout Analysis
* Work in progress 
  Access Paper or Ask Questions

Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks


May 18, 2020
Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, Yejin Choi, Jianfeng Gao

Figure 1 for Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks
Figure 2 for Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks
Figure 3 for Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks
Figure 4 for Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks
* Code and pre-trained models are released: https://github.com/microsoft/Oscar 
  Access Paper or Ask Questions

Harvesting and Refining Question-Answer Pairs for Unsupervised QA


May 06, 2020
Zhongli Li, Wenhui Wang, Li Dong, Furu Wei, Ke Xu

Figure 1 for Harvesting and Refining Question-Answer Pairs for Unsupervised QA
Figure 2 for Harvesting and Refining Question-Answer Pairs for Unsupervised QA
Figure 3 for Harvesting and Refining Question-Answer Pairs for Unsupervised QA
Figure 4 for Harvesting and Refining Question-Answer Pairs for Unsupervised QA
* Accepted by ACL-20 
  Access Paper or Ask Questions

Scheduled DropHead: A Regularization Method for Transformer Models


Apr 28, 2020
Wangchunshu Zhou, Tao Ge, Ke Xu, Furu Wei, Ming Zhou

Figure 1 for Scheduled DropHead: A Regularization Method for Transformer Models
Figure 2 for Scheduled DropHead: A Regularization Method for Transformer Models
Figure 3 for Scheduled DropHead: A Regularization Method for Transformer Models
Figure 4 for Scheduled DropHead: A Regularization Method for Transformer Models

  Access Paper or Ask Questions

Self-Attention Attribution: Interpreting Information Interactions Inside Transformer


Apr 23, 2020
Yaru Hao, Li Dong, Furu Wei, Ke Xu

Figure 1 for Self-Attention Attribution: Interpreting Information Interactions Inside Transformer
Figure 2 for Self-Attention Attribution: Interpreting Information Interactions Inside Transformer
Figure 3 for Self-Attention Attribution: Interpreting Information Interactions Inside Transformer
Figure 4 for Self-Attention Attribution: Interpreting Information Interactions Inside Transformer
* 11 pages 
  Access Paper or Ask Questions

At Which Level Should We Extract? An Empirical Study on Extractive Document Summarization


Apr 06, 2020
Qingyu Zhou, Furu Wei, Ming Zhou

Figure 1 for At Which Level Should We Extract? An Empirical Study on Extractive Document Summarization
Figure 2 for At Which Level Should We Extract? An Empirical Study on Extractive Document Summarization
Figure 3 for At Which Level Should We Extract? An Empirical Study on Extractive Document Summarization
Figure 4 for At Which Level Should We Extract? An Empirical Study on Extractive Document Summarization

  Access Paper or Ask Questions

Learning to Summarize Passages: Mining Passage-Summary Pairs from Wikipedia Revision Histories


Apr 06, 2020
Qingyu Zhou, Furu Wei, Ming Zhou

Figure 1 for Learning to Summarize Passages: Mining Passage-Summary Pairs from Wikipedia Revision Histories
Figure 2 for Learning to Summarize Passages: Mining Passage-Summary Pairs from Wikipedia Revision Histories
Figure 3 for Learning to Summarize Passages: Mining Passage-Summary Pairs from Wikipedia Revision Histories
Figure 4 for Learning to Summarize Passages: Mining Passage-Summary Pairs from Wikipedia Revision Histories

  Access Paper or Ask Questions

STEP: Sequence-to-Sequence Transformer Pre-training for Document Summarization


Apr 04, 2020
Yanyan Zou, Xingxing Zhang, Wei Lu, Furu Wei, Ming Zhou

Figure 1 for STEP: Sequence-to-Sequence Transformer Pre-training for Document Summarization
Figure 2 for STEP: Sequence-to-Sequence Transformer Pre-training for Document Summarization
Figure 3 for STEP: Sequence-to-Sequence Transformer Pre-training for Document Summarization
Figure 4 for STEP: Sequence-to-Sequence Transformer Pre-training for Document Summarization

  Access Paper or Ask Questions

UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training


Feb 28, 2020
Hangbo Bao, Li Dong, Furu Wei, Wenhui Wang, Nan Yang, Xiaodong Liu, Yu Wang, Songhao Piao, Jianfeng Gao, Ming Zhou, Hsiao-Wuen Hon

Figure 1 for UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training
Figure 2 for UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training
Figure 3 for UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training
Figure 4 for UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training
* 11 pages 
  Access Paper or Ask Questions

MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers


Feb 25, 2020
Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, Ming Zhou

Figure 1 for MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
Figure 2 for MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
Figure 3 for MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
Figure 4 for MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
* Code and models: https://github.com/microsoft/unilm/tree/master/minilm 
  Access Paper or Ask Questions

LayoutLM: Pre-training of Text and Layout for Document Image Understanding


Feb 19, 2020
Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou

Figure 1 for LayoutLM: Pre-training of Text and Layout for Document Image Understanding
Figure 2 for LayoutLM: Pre-training of Text and Layout for Document Image Understanding
Figure 3 for LayoutLM: Pre-training of Text and Layout for Document Image Understanding
Figure 4 for LayoutLM: Pre-training of Text and Layout for Document Image Understanding
* Work in progress 
  Access Paper or Ask Questions