Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox
Picture for Furu Wei

Furu Wei

UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data


Jan 19, 2021
Chengyi Wang, Yu Wu, Yao Qian, Kenichi Kumatani, Shujie Liu, Furu Wei, Michael Zeng, Xuedong Huang

Figure 1 for UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data
Figure 2 for UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data
Figure 3 for UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data
Figure 4 for UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data

  Access Paper or Ask Questions

Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting


Jan 02, 2021
Wangchunshu Zhou, Tao Ge, Ke Xu, Furu Wei

Figure 1 for Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting
Figure 2 for Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting
Figure 3 for Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting
Figure 4 for Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting

  Access Paper or Ask Questions

MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers


Dec 31, 2020
Wenhui Wang, Hangbo Bao, Shaohan Huang, Li Dong, Furu Wei

Figure 1 for MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers
Figure 2 for MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers
Figure 3 for MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers
Figure 4 for MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers

  Access Paper or Ask Questions

XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders


Dec 31, 2020
Shuming Ma, Jian Yang, Haoyang Huang, Zewen Chi, Li Dong, Dongdong Zhang, Hany Hassan Awadalla, Alexandre Muzio, Akiko Eriguchi, Saksham Singhal, Xia Song, Arul Menezes, Furu Wei

Figure 1 for XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders
Figure 2 for XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders
Figure 3 for XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders
Figure 4 for XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders

  Access Paper or Ask Questions

LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding


Dec 29, 2020
Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, Min Zhang, Lidong Zhou

Figure 1 for LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding
Figure 2 for LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding
Figure 3 for LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding
Figure 4 for LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding
* Work in progress 
  Access Paper or Ask Questions

Unsupervised Extractive Summarization by Pre-training Hierarchical Transformers


Oct 16, 2020
Shusheng Xu, Xingxing Zhang, Yi Wu, Furu Wei, Ming Zhou

Figure 1 for Unsupervised Extractive Summarization by Pre-training Hierarchical Transformers
Figure 2 for Unsupervised Extractive Summarization by Pre-training Hierarchical Transformers
Figure 3 for Unsupervised Extractive Summarization by Pre-training Hierarchical Transformers
Figure 4 for Unsupervised Extractive Summarization by Pre-training Hierarchical Transformers
* 9 pages, 4 figures 
  Access Paper or Ask Questions

Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction


Oct 07, 2020
Mengyun Chen, Tao Ge, Xingxing Zhang, Furu Wei, Ming Zhou

Figure 1 for Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction
Figure 2 for Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction
Figure 3 for Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction
Figure 4 for Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction
* Accepted by EMNLP 2020 
  Access Paper or Ask Questions

Generating Commonsense Explanation by Extracting Bridge Concepts from Reasoning Paths


Sep 24, 2020
Haozhe Ji, Pei Ke, Shaohan Huang, Furu Wei, Minlie Huang

Figure 1 for Generating Commonsense Explanation by Extracting Bridge Concepts from Reasoning Paths
Figure 2 for Generating Commonsense Explanation by Extracting Bridge Concepts from Reasoning Paths
Figure 3 for Generating Commonsense Explanation by Extracting Bridge Concepts from Reasoning Paths
Figure 4 for Generating Commonsense Explanation by Extracting Bridge Concepts from Reasoning Paths
* Accepted by AACL-IJCNLP 2020 
  Access Paper or Ask Questions

Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph


Sep 24, 2020
Haozhe Ji, Pei Ke, Shaohan Huang, Furu Wei, Xiaoyan Zhu, Minlie Huang

Figure 1 for Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph
Figure 2 for Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph
Figure 3 for Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph
Figure 4 for Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph
* accepted by EMNLP 2020 
  Access Paper or Ask Questions

InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training


Jul 15, 2020
Zewen Chi, Li Dong, Furu Wei, Nan Yang, Saksham Singhal, Wenhui Wang, Xia Song, Xian-Ling Mao, Heyan Huang, Ming Zhou

Figure 1 for InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training
Figure 2 for InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training
Figure 3 for InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training
Figure 4 for InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training
* 11 pages 
  Access Paper or Ask Questions

BERT Loses Patience: Fast and Robust Inference with Early Exit


Jun 29, 2020
Wangchunshu Zhou, Canwen Xu, Tao Ge, Julian McAuley, Ke Xu, Furu Wei

Figure 1 for BERT Loses Patience: Fast and Robust Inference with Early Exit
Figure 2 for BERT Loses Patience: Fast and Robust Inference with Early Exit
Figure 3 for BERT Loses Patience: Fast and Robust Inference with Early Exit
Figure 4 for BERT Loses Patience: Fast and Robust Inference with Early Exit
* 13 pages 
  Access Paper or Ask Questions

DocBank: A Benchmark Dataset for Document Layout Analysis


Jun 01, 2020
Minghao Li, Yiheng Xu, Lei Cui, Shaohan Huang, Furu Wei, Zhoujun Li, Ming Zhou

Figure 1 for DocBank: A Benchmark Dataset for Document Layout Analysis
Figure 2 for DocBank: A Benchmark Dataset for Document Layout Analysis
Figure 3 for DocBank: A Benchmark Dataset for Document Layout Analysis
Figure 4 for DocBank: A Benchmark Dataset for Document Layout Analysis
* Work in progress 
  Access Paper or Ask Questions

Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks


May 18, 2020
Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, Yejin Choi, Jianfeng Gao

Figure 1 for Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks
Figure 2 for Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks
Figure 3 for Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks
Figure 4 for Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks
* Code and pre-trained models are released: https://github.com/microsoft/Oscar 
  Access Paper or Ask Questions

Harvesting and Refining Question-Answer Pairs for Unsupervised QA


May 06, 2020
Zhongli Li, Wenhui Wang, Li Dong, Furu Wei, Ke Xu

Figure 1 for Harvesting and Refining Question-Answer Pairs for Unsupervised QA
Figure 2 for Harvesting and Refining Question-Answer Pairs for Unsupervised QA
Figure 3 for Harvesting and Refining Question-Answer Pairs for Unsupervised QA
Figure 4 for Harvesting and Refining Question-Answer Pairs for Unsupervised QA
* Accepted by ACL-20 
  Access Paper or Ask Questions

Scheduled DropHead: A Regularization Method for Transformer Models


Apr 28, 2020
Wangchunshu Zhou, Tao Ge, Ke Xu, Furu Wei, Ming Zhou

Figure 1 for Scheduled DropHead: A Regularization Method for Transformer Models
Figure 2 for Scheduled DropHead: A Regularization Method for Transformer Models
Figure 3 for Scheduled DropHead: A Regularization Method for Transformer Models
Figure 4 for Scheduled DropHead: A Regularization Method for Transformer Models

  Access Paper or Ask Questions

Self-Attention Attribution: Interpreting Information Interactions Inside Transformer


Apr 23, 2020
Yaru Hao, Li Dong, Furu Wei, Ke Xu

Figure 1 for Self-Attention Attribution: Interpreting Information Interactions Inside Transformer
Figure 2 for Self-Attention Attribution: Interpreting Information Interactions Inside Transformer
Figure 3 for Self-Attention Attribution: Interpreting Information Interactions Inside Transformer
Figure 4 for Self-Attention Attribution: Interpreting Information Interactions Inside Transformer
* 11 pages 
  Access Paper or Ask Questions

At Which Level Should We Extract? An Empirical Study on Extractive Document Summarization


Apr 06, 2020
Qingyu Zhou, Furu Wei, Ming Zhou

Figure 1 for At Which Level Should We Extract? An Empirical Study on Extractive Document Summarization
Figure 2 for At Which Level Should We Extract? An Empirical Study on Extractive Document Summarization
Figure 3 for At Which Level Should We Extract? An Empirical Study on Extractive Document Summarization
Figure 4 for At Which Level Should We Extract? An Empirical Study on Extractive Document Summarization

  Access Paper or Ask Questions

Learning to Summarize Passages: Mining Passage-Summary Pairs from Wikipedia Revision Histories


Apr 06, 2020
Qingyu Zhou, Furu Wei, Ming Zhou

Figure 1 for Learning to Summarize Passages: Mining Passage-Summary Pairs from Wikipedia Revision Histories
Figure 2 for Learning to Summarize Passages: Mining Passage-Summary Pairs from Wikipedia Revision Histories
Figure 3 for Learning to Summarize Passages: Mining Passage-Summary Pairs from Wikipedia Revision Histories
Figure 4 for Learning to Summarize Passages: Mining Passage-Summary Pairs from Wikipedia Revision Histories

  Access Paper or Ask Questions

STEP: Sequence-to-Sequence Transformer Pre-training for Document Summarization


Apr 04, 2020
Yanyan Zou, Xingxing Zhang, Wei Lu, Furu Wei, Ming Zhou

Figure 1 for STEP: Sequence-to-Sequence Transformer Pre-training for Document Summarization
Figure 2 for STEP: Sequence-to-Sequence Transformer Pre-training for Document Summarization
Figure 3 for STEP: Sequence-to-Sequence Transformer Pre-training for Document Summarization
Figure 4 for STEP: Sequence-to-Sequence Transformer Pre-training for Document Summarization

  Access Paper or Ask Questions

UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training


Feb 28, 2020
Hangbo Bao, Li Dong, Furu Wei, Wenhui Wang, Nan Yang, Xiaodong Liu, Yu Wang, Songhao Piao, Jianfeng Gao, Ming Zhou, Hsiao-Wuen Hon

Figure 1 for UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training
Figure 2 for UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training
Figure 3 for UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training
Figure 4 for UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training
* 11 pages 
  Access Paper or Ask Questions

MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers


Feb 25, 2020
Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, Ming Zhou

Figure 1 for MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
Figure 2 for MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
Figure 3 for MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
Figure 4 for MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
* Code and models: https://github.com/microsoft/unilm/tree/master/minilm 
  Access Paper or Ask Questions

LayoutLM: Pre-training of Text and Layout for Document Image Understanding


Feb 19, 2020
Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou

Figure 1 for LayoutLM: Pre-training of Text and Layout for Document Image Understanding
Figure 2 for LayoutLM: Pre-training of Text and Layout for Document Image Understanding
Figure 3 for LayoutLM: Pre-training of Text and Layout for Document Image Understanding
Figure 4 for LayoutLM: Pre-training of Text and Layout for Document Image Understanding
* Work in progress 
  Access Paper or Ask Questions

Self-Adversarial Learning with Comparative Discrimination for Text Generation


Feb 12, 2020
Wangchunshu Zhou, Tao Ge, Ke Xu, Furu Wei, Ming Zhou

Figure 1 for Self-Adversarial Learning with Comparative Discrimination for Text Generation
Figure 2 for Self-Adversarial Learning with Comparative Discrimination for Text Generation
Figure 3 for Self-Adversarial Learning with Comparative Discrimination for Text Generation
Figure 4 for Self-Adversarial Learning with Comparative Discrimination for Text Generation
* ICLR 2020 
  Access Paper or Ask Questions

BERT-of-Theseus: Compressing BERT by Progressive Module Replacing


Feb 10, 2020
Canwen Xu, Wangchunshu Zhou, Tao Ge, Furu Wei, Ming Zhou

Figure 1 for BERT-of-Theseus: Compressing BERT by Progressive Module Replacing
Figure 2 for BERT-of-Theseus: Compressing BERT by Progressive Module Replacing
Figure 3 for BERT-of-Theseus: Compressing BERT by Progressive Module Replacing
Figure 4 for BERT-of-Theseus: Compressing BERT by Progressive Module Replacing
* 11 pages; typo fixed 
  Access Paper or Ask Questions

Multimodal Matching Transformer for Live Commenting


Feb 07, 2020
Chaoqun Duan, Lei Cui, Shuming Ma, Furu Wei, Conghui Zhu, Tiejun Zhao

Figure 1 for Multimodal Matching Transformer for Live Commenting
Figure 2 for Multimodal Matching Transformer for Live Commenting
Figure 3 for Multimodal Matching Transformer for Live Commenting
Figure 4 for Multimodal Matching Transformer for Live Commenting

  Access Paper or Ask Questions