Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Furu Wei

UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data


Jan 19, 2021
Chengyi Wang, Yu Wu, Yao Qian, Kenichi Kumatani, Shujie Liu, Furu Wei, Michael Zeng, Xuedong Huang


  Access Paper or Ask Questions

Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting


Jan 02, 2021
Wangchunshu Zhou, Tao Ge, Ke Xu, Furu Wei


  Access Paper or Ask Questions

MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers


Dec 31, 2020
Wenhui Wang, Hangbo Bao, Shaohan Huang, Li Dong, Furu Wei


  Access Paper or Ask Questions

XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders


Dec 31, 2020
Shuming Ma, Jian Yang, Haoyang Huang, Zewen Chi, Li Dong, Dongdong Zhang, Hany Hassan Awadalla, Alexandre Muzio, Akiko Eriguchi, Saksham Singhal, Xia Song, Arul Menezes, Furu Wei


  Access Paper or Ask Questions

LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding


Dec 29, 2020
Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, Min Zhang, Lidong Zhou

* Work in progress 

  Access Paper or Ask Questions

Unsupervised Extractive Summarization by Pre-training Hierarchical Transformers


Oct 16, 2020
Shusheng Xu, Xingxing Zhang, Yi Wu, Furu Wei, Ming Zhou

* 9 pages, 4 figures 

  Access Paper or Ask Questions

Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction


Oct 07, 2020
Mengyun Chen, Tao Ge, Xingxing Zhang, Furu Wei, Ming Zhou

* Accepted by EMNLP 2020 

  Access Paper or Ask Questions

Generating Commonsense Explanation by Extracting Bridge Concepts from Reasoning Paths


Sep 24, 2020
Haozhe Ji, Pei Ke, Shaohan Huang, Furu Wei, Minlie Huang

* Accepted by AACL-IJCNLP 2020 

  Access Paper or Ask Questions

Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph


Sep 24, 2020
Haozhe Ji, Pei Ke, Shaohan Huang, Furu Wei, Xiaoyan Zhu, Minlie Huang

* accepted by EMNLP 2020 

  Access Paper or Ask Questions

InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training


Jul 15, 2020
Zewen Chi, Li Dong, Furu Wei, Nan Yang, Saksham Singhal, Wenhui Wang, Xia Song, Xian-Ling Mao, Heyan Huang, Ming Zhou

* 11 pages 

  Access Paper or Ask Questions

BERT Loses Patience: Fast and Robust Inference with Early Exit


Jun 29, 2020
Wangchunshu Zhou, Canwen Xu, Tao Ge, Julian McAuley, Ke Xu, Furu Wei

* 13 pages 

  Access Paper or Ask Questions

DocBank: A Benchmark Dataset for Document Layout Analysis


Jun 01, 2020
Minghao Li, Yiheng Xu, Lei Cui, Shaohan Huang, Furu Wei, Zhoujun Li, Ming Zhou

* Work in progress 

  Access Paper or Ask Questions

Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks


May 18, 2020
Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, Yejin Choi, Jianfeng Gao

* Code and pre-trained models are released: https://github.com/microsoft/Oscar 

  Access Paper or Ask Questions

Harvesting and Refining Question-Answer Pairs for Unsupervised QA


May 06, 2020
Zhongli Li, Wenhui Wang, Li Dong, Furu Wei, Ke Xu

* Accepted by ACL-20 

  Access Paper or Ask Questions

Scheduled DropHead: A Regularization Method for Transformer Models


Apr 28, 2020
Wangchunshu Zhou, Tao Ge, Ke Xu, Furu Wei, Ming Zhou


  Access Paper or Ask Questions

Self-Attention Attribution: Interpreting Information Interactions Inside Transformer


Apr 23, 2020
Yaru Hao, Li Dong, Furu Wei, Ke Xu

* 11 pages 

  Access Paper or Ask Questions

At Which Level Should We Extract? An Empirical Study on Extractive Document Summarization


Apr 06, 2020
Qingyu Zhou, Furu Wei, Ming Zhou


  Access Paper or Ask Questions

Learning to Summarize Passages: Mining Passage-Summary Pairs from Wikipedia Revision Histories


Apr 06, 2020
Qingyu Zhou, Furu Wei, Ming Zhou


  Access Paper or Ask Questions

STEP: Sequence-to-Sequence Transformer Pre-training for Document Summarization


Apr 04, 2020
Yanyan Zou, Xingxing Zhang, Wei Lu, Furu Wei, Ming Zhou


  Access Paper or Ask Questions

UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training


Feb 28, 2020
Hangbo Bao, Li Dong, Furu Wei, Wenhui Wang, Nan Yang, Xiaodong Liu, Yu Wang, Songhao Piao, Jianfeng Gao, Ming Zhou, Hsiao-Wuen Hon

* 11 pages 

  Access Paper or Ask Questions

MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers


Feb 25, 2020
Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, Ming Zhou

* Code and models: https://github.com/microsoft/unilm/tree/master/minilm 

  Access Paper or Ask Questions

LayoutLM: Pre-training of Text and Layout for Document Image Understanding


Feb 19, 2020
Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou

* Work in progress 

  Access Paper or Ask Questions

Self-Adversarial Learning with Comparative Discrimination for Text Generation


Feb 12, 2020
Wangchunshu Zhou, Tao Ge, Ke Xu, Furu Wei, Ming Zhou

* ICLR 2020 

  Access Paper or Ask Questions

BERT-of-Theseus: Compressing BERT by Progressive Module Replacing


Feb 10, 2020
Canwen Xu, Wangchunshu Zhou, Tao Ge, Furu Wei, Ming Zhou

* 11 pages; typo fixed 

  Access Paper or Ask Questions

Multimodal Matching Transformer for Live Commenting


Feb 07, 2020
Chaoqun Duan, Lei Cui, Shuming Ma, Furu Wei, Conghui Zhu, Tiejun Zhao


  Access Paper or Ask Questions