Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Furu Wei

Sequence Level Contrastive Learning for Text Summarization


Sep 08, 2021
Shusheng Xu, Xingxing Zhang, Yi Wu, Furu Wei

* 2 figures, 12 tables 

  Access Paper or Ask Questions

Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compression


Sep 07, 2021
Canwen Xu, Wangchunshu Zhou, Tao Ge, Ke Xu, Julian McAuley, Furu Wei

* Accepted to EMNLP 2021 (main conference) 

  Access Paper or Ask Questions

LayoutReader: Pre-training of Text and Layout for Reading Order Detection


Aug 27, 2021
Zilong Wang, Yiheng Xu, Lei Cui, Jingbo Shang, Furu Wei

* EMNLP 2021 

  Access Paper or Ask Questions

UniSpeech at scale: An Empirical Study of Pre-training Method on Large-Scale Speech Recognition Dataset


Jul 12, 2021
Chengyi Wang, Yu Wu, Shujie Liu, Jinyu Li, Yao Qian, Kenichi Kumatani, Furu Wei


  Access Paper or Ask Questions

XLM-E: Cross-lingual Language Model Pre-training via ELECTRA


Jun 30, 2021
Zewen Chi, Shaohan Huang, Li Dong, Shuming Ma, Saksham Singhal, Payal Bajaj, Xia Song, Furu Wei


  Access Paper or Ask Questions

Adapt-and-Distill: Developing Small, Fast and Effective Pretrained Language Models for Domains


Jun 29, 2021
Yunzhi Yao, Shaohan Huang, Wenhui Wang, Li Dong, Furu Wei

* accepted as ACL2021 Findings 

  Access Paper or Ask Questions

DeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoders


Jun 25, 2021
Shuming Ma, Li Dong, Shaohan Huang, Dongdong Zhang, Alexandre Muzio, Saksham Singhal, Hany Hassan Awadalla, Xia Song, Furu Wei

* Work in progress 

  Access Paper or Ask Questions

Learning to Sample Replacements for ELECTRA Pre-Training


Jun 25, 2021
Yaru Hao, Li Dong, Hangbo Bao, Ke Xu, Furu Wei

* Accepted by Findings of ACL 2021 

  Access Paper or Ask Questions

BEiT: BERT Pre-Training of Image Transformers


Jun 15, 2021
Hangbo Bao, Li Dong, Furu Wei

* A Path to the BERT Moment of CV. Work in progress 

  Access Paper or Ask Questions

Consistency Regularization for Cross-Lingual Fine-Tuning


Jun 15, 2021
Bo Zheng, Li Dong, Shaohan Huang, Wenhui Wang, Zewen Chi, Saksham Singhal, Wanxiang Che, Ting Liu, Xia Song, Furu Wei

* ACL-2021 

  Access Paper or Ask Questions

Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment


Jun 11, 2021
Zewen Chi, Li Dong, Bo Zheng, Shaohan Huang, Xian-Ling Mao, Heyan Huang, Furu Wei

* ACL-2021 

  Access Paper or Ask Questions

VT-SSum: A Benchmark Dataset for Video Transcript Segmentation and Summarization


Jun 10, 2021
Tengchao Lv, Lei Cui, Momcilo Vasilijevic, Furu Wei

* Work in progress 

  Access Paper or Ask Questions

Instantaneous Grammatical Error Correction with Shallow Aggressive Decoding


Jun 09, 2021
Xin Sun, Tao Ge, Furu Wei, Houfeng Wang

* Accepted by ACL2021 (main conference) 

  Access Paper or Ask Questions

Attention Temperature Matters in Abstractive Summarization Distillation


Jun 08, 2021
Shengqiang Zhang, Xingxing Zhang, Hangbo Bao, Furu Wei


  Access Paper or Ask Questions

Memory-Efficient Differentiable Transformer Architecture Search


May 31, 2021
Yuekai Zhao, Li Dong, Yelong Shen, Zhihua Zhang, Furu Wei, Weizhu Chen

* Accepted by Findings of ACL 2021 

  Access Paper or Ask Questions

LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understanding


Apr 18, 2021
Yiheng Xu, Tengchao Lv, Lei Cui, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Furu Wei

* Work in progress 

  Access Paper or Ask Questions

Zero-shot Cross-lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoders


Apr 18, 2021
Guanhua Chen, Shuming Ma, Yun Chen, Li Dong, Dongdong Zhang, Jia Pan, Wenping Wang, Furu Wei

* Preprint 

  Access Paper or Ask Questions

Knowledge Neurons in Pretrained Transformers


Apr 18, 2021
Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Furu Wei


  Access Paper or Ask Questions

mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs


Apr 18, 2021
Zewen Chi, Li Dong, Shuming Ma, Shaohan Huang Xian-Ling Mao, Heyan Huang, Furu Wei


  Access Paper or Ask Questions

Blow the Dog Whistle: A Chinese Dataset for Cant Understanding with Common Sense and World Knowledge


Apr 06, 2021
Canwen Xu, Wangchunshu Zhou, Tao Ge, Ke Xu, Julian McAuley, Furu Wei

* NAACL 2021 

  Access Paper or Ask Questions

UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data


Jan 19, 2021
Chengyi Wang, Yu Wu, Yao Qian, Kenichi Kumatani, Shujie Liu, Furu Wei, Michael Zeng, Xuedong Huang


  Access Paper or Ask Questions

Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting


Jan 02, 2021
Wangchunshu Zhou, Tao Ge, Ke Xu, Furu Wei


  Access Paper or Ask Questions

MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers


Dec 31, 2020
Wenhui Wang, Hangbo Bao, Shaohan Huang, Li Dong, Furu Wei


  Access Paper or Ask Questions

XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders


Dec 31, 2020
Shuming Ma, Jian Yang, Haoyang Huang, Zewen Chi, Li Dong, Dongdong Zhang, Hany Hassan Awadalla, Alexandre Muzio, Akiko Eriguchi, Saksham Singhal, Xia Song, Arul Menezes, Furu Wei


  Access Paper or Ask Questions

LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding


Dec 29, 2020
Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, Min Zhang, Lidong Zhou

* Work in progress 

  Access Paper or Ask Questions

Unsupervised Extractive Summarization by Pre-training Hierarchical Transformers


Oct 16, 2020
Shusheng Xu, Xingxing Zhang, Yi Wu, Furu Wei, Ming Zhou

* 9 pages, 4 figures 

  Access Paper or Ask Questions

Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction


Oct 07, 2020
Mengyun Chen, Tao Ge, Xingxing Zhang, Furu Wei, Ming Zhou

* Accepted by EMNLP 2020 

  Access Paper or Ask Questions