Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Hangbo Bao

Learning to Sample Replacements for ELECTRA Pre-Training


Jun 25, 2021
Yaru Hao, Li Dong, Hangbo Bao, Ke Xu, Furu Wei

* Accepted by Findings of ACL 2021 

  Access Paper or Ask Questions

BEiT: BERT Pre-Training of Image Transformers


Jun 15, 2021
Hangbo Bao, Li Dong, Furu Wei

* A Path to the BERT Moment of CV. Work in progress 

  Access Paper or Ask Questions

Attention Temperature Matters in Abstractive Summarization Distillation


Jun 08, 2021
Shengqiang Zhang, Xingxing Zhang, Hangbo Bao, Furu Wei


  Access Paper or Ask Questions

MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers


Dec 31, 2020
Wenhui Wang, Hangbo Bao, Shaohan Huang, Li Dong, Furu Wei


  Access Paper or Ask Questions

UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training


Feb 28, 2020
Hangbo Bao, Li Dong, Furu Wei, Wenhui Wang, Nan Yang, Xiaodong Liu, Yu Wang, Songhao Piao, Jianfeng Gao, Ming Zhou, Hsiao-Wuen Hon

* 11 pages 

  Access Paper or Ask Questions

MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers


Feb 25, 2020
Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, Ming Zhou

* Code and models: https://github.com/microsoft/unilm/tree/master/minilm 

  Access Paper or Ask Questions

Neural Melody Composition from Lyrics


Sep 12, 2018
Hangbo Bao, Shaohan Huang, Furu Wei, Lei Cui, Yu Wu, Chuanqi Tan, Songhao Piao, Ming Zhou


  Access Paper or Ask Questions

Neural Question Generation from Text: A Preliminary Study


Apr 18, 2017
Qingyu Zhou, Nan Yang, Furu Wei, Chuanqi Tan, Hangbo Bao, Ming Zhou

* Submitted to EMNLP 2017 

  Access Paper or Ask Questions