Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Wenhui Wang

Adapt-and-Distill: Developing Small, Fast and Effective Pretrained Language Models for Domains


Jun 29, 2021
Yunzhi Yao, Shaohan Huang, Wenhui Wang, Li Dong, Furu Wei

* accepted as ACL2021 Findings 

  Access Paper or Ask Questions

Consistency Regularization for Cross-Lingual Fine-Tuning


Jun 15, 2021
Bo Zheng, Li Dong, Shaohan Huang, Wenhui Wang, Zewen Chi, Saksham Singhal, Wanxiang Che, Ting Liu, Xia Song, Furu Wei

* ACL-2021 

  Access Paper or Ask Questions

MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers


Dec 31, 2020
Wenhui Wang, Hangbo Bao, Shaohan Huang, Li Dong, Furu Wei


  Access Paper or Ask Questions

InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training


Jul 15, 2020
Zewen Chi, Li Dong, Furu Wei, Nan Yang, Saksham Singhal, Wenhui Wang, Xia Song, Xian-Ling Mao, Heyan Huang, Ming Zhou

* 11 pages 

  Access Paper or Ask Questions

Harvesting and Refining Question-Answer Pairs for Unsupervised QA


May 06, 2020
Zhongli Li, Wenhui Wang, Li Dong, Furu Wei, Ke Xu

* Accepted by ACL-20 

  Access Paper or Ask Questions

Comparing SNNs and RNNs on Neuromorphic Vision Datasets: Similarities and Differences


May 02, 2020
Weihua He, YuJie Wu, Lei Deng, Guoqi Li, Haoyu Wang, Yang Tian, Wei Ding, Wenhui Wang, Yuan Xie


  Access Paper or Ask Questions

UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training


Feb 28, 2020
Hangbo Bao, Li Dong, Furu Wei, Wenhui Wang, Nan Yang, Xiaodong Liu, Yu Wang, Songhao Piao, Jianfeng Gao, Ming Zhou, Hsiao-Wuen Hon

* 11 pages 

  Access Paper or Ask Questions

MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers


Feb 25, 2020
Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, Ming Zhou

* Code and models: https://github.com/microsoft/unilm/tree/master/minilm 

  Access Paper or Ask Questions

Cross-Lingual Natural Language Generation via Pre-Training


Sep 24, 2019
Zewen Chi, Li Dong, Furu Wei, Wenhui Wang, Xian-Ling Mao, Heyan Huang

* 10 pages 

  Access Paper or Ask Questions

Learning to Ask Unanswerable Questions for Machine Reading Comprehension


Jun 14, 2019
Haichao Zhu, Li Dong, Furu Wei, Wenhui Wang, Bing Qin, Ting Liu

* ACL 2019 (long paper) 

  Access Paper or Ask Questions

Unified Language Model Pre-training for Natural Language Understanding and Generation


May 08, 2019
Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, Hsiao-Wuen Hon


  Access Paper or Ask Questions