Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Wenhui Wang

Adapt-and-Distill: Developing Small, Fast and Effective Pretrained Language Models for Domains

Jun 29, 2021
Yunzhi Yao, Shaohan Huang, Wenhui Wang, Li Dong, Furu Wei

* accepted as ACL2021 Findings 

  Access Paper or Ask Questions

Consistency Regularization for Cross-Lingual Fine-Tuning

Jun 15, 2021
Bo Zheng, Li Dong, Shaohan Huang, Wenhui Wang, Zewen Chi, Saksham Singhal, Wanxiang Che, Ting Liu, Xia Song, Furu Wei

* ACL-2021 

  Access Paper or Ask Questions

MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers

Dec 31, 2020
Wenhui Wang, Hangbo Bao, Shaohan Huang, Li Dong, Furu Wei

  Access Paper or Ask Questions

InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training

Jul 15, 2020
Zewen Chi, Li Dong, Furu Wei, Nan Yang, Saksham Singhal, Wenhui Wang, Xia Song, Xian-Ling Mao, Heyan Huang, Ming Zhou

* 11 pages 

  Access Paper or Ask Questions

Harvesting and Refining Question-Answer Pairs for Unsupervised QA

May 06, 2020
Zhongli Li, Wenhui Wang, Li Dong, Furu Wei, Ke Xu

* Accepted by ACL-20 

  Access Paper or Ask Questions

Comparing SNNs and RNNs on Neuromorphic Vision Datasets: Similarities and Differences

May 02, 2020
Weihua He, YuJie Wu, Lei Deng, Guoqi Li, Haoyu Wang, Yang Tian, Wei Ding, Wenhui Wang, Yuan Xie

  Access Paper or Ask Questions

UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training

Feb 28, 2020
Hangbo Bao, Li Dong, Furu Wei, Wenhui Wang, Nan Yang, Xiaodong Liu, Yu Wang, Songhao Piao, Jianfeng Gao, Ming Zhou, Hsiao-Wuen Hon

* 11 pages 

  Access Paper or Ask Questions

MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers

Feb 25, 2020
Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, Ming Zhou

* Code and models: 

  Access Paper or Ask Questions

Cross-Lingual Natural Language Generation via Pre-Training

Sep 24, 2019
Zewen Chi, Li Dong, Furu Wei, Wenhui Wang, Xian-Ling Mao, Heyan Huang

* 10 pages 

  Access Paper or Ask Questions

Learning to Ask Unanswerable Questions for Machine Reading Comprehension

Jun 14, 2019
Haichao Zhu, Li Dong, Furu Wei, Wenhui Wang, Bing Qin, Ting Liu

* ACL 2019 (long paper) 

  Access Paper or Ask Questions

Unified Language Model Pre-training for Natural Language Understanding and Generation

May 08, 2019
Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, Hsiao-Wuen Hon

  Access Paper or Ask Questions