Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Pengcheng He

Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization


Jun 08, 2021
Chen Liang, Simiao Zuo, Minshuo Chen, Haoming Jiang, Xiaodong Liu, Pengcheng He, Tuo Zhao, Weizhu Chen

* The 59th annual meeting of the Association for Computational Linguistics (ACL 2021) 

  Access Paper or Ask Questions

Adversarial Training as Stackelberg Game: An Unrolled Optimization Approach


Apr 11, 2021
Simiao Zuo, Chen Liang, Haoming Jiang, Xiaodong Liu, Pengcheng He, Jianfeng Gao, Weizhu Chen, Tuo Zhao


  Access Paper or Ask Questions

Token-wise Curriculum Learning for Neural Machine Translation


Mar 20, 2021
Chen Liang, Haoming Jiang, Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao, Tuo Zhao


  Access Paper or Ask Questions

Greedy Multi-step Off-Policy Reinforcement Learning


Mar 07, 2021
Yuhui Wang, Pengcheng He, Xiaoyang Tan


  Access Paper or Ask Questions

Reader-Guided Passage Reranking for Open-Domain Question Answering


Jan 01, 2021
Yuning Mao, Pengcheng He, Xiaodong Liu, Yelong Shen, Jianfeng Gao, Jiawei Han, Weizhu Chen

* TLDR: Reranking retrieved passages by reader predictions can achieve 10~20 gains in top-1 retrieval accuracy and 1~4 gains in Exact Match (EM) without any training 

  Access Paper or Ask Questions

UnitedQA: A Hybrid Approach for Open Domain Question Answering


Jan 01, 2021
Hao Cheng, Yelong Shen, Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao


  Access Paper or Ask Questions

NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned


Jan 01, 2021
Sewon Min, Jordan Boyd-Graber, Chris Alberti, Danqi Chen, Eunsol Choi, Michael Collins, Kelvin Guu, Hannaneh Hajishirzi, Kenton Lee, Jennimaria Palomaki, Colin Raffel, Adam Roberts, Tom Kwiatkowski, Patrick Lewis, Yuxiang Wu, Heinrich Küttler, Linqing Liu, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel, Sohee Yang, Minjoon Seo, Gautier Izacard, Fabio Petroni, Lucas Hosseini, Nicola De Cao, Edouard Grave, Ikuya Yamada, Sonse Shimaoka, Masatoshi Suzuki, Shumpei Miyawaki, Shun Sato, Ryo Takahashi, Jun Suzuki, Martin Fajcik, Martin Docekal, Karel Ondrej, Pavel Smrz, Hao Cheng, Yelong Shen, Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao, Barlas Oguz, Xilun Chen, Vladimir Karpukhin, Stan Peshterliev, Dmytro Okhonko, Michael Schlichtkrull, Sonal Gupta, Yashar Mehdad, Wen-tau Yih

* 26 pages 

  Access Paper or Ask Questions

Generation-Augmented Retrieval for Open-domain Question Answering


Sep 17, 2020
Yuning Mao, Pengcheng He, Xiaodong Liu, Yelong Shen, Jianfeng Gao, Jiawei Han, Weizhu Chen


  Access Paper or Ask Questions

DeBERTa: Decoding-enhanced BERT with Disentangled Attention


Jun 05, 2020
Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen

* 17 pages,4 figures, 8 tables 

  Access Paper or Ask Questions

Adversarial Training for Large Neural Language Models


Apr 29, 2020
Xiaodong Liu, Hao Cheng, Pengcheng He, Weizhu Chen, Yu Wang, Hoifung Poon, Jianfeng Gao

* 13 pages, 9 tables, 2 figures 

  Access Paper or Ask Questions

Exploiting Structured Knowledge in Text via Graph-Guided Representation Learning


Apr 29, 2020
Tao Shen, Yi Mao, Pengcheng He, Guodong Long, Adam Trischler, Weizhu Chen


  Access Paper or Ask Questions

The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural Language Understanding


Feb 19, 2020
Xiaodong Liu, Yu Wang, Jianshu Ji, Hao Cheng, Xueyun Zhu, Emmanuel Awa, Pengcheng He, Weizhu Chen, Hoifung Poon, Guihong Cao, Jianfeng Gao

* 9 pages, 3 figures and 3 tables 

  Access Paper or Ask Questions

SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization


Nov 08, 2019
Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, Tuo Zhao


  Access Paper or Ask Questions

X-SQL: reinforce schema representation with context


Aug 21, 2019
Pengcheng He, Yi Mao, Kaushik Chakrabarti, Weizhu Chen


  Access Paper or Ask Questions

On the Variance of the Adaptive Learning Rate and Beyond


Aug 08, 2019
Liyuan Liu, Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, Jiawei Han


  Access Paper or Ask Questions

A Hybrid Neural Network Model for Commonsense Reasoning


Jul 27, 2019
Pengcheng He, Xiaodong Liu, Weizhu Chen, Jianfeng Gao

* 9 pages, 3 figures, 6 tables 

  Access Paper or Ask Questions

Improving Multi-Task Deep Neural Networks via Knowledge Distillation for Natural Language Understanding


Apr 20, 2019
Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao

* 8 pages, 2 figures and 3 tables 

  Access Paper or Ask Questions

Multi-Task Deep Neural Networks for Natural Language Understanding


Jan 31, 2019
Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao

* 10 pages, 2 figures and 5 tables 

  Access Paper or Ask Questions