Alert button
Picture for Pengcheng He

Pengcheng He

Alert button

HyperTuning: Toward Adapting Large Language Models without Back-propagation

Nov 22, 2022
Jason Phang, Yi Mao, Pengcheng He, Weizhu Chen

Figure 1 for HyperTuning: Toward Adapting Large Language Models without Back-propagation
Figure 2 for HyperTuning: Toward Adapting Large Language Models without Back-propagation
Figure 3 for HyperTuning: Toward Adapting Large Language Models without Back-propagation
Figure 4 for HyperTuning: Toward Adapting Large Language Models without Back-propagation
Viaarxiv icon

Less is More: Task-aware Layer-wise Distillation for Language Model Compression

Oct 05, 2022
Chen Liang, Simiao Zuo, Qingru Zhang, Pengcheng He, Weizhu Chen, Tuo Zhao

Figure 1 for Less is More: Task-aware Layer-wise Distillation for Language Model Compression
Figure 2 for Less is More: Task-aware Layer-wise Distillation for Language Model Compression
Figure 3 for Less is More: Task-aware Layer-wise Distillation for Language Model Compression
Figure 4 for Less is More: Task-aware Layer-wise Distillation for Language Model Compression
Viaarxiv icon

Z-Code++: A Pre-trained Language Model Optimized for Abstractive Summarization

Aug 21, 2022
Pengcheng He, Baolin Peng, Liyang Lu, Song Wang, Jie Mei, Yang Liu, Ruochen Xu, Hany Hassan Awadalla, Yu Shi, Chenguang Zhu, Wayne Xiong, Michael Zeng, Jianfeng Gao, Xuedong Huang

Figure 1 for Z-Code++: A Pre-trained Language Model Optimized for Abstractive Summarization
Figure 2 for Z-Code++: A Pre-trained Language Model Optimized for Abstractive Summarization
Figure 3 for Z-Code++: A Pre-trained Language Model Optimized for Abstractive Summarization
Figure 4 for Z-Code++: A Pre-trained Language Model Optimized for Abstractive Summarization
Viaarxiv icon

OmniTab: Pretraining with Natural and Synthetic Data for Few-shot Table-based Question Answering

Jul 08, 2022
Zhengbao Jiang, Yi Mao, Pengcheng He, Graham Neubig, Weizhu Chen

Figure 1 for OmniTab: Pretraining with Natural and Synthetic Data for Few-shot Table-based Question Answering
Figure 2 for OmniTab: Pretraining with Natural and Synthetic Data for Few-shot Table-based Question Answering
Figure 3 for OmniTab: Pretraining with Natural and Synthetic Data for Few-shot Table-based Question Answering
Figure 4 for OmniTab: Pretraining with Natural and Synthetic Data for Few-shot Table-based Question Answering
Viaarxiv icon

PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance

Jun 25, 2022
Qingru Zhang, Simiao Zuo, Chen Liang, Alexander Bukharin, Pengcheng He, Weizhu Chen, Tuo Zhao

Figure 1 for PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance
Figure 2 for PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance
Figure 3 for PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance
Figure 4 for PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance
Viaarxiv icon

GODEL: Large-Scale Pre-Training for Goal-Directed Dialog

Jun 22, 2022
Baolin Peng, Michel Galley, Pengcheng He, Chris Brockett, Lars Liden, Elnaz Nouri, Zhou Yu, Bill Dolan, Jianfeng Gao

Figure 1 for GODEL: Large-Scale Pre-Training for Goal-Directed Dialog
Figure 2 for GODEL: Large-Scale Pre-Training for Goal-Directed Dialog
Figure 3 for GODEL: Large-Scale Pre-Training for Goal-Directed Dialog
Figure 4 for GODEL: Large-Scale Pre-Training for Goal-Directed Dialog
Viaarxiv icon

Diffusion-GAN: Training GANs with Diffusion

Jun 05, 2022
Zhendong Wang, Huangjie Zheng, Pengcheng He, Weizhu Chen, Mingyuan Zhou

Figure 1 for Diffusion-GAN: Training GANs with Diffusion
Figure 2 for Diffusion-GAN: Training GANs with Diffusion
Figure 3 for Diffusion-GAN: Training GANs with Diffusion
Figure 4 for Diffusion-GAN: Training GANs with Diffusion
Viaarxiv icon

ALLSH: Active Learning Guided by Local Sensitivity and Hardness

May 10, 2022
Shujian Zhang, Chengyue Gong, Xingchao Liu, Pengcheng He, Weizhu Chen, Mingyuan Zhou

Figure 1 for ALLSH: Active Learning Guided by Local Sensitivity and Hardness
Figure 2 for ALLSH: Active Learning Guided by Local Sensitivity and Hardness
Figure 3 for ALLSH: Active Learning Guided by Local Sensitivity and Hardness
Figure 4 for ALLSH: Active Learning Guided by Local Sensitivity and Hardness
Viaarxiv icon

MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation

Apr 28, 2022
Simiao Zuo, Qingru Zhang, Chen Liang, Pengcheng He, Tuo Zhao, Weizhu Chen

Figure 1 for MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation
Figure 2 for MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation
Figure 3 for MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation
Figure 4 for MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation
Viaarxiv icon

CAMERO: Consistency Regularized Ensemble of Perturbed Language Models with Weight Sharing

Apr 18, 2022
Chen Liang, Pengcheng He, Yelong Shen, Weizhu Chen, Tuo Zhao

Figure 1 for CAMERO: Consistency Regularized Ensemble of Perturbed Language Models with Weight Sharing
Figure 2 for CAMERO: Consistency Regularized Ensemble of Perturbed Language Models with Weight Sharing
Figure 3 for CAMERO: Consistency Regularized Ensemble of Perturbed Language Models with Weight Sharing
Figure 4 for CAMERO: Consistency Regularized Ensemble of Perturbed Language Models with Weight Sharing
Viaarxiv icon