Get our free extension to see links to code for papers anywhere online!

Chrome logo  Add to Chrome

Firefox logo Add to Firefox

Delta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Models



Ning Ding , Yujia Qin , Guang Yang , Fuchao Wei , Zonghan Yang , Yusheng Su , Shengding Hu , Yulin Chen , Chi-Min Chan , Weize Chen , Jing Yi , Weilin Zhao , Xiaozhi Wang , Zhiyuan Liu , Hai-Tao Zheng , Jianfei Chen , Yang Liu , Jie Tang , Juanzi Li , Maosong Sun

* 49 pages 

   Access Paper or Ask Questions

On Transferability of Prompt Tuning for Natural Language Understanding



Yusheng Su , Xiaozhi Wang , Yujia Qin , Chi-Min Chan , Yankai Lin , Zhiyuan Liu , Peng Li , Juanzi Li , Lei Hou , Maosong Sun , Jie Zhou


   Access Paper or Ask Questions

Exploring Low-dimensional Intrinsic Task Subspace via Prompt Tuning



Yujia Qin , Xiaozhi Wang , Yusheng Su , Yankai Lin , Ning Ding , Zhiyuan Liu , Juanzi Li , Lei Hou , Peng Li , Maosong Sun , Jie Zhou


   Access Paper or Ask Questions

Knowledge Inheritance for Pre-trained Language Models



Yujia Qin , Yankai Lin , Jing Yi , Jiajie Zhang , Xu Han , Zhengyan Zhang , Yusheng Su , Zhiyuan Liu , Peng Li , Maosong Sun , Jie Zhou

* preprint 

   Access Paper or Ask Questions

CSS-LM: A Contrastive Framework for Semi-supervised Fine-tuning of Pre-trained Language Models



Yusheng Su , Xu Han , Yankai Lin , Zhengyan Zhang , Zhiyuan Liu , Peng Li , Jie Zhou , Maosong Sun


   Access Paper or Ask Questions

CPM: A Large-scale Generative Chinese Pre-trained Language Model



Zhengyan Zhang , Xu Han , Hao Zhou , Pei Ke , Yuxian Gu , Deming Ye , Yujia Qin , Yusheng Su , Haozhe Ji , Jian Guan , Fanchao Qi , Xiaozhi Wang , Yanan Zheng , Guoyang Zeng , Huanqi Cao , Shengqi Chen , Daixuan Li , Zhenbo Sun , Zhiyuan Liu , Minlie Huang , Wentao Han , Jie Tang , Juanzi Li , Xiaoyan Zhu , Maosong Sun


   Access Paper or Ask Questions

Contextual Knowledge Selection and Embedding towards Enhanced Pre-Trained Language Models



Yusheng Su , Xu Han , Zhengyan Zhang , Peng Li , Zhiyuan Liu , Yankai Lin , Jie Zhou , Maosong Sun


   Access Paper or Ask Questions