Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Weizhu Chen

LoRA: Low-Rank Adaptation of Large Language Models


Jun 17, 2021
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Weizhu Chen


  Access Paper or Ask Questions

Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization


Jun 08, 2021
Chen Liang, Simiao Zuo, Minshuo Chen, Haoming Jiang, Xiaodong Liu, Pengcheng He, Tuo Zhao, Weizhu Chen

* The 59th annual meeting of the Association for Computational Linguistics (ACL 2021) 

  Access Paper or Ask Questions

HiddenCut: Simple Data Augmentation for Natural Language Understanding with Better Generalization


May 31, 2021
Jiaao Chen, Dinghan Shen, Weizhu Chen, Diyi Yang

* ACL 2021 

  Access Paper or Ask Questions

Memory-Efficient Differentiable Transformer Architecture Search


May 31, 2021
Yuekai Zhao, Li Dong, Yelong Shen, Zhihua Zhang, Furu Wei, Weizhu Chen

* Accepted by Findings of ACL 2021 

  Access Paper or Ask Questions

Poolingformer: Long Document Modeling with Pooling Attention


May 10, 2021
Hang Zhang, Yeyun Gong, Yelong Shen, Weisheng Li, Jiancheng Lv, Nan Duan, Weizhu Chen

* Accepted by ICML 2021 

  Access Paper or Ask Questions

A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation


Apr 18, 2021
Tianyu Liu, Yizhe Zhang, Chris Brockett, Yi Mao, Zhifang Sui, Weizhu Chen, Bill Dolan


  Access Paper or Ask Questions

Adversarial Training as Stackelberg Game: An Unrolled Optimization Approach


Apr 11, 2021
Simiao Zuo, Chen Liang, Haoming Jiang, Xiaodong Liu, Pengcheng He, Jianfeng Gao, Weizhu Chen, Tuo Zhao


  Access Paper or Ask Questions

Finetuning Pretrained Transformers into RNNs


Mar 24, 2021
Jungo Kasai, Hao Peng, Yizhe Zhang, Dani Yogatama, Gabriel Ilharco, Nikolaos Pappas, Yi Mao, Weizhu Chen, Noah A. Smith


  Access Paper or Ask Questions

Token-wise Curriculum Learning for Neural Machine Translation


Mar 20, 2021
Chen Liang, Haoming Jiang, Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao, Tuo Zhao


  Access Paper or Ask Questions

What Makes Good In-Context Examples for GPT-$3$?


Jan 17, 2021
Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, Weizhu Chen


  Access Paper or Ask Questions

Reader-Guided Passage Reranking for Open-Domain Question Answering


Jan 01, 2021
Yuning Mao, Pengcheng He, Xiaodong Liu, Yelong Shen, Jianfeng Gao, Jiawei Han, Weizhu Chen

* TLDR: Reranking retrieved passages by reader predictions can achieve 10~20 gains in top-1 retrieval accuracy and 1~4 gains in Exact Match (EM) without any training 

  Access Paper or Ask Questions

UnitedQA: A Hybrid Approach for Open Domain Question Answering


Jan 01, 2021
Hao Cheng, Yelong Shen, Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao


  Access Paper or Ask Questions

NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned


Jan 01, 2021
Sewon Min, Jordan Boyd-Graber, Chris Alberti, Danqi Chen, Eunsol Choi, Michael Collins, Kelvin Guu, Hannaneh Hajishirzi, Kenton Lee, Jennimaria Palomaki, Colin Raffel, Adam Roberts, Tom Kwiatkowski, Patrick Lewis, Yuxiang Wu, Heinrich Küttler, Linqing Liu, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel, Sohee Yang, Minjoon Seo, Gautier Izacard, Fabio Petroni, Lucas Hosseini, Nicola De Cao, Edouard Grave, Ikuya Yamada, Sonse Shimaoka, Masatoshi Suzuki, Shumpei Miyawaki, Shun Sato, Ryo Takahashi, Jun Suzuki, Martin Fajcik, Martin Docekal, Karel Ondrej, Pavel Smrz, Hao Cheng, Yelong Shen, Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao, Barlas Oguz, Xilun Chen, Vladimir Karpukhin, Stan Peshterliev, Dmytro Okhonko, Michael Schlichtkrull, Sonal Gupta, Yashar Mehdad, Wen-tau Yih

* 26 pages 

  Access Paper or Ask Questions

BANG: Bridging Autoregressive and Non-autoregressive Generation with Large Scale Pretraining


Dec 31, 2020
Weizhen Qi, Yeyun Gong, Jian Jiao, Yu Yan, Dayiheng Liu, Weizhu Chen, Kewen Tang, Houqiang Li, Jiusheng Chen, Ruofei Zhang, Ming Zhou, Nan Duan


  Access Paper or Ask Questions

Few-Shot Named Entity Recognition: A Comprehensive Study


Dec 29, 2020
Jiaxin Huang, Chunyuan Li, Krishan Subudhi, Damien Jose, Shobana Balakrishnan, Weizhu Chen, Baolin Peng, Jianfeng Gao, Jiawei Han


  Access Paper or Ask Questions

GLGE: A New General Language Generation Evaluation Benchmark


Nov 24, 2020
Dayiheng Liu, Yu Yan, Yeyun Gong, Weizhen Qi, Hang Zhang, Jian Jiao, Weizhu Chen, Jie Fu, Linjun Shou, Ming Gong, Pengcheng Wang, Jiusheng Chen, Daxin Jiang, Jiancheng Lv, Ruofei Zhang, Winnie Wu, Ming Zhou, Nan Duan

* 11 pages 

  Access Paper or Ask Questions

MixKD: Towards Efficient Distillation of Large-scale Language Models


Nov 01, 2020
Kevin J Liang, Weituo Hao, Dinghan Shen, Yufan Zhou, Weizhu Chen, Changyou Chen, Lawrence Carin


  Access Paper or Ask Questions

A Simple but Tough-to-Beat Data Augmentation Approach for Natural Language Understanding and Generation


Oct 23, 2020
Dinghan Shen, Mingzhi Zheng, Yelong Shen, Yanru Qu, Weizhu Chen

* Source code is available at: https://github.com/dinghanshen/cutoff 

  Access Paper or Ask Questions

CoDA: Contrast-enhanced and Diversity-promoting Data Augmentation for Natural Language Understanding


Oct 16, 2020
Yanru Qu, Dinghan Shen, Yelong Shen, Sandra Sajeev, Jiawei Han, Weizhu Chen


  Access Paper or Ask Questions

Improving Self-supervised Pre-training via a Fully-Explored Masked Language Model


Oct 14, 2020
Mingzhi Zheng, Dinghan Shen, Yelong Shen, Weizhu Chen, Lin Xiao


  Access Paper or Ask Questions

Generation-Augmented Retrieval for Open-domain Question Answering


Sep 17, 2020
Yuning Mao, Pengcheng He, Xiaodong Liu, Yelong Shen, Jianfeng Gao, Jiawei Han, Weizhu Chen


  Access Paper or Ask Questions

Example-Based Named Entity Recognition


Aug 24, 2020
Morteza Ziyadi, Yuting Sun, Abhishek Goswami, Jade Huang, Weizhu Chen

* 15 pages, 6 figures, 5 tables with appendix 

  Access Paper or Ask Questions

DeBERTa: Decoding-enhanced BERT with Disentangled Attention


Jun 05, 2020
Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen

* 17 pages,4 figures, 8 tables 

  Access Paper or Ask Questions

Adversarial Training for Large Neural Language Models


Apr 29, 2020
Xiaodong Liu, Hao Cheng, Pengcheng He, Weizhu Chen, Yu Wang, Hoifung Poon, Jianfeng Gao

* 13 pages, 9 tables, 2 figures 

  Access Paper or Ask Questions

Exploiting Structured Knowledge in Text via Graph-Guided Representation Learning


Apr 29, 2020
Tao Shen, Yi Mao, Pengcheng He, Guodong Long, Adam Trischler, Weizhu Chen


  Access Paper or Ask Questions

Understanding the Difficulty of Training Transformers


Apr 17, 2020
Liyuan Liu, Xiaodong Liu, Jianfeng Gao, Weizhu Chen, Jiawei Han


  Access Paper or Ask Questions