Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Qun Liu

PanGu-$α$: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation


Apr 26, 2021
Wei Zeng, Xiaozhe Ren, Teng Su, Hui Wang, Yi Liao, Zhiwei Wang, Xin Jiang, ZhenZhang Yang, Kaisheng Wang, Xiaoda Zhang, Chen Li, Ziyan Gong, Yifan Yao, Xinjing Huang, Jun Wang, Jianfeng Yu, Qi Guo, Yue Yu, Yan Zhang, Jin Wang, Hengtao Tao, Dasen Yan, Zexuan Yi, Fang Peng, Fangqing Jiang, Han Zhang, Lingfeng Deng, Yehong Zhang, Zhe Lin, Chao Zhang, Shaojie Zhang, Mingyue Guo, Shanzhi Gu, Gaojun Fan, Yaowei Wang, Xuefeng Jin, Qun Liu, Yonghong Tian

* The technique report for PanGu-$\alpha$ 

  Access Paper or Ask Questions

Extract then Distill: Efficient and Effective Task-Agnostic BERT Distillation


Apr 24, 2021
Cheng Chen, Yichun Yin, Lifeng Shang, Zhi Wang, Xin Jiang, Xiao Chen, Qun Liu


  Access Paper or Ask Questions

Improving Neural Machine Translation with Compact Word Embedding Tables


Apr 18, 2021
Krtin Kumar, Mehdi Rezagholizadeh, Yiu Sing Lau, Qun Liu


  Access Paper or Ask Questions

An Approach to Improve Robustness of NLP Systems against ASR Errors


Mar 25, 2021
Tong Cui, Jinghui Xiao, Liangyou Li, Xin Jiang, Qun Liu

* 9 pages, 3 figures 

  Access Paper or Ask Questions

Dependency Graph-to-String Statistical Machine Translation


Mar 20, 2021
Liangyou Li, Andy Way, Qun Liu


  Access Paper or Ask Questions

Reweighting Augmented Samples by Minimizing the Maximal Expected Loss


Mar 16, 2021
Mingyang Yi, Lu Hou, Lifeng Shang, Xin Jiang, Qun Liu, Zhi-Ming Ma

* published in ICLR2021 
* 14 pages 

  Access Paper or Ask Questions

LightMBERT: A Simple Yet Effective Method for Multilingual BERT Distillation


Mar 11, 2021
Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, Qun Liu


  Access Paper or Ask Questions

Training Multilingual Pre-trained Language Model with Byte-level Subwords


Jan 23, 2021
Junqiu Wei, Qun Liu, Yinpeng Guo, Xin Jiang


  Access Paper or Ask Questions

Revisiting Robust Neural Machine Translation: A Transformer Case Study


Dec 31, 2020
Peyman Passban, Puneeth S. M. Saladi, Qun Liu


  Access Paper or Ask Questions

BinaryBERT: Pushing the Limit of BERT Quantization


Dec 31, 2020
Haoli Bai, Wei Zhang, Lu Hou, Lifeng Shang, Jing Jin, Xin Jiang, Qun Liu, Michael Lyu, Irwin King


  Access Paper or Ask Questions

Better Robustness by More Coverage: Adversarial Training with Mixup Augmentation for Robust Fine-tuning


Dec 31, 2020
Chenglei Si, Zhengyan Zhang, Fanchao Qi, Zhiyuan Liu, Yasheng Wang, Qun Liu, Maosong Sun

* 9 pages 

  Access Paper or Ask Questions

HopRetriever: Retrieve Hops over Wikipedia to Answer Complex Questions


Dec 31, 2020
Shaobo Li, Xiaoguang Li, Lifeng Shang, Xin Jiang, Qun Liu, Chengjie Sun, Zhenzhou Ji, Bingquan Liu

* Accepted at AAAI 2021 

  Access Paper or Ask Questions

ALP-KD: Attention-Based Layer Projection for Knowledge Distillation


Dec 27, 2020
Peyman Passban, Yimeng Wu, Mehdi Rezagholizadeh, Qun Liu

* AAAI 2021. This work has been done while Peyman Passban was at Huawei 

  Access Paper or Ask Questions

Improving Task-Agnostic BERT Distillation with Layer Mapping Search


Dec 11, 2020
Xiaoqi Jiao, Huating Chang, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, Qun Liu


  Access Paper or Ask Questions

Document Graph for Neural Machine Translation


Dec 08, 2020
Mingzhou Xu, Liangyou Li, Derek. F. Wong, Qun Liu, Lidia S. Chao


  Access Paper or Ask Questions

PPKE: Knowledge Representation Learning by Path-based Pre-training


Dec 07, 2020
Bin He, Di Zhou, Jing Xie, Jinghui Xiao, Xin Jiang, Qun Liu


  Access Paper or Ask Questions

KgPLM: Knowledge-guided Language Model Pre-training via Generative and Discriminative Learning


Dec 07, 2020
Bin He, Xin Jiang, Jinghui Xiao, Qun Liu

* 10 pages, 3 figures 

  Access Paper or Ask Questions

From Unsupervised Machine Translation To Adversarial Text Generation


Nov 10, 2020
Ahmad Rashid, Alan Do-Omri, Md. Akmal Haidar, Qun Liu, Mehdi Rezagholizadeh

* Accepted at ICASSP 2020 

  Access Paper or Ask Questions

Know What You Don't Need: Single-Shot Meta-Pruning for Attention Heads


Nov 07, 2020
Zhengyan Zhang, Fanchao Qi, Zhiyuan Liu, Qun Liu, Maosong Sun


  Access Paper or Ask Questions

HyperText: Endowing FastText with Hyperbolic Geometry


Oct 30, 2020
Yudong Zhu, Di Zhou, Jinghui Xiao, Xin Jiang, Xiao Chen, Qun Liu

* Findings of EMNLP 2020 

  Access Paper or Ask Questions

LiDAM: Semi-Supervised Learning with Localized Domain Adaptation and Iterative Matching


Oct 13, 2020
Qun Liu, Matthew Shreve, Raja Bala


  Access Paper or Ask Questions

TernaryBERT: Distillation-aware Ultra-low Bit BERT


Oct 10, 2020
Wei Zhang, Lu Hou, Yichun Yin, Lifeng Shang, Xiao Chen, Xin Jiang, Qun Liu

* Accepted by EMNLP 2020 

  Access Paper or Ask Questions

Why Skip If You Can Combine: A Simple Knowledge Distillation Technique for Intermediate Layers


Oct 06, 2020
Yimeng Wu, Peyman Passban, Mehdi Rezagholizade, Qun Liu

* The first two authors contributed equally 

  Access Paper or Ask Questions

TensorCoder: Dimension-Wise Attention via Tensor Representation for Natural Language Modeling


Aug 12, 2020
Shuai Zhang, Peng Zhang, Xindian Ma, Junqiu Wei, Ningning Wang, Qun Liu


  Access Paper or Ask Questions

Learning to Detect Unacceptable Machine Translations for Downstream Tasks


May 08, 2020
Meng Zhang, Xin Jiang, Yang Liu, Qun Liu


  Access Paper or Ask Questions

Accurate Word Alignment Induction from Neural Machine Translation


Apr 30, 2020
Yun Chen, Yang Liu, Guanhua Chen, Xin Jiang, Qun Liu


  Access Paper or Ask Questions

Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT


Apr 30, 2020
Zhiyong Wu, Yun Chen, Ben Kao, Qun Liu

* Accepted to ACL2020 as a long paper 

  Access Paper or Ask Questions

Probabilistically Masked Language Model Capable of Autoregressive Generation in Arbitrary Word Order


Apr 24, 2020
Yi Liao, Xin Jiang, Qun Liu

* Accepted by ACL 2020 

  Access Paper or Ask Questions