Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox
Picture for Qun Liu

Qun Liu

PanGu-$α$: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation


Apr 26, 2021
Wei Zeng, Xiaozhe Ren, Teng Su, Hui Wang, Yi Liao, Zhiwei Wang, Xin Jiang, ZhenZhang Yang, Kaisheng Wang, Xiaoda Zhang, Chen Li, Ziyan Gong, Yifan Yao, Xinjing Huang, Jun Wang, Jianfeng Yu, Qi Guo, Yue Yu, Yan Zhang, Jin Wang, Hengtao Tao, Dasen Yan, Zexuan Yi, Fang Peng, Fangqing Jiang, Han Zhang, Lingfeng Deng, Yehong Zhang, Zhe Lin, Chao Zhang, Shaojie Zhang, Mingyue Guo, Shanzhi Gu, Gaojun Fan, Yaowei Wang, Xuefeng Jin, Qun Liu, Yonghong Tian

Figure 1 for PanGu-$α$: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation
Figure 2 for PanGu-$α$: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation
Figure 3 for PanGu-$α$: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation
Figure 4 for PanGu-$α$: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation
* The technique report for PanGu-$\alpha$ 
  Access Paper or Ask Questions

Extract then Distill: Efficient and Effective Task-Agnostic BERT Distillation


Apr 24, 2021
Cheng Chen, Yichun Yin, Lifeng Shang, Zhi Wang, Xin Jiang, Xiao Chen, Qun Liu

Figure 1 for Extract then Distill: Efficient and Effective Task-Agnostic BERT Distillation
Figure 2 for Extract then Distill: Efficient and Effective Task-Agnostic BERT Distillation
Figure 3 for Extract then Distill: Efficient and Effective Task-Agnostic BERT Distillation
Figure 4 for Extract then Distill: Efficient and Effective Task-Agnostic BERT Distillation

  Access Paper or Ask Questions

Improving Neural Machine Translation with Compact Word Embedding Tables


Apr 18, 2021
Krtin Kumar, Mehdi Rezagholizadeh, Yiu Sing Lau, Qun Liu

Figure 1 for Improving Neural Machine Translation with Compact Word Embedding Tables
Figure 2 for Improving Neural Machine Translation with Compact Word Embedding Tables
Figure 3 for Improving Neural Machine Translation with Compact Word Embedding Tables
Figure 4 for Improving Neural Machine Translation with Compact Word Embedding Tables

  Access Paper or Ask Questions

An Approach to Improve Robustness of NLP Systems against ASR Errors


Mar 25, 2021
Tong Cui, Jinghui Xiao, Liangyou Li, Xin Jiang, Qun Liu

Figure 1 for An Approach to Improve Robustness of NLP Systems against ASR Errors
Figure 2 for An Approach to Improve Robustness of NLP Systems against ASR Errors
Figure 3 for An Approach to Improve Robustness of NLP Systems against ASR Errors
Figure 4 for An Approach to Improve Robustness of NLP Systems against ASR Errors
* 9 pages, 3 figures 
  Access Paper or Ask Questions

Dependency Graph-to-String Statistical Machine Translation


Mar 20, 2021
Liangyou Li, Andy Way, Qun Liu

Figure 1 for Dependency Graph-to-String Statistical Machine Translation
Figure 2 for Dependency Graph-to-String Statistical Machine Translation
Figure 3 for Dependency Graph-to-String Statistical Machine Translation
Figure 4 for Dependency Graph-to-String Statistical Machine Translation

  Access Paper or Ask Questions

Reweighting Augmented Samples by Minimizing the Maximal Expected Loss


Mar 16, 2021
Mingyang Yi, Lu Hou, Lifeng Shang, Xin Jiang, Qun Liu, Zhi-Ming Ma

Figure 1 for Reweighting Augmented Samples by Minimizing the Maximal Expected Loss
Figure 2 for Reweighting Augmented Samples by Minimizing the Maximal Expected Loss
Figure 3 for Reweighting Augmented Samples by Minimizing the Maximal Expected Loss
Figure 4 for Reweighting Augmented Samples by Minimizing the Maximal Expected Loss
* published in ICLR2021 
 * 14 pages 
  Access Paper or Ask Questions

LightMBERT: A Simple Yet Effective Method for Multilingual BERT Distillation


Mar 11, 2021
Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, Qun Liu

Figure 1 for LightMBERT: A Simple Yet Effective Method for Multilingual BERT Distillation
Figure 2 for LightMBERT: A Simple Yet Effective Method for Multilingual BERT Distillation
Figure 3 for LightMBERT: A Simple Yet Effective Method for Multilingual BERT Distillation
Figure 4 for LightMBERT: A Simple Yet Effective Method for Multilingual BERT Distillation

  Access Paper or Ask Questions

Training Multilingual Pre-trained Language Model with Byte-level Subwords


Jan 23, 2021
Junqiu Wei, Qun Liu, Yinpeng Guo, Xin Jiang

Figure 1 for Training Multilingual Pre-trained Language Model with Byte-level Subwords
Figure 2 for Training Multilingual Pre-trained Language Model with Byte-level Subwords
Figure 3 for Training Multilingual Pre-trained Language Model with Byte-level Subwords
Figure 4 for Training Multilingual Pre-trained Language Model with Byte-level Subwords

  Access Paper or Ask Questions

Revisiting Robust Neural Machine Translation: A Transformer Case Study


Dec 31, 2020
Peyman Passban, Puneeth S. M. Saladi, Qun Liu

Figure 1 for Revisiting Robust Neural Machine Translation: A Transformer Case Study
Figure 2 for Revisiting Robust Neural Machine Translation: A Transformer Case Study
Figure 3 for Revisiting Robust Neural Machine Translation: A Transformer Case Study
Figure 4 for Revisiting Robust Neural Machine Translation: A Transformer Case Study

  Access Paper or Ask Questions

BinaryBERT: Pushing the Limit of BERT Quantization


Dec 31, 2020
Haoli Bai, Wei Zhang, Lu Hou, Lifeng Shang, Jing Jin, Xin Jiang, Qun Liu, Michael Lyu, Irwin King

Figure 1 for BinaryBERT: Pushing the Limit of BERT Quantization
Figure 2 for BinaryBERT: Pushing the Limit of BERT Quantization
Figure 3 for BinaryBERT: Pushing the Limit of BERT Quantization
Figure 4 for BinaryBERT: Pushing the Limit of BERT Quantization

  Access Paper or Ask Questions

Better Robustness by More Coverage: Adversarial Training with Mixup Augmentation for Robust Fine-tuning


Dec 31, 2020
Chenglei Si, Zhengyan Zhang, Fanchao Qi, Zhiyuan Liu, Yasheng Wang, Qun Liu, Maosong Sun

Figure 1 for Better Robustness by More Coverage: Adversarial Training with Mixup Augmentation for Robust Fine-tuning
Figure 2 for Better Robustness by More Coverage: Adversarial Training with Mixup Augmentation for Robust Fine-tuning
Figure 3 for Better Robustness by More Coverage: Adversarial Training with Mixup Augmentation for Robust Fine-tuning
Figure 4 for Better Robustness by More Coverage: Adversarial Training with Mixup Augmentation for Robust Fine-tuning
* 9 pages 
  Access Paper or Ask Questions

HopRetriever: Retrieve Hops over Wikipedia to Answer Complex Questions


Dec 31, 2020
Shaobo Li, Xiaoguang Li, Lifeng Shang, Xin Jiang, Qun Liu, Chengjie Sun, Zhenzhou Ji, Bingquan Liu

Figure 1 for HopRetriever: Retrieve Hops over Wikipedia to Answer Complex Questions
Figure 2 for HopRetriever: Retrieve Hops over Wikipedia to Answer Complex Questions
Figure 3 for HopRetriever: Retrieve Hops over Wikipedia to Answer Complex Questions
Figure 4 for HopRetriever: Retrieve Hops over Wikipedia to Answer Complex Questions
* Accepted at AAAI 2021 
  Access Paper or Ask Questions

ALP-KD: Attention-Based Layer Projection for Knowledge Distillation


Dec 27, 2020
Peyman Passban, Yimeng Wu, Mehdi Rezagholizadeh, Qun Liu

Figure 1 for ALP-KD: Attention-Based Layer Projection for Knowledge Distillation
Figure 2 for ALP-KD: Attention-Based Layer Projection for Knowledge Distillation
Figure 3 for ALP-KD: Attention-Based Layer Projection for Knowledge Distillation
Figure 4 for ALP-KD: Attention-Based Layer Projection for Knowledge Distillation
* AAAI 2021. This work has been done while Peyman Passban was at Huawei 
  Access Paper or Ask Questions

Improving Task-Agnostic BERT Distillation with Layer Mapping Search


Dec 11, 2020
Xiaoqi Jiao, Huating Chang, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, Qun Liu

Figure 1 for Improving Task-Agnostic BERT Distillation with Layer Mapping Search
Figure 2 for Improving Task-Agnostic BERT Distillation with Layer Mapping Search
Figure 3 for Improving Task-Agnostic BERT Distillation with Layer Mapping Search
Figure 4 for Improving Task-Agnostic BERT Distillation with Layer Mapping Search

  Access Paper or Ask Questions

Document Graph for Neural Machine Translation


Dec 08, 2020
Mingzhou Xu, Liangyou Li, Derek. F. Wong, Qun Liu, Lidia S. Chao

Figure 1 for Document Graph for Neural Machine Translation
Figure 2 for Document Graph for Neural Machine Translation
Figure 3 for Document Graph for Neural Machine Translation
Figure 4 for Document Graph for Neural Machine Translation

  Access Paper or Ask Questions

PPKE: Knowledge Representation Learning by Path-based Pre-training


Dec 07, 2020
Bin He, Di Zhou, Jing Xie, Jinghui Xiao, Xin Jiang, Qun Liu

Figure 1 for PPKE: Knowledge Representation Learning by Path-based Pre-training
Figure 2 for PPKE: Knowledge Representation Learning by Path-based Pre-training
Figure 3 for PPKE: Knowledge Representation Learning by Path-based Pre-training
Figure 4 for PPKE: Knowledge Representation Learning by Path-based Pre-training

  Access Paper or Ask Questions

KgPLM: Knowledge-guided Language Model Pre-training via Generative and Discriminative Learning


Dec 07, 2020
Bin He, Xin Jiang, Jinghui Xiao, Qun Liu

Figure 1 for KgPLM: Knowledge-guided Language Model Pre-training via Generative and Discriminative Learning
Figure 2 for KgPLM: Knowledge-guided Language Model Pre-training via Generative and Discriminative Learning
Figure 3 for KgPLM: Knowledge-guided Language Model Pre-training via Generative and Discriminative Learning
Figure 4 for KgPLM: Knowledge-guided Language Model Pre-training via Generative and Discriminative Learning
* 10 pages, 3 figures 
  Access Paper or Ask Questions

From Unsupervised Machine Translation To Adversarial Text Generation


Nov 10, 2020
Ahmad Rashid, Alan Do-Omri, Md. Akmal Haidar, Qun Liu, Mehdi Rezagholizadeh

Figure 1 for From Unsupervised Machine Translation To Adversarial Text Generation
Figure 2 for From Unsupervised Machine Translation To Adversarial Text Generation
Figure 3 for From Unsupervised Machine Translation To Adversarial Text Generation
Figure 4 for From Unsupervised Machine Translation To Adversarial Text Generation
* Accepted at ICASSP 2020 
  Access Paper or Ask Questions

Know What You Don't Need: Single-Shot Meta-Pruning for Attention Heads


Nov 07, 2020
Zhengyan Zhang, Fanchao Qi, Zhiyuan Liu, Qun Liu, Maosong Sun

Figure 1 for Know What You Don't Need: Single-Shot Meta-Pruning for Attention Heads
Figure 2 for Know What You Don't Need: Single-Shot Meta-Pruning for Attention Heads
Figure 3 for Know What You Don't Need: Single-Shot Meta-Pruning for Attention Heads
Figure 4 for Know What You Don't Need: Single-Shot Meta-Pruning for Attention Heads

  Access Paper or Ask Questions

HyperText: Endowing FastText with Hyperbolic Geometry


Oct 30, 2020
Yudong Zhu, Di Zhou, Jinghui Xiao, Xin Jiang, Xiao Chen, Qun Liu

Figure 1 for HyperText: Endowing FastText with Hyperbolic Geometry
Figure 2 for HyperText: Endowing FastText with Hyperbolic Geometry
Figure 3 for HyperText: Endowing FastText with Hyperbolic Geometry
Figure 4 for HyperText: Endowing FastText with Hyperbolic Geometry
* Findings of EMNLP 2020 
  Access Paper or Ask Questions

LiDAM: Semi-Supervised Learning with Localized Domain Adaptation and Iterative Matching


Oct 13, 2020
Qun Liu, Matthew Shreve, Raja Bala

Figure 1 for LiDAM: Semi-Supervised Learning with Localized Domain Adaptation and Iterative Matching
Figure 2 for LiDAM: Semi-Supervised Learning with Localized Domain Adaptation and Iterative Matching
Figure 3 for LiDAM: Semi-Supervised Learning with Localized Domain Adaptation and Iterative Matching
Figure 4 for LiDAM: Semi-Supervised Learning with Localized Domain Adaptation and Iterative Matching

  Access Paper or Ask Questions

TernaryBERT: Distillation-aware Ultra-low Bit BERT


Oct 10, 2020
Wei Zhang, Lu Hou, Yichun Yin, Lifeng Shang, Xiao Chen, Xin Jiang, Qun Liu

Figure 1 for TernaryBERT: Distillation-aware Ultra-low Bit BERT
Figure 2 for TernaryBERT: Distillation-aware Ultra-low Bit BERT
Figure 3 for TernaryBERT: Distillation-aware Ultra-low Bit BERT
Figure 4 for TernaryBERT: Distillation-aware Ultra-low Bit BERT
* Accepted by EMNLP 2020 
  Access Paper or Ask Questions

Why Skip If You Can Combine: A Simple Knowledge Distillation Technique for Intermediate Layers


Oct 06, 2020
Yimeng Wu, Peyman Passban, Mehdi Rezagholizade, Qun Liu

Figure 1 for Why Skip If You Can Combine: A Simple Knowledge Distillation Technique for Intermediate Layers
Figure 2 for Why Skip If You Can Combine: A Simple Knowledge Distillation Technique for Intermediate Layers
Figure 3 for Why Skip If You Can Combine: A Simple Knowledge Distillation Technique for Intermediate Layers
Figure 4 for Why Skip If You Can Combine: A Simple Knowledge Distillation Technique for Intermediate Layers
* The first two authors contributed equally 
  Access Paper or Ask Questions

TensorCoder: Dimension-Wise Attention via Tensor Representation for Natural Language Modeling


Aug 12, 2020
Shuai Zhang, Peng Zhang, Xindian Ma, Junqiu Wei, Ningning Wang, Qun Liu

Figure 1 for TensorCoder: Dimension-Wise Attention via Tensor Representation for Natural Language Modeling
Figure 2 for TensorCoder: Dimension-Wise Attention via Tensor Representation for Natural Language Modeling
Figure 3 for TensorCoder: Dimension-Wise Attention via Tensor Representation for Natural Language Modeling
Figure 4 for TensorCoder: Dimension-Wise Attention via Tensor Representation for Natural Language Modeling

  Access Paper or Ask Questions

Learning to Detect Unacceptable Machine Translations for Downstream Tasks


May 08, 2020
Meng Zhang, Xin Jiang, Yang Liu, Qun Liu

Figure 1 for Learning to Detect Unacceptable Machine Translations for Downstream Tasks
Figure 2 for Learning to Detect Unacceptable Machine Translations for Downstream Tasks
Figure 3 for Learning to Detect Unacceptable Machine Translations for Downstream Tasks
Figure 4 for Learning to Detect Unacceptable Machine Translations for Downstream Tasks

  Access Paper or Ask Questions

Accurate Word Alignment Induction from Neural Machine Translation


Apr 30, 2020
Yun Chen, Yang Liu, Guanhua Chen, Xin Jiang, Qun Liu

Figure 1 for Accurate Word Alignment Induction from Neural Machine Translation
Figure 2 for Accurate Word Alignment Induction from Neural Machine Translation
Figure 3 for Accurate Word Alignment Induction from Neural Machine Translation
Figure 4 for Accurate Word Alignment Induction from Neural Machine Translation

  Access Paper or Ask Questions

Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT


Apr 30, 2020
Zhiyong Wu, Yun Chen, Ben Kao, Qun Liu

Figure 1 for Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT
Figure 2 for Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT
Figure 3 for Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT
Figure 4 for Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT
* Accepted to ACL2020 as a long paper 
  Access Paper or Ask Questions

Probabilistically Masked Language Model Capable of Autoregressive Generation in Arbitrary Word Order


Apr 24, 2020
Yi Liao, Xin Jiang, Qun Liu

Figure 1 for Probabilistically Masked Language Model Capable of Autoregressive Generation in Arbitrary Word Order
Figure 2 for Probabilistically Masked Language Model Capable of Autoregressive Generation in Arbitrary Word Order
Figure 3 for Probabilistically Masked Language Model Capable of Autoregressive Generation in Arbitrary Word Order
Figure 4 for Probabilistically Masked Language Model Capable of Autoregressive Generation in Arbitrary Word Order
* Accepted by ACL 2020 
  Access Paper or Ask Questions