Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Mehdi Rezagholizadeh

Mehdi Rezagholizadeh

Huawei Noah's Ark Lab

Pro-KD: Progressive Distillation by Following the Footsteps of the Teacher


Oct 16, 2021
Mehdi Rezagholizadeh, Aref Jafari, Puneeth Salad, Pranav Sharma, Ali Saheb Pasand, Ali Ghodsi


  Access Paper or Ask Questions

A Short Study on Compressing Decoder-Based Language Models


Oct 16, 2021
Tianda Li, Yassir El Mesbahi, Ivan Kobyzev, Ahmad Rashid, Atif Mahmud, Nithin Anchuri, Habib Hajimolahoseini, Yang Liu, Mehdi Rezagholizadeh


  Access Paper or Ask Questions

Kronecker Decomposition for GPT Compression


Oct 15, 2021
Ali Edalati, Marzieh Tahaei, Ahmad Rashid, Vahid Partovi Nia, James J. Clark, Mehdi Rezagholizadeh


  Access Paper or Ask Questions

RAIL-KD: RAndom Intermediate Layer Mapping for Knowledge Distillation


Oct 01, 2021
Md Akmal Haidar, Nithin Anchuri, Mehdi Rezagholizadeh, Abbas Ghaddar, Philippe Langlais, Pascal Poupart


  Access Paper or Ask Questions

Knowledge Distillation with Noisy Labels for Natural Language Understanding


Sep 21, 2021
Shivendra Bhardwaj, Abbas Ghaddar, Ahmad Rashid, Khalil Bibi, Chengyang Li, Ali Ghodsi, Philippe Langlais, Mehdi Rezagholizadeh


  Access Paper or Ask Questions

How to Select One Among All? An Extensive Empirical Study Towards the Robustness of Knowledge Distillation in Natural Language Understanding


Sep 20, 2021
Tianda Li, Ahmad Rashid, Aref Jafari, Pranav Sharma, Ali Ghodsi, Mehdi Rezagholizadeh

* Accepted as EMNLP 2021 Findings 

  Access Paper or Ask Questions

KroneckerBERT: Learning Kronecker Decomposition for Pre-trained Language Models via Knowledge Distillation


Sep 13, 2021
Marzieh S. Tahaei, Ella Charlaix, Vahid Partovi Nia, Ali Ghodsi, Mehdi Rezagholizadeh


  Access Paper or Ask Questions

End-to-End Self-Debiasing Framework for Robust NLU Training


Sep 05, 2021
Abbas Ghaddar, Philippe Langlais, Mehdi Rezagholizadeh, Ahmad Rashid

* Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021; August; 2021; pages 1923--1929 
* Findings ACL 2021 

  Access Paper or Ask Questions

Context-aware Adversarial Training for Name Regularity Bias in Named Entity Recognition


Jul 24, 2021
Abbas Ghaddar, Philippe Langlais, Ahmad Rashid, Mehdi Rezagholizadeh

* journal={Transactions of the Association for Computational Linguistics}, volume={9}, pages={586--604}, year={2021}, 
* MIT Press\TACL 2021\Presented at ACL 2021 This is the exact same content of the TACL version, except the figures and tables are better aligned 

  Access Paper or Ask Questions

Not Far Away, Not So Close: Sample Efficient Nearest Neighbour Data Augmentation via MiniMax


Jun 02, 2021
Ehsan Kamalloo, Mehdi Rezagholizadeh, Peyman Passban, Ali Ghodsi

* Findings of ACL 2021 

  Access Paper or Ask Questions

MATE-KD: Masked Adversarial TExt, a Companion to Knowledge Distillation


May 12, 2021
Ahmad Rashid, Vasileios Lioutas, Mehdi Rezagholizadeh

* Accepted at ACL 2021 

  Access Paper or Ask Questions

Improving Neural Machine Translation with Compact Word Embedding Tables


Apr 18, 2021
Krtin Kumar, Mehdi Rezagholizadeh, Yiu Sing Lau, Qun Liu


  Access Paper or Ask Questions

Robust Embeddings Via Distributions


Apr 17, 2021
Kira A. Selby, Yinong Wang, Ruizhe Wang, Peyman Passban, Ahmad Rashid, Mehdi Rezagholizadeh, Pascal Poupart


  Access Paper or Ask Questions

Annealing Knowledge Distillation


Apr 14, 2021
Aref Jafari, Mehdi Rezagholizadeh, Pranav Sharma, Ali Ghodsi


  Access Paper or Ask Questions

Transformer-based ASR Incorporating Time-reduction Layer and Fine-tuning with Self-Knowledge Distillation


Mar 17, 2021
Md Akmal Haidar, Chao Xing, Mehdi Rezagholizadeh


  Access Paper or Ask Questions

Fine-tuning of Pre-trained End-to-end Speech Recognition with Generative Adversarial Networks


Mar 10, 2021
Md Akmal Haidar, Mehdi Rezagholizadeh

* Accepted in ICASSP 2021 conference 

  Access Paper or Ask Questions

Towards Zero-Shot Knowledge Distillation for Natural Language Processing


Dec 31, 2020
Ahmad Rashid, Vasileios Lioutas, Abbas Ghaddar, Mehdi Rezagholizadeh

* 13 pages, 8 tables, 2 algorithms and 1 figure 

  Access Paper or Ask Questions

ALP-KD: Attention-Based Layer Projection for Knowledge Distillation


Dec 27, 2020
Peyman Passban, Yimeng Wu, Mehdi Rezagholizadeh, Qun Liu

* AAAI 2021. This work has been done while Peyman Passban was at Huawei 

  Access Paper or Ask Questions

From Unsupervised Machine Translation To Adversarial Text Generation


Nov 10, 2020
Ahmad Rashid, Alan Do-Omri, Md. Akmal Haidar, Qun Liu, Mehdi Rezagholizadeh

* Accepted at ICASSP 2020 

  Access Paper or Ask Questions

Fully Quantized Transformer for Improved Translation


Nov 23, 2019
Gabriele Prato, Ella Charlaix, Mehdi Rezagholizadeh


  Access Paper or Ask Questions

Fully Quantizing a Simplified Transformer for End-to-end Speech Recognition


Nov 09, 2019
Alex Bie, Bharat Venkitesh, Joao Monteiro, Md. Akmal Haidar, Mehdi Rezagholizadeh

* Submitted to ICASSP 2020 

  Access Paper or Ask Questions

Distilled embedding: non-linear embedding factorization using knowledge distillation


Oct 02, 2019
Vasileios Lioutas, Ahmad Rashid, Krtin Kumar, Md Akmal Haidar, Mehdi Rezagholizadeh

* 11 pages, 1 figure, 10 tables 

  Access Paper or Ask Questions

EditNTS: An Neural Programmer-Interpreter Model for Sentence Simplification through Explicit Editing


Jun 19, 2019
Yue Dong, Zichao Li, Mehdi Rezagholizadeh, Jackie Chi Kit Cheung

* 9 pages, 1 figure, accepted at ACL2019 

  Access Paper or Ask Questions

Bilingual-GAN: A Step Towards Parallel Text Generation


May 14, 2019
Ahmad Rashid, Alan Do-Omri, Md. Akmal Haidar, Qun Liu, Mehdi Rezagholizadeh


  Access Paper or Ask Questions

TextKD-GAN: Text Generation using KnowledgeDistillation and Generative Adversarial Networks


Apr 23, 2019
Md. Akmal Haidar, Mehdi Rezagholizadeh

* 32nd Canadian Conference on Artificial Intelligence 2019 
* arXiv admin note: text overlap with arXiv:1904.07293 

  Access Paper or Ask Questions

Latent Code and Text-based Generative Adversarial Networks for Soft-text Generation


Apr 23, 2019
Md. Akmal Haidar, Mehdi Rezagholizadeh, Alan Do-Omri, Ahmad Rashid

* 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics 

  Access Paper or Ask Questions