Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox
Picture for Di He

Di He

Transformers with Competitive Ensembles of Independent Mechanisms


Feb 27, 2021
Alex Lamb, Di He, Anirudh Goyal, Guolin Ke, Chien-Feng Liao, Mirco Ravanelli, Yoshua Bengio

Figure 1 for Transformers with Competitive Ensembles of Independent Mechanisms
Figure 2 for Transformers with Competitive Ensembles of Independent Mechanisms
Figure 3 for Transformers with Competitive Ensembles of Independent Mechanisms
Figure 4 for Transformers with Competitive Ensembles of Independent Mechanisms
* Under Review, ICML 2021 
  Access Paper or Ask Questions

LazyFormer: Self Attention with Lazy Update


Feb 25, 2021
Chengxuan Ying, Guolin Ke, Di He, Tie-Yan Liu

Figure 1 for LazyFormer: Self Attention with Lazy Update
Figure 2 for LazyFormer: Self Attention with Lazy Update
Figure 3 for LazyFormer: Self Attention with Lazy Update
Figure 4 for LazyFormer: Self Attention with Lazy Update

  Access Paper or Ask Questions

Less is More: Pre-training a Strong Siamese Encoder Using a Weak Decoder


Feb 18, 2021
Shuqi Lu, Chenyan Xiong, Di He, Guolin Ke, Waleed Malik, Zhicheng Dou, Paul Bennett, Tieyan Liu, Arnold Overwijk

Figure 1 for Less is More: Pre-training a Strong Siamese Encoder Using a Weak Decoder
Figure 2 for Less is More: Pre-training a Strong Siamese Encoder Using a Weak Decoder
Figure 3 for Less is More: Pre-training a Strong Siamese Encoder Using a Weak Decoder
Figure 4 for Less is More: Pre-training a Strong Siamese Encoder Using a Weak Decoder

  Access Paper or Ask Questions

Revisiting Language Encoding in Learning Multilingual Representations


Feb 16, 2021
Shengjie Luo, Kaiyuan Gao, Shuxin Zheng, Guolin Ke, Di He, Liwei Wang, Tie-Yan Liu


  Access Paper or Ask Questions

Towards Certifying $\ell_\infty$ Robustness using Neural Networks with $\ell_\infty$-dist Neurons


Feb 10, 2021
Bohang Zhang, Tianle Cai, Zhou Lu, Di He, Liwei Wang

Figure 1 for Towards Certifying $\ell_\infty$ Robustness using Neural Networks with $\ell_\infty$-dist Neurons
Figure 2 for Towards Certifying $\ell_\infty$ Robustness using Neural Networks with $\ell_\infty$-dist Neurons
Figure 3 for Towards Certifying $\ell_\infty$ Robustness using Neural Networks with $\ell_\infty$-dist Neurons
Figure 4 for Towards Certifying $\ell_\infty$ Robustness using Neural Networks with $\ell_\infty$-dist Neurons

  Access Paper or Ask Questions

CODE-AE: A Coherent De-confounding Autoencoder for Predicting Patient-Specific Drug Response From Cell Line Transcriptomics


Jan 31, 2021
Di He, Lei Xie

Figure 1 for CODE-AE: A Coherent De-confounding Autoencoder for Predicting Patient-Specific Drug Response From Cell Line Transcriptomics
Figure 2 for CODE-AE: A Coherent De-confounding Autoencoder for Predicting Patient-Specific Drug Response From Cell Line Transcriptomics
Figure 3 for CODE-AE: A Coherent De-confounding Autoencoder for Predicting Patient-Specific Drug Response From Cell Line Transcriptomics
Figure 4 for CODE-AE: A Coherent De-confounding Autoencoder for Predicting Patient-Specific Drug Response From Cell Line Transcriptomics

  Access Paper or Ask Questions

A Cross-Level Information Transmission Network for Predicting Phenotype from New Genotype: Application to Cancer Precision Medicine


Oct 09, 2020
Di He, Lei Xie


  Access Paper or Ask Questions

GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training


Sep 07, 2020
Tianle Cai, Shengjie Luo, Keyulu Xu, Di He, Tie-yan Liu, Liwei Wang

Figure 1 for GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training
Figure 2 for GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training
Figure 3 for GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training
Figure 4 for GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training

  Access Paper or Ask Questions

Variance-reduced Language Pretraining via a Mask Proposal Network


Aug 12, 2020
Liang Chen, Tianyuan Zhang, Di He, Guolin Ke, Liwei Wang, Tie-Yan Liu

Figure 1 for Variance-reduced Language Pretraining via a Mask Proposal Network
Figure 2 for Variance-reduced Language Pretraining via a Mask Proposal Network
Figure 3 for Variance-reduced Language Pretraining via a Mask Proposal Network
Figure 4 for Variance-reduced Language Pretraining via a Mask Proposal Network

  Access Paper or Ask Questions

Taking Notes on the Fly Helps BERT Pre-training


Aug 04, 2020
Qiyu Wu, Chen Xing, Yatao Li, Guolin Ke, Di He, Tie-Yan Liu

* Qiyu Wu and Chen Xing contribute equally 
  Access Paper or Ask Questions

Transferred Discrepancy: Quantifying the Difference Between Representations


Jul 24, 2020
Yunzhen Feng, Runtian Zhai, Di He, Liwei Wang, Bin Dong

Figure 1 for Transferred Discrepancy: Quantifying the Difference Between Representations
Figure 2 for Transferred Discrepancy: Quantifying the Difference Between Representations
Figure 3 for Transferred Discrepancy: Quantifying the Difference Between Representations
Figure 4 for Transferred Discrepancy: Quantifying the Difference Between Representations
* 23 pages, 3 figures 
  Access Paper or Ask Questions

Rethinking Positional Encoding in Language Pre-training


Jul 09, 2020
Guolin Ke, Di He, Tie-Yan Liu

Figure 1 for Rethinking Positional Encoding in Language Pre-training
Figure 2 for Rethinking Positional Encoding in Language Pre-training
Figure 3 for Rethinking Positional Encoding in Language Pre-training
Figure 4 for Rethinking Positional Encoding in Language Pre-training
* refine writing; update down-stream results due to bug fix 
  Access Paper or Ask Questions

Rethinking the Positional Encoding in Language Pre-training


Jun 28, 2020
Guolin Ke, Di He, Tie-Yan Liu

Figure 1 for Rethinking the Positional Encoding in Language Pre-training
Figure 2 for Rethinking the Positional Encoding in Language Pre-training
Figure 3 for Rethinking the Positional Encoding in Language Pre-training
Figure 4 for Rethinking the Positional Encoding in Language Pre-training

  Access Paper or Ask Questions

MC-BERT: Efficient Language Pre-Training via a Meta Controller


Jun 16, 2020
Zhenhui Xu, Linyuan Gong, Guolin Ke, Di He, Shuxin Zheng, Liwei Wang, Jiang Bian, Tie-Yan Liu

Figure 1 for MC-BERT: Efficient Language Pre-Training via a Meta Controller
Figure 2 for MC-BERT: Efficient Language Pre-Training via a Meta Controller
Figure 3 for MC-BERT: Efficient Language Pre-Training via a Meta Controller
Figure 4 for MC-BERT: Efficient Language Pre-Training via a Meta Controller

  Access Paper or Ask Questions

Invertible Image Rescaling


May 12, 2020
Mingqing Xiao, Shuxin Zheng, Chang Liu, Yaolong Wang, Di He, Guolin Ke, Jiang Bian, Zhouchen Lin, Tie-Yan Liu

Figure 1 for Invertible Image Rescaling
Figure 2 for Invertible Image Rescaling
Figure 3 for Invertible Image Rescaling
Figure 4 for Invertible Image Rescaling

  Access Paper or Ask Questions

Incorporating BERT into Neural Machine Translation


Feb 17, 2020
Jinhua Zhu, Yingce Xia, Lijun Wu, Di He, Tao Qin, Wengang Zhou, Houqiang Li, Tie-Yan Liu

Figure 1 for Incorporating BERT into Neural Machine Translation
Figure 2 for Incorporating BERT into Neural Machine Translation
Figure 3 for Incorporating BERT into Neural Machine Translation
Figure 4 for Incorporating BERT into Neural Machine Translation
* Accepted to ICLR-2020 
  Access Paper or Ask Questions

MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius


Feb 15, 2020
Runtian Zhai, Chen Dan, Di He, Huan Zhang, Boqing Gong, Pradeep Ravikumar, Cho-Jui Hsieh, Liwei Wang

Figure 1 for MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius
Figure 2 for MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius
Figure 3 for MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius
Figure 4 for MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius
* In ICLR 2020. 20 Pages 
  Access Paper or Ask Questions

On Layer Normalization in the Transformer Architecture


Feb 12, 2020
Ruibin Xiong, Yunchang Yang, Di He, Kai Zheng, Shuxin Zheng, Chen Xing, Huishuai Zhang, Yanyan Lan, Liwei Wang, Tie-Yan Liu

Figure 1 for On Layer Normalization in the Transformer Architecture
Figure 2 for On Layer Normalization in the Transformer Architecture
Figure 3 for On Layer Normalization in the Transformer Architecture
Figure 4 for On Layer Normalization in the Transformer Architecture

  Access Paper or Ask Questions

Defective Convolutional Layers Learn Robust CNNs


Nov 19, 2019
Tiange Luo, Tianle Cai, Mengxiao Zhang, Siyu Chen, Di He, Liwei Wang

Figure 1 for Defective Convolutional Layers Learn Robust CNNs
Figure 2 for Defective Convolutional Layers Learn Robust CNNs
Figure 3 for Defective Convolutional Layers Learn Robust CNNs
Figure 4 for Defective Convolutional Layers Learn Robust CNNs

  Access Paper or Ask Questions

Fast Structured Decoding for Sequence Models


Oct 25, 2019
Zhiqing Sun, Zhuohan Li, Haoqing Wang, Zi Lin, Di He, Zhi-Hong Deng

Figure 1 for Fast Structured Decoding for Sequence Models
Figure 2 for Fast Structured Decoding for Sequence Models
Figure 3 for Fast Structured Decoding for Sequence Models
Figure 4 for Fast Structured Decoding for Sequence Models
* Accepted to NeurIPS 2019 (Previous title: Structured Decoding for Non-Autoregressive Machine Translation) 
  Access Paper or Ask Questions

On the Anomalous Generalization of GANs


Oct 06, 2019
Jinchen Xuan, Yunchang Yang, Ze Yang, Di He, Liwei Wang

Figure 1 for On the Anomalous Generalization of GANs
Figure 2 for On the Anomalous Generalization of GANs
Figure 3 for On the Anomalous Generalization of GANs
Figure 4 for On the Anomalous Generalization of GANs

  Access Paper or Ask Questions

Hint-Based Training for Non-Autoregressive Machine Translation


Sep 15, 2019
Zhuohan Li, Zi Lin, Di He, Fei Tian, Tao Qin, Liwei Wang, Tie-Yan Liu

Figure 1 for Hint-Based Training for Non-Autoregressive Machine Translation
Figure 2 for Hint-Based Training for Non-Autoregressive Machine Translation
Figure 3 for Hint-Based Training for Non-Autoregressive Machine Translation
Figure 4 for Hint-Based Training for Non-Autoregressive Machine Translation
* EMNLP-IJCNLP 2019 
  Access Paper or Ask Questions

Multilingual Neural Machine Translation with Language Clustering


Aug 25, 2019
Xu Tan, Jiale Chen, Di He, Yingce Xia, Tao Qin, Tie-Yan Liu

Figure 1 for Multilingual Neural Machine Translation with Language Clustering
Figure 2 for Multilingual Neural Machine Translation with Language Clustering
Figure 3 for Multilingual Neural Machine Translation with Language Clustering
Figure 4 for Multilingual Neural Machine Translation with Language Clustering
* Accepted by EMNLP 2019 
  Access Paper or Ask Questions

Representation Degeneration Problem in Training Natural Language Generation Models


Jul 28, 2019
Jun Gao, Di He, Xu Tan, Tao Qin, Liwei Wang, Tie-Yan Liu

Figure 1 for Representation Degeneration Problem in Training Natural Language Generation Models
Figure 2 for Representation Degeneration Problem in Training Natural Language Generation Models
Figure 3 for Representation Degeneration Problem in Training Natural Language Generation Models
Figure 4 for Representation Degeneration Problem in Training Natural Language Generation Models
* ICLR 2019 
  Access Paper or Ask Questions

Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View


Jun 06, 2019
Yiping Lu, Zhuohan Li, Di He, Zhiqing Sun, Bin Dong, Tao Qin, Liwei Wang, Tie-Yan Liu

Figure 1 for Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View
Figure 2 for Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View
Figure 3 for Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View
Figure 4 for Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View

  Access Paper or Ask Questions