Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Di He

Do Transformers Really Perform Bad for Graph Representation?


Jun 09, 2021
Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, Tie-Yan Liu


  Access Paper or Ask Questions

How could Neural Networks understand Programs?


May 31, 2021
Dinglan Peng, Shuxin Zheng, Yatao Li, Guolin Ke, Di He, Tie-Yan Liu

* ICML 2021 

  Access Paper or Ask Questions

Adversarial Training with Rectified Rejection


May 31, 2021
Tianyu Pang, Huishuai Zhang, Di He, Yinpeng Dong, Hang Su, Wei Chen, Jun Zhu, Tie-Yan Liu


  Access Paper or Ask Questions

Wav2vec-C: A Self-supervised Model for Speech Representation Learning


Mar 09, 2021
Samik Sadhu, Di He, Che-Wei Huang, Sri Harish Mallidi, Minhua Wu, Ariya Rastrow, Andreas Stolcke, Jasha Droppo, Roland Maas


  Access Paper or Ask Questions

Transformers with Competitive Ensembles of Independent Mechanisms


Feb 27, 2021
Alex Lamb, Di He, Anirudh Goyal, Guolin Ke, Chien-Feng Liao, Mirco Ravanelli, Yoshua Bengio

* Under Review, ICML 2021 

  Access Paper or Ask Questions

LazyFormer: Self Attention with Lazy Update


Feb 25, 2021
Chengxuan Ying, Guolin Ke, Di He, Tie-Yan Liu


  Access Paper or Ask Questions

Less is More: Pre-training a Strong Siamese Encoder Using a Weak Decoder


Feb 18, 2021
Shuqi Lu, Chenyan Xiong, Di He, Guolin Ke, Waleed Malik, Zhicheng Dou, Paul Bennett, Tieyan Liu, Arnold Overwijk


  Access Paper or Ask Questions

Revisiting Language Encoding in Learning Multilingual Representations


Feb 16, 2021
Shengjie Luo, Kaiyuan Gao, Shuxin Zheng, Guolin Ke, Di He, Liwei Wang, Tie-Yan Liu


  Access Paper or Ask Questions

Towards Certifying $\ell_\infty$ Robustness using Neural Networks with $\ell_\infty$-dist Neurons


Feb 10, 2021
Bohang Zhang, Tianle Cai, Zhou Lu, Di He, Liwei Wang


  Access Paper or Ask Questions

CODE-AE: A Coherent De-confounding Autoencoder for Predicting Patient-Specific Drug Response From Cell Line Transcriptomics


Jan 31, 2021
Di He, Lei Xie


  Access Paper or Ask Questions

A Cross-Level Information Transmission Network for Predicting Phenotype from New Genotype: Application to Cancer Precision Medicine


Oct 09, 2020
Di He, Lei Xie


  Access Paper or Ask Questions

GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training


Sep 07, 2020
Tianle Cai, Shengjie Luo, Keyulu Xu, Di He, Tie-yan Liu, Liwei Wang


  Access Paper or Ask Questions

Variance-reduced Language Pretraining via a Mask Proposal Network


Aug 12, 2020
Liang Chen, Tianyuan Zhang, Di He, Guolin Ke, Liwei Wang, Tie-Yan Liu


  Access Paper or Ask Questions

Taking Notes on the Fly Helps BERT Pre-training


Aug 04, 2020
Qiyu Wu, Chen Xing, Yatao Li, Guolin Ke, Di He, Tie-Yan Liu

* Qiyu Wu and Chen Xing contribute equally 

  Access Paper or Ask Questions

Transferred Discrepancy: Quantifying the Difference Between Representations


Jul 24, 2020
Yunzhen Feng, Runtian Zhai, Di He, Liwei Wang, Bin Dong

* 23 pages, 3 figures 

  Access Paper or Ask Questions

Rethinking Positional Encoding in Language Pre-training


Jul 09, 2020
Guolin Ke, Di He, Tie-Yan Liu

* refine writing; update down-stream results due to bug fix 

  Access Paper or Ask Questions

Rethinking the Positional Encoding in Language Pre-training


Jun 28, 2020
Guolin Ke, Di He, Tie-Yan Liu


  Access Paper or Ask Questions

MC-BERT: Efficient Language Pre-Training via a Meta Controller


Jun 16, 2020
Zhenhui Xu, Linyuan Gong, Guolin Ke, Di He, Shuxin Zheng, Liwei Wang, Jiang Bian, Tie-Yan Liu


  Access Paper or Ask Questions

Invertible Image Rescaling


May 12, 2020
Mingqing Xiao, Shuxin Zheng, Chang Liu, Yaolong Wang, Di He, Guolin Ke, Jiang Bian, Zhouchen Lin, Tie-Yan Liu


  Access Paper or Ask Questions

Incorporating BERT into Neural Machine Translation


Feb 17, 2020
Jinhua Zhu, Yingce Xia, Lijun Wu, Di He, Tao Qin, Wengang Zhou, Houqiang Li, Tie-Yan Liu

* Accepted to ICLR-2020 

  Access Paper or Ask Questions

MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius


Feb 15, 2020
Runtian Zhai, Chen Dan, Di He, Huan Zhang, Boqing Gong, Pradeep Ravikumar, Cho-Jui Hsieh, Liwei Wang

* In ICLR 2020. 20 Pages 

  Access Paper or Ask Questions

On Layer Normalization in the Transformer Architecture


Feb 12, 2020
Ruibin Xiong, Yunchang Yang, Di He, Kai Zheng, Shuxin Zheng, Chen Xing, Huishuai Zhang, Yanyan Lan, Liwei Wang, Tie-Yan Liu


  Access Paper or Ask Questions

Defective Convolutional Layers Learn Robust CNNs


Nov 19, 2019
Tiange Luo, Tianle Cai, Mengxiao Zhang, Siyu Chen, Di He, Liwei Wang


  Access Paper or Ask Questions

Fast Structured Decoding for Sequence Models


Oct 25, 2019
Zhiqing Sun, Zhuohan Li, Haoqing Wang, Zi Lin, Di He, Zhi-Hong Deng

* Accepted to NeurIPS 2019 (Previous title: Structured Decoding for Non-Autoregressive Machine Translation) 

  Access Paper or Ask Questions

On the Anomalous Generalization of GANs


Oct 06, 2019
Jinchen Xuan, Yunchang Yang, Ze Yang, Di He, Liwei Wang


  Access Paper or Ask Questions