Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Zhuohan Li

TeraPipe: Token-Level Pipeline Parallelism for Training Large-Scale Language Models

Feb 16, 2021
Zhuohan Li, Siyuan Zhuang, Shiyuan Guo, Danyang Zhuo, Hao Zhang, Dawn Song, Ion Stoica

  Access Paper or Ask Questions

Train Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers

Feb 26, 2020
Zhuohan Li, Eric Wallace, Sheng Shen, Kevin Lin, Kurt Keutzer, Dan Klein, Joseph E. Gonzalez

  Access Paper or Ask Questions

Hoplite: Efficient Collective Communication for Task-Based Distributed Systems

Feb 13, 2020
Siyuan Zhuang, Zhuohan Li, Danyang Zhuo, Stephanie Wang, Eric Liang, Robert Nishihara, Philipp Moritz, Ion Stoica

  Access Paper or Ask Questions

Fast Structured Decoding for Sequence Models

Oct 25, 2019
Zhiqing Sun, Zhuohan Li, Haoqing Wang, Zi Lin, Di He, Zhi-Hong Deng

* Accepted to NeurIPS 2019 (Previous title: Structured Decoding for Non-Autoregressive Machine Translation) 

  Access Paper or Ask Questions

Hint-Based Training for Non-Autoregressive Machine Translation

Sep 15, 2019
Zhuohan Li, Zi Lin, Di He, Fei Tian, Tao Qin, Liwei Wang, Tie-Yan Liu


  Access Paper or Ask Questions

Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View

Jun 06, 2019
Yiping Lu, Zhuohan Li, Di He, Zhiqing Sun, Bin Dong, Tao Qin, Liwei Wang, Tie-Yan Liu

  Access Paper or Ask Questions

Towards Binary-Valued Gates for Robust LSTM Training

Jun 08, 2018
Zhuohan Li, Di He, Fei Tian, Wei Chen, Tao Qin, Liwei Wang, Tie-Yan Liu

* ICML 2018 

  Access Paper or Ask Questions