Get our free extension to see links to code for papers anywhere online!

Chrome logo  Add to Chrome

Firefox logo Add to Firefox

Colossal-Auto: Unified Automation of Parallelization and Activation Checkpoint for Large-scale Models


Feb 22, 2023
Yuliang Liu, Shenggui Li, Jiarui Fang, Yanjun Shao, Boyuan Yao, Yang You

Add code


   Access Paper or Ask Questions

MAP: Memory-aware Automated Intra-op Parallel Training For Foundation Models


Feb 06, 2023
Yuliang Liu, Shenggui Li, Jiarui Fang, Yanjun Shao, Boyuan Yao, Yang You

Add code


   Access Paper or Ask Questions

Elixir: Train a Large Language Model on a Small GPU Cluster


Dec 10, 2022
Haichen Huang, Jiarui Fang, Hongxin Liu, Shenggui Li, Yang You

Add code


   Access Paper or Ask Questions

EnergonAI: An Inference System for 10-100 Billion Parameter Transformer Models


Sep 06, 2022
Jiangsu Du, Ziming Liu, Jiarui Fang, Shenggui Li, Yongbin Li, Yutong Lu, Yang You

Add code


   Access Paper or Ask Questions

A Frequency-aware Software Cache for Large Recommendation System Embeddings


Aug 08, 2022
Jiarui Fang, Geng Zhang, Jiatong Han, Shenggui Li, Zhengda Bian, Yongbin Li, Jin Liu, Yang You

Add code


   Access Paper or Ask Questions

PatrickStar: Parallel Training of Pre-trained Models via a Chunk-based Memory Management


Aug 12, 2021
Jiarui Fang, Yang Yu, Shenggui Li, Yang You, Jie Zhou

Add code

* 11 pages 

   Access Paper or Ask Questions

TurboTransformers: An Efficient GPU Serving System For Transformer Models


Oct 09, 2020
Jiarui Fang, Yang Yu, Chengduo Zhao, Jie Zhou

Add code


   Access Paper or Ask Questions

swCaffe: a Parallel Framework for Accelerating Deep Learning Applications on Sunway TaihuLight


Mar 16, 2019
Jiarui Fang, Liandeng Li, Haohuan Fu, Jinlei Jiang, Wenlai Zhao, Conghui He, Xin You, Guangwen Yang

Add code

* 10 pages 

   Access Paper or Ask Questions

RedSync : Reducing Synchronization Traffic for Distributed Deep Learning


Aug 13, 2018
Jiarui Fang, Haohuan Fu, Guangwen Yang, Cho-Jui Hsieh

Add code

* 20 pages 

   Access Paper or Ask Questions