Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox
Picture for Shaohuai Shi

Shaohuai Shi

Accelerating Distributed K-FAC with Smart Parallelism of Computing and Communication Tasks


Jul 14, 2021
Shaohuai Shi, Lin Zhang, Bo Li

Figure 1 for Accelerating Distributed K-FAC with Smart Parallelism of Computing and Communication Tasks
Figure 2 for Accelerating Distributed K-FAC with Smart Parallelism of Computing and Communication Tasks
Figure 3 for Accelerating Distributed K-FAC with Smart Parallelism of Computing and Communication Tasks
Figure 4 for Accelerating Distributed K-FAC with Smart Parallelism of Computing and Communication Tasks
* 11 pages. Accepted to IEEE ICDCS 2021 
  Access Paper or Ask Questions

Automated Model Design and Benchmarking of 3D Deep Learning Models for COVID-19 Detection with Chest CT Scans


Feb 12, 2021
Xin He, Shihao Wang, Xiaowen Chu, Shaohuai Shi, Jiangping Tang, Xin Liu, Chenggang Yan, Jiyong Zhang, Guiguang Ding

Figure 1 for Automated Model Design and Benchmarking of 3D Deep Learning Models for COVID-19 Detection with Chest CT Scans
Figure 2 for Automated Model Design and Benchmarking of 3D Deep Learning Models for COVID-19 Detection with Chest CT Scans
Figure 3 for Automated Model Design and Benchmarking of 3D Deep Learning Models for COVID-19 Detection with Chest CT Scans
Figure 4 for Automated Model Design and Benchmarking of 3D Deep Learning Models for COVID-19 Detection with Chest CT Scans
* Accepted by AAAI 2021, COVID-19, Neural Architecture Search, AutoML 
  Access Paper or Ask Questions

Towards Scalable Distributed Training of Deep Learning on Public Cloud Clusters


Oct 20, 2020
Shaohuai Shi, Xianhao Zhou, Shutao Song, Xingyao Wang, Zilin Zhu, Xue Huang, Xinan Jiang, Feihu Zhou, Zhenyu Guo, Liqiang Xie, Rui Lan, Xianbin Ouyang, Yan Zhang, Jieqian Wei, Jing Gong, Weiliang Lin, Ping Gao, Peng Meng, Xiaomin Xu, Chenyang Guo, Bo Yang, Zhibo Chen, Yongjian Wu, Xiaowen Chu

Figure 1 for Towards Scalable Distributed Training of Deep Learning on Public Cloud Clusters
Figure 2 for Towards Scalable Distributed Training of Deep Learning on Public Cloud Clusters
Figure 3 for Towards Scalable Distributed Training of Deep Learning on Public Cloud Clusters
Figure 4 for Towards Scalable Distributed Training of Deep Learning on Public Cloud Clusters
* 13 pages 
  Access Paper or Ask Questions

Communication-Efficient Distributed Deep Learning: Survey, Evaluation, and Challenges


May 27, 2020
Shaohuai Shi, Zhenheng Tang, Xiaowen Chu, Chengjian Liu, Wei Wang, Bo Li

Figure 1 for Communication-Efficient Distributed Deep Learning: Survey, Evaluation, and Challenges
Figure 2 for Communication-Efficient Distributed Deep Learning: Survey, Evaluation, and Challenges
Figure 3 for Communication-Efficient Distributed Deep Learning: Survey, Evaluation, and Challenges
Figure 4 for Communication-Efficient Distributed Deep Learning: Survey, Evaluation, and Challenges
* 10 pages, 6 figures & tables. Code at: https://github.com/HKBU-HPML/ddl-benchmarks 
  Access Paper or Ask Questions

FADNet: A Fast and Accurate Network for Disparity Estimation


Mar 24, 2020
Qiang Wang, Shaohuai Shi, Shizhen Zheng, Kaiyong Zhao, Xiaowen Chu

Figure 1 for FADNet: A Fast and Accurate Network for Disparity Estimation
Figure 2 for FADNet: A Fast and Accurate Network for Disparity Estimation
Figure 3 for FADNet: A Fast and Accurate Network for Disparity Estimation
Figure 4 for FADNet: A Fast and Accurate Network for Disparity Estimation

  Access Paper or Ask Questions

Communication-Efficient Distributed Deep Learning: A Comprehensive Survey


Mar 10, 2020
Zhenheng Tang, Shaohuai Shi, Xiaowen Chu, Wei Wang, Bo Li

Figure 1 for Communication-Efficient Distributed Deep Learning: A Comprehensive Survey
Figure 2 for Communication-Efficient Distributed Deep Learning: A Comprehensive Survey
Figure 3 for Communication-Efficient Distributed Deep Learning: A Comprehensive Survey
Figure 4 for Communication-Efficient Distributed Deep Learning: A Comprehensive Survey

  Access Paper or Ask Questions

Communication Contention Aware Scheduling of Multiple Deep Learning Training Jobs


Feb 24, 2020
Qiang Wang, Shaohuai Shi, Canhui Wang, Xiaowen Chu

Figure 1 for Communication Contention Aware Scheduling of Multiple Deep Learning Training Jobs
Figure 2 for Communication Contention Aware Scheduling of Multiple Deep Learning Training Jobs
Figure 3 for Communication Contention Aware Scheduling of Multiple Deep Learning Training Jobs
Figure 4 for Communication Contention Aware Scheduling of Multiple Deep Learning Training Jobs

  Access Paper or Ask Questions

Communication-Efficient Decentralized Learning with Sparsification and Adaptive Peer Selection


Feb 22, 2020
Zhenheng Tang, Shaohuai Shi, Xiaowen Chu

Figure 1 for Communication-Efficient Decentralized Learning with Sparsification and Adaptive Peer Selection
Figure 2 for Communication-Efficient Decentralized Learning with Sparsification and Adaptive Peer Selection
Figure 3 for Communication-Efficient Decentralized Learning with Sparsification and Adaptive Peer Selection
Figure 4 for Communication-Efficient Decentralized Learning with Sparsification and Adaptive Peer Selection

  Access Paper or Ask Questions

MG-WFBP: Merging Gradients Wisely for Efficient Communication in Distributed Deep Learning


Dec 18, 2019
Shaohuai Shi, Xiaowen Chu, Bo Li

Figure 1 for MG-WFBP: Merging Gradients Wisely for Efficient Communication in Distributed Deep Learning
Figure 2 for MG-WFBP: Merging Gradients Wisely for Efficient Communication in Distributed Deep Learning
Figure 3 for MG-WFBP: Merging Gradients Wisely for Efficient Communication in Distributed Deep Learning
Figure 4 for MG-WFBP: Merging Gradients Wisely for Efficient Communication in Distributed Deep Learning
* Under review as a journal paper. 14 pages. arXiv admin note: substantial text overlap with arXiv:1811.11141 
  Access Paper or Ask Questions

Layer-wise Adaptive Gradient Sparsification for Distributed Deep Learning with Convergence Guarantees


Nov 21, 2019
Shaohuai Shi, Zhenheng Tang, Qiang Wang, Kaiyong Zhao, Xiaowen Chu

Figure 1 for Layer-wise Adaptive Gradient Sparsification for Distributed Deep Learning with Convergence Guarantees
Figure 2 for Layer-wise Adaptive Gradient Sparsification for Distributed Deep Learning with Convergence Guarantees
Figure 3 for Layer-wise Adaptive Gradient Sparsification for Distributed Deep Learning with Convergence Guarantees
Figure 4 for Layer-wise Adaptive Gradient Sparsification for Distributed Deep Learning with Convergence Guarantees
* 11 pages 
  Access Paper or Ask Questions

Understanding Top-k Sparsification in Distributed Deep Learning


Nov 20, 2019
Shaohuai Shi, Xiaowen Chu, Ka Chun Cheung, Simon See

Figure 1 for Understanding Top-k Sparsification in Distributed Deep Learning
Figure 2 for Understanding Top-k Sparsification in Distributed Deep Learning
Figure 3 for Understanding Top-k Sparsification in Distributed Deep Learning
Figure 4 for Understanding Top-k Sparsification in Distributed Deep Learning
* 14 pages 
  Access Paper or Ask Questions

Computer-Aided Clinical Skin Disease Diagnosis Using CNN and Object Detection Models


Nov 20, 2019
Xin He, Shihao Wang, Shaohuai Shi, Zhenheng Tang, Yuxin Wang, Zhihao Zhao, Jing Dai, Ronghao Ni, Xiaofeng Zhang, Xiaoming Liu, Zhili Wu, Wu Yu, Xiaowen Chu

Figure 1 for Computer-Aided Clinical Skin Disease Diagnosis Using CNN and Object Detection Models
Figure 2 for Computer-Aided Clinical Skin Disease Diagnosis Using CNN and Object Detection Models
Figure 3 for Computer-Aided Clinical Skin Disease Diagnosis Using CNN and Object Detection Models
Figure 4 for Computer-Aided Clinical Skin Disease Diagnosis Using CNN and Object Detection Models
* KDDBHI Workshop 2019, IEEE BigData Conference 
  Access Paper or Ask Questions

Performance and Power Evaluation of AI Accelerators for Training Deep Learning Models


Sep 22, 2019
Yuxin Wang, Qiang Wang, Shaohuai Shi, Xin He, Zhenheng Tang, Kaiyong Zhao, Xiaowen Chu

Figure 1 for Performance and Power Evaluation of AI Accelerators for Training Deep Learning Models
Figure 2 for Performance and Power Evaluation of AI Accelerators for Training Deep Learning Models
Figure 3 for Performance and Power Evaluation of AI Accelerators for Training Deep Learning Models
Figure 4 for Performance and Power Evaluation of AI Accelerators for Training Deep Learning Models
* There are some technical mistakes, and need to be carefully revised. So we choose to withdraw 
  Access Paper or Ask Questions

A Distributed Synchronous SGD Algorithm with Global Top-$k$ Sparsification for Low Bandwidth Networks


Jan 14, 2019
Shaohuai Shi, Qiang Wang, Kaiyong Zhao, Zhenheng Tang, Yuxin Wang, Xiang Huang, Xiaowen Chu

Figure 1 for A Distributed Synchronous SGD Algorithm with Global Top-$k$ Sparsification for Low Bandwidth Networks
Figure 2 for A Distributed Synchronous SGD Algorithm with Global Top-$k$ Sparsification for Low Bandwidth Networks
Figure 3 for A Distributed Synchronous SGD Algorithm with Global Top-$k$ Sparsification for Low Bandwidth Networks
Figure 4 for A Distributed Synchronous SGD Algorithm with Global Top-$k$ Sparsification for Low Bandwidth Networks
* 12 pages 
  Access Paper or Ask Questions

Highly Scalable Deep Learning Training System with Mixed-Precision: Training ImageNet in Four Minutes


Jul 30, 2018
Xianyan Jia, Shutao Song, Wei He, Yangzihao Wang, Haidong Rong, Feihu Zhou, Liqiang Xie, Zhenyu Guo, Yuanzhou Yang, Liwei Yu, Tiegang Chen, Guangxiao Hu, Shaohuai Shi, Xiaowen Chu

Figure 1 for Highly Scalable Deep Learning Training System with Mixed-Precision: Training ImageNet in Four Minutes
Figure 2 for Highly Scalable Deep Learning Training System with Mixed-Precision: Training ImageNet in Four Minutes
Figure 3 for Highly Scalable Deep Learning Training System with Mixed-Precision: Training ImageNet in Four Minutes
* arXiv admin note: text overlap with arXiv:1803.03383 by other authors 
  Access Paper or Ask Questions

Performance Evaluation of Deep Learning Tools in Docker Containers


Nov 09, 2017
Pengfei Xu, Shaohuai Shi, Xiaowen Chu

Figure 1 for Performance Evaluation of Deep Learning Tools in Docker Containers
Figure 2 for Performance Evaluation of Deep Learning Tools in Docker Containers
Figure 3 for Performance Evaluation of Deep Learning Tools in Docker Containers
Figure 4 for Performance Evaluation of Deep Learning Tools in Docker Containers
* Conference: BIgCom2017, 9 pages 
  Access Paper or Ask Questions

Speeding up Convolutional Neural Networks By Exploiting the Sparsity of Rectifier Units


May 15, 2017
Shaohuai Shi, Xiaowen Chu

Figure 1 for Speeding up Convolutional Neural Networks By Exploiting the Sparsity of Rectifier Units
Figure 2 for Speeding up Convolutional Neural Networks By Exploiting the Sparsity of Rectifier Units
Figure 3 for Speeding up Convolutional Neural Networks By Exploiting the Sparsity of Rectifier Units
Figure 4 for Speeding up Convolutional Neural Networks By Exploiting the Sparsity of Rectifier Units
* 7 pages 
  Access Paper or Ask Questions

Benchmarking State-of-the-Art Deep Learning Software Tools


Feb 17, 2017
Shaohuai Shi, Qiang Wang, Pengfei Xu, Xiaowen Chu

Figure 1 for Benchmarking State-of-the-Art Deep Learning Software Tools
Figure 2 for Benchmarking State-of-the-Art Deep Learning Software Tools
Figure 3 for Benchmarking State-of-the-Art Deep Learning Software Tools
Figure 4 for Benchmarking State-of-the-Art Deep Learning Software Tools
* Revision history: 1. Revise ResNet-50 configuration in MXNet. 2. Add faster implementation of ResNet-56 in TensorFlow with multiple GPUs 
  Access Paper or Ask Questions