Alert button
Picture for Zhenheng Tang

Zhenheng Tang

Alert button

FedImpro: Measuring and Improving Client Update in Federated Learning

Feb 10, 2024
Zhenheng Tang, Yonggang Zhang, Shaohuai Shi, Xinmei Tian, Tongliang Liu, Bo Han, Xiaowen Chu

Viaarxiv icon

FusionAI: Decentralized Training and Deploying LLMs with Massive Consumer-Level GPUs

Sep 03, 2023
Zhenheng Tang, Yuxin Wang, Xin He, Longteng Zhang, Xinglin Pan, Qiang Wang, Rongfei Zeng, Kaiyong Zhao, Shaohuai Shi, Bingsheng He, Xiaowen Chu

Figure 1 for FusionAI: Decentralized Training and Deploying LLMs with Massive Consumer-Level GPUs
Figure 2 for FusionAI: Decentralized Training and Deploying LLMs with Massive Consumer-Level GPUs
Figure 3 for FusionAI: Decentralized Training and Deploying LLMs with Massive Consumer-Level GPUs
Figure 4 for FusionAI: Decentralized Training and Deploying LLMs with Massive Consumer-Level GPUs
Viaarxiv icon

FedML Parrot: A Scalable Federated Learning System via Heterogeneity-aware Scheduling on Sequential and Hierarchical Training

Mar 03, 2023
Zhenheng Tang, Xiaowen Chu, Ryan Yide Ran, Sunwoo Lee, Shaohuai Shi, Yonggang Zhang, Yuxin Wang, Alex Qiaozhong Liang, Salman Avestimehr, Chaoyang He

Figure 1 for FedML Parrot: A Scalable Federated Learning System via Heterogeneity-aware Scheduling on Sequential and Hierarchical Training
Figure 2 for FedML Parrot: A Scalable Federated Learning System via Heterogeneity-aware Scheduling on Sequential and Hierarchical Training
Figure 3 for FedML Parrot: A Scalable Federated Learning System via Heterogeneity-aware Scheduling on Sequential and Hierarchical Training
Figure 4 for FedML Parrot: A Scalable Federated Learning System via Heterogeneity-aware Scheduling on Sequential and Hierarchical Training
Viaarxiv icon

NAS-LID: Efficient Neural Architecture Search with Local Intrinsic Dimension

Nov 24, 2022
Xin He, Jiangchao Yao, Yuxin Wang, Zhenheng Tang, Ka Chu Cheung, Simon See, Bo Han, Xiaowen Chu

Figure 1 for NAS-LID: Efficient Neural Architecture Search with Local Intrinsic Dimension
Figure 2 for NAS-LID: Efficient Neural Architecture Search with Local Intrinsic Dimension
Figure 3 for NAS-LID: Efficient Neural Architecture Search with Local Intrinsic Dimension
Figure 4 for NAS-LID: Efficient Neural Architecture Search with Local Intrinsic Dimension
Viaarxiv icon

Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learning

Jun 06, 2022
Zhenheng Tang, Yonggang Zhang, Shaohuai Shi, Xin He, Bo Han, Xiaowen Chu

Figure 1 for Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learning
Figure 2 for Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learning
Figure 3 for Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learning
Figure 4 for Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learning
Viaarxiv icon

FedCV: A Federated Learning Framework for Diverse Computer Vision Tasks

Nov 22, 2021
Chaoyang He, Alay Dilipbhai Shah, Zhenheng Tang, Di Fan1Adarshan Naiynar Sivashunmugam, Keerti Bhogaraju, Mita Shimpi, Li Shen, Xiaowen Chu, Mahdi Soltanolkotabi, Salman Avestimehr

Figure 1 for FedCV: A Federated Learning Framework for Diverse Computer Vision Tasks
Figure 2 for FedCV: A Federated Learning Framework for Diverse Computer Vision Tasks
Figure 3 for FedCV: A Federated Learning Framework for Diverse Computer Vision Tasks
Figure 4 for FedCV: A Federated Learning Framework for Diverse Computer Vision Tasks
Viaarxiv icon

Communication-Efficient Distributed Deep Learning: Survey, Evaluation, and Challenges

May 27, 2020
Shaohuai Shi, Zhenheng Tang, Xiaowen Chu, Chengjian Liu, Wei Wang, Bo Li

Figure 1 for Communication-Efficient Distributed Deep Learning: Survey, Evaluation, and Challenges
Figure 2 for Communication-Efficient Distributed Deep Learning: Survey, Evaluation, and Challenges
Figure 3 for Communication-Efficient Distributed Deep Learning: Survey, Evaluation, and Challenges
Figure 4 for Communication-Efficient Distributed Deep Learning: Survey, Evaluation, and Challenges
Viaarxiv icon

Communication-Efficient Distributed Deep Learning: A Comprehensive Survey

Mar 10, 2020
Zhenheng Tang, Shaohuai Shi, Xiaowen Chu, Wei Wang, Bo Li

Figure 1 for Communication-Efficient Distributed Deep Learning: A Comprehensive Survey
Figure 2 for Communication-Efficient Distributed Deep Learning: A Comprehensive Survey
Figure 3 for Communication-Efficient Distributed Deep Learning: A Comprehensive Survey
Figure 4 for Communication-Efficient Distributed Deep Learning: A Comprehensive Survey
Viaarxiv icon

Communication-Efficient Decentralized Learning with Sparsification and Adaptive Peer Selection

Feb 22, 2020
Zhenheng Tang, Shaohuai Shi, Xiaowen Chu

Figure 1 for Communication-Efficient Decentralized Learning with Sparsification and Adaptive Peer Selection
Figure 2 for Communication-Efficient Decentralized Learning with Sparsification and Adaptive Peer Selection
Figure 3 for Communication-Efficient Decentralized Learning with Sparsification and Adaptive Peer Selection
Figure 4 for Communication-Efficient Decentralized Learning with Sparsification and Adaptive Peer Selection
Viaarxiv icon

Layer-wise Adaptive Gradient Sparsification for Distributed Deep Learning with Convergence Guarantees

Nov 21, 2019
Shaohuai Shi, Zhenheng Tang, Qiang Wang, Kaiyong Zhao, Xiaowen Chu

Figure 1 for Layer-wise Adaptive Gradient Sparsification for Distributed Deep Learning with Convergence Guarantees
Figure 2 for Layer-wise Adaptive Gradient Sparsification for Distributed Deep Learning with Convergence Guarantees
Figure 3 for Layer-wise Adaptive Gradient Sparsification for Distributed Deep Learning with Convergence Guarantees
Figure 4 for Layer-wise Adaptive Gradient Sparsification for Distributed Deep Learning with Convergence Guarantees
Viaarxiv icon