Alert button

Federated Learning with Unbiased Gradient Aggregation and Controllable Meta Updating

Oct 18, 2019
Xin Yao, Tianchi Huang, Rui-Xiao Zhang, Ruiyu Li, Lifeng Sun

Figure 1 for Federated Learning with Unbiased Gradient Aggregation and Controllable Meta Updating
Figure 2 for Federated Learning with Unbiased Gradient Aggregation and Controllable Meta Updating
Figure 3 for Federated Learning with Unbiased Gradient Aggregation and Controllable Meta Updating
Figure 4 for Federated Learning with Unbiased Gradient Aggregation and Controllable Meta Updating

Share this with someone who'll enjoy it:

Federated Averaging (FedAvg) serves as the fundamental framework in Federated Learning (FL) settings. However, we argue that 1) the multiple steps of local updating will result in gradient biases and 2) there is an inconsistency between the target distribution and the optimization objectives following the training paradigm in FedAvg. To tackle these problems, we first propose an unbiased gradient aggregation algorithm with the keep-trace gradient descent and gradient evaluation strategy. Then we introduce a meta updating procedure with a controllable meta training set to provide a clear and consistent optimization objective. Experimental results demonstrate that the proposed methods outperform compared ones with various network architectures in both the IID and non-IID FL settings.

* This manuscript has been accepted to the Workshop on Federated Learning for Data Privacy and Confidentiality (FL - NeurIPS 2019, in Conjunction with NeurIPS 2019)  
View paper onarxiv icon

Share this with someone who'll enjoy it: