Alert button

FedNNNN: Norm-Normalized Neural Network Aggregation for Fast and Accurate Federated Learning

Aug 11, 2020
Kenta Nagura, Song Bian, Takashi Sato

Figure 1 for FedNNNN: Norm-Normalized Neural Network Aggregation for Fast and Accurate Federated Learning
Figure 2 for FedNNNN: Norm-Normalized Neural Network Aggregation for Fast and Accurate Federated Learning
Figure 3 for FedNNNN: Norm-Normalized Neural Network Aggregation for Fast and Accurate Federated Learning
Figure 4 for FedNNNN: Norm-Normalized Neural Network Aggregation for Fast and Accurate Federated Learning

Share this with someone who'll enjoy it:

Federated learning (FL) is a distributed learning protocol in which a server needs to aggregate a set of models learned some independent clients to proceed the learning process. At present, model averaging, known as FedAvg, is one of the most widely adapted aggregation techniques. However, it is known to yield the models with degraded prediction accuracy and slow convergence. In this work, we find out that averaging models from different clients significantly diminishes the norm of the update vectors, resulting in slow learning rate and low prediction accuracy. Therefore, we propose a new aggregation method called FedNNNN. Instead of simple model averaging, we adjust the norm of the update vector and introduce momentum control techniques to improve the aggregation effectiveness of FL. As a demonstration, we evaluate FedNNNN on multiple datasets and scenarios with different neural network models, and observe up to 5.4% accuracy improvement.

View paper onarxiv icon

Share this with someone who'll enjoy it: