A Practical Data-Free Approach to One-shot Federated Learning with Heterogeneity

One-shot Federated Learning (FL) has recently emerged as a promising approach, which allows the central server to learn a model in a single communication round. Despite the low communication cost, existing one-shot FL methods are mostly impractical or face inherent limitations, e.g., a public dataset is required, clients' models are homogeneous, need to upload additional data/model information. To overcome these issues, we propose a more practical data-free approach named FedSyn for one-shot FL framework with heterogeneity. Our FedSyn trains the global model by a data generation stage and a model distillation stage. To the best of our knowledge, FedSyn is the first method that can be practically applied to various real-world applications due to the following advantages: (1) FedSyn requires no additional information (except the model parameters) to be transferred between clients and the server; (2) FedSyn does not require any auxiliary dataset for training; (3) FedSyn is the first to consider both model and statistical heterogeneities in FL, i.e., the clients' data are non-iid and different clients may have different model architectures. Experiments on a variety of real-world datasets demonstrate the superiority of our FedSyn. For example, FedSyn outperforms the best baseline method Fed-ADI by 5.08% on CIFAR10 dataset when data are non-iid.