Over-the-Air Federated Learning Over MIMO Channels: A Sparse-Coded Multiplexing Approach

The communication bottleneck of over-the-air federated learning (OA-FL) lies in uploading the gradients of local learning models. In this paper, we study the reduction of the communication overhead in the gradients uploading by using the multiple-input multiple-output (MIMO) technique. We propose a novel sparse-coded multiplexing (SCoM) approach that employs sparse-coding compression and MIMO multiplexing to balance the communication overhead and the learning performance of the FL model. We derive an upper bound on the learning performance loss of the SCoM-based MIMO OA-FL scheme by quantitatively characterizing the gradient aggregation error. Based on the analysis results, we show that the optimal number of multiplexed data streams to minimize the upper bound on the FL learning performance loss is given by the minimum of the numbers of transmit and receive antennas. We then formulate an optimization problem for the design of precoding and post-processing matrices to minimize the gradient aggregation error. To solve this problem, we develop a low-complexity algorithm based on alternating optimization (AO) and alternating direction method of multipliers (ADMM), which effectively mitigates the impact of the gradient aggregation error. Numerical results demonstrate the superb performance of the proposed SCoM approach.

UAV-Assisted Hierarchical Aggregation for Over-the-Air Federated Learning

With huge amounts of data explosively increasing in the mobile edge, over-the-air federated learning (OA-FL) emerges as a promising technique to reduce communication costs and privacy leak risks. However, when devices in a relatively large area cooperatively train a machine learning model, the attendant straggler issues will significantly reduce the learning performance. In this paper, we propose an unmanned aerial vehicle (UAV) assisted OA-FL system, where the UAV acts as a parameter server (PS) to aggregate the local gradients hierarchically for global model updating. Under this UAV-assisted hierarchical aggregation scheme, we carry out a gradient-correlation-aware FL performance analysis. We then formulate a mean squared error (MSE) minimization problem to tune the UAV trajectory and the global aggregation coefficients based on the analysis results. An algorithm based on alternating optimization (AO) and successive convex approximation (SCA) is developed to solve the formulated problem. Simulation results demonstrate the great potential of our UAV-assisted hierarchical aggregation scheme.

Over-the-Air Multi-Task Federated Learning Over MIMO Interference Channel

With the explosive growth of data and wireless devices, federated learning (FL) has emerged as a promising technology for large-scale intelligent systems. Utilizing the analog superposition of electromagnetic waves, over-the-air computation is an appealing approach to reduce the burden of communication in the FL model aggregation. However, with the urgent demand for intelligent systems, the training of multiple tasks with over-the-air computation further aggravates the scarcity of communication resources. This issue can be alleviated to some extent by training multiple tasks simultaneously with shared communication resources, but the latter inevitably brings about the problem of inter-task interference. In this paper, we study over-the-air multi-task FL (OA-MTFL) over the multiple-input multiple-output (MIMO) interference channel. We propose a novel model aggregation method for the alignment of local gradients for different devices, which alleviates the straggler problem that exists widely in over-the-air computation due to the channel heterogeneity. We establish a unified communication-computation analysis framework for the proposed OA-MTFL scheme by considering the spatial correlation between devices, and formulate an optimization problem of designing transceiver beamforming and device selection. We develop an algorithm by using alternating optimization (AO) and fractional programming (FP) to solve this problem, which effectively relieves the impact of inter-task interference on the FL learning performance. We show that due to the use of the new model aggregation method, device selection is no longer essential to our scheme, thereby avoiding the heavy computational burden caused by implementing device selection. The numerical results demonstrate the correctness of the analysis and the outstanding performance of the proposed scheme.