Beamforming for Massive MIMO Aerial Communications: A Robust and Scalable DRL Approach

This paper presents a distributed beamforming framework for a constellation of airborne platform stations (APSs) in a massive Multiple-Input and Multiple-Output (MIMO) non-terrestrial network (NTN) that targets the downlink sum-rate maximization under imperfect local channel state information (CSI). We propose a novel entropy-based multi-agent deep reinforcement learning (DRL) approach where each non-terrestrial base station (NTBS) independently computes its beamforming vector using a Fourier Neural Operator (FNO) to capture long-range dependencies in the frequency domain. To ensure scalability and robustness, the proposed framework integrates transfer learning based on a conjugate prior mechanism and a low-rank decomposition (LRD) technique, thus enabling efficient support for large-scale user deployments and aerial layers. Our simulation results demonstrate the superiority of the proposed method over baseline schemes including WMMSE, ZF, MRT, CNN-based DRL, and the deep deterministic policy gradient (DDPG) method in terms of average sum rate, robustness to CSI imperfection, user mobility, and scalability across varying network sizes and user densities. Furthermore, we show that the proposed method achieves significant computational efficiency compared to CNN-based and WMMSE methods, while reducing communication overhead in comparison with shared-critic DRL approaches.