Sub-array Selection Optimization for Joint Self-Interference and Multi-User Interference Suppression in FD mMIMO

This paper proposes a beamforming optimization scheme with joint antenna sub-array selection (SAS) and angular perturbation-based nulling (APN) for full-duplex (FD) massive multiple-input multiple-output (mMIMO) systems, to simultaneously suppress self-interference (SI) and multi-user interference (MUI). A comprehensive over-the-air SI channel measurement campaign, conducted with an 8x8Tx-8x8Rx FD array prototype, reveals significant variations across sub-arrays at different spatial locations, as well as reconfigurable characteristics of the SI channel under diverse Tx and Rx sub-array configurations. To exploit the selective SI channels, a particle swarm optimization (PSO)-based algorithm is developed to jointly determine optimal sub-array indices and perturbed steering angles, thereby effectively nullifying potential interference. Selecting sub-arrays with inherently lower SI channels notably enhances the beam-level isolation, while the added selection flexibility among comparable SI channels ensures more uniform SI suppression across diverse DL/UL locations and significantly improves worst-case isolation. Experimental evaluation based on the measured SI channel demonstrates that the proposed SAS technique achieves residual Tx-Rx beam-level SI suppression improvements of 29.2 dB and 26.6 dB for the sample 1x2 and 1x4 sub-arrays, respectively. A worst-case improvement greater than 30.7 dB is observed. Overall, the joint SAS and APN optimization scheme achieves average beam-level isolation of 85.2 dB and 83.3 dB with the 1x2 and 1x4 sub-arrays, respectively. With the application of a baseband precoder, all tested sub-array configurations achieve average MUI suppression better than -181.3 dB. These results confirm the potential of the proposed optimization algorithm to successfully reduce interference to the noise floor, thereby guaranteeing reliable FD mMIMO operation.