Hybrid Beamforming Optimization for MIMO ISAC Exploiting Prior Information: A PCRB-based Approach

This paper considers a multiple-input multiple-output (MIMO) integrated sensing and communication (ISAC) system, where a multi-antenna base station (BS) with transceiver hybrid analog-digital arrays transmits dual-functional signals to communicate with a multi-antenna user and simultaneously sense the unknown and random location information of a target based on the reflected echo signals and the prior distribution information on the target's location. Under transceiver hybrid arrays, we characterize the sensing performance by deriving the posterior Cram\'{e}r-Rao bound (PCRB) of the mean-squared error which is a function of the transmit hybrid beamforming and receive analog beamforming. We study joint transmit hybrid beamforming and receive analog beamforming optimization to minimize the PCRB subject to a communication rate requirement. We first consider a sensing-only system and derive the optimal solution to each element in the transmit/receive analog beamforming matrices that minimizes the PCRB in closed form. Then, we develop an alternating optimization (AO) based algorithm. Next, we study a narrowband MIMO ISAC system and devise an efficient AO-based hybrid beamforming algorithm by leveraging weighted minimum mean-squared error and feasible point pursuit successive convex approximation methods. Furthermore, we extend the results for narrowband systems to a MIMO orthogonal frequency-division multiplexing (OFDM) ISAC system. Numerical results validate the effectiveness of our proposed hybrid beamforming designs. It is revealed that the number of receive RF chains has more significant impact on the sensing performance than its transmit counterpart. Under a given budget on the total number of transmit/receive RF chains at the BS, the optimal number of transmit RF chains increases as the communication rate target increases due to the non-trivial PCRB-rate trade-off.