Tri-Hybrid Holographic Beamforming for Integrated Sensing and Communication

Integrated sensing and communication (ISAC) can perform both communication and sensing tasks using the same frequency band and hardware, making it a key technology for 6G. As a low-cost implementation for large-scale antenna arrays, reconfigurable holographic surfaces (RHSs) can be integrated into ISAC systems to realize the holographic ISAC paradigm, where enlarged radiation apertures achieve significant beamforming gains. In this paper, we investigate the tri-hybrid holographic ISAC framework, where the beamformer comprises digital, analog, and RHS-based electromagnetic (EM) layers. The analog layer employs a small number of phase shifters (PSs) to provide subarray-level phase control for the amplitude-modulated RHSs. Tri-hybrid beamforming provides a pathway for low-cost large-scale holographic ISAC. However, compared to conventional ISAC systems, it is challenging to achieve joint subarray-level phase control via PSs and element-level radiation amplitude control via RHSs for holographic ISAC. To address this, we present a tri-hybrid holographic ISAC scheme that minimizes sensing waveform error while satisfying the minimum user rate requirement. A joint optimization approach for PS phases and RHS amplitude responses is designed to address inter-layer coupling and distinct feasible regions. Theoretical analyses reveal that the optimized amplitude responses cluster near boundary values, i.e., 1-bit amplitude control, to reduce hardware and algorithmic complexity. Simulation results show that the proposed scheme achieves a controllable performance trade-off between communication and sensing tasks. Measured RHS beam gain validates the enhancement of holographic beamforming through subarray-level phase shifting. Moreover, as the number of RHS elements increases, the proposed approach exceeds the performance of conventional hybrid beamforming while significantly reducing the number of PSs.