Constellation Design and Detection under Generalized Hardware Impairments

This paper presents a maximum-likelihood detection framework that jointly mitigates hardware (HW) impairments in both amplitude and phase. By modeling transceiver distortions as residual amplitude and phase noise, we introduce the approximate phase-and-amplitude distortion detector (PAD-D), which operates in the polar domain and effectively mitigates both distortion components through distortion-aware weighting. The proposed detector performs reliable detection under generalized HW impairment conditions, achieving substantial performance gains over the conventional Euclidean detector (EUC-D) and the Gaussian-assumption phase noise detector (GAP-D), which is primarily designed to address phase distortions. In addition, we derive a closed-form high-SNR symbol error probability (SEP) approximation, which offers a generic analytical expression applicable to arbitrary constellations. Simulation results demonstrate that the PAD-D achieves up to an order-of-magnitude reduction in the error floor relative to EUC-D and GAP-D for both high-order quadrature amplitude modulation (QAM) and super amplitude phase-shift keying (SAPSK) constellations, establishing a unified and practical framework for detection under realistic transceiver impairments. Building on this framework, we further develop optimized constellations tailored to PAD-D, where the symbol positions are optimized in the complex plane to minimize SEP. The optimality of these constellations is confirmed through extensive simulations, which also verify the accuracy of the proposed analytical SEP approximation, even for the optimized designs.