Energy efficiency is critical for future sustainable cellular systems. Power saving optimization has been a key part of the fifth generation (5G) new radio specifications. For 5Gadvanced and future 6G, with the anticipation of a trillion internet of things (IoTs) devices with non-rechargeable or low-density batteries, device power efficiency is rather essential. There are numerous contributions from industry and academia which present the potential power saving gains of the various 5G power saving techniques; however, there is a lack of art on the performance cost paid to achieve such power saving gains. Therefore, this paper presents a comprehensive evaluation of the radio latency and reliability cost, which is lost due to a certain 5G new radio power saving feature. A thorough review of the state of-the-art 5G power saving techniques is introduced. Extensive system level simulations are performed to evaluate the latency and reliability cost of the considered power saving features. The paper offers valuable recommendations for supporting power efficient latency-critical traffic for beyond 5G-advanced systems.