For intelligent reflecting surface (IRS) aided downlink communication in frequency division duplex (FDD) systems, the overhead for the base station (BS) to acquire channel state information (CSI) is extremely high under the conventional ``estimate-then-quantize'' scheme, where the users first estimate and then feed back their channels to the BS. Recently, [1] revealed a strong correlation in different users' cascaded channels stemming from their common BS-IRS channel component, and leveraged such a correlation to significantly reduce the pilot transmission overhead in IRS-aided uplink communication. In this paper, we aim to exploit the above channel property for reducing the overhead of both pilot transmission and feedback transmission in IRS-aided downlink communication. Different from the uplink counterpart where the BS possesses the pilot signals containing the CSI of all the users, in downlink communication, the distributed users merely receive the pilot signals containing their own CSI and cannot leverage the correlation in different users' channels revealed in [1]. To tackle this challenge, this paper proposes a novel ``quantize-then-estimate'' protocol in FDD IRS-aided downlink communication. Specifically, the users first quantize their received pilot signals, instead of the channels estimated from the pilot signals, and then transmit the quantization bits to the BS. After de-quantizing the pilot signals received by all the users, the BS estimates all the cascaded channels by leveraging the correlation embedded in them, similar to the uplink scenario. Furthermore, we manage to show both analytically and numerically the great overhead reduction in terms of pilot transmission and feedback transmission arising from our proposed ``quantize-then-estimate'' protocol.