Intelligent reflecting surface (IRS)-aided communication is a promising technology for beyond 5G (B5G) systems, to reconfigure the radio environment proactively. However, IRS-aided communication in practice requires efficient channel estimation or passive beam training, whose overhead and complexity increase drastically with the number of reflecting elements/beam directions. To tackle this challenge, we propose in this paper a novel environment-aware joint active and passive beam selection scheme for IRS-aided wireless communication, based on the new concept of channel knowledge map (CKM). Specifically, by utilizing both the location information of the user equipment (UE), which is readily available in contemporary wireless systems with ever-increasing accuracy, and the environment information offered by CKM, the proposed scheme achieves efficient beam selection with either no real-time training required (training-free beam selection) or only moderate training overhead (light-training beam selection). Numerical results based on practical channels obtained using commercial ray tracing software are presented, which demonstrate the superior performance of the proposed scheme over various benchmark schemes.