Reconfigurable intelligent surface (RIS) has recently gained popularity as a promising solution for improving the signal transmission quality of wireless communications with less hardware cost and energy consumption. This letter offers a novel deep reinforcement learning (DRL) algorithm based on a location-aware imitation environment for the joint beamforming design in an RIS-aided mmWave multiple-input multiple-output system. Specifically, we design a neural network to imitate the transmission environment based on the geometric relationship between the user's location and the mmWave channel. Following this, a novel DRL-based method is developed that interacts with the imitation environment using the easily available location information. Finally, simulation results demonstrate that the proposed DRL-based algorithm provides more robust performance without excessive interaction overhead compared to the existing DRL-based approaches.