Mobile robots operating in outdoor environments frequently encounter the issue of undesired traces left by dynamic objects and manifested as obstacles on the map, impeding the robot's ability to achieve accurate localization and navigation performance. To address this problem, we present a novel online map construction framework called RH-Map. Our framework leverages a newly proposed 3D region-wise hash map data structure for efficiently removing dynamic objects in real-time. It comprises a real-time dynamic object removal front-end module S2M-R and a lightweight back-end module for further removal. We conducted extensive experiments on the SemanticKITTI dataset, and the results demonstrate that our proposed method performs favorably compared to state-of-the-art approaches, and we further validated the proposed framework in real-world environment. The source code is released and available for the community.