Accurate pose estimation is a fundamental ability that all mobile robots must posses in order to traverse robustly in a given environment. Much like a human, this ability is dependent on the robot's understanding of a given scene. For Autonomous Vehicles (AV's), detailed 3D maps created beforehand are widely used to augment the perceptive abilities and estimate pose based on current sensor measurements. This approach however is less suited for rural communities that are sparsely connected and cover large areas. To deal with the challenge of localizing a vehicle in a rural setting, this paper presents a data-set of rural road scenes, along with an approach for fast segmentation of roads using LIDAR point clouds. The segmented point cloud in concert with road network information from Open Street Maps (OSM) is used for pose estimation. We propose two measurement models which are compared with state of the art methods for localization on OSM for tracking as well as global localization. The results show that the proposed algorithm is able to estimate pose within a 2 sq. km area with mean accuracy of 6.5 meters.