This paper mainly studies the localization and mapping of range sensing robots in the confidence-rich map (CRM), a dense environmental representation with continuous belief, and then extends to information-theoretic exploration to reduce the pose uncertainty. Most previous works about active simultaneous localization and mapping (SLAM) and exploration always assumed the known robot poses or utilized inaccurate information metrics to approximate pose uncertainty, resulting in imbalanced exploration performance and efficiency in the unknown environment. This inspires us to extend the confidence-rich mutual information (CRMI) with measurable pose uncertainty. Specifically, we propose a Rao-Blackwellized particle filter-based localization and mapping scheme (RBPF-CLAM) for CRMs, then we develop a new closed-form weighting method to improve the localization accuracy without scan matching. We further compute the uncertain CRMI (UCRMI) with the weighted particles by a more accurate approximation. Simulations and experimental evaluations show the localization accuracy and exploration performance of the proposed methods in unstructured and confined scenes.