This paper presents a continually self-improving lifelong learning framework for a mobile robot navigating in different environments. Classical static navigation methods require environment-specific in-situ system adjustment, e.g. from human experts, or may repeat their mistakes regardless of how many times they have navigated in the same environment. Having the potential to improve with experience, learning-based navigation is highly dependent on access to training resources, e.g. sufficient memory and fast computation, and is prone to forgetting previously learned capability, especially when facing different environments. In this work, we propose a lifelong learning framework for mobile robot navigation which (1) improves from its own experience without expert demonstrations, and (2) retains capability to navigate in previous environments after learning in new ones. This framework is implemented and tested entirely onboard a physical robot with a limited memory and computation budget.