The segmentation of drivable areas and road anomalies are critical capabilities to achieve autonomous navigation for robotic wheelchairs. The recent progress of semantic segmentation using deep learning techniques has presented effective results. However, the acquisition of large-scale datasets with hand-labeled ground truth is time-consuming and labor-intensive, making the deep learning-based methods often hard to implement in practice. We contribute to the solution of this problem for the task of drivable area and road anomaly segmentation by proposing a self-supervised learning approach. We develop a pipeline that can automatically generate segmentation labels for drivable areas and road anomalies. Then, we train RGB-D data-based semantic segmentation neural networks and get predicted labels. Experimental results show that our proposed automatic labeling pipeline achieves an impressive speed-up compared to manual labeling. In addition, our proposed self-supervised approach exhibits more robust and accurate results than the state-of-the-art traditional algorithms as well as the state-of-the-art self-supervised algorithms.