The binary segmentation of roads in very high resolution (VHR) remote sensing images (RSIs) has always been a challenging task due to factors such as occlusions (caused by shadows, trees, buildings, etc.) and the intra-class variances of road surfaces. The wide use of convolutional neural networks (CNNs) has greatly improved the segmentation accuracy and made the task end-to-end trainable. However, there are still margins to improve in terms of the completeness and connectivity of the results. In this paper, we consider the specific context of road extraction and present a direction-aware residual network that includes three main contributions: 1) ResDec: an asymmetric residual segmentation network with deconvolutional layers and a structural supervision to enhance the learning of road topology; 2) a pixel-level supervision of local directions to enhance the embedding of linear features; 3) Refnet: a refinement network to optimize the segmentation results. An ablation study on the benchmark Massachusetts dataset has confirmed the effectiveness of the presented designs. Comparative experiments with other approaches show that the proposed method has advantages in both overall accuracy and F1-score.