Automatic detection of shadow regions in an image is a difficult task due to the lack of prior information about the illumination source and the dynamic of the scene objects. To address this problem, in this paper, a deep-learning based segmentation method is proposed that identifies shadow regions at the pixel-level in a single RGB image. We exploit a novel Convolutional Neural Network (CNN) architecture to identify and extract shadow features in an end-to-end manner. This network preserves learned contexts during the training and observes the entire image to detect global and local shadow patterns simultaneously. The proposed method is evaluated on two publicly available datasets of SBU and UCF. We have improved the state-of-the-art Balanced Error Rate (BER) on these datasets by 22\% and 14\%, respectively.