Computer-aided diagnosis systems for classification of different type of skin lesions have been an active field of research in recent decades. It has been shown that introducing lesions and their attributes masks into lesion classification pipeline can greatly improve the performance. In this paper, we propose a framework by incorporating transfer learning for segmenting lesions and their attributes based on the convolutional neural networks. The proposed framework is inspired by the well-known UNet architecture. It utilizes a variety of pre-trained networks in the encoding path and generates the prediction map by combining multi-scale information in decoding path using a pyramid pooling manner. To circumvent the lack of training data and increase the proposed model generalization, an extensive set of novel augmentation routines have been applied during the training of the network. Moreover, for each task of lesion and attribute segmentation, a specific loss function has been designed to obviate the training phase difficulties. Finally, the prediction for each task is generated by ensembling the outputs from different models. The proposed approach achieves promising results on the cross-validation experiments on the ISIC2018- Task1 and Task2 data sets.