Skin cancer is one of the most threatening diseases worldwide. However, diagnosing a skin cancer correctly is challenging. Recently, deep learning algorithms have achieved excellent performance on various tasks. Particularly, they have been also implemented for the tasks of skin disease diagnosis. In this paper, we present a review on deep learning methods and their applications in skin disease diagnosis. We first introduce skin diseases and image acquisition methods in dermatology, and list several publicly available datasets for training and testing algorithms for skin disease diagnosis. Then, we introduce the conception of deep learning and review popular deep learning architectures. Thereafter, popular deep learning frameworks that facilitate the implementation of deep learning algorithms and performance evaluation metrics are presented. As an important part of this article, we then review the literatures involving deep learning methods for skin disease diagnosis from several aspects according to the specific tasks. Additionally, we discuss the challenges faced in the area of skin disease diagnosis with deep learning and suggest possible future research directions. Finally, we summarize the article. The major purpose of this article is to provide a conceptual and systematically review of the recent works on skin disease diagnosis with deep learning. Given the popularity of deep learning, there remains great challenges in the area, as well as opportunities that we can explore in the future.