In the recent time deep learning has achieved huge popularity due to its performance in various machine learning algorithms. Deep learning as hierarchical or structured learning attempts to model high level abstractions in data by using a group of processing layers. The foundation of deep learning architectures is inspired by the understanding of information processing and neural responses in human brain. The architectures are created by stacking multiple linear or non-linear operations. The article mainly focuses on the state-of-art deep learning models and various real world applications specific training methods. Selecting optimal architecture for specific problem is a challenging task, at a closing stage of the article we proposed optimal approach to deep convolutional architecture for the application of image recognition.