Face recognition in real-time scenarios is mainly affected by illumination, expression and pose variations and also by occlusion. This paper presents the framework for pose adaptive component-based face recognition system. The framework proposed deals with all the above mentioned issues. The steps involved in the presented framework are (i) facial landmark localisation, (ii) facial component extraction, (iii) pre-processing of facial image (iv) facial pose estimation (v) feature extraction using Local Binary Pattern Histograms of each component followed by (vi) fusion of pose adaptive classification of components. By employing pose adaptive classification, the recognition process is carried out on some part of database, based on estimated pose, instead of applying the recognition process on the whole database. Pre-processing techniques employed to overcome the problems due to illumination variation are also discussed in this paper. Component-based techniques provide better recognition rates when face images are occluded compared to the holistic methods. Our method is simple, feasible and provides better results when compared to other holistic methods.