Face recognition system is one of the esteemed research areas in pattern recognition and computer vision as long as its major challenges. A few challenges in recognizing faces are blur, illumination, and varied expressions. Blur is natural while taking photographs using cameras, mobile phones, etc. Blur can be uniform and non-uniform. Usually non-uniform blur happens in images taken using handheld image devices. Distinguishing or handling a blurred image in a face recognition system is generally tough. Under varying lighting conditions, it is challenging to identify the person correctly. Diversified facial expressions such as happiness, sad, surprise, fear, anger changes or deforms the faces from normal images. Identifying faces with facial expressions is also a challenging task, due to the deformation caused by the facial expressions. To solve these issues, a pre-processing step was carried out after which Blur and Illumination-Robust Face recognition (BIRFR) algorithm was performed. The test image and training images with facial expression are transformed to neutral face using Facial expression removal (FER) peration. Every training image is transformed based on the optimal Transformation Spread Function (TSF), and illumination coefficients. Local Binary Pattern (LBP) features extracted from test image and transformed training image is used for classification.