In this paper, a high performance face recognition system based on local binary pattern (LBP) using the probability distribution functions (PDF) of pixels in different mutually independent color channels which are robust to frontal homogenous illumination and planer rotation is proposed. The illumination of faces is enhanced by using the state-of-the-art technique which is using discrete wavelet transform (DWT) and singular value decomposition (SVD). After equalization, face images are segmented by use of local Successive Mean Quantization Transform (SMQT) followed by skin color based face detection system. Kullback-Leibler Distance (KLD) between the concatenated PDFs of a given face obtained by LBP and the concatenated PDFs of each face in the database is used as a metric in the recognition process. Various decision fusion techniques have been used in order to improve the recognition rate. The proposed system has been tested on the FERET, HP, and Bosphorus face databases. The proposed system is compared with conventional and thestate-of-the-art techniques. The recognition rates obtained using FVF approach for FERET database is 99.78% compared with 79.60% and 68.80% for conventional gray scale LBP and Principle Component Analysis (PCA) based face recognition techniques respectively.