Recommender system recommends interesting items to users based on users' past information history. Researchers have been paying attention to improvement of algorithmic performance such as MAE and precision@K. Major techniques such as matrix factorization and learning to rank are optimized based on such evaluation metrics. However, the intrinsic Matthew Effect problem poses great threat to the fairness of the recommender system, and the unfairness problem cannot be resolved by optimization of traditional metrics. In this paper, we propose a novel algorithm that incorporates Matthew Effect reduction with the matrix factorization framework. We demonstrate that our approach can boost the fairness of the algorithm and enhances performance evaluated by traditional metrics.