Recommender Systems are giving increasing importance to the beyond accuracy quality measures, and reliability is one of the most important in the Collaborative Filtering context. This paper proposes Bernoulli Matrix Factorization (BeMF), a matrix factorization model to provide both prediction values and reliability ones. This is a very innovative approach from several perspectives: a) it acts on the model-based Collaborative Filtering, rather than in the memory-based one, b) it does not use external methods or extended architectures for providing reliability such as the existing solutions, c) it is based on a classification-based model, instead of the usual regression-based ones, and d) the matrix factorization formalism is supported by the Bernoulli distribution, to exploit the binary nature of the designed classification model. As expected, results show that the more reliable a prediction is, the less liable to be wrong: recommendation quality has been improved by selecting the most reliable predictions. State-of-the-Art quality measures for reliability have been tested, showing improved results compared to the baseline methods and models.