A recommender system recommends a few items for a user by sorting items according to their predicted preferences and filter items with the highest predicted preferences. While sorting and selecting top-K items are an inherent part of the personalized recommendation, it is nontrivial to incorporate them in the process of end-to-end model training since sorting is not differentiable and impossible to optimize with gradient based updates. Instead, existing recommenders optimize surrogate objectives, often rendering suboptimal quality of recommendations. In this paper, we propose the differentiable ranking metrics (DRM), a differentiable relaxation of evaluation metrics such as Precision and Recall. DRM maximizes the evaluation metrics for recommendation models directly. Via experiments with several real-world datasets, we demonstrate that the joint learning of the DRM cost function upon existing factor-based recommendation models improves the quality of recommendations significantly, in comparison with other state-of-the-art recommendation methods.