Deep candidate generation has become an increasingly popular choice deployed in many industrial search, recommendation and ads systems. Standard deep candidate generation models rely on sophisticated sampling techniques to approximately conduct maximum likelihood estimation of user-item interactions, following the advances in the language modeling community. However, it is unclear whether these sampling strategies are the best choice for candidate generation in recommender system, where we face severe selection bias in the training data with an extremely large candidate set and rich features of various types. In this paper, we propose CLRec, a Contrastive Learning paradigm for large scale candidate generation in Recommender systems. CLRec employs a queue based buffer to hold previous examples or representations as negative labels, within which a contrastive loss is optimized. This framework achieves better performance while requiring no explicit sampling, providing great efficiency in encoding rich types of features on the label side. We analyze both theoretically and empirically that CLRec can in fact alleviate the selection bias, leading to a more diversified and fairer recommendation. We deploy CLRec in Taobao and conduct online A/B test on a traffic-intensive scenario, showing a large margin improvement on both performance and efficiency, as well as a dramatic reduction on the rich-get-richer phenomenon.