In recent years, significant progress has been made in face recognition due to the availability of large-scale labeled face datasets. However, since the faces in these datasets usually contain limited degree and types of variation, the models trained on them generalize poorly to more realistic unconstrained face datasets. While collecting labeled faces with larger variations could be helpful, it is practically infeasible due to privacy and labor cost. In comparison, it is easier to acquire a large number of unlabeled faces from different domains which would better represent the testing scenarios in real-world problems. We present an approach to use such unlabeled faces to learn generalizable face representations, which can be viewed as an unsupervised domain generalization framework. Experimental results on unconstrained datasets show that a small amount of unlabeled data with sufficient diversity can (i) lead to an appreciable gain in recognition performance and (ii) outperform the supervised baseline when combined with less than half of the labeled data. Compared with the state-of-the-art face recognition methods, our method further improves their performance on challenging benchmarks, such as IJB-B, IJB-C and IJB-S.