The use of large-scale vision-language datasets is limited for object detection due to the negative impact of label noise on localization. Prior methods have shown how such large-scale datasets can be used for pretraining, which can provide initial signal for localization, but is insufficient without clean bounding-box data for at least some categories. We propose a technique to "vet" labels extracted from noisy captions. Our method trains a classifier that predicts if an extracted label is actually present in the image or not. Our classifier generalizes across dataset boundaries and shows promise for generalizing across categories as well. We compare the classifier to eleven baselines on five datasets, and demonstrate that it can improve weakly-supervised detection without label vetting by 80% (16.0 to 29.1 mAP when evaluated on PASCAL VOC).