Alert button

Active Surrogate Estimators: An Active Learning Approach to Label-Efficient Model Evaluation

Feb 14, 2022
Jannik Kossen, Sebastian Farquhar, Yarin Gal, Tom Rainforth

Figure 1 for Active Surrogate Estimators: An Active Learning Approach to Label-Efficient Model Evaluation
Figure 2 for Active Surrogate Estimators: An Active Learning Approach to Label-Efficient Model Evaluation
Figure 3 for Active Surrogate Estimators: An Active Learning Approach to Label-Efficient Model Evaluation
Figure 4 for Active Surrogate Estimators: An Active Learning Approach to Label-Efficient Model Evaluation

Share this with someone who'll enjoy it:

We propose Active Surrogate Estimators (ASEs), a new method for label-efficient model evaluation. Evaluating model performance is a challenging and important problem when labels are expensive. ASEs address this active testing problem using a surrogate-based estimation approach, whereas previous methods have focused on Monte Carlo estimates. ASEs actively learn the underlying surrogate, and we propose a novel acquisition strategy, XWING, that tailors this learning to the final estimation task. We find that ASEs offer greater label-efficiency than the current state-of-the-art when applied to challenging model evaluation problems for deep neural networks. We further theoretically analyze ASEs' errors.

View paper onarxiv icon

Share this with someone who'll enjoy it: