Alert button

Neural Ensemble Search for Performant and Calibrated Predictions

Jun 15, 2020
Sheheryar Zaidi, Arber Zela, Thomas Elsken, Chris Holmes, Frank Hutter, Yee Whye Teh

Figure 1 for Neural Ensemble Search for Performant and Calibrated Predictions
Figure 2 for Neural Ensemble Search for Performant and Calibrated Predictions
Figure 3 for Neural Ensemble Search for Performant and Calibrated Predictions
Figure 4 for Neural Ensemble Search for Performant and Calibrated Predictions

Share this with someone who'll enjoy it:

Ensembles of neural networks achieve superior performance compared to stand-alone networks not only in terms of accuracy on in-distribution data but also on data with distributional shift alongside improved uncertainty calibration. Diversity among networks in an ensemble is believed to be key for building strong ensembles, but typical approaches only ensemble different weight vectors of a fixed architecture. Instead, we investigate neural architecture search (NAS) for explicitly constructing ensembles to exploit diversity among networks of varying architectures and to achieve robustness against distributional shift. By directly optimizing ensemble performance, our methods implicitly encourage diversity among networks, without the need to explicitly define diversity. We find that the resulting ensembles are more diverse compared to ensembles composed of a fixed architecture and are therefore also more powerful. We show significant improvements in ensemble performance on image classification tasks both for in-distribution data and during distributional shift with better uncertainty calibration.

View paper onarxiv icon

Share this with someone who'll enjoy it: