Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Hyperboost: Hyperparameter Optimization by Gradient Boosting surrogate models

Jan 06, 2021
Jeroen van Hoof, Joaquin Vanschoren

Bayesian Optimization is a popular tool for tuning algorithms in automatic machine learning (AutoML) systems. Current state-of-the-art methods leverage Random Forests or Gaussian processes to build a surrogate model that predicts algorithm performance given a certain set of hyperparameter settings. In this paper, we propose a new surrogate model based on gradient boosting, where we use quantile regression to provide optimistic estimates of the performance of an unobserved hyperparameter setting, and combine this with a distance metric between unobserved and observed hyperparameter settings to help regulate exploration. We demonstrate empirically that the new method is able to outperform some state-of-the art techniques across a reasonable sized set of classification problems.

* ECMLPKDD 2019 Workshop on Automating Data Science 

Share this with someone who'll enjoy it:

   Access Paper Source

Share this with someone who'll enjoy it: