论文信息 - Evaluation System for a Bayesian Optimization Service

Evaluation System for a Bayesian Optimization Service

Bayesian optimization is an elegant solution to the hyperparameter optimization problem in machine learning. Building a reliable and robust Bayesian optimization service requires careful testing methodology and sound statistical analysis. In this talk we will outline our development of an evaluation framework to rigorously test and measure the impact of changes to the SigOpt optimization service. We present an overview of our evaluation system and discuss how this framework empowers our research engineers to confidently and quickly make changes to our core optimization engine

[1] Kevin Leyton-Brown,et al. Sequential Model-Based Optimization for General Algorithm Configuration , 2011, LION.

[2] Ian Dewancker,et al. A Strategy for Ranking Optimization Methods using Multiple Criteria , 2016, AutoML@ICML.

[3] Ian Dewancker,et al. A Stratified Analysis of Bayesian Optimization Methods , 2016, ArXiv.

[4] Nando de Freitas,et al. Taking the Human Out of the Loop: A Review of Bayesian Optimization , 2016, Proceedings of the IEEE.

[5] Jasper Snoek,et al. Practical Bayesian Optimization of Machine Learning Algorithms , 2012, NIPS.

[6] Yoshua Bengio,et al. Algorithms for Hyper-Parameter Optimization , 2011, NIPS.