Automated Machine Learning and Bayesian Optimization

Automated machine learning (AutoML) is introduced in this chapter to illustrate model selection and hyperparameter tuning and the specific features of the resulting optimization problems.

[1]  Harold J. Kushner,et al.  A New Method of Locating the Maximum Point of an Arbitrary Multipeak Curve in the Presence of Noise , 1964 .

[2]  Francesco Archetti,et al.  Global optimization in machine learning: the design of a predictive analytics application , 2018, Soft Computing.

[3]  Aaron Klein,et al.  Efficient and Robust Automated Machine Learning , 2015, NIPS.

[4]  Kirthevasan Kandasamy,et al.  High Dimensional Bayesian Optimisation and Bandits via Additive Models , 2015, ICML.

[5]  Antonio Candelieri.  Clustering and Support Vector Regression for Water Demand Forecasting and Anomaly Detection , 2017 .

[6]  Ameet Talwalkar,et al.  Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization , 2016, J. Mach. Learn. Res..

[7]  Lars Kotthoff,et al.  Auto-WEKA 2.0: Automatic model selection and hyperparameter optimization in WEKA , 2017, J. Mach. Learn. Res..

[8]  Wenjian Wang,et al.  Error estimation based on variance analysis of k-fold cross-validation , 2017, Pattern Recognit..

[9]  Nando de Freitas,et al.  Bayesian Optimization in a Billion Dimensions via Random Embeddings , 2013, J. Artif. Intell. Res..

[10]  Yoshua Bengio,et al.  No Unbiased Estimator of the Variance of K-Fold Cross-Validation , 2003, J. Mach. Learn. Res..

[11]  D. Wolpert.  The Supervised Learning No-Free-Lunch Theorems , 2002 .

[12]  Nando de Freitas,et al.  Taking the Human Out of the Loop: A Review of Bayesian Optimization , 2016, Proceedings of the IEEE.

[13]  Panos M. Pardalos,et al.  No Free Lunch Theorem: A Review , 2019, Approximation and Optimization.

[14]  D. Wolpert,et al.  No Free Lunch Theorems for Search , 1995 .

[15]  Kevin Leyton-Brown,et al.  Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms , 2012, KDD.

[16]  Lars Kotthoff,et al.  Automated Machine Learning: Methods, Systems, Challenges , 2019, The Springer Series on Challenges in Machine Learning.

[17]  Razvan Andonie,et al.  A Dynamic Early Stopping Criterion for Random Search in SVM Hyperparameter Optimization , 2018, AIAI.

[18]  David C. Miller,et al.  Learning surrogate models for simulation‐based optimization , 2014 .