Hierarchical surrogate modeling for illumination algorithms

Evolutionary illumination is a recent technique that allows producing many diverse, optimal solutions in a map of manually defined features. To support the large amount of objective function evaluations, surrogate model assistance was recently introduced. Illumination models need to represent many more, diverse optimal regions than classical surrogate models. In this PhD thesis, we propose to decompose the sample set, decreasing model complexity, by hierarchically segmenting the training set according to their coordinates in feature space. An ensemble of diverse models can then be trained to serve as a surrogate to illumination.

[1]  Nando de Freitas,et al.  A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning , 2010, ArXiv.

[2]  Carl E. Rasmussen,et al.  A Unifying View of Sparse Approximate Gaussian Process Regression , 2005, J. Mach. Learn. Res..

[3]  Shenkai Gu,et al.  Multi-objective and semi-supervised heterogeneous classifier ensembles , 2017 .

[4]  Kenneth Levenberg.  A METHOD FOR THE SOLUTION OF CERTAIN NON – LINEAR PROBLEMS IN LEAST SQUARES , 1944 .

[5]  Jean-Baptiste Mouret,et al.  Data-efficient exploration, optimization, and modeling of diverse designs through surrogate-assisted illumination , 2017, GECCO.

[6]  Antoine Cully,et al.  Robots that can adapt like animals , 2014, Nature.

[7]  Michael Emmerich,et al.  Metamodel Assisted Multiobjective Optimisation Strategies and their Application in Airfoil Design , 2004 .

[8]  Jean-Baptiste Mouret,et al.  Illuminating search spaces by mapping elites , 2015, ArXiv.

[9]  Chao-Ning Liu,et al.  Computer-Automated Design of Multifont Print Recognition Logic , 1963, IBM J. Res. Dev..

[10]  Zoubin Ghahramani,et al.  Sparse Gaussian Processes using Pseudo-inputs , 2005, NIPS.

[11]  Yaochu Jin,et al.  Surrogate-assisted evolutionary computation: Recent advances and future challenges , 2011, Swarm Evol. Comput..

[12]  Richard P. Dwight,et al.  Aerodynamic Shape Optimization Using the Discrete Adjoint of the Navier-Stokes Equations : Applications towards Complex 3 D Configurations , 2009 .

[13]  Marjan Mernik,et al.  Exploration and exploitation in evolutionary algorithms: A survey , 2013, CSUR.

[14]  Sven Behnke,et al.  Hierarchical Neural Networks for Image Interpretation , 2003, Lecture Notes in Computer Science.

[15]  Yew-Soon Ong,et al.  Curse and Blessing of Uncertainty in Evolutionary Algorithm Using Approximation , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[16]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Kenneth O. Stanley,et al.  Simple Evolutionary Optimization Can Rival Stochastic Gradient Descent in Neural Networks , 2016, GECCO.

[18]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[19]  Richard P. Dwight,et al.  Aerodynamic Shape Optimization Using the Discrete Adjoint of the Navier-Stokes Equations: Applications towards Complex 3D Configurations , 2009 .

[20]  Peter Auer,et al.  Using Confidence Bounds for Exploitation-Exploration Trade-offs , 2003, J. Mach. Learn. Res..

[21]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[22]  Jean-Baptiste Mouret,et al.  Feature Space Modeling Through Surrogate Illumination , 2017, ArXiv.

[23]  Bernhard Sendhoff,et al.  Three dimensional evolutionary aerodynamic design optimization with CMA-ES , 2005, GECCO '05.