Open Issues in Surrogate-Assisted Optimization

Surrogate-assisted optimization was developed for handling complex and costly problems, which arise from real-world applications. The main idea behind surrogate-assisted optimization is to optimally exhaust the available information to lower the amount of required expensive function evaluations thus saving time, resources and the related costs. This chapter outlines the existing challenges in this field that include benchmarking, constraint handling, constructing ensembles of surrogates and solving discrete and/or multi-objective optimization problems. We discuss shortcomings of existing techniques, propose suggestions for improvements and give an outlook on promising research directions. This is valuable for practitioners and researchers alike, since the increased availability of computational resources on the one hand and the continuous development of new approaches on the other hand raise many intricate new problems in this field.

[1]  Tom Dhaene,et al.  A constrained multi-objective surrogate-based optimization algorithm , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[2]  N. Cressie.  Spatial prediction and ordinary kriging , 1988 .

[3]  James Theiler,et al.  Using surrogate data to detect nonlinearity in time series , 1991 .

[4]  Khaled Rasheed,et al.  A Survey of Fitness Approximation Methods Applied in Evolutionary Algorithms , 2010 .

[5]  Dimitrios Gunopulos,et al.  Automatic subspace clustering of high dimensional data for data mining applications , 1998, SIGMOD '98.

[6]  Salvador A. Pintos,et al.  Toward an optimal ensemble of kernel-based approximations with engineering applications , 2006 .

[7]  R. Haftka,et al.  Ensemble of surrogates , 2007 .

[8]  Juan J. Alonso,et al.  Multi-Fidelity Aerodynamic Optimization Using Treed Metamodels with Adaptive Sampling , 2007 .

[9]  Thomas Bartz-Beielstein,et al.  Simulation-based test functions for optimization algorithms , 2017, GECCO.

[10]  Jürgen Branke,et al.  Evolutionary optimization in uncertain environments-a survey , 2005, IEEE Transactions on Evolutionary Computation.

[11]  Andy J. Keane,et al.  Recent advances in surrogate-based optimization , 2009 .

[12]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[13]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[14]  Y. Ho,et al.  Simple Explanation of the No-Free-Lunch Theorem and Its Implications , 2002 .

[15]  Bernhard Sendhoff,et al.  Robust Optimization - A Comprehensive Survey , 2007 .

[16]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[17]  Jie Xu,et al.  Multi-fidelity Model Integration for Engineering Design☆ , 2015 .

[18]  Yaochu Jin,et al.  Surrogate-assisted evolutionary computation: Recent advances and future challenges , 2011, Swarm Evol. Comput..

[19]  Yoel Tenne,et al.  A Versatile Surrogate-Assisted Memetic Algorithm for Optimization of Computationally Expensive Functions and its Engineering Applications , 2008 .

[20]  Jürgen Branke,et al.  A Multi-population Approach to Dynamic Optimization Problems , 2000 .

[21]  Thomas Bartz-Beielstein,et al.  Efficient Global Optimization with Indefinite Kernels , 2016, PPSN.

[22]  Andreas Krause,et al.  Navigating the protein fitness landscape with Gaussian processes , 2012, Proceedings of the National Academy of Sciences.

[23]  Christine A. Shoemaker,et al.  Influence of ensemble surrogate models and sampling strategy on the solution quality of algorithms for computationally expensive black-box global optimization problems , 2014, J. Glob. Optim..

[24]  Bernhard Sendhoff,et al.  Evolution by Adapting Surrogates , 2013, Evolutionary Computation.

[25]  G. G. Wang,et al.  Space exploration and global optimization for computationally intensive design problems: a rough set based approach , 2004 .

[26]  Jürgen Branke,et al.  On Using Surrogates with Genetic Programming , 2015, Evolutionary Computation.

[27]  S. V. N. RAO,et al.  Optimal pumping locations of skimming wells , 2007 .

[28]  Anne Auger,et al.  COCO: a platform for comparing continuous optimizers in a black-box setting , 2016, Optim. Methods Softw..

[29]  Hirotaka Nakayama,et al.  Meta-Modeling in Multiobjective Optimization , 2008, Multiobjective Optimization.

[30]  Jonathan A. Wright,et al.  Constrained, mixed-integer and multi-objective optimisation of building designs by NSGA-II with fitness approximation , 2015, Appl. Soft Comput..

[31]  Robert B. Gramacy,et al.  tgp: An R Package for Bayesian Nonstationary, Semiparametric Nonlinear Regression and Design by Treed Gaussian Process Models , 2007 .

[32]  X. Chen,et al.  High-fidelity global optimization of shape design by dimensionality reduction, metamodels and deterministic particle swarm , 2015 .

[33]  Thomas Bartz-Beielstein,et al.  Efficient global optimization for combinatorial problems , 2014, GECCO.

[34]  Hansheng Wang.  Forward Regression for Ultra-High Dimensional Variable Screening , 2009 .

[35]  John Goldak,et al.  Combinatorial optimization of weld sequence by using a surrogate model to mitigate a weld distortion , 2011 .

[36]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[37]  Thomas Bartz-Beielstein,et al.  Particle Swarm Optimization and Sequential Sampling in Noisy Environments , 2007, Metaheuristics.

[38]  Thomas Bartz-Beielstein.  How to Create Generalizable Results , 2015, Handbook of Computational Intelligence.

[39]  G. Gary Wang,et al.  Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions , 2010 .

[40]  Helio J. C. Barbosa,et al.  Full-atom ab initio protein structure prediction with a Genetic Algorithm using a similarity-based surrogate model , 2010, IEEE Congress on Evolutionary Computation.

[41]  Timothy W. Simpson,et al.  Design and Analysis of Computer Experiments in Multidisciplinary Design Optimization: A Review of How Far We Have Come - Or Not , 2008 .

[42]  Theresa Dawn Robinson,et al.  Surrogate-Based Optimization Using Multifidelity Models with Variable Parameterization and Corrected Space Mapping , 2008 .

[43]  Xiaodong Li,et al.  Improving Local Convergence in Particle Swarms by Fitness Approximation Using Regression , 2010 .

[44]  Michael A. Osborne,et al.  Raiders of the Lost Architecture: Kernels for Bayesian Optimization in Conditional Parameter Spaces , 2014, 1409.4011.

[45]  Thomas Stützle,et al.  Multi-objective ant colony optimization for the twin-screw configuration problem , 2012 .

[46]  Salvador Pintos,et al.  An Optimization Methodology of Alkaline-Surfactant-Polymer Flooding Processes Using Field Scale Numerical Simulation and Multiple Surrogates , 2005 .

[47]  Loo Hay Lee,et al.  Stochastic Simulation Optimization - An Optimal Computing Budget Allocation , 2010, System Engineering and Operations Research.

[48]  Carlos A. Coello Coello.  Constraint-handling techniques used with evolutionary algorithms , 2007, GECCO '07.

[49]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[50]  J.W. Bandler,et al.  Space mapping: the state of the art , 2004, IEEE Transactions on Microwave Theory and Techniques.

[51]  Rasmus K. Ursem,et al.  Multinational GAs: Multimodal Optimization Techniques in Dynamic Environments , 2000, GECCO.

[52]  Gary B. Lamont,et al.  Applications Of Multi-Objective Evolutionary Algorithms , 2004 .

[53]  Luis Mateus Rocha,et al.  Singular value decomposition and principal component analysis , 2003 .

[54]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[55]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.

[56]  Kalyanmoy Deb,et al.  A Generative Kriging Surrogate Model for Constrained and Unconstrained Multi-objective Optimization , 2016, GECCO.

[57]  Christopher Stone,et al.  Exploiting Diverse Distance Metrics for Surrogate-Based Optimisation of Ordering Problems: A Case Study , 2016, GECCO.

[58]  Michèle Sebag,et al.  Dominance-Based Pareto-Surrogate for Multi-Objective Optimization , 2010, SEAL.

[59]  Joshua D. Knowles,et al.  ParEGO: a hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems , 2006, IEEE Transactions on Evolutionary Computation.

[60]  L. Darrell Whitley,et al.  Building Better Test Functions , 1995, ICGA.

[61]  Thomas Bartz-Beielstein,et al.  Model-based methods for continuous and discrete global optimization , 2017, Appl. Soft Comput..

[62]  Andy J. Keane,et al.  Engineering Design via Surrogate Modelling - A Practical Guide , 2008 .

[63]  Alexander J. Smola,et al.  Support Vector Regression Machines , 1996, NIPS.

[64]  Michael T. M. Emmerich,et al.  Single- and multiobjective evolutionary optimization assisted by Gaussian random field metamodels , 2006, IEEE Transactions on Evolutionary Computation.

[65]  T. Simpson,et al.  Comparative studies of metamodelling techniques under multiple modelling criteria , 2001 .

[66]  Anirban Chaudhuri,et al.  Parallel surrogate-assisted global optimization with expensive functions – a survey , 2016 .

[67]  Thomas Bartz-Beielstein,et al.  Noisy optimization with sequential parameter optimization and optimal computational budget allocation , 2011, GECCO.

[68]  Alán Aspuru-Guzik,et al.  Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules , 2016, ACS central science.

[69]  Kalyanmoy Deb,et al.  Multi-objective optimization using evolutionary algorithms , 2001, Wiley-Interscience series in systems and optimization.

[70]  Andy J. Keane,et al.  Weld sequence optimization: The use of surrogate models for solving sequential combinatorial problems , 2005 .

[71]  Thorsten Joachims,et al.  A support vector method for multivariate performance measures , 2005, ICML.

[72]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[73]  Filip De Turck,et al.  Automatic surrogate model type selection during the optimization of expensive black-box problems , 2011, Proceedings of the 2011 Winter Simulation Conference (WSC).

[74]  Krzysztof Michalak,et al.  Infeasibility Driven Evolutionary Algorithm with ARIMA-Based Prediction Mechanism , 2011, IDEAL.