A comparison of quality measures for model selection in surrogate-assisted evolutionary algorithm

AbstractChoosing a proper approximation model should be the first and the most fundamental problem to be solved when dealing with surrogate-assisted evolutionary algorithms. Till now, most of the model selection methods emphasize on obtaining the best surrogate model basing on model accuracy assessments. As the population ranking is of the most important part in evolutionary optimization, the target function of surrogate model should focus on the right ranking of candidate solutions. Therefore, in this paper, we make a comparison study on several model quality measures which basically dedicated to measuring the capability of surrogate model in selecting and ranking the candidate solutions. In order to investigate the compatibility between accuracy assessments and ranking correlation methods, four algorithms with different model selection strategies based on different quality measures are designed and comparative study is made by contrasting them to three specific surrogate-assisted evolutionary algorithms as well as the standard particle swarm optimization. Simulation results on ten commonly used benchmark problems and one engineering case demonstrate the efficacy of the designed model selection strategies and meanwhile provide further insight into the three model quality measures studied in this paper in model selection.

[1]  Francisco Herrera,et al.  A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms , 2011, Swarm Evol. Comput..

[2]  John Doherty,et al.  Offline Data-Driven Evolutionary Optimization Using Selective Surrogate Ensembles , 2019, IEEE Transactions on Evolutionary Computation.

[3]  Zuomin Dong,et al.  Trends, features, and tests of common and recently introduced global optimization methods , 2010 .

[4]  Andy J. Keane,et al.  Recent advances in surrogate-based optimization , 2009 .

[5]  Weihua Zhang,et al.  Global sensitivity analysis using a Gaussian Radial Basis Function metamodel , 2016, Reliab. Eng. Syst. Saf..

[6]  Ying Tan,et al.  Surrogate-assisted hierarchical particle swarm optimization , 2018, Inf. Sci..

[7]  Raphael T. Haftka,et al.  Surrogate-based Analysis and Optimization , 2005 .

[8]  Chao Hu,et al.  Sequential exploration-exploitation with dynamic trade-off for efficient reliability analysis of complex engineered systems , 2017 .

[9]  Michael T. M. Emmerich,et al.  Single- and multiobjective evolutionary optimization assisted by Gaussian random field metamodels , 2006, IEEE Transactions on Evolutionary Computation.

[10]  Mohammad Reza Banan,et al.  A new PSO-based algorithm for multi-objective optimization with continuous and discrete design variables , 2018 .

[11]  Deyi Xue,et al.  A semi-empirical model considering the influence of operating parameters on performance for a direct methanol fuel cell , 2011 .

[12]  G. Matheron.  Principles of geostatistics , 1963 .

[13]  Hirotaka Nakayama,et al.  A Computational Intelligence Approach to Optimization with Unknown Objective Functions , 2001, ICANN.

[14]  Yin Tan,et al.  An adaptive model selection strategy for surrogate-assisted particle swarm optimization algorithm , 2016, 2016 IEEE Symposium Series on Computational Intelligence (SSCI).

[15]  T. Simpson,et al.  Comparative studies of metamodelling techniques under multiple modelling criteria , 2001 .

[16]  Donghui Wang,et al.  Review: Structural design employing a sequential approximation optimization approach , 2014 .

[17]  Carlos A. Coello Coello,et al.  Comparison of metamodeling techniques in evolutionary algorithms , 2017, Soft Comput..

[18]  H. Takagi,et al.  Accelerating a GA convergence by fitting a single-peak function , 1999 .

[19]  Timothy W. Simpson,et al.  Metamodels for Computer-based Engineering Design: Survey and recommendations , 2001, Engineering with Computers.

[20]  Koichi Hori,et al.  Preface Information technology support for creativity , 1997, Knowl. Based Syst..

[21]  Dekun Yang,et al.  Evolutionary algorithms with a coarse-to-fine function smoothing , 1995, Proceedings of 1995 IEEE International Conference on Evolutionary Computation.

[22]  Haitao Liu,et al.  An adaptive RBF-HDMR modeling approach under limited computational budget , 2018 .

[23]  Régis Duvigneau,et al.  Low cost PSO using metamodels and inexact pre-evaluation: Application to aerodynamic shape design , 2009 .

[24]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[25]  Deyi Xue,et al.  Comparative study on influencing factors in adaptive metamodeling , 2014, Engineering with Computers.

[26]  M. J. D. Powell,et al.  Radial basis function methods for interpolation to functions of many variables , 2001, HERCMA.

[27]  Bernhard Sendhoff,et al.  Evolutionary Optimization with Dynamic Fidelity Computational Models , 2008, ICIC.

[28]  Yaochu Jin,et al.  A comprehensive survey of fitness approximation in evolutionary computation , 2005, Soft Comput..

[29]  John Doherty,et al.  Committee-Based Active Learning for Surrogate-Assisted Particle Swarm Optimization of Expensive Problems , 2017, IEEE Transactions on Cybernetics.

[30]  Michel Verleysen,et al.  Model Selection with Cross-Validations and Bootstraps - Application to Time Series Prediction with RBFN Models , 2003, ICANN.

[31]  Petros Koumoutsakos,et al.  Accelerating evolutionary algorithms with Gaussian process fitness function models , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[32]  Liang Shi,et al.  ASAGA: an adaptive surrogate-assisted genetic algorithm , 2008, GECCO '08.

[33]  Yaochu Jin,et al.  Surrogate-assisted evolutionary computation: Recent advances and future challenges , 2011, Swarm Evol. Comput..

[34]  Bernhard Sendhoff,et al.  Evolution by Adapting Surrogates , 2013, Evolutionary Computation.

[35]  Fred H. Lesh,et al.  Multi-dimensional least-squares polynomial curve fitting , 1959, CACM.

[36]  Thomas J. Santner,et al.  The Design and Analysis of Computer Experiments , 2003, Springer Series in Statistics.

[37]  M. Clerc,et al.  The swarm and the queen: towards a deterministic and adaptive particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[38]  S. T. Buckland,et al.  An Introduction to the Bootstrap. , 1994 .

[39]  Jianqiao Chen,et al.  A surrogate-based particle swarm optimization algorithm for solving optimization problems with expensive black box functions , 2013 .

[40]  Bernhard Sendhoff,et al.  Generalizing Surrogate-Assisted Evolutionary Computation , 2010, IEEE Transactions on Evolutionary Computation.

[41]  Junho Song,et al.  Accelerated Monte Carlo system reliability analysis through machine-learning-based surrogate models of network connectivity , 2017, Reliab. Eng. Syst. Saf..

[42]  Ying Tan,et al.  A generation-based optimal restart strategy for surrogate-assisted social learning particle swarm optimization , 2019, Knowl. Based Syst..

[43]  G. Gary Wang,et al.  Review of Metamodeling Techniques in Support of Engineering Design Optimization , 2007 .

[44]  Wilfrido Gómez-Flores,et al.  On the selection of surrogate models in evolutionary optimization algorithms , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[45]  G. Steven,et al.  Topology and shape optimization methods using evolutionary algorithms: a review , 2015 .

[46]  Craig T. Lawrence,et al.  A Computationally Efficient Feasible Sequential Quadratic Programming Algorithm , 2000, SIAM J. Optim..

[47]  N. Fisher,et al.  Bootstrap algorithms for small samples , 1991 .

[48]  Khaled Rasheed,et al.  A Survey of Fitness Approximation Methods Applied in Evolutionary Algorithms , 2010 .

[49]  Bernhard Sendhoff,et al.  A study on metamodeling techniques, ensembles, and multi-surrogates in evolutionary computation , 2007, GECCO '07.

[50]  Bernhard Sendhoff,et al.  Structure optimization of neural networks for evolutionary design optimization , 2005, Soft Comput..

[51]  Søren Nymand Lophaven,et al.  DACE - A Matlab Kriging Toolbox, Version 2.0 , 2002 .

[52]  Andy J. Keane,et al.  Meta-Lamarckian learning in memetic algorithms , 2004, IEEE Transactions on Evolutionary Computation.