Efficient Generalized Surrogate-Assisted Evolutionary Algorithm for High-Dimensional Expensive Problems

Engineering optimization problems usually involve computationally expensive simulations and many design variables. Solving such problems in an efficient manner is still a major challenge. In this paper, a generalized surrogate-assisted evolutionary algorithm is proposed to solve such high-dimensional expensive problems. The proposed algorithm is based on the optimization framework of the genetic algorithm (GA). This algorithm proposes to use a surrogate-based trust region local search method, a surrogate-guided GA (SGA) updating mechanism with a neighbor region partition strategy and a prescreening strategy based on the expected improvement infilling criterion of a simplified Kriging in the optimization process. The SGA updating mechanism is a special characteristic of the proposed algorithm. This mechanism makes a fusion between surrogates and the evolutionary algorithm. The neighbor region partition strategy effectively retains the diversity of the population. Moreover, multiple surrogates used in the SGA updating mechanism make the proposed algorithm optimize robustly. The proposed algorithm is validated by testing several high-dimensional numerical benchmark problems with dimensions varying from 30 to 100, and an overall comparison is made between the proposed algorithm and other optimization algorithms. The results show that the proposed algorithm is very efficient and promising for optimizing high-dimensional expensive problems.

[1]  T. Simpson,et al.  Comparative studies of metamodelling techniques under multiple modelling criteria , 2001 .

[2]  Vic Grout,et al.  Efficient Global Optimization of Actuator Based on a Surrogate Model Assisted Hybrid Algorithm , 2018, IEEE Transactions on Industrial Electronics.

[3]  John L. Nazareth,et al.  Introduction to derivative-free optimization , 2010, Math. Comput..

[4]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[5]  Ponnuthurai N. Suganthan,et al.  Recent advances in differential evolution - An updated survey , 2016, Swarm Evol. Comput..

[6]  A. Keane,et al.  Evolutionary Optimization of Computationally Expensive Problems via Surrogate Modeling , 2003 .

[7]  Li Wei,et al.  Adaptive Radial-Basis-Function-Based Multifidelity Metamodeling for Expensive Black-Box Problems , 2017 .

[8]  Yew-Soon Ong,et al.  A study on polynomial regression and Gaussian process global surrogate model in hierarchical surrogate-assisted evolutionary algorithm , 2005, 2005 IEEE Congress on Evolutionary Computation.

[9]  Hans-Martin Gutmann,et al.  A Radial Basis Function Method for Global Optimization , 2001, J. Glob. Optim..

[10]  Kambiz Haji Hajikolaei,et al.  Employing partial metamodels for optimization with scarce samples , 2018 .

[11]  Régis Duvigneau,et al.  Low cost PSO using metamodels and inexact pre-evaluation: Application to aerodynamic shape design , 2009 .

[12]  Liang Gao,et al.  Metamodeling for high dimensional design problems by multi-fidelity simulations , 2017 .

[13]  Joseph Morlier,et al.  Efficient global optimization for high-dimensional constrained problems by using the Kriging models combined with the partial least squares method , 2018 .

[14]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[15]  Mehmet Fatih Tasgetiren,et al.  Differential evolution algorithm with ensemble of parameters and mutation strategies , 2011, Appl. Soft Comput..

[16]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[17]  Dorothea Heiss-Czedik,et al.  An Introduction to Genetic Algorithms. , 1997, Artificial Life.

[18]  Henry P. Wynn,et al.  Screening, predicting, and computer experiments , 1992 .

[19]  Yuping Wang,et al.  An orthogonal genetic algorithm with quantization for global numerical optimization , 2001, IEEE Trans. Evol. Comput..

[20]  Liang Gao,et al.  An enhanced RBF-HDMR integrated with an adaptive sampling method for approximating high dimensional problems in engineering design , 2016 .

[21]  Jianqiao Chen,et al.  A surrogate-based particle swarm optimization algorithm for solving optimization problems with expensive black box functions , 2013 .

[22]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[23]  Hui Wang,et al.  Diversity enhanced particle swarm optimization with neighborhood search , 2013, Inf. Sci..

[24]  Jian Liu,et al.  Gradient-Free Trust-Region-Based Adaptive Response Surface Method for Expensive Aircraft Optimization , 2017 .

[25]  Anirban Chaudhuri,et al.  Parallel surrogate-assisted global optimization with expensive functions – a survey , 2016 .

[26]  Carlos A. Coello Coello,et al.  Comparison of metamodeling techniques in evolutionary algorithms , 2017, Soft Comput..

[27]  Yong Wang,et al.  Global and Local Surrogate-Assisted Differential Evolution for Expensive Constrained Optimization Problems With Inequality Constraints , 2019, IEEE Transactions on Cybernetics.

[28]  Tianyou Chai,et al.  Heterogeneous Ensemble-Based Infill Criterion for Evolutionary Multiobjective Optimization of Expensive Problems , 2019, IEEE Transactions on Cybernetics.

[29]  Thomas Hemker,et al.  Applicability of surrogates to improve efficiency of particle swarm optimization for simulation-based problems , 2012 .

[30]  Qingfu Zhang,et al.  Expensive Multiobjective Optimization by MOEA/D With Gaussian Process Model , 2010, IEEE Transactions on Evolutionary Computation.

[31]  P. N. Suganthan,et al.  Differential Evolution Algorithm With Strategy Adaptation for Global Numerical Optimization , 2009, IEEE Transactions on Evolutionary Computation.

[32]  G. G. Wang,et al.  Adaptive Response Surface Method Using Inherited Latin Hypercube Design Points , 2003 .

[33]  Tapabrata Ray,et al.  A surrogate-assisted differential evolution algorithm with dynamic parameters selection for solving expensive optimization problems , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[34]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[35]  Andy J. Keane,et al.  Combining Global and Local Surrogate Models to Accelerate Evolutionary Optimization , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[36]  Songqing Shan,et al.  Development of Adaptive RBF-HDMR Model for Approximating High Dimensional Problems , 2009, DAC 2009.

[37]  Qingfu Zhang,et al.  A surrogate-model-assisted evolutionary algorithm for computationally expensive design optimization problems with inequality constraints , 2016 .

[38]  Ren-Jye Yang,et al.  Approximation methods in multidisciplinary analysis and optimization: a panel discussion , 2004 .

[39]  Bu-Sung Lee,et al.  Memetic algorithm using multi-surrogates for computationally expensive optimization problems , 2007, Soft Comput..

[40]  G. Gary Wang,et al.  Trust Region Based Mode Pursuing Sampling Method for Global Optimization of High Dimensional Design Problems , 2015 .

[41]  Aimin Zhou,et al.  A Multioperator Search Strategy Based on Cheap Surrogate Models for Evolutionary Optimization , 2015, IEEE Transactions on Evolutionary Computation.

[42]  Bo Liu,et al.  Global Optimization of Microwave Filters Based on a Surrogate Model-Assisted Evolutionary Algorithm , 2017, IEEE Transactions on Microwave Theory and Techniques.

[43]  Songqing Shan,et al.  Turning Black-Box Functions Into White Functions , 2011 .

[44]  Hu Wang,et al.  Adaptive MLS-HDMR metamodeling techniques for high dimensional problems , 2011, Expert Syst. Appl..

[45]  Jin Yeon Cho,et al.  Sequential Approximate Optimization Procedure based on Sample-reusable Moving Least Squares Meta-model and its Application to Design Optimizations , 2010 .

[46]  Yang Yu,et al.  A two-layer surrogate-assisted particle swarm optimization algorithm , 2014, Soft Computing.

[47]  Stefan M. Wild,et al.  CONORBIT: constrained optimization by radial basis function interpolation in trust regions† , 2017, Optim. Methods Softw..

[48]  Christine A. Shoemaker,et al.  Global Convergence of Radial Basis Function Trust-Region Algorithms for Derivative-Free Optimization , 2013, SIAM Rev..

[49]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[50]  G. G. Wang,et al.  Metamodeling for High Dimensional Simulation-Based Design Problems , 2010 .

[51]  M. R. Osborne,et al.  Methods for unconstrained optimization problems , 1968 .

[52]  T. Simpson,et al.  Computationally Inexpensive Metamodel Assessment Strategies , 2002 .

[53]  Ying Tan,et al.  Surrogate-assisted hierarchical particle swarm optimization , 2018, Inf. Sci..

[54]  Michael T. M. Emmerich,et al.  Single- and multiobjective evolutionary optimization assisted by Gaussian random field metamodels , 2006, IEEE Transactions on Evolutionary Computation.

[55]  K. Yamazaki,et al.  Sequential Approximate Optimization using Radial Basis Function network for engineering optimization , 2011 .

[56]  Helio J. C. Barbosa,et al.  A study on fitness inheritance for enhanced efficiency in real-coded genetic algorithms , 2012, 2012 IEEE Congress on Evolutionary Computation.

[57]  Yaochu Jin,et al.  Surrogate-assisted evolutionary computation: Recent advances and future challenges , 2011, Swarm Evol. Comput..

[58]  Jianchao Zeng,et al.  Surrogate-Assisted Cooperative Swarm Optimization of High-Dimensional Expensive Problems , 2017, IEEE Transactions on Evolutionary Computation.

[59]  Bernhard Sendhoff,et al.  Evolution by Adapting Surrogates , 2013, Evolutionary Computation.

[60]  Achille Messac,et al.  Metamodeling using extended radial basis functions: a comparative approach , 2006, Engineering with Computers.

[61]  N. Cressie.  The origins of kriging , 1990 .

[62]  Khaled Rasheed,et al.  Guided crossover: a new operator for genetic algorithm based optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[63]  Rommel G. Regis,et al.  Evolutionary Programming for High-Dimensional Constrained Expensive Black-Box Optimization Using Radial Basis Functions , 2014, IEEE Transactions on Evolutionary Computation.

[64]  Liang Gao,et al.  Ensemble of surrogates assisted particle swarm optimization of medium scale expensive problems , 2019, Appl. Soft Comput..

[65]  Qingfu Zhang,et al.  A Gaussian Process Surrogate Model Assisted Evolutionary Algorithm for Medium Scale Expensive Optimization Problems , 2014, IEEE Transactions on Evolutionary Computation.

[66]  Tung-Kuan Liu,et al.  Hybrid Taguchi-genetic algorithm for global numerical optimization , 2004, IEEE Transactions on Evolutionary Computation.

[67]  Qingfu Zhang,et al.  Behavioral study of the surrogate model-aware evolutionary search framework , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[68]  John Doherty,et al.  Committee-Based Active Learning for Surrogate-Assisted Particle Swarm Optimization of Expensive Problems , 2017, IEEE Transactions on Cybernetics.

[69]  Rammohan Mallipeddi,et al.  An evolving surrogate model-based differential evolution algorithm , 2015, Appl. Soft Comput..

[70]  Kambiz Haji Hajikolaei,et al.  Optimization on Metamodeling-Supported Iterative Decomposition , 2015, DAC 2015.

[71]  Rommel G. Regis,et al.  Particle swarm with radial basis function surrogates for expensive black-box optimization , 2014, J. Comput. Sci..

[72]  Guangyao Li,et al.  Advanced high strength steel springback optimization by projection-based heuristic global search algorithm , 2013 .