A generation-based optimal restart strategy for surrogate-assisted social learning particle swarm optimization

Abstract Evolutionary algorithm provides a powerful tool to the solution of modern complex engineering optimization problems. In general, a great deal of evaluation effort often requires to be made in evolutionary optimization to locate a reasonable optimum. This poses a serious challenge to extend its application to computationally expensive problems. To alleviate this difficulty, surrogate-assisted evolutionary algorithms (SAEAs) have drawn great attention over the past decades. However, in order to ensure the performance of SAEAs, the use of appropriate model management is indispensable. This paper proposes a generation-based optimal restart strategy for a surrogate-assisted social learning particle swarm optimization (SL-PSO). In the proposed method, the SL-PSO restarts every few generations in the global radial-basis-function model landscape, and the best sample points archived in the database are employed to reinitialize the swarm at each restart. Promising individual with the best estimated fitness value is chosen for exact evaluation before each restart of the SL-PSO. The proposed method skillfully integrates the restart strategy, generation-based and individual-based model managements into a whole, whilst those three ingredients coordinate with each other, thus offering a powerful optimizer for the computationally expensive problems. To assess the performance of the proposed method, comprehensive experiments are conducted on a benchmark test suit of dimensions ranging from 10 to 100. Experimental results demonstrate that the proposed method shows superior performance in comparison with four state-of-the-art algorithms in a majority of benchmarks when only a limited computational budget is available.

[1]  Rommel G. Regis,et al.  Evolutionary Programming for High-Dimensional Constrained Expensive Black-Box Optimization Using Radial Basis Functions , 2014, IEEE Transactions on Evolutionary Computation.

[2]  Jeng-Shyang Pan,et al.  A new fitness estimation strategy for particle swarm optimization , 2013, Inf. Sci..

[3]  Xin Yao,et al.  Meta-Heuristic Algorithms in Car Engine Design: A Literature Survey , 2015, IEEE Transactions on Evolutionary Computation.

[4]  Yaochu Jin,et al.  Surrogate-assisted evolutionary computation: Recent advances and future challenges , 2011, Swarm Evol. Comput..

[5]  Jianchao Zeng,et al.  Surrogate-Assisted Cooperative Swarm Optimization of High-Dimensional Expensive Problems , 2017, IEEE Transactions on Evolutionary Computation.

[6]  Francisco Herrera,et al.  A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 Special Session on Real Parameter Optimization , 2009, J. Heuristics.

[7]  Loris Vincenzi,et al.  A proper infill sampling strategy for improving the speed performance of a Surrogate-Assisted Evolutionary Algorithm , 2017 .

[8]  Bernhard Sendhoff,et al.  Evolution by Adapting Surrogates , 2013, Evolutionary Computation.

[9]  Yudong Zhang,et al.  Detection of Alzheimer's disease and mild cognitive impairment based on structural volumetric MR images using 3D-DWT and WTA-KSVM trained by PSOTVAC , 2015, Biomed. Signal Process. Control..

[10]  Xin Yao,et al.  Evolutionary Multiobjective Optimization-Based Multimodal Optimization: Fitness Landscape Approximation and Peak Detection , 2018, IEEE Transactions on Evolutionary Computation.

[11]  Hans-Martin Gutmann,et al.  A Radial Basis Function Method for Global Optimization , 2001, J. Glob. Optim..

[12]  Ahmed Kattan,et al.  Surrogate Genetic Programming: A semantic aware evolutionary search , 2015, Inf. Sci..

[13]  Haitao Liu,et al.  A survey of adaptive sampling for global metamodeling in support of simulation-based complex engineering design , 2017, Structural and Multidisciplinary Optimization.

[14]  Fred H. Lesh,et al.  Multi-dimensional least-squares polynomial curve fitting , 1959, CACM.

[15]  G. Gary Wang,et al.  Review of Metamodeling Techniques in Support of Engineering Design Optimization , 2007 .

[16]  Haitao Liu,et al.  An adaptive sampling approach for Kriging metamodeling by maximizing expected prediction error , 2017, Comput. Chem. Eng..

[17]  G. Steven,et al.  Topology and shape optimization methods using evolutionary algorithms: a review , 2015 .

[18]  T. Simpson,et al.  Comparative studies of metamodelling techniques under multiple modelling criteria , 2001 .

[19]  G. Gary Wang,et al.  Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions , 2010 .

[20]  Bernhard Sendhoff,et al.  A framework for evolutionary optimization with approximate fitness functions , 2002, IEEE Trans. Evol. Comput..

[21]  Jack P. C. Kleijnen,et al.  Regression and Kriging metamodels with their experimental designs in simulation: A review , 2017, Eur. J. Oper. Res..

[22]  Fan Ye,et al.  Sheet metal forming optimization by using surrogate modeling techniques , 2017 .

[23]  Liang Gao,et al.  A multi-point sampling method based on kriging for global optimization , 2017 .

[24]  Christine A. Shoemaker,et al.  A Stochastic Radial Basis Function Method for the Global Optimization of Expensive Functions , 2007, INFORMS J. Comput..

[25]  Carlos A. Coello Coello,et al.  Comparison of metamodeling techniques in evolutionary algorithms , 2017, Soft Comput..

[26]  Yaochu Jin,et al.  A comprehensive survey of fitness approximation in evolutionary computation , 2005, Soft Comput..

[27]  John Doherty,et al.  Committee-Based Active Learning for Surrogate-Assisted Particle Swarm Optimization of Expensive Problems , 2017, IEEE Transactions on Cybernetics.

[28]  Genlin Ji,et al.  Preliminary research on abnormal brain detection by wavelet-energy and quantum- behaved PSO. , 2016, Technology and health care : official journal of the European Society for Engineering and Medicine.

[29]  Handing Wang,et al.  Data-Driven Surrogate-Assisted Multiobjective Evolutionary Optimization of a Trauma System , 2016, IEEE Transactions on Evolutionary Computation.

[30]  R. Regis Constrained optimization by radial basis function interpolation for high-dimensional expensive black-box problems with infeasible initial points , 2014 .

[31]  C. Shoemaker,et al.  Combining radial basis function surrogates and dynamic coordinate search in high-dimensional expensive black-box optimization , 2013 .

[32]  Selen Cremaschi,et al.  Adaptive sequential sampling for surrogate model generation with artificial neural networks , 2014, Comput. Chem. Eng..

[33]  Petros Koumoutsakos,et al.  Accelerating evolutionary algorithms with Gaussian process fitness function models , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[34]  Yang Yu,et al.  A two-layer surrogate-assisted particle swarm optimization algorithm , 2014, Soft Computing.

[35]  Guangyao Li,et al.  Variable stiffness composite material design by using support vector regression assisted efficient global optimization method , 2017 .

[36]  Bernhard Sendhoff,et al.  Structure optimization of neural networks for evolutionary design optimization , 2005, Soft Comput..

[37]  Zuomin Dong,et al.  Trends, features, and tests of common and recently introduced global optimization methods , 2010 .

[38]  Antonio Bolufé Röhler,et al.  Measuring the curse of dimensionality and its effects on particle swarm optimization and differential evolution , 2014, Applied Intelligence.

[39]  Francisco Herrera,et al.  A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms , 2011, Swarm Evol. Comput..

[40]  Yan Wang,et al.  An active learning radial basis function modeling method based on self-organization maps for simulation-based design problems , 2017, Knowl. Based Syst..

[41]  Weihua Zhang,et al.  Global sensitivity analysis using a Gaussian Radial Basis Function metamodel , 2016, Reliab. Eng. Syst. Saf..

[42]  Yaochu Jin,et al.  A social learning particle swarm optimization algorithm for scalable optimization , 2015, Inf. Sci..

[43]  M. Powell Recent research at Cambridge on radial basis functions , 1999 .

[44]  Antonin Ponsich,et al.  A Survey on Multiobjective Evolutionary Algorithms for the Solution of the Portfolio Optimization Problem and Other Finance and Economics Applications , 2013, IEEE Transactions on Evolutionary Computation.

[45]  Tianyou Chai,et al.  Generalized Multitasking for Evolutionary Optimization of Expensive Problems , 2019, IEEE Transactions on Evolutionary Computation.

[46]  Kaisa Miettinen,et al.  A Surrogate-Assisted Reference Vector Guided Evolutionary Algorithm for Computationally Expensive Many-Objective Optimization , 2018, IEEE Transactions on Evolutionary Computation.

[47]  Thomas J. Santner,et al.  The Design and Analysis of Computer Experiments , 2003, Springer Series in Statistics.

[48]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[49]  Ying Tan,et al.  Surrogate-assisted hierarchical particle swarm optimization , 2018, Inf. Sci..

[50]  Michael T. M. Emmerich,et al.  Single- and multiobjective evolutionary optimization assisted by Gaussian random field metamodels , 2006, IEEE Transactions on Evolutionary Computation.

[51]  Hirotaka Nakayama,et al.  Meta-Modeling in Multiobjective Optimization , 2008, Multiobjective Optimization.

[52]  J. Havinga,et al.  Sequential improvement for robust optimization using an uncertainty measure for radial basis functions , 2017 .

[53]  Layne T. Watson,et al.  Efficient global optimization algorithm assisted by multiple surrogate techniques , 2012, Journal of Global Optimization.