A two-layer surrogate-assisted particle swarm optimization algorithm

Like most evolutionary algorithms, particle swarm optimization (PSO) usually requires a large number of fitness evaluations to obtain a sufficiently good solution. This poses an obstacle for applying PSO to computationally expensive problems. This paper proposes a two-layer surrogate-assisted PSO (TLSAPSO) algorithm, in which a global and a number of local surrogate models are employed for fitness approximation. The global surrogate model aims to smooth out the local optima of the original multimodal fitness function and guide the swarm to fly quickly to an optimum or the global optimum. In the meantime, a local surrogate model constructed using the data samples near each particle is built to achieve a fitness estimation as accurate as possible. The contribution of each surrogate in the search is empirically verified by experiments on uni- and multi-modal problems. The performance of the proposed TLSAPSO algorithm is examined on ten widely used benchmark problems, and the experimental results show that the proposed algorithm is effective and highly competitive with the state-of-the-art, especially for multimodal optimization problems.

[1]  Tim Hendtlass.  Fitness estimation and the particle swarm optimisation algorithm , 2007, 2007 IEEE Congress on Evolutionary Computation.

[2]  Jeng-Shyang Pan,et al.  Similarity-based evolution control for fitness estimation in particle swarm optimization , 2013, 2013 IEEE Symposium on Computational Intelligence in Dynamic and Uncertain Environments (CIDUE).

[3]  Andreas Zell,et al.  Evolution strategies assisted by Gaussian processes with improved preselection criterion , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[4]  Thomas Hemker,et al.  Applicability of surrogates to improve efficiency of particle swarm optimization for simulation-based problems , 2012 .

[5]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[6]  Andy J. Keane,et al.  Combining Global and Local Surrogate Models to Accelerate Evolutionary Optimization , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[7]  Yaochu Jin,et al.  A comprehensive survey of fitness approximation in evolutionary computation , 2005, Soft Comput..

[8]  Jianqiao Chen,et al.  A surrogate-based particle swarm optimization algorithm for solving optimization problems with expensive black box functions , 2013 .

[9]  Agus Sudjianto,et al.  Blind Kriging: A New Method for Developing Metamodels , 2008 .

[10]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[11]  Ahmed Kattan,et al.  Evolving radial basis function networks via GP for estimating fitness values using surrogate models , 2012, 2012 IEEE Congress on Evolutionary Computation.

[12]  Carlos A. Coello Coello,et al.  A study of fitness inheritance and approximation techniques for multi-objective particle swarm optimization , 2005, 2005 IEEE Congress on Evolutionary Computation.

[13]  M. Rais-Rohani,et al.  Ensemble of metamodels with optimized weight factors , 2008 .

[14]  Kai-Yew Lum,et al.  Max-min surrogate-assisted evolutionary algorithm for robust design , 2006, IEEE Transactions on Evolutionary Computation.

[15]  M. A. Bishr,et al.  Power systems operation using particle swarm optimization technique , 2008 .

[16]  R. Eberhart,et al.  Comparing inertia weights and constriction factors in particle swarm optimization , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[17]  Qingfu Zhang,et al.  A Gaussian Process Surrogate Model Assisted Evolutionary Algorithm for Medium Scale Expensive Optimization Problems , 2014, IEEE Transactions on Evolutionary Computation.

[18]  Yew-Soon Ong,et al.  A study on polynomial regression and Gaussian process global surrogate model in hierarchical surrogate-assisted evolutionary algorithm , 2005, 2005 IEEE Congress on Evolutionary Computation.

[19]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[20]  Rommel G. Regis,et al.  Particle swarm with radial basis function surrogates for expensive black-box optimization , 2014, J. Comput. Sci..

[21]  R. Storn,et al.  On the usage of differential evolution for function optimization , 1996, Proceedings of North American Fuzzy Information Processing.

[22]  Helio J. C. Barbosa,et al.  A study on fitness inheritance for enhanced efficiency in real-coded genetic algorithms , 2012, 2012 IEEE Congress on Evolutionary Computation.

[23]  A. Keane,et al.  Evolutionary Optimization of Computationally Expensive Problems via Surrogate Modeling , 2003 .

[24]  Kok Wai Wong,et al.  Surrogate-Assisted Evolutionary Optimization Frameworks for High-Fidelity Engineering Design Problems , 2005 .

[25]  Xin Yao,et al.  Classification-assisted Differential Evolution for computationally expensive problems , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[26]  Yoel Tenne,et al.  A framework for memetic optimization using variable global and local surrogate models , 2009, Soft Comput..

[27]  Xiaoyan Sun,et al.  A New Surrogate-Assisted Interactive Genetic Algorithm With Weighted Semisupervised Learning , 2013, IEEE Transactions on Cybernetics.

[28]  Bernhard Sendhoff,et al.  Reducing Fitness Evaluations Using Clustering Techniques and Neural Network Ensembles , 2004, GECCO.

[29]  Shang He,et al.  An improved particle swarm optimizer for mechanical design optimization problems , 2004 .

[30]  D. Y. Sha,et al.  A new particle swarm optimization for the open shop scheduling problem , 2008, Comput. Oper. Res..

[31]  Julian F. Miller,et al.  Genetic and Evolutionary Computation — GECCO 2003 , 2003, Lecture Notes in Computer Science.

[32]  R. Haftka,et al.  Ensemble of surrogates , 2007 .

[33]  M. Farina.  A neural network based generalized response surface multiobjective evolutionary algorithm , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[34]  Régis Duvigneau,et al.  Low cost PSO using metamodels and inexact pre-evaluation: Application to aerodynamic shape design , 2009 .

[35]  Masoud Rais-Rohani,et al.  Ensemble of Metamodels with Optimized Weight Factors , 2008 .

[36]  Xiaodong Li,et al.  Improving Local Convergence in Particle Swarms by Fitness Approximation Using Regression , 2010 .

[37]  Bin Li,et al.  An evolution strategy assisted by an ensemble of local Gaussian process models , 2013, GECCO '13.

[38]  Jeng-Shyang Pan,et al.  A new fitness estimation strategy for particle swarm optimization , 2013, Inf. Sci..

[39]  Meng-Sing Liou,et al.  Multiobjective optimization using coupled response surface model and evolutionary algorithm , 2004 .

[40]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[41]  Petros Koumoutsakos,et al.  Accelerating evolutionary algorithms with Gaussian process fitness function models , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[42]  Robert E. Smith,et al.  Fitness inheritance in genetic algorithms , 1995, SAC '95.

[43]  Bernhard Sendhoff,et al.  Generalizing Surrogate-Assisted Evolutionary Computation , 2010, IEEE Transactions on Evolutionary Computation.

[44]  Bernhard Sendhoff,et al.  A framework for evolutionary optimization with approximate fitness functions , 2002, IEEE Trans. Evol. Comput..

[45]  Alain Ratle,et al.  Kriging as a surrogate fitness landscape in evolutionary optimization , 2001, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.