Improving Local Convergence in Particle Swarms by Fitness Approximation Using Regression

In this chapter we present a technique that helps Particle Swarm Optimisers (PSOs) locate an optimum more quickly, through fitness approximation using regression. A least-squares regression is used to estimate the shape of the local fitness landscape. From this shape, the expected location of the peak is calculated and the information given to the PSO. By guiding the PSO to the optimum, the local convergence speed can be vastly improved. We demonstrate the effectiveness of using regression on several static multimodal test functions as well as dynamicmultimodal test scenarios (Moving Peaks). This chapter also extends theMoving Peaks test suite by enhancing the standard conic peak function to allow the creation of asymmetrical and additional multiple local peaks. The combination of this technique and a speciation-based PSO compares favourably to another multi-swarm PSO algorithm that has proven to be working well on the Moving peaks test functions.

[1]  Arnold Neumaier,et al.  Molecular Modeling of Proteins and Mathematical Prediction of Protein Structure , 1997, SIAM Rev..

[2]  Mauro Birattari,et al.  Swarm Intelligence , 2012, Lecture Notes in Computer Science.

[3]  Zbigniew Michalewicz,et al.  Evolutionary Computation 1 , 2018 .

[4]  J. Kennedy,et al.  Population structure and particle swarm performance , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[5]  X. Yao.  Evolutionary Search of Approximated N-dimensional Landscapes , 2000 .

[6]  A. Engelbrecht,et al.  A new locally convergent particle swarm optimiser , 2002, IEEE International Conference on Systems, Man and Cybernetics.

[7]  Xiaodong Li,et al.  Enhancing the robustness of a speciation-based PSO , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[8]  A. Keane,et al.  Evolutionary Optimization of Computationally Expensive Problems via Surrogate Modeling , 2003 .

[9]  Edmund K. Burke,et al.  Parallel Problem Solving from Nature - PPSN IX: 9th International Conference, Reykjavik, Iceland, September 9-13, 2006, Proceedings , 2006, PPSN.

[10]  P. Toint,et al.  An Algorithm using Quadratic Interpolation for Unconstrained Derivative Free Optimization , 1996 .

[11]  J. -F. M. Barthelemy,et al.  Approximation concepts for optimum structural design — a review , 1993 .

[12]  P. John Clarkson,et al.  A Species Conserving Genetic Algorithm for Multimodal Function Optimization , 2002, Evolutionary Computation.

[13]  Yaochu Jin,et al.  Managing approximate models in evolutionary aerodynamic design optimization , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[14]  Xiaodong Li,et al.  Using regression to improve local convergence , 2007, 2007 IEEE Congress on Evolutionary Computation.

[15]  Kalyanmoy Deb,et al.  An Investigation of Niche and Species Formation in Genetic Function Optimization , 1989, ICGA.

[16]  Bernhard Sendhoff,et al.  A framework for evolutionary optimization with approximate fitness functions , 2002, IEEE Trans. Evol. Comput..

[17]  Xiaodong Li,et al.  Particle swarm with speciation and adaptation in a dynamic environment , 2006, GECCO.

[18]  Adi Ben-Israel,et al.  Generalized inverses: theory and applications , 1974 .

[19]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1992, Artificial Intelligence.

[20]  Jürgen Branke,et al.  Multi-swarm Optimization in Dynamic Environments , 2004, EvoWorkshops.

[21]  D. Winfield,et al.  Function Minimization by Interpolation in a Data Table , 1973 .

[22]  Alain Ratle,et al.  Accelerating the Convergence of Evolutionary Algorithms by Fitness Landscape Approximation , 1998, PPSN.

[23]  Andries Petrus Engelbrecht,et al.  Using neighbourhoods with the guaranteed convergence PSO , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[24]  Thomas Philip Runarsson.  Ordinal Regression in Evolutionary Computation , 2006, PPSN.

[25]  J. S. Vesterstrom,et al.  Division of labor in particle swarm optimisation , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[26]  F. H. Branin.  Widely convergent method for finding multiple solutions of simultaneous nonlinear equations , 1972 .

[27]  Xiaodong Li,et al.  This article has been accepted for inclusion in a future issue. IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 1 Locating and Tracking Multiple Dynamic Optima by a Particle Swarm Model Using Speciation , 2022 .

[28]  Ralph R. Martin,et al.  A Sequential Niche Technique for Multimodal Function Optimization , 1993, Evolutionary Computation.

[29]  G. Mahinthakumar,et al.  Hybrid Genetic Algorithm—Local Search Methods for Solving Groundwater Source Identification Inverse Problems , 2005 .

[30]  Riccardo Poli,et al.  Genetic and Evolutionary Computation – GECCO 2004 , 2004, Lecture Notes in Computer Science.

[31]  Peng-Yeng Yin,et al.  A hybrid particle swarm optimization algorithm for optimal task assignment in distributed systems , 2006, Comput. Stand. Interfaces.

[32]  Jürgen Branke,et al.  Memory enhanced evolutionary algorithms for changing optimization problems , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[33]  Joel Weisman,et al.  Introduction to optimization theory , 1973 .

[34]  Yaochu Jin,et al.  A comprehensive survey of fitness approximation in evolutionary computation , 2005, Soft Comput..

[35]  Yew-Soon Ong,et al.  A study on polynomial regression and Gaussian process global surrogate model in hierarchical surrogate-assisted evolutionary algorithm , 2005, 2005 IEEE Congress on Evolutionary Computation.

[36]  Xiaodong Li,et al.  Adaptively Choosing Neighbourhood Bests Using Species in a Particle Swarm Optimizer for Multimodal Function Optimization , 2004, GECCO.

[37]  Dr. Zbigniew Michalewicz,et al.  How to Solve It: Modern Heuristics , 2004 .

[38]  Xiaodong Li,et al.  Graph Algorithms , 1979 .

[39]  Kalyanmoy Deb,et al.  Comparison of multi-modal optimization algorithms based on evolutionary algorithms , 2006, GECCO.

[40]  Thomas Bäck,et al.  Parallel Problem Solving from Nature — PPSN V , 1998, Lecture Notes in Computer Science.

[41]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[42]  J. Meyer.  Generalized Inverses (Theory And Applications) (Adi Ben-Israel and Thomas N. E. Greville) , 1976 .