Self-adaptive particle swarm optimization: a review and analysis of convergence

Particle swarm optimization (PSO) is a population-based, stochastic search algorithm inspired by the flocking behaviour of birds. The PSO algorithm has been shown to be rather sensitive to its control parameters, and thus, performance may be greatly improved by employing appropriately tuned parameters. However, parameter tuning is typically a time-intensive empirical process. Furthermore, a priori parameter tuning makes the implicit assumption that the optimal parameters of the PSO algorithm are not time-dependent. To address these issues, self-adaptive particle swarm optimization (SAPSO) algorithms adapt their control parameters throughout execution. While there is a wide variety of such SAPSO algorithms in the literature, their behaviours are not well understood. Specifically, it is unknown whether these SAPSO algorithms will even exhibit convergent behaviour. This paper addresses this lack of understanding by investigating the convergence behaviours of 18 SAPSO algorithms both analytically and empirically. This paper also empirically examines whether the adapted parameters reach a stable point and whether the final parameter values adhere to a well-known convergence criterion. The results depict a grim state for SAPSO algorithms; over half of the SAPSO algorithms exhibit divergent behaviour while many others prematurely converge.

[1]  Deng Ju-Long,et al.  Control problems of grey systems , 1982 .

[2]  Thomas Stützle,et al.  A Racing Algorithm for Configuring Metaheuristics , 2002, GECCO.

[3]  Mohammad Reza Meybodi,et al.  A note on the learning automata based algorithms for adaptive parameter selection in PSO , 2011, Appl. Soft Comput..

[4]  Li Jian,et al.  An Improved Self-Adaptive Particle Swarm Optimization Algorithm with Simulated Annealing , 2009, 2009 Third International Symposium on Intelligent Information Technology Application.

[5]  Mohammad Mehdi Ebadzadeh,et al.  A novel particle swarm optimization algorithm with adaptive inertia weight , 2011, Appl. Soft Comput..

[6]  Bo Liu,et al.  Improved particle swarm optimization combined with chaos , 2005 .

[7]  Jianzhong Zhou,et al.  A Self-Adaptive Particle Swarm Optimization Algorithm with Individual Coefficients Adjustment , 2007, 2007 International Conference on Computational Intelligence and Security (CIS 2007).

[8]  Jianzhong Zhou,et al.  A Self-Adaptive Particle Swarm Optimization Algorithm with Individual Coefficients Adjustment , 2007 .

[9]  Visakan Kadirkamanathan,et al.  Stability analysis of the particle dynamics in particle swarm optimizer , 2006, IEEE Transactions on Evolutionary Computation.

[10]  Gang Xu,et al.  An adaptive parameter tuning of particle swarm optimization algorithm , 2013, Appl. Math. Comput..

[11]  Yu Wang,et al.  Self-adaptive learning based particle swarm optimization , 2011, Inf. Sci..

[12]  Andries Petrus Engelbrecht,et al.  A study of particle swarm optimization particle trajectories , 2006, Inf. Sci..

[13]  Andries Petrus Engelbrecht,et al.  A self-adaptive heterogeneous pso for real-parameter optimization , 2013, 2013 IEEE Congress on Evolutionary Computation.

[14]  Bijaya Ketan Panigrahi,et al.  Adaptive particle swarm optimization approach for static and dynamic economic load dispatch , 2008 .

[15]  Jun Zhang,et al.  Adaptive Particle Swarm Optimization , 2008, ANTS Conference.

[16]  Ming-Feng Yeh,et al.  Grey particle swarm optimization , 2012, Appl. Soft Comput..

[17]  Andries Petrus Engelbrecht,et al.  Inertia weight control strategies for particle swarm optimization , 2016, Swarm Intelligence.

[18]  M. R. Meybodi,et al.  Adaptive parameter selection scheme for PSO: A learning automata approach , 2009, 2009 14th International CSI Computer Conference.

[19]  D. Broomhead,et al.  Exact analysis of the sampling distribution for the canonical particle swarm optimiser and its convergence during stagnation , 2007, GECCO '07.

[20]  Andries Petrus Engelbrecht,et al.  Comparison of self-adaptive particle swarm optimizers , 2014, 2014 IEEE Symposium on Swarm Intelligence.

[21]  Guoqing Li,et al.  A Self-Adaptive Improved Particle Swarm Optimization Algorithm and Its Application in Available Transfer Capability Calculation , 2009, 2009 Fifth International Conference on Natural Computation.

[22]  Narasimhan Sundararajan,et al.  Self regulating particle swarm optimization algorithm , 2015, Inf. Sci..

[23]  A. P. Engelbrecht.  Roaming Behavior of Unconstrained Particles , 2013, 2013 BRICS Congress on Computational Intelligence and 11th Brazilian Congress on Computational Intelligence.

[24]  Andries Petrus Engelbrecht,et al.  A generalized theoretical deterministic particle swarm model , 2014, Swarm Intelligence.

[25]  R. Eberhart,et al.  Empirical study of particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[26]  Qunfeng Liu,et al.  Order-2 Stability Analysis of Particle Swarm Optimization , 2015, Evolutionary Computation.

[27]  Andries Petrus Engelbrecht,et al.  The sad state of self-adaptive particle swarm optimizers , 2016, 2016 IEEE Congress on Evolutionary Computation (CEC).

[28]  Xiufen Li,et al.  A Self-Adaptive Particle Swarm Optimization Algorithm , 2008, 2008 International Conference on Computer Science and Software Engineering.

[29]  Andries Petrus Engelbrecht,et al.  Particle swarm stability: a theoretical extension using the non-stagnate distribution assumption , 2018, Swarm Intelligence.

[30]  Andries Petrus Engelbrecht,et al.  Particle swarm optimizer: The impact of unstable particles on performance , 2016, 2016 IEEE Symposium Series on Computational Intelligence (SSCI).

[31]  YangShengxiang,et al.  A Self-Learning Particle Swarm Optimizer for Global Optimization Problems , 2012 .

[32]  Changhe Li,et al.  A Self-Learning Particle Swarm Optimizer for Global Optimization Problems , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[33]  Veysel Gazi,et al.  Stochastic stability analysis of the particle dynamics in the PSO algorithm , 2012, 2012 IEEE International Symposium on Intelligent Control.

[34]  Andries Petrus Engelbrecht,et al.  Particle swarm variants: standardized convergence analysis , 2015, Swarm Intelligence.

[35]  Yusong Yan,et al.  A Novel Particle Swarm Optimization Algorithm ? , 2014 .

[36]  Xuehu Yan,et al.  An Improved Algorithm for Iris Location , 2007 .

[37]  Guojun Tan,et al.  A Self-Adaptive Mutation-Particle Swarm Optimization Algorithm , 2008, 2008 Fourth International Conference on Natural Computation.

[38]  E. Aiyoshi,et al.  Particle swarm optimization: a numerical stability analysis and parameter adjustment based on swarm activity , 2008 .

[39]  Andries Petrus Engelbrecht,et al.  On the optimality of particle swarm parameters in dynamic environments , 2013, 2013 IEEE Congress on Evolutionary Computation.

[40]  Andries Petrus Engelbrecht,et al.  Particle swarm convergence: An empirical investigation , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[41]  Thomas Stützle,et al.  Heterogeneous particle swarm optimizers , 2009, 2009 IEEE Congress on Evolutionary Computation.

[42]  Jiangye Yuan,et al.  A modified particle swarm optimizer with dynamic adaptation , 2007, Appl. Math. Comput..

[43]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[44]  Saman K. Halgamuge,et al.  Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients , 2004, IEEE Transactions on Evolutionary Computation.

[45]  Riccardo Poli,et al.  Mean and Variance of the Sampling Distribution of Particle Swarm Optimizers During Stagnation , 2009, IEEE Transactions on Evolutionary Computation.

[46]  A. P. Engelbrecht,et al.  Particle Swarm Optimization: Global Best or Local Best? , 2013, 2013 BRICS Congress on Computational Intelligence and 11th Brazilian Congress on Computational Intelligence.

[47]  Andries Petrus Engelbrecht,et al.  Particle swarm optimization: Velocity initialization , 2012, 2012 IEEE Congress on Evolutionary Computation.

[48]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[49]  D. Devaraj,et al.  Adaptive Particle Swarm Optimization Approach for Optimal Reactive Power Planning , 2008, 2008 Joint International Conference on Power System Technology and IEEE Power India Conference.

[50]  Andries Petrus Engelbrecht,et al.  Optimal parameter regions for particle swarm optimization algorithms , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[51]  Gaofeng Wang,et al.  A Method of Self-Adaptive Inertia Weight for PSO , 2008, 2008 International Conference on Computer Science and Software Engineering.

[52]  Ioan Cristian Trelea,et al.  The particle swarm optimization algorithm: convergence analysis and parameter selection , 2003, Inf. Process. Lett..