Particle Swarm Optimization: Understanding Order-2 Stability Guarantees

This paper’s primary aim is to provide clarity on which guarantees about particle stability can actually be made. The particle swarm optimization algorithm has undergone a considerable amount of theoretical analysis. However, with this abundance of theory has come some terminological inconstancies, and as a result it is easy for a practitioner to be misguided by overloaded terminology. Specifically, the criteria for both order-1 and order-2 stability are well studied, but the exact definition of order-2 stability is not consistent amongst researchers. A consequence of this inconsistency in terminology is that the existing theory may in fact misguide practitioners instead of assisting them. In this paper it is theoretically and empirically demonstrated which practical guarantees can in fact be made about particle stability. Specifically, it is shown that the definition of order-2 stability which accurately reflects PSO behavior is that of convergence in second order moment to a constant, and not to zero.

[1]  E. Ozcan,et al.  Particle swarm optimization: surfing the waves , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[2]  Visakan Kadirkamanathan,et al.  Stability analysis of the particle dynamics in particle swarm optimizer , 2006, IEEE Transactions on Evolutionary Computation.

[3]  A. P. Engelbrecht,et al.  Particle Swarm Optimization: Global Best or Local Best? , 2013, 2013 BRICS Congress on Computational Intelligence and 11th Brazilian Congress on Computational Intelligence.

[4]  Andries Petrus Engelbrecht,et al.  Particle swarm optimizer: The impact of unstable particles on performance , 2016, 2016 IEEE Symposium Series on Computational Intelligence (SSCI).

[5]  Andries Petrus Engelbrecht,et al.  Particle swarm variants: standardized convergence analysis , 2015, Swarm Intelligence.

[6]  Ponnuthurai Nagaratnam Suganthan,et al.  Problem Definitions and Evaluation Criteria for the CEC 2014 Special Session and Competition on Single Objective Real-Parameter Numerical Optimization , 2014 .

[7]  Zbigniew Michalewicz,et al.  Stability Analysis of the Particle Swarm Optimization Without Stagnation Assumption , 2016, IEEE Transactions on Evolutionary Computation.

[8]  Chilukuri K. Mohan,et al.  Analysis of a simple particle swarm optimization system , 1998 .

[9]  Ioan Cristian Trelea,et al.  The particle swarm optimization algorithm: convergence analysis and parameter selection , 2003, Inf. Process. Lett..

[10]  Shiyuan Yang,et al.  Stochastic convergence analysis and parameter selection of the standard particle swarm optimization algorithm , 2007, Inf. Process. Lett..

[11]  Qunfeng Liu,et al.  Order-2 Stability Analysis of Particle Swarm Optimization , 2015, Evolutionary Computation.

[12]  Riccardo Poli,et al.  Mean and Variance of the Sampling Distribution of Particle Swarm Optimizers During Stagnation , 2009, IEEE Transactions on Evolutionary Computation.

[13]  Veysel Gazi,et al.  Stochastic stability analysis of the particle dynamics in the PSO algorithm , 2012, 2012 IEEE International Symposium on Intelligent Control.

[14]  D. Broomhead,et al.  Exact analysis of the sampling distribution for the canonical particle swarm optimiser and its convergence during stagnation , 2007, GECCO '07.

[15]  Andries Petrus Engelbrecht,et al.  Fully informed particle swarm optimizer: Convergence analysis , 2015, 2015 IEEE Congress on Evolutionary Computation (CEC).

[16]  Zbigniew Michalewicz,et al.  Analysis of Stability, Local Convergence, and Transformation Sensitivity of a Variant of the Particle Swarm Optimization Algorithm , 2016, IEEE Transactions on Evolutionary Computation.

[17]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[18]  Tim Blackwell,et al.  A Study of Collapse in Bare Bones Particle Swarm Optimization , 2012, IEEE Transactions on Evolutionary Computation.

[19]  Andries Petrus Engelbrecht,et al.  Unified particle swarm optimizer: Convergence analysis , 2016, 2016 IEEE Congress on Evolutionary Computation (CEC).

[20]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[21]  Juan Luis Fernández-Martínez,et al.  Convergence and stochastic stability analysis of particle swarm optimization variants with generic parameter distributions , 2014 .

[22]  Riccardo Poli,et al.  Analysis of the publications on the applications of particle swarm optimisation , 2008 .

[23]  Andries Petrus Engelbrecht,et al.  Fitness-distance-ratio particle swarm optimization: stability analysis , 2017, GECCO.

[24]  E. Kreyszig.  Introductory Functional Analysis With Applications , 1978 .

[25]  Andries Petrus Engelbrecht,et al.  Particle swarm convergence: An empirical investigation , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[26]  Esperanza García Gonzalo,et al.  Convergence and stochastic stability analysis of particle swarm optimization variants with generic parameter distributions , 2014, Appl. Math. Comput..

[27]  A. P. Engelbrecht.  Roaming Behavior of Unconstrained Particles , 2013, 2013 BRICS Congress on Computational Intelligence and 11th Brazilian Congress on Computational Intelligence.

[28]  Andries Petrus Engelbrecht,et al.  A generalized theoretical deterministic particle swarm model , 2014, Swarm Intelligence.

[29]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[30]  Andries Petrus Engelbrecht,et al.  A study of particle swarm optimization particle trajectories , 2006, Inf. Sci..

[31]  Andries Petrus Engelbrecht,et al.  Particle swarm stability: a theoretical extension using the non-stagnate distribution assumption , 2018, Swarm Intelligence.

[32]  Zbigniew Michalewicz,et al.  Particle Swarm Optimization for Single Objective Continuous Space Problems: A Review , 2017, Evolutionary Computation.