Analysis of Stability, Local Convergence, and Transformation Sensitivity of a Variant of the Particle Swarm Optimization Algorithm

In this paper, we investigate three important properties (stability, local convergence, and transformation invariance) of a variant of particle swarm optimization (PSO) called standard PSO 2011 (SPSO2011). Through some experiments, we identify boundaries of coefficients for this algorithm that ensure particles converge to their equilibrium. Our experiments show that these convergence boundaries for this algorithm are: 1) dependent on the number of dimensions of the problem; 2) different from that of some other PSO variants; and 3) not affected by the stagnation assumption. We also determine boundaries for coefficients associated with different behaviors, e.g., nonoscillatory and zigzagging, of particles before convergence through analysis of particle positions in the frequency domain. In addition, we investigate the local convergence property of this algorithm and we prove that it is not locally convergent. We provide a sufficient condition and related proofs for local convergence for a formulation that represents updating rules of a large class of PSO variants. We modify the SPSO2011 in such a way that it satisfies that sufficient condition; hence, the modified algorithm is locally convergent. Also, we prove that the original standard PSO algorithm is not sensitive to rotation, scaling, and translation of the search space.

[1]  Per Kristian Lehre,et al.  Finite First Hitting Time versus Stochastic Convergence in Particle Swarm Optimisation , 2011, ArXiv.

[2]  Enrique Alba,et al.  Restart particle swarm optimization with velocity modulation: a scalability test , 2011, Soft Comput..

[3]  Rolf Wanka,et al.  Particle swarm optimization almost surely finds local optima , 2013, GECCO '13.

[4]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[5]  Jürgen Branke,et al.  Experimental Analysis of Bound Handling Techniques in Particle Swarm Optimization , 2013, IEEE Transactions on Evolutionary Computation.

[6]  Ying Lin,et al.  Particle Swarm Optimization With an Aging Leader and Challengers , 2013, IEEE Transactions on Evolutionary Computation.

[7]  Zbigniew Michalewicz,et al.  SPSO 2011: analysis of stability; local convergence; and rotation sensitivity , 2014, GECCO.

[8]  Maurice Clerc,et al.  Standard Particle Swarm Optimisation , 2012 .

[9]  Xiang Li,et al.  A hybrid particle swarm with velocity mutation for constraint optimization problems , 2013, GECCO '13.

[10]  Andries Petrus Engelbrecht,et al.  A study of particle swarm optimization particle trajectories , 2006, Inf. Sci..

[11]  Andries Petrus Engelbrecht,et al.  Particle swarm convergence: An empirical investigation , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[12]  Daniel N. Wilke,et al.  Analysis of the particle swarm optimization algorithm , 2007 .

[13]  N. Baba.  Convergence of a random optimization method for constrained optimization problems , 1981 .

[14]  Nikolaus Hansen,et al.  Adapting arbitrary normal mutation distributions in evolution strategies: the covariance matrix adaptation , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[15]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[16]  Thomas Stützle,et al.  Incremental Social Learning in Particle Swarms , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[17]  Zbigniew Michalewicz,et al.  A locally convergent rotationally invariant particle swarm optimization algorithm , 2014, Swarm Intelligence.

[18]  Andries Petrus Engelbrecht,et al.  Particle Swarm Convergence: Standardized Analysis and Topological Influence , 2014, ANTS Conference.

[19]  Giovanni Fasano,et al.  Dynamic analysis for the selection of parameters and initial population, in particle swarm optimization , 2010, J. Glob. Optim..

[20]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[21]  Derek T. Green,et al.  Biases in Particle Swarm Optimization , 2010 .

[22]  Andries Petrus Engelbrecht,et al.  A Convergence Proof for the Particle Swarm Optimiser , 2010, Fundam. Informaticae.

[23]  Shiyuan Yang,et al.  Stochastic convergence analysis and parameter selection of the standard particle swarm optimization algorithm , 2007, Inf. Process. Lett..

[24]  Carsten Witt,et al.  Why standard particle swarm optimisers elude a theoretical runtime analysis , 2009, FOGA '09.

[25]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[26]  Qunfeng Liu,et al.  Order-2 Stability Analysis of Particle Swarm Optimization , 2015, Evolutionary Computation.

[27]  R. Salomon.  Re-evaluating genetic algorithm performance under coordinate rotation of benchmark functions. A survey of some theoretical and practical aspects of genetic algorithms. , 1996, Bio Systems.

[28]  Mauricio Zambrano-Bigiarini,et al.  Standard Particle Swarm Optimisation 2011 at CEC-2013: A baseline for future PSO improvements , 2013, 2013 IEEE Congress on Evolutionary Computation.

[29]  Thomas Stützle,et al.  Frankenstein's PSO: A Composite Particle Swarm Optimization Algorithm , 2009, IEEE Transactions on Evolutionary Computation.

[30]  Roger J.-B. Wets,et al.  Minimization by Random Search Techniques , 1981, Math. Oper. Res..

[31]  A. Groenwold,et al.  Comparison of linear and classical velocity update rules in particle swarm optimization: notes on scale and frame invariance , 2007 .

[32]  Ioan Cristian Trelea,et al.  The particle swarm optimization algorithm: convergence analysis and parameter selection , 2003, Inf. Process. Lett..

[33]  Riccardo Poli,et al.  Mean and Variance of the Sampling Distribution of Particle Swarm Optimizers During Stagnation , 2009, IEEE Transactions on Evolutionary Computation.

[34]  Anne Auger,et al.  Impacts of invariance in search: When CMA-ES and PSO face ill-conditioned and non-separable problems , 2011, Appl. Soft Comput..

[35]  Zbigniew Michalewicz,et al.  An analysis of the velocity updating rule of the particle swarm optimization algorithm , 2014, Journal of Heuristics.

[36]  Riccardo Poli,et al.  Particle Swarm Optimisation , 2011 .

[37]  James Kennedy,et al.  Defining a Standard for Particle Swarm Optimization , 2007, 2007 IEEE Swarm Intelligence Symposium.

[38]  M. Clerc.  Standard Particle Swarm Optimisation From 2006 to 2011 , 2012 .

[39]  Anne Auger,et al.  Empirical comparisons of several derivative free optimization algorithms , 2009 .

[40]  Andries Petrus Engelbrecht,et al.  Fundamentals of Computational Swarm Intelligence , 2005 .

[41]  G. Marsaglia.  Choosing a Point from the Surface of a Sphere , 1972 .

[42]  Riccardo Poli,et al.  Analysis of the publications on the applications of particle swarm optimisation , 2008 .

[43]  Andries Petrus Engelbrecht,et al.  A generalized theoretical deterministic particle swarm model , 2014, Swarm Intelligence.

[44]  James Kennedy,et al.  Particle swarm optimization , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[45]  M. Clerc,et al.  The swarm and the queen: towards a deterministic and adaptive particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).