Dynamic Trajectory and Convergence Analysis of Swarm Algorithm

Swarm Intelligence (SI) is an innovative distributed intelligent paradigm whereby the collective behaviors of unsophisticated individuals interacting locally with their environment cause coherent functional global patterns to emerge. Although the swarm algorithms have exhibited good performance across a wide range of application problems, it is difficult to analyze the convergence. In this paper, we discuss the dynamic trajectory and convergence of the swarm intelligent model, namely the particle swarm algorithm. We explore the tradeoff between exploration and exploitation using differential analysis and Laplace transform. The trajectories are parsed into first-order inertial element and second-order oscillation element. Their transfer functions are derived, and the trajectories are described in explicit time functions. The first-order inertial element is helpful to maintain the trajectory's stability and algorithm convergence, while the second-order oscillation element trends to explore some new search spaces for the better solutions. The convergence regions of the swarm system are analyzed using the spectral radius and Lyapunov second theorem on stability.

[1]  D. Broomhead,et al.  Exact analysis of the sampling distribution for the canonical particle swarm optimiser and its convergence during stagnation , 2007, GECCO '07.

[2]  Giandomenico Spezzano,et al.  So-Grid: A self-organizing Grid featuring bio-inspired algorithms , 2008, TAAS.

[3]  Visakan Kadirkamanathan,et al.  Stability analysis of the particle dynamics in particle swarm optimizer , 2006, IEEE Transactions on Evolutionary Computation.

[4]  Frantisek Capkovic,et al.  Modelling, Analysing and Control of Interactions Among Agents in MAS , 2006, Comput. Informatics.

[5]  Manoj Kumar Tiwari,et al.  Interactive Particle Swarm: A Pareto-Adaptive Metaheuristic to Multiobjective Optimization , 2008, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[6]  Karl-Dirk Kammeyer,et al.  Optimization of Power Allocation for Interference Cancellation With Particle Swarm Optimization , 2009, IEEE Transactions on Evolutionary Computation.

[7]  JieCHEN,et al.  Stability analysis of particle swarm optimization without Lipschitz constraint , 2003 .

[8]  Zhao Xinchao.  A perturbed particle swarm algorithm for numerical optimization , 2010 .

[9]  Martijn C. Schut,et al.  On model design for simulation of collective intelligence , 2010, Inf. Sci..

[10]  Ying Tan,et al.  Analysis of particle swarm optimization based on discrete time linear system theory , 2004, Fifth World Congress on Intelligent Control and Automation (IEEE Cat. No.04EX788).

[11]  Andries Petrus Engelbrecht,et al.  Fundamentals of Computational Swarm Intelligence , 2005 .

[12]  Kusum Deep,et al.  Mean particle swarm optimisation for function optimisation , 2009, Int. J. Comput. Intell. Stud..

[13]  D. Jeya Mala,et al.  A Hybrid Test Optimization Framework -- Coupling Genetic Algorithm with Local Search Technique , 2010, Comput. Informatics.

[14]  Mu-Chun Su,et al.  A swarm-inspired projection algorithm , 2009, Pattern Recognit..

[15]  Jun Zhang,et al.  Adaptive Particle Swarm Optimization , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[16]  Milan R. Rapaic,et al.  Time-varying PSO - convergence analysis, convergence-related parameterization and new parameter adjustment schemes , 2009, Inf. Process. Lett..

[17]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[18]  Ting-Yu Chen,et al.  On the improvements of the particle swarm optimization algorithm , 2010, Adv. Eng. Softw..

[19]  Eliot Winer,et al.  Synchronous parallelization of Particle Swarm Optimization with digital pheromones , 2009, Adv. Eng. Softw..

[20]  E. Polak,et al.  System Theory , 1963 .

[21]  L Hongbo,et al.  An Hybrid Fuzzy Variable Neighborhood Particle Swarm Optimization Algorithm for Solving Quadratic Assignment Problems , 2007 .

[22]  Leandro dos Santos Coelho,et al.  Gaussian quantum-behaved particle swarm optimization approaches for constrained engineering design problems , 2010, Expert Syst. Appl..

[23]  Gary G. Yen,et al.  PSO-Based Multiobjective Optimization With Dynamic Population Size and Adaptive Local Archives , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[24]  Riccardo Poli,et al.  Analysis of the publications on the applications of particle swarm optimisation , 2008 .

[25]  Andrew J. Chipperfield,et al.  Simplifying Particle Swarm Optimization , 2010, Appl. Soft Comput..

[26]  H. K. Wimmer.  Spectral Radius and Radius of Convergence , 1974 .

[27]  Jie Chen,et al.  Optimal Contraction Theorem for Exploration–Exploitation Tradeoff in Search and Optimization , 2009, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[28]  Keiichiro Yasuda,et al.  Analysis of the Dynamics of Particle Swarm Optimization , 2005 .

[29]  Ajith Abraham,et al.  Chaotic dynamic characteristics in swarm intelligence , 2007, Appl. Soft Comput..

[30]  A. Fuller,et al.  Stability of Motion , 1976, IEEE Transactions on Systems, Man, and Cybernetics.

[31]  Li Ning,et al.  An Analysis for a Particle's Trajectory of PSO Based on Difference Equation , 2006 .

[32]  Weiping Li,et al.  Applied Nonlinear Control , 1991 .

[33]  Andries Petrus Engelbrecht,et al.  Measuring exploration/exploitation in particle swarms using swarm diversity , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[34]  J. Gillis,et al.  Matrix Iterative Analysis , 1961 .

[35]  Shiyuan Yang,et al.  Stochastic convergence analysis and parameter selection of the standard particle swarm optimization algorithm , 2007, Inf. Process. Lett..

[36]  Slawomir J. Nasuto,et al.  Exploration vs. exploitation in naturally inspired search , 2006 .

[37]  Xin Chen,et al.  A Modified PSO Structure Resulting in High Exploration Ability With Convergence Guaranteed , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[38]  Oguz Findik,et al.  A comparison of feature selection models utilizing binary particle swarm optimization and genetic algorithm in determining coronary artery disease using support vector machine , 2010, Expert Syst. Appl..

[39]  E. Ozcan,et al.  Particle swarm optimization: surfing the waves , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[40]  Ajith Abraham,et al.  A Multi-swarm Approach to Multi-objective Flexible Job-shop Scheduling Problems , 2009, Fundam. Informaticae.

[41]  Bo Li,et al.  Particle Swarm Optimisation from lbest to gbest , 2004, WSC.

[42]  Michael N. Vrahatis,et al.  Recent approaches to global optimization problems through Particle Swarm Optimization , 2002, Natural Computing.

[43]  Roy M. Howard,et al.  Linear System Theory , 1992 .

[44]  Ioan Cristian Trelea,et al.  The particle swarm optimization algorithm: convergence analysis and parameter selection , 2003, Inf. Process. Lett..

[45]  Mauro Birattari,et al.  Swarm Intelligence , 2012, Lecture Notes in Computer Science.

[46]  Keith Phalp,et al.  Natural strategies for search , 2009, Natural Computing.

[47]  Alberto García-Villoria,et al.  Introducing dynamic diversity into a discrete particle swarm optimization , 2009, Comput. Oper. Res..

[48]  C. A. Desoer,et al.  Nonlinear Systems Analysis , 1978 .

[49]  Barbara Webb,et al.  Swarm Intelligence: From Natural to Artificial Systems , 2002, Connect. Sci..

[50]  Andries Petrus Engelbrecht,et al.  A study of particle swarm optimization particle trajectories , 2006, Inf. Sci..

[51]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[52]  Mohamed E. El-Hawary,et al.  A Survey of Particle Swarm Optimization Applications in Electric Power Systems , 2009, IEEE Transactions on Evolutionary Computation.

[53]  M. Cotsaftis.  On Stability of Motion , 1969 .

[54]  Bijaya K. Panigrahi,et al.  On Some Properties of the lbest Topology in Particle Swarm Optimization , 2009, 2009 Ninth International Conference on Hybrid Intelligent Systems.

[55]  Xinchao Zhao,et al.  A perturbed particle swarm algorithm for numerical optimization , 2010, Appl. Soft Comput..

[56]  Richard S. Varga,et al.  Matrix Iterative Analysis , 2000, The Mathematical Gazette.

[57]  Ajith Abraham,et al.  An Hybrid Fuzzy Variable Neighborhood Particle Swarm Optimization Algorithm for Solving Quadratic Assignment Problems , 2007, J. Univers. Comput. Sci..

[58]  Andries Petrus Engelbrecht,et al.  A Cooperative approach to particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[59]  J. F. Martínez,et al.  The generalized PSO: a new door to PSO evolution , 2008 .

[60]  Swagatam Das,et al.  A closed loop stability analysis and parameter selection of the Particle Swarm Optimization dynamics for faster convergence , 2007, 2007 IEEE Congress on Evolutionary Computation.

[61]  Masafumi Hagiwara,et al.  Balancing Exploitation and Exploration in Particle Swarm Optimization: Velocity-based Reinitialization , 2008 .