Simple and Adaptive Particle Swarms

The substantial advances that have been made to both the theoretical and practical aspects of particle swarm optimization over the past 10 years have taken it far beyond its original intent as a biological swarm simulation. This thesis details and explains these advances in the context of what has been achieved to this point, as well as what has yet to be understood or solidified within the research community. Taking into account the state of the modern field, a standardized PSO algorithm is defined for benchmarking and comparative purposes both within the work, and for the community as a whole. This standard is refined and simplified over several iterations into a form that does away with potentially undesirable properties of the standard algorithm while retaining equivalent or superior performance on the common set of benchmarks. This refinement, referred to as a discrete recombinant swarm (PSODRS) requires only a single user-defined parameter in the positional update equation, and uses minimal additive stochasticity, rather than the multiplicative stochasticity inherent in the standard PSO. After a mathematical analysis of the PSO-DRS algorithm, an adaptive framework is developed and rigorously tested, demonstrating the effects of the tunable particle- and swarm-level parameters. This adaptability shows practical benefit by broadening the range of problems which the PSO-DRS algorithm is wellsuited to optimize.

[1]  Visakan Kadirkamanathan,et al.  Stability analysis of the particle dynamics in particle swarm optimizer , 2006, IEEE Transactions on Evolutionary Computation.

[2]  Alex A. Freitas,et al.  A hybrid PSO/ACO algorithm for discovering classification rules in data mining , 2008 .

[3]  Ulf Grenander,et al.  A stochastic nonlinear model for coordinated bird flocks , 1990 .

[4]  Gerard T. McKee,et al.  Locating the mouth region in images of human faces , 1993, Other Conferences.

[5]  José Neves,et al.  The fully informed particle swarm: simpler, maybe better , 2004, IEEE Transactions on Evolutionary Computation.

[6]  Russell C. Eberhart,et al.  Tracking and optimizing dynamic systems with particle swarms , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[7]  Yuhui Shi,et al.  Particle swarm optimization: developments, applications and resources , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[8]  R. Storn,et al.  Differential Evolution - A simple and efficient adaptive scheme for global optimization over continuous spaces , 2004 .

[9]  Dmitry Yu. Zubarev,et al.  Global minimum structure searches via particle swarm optimization , 2007, J. Comput. Chem..

[10]  Rakesh Angira,et al.  A Comparative Study of Differential Evolution Algorithms for Estimation of Kinetic Parameters , 2012 .

[11]  Andres Upegui,et al.  Particle Swarm Optimization with Discrete Recombination: An Online Optimizer for Evolvable Hardware , 2006, First NASA/ESA Conference on Adaptive Hardware and Systems (AHS'06).

[12]  A. Carlisle,et al.  Tracking changing extrema with adaptive particle swarm optimizer , 2002, Proceedings of the 5th Biannual World Automation Congress.

[13]  J. Salerno,et al.  Using the particle swarm optimization technique to train a recurrent neural model , 1997, Proceedings Ninth IEEE International Conference on Tools with Artificial Intelligence.

[14]  Gary B. Lamont,et al.  Visualizing particle swarm optimization - Gaussian particle swarm optimization , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[15]  Kevin D. Seppi,et al.  Exposing origin-seeking bias in PSO , 2005, GECCO '05.

[16]  Riccardo Poli,et al.  Theoretical derivation, analysis and empirical evaluation of a simpler Particle Swarm Optimiser , 2007, 2007 IEEE Congress on Evolutionary Computation.

[17]  Peter J. Bentley,et al.  Don't push me! Collision-avoiding swarms , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[18]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[19]  Fabio Schoen,et al.  Fast Global Optimization of Difficult Lennard-Jones Clusters , 2002, Comput. Optim. Appl..

[20]  Jun Zhang,et al.  Adaptive Particle Swarm Optimization , 2008, ANTS Conference.

[21]  Mauro Birattari,et al.  Swarm Intelligence , 2012, Lecture Notes in Computer Science.

[22]  S. Ronald,et al.  Robust encodings in genetic algorithms: a survey of encoding issues , 1997, Proceedings of 1997 IEEE International Conference on Evolutionary Computation (ICEC '97).

[23]  M. Clerc.  Stagnation Analysis in Particle Swarm Optimisation or What Happens When Nothing Happens , 2006 .

[24]  J. Bishop.  Stochastic searching networks , 1989 .

[25]  T. Blackwell,et al.  Particle swarms and population diversity , 2005, Soft Comput..

[26]  David B. Fogel,et al.  Evolutionary Computation: Towards a New Philosophy of Machine Intelligence , 1995 .

[27]  Peter J. Angeline,et al.  Adaptive and Self-adaptive Evolutionary Computations , 1995 .

[28]  Vladimiro Miranda,et al.  NEW EVOLUTIONARY PARTICLE SWARM ALGORITHM (EPSO) APPLIED TO VOLTAGE/VAR CONTROL , 2002 .

[29]  Peter J. Bentley,et al.  Dynamic Search With Charged Swarms , 2002, GECCO.

[30]  James Kennedy,et al.  The Behavior of Particles , 1998, Evolutionary Programming.

[31]  Russell C. Eberhart,et al.  Adaptive particle swarm optimization: detection and response to dynamic systems , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[32]  Yutian Liu,et al.  An adaptive PSO algorithm for reactive power optimization , 2003 .

[33]  J. S. F. Barker,et al.  Simulation of Genetic Systems by Automatic Digital Computers , 1958 .

[34]  Riccardo Poli,et al.  On the moments of the sampling distribution of particle swarm optimisers , 2007, GECCO '07.

[35]  Jürgen Branke,et al.  Multi-swarm Optimization in Dynamic Environments , 2004, EvoWorkshops.

[36]  M. R. AlRashidi,et al.  A Survey of Particle Swarm Optimization Applications in Power System Operations , 2006 .

[37]  Zbigniew Michalewicz,et al.  Evolutionary algorithms , 1997, Emerging Evolutionary Algorithms for Antennas and Wireless Communications.

[38]  Xin Yao,et al.  Fast Evolutionary Programming , 1996, Evolutionary Programming.

[39]  Nico Karssemeijer,et al.  Parameter Estimation in Stochastic Mammogram Model by Heuristic Optimization Techniques , 2006, IEEE Transactions on Information Technology in Biomedicine.

[40]  E. Ozcan,et al.  Particle swarm optimization: surfing the waves , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[41]  Xin Yao,et al.  Evolutionary programming using mutations based on the Levy probability distribution , 2004, IEEE Transactions on Evolutionary Computation.

[42]  Zbigniew Michalewicz,et al.  Evolutionary Algorithms, Homomorphous Mappings, and Constrained Parameter Optimization , 1999, Evolutionary Computation.

[43]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[44]  Tim Blackwell,et al.  Examination of particle tails , 2008 .

[45]  J. Hartigan,et al.  The Dip Test of Unimodality , 1985 .

[46]  L. D. Whitley,et al.  The No Free Lunch and problem description length , 2001 .

[47]  J. Kennedy,et al.  Population structure and particle swarm performance , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[48]  R. Eberhart,et al.  Comparing inertia weights and constriction factors in particle swarm optimization , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[49]  J. Doye,et al.  Global Optimization by Basin-Hopping and the Lowest Energy Structures of Lennard-Jones Clusters Containing up to 110 Atoms , 1997, cond-mat/9803344.

[50]  M. Hoare.  Structure and Dynamics of Simple Microclusters , 2007 .

[51]  Tim Blackwell,et al.  Origin of bursts , 2007, GECCO '07.

[52]  Andrew M. Sutton,et al.  PSO and multi-funnel landscapes: how cooperation might limit exploration , 2006, GECCO.

[53]  J. Northby.  Structure and binding of Lennard‐Jones clusters: 13≤N≤147 , 1987 .

[54]  Russell C. Eberhart,et al.  Solving Constrained Nonlinear Optimization Problems with Particle Swarm Optimization , 2002 .

[55]  Yaobin Chen,et al.  Battery pack state of charge estimator design using computational intelligence approaches , 2000, Fifteenth Annual Battery Conference on Applications and Advances (Cat. No.00TH8490).

[56]  Chilukuri K. Mohan,et al.  Analysis of a simple particle swarm optimization system , 1998 .

[57]  Marco Dorigo,et al.  Optimization, Learning and Natural Algorithms , 1992 .

[58]  A. Neumaier.  Complete search in continuous global optimization and constraint satisfaction , 2004, Acta Numerica.

[59]  Narendra Karmarkar,et al.  A new polynomial-time algorithm for linear programming , 1984, STOC '84.

[60]  Tim Blackwell,et al.  Understanding particle swarms through simplification: a study of recombinant PSO , 2007, GECCO '07.

[61]  René Thomsen,et al.  A comparative study of differential evolution, particle swarm optimization, and evolutionary algorithms on numerical benchmark problems , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[62]  R. Eberhart,et al.  Empirical study of particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[63]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[64]  James Kennedy,et al.  Particle swarm optimization , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[65]  K. Ho,et al.  Structural optimization of Lennard-Jones clusters by a genetic algorithm , 1996 .

[66]  Xiaohui Hu,et al.  Engineering optimization with particle swarm , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[67]  Janet E. Jones.  On the determination of molecular fields. III.—From crystal measurements and kinetic theory data , 1924 .

[68]  J. Jaccard,et al.  LISREL Approaches to Interaction Effects in Multiple Regression , 1998 .

[69]  S. Holm.  A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[70]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[71]  C.A. Coello Coello,et al.  MOPSO: a proposal for multiple objective particle swarm optimization , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[72]  Suganthan.  [IEEE 1999. Congress on Evolutionary Computation-CEC99 - Washington, DC, USA (6-9 July 1999)] Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406) - Particle swarm optimiser with neighbourhood operator , 1999 .

[73]  Konstantinos E. Parsopoulos,et al.  PARTICLE SWARM OPTIMIZER IN NOISY AND CONTINUOUSLY CHANGING ENVIRONMENTS , 2001 .

[74]  Riley H. Lunn.  Tribes , 2006, Cranio : the journal of craniomandibular practice.

[75]  David B. Fogel,et al.  Tuning Evolutionary Programming for Conformationally Flexible Molecular Docking , 1996, Evolutionary Programming.

[76]  Sharon L. Milgram,et al.  The Small World Problem , 1967 .

[77]  J. Kennedy,et al.  Neighborhood topologies in fully informed and best-of-neighborhood particle swarms , 2003, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[78]  D. Watts,et al.  Small Worlds: The Dynamics of Networks between Order and Randomness , 2001 .

[79]  Holger H. Hoos,et al.  An Ant Colony Optimization Algorithm for the 2D HP Protein Folding Problem , 2002, Ant Algorithms.

[80]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[81]  Yongling Zheng,et al.  On the convergence analysis and parameter selection in particle swarm optimization , 2003, Proceedings of the 2003 International Conference on Machine Learning and Cybernetics (IEEE Cat. No.03EX693).

[82]  Spyros G. Tzafestas,et al.  The autonomous mobile robot SENARIO: a sensor aided intelligent navigation system for powered wheelchairs , 1997, IEEE Robotics Autom. Mag..

[83]  R. W. Dobbins,et al.  Computational intelligence PC tools , 1996 .

[84]  D. Broomhead,et al.  Exact analysis of the sampling distribution for the canonical particle swarm optimiser and its convergence during stagnation , 2007, GECCO '07.

[85]  Craig W. Reynolds.  Flocks, herds, and schools: a distributed behavioral model , 1987, SIGGRAPH.

[86]  W. Vent,et al.  Rechenberg, Ingo, Evolutionsstrategie — Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. 170 S. mit 36 Abb. Frommann‐Holzboog‐Verlag. Stuttgart 1973. Broschiert , 1975 .

[87]  James Kennedy,et al.  Defining a Standard for Particle Swarm Optimization , 2007, 2007 IEEE Swarm Intelligence Symposium.

[88]  Lawrence J. Fogel,et al.  Artificial Intelligence through Simulated Evolution , 1966 .

[89]  Xiaodong Li,et al.  Particle swarm with speciation and adaptation in a dynamic environment , 2006, GECCO.

[90]  R. Poli.  An Analysis of Publications on Particle Swarm Optimisation Applications , 2007 .

[91]  Keiichiro Yasuda,et al.  Adaptive Particle Swarm Optimization via Velocity Feedback , 2005 .

[92]  Xin Yao,et al.  Dynamic Control of Adaptive Parameters in Evolutionary Programming , 1998, SEAL.

[93]  Andrzej Osyczka,et al.  7 – Multicriteria optimization for engineering design , 1985 .

[94]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[95]  Joseph C. Culberson,et al.  On the Futility of Blind Search: An Algorithmic View of No Free Lunch , 1998, Evolutionary Computation.

[96]  Slawomir J. Nasuto,et al.  Time Complexity Analysis of the Stochastic Diffusion Search , 1998, NC.

[97]  Michael N. Vrahatis,et al.  Recent approaches to global optimization problems through Particle Swarm Optimization , 2002, Natural Computing.

[98]  P. Levy.  Théorie de l'addition des variables aléatoires , 1955 .

[99]  Jack L. Crosby,et al.  Computer simulation in genetics. , 1973 .

[100]  Ioan Cristian Trelea,et al.  The particle swarm optimization algorithm: convergence analysis and parameter selection , 2003, Inf. Process. Lett..

[101]  Cheng-Yan Kao,et al.  Applying Family Competition to Evolution Strategies for Constrained Optimization , 1997, Evolutionary Programming.

[102]  Jürgen Branke,et al.  Multiswarms, exclusion, and anti-convergence in dynamic environments , 2006, IEEE Transactions on Evolutionary Computation.

[103]  Zbigniew Michalewicz,et al.  Evolutionary Algorithms for Constrained Parameter Optimization Problems , 1996, Evolutionary Computation.

[104]  John Mark Bishop,et al.  Minimum stable convergence criteria for Stochastic Diffusion Search , 2004 .

[105]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[106]  Bernd Hartke.  Global geometry optimization of atomic and molecular clusters by genetic algorithms , 2001 .

[107]  Tim M. Blackwell,et al.  The Lévy Particle Swarm , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[108]  Mohamed E. El-Hawary,et al.  A Survey of Particle Swarm Optimization Applications in Electric Power Systems , 2009, IEEE Transactions on Evolutionary Computation.

[109]  Mark M. Millonas,et al.  Swarms, Phase Transitions, and Collective Intelligence , 1993, adap-org/9306002.

[110]  G. Dantzig.  Programming of Interdependent Activities: II Mathematical Model , 1949 .

[111]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[112]  A. Fraser,et al.  Computer models in genetics , 1970 .

[113]  Tim Blackwell,et al.  A simplified recombinant PSO , 2008 .

[114]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[115]  Thomas Bäck,et al.  Evolutionary computation: Toward a new philosophy of machine intelligence , 1997, Complex..

[116]  Roger M. Whitaker,et al.  An agent based approach to site selection for wireless networks , 2002, SAC '02.

[117]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[118]  James Kennedy,et al.  Probability and dynamics in the particle swarm , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[119]  H. Yoshida,et al.  A particle swarm optimization for reactive power and voltage control considering voltage security assessment , 1999, 2001 IEEE Power Engineering Society Winter Meeting. Conference Proceedings (Cat. No.01CH37194).

[120]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[121]  James Kennedy,et al.  Small worlds and mega-minds: effects of neighborhood topology on particle swarm performance , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[122]  R. Steele.  Optimization , 2005 .

[123]  Slawomir J. Nasuto,et al.  Convergence Analysis of Stochastic Diffusion Search , 1999, Parallel Algorithms Appl..

[124]  D. Fogel.  Evolutionary algorithms in theory and practice , 1997, Complex..

[125]  D. Wolpert,et al.  No Free Lunch Theorems for Search , 1995 .

[126]  M. N. Vrahatis,et al.  Particle swarm optimization method in multiobjective problems , 2002, SAC '02.

[127]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[128]  Russell C. Eberhart,et al.  Human tremor analysis using particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[129]  D. Wunsch,et al.  Multiclass Cancer Classification Using Semisupervised Ellipsoid ARTMAP and Particle Swarm Optimization with Gene Expression Data , 2007, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[130]  Wenzhong Guo,et al.  Two Improved Algorithms for Multiple Sequence Alignment in a Remote Diagnose System for Colonic Cancer in Pervasive Environment , 2006, 2006 First International Symposium on Pervasive Computing and Applications.

[131]  Russell C. Eberhart,et al.  Comparison between Genetic Algorithms and Particle Swarm Optimization , 1998, Evolutionary Programming.

[132]  A. Rama Mohan Rao,et al.  Multi-objective optimal design of fuzzy logic controller using a self configurable swarm intelligence algorithm , 2008 .

[133]  Frans van den Bergh,et al.  An analysis of particle swarm optimizers , 2002 .

[134]  Bo Zhao,et al.  A Survey on Application of Swarm Intelligence Computation to Electric Power System , 2006, 2006 6th World Congress on Intelligent Control and Automation.

[135]  James Kennedy,et al.  Bare bones particle swarms , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[136]  Y. Rahmat-Samii,et al.  Advances in Particle Swarm Optimization for Antenna Designs: Real-Number, Binary, Single-Objective and Multiobjective Implementations , 2007, IEEE Transactions on Antennas and Propagation.

[137]  R. J. W. Hodgson,et al.  Partical Swarm Optimization Applied To The Atomic Cluster Optimization Problem , 2002, GECCO.