Asynchronous parallelization of particle swarm optimization through digital pheromone sharing

In this paper, a model for sharing digital pheromones between multiple particle swarms to search n-dimensional design spaces in an asynchronous parallel computing environment is presented. Particle swarm optimization (PSO) is an evolutionary technique used to effectively search multi-modal design spaces. With the aid of digital pheromones, members in a swarm can better communicate with each other to improve search performance. Previous work by the authors demonstrated the capability of digital pheromones within PSO for searching the global optimum in both single and coarse grain synchronous parallel computing environments. In the coarse grain approach, multiple swarms are simultaneously deployed across various processors and synchronization is carried out only when all swarms achieved convergence, in an effort to reduce processor-to-processor communication and network latencies. However, it is theorized that with an appropriate parallelization scheme, the benefits of digital pheromones and communication between swarms can outweigh the network bandwidth latencies resulting in improved search efficiency and accuracy. To explore this idea, a swarm is deployed in the design space across different processors. Through an additional processor, each part of the swarm can communicate with the others. While digital pheromones aid communication within a swarm, the developed parallelization model facilitates communication between multiple swarms resulting in improved search accuracy and efficiency. The development of this method and results from solving several multi-modal test problems are presented.

[1]  Russell C. Eberhart,et al.  Swarm intelligence for permutation optimization: a case study of n-queens problem , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[2]  Gabriela Ciuprina,et al.  Use of intelligent-particle swarm optimization in electromagnetics. IEEE Trans Mag , 2002 .

[3]  Panos M. Pardalos,et al.  Filled functions for unconstrained global optimization , 2001, J. Glob. Optim..

[4]  V. Cerný.  Thermodynamical approach to the traveling salesman problem: An efficient simulation algorithm , 1985 .

[5]  Michael N. Vrahatis,et al.  Recent approaches to global optimization problems through Particle Swarm Optimization , 2002, Natural Computing.

[6]  Jigui Sun,et al.  A Hybrid Particle Swarm Optimization for Binary CSPs , 2006, ICIC.

[7]  Rui Huang,et al.  Mobile Agent Routing Based on a Two-Stage Optimization Model and a Hybrid Evolutionary Algorithm in Wireless Sensor Networks , 2006, ICNC.

[8]  Chunguang Zhou,et al.  Particle swarm optimization for traveling salesman problem , 2003, Proceedings of the 2003 International Conference on Machine Learning and Cybernetics (IEEE Cat. No.03EX693).

[9]  A. Groenwold,et al.  Comparison of linear and classical velocity update rules in particle swarm optimization: notes on scale and frame invariance , 2007 .

[10]  Ning Zhong,et al.  A Hybrid Discrete Particle Swarm Optimization for the Traveling Salesman Problem , 2006, SEAL.

[11]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[12]  Hongwei Liu,et al.  Virus-Evolutionary Particle Swarm Optimization Algorithm , 2006, ICNC.

[13]  S. N. Sivanandam,et al.  Introduction to genetic algorithms , 2007 .

[14]  Lalit M. Patnaik,et al.  Genetic algorithms: a survey , 1994, Computer.

[15]  Russell C. Eberhart,et al.  Parameter Selection in Particle Swarm Optimization , 1998, Evolutionary Programming.

[16]  B J Fregly,et al.  Parallel global optimization with the particle swarm algorithm , 2004, International journal for numerical methods in engineering.

[17]  Marco Dorigo,et al.  Distributed Optimization by Ant Colonies , 1992 .

[18]  Ali Kaveh,et al.  A HYBRID PARTICLE SWARM AND ANT COLONY OPTIMIZATION FOR DESIGN OF TRUSS STRUCTURES , 2008 .

[19]  Bin Shen,et al.  Heuristic Information Based Improved Fuzzy Discrete PSO Method for Solving TSP , 2006, PRICAI.

[20]  Eliot Winer,et al.  Multimodal UAV ground control system , 2006 .

[21]  H. Van Dyke Parunak,et al.  DIGITAL PHEROMONES FOR AUTONOMOUS COORDINATION OF SWARMING UAV'S , 2002 .

[22]  Russell C. Eberhart,et al.  Solving Constrained Nonlinear Optimization Problems with Particle Swarm Optimization , 2002 .

[23]  Xiaohui Hu,et al.  Engineering optimization with particle swarm , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[24]  Shang He,et al.  An improved particle swarm optimizer for mechanical design optimization problems , 2004 .

[25]  B. P. Wang,et al.  Particle Swarm Optimization for Mixed Discrete, Integer and Continuous Variables , 2004 .

[26]  Jaroslaw Sobieszczanski-Sobieski,et al.  Particle swarm optimization , 2002 .

[27]  Russell C. Eberhart,et al.  Particle swarm with extended memory for multiobjective optimization , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[28]  Albert A. Groenwold,et al.  A Study of Global Optimization Using Particle Swarms , 2005, J. Glob. Optim..

[29]  Hwa-Seok Lee,et al.  PC Cluster based Parallel PSO Algorithm for Optimal Power Flow , 2007, 2007 International Conference on Intelligent Systems Applications to Power Systems.

[30]  Andries Petrus Engelbrecht,et al.  Fundamentals of Computational Swarm Intelligence , 2005 .

[31]  Jaroslaw Sobieszczanski-Sobieski,et al.  A Parallel Particle Swarm Optimization Algorithm Accelerated by Asynchronous Evaluations , 2005 .

[32]  James Kennedy,et al.  Proceedings of the 1998 IEEE International Conference on Evolutionary Computation [Book Review] , 1999, IEEE Transactions on Evolutionary Computation.

[33]  Randy L. Haupt,et al.  Practical Genetic Algorithms , 1998 .

[34]  Hitoshi Iba,et al.  Particle swarm optimization with Gaussian mutation , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[35]  Yuhui Shi,et al.  Particle swarm optimization: developments, applications and resources , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[36]  P. Fourie,et al.  The particle swarm optimization algorithm in size and shape optimization , 2002 .

[37]  B. Bochenek,et al.  Structural optimization for post-buckling behavior using particle swarms , 2006 .

[38]  Y. Rahmat-Samii,et al.  Reconfigurable array design using parallel particle swarm optimization , 2003, IEEE Antennas and Propagation Society International Symposium. Digest. Held in conjunction with: USNC/CNC/URSI North American Radio Sci. Meeting (Cat. No.03CH37450).

[39]  S. Jayanti,et al.  Corrosion fatigue through particle swarm optimization , 2003 .

[40]  Wang Yi,et al.  An Adaptive Stochastic Collision Detection Between Deformable Objects Using Particle Swarm Optimization , 2006, EvoWorkshops.

[41]  James H. Oliver,et al.  UAV Swarm Control: Calculating Digital Pheromone Fields with the GPU , 2006 .

[42]  Eliot Winer,et al.  Improving solution characteristics of particle swarm optimization using digital pheromones , 2009 .

[43]  Godfrey C. Onwubolu,et al.  Optimal path for automated drilling operations by a new heuristic approach using particle swarm optimization , 2004 .

[44]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[45]  Tapabrata Ray,et al.  ENGINEERING DESIGN OPTIMIZATION USING A SWARM WITH AN INTELLIGENT INFORMATION SHARING AMONG INDIVIDUALS , 2001 .

[46]  Yoshikazu Fukuyama,et al.  A particle swarm optimization for reactive power and voltage control considering voltage security assessment , 2000 .

[47]  A. Groenwold,et al.  Comparison of linear and classical velocity update rules in particle swarm optimization: notes on diversity , 2007 .

[48]  Imtiaz Ahmad,et al.  Particle swarm optimization for task assignment problem , 2002, Microprocess. Microsystems.

[49]  R. K. Suresh,et al.  Discrete Particle Swarm Optimization (DPSO) Algorithm for Permutation Flowshop Scheduling to Minimize Makespan , 2005, ICNC.

[50]  Koetsu Yamazaki,et al.  Penalty function approach for the mixed discrete nonlinear problems by particle swarm optimization , 2006 .

[51]  Jiang Chuanwen,et al.  A hybrid method of chaotic particle swarm optimization and linear interior for reactive power optimisation , 2005, Math. Comput. Simul..

[52]  D.S. Weile,et al.  Application of a parallel particle swarm optimization scheme to the design of electromagnetic absorbers , 2005, IEEE Transactions on Antennas and Propagation.

[53]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[54]  Singiresu S. Rao.  Engineering Optimization : Theory and Practice , 2010 .

[55]  Michael N. Vrahatis,et al.  On the computation of all global minimizers through particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[56]  Carlos A. Coello Coello,et al.  Handling multiple objectives with particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[57]  Tony White,et al.  Towards multi-swarm problem solving in networks , 1998, Proceedings International Conference on Multi Agent Systems (Cat. No.98EX160).

[58]  Eliot Winer,et al.  A parallel implementation of particle swarm optimization using digital pheromones , 2006 .

[59]  Godfrey C. Onwubolu,et al.  New optimization techniques in engineering , 2004, Studies in Fuzziness and Soft Computing.

[60]  Jing Liu,et al.  Quantum-Behaved Particle Swarm Optimization for Integer Programming , 2006, ICONIP.

[61]  P. Eberhard,et al.  Using Augmented Lagrangian Particle Swarm Optimization for Constrained Problems in Engineering , 2009 .

[62]  Byung-Il Koh,et al.  Parallel asynchronous particle swarm optimization , 2006, International journal for numerical methods in engineering.

[63]  Eliot Winer,et al.  Three-Dimensional Path Planning of Unmanned Aerial Vehicles Using Particle Swarm Optimization , 2006 .

[64]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[65]  Anthony Skjellum,et al.  Using MPI - portable parallel programming with the message-parsing interface , 1994 .

[66]  Eric Bonabeau,et al.  Swarm Intelligence: A New C2 Paradigm with an Application to Control Swarms of UAVs , 2003 .

[67]  Yanchun Liang,et al.  Hybrid evolutionary algorithms based on PSO and GA , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[68]  Y. Rahmat-Samii,et al.  Particle swarm optimization in electromagnetics , 2004, IEEE Transactions on Antennas and Propagation.

[69]  Marco Dorigo,et al.  Ant system: optimization by a colony of cooperating agents , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[70]  Subramaniam Rajan,et al.  Design optimization of discrete structural systems using MPI-enabled genetic algorithm , 2004 .

[71]  Y. Rahmat-Samii,et al.  Parallel particle swarm optimization and finite- difference time-domain (PSO/FDTD) algorithm for multiband and wide-band patch antenna designs , 2005, IEEE Transactions on Antennas and Propagation.

[72]  Jigui Sun,et al.  An Improved Discrete Particle Swarm Optimization Algorithm for TSP , 2007, 2007 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology - Workshops.

[73]  J. Sobieszczanski-Sobieski,et al.  Multidisciplinary optimization of a transport aircraft wing using particle swarm optimization , 2004 .

[74]  Albert A. Groenwold,et al.  Sizing design of truss structures using particle swarms , 2003 .

[75]  P. Eberhard,et al.  Using augmented Lagrangian particle swarm optimization for constrained problems in engineering">Using augmented Lagrangian particle swarm optimization for constrained problems in engineering , 2006 .

[76]  Barron J. Bichon,et al.  Design of Steel Frames Using Ant Colony Optimization , 2005 .

[77]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).