A study of particle swarm optimization for cognitive machines

This paper presents a study of the properties of optimization algorithms for use in cognitive machines through five key measures: (i) speed of convergence, (ii) degree of exploration of the parameter space, (iii) storage and system size, (iv) adaptability, and (v) multi-scale capabilities. Based on these factors, a novel study of the trajectories of a particle in the particle swarm optimization algorithm is performed both in the time and frequency domain. The analysis shows that the trajectories of particles can be separated into a transient and a steady state periods where the transient is wide-sense stationary with long term dependancies that show the evolutionary properties of the algorithm as it converges on a solution. The steady state shows an increased degree of exploration of the parameter space that allow the algorithm to improve on the solution found over time. The results show the advantages of particle swarm optimization and inherent properties that make this optimization algorithm a suitable choice for use in cognitive machines. The information learned from this analysis can further be used to extract complexity measures to classify the behavior and control of particle swarm optimization, and make proper quick decisions on what to do next. The decision process often requires more alternatives to be considered in a short window of time than it is physically possible for a real-time system [Kins04]. Thus, in order to make good decisions without exploring all possible paths, a cognitive system requires optimization techniques that can survey the possible options, and quickly select the best option possible. The paper reviews the requirements for an ideal optimization technique for use in cognitive systems and proposes the particle swarm optimization algorithm as one technique that is designed to satisfy these requirements. In order to show the properties of PSO, a novel trajectory, time, and frequency domain analyses of single particles along individual dimensions are presented.

[1]  José Neves,et al.  The fully informed particle swarm: simpler, maybe better , 2004, IEEE Transactions on Evolutionary Computation.

[2]  J. Kennedy,et al.  Population structure and particle swarm performance , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[3]  James Kennedy,et al.  Some Issues and Practices for Particle Swarms , 2007, 2007 IEEE Swarm Intelligence Symposium.

[4]  刘秀丽,et al.  Kennedy病一例报告 , 2005 .

[5]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[6]  Frans van den Bergh,et al.  An analysis of particle swarm optimizers , 2002 .

[7]  R. Eberhart,et al.  Empirical study of particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[8]  Peter J. Angeline,et al.  Evolutionary Optimization Versus Particle Swarm Optimization: Philosophy and Performance Differences , 1998, Evolutionary Programming.

[9]  Simon Haykin.  Cognitive Radar Networks , 2006 .

[10]  W. Kinsner,et al.  Chaotic simulated annealing in multilayer feedforward networks , 1996, Proceedings of 1996 Canadian Conference on Electrical and Computer Engineering.

[11]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[12]  Donald E. Grierson,et al.  Comparison among five evolutionary-based optimization algorithms , 2005, Adv. Eng. Informatics.

[13]  Ioan Cristian Trelea,et al.  The particle swarm optimization algorithm: convergence analysis and parameter selection , 2003, Inf. Process. Lett..

[14]  James Kennedy,et al.  Defining a Standard for Particle Swarm Optimization , 2007, 2007 IEEE Swarm Intelligence Symposium.

[15]  Bo Liu,et al.  Improved particle swarm optimization combined with chaos , 2005 .

[16]  James Kennedy,et al.  The particle swarm: social adaptation of knowledge , 1997, Proceedings of 1997 IEEE International Conference on Evolutionary Computation (ICEC '97).

[17]  R. Eberhart,et al.  Comparing inertia weights and constriction factors in particle swarm optimization , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[18]  Simon Haykin,et al.  Cognitive radio: brain-empowered wireless communications , 2005, IEEE Journal on Selected Areas in Communications.

[19]  H. Kantz,et al.  Nonlinear time series analysis , 1997 .

[20]  Vijay K. Bhargava,et al.  Cognitive Wireless Communication Networks , 2007 .

[21]  K. Fernow.  New York , 1896, American Potato Journal.

[22]  James Kennedy,et al.  Small worlds and mega-minds: effects of neighborhood topology on particle swarm performance , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[23]  Andries Petrus Engelbrecht,et al.  A study of particle swarm optimization particle trajectories , 2006, Inf. Sci..

[24]  Dario Floreano,et al.  Bio-inspired artificial intelligence , 2008 .

[25]  Witold Kinsner,et al.  A study of optimal topologies in swarm intelligence , 2010, CCECE 2010.

[26]  Witold Kinsner.  A Unified Approach To Fractal Dimensions , 2008, J. Inf. Technol. Res..

[27]  Feng Quanyuan,et al.  The Standard Particle Swarm Optimization Algorithm Convergence Analysis and Parameter Selection , 2007, Third International Conference on Natural Computation (ICNC 2007).

[28]  Witold Kinsner.  Is entropy suitable to characterize data and signals for cognitive informatics? , 2004, Proceedings of the Third IEEE International Conference on Cognitive Informatics, 2004..

[29]  Yingxu Wang.  On Cognitive Informatics , 2003 .

[30]  Rainer Laur,et al.  Stopping Criteria for Single-Objective Optimization , 2005 .

[31]  S. Haykin,et al.  Cognitive radar: a way of the future , 2006, IEEE Signal Processing Magazine.

[32]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[33]  R. Eberhart,et al.  Particle Swarm Optimization-Neural Networks, 1995. Proceedings., IEEE International Conference on , 2004 .