Towards Swarm Diversity: Random Sampling in Variable Neighborhoods Procedure Using a Lévy Distribution

Particle Swarm Optimization (PSO) is a non- direct search method for numerical optimization. The key advantages of this metaheuristic are principally associated to its simplicity, few parameters and high convergence rate. In the canonical PSO using a fully connected topology, a particle adjusts its position by using two attractors: the best record stored for the current agent, and the best point discovered for the entire swarm. It leads to a high convergence rate, but also progressively deteriorates the swarm diversity. As a result, the particle swarm frequently gets attracted by sub-optimal points. Once the particles have been attracted to a local optimum, they continue the search process within a small region of the solution space, thus reducing the algorithm exploration. To deal with this issue, this paper presents a variant of the Random Sampling in Variable Neighborhoods (RSVN) procedure using a Levy distribution, which is able to notably improve the PSO search ability in multimodal problems.

[1]  Rui Mendes,et al.  Neighborhood topologies in fully informed and best-of-neighborhood particle swarms , 2006 .

[2]  Jun Zhang,et al.  Adaptive Particle Swarm Optimization , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[3]  Hamid R. Tizhoosh,et al.  Opposition-Based Reinforcement Learning , 2006, J. Adv. Comput. Intell. Intell. Informatics.

[4]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[5]  Yu Liu,et al.  A Survey on Particle Swarm Optimization Algorithms for Multimodal Function Optimization , 2011, J. Softw..

[6]  Gonzalo Nápoles,et al.  Modelling, Aggregation and Simulation of a Dynamic Biological System through Fuzzy Cognitive Maps , 2012, MICAI.

[7]  Rafael Bello,et al.  Constricted Particle Swarm Optimization based Algorithm for Global Optimization , 2012, Polibits.

[8]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[9]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[10]  M. Friedman.  The Use of Ranks to Avoid the Assumption of Normality Implicit in the Analysis of Variance , 1937 .

[11]  James Kennedy,et al.  Particle swarm optimization , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[12]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[13]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[14]  Francisco Herrera,et al.  A study of statistical techniques and performance measures for genetics-based machine learning: accuracy and interpretability , 2009, Soft Comput..

[15]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[16]  Rafael Bello,et al.  Particle Swarm Optimization with Random Sampling in Variable Neighbourhoods for Solving Global Minimization Problems , 2012, ANTS.

[17]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[18]  Tim M. Blackwell,et al.  The Lévy Particle Swarm , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[19]  R. Thangaraj,et al.  A New Particle Swarm Optimization with Quadratic Crossover , 2007, 15th International Conference on Advanced Computing and Communications (ADCOM 2007).

[20]  Mounir Ben Ghalia,et al.  Regrouping particle swarm optimization: A new global optimization algorithm with improved performance consistency across benchmarks , 2009, 2009 IEEE International Conference on Systems, Man and Cybernetics.

[21]  Ilya Pavlyukevich.  Lévy flights, non-local search and simulated annealing , 2007, J. Comput. Phys..

[22]  Yu Wang,et al.  Self-adaptive learning based particle swarm optimization , 2011, Inf. Sci..

[23]  Andries Petrus Engelbrecht,et al.  Measuring exploration/exploitation in particle swarms using swarm diversity , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[24]  Xin Yao,et al.  Evolutionary programming using mutations based on the Levy probability distribution , 2004, IEEE Transactions on Evolutionary Computation.

[25]  B. Gnedenko,et al.  Limit Distributions for Sums of Independent Random Variables , 1955 .

[26]  Hui Wang,et al.  Opposition-based particle swarm algorithm with cauchy mutation , 2007, 2007 IEEE Congress on Evolutionary Computation.

[27]  F. Wilcoxon.  Individual Comparisons by Ranking Methods , 1945 .

[28]  Jing J. Liang,et al.  Comprehensive learning particle swarm optimizer for global optimization of multimodal functions , 2006, IEEE Transactions on Evolutionary Computation.

[29]  Francisco Herrera,et al.  A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 Special Session on Real Parameter Optimization , 2009, J. Heuristics.

[30]  R. Mantegna,et al.  Fast, accurate algorithm for numerical simulation of Lévy stable stochastic processes. , 1994, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[31]  James Montgomery,et al.  A Simple Strategy to Maintain Diversity and Reduce Crowding in Particle Swarm Optimization , 2011, Australasian Conference on Artificial Intelligence.

[32]  James Kennedy,et al.  Bare bones particle swarms , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[33]  James Kennedy,et al.  Defining a Standard for Particle Swarm Optimization , 2007, 2007 IEEE Swarm Intelligence Symposium.

[34]  James Kennedy,et al.  Probability and dynamics in the particle swarm , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[35]  Xin-She Yang,et al.  Nature-Inspired Metaheuristic Algorithms , 2008 .

[36]  J. J. Higgins.  Introduction to Modern Nonparametric Statistics , 2003 .

[37]  Ioan Cristian Trelea,et al.  The particle swarm optimization algorithm: convergence analysis and parameter selection , 2003, Inf. Process. Lett..