Population statistics for particle swarm optimization: Hybrid methods in noisy optimization problems

Particle swarm optimization (PSO) is a metaheuristic designed to find good solutions to optimization problems. However, when optimization problems are subject to noise, the quality of the resulting solutions significantly deteriorates, hence prompting the need to incorporate noise mitigation mechanisms into PSO. Based on the allocation of function evaluations, two opposite approaches are generally taken. On the one hand, resampling-based PSO algorithms incorporate resampling methods to better estimate the objective function values of the solutions at the cost of performing fewer iterations. On the other hand, single-evaluation PSO algorithms perform more iterations at the cost of dealing with very inaccurately estimated objective function values. In this paper, we propose a new approach in which hybrid PSO algorithms incorporate noise mitigation mechanisms from the other two approaches, and the quality of their results is better than that of the state of the art with a few exceptions. The performance of the algorithms is analyzed by means of a set of population statistics that measure different characteristics of the swarms throughout the search process. Amongst the hybrid PSO algorithms, we find a promising algorithm whose simplicity, flexibility and quality of results questions the importance of incorporating complex resampling methods into state-of-the-art PSO algorithms. & 2015 Elsevier B.V. All rights reserved.

[1]  Andries Petrus Engelbrecht,et al.  Heterogeneous particle swarms in dynamic environments , 2011, 2011 IEEE Symposium on Swarm Intelligence.

[2]  J. Proudfoot,et al.  Noise , 1931, The Indian medical gazette.

[3]  Xiaodong Li,et al.  Benchmark Functions for the CEC'2010 Special Session and Competition on Large-Scale , 2009 .

[4]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[5]  Winston Khoon Guan Seah,et al.  A performance study on synchronicity and neighborhood size in particle swarm optimization , 2013, Soft Comput..

[6]  Andries Petrus Engelbrecht,et al.  Fundamentals of Computational Swarm Intelligence , 2005 .

[7]  Chrysostomos D. Stylios,et al.  Integrating particle swarm optimization with reinforcement learning in noisy problems , 2012, GECCO '12.

[8]  Andries P. Engelbrecht.  Heterogeneous Particle Swarm Optimization , 2010, ANTS Conference.

[9]  Chukwudi Anyakoha,et al.  A review of particle swarm optimization. Part II: hybridisation, combinatorial, multicriteria and constrained optimization, and indicative applications , 2008, Natural Computing.

[10]  Mark Johnston,et al.  Resampling in Particle Swarm Optimization , 2013, 2013 IEEE Congress on Evolutionary Computation.

[11]  Dirk V. Arnold,et al.  Noisy Optimization With Evolution Strategies , 2002, Genetic Algorithms and Evolutionary Computation.

[12]  David E. Goldberg,et al.  Genetic Algorithms, Tournament Selection, and the Effects of Noise , 1995, Complex Syst..

[13]  Kalyanmoy Deb,et al.  Genetic Algorithms, Noise, and the Sizing of Populations , 1992, Complex Syst..

[14]  Mengjie Zhang,et al.  Population statistics for particle swarm optimization: Resampling methods in noisy optimization problems , 2014, Swarm Evol. Comput..

[15]  David E. Goldberg,et al.  Genetic Algorithms, Selection Schemes, and the Varying Effects of Noise , 1996, Evolutionary Computation.

[16]  A. P. Engelbrecht,et al.  Particle Swarm Optimization: Global Best or Local Best? , 2013, 2013 BRICS Congress on Computational Intelligence and 11th Brazilian Congress on Computational Intelligence.

[17]  T. T. Soong,et al.  Fundamentals of Probability and Statistics for Engineers , 2004 .

[18]  Ling Wang,et al.  Particle swarm optimization for function optimization in noisy environment , 2006, Appl. Math. Comput..

[19]  Jose Luis Fernandez-Marquez,et al.  Adapting Particle Swarm Optimization in dynamic and noisy environments , 2010, IEEE Congress on Evolutionary Computation.

[20]  Dan Boneh,et al.  On genetic algorithms , 1995, COLT '95.

[21]  Jose Luis Fernandez-Marquez,et al.  An evaporation mechanism for dynamic and noisy multimodal optimization , 2009, GECCO.

[22]  Riccardo Poli,et al.  Analysis of the publications on the applications of particle swarm optimisation , 2008 .

[23]  Andries Petrus Engelbrecht.  Scalability of a heterogeneous particle swarm optimizer , 2011, 2011 IEEE Symposium on Swarm Intelligence.

[24]  Thomas Bartz-Beielstein,et al.  Particle Swarm Optimization and Sequential Sampling in Noisy Environments , 2007, Metaheuristics.

[25]  Anthony Di Pietro.  Optimising evolutionary strategies for problems with varying noise strength , 2007 .

[26]  Winston Khoon Guan Seah,et al.  Evaporation Mechanisms for Particle Swarm Optimization , 2012, SEAL.

[27]  Winston Khoon Guan Seah,et al.  A performance study on the effects of noise and evaporation in Particle Swarm Optimization , 2012, 2012 IEEE Congress on Evolutionary Computation.

[28]  Thomas E. Potok,et al.  Tracking non-stationary optimal solution by particle swarm optimizer , 2005, Sixth International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing and First ACIS International Workshop on Self-Assembling Wireless Network.

[29]  Mark Johnston,et al.  Optimal computing budget allocation in particle swarm optimization , 2013, GECCO '13.

[30]  Thomas E. Potok,et al.  A Simple Distributed Particle Swarm Optimization for Dynamic and Noisy Environments , 2008, NICSO.

[31]  Frans van den Bergh,et al.  An analysis of particle swarm optimizers , 2002 .

[32]  Andries Petrus Engelbrecht,et al.  A self-adaptive heterogeneous pso for real-parameter optimization , 2013, 2013 IEEE Congress on Evolutionary Computation.

[33]  Bo Liu,et al.  Hybrid Particle Swarm Optimization for Flow Shop Scheduling with Stochastic Processing Time , 2005, CIS.

[34]  Salman Mohagheghi,et al.  Particle Swarm Optimization: Basic Concepts, Variants and Applications in Power Systems , 2008, IEEE Transactions on Evolutionary Computation.

[35]  Jenny Mead,et al.  Deception , 2007, Journal of empirical research on human research ethics : JERHRE.

[36]  Mark Johnston,et al.  Deception, blindness and disorientation in particle swarm optimization applied to noisy problems , 2014, Swarm Intelligence.

[37]  Andries Petrus Engelbrecht,et al.  A Self-adaptive Heterogeneous PSO Inspired by Ants , 2012, ANTS.

[38]  Jürgen Branke,et al.  Evolutionary optimization in uncertain environments-a survey , 2005, IEEE Transactions on Evolutionary Computation.

[39]  Hans-Georg Beyer,et al.  Toward a Theory of Evolution Strategies: On the Benefits of Sex the (/, ) Theory , 1995, Evolutionary Computation.

[40]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[41]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[42]  Chun-Hung Chen,et al.  Simulation Budget Allocation for Further Enhancing the Efficiency of Ordinal Optimization , 2000, Discret. Event Dyn. Syst..

[43]  Thomas E. Potok,et al.  Distributed Adaptive Particle Swarm Optimizer in Dynamic Environment , 2007, 2007 IEEE International Parallel and Distributed Processing Symposium.

[44]  Loo Hay Lee,et al.  Simulation optimization using the Particle Swarm Optimization with optimal computing budget allocation , 2011, Proceedings of the 2011 Winter Simulation Conference (WSC).

[45]  Juan Rada-Vilela,et al.  Population Statistics for Particle Swarm Optimization on Problems Subject to Noise , 2014 .

[46]  B. Latané,et al.  From private attitude to public opinion: A dynamic theory of social impact. , 1990 .

[47]  Loo Hay Lee,et al.  Stochastic Simulation Optimization - An Optimal Computing Budget Allocation , 2010, System Engineering and Operations Research.