Parallel Evolutionary Algorithms Performing Pairwise Comparisons

We study mathematically and experimentally the convergence rate of differential evolution and particle swarm optimization for simple unimodal functions. Due to parallelization concerns, the focus is on lower bounds on the runtime, i.e. upper bounds on the speed-up, as a function of the population size. Two cases are particularly relevant: A population size of the same order of magnitude as the dimension and larger population sizes. We use the branching factor as a tool for proving bounds and get, as upper bounds, a linear speed-up for a population size similar to the dimension, and a logarithmic speed-up for larger population sizes. We then propose parametrizations for differential evolution and particle swarm optimization that reach these bounds.

[1]  Olivier Teytaud,et al.  Log(lambda) Modifications for Optimal Parallelism , 2010, PPSN.

[2]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[3]  B J Fregly,et al.  Parallel global optimization with the particle swarm algorithm , 2004, International journal for numerical methods in engineering.

[4]  Jouni Lampinen,et al.  A Fuzzy Adaptive Differential Evolution Algorithm , 2005, Soft Comput..

[5]  László Györfi,et al.  A Probabilistic Theory of Pattern Recognition , 1996, Stochastic Modelling and Applied Probability.

[6]  A. Auger.  Convergence results for the ( 1 , )-SA-ES using the theory of-irreducible Markov chains , 2005 .

[7]  Wei-jie Yu,et al.  Multi-population differential evolution with adaptive parameter control for global optimization , 2011, GECCO '11.

[8]  A. Kai Qin,et al.  An improved CUDA-based implementation of differential evolution on GPU , 2012, GECCO '12.

[9]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[10]  Janez Brest,et al.  Self-Adapting Control Parameters in Differential Evolution: A Comparative Study on Numerical Benchmark Problems , 2006, IEEE Transactions on Evolutionary Computation.

[11]  Lu Wang,et al.  Self-adapting Differential Evolution Algorithm with Chaos Random for Global Numerical Optimization , 2010, ISICA.

[12]  Olivier Teytaud,et al.  General Lower Bounds for Evolutionary Algorithms , 2006, PPSN.

[13]  David Ardia,et al.  Differential Evolution with DEoptim: An Application to Non-Convex Portfolio Optimization , 2010, R J..

[14]  Maurice Clerc,et al.  Beyond Standard Particle Swarm Optimisation , 2010, Int. J. Swarm Intell. Res..

[15]  Belkacem Mahdad,et al.  Fuzzy controlled parallel PSO to solving large practical economic dispatch , 2010, 2010 IEEE International Energy Conference.

[16]  T. Zaslavsky.  Facing Up to Arrangements: Face-Count Formulas for Partitions of Space by Hyperplanes , 1975 .

[17]  David Ardia,et al.  Jump-Diffusion Calibration Using Differential Evolution , 2010 .

[18]  Pierre Collet,et al.  PARADE: A Massively Parallel Differential Evolution Template for EASEA , 2012, ICAISC.

[19]  Ioan Cristian Trelea,et al.  The particle swarm optimization algorithm: convergence analysis and parameter selection , 2003, Inf. Process. Lett..

[20]  Ville Tirronen,et al.  Parallel Random Injection Differential Evolution , 2010, EvoApplications.

[21]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[22]  Jeng-Shyang Pan,et al.  A Parallel Particle Swarm Optimization Algorithm with Communication Strategies , 2005, J. Inf. Sci. Eng..

[23]  Per Kristian Lehre,et al.  Unbiased Black-Box Complexity of Parallel Search , 2014, PPSN.

[24]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[25]  Mehmet Fatih Tasgetiren,et al.  Differential evolution algorithm with ensemble of parameters and mutation strategies , 2011, Appl. Soft Comput..

[26]  Hans-Georg Beyer,et al.  Toward a Theory of Evolution Strategies: Some Asymptotical Results from the (1,+ )-Theory , 1993, Evolutionary Computation.

[27]  David Ardia,et al.  Differential Evolution (DEoptim) for Non-Convex Portfolio Optimization , 2010 .

[28]  Anne Auger,et al.  Convergence results for the (1, lambda)-SA-ES using the theory of phi-irreducible Markov chains , 2005, Theor. Comput. Sci..

[29]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[30]  Michael N. Vrahatis,et al.  Parameter selection and adaptation in Unified Particle Swarm Optimization , 2007, Math. Comput. Model..

[31]  Kevin D. Seppi,et al.  Parallel PSO using MapReduce , 2007, 2007 IEEE Congress on Evolutionary Computation.

[32]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[33]  Vladimir Vapnik,et al.  The Nature of Statistical Learning , 1995 .

[34]  M. E. H. Pedersen,et al.  Tuning & simplifying heuristical optimization , 2010 .

[35]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[36]  Janez Puhan,et al.  A new asynchronous parallel global optimization method based on simulated annealing and differential evolution , 2011, Appl. Soft Comput..

[37]  Olivier Teytaud,et al.  Algorithms (X, sigma, eta): Quasi-random Mutations for Evolution Strategies , 2005, Artificial Evolution.

[38]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[39]  James Kennedy,et al.  Defining a Standard for Particle Swarm Optimization , 2007, 2007 IEEE Swarm Intelligence Symposium.

[40]  Hans-Georg Beyer,et al.  The Theory of Evolution Strategies , 2001, Natural Computing Series.

[41]  Petr Posík,et al.  JADE, an adaptive differential evolution algorithm, benchmarked on the BBOB noiseless testbed , 2012, GECCO '12.

[42]  Ponnuthurai N. Suganthan,et al.  An Adaptive Differential Evolution Algorithm With Novel Mutation and Crossover Strategies for Global Numerical Optimization , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[43]  Mauricio Zambrano-Bigiarini,et al.  Standard Particle Swarm Optimisation 2011 at CEC-2013: A baseline for future PSO improvements , 2013, 2013 IEEE Congress on Evolutionary Computation.

[44]  Hans-Georg Beyer,et al.  Toward a Theory of Evolution Strategies: The (, )-Theory , 1994, Evolutionary Computation.

[45]  Terence General Chair-Moore Jason H. Soule,et al.  Proceedings of the fourteenth international conference on Genetic and evolutionary computation conference companion , 2012 .

[46]  Hans-Georg Beyer,et al.  On the Dynamics of EAs without Selection , 1998, FOGA.

[47]  Kevin D. Seppi,et al.  A speculative approach to parallelization in particle swarm optimization , 2012, Swarm Intelligence.

[48]  Olivier Teytaud,et al.  Lower Bounds for Comparison Based Evolution Strategies Using VC-dimension and Sign Patterns , 2011, Algorithmica.

[49]  Arthur C. Sanderson,et al.  JADE: Adaptive Differential Evolution With Optional External Archive , 2009, IEEE Transactions on Evolutionary Computation.