Measuring the curse of dimensionality and its effects on particle swarm optimization and differential evolution

The existence of the curse of dimensionality is well known, and its general effects are well acknowledged. However, and perhaps due to this colloquial understanding, specific measurements on the curse of dimensionality and its effects are not as extensive. In continuous domains, the volume of the search space grows exponentially with dimensionality. Conversely, the number of function evaluations budgeted to explore this search space usually grows only linearly. The divergence of these growth rates has important effects on the parameters used in particle swarm optimization and differential evolution as dimensionality increases. New experiments focus on the effects of population size and key changes to the search characteristics of these popular metaheuristics when population size is less than the dimensionality of the search space. Results show how design guidelines developed for low-dimensional implementations can become unsuitable for high-dimensional search spaces.

[1]  Lixing Han,et al.  Implementing the Nelder-Mead simplex algorithm with adaptive parameters , 2010, Computational Optimization and Applications.

[2]  El-Ghazali Talbi,et al.  Metaheuristics - From Design to Implementation , 2009 .

[3]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[4]  James Montgomery,et al.  An analysis of the operation of differential evolution at high and low crossover rates , 2010, IEEE Congress on Evolutionary Computation.

[5]  Anne Auger,et al.  Real-Parameter Black-Box Optimization Benchmarking 2009: Noiseless Functions Definitions , 2009 .

[6]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[7]  James Kennedy,et al.  Defining a Standard for Particle Swarm Optimization , 2007, 2007 IEEE Swarm Intelligence Symposium.

[8]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[9]  M. R. Meybodi,et al.  CLA-DE: a hybrid model based on cellular learning automata for numerical optimization , 2012, Applied Intelligence.

[10]  Gerhard W. Dueck,et al.  Threshold accepting: a general purpose optimization algorithm appearing superior to simulated anneal , 1990 .

[11]  James Montgomery.  Differential evolution: Difference vectors and movement in solution space , 2009, 2009 IEEE Congress on Evolutionary Computation.

[12]  Kenya Jin'no,et al.  Analysis of dynamical characteristic of canonical deterministic PSO , 2010, IEEE Congress on Evolutionary Computation.

[13]  Ponnuthurai N. Suganthan,et al.  Empirical study on the effect of population size on Differential evolution Algorithm , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[14]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[15]  R. Eberhart,et al.  Empirical study of particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[16]  John J. Grefenstette,et al.  Optimization of Control Parameters for Genetic Algorithms , 1986, IEEE Transactions on Systems, Man, and Cybernetics.

[17]  Jürgen Branke,et al.  Experimental Analysis of Bound Handling Techniques in Particle Swarm Optimization , 2013, IEEE Transactions on Evolutionary Computation.

[18]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[19]  Silvia Curteanu,et al.  Modeling of oxygen mass transfer in the presence of oxygen-vectors using neural networks developed by differential evolution algorithm , 2011, Eng. Appl. Artif. Intell..

[20]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[21]  M. Clerc,et al.  The swarm and the queen: towards a deterministic and adaptive particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[22]  Bogdan M. Wilamowski,et al.  Improved nelder mead's simplex method and applications , 2012 .

[23]  R. Storn,et al.  On the usage of differential evolution for function optimization , 1996, Proceedings of North American Fuzzy Information Processing.

[24]  Tim Hendtlass,et al.  Particle Swarm Optimisation and high dimensional problem spaces , 2009, 2009 IEEE Congress on Evolutionary Computation.

[25]  Wei Chu,et al.  A new evolutionary search strategy for global optimization of high-dimensional problems , 2011, Inf. Sci..

[26]  Zhang Li-ping,et al.  Optimal choice of parameters for particle swarm optimization , 2005 .

[27]  Andries Petrus Engelbrecht,et al.  Particle swarm optimization: Velocity initialization , 2012, 2012 IEEE Congress on Evolutionary Computation.