A Comparative Study of CMA-ES on Large Scale Global Optimisation

In this paper, we investigate the performance of CMA-ES on large scale non-separable optimisation problems. CMA-ES is a robust local optimiser that has shown great performance on small-scale non-separable optimisation problems. Self-adaptation of a covariance matrix makes it rotational invariant which is a desirable property, especially for solving non-separable problems. The focus of this paper is to compare the performance of CMA-ES with Cooperative Co-evolutionary Algorithms (CCEAs) for large scale global optimisation (on problems with up to 1000 real-valued variables). Since the original CMA-ES is incapable of handling problems with more than several hundreds dimensions, sep-CMA-ES was developed using only the diagonal elements of the covariance matrix. In this paper sep-CMA-ES is compared with several existing CCEAs. Experimental results revealed that the performance of sep-CMA-ES drops significantly when the dimensionality of the problem increases. However, our results suggest that the rotational invariant property of CMA-ES can be utilised in conjunction with a CCEA to further enhance its capability to handle large scale optimisation problems.

[1]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[2]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[3]  Kenneth A. De Jong,et al.  A Cooperative Coevolutionary Approach to Function Optimization , 1994, PPSN.

[4]  Simon M. Lucas,et al.  Parallel Problem Solving from Nature - PPSN X, 10th International Conference Dortmund, Germany, September 13-17, 2008, Proceedings , 2008, PPSN.

[5]  Nikolaus Hansen,et al.  Evaluating the CMA Evolution Strategy on Multimodal Test Functions , 2004, PPSN.

[6]  Hans-Paul Schwefel,et al.  Evolution strategies – A comprehensive introduction , 2002, Natural Computing.

[7]  Xin Yao,et al.  Large scale evolutionary optimization using cooperative coevolution , 2008, Inf. Sci..

[8]  Hongfei Teng,et al.  Cooperative Co-evolutionary Differential Evolution for Function Optimization , 2005, ICNC.

[9]  Xiaodong Li,et al.  Cooperative Co-evolution with delta grouping for large scale non-separable function optimization , 2010, IEEE Congress on Evolutionary Computation.

[10]  R. Salomon.  Re-evaluating genetic algorithm performance under coordinate rotation of benchmark functions. A survey of some theoretical and practical aspects of genetic algorithms. , 1996, Bio Systems.

[11]  Andries Petrus Engelbrecht,et al.  A Cooperative approach to particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[12]  Raymond Ros,et al.  A Simple Modification in CMA-ES Achieving Linear Time and Space Complexity , 2008, PPSN.

[13]  Christian Igel,et al.  A computational efficient covariance matrix update and a (1+1)-CMA for evolution strategies , 2006, GECCO.

[14]  Nikolaus Hansen,et al.  Benchmarking a BI-population CMA-ES on the BBOB-2009 function testbed , 2009, GECCO '09.

[15]  Xiaodong Li,et al.  Benchmark Functions for the CEC'2010 Special Session and Competition on Large-Scale , 2009 .

[16]  Tapabrata Ray,et al.  A cooperative coevolutionary algorithm with Correlation based Adaptive Variable Partitioning , 2009, 2009 IEEE Congress on Evolutionary Computation.

[17]  Zbigniew Michalewicz,et al.  Handbook of Evolutionary Computation , 1997 .

[18]  Nikolaus Hansen,et al.  A restart CMA evolution strategy with increasing population size , 2005, 2005 IEEE Congress on Evolutionary Computation.

[19]  Thomas Bäck,et al.  Evolutionary algorithms in theory and practice - evolution strategies, evolutionary programming, genetic algorithms , 1996 .

[20]  X. Yao,et al.  Scaling up fast evolutionary programming with cooperative coevolution , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[21]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[22]  Xiaodong Li,et al.  Cooperative Co-evolution for large scale optimization through more frequent random grouping , 2010, IEEE Congress on Evolutionary Computation.

[23]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[24]  Petros Koumoutsakos,et al.  Reducing the Time Complexity of the Derandomized Evolution Strategy with Covariance Matrix Adaptation (CMA-ES) , 2003, Evolutionary Computation.