Model complex control CMA-ES

Abstract Covariance Matrix Adaptation Evolution Strategy (CMA-ES) has shown great performance on nonseparable optimization problems largely due to its rotation-invariant feature. However, as the computational cost of the self-adaption operation is sensitive to the scale of problems, the performance of CMA-ES heavily suffers from the well-known curse of dimensionality, which makes it impractical to many Large Scale Global Optimization (LSGO) problems. In this paper, a correlation coefficient based grouping (CCG) strategy is proposed to detect the correlations between variables in a simple yet efficient way. Then coupled with a model complexity control (MCC) framework, a new variant of CMA-ES, named MCC-CCG-CMAES, is presented for LSGO problems, which suffers less from curse of dimensionality and significantly reduces the computational cost compared with the standard CMA-ES. To the best of our knowledge, this work is the first attempt at enhancing CMA-ES with the MCC framework rather than the cooperative coevolution (CC) framework. Experimental results on the CEC′2010 large-scale global optimization (LSGO) benchmark functions show that the performance of MCC-CCG-CMAES outperforms the state-of-the-art counterparts.

[1]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[2]  Ilya Loshchilov,et al.  A computationally efficient limited memory CMA-ES for large scale optimization , 2014, GECCO.

[3]  Xiaodong Li,et al.  A Comparative Study of CMA-ES on Large Scale Global Optimisation , 2010, Australasian Conference on Artificial Intelligence.

[4]  Xiaodong Li,et al.  Benchmark Functions for the CEC'2010 Special Session and Competition on Large-Scale , 2009 .

[5]  J. MacQueen Some methods for classification and analysis of multivariate observations , 1967 .

[6]  Ata Kabán,et al.  Heavy tails with parameter adaptation in random projection based continuous EDA , 2015, 2015 IEEE Congress on Evolutionary Computation (CEC).

[7]  Delbert Dueck,et al.  Clustering by Passing Messages Between Data Points , 2007, Science.

[8]  M Omidvar,et al.  Cooperative co-evolutionary algorithms for large-scale optimization , 2015 .

[9]  David J. Sheskin,et al.  Handbook of Parametric and Nonparametric Statistical Procedures , 1997 .

[10]  Tapabrata Ray,et al.  A cooperative coevolutionary algorithm with Correlation based Adaptive Variable Partitioning , 2009, 2009 IEEE Congress on Evolutionary Computation.

[11]  R. Paul Wiegand,et al.  An empirical analysis of collaboration methods in cooperative coevolutionary algorithms , 2001 .

[12]  Andries Petrus Engelbrecht,et al.  A Cooperative approach to particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[13]  Peter Tiño,et al.  Scaling Up Estimation of Distribution Algorithms for Continuous Optimization , 2011, IEEE Transactions on Evolutionary Computation.

[14]  Kenneth A. De Jong,et al.  A Cooperative Coevolutionary Approach to Function Optimization , 1994, PPSN.

[15]  H. Cabral,et al.  Multiple Comparisons Procedures , 2008, Circulation.

[16]  Ata Kabán,et al.  Multivariate Cauchy EDA Optimisation , 2014, IDEAL.

[17]  Xin Yao,et al.  Large scale evolutionary optimization using cooperative coevolution , 2008, Inf. Sci..

[18]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Tutorial , 2016, ArXiv.

[19]  Youhei Akimoto,et al.  Projection-Based Restricted Covariance Matrix Adaptation for High Dimension , 2016, GECCO.

[20]  Xin Yao,et al.  Multilevel cooperative coevolution for large scale optimization , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[21]  Ilya Loshchilov,et al.  LM-CMA: An Alternative to L-BFGS for Large-Scale Black Box Optimization , 2015, Evolutionary Computation.

[22]  Ke Tang,et al.  Scaling Up Covariance Matrix Adaptation Evolution Strategy Using Cooperative Coevolution , 2013, IDEAL.

[23]  Xiaodong Li,et al.  Cooperative Co-Evolution With Differential Grouping for Large Scale Optimization , 2014, IEEE Transactions on Evolutionary Computation.

[24]  Ponnuthurai Nagaratnam Suganthan,et al.  Benchmark Functions for the CEC'2013 Special Session and Competition on Large-Scale Global Optimization , 2008 .

[25]  Xin Yao,et al.  Self-adaptive differential evolution with neighborhood search , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[26]  Raymond Ros,et al.  A Simple Modification in CMA-ES Achieving Linear Time and Space Complexity , 2008, PPSN.

[27]  Xiaodong Li,et al.  A Competitive Divide-and-Conquer Algorithm for Unconstrained Large-Scale Black-Box Optimization , 2016, ACM Trans. Math. Softw..

[28]  Xiaodong Li,et al.  Effective decomposition of large-scale separable continuous functions for cooperative co-evolutionary algorithms , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[29]  Ata Kabán,et al.  Toward Large-Scale Continuous EDA: A Random Matrix Theory Perspective , 2013, Evolutionary Computation.

[30]  Ata Kabán,et al.  REMEDA: Random Embedding EDA for Optimising Functions with Intrinsic Dimension , 2016, PPSN.