Mirrored Orthogonal Sampling for Covariance Matrix Adaptation Evolution Strategies

Generating more evenly distributed samples in high dimensional search spaces is the major purpose of the recently proposed mirrored sampling technique for evolution strategies. The diversity of the mutation samples is enlarged and the convergence rate is therefore improved by the mirrored sampling. Motivated by the mirrored sampling technique, this article introduces a new derandomized sampling technique called mirrored orthogonal sampling. The performance of this new technique is both theoretically analyzed and empirically studied on the sphere function. In particular, the mirrored orthogonal sampling technique is applied to the well-known Covariance Matrix Adaptation Evolution Strategy (CMA-ES). The resulting algorithm is experimentally tested on the well-known Black-Box Optimization Benchmark (BBOB). By comparing the results from the benchmark, mirrored orthogonal sampling is found to outperform both the standard CMA-ES and its variant using mirrored sampling.

[1]  Hao Wang,et al.  Mirrored orthogonal sampling with pairwise selection in evolution strategies , 2014, SAC.

[2]  Shuhei Kimura,et al.  Genetic algorithms using low-discrepancy sequences , 2005, GECCO '05.

[3]  Anne Auger,et al.  Theory of Evolution Strategies: A New Perspective , 2011, Theory of Randomized Search Heuristics.

[4]  Å. Björck Numerics of Gram-Schmidt orthogonalization , 1994 .

[5]  Harald Niederreiter,et al.  Random number generation and Quasi-Monte Carlo methods , 1992, CBMS-NSF regional conference series in applied mathematics.

[6]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Comparing Review , 2006, Towards a New Evolutionary Computation.

[7]  Anne Auger,et al.  Analyzing the impact of mirrored sampling and sequential selection in elitist evolution strategies , 2011, FOGA '11.

[8]  Petros Koumoutsakos,et al.  Reducing the Time Complexity of the Derandomized Evolution Strategy with Covariance Matrix Adaptation (CMA-ES) , 2003, Evolutionary Computation.

[9]  Thomas Bäck,et al.  Order Statistics for Convergence Velocity Analysis of Simplified Evolutionary Algorithms , 1994, FOGA.

[10]  T. J. Page Multivariate Statistics: A Vector Space Approach , 1984 .

[11]  Shahryar Rahnamayan,et al.  Opposition-Based Differential Evolution Algorithms , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[12]  Hans-Paul Schwefel,et al.  Evolution and Optimum Seeking: The Sixth Generation , 1993 .

[13]  Anne Auger,et al.  Real-Parameter Black-Box Optimization Benchmarking 2009: Noiseless Functions Definitions , 2009 .

[14]  H. H. Rosenbrock,et al.  An Automatic Method for Finding the Greatest or Least Value of a Function , 1960, Comput. J..

[15]  F. Pillichshammer,et al.  Digital Nets and Sequences: Discrepancy Theory and Quasi-Monte Carlo Integration , 2010 .

[16]  Hans-Georg Beyer,et al.  Toward a Theory of Evolution Strategies: Some Asymptotical Results from the (1,+ )-Theory , 1993, Evolutionary Computation.

[17]  N. Hansen,et al.  Real-Parameter Black-Box Optimization Benchmarking: Experimental Setup , 2010 .

[18]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[19]  Anne Auger,et al.  Mirrored variants of the (1,2)-CMA-ES compared on the noiseless BBOB-2010 testbed , 2010, GECCO '10.

[20]  Anne Auger,et al.  Mirrored Sampling and Sequential Selection for Evolution Strategies , 2010, PPSN.

[21]  Thomas Bäck,et al.  Contemporary Evolution Strategies , 2013, Natural Computing Series.

[22]  Michèle Sebag,et al.  Adaptive coordinate descent , 2011, GECCO '11.

[23]  Nikolaus Hansen,et al.  Adaptive Encoding: How to Render Search Coordinate System Invariant , 2008, PPSN.

[24]  Anne Auger,et al.  Mirrored sampling in evolution strategies with weighted recombination , 2011, GECCO '11.

[25]  Olivier Teytaud,et al.  DCMA: yet another derandomization in covariance-matrix-adaptation , 2007, GECCO '07.