Markov Chain Analysis of Cumulative Step-Size Adaptation on a Linear Constrained Problem

This paper analyzes a -Evolution Strategy, a randomized comparison-based adaptive search algorithm optimizing a linear function with a linear constraint. The algorithm uses resampling to handle the constraint. Two cases are investigated: first, the case where the step-size is constant, and second, the case where the step-size is adapted using cumulative step-size adaptation. We exhibit for each case a Markov chain describing the behavior of the algorithm. Stability of the chain implies, by applying a law of large numbers, either convergence or divergence of the algorithm. Divergence is the desired behavior. In the constant step-size case, we show stability of the Markov chain and prove the divergence of the algorithm. In the cumulative step-size adaptation case, we prove stability of the Markov chain in the simplified case where the cumulation parameter equals 1, and discuss steps to obtain similar results for the full (default) algorithm where the cumulation parameter is smaller than 1. The stability of the Markov chain allows us to deduce geometric divergence or convergence, depending on the dimension, constraint angle, population size, and damping parameter, at a rate that we estimate. Our results complement previous studies where stability was assumed.

[1]  Hans-Georg Beyer,et al.  Performance analysis of evolutionary optimization with cumulative step length adaptation , 2004, IEEE Transactions on Automatic Control.

[2]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[3]  Dirk V. Arnold,et al.  Step Length Adaptation on Ridge Functions , 2008, Evolutionary Computation.

[4]  Carlos A. Coello Coello,et al.  Constraint-handling in nature-inspired numerical optimization: Past, present and future , 2011, Swarm Evol. Comput..

[5]  Dirk V. Arnold,et al.  Analysis of a repair mechanism for the (1,λ)-ES applied to a simple constrained problem , 2011, GECCO '11.

[6]  Anne Auger,et al.  Cumulative Step-size Adaptation on Linear Functions: Technical Report , 2012, PPSN 2012.

[7]  Dirk V. Arnold,et al.  Resampling versus Repair in Evolution Strategies Applied to a Constrained Linear Problem , 2013, Evolutionary Computation.

[8]  Carlos A. Coello Coello,et al.  Constrained Optimization via Multiobjective Evolutionary Algorithms , 2008, Multiobjective Problem Solving from Nature.

[9]  Anne Auger,et al.  Cumulative Step-Size Adaptation on Linear Functions , 2012, PPSN.

[10]  Anne Auger,et al.  Markov chain analysis of evolution strategies on a linear constraint optimization problem , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[11]  Alex A. Freitas,et al.  Evolutionary Computation , 2002 .

[12]  Petros Koumoutsakos,et al.  A Method for Handling Uncertainty in Evolutionary Optimization With an Application to Feedback Control of Combustion , 2009, IEEE Transactions on Evolutionary Computation.

[13]  Dirk V. Arnold,et al.  On the Behaviour of the (1, λ)-σSA-ES for a Constrained Linear Problem , 2012, PPSN.

[14]  Dirk V. Arnold,et al.  Noisy Optimization With Evolution Strategies , 2002, Genetic Algorithms and Evolutionary Computation.

[15]  Anne Auger,et al.  Verifiable Conditions for Irreducibility, Aperiodicity and T-chain Property of a General Markov Chain , 2015 .

[16]  Kalyanmoy Deb,et al.  Multiobjective Problem Solving from Nature: From Concepts to Applications , 2008, Natural Computing Series.

[17]  Martin Holena,et al.  A Generalized Markov-Chain Modelling Approach to (1, λ)-ES Linear Optimization , 2013, PPSN.

[18]  Dirk V. Arnold,et al.  On the behaviour of the (1,λ)-es for a simple constrained problem , 2011, FOGA '11.

[19]  N. Hansen,et al.  Markov Chain Analysis of Cumulative Step-Size Adaptation on a Linear Constrained Problem , 2015, Evolutionary Computation.

[20]  Dirk V. Arnold,et al.  Towards an Augmented Lagrangian Constraint Handling Approach for the (1+1)-ES , 2015, GECCO.

[21]  Xin Yao,et al.  Stochastic ranking for constrained evolutionary optimization , 2000, IEEE Trans. Evol. Comput..

[22]  Dirk V. Arnold,et al.  On the Behaviour of the (1+1)-ES for a Simple Constrained Problem , 2008, PPSN.

[23]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[24]  Martin Holena,et al.  A Generalized Markov-Chain Modelling Approach to $(1,\lambda )$-ES Linear Optimization: Technical Report , 2013 .

[25]  Dirk V. Arnold,et al.  A (1+1)-CMA-ES for constrained optimisation , 2012, GECCO '12.