A modified Covariance Matrix Adaptation Evolution Strategy with adaptive penalty function and restart for constrained optimization

In the last decades, a number of novel meta-heuristics and hybrid algorithms have been proposed to solve a great variety of optimization problems. Among these, constrained optimization problems are considered of particular interest in applications from many different domains. The presence of multiple constraints can make optimization problems particularly hard to solve, thus imposing the use of specific techniques to handle fitness landscapes which generally show complex properties. In this paper, we introduce a modified Covariance Matrix Adaptation Evolution Strategy (CMA-ES) specifically designed for solving constrained optimization problems. The proposed method makes use of the restart mechanism typical of most modern variants of CMA-ES, and handles constraints by means of an adaptive penalty function. This novel CMA-ES scheme presents competitive results on a broad set of benchmark functions and engineering problems, outperforming most state-of-the-art algorithms as for both efficiency and constraint handling.

[1]  Xiaohui Hu,et al.  Engineering optimization with particle swarm , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[2]  Amir Hossein Gandomi,et al.  Cuckoo search algorithm: a metaheuristic approach to solve structural optimization problems , 2011, Engineering with Computers.

[3]  Leandro dos Santos Coelho,et al.  Gaussian quantum-behaved particle swarm optimization approaches for constrained engineering design problems , 2010, Expert Syst. Appl..

[4]  Carlos A. Coello Coello,et al.  A simple multimembered evolution strategy to solve constrained optimization problems , 2005, IEEE Transactions on Evolutionary Computation.

[5]  Marc Schoenauer,et al.  Multidisciplinary Optimization in the Design of Future Space Launchers , 2013 .

[6]  Efrén Mezura-Montes,et al.  Self-adaptive and Deterministic Parameter Control in Differential Evolution for Constrained Optimization , 2009 .

[7]  Xin-She Yang,et al.  Engineering optimisation by cuckoo search , 2010, Int. J. Math. Model. Numer. Optimisation.

[8]  Giovanni Iacca,et al.  Parallel memetic structures , 2013, Inf. Sci..

[9]  Ali Wagdy Mohamed,et al.  Constrained optimization based on modified differential evolution algorithm , 2012, Inf. Sci..

[10]  Ruhul A. Sarker,et al.  An agent-based memetic algorithm (AMA) for solving constrained optimazation problems , 2007, 2007 IEEE Congress on Evolutionary Computation.

[11]  Nikolaus Hansen,et al.  Adapting arbitrary normal mutation distributions in evolution strategies: the covariance matrix adaptation , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[12]  Carlos A. Coello Coello,et al.  Solving Engineering Optimization Problems with the Simple Constrained Particle Swarm Optimizer , 2008, Informatica.

[13]  Jing J. Liang,et al.  Problem Deflnitions and Evaluation Criteria for the CEC 2006 Special Session on Constrained Real-Parameter Optimization , 2006 .

[14]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[15]  Carlos García-Martínez,et al.  Memetic Algorithms for Continuous Optimisation Based on Local Search Chains , 2010, Evolutionary Computation.

[16]  Ling Wang,et al.  An effective differential evolution with level comparison for constrained engineering design , 2010 .

[17]  Ali Husseinzadeh Kashan,et al.  An efficient algorithm for constrained global optimization and application to mechanical engineering design: League championship algorithm (LCA) , 2011, Comput. Aided Des..

[18]  Ali Osman Kusakci,et al.  An adaptive penalty based covariance matrix adaptation-evolution strategy , 2013, Comput. Oper. Res..

[19]  Sabine Fenstermacher,et al.  Genetic Algorithms Data Structures Evolution Programs , 2016 .

[20]  Vinicius Veloso de Melo,et al.  Investigating Multi-View Differential Evolution for solving constrained engineering design problems , 2013, Expert Syst. Appl..

[21]  Carlos A. Coello Coello,et al.  Use of a self-adaptive penalty approach for engineering optimization problems , 2000 .

[22]  Carlos A. Coello Coello,et al.  Modified Differential Evolution for Constrained Optimization , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[23]  R. Venkata Rao,et al.  Teaching-learning-based optimization: A novel method for constrained mechanical design optimization problems , 2011, Comput. Aided Des..

[24]  Ponnuthurai N. Suganthan,et al.  A differential covariance matrix adaptation evolutionary algorithm for global optimization , 2011, 2011 IEEE Symposium on Differential Evolution (SDE).

[25]  Tapabrata Ray,et al.  Society and civilization: An optimization algorithm based on the simulation of social behavior , 2003, IEEE Trans. Evol. Comput..

[26]  Carlos A. Coello Coello,et al.  Simple Feasibility Rules and Differential Evolution for Constrained Optimization , 2004, MICAI.

[27]  Ricardo Landa Becerra,et al.  Efficient evolutionary optimization through the use of a cultural algorithm , 2004 .

[28]  Erwie Zahara,et al.  Hybrid Nelder-Mead simplex search and particle swarm optimization for constrained engineering design problems , 2009, Expert Syst. Appl..

[29]  Jing J. Liang,et al.  Dynamic Multi-Swarm Particle Swarm Optimizer with a Novel Constraint-Handling Mechanism , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[30]  C. Coello,et al.  Cultured differential evolution for constrained optimization , 2006 .

[31]  Christian Igel,et al.  A computational efficient covariance matrix update and a (1+1)-CMA for evolution strategies , 2006, GECCO.

[32]  Nikolaus Hansen,et al.  Benchmarking a BI-population CMA-ES on the BBOB-2009 function testbed , 2009, GECCO '09.

[33]  Michèle Sebag,et al.  Intensive surrogate model exploitation in self-adaptive surrogate-assisted cma-es (saacm-es) , 2013, GECCO '13.

[34]  P. Suganthan,et al.  Problem Definitions and Evaluation Criteria for the CEC 2010 Competition on Constrained Real- Parameter Optimization , 2010 .

[35]  Ioan Cristian Trelea,et al.  The particle swarm optimization algorithm: convergence analysis and parameter selection , 2003, Inf. Process. Lett..

[36]  A. Kai Qin,et al.  Self-adaptive Differential Evolution Algorithm for Constrained Real-Parameter Optimization , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[37]  Vinicius Veloso de Melo,et al.  Evaluating differential evolution with penalty function to solve constrained engineering problems , 2012, Expert Syst. Appl..

[38]  Oliver Kramer,et al.  Surrogate Constraint Functions for CMA Evolution Strategies , 2009, KI.

[39]  Hitoshi Iba,et al.  Accelerating Differential Evolution Using an Adaptive Local Search , 2008, IEEE Transactions on Evolutionary Computation.

[40]  Hans-Georg Beyer,et al.  On the Design of Constraint Covariance Matrix Self-Adaptation Evolution Strategies Including a Cardinality Constraint , 2012, IEEE Transactions on Evolutionary Computation.

[41]  Jonathan M. Garibaldi,et al.  A novel memetic algorithm for constrained optimization , 2010, IEEE Congress on Evolutionary Computation.

[42]  Efrén Mezura-Montes,et al.  Parameter control in Differential Evolution for constrained optimization , 2009, 2009 IEEE Congress on Evolutionary Computation.

[43]  Yong Wang,et al.  Constrained Evolutionary Optimization by Means of ( + )-Differential Evolution and Improved Adaptive Trade-Off Model , 2011, Evolutionary Computation.

[44]  Petros Koumoutsakos,et al.  Reducing the Time Complexity of the Derandomized Evolution Strategy with Covariance Matrix Adaptation (CMA-ES) , 2003, Evolutionary Computation.

[45]  José Mario Martínez,et al.  Augmented Lagrangian method with nonmonotone penalty parameters for constrained optimization , 2011, Computational Optimization and Applications.

[46]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[47]  Ling Wang,et al.  An effective co-evolutionary differential evolution for constrained optimization , 2007, Appl. Math. Comput..

[48]  Qinghua Wu,et al.  An improved group search optimizer for mechanical design optimization problems , 2009 .

[49]  Carlos A. Coello Coello,et al.  Useful Infeasible Solutions in Engineering Optimization with Evolutionary Algorithms , 2005, MICAI.

[50]  William E. Hart,et al.  A Filter-Based Evolutionary Algorithm for Constrained Optimization , 2004, Evolutionary Computation.

[51]  Michael N. Vrahatis,et al.  Unified Particle Swarm Optimization for Solving Constrained Engineering Optimization Problems , 2005, ICNC.

[52]  Petros Koumoutsakos,et al.  A Method for Handling Uncertainty in Evolutionary Optimization With an Application to Feedback Control of Combustion , 2009, IEEE Transactions on Evolutionary Computation.

[53]  Wenjian Luo,et al.  Differential evolution with dynamic stochastic selection for constrained optimization , 2008, Inf. Sci..

[54]  Yong Wang,et al.  Hybridizing particle swarm optimization with differential evolution for constrained numerical and engineering optimization , 2010, Appl. Soft Comput..

[55]  Tetsuyuki Takahama,et al.  Constrained Optimization by the ε Constrained Differential Evolution with Gradient-Based Mutation and Feasible Elites , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[56]  Mehmet Fatih Tasgetiren,et al.  A Multi-Populated Differential Evolution Algorithm for Solving Constrained Optimization Problem , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[57]  Efrén Mezura-Montes,et al.  Comparing bio-inspired algorithms in constrained optimization problems , 2007, 2007 IEEE Congress on Evolutionary Computation.

[58]  Ling Wang,et al.  An effective co-evolutionary particle swarm optimization for constrained engineering design problems , 2007, Eng. Appl. Artif. Intell..

[59]  C. A. Coello Coello,et al.  A memetic algorithm with simplex crossover for solving constrained optimization problems , 2012, World Automation Congress 2012.

[60]  Dervis Karaboga,et al.  Artificial Bee Colony (ABC) Optimization Algorithm for Solving Constrained Optimization Problems , 2007, IFSA.

[61]  Ling Wang,et al.  A hybrid particle swarm optimization with a feasibility-based rule for constrained optimization , 2007, Appl. Math. Comput..

[62]  Nikolaus Hansen,et al.  A restart CMA evolution strategy with increasing population size , 2005, 2005 IEEE Congress on Evolutionary Computation.

[63]  C. Coello,et al.  Increasing Successful Offspring and Diversity in Differential Evolution for Engineering Design , 2006 .

[64]  Witold Pedrycz,et al.  Foundations of Fuzzy Logic and Soft Computing, 12th International Fuzzy Systems Association World Congress, IFSA 2007, Cancun, Mexico, June 18-21, 2007, Proceedings , 2007, IFSA.

[65]  Dirk V. Arnold,et al.  A (1+1)-CMA-ES for constrained optimisation , 2012, GECCO '12.

[66]  Aravind Srinivasan,et al.  A Population-Based, Parent Centric Procedure for Constrained Real-Parameter Optimization , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[67]  Ajith Abraham,et al.  Low Discrepancy Initialized Particle Swarm Optimization for Solving Constrained Optimization Problems , 2009, Fundam. Informaticae.

[68]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1996, Springer Berlin Heidelberg.

[69]  Jouni Lampinen,et al.  Constrained Real-Parameter Optimization with Generalized Differential Evolution , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[70]  Ivona Brajevic,et al.  Performance of the improved artificial bee colony algorithm on standard engineering constrained problems , 2011 .

[71]  Marc Schoenauer,et al.  ASCHEA: new results using adaptive segregational constraint handling , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[72]  Carlos A. Coello Coello,et al.  Constraint-handling in genetic algorithms through the use of dominance-based tournament selection , 2002, Adv. Eng. Informatics.

[73]  Ardeshir Bahreininejad,et al.  Mine blast algorithm: A new population based algorithm for solving constrained engineering optimization problems , 2013, Appl. Soft Comput..

[74]  Giovanni Iacca,et al.  A CMA-ES super-fit scheme for the re-sampled inheritance search , 2013, 2013 IEEE Congress on Evolutionary Computation.

[75]  Giovanni Iacca,et al.  Ockham's Razor in memetic computing: Three stage optimal memetic exploration , 2012, Inf. Sci..

[76]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[77]  Michèle Sebag,et al.  Alternative Restart Strategies for CMA-ES , 2012, PPSN.

[78]  Zhun Fan,et al.  Constrained optimization based on hybrid evolutionary algorithm and adaptive constraint-handling technique , 2009 .