Trust Regions in Surrogate-Assisted Evolutionary Programming for Constrained Expensive Black-Box Optimization

This paper develops a new surrogate-assisted evolutionary programming (EP) algorithm for computationally expensive constrained black-box optimization. The proposed algorithm, TRICEPS (Trust Regions In Constrained Evolutionary Programming using Surrogates) builds surrogates for the black-box objective function and inequality constraint functions in every generation of the EP and uses a trust-region-like approach to refine the best solution at the end of each generation. Each parent produces a large number of trial offspring in each generation, and then the surrogates are used to identify promising trial offspring, which become the actual offspring where the objective and constraint functions are evaluated. After the function evaluations at these offspring, TRICEPS finds a minimizer of the surrogate of the objective function within a trust region centered at the current best solution and subject to surrogate inequality constraints with a small margin and with a distance requirement from previously evaluated points. The trust region is either expanded or reduced depending on whether the subproblem solution turned out to be feasible and whether the ratio of the actual improvement to the predicted improvement exceeds or falls below certain thresholds. TRICEPS is implemented using a cubic radial basis function (RBF) model with a linear polynomial tail and is compared to an RBF-assisted EP called CEP-RBF (Regis 2014b) and to other alternatives on 18 benchmark problems and on an automotive application with 124 decision variables and 68 black-box inequality constraints. Performance and data profiles show that TRICEPS is a substantial improvement over CEP-RBF and it is much better than the other alternatives on the test problems used.

[1]  Christine A. Shoemaker,et al.  A quasi-multistart framework for global optimization of expensive functions using response surface models , 2013, J. Glob. Optim..

[2]  Yaochu Jin,et al.  Surrogate-assisted evolutionary computation: Recent advances and future challenges , 2011, Swarm Evol. Comput..

[3]  Rommel G. Regis,et al.  Stochastic radial basis function algorithms for large-scale optimization involving expensive black-box objective and constraint functions , 2011, Comput. Oper. Res..

[4]  R. A. Cuninghame-Green,et al.  Applied geometric programming , 1976 .

[5]  K. Gupta,et al.  Metamodel-Based Optimization for Problems With Expensive Objective and Constraint Functions , 2011 .

[6]  Marco Sciandrone,et al.  Sequential Penalty Derivative-Free Methods for Nonlinear Constrained Optimization , 2010, SIAM J. Optim..

[7]  Carlos A. Coello Coello,et al.  Constraint-handling in nature-inspired numerical optimization: Past, present and future , 2011, Swarm Evol. Comput..

[8]  R. Regis.  Constrained optimization by radial basis function interpolation for high-dimensional expensive black-box problems with infeasible initial points , 2014 .

[9]  Rommel G. Regis,et al.  Particle swarm with radial basis function surrogates for expensive black-box optimization , 2014, J. Comput. Sci..

[10]  Frederico G. Guimarães,et al.  Constraint quadratic approximation operator for treating equality constraints with genetic algorithms , 2005, 2005 IEEE Congress on Evolutionary Computation.

[11]  A. Keane,et al.  Evolutionary Optimization of Computationally Expensive Problems via Surrogate Modeling , 2003 .

[12]  Hans-Martin Gutmann,et al.  A Radial Basis Function Method for Global Optimization , 2001, J. Glob. Optim..

[13]  Bernhard Sendhoff,et al.  A framework for evolutionary optimization with approximate fitness functions , 2002, IEEE Trans. Evol. Comput..

[14]  Ricardo Landa Becerra,et al.  Efficient evolutionary optimization through the use of a cultural algorithm , 2004 .

[15]  Xin Yao,et al.  Stochastic ranking for constrained evolutionary optimization , 2000, IEEE Trans. Evol. Comput..

[16]  Panos M. Pardalos,et al.  A Collection of Test Problems for Constrained Global Optimization Algorithms , 1990, Lecture Notes in Computer Science.

[17]  K. Deb,et al.  A bi-objective constrained optimization algorithm using a hybrid evolutionary and penalty function approach , 2013 .

[18]  Thomas Philip Runarsson,et al.  Constrained Evolutionary Optimization by Approximate Ranking and Surrogate Models , 2004, PPSN.

[19]  Julio R. Banga,et al.  Improved scatter search for the global optimization of computationally expensive dynamic models , 2009, J. Glob. Optim..

[20]  Liang Shi,et al.  ASAGA: an adaptive surrogate-assisted genetic algorithm , 2008, GECCO '08.

[21]  Oliver Kramer,et al.  Towards Non-linear Constraint Estimation for Expensive Optimization , 2013, EvoApplications.

[22]  Tapabrata Ray,et al.  Multi-objective design optimisation using multiple adaptive spatially distributed surrogates , 2009 .

[23]  Dirk V. Arnold,et al.  A (1+1)-CMA-ES for constrained optimisation , 2012, GECCO '12.

[24]  Christine A. Shoemaker,et al.  A Stochastic Radial Basis Function Method for the Global Optimization of Expensive Functions , 2007, INFORMS J. Comput..

[25]  Yong Wang,et al.  Combining Multiobjective Optimization With Differential Evolution to Solve Constrained Optimization Problems , 2012, IEEE Transactions on Evolutionary Computation.

[26]  Tetsuyuki Takahama,et al.  Efficient Constrained Optimization by the ε Constrained Rank-Based Differential Evolution , 2012, 2012 IEEE Congress on Evolutionary Computation.

[27]  Tapabrata Ray,et al.  An Evolutionary Algorithm with Spatially Distributed Surrogates for Multiobjective Optimization , 2007, ACAL.

[28]  Christine A. Shoemaker,et al.  Comparison of function approximation, heuristic, and derivative‐based methods for automatic calibration of computationally expensive groundwater bioremediation models , 2005 .

[29]  Christine A. Shoemaker,et al.  Local function approximation in evolutionary algorithms for the optimization of costly functions , 2004, IEEE Transactions on Evolutionary Computation.

[30]  Christine A. Shoemaker,et al.  Global Convergence of Radial Basis Function Trust Region Derivative-Free Algorithms , 2011, SIAM J. Optim..

[31]  Carlos A. Coello Coello,et al.  Constraint-handling in genetic algorithms through the use of dominance-based tournament selection , 2002, Adv. Eng. Informatics.

[32]  A. Basudhar,et al.  Constrained efficient global optimization with support vector machines , 2012, Structural and Multidisciplinary Optimization.

[33]  Michael T. M. Emmerich,et al.  Single- and multiobjective evolutionary optimization assisted by Gaussian random field metamodels , 2006, IEEE Transactions on Evolutionary Computation.

[34]  Thomas Bäck,et al.  Metamodel-Assisted Evolution Strategies , 2002, PPSN.

[35]  Elizabeth F. Wanner,et al.  Constrained Optimization Based on Quadratic Approximations in Genetic Algorithms , 2009 .

[36]  Susana Gomez,et al.  Advances in Optimization and Numerical Analysis , 1994 .

[37]  Andy J. Keane,et al.  Combining Global and Local Surrogate Models to Accelerate Evolutionary Optimization , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[38]  Bryan A. Tolson,et al.  Dynamically dimensioned search algorithm for computationally efficient watershed model calibration , 2007 .

[39]  Rommel G. Regis,et al.  Evolutionary Programming for High-Dimensional Constrained Expensive Black-Box Optimization Using Radial Basis Functions , 2014, IEEE Transactions on Evolutionary Computation.

[40]  Julio R. Banga,et al.  Scatter search for chemical and bio-process optimization , 2007, J. Glob. Optim..

[41]  Carlos A. Coello Coello.  Constraint-handling techniques used with evolutionary algorithms , 2007, GECCO '07.

[42]  Abdel-Rahman Hedar,et al.  Studies on Metaheuristics for Continuous Global Optimization Problems , 2004 .

[43]  Michèle Sebag,et al.  Self-adaptive surrogate-assisted covariance matrix adaptation evolution strategy , 2012, GECCO '12.

[44]  Oliver Kramer,et al.  Surrogate Constraint Functions for CMA Evolution Strategies , 2009, KI.

[45]  C. Shoemaker,et al.  Combining radial basis function surrogates and dynamic coordinate search in high-dimensional expensive black-box optimization , 2013 .

[46]  Christine A. Shoemaker,et al.  ORBIT: Optimization by Radial Basis Function Interpolation in Trust-Regions , 2008, SIAM J. Sci. Comput..

[47]  Stefan M. Wild,et al.  Benchmarking Derivative-Free Optimization Algorithms , 2009, SIAM J. Optim..

[48]  Kalyanmoy Deb,et al.  Individual penalty based constraint handling using a hybrid bi-objective and penalty function approach , 2013, 2013 IEEE Congress on Evolutionary Computation.

[49]  N. Hansen,et al.  Markov Chain Analysis of Cumulative Step-Size Adaptation on a Linear Constrained Problem , 2015, Evolutionary Computation.

[50]  Zbigniew Michalewicz,et al.  Evolutionary Algorithms for Constrained Parameter Optimization Problems , 1996, Evolutionary Computation.

[51]  Carlos A. Coello Coello,et al.  Multi-objective airfoil shape optimization using a multiple-surrogate approach , 2012, 2012 IEEE Congress on Evolutionary Computation.

[52]  Thomas Hemker,et al.  Applicability of surrogates to improve efficiency of particle swarm optimization for simulation-based problems , 2012 .

[53]  L. Watson,et al.  Why Not Run the Efficient Global Optimization Algorithm with Multiple Surrogates , 2010 .

[54]  Carlos A. Coello Coello,et al.  Engineering optimization using simple evolutionary algorithm , 2003, Proceedings. 15th IEEE International Conference on Tools with Artificial Intelligence.

[55]  Mattias Björkman,et al.  Global Optimization of Costly Nonconvex Functions Using Radial Basis Functions , 2000 .

[56]  Rick Hesse.  A Heuristic Search Procedure for Estimating a Global Solution of Nonconvex Programming Problems , 1973, Oper. Res..

[57]  Carlos A. Coello Coello,et al.  A simple multimembered evolution strategy to solve constrained optimization problems , 2005, IEEE Transactions on Evolutionary Computation.

[58]  M. Powell.  A Direct Search Optimization Method That Models the Objective and Constraint Functions by Linear Interpolation , 1994 .

[59]  Gary G. Yen,et al.  A Self Adaptive Penalty Function Based Algorithm for Constrained Optimization , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[60]  P. Suganthan,et al.  Problem Definitions and Evaluation Criteria for the CEC 2010 Competition on Constrained Real- Parameter Optimization , 2010 .