Enhancing infill sampling criteria for surrogate-based constrained optimization

A popular approach to handling constraints in surrogate-based optimization is through the addition of penalty functions to an infill sampling criterion that seeks objective improvement. Typical sampling metrics, such as expected improvement tend to have multi modal landscapes and can be difficult to search. When the problem is transformed using a penalty approach the search can become riddled with cliffs and further increases the complexity of the landscape. Here we avoid searching this aggregated space by treating objective improvement and constraint satisfaction as separate goals, using multiobjective optimization. This approach is used to enhance the efficiency and reliability of infill sampling and shows some promising results. Further to this, by selecting model update points in close proximity to the constraint boundaries, the regions that are likely to contain the feasible optimum can be better modelled. The resulting enhanced probability of feasibility is used to encourage the exploitation of constraint boundaries.

[1]  Thomas J. Santner,et al.  The Design and Analysis of Computer Experiments , 2003, Springer Series in Statistics.

[2]  Kalyanmoy Deb,et al.  A Fast Elitist Non-dominated Sorting Genetic Algorithm for Multi-objective Optimisation: NSGA-II , 2000, PPSN.

[3]  M. D. McKay,et al.  A comparison of three methods for selecting values of input variables in the analysis of output from a computer code , 2000 .

[4]  Kalyanmoy Deb,et al.  Multiobjective Problem Solving from Nature: From Concepts to Applications , 2008, Natural Computing Series.

[5]  D. Ginsbourger,et al.  Kriging is well-suited to parallelize optimization , 2010 .

[6]  Michael R. Anderberg,et al.  Cluster Analysis for Applications , 1973 .

[7]  Carlos A. Coello Coello,et al.  On the Influence of the Number of Objectives on the Hardness of a Multiobjective Optimization Problem , 2011, IEEE Transactions on Evolutionary Computation.

[8]  A. Oyama,et al.  NEW CONSTRAINT-HANDLING METHOD FOR MULTI-OBJECTIVE MULTI-CONSTRAINT EVOLUTIONARY OPTIMIZATION AND ITS APPLICATION TO SPACE PLANE DESIGN , 2005 .

[9]  William J. Welch,et al.  Computer experiments and global optimization , 1997 .

[10]  John H. Holland,et al.  Outline for a Logical Theory of Adaptive Systems , 1962, JACM.

[11]  Richard J. Beckman,et al.  A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output From a Computer Code , 2000, Technometrics.

[12]  James M. Parr,et al.  Infill sampling criteria for surrogate-based optimization with constraint handling , 2012 .

[13]  Sven Leyffer,et al.  Nonlinear programming without a penalty function , 2002, Math. Program..

[14]  N. M. Alexandrov,et al.  A trust-region framework for managing the use of approximation models in optimization , 1997 .

[15]  Michael R Chernick,et al.  Bootstrap Methods: A Guide for Practitioners and Researchers , 2007 .

[16]  Ingo Wegener,et al.  Can Single-Objective Optimization Profit from Multiobjective Optimization? , 2008, Multiobjective Problem Solving from Nature.

[17]  Chi-Keong Goh,et al.  Computational Intelligence in Expensive Optimization Problems , 2010 .

[18]  A. Forrester,et al.  Design and analysis of 'noisy' computer experiments , 2006 .

[19]  R. K. Ursem.  Multi-objective Optimization using Evolutionary Algorithms , 2009 .

[20]  Victor Picheny,et al.  Adaptive Designs of Experiments for Accurate Approximation of a Target Region , 2010 .

[21]  Charles Audet,et al.  A surrogate-model-based method for constrained optimization , 2000 .

[22]  T. J. Mitchell,et al.  Exploratory designs for computational experiments , 1995 .

[23]  Richard A. Watson,et al.  Reducing Local Optima in Single-Objective Problems by Multi-objectivization , 2001, EMO.

[24]  Michael James Sasena,et al.  Flexibility and efficiency enhancements for constrained global design optimization with kriging approximations. , 2002 .

[25]  M. Eldred,et al.  Multimodal Reliability Assessment for Complex Engineering Applications using Efficient Global Optimization , 2007 .

[26]  David J. J. Toal,et al.  Kriging Hyperparameter Tuning Strategies , 2008 .

[27]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[28]  M. Sasena,et al.  Exploration of Metamodeling Sampling Criteria for Constrained Global Optimization , 2002 .

[29]  Singiresu S. Rao.  Engineering Optimization : Theory and Practice , 2010 .

[30]  P. J. Fleming,et al.  The good of the many outweighs the good of the one: evolutionary multi-objective optimization , 2003 .

[31]  Donald R. Jones,et al.  A Taxonomy of Global Optimization Methods Based on Response Surfaces , 2001, J. Glob. Optim..

[32]  Mikkel T. Jensen,et al.  Guiding Single-Objective Optimization Using Multi-objective Methods , 2003, EvoWorkshops.

[33]  George Michailidis,et al.  Sequential Experiment Design for Contour Estimation From Complex Computer Codes , 2008, Technometrics.

[34]  David W. Corne,et al.  Multiple objective optimisation applied to route planning , 2011, GECCO '11.

[35]  Kalyanmoy Deb,et al.  Muiltiobjective Optimization Using Nondominated Sorting in Genetic Algorithms , 1994, Evolutionary Computation.

[36]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[37]  Xin Yao,et al.  Performance Scaling of Multi-objective Evolutionary Algorithms , 2003, EMO.

[38]  Deniz Yuret,et al.  Dynamic Hill Climbing: Overcoming the limitations of optimization techniques , 1993 .

[39]  Andy J. Keane,et al.  On the Design of Optimization Strategies Based on Global Response Surface Approximation Models , 2005, J. Glob. Optim..

[40]  Yaochu Jin,et al.  A comprehensive survey of fitness approximation in evolutionary computation , 2005, Soft Comput..

[41]  Rommel G. Regis,et al.  Stochastic radial basis function algorithms for large-scale optimization involving expensive black-box objective and constraint functions , 2011, Comput. Oper. Res..

[42]  Sonja Kuhnt,et al.  Design and analysis of computer experiments , 2010 .

[43]  Christine A. Shoemaker,et al.  Constrained Global Optimization of Expensive Black Box Functions Using Radial Basis Functions , 2005, J. Glob. Optim..

[44]  Andres Angantyr,et al.  Constrained optimization based on a multiobjective evolutionary algorithm , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[45]  Andy J. Keane,et al.  Recent advances in surrogate-based optimization , 2009 .

[46]  C. Coello.  TREATING CONSTRAINTS AS OBJECTIVES FOR SINGLE-OBJECTIVE EVOLUTIONARY OPTIMIZATION , 2000 .

[47]  Xin Yao,et al.  How well do multi-objective evolutionary algorithms scale to large problems , 2007, 2007 IEEE Congress on Evolutionary Computation.

[48]  Andy J. Keane,et al.  Review of efficient surrogate infill sampling criteria with constraint handling , 2010 .

[49]  Tapabrata Ray,et al.  An Evolutionary Algorithm for Constrained Optimization , 2000, GECCO.

[50]  A. J. Booker,et al.  A rigorous framework for optimization of expensive functions by surrogates , 1998 .