Constrained efficient global optimization with support vector machines

This paper presents a methodology for constrained efficient global optimization (EGO) using support vector machines (SVMs). While the objective function is approximated using Kriging, as in the original EGO formulation, the boundary of the feasible domain is approximated explicitly as a function of the design variables using an SVM. Because SVM is a classification approach and does not involve response approximations, this approach alleviates issues due to discontinuous or binary responses. More importantly, several constraints, even correlated, can be represented using one unique SVM, thus considerably simplifying constrained problems. In order to account for constraints, this paper introduces an SVM-based “probability of feasibility” using a new Probabilistic SVM model. The proposed optimization scheme is constituted of two levels. In a first stage, a global search for the optimal solution is performed based on the “expected improvement” of the objective function and the probability of feasibility. In a second stage, the SVM boundary is locally refined using an adaptive sampling scheme. An unconstrained and a constrained formulation of the optimization problem are presented and compared. Several analytical examples are used to test the formulations. In particular, a problem with 99 constraints and an aeroelasticity problem with binary output are presented. Overall, the results indicate that the constrained formulation is more robust and efficient.

[1]  N. Cressie.  The origins of kriging , 1990 .

[2]  William J. Welch,et al.  Computer experiments and global optimization , 1997 .

[3]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[4]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[5]  John Platt,et al.  Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods , 1999 .

[6]  J. Chilès,et al.  Geostatistics: Modeling Spatial Uncertainty , 1999 .

[7]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[8]  Roger Woodard,et al.  Interpolation of Spatial Data: Some Theory for Kriging , 1999, Technometrics.

[9]  Alexander J. Smola,et al.  Advances in Large Margin Classifiers , 2000 .

[10]  Charles Audet,et al.  A surrogate-model-based method for constrained optimization , 2000 .

[11]  Xiaoping Du,et al.  The use of metamodeling techniques for optimization under uncertainty , 2001 .

[12]  P. A. Newman,et al.  Approximation and Model Management in Aerodynamic Optimization with Variable-Fidelity Models , 2001 .

[13]  Michael James Sasena,et al.  Flexibility and efficiency enhancements for constrained global design optimization with kriging approximations. , 2002 .

[14]  M. Sasena,et al.  Exploration of Metamodeling Sampling Criteria for Constrained Global Optimization , 2002 .

[15]  M. Sasena,et al.  Global optimization of problems with disconnected feasible regions via surrogate modeling , 2002 .

[16]  Thomas J. Santner,et al.  The Design and Analysis of Computer Experiments , 2003, Springer Series in Statistics.

[17]  T. Simpson,et al.  Use of Kriging Models to Approximate Deterministic Computer Models , 2005 .

[18]  Raphael T. Haftka,et al.  Surrogate-based Analysis and Optimization , 2005 .

[19]  A. Atiya,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[20]  G. Gary Wang,et al.  Review of Metamodeling Techniques in Support of Engineering Design Optimization , 2007, DAC 2006.

[21]  R. A. Miller,et al.  Sequential kriging optimization using multiple-fidelity evaluations , 2006 .

[22]  Raphael T. Haftka,et al.  A convex hull approach for the reliability-based design optimization of nonlinear transient dynamic problems , 2007 .

[23]  David Ginsbourger,et al.  A Multi-points Criterion for Deterministic Parallel Global Optimization based on Kriging , 2007 .

[24]  Joachim Kunert,et al.  An Efficient Sequential Optimization Approach Based on the Multivariate Expected Improvement Criterion , 2007 .

[25]  Hsuan-Tien Lin,et al.  A note on Platt’s probabilistic outputs for support vector machines , 2007, Machine Learning.

[26]  Victor Picheny,et al.  Conservative Predictions Using Surrogate Modeling , 2008 .

[27]  Andy J. Keane,et al.  Engineering Design via Surrogate Modelling - A Practical Guide , 2008 .

[28]  A. Basudhar,et al.  Adaptive explicit decision functions for probabilistic design and optimization using support vector machines , 2008 .

[29]  Antonio Harrison Sánchez,et al.  Limit state function identification using Support Vector Machines for discontinuous responses and disjoint failure domains , 2008 .

[30]  Wolfgang Ponweiser,et al.  Clustered multiple generalized expected improvement: A novel infill sampling criterion for surrogate models , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[31]  Timothy W. Simpson,et al.  Design and Analysis of Computer Experiments in Multidisciplinary Design Optimization: A Review of How Far We Have Come - Or Not , 2008 .

[32]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[33]  Michael S. Eldred,et al.  Reliability-Based Design Optimization Using Efficient Global Reliability Analysis , 2009 .

[34]  Samy Missoum,et al.  A Sampling-Based Approach for Probabilistic Design with Random Fields , 2008 .

[35]  Andy J. Keane,et al.  Recent advances in surrogate-based optimization , 2009 .

[36]  Jack P. C. Kleijnen,et al.  Kriging Metamodeling in Simulation: A Review , 2007, Eur. J. Oper. Res..

[37]  Parviz E. Nikravesh,et al.  Reliability-Based Optimal Design and Tolerancing for Multibody Systems Using Explicit Design Space Decomposition , 2010 .

[38]  Raphael T. Haftka,et al.  Making the Most Out of Surrogate Models: Tricks of the Trade , 2010, DAC 2010.

[39]  L. Watson,et al.  Why Not Run the Efficient Global Optimization Algorithm with Multiple Surrogates , 2010 .

[40]  R. Brereton,et al.  Support vector machines for classification and regression. , 2010, The Analyst.

[41]  Robert B. Gramacy,et al.  Optimization Under Unknown Constraints , 2010, 1004.4027.

[42]  A. Basudhar,et al.  An improved adaptive sampling scheme for the construction of explicit boundaries , 2010 .