Efficient global optimization of constrained mixed variable problems

Due to the increasing demand for high performance and cost reduction within the framework of complex system design, numerical optimization of computationally costly problems is an increasingly popular topic in most engineering fields. In this paper, several variants of the Efficient Global Optimization algorithm for costly constrained problems depending simultaneously on continuous decision variables as well as on quantitative and/or qualitative discrete design parameters are proposed. The adaptation that is considered is based on a redefinition of the Gaussian Process kernel as a product between the standard continuous kernel and a second kernel representing the covariance between the discrete variable values. Several parameterizations of this discrete kernel, with their respective strengths and weaknesses, are discussed in this paper. The novel algorithms are tested on a number of analytical test-cases and an aerospace related design problem, and it is shown that they require fewer function evaluations in order to converge towards the neighborhoods of the problem optima when compared to more commonly used optimization algorithms.

[1]  Donald R. Jones,et al.  Global versus local search in constrained optimization of computer models , 1998 .

[2]  Martin Holena,et al.  Surrogate Model for Mixed-Variables Evolutionary Optimization Based on GLM and RBF Networks , 2013, SOFSEM.

[3]  R. Webster,et al.  Kriging: a method of interpolation for geographical information systems , 1990, Int. J. Geogr. Inf. Sci..

[4]  Dries Verstraete,et al.  Hypersonic cryogenic tank design using mixed-variable surrogate-based optimization , 2014 .

[5]  Nils-Hassan Quttineh,et al.  An adaptive radial basis algorithm (ARBF) for expensive black-box mixed-integer constrained global optimization , 2008 .

[6]  Shiyu Zhou,et al.  A Simple Approach to Emulation for Computer Models With Qualitative and Quantitative Factors , 2011, Technometrics.

[7]  Rommel G. Regis,et al.  Evolutionary Programming for High-Dimensional Constrained Expensive Black-Box Optimization Using Radial Basis Functions , 2014, IEEE Transactions on Evolutionary Computation.

[8]  Christine A. Shoemaker,et al.  SO-MI: A surrogate model algorithm for computationally expensive nonlinear mixed-integer black-box global optimization problems , 2013, Comput. Oper. Res..

[9]  Michael James Sasena,et al.  Flexibility and efficiency enhancements for constrained global design optimization with kriging approximations. , 2002 .

[10]  Pedro Larrañaga,et al.  Towards a New Evolutionary Computation - Advances in the Estimation of Distribution Algorithms , 2006, Towards a New Evolutionary Computation.

[11]  Marc Parizeau,et al.  DEAP: evolutionary algorithms made easy , 2012, J. Mach. Learn. Res..

[12]  R. Rebonato,et al.  The Most General Methodology to Create a Valid Correlation Matrix for Risk Management and Option Pricing Purposes , 2011 .

[13]  Xu Xu,et al.  Surrogate Models for Mixed Discrete-Continuous Variables , 2014, Constraint Programming and Decision Making.

[14]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2004 .

[15]  Timothy W. Simpson,et al.  Metamodels for Computer-based Engineering Design: Survey and recommendations , 2001, Engineering with Computers.

[16]  Yulei Zhang,et al.  Computer Experiments with Qualitative and Quantitative Variables: A Review and Reexamination , 2015 .

[17]  Thomas J. Santner,et al.  Design and analysis of computer experiments , 1998 .

[18]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[19]  Peter Z. G. Qian,et al.  Gaussian Process Models for Computer Experiments With Qualitative and Quantitative Factors , 2008, Technometrics.

[20]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[21]  G. Gary Wang,et al.  Review of Metamodeling Techniques in Support of Engineering Design Optimization , 2007, DAC 2006.

[22]  Julien Marzat,et al.  Analysis of multi-objective Kriging-based methods for constrained global optimization , 2016, Comput. Optim. Appl..

[23]  Raphael T. Haftka,et al.  Optimization and Experiments: A Survey , 1998 .

[24]  Momchil Halstrup,et al.  Black-box optimization of mixed discrete-continuous optimization problems , 2016 .

[25]  Marc A. Stelmack,et al.  GENETIC ALGORTIHMS FOR MIXED DISCRETE/CONTINUOUS OPTIMIZATION IN MULTIDISCIPLI NARY DESIGN , 1998 .

[26]  A. Agresti.  An introduction to categorical data analysis , 1997 .

[27]  Raphael T. Haftka,et al.  Surrogate-based Analysis and Optimization , 2005 .

[28]  Charles Audet,et al.  Mesh adaptive direct search algorithms for mixed variable optimization , 2007, Optim. Lett..

[29]  Douglas M. Bates,et al.  Unconstrained parametrizations for variance-covariance matrices , 1996, Stat. Comput..

[30]  Yves Deville,et al.  Group Kernels for Gaussian Process Metamodels with Categorical Inputs , 2018, SIAM/ASA J. Uncertain. Quantification.

[31]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Comparing Review , 2006, Towards a New Evolutionary Computation.

[32]  Loïc Brevault,et al.  Overview and Comparison of Gaussian Process-Based Surrogate Models for Mixed Continuous and Discrete Variables: Application on Aerospace Design Problems , 2020, High-Performance Simulation-Based Optimization.

[33]  Richard J. Beckman,et al.  A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output From a Computer Code , 2000, Technometrics.

[34]  D. Bates,et al.  Mixed-Effects Models in S and S-PLUS , 2001 .

[35]  K. Rashid,et al.  An adaptive multiquadric radial basis function method for expensive black-box mixed-integer nonlinear constrained optimization , 2013 .

[36]  J. Gower.  A General Coefficient of Similarity and Some of Its Properties , 1971 .

[37]  M. R. Osborne,et al.  Estimation of covariance parameters in kriging via restricted maximum likelihood , 1991 .