A mixed-categorical correlation kernel for Gaussian process

Recently, there has been a growing interest for mixed-categorical meta-models based on Gaussian process (GP) surrogates. In this setting, several existing approaches use different strategies either by using continuous kernels ( e.g. , continuous relaxation and Gower distance based GP) or by using a direct estimation of the correlation matrix. In this paper, we present a kernel-based approach that extends continuous exponential kernels to handle mixed-categorical variables. The proposed kernel leads to a new GP surrogate that generalizes both the continuous relaxation and the Gower distance based GP models. We demonstrate, on both analytical and engineering problems, that our proposed GP model gives a higher likelihood and a smaller residual error than the other kernel-based state-of-the-art models. Our method is available in the open-source software SMT.

[1]  J. Morlier,et al.  A mixed-categorical data-driven approach for prediction and optimization of hybrid discontinuous composites performance , 2022, AIAA AVIATION 2022 Forum.

[2]  M. Bauerheim,et al.  Coaxial-Injector Surrogate Modeling Based on Reynolds-Averaged Navier–Stokes Simulations Using Deep Learning , 2022, Journal of Propulsion and Power.

[3]  P. S. Palar,et al.  Modeling Non-Stationarity with Deep Gaussian Processes: Applications in Aerospace Engineering , 2022, AIAA SCITECH 2022 Forum.

[4]  R. L. Riche,et al.  A comparison of mixed-variables Bayesian optimization approaches , 2021, Adv. Model. Simul. Eng. Sci..

[5]  Juan Pablo Vielma,et al.  Constrained Discrete Black-Box Optimization using Mixed-Integer Programming , 2021, ICML.

[6]  D. Williamson,et al.  Deep Gaussian Process Emulation using Stochastic Imputation , 2021, Technometrics.

[7]  A. F. López-Lopera,et al.  Multi-Output Gaussian Processes with Functional Data: A Study on Coastal Flood Hazard Assessment , 2020, Reliab. Eng. Syst. Saf..

[8]  B. Khoo,et al.  Low-Reynolds-number airfoil design optimization using deep-learning-based tailored airfoil modes , 2021, Aerospace Science and Technology.

[9]  Ramin Bostanabad,et al.  Latent Map Gaussian Processes for Mixed Variable Metamodeling , 2021, Computer Methods in Applied Mechanics and Engineering.

[10]  Mathijs de Weerdt,et al.  Black-box mixed-variable optimisation using a surrogate model that satisfies integer constraints , 2020, GECCO Companion.

[11]  Hazi Mohammad Azamathulla,et al.  Application of Gaussian process regression to forecast multi-step ahead SPEI drought index , 2021 .

[12]  Sebastien Defoort,et al.  From FAST to FAST-OAD: An open source framework for rapid Overall Aircraft Design , 2021 .

[13]  Joaquim R. R. A. Martins,et al.  Efficient Aerodynamic Shape Optimization with Deep-Learning-Based Geometric Filtering , 2020 .

[14]  Delphine Sinoquet,et al.  Global optimization for mixed categorical-continuous variables based on Gaussian process models with a randomized categorical space exploration step , 2020, INFOR Inf. Syst. Oper. Res..

[15]  Santu Rana,et al.  Bayesian Optimization for Categorical and Category-Specific Continuous Inputs , 2019, AAAI.

[16]  Yves Deville,et al.  Group Kernels for Gaussian Process Metamodels with Categorical Inputs , 2018, SIAM/ASA J. Uncertain. Quantification.

[17]  Eduardo C. Garrido-Merchán,et al.  Dealing with Categorical and Integer-valued Variables in Bayesian Optimization with Gaussian Processes , 2017, Neurocomputing.

[18]  Joaquim R. R. A. Martins,et al.  A Python surrogate modeling framework with derivatives , 2019, Adv. Eng. Softw..

[19]  G. A. Young,et al.  Mathematical Statistics: An Introduction to Likelihood Based InferenceRichard J.RossiJohn Wiley & Sons, 2018, xv + 422 pages, £85.99, ebook ISBN: 978‐1‐118‐77104‐4, LCCN 2018010628 (ebook) , 2019, International Statistical Review.

[20]  Peter Schmollgruber,et al.  Multidisciplinary Exploration of DRAGON: an ONERA Hybrid Electric Distributed Propulsion Concept , 2019, AIAA Scitech 2019 Forum.

[21]  William A. Crossley,et al.  A Mixed Integer Efficient Global Optimization Algorithm with Multiple Infill Strategy - Applied to a Wing Topology Optimization Problem , 2019, AIAA Scitech 2019 Forum.

[22]  Loïc Brevault,et al.  Efficient global optimization of constrained mixed variable problems , 2018, Journal of Global Optimization.

[23]  Joseph Morlier,et al.  Efficient global optimization for high-dimensional constrained problems by using the Kriging models combined with the partial least squares method , 2018 .

[24]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[25]  D. Sculley,et al.  Google Vizier: A Service for Black-Box Optimization , 2017, KDD.

[26]  R. K. Rowe,et al.  Additive Gaussian Process for Computer Models With Qualitative and Quantitative Factors , 2017, Technometrics.

[27]  Henri Calandra,et al.  A parallel evolution strategy for an earth imaging problem in geophysics , 2016 .

[28]  Momchil Halstrup,et al.  Black-box optimization of mixed discrete-continuous optimization problems , 2016 .

[29]  G. Gary Wang,et al.  Trust Region Based Mode Pursuing Sampling Method for Global Optimization of High Dimensional Design Problems , 2015 .

[30]  David Duvenaud,et al.  Automatic model construction with Gaussian processes , 2014 .

[31]  M. Herrera,et al.  Metamodel-assisted optimization based on multiple kernel regression for mixed variables , 2014, Structural and Multidisciplinary Optimization.

[32]  Jeffrey S. Racine,et al.  The crs Package: Nonparametric Regression Splines for Continuous and Categorical Predictors , 2012, R J..

[33]  Yoshua Bengio,et al.  Algorithms for Hyper-Parameter Optimization , 2011, NIPS.

[34]  Shiyu Zhou,et al.  A Simple Approach to Emulation for Computer Models With Qualitative and Quantitative Factors , 2011, Technometrics.

[35]  Ahmed Kattan,et al.  Geometric Generalisation of Surrogate Model Based Optimisation to Combinatorial Spaces , 2011, EvoCOP.

[36]  Kevin Leyton-Brown,et al.  Sequential Model-Based Optimization for General Algorithm Configuration , 2011, LION.

[37]  Herbert K. H. Lee,et al.  Gaussian Processes , 2011, International Encyclopedia of Statistical Science.

[38]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[39]  Peter Z. G. Qian,et al.  Gaussian Process Models for Computer Experiments With Qualitative and Quantitative Factors , 2008, Technometrics.

[40]  Andy J. Keane,et al.  Engineering Design via Surrogate Modelling - A Practical Guide , 2008 .

[41]  D. Brigo,et al.  Parameterizing correlations: a geometric interpretation , 2007 .

[42]  J. Dennis,et al.  Filter Pattern Search Algorithms for Mixed Variable Constrained Optimization Problems , 2004 .

[43]  Wei Chen,et al.  An Efficient Algorithm for Constructing Optimal Design of Computer Experiments , 2005, DAC 2003.

[44]  R. Rebonato,et al.  The Most General Methodology to Create a Valid Correlation Matrix for Risk Management and Option Pricing Purposes , 2011 .

[45]  T. Raghavan,et al.  Nonnegative Matrices and Applications , 1997 .

[46]  M. Powell A Direct Search Optimization Method That Models the Objective and Constraint Functions by Linear Interpolation , 1994 .

[47]  D. Krige A statistical approach to some basic mine valuation problems on the Witwatersrand, by D.G. Krige, published in the Journal, December 1951 : introduction by the author , 1951 .

[48]  I. J. Schoenberg,et al.  Metric spaces and positive definite functions , 1938 .