Multi-output Gaussian process prediction for computationally expensive problems with multiple levels of fidelity

Abstract The multi-output Gaussian process (MOGP) modeling approach is a promising way to deal with multiple correlated outputs since it can capture useful information across outputs to provide more accurate predictions than simply modeling these outputs separately. While the existing MOGP models are considered as single-fidelity models and need a large number of high-fidelity samples to construct a relatively accurate surrogate model. The computational cost is often prohibitive, especially for computationally expensive problems. In this paper, a multi-fidelity multi-output Gaussian process (MMGP) model is proposed to deal with multi-fidelity (MF) data with multiple correlated outputs. It aims to approximate multiple correlated high-fidelity (HF) functions enhanced by the low-fidelity (LF) data. The MMGP model can make a trade-off between high prediction accuracy and low computational cost to relieve the computational burden. The performance of the MMGP model is demonstrated with two analytical examples with different data structures and an aerodynamic prediction example for a NACA 0012 airfoil. Another four well-known surrogate modeling methods are also tested to compare with the proposed approach. Results show that the proposed model is capable of dealing with diverse data structures, heterotopic data, or HF samples generated in different regions. The proposed model provides more accurate predictions than the other approaches for most test cases, especially when using the heterotopic training data. In the NACA0012 airfoil case, the root-mean-square error (RMSE) for the drag coefficient can be reduced by 30%–40% using the MMGP model compared with that of the other MF modeling methods.

[1]  Jianmei Song,et al.  Multi-fidelity uncertainty propagation using polynomial chaos and Gaussian process modeling , 2019, Structural and Multidisciplinary Optimization.

[2]  Zhong-Hua Han,et al.  A New Cokriging Method for Variable-Fidelity Surrogate Modeling of Aerodynamic Data , 2010 .

[3]  Felipe A. Tobar,et al.  MOGPTK: The Multi-Output Gaussian Process Toolkit , 2020, Neurocomputing.

[4]  Qi Zhou,et al.  A generalized hierarchical co-Kriging model for multi-fidelity data fusion , 2020 .

[5]  Haitao Liu,et al.  Remarks on multi-output Gaussian process regression , 2018, Knowl. Based Syst..

[6]  Shawn E. Gano,et al.  Hybrid Variable Fidelity Optimization by Using a Kriging-Based Scaling Function , 2005 .

[7]  Jun Liu,et al.  An active-learning method based on multi-fidelity Kriging model for structural reliability analysis , 2020 .

[8]  Wei Chen,et al.  An Efficient Algorithm for Constructing Optimal Design of Computer Experiments , 2005, DAC 2003.

[9]  David A. Clifton,et al.  Multitask Gaussian Processes for Multivariate Physiological Time-Series Analysis , 2015, IEEE Transactions on Biomedical Engineering.

[10]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[11]  Zhong-Hua Han,et al.  Efficient aerodynamic shape optimization using variable-fidelity surrogate models and multilevel computational grids , 2020, Chinese Journal of Aeronautics.

[12]  Loic Le Gratiet,et al.  RECURSIVE CO-KRIGING MODEL FOR DESIGN OF COMPUTER EXPERIMENTS WITH MULTIPLE LEVELS OF FIDELITY , 2012, 1210.0686.

[13]  Nathalie Bartoli,et al.  Linear regression-based multifidelity surrogate for disturbance amplification in multiphase explosion , 2019, Structural and Multidisciplinary Optimization.

[14]  Shaojun Feng,et al.  Adaptive infill sampling criterion for multi-fidelity gradient-enhanced kriging model , 2020 .

[15]  Edwin V. Bonilla,et al.  Multi-task Gaussian Process Prediction , 2007, NIPS.

[16]  Dick den Hertog,et al.  Space-filling Latin hypercube designs for computer experiments , 2008 .

[17]  Liang Gao,et al.  A screening-based gradient-enhanced Kriging modeling method for high-dimensional problems , 2019, Applied Mathematical Modelling.

[18]  Oliver Stegle,et al.  It is all in the noise: Efficient multi-task Gaussian process inference with structured residuals , 2013, NIPS.

[19]  Timothy C. Coburn,et al.  Geostatistics for Natural Resources Evaluation , 2000, Technometrics.

[20]  Jasper Snoek,et al.  Multi-Task Bayesian Optimization , 2013, NIPS.

[21]  Liang Gao,et al.  Metamodeling for high dimensional design problems by multi-fidelity simulations , 2017 .

[22]  El-Ghazali Talbi,et al.  Multi-fidelity modeling with different input domain definitions using deep Gaussian processes , 2020, Structural and Multidisciplinary Optimization.

[23]  Stefan Görtz,et al.  Improving variable-fidelity surrogate modeling via gradient-enhanced kriging and a generalized hybrid bridge function , 2013 .

[24]  Jose E. Rayas-Sanchez,et al.  Power in Simplicity with ASM: Tracing the Aggressive Space Mapping Algorithm Over Two Decades of Development and Engineering Applications , 2016, IEEE Microwave Magazine.

[25]  Stefan Görtz,et al.  Hierarchical Kriging Model for Variable-Fidelity Surrogate Modeling , 2012 .

[26]  A. O'Hagan,et al.  Predicting the output from a complex computer code when fast approximations are available , 2000 .

[27]  Shiyu Zhou,et al.  Minimizing Negative Transfer of Knowledge in Multivariate Gaussian Processes: A Scalable and Regularized Approach , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Raphael T. Haftka,et al.  Sensitivity-based scaling for approximating. Structural response , 1993 .

[29]  Marcus R. Frean,et al.  Dependent Gaussian Processes , 2004, NIPS.

[30]  Arthur C. Sanderson,et al.  JADE: Adaptive Differential Evolution With Optional External Archive , 2009, IEEE Transactions on Evolutionary Computation.

[31]  Alexander I. J. Forrester,et al.  Multi-fidelity optimization via surrogate modelling , 2007, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[32]  Yee Whye Teh,et al.  Semiparametric latent factor models , 2005, AISTATS.

[33]  Peng Jin,et al.  A two-stage adaptive multi-fidelity surrogate model-assisted multi-objective genetic algorithm for computationally expensive problems , 2019, Engineering with Computers.

[34]  Haitao Liu,et al.  Cope with diverse data structures in multi-fidelity modeling: A Gaussian process method , 2018, Eng. Appl. Artif. Intell..

[35]  Jay Lee,et al.  Gaussian Process Regression for numerical wind speed prediction enhancement , 2020 .

[36]  Yu Zhang,et al.  Weighted Gradient-Enhanced Kriging for High-Dimensional Surrogate Modeling and Design Optimization , 2017 .

[37]  Neil D. Lawrence,et al.  Computationally Efficient Convolved Multiple Output Gaussian Processes , 2011, J. Mach. Learn. Res..

[38]  Thomas J. Santner,et al.  Design and analysis of computer experiments , 1998 .

[39]  Loic Le Gratiet,et al.  Multi-fidelity Gaussian process regression for computer experiments , 2013 .

[40]  Liang Gao,et al.  Optimization of expensive black-box problems via Gradient-enhanced Kriging , 2020 .

[41]  J. Alonso,et al.  SU2: An Open-Source Suite for Multiphysics Simulation and Design , 2016 .

[42]  N. M. Alexandrov,et al.  A trust-region framework for managing the use of approximation models in optimization , 1997 .