Cope with diverse data structures in multi-fidelity modeling: A Gaussian process method

Abstract Multi-fidelity modeling (MFM) frameworks, especially the Bayesian MFM, have gained popularity in simulation based modeling, uncertainty quantification and optimization, due to the potential for reducing computational budget. In the view of multi-output modeling, the MFM approximates the high-/low-fidelity outputs simultaneously by considering the output correlations, and particularly, it transfers knowledge from the inexpensive low-fidelity outputs that have many training points to enhance the modeling of the expensive high-fidelity output that has a few training points. This article presents a novel multi-fidelity Gaussian process for modeling with diverse data structures. The diverse data structures mainly refer to the diversity of high-fidelity sample distributions, i.e., the high-fidelity points may randomly fill the domain, or more challengingly, they may cluster in some subregions. The proposed multi-fidelity model is composed of a global trend term and a local residual term. Particularly, the flexible residual term extracts both the shared and output-specific residual information via a data-driven weight parameter. Numerical experiments on two synthetic examples, an aircraft example and a stochastic incompressible flow example reveal that this very promising Bayesian MFM approach is capable of effectively extracting the low-fidelity information for facilitating the modeling of the high-fidelity output using diverse data structures.

[1]  Yee Whye Teh,et al.  Semiparametric latent factor models , 2005, AISTATS.

[2]  Ronald P. Barry,et al.  Constructing and fitting models for cokriging and multivariable spatial prediction , 1998 .

[3]  Jerome Sacks,et al.  Choosing the Sample Size of a Computer Experiment: A Practical Guide , 2009, Technometrics.

[4]  Peter Z. G. Qian Nested Latin hypercube designs , 2009 .

[5]  Loic Le Gratiet,et al.  RECURSIVE CO-KRIGING MODEL FOR DESIGN OF COMPUTER EXPERIMENTS WITH MULTIPLE LEVELS OF FIDELITY , 2012, 1210.0686.

[6]  Nicholas R. Jennings,et al.  Real-time information processing of environmental sensor network data using bayesian gaussian processes , 2012, ACM Trans. Sens. Networks.

[7]  Ilias Bilionis,et al.  Multi-output local Gaussian process regression: Applications to uncertainty quantification , 2012, J. Comput. Phys..

[8]  Stefan Görtz,et al.  Hierarchical Kriging Model for Variable-Fidelity Surrogate Modeling , 2012 .

[9]  P Perdikaris,et al.  Multi-fidelity modelling via recursive co-kriging and Gaussian–Markov random fields , 2015, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[10]  Neil D. Lawrence,et al.  Computationally Efficient Convolved Multiple Output Gaussian Processes , 2011, J. Mach. Learn. Res..

[11]  Alexey Zaytsev,et al.  Minimax Error of Interpolation and Optimal Design of Experiments for Variable Fidelity Data , 2016 .

[12]  Caroline Sainvitu,et al.  MULTI-FIDELITY EXTENSION TO NON-INTRUSIVE PROPER ORTHOGONAL DECOMPOSITION BASED SURROGATES , 2016 .

[13]  Tom Dhaene,et al.  Performance study of multi-fidelity gradient enhanced kriging , 2015 .

[14]  Zhong-Hua Han,et al.  A New Cokriging Method for Variable-Fidelity Surrogate Modeling of Aerodynamic Data , 2010 .

[15]  Jack P. C. Kleijnen,et al.  Multivariate versus Univariate Kriging Metamodels for Multi-Response Simulation Models , 2014, Eur. J. Oper. Res..

[16]  David J. J. Toal,et al.  Some considerations regarding the use of multi-fidelity Kriging in the construction of surrogate models , 2015 .

[17]  Raphael T. Haftka,et al.  Remarks on multi-fidelity surrogates , 2016, Structural and Multidisciplinary Optimization.

[18]  R. Haftka,et al.  Review of multi-fidelity models , 2016, Advances in Computational Science and Engineering.

[19]  A. O'Hagan,et al.  Bayesian calibration of computer models , 2001 .

[20]  Andreas C. Damianou,et al.  Nonlinear information fusion algorithms for data-efficient multi-fidelity modelling , 2017, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[21]  Neil D. Lawrence,et al.  Sparse Convolved Gaussian Processes for Multi-output Regression , 2008, NIPS.

[22]  G. Karniadakis,et al.  Spectral/hp Element Methods for Computational Fluid Dynamics , 2005 .

[23]  D. Myers Matrix formulation of co-kriging , 1982 .

[24]  J. Vargas-Guzmán,et al.  Coregionalization by Linear Combination of Nonorthogonal Components , 2002 .

[25]  Samuel Kaski,et al.  Focused multi-task learning in a Gaussian process framework , 2012, Machine Learning.

[26]  Oliver Stegle,et al.  It is all in the noise: Efficient multi-task Gaussian process inference with structured residuals , 2013, NIPS.

[27]  David A. Clifton,et al.  Multitask Gaussian Processes for Multivariate Physiological Time-Series Analysis , 2015, IEEE Transactions on Biomedical Engineering.

[28]  A. O'Hagan,et al.  Predicting the output from a complex computer code when fast approximations are available , 2000 .

[29]  Edwin V. Bonilla,et al.  Collaborative Multi-output Gaussian Processes , 2014, UAI.

[30]  Neil D. Lawrence,et al.  Kernels for Vector-Valued Functions: a Review , 2011, Found. Trends Mach. Learn..

[31]  Timoleon Kipouros,et al.  A Multi-Objective Multi-Fidelity framework for global optimization , 2017 .

[32]  Robert B. Gramacy,et al.  Cases for the nugget in modeling computer experiments , 2010, Statistics and Computing.

[33]  Stefan Görtz,et al.  Improving variable-fidelity surrogate modeling via gradient-enhanced kriging and a generalized hybrid bridge function , 2013 .

[34]  Hisashi Kashima,et al.  Self-measuring Similarity for Multi-task Gaussian Process , 2011, ICML Unsupervised and Transfer Learning.

[35]  Kaworu Ebana,et al.  Multi-task Gaussian process for imputing missing data in multi-trait and multi-environment trials , 2016, Theoretical and Applied Genetics.

[36]  Marcus R. Frean,et al.  Dependent Gaussian Processes , 2004, NIPS.

[37]  Loïc Le Gratiet,et al.  Cokriging-Based Sequential Design Strategies Using Fast Cross-Validation Techniques for Multi-Fidelity Computer Codes , 2015, Technometrics.

[38]  Edwin V. Bonilla,et al.  Multi-task Gaussian Process Prediction , 2007, NIPS.

[39]  R. Ababou,et al.  On the condition number of covariance matrices in kriging, estimation, and simulation of random fields , 1994 .

[40]  Radford M. Neal Monte Carlo Implementation of Gaussian Process Models for Bayesian Regression and Classification , 1997, physics/9701026.

[41]  Alexander I. J. Forrester,et al.  Multi-fidelity optimization via surrogate modelling , 2007, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[42]  Søren Nymand Lophaven,et al.  Aspects of the Matlab toolbox DACE , 2002 .

[43]  Peter Z. G. Qian,et al.  Bayesian Hierarchical Modeling for Integrating Low-Accuracy and High-Accuracy Experiments , 2008, Technometrics.

[44]  G. Karniadakis,et al.  Model inversion via multi-fidelity Bayesian optimization: a new paradigm for parameter estimation in haemodynamics, and beyond , 2016, Journal of The Royal Society Interface.

[45]  Andy J. Keane,et al.  Cokriging for Robust Design Optimization , 2012 .