Clustered active-subspace based local Gaussian Process emulator for high-dimensional and complex computer models

Quantifying uncertainties in physical or engineering systems often requires a large number of simulations of the underlying computer models that are computationally intensive. Emulators or surrogate models are often used to accelerate the computation in such problems, and in this regard the Gaussian Process (GP) emulator is a popular choice for its ability to quantify the approximation error in the emulator itself. However, a major limitation of the GP emulator is that it can not handle problems of very high dimensions, which is often addressed with dimension reduction techniques. In this work we hope to address an issue that the models of interest are so complex that they admit different low dimensional structures in different parameter regimes. Building upon the active subspace method for dimension reduction, we propose a clustered active subspace method which identifies the local low-dimensional structures as well as the parameter regimes they are in (represented as clusters), and then construct low dimensional and local GP emulators within the clusters. Specifically we design a clustering method based on the gradient information to identify these clusters, and a local GP construction procedure to construct the GP emulator within a local cluster. With numerical examples, we demonstrate Preprint submitted to Elsevier 5 January 2021 that the proposed method is effective when the underlying models are of complex low-dimensional structures.

[1]  Jinglai Li,et al.  Gaussian process surrogates for failure detection: A Bayesian experimental design approach , 2015, J. Comput. Phys..

[2]  Sonja Kuhnt,et al.  Design and analysis of computer experiments , 2010 .

[3]  Kailiang Wu,et al.  Data Driven Governing Equations Approximation Using Deep Neural Networks , 2018, J. Comput. Phys..

[4]  Nicolas Le Roux,et al.  The Curse of Highly Variable Functions for Local Kernel Machines , 2005, NIPS.

[5]  Xiang Ma,et al.  An adaptive hierarchical sparse grid collocation algorithm for the solution of stochastic differential equations , 2009, J. Comput. Phys..

[6]  Temple F. Smith Occam's razor , 1980, Nature.

[7]  Paul G. Constantine,et al.  Active Subspaces - Emerging Ideas for Dimension Reduction in Parameter Studies , 2015, SIAM spotlights.

[8]  Paris Perdikaris,et al.  Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data , 2019, J. Comput. Phys..

[9]  Christine A. Shoemaker,et al.  Constrained Global Optimization of Expensive Black Box Functions Using Radial Basis Functions , 2005, J. Glob. Optim..

[10]  A. O'Hagan,et al.  Bayesian inference for the uncertainty distribution of computer model outputs , 2002 .

[11]  Ilias Bilionis,et al.  Multi-output local Gaussian process regression: Applications to uncertainty quantification , 2012, J. Comput. Phys..

[12]  George Em Karniadakis,et al.  Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems , 2018, J. Comput. Phys..

[13]  Bing Li,et al.  Sufficient Dimension Reduction: Methods and Applications with R , 2018 .

[14]  Qiqi Wang,et al.  Erratum: Active Subspace Methods in Theory and Practice: Applications to Kriging Surfaces , 2013, SIAM J. Sci. Comput..

[15]  Jinglai Li,et al.  A surrogate accelerated multicanonical Monte Carlo method for uncertainty quantification , 2016, J. Comput. Phys..

[16]  D. Xiu Efficient collocational approach for parametric uncertainty analysis , 2007 .

[17]  Andreas Krause,et al.  High-Dimensional Gaussian Process Bandits , 2013, NIPS.

[18]  Ilias Bilionis,et al.  Gaussian processes with built-in dimensionality reduction: Applications in high-dimensional uncertainty propagation , 2016, 1602.04550.

[19]  Hongqiao Wang,et al.  Adaptive Gaussian Process Approximation for Bayesian Inference with Expensive Likelihood Functions , 2017, Neural Computation.

[20]  Jing Li,et al.  When Bifidelity Meets CoKriging: An Efficient Physics-Informed Multifidelity Method , 2018, SIAM J. Sci. Comput..

[21]  Dongbin Xiu,et al.  The Wiener-Askey Polynomial Chaos for Stochastic Differential Equations , 2002, SIAM J. Sci. Comput..

[22]  Serge Guillas,et al.  Dimension Reduction for Gaussian Process Emulation: An Application to the Influence of Bathymetry on Tsunami Heights , 2016, SIAM/ASA J. Uncertain. Quantification.

[23]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[24]  Kirthevasan Kandasamy,et al.  Bayesian active learning for posterior estimation , 2015 .

[25]  Hans-Martin Gutmann,et al.  A Radial Basis Function Method for Global Optimization , 2001, J. Glob. Optim..

[26]  Trent Michael Russi,et al.  Uncertainty Quantification with Experimental Data and Complex System Models , 2010 .

[27]  A. O'Hagan,et al.  Probabilistic sensitivity analysis of complex models: a Bayesian approach , 2004 .

[28]  James O. Berger,et al.  Uncertainty analysis and other inference tools for complex computer codes , 1998 .

[29]  Ling Li,et al.  Sequential design of computer experiments for the estimation of a probability of failure , 2010, Statistics and Computing.

[30]  M. Heinkenschloss,et al.  Large-Scale PDE-Constrained Optimization: An Introduction , 2003 .

[31]  Raúl Tempone,et al.  Galerkin Finite Element Approximations of Stochastic Elliptic Partial Differential Equations , 2004, SIAM J. Numer. Anal..

[32]  Alex A. Gorodetsky,et al.  Mercer kernels and integrated variance experimental design: connections between Gaussian process regression and polynomial approximation , 2015, SIAM/ASA J. Uncertain. Quantification.

[33]  Ker-Chau Li,et al.  Sliced Inverse Regression for Dimension Reduction , 1991 .

[34]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.