Clustered active-subspace based local Gaussian Process emulator for high-dimensional and complex computer models

Quantifying uncertainties in physical or engineering systems often requires a large number of simulations of the underlying computer models that are computationally intensive. Emulators or surrogate models are often used to accelerate the computation in such problems, and in this regard the Gaussian Process (GP) emulator is a popular choice for its ability to quantify the approximation error in the emulator itself. However, a major limitation of the GP emulator is that it can not handle problems of very high dimensions, which is often addressed with dimension reduction techniques. In this work we hope to address an issue that the models of interest are so complex that they admit different low dimensional structures in different parameter regimes. Building upon the active subspace method for dimension reduction, we propose a clustered active subspace method which identifies the local low-dimensional structures as well as the parameter regimes they are in (represented as clusters), and then construct low dimensional and local GP emulators within the clusters. Specifically we design a clustering method based on the gradient information to identify these clusters, and a local GP construction procedure to construct the GP emulator within a local cluster. With numerical examples, we demonstrate Preprint submitted to Elsevier 11 November 2021 that the proposed method is effective when the underlying models are of complex low-dimensional structures.

[1]  Jinglai Li,et al.  Gaussian process surrogates for failure detection: A Bayesian experimental design approach , 2015, J. Comput. Phys..

[2]  Jing Li,et al.  When Bifidelity Meets CoKriging: An Efficient Physics-Informed Multifidelity Method , 2018, SIAM J. Sci. Comput..

[3]  Ker-Chau Li,et al.  Sliced Inverse Regression for Dimension Reduction , 1991 .

[4]  Peter J. Rousseeuw,et al.  Finding Groups in Data: An Introduction to Cluster Analysis , 1990 .

[5]  Kailiang Wu,et al.  Data Driven Governing Equations Approximation Using Deep Neural Networks , 2018, J. Comput. Phys..

[6]  Sylvain Arlot,et al.  A survey of cross-validation procedures for model selection , 2009, 0907.4728.

[7]  Dongbin Xiu,et al.  The Wiener-Askey Polynomial Chaos for Stochastic Differential Equations , 2002, SIAM J. Sci. Comput..

[8]  A. O'Hagan,et al.  Bayesian inference for the uncertainty distribution of computer model outputs , 2002 .

[9]  Ilias Bilionis,et al.  Gaussian processes with built-in dimensionality reduction: Applications in high-dimensional uncertainty propagation , 2016, 1602.04550.

[10]  M. Heinkenschloss,et al.  Large-Scale PDE-Constrained Optimization: An Introduction , 2003 .

[11]  George M. Saviello Comment … , 1969, Comparative Studies in Society and History.

[12]  Ling Li,et al.  Sequential design of computer experiments for the estimation of a probability of failure , 2010, Statistics and Computing.

[13]  Thomas J. Santner,et al.  Design and analysis of computer experiments , 1998 .

[14]  Christine A. Shoemaker,et al.  Constrained Global Optimization of Expensive Black Box Functions Using Radial Basis Functions , 2005, J. Glob. Optim..

[15]  Ilias Bilionis,et al.  Multi-output local Gaussian process regression: Applications to uncertainty quantification , 2012, J. Comput. Phys..

[16]  Paris Perdikaris,et al.  Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data , 2019, J. Comput. Phys..

[17]  D. Xiu Efficient collocational approach for parametric uncertainty analysis , 2007 .

[18]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[19]  Paul G. Constantine,et al.  Active Subspaces - Emerging Ideas for Dimension Reduction in Parameter Studies , 2015, SIAM spotlights.

[20]  Hongqiao Wang,et al.  Adaptive Gaussian Process Approximation for Bayesian Inference with Expensive Likelihood Functions , 2017, Neural Computation.

[21]  Hans-Martin Gutmann,et al.  A Radial Basis Function Method for Global Optimization , 2001, J. Glob. Optim..

[22]  James O. Berger,et al.  Uncertainty analysis and other inference tools for complex computer codes , 1998 .

[23]  Temple F. Smith Occam's razor , 1980, Nature.

[24]  George Em Karniadakis,et al.  Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems , 2018, J. Comput. Phys..

[25]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[26]  A. O'Hagan,et al.  Probabilistic sensitivity analysis of complex models: a Bayesian approach , 2004 .

[27]  Alex A. Gorodetsky,et al.  Mercer kernels and integrated variance experimental design: connections between Gaussian process regression and polynomial approximation , 2015, SIAM/ASA J. Uncertain. Quantification.

[28]  Xiang Ma,et al.  An adaptive hierarchical sparse grid collocation algorithm for the solution of stochastic differential equations , 2009, J. Comput. Phys..

[29]  Serge Guillas,et al.  Dimension Reduction for Gaussian Process Emulation: An Application to the Influence of Bathymetry on Tsunami Heights , 2016, SIAM/ASA J. Uncertain. Quantification.

[30]  Andreas Krause,et al.  High-Dimensional Gaussian Process Bandits , 2013, NIPS.

[31]  Nicolas Le Roux,et al.  The Curse of Highly Variable Functions for Local Kernel Machines , 2005, NIPS.

[32]  Jinglai Li,et al.  A surrogate accelerated multicanonical Monte Carlo method for uncertainty quantification , 2016, J. Comput. Phys..

[33]  Trent Michael Russi,et al.  Uncertainty Quantification with Experimental Data and Complex System Models , 2010 .

[34]  Sufficient Dimension Reduction: Methods and Applications With R , 2020 .

[35]  Qiqi Wang,et al.  Erratum: Active Subspace Methods in Theory and Practice: Applications to Kriging Surfaces , 2013, SIAM J. Sci. Comput..

[36]  Kirthevasan Kandasamy,et al.  Bayesian active learning for posterior estimation , 2015 .

[37]  Raúl Tempone,et al.  Galerkin Finite Element Approximations of Stochastic Elliptic Partial Differential Equations , 2004, SIAM J. Numer. Anal..

[38]  Bing Li,et al.  Sufficient Dimension Reduction: Methods and Applications with R , 2018 .