Local Gaussian Process Approximation for Large Computer Experiments

We provide a new approach to approximate emulation of large computer experiments. By focusing expressly on desirable properties of the predictive equations, we derive a family of local sequential design schemes that dynamically define the support of a Gaussian process predictor based on a local subset of the data. We further derive expressions for fast sequential updating of all needed quantities as the local designs are built up iteratively. Then we show how independent application of our local design strategy across the elements of a vast predictive grid facilitates a trivially parallel implementation. The end result is a global predictor able to take advantage of modern multicore architectures, providing a nonstationary modeling feature as a bonus. We demonstrate our method on two examples using designs with thousands of data points, and compare to the method of compactly supported covariances. Supplementary materials for this article are available online.

[1]  A. O'Hagan,et al.  Bayesian inference for non‐stationary spatial covariance structure via spatial deformations , 2003 .

[2]  Robert B. Gramacy,et al.  Optimization Under Unknown Constraints , 2010, 1004.4027.

[3]  Robert B. Gramacy,et al.  Massively parallel approximate Gaussian process regression , 2013, SIAM/ASA J. Uncertain. Quantification.

[4]  Michael L. Stein,et al.  Interpolation of spatial data , 1999 .

[5]  Carl E. Rasmussen,et al.  A Unifying View of Sparse Approximate Gaussian Process Regression , 2005, J. Mach. Learn. Res..

[6]  Derek Bingham,et al.  Efficient emulators of computer experiments using compactly supported correlation functions, with an application to cosmology , 2011, 1107.0749.

[7]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[8]  David A. Cohn,et al.  Neural Network Exploration Using Optimal Experiment Design , 1993, NIPS.

[9]  Jianhua Z. Huang,et al.  A full scale approximation of covariance functions for large spatial data sets , 2012 .

[10]  Robert B. Gramacy,et al.  Particle Learning of Gaussian Process Models for Sequential Design and Optimization , 2009, 0909.5262.

[11]  J. Berger,et al.  Objective Bayesian Analysis of Spatially Correlated Data , 2001 .

[12]  Zoubin Ghahramani,et al.  Sparse Gaussian Processes using Pseudo-inputs , 2005, NIPS.

[13]  N. Cressie,et al.  Fixed rank kriging for very large spatial data sets , 2008 .

[14]  Robert B. Gramacy,et al.  Adaptive Design and Analysis of Supercomputer Experiments , 2008, Technometrics.

[15]  Zhiyi Chi,et al.  Approximating likelihoods for large spatial data sets , 2004 .

[16]  P. E. O'connell,et al.  River flow forecasting through conceptual models part III - The Ray catchment at Grendon Underwood , 1970 .

[17]  J. Nash,et al.  River flow forecasting through conceptual models part I — A discussion of principles☆ , 1970 .

[18]  X. Emery The kriging update equations and their application to the selection of neighboring data , 2009 .

[19]  M. D. McKay,et al.  A comparison of three methods for selecting values of input variables in the analysis of output from a computer code , 2000 .

[20]  D. Nychka,et al.  Covariance Tapering for Interpolation of Large Spatial Datasets , 2006 .

[21]  A. V. Vecchia Estimation and model identification for continuous spatial processes , 1988 .

[22]  Pritam Ranjan,et al.  A Short Note on Gaussian Process Modeling for Large Datasets using Graphics Processing Units , 2011 .

[23]  Christoforos Anagnostopoulos,et al.  Dynamic trees for streaming and massive data contexts , 2012, 1201.5568.

[24]  Peter Z. G. Qian,et al.  Accurate emulators for large-scale computer experiments , 2011, 1203.2433.

[25]  Robert B. Gramacy,et al.  Cases for the nugget in modeling computer experiments , 2010, Statistics and Computing.

[26]  Stephen Barnett,et al.  Matrix Methods for Engineers and Scientists , 1982 .

[27]  Christoforos Anagnostopoulos,et al.  Information-Theoretic Data Discarding for Dynamic Trees on Data Streams , 2013, Entropy.

[28]  T. J. Mitchell,et al.  Bayesian design and analysis of computer experiments: Use of derivatives in surface prediction , 1993 .

[29]  Christopher J Paciorek,et al.  Spatial modelling using a new class of nonstationary covariance functions , 2006, Environmetrics.

[30]  Klaus Obermayer,et al.  Gaussian process regression: active data selection and test point rejection , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[31]  Robert B. Gramacy,et al.  Calibrating a large computer experiment simulating radiative shock hydrodynamics , 2014, 1410.3293.

[32]  Pritam Ranjan,et al.  A Computationally Stable Approach to Gaussian Process Interpolation of Deterministic Computer Simulation Data , 2010, Technometrics.

[33]  Noel A Cressie,et al.  Statistics for Spatial Data, Revised Edition. , 1994 .

[34]  Jo Eidsvik,et al.  Estimation and Prediction in Spatial Models With Block Composite Likelihoods , 2014 .

[35]  J. Andrew Royle,et al.  Multiresolution models for nonstationary spatial covariance functions , 2002 .

[36]  Elad Gilboa,et al.  Scaling Multidimensional Inference for Structured Gaussian Processes , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  B. A. Worley Deterministic uncertainty analysis , 1987 .

[38]  Sonja Kuhnt,et al.  Design and analysis of computer experiments , 2010 .