Speeding Up Neighborhood Search in Local Gaussian Process Prediction

Recent implementations of local approximate Gaussian process models have pushed computational boundaries for nonlinear, nonparametric prediction problems, particularly when deployed as emulators for computer experiments. Their flavor of spatially independent computation accommodates massive parallelization, meaning that they can handle designs two or more orders of magnitude larger than previously. However, accomplishing that feat can still require massive computational horsepower. Here we aim to ease that burden. We study how predictive variance is reduced as local designs are built up for prediction. We then observe how the exhaustive and discrete nature of an important search subroutine involved in building such local designs may be overly conservative. Rather, we suggest that searching the space radially, that is, continuously along rays emanating from the predictive location of interest, is a far thriftier alternative. Our empirical work demonstrates that ray-based search yields predictors with accuracy comparable to exhaustive search, but in a fraction of the time—for many problems bringing a supercomputer implementation back onto the desktop. Supplementary materials for this article are available online.

[1]  Jianhua Z. Huang,et al.  A full scale approximation of covariance functions for large spatial data sets , 2012 .

[2]  B. A. Worley Deterministic uncertainty analysis , 1987 .

[3]  Sonja Kuhnt,et al.  Design and analysis of computer experiments , 2010 .

[4]  Robert B. Gramacy,et al.  Adaptive Design and Analysis of Supercomputer Experiments , 2008, Technometrics.

[5]  Klaus Obermayer,et al.  Gaussian process regression: active data selection and test point rejection , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[6]  R. Brent Table errata: Algorithms for minimization without derivatives (Prentice-Hall, Englewood Cliffs, N. J., 1973) , 1975 .

[7]  Sudipto Banerjee,et al.  Hierarchical Nearest-Neighbor Gaussian Process Models for Large Geostatistical Datasets , 2014, Journal of the American Statistical Association.

[8]  J. Berger,et al.  Objective Bayesian Analysis of Spatially Correlated Data , 2001 .

[9]  Robert B. Gramacy,et al.  Particle Learning of Gaussian Process Models for Sequential Design and Optimization , 2009, 0909.5262.

[10]  Peter Challenor,et al.  Computational Statistics and Data Analysis the Effect of the Nugget on Gaussian Process Emulators of Computer Models , 2022 .

[11]  Robert B. Gramacy,et al.  Calibrating a large computer experiment simulating radiative shock hydrodynamics , 2014, 1410.3293.

[12]  Rae. Z.H. Aliyev,et al.  Interpolation of Spatial Data , 2018, Biomedical Journal of Scientific & Technical Research.

[13]  Robert B. Gramacy,et al.  Cases for the nugget in modeling computer experiments , 2010, Statistics and Computing.

[14]  Stephen A. Freitas,et al.  Modern Industrial Statistics: Design and Control of Quality and Reliability , 1999, Technometrics.

[15]  X. Emery The kriging update equations and their application to the selection of neighboring data , 2009 .

[16]  Stefan M. Wild,et al.  Variable selection and sensitivity analysis using dynamic trees, with an application to computer code performance tuning , 2011, 1108.4739.

[17]  Zhiyi Chi,et al.  Approximating likelihoods for large spatial data sets , 2004 .

[18]  Daniel W. Apley,et al.  Local Gaussian Process Approximation for Large Computer Experiments , 2013, 1303.0383.

[19]  Zoubin Ghahramani,et al.  Sparse Gaussian Processes using Pseudo-inputs , 2005, NIPS.

[20]  Derek Bingham,et al.  Efficient emulators of computer experiments using compactly supported correlation functions, with an application to cosmology , 2011, 1107.0749.

[21]  James R. Gattiker,et al.  Parallel Bayesian Additive Regression Trees , 2013, 1309.1906.

[22]  Jo Eidsvik,et al.  Estimation and Prediction in Spatial Models With Block Composite Likelihoods , 2014 .

[23]  Noel A Cressie,et al.  Statistics for Spatial Data, Revised Edition. , 1994 .

[24]  Stanley H. Cohen,et al.  Design and Analysis , 2010 .

[25]  Robert B. Gramacy,et al.  Massively parallel approximate Gaussian process regression , 2013, SIAM/ASA J. Uncertain. Quantification.

[26]  Noel A Cressie,et al.  Statistics for Spatial Data. , 1992 .

[27]  Jason L. Loeppky,et al.  Analysis Methods for Computer Experiments: How to Assess and What Counts? , 2016 .

[28]  N. Cressie,et al.  Fixed rank kriging for very large spatial data sets , 2008 .

[29]  David M. Steinberg,et al.  Modeling Data from Computer Experiments: An Empirical Comparison of Kriging with MARS and Projection Pursuit Regression , 2007 .

[30]  Weiwei Wang,et al.  Sequential design for computer experiments with a flexible Bayesian additive model , 2012, 1203.1078.

[31]  Prabhat,et al.  Parallelizing Gaussian Process Calculations in R , 2013, ArXiv.

[32]  A. V. Vecchia Estimation and model identification for continuous spatial processes , 1988 .

[33]  Robert B. Gramacy,et al.  laGP: Large-Scale Spatial Modeling via Local Approximate Gaussian Processes in R , 2016 .

[34]  Pritam Ranjan,et al.  A Short Note on Gaussian Process Modeling for Large Datasets using Graphics Processing Units , 2011 .

[35]  T. J. Mitchell,et al.  Bayesian design and analysis of computer experiments: Use of derivatives in surface prediction , 1993 .

[36]  Peter Z. G. Qian,et al.  Accurate emulators for large-scale computer experiments , 2011, 1203.2433.