A General Framework for Vecchia Approximations of Gaussian Processes

Gaussian processes (GPs) are commonly used as models for functions, time series, and spatial fields, but they are computationally infeasible for large datasets. Focusing on the typical setting of modeling data as a GP plus an additive noise term, we propose a generalization of the Vecchia (1988) approach as a framework for GP approximations. We show that our general Vecchia approach contains many popular existing GP approximations as special cases, allowing for comparisons among the different methods within a unified framework. Representing the models by directed acyclic graphs, we determine the sparsity of the matrices necessary for inference, which leads to new insights regarding the computational properties. Based on these results, we propose a novel sparse general Vecchia approximation, which ensures computational feasibility for large spatial datasets but can lead to considerable improvements in approximation accuracy over Vecchia's original approach. We provide several theoretical results and conduct numerical comparisons. We conclude with guidelines for the use of Vecchia approximations in spatial statistics.

[1]  C. Striebel,et al.  On the maximum likelihood estimates for linear dynamic systems , 1965 .

[2]  W. F. Tinney,et al.  On computing certain elements of the inverse of a sparse matrix , 1975, Commun. ACM.

[3]  A. V. Vecchia.  Estimation and model identification for continuous spatial processes , 1988 .

[4]  N. Cressie,et al.  Image analysis with partially ordered Markov models , 1998 .

[5]  Michael I. Jordan.  Graphical Models , 2003 .

[6]  N. Cressie,et al.  A dimension-reduced approach to space-time Kalman filtering , 1999 .

[7]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[8]  R. Eubank,et al.  The Equivalence Between the Cholesky Decomposition and the Kalman Filter , 2002 .

[9]  Sw. Banerjee,et al.  Hierarchical Modeling and Analysis for Spatial Data , 2003 .

[10]  Zhiyi Chi,et al.  Approximating likelihoods for large spatial data sets , 2004 .

[11]  David Higdon,et al.  A process-convolution approach to modelling temperatures in the North Atlantic Ocean , 1998, Environmental and Ecological Statistics.

[12]  Carl E. Rasmussen,et al.  A Unifying View of Sparse Approximate Gaussian Process Regression , 2005, J. Mach. Learn. Res..

[13]  Leonhard Held,et al.  Gaussian Markov Random Fields: Theory and Applications , 2005 .

[14]  D. Nychka,et al.  Covariance Tapering for Interpolation of Large Spatial Datasets , 2006 .

[15]  Zoubin Ghahramani,et al.  Local and global sparse Gaussian process approximations , 2007, AISTATS.

[16]  A. Gelfand,et al.  Gaussian predictive process models for large spatial data sets , 2008, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[17]  N. Cressie,et al.  Fixed rank kriging for very large spatial data sets , 2008 .

[18]  Eric Darve,et al.  Computing entries of the inverse of a sparse matrix using the FIND algorithm , 2008, J. Comput. Phys..

[19]  Douglas W. Nychka,et al.  Covariance Tapering for Likelihood-Based Estimation in Large Spatial Data Sets , 2008 .

[20]  Peter Buhlmann,et al.  High dimensional sparse covariance estimation via directed acyclic graphs , 2009, 0911.2375.

[21]  V. Mandrekar,et al.  Fixed-domain asymptotic properties of tapered maximum likelihood estimators , 2009, 0909.0359.

[22]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[23]  Andrew O. Finley,et al.  Improving the performance of predictive process modeling for large datasets , 2009, Comput. Stat. Data Anal..

[24]  Leonhard Held,et al.  Discrete Spatial Variation , 2010 .

[25]  Stephan R. Sain,et al.  spam: A Sparse Matrix R Package with Emphasis on MCMC Methods for Gaussian Markov Random Fields , 2010 .

[26]  N. Reid,et al.  AN OVERVIEW OF COMPOSITE LIKELIHOOD METHODS , 2011 .

[27]  Matthias Katzfuss,et al.  Spatio‐temporal smoothing and EM estimation for massive remote‐sensing data sets , 2011 .

[28]  H. Rue,et al.  An explicit link between Gaussian fields and Gaussian Markov random fields: the stochastic partial differential equation approach , 2011 .

[29]  Michael L. Stein,et al.  2010 Rietz lecture: When does the screening effect hold? , 2011, 1203.1801.

[30]  Jianhua Z. Huang,et al.  Covariance approximation for large multivariate spatial data sets with an application to multiple climate model errors , 2011, 1203.0133.

[31]  Lexing Ying,et al.  SelInv---An Algorithm for Selected Inversion of a Sparse Symmetric Matrix , 2011, TOMS.

[32]  Jianhua Z. Huang,et al.  A full scale approximation of covariance functions for large spatial data sets , 2012 .

[33]  Hao Zhang.  Asymptotics and Computation for Spatial Statistics , 2012 .

[34]  Robert B. Gramacy,et al.  Cases for the nugget in modeling computer experiments , 2010, Statistics and Computing.

[35]  Dorit Hammerling,et al.  Explorer A Multi-resolution Gaussian process model for the analysis of large spatial data sets , 2012 .

[36]  Jorge Mateu,et al.  Estimating Space and Space-Time Covariance Functions for Large Data Sets: A Weighted Composite Likelihood Approach , 2012 .

[37]  Daniel W. Apley,et al.  Local Gaussian Process Approximation for Large Computer Experiments , 2013, 1303.0383.

[38]  B. Shaby,et al.  The Open-Faced Sandwich Adjustment for MCMC Using Estimating Functions , 2012, 1204.3687.

[39]  Christian P. Robert,et al.  Statistics for Spatio-Temporal Data , 2014 .

[40]  Michael L. Stein,et al.  Limitations on low rank approximations for covariance matrices of spatial data , 2014 .

[41]  Matthias Katzfuss,et al.  A Multi-Resolution Approximation for Massive Spatial Datasets , 2015, 1507.04789.

[42]  N. Hamm,et al.  NONSEPARABLE DYNAMIC NEAREST NEIGHBOR GAUSSIAN PROCESS MODELS FOR LARGE SPATIO-TEMPORAL DATA WITH AN APPLICATION TO PARTICULATE MATTER ANALYSIS. , 2015, The annals of applied statistics.

[43]  Joseph Guinness.  Permutation Methods for Sharpening Gaussian Process Approximations , 2016 .

[44]  Sudipto Banerjee,et al.  On nearest‐neighbor Gaussian process models for massive spatial data , 2016, Wiley interdisciplinary reviews. Computational statistics.

[45]  Sudipto Banerjee,et al.  Hierarchical Nearest-Neighbor Gaussian Process Models for Large Geostatistical Datasets , 2014, Journal of the American Statistical Association.

[46]  Ying Sun,et al.  Statistically and Computationally Efficient Estimating Equations for Large Spatial Datasets , 2016 .

[47]  Huang Huang,et al.  Hierarchical Low Rank Approximation of Likelihoods for Large Spatial Datasets , 2016, 1605.08898.

[48]  Matthias Katzfuss,et al.  A class of multi-resolution approximations for large spatial datasets , 2017, Statistica Sinica.

[49]  Andrew O. Finley,et al.  Applying Nearest Neighbor Gaussian Processes to Massive Spatial Data Sets: Forest Canopy Height Prediction Across Tanana Valley Alaska , 2017 .

[50]  Lexing Ying,et al.  Fast Spatial Gaussian Process Maximum Likelihood Estimation via Skeletonization Factorizations , 2016, Multiscale Model. Simul..

[51]  Joseph Guinness,et al.  Permutation and Grouping Methods for Sharpening Gaussian Process Approximations , 2016, Technometrics.

[52]  Matthias Katzfuss,et al.  Vecchia Approximations of Gaussian-Process Predictions , 2018, Journal of Agricultural, Biological and Environmental Statistics.

[53]  Michael E. Schaepman,et al.  Predicting Missing Values in Spatio-Temporal Remote Sensing Data , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[54]  Sudipto Banerjee,et al.  Web Appendix: Meta-Kriging: Scalable Bayesian Modeling and Inference for Massive Spatial Datasets , 2018 .

[55]  Dorit Hammerling,et al.  A Case Study Competition Among Methods for Analyzing Large Spatial Data , 2017, Journal of Agricultural, Biological and Environmental Statistics.

[56]  Jianhua Z. Huang,et al.  Smoothed Full-Scale Approximation of Gaussian Process Models for Computation of Large Spatial Datasets , 2019, Statistica Sinica.

[57]  Matthias Katzfuss,et al.  Multi-Resolution Filters for Massive Spatio-Temporal Data , 2018, Journal of Computational and Graphical Statistics.