An explicit link between Gaussian fields and Gaussian Markov random fields: the stochastic partial differential equation approach

Summary.  Continuously indexed Gaussian fields (GFs) are the most important ingredient in spatial statistical modelling and geostatistics. The specification through the covariance function gives an intuitive interpretation of the field properties. On the computational side, GFs are hampered with the big n problem, since the cost of factorizing dense matrices is cubic in the dimension. Although computational power today is at an all time high, this fact seems still to be a computational bottleneck in many applications. Along with GFs, there is the class of Gaussian Markov random fields (GMRFs) which are discretely indexed. The Markov property makes the precision matrix involved sparse, which enables the use of numerical algorithms for sparse matrices, that for fields in only use the square root of the time required by general algorithms. The specification of a GMRF is through its full conditional distributions but its marginal properties are not transparent in such a parameterization. We show that, using an approximate stochastic weak solution to (linear) stochastic partial differential equations, we can, for some GFs in the Matérn class, provide an explicit link, for any triangulation of , between GFs and GMRFs, formulated as a basis function representation. The consequence is that we can take the best from the two worlds and do the modelling by using GFs but do the computations by using GMRFs. Perhaps more importantly, our approach generalizes to other covariance functions generated by SPDEs, including oscillating and non‐stationary GFs, as well as GFs on manifolds. We illustrate our approach by analysing global temperature data with a non‐stationary model defined on a sphere.

[1]  Change of variables in Laplace's and other second-order differential equations , 1934 .

[2]  H. Fédérer,et al.  Hausdorff Measure and Lebesgue Area. , 1951, Proceedings of the National Academy of Sciences of the United States of America.

[3]  P. Whittle ON STATIONARY PROCESSES IN THE PLANE , 1954 .

[4]  V. Heine MODELS FOR TWO-DIMENSIONAL STATIONARY STOCHASTIC PROCESSES , 1955 .

[5]  Richard H. Jones,et al.  Stochastic Processes on a Sphere , 1963 .

[6]  L. Auslander,et al.  Introduction to Differentiable Manifolds , 1963 .

[7]  Irene A. Stegun,et al.  Handbook of Mathematical Functions. , 1966 .

[8]  P. Clark A Subordinated Stochastic Process Model with Finite Variance for Speculative Prices , 1973 .

[9]  Benoit B. Mandelbrot,et al.  A Subordinated Stochastic Process Model with Finite Variance for Speculative Prices: Comment , 1973 .

[10]  J. Besag Spatial Interaction and the Statistical Analysis of Lattice Systems , 1974 .

[11]  J. Besag Statistical Analysis of Non-Lattice Data , 1975 .

[12]  Ju A Rozanov MARKOV RANDOM FIELDS AND STOCHASTIC PARTIAL DIFFERENTIAL EQUATIONS , 1977 .

[13]  O. Barndorff-Nielsen,et al.  Infinite divisibility of the hyperbolic and generalized inverse Gaussian distributions , 1977 .

[14]  H. Fédérer,et al.  Colloquium lectures on geometric measure theory , 1978 .

[15]  Julian Besag,et al.  On a System of Two-dimensional Recurrence Equations , 1981 .

[16]  G. Wahba Spline Interpolation and Smoothing on the Sphere , 1981 .

[17]  X. Guyon Parameter estimation for a stationary process on a d-dimensional lattice , 1982 .

[18]  H. B. Mitchell Markov Random Fields , 1982 .

[19]  J. Pasciak,et al.  Computer solution of large sparse positive definite systems , 1982 .

[20]  R. Bhattacharya,et al.  THE HURST EFFECT UNDER TRENDS , 1983 .

[21]  M. Ghil,et al.  A stochastic-dynamic model for the spatial structure of forecast error statistics , 1983 .

[22]  A. Balakrishnan,et al.  Spectral theory of random fields , 1983 .

[23]  V. Thomée,et al.  The lumped mass finite element method for a parabolic problem , 1985, The Journal of the Australian Mathematical Society. Series B. Applied Mathematics.

[24]  I. Duff,et al.  Direct Methods for Sparse Matrices , 1987 .

[25]  R. Dahlhaus,et al.  Edge effects and efficient parameter estimation for stationary random fields , 1987 .

[26]  References to discussion , 1988 .

[27]  A. V. Vecchia Estimation and model identification for continuous spatial processes , 1988 .

[28]  J. Keller,et al.  Exact non-reflecting boundary conditions , 1989 .

[29]  Fred L. Bookstein,et al.  Principal Warps: Thin-Plate Splines and the Decomposition of Deformations , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  P. McCullagh,et al.  Generalized Linear Models, 2nd Edn. , 1990 .

[31]  J. Besag,et al.  Bayesian image restoration, with two applications in spatial statistics , 1991 .

[32]  P. Guttorp,et al.  Nonparametric Estimation of Nonstationary Spatial Covariance Structure , 1992 .

[33]  Robert Haining,et al.  Statistics for spatial data: by Noel Cressie, 1991, John Wiley & Sons, New York, 900 p., ISBN 0-471-84336-9, US $89.95 , 1993 .

[34]  O. Marichev,et al.  Fractional Integrals and Derivatives: Theory and Applications , 1993 .

[35]  Mike Rees,et al.  5. Statistics for Spatial Data , 1993 .

[36]  R. J. Martin,et al.  Approximations to the covariance properties of processes averaged over irregular spatial regions , 1994 .

[37]  L. R. Scott,et al.  The Mathematical Theory of Finite Element Methods , 1994 .

[38]  J. Besag,et al.  On conditional and intrinsic autoregressions , 1995 .

[39]  Douglas W. Nychka,et al.  Splines as Local Smoothers , 1995 .

[40]  P. Green Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[41]  F. J. Narcowich,et al.  Generalized Hermite interpolation and positive definite kernels on a Riemannian manifold , 1995 .

[42]  Vladimir I. Piterbarg,et al.  Asymptotic Methods in the Theory of Gaussian Processes and Fields , 1995 .

[43]  R. Tweedie,et al.  Exponential convergence of Langevin distributions and their discrete approximations , 1996 .

[44]  Elja Arjas,et al.  Bayesian Inference of Survival Probabilities, under Stochastic Ordering Constraints , 1996 .

[45]  Michael I. Miller,et al.  Deformable templates using large deformation kinematics , 1996, IEEE Trans. Image Process..

[46]  Eulogio Pardo-Igúzquiza,et al.  AMLE3D: a computer program for the inference of spatial covariance parameters by approximate maximum likelihood estimation , 1997 .

[47]  R. Tweedie,et al.  Exponential Convergence of Langevin Diiusions and Their Discrete Approximations , 1997 .

[48]  R. Vose,et al.  An Overview of the Global Historical Climatology Network Temperature Database , 1997 .

[49]  J. Møller,et al.  Log Gaussian Cox Processes , 1998 .

[50]  T. Gneiting Simple tests for the validity of correlation function models on the circle , 1998 .

[51]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[52]  P. Diggle,et al.  Model-based geostatistics (with discussion). , 1998 .

[53]  T. C. Haas,et al.  Model-based geostatistics. Discussion. Authors' reply , 1998 .

[54]  T. C. Haas,et al.  Model-based geostatistics - Discussion , 1998 .

[55]  Jye-Chyi Lu,et al.  Parametric nonstationary correlation models , 1998 .

[56]  Vipin Kumar,et al.  A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs , 1998, SIAM J. Sci. Comput..

[57]  David Higdon,et al.  Non-Stationary Spatial Modeling , 2022, 2212.08043.

[58]  N. Cressie,et al.  Classes of nonseparable, spatio-temporal stationary covariance functions , 1999 .

[59]  Dan Givoli,et al.  Recent advances in the DtN FE Method , 1999 .

[60]  Makiko Sato,et al.  GISS analysis of surface temperature change , 1999 .

[61]  J. Chilès,et al.  Geostatistics: Modeling Spatial Uncertainty , 1999 .

[62]  Barnali Das Global covariance modeling: A deformation approach to anisotropy , 2000 .

[63]  J. Oden,et al.  A Posteriori Error Estimation in Finite Element Analysis , 2000 .

[64]  Miscellanea. Exact Gaussian maximum likelihood and simulation for regularly-spaced observations with Gaussian correlations , 2000 .

[65]  Anthony N. Pettitt,et al.  Binary probability maps using a hidden conditional autoregressive Gaussian process with an application to Finnish common toad data , 2000 .

[66]  Roger Woodard,et al.  Interpolation of Spatial Data: Some Theory for Kriging , 1999, Technometrics.

[67]  Herbert Edelsbrunner,et al.  Geometry and Topology for Mesh Generation , 2001, Cambridge monographs on applied and computational mathematics.

[68]  Montserrat Fuentes,et al.  A high frequency kriging approach for non‐stationary environmental processes , 2001 .

[69]  L. Mark Berliner,et al.  Spatiotemporal Hierarchical Bayesian Modeling Tropical Ocean Surface Winds , 2001 .

[70]  J. Berger,et al.  Objective Bayesian Analysis of Spatially Correlated Data , 2001 .

[71]  Makiko Sato,et al.  A closer look at United States and global surface temperature change , 2001 .

[72]  Herbert Edelsbrunner,et al.  Geometry and Topology for Mesh Generation , 2001, Cambridge monographs on applied and computational mathematics.

[73]  A. O'Hagan,et al.  Bayesian calibration of computer models , 2001 .

[74]  I. Sokolov Lévy flights from a continuous-time process. , 2000, Physical review. E, Statistical, nonlinear, and soft matter physics.

[75]  H. Rue Fast sampling of Gaussian Markov random fields , 2000 .

[76]  M. Yor,et al.  Stochastic Volatility for Lévy Processes , 2003 .

[77]  Eric R. Ziegel,et al.  Generalized Linear Models , 2002, Technometrics.

[78]  Tim Ramsay,et al.  Spline smoothing over difficult regions , 2002 .

[79]  R. Henderson,et al.  Modelling spatial variation in leukaemia survival data. , 2002 .

[80]  H. Rue,et al.  Fitting Gaussian Markov Random Fields to Gaussian Fields , 2002 .

[81]  Anthony N. Pettitt,et al.  A Conditional Autoregressive Gaussian Process for Irregularly Spaced Multivariate Data with Application to Modelling Large Sets of Binary Data , 2002, Stat. Comput..

[82]  T. Gneiting Nonseparable, Stationary Covariance Functions for Space–Time Data , 2002 .

[83]  R. LeVeque Finite Volume Methods for Hyperbolic Problems: Characteristics and Riemann Problems for Linear Hyperbolic Equations , 2002 .

[84]  M. Wiktorsson Simulation of stochastic integrals with respect to Lévy processes of type G , 2002 .

[85]  T. Gneiting Compactly Supported Correlation Functions , 2002 .

[86]  Philippe G. Ciarlet,et al.  The finite element method for elliptic problems , 2002, Classics in applied mathematics.

[87]  R. Henderson,et al.  Modeling Spatial Variation in Leukemia Survival Data , 2002 .

[88]  J. Møller,et al.  Statistical Inference and Simulation for Spatial Point Processes , 2003 .

[89]  Chris A. Glasbey,et al.  A latent Gaussian Markov random‐field model for spatiotemporal rainfall disaggregation , 2003 .

[90]  Christopher K. Wikle,et al.  Hierarchical Bayesian Models for Predicting The Spread of Ecological Processes , 2003 .

[91]  Thomas J. Santner,et al.  Design and analysis of computer experiments , 1998 .

[92]  Sw. Banerjee,et al.  Hierarchical Modeling and Analysis for Spatial Data , 2003 .

[93]  A. O'Hagan,et al.  Bayesian inference for non‐stationary spatial covariance structure via spatial deformations , 2003 .

[94]  M. Wall A close look at the spatial structure implied by the CAR and SAR models , 2004 .

[95]  Hao Zhang Inconsistent Estimation and Asymptotically Equal Interpolations in Model-Based Geostatistics , 2004 .

[96]  Holger Wendland,et al.  Scattered Data Approximation: Conditionally positive definite functions , 2004 .

[97]  Zhiyi Chi,et al.  Approximating likelihoods for large spatial data sets , 2004 .

[98]  R. Coifman,et al.  Diffusion Wavelets , 2004 .

[99]  Tilmann Gneiting,et al.  Stochastic Models That Separate Fractal Dimension and the Hurst Effect , 2001, SIAM Rev..

[100]  J. Klafter,et al.  Spatial gliding, temporal trapping, and anomalous transport , 2004 .

[101]  N. Cressie,et al.  Hierarchical modeling of count data with application to nuclear fall-out , 2003, Environmental and Ecological Statistics.

[102]  David Higdon,et al.  A process-convolution approach to modelling temperatures in the North Atlantic Ocean , 1998, Environmental and Ecological Statistics.

[103]  H. Rue,et al.  On the Second‐Order Random Walk Model for Irregular Locations , 2008 .

[104]  J. Klafter,et al.  Lévy, Ornstein–Uhlenbeck, and Subordination: Spectral vs. Jump Description , 2005 .

[105]  M. Stein Space–Time Covariance Functions , 2005 .

[106]  M. Stein Nonstationary spatial covariance functions , 2005 .

[107]  A. Gelfand,et al.  Spatio-temporal change-point modeling , 2005 .

[108]  Leonhard Held,et al.  Gaussian Markov Random Fields: Theory and Applications , 2005 .

[109]  Debashis Mondal,et al.  First-order intrinsic autoregressions and the de Wijs process , 2005 .

[110]  Oyvind Hjelle,et al.  Triangulations and applications , 2006 .

[111]  M. Stein,et al.  Spatial sampling design for prediction with estimated parameters , 2006 .

[112]  Dale L. Zimmerman,et al.  Optimal network design for spatial prediction, covariance parameter estimation, and empirical prediction , 2006 .

[113]  Christopher J Paciorek,et al.  Spatial modelling using a new class of nonstationary covariance functions , 2006, Environmetrics.

[114]  James V. Zidek,et al.  Statistical Analysis of Environmental Space-Time Processes , 2006 .

[115]  Mevin B. Hooten,et al.  Hierarchical Bayesian Spatio-Temporal Models for Population Spread , 2006 .

[116]  D. Nychka,et al.  Covariance Tapering for Interpolation of Large Spatial Datasets , 2006 .

[117]  Gennady Samorodnitsky,et al.  Long Range Dependence , 2007, Found. Trends Stoch. Syst..

[118]  C. Czado,et al.  Modelling count data with overdispersion and spatial effects , 2008 .

[119]  P. Guttorp,et al.  Studies in the history of probability and statistics XLIX On the Matérn correlation family , 2006 .

[120]  Timothy A. Davis,et al.  Direct methods for sparse linear systems , 2006, Fundamentals of algorithms.

[121]  P. Jones,et al.  Uncertainty estimates in regional and global observed temperature changes: A new data set from 1850 , 2006 .

[122]  Mikyoung Jun,et al.  An Approach to Producing Space–Time Covariance Functions on Spheres , 2007, Technometrics.

[123]  M. Hegland Approximate Maximum a Posteriori with Gaussian Process Priors , 2007 .

[124]  Doug Nychka,et al.  A framework to understand the asymptotic properties of Kriging and splines , 2007 .

[125]  Hongtu Zhu,et al.  Statistical Analyses of Brain Surfaces Using Gaussian Random Fields on 2-D Manifolds , 2007, IEEE Transactions on Medical Imaging.

[126]  R. Adler,et al.  Random Fields and Geometry , 2007 .

[127]  L. Fahrmeir,et al.  A Mixed Model Approach for Geoadditive Hazard Regression , 2007 .

[128]  J. Hesthaven,et al.  Nodal Discontinuous Galerkin Methods: Algorithms, Analysis, and Applications , 2007 .

[129]  Kathryn M. Irvine,et al.  Spatial designs and properties of spatial correlation: Effects on covariance estimation , 2007 .

[130]  R. Waagepetersen,et al.  Modern Statistics for Spatial Point Processes * , 2007 .

[131]  P. Diggle,et al.  Model‐based geostatistics , 2007 .

[132]  K. Mardia Should Geostatistics Be Model-Based? , 2007 .

[133]  M. Fuentes Approximate Likelihood for Large Irregularly Spaced Spatial Data , 2007, Journal of the American Statistical Association.

[134]  Mevin B. Hooten,et al.  A hierarchical Bayesian non-linear spatio-temporal model for the spread of invasive species with application to the Eurasian Collared-Dove , 2008, Environmental and Ecological Statistics.

[135]  Hebert Montegranario,et al.  A regularization approach for surface reconstruction from point clouds , 2007, Appl. Math. Comput..

[136]  S. Martino Approximate Bayesian Inference for Latent Gaussian Models , 2007 .

[137]  Montserrat Fuentes,et al.  A comparative study of Gaussian geostatistical models and Gaussian Markov random field models1. , 2008, Journal of multivariate analysis.

[138]  Ola Hössjer,et al.  Fast kriging of large data sets with Gaussian Markov random fields , 2008, Comput. Stat. Data Anal..

[139]  P. Diggle,et al.  Bivariate Binomial Spatial Modeling of Loa loa Prevalence in Tropical Africa , 2008 .

[140]  Jorge Mateu,et al.  The Dagum family of isotropic correlation functions , 2007, 0705.0456.

[141]  A. Gelfand,et al.  Gaussian predictive process models for large spatial data sets , 2008, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[142]  A. Quarteroni,et al.  Numerical Approximation of Partial Differential Equations , 2008 .

[143]  Simon N. Wood,et al.  Soap film smoothing , 2008 .

[144]  N. Cressie,et al.  Fixed rank kriging for very large spatial data sets , 2008 .

[145]  I. Rychlik,et al.  Fatigue damage assessment for a spectral model of non-Gaussian random loads, Åberk, S. Podgorski, K. Rychlik; I. , 2009 .

[146]  Noel Cressie,et al.  Conditional-mean least-squares fitting of Gaussian Markov random fields to Gaussian fields , 2008, Comput. Stat. Data Anal..

[147]  Asymptotic Behavior of a Continuous Approximation to the Kriging Weighting Function , 2008 .

[148]  Mikyoung Jun,et al.  Nonstationary covariance models for global data , 2008, 0901.3980.

[149]  Harold R. Parks,et al.  Geometric Integration Theory , 2008 .

[150]  Vo Anh,et al.  A Numerical Solution Using an Adaptively Preconditioned Lanczos Method for a Class of Linear Systems Related with the Fractional Poisson Equation , 2008 .

[151]  Milan Stehlík,et al.  Compound optimal spatial designs , 2009 .

[152]  Computationally efficient methods in spatial statistics : Applications in environmental modeling , 2009 .

[153]  Janine B. Illian,et al.  Gibbs point process models with mixed effects , 2009 .

[154]  H. Rue,et al.  Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations , 2009 .

[155]  A. Gelfand,et al.  Multivariate Spatial Process Models , 2021, Handbook of Regional Science.

[156]  Mario Chica-Olmo,et al.  MLMATERN: A computer program for maximum likelihood inference with the spatial Matérn covariance model , 2009, Comput. Geosci..

[157]  Szu-Lang Liao,et al.  Closed-form valuations of basket options using a multivariate normal inverse Gaussian model , 2009 .

[158]  R. Adler The Geometry of Random Fields , 2009 .

[159]  Jerome Sacks,et al.  Choosing the Sample Size of a Computer Experiment: A Practical Guide , 2009, Technometrics.

[160]  Werner G. Müller,et al.  Designs for Detecting Spatial Dependence , 2009 .

[161]  Michael Höhle,et al.  Additive‐Multiplicative Regression Models for Spatio‐Temporal Epidemics , 2009, Biometrical journal. Biometrische Zeitschrift.

[162]  K. Mardia,et al.  Maximum likelihood estimation using composite likelihoods for closed exponential families , 2009 .

[163]  Wavelet Markov models as efficient alternatives to tapering and convolution fields , 2009 .

[164]  Non-traditional stochastic models for ocean waves Lagrange models and nested SPDE models , 2010 .

[165]  Venkata K. Jandhyala,et al.  EXACT ASYMPTOTIC DISTRIBUTION OF CHANGE-POINT MLE FOR CHANGE IN THE MEAN OF GAUSSIAN SEQUENCES , 2010, 1011.2322.

[166]  Jörg Wegener Noise Convolution Models: Fluids in Stochastic Motion, Non-Gaussian Tempo-Spatial Fields, and a Notion of Tilting , 2010 .

[167]  Paul D. Sampson,et al.  Constructions for Nonstationary Spatial Processes , 2010 .

[168]  H. Rue,et al.  Fitting a log Gaussian Cox process with temporally varying effects – a case study , 2010 .

[169]  A. O'Hagan,et al.  Bayesian emulation of complex multi-output and dynamic computer models , 2010 .

[170]  L. Hervella-Nieto,et al.  Perfectly Matched Layers for Time-Harmonic Second Order Elliptic Problems , 2010 .

[171]  M. Hooten,et al.  A general science-based framework for dynamical spatio-temporal models , 2010 .

[172]  David Bolin,et al.  Non-traditional stochastic models for ocean waves , 2010 .

[173]  Yu Yue,et al.  Nonstationary Spatial Gaussian Markov Random Fields , 2010 .

[174]  P. Diggle,et al.  Geostatistical inference under preferential sampling , 2010 .

[175]  Markus Hegland,et al.  A Finite Element Method for Density Estimation with Gaussian Process Priors , 2010, SIAM J. Numer. Anal..

[176]  James R. Gattiker,et al.  Assessing the probability of rare climate events , 2010 .

[177]  J. Mateu,et al.  Ordinary kriging for function-valued spatial data , 2011, Environmental and Ecological Statistics.

[178]  D. Nychka,et al.  The Value of Multiproxy Reconstruction of Past Climate , 2010 .

[179]  Håvard Rue,et al.  A toolbox for fitting complex spatial point process models using integrated Laplace transformation (INLA) , 2010 .

[180]  Michela Cameletti,et al.  Comparing air quality statistical models , 2010, 1011.1845.

[181]  T. Gneiting,et al.  Matérn Cross-Covariance Functions for Multivariate Random Fields , 2010 .

[182]  F. Lindgren,et al.  Spatial models generated by nested stochastic partial differential equations, with an application to global ozone mapping , 2011, 1104.3436.

[183]  Michael L. Stein,et al.  Local likelihood estimation for nonstationary random fields , 2011, J. Multivar. Anal..

[184]  E. Porcu,et al.  The Dagum and auxiliary covariance families: Towards reconciling two-parameter models that separate fractal dimension and the Hurst effect , 2011 .

[185]  C. Wikle,et al.  Polynomial nonlinear spatio‐temporal integro‐difference equation models , 2011 .

[186]  Marc G. Genton,et al.  Correlation Models for Temperature Fields , 2011 .

[187]  Fractional normal inverse Gaussian diffusion , 2011 .

[188]  J. Mateu,et al.  On a class of non-stationary, compactly supported spatial covariance functions , 2013, Stochastic Environmental Research and Risk Assessment.

[189]  Krzysztof Podgórski,et al.  A class of non-Gaussian second order random fields , 2011 .

[190]  F. Lindgren,et al.  Spatial wavelet Markov models are more efficient than covariance tapering and process convolutions , 2011 .

[191]  Noel A Cressie,et al.  Statistics for Spatio-Temporal Data , 2011 .

[192]  The Genetics of Conflict: Low Level Interaction between Conflict Events , 2011 .

[193]  Anthony O'Hagan,et al.  Considering covariates in the covariance structure of spatial processes , 2011 .

[194]  H. Rue,et al.  In order to make spatial statistics computationally feasible, we need to forget about the covariance function , 2012 .

[195]  Andrew O. Finley,et al.  Norges Teknisk-naturvitenskapelige Universitet Approximate Bayesian Inference for Large Spatial Datasets Using Predictive Process Models Approximate Bayesian Inference for Large Spatial Datasets Using Predictive Process Models , 2022 .

[196]  Haavard Rue,et al.  Using INLA to fit a complex point process model with temporally varying effects – a case study , 2012 .

[197]  Ying Sun,et al.  Geostatistics for Large Datasets , 2012 .

[198]  Finn Lindgren,et al.  Bayesian computing with INLA: New features , 2012, Comput. Stat. Data Anal..