Posterior and Cross-validatory Predictive Checks: A Comparison of MCMC and INLA

Model criticism and comparison of Bayesian hierarchical models is often based on posterior or leave-one-out cross-validatory predictive checks. Cross-validatory checks are usually preferred because posterior predictive checks are difficult to assess and tend to be too conservative. However, techniques for statistical inference in such models often try to avoid full (manual) leave-one-out cross-validation, since it is very time-consuming. In this paper we will compare two approaches for estimating Bayesian hierarchical models: Markov chain Monte Carlo (MCMC) and integrated nested Laplace approximations (INLA). We review how both approaches allow for the computation of leave-one-out cross-validatory checks without re-running the model for each observation in turn. We then empirically compare the two approaches in an extensive case study analysing the spatial distribution of bovine viral diarrhoe (BVD) among cows in Switzerland.

[1]  D J Spiegelhalter,et al.  Approximate cross‐validatory predictive checks in disease mapping models , 2003, Statistics in medicine.

[2]  Leonhard Held,et al.  Gaussian Markov Random Fields: Theory and Applications , 2005 .

[3]  A. Azzalini,et al.  Statistical applications of the multivariate skew normal distribution , 2009, 0911.2093.

[4]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[5]  D. Altman,et al.  STATISTICAL METHODS FOR ASSESSING AGREEMENT BETWEEN TWO METHODS OF CLINICAL MEASUREMENT , 1986, The Lancet.

[6]  A. P. Dawid,et al.  Present position and potential developments: some personal views , 1984 .

[7]  H. Rue,et al.  Approximate Bayesian inference for hierarchical Gaussian Markov random field models , 2007 .

[8]  Claudia Czado,et al.  Predictive Model Assessment for Count Data , 2009, Biometrics.

[9]  N. Hjort,et al.  Post-Processing Posterior Predictive p Values , 2006 .

[10]  D. J. Spiegelhalter,et al.  Identifying outliers in Bayesian hierarchical models: a simulation-based approach , 2007 .

[11]  J. Besag,et al.  Bayesian image restoration, with two applications in spatial statistics , 1991 .

[12]  L. Tierney,et al.  Accurate Approximations for Posterior Moments and Marginal Densities , 1986 .

[13]  P. Armitage,et al.  Mid-P confidence intervals: a brief review , 1995 .

[14]  Christian P. Robert,et al.  Monte Carlo Statistical Methods , 2005, Springer Texts in Statistics.

[15]  M. Newton.  Approximate Bayesian-inference With the Weighted Likelihood Bootstrap , 1994 .

[16]  M. Plummer.  Penalized loss functions for Bayesian model comparison. , 2008, Biostatistics.

[17]  L Bernardinelli,et al.  Bayesian estimates of disease maps: how important are priors? , 1995, Statistics in medicine.

[18]  L. Pettit.  The Conditional Predictive Ordinate for the Normal Distribution , 1990 .

[19]  H. Rue,et al.  On Block Updating in Markov Random Field Models for Disease Mapping , 2002 .

[20]  H S Stern,et al.  Posterior predictive model checks for disease mapping models. , 2000, Statistics in medicine.

[21]  L. Fahrmeir,et al.  PENALIZED STRUCTURED ADDITIVE REGRESSION FOR SPACE-TIME DATA: A BAYESIAN PERSPECTIVE , 2004 .

[22]  L. Fahrmeir,et al.  Bayesian inference for generalized additive mixed models based on Markov random field priors , 2001 .

[23]  C Gössl,et al.  Bayesian Spatiotemporal Inference in Functional Magnetic Resonance Imaging , 2001, Biometrics.

[24]  Walter R. Gilks,et al.  Hypothesis testing and model selection , 1995 .

[25]  H. Rue,et al.  Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations , 2009 .

[26]  Bradley P. Carlin,et al.  Bayesian measures of model complexity and fit , 2002 .

[27]  Leonhard Held,et al.  The analysis of heterogeneous time trends in multivariate age-period-cohort models. , 2010, Biostatistics.

[28]  Xiao-Li Meng,et al.  POSTERIOR PREDICTIVE ASSESSMENT OF MODEL FITNESS VIA REALIZED DISCREPANCIES , 1996 .

[29]  Håvard Rue,et al.  Norges Teknisk-naturvitenskapelige Universitet Implementing Approximate Bayesian Inference Using Integrated Nested Laplace Approximation: a Manual for the Inla Program Implementing Approximate Bayesian Inference Using Integrated Nested Laplace Approximation: a Manual for the Inla Program , 2022 .

[30]  Seymour Geisser,et al.  8. Predictive Inference: An Introduction , 1995 .

[31]  G. Casella,et al.  Rao-Blackwellisation of sampling schemes , 1996 .

[32]  L. Bernardinelli,et al.  Bayesian methods for mapping disease risk , 1996 .

[33]  N. Breslow,et al.  Approximate inference in generalized linear mixed models , 1993 .

[34]  L. Fahrmeir.  Posterior Mode Estimation by Extended Kalman Filtering for Multivariate Dynamic Generalized Linear Models , 1992 .

[35]  J. Nelder,et al.  Generalized Linear Models with Random Effects: Unified Analysis via H-likelihood , 2006 .

[36]  A. Raftery,et al.  Strictly Proper Scoring Rules, Prediction, and Estimation , 2007 .

[37]  Leonhard Knorr-Held,et al.  Non‐Parametric Ecological Regression and Spatial Variation , 2003 .