Bayesian leave-one-out cross-validation for non-factorizable normal models

Cross-validation can be used to measure a model's predictive accuracy for the purpose of model comparison, averaging, or selection. Standard leave-one-out cross-validation (LOO-CV) requires the likelihood to be factorizable, but many important models in temporal and spatial statistics do not have this property. We derive how to compute and validate both exact and approximate LOO-CV for Bayesian non-factorizable models with a multivariate normal likelihood.

[1]  S. Sundararajan,et al.  Predictive Approaches for Choosing Hyperparameters in Gaussian Processes , 1999, Neural Computation.

[2]  Roger Bivand,et al.  Comparing Implementations of Estimation Methods for Spatial Econometrics , 2015 .

[3]  Paul-Christian Bürkner,et al.  Advanced Bayesian Multilevel Modeling with the R Package brms , 2017, R J..

[4]  Aki Vehtari,et al.  A survey of Bayesian predictive methods for model assessment, selection and comparison , 2012 .

[5]  Tomohiro Ando,et al.  Predictive likelihood for Bayesian model selection and averaging , 2010 .

[6]  A. Gelman,et al.  Pareto Smoothed Importance Sampling , 2015, 1507.02646.

[7]  Adrian E. Raftery,et al.  Bayesian model averaging: a tutorial (with comments by M. Clyde, David Draper and E. I. George, and a rejoinder by the authors , 1999 .

[8]  Paul-Christian Bürkner,et al.  brms: An R Package for Bayesian Multilevel Models Using Stan , 2017 .

[9]  Jiqiang Guo,et al.  Stan: A Probabilistic Programming Language. , 2017, Journal of statistical software.

[10]  S. Geisser,et al.  A Predictive Approach to Model Selection , 1979 .

[11]  Adrian E. Raftery,et al.  Bayesian Model Averaging: A Tutorial , 2016 .

[12]  L. Anselin.  Spatial Econometrics: Methods and Models , 1988 .

[13]  Aki Vehtari,et al.  Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC , 2015, Statistics and Computing.

[14]  Hadley Wickham,et al.  ggplot2 - Elegant Graphics for Data Analysis (2nd Edition) , 2017 .

[15]  Hong Chang,et al.  Model Determination Using Predictive Distributions with Implementation via Sampling-Based Methods , 1992 .

[16]  Ole Winther,et al.  Bayesian Leave-One-Out Cross-Validation Approximations for Gaussian Latent Variable Models , 2014, J. Mach. Learn. Res..

[17]  Jouko Lampinen,et al.  Bayesian Model Assessment and Comparison Using Cross-Validation Predictive Densities , 2002, Neural Computation.