Approximate inference for disease mapping with sparse Gaussian processes

Gaussian process (GP) models are widely used in disease mapping as they provide a natural framework for modeling spatial correlations. Their challenges, however, lie in computational burden and memory requirements. In disease mapping models, the other difficulty is inference, which is analytically intractable due to the non‐Gaussian observation model. In this paper, we address both these challenges. We show how to efficiently build fully and partially independent conditional (FIC/PIC) sparse approximations for the GP in two‐dimensional surface, and how to conduct approximate inference using expectation propagation (EP) algorithm and Laplace approximation (LA). We also propose to combine FIC with a compactly supported covariance function to construct a computationally efficient additive model that can model long and short length‐scale spatial correlations simultaneously. The benefit of these approximations is computational. The sparse GPs speed up the computations and reduce the memory requirements. The posterior inference via EP and Laplace approximation is much faster and is practically as accurate as via Markov chain Monte Carlo. Copyright © 2010 John Wiley & Sons, Ltd.

[1]  J. Hammersley MONTE CARLO METHODS FOR SOLVING MULTIVARIABLE PROBLEMS , 1960 .

[2]  William G. Poole,et al.  Incomplete Nested Dissection for Solving n by n Grid Problems , 1978 .

[3]  พงศ์ศักดิ์ บินสมประสงค์,et al.  FORMATION OF A SPARSE BUS IMPEDANCE MATRIX AND ITS APPLICATION TO SHORT CIRCUIT STUDY , 1980 .

[4]  H. Niessner,et al.  On computing the inverse of a sparse matrix , 1983 .

[5]  L. Tierney,et al.  Accurate Approximations for Posterior Moments and Marginal Densities , 1986 .

[6]  J. Geweke,et al.  Bayesian Inference in Econometric Models Using Monte Carlo Integration , 1989 .

[7]  Charles J. Geyer,et al.  Practical Markov Chain Monte Carlo , 1992 .

[8]  Harald Niederreiter,et al.  Random number generation and Quasi-Monte Carlo methods , 1992, CBMS-NSF regional conference series in applied mathematics.

[9]  Hong Chang,et al.  Model Determination Using Predictive Distributions with Implementation via Sampling-Based Methods , 1992 .

[10]  Geoffrey E. Hinton,et al.  Bayesian Learning for Neural Networks , 1995 .

[11]  R. Caflisch,et al.  Quasi-Monte Carlo integration , 1995 .

[12]  J. Møller,et al.  Log Gaussian Cox Processes , 1998 .

[13]  D. Harville Matrix Algebra From a Statistician's Perspective , 1998 .

[14]  Andrew Gelman,et al.  General methods for monitoring convergence of iterative simulations , 1998 .

[15]  David Barber,et al.  Bayesian Classification With Gaussian Processes , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Amos Storkey,et al.  Efficient Covariance Matrix Methods for Bayesian Gaussian Processes and Hopfield Neural Networks , 1999 .

[17]  N. Cressie,et al.  A dimension-reduced approach to space-time Kalman filtering , 1999 .

[18]  Ole Winther,et al.  Efficient Approaches to Gaussian Process Classification , 1999, NIPS.

[19]  Hoon Kim,et al.  Monte Carlo Statistical Methods , 2000, Technometrics.

[20]  David J. C. MacKay,et al.  Variational Gaussian process classifiers , 2000, IEEE Trans. Neural Networks Learn. Syst..

[21]  Rafael Lozano,et al.  AGE STANDARDIZATION OF RATES: A NEW WHO STANDARD , 2000 .

[22]  Radford M. Neal Annealed importance sampling , 1998, Stat. Comput..

[23]  Aki Vehtari,et al.  Bayesian model assessment and selection using expected utilities , 2001 .

[24]  Tom Minka,et al.  A family of algorithms for approximate Bayesian inference , 2001 .

[25]  Lehel Csató,et al.  Sparse On-Line Gaussian Processes , 2002, Neural Computation.

[26]  Jouko Lampinen,et al.  Bayesian Model Assessment and Comparison Using Cross-Validation Predictive Densities , 2002, Neural Computation.

[27]  Bradley P. Carlin,et al.  Bayesian measures of model complexity and fit , 2002 .

[28]  T. Gneiting Compactly Supported Correlation Functions , 2002 .

[29]  Neil D. Lawrence,et al.  Fast Forward Selection to Speed Up Sparse Gaussian Process Regression , 2003, AISTATS.

[30]  S. Scobie Spatial epidemiology: methods and applications , 2003 .

[31]  Timothy A. Davis,et al.  Algorithm 837: AMD, an approximate minimum degree ordering algorithm , 2004, TOMS.

[32]  Haotian Hang,et al.  Inconsistent Estimation and Asymptotically Equal Interpolations in Model-Based Geostatistics , 2004 .

[33]  A. Gelman Prior distributions for variance parameters in hierarchical models (comment on article by Browne and Draper) , 2004 .

[34]  Tom Heskes,et al.  Gaussian Quadrature Based Expectation Propagation , 2005, AISTATS.

[35]  Timothy A. Davis,et al.  Row Modifications of a Sparse Cholesky Factorization , 2005, SIAM J. Matrix Anal. Appl..

[36]  Dan Cornford,et al.  Sequential, Bayesian Geostatistics: A Principled Method for Large Data Sets , 2005 .

[37]  Carl E. Rasmussen,et al.  Assessing Approximate Inference for Binary Gaussian Process Classification , 2005, J. Mach. Learn. Res..

[38]  Carl E. Rasmussen,et al.  A Unifying View of Sparse Approximate Gaussian Process Regression , 2005, J. Mach. Learn. Res..

[39]  M. Seeger Expectation Propagation for Exponential Families , 2005 .

[40]  Sylvia Richardson,et al.  A comparison of Bayesian spatial models for disease mapping , 2005, Statistical methods in medical research.

[41]  Zoubin Ghahramani,et al.  Sparse Gaussian Processes using Pseudo-inputs , 2005, NIPS.

[42]  Susan M. Sanchez,et al.  Very large fractional factorial and central composite designs , 2005, TOMC.

[43]  Leonhard Held,et al.  Gaussian Markov Random Fields: Theory and Applications , 2005 .

[44]  P. Gustafson,et al.  Conservative prior distributions for variance parameters in hierarchical models , 2006 .

[45]  Timothy A. Davis,et al.  Direct methods for sparse linear systems , 2006, Fundamentals of algorithms.

[46]  Neil D. Lawrence,et al.  Learning for Larger Datasets with the Gaussian Process Latent Variable Model , 2007, AISTATS.

[47]  Aki Vehtari,et al.  Sparse Log Gaussian Processes via MCMC for Spatial Epidemiology , 2007, Gaussian Processes in Practice.

[48]  Sean B. Holden,et al.  The Generalized FITC Approximation , 2007, NIPS.

[49]  H. Rue,et al.  Approximate Bayesian inference for hierarchical Gaussian Markov random field models , 2007 .

[50]  P. Diggle,et al.  Model‐based geostatistics , 2007 .

[51]  Christopher J. Paciorek,et al.  Computational techniques for spatial logistic regression with large data sets , 2007, Comput. Stat. Data Anal..

[52]  S. Martino Approximate Bayesian Inference for Latent Gaussian Models , 2007 .

[53]  Zoubin Ghahramani,et al.  Local and global sparse Gaussian process approximations , 2007, AISTATS.

[54]  Edward Lloyd Snelson,et al.  Flexible and efficient Gaussian process models for machine learning , 2007 .

[55]  M. Opper Sparse Online Gaussian Processes , 2008 .

[56]  A. Gelfand,et al.  Gaussian predictive process models for large spatial data sets , 2008, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[57]  L. Shampine Vectorized adaptive quadrature in MATLAB , 2008 .

[58]  Aki Vehtari,et al.  Modelling local and global phenomena with sparse Gaussian processes , 2008, UAI.

[59]  C. Rasmussen,et al.  Approximations for Binary Gaussian Process Classification , 2008 .

[60]  Ole Winther,et al.  Perturbation Corrections in Approximate Inference: Mixture Modelling Applications , 2009, J. Mach. Learn. Res..

[61]  H. Rue,et al.  Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations , 2009 .