Conditional simulation from highly structured Gaussian systems, with application to blocking-MCMC for the Bayesian analysis of very large linear models

This paper examines strategies for simulating exactly from large Gaussian linear models conditional on some Gaussian observations. Local computation strategies based on the conditional independence structure of the model are developed in order to reduce costs associated with storage and computation. Application of these algorithms to simulation from nested hierarchical linear models is considered, and the construction of efficient MCMC schemes for Bayesian inference in high-dimensional linear models is outlined.

[1]  Gene H. Golub,et al.  Matrix computations , 1983 .

[2]  W. Wong,et al.  The calculation of posterior distributions by data augmentation , 1987 .

[3]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .

[4]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .

[5]  M. West,et al.  Bayesian forecasting and dynamic models , 1989 .

[6]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[7]  J. N. R. Jeffers,et al.  Graphical Models in Applied Multivariate Statistics. , 1990 .

[8]  S. Lauritzen.  Propagation of Probabilities, Means, and Variances in Mixed Graphical Association Models , 1992 .

[9]  A. P. Dawid,et al.  Applications of a general propagation algorithm for probabilistic expert systems , 1992 .

[10]  S. Normand,et al.  Parameter Updating in a Bayes Network , 1992 .

[11]  Jun S. Liu,et al.  Covariance structure of the Gibbs sampler with applications to the comparisons of estimators and augmentation schemes , 1994 .

[12]  Steffen L. Lauritzen,et al.  Hybrid Propagation in Junction Trees , 1994, IPMU.

[13]  A. Gelfand,et al.  Efficient parametrisations for normal linear mixed models , 1995 .

[14]  Uffe Kjærulff,et al.  Blocking Gibbs sampling in very large probabilistic expert systems , 1995, Int. J. Hum. Comput. Stud..

[15]  N. Shephard,et al.  The simulation smoother for time series models , 1995 .

[16]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[17]  S. Chib.  Calculating posterior distributions and modal estimates in Markov mixture models , 1996 .

[18]  Steffen L. Lauritzen,et al.  Graphical models in R , 1996 .

[19]  L. García-Cortés,et al.  On a multivariate implementation of the Gibbs sampler , 1996, Genetics Selection Evolution.

[20]  G. Roberts,et al.  Updating Schemes, Correlation Structure, Blocking and Parameterization for the Gibbs Sampler , 1997 .

[21]  Lain L. MacDonald,et al.  Hidden Markov and Other Models for Discrete- valued Time Series , 1997 .

[22]  Michael A. West,et al.  Bayesian Forecasting and Dynamic Models (2nd edn) , 1997, J. Oper. Res. Soc..

[23]  S. P. Pederson,et al.  Hidden Markov and Other Models for Discrete-Valued Time Series , 1998 .

[24]  Darren J. Wilkinson,et al.  An Object-Oriented Approach to Local Computation in Bayes Linear Belief Networks , 1998, COMPSTAT.

[25]  C. Geyer,et al.  Geometric Ergodicity of Gibbs and Block Gibbs Samplers for a Hierarchical Random Effects Model , 1998 .

[26]  Bradley P. Carlin,et al.  On MCMC sampling in hierarchical longitudinal models , 1999, Stat. Comput..

[27]  A. Rue.  A Fast and Exact Simulation Algorithm for General Gaussian Markov Random Fields , 1999 .

[28]  C S Jensen,et al.  Blocking Gibbs sampling for linkage analysis in large pedigrees with many loops. , 1999, American journal of human genetics.

[29]  Michael I. Jordan,et al.  Probabilistic Networks and Expert Systems , 1999 .

[30]  Darren J. Wilkinson,et al.  Detecting homogeneous segments in DNA sequences by using hidden Markov models , 2000 .

[31]  H. Rue.  Fast Sampling of Gaussian Markov Random Fields with Applications , 2000 .

[32]  Darren J. Wilkinson,et al.  Bayes linear analysis for graphical models: The geometric approach to local computation and interpretive graphics , 2000, Stat. Comput..

[33]  D. Wilkinson,et al.  Multivariate DLMs for forecasting financial time series, with application to the management of portfolios , 2000 .

[34]  Roldan Pozo,et al.  NIST sparse BLAS user's guide , 2001 .

[35]  P. Barone,et al.  General over-relaxation Markov chain Monte Carlo algorithms for Gaussian densities , 2001 .