Gibbs sampling for a Bayesian hierarchical general linear model

We consider a Bayesian hierarchical version of the normal theory general linear model which is practically relevant in the sense that it is general enough to have many applications and it is not straightforward to sample directly from the corresponding posterior distribution. Thus we study a block Gibbs sampler that has the posterior as its invariant distribution. In particular, we establish that the Gibbs sampler converges at a geometric rate. This allows us to establish conditions for a central limit theorem for the ergodic averages used to estimate features of the posterior. Geometric ergodicity is also a key component for using batch means methods to consistently estimate the variance of the asymptotic normal distribution. Together, our results give practitioners the tools to be as confident in inferences based on the observations from the Gibbs sampler as they would be with inferences based on random samples from the posterior. Our theoretical results are illustrated with an application to data on the cost of health plans issued by health maintenance organizations.

[1]  David R. Cox,et al.  Some algebra and geometry for hierarchical models, applied to diagnostics , 1998 .

[2]  Galin L. Jones,et al.  Fixed-Width Output Analysis for Markov Chain , 2005 .

[3]  C. Geyer,et al.  Discussion: Markov Chains for Exploring Posterior Distributions , 1994 .

[4]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[5]  Bin Yu,et al.  Regeneration in Markov chain samplers , 1995 .

[6]  Galin L. Jones,et al.  Sufficient burn-in for Gibbs samplers for a hierarchical random effects model , 2004, math/0406454.

[7]  R. Tweedie,et al.  Rates of convergence of the Hastings and Metropolis algorithms , 1996 .

[8]  S. R. Searle,et al.  On Deriving the Inverse of a Sum of Matrices , 1981 .

[9]  J. Rosenthal,et al.  On convergence rates of Gibbs samplers for uniform distributions , 1998 .

[10]  J. Rosenthal.  Minorization Conditions and Convergence Rates for Markov Chain Monte Carlo , 1995 .

[11]  Ranjini Natarajan,et al.  Gibbs Sampling with Diffuse Proper Priors: A Valid Approach to Data-Driven Inference? , 1998 .

[12]  S. F. Jarner,et al.  Geometric ergodicity of Metropolis algorithms , 2000 .

[13]  Christian P. Robert,et al.  Using a Markov Chain to Construct a Tractable Approximation of an Intractable Probability Distribution , 2006 .

[14]  Jerry Nedelman,et al.  Book review: “Bayesian Data Analysis,” Second Edition by A. Gelman, J.B. Carlin, H.S. Stern, and D.B. Rubin Chapman & Hall/CRC, 2004 , 2005, Comput. Stat..

[15]  William Watkins.  CONVEX MATRIX FUNCTIONS , 1974 .

[16]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[17]  J. Hobert,et al.  Block Gibbs Sampling for Bayesian Random Effects Models With Improper Priors: Convergence and Regeneration , 2009 .

[18]  S. Meyn,et al.  Computable Bounds for Geometric Convergence Rates of Markov Chains , 1994 .

[19]  Galin L. Jones,et al.  Honest Exploration of Intractable Probability Distributions via Markov Chain Monte Carlo , 2001 .

[20]  Galin L. Jones,et al.  On the applicability of regenerative simulation in Markov chain Monte Carlo , 2002 .

[21]  R. Tweedie,et al.  Geometric convergence and central limit theorems for multidimensional Hastings and Metropolis algorithms , 1996 .

[22]  Gareth O. Roberts,et al.  Markov Chains and De‐initializing Processes , 2001 .

[23]  J. Rosenthal,et al.  Convergence of Slice Sampler Markov Chains , 1999 .

[24]  G. Roberts,et al.  Stability of the Gibbs sampler for Bayesian hierarchical models , 2007, 0710.4234.

[25]  Murali Haran,et al.  Markov chain Monte Carlo: Can we trust the third significant figure? , 2007, math/0703746.

[26]  J. Rosenthal.  RATES OF CONVERGENCE FOR GIBBS SAMPLING FOR VARIANCE COMPONENT MODELS , 1995 .

[27]  A Few Remarks on “Fixed-Width Output Analysis for Markov Chain Monte Carlo” by Jones et al , 2007 .

[28]  G. Roberts,et al.  Adaptive Markov Chain Monte Carlo through Regeneration , 1998 .

[29]  C. Geyer,et al.  Geometric Ergodicity of Gibbs and Block Gibbs Samplers for a Hierarchical Random Effects Model , 1998 .

[30]  P. Glynn,et al.  The Asymptotic Validity of Sequential Stopping Rules for Stochastic Simulations , 1992 .

[31]  R. Tweedie,et al.  Bounds on regeneration times and convergence rates for Markov chains fn1 fn1 Work supported in part , 1999 .

[32]  Galin L. Jones.  On the Markov chain central limit theorem , 2004, math/0409112.

[33]  Galin L. Jones,et al.  Fixed-Width Output Analysis for Markov Chain Monte Carlo , 2006, math/0601446.