Explaining the Gibbs Sampler

Abstract Computer-intensive algorithms, such as the Gibbs sampler, have become increasingly popular statistical tools, both in applied and theoretical work. The properties of such algorithms, however, may sometimes not be obvious. Here we give a simple explanation of how and why the Gibbs sampler works. We analytically establish its properties in a simple case and provide insight for more complicated cases. There are also a number of examples.

[1]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[2]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[3]  C. J. Stone,et al.  Introduction to Stochastic Processes , 1972 .

[4]  J. Besag.  Spatial Interaction and the Statistical Analysis of Lattice Systems , 1974 .

[5]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[6]  B. Efron,et al.  The Jackknife: The Bootstrap and Other Resampling Plans. , 1983 .

[7]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  L. Devroye.  Non-Uniform Random Variate Generation , 1986 .

[9]  Brian D. Ripley,et al.  Stochastic Simulation , 2005 .

[10]  W. Wong,et al.  The calculation of posterior distributions by data augmentation , 1987 .

[11]  B. Efron.  The jackknife, the bootstrap, and other resampling plans , 1987 .

[12]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[13]  S. E. Hills,et al.  Illustration of Bayesian Inference in Normal Data Models Using Gibbs Sampling , 1990 .

[14]  C. Geyer.  Markov Chain Monte Carlo Maximum Likelihood , 1991 .

[15]  Scott L. Zeger,et al.  Generalized linear models with random e ects: a Gibbs sampling approach , 1991 .

[16]  Robert C. Blattberg,et al.  Shrinkage Estimation of Price and Promotional Elasticities: Seemingly Unrelated Equations , 1991 .

[17]  John Geweke,et al.  Evaluating the accuracy of sampling-based approaches to the calculation of posterior moments , 1991 .

[18]  Adrian F. M. Smith,et al.  Bayesian Analysis of Constrained Parameter and Truncated Data Problems , 1991 .

[19]  Andrew L. Rukhin,et al.  Tools for statistical inference , 1991 .

[20]  L. Wasserman,et al.  Bayesian analysis of outlier problems using the Gibbs sampler , 1991 .

[21]  A. Raftery,et al.  Stopping the Gibbs Sampler,the Use of Morphology,and Other Issues in Spatial Statistics (Bayesian image restoration,with two applications in spatial statistics) -- (Discussion) , 1991 .

[22]  B. Carlin,et al.  On the Convergence of Successive Substitution Sampling , 1992 .

[23]  A. Gelfand,et al.  Hierarchical Bayes Models for the Progression of HIV Infection Using Longitudinal CD4 T-Cell Numbers , 1992 .

[24]  Alan E. Gelfand,et al.  Bayesian statistics without tears: A sampling-resampling perspective , 1992 .

[25]  Bradley P. Carlin,et al.  Hierarchical Bayes Models for the Progression of HIV Infection Using Longitudinal CD4+ Counts. , 1992 .

[26]  S. Chib,et al.  Bayesian analysis of binary and polychotomous response data , 1993 .

[27]  E. George,et al.  Journal of the American Statistical Association is currently published by American Statistical Association. , 2007 .

[28]  Draft: Sampling Based Methods for the Estimation of DNA Sequence Accuracy , 1993 .

[29]  Nicholas G. Polson,et al.  On the Geometric Convergence of the Gibbs Sampler , 1994 .

[30]  C. Robert,et al.  Estimation of Finite Mixture Distributions Through Bayesian Sampling , 1994 .

[31]  L. Tierney.  Markov Chains for Exploring Posterior Distributions , 1994 .