Analysis of the Gibbs Sampler for a Model Related to James-stein Estimators

We analyse a hierarchical Bayes model which is related to the usual empirical Bayes formulation of James-Stein estimators. We consider running a Gibbs sampler on this model. Using previous results about convergence rates of Markov chains, we provide rigorous, numerical, reasonable bounds on the running time of the Gibbs sampler, for a suitable range of prior distributions. We apply these results to baseball data from Efron and Morris (1975). For a different range of prior distributions, we prove that the Gibbs sampler will fail to converge, and use this information to prove that in this case the associated posterior distribution is non-normalizable.

[1]  G. Box,et al.  Scientific Inference, Data Analysis and Robustness. , 1985 .

[2]  C. Hwang,et al.  Convergence rates of the Gibbs sampler, the Metropolis algorithm and other single-site updating dynamics , 1993 .

[3]  Nicholas G. Polson,et al.  Sampling from log-concave distributions , 1994 .

[4]  W. Wong,et al.  The calculation of posterior distributions by data augmentation , 1987 .

[5]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[6]  Mark Jerrum,et al.  Approximating the Permanent , 1989, SIAM J. Comput..

[7]  Richard L. Tweedie,et al.  Geometric Convergence Rates for Stochastically Ordered Markov Chains , 1996, Math. Oper. Res..

[8]  Jun S. Liu,et al.  Covariance Structure and Convergence Rate of the Gibbs Sampler with Various Scans , 1995 .

[9]  Jeffrey S. Rosenthal,et al.  RATES OF CONVERGENCE FOR EVERYWHERE-POSITIVE MARKOV , 1995 .

[10]  B. Carlin,et al.  On the Convergence of Successive Substitution Sampling , 1992 .

[11]  Joong-Kweon Sohn,et al.  Convergence Diagnostics for the Gibbs Sampler , 1996 .

[12]  U. Grenander,et al.  Comparing sweep strategies for stochastic relaxation , 1991 .

[13]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[14]  B. Efron,et al.  Data Analysis Using Stein's Estimator and its Generalizations , 1975 .

[15]  Patrick Billingsley,et al.  Probability and Measure. , 1986 .

[16]  R. Tweedie,et al.  Rates of convergence of the Hastings and Metropolis algorithms , 1996 .

[17]  J. Rosenthal.  Minorization Conditions and Convergence Rates for Markov Chain Monte Carlo , 1995 .

[18]  J. Rosenthal,et al.  Rates of convergence for everywhere-positive Markov chains , 1995 .

[19]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[20]  Jun S. Liu,et al.  Covariance structure of the Gibbs sampler with applications to the comparisons of estimators and augmentation schemes , 1994 .

[21]  C. Morris.  Parametric Empirical Bayes Confidence Intervals , 1983 .

[22]  J. Rosenthal.  RATES OF CONVERGENCE FOR GIBBS SAMPLING FOR VARIANCE COMPONENT MODELS , 1995 .

[23]  L. Tierney.  Markov Chains for Exploring Posterior Distributions , 1994 .

[24]  J. Rosenthal.  Rates of Convergence for Data Augmentation on Finite Sample Spaces , 1993 .

[25]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  S. E. Hills,et al.  Illustration of Bayesian Inference in Normal Data Models Using Gibbs Sampling , 1990 .

[27]  C. Stein,et al.  Estimation with Quadratic Loss , 1992 .

[28]  B. Efron,et al.  Stein's Estimation Rule and Its Competitors- An Empirical Bayes Approach , 1973 .

[29]  Bin Yu,et al.  Regeneration in Markov chain samplers , 1995 .

[30]  A. O'Hagan,et al.  The Calculation of Posterior Distributions by Data Augmentation: Comment , 1987 .

[31]  S. Meyn,et al.  Computable Bounds for Geometric Convergence Rates of Markov Chains , 1994 .