Marginal Likelihood From the Metropolis–Hastings Output

This article provides a framework for estimating the marginal likelihood for the purpose of Bayesian model comparisons. The approach extends and completes the method presented in Chib (1995) by overcoming the problems associated with the presence of intractable full conditional densities. The proposed method is developed in the context of MCMC chains produced by the Metropolis–Hastings algorithm, whose building blocks are used both for sampling and marginal likelihood estimation, thus economizing on prerun tuning effort and programming. Experiments involving the logit model for binary data, hierarchical random effects model for clustered Gaussian data, Poisson regression model for clustered count data, and the multivariate probit model for correlated binary data, are used to illustrate the performance and implementation of the method. These examples demonstrate that the method is practical and widely applicable.

[1]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[2]  W. Newey,et al.  A Simple, Positive Semi-Definite, Heteroskedasticity and Autocorrelationconsistent Covariance Matrix , 1986 .

[3]  T. Mroz,et al.  The Sensitivity of an Empirical Model of Married Women's Hours of Work to Economic and Statistical Assumptions , 1987 .

[4]  W. Wong,et al.  The calculation of posterior distributions by data augmentation , 1987 .

[5]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[6]  M. Tanner,et al.  Facilitating the Gibbs Sampler: The Gibbs Stopper and the Griddy-Gibbs Sampler , 1992 .

[7]  S. Chib,et al.  Bayesian analysis of binary and polychotomous response data , 1993 .

[8]  A. Gelfand,et al.  Bayesian Model Choice: Asymptotics and Exact Calculations , 1994 .

[9]  L. Tierney.  Markov Chains for Exploring Posterior Distributions , 1994 .

[10]  B. Carlin,et al.  Bayesian Model Choice Via Markov Chain Monte Carlo Methods , 1995 .

[11]  L. Wasserman,et al.  Computing Bayes Factors Using a Generalization of the Savage-Dickey Density Ratio , 1995 .

[12]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[13]  S. Chib,et al.  Understanding the Metropolis-Hastings Algorithm , 1995 .

[14]  S. Chib.  Marginal Likelihood from the Gibbs Output , 1995 .

[15]  Xiao-Li Meng,et al.  SIMULATING RATIOS OF NORMALIZING CONSTANTS VIA A SIMPLE IDENTITY: A THEORETICAL EXPLORATION , 1996 .

[16]  S. Chib,et al.  Posterior Simulation and Bayes Factors in Panel Count Data Models , 1998 .

[17]  L. Wasserman,et al.  Computing Bayes Factors by Combining Simulation and Asymptotic Approximations , 1997 .

[18]  Ming-Hui Chen,et al.  On Monte Carlo methods for estimating ratios of normalizing constants , 1997 .

[19]  B. Everitt,et al.  Analysis of longitudinal data , 1998, British Journal of Psychiatry.

[20]  S. Chib,et al.  Analysis of multivariate probit models , 1998 .

[21]  John M. Olin.  On MCMC sampling in hierarchical longitudinal models , 1999 .

[22]  Cong Han,et al.  MCMC Methods for Computing Bayes Factors: A Comparative Review , 2000 .

[23]  A. Shapiro.  Monte Carlo Sampling Methods , 2003 .