Bayesian Multimodel Inference by RJMCMC: A Gibbs Sampling Approach

Bayesian multimodel inference treats a set of candidate models as the sample space of a latent categorical random variable, sampled once; the data at hand are modeled as having been generated according to the sampled model. Model selection and model averaging are based on the posterior probabilities for the model set. Reversible-jump Markov chain Monte Carlo (RJMCMC) extends ordinary MCMC methods to this meta-model. We describe a version of RJMCMC that intuitively represents the process as Gibbs sampling with alternating updates of a categorical variable M (for Model) and a “palette” of parameters , from which any of the model-specific parameters can be calculated. Our representation makes plain how model-specific Monte Carlo outputs (analytical or numerical) can be post-processed to compute model weights or Bayes factors. We illustrate the procedure with several examples.

[1]  William A. Link,et al.  Bayes Factors and Multimodel Inference , 2009 .

[2]  Xiao-Li Meng,et al.  SIMULATING RATIOS OF NORMALIZING CONSTANTS VIA A SIMPLE IDENTITY: A THEORETICAL EXPLORATION , 1996 .

[3]  G. Casella,et al.  Rao-Blackwellisation of sampling schemes , 1996 .

[4]  M. Newton.  Approximate Bayesian-inference With the Weighted Likelihood Bootstrap , 1994 .

[5]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[6]  S. Chib.  Marginal Likelihood from the Gibbs Output , 1995 .

[7]  William A Link,et al.  Model weights and the foundations of multimodel inference. , 2006, Ecology.

[8]  R. O’Hara,et al.  A review of Bayesian variable selection methods: what, how and which , 2009 .

[9]  George A. F. Seber,et al.  A matrix handbook for statisticians , 2007 .

[10]  S. Godsill.  On the Relationship Between Markov chain Monte Carlo Methods for Model Uncertainty , 2001 .

[11]  Peter Congdon,et al.  Bayesian model choice based on Monte Carlo estimates of posterior model probabilities , 2006, Comput. Stat. Data Anal..

[12]  B. Carlin,et al.  Bayesian Model Choice Via Markov Chain Monte Carlo Methods , 1995 .

[13]  Jean-Michel Marin,et al.  On some difficulties with a posterior probability approximation technique , 2008 .

[14]  L. Wasserman,et al.  Computing Bayes Factors by Combining Simulation and Asymptotic Approximations , 1997 .

[15]  A. Mira,et al.  Efficient Bayes factor estimation from the reversible jump output , 2006 .

[16]  Bradley P. Carlin,et al.  Bayesian measures of model complexity and fit , 2002 .