ESTIMATION OF LARGE FAMILIES OF BAYES FACTORS FROM MARKOV CHAIN OUTPUT

We consider situations in Bayesian analysis where the prior is indexed by a hyperparameter taking on a continuum of values. We distinguish some arbitrary value of the hyperparameter, and consider the problem of estimating the Bayes factor for the model indexed by the hyperparameter vs. the model specified by the distinguished point, as the hyperparameter varies. We assume that we have Markov chain output from the posterior for a finite number of the priors, and develop a method for efficiently computing estimates of the entire family of Bayes factors. As an application of the ideas, we consider some commonly used hierarchical Bayesian models and show that the parametric assumptions in these models can be recast as assumptions regarding the prior. Therefore, our method can be used as a model selection criterion in a Bayesian framework. We illustrate our methodology through a detailed example involving Bayesian model selection.

[1]  Hani Doss,et al.  Aspirin, ibuprofen, and other non-steroidal anti-inflammatory drugs in cancer prevention: a critical review of non-selective COX-2 blockade (review). , 2005, Oncology reports.

[2]  James O. Berger,et al.  Statistical Decision Theory and Bayesian Analysis, Second Edition , 1985 .

[3]  J. Rosenthal,et al.  Geometric Ergodicity and Hybrid Markov Chains , 1997 .

[4]  L. Wasserman,et al.  Computing Bayes Factors by Combining Simulation and Asymptotic Approximations , 1997 .

[5]  C. Geyer,et al.  Discussion: Markov Chains for Exploring Posterior Distributions , 1994 .

[6]  Charles J. Geyer,et al.  Practical Markov Chain Monte Carlo , 1992 .

[7]  Xiao-Li Meng,et al.  Warp Bridge Sampling , 2002 .

[8]  K. Athreya,et al.  ON THE CONVERGENCE OF THE MARKOV CHAIN SIMULATION METHOD , 1996 .

[9]  Zhiqiang Tan,et al.  On a Likelihood Approach for Monte Carlo Integration , 2004 .

[10]  Bayesian Model Selection: Some Thoughts on Future Directions , 2007 .

[11]  R. Cogburn,et al.  The central limit theorem for Markov processes , 1972 .

[12]  C. Antoniak.  Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric Problems , 1974 .

[13]  N. Bingham.  INDEPENDENT AND STATIONARY SEQUENCES OF RANDOM VARIABLES , 1973 .

[14]  P. McCullagh,et al.  A theory of statistical models for Monte Carlo integration , 2003 .

[15]  A. Owen,et al.  Safe and Effective Importance Sampling , 2000 .

[16]  Christian P. Robert,et al.  The Bayesian choice : from decision-theoretic foundations to computational implementation , 2007 .

[17]  Dean Phillips Foster,et al.  Calibration and Empirical Bayes Variable Selection , 1997 .

[18]  C. Robert,et al.  A mixture representation of π with applications in Markov chain Monte Carlo and perfect sampling , 2004, math/0407120.

[19]  W. Michael Conklin,et al.  Monte Carlo Methods in Bayesian Computation , 2001, Technometrics.

[20]  S. Chib.  Marginal Likelihood from the Gibbs Output , 1995 .

[21]  A. Gelman.  Prior distributions for variance parameters in hierarchical models (comment on article by Browne and Draper) , 2004 .

[22]  Zhiqiang Tan,et al.  Monte Carlo Integration With Acceptance-Rejection , 2006 .

[23]  Xiao-Li Meng,et al.  SIMULATING RATIOS OF NORMALIZING CONSTANTS VIA A SIMPLE IDENTITY: A THEORETICAL EXPLORATION , 1996 .

[24]  S. Chib,et al.  Marginal Likelihood From the Metropolis–Hastings Output , 2001 .

[25]  J. Berger.  Statistical Decision Theory and Bayesian Analysis , 1988 .

[26]  Zhiqiang Tan,et al.  Monte Carlo integration with Markov chain , 2008 .

[27]  Bradley P. Carlin,et al.  Markov Chain Monte Carlo Methods for Computing Bayes Factors , 2001 .