Easily Computed Marginal Likelihoods from Posterior Simulation Using the THAMES Estimator

We propose an easily computed estimator of marginal likelihoods from posterior simulation output, via reciprocal importance sampling, combining earlier proposals of DiCiccio et al (1997) and Robert and Wraith (2009). This involves only the unnormalized posterior densities from the sampled parameter values, and does not involve additional simulations beyond the main posterior simulation, or additional complicated calculations. It is unbiased for the reciprocal of the marginal likelihood, consistent, has finite variance, and is asymptotically normal. It involves one user-specified control parameter, and we derive an optimal way of specifying this. We illustrate it with several numerical examples.

[1]  F. Llorente,et al.  Marginal likelihood computation for model selection and hypothesis testing: an extensive review , 2020, SIAM Rev..

[2]  Jeffrey W. Miller.  Asymptotic Normality, Concentration, and Coverage of Generalized Posteriors , 2019, J. Mach. Learn. Res..

[3]  G. Hajargasht,et al.  Accurate Computation of Marginal Data Densities Using Variational Bayes , 2018, 1805.10036.

[4]  Henrik Singmann,et al.  bridgesampling: An R Package for Estimating Normalizing Constants , 2017, Journal of Statistical Software.

[5]  Eric Moulines,et al.  Efficient Bayesian Computation by Proximal Markov Chain Monte Carlo: When Langevin Meets Moreau , 2016, SIAM J. Imaging Sci..

[6]  C. Robert,et al.  Computational methods for Bayesian model choice , 2009, 0907.5123.

[7]  John D. Lafferty,et al.  A correlated topic model of Science , 2007, 0708.3601.

[8]  Tony O’Hagan.  Bayes factors , 2006 .

[9]  J. Skilling.  Nested sampling for general Bayesian computation , 2006 .

[10]  Daniel F. Waggoner,et al.  Methods for Inference in Large Multiple-Equation Markov-Switching Models , 2006 .

[11]  S. Frühwirth-Schnatter.  Estimating Marginal Likelihoods for Mixture and Markov Switching Models Using Bridge Sampling Techniques , 2004 .

[12]  R. Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[13]  Xiaotong Shen.  Asymptotic Normality of Semiparametric and Nonparametric Posterior Distributions , 2002 .

[14]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[15]  S. Ghosal.  Asymptotic Normality of Posterior Distributions for Exponential Families when the Number of Parameters Tends to Infinity , 2000 .

[16]  L. Wasserman,et al.  Computing Bayes Factors by Combining Simulation and Asymptotic Approximations , 1997 .

[17]  S. Chib.  Marginal Likelihood from the Gibbs Output , 1995 .

[18]  A. Gelfand,et al.  Bayesian Model Choice: Asymptotics and Exact Calculations , 1994 .

[19]  David J. C. MacKay,et al.  Bayesian Interpolation , 1992, Neural Computation.

[20]  T. Stamey,et al.  Prostate specific antigen in the diagnosis and treatment of adenocarcinoma of the prostate. II. Radical prostatectomy treated patients. , 1989, The Journal of urology.

[21]  Trevor J. Sweeting,et al.  On a Converse to Scheffe's Theorem , 1986 .

[22]  J. Ferron,et al.  Multilevel Analysis , 2020, Encyclopedia of Behavioral Medicine.

[23]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[24]  Raghu Kacker,et al.  Digital Library of Mathematical Functions , 2003 .

[25]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[26]  Roel Bosker,et al.  Multilevel analysis : an introduction to basic and advanced multilevel modeling , 1999 .

[27]  Xiao-Li Meng,et al.  SIMULATING RATIOS OF NORMALIZING CONSTANTS VIA A SIMPLE IDENTITY: A THEORETICAL EXPLORATION , 1996 .

[28]  M. Newton.  Approximate Bayesian-inference With the Weighted Likelihood Bootstrap , 1994 .

[29]  I. Johnstone,et al.  On Asymptotic Posterior Normality for Stochastic Processes , 1979 .