A Comparison of Bayes Factor Approximation Methods Including Two New Methods

Bayes factors (BFs) play an important role in comparing the fit of statistical models. However, computational limitations or lack of an appropriate prior sometimes prevent researchers from using exact BFs. Instead, it is approximated, often using the Bayesian Information Criterion (BIC) or a variant of BIC. The authors provide a comparison of several BF approximations, including two new approximations, the Scaled Unit Information Prior Bayesian Information Criterion (SPBIC) and Information matrix-based Bayesian Information Criterion (IBIC). The SPBIC uses a scaled unit information prior that is more general than the BIC’s unit information prior, and the IBIC utilizes more terms of approximation than the BIC. Through simulation, the authors show that several measures perform well in large samples, that performance declines in smaller samples, and that SPBIC and IBIC provide improvement to existing measures under some conditions, including small sample sizes. The authors illustrate the use of the fit measures with the crime data of Ehrlich and then conclude with recommendations for researchers.

[1]  D. Weakliem.  A Critique of the Bayesian Information Criterion for Model Selection , 1999 .

[2]  B. Carlin,et al.  Bayesian Model Choice Via Markov Chain Monte Carlo Methods , 1995 .

[3]  R. Kass,et al.  Approximate Bayes Factors and Orthogonal Parameters, with Application to Testing Equality of Two Binomial Proportions , 1992 .

[4]  Bradley P. Carlin,et al.  BAYES AND EMPIRICAL BAYES METHODS FOR DATA ANALYSIS , 1996, Stat. Comput..

[5]  Adrian E. Raftery,et al.  Bayesian Model Selection in Structural Equation Models , 1992 .

[6]  P. Green,et al.  On Bayesian Analysis of Mixtures with an Unknown Number of Components (with discussion) , 1997 .

[7]  I. Ehrlich.  Participation in Illegitimate Activities: A Theoretical and Empirical Investigation , 1973, Journal of Political Economy.

[8]  Hua Lee,et al.  Maximum Entropy and Bayesian Methods. , 1996 .

[9]  L. Tierney,et al.  Accurate Approximations for Posterior Moments and Marginal Densities , 1986 .

[10]  A. Rukhin.  Bayes and Empirical Bayes Methods for Data Analysis , 1997 .

[11]  A. Raftery.  Bayesian Model Selection in Social Research , 1995 .

[12]  David L. Weakliem,et al.  Introduction to the Special Issue on Model Selection , 2004 .

[13]  John K Kruschke,et al.  Bayesian data analysis. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[14]  J. Kuha.  AIC and BIC , 2004 .

[15]  L. M. M.-T..  Theory of Probability , 1929, Nature.

[16]  Ward Edwards,et al.  Bayesian statistical inference for psychological research. , 1963 .

[17]  D. Spiegelhalter,et al.  Bayes Factors for Linear and Log‐Linear Models with Vague Prior Information , 1982 .

[18]  David R. Anderson,et al.  Multimodel Inference , 2004 .

[19]  D. Haughton.  On the Choice of a Model to Fit Data from an Exponential Family , 1988 .

[20]  A. Raftery,et al.  Bayesian Information Criterion for Censored Survival Models , 2000, Biometrics.

[21]  Dominique Haughton,et al.  Information and other criteria in structural equation model selection , 1997 .

[22]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[23]  P. Green,et al.  Corrigendum: On Bayesian analysis of mixtures with an unknown number of components , 1997 .

[24]  A. Blumstein,et al.  Deterrence and incapacitation : estimating the effects of criminal sanctions on crime rates , 1980 .

[25]  D. Hunter,et al.  Variable Selection using MM Algorithms. , 2005, Annals of statistics.

[26]  A. Raftery.  Approximate Bayes factors and accounting for model uncertainty in generalised linear models , 1996 .

[27]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[28]  Christopher Winship.  Editor's Introduction to the Special Issue on the Bayesian Information Criterion , 1999 .

[29]  B. Efron,et al.  Assessing the accuracy of the maximum likelihood estimator: Observed versus expected Fisher information , 1978 .

[30]  G. Fitzmaurice.  Model selection with overdispersed data , 1997 .

[31]  James O. Berger,et al.  An overview of robust Bayesian analysis , 1994 .

[32]  Rangasami L. Kashyap,et al.  Optimal Choice of AR and MA Parts in Autoregressive Moving Average Models , 1982, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  John Skilling,et al.  Maximum Entropy and Bayesian Methods , 1989 .