Bayes factors

In a 1935 paper, and in his book Theory of Probability, Je reys developed a methodology for quantifying the evidence in favor of a scienti c theory. The centerpiece was a number, now called the Bayes factor, which is the posterior odds of the null hypothesis when the prior probability on the null is one-half. Although there has been much discussion of Bayesian hypothesis testing in the context of criticism of P -values, less attention has been given to the Bayes factor as a practical tool of applied statistics. In this paper we review and discuss the uses of Bayes factors in the context of ve scienti c applications in genetics, sports, ecology, sociology and psychology. The points we emphasize are: | from Je reys' Bayesian point of view, the purpose of hypothesis testing is to evaluate the evidence in favor of a scienti c theory; | Bayes factors o er a way of evaluating evidence in favor of a null hypothesis; | Bayes factors provide a way of incorporating external information into the evaluation of evidence about a hypothesis; | Bayes factors are very general, and do not require alternative models to be nested; | several techniques are available for computing Bayes factors, including asymptotic approximations which are easy to compute using the output from standard packages that maximize likelihoods; | in \non-standard" statistical models that do not satisfy common regularity conditions, it can be technically simpler to calculate Bayes factors than to derive non-Bayesian signi cance tests; | the Schwarz criterion (or BIC) gives a crude approximation to the logarithm of the Bayes factor, which is easy to use and does not require evaluation of prior distributions; | when one is interested in estimation or prediction, Bayes factors may be converted to weights to be attached to various models so that a composite estimate or prediction may be obtained that takes account of structural or model uncertainty; | algorithms have been proposed that allow model uncertainty to be taken into account when the class of models initially considered is very large; | Bayes factors are useful for guiding an evolutionary model-building process; | and, nally, it is important, and feasible, to assess the sensitivity of conclusions to the prior distributions used.

[1]  Comparing probabilistic methods for outlier detection , 1992 .

[2]  A. Raftery,et al.  Bayesian analysis of a Poisson process with a change-point , 1986 .

[3]  A. Raftery,et al.  Model-based Gaussian and non-Gaussian clustering , 1993 .

[4]  J. Berger,et al.  Testing Precise Hypotheses , 1987 .

[5]  A. Raftery,et al.  Bayes Factors for Non‐Homogeneous Poisson Processes with Vague Prior Information , 1986 .

[6]  Adrian E. Raftery,et al.  Bayesian Model Selection in Structural Equation Models , 1992 .

[7]  C. N. Morris,et al.  The calculation of posterior distributions by data augmentation , 1987 .

[8]  G. C. Tiao,et al.  A Further Look at Robustness via Bayes's Theorem , 1962 .

[9]  Brent R. Moulton.  A Bayesian approach to regression selection and estimation, with application to a price index for radio services , 1991 .

[10]  M. P. Dumont.  Comment … , 1970 .

[11]  H. Jeffreys.  Some Tests of Significance, Treated by the Theory of Probability , 1935, Mathematical Proceedings of the Cambridge Philosophical Society.

[12]  D. Lindley.  A STATISTICAL PARADOX , 1957 .

[13]  R. Shibata.  An optimal selection of regression variables , 1981 .

[14]  J. Dickey.  The Weighted Likelihood Ratio, Linear Hypotheses on Normal Location Parameters , 1971 .

[15]  J. Hartigan.  Locally uniform prior distributions , 1996 .

[16]  A. Atkinson.  Subset Selection in Regression , 1992 .

[17]  E. Slate.  Parameterizations for Natural Exponential Families with Quadratic Variance Functions , 1994 .

[18]  D. Lindley.  The Use of Prior Probability Distributions in Statistical Inference and Decisions , 1961 .

[19]  P. Garthwaite.  Preposterior expected loss as a scoring rule for prior distributions , 1992 .

[20]  Ming-Hui Chen.  Importance-Weighted Marginal Bayesian Posterior Density Estimation , 1994 .

[21]  J. Berger,et al.  The Intrinsic Bayes Factor for Model Selection and Prediction , 1996 .

[22]  Adrian E. Raftery,et al.  Accounting for Model Uncertainty in Survival Analysis Improves Predictive Performance , 1995 .

[23]  D. Haughton.  On the Choice of a Model to Fit Data from an Exponential Family , 1988 .

[24]  Arnold Zellner,et al.  Bayesian analysis in econometrics and statistics , 1997 .

[25]  D. Freedman.  A Note on Screening Regression Equations , 1983 .

[26]  R. Hauser,et al.  COMPARATIVE SOCIAL MOBILITY REVISITED: MODELS OF CONVERGENCE AND DIVERGENCE IN 16 COUNTRIES* , 1984 .

[27]  J. York,et al.  Bayesian Graphical Models for Discrete Data , 1995 .

[28]  J. Berger,et al.  Testing a Point Null Hypothesis: The Irreconcilability of P Values and Evidence , 1987 .

[29]  V. Greaney.  Equality of opportunity in Irish schools , 1984 .

[30]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[31]  A. Raftery.  Analysis of a Simple Debugging Model. , 1988 .

[32]  James O. Berger,et al.  The Relevance of Stopping Rules in Statistical Inference , 1988 .

[33]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[34]  Nicholas G. Polson,et al.  Bayes factors for discrete observations from diffusion processes , 1994 .

[35]  R. Rust,et al.  Incorporating prior theory in covariance structure analysis: A bayesian approach , 1989 .

[36]  P. Groenewald,et al.  BAYESIAN TESTS FOR SOME PRECISE HYPOTHESES ON MULTI‐NORMAL MEANS , 1989 .

[37]  D. Findley.  Counterexamples to parsimony and BIC , 1991 .

[38]  S. Fienberg,et al.  Identification and estimation of age-period-cohort models in the analysis of discrete archival data , 1979 .

[39]  R. Kass.  Bayes Factors in Practice , 1993 .

[40]  J. Geweke,et al.  Bayesian Inference in Econometric Models Using Monte Carlo Integration , 1989 .

[41]  I. Good.  A Bayesian Significance Test for Multinomial Distributions , 1967 .

[42]  R. Shibata.  Asymptotically Efficient Selection of the Order of the Model for Estimating Parameters of a Linear Process , 1980 .

[43]  Jim Albert,et al.  A Bayesian test for a two-way contingency table using a independence priors , 1990 .

[44]  I. Good.  Good Thinking: The Foundations of Probability and Its Applications , 1983 .

[45]  A. F. Smith,et al.  Bayesian Methods in Practice: Experiences in the Pharmaceutical Industry , 1986 .

[46]  R. Shibata.  Selection of the order of an autoregressive model by Akaike's information criterion , 1976 .

[47]  Leland Stewart,et al.  Hierarchical Bayesian Analysis using Monte Carlo Integration: Computing Posterior Distributions when , 1987 .

[48]  James O. Berger,et al.  Lower bounds on Bayes factors for multinomial distributions, with application to chi-squared tests of fit , 1990 .

[49]  D. Spiegelhalter,et al.  Bayes Factors for Linear and Log‐Linear Models with Vague Prior Information , 1982 .

[50]  A. Raftery.  Inference and Prediction for a General Order Statistic Model with Unknown Population Size. , 1987 .

[51]  T. J. Mitchell,et al.  Bayesian variable selection in regression , 1987 .

[52]  K. Worsley.  Confidence regions and tests for a change-point in a sequence of exponential family random variables , 1986 .

[53]  A. Atkinson.  Posterior probabilities for choosing a regression model , 1978 .

[54]  A. Raftery.  Choosing Models for Cross-Classifications , 1986 .

[55]  L. Tierney,et al.  Fully Exponential Laplace Approximations to Expectations and Variances of Nonpositive Functions , 1989 .

[56]  David Draper,et al.  Assessment and Propagation of Model Uncertainty , 2011 .

[57]  C. Robert,et al.  Noninformative Bayesian testing and neutral Bayes factors , 1996 .

[58]  I. Guttman,et al.  Comparing probabilistic methods for outlier detection in linear models , 1993 .

[59]  I. Good.  The Bayes/Non-Bayes Compromise: A Brief Review , 1992 .

[60]  A. Raftery,et al.  Does Irish education approach the meritocratic ideal - a logistic analysis , 1985 .

[61]  Nicholas G. Polson,et al.  Inference for nonconjugate Bayesian Models using the Gibbs sampler , 1991 .

[62]  Peter E. Rossi.  Comparison of Alternative Functional Forms in Production , 1985 .

[63]  R. Kass,et al.  Approximate Bayes Factors and Orthogonal Parameters, with Application to Testing Equality of Two Binomial Proportions , 1992 .

[64]  A. Raftery.  A Note on Bayes Factors for Log‐Linear Contingency Table Models with Vague Prior Information , 1986 .

[65]  R. Sklar,et al.  Role of the uvrE gene product and of inducible O6-methylguanine removal in the induction of mutations by N-methyl-N'-nitro-N-nitrosoguanidine in Escherichia coli. , 1980, Journal of molecular biology.

[66]  E. Parzen.  Some recent advances in time series modeling , 1974 .

[67]  Peter E. Rossi,et al.  A bayesian approach to testing the arbitrage pricing theory , 1991 .

[68]  M. Degroot.  Optimal Statistical Decisions , 1970 .

[69]  B. Carlin,et al.  Bayesian Model Choice Via Markov Chain Monte Carlo Methods , 1995 .

[70]  Edward E. Leamer,et al.  Specification Searches: Ad Hoc Inference with Nonexperimental Data , 1980 .

[71]  M. Stone.  Comments on Model Selection Criteria of Akaike and Schwarz , 1979 .

[72]  D. Cox.  Tests of Separate Families of Hypotheses , 1961 .

[73]  A. O'Hagan,et al.  Fractional Bayes factors for model comparison , 1995 .

[74]  J. Dickey,et al.  Bayes factors for independence in contingency tables , 1974 .

[75]  D. Spiegelhalter,et al.  Bayes Factors and Choice Criteria for Linear Models , 1980 .

[76]  Xiao-Li Meng,et al.  SIMULATING RATIOS OF NORMALIZING CONSTANTS VIA A SIMPLE IDENTITY: A THEORETICAL EXPLORATION , 1996 .

[77]  A. Tversky,et al.  The hot hand in basketball: On the misperception of random sequences , 1985, Cognitive Psychology.

[78]  L. Stein,et al.  Probability and the Weighing of Evidence , 1950 .

[79]  R. Kass,et al.  Approximate Bayesian Inference in Conditionally Independent Hierarchical Models (Parametric Empirical Bayes Models) , 1989 .

[80]  J. Cornfield.  Sequential Trials, Sequential Analysis and the Likelihood Principle , 1966 .

[81]  Peter E. Rossi,et al.  Bayes factors for nonlinear hypotheses and likelihood distributions , 1992 .

[82]  John Aitchison,et al.  Statistical Prediction Analysis , 1975 .

[83]  John W. Pratt,et al.  Testing a Point Null Hypothesis: The Irreconcilability of P Values and Evidence: Comment , 1987 .

[84]  Wayne S. Smith,et al.  Interactive Elicitation of Opinion for a Normal Linear Model , 1980 .

[85]  L. Pettit,et al.  Measuring the effect of observations on Bayes factors , 1990 .

[86]  L. Tierney,et al.  Accurate Approximations for Posterior Moments and Marginal Densities , 1986 .

[87]  S. Geisser,et al.  A Predictive Approach to Model Selection , 1979 .

[88]  G. Chow.  A comparison of the information and posterior probability criteria for model selection , 1981 .

[89]  A. Zellner.  Posterior odds ratios for regression hypotheses : General considerations and some specific results , 1981 .

[90]  A. Zellner.  An Introduction to Bayesian Inference in Econometrics , 1971 .

[91]  Adrian F. M. Smith,et al.  Bayesian computation via the gibbs sampler and related markov chain monte carlo methods (with discus , 1993 .

[92]  Edward E. Leamer,et al.  Model choice and specification analysis , 1983 .

[93]  A. Gelfand,et al.  Bayesian Model Choice: Asymptotics and Exact Calculations , 1994 .

[94]  E. George,et al.  Journal of the American Statistical Association is currently published by American Statistical Association. , 2007 .

[95]  J. Berger.  Statistical Decision Theory and Bayesian Analysis , 1988 .

[96]  K. Gaver,et al.  Posterior probabilities of alternative linear models , 1971 .

[97]  A. Raftery,et al.  Maximally Maintained Inequality: Expansion, Reform, and Opportunity in Irish Education, 1921-75. , 1993 .

[98]  J. Berger,et al.  Interpreting the stars in precise hypothesis testing , 1991 .

[99]  J. Dickey.  Approximate Posterior Distributions , 1976 .

[100]  A. Raftery.  Approximate Bayes factors and accounting for model uncertainty in generalised linear models , 1996 .

[101]  M. Bartlett.  A comment on D. V. Lindley's statistical paradox , 1957 .

[102]  D. Madigan,et al.  Model Selection and Accounting for Model Uncertainty in Graphical Models Using Occam's Window , 1994 .

[103]  James S. Hodges,et al.  Uncertainty, Policy Analysis and Statistics , 1987 .

[104]  H. V. Dijk,et al.  A Bayesian analysis of the unit root in real exchange rates , 1991 .

[105]  Bradley P. Carlin,et al.  Predicting Working Memory Failure: A Subjective Bayesian Approach to Model Selection , 1992 .

[106]  F. Mosteller,et al.  Inference and Disputed Authorship: The Federalist , 1966 .

[107]  Hong Chang,et al.  Model Determination Using Predictive Distributions with Implementation via Sampling-Based Methods , 1992 .

[108]  Alan J. Miller.  Sélection of subsets of regression variables , 1984 .

[109]  J. Cornfield.  A BAYESIAN TEST OF SOME CLASSICAL HYPOTHESES- WITH APPLICATIONS TO SEQUENTIAL CLINICAL TRIALS , 1966 .

[110]  A. Raftery,et al.  Stopping the Gibbs Sampler,the Use of Morphology,and Other Issues in Spatial Statistics (Bayesian image restoration,with two applications in spatial statistics) -- (Discussion) , 1991 .

[111]  A. Zellner.  Jeffreys-Bayes posterior odds ratio and the Akaike information criterion for discriminating between models , 1978 .

[112]  R. Katz.  On Some Criteria for Estimating the Order of a Markov Chain , 1981 .

[113]  J. F. Crook,et al.  The Robustness and Sensitivity of the Mixed-Dirichlet Bayesian Test for "Independence" in Contingency Tables , 1987 .

[114]  Peter E. Rossi.  Dynamic econometric modeling: Comparison of dynamic factor demand models , 1988 .

[115]  Adrian E. Raftery,et al.  Approximate Bayes factors for generalized linear models , 1988 .

[116]  D. Rubin.  Bayesianly Justifiable and Relevant Frequency Calculations for the Applied Statistician , 1984 .

[117]  Russell G. Almond,et al.  Strategies for Graphical Model Selection , 1994 .

[118]  J. Dickey,et al.  The Weighted Likelihood Ratio, Sharp Hypotheses about Chances, the Order of a Markov Chain , 1970 .

[119]  N. Draper,et al.  A Common Model Selection Criterion , 1987 .

[120]  David J. Spiegelhalter,et al.  Sequential Model Criticism in Probabilistic Expert Systems , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[121]  A. P. Dawid,et al.  Present position and potential developments: some personal views , 1984 .

[122]  R. Kass,et al.  Subregion-Adaptive Integration of Functions Having a Dominant Peak , 1997 .

[123]  J. Dickey,et al.  Bayesian Decision Theory and the Simplification of Models , 1980 .