Bayes Factors and Choice Criteria for Linear Models

SUMMARY Global and local Bayes factors are defined and their respective roles examined as choice criteria among alternative linear models. The global Bayes factor is seen to function, in appropriate contexts, as a fully automatic Occam's razor and to be closely related to the Schwarz model choice criterion. The local Bayes factor is shown to have a close relationship with the Akaike Information Criterion. THE problem of choosing between alternative models continues to attract a good deal of theoretical attention, much of which has been stimulated by the appearance of the Akaike Information Criterion (Akaike, 1973). The AIC and its variants (see, for example, Bhansali and Downham, 1977) essentially adjust the likelihood ratio test statistic by a constant multiple of the difference in dimensionalities of the two models under consideration. Schwarz (1978) has recently proposed a fundamentally different criterion which replaces the constant multiplier of the AIC by the logarithm of the sample size. Stone (1979) has compared and contrasted these two approaches in terms of certain of their asymptotic properties, and a further contribution in this area is provided by Hannan and Quinn (1979). In this paper, which deals with the important special case of choice between alternative nested linear models, we shall also be concerned with comparing and contrasting these two forms of model choice criteria, but from a rather different, non-asymptotic, perspective. Our starting point is not a consideration of the criteria themselves, but, instead, a discussion of the Bayesian approach to comparing alternative nested linear models on the basis of their posterior probabilities, or, equivalently, on the basis of ratios of posterior to prior odds. Stone has, in effect, argued that the comparison of choice criteria for linear models on the basis of their asymptotic properties is rather arbitrarily dependent on the assumptions made about the embedded sequence of design matrices. Our approach avoids the arbitrary asymptotics and concentrates, instead, on the way in which the dependence of the prior specification on the design matrix influences the forms of model choice criteria which arise from consideration of posterior probabilities. It will be shown that, depending on the nature of the prior specification adopted for model parameters, two fundamentally different forms of odds ratio, or Bayesfactor, arise. The first of these, which we shall call the global Bayes factor, will be discussed in Section 2, and will be shown to lead, essentially, to the Schwarz-type of criterion. The relationship of the global Bayes factor to the so-called Lindley Paradox (Lindley, 1957) will also be examined and, motivated by this, in Section 3 we derive what we shall call the local Bayes factor. This will be shown to lead to a variant of the Akaike criterion, and a comparison will then be made with various Akaike-type procedures, including those of Bhansali and Downham (1977).