A note on the generalized information criterion for choice of a model

SUMMARY One way of selecting models is to choose that model for which the maximized log likelihood minus a multiple of the number of parameters estimated is a maximum. This note explores the choice of values for the multiplier, with the value one corresponding to Akaike's information criterion. The relationship with Bayesian procedures is mentioned. Suppose that there are a number of competing models which may be fitted to some data. If the log likelihood of the ith model maximized over q* parameters is Li, the generalized information criterion is to choose the model for which Li - -1cq* is a maximum. In the criterion suggested by Akaike as equals 2. Values of os in the range 1-4 are considered by Bhansali & Downham (1977) for the choice of a time series model. As have many other authors, including Geisser & Eddy (1979) and McClave (1978), Bhansali & Downham assessed their criteria by the frequency of choice of the correct model. One purpose of the present note is to suggest that this. may not be an appropriate basis for choice: the objectives of the analysis need more explicit formulation. A similar point is made by Akaike (1979) who com- pares values of ot on the basis of squared prediction error. His simulation is, however, un- informative about the conditions under which various values of a are optimum. In this note a simple example is simulated to investigate the taxonomy of optimum of values for prediction purposes. For simplicity of structure and ease of simulation we work with linear regression models, for which the information criterion reduces to a generalized Cp statistic. The behaviour of this statistic as a function of ot is investigated in the next section. Significance testing and Bayesian alternatives are discussed in ? 3. Section 4 is concerned with asymptotics. The note closes with some general comments which allude to ridge regression and simulation.

[1]  G. Box,et al.  A Basis for the Selection of a Response Surface Design , 1959 .

[2]  D. Cox.  Tests of Separate Families of Hypotheses , 1961 .

[3]  G. Box,et al.  THE CHOICE OF A SECOND ORDER ROTATABLE DESIGN , 1963 .

[4]  Williams Da,et al.  Discrimination between regression models to determine the pattern of enzyme synthesis in synchronous cell cultures. , 1970 .

[5]  M. Stone.  An Asymptotic Equivalence of Choice of Model by Cross‐Validation and Akaike's Criterion , 1977 .

[6]  R. Bhansali,et al.  Some properties of the order of an autoregressive model selected by a generalization of Akaike∘s EPF criterion , 1977 .

[7]  N. Wermuth,et al.  A Simulation Study of Alternatives to Ordinary Least Squares , 1977 .

[8]  R. Kashyap.  A Bayesian comparison of different classes of dynamic models using empirical data , 1977 .

[9]  T. Fine,et al.  Consistent estimation of system order , 1978, 1978 IEEE Conference on Decision and Control including the 17th Symposium on Adaptive Processes.

[10]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[11]  A. Atkinson.  Posterior probabilities for choosing a regression model , 1978 .

[12]  P. J. Brown,et al.  Prediction with shrinkage estimators , 1978 .

[13]  James T. McClave.  Estimating the order of moving average models: the max X2 method , 1978 .

[14]  H. Akaike.  A Bayesian analysis of the minimum AIC procedure , 1978 .

[15]  H. Akaike.  A Bayesian extension of the minimum AIC procedure of autoregressive model fitting , 1979 .

[16]  M. Stone.  Comments on Model Selection Criteria of Akaike and Schwarz , 1979 .

[17]  B. G. Quinn,et al.  The determination of the order of an autoregression , 1979 .

[18]  Russell C. H. Cheng,et al.  Some Simple Gamma Variate Generators , 1979 .

[19]  S. Geisser,et al.  A Predictive Approach to Model Selection , 1979 .

[20]  D. Spiegelhalter,et al.  Bayes Factors and Choice Criteria for Linear Models , 1980 .