Model selection and Akaike's Information Criterion (AIC): The general theory and its analytical extensions

During the last fifteen years, Akaike's entropy-based Information Criterion (AIC) has had a fundamental impact in statistical model evaluation problems. This paper studies the general theory of the AIC procedure and provides its analytical extensions in two ways without violating Akaike's main principles. These extensions make AIC asymptotically consistent and penalize overparameterization more stringently to pick only the simplest of the “true” models. These selection criteria are called CAIC and CAICF. Asymptotic properties of AIC and its extensions are investigated, and empirical performances of these criteria are studied in choosing the correct degree of a polynomial model in two different Monte Carlo experiments under different conditions.

[1]  R. Fisher,et al.  On the Mathematical Foundations of Theoretical Statistics , 1922 .

[2]  E. S. Pearson,et al.  ON THE USE AND INTERPRETATION OF CERTAIN TEST CRITERIA FOR PURPOSES OF STATISTICAL INFERENCE PART I , 1928 .

[3]  E. S. Pearson,et al.  On the Problem of the Most Efficient Tests of Statistical Hypotheses , 1933 .

[4]  E. S. Pearson,et al.  On the Problem of the Most Efficient Tests of Statistical Hypotheses , 1933 .

[5]  A. Wald.  Tests of statistical hypotheses concerning several parameters when the number of observations is large , 1943 .

[6]  M. G. Kendall,et al.  The advanced theory of statistics. Vols. 2. , 1969 .

[7]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[8]  E. Jaynes.  Information Theory and Statistical Mechanics , 1957 .

[9]  Maurice G. Kendall.  The advanced theory of statistics , 1958 .

[10]  Solomon Kullback,et al.  Information Theory and Statistics , 1960 .

[11]  T. W. Anderson.  The Choice of the Degree of a Polynomial Regression as a Multiple Decision Problem , 1962 .

[12]  G. Enderlein.  Wilks, S. S.: Mathematical Statistics. J. Wiley and Sons, New York–London 1962; 644 S., 98 s , 1964 .

[13]  The Power of the Likelihood Ratio Test , 1967 .

[14]  D. Lindley.  The Choice of Variables in Multiple Regression , 1968 .

[15]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[16]  H. Akaike.  A new look at the statistical model identification , 1974 .

[17]  H. Akaike.  Canonical Correlation Analysis of Time Series and the Use of an Information Criterion , 1976 .

[18]  Franklin A. Graybill,et al.  Theory and Application of the Linear Model , 1976 .

[19]  R. Bhansali,et al.  Some properties of the order of an autoregressive model selected by a generalization of Akaike∘s EPF criterion , 1977 .

[20]  Hirotugu Akaike,et al.  On entropy maximization principle , 1977 .

[21]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[22]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[23]  Hirotugu Akaike,et al.  On Newer Statistical Approaches to Parameter Estimation and Structure Determination , 1978 .

[24]  H. Akaike.  A Bayesian extension of the minimum AIC procedure of autoregressive model fitting , 1979 .

[25]  B. G. Quinn,et al.  The determination of the order of an autoregression , 1979 .

[26]  G. Kitagawa.  On the Use of AIC for the Detection of Outliers , 1979 .

[27]  Barry G. Quinn,et al.  Order Determination for a Multivariate Autoregression , 1980 .

[28]  Data Modeling Using Quantile and Density-Quantile Functions. , 1980 .

[29]  A. Atkinson.  A note on the generalized information criterion for choice of a model , 1980 .

[30]  C. J. Stone,et al.  Admissible Selection of an Accurate and Parsimonious Normal Linear Regression Model , 1981 .

[31]  H. Akaike.  Likelihood of a model and information criteria , 1981 .

[32]  Hirotugu Akaike,et al.  MODERN DEVELOPMENT OF STATISTICAL METHODS , 1981 .

[33]  Rangasami L. Kashyap,et al.  Optimal Choice of AR and MA Parts in Autoregressive Moving Average Models , 1982, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  H. White.  Maximum Likelihood Estimation of Misspecified Models , 1982 .

[35]  M. Woodroofe.  On Model Selection and the ARC Sine Laws , 1982 .

[36]  P. Coiffet.  Modelling and Control , 1983 .

[37]  H. Clergeot.  Filter-order selection in adaptive maximum likelihood estimation , 1984, IEEE Trans. Inf. Theory.

[38]  R. Hartley.  Stochastic Modelling and Control , 1985 .

[39]  E. Hannan.  Remembrance of Things Past , 1986 .

[40]  T. Teräsvirta,et al.  Model selection criteria and model selection tests in regression models , 1986 .

[41]  S. Sclove.  Application of model-selection criteria to some problems in multivariate analysis , 1987 .

[42]  H. Akaike.  Factor analysis and AIC , 1987 .

[43]  D. Haughton.  On the Choice of a Model to Fit Data from an Exponential Family , 1988 .