Model selection with data-oriented penalty

We consider the problem of model (or variable) selection in the classical regression model using the GIC (general information criterion). In this method the maximum likelihood is used with a penalty function denoted by C n , depending on the sample size n and chosen to ensure consistency in the selection of the true model. There are various choices of C n suggested in the literature on model selection. In this paper we show that a particular choice of C n based on observed data, which makes it random, preserves the consistency property and provides improved performance over a fixed choice of C n .

[1]  Calyampudi R. Rao,et al.  A strongly consistent procedure for model selection in a regression problem , 1989 .

[2]  H. Akaike.  Statistical predictor identification , 1970 .

[3]  R. Shibata.  Approximate efficiency of a selection procedure for the number of regression variables , 1984 .

[4]  Robert B. Ash,et al.  Information Theory , 2020, The SAGE International Encyclopedia of Mass Media and Society.

[5]  Ritei Shibata.  Selection of the number of regression variables; A minimax choice of generalized FPE , 1986 .

[6]  A. Atkinson.  A note on the generalized information criterion for choice of a model , 1980 .

[7]  B. G. Quinn,et al.  The determination of the order of an autoregression , 1979 .

[8]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[9]  Zhi-Dong Bai,et al.  On rates of convergence of efficient detection criteria in signal processing with white noise , 1989, IEEE Trans. Inf. Theory.

[10]  C. R. Rao,et al.  Linear Statistical Inference and its Applications , 1968 .

[11]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[12]  Hans-Hermann Bock,et al.  Classification and Related Methods of Data Analysis , 1988 .

[13]  W. Loh,et al.  Consistent Variable Selection in Linear Models , 1995 .

[14]  J. Shao.  Linear Model Selection by Cross-validation , 1993 .

[15]  Calyampudi R. Rao,et al.  Linear Statistical Inference and Its Applications. , 1975 .

[16]  H. Akaike.  A new look at the statistical model identification , 1974 .

[17]  Z. Bai,et al.  On detection of the number of signals in presence of white noise , 1985 .