Model order estimation via penalizing adaptively the likelihood (PAL)

This paper introduces a novel rule for model order estimation based on penalizing adatively the likelihood (PAL). The penalty term of PAL, which is data adaptive (as the name suggests), has several unique features: it is ''small'' (e.g. comparable to AIC penalty) for model orders, let us say n, less than or equal to the true order, denoted by n"0, and it is ''large'' (e.g. of the same order as BIC penalty) for n>n"0; furthermore this is true not only as the data sample length increases (which is the case most often considered in the literature) but also as the signal-to-noise ratio (SNR) increases (the harder case for AIC, BIC and the like); and this ''oracle-like'' behavior of PAL's penalty is achieved without any knowledge about n"0. The paper presents a number of simulation examples to show that PAL has an excellent performance also in non-asymptotic regimes and compare this performance with that of AIC and BIC.

[1]  Walter Zucchini,et al.  Model Selection , 2011, International Encyclopedia of Statistical Science.

[2]  Mahmood Karimi,et al.  Order selection criteria for vector autoregressive models , 2011, Signal Process..

[3]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[4]  Steven Kay,et al.  Inconsistency of the MDL: On the Performance of Model Order Selection Criteria With Increasing Signal-to-Noise Ratio , 2011, IEEE Transactions on Signal Processing.

[5]  E. Lehmann.  Testing Statistical Hypotheses , 1960 .

[6]  S. S. Wilks.  Sample Criteria for Testing Equality of Means, Equality of Variances, and Equality of Covariances in a Normal Multivariate Distribution , 1946 .

[7]  C. Raghavendra Rao,et al.  On model selection , 2001 .

[8]  R. Shibata.  Asymptotically Efficient Selection of the Order of the Model for Estimating Parameters of a Linear Process , 1980 .

[9]  Yuhong Yang.  Can the Strengths of AIC and BIC Be Shared , 2005 .

[10]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[11]  H. Akaike.  A new look at the statistical model identification , 1974 .

[12]  Xiaotong Shen,et al.  Adaptive Model Selection , 2002 .

[13]  Petre Stoica,et al.  On the Proper Forms of BIC for Model Order Selection , 2012, IEEE Transactions on Signal Processing.

[14]  Tiee-Jian Wu,et al.  The weighted average information criterion for multivariate regression model selection , 2013, Signal Process..

[15]  Y. Selen,et al.  Model-order selection: a review of information criterion rules , 2004, IEEE Signal Processing Magazine.