Multivariate regression model selection from small samples using Kullback's symmetric divergence

The Kullback Information Criterion, KIC, and its univariate bias-corrected version, KICc, are two new developed criteria for model selection. The two criteria can be viewed as estimators of the expected Kullback symmetric divergence and they have a fixed bias corrected term. In this paper, a new small sample model selection criterion for multivariate regression models is developed. The proposed criterion is named KICvc, where the notation "vc" stands for vector correction, and it can be considered as an extension of KIC for multivariate regression models. KICvc adjusts KIC to be an unbiased estimator for the variant of the Kullback symmetric divergence, assuming that the true model is correctly specified or overfitted. Furthermore, KICvc provides better multivariate regression model order or dimension choices than KIC in small samples. Simulation results shows that the proposed criterion estimates the model order more accurately than other asymptotically efficient methods when applied to multivariate regression model selection in small samples. As a result, KICvc serves as an effective tool for selecting a multivariate regression model of appropriate dimension. A theoretical justification of the proposed criterion is presented.

[1]  H. Akaike.  Statistical predictor identification , 1970 .

[2]  Abd-Krim Seghouane,et al.  A criterion for model selection in the presence of incomplete data based on Kullback's symmetric divergence , 2005, Signal Process..

[3]  Piet M. T. Broersen,et al.  On the penalty factor for autoregressive order selection in finite samples , 1996, IEEE Trans. Signal Process..

[4]  Chih-Ling Tsai,et al.  MODEL SELECTION FOR MULTIVARIATE REGRESSION IN SMALL SAMPLES , 1994 .

[5]  Abd-Krim Seghouane,et al.  A criterion for vector autoregressive model selection based on Kullback's symmetric divergence , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[6]  Hidetoshi Shimodaira.  A new criterion for selecting models from partially observed data , 1994 .

[7]  J. Bernardo,et al.  Psi (Digamma) Function , 1976 .

[8]  J. Cavanaugh.  Criteria for Linear Model Selection Based on Kullback's Symmetric Divergence , 2004 .

[9]  T. W. Anderson.  An Introduction to Multivariate Statistical Analysis , 1959 .

[10]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[11]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[12]  J. Cavanaugh.  A large-sample model selection criterion based on Kullback's symmetric divergence , 1999 .

[13]  R. Muirhead.  Aspects of Multivariate Statistical Theory , 1982, Wiley Series in Probability and Statistics.

[14]  Clifford M. Hurvich,et al.  Regression and time series model selection in small samples , 1989 .

[15]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[16]  H. Jeffreys.  An invariant form for the prior probability in estimation problems , 1946, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences.

[17]  R. Bhansali,et al.  Some properties of the order of an autoregressive model selected by a generalization of Akaike∘s EPF criterion , 1977 .

[18]  J. Cavanaugh,et al.  An Akaike information criterion for model selection in the presence of incomplete data , 1998 .

[19]  Abd-Krim Seghouane,et al.  A small sample model selection criterion based on Kullback's symmetric divergence , 2004, IEEE Transactions on Signal Processing.