Akaike's Information Criterion, Cp and Estimators of Loss for Elliptically Symmetric Distributions

type="main" xml:id="insr12052-abs-0001"> In this article, we develop a modern perspective on Akaike's information criterion and Mallows's C p for model selection, and propose generalisations to spherically and elliptically symmetric distributions. Despite the differences in their respective motivation, C p and Akaike's information criterion are equivalent in the special case of Gaussian linear regression. In this case, they are also equivalent to a third criterion, an unbiased estimator of the quadratic prediction loss, derived from loss estimation theory. We then show that the form of the unbiased estimator of the quadratic prediction loss under a Gaussian assumption still holds under a more general distributional assumption, the family of spherically symmetric distributions. One of the features of our results is that our criterion does not rely on the specificity of the distribution, but only on its spherical symmetry. The same kind of criterion can be derived for a family of elliptically contoured distribution, which allows correlations, when considering the invariant loss. More specifically, the unbiasedness property is relative to a distribution associated to the original density.

[1]  Tatsuya Kubokawa,et al.  Robust Improvement in Estimation of a Mean Matrix in an Elliptically Contoured Distribution , 2001 .

[2]  C. Stein Estimation of the Mean of a Multivariate Normal Distribution , 1981 .

[3]  Martin T. Wells,et al.  Robust shrinkage estimation for elliptically symmetric distributions with unknown covariance matrix , 2003 .

[4]  Dean P. Foster,et al.  The risk inflation criterion for multiple regression , 1994 .

[5]  Tatsuya Kubokawa,et al.  Robust improvement in estimation of a covariance matrix in an elliptically contoured distribution , 1999 .

[6]  M. A. Chmielewski,et al.  Elliptically Symmetric Distributions: A Review and Bibliography , 1981 .

[7]  H. Akaike A new look at the statistical model identification , 1974 .

[8]  Joseph E. Cavanaugh,et al.  Estimation Optimality of Corrected AIC and Modified Cp in Linear Regression , 2006 .

[9]  Mary C. Meyer,et al.  ON THE DEGREES OF FREEDOM IN SHAPE-RESTRICTED REGRESSION , 2000 .

[10]  C. Robert,et al.  Robust shrinkage estimators of the location parameter for elliptically symmetric distributions , 1989 .

[11]  Ker-Chau Li,et al.  From Stein's Unbiased Risk Estimates to the Method of Generalized Cross Validation , 1985 .

[12]  D. Cellier,et al.  Shrinkage Estimators under Spherical Symmetry for the General Linear Model , 1995 .

[13]  On Inadmissibility of Some Unbiased Estimates of Loss , 1988 .

[14]  R. Shibata Asymptotically Efficient Selection of the Order of the Model for Estimating Parameters of a Linear Process , 1980 .

[15]  Power analysis for linear models with spherical errors , 2002 .

[16]  M. Wells,et al.  Estimation of a Loss Function for Spherically Symmetric Distributions in the General Linear Model , 1995 .

[17]  Dominique Fourdrinier,et al.  On Bayes and unbiased estimators of loss , 2003 .

[18]  Sylvain Arlot,et al.  A survey of cross-validation procedures for model selection , 2009, 0907.4728.

[19]  Yunqian Ma,et al.  Comparison of Model Selection for Regression , 2003, Neural Computation.

[20]  Robust generalized Bayes minimax estimators of location vectors for spherically symmetric distributions with unknown scale , 2010 .

[21]  William E. Strawderman,et al.  An extended class of minimax generalized Bayes estimators of regression coefficients , 2009, J. Multivar. Anal..

[22]  B. Efron The Estimation of Prediction Error , 2004 .

[23]  C. L. Mallows Some comments on C_p , 1973 .

[24]  H. Hudson A Natural Identity for Exponential Families with Applications in Multiparameter Estimation , 1978 .

[25]  Jianming Ye On Measuring and Correcting the Effects of Data Mining and Model Selection , 1998 .

[26]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[27]  Martin T. Wells,et al.  On Improved Loss Estimation for Shrinkage Estimators , 2012, 1203.4989.

[28]  A robust generalized Bayes estimator improving on the James-Stein estimator for spherically symmetric distributions , 2003 .