Improved Empirical Bayes Ridge Regression Estimators Under Multicollinearity

Abstract In this paper, we consider the problem of estimating the regression parameters in a multiple linear regression model when the multicollinearity is present. Under the assumption of normality, we present three empirical Bayes estimators. One of them shrinks the least squares (LS) estimator towards the principal component. The second one is a hierarchical empirical Bayes estimator shrinking the LS estimator twice. The third one is obtained by choosing different priors for the two sets of regression parameters that arise in the case of multicollinearity; this estimator is termed decomposed empirical Bayes estimator. These proposed estimators are not only proved to be uniformly better than the LS estimator, that is, minimax in terms of risk under the Strawderman loss function, but also shown to be useful in the multicollinearity cases through simulation and empirical studies.

[1]  William E. Strawderman,et al.  Minimax Adaptive Generalized Ridge Regression Estimators , 1978 .

[2]  Bradley Efron,et al.  Families of Minimax Estimators of the Mean of a Multivariate Normal Distribution , 1976 .

[3]  R. Sundberg,et al.  Continuum regression is not always continuous , 1996 .

[4]  A. E. Hoerl,et al.  Ridge regression: biased estimation for nonorthogonal problems , 2000 .

[5]  Anders Björkström,et al.  A Generalized View on Continuum Regression , 1999 .

[6]  George Casella,et al.  Minimax Ridge Regression Estimation , 1980 .

[7]  Nobuo Shinozaki,et al.  Minimaxity of empirical bayes estimators of the means of independent normal variables with unequal variances , 1993 .

[8]  R. Sundberg.  Continuum Regression and Ridge Regression , 1993 .

[9]  Nobuo Shinozaki,et al.  Minimaxity of empirical bayes estimators shrinking toward the grand mean when variances are unequal , 1996 .

[10]  Muni S. Srivastava,et al.  Regression Analysis: Theory, Methods, and Applications , 1991 .

[11]  D. Lindley,et al.  Bayes Estimates for the Linear Model , 1972 .

[12]  R. Snee,et al.  Ridge Regression in Practice , 1975 .

[13]  Stephen Taylor,et al.  Regression Analysis with Applications , 1987 .

[14]  M. Stone.  Continuum regression: Cross-validated sequentially constructed prediction embracing ordinary least s , 1990 .

[15]  M. Srivastava.  Methods of Multivariate Statistics , 2002 .

[16]  C. Stein.  Estimation of the Mean of a Multivariate Normal Distribution , 1981 .

[17]  David A. Belsley.  Demeaning Conditioning Diagnostics through Centering , 1984 .