Combining Unbiased Ridge and Principal Component Regression Estimators

In the presence of multicollinearity problem, ordinary least squares (OLS) estimation is inadequate. To circumvent this problem, two well-known estimation procedures often suggested are the unbiased ridge regression (URR) estimator given by Crouse et al. (1995) and the (r, k) class estimator given by Baye and Parker (1984). In this article, we proposed a new class of estimators, namely modified (r, k) class ridge regression (MCRR) which includes the OLS, the URR, the (r, k) class, and the principal components regression (PCR) estimators. It is based on a criterion that combines the ideas underlying the URR and the PCR estimators. The standard properties of this new class estimator have been investigated and a numerical illustration is done. The conditions under which the MCRR estimator is better than the other two estimators have been investigated.

[1]  R. W. Farebrother,et al.  Further Results on the Mean Square Error of Ridge Regression , 1976 .

[2]  Donald W. Marquaridt.  Generalized Inverses, Ridge Regression, Biased Linear Estimation, and Nonlinear Estimation , 1970 .

[3]  Chun Jin,et al.  Unbiased ridge estimation with prior information and ridge trace , 1995 .

[4]  Arthur E. Hoerl,et al.  Ridge Regression: Biased Estimation for Nonorthogonal Problems , 2000, Technometrics.

[5]  Selahattin Kaçıranlar,et al.  Comparisons of the Unbiased Ridge Estimation to the Other Estimations , 2007 .

[6]  Feras Sh. Mahmood Batah,et al.  Effect of jackknifing on various ridge type estimators , 2008, Model. Assist. Stat. Appl..

[7]  Robert L. Mason,et al.  Biased Estimation in Regression: An Evaluation Using Mean Squared Error , 1977 .

[8]  Lawrence S. Mayer,et al.  On Biased Estimation in Linear Models , 1973 .

[9]  Jeffrey Pliskin.  A ridge-type estimator and good prior means , 1987 .

[10]  Michael R. Baye,et al.  Combining ridge and principal component regression:a money demand illustration , 1984 .

[11]  A. E. Hoerl,et al.  Ridge regression:some simulations , 1975 .

[12]  Nityananda Sarkar,et al.  Mean square error matrix comparison of some estimators in linear regressions with multicollinearity , 1996 .

[13]  B. F. Swindel.  Good ridge estimators based on prior information , 1976 .

[14]  Dale Borowiak,et al.  Linear Models, Least Squares and Alternatives , 2001, Technometrics.

[15]  Zhi-Fu Wang,et al.  On Biased Estimation in Linear Models , 2006, 2006 International Conference on Machine Learning and Cybernetics.