Multiple-shrinkage principal component regression

SUMMARY Consider the multiple-regression framework where the goal is 'black box' prediction and the explanatory variables are (approximately) multicollinear. For this situation we propose a multiple-shrinkage estimator which adaptively mimics the best principal components shrinkage estimator. The estimator proposed is a variant of the Stein estimator which is also guaranteed to give lower prediction mean-squared error than the least squares estimator, thus avoiding the danger of overshrinking when using principal components regression. The predictive performance of this estimator is illustrated and compared with that of principal components regression on three data sets by using random crossvalidation.

[1]  J. Shao.  Linear Model Selection by Cross-validation , 1993 .

[2]  J. Friedman,et al.  A Statistical View of Some Chemometrics Regression Tools , 1993 .

[3]  S. Oman.  Random calibration with many measurements: an application of Stein estimation , 1991 .

[4]  J. Chambers,et al.  The New S Language , 1989 .

[5]  G. Stewart.  Collinearity and Least Squares Regression , 1987 .

[6]  E. George.  Combining Minimax Shrinkage Estimators , 1986 .

[7]  R. Jennrich,et al.  How much does stein estimation help in multiple linear regression , 1986 .

[8]  E. George.  Minimax Multiple Shrinkage Estimation , 1986 .

[9]  S. Wold,et al.  The Collinearity Problem in Linear Regression. The Partial Least Squares (PLS) Approach to Generalized Inverses , 1984 .

[10]  G. Casella,et al.  Minimax Confidence Sets for the Mean of a Multivariate Normal Distribution , 1982 .

[11]  C. Stein.  Estimation of the Mean of a Multivariate Normal Distribution , 1981 .

[12]  H. V. Henderson,et al.  Building Multiple Regression Models Interactively , 1981 .

[13]  T. Hassard,et al.  Applied Linear Regression , 2005 .

[14]  Norman R. Draper,et al.  Ridge Regression and James-Stein Estimation: Review and Comments , 1979 .

[15]  G. Judge,et al.  A Monte Carlo comparison of traditional and Stein-rule estimators under squared error loss , 1976 .

[16]  R. R. Hocking.  The analysis and selection of variables in linear regression , 1976 .

[17]  G. C. McDonald,et al.  Instabilities of Regression Estimates Relating Air Pollution to Mortality , 1973 .

[18]  George G. Judge,et al.  The Statistical Consequences of Preliminary Test Estimators in Regression , 1973 .

[19]  C. Morris,et al.  Non-Optimality of Preliminary-Test Estimators for the Mean of a Multivariate Normal Distribution , 1972 .

[20]  S. Silvey.  Multicollinearity and Imprecise Estimation , 1969 .

[21]  Stanley L. Sclove,et al.  Improved Estimators for Coefficients in Linear Regression , 1968 .

[22]  W. Massy.  Principal Components Regression in Exploratory Statistical Research , 1965 .

[23]  Maurice G. Kendall,et al.  A course in multivariate analysis , 1958 .

[24]  I. Helland.  Maximum likelihood regression on relevant components , 1992 .

[25]  C. Stein,et al.  Estimation with Quadratic Loss , 1992 .

[26]  G. Seber,et al.  Linear Regression Analysis , 1980 .