A Simulation Study of Alternatives to Ordinary Least Squares

Abstract Estimated regression coefficients and errors in these estimates are computed for 160 artificial data sets drawn from 160 normal linear models structured according to factorial designs. Ordinary multiple regression (OREG) is compared with 56 alternatives which pull some or all estimated regression coefficients some or all the way to zero. Substantial improvements over OREG are exhibited when collinearity effects are present, noncentrality in the original model is small, and selected true regression coefficients are small. Ridge regression emerges as an important tool, while a Bayesian extension of variable selection proves valuable when the true regression coefficients vary widely in importance.

[1]  T. A. Bray,et al.  A Convenient Method for Generating Normal Variables , 1964 .

[2]  Stanley L. Sclove,et al.  Improved Estimators for Coefficients in Linear Regression , 1968 .

[3]  P. A. W. Lewis,et al.  A Pseudo-Random Number Generator for the System/360 , 1969, IBM Syst. J..

[4]  A. Baranchik,et al.  A Family of Minimax Estimators of the Mean of a Multivariate Normal Distribution , 1970 .

[5]  Donald W. Marquaridt.  Generalized Inverses, Ridge Regression, Biased Linear Estimation, and Nonlinear Estimation , 1970 .

[6]  A. E. Hoerl,et al.  Ridge Regression: Applications to Nonorthogonal Problems , 1970 .

[7]  B. Efron,et al.  Limiting the Risk of Bayes and Empirical Bayes Estimators—Part I: The Bayes Case , 1971 .

[8]  D. Lindley,et al.  Bayes Estimates for the Linear Model , 1972 .

[9]  B. Efron,et al.  Limiting the Risk of Bayes and Empirical Bayes Estimators—Part II: The Empirical Bayes Case , 1972 .

[10]  B. Efron,et al.  Empirical Bayes on vector observations: An extension of Stein's method , 1972 .

[11]  B. Efron,et al.  Combining Possibly Related Estimation Problems , 1973 .

[12]  D. Conniffe,et al.  A Critical View of Ridge Regression , 1973 .

[13]  B. Efron,et al.  Stein's Estimation Rule and Its Competitors- An Empirical Bayes Approach , 1973 .

[14]  A. E. Hoerl,et al.  Ridge regression:some simulations , 1975 .

[15]  B. Efron,et al.  Data Analysis Using Stein's Estimator and its Generalizations , 1975 .

[16]  P. Brown,et al.  Election Night Forecasting , 1975 .

[17]  W. Hemmerle.  An Explicit Solution for Generalized Ridge Regression , 1975 .

[18]  J. Rolph.  Choosing shrinkage estimators for regression problems , 1976 .