SHRINKAGE ESTIMATION FOR NEARLY SINGULAR DESIGNS

Shrinkage estimation procedures such as ridge regression and the lasso have been proposed for stabilizing estimation in linear models when high collinearity exists in the design. In this paper, we consider asymptotic properties of shrinkage estimators in the case of “nearly singular” designs.I thank Hannes Leeb and Benedikt Pötscher and also the referees for their valuable comments. This research was supported by a grant from the Natural Sciences and Engineering Research Council of Canada.

[1]  J. Friedman,et al.  A Statistical View of Some Chemometrics Regression Tools , 1993 .

[2]  Jon A. Wellner,et al.  Weak Convergence and Empirical Processes: With Applications to Statistics , 1996 .

[3]  P. Phillips.  Regression with Slowly Varying Regressors , 2001 .

[4]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[5]  Mehmet Caner,et al.  LASSO-TYPE GMM ESTIMATOR , 2009, Econometric Theory.

[6]  Wenjiang J. Fu.  Penalized Regressions: The Bridge versus the Lasso , 1998 .

[7]  Georg Ch. Pflug,et al.  Asymptotic Stochastic Programs , 1995, Math. Oper. Res..

[8]  C. Geyer.  On the Asymptotics of Constrained $M$-Estimation , 1994 .

[9]  Hannes Leeb,et al.  Performance Limits for Estimators of the Risk or Distribution of Shrinkage-Type Estimators, and Some General Lower Risk-Bound Results , 2002 .

[10]  M. Stone.  Continuum regression: Cross-validated sequentially constructed prediction embracing ordinary least s , 1990 .

[11]  M. Srivastava.  On Fixed-Width Confidence Bounds for Regression Parameters , 1971 .

[12]  Benedikt M. Pötscher,et al.  On the Distribution of Penalized Maximum Likelihood Estimators: The LASSO, SCAD, and Thresholding , 2007, J. Multivar. Anal..

[13]  Jonathan H. Wright,et al.  A Survey of Weak Instruments and Weak Identification in Generalized Method of Moments , 2002 .

[14]  J. Robins,et al.  Likelihood-based inference with singular , 2000 .

[15]  A. E. Hoerl,et al.  Ridge regression: biased estimation for nonorthogonal problems , 2000 .

[16]  L. E. Wangen,et al.  A theoretical foundation for the PLS algorithm , 1987 .

[17]  Mehmet Caner.  Nearly Singular Design in GMM and Generalized Empirical Likelihood Estimators , 2006 .

[18]  Wenjiang J. Fu,et al.  Asymptotics for lasso-type estimators , 2000 .

[19]  V. Chernozhukov.  Extremal Quantile Regression , 2005 .

[20]  J. Robins,et al.  Likelihood-based inference with singular information matrix , 2000 .

[21]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[22]  Victor Chernozhukov,et al.  Likelihood Estimation and Inference in a Class of Nonregular Econometric Models , 2003 .