Shrinkage estimation for identification of linear components in additive models

In this short paper, we demonstrate that the popular penalized estimation method typically used for variable selection in parametric or semiparametric models can actually provide a way to identify linear components in additive models. Unlike most studies in the literature, we are NOT performing variable selection. Due to the difficulty in a priori deciding which predictors should enter the partially linear additive model as the linear components, such a method will prove useful in practice.

[1]  Jianhua Z. Huang,et al.  Variable Selection in Nonparametric Varying-Coefficient Models for Analysis of Repeated Measurements , 2008, Journal of the American Statistical Association.

[2]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[3]  Peng Zhao,et al.  On Model Selection Consistency of Lasso , 2006, J. Mach. Learn. Res..

[4]  Lie Wang,et al.  Adaptive variance function estimation in heteroscedastic nonparametric regression , 2008, 0810.4780.

[5]  Ji Zhu,et al.  Regularized Multivariate Regression for Identifying Master Predictors with Application to Integrative Genomics Study of Breast Cancer. , 2008, The annals of applied statistics.

[6]  H. Zou.  The Adaptive Lasso and Its Oracle Properties , 2006 .

[7]  Cun-Hui Zhang,et al.  The sparsity and bias of the Lasso selection in high-dimensional linear regression , 2008, 0808.0967.

[8]  H. Zou,et al.  Structured variable selection and estimation , 2009, 1011.0610.

[9]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[10]  O. Lepskii.  On a Problem of Adaptive Estimation in Gaussian White Noise , 1991 .

[11]  G. Wahba.  Spline models for observational data , 1990 .

[12]  Carl de Boor,et al.  A Practical Guide to Splines , 1978, Applied Mathematical Sciences.

[13]  Yufeng Liu,et al.  VARIABLE SELECTION IN QUANTILE REGRESSION , 2009 .

[14]  H. Zou,et al.  Addendum: Regularization and variable selection via the elastic net , 2005 .

[15]  I. Johnstone,et al.  Adapting to Unknown Smoothness via Wavelet Shrinkage , 1995 .

[16]  Qi Li,et al.  Efficient Estimation of Additive Partially Linear Models , 2000 .

[17]  P. Bickel,et al.  SIMULTANEOUS ANALYSIS OF LASSO AND DANTZIG SELECTOR , 2008, 0801.1095.

[18]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[19]  I. Johnstone,et al.  Wavelet Shrinkage: Asymptopia? , 1995 .

[20]  K. Doksum,et al.  SHRINKAGE, PRETEST AND ABSOLUTE PENALTY ESTIMATORS IN PARTIALLY LINEAR MODELS , 2007 .