Variable selection in partially linear wavelet models

Variable selection is fundamental in high-dimensional statistical modelling, including non-and semiparametric regression. However, little work has been done for variable selection in a partially linear model (PLM). We propose and study a unified approach via double penalized least squares, retaining good features of both variable selection and model estimation in the framework of PLM. The proposed method is distinguished from others in that the penalty functions combine the l 1 penalty coming from wavelet thresholding in the non-parametric component with the l 1 penalty from the lasso in the parametric component. Simulations are used to investigate the performances of the proposed estimator in various settings, illustrating its effectiveness for simultaneous variable selection as well as estimation.

[1]  Wim Sweldens,et al.  The lifting scheme: a construction of second generation wavelets , 1998 .

[2]  C. L. Mallows.  Some comments on C_p , 1973 .

[3]  D. L. Donoho,et al.  Ideal spacial adaptation via wavelet shrinkage , 1994 .

[4]  H. Akaike.  Maximum likelihood identification of Gaussian autoregressive moving average models , 1973 .

[5]  S. Pandey,et al.  What Are Degrees of Freedom , 2008 .

[6]  I. Johnstone,et al.  Adapting to Unknown Smoothness via Wavelet Shrinkage , 1995 .

[7]  Young K. Truong,et al.  Local Linear Estimation in Partly Linear Models , 1997 .

[8]  I. Johnstone,et al.  Wavelet Shrinkage: Asymptopia? , 1995 .

[9]  Arne Kovac,et al.  Extending the Scope of Wavelet Regression Methods by Coefficient-Dependent Thresholding , 2000 .

[10]  Wolfgang Härdle,et al.  Partially Linear Models , 2000 .

[11]  Xiao-Wen Chang,et al.  Wavelet estimation of partially linear models , 2004, Comput. Stat. Data Anal..

[12]  Hao Helen Zhang,et al.  Adaptive Lasso for Cox's proportional hazards model , 2007 .

[13]  Runze Li,et al.  Variable Selection in Semiparametric Regression Modeling. , 2008, Annals of statistics.

[14]  Wenjiang J. Fu,et al.  Asymptotics for lasso-type estimators , 2000 .

[15]  Adhemar Bultheel,et al.  Multiple wavelet threshold estimation by generalized cross validation for images with correlated noise , 1999, IEEE Trans. Image Process..

[16]  T. Tony Cai,et al.  WAVELET SHRINKAGE FOR NONEQUISPACED SAMPLES , 1998 .

[17]  R. Tibshirani,et al.  On the “degrees of freedom” of the lasso , 2007, 0712.0881.

[18]  I. Johnstone,et al.  Ideal spatial adaptation by wavelet shrinkage , 1994 .

[19]  Peter Craven,et al.  Smoothing noisy data with spline functions , 1978 .

[20]  Adhemar Bultheel,et al.  Generalized cross validation for wavelet thresholding , 1997, Signal Process..

[21]  Nancy E. Heckman,et al.  Spline Smoothing in a Partly Linear Model , 1986 .

[22]  R. Tibshirani,et al.  Sparsity and smoothness via the fused lasso , 2005 .

[23]  Peng Zhao,et al.  On Model Selection Consistency of Lasso , 2006, J. Mach. Learn. Res..

[24]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[25]  Chih-Ling Tsai,et al.  Regression coefficient and autoregressive order shrinkage and selection via the lasso , 2007 .

[26]  Fuxia Cheng.  On Variance Estimation in Semiparametric Regression Models , 2005 .

[27]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[28]  G. Nason.  Wavelet Shrinkage using Cross-validation , 1996 .

[29]  P. Speckman.  Kernel smoothing in partial linear models , 1988 .

[30]  J. Rice.  Convergence rates for partially splined models , 1986 .

[31]  C. Mallows.  Some Comments on Cp , 2000, Technometrics.

[32]  Anestis Antoniadis,et al.  Wavelet regression for random or irregular design , 1998 .

[33]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[34]  E. Bullmore,et al.  Penalized partially linear models using sparse representations with an application to fMRI time series , 2005, IEEE Transactions on Signal Processing.

[35]  Adhemar Bultheel,et al.  Stabilised wavelet transforms for non-equispaced data smoothing , 2002, Signal Process..

[36]  Wenjiang J. Fu.  Penalized Regressions: The Bridge versus the Lasso , 1998 .

[37]  C. Mallows.  More comments on C p , 1995 .

[38]  Chih-Ling Tsai,et al.  Semiparametric and Additive Model Selection Using an Improved Akaike Information Criterion , 1999 .

[39]  Mohamed-Jalal Fadili,et al.  Penalized partially linear models using orthonormal wavelet bases with an application to fMRI time series , 2004, 2004 2nd IEEE International Symposium on Biomedical Imaging: Nano to Macro (IEEE Cat No. 04EX821).

[40]  H. Zou.  The Adaptive Lasso and Its Oracle Properties , 2006 .