Variable selection in high-dimensional quantile varying coefficient models

In this paper, we propose a two-stage variable selection procedure for high dimensional quantile varying coefficient models. The proposed method is based on basis function approximation and LASSO-type penalties. We show that the first stage penalized estimator with LASSO penalty reduces the model from ultra-high dimensional to a model that has size close to the true model, but contains the true model as a valid sub model. By applying adaptive LASSO penalty to the reduced model, the second stage excludes the remained irrelevant covariates, leading to an estimator consistent in variable selection. A simulation study and the analysis of a real data demonstrate that the proposed method performs quite well in finite samples, with regard to dimension reduction and variable selection.

[1]  J. Horowitz,et al.  VARIABLE SELECTION IN NONPARAMETRIC ADDITIVE MODELS. , 2010, Annals of statistics.

[2]  Jianqing Fan,et al.  Sure independence screening for ultrahigh dimensional feature space , 2006, math/0612857.

[3]  Jianhui Zhou,et al.  Quantile regression in partially linear varying coefficient models , 2009, 0911.3501.

[4]  P. Shi,et al.  Convergence rate of b-spline estimators of nonparametric conditional quantile functions ∗ , 1994 .

[5]  R. Koenker,et al.  Regression Quantiles , 2007 .

[6]  Cun-Hui Zhang,et al.  The sparsity and bias of the Lasso selection in high-dimensional linear regression , 2008, 0808.0967.

[7]  Runze Li,et al.  Quantile Regression for Analyzing Heterogeneity in Ultra-High Dimension , 2012, Journal of the American Statistical Association.

[8]  Yingcun Xia,et al.  Shrinkage Estimation of the Varying Coefficient Model , 2008 .

[9]  N. Meinshausen,et al.  High-dimensional graphs and variable selection with the Lasso , 2006, math/0608017.

[10]  L. Schumaker.  Spline Functions: Basic Theory , 1981 .

[11]  Keith Knight,et al.  Limiting distributions for $L\sb 1$ regression estimators under general conditions , 1998 .

[12]  Hansheng Wang.  Forward Regression for Ultra-High Dimensional Variable Screening , 2009 .

[13]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[14]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[15]  N. Meinshausen,et al.  LASSO-TYPE RECOVERY OF SPARSE REPRESENTATIONS FOR HIGH-DIMENSIONAL DATA , 2008, 0806.0145.

[16]  Jian Huang,et al.  VARIABLE SELECTION AND ESTIMATION IN HIGH-DIMENSIONAL VARYING-COEFFICIENT MODELS. , 2011, Statistica Sinica.

[17]  Jianhua Z. Huang,et al.  Variable Selection in Nonparametric Varying-Coefficient Models for Analysis of Repeated Measurements , 2008, Journal of the American Statistical Association.

[18]  Jianqing Fan,et al.  Profile likelihood inferences on semiparametric varying-coefficient partially linear models , 2005 .

[19]  H. Zou.  The Adaptive Lasso and Its Oracle Properties , 2006 .

[20]  D. Rubinfeld,et al.  Hedonic housing prices and the demand for clean air , 1978 .

[21]  Heng Lian.  Flexible Shrinkage Estimation in High-Dimensional Varying Coefficient Models , 2010 .

[22]  Zhongyi Zhu,et al.  A UNIFIED VARIABLE SELECTION APPROACH FOR VARYING COEFFICIENT MODELS , 2012 .

[23]  Yufeng Liu,et al.  VARIABLE SELECTION IN QUANTILE REGRESSION , 2009 .

[24]  Stanley R. Johnson,et al.  Varying Coefficient Models , 1984 .

[25]  Zongwu Cai,et al.  Nonparametric Quantile Estimations for Dynamic Smooth Coefficient Models , 2008 .

[26]  Jiahua Chen,et al.  Extended Bayesian information criteria for model selection with large model spaces , 2008 .

[27]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[28]  John N. Tsitsiklis,et al.  Introduction to linear optimization , 1997, Athena scientific optimization and computation series.

[29]  A. Belloni,et al.  L1-Penalized Quantile Regression in High Dimensional Sparse Models , 2009, 0904.2931.

[30]  W Y Zhang,et al.  Discussion on `Sure independence screening for ultra-high dimensional feature space' by Fan, J and Lv, J. , 2008 .