Dantzig-type penalization for multiple quantile regression with high dimensional covariates

We study joint quantile regression at multiple quantile levels with highdimensional covariates. Variable selection performed at individual quantile levels may lack stability across neighboring quantiles, making it difficult to understand and to interpret the impact of a given covariate on conditional quantile functions. We propose a Dantzig–type penalization method for sparse model selection at each quantile level which, at the same time, aims to shrink differences of the selected models across neighboring quantiles. We show model selection consistency, and investigate the stability of the selected models across quantiles. We also provide asymptotic normality of post–model–selection parameter estimation in the multiple quantile framework. We use numerical examples and data analysis to demonstrate that the proposed Dantzig–type quantile regression model selection method provides stable models for both homogeneous and heterogeneous cases.

[1]  Jinfeng Xu,et al.  Shrinkage estimation of varying covariate effects based on quantile regression , 2014, Stat. Comput..

[2]  H. Zou,et al.  Composite quantile regression and the oracle Model Selection Theory , 2008, 0806.2905.

[3]  H. Zou,et al.  STRONG ORACLE OPTIMALITY OF FOLDED CONCAVE PENALIZED ESTIMATION. , 2012, Annals of statistics.

[4]  Soumendu Sundar Mukherjee.  Weak convergence and empirical processes , 2019 .

[5]  Jianqing Fan,et al.  ADAPTIVE ROBUST VARIABLE SELECTION. , 2012, Annals of statistics.

[6]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[7]  Runze Li,et al.  Quantile Regression for Analyzing Heterogeneity in Ultra-High Dimension , 2012, Journal of the American Statistical Association.

[8]  Cun-Hui Zhang,et al.  Adaptive Lasso for sparse high-dimensional regression models , 2008 .

[9]  Yongdai Kim,et al.  Smoothly Clipped Absolute Deviation on High Dimensions , 2008 .

[10]  Shuheng Zhou.  Restricted Eigenvalue Conditions on Subgaussian Random Matrices , 2009, 0912.4045.

[11]  Ji Zhu,et al.  L1-Norm Quantile Regression , 2008 .

[12]  Martin J. Wainwright,et al.  Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $\ell _{1}$ -Constrained Quadratic Programming (Lasso) , 2009, IEEE Transactions on Information Theory.

[13]  Jianqing Fan,et al.  Sure independence screening for ultrahigh dimensional feature space , 2006, math/0612857.

[14]  S. Geer,et al.  The adaptive and the thresholded Lasso for potentially misspecified models (and a lower bound for the Lasso) , 2011 .

[15]  Howard D Bondell,et al.  Interquantile Shrinkage in Regression Models , 2013, Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America.

[16]  Lie Wang,et al.  L1 penalized LAD estimator for high dimensional linear , 2012, 1202.6347.

[17]  Sungwan Bang,et al.  Simultaneous estimation and factor selection in quantile regression via adaptive sup-norm regularization , 2012, Comput. Stat. Data Anal..

[18]  Shuheng Zhou,et al.  25th Annual Conference on Learning Theory Reconstruction from Anisotropic Random Measurements , 2022 .

[19]  Cun-Hui Zhang,et al.  The sparsity and bias of the Lasso selection in high-dimensional linear regression , 2008, 0808.0967.

[20]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[21]  R. Tibshirani,et al.  Sparsity and smoothness via the fused lasso , 2005 .

[22]  V. Sheffield,et al.  Regulation of gene expression in the mammalian eye and its relevance to eye disease , 2006, Proceedings of the National Academy of Sciences.

[23]  Xuming He.  Quantile Curves without Crossing , 1997 .

[24]  Peng Zhao,et al.  On Model Selection Consistency of Lasso , 2006, J. Mach. Learn. Res..

[25]  Hui Zou,et al.  Computational Statistics and Data Analysis Regularized Simultaneous Model Selection in Multiple Quantiles Regression , 2022 .

[26]  Hannes Leeb,et al.  The Finite-Sample Distribution of Post-Model-Selection Estimators, and Uniform Versus Non-Uniform Approximations , 2000 .

[27]  R. Koenker,et al.  Regression Quantiles , 2007 .

[28]  Sara van de Geer,et al.  Statistics for High-Dimensional Data: Methods, Theory and Applications , 2011 .

[29]  Victor Chernozhukov,et al.  Uniform post-selection inference for least absolute deviation regression and other Z-estimation problems , 2013, 1304.0282.

[30]  Yufeng Liu,et al.  VARIABLE SELECTION IN QUANTILE REGRESSION , 2009 .

[31]  M. Talagrand,et al.  Probability in Banach Spaces: Isoperimetry and Processes , 1991 .

[32]  S. Geer,et al.  On the conditions used to prove oracle results for the Lasso , 2009, 0910.0722.

[33]  N. Meinshausen,et al.  High-dimensional graphs and variable selection with the Lasso , 2006, math/0608017.

[34]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[35]  Holger Dette,et al.  Censored quantile regression processes under dependence and penalization , 2012 .

[36]  A. Belloni,et al.  L1-Penalized Quantile Regression in High Dimensional Sparse Models , 2009, 0904.2931.

[37]  P. Bickel,et al.  SIMULTANEOUS ANALYSIS OF LASSO AND DANTZIG SELECTOR , 2008, 0801.1095.

[38]  Qi Zheng,et al.  GLOBALLY ADAPTIVE QUANTILE REGRESSION WITH ULTRA-HIGH DIMENSIONAL DATA. , 2015, Annals of statistics.

[39]  R. Koenker.  Quantile Regression: Fundamentals of Quantile Regression , 2005 .

[40]  B. M. Pötscher,et al.  MODEL SELECTION AND INFERENCE: FACTS AND FICTION , 2005, Econometric Theory.

[41]  Holger Dette,et al.  Non‐crossing non‐parametric estimates of quantile curves , 2008 .

[42]  Keith Knight,et al.  Limiting distributions for $L\sb 1$ regression estimators under general conditions , 1998 .

[43]  K. B. Kulasekera,et al.  Adaptive penalized quantile regression for high dimensional data , 2013 .

[44]  Martin J. Wainwright,et al.  Restricted Eigenvalue Properties for Correlated Gaussian Designs , 2010, J. Mach. Learn. Res..

[45]  Huixia Judy Wang,et al.  A semiparametric Bayesian approach for joint-quantile regression with clustered data , 2015, Comput. Stat. Data Anal..

[46]  H. Zou.  The Adaptive Lasso and Its Oracle Properties , 2006 .

[47]  Rian,et al.  Non-crossing quantile regression curve estimation , 2010 .

[48]  N. Meinshausen,et al.  LASSO-TYPE RECOVERY OF SPARSE REPRESENTATIONS FOR HIGH-DIMENSIONAL DATA , 2008, 0806.0145.