Adaptive sup-norm regularized simultaneous multiple quantiles regression

When modelling multiple conditional quantiles of univariate and/or multivariate responses, it is of great importance to share strength among them. The simultaneous multiple quantiles regression (SMQR) technique is a novel regularization method that explores the similarity among multiple conditional quantiles and performs simultaneous model selection. However, the SMQR suffers from estimation inefficiency and model selection inconsistency because it applies the same amount of shrinkage to each predictor variable without assessing its relative importance. To overcome such a limitation, we propose the adaptive sup-norm regularized SMQR (ASMQR) method, which allows different amounts of shrinkage to be imposed on different variables according to their relative importance. We show that the proposed ASMQR method, a generalized form of the adaptive lasso regularized quantile regression (ALQR) method, possesses the oracle property and that it is a better tool for selecting a common subset of significant variables than the ALQR and SMQR methods through a simulation study.

[1]  Keith Knight,et al.  Limiting distributions for $L\sb 1$ regression estimators under general conditions , 1998 .

[2]  Hansheng Wang,et al.  Computational Statistics and Data Analysis a Note on Adaptive Group Lasso , 2022 .

[3]  Pin T. Ng,et al.  Quantile smoothing splines , 1994 .

[4]  Hao Helen Zhang,et al.  Variable selection for the multicategory SVM via adaptive sup-norm regularization , 2008, 0803.3676.

[5]  Sungwan Bang,et al.  Simultaneous estimation and factor selection in quantile regression via adaptive sup-norm regularization , 2012, Comput. Stat. Data Anal..

[6]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[7]  Hansheng Wang,et al.  Robust Regression Shrinkage and Consistent Variable Selection Via the LAD-Lasso , 2008 .

[8]  H. Zou,et al.  The F ∞ -norm support vector machine , 2008 .

[9]  R. Koenker,et al.  Regression Quantiles , 2007 .

[10]  R. Koenker,et al.  Hierarchical Spline Models for Conditional Quantiles and the Demand for Electricity , 1990 .

[11]  R. Koenker.  Quantile regression for longitudinal data , 2004 .

[12]  Hansheng Wang,et al.  Robust Regression Shrinkage and Consistent Variable Selection Through the LAD-Lasso , 2007 .

[13]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[14]  J. Friedman,et al.  Predicting Multivariate Responses in Multiple Linear Regression , 1997 .

[15]  Jinfeng Xu,et al.  Simultaneous estimation and variable selection in median regression using Lasso-type penalty , 2010, Annals of the Institute of Statistical Mathematics.

[16]  Hui Zou,et al.  Computational Statistics and Data Analysis Regularized Simultaneous Model Selection in Multiple Quantiles Regression , 2022 .

[17]  Stephen J. Wright,et al.  Simultaneous Variable Selection , 2005, Technometrics.

[18]  Achievement Motivation for Introductory College Biology , 2013 .

[19]  Ji Zhu,et al.  Variable selection for multicategory SVM via sup-norm regularization , 2006 .

[20]  Song Yang,et al.  Censored Median Regression Using Weighted Empirical Survival and Hazard Functions , 1999 .

[21]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[22]  R. Koenker.  Quantile Regression: Name Index , 2005 .

[23]  Yufeng Liu,et al.  Simultaneous multiple non-crossing quantile regression estimation using kernel constraints , 2011, Journal of nonparametric statistics.

[24]  Ming Yuan,et al.  GACV for quantile smoothing splines , 2006, Comput. Stat. Data Anal..

[25]  Xuming He,et al.  Detecting Differential Expressions in GeneChip Microarray Studies , 2007 .

[26]  Alexander J. Smola,et al.  Nonparametric Quantile Estimation , 2006, J. Mach. Learn. Res..

[27]  C. Geyer.  On the Asymptotics of Constrained $M$-Estimation , 1994 .

[28]  Yufeng Liu,et al.  Stepwise multiple quantile regression estimation using non-crossing constraints , 2009 .

[29]  M. Yuan,et al.  On the non‐negative garrotte estimator , 2007 .

[30]  H. Zou.  The Adaptive Lasso and Its Oracle Properties , 2006 .

[31]  Rian,et al.  Non-crossing quantile regression curve estimation , 2010 .

[32]  Yufeng Liu,et al.  VARIABLE SELECTION IN QUANTILE REGRESSION , 2009 .

[33]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[34]  H. Bondell,et al.  Noncrossing quantile regression curve estimation. , 2010, Biometrika.

[35]  M. Yuan,et al.  On the Nonnegative Garrote Estimator , 2005 .

[36]  G. Wahba,et al.  A NOTE ON THE LASSO AND RELATED PROCEDURES IN MODEL SELECTION , 2006 .

[37]  Timo Similä,et al.  Input selection and shrinkage in multiresponse linear regression , 2007, Comput. Stat. Data Anal..

[38]  Ji Zhu,et al.  L1-Norm Quantile Regression , 2008 .

[39]  B. Skagerberg,et al.  Multivariate data analysis applied to low-density polyethylene reactors , 1992 .

[40]  Roger W. Johnson.  Fitting Percentage of Body Fat to Simple Body Measurements: College Women , 1996, Journal of Statistics and Data Science Education.