Trace Pursuit: A General Framework for Model-Free Variable Selection

ABSTRACT We propose trace pursuit for model-free variable selection under the sufficient dimension-reduction paradigm. Two distinct algorithms are proposed: stepwise trace pursuit and forward trace pursuit. Stepwise trace pursuit achieves selection consistency with fixed p. Forward trace pursuit can serve as an initial screening step to speed up the computation in the case of ultrahigh dimensionality. The screening consistency property of forward trace pursuit based on sliced inverse regression is established. Finite sample performances of trace pursuit and other model-free variable selection methods are compared through numerical studies. Supplementary materials for this article are available online.

[1]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[2]  Xiangrong Yin,et al.  Sliced Inverse Regression with Regularizations , 2008, Biometrics.

[3]  Bo Jiang,et al.  Variable selection for general index models via sliced inverse regression , 2013, 1304.4056.

[4]  B. Li,et al.  Dimension reduction for nonelliptically distributed predictors , 2009, 0904.3842.

[5]  R. Dennis Cook,et al.  Using Dimension-Reduction Subspaces to Identify Important Inputs in Models of Physical Systems ∗ , 2009 .

[6]  T. Cai,et al.  A Constrained ℓ1 Minimization Approach to Sparse Precision Matrix Estimation , 2011, 1102.2233.

[7]  Tail areas of linear combinations of chi-squares and non-central chi-squares , 1993 .

[8]  Jun Zhang,et al.  Robust rank correlation based screening , 2010, 1012.4255.

[9]  Cun-Hui Zhang.  Nearly unbiased variable selection under minimax concave penalty , 2010, 1002.4734.

[10]  Jianqing Fan,et al.  Sure independence screening for ultrahigh dimensional feature space , 2006, math/0612857.

[11]  Ker-Chau Li.  Sliced inverse regression for dimension reduction (with discussion) , 1991 .

[12]  Xiangrong Yin,et al.  Sequential sufficient dimension reduction for large p, small n problems , 2015 .

[13]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[14]  Marcel Dettling,et al.  BagBoosting for tumor classification with gene expression data , 2004, Bioinform..

[15]  C. Nachtsheim,et al.  Model‐free variable selection , 2005 .

[16]  Wenxuan Zhong,et al.  Correlation pursuit: forward stepwise variable selection for index models , 2012, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[17]  Bing Li,et al.  Dimension reduction for non-elliptically distributed predictors: second-order methods , 2010 .

[18]  Bing Li,et al.  Successive direction extraction for estimating the central subspace in a multiple-index regression , 2008 .

[19]  Lexin Li,et al.  Sparse sufficient dimension reduction , 2007 .

[20]  R. Cook,et al.  Coordinate-independent sparse sufficient dimension reduction and variable selection , 2010, 1211.3215.

[21]  R. Dennis Cook,et al.  Dimension Reduction in Regressions With Exponential Family Predictors , 2009 .

[22]  Hansheng Wang.  Forward Regression for Ultra-High Dimensional Variable Screening , 2009 .

[23]  Jeffrey S. Morris,et al.  Sure independence screening for ultrahigh dimensional feature space Discussion , 2008 .

[24]  R. H. Moore,et al.  Regression Graphics: Ideas for Studying Regressions Through Graphics , 1998, Technometrics.

[25]  Lan Wang,et al.  Quantile-adaptive model-free variable screening for high-dimensional heterogeneous data , 2013, 1304.2186.

[26]  Yuexiao Dong.  Dimension reduction for non-elliptically distributed predictors , 2009 .

[27]  H. Zou.  The Adaptive Lasso and Its Oracle Properties , 2006 .

[28]  J. Mesirov,et al.  Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. , 1999, Science.

[29]  W Y Zhang,et al.  Discussion on `Sure independence screening for ultra-high dimensional feature space' by Fan, J and Lv, J. , 2008 .

[30]  R. Cook,et al.  Reweighting to Achieve Elliptically Contoured Covariates in Regression , 1994 .

[31]  S. Weisberg,et al.  Comments on "Sliced inverse regression for dimension reduction" by K. C. Li , 1991 .

[32]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[33]  L. Fernholz.  von Mises Calculus For Statistical Functionals , 1983 .

[34]  Ker-Chau Li,et al.  Sliced Inverse Regression for Dimension Reduction , 1991 .

[35]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[36]  F. Portier.  Dimension reduction in regression , 2013 .

[37]  L. Breiman.  Better subset regression using the nonnegative garrote , 1995 .

[38]  Zhou Yu,et al.  Dimension reduction and predictor selection in semiparametric models , 2013 .

[39]  Lixing Zhu,et al.  Nonparametric feature screening , 2013, Comput. Stat. Data Anal..

[40]  Xuming He,et al.  Dimension reduction based on constrained canonical correlation and variable filtering , 2008, 0808.0977.

[41]  R. Cook.  Graphics for regressions with a binary response , 1996 .

[42]  Runze Li,et al.  Feature Screening via Distance Correlation Learning , 2012, Journal of the American Statistical Association.

[43]  T. Cai,et al.  A Direct Estimation Approach to Sparse Linear Discriminant Analysis , 2011, 1107.3442.

[44]  Jiahua Chen,et al.  Extended Bayesian information criteria for model selection with large model spaces , 2008 .

[45]  Yichao Wu,et al.  MARGINAL EMPIRICAL LIKELIHOOD AND SURE INDEPENDENCE FEATURE SCREENING. , 2013, Annals of statistics.

[46]  P. Bentler,et al.  Corrections to test statistics in principal Hessian directions , 2000 .

[47]  Shaoli Wang,et al.  On Directional Regression for Dimension Reduction , 2007 .

[48]  Jun S. Liu,et al.  SLICED INVERSE REGRESSION WITH VARIABLE SELECTION AND INTERACTION DETECTION , 2013 .

[49]  R. Dennis Cook,et al.  A note on shrinkage sliced inverse regression , 2005 .

[50]  E. Petricoin,et al.  Clinical proteomics: translating benchside promise into bedside reality , 2002, Nature Reviews Drug Discovery.

[51]  R. Dennis Cook,et al.  A Model-Free Test for Reduced Rank in Multivariate Regression , 2003 .

[52]  R. Dennis Cook,et al.  Marginal tests with sliced average variance estimation , 2007 .

[53]  R. Dennis Cook,et al.  Testing predictor contributions in sufficient dimension reduction , 2004, math/0406520.

[54]  Runze Li,et al.  Model-Free Feature Screening for Ultrahigh-Dimensional Data , 2011, Journal of the American Statistical Association.