Sufficient dimension reduction and prediction in regression

Dimension reduction for regression is a prominent issue today because technological advances now allow scientists to routinely formulate regressions in which the number of predictors is considerably larger than in the past. While several methods have been proposed to deal with such regressions, principal components (PCs) still seem to be the most widely used across the applied sciences. We give a broad overview of ideas underlying a particular class of methods for dimension reduction that includes PCs, along with an introduction to the corresponding methodology. New methods are proposed for prediction in regressions with many predictors.

[1]  R. J. Adcock.  A Problem in Least Squares , 1878 .

[2]  R. Fisher,et al.  On the Mathematical Foundations of Theoretical Statistics , 1922 .

[3]  D. Cox,et al.  Notes on Some Aspects of Regression Analysis , 1968 .

[4]  S. Weisberg,et al.  Residuals and Influence in Regression , 1982 .

[5]  I. Helland.  Partial least squares regression and statistical models , 1990 .

[6]  S. Weisberg,et al.  Comments on "Sliced inverse regression for dimension reduction" by K. C. Li , 1991 .

[7]  Ker-Chau Li,et al.  Sliced Inverse Regression for Dimension Reduction , 1991 .

[8]  L. Zhao,et al.  Estimating equations for parameters in means and covariances of multivariate discrete and continuous responses. , 1991, Biometrics.

[9]  Matthew P. Wand,et al.  Kernel Smoothing , 1995 .

[10]  R. Cook.  Graphics for regressions with a binary response , 1996 .

[11]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[12]  J. Simonoff.  Smoothing Methods in Statistics , 1998 .

[13]  D. Botstein,et al.  Singular value decomposition for genome-wide expression data processing and modeling. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[14]  Prasad A. Naik,et al.  Partial least squares estimator for single‐index models , 2000 .

[15]  J. Brian Gray,et al.  Applied Regression Including Computing and Graphics , 1999, Technometrics.

[16]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[17]  Robert Tibshirani,et al.  The Elements of Statistical Learning , 2001 .

[18]  R. Weiss,et al.  Using the Bootstrap to Select One of a New Class of Dimension Reduction Methods , 2003 .

[19]  R. Cook,et al.  Sufficient Dimension Reduction via Inverse Regression , 2005 .

[20]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[21]  Jianqing Fan,et al.  Sure independence screening for ultrahigh dimensional feature space , 2006, math/0612857.

[22]  Lexin Li,et al.  Sparse sufficient dimension reduction , 2007 .

[23]  Yingxing Li,et al.  On hybrid methods of inverse regression-based algorithms , 2007, Comput. Stat. Data Anal..

[24]  Bing Li,et al.  Dimension reduction in regression without matrix inversion , 2007 .

[25]  Shaoli Wang,et al.  On Directional Regression for Dimension Reduction , 2007 .

[26]  John D. Storey,et al.  Capturing Heterogeneity in Gene Expression Studies by Surrogate Variable Analysis , 2007, PLoS genetics.

[27]  R. Christensen,et al.  Fisher Lecture: Dimension Reduction in Regression , 2007, 0708.3774.

[28]  R. Cook,et al.  Principal fitted components for dimension reduction in regression , 2008, 0906.3953.

[29]  C. Robert.  Discussion of "Sure independence screening for ultra-high dimensional feature space" by Fan and Lv. , 2008 .

[30]  Jeffrey S. Morris,et al.  Sure independence screening for ultrahigh dimensional feature space Discussion , 2008 .

[31]  Oliver Johnson.  Theoretical properties of Cook’s PFC dimension reduction algorithm for linear regression , 2008 .

[32]  Xiangrong Yin,et al.  Sliced Inverse Regression with Regularizations , 2008, Biometrics.

[33]  R. Dennis Cook,et al.  Using Dimension-Reduction Subspaces to Identify Important Inputs in Models of Physical Systems ∗ , 2009 .

[34]  R. Cook,et al.  Likelihood-Based Sufficient Dimension Reduction , 2009 .

[35]  Michael I. Jordan,et al.  Kernel dimension reduction in regression , 2009, 0908.1854.

[36]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.