Sparse principal component regression with adaptive loading

Principal component regression (PCR) is a two-stage procedure that selects some principal components and then constructs a regression model regarding them as new explanatory variables. Note that the principal components are obtained from only explanatory variables and not considered with the response variable. To address this problem, we propose the sparse principal component regression (SPCR) that is a one-stage procedure for PCR. SPCR enables us to adaptively obtain sparse principal component loadings that are related to the response variable and select the number of principal components simultaneously. SPCR can be obtained by the convex optimization problem for each parameter with the coordinate descent algorithm. Monte Carlo simulations and real data analyses are performed to illustrate the effectiveness of SPCR.

[1]  Kai Wang,et al.  A principal components regression approach to multilocus genetic association studies , 2008, Genetic epidemiology.

[2]  W. Massy.  Principal Components Regression in Exploratory Statistical Research , 1965 .

[3]  Marina Vannucci,et al.  Regularized partial least squares with an application to NMR spectroscopy , 2012, Stat. Anal. Data Min..

[4]  P. Reiss,et al.  Functional Principal Component Regression and Functional Partial Least Squares , 2007 .

[5]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[6]  S. Keleş,et al.  Sparse partial least squares regression for simultaneous dimension reduction and variable selection , 2010, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[7]  Fang Han,et al.  Robust Sparse Principal Component Regression under the High Dimensional Elliptical Model , 2013, NIPS.

[8]  Trevor Hastie,et al.  Regularization Paths for Generalized Linear Models via Coordinate Descent. , 2010, Journal of statistical software.

[9]  Seokho Lee,et al.  A coordinate descent MM algorithm for fast computation of sparse logistic PCA , 2013, Comput. Stat. Data Anal..

[10]  Jianhua Z. Huang,et al.  SPARSE LOGISTIC PRINCIPAL COMPONENTS ANALYSIS FOR BINARY DATA. , 2010, The annals of applied statistics.

[11]  Hui Zou,et al.  A Penalized Maximum Likelihood Approach to Sparse Factor Analysis , 2010 .

[12]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[13]  G. Irwin,et al.  Dynamic inferential estimation using principal components regression (PCR) , 1998 .

[14]  Kei Hirose,et al.  Sparse estimation via nonconcave penalized likelihood in factor analysis model , 2012, Stat. Comput..

[15]  Jianhua Z. Huang,et al.  Sparse Reduced-Rank Regression for Simultaneous Dimension Reduction and Variable Selection , 2012 .

[16]  Ian T. Jolliffe,et al.  Principal Component Analysis , 2002, International Encyclopedia of Statistical Science.

[17]  Robert I. Jennrich,et al.  Rotation to Simple Loadings Using Component Loss Functions: The Oblique Case , 2006 .

[18]  H. Zou.  The Adaptive Lasso and Its Oracle Properties , 2006 .

[19]  R. Tibshirani,et al.  Sparse Principal Component Analysis , 2006 .

[20]  Hans-Peter Kriegel,et al.  Supervised probabilistic principal component analysis , 2006, KDD '06.

[21]  Chenlei Leng,et al.  VARIABLE SELECTION AND COEFFICIENT ESTIMATION VIA REGULARIZED RANK REGRESSION , 2010 .

[22]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[23]  J. Friedman,et al.  A Statistical View of Some Chemometrics Regression Tools , 1993 .

[24]  I. Jolliffe,et al.  A Modified Principal Component Technique Based on the LASSO , 2003 .

[25]  Karl Pearson F.R.S..  LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[26]  I. Jolliffe.  A Note on the Use of Principal Components in Regression , 1982 .

[27]  S. D. Jong.  SIMPLS: an alternative approach to partial least squares regression , 1993 .

[28]  Jun Shao,et al.  The gic for model selection : a hypothesis testing approach , 2000 .

[29]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[30]  J. Friedman,et al.  [A Statistical View of Some Chemometrics Regression Tools]: Response , 1993 .

[31]  Abbas Khalili.  New estimation and feature selection methods in mixture‐of‐experts models , 2010 .

[32]  Andrzej Cichocki,et al.  Kernel PCA for Feature Extraction and De-Noising in Nonlinear Regression , 2001, Neural Computing & Applications.

[33]  R. Tibshirani,et al.  Prediction by Supervised Principal Components , 2006 .