Principal weighted logistic regression for sufficient dimension reduction in binary classification

Abstract Sufficient dimension reduction (SDR) is a popular supervised machine learning technique that reduces the predictor dimension and facilitates subsequent data analysis in practice. In this article, we propose principal weighted logistic regression (PWLR), an efficient SDR method in binary classification where inverse-regression-based SDR methods often suffer. We first develop linear PWLR for linear SDR and study its asymptotic properties. We then extend it to nonlinear SDR and propose the kernel PWLR. Evaluations with both simulated and real data show the promising performance of the PWLR for SDR in binary classification.

[1]  Su-Yun Huang,et al.  Nonlinear Dimension Reduction with Kernel Sliced Inverse Regression , 2009, IEEE Transactions on Knowledge and Data Engineering.

[2]  G. Wahba,et al.  Some results on Tchebycheffian spline functions , 1971 .

[3]  Ruth M. Pfeiffer,et al.  On the distribution of the left singular vectors of a random matrix and its applications , 2008 .

[4]  Han-Ming Wu.  Kernel Sliced Inverse Regression with Applications to Classification , 2008 .

[5]  R. Dennis Cook,et al.  Testing predictor contributions in sufficient dimension reduction , 2004, math/0406520.

[6]  Ker-Chau Li,et al.  Sliced Inverse Regression for Dimension Reduction , 1991 .

[7]  B. Li,et al.  Dimension reduction for nonelliptically distributed predictors , 2009, 0904.3842.

[8]  Shaoli Wang,et al.  On Directional Regression for Dimension Reduction , 2007 .

[9]  R. Cook,et al.  Fused Estimators of the Central Subspace in Sufficient Dimension Reduction , 2014 .

[10]  Ker-Chau Li,et al.  Regression Analysis Under Link Violation , 1989 .

[11]  R. Cook,et al.  Using intraslice covariances for improved estimation of the central subspace in regression , 2006 .

[12]  S. Weisberg,et al.  Comments on "Sliced inverse regression for dimension reduction" by K. C. Li , 1991 .

[13]  G. Wahba.  Spline models for observational data , 1990 .

[14]  Liping Zhu,et al.  A Review on Dimension Reduction , 2013, International statistical review = Revue internationale de statistique.

[15]  Ker-Chau Li,et al.  On Principal Hessian Directions for Data Visualization and Dimension Reduction: Another Application of Stein's Lemma , 1992 .

[16]  Yufeng Liu,et al.  Principal weighted support vector machines for sufficient dimension reduction in binary classification , 2017, Biometrika.

[17]  Bing Li,et al.  A general theory for nonlinear sufficient dimension reduction: Formulation and estimation , 2013, 1304.0580.

[18]  R. Christensen,et al.  Fisher Lecture: Dimension Reduction in Regression , 2007, 0708.3774.

[19]  Lixing Zhu,et al.  Dimension Reduction in Regressions Through Cumulative Slicing Estimation , 2010 .

[20]  Bing Li,et al.  Principal support vector machines for linear and nonlinear sufficient dimension reduction , 2011, 1203.2790.

[21]  Wei Luo,et al.  Combining eigenvalues and variation of eigenvectors for order determination , 2016 .

[22]  Yufeng Liu,et al.  Probability-enhanced sufficient dimension reduction for binary classification. , 2014, Biometrics.

[23]  H. Tong,et al.  Article: 2 , 2002, European Financial Services Law.

[24]  R. Cook,et al.  Dimension Reduction in Binary Response Regression , 1999 .

[25]  D. Pollard.  Asymptotics for Least Absolute Deviation Regression Estimators , 1991, Econometric Theory.

[26]  Seung Jun Shin,et al.  Penalized principal logistic regression for sparse sufficient dimension reduction , 2017, Comput. Stat. Data Anal..

[27]  W. Newey,et al.  Large sample estimation and hypothesis testing , 1986 .