Real-time sufficient dimension reduction through principal least squares support vector machines

Abstract We propose a real-time approach for sufficient dimension reduction. Compared with popular sufficient dimension reduction methods including sliced inverse regression and principal support vector machines, the proposed principal least squares support vector machines approach enjoys better estimation of the central subspace. Furthermore, this new proposal can be used in the presence of streamed data for quick real-time updates. It is demonstrated through simulations and real data applications that our proposal performs better and faster than existing algorithms in the literature.

[1]  Jang-Gyu Lee,et al.  On updating the singular value decomposition , 1996, Proceedings of International Conference on Communication Technology. ICCT '96.

[2]  Li Xiaolin.  Real time regression analysis in internet of stock market cycles , 2018, Cognitive Systems Research.

[3]  David P. Woodruff,et al.  The Fast Cauchy Transform and Faster Robust Linear Regression , 2012, SIAM J. Comput..

[4]  Bing Li,et al.  Sufficient Dimension Reduction: Methods and Applications with R , 2018 .

[5]  Johan A. K. Suykens,et al.  Least squares support vector machine classifiers: a large scale algorithm , 1999 .

[6]  Ruth M. Pfeiffer,et al.  On the distribution of the left singular vectors of a random matrix and its applications , 2008 .

[7]  Ker-Chau Li,et al.  Sliced Inverse Regression for Dimension Reduction , 1991 .

[8]  A. Artemiou,et al.  Sufficient dimension reduction via principal Lq support vector machine , 2016 .

[9]  Serkan Günal,et al.  Subspace based feature selection for pattern recognition , 2008, Inf. Sci..

[10]  Bing Li,et al.  Principal support vector machines for linear and nonlinear sufficient dimension reduction , 2011, 1203.2790.

[11]  James R. Schott,et al.  Determining the Dimensionality in Sliced Inverse Regression , 1994 .

[12]  Lixing Zhu,et al.  Dimension Reduction in Regressions Through Cumulative Slicing Estimation , 2010 .

[13]  Hadi Fanaee-T,et al.  Event labeling combining ensemble detectors and background knowledge , 2014, Progress in Artificial Intelligence.

[14]  Andreas Artemiou,et al.  A study on imbalance support vector machine algorithms for sufficient dimension reduction , 2017 .

[15]  S. R. Searle,et al.  On Deriving the Inverse of a Sum of Matrices , 1981 .

[16]  Vladimir Vapnik,et al.  Support-vector networks , 2004, Machine Learning.

[17]  Q. M. Jonathan Wu,et al.  Curvelet based face recognition via dimension reduction , 2009, Signal Process..

[18]  Runze Li,et al.  Online Sufficient Dimension Reduction Through Sliced Inverse Regression , 2020, J. Mach. Learn. Res..

[19]  Wei Luo,et al.  Combining eigenvalues and variation of eigenvectors for order determination , 2016 .

[20]  J. Magnus,et al.  The Commutation Matrix: Some Properties and Applications , 1979 .

[21]  A. Artemiou,et al.  A Cost Based Reweighted Scheme of Principal Support Vector Machine , 2014 .

[22]  Hyunsoo Kim,et al.  Dimension Reduction in Text Classification with Support Vector Machines , 2005, J. Mach. Learn. Res..

[23]  Xingye Qiao,et al.  Sufficient dimension reduction based on distance‐weighted discrimination , 2020, Scandinavian Journal of Statistics.

[24]  Dean P. Foster,et al.  VIF Regression: A Fast Regression Algorithm for Large Data , 2009, 2009 Ninth IEEE International Conference on Data Mining.

[25]  Raul Fonseca Neto,et al.  Online algorithm based on support vectors for orthogonal regression , 2013, Pattern Recognit. Lett..

[26]  Lixing Zhu,et al.  Principal minimax support vector machine for sufficient dimension reduction with contaminated data , 2016, Comput. Stat. Data Anal..

[27]  Dimitrios Makris,et al.  Linear latent low dimensional space for online early action recognition and prediction , 2017, Pattern Recognit..

[28]  Yuexiao Dong,et al.  A brief review of linear sufficient dimension reduction through optimization , 2021 .

[29]  S. Weisberg,et al.  Comments on "Sliced inverse regression for dimension reduction" by K. C. Li , 1991 .

[30]  Ker-Chau Li,et al.  On Principal Hessian Directions for Data Visualization and Dimension Reduction: Another Application of Stein's Lemma , 1992 .

[31]  Yufeng Liu,et al.  Principal weighted support vector machines for sufficient dimension reduction in binary classification , 2017, Biometrika.

[32]  Luc Van Gool,et al.  Real time head pose estimation with random regression forests , 2011, CVPR 2011.

[33]  Glenn Fung,et al.  Proximal support vector machine classifiers , 2001, KDD '01.

[34]  Lixing Zhu,et al.  On Sliced Inverse Regression With High-Dimensional Covariates , 2006 .

[35]  R. H. Moore,et al.  Regression Graphics: Ideas for Studying Regressions Through Graphics , 1998, Technometrics.

[36]  Bing Li,et al.  Successive direction extraction for estimating the central subspace in a multiple-index regression , 2008 .

[37]  Sarajane Marques Peres,et al.  Grammatical Facial Expressions Recognition with Machine Learning , 2014, FLAIRS Conference.

[38]  Johan A. K. Suykens,et al.  Benchmarking Least Squares Support Vector Machine Classifiers , 2004, Machine Learning.

[39]  Seung Jun Shin,et al.  Penalized principal logistic regression for sparse sufficient dimension reduction , 2017, Comput. Stat. Data Anal..

[40]  Michael I. Jordan,et al.  Kernel dimension reduction in regression , 2009, 0908.1854.

[41]  M. Wand,et al.  Real-Time Semiparametric Regression , 2012, 1209.3550.