Improving the Efficiency of the Support Vector Decomposition Machine

The Support Vector Decomposition Machine is a supervised dimensionality reduction technique which simultaneously minimizes reconstruction error and classification loss. To guarantee a unique minimum, a set of arbitrary constraints are introduced. We propose a different set of constraints, which result in a much more efficient implementation, drastically reducing both training and inference time in simulations with synthetic data.

[1]  Geoffrey J. Gordon,et al.  The support vector decomposition machine , 2006, ICML.

[2]  C. R. Rao,et al.  Generalized Inverse of Matrices and its Applications , 1972 .

[3]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[4]  E. M. Wright,et al.  Adaptive Control Processes: A Guided Tour , 1961, The Mathematical Gazette.

[5]  Mykola Pechenizkiy,et al.  PCA-based feature transformation for classification: issues in medical diagnostics , 2004 .

[6]  S. Lai,et al.  A novel local PCA-Based method for detecting activation signals in fMRI , 1999 .

[7]  Aapo Hyvärinen,et al.  Fast and robust fixed-point algorithms for independent component analysis , 1999, IEEE Trans. Neural Networks.

[8]  Rui Xu,et al.  Survey of clustering algorithms , 2005, IEEE Transactions on Neural Networks.

[9]  Russell Greiner,et al.  Kernel Principal Component Analysis for dimensionality reduction in fMRI-based diagnosis of ADHD , 2012, Front. Syst. Neurosci..

[10]  Edward R. Dougherty,et al.  Performance of feature-selection methods in the classification of high-dimension data , 2009, Pattern Recognit..

[11]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[12]  K. S. Banerjee.  Generalized Inverse of Matrices and Its Applications , 1973 .

[13]  Lijuan Cao,et al.  A comparison of PCA, KPCA and ICA for dimensionality reduction in support vector machine , 2003, Neurocomputing.

[14]  Jing Wang,et al.  Independent component analysis-based dimensionality reduction with applications in hyperspectral image analysis , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..