Nonparallel plane proximal classifier

We observed that the two costly optimization problems of twin support vector machine (TWSVM) classifier can be avoided by introducing a technique as used in proximal support vector machine (PSVM) classifier. With this modus operandi we formulate a much simpler nonparallel plane proximal classifier (NPPC) for speeding up the training of it by reducing significant computational burden over TWSVM. The formulation of NPPC for binary data classification is based on two identical mean square error (MSE) optimization problems which lead to solving two small systems of linear equations in input space. Thus it eliminates the need of any specialized software for solving the quadratic programming problems (QPPs). The formulation is also extended for nonlinear kernel classifier. Our computations show that a MATLAB implementation of NPPC can be trained with a data set of 3 million points with 10 attributes in less than 3s. Computational results on synthetic as well as on several bench mark data sets indicate the advantages of the proposed classifier in both computational time and test accuracy. The experimental results also indicate that performances of classifiers obtained by MSE approach are sufficient in many cases than the classifiers obtained by standard SVM approach.

[1]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[3]  Mokhtar S. Bazaraa,et al.  Nonlinear Programming: Theory and Algorithms , 1993 .

[4]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[5]  J. Chambers,et al.  A robust mixed-norm adaptive filter algorithm , 1997, IEEE Signal Processing Letters.

[6]  Tomaso A. Poggio,et al.  Regularization Networks and Support Vector Machines , 2000, Adv. Comput. Math..

[7]  Gene H. Golub,et al.  Matrix computations , 1983 .

[8]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Bernhard Schölkopf,et al.  Support vector channel selection in BCI , 2004, IEEE Transactions on Biomedical Engineering.

[10]  Theodore B. Trafalis,et al.  Support vector machine for regression and applications to financial forecasting , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[11]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[12]  Thorsten Joachims,et al.  Making large-scale support vector machine learning practical , 1999 .

[13]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[14]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[15]  D Haussler,et al.  Knowledge-based analysis of microarray gene expression data by using support vector machines. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[16]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Kok Seng Chua,et al.  Efficient computations for large least square support vector machine classifiers , 2003, Pattern Recognit. Lett..

[18]  Nello Cristianini,et al.  An introduction to Support Vector Machines , 2000 .

[19]  Alessandro Verri,et al.  Pattern recognition with support vector machines : First International Workshop, SVM 2002, Niagara Falls, Canada, August 10, 2002 : proceedings , 2002 .

[20]  Soushan Wu,et al.  Credit rating analysis with support vector machines and neural networks: a market comparative study , 2004, Decis. Support Syst..

[21]  Glenn Fung,et al.  Proximal support vector machine classifiers , 2001, KDD '01.

[22]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[23]  Alexander J. Smola,et al.  Advances in Large Margin Classifiers , 2000 .

[24]  Olvi L. Mangasarian,et al.  Multisurface proximal support vector machine classification via generalized eigenvalues , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Touradj Ebrahimi,et al.  Joint Time-Frequency-Space Classification of EEG in a Brain-Computer Interface Application , 2003, EURASIP J. Adv. Signal Process..

[26]  Thorsten Joachims,et al.  Text Categorization with Support Vector Machines: Learning with Many Relevant Features , 1998, ECML.

[27]  Yuh-Jye Lee,et al.  RSVM: Reduced Support Vector Machines , 2001, SDM.

[28]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[29]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[30]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machines , 2002 .

[31]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[32]  Christopher J. C. Burges,et al.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998, Data Mining and Knowledge Discovery.