Distance difference and linear programming nonparallel plane classifier

We first propose Distance Difference GEPSVM (DGEPSVM), a binary classifier that obtains two nonparallel planes by solving two standard eigenvalue problems. Compared with GEPSVM, this algorithm does not need to care about the singularity occurring in GEPSVM, but with better classification correctness. This formulation is capable of dealing with XOR problems with different distribution for keeping the genuine geometrical interpretation of primal GEPSVM. Moreover, the proposed algorithm gives classification correctness comparable to that of LSTSVM and TWSVM, but with lesser unknown parameters. Then, the regularization techniques are incorporated to the TWSVM. With the help of the regularized formulation, a linear programming formation for TWSVM is proposed, called FETSVM, to improve TWSVM sparsity, thereby suppressing input features. This means FETSVM is capable of reducing the number of input features, for linear case. When a nonlinear classifier is used, this means few kernel functions determine the classifier. Lastly, this algorithm is compared on artificial and public datasets. To further illustrate the effectiveness of our proposed algorithms, we also apply these algorithms to USPS handwritten digits.

[1]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[2]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[3]  David Page.  Comparative Data Mining for Microarrays : A Case Study Based on Multiple Myeloma , 2002 .

[4]  Anirban Mukherjee,et al.  Nonparallel plane proximal classifier , 2009, Signal Process..

[5]  Gene H. Golub,et al.  Matrix computations , 1983 .

[6]  Daoqiang Zhang,et al.  Efficient and robust feature extraction by maximum margin criterion , 2003, IEEE Transactions on Neural Networks.

[7]  Li Zhang,et al.  Linear programming support vector machines , 2002, Pattern Recognit..

[8]  Bin Chen,et al.  Proximal support vector machine using local information , 2009, Neurocomputing.

[9]  Olvi L. Mangasarian,et al.  Multisurface proximal support vector machine classification via generalized eigenvalues , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Madan Gopal,et al.  Least squares twin support vector machines for pattern classification , 2009, Expert Syst. Appl..

[11]  Hui Zou.  An Improved 1-norm SVM for Simultaneous Classification and Variable Selection , 2007, AISTATS.

[12]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Glenn Fung,et al.  Proximal support vector machine classifiers , 2001, KDD '01.

[14]  Panos M. Pardalos,et al.  A classification method based on generalized eigenvalue problems , 2007, Optim. Methods Softw..

[15]  Yuh-Jye Lee,et al.  RSVM: Reduced Support Vector Machines , 2001, SDM.

[16]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).