Combined outputs framework for twin support vector machines

Twin support vector machine (TWSVM) is regarded as a milestone in the development of powerful SVMs. However, there are some inconsistencies with TWSVM that can lead to many reasonable modifications with different outputs. In order to obtain better performance, we propose a novel combined outputs framework that combines rational outputs. Based on this framework, an optimal output model, called the linearly combined twin bounded support vector machine (LCTBSVM), is presented. Our LCTBSVM is based on the outputs of several TWSVMs, and produces the optimal output by solving an optimization problem. Furthermore, two heuristic algorithms are suggested in order to solve the optimization problem. Our comprehensive experiments show the superior generalization performance of our LCTBSVM compared with SVM, PSVM, GEPSVM, and some current TWSVMs, thus confirming the value of our theoretical analysis approach.

[1]  S. Sathiya Keerthi,et al.  Deterministic annealing for semi-supervised kernel machines , 2006, ICML.

[2]  Reshma Khemchandani,et al.  Regularized least squares fuzzy support vector regression for financial time series forecasting , 2009, Expert Syst. Appl..

[3]  Emile H. L. Aarts,et al.  Simulated Annealing: Theory and Applications , 1987, Mathematics and Its Applications.

[4]  Yuan-Hai Shao,et al.  A proximal classifier with consistency , 2013, Knowl. Based Syst..

[5]  Yuan-Hai Shao,et al.  Laplacian unit-hyperplane learning from positive and unlabeled examples , 2015, Inf. Sci..

[6]  Yuan-Hai Shao,et al.  A regularization for the projection twin support vector machine , 2013, Knowl. Based Syst..

[7]  Shuyin Xia,et al.  Relative density based support vector machine , 2015, Neurocomputing.

[8]  Sridha Sridharan,et al.  Improved facial expression recognition via uni-hyperplane classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Yuan-Hai Shao,et al.  Probabilistic outputs for twin support vector machines , 2012, Knowl. Based Syst..

[11]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[12]  Bernhard Schölkopf,et al.  Kernel Methods in Computational Biology , 2005 .

[13]  Anamika Yadav,et al.  Iris recognition using combined support vector machine and Hamming distance approach , 2014, Expert Syst. Appl..

[14]  Zhi-Hua Zhou,et al.  Ensemble Methods: Foundations and Algorithms , 2012 .

[15]  David G. Stork,et al.  Pattern Classification , 1973 .

[16]  Nello Cristianini,et al.  Convex Methods for Transduction , 2003, NIPS.

[17]  Anirban Mukherjee,et al.  Newton's method for nonparallel plane proximal classifier with unity norm hyperplanes , 2010, Signal Process..

[18]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[19]  Yuan-Hai Shao,et al.  A coordinate descent margin based-twin support vector machine for classification , 2012, Neural Networks.

[20]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[21]  Yong Shi,et al.  ν-Nonparallel support vector machine for pattern classification , 2014, Neural Computing and Applications.

[22]  Santanu Ghorai,et al.  Unity norm twin support vector machine classifier , 2010, 2010 Annual IEEE India Conference (INDICON).

[23]  Yuan-Hai Shao,et al.  A novel margin-based twin support vector machine with unity norm hyperplanes , 2012, Neural Computing and Applications.

[24]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[25]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[26]  Yuan-Hai Shao,et al.  Least squares recursive projection twin support vector machine for classification , 2012, Pattern Recognit..

[27]  Yong Shi,et al.  Robust twin support vector machine for pattern classification , 2013, Pattern Recognit..

[28]  Francisco Herrera,et al.  Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power , 2010, Inf. Sci..

[29]  N. Deng,et al.  Improved prediction of palmitoylation sites using PWMs and SVM. , 2011, Protein and peptide letters.

[30]  O. Mangasarian,et al.  Massive data discrimination via linear support vector machines , 2000 .

[31]  Nai-Yang Deng,et al.  Support Vector Machines: Optimization Based Theory, Algorithms, and Extensions , 2012 .

[32]  Yuan-Hai Shao,et al.  Nonparallel hyperplane support vector machine for binary classification problems , 2014, Inf. Sci..

[33]  Olvi L. Mangasarian,et al.  Multisurface proximal support vector machine classification via generalized eigenvalues , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Lei Wang,et al.  AdaBoost with SVM-based component classifiers , 2008, Eng. Appl. Artif. Intell..

[35]  Zhi-Hua Zhou,et al.  Towards Making Unlabeled Data Never Hurt , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  S. García,et al.  An Extension on "Statistical Comparisons of Classifiers over Multiple Data Sets" for all Pairwise Comparisons , 2008 .

[37]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[38]  I ScottKirkpatrick.  Optimization by Simulated Annealing: Quantitative Studies , 1984 .

[39]  S. Sathiya Keerthi,et al.  Branch and Bound for Semi-Supervised Support Vector Machines , 2006, NIPS.

[40]  C. M. Bishop,et al.  Improvements on Twin Support Vector Machines , 2011 .

[41]  Glenn Fung,et al.  Proximal support vector machine classifiers , 2001, KDD '01.

[42]  Sang Won Yoon,et al.  Breast cancer diagnosis based on feature extraction using a hybrid of K-means and support vector machine algorithms , 2014, Expert Syst. Appl..