Multiple birth support vector machine for multi-class classification

For multi-class classification problem, a novel algorithm, called as multiple birth support vector machine (MBSVM), is proposed, which can be considered as an extension of twin support vector machine. Our MBSVM has been compared with the several typical support vector machines. From theoretical point of view, it has been shown that its computational complexity is remarkably low, especially when the class number K is large. Based on our MBSVM, the dual problems of MBSVM are equivalent to symmetric mixed linear complementarity problems to which successive overrelaxation (SOR) can be directly applied. We establish our SOR algorithm for MBSVM. The SOR algorithm handles one data point at a time, so it can process large dataset that need no reside in memory. From practical point of view, its accuracy has been validated by the preliminary numerical experiments.

[1]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[2]  Madan Gopal,et al.  Application of smoothing technique on twin support vector machines , 2008, Pattern Recognit. Lett..

[3]  Xinjun Peng,et al.  TSVR: An efficient Twin Support Vector Machine for regression , 2010, Neural Networks.

[4]  Anirban Mukherjee,et al.  Nonparallel plane proximal classifier , 2009, Signal Process..

[5]  Simon Rogers,et al.  Disease Classification from Capillary Electrophoresis: Mass Spectrometry , 2005, ICAPR.

[6]  Xinjun Peng,et al.  Least squares twin support vector hypersphere (LS-TSVH) for pattern recognition , 2010, Expert Syst. Appl..

[7]  Glenn Fung,et al.  Proximal support vector machine classifiers , 2001, KDD '01.

[8]  Vladimir Vapnik,et al.  The Nature of Statistical Learning , 1995 .

[9]  S. Balasundaram,et al.  Application of Lagrangian Twin Support Vector Machines for Classification , 2010, 2010 Second International Conference on Machine Learning and Computing.

[10]  Xinjun Peng,et al.  Primal twin support vector regression and its sparse approximation , 2010, Neurocomputing.

[11]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[12]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Yuan-Hai Shao,et al.  Improvements on Twin Support Vector Machines , 2011, IEEE Transactions on Neural Networks.

[14]  Koby Crammer,et al.  On the Algorithmic Implementation of Multiclass Kernel-based Vector Machines , 2002, J. Mach. Learn. Res..

[15]  Alexander J. Smola,et al.  Advances in Large Margin Classifiers , 2000 .

[16]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[17]  Isabelle Guyon,et al.  Comparison of classifier methods: a case study in handwritten digit recognition , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[18]  Jason Weston,et al.  Multi-Class Support Vector Machines , 1998 .

[19]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[20]  Reshma Khemchandani,et al.  Optimal kernel selection in twin support vector machines , 2009, Optim. Lett..

[21]  Gene H. Golub,et al.  Matrix computations , 1983 .

[22]  Madan Gopal,et al.  Knowledge based Least Squares Twin support vector machines , 2010, Inf. Sci..

[23]  Chris H. Q. Ding,et al.  Multi-class protein fold recognition using support vector machines and neural networks , 2001, Bioinform..

[24]  Robert Tibshirani,et al.  Classification by Pairwise Coupling , 1997, NIPS.

[25]  Chunxia Zhao,et al.  Localized twin SVM via convex minimization , 2011, Neurocomputing.

[26]  Koby Crammer,et al.  On the Learnability and Design of Output Codes for Multiclass Problems , 2002, Machine Learning.

[27]  Chunxia Zhao,et al.  A feature selection method for nonparallel plane support vector machine classification , 2012, Optim. Methods Softw..

[28]  David R. Musicant,et al.  Successive overrelaxation for support vector machines , 1999, IEEE Trans. Neural Networks.

[29]  G. Wahba,et al.  Multicategory Support Vector Machines , Theory , and Application to the Classification of Microarray Data and Satellite Radiance Data , 2004 .

[30]  Chunxia Zhao,et al.  Least squares twin support vector machine classification via maximum one-class within class variance , 2012, Optim. Methods Softw..

[31]  Xinjun Peng,et al.  A nu-twin support vector machine (nu-TSVM) classifier and its geometric algorithms , 2010, Inf. Sci..

[32]  Olvi L. Mangasarian,et al.  Multisurface proximal support vector machine classification via generalized eigenvalues , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Thomas G. Dietterich,et al.  Solving Multiclass Learning Problems via Error-Correcting Output Codes , 1994, J. Artif. Intell. Res..

[34]  Yoram Singer,et al.  Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers , 2000, J. Mach. Learn. Res..

[35]  Ulrich H.-G. Kreßel,et al.  Pairwise classification and support vector machines , 1999 .

[36]  Eddy Mayoraz,et al.  Improved Pairwise Coupling Classification with Correcting Classifiers , 1998, ECML.

[37]  Nello Cristianini,et al.  Large Margin DAGs for Multiclass Classification , 1999, NIPS.

[38]  Kristin P. Bennett,et al.  Combining support vector and mathematical programming methods for classification , 1999 .

[39]  Z.-Q. Luo,et al.  Error bounds and convergence analysis of feasible descent methods: a general approach , 1993, Ann. Oper. Res..

[40]  Hassiba Nemmour,et al.  Multi-Class SVMs Based on Fuzzy Integral Mixture for Handwritten Digit Recognition , 2006, Geometric Modeling and Imaging--New Trends (GMAI'06).