A New Orientation for Multi-Class SVM

Combined by several binary-class SVMs, present multi-class SVMs are usually inefficient in training process. When there is large number of categories of data to classify, training it would be very difficult. Expanded from hyper-sphere one-class SVM (HS-SVM), hyper-sphere multi-class SIM (HSMC-SVM), which builds a HS-SVM for every category of data, is a direct classifier. Its training speed is faster than the combined multi-class classifiers. In order to fast train the HSMC-SVM, a training algorithm following the idea of SMO is proposed. For researching the generalization performance of HSMC-SVM, the theoretic upper bound of generalizing error of HSMC-SVM is analyzed too. As shown in the numeric experiments, the training speed of HSMC-SVM is faster than 1-v-r and 1-v-1, but the classification precision is lower than them. HSMC-SIM provides a new idea on researching fast directed multi-class classifiers in machine learning area.

[1]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[2]  S. Abe,et al.  Decision-tree-based multiclass support vector machines , 2002, Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02..

[3]  Ulrich H.-G. Kreßel,et al.  Pairwise classification and support vector machines , 1999 .

[4]  Nello Cristianini,et al.  Large Margin DAGs for Multiclass Classification , 1999, NIPS.

[5]  Chen Mianyun.  On Multiclass Classification Methods for Support Vector Machines , 2005 .

[6]  John Shawe-Taylor,et al.  Structural Risk Minimization Over Data-Dependent Hierarchies , 1998, IEEE Trans. Inf. Theory.

[7]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[8]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[9]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[10]  Friedhelm Schwenker,et al.  Hierarchical support vector machines for multi-class pattern recognition , 2000, KES'2000. Fourth International Conference on Knowledge-Based Intelligent Engineering Systems and Allied Technologies. Proceedings (Cat. No.00TH8516).

[11]  Jason Weston,et al.  Multi-Class Support Vector Machines , 1998 .