Reduced one-against-all method for multiclass SVM classification

Abstract We present an improved version of one-against-all method for multiclass SVM classification based on subset sample selection, named reduced one-against-all, to achieve high performance in large multiclass problems. Reduced one-against-all drastically decreases the computing effort involved in training one-against-all classifiers, without any compromise in classification accuracy. Computational comparisons on publicly available datasets indicate that the proposed method has comparable accuracy to that of conventional one-against-all method, but with an order of magnitude faster. On the largest dataset considered, reduced one-against-all method achieved 50% reduction in computing time over one-against-all method for almost the same classification accuracy. We further investigated reduced one-against-all with linear kernel for multi-label text categorization applications. Computational results demonstrate the effectiveness of the proposed method on both the text corpuses considered.

[1]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[2]  M. F. Porter,et al.  An algorithm for suffix stripping , 1997 .

[3]  Jonathan J. Hull,et al.  A Database for Handwritten Text Recognition Research , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  David R. Musicant,et al.  Successive overrelaxation for support vector machines , 1999, IEEE Trans. Neural Networks.

[5]  Yuh-Jye Lee,et al.  RSVM: Reduced Support Vector Machines , 2001, SDM.

[6]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[7]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[8]  Thorsten Joachims,et al.  Text Categorization with Support Vector Machines: Learning with Many Relevant Features , 1998, ECML.

[9]  Bernhard Schölkopf,et al.  Extracting Support Data for a Given Task , 1995, KDD.

[10]  Xiaodan Wang,et al.  An Improved Algorithm for Decision-Tree-Based SVM , 2006, 2006 6th World Congress on Intelligent Control and Automation.

[11]  Robert Tibshirani,et al.  Classification by Pairwise Coupling , 1997, NIPS.

[12]  Gérard Dreyfus,et al.  Single-layer learning revisited: a stepwise procedure for building and training a neural network , 1989, NATO Neurocomputing.

[13]  Irwin King,et al.  Locating support vectors via /spl beta/-skeleton technique , 2002, Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02..

[14]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[16]  B. Fei,et al.  Binary tree of SVM: a new fast multiclass training and classification algorithm , 2006, IEEE Transactions on Neural Networks.

[17]  Zheng Nanning,et al.  Unsupervised clustering based reduced support vector machines , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[18]  Antônio de Pádua Braga,et al.  SVM-KM: speeding SVMs learning with a priori cluster selection and k-means , 2000, Proceedings. Vol.1. Sixth Brazilian Symposium on Neural Networks.

[19]  Shigeo Abe,et al.  Fast Training of Support Vector Machines by Extracting Boundary Data , 2001, ICANN.

[20]  S. Halgamuge,et al.  Reducing the Number of Training Samples for Fast Support Vector Machine Classification , 2004 .

[21]  Nello Cristianini,et al.  Large Margin DAGs for Multiclass Classification , 1999, NIPS.

[22]  Narendra Ahuja,et al.  A geometric approach to train support vector machines , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[23]  Ryan M. Rifkin,et al.  In Defense of One-Vs-All Classification , 2004, J. Mach. Learn. Res..

[24]  Sungzoon Cho,et al.  Invariance of neighborhood relation under input space to feature space mapping , 2005, Pattern Recognit. Lett..

[25]  Olvi L. Mangasarian,et al.  Nonlinear Programming , 1969 .

[26]  Nicolás García-Pedrajas,et al.  Improving multiclass pattern recognition by the combination of two strategies , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Leon N. Cooper,et al.  Selecting Data for Fast Support Vector Machines Training , 2007, Trends in Neural Computation.

[28]  Susan T. Dumais,et al.  Inductive learning algorithms and representations for text categorization , 1998, CIKM '98.

[29]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.