Adaptive binary tree for fast SVM multiclass classification

This paper presents an adaptive binary tree (ABT) to reduce the test computational complexity of multiclass support vector machine (SVM). It achieves a fast classification by: (1) reducing the number of binary SVMs for one classification by using separating planes of some binary SVMs to discriminate other binary problems; (2) selecting the binary SVMs with the fewest average number of support vectors (SVs). The average number of SVs is proposed to denote the computational complexity to exclude one class. Compared with five well-known methods, experiments on many benchmark data sets demonstrate our method can speed up the test phase while remain the high accuracy of SVMs.

[1]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[2]  B. Fei,et al.  Binary tree of SVM: a new fast multiclass training and classification algorithm , 2006, IEEE Transactions on Neural Networks.

[3]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[4]  Patrick Haffner,et al.  Support vector machines for histogram-based image classification , 1999, IEEE Trans. Neural Networks.

[5]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[6]  Christopher J. C. Burges,et al.  Simplified Support Vector Decision Rules , 1996, ICML.

[7]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[8]  Farid Melgani,et al.  Toward an Optimal SVM Classification System for Hyperspectral Remote Sensing Images , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[9]  Nello Cristianini,et al.  Large Margin DAGs for Multiclass Classification , 1999, NIPS.

[10]  Harris Drucker,et al.  A Case Study in Handwritten Digit Recognition , 1994 .

[11]  Koby Crammer,et al.  On the Algorithmic Implementation of Multiclass Kernel-based Vector Machines , 2002, J. Mach. Learn. Res..

[12]  Tu Bao Ho,et al.  A bottom-up method for simplifying support vector solutions , 2006, IEEE Transactions on Neural Networks.

[13]  Bin Zhao,et al.  Support Vector Machine and its Application in Handwritten Numeral Recognition , 2000, ICPR.

[14]  Thomas G. Dietterich,et al.  Solving Multiclass Learning Problems via Error-Correcting Output Codes , 1994, J. Artif. Intell. Res..

[15]  J. Chen,et al.  A pairwise decision tree framework for hyperspectral classification , 2007 .

[16]  Lorenzo Bruzzone,et al.  Classification of hyperspectral remote sensing images with support vector machines , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Nicolás García-Pedrajas,et al.  Improving multiclass pattern recognition by the combination of two strategies , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Hang Joon Kim,et al.  Support Vector Machines for Texture Classification , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Ryan M. Rifkin,et al.  In Defense of One-Vs-All Classification , 2004, J. Mach. Learn. Res..

[20]  Xia Shaowei,et al.  Support vector machine and its application in handwritten numeral recognition , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[21]  Yi Lu Murphey,et al.  Multi-class pattern classification using neural networks , 2007, Pattern Recognit..

[22]  Urszula Markowska-Kaczmar,et al.  Support vector machines in handwritten digits classification , 2005, 5th International Conference on Intelligent Systems Design and Applications (ISDA'05).

[23]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[24]  David J. Spiegelhalter,et al.  Machine Learning, Neural and Statistical Classification , 2009 .

[25]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[26]  Tom Downs,et al.  Exact Simplification of Support Vector Solutions , 2002, J. Mach. Learn. Res..

[27]  Jason Weston,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2002, Machine Learning.

[29]  Ulrich H.-G. Kreßel,et al.  Pairwise classification and support vector machines , 1999 .

[30]  Yoram Singer,et al.  Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers , 2000, J. Mach. Learn. Res..

[31]  Cheng Wang,et al.  Combining Support Vector Machines With a Pairwise Decision Tree , 2008, IEEE Geoscience and Remote Sensing Letters.